Hello All,
I am playing around with the different data normalization modes for dividends and noticed something. When using DataNormalizationMode.TotalReturn I noticed that the price is not adjusted until a couple of days after the dividend. Just curious what the logic behind this is?
Using the following code to test. I am switching between Raw handling and Total Return.
import numpy as np
### <summary>
### Basic template algorithm simply initializes the date range and cash. This is a skeleton
### framework you can use for designing an algorithm.
### </summary>
class BasicTemplateAlgorithm(QCAlgorithm):
'''Basic template algorithm simply initializes the date range and cash'''
def Initialize(self):
'''Initialise the data and resolution required, as well as the cash and start-end dates for your algorithm. All algorithms must initialized.'''
self.ticker = 'MSFT'
self.raw_handling = True
self.SetStartDate(2018,2,8) #Set Start Date
self.SetEndDate(2018,2,20) #Set End Date
self.SetCash(10000) #Set Strategy Cash
# Find more symbols here: http://quantconnect.com/data
self.AddEquity(self.ticker, Resolution.Daily)
# Set Dividend Handling Method
# https://www.quantconnect.com/forum/discussion/508/update-dividends-splits-and-custom-price-normalization/p1
if self.raw_handling:
self.Securities[self.ticker].SetDataNormalizationMode(DataNormalizationMode.Raw)
else:
self.Securities[self.ticker].SetDataNormalizationMode(DataNormalizationMode.TotalReturn)
def OnData(self, data):
'''OnData event is the primary entry point for your algorithm. Each new data point will be pumped in here.
Arguments:
data: Slice object keyed by symbol containing the stock data
'''
if not self.Portfolio.Invested:
self.SetHoldings(self.ticker, 1)
for kvp in data.Dividends: # update this to Dividends dictionary
div_ticker = kvp.Key
div_distribution = kvp.Value.Distribution
div_total_value = div_distribution * self.Portfolio[self.ticker].Quantity
self.Log("{0} >> DIVIDEND >> {1} - ${2} - ${3}".format(self.Time, div_ticker, div_distribution, div_total_value))
self.Log("{0} >> SUMMARY >> {1} | Port Cash: {2} | Port Value: {3} | Holdings: {4} | Price {5}".format(self.Time,
self.ticker, self.Portfolio.Cash, self.Portfolio.TotalPortfolioValue, self.Portfolio[self.ticker].Quantity, self.Portfolio[self.ticker].Price))
The final results are correct from what I can see. I just noticed that the price is not adjusted the day after. Instead it is adjusted 2 days after.
RAW Log
2018-02-08 00:00:00 Launching analysis for 524f7e92d0fe02169f8772aff24da261 with LEAN Engine v2.4.0.0.4710
2018-02-08 00:00:00 2018-02-08 00:00:00 >> SUMMARY >> MSFT | Port Cash: 10000.0 | Port Value: 10000.00000 | Holdings: 0 | Price 89.5800
2018-02-09 00:00:00 2018-02-09 00:00:00 >> SUMMARY >> MSFT | Port Cash: 41.190 | Port Value: 9475.08000 | Holdings: 111 | Price 84.9900
2018-02-10 00:00:00 2018-02-10 00:00:00 >> SUMMARY >> MSFT | Port Cash: 41.190 | Port Value: 9825.84000 | Holdings: 111 | Price 88.1500
2018-02-13 00:00:00 2018-02-13 00:00:00 >> SUMMARY >> MSFT | Port Cash: 41.190 | Port Value: 9936.84000 | Holdings: 111 | Price 89.1500
2018-02-14 00:00:00 2018-02-14 00:00:00 >> DIVIDEND >> MSFT - $0.42 - $46.62
2018-02-14 00:00:00 2018-02-14 00:00:00 >> SUMMARY >> MSFT | Port Cash: 87.810 | Port Value: 10053.39000 | Holdings: 111 | Price 89.7800
2018-02-15 00:00:00 2018-02-15 00:00:00 >> SUMMARY >> MSFT | Port Cash: 87.810 | Port Value: 10053.39000 | Holdings: 111 | Price 89.7800
2018-02-16 00:00:00 2018-02-16 00:00:00 >> SUMMARY >> MSFT | Port Cash: 87.810 | Port Value: 10379.73000 | Holdings: 111 | Price 92.7200
2018-02-17 00:00:00 2018-02-17 00:00:00 >> SUMMARY >> MSFT | Port Cash: 87.810 | Port Value: 10292.04000 | Holdings: 111 | Price 91.9300
2018-02-21 00:00:00 2018-02-21 00:00:00 >> SUMMARY >> MSFT | Port Cash: 87.810 | Port Value: 10379.73000 | Holdings: 111 | Price 92.7200
2018-02-21 00:00:00 Algorithm Id:(524f7e92d0fe02169f8772aff24da261) completed in 1.09 seconds at 0k data points per second. Processing total of 11 data points.
Total Return Log
2018-02-08 00:00:00 Launching analysis for 1245afb9bd9840408d99861cd4c6661c with LEAN Engine v2.4.0.0.4710
2018-02-08 00:00:00 2018-02-08 00:00:00 >> SUMMARY >> MSFT | Port Cash: 10000.0 | Port Value: 10000.00000 | Holdings: 0 | Price 89.5800
2018-02-09 00:00:00 2018-02-09 00:00:00 >> SUMMARY >> MSFT | Port Cash: 41.190 | Port Value: 9475.08000 | Holdings: 111 | Price 84.9900
2018-02-10 00:00:00 2018-02-10 00:00:00 >> SUMMARY >> MSFT | Port Cash: 41.190 | Port Value: 9825.84000 | Holdings: 111 | Price 88.1500
2018-02-13 00:00:00 2018-02-13 00:00:00 >> SUMMARY >> MSFT | Port Cash: 41.190 | Port Value: 9936.84000 | Holdings: 111 | Price 89.1500
2018-02-14 00:00:00 2018-02-14 00:00:00 >> DIVIDEND >> MSFT - $0.42 - $46.62
2018-02-14 00:00:00 2018-02-14 00:00:00 >> SUMMARY >> MSFT | Port Cash: 41.190 | Port Value: 10006.77000 | Holdings: 111 | Price 89.7800
2018-02-15 00:00:00 2018-02-15 00:00:00 >> SUMMARY >> MSFT | Port Cash: 41.190 | Port Value: 10006.77000 | Holdings: 111 | Price 89.7800
2018-02-16 00:00:00 2018-02-16 00:00:00 >> SUMMARY >> MSFT | Port Cash: 41.190 | Port Value: 10379.73000 | Holdings: 111 | Price 93.1400
2018-02-17 00:00:00 2018-02-17 00:00:00 >> SUMMARY >> MSFT | Port Cash: 41.190 | Port Value: 10292.04000 | Holdings: 111 | Price 92.3500
2018-02-21 00:00:00 2018-02-21 00:00:00 >> SUMMARY >> MSFT | Port Cash: 41.190 | Port Value: 10379.73000 | Holdings: 111 | Price 93.1400
2018-02-21 00:00:00 Algorithm Id:(1245afb9bd9840408d99861cd4c6661c) completed in 1.11 seconds at 0k data points per second. Processing total
As we can see the distribution happens on the 14th of Feb. Our cash balance is immediately updated in Raw mode as expected. However, in the total return log, the price remains the same as the Raw log until the 16th of Feb.
BONUS semi-related question
Is there a way to account for ex-div dates? For example in this example, I probably should not have received the dividend as it would have gone ex-div before the test start date.
ThatBlokeDave
One other thing, I just noticed that if I comment out the `SetDataNormalizationMode` lines and run the script, I get different close prices to the Raw, Split Adjusted and Total Return methods:
See here:
2018-02-08 00:00:00 : Launching analysis for 16e612198ee190d2bc12ea6c648d1056 with LEAN Engine v2.4.0.0.4721 2018-02-08 00:00:00 : 2018-02-08 00:00:00 >> SUMMARY >> MSFT | Port Cash: 10000.0 | Port Value: 10000.0 | Holdings: 0 | Price 88.43586632400 2018-02-09 00:00:00 : 2018-02-09 00:00:00 >> SUMMARY >> MSFT | Port Cash: 79.8089349440 | Port Value: 9477.111895808000 | Holdings: 112 | Price 83.90449072200 2018-02-10 00:00:00 : 2018-02-10 00:00:00 >> SUMMARY >> MSFT | Port Cash: 79.8089349440 | Port Value: 9826.511558784000 | Holdings: 112 | Price 87.02413057000 2018-02-13 00:00:00 : 2018-02-13 00:00:00 >> SUMMARY >> MSFT | Port Cash: 79.8089349440 | Port Value: 9937.081072384000 | Holdings: 112 | Price 88.01135837000 2018-02-14 00:00:00 : 2018-02-14 00:00:00 >> DIVIDEND >> MSFT - $0.42 - $47.04 2018-02-14 00:00:00 : 2018-02-14 00:00:00 >> SUMMARY >> MSFT | Port Cash: 79.8089349440 | Port Value: 10006.739865952000 | Holdings: 112 | Price 88.63331188400 2018-02-15 00:00:00 : 2018-02-15 00:00:00 >> SUMMARY >> MSFT | Port Cash: 79.8089349440 | Port Value: 10006.739865952000 | Holdings: 112 | Price 88.63331188400 2018-02-16 00:00:00 : 2018-02-16 00:00:00 >> SUMMARY >> MSFT | Port Cash: 79.8089349440 | Port Value: 10379.998965536000 | Holdings: 112 | Price 91.96598241600 2018-02-17 00:00:00 : 2018-02-17 00:00:00 >> SUMMARY >> MSFT | Port Cash: 79.8089349440 | Port Value: 10292.238502592000 | Holdings: 112 | Price 91.18240685400 2018-02-21 00:00:00 : 2018-02-21 00:00:00 >> SUMMARY >> MSFT | Port Cash: 79.8089349440 | Port Value: 10379.998965536000 | Holdings: 112 | Price 91.96598241600 2018-02-21 00:00:00 : Algorithm Id:(16e612198ee190d2bc12ea6c648d1056) completed in 1.10 seconds at 0k data points per second. Processing total of 11 data points.
When looking into this, I was working from this post:
update-dividends-splits-and-custom-price-normalization/p1
Jared notes that:
"The current, default behavior of QuantConnect is to fully adjust for splits and dividends, giving you an adjusted price which always equals today's price. "
So to me that sounds like the Raw mode as the price today price I see today (for any stock) is affected by the dividend payments and splits on that day (if any)...
However, since the close prices in the log above does not reflect the results from any of the data normalization modes in the post (Raw, SplitAdjusted or TotalReturn), I assume that I have missing a piece of information.
ThatBlokeDave
Ok - So seems my ex-div question is not valid. The data provided IS the ex-div date according to this:
MSFT - Dividend Payout Table
I was slightly confused since when I was testing in Raw Mode, the cash was paid into the account immediately. However, this would not be the case in the real world. According to the table, I would not have been paid until the 8th of March.
If I am looking at this correctly, the issue I see is this.... because the distribution is paid in cash and added to my balance immediately on the ex-div date, I can re-invest that money earlier and have more time in the market than I would have in the real world.
I imagine this could skew the backtest results significantly if I have a divdend heavy portfolio.
Note: This still does not get me closer to understanding the adjustment logic and Naormalization mode.
ThatBlokeDave
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
To unlock posting to the community forums please complete at least 30% of Boot Camp.
You can continue your Boot Camp training progress from the terminal. We hope to see you in the community soon!