Comparison of QC MOMP with momentum calculated using self.History.

Hi,

changing period + 1 to period give perfect result

Hi Vladimir,

The values are different above because the indicator period is 126 while the manual calculation period is 127. We need to replace

with

See the attached backtest for reference.

Best,
Derek Melchin

Alexander Ermushev, Derek Melchin,

To get momentum for 1 day, we need 2 bars of data (today and yesterday), one more bar than RETURN_PERIOD.
The same is true if you want a momentum of 21, 63, 126, 252 days.
Calculating momentum using self.History in my backtest is correct.
To prove that I ran the test with RETURN_PERIOD = 1 in my version of the code.
The problem is somewhere else.

Derek Melchin,

Here is the log from backtest above

2021-01-01 00:00:00 :     Ending values: -0.0 0.24602997092371126
2021-01-05 00:00:00 :     Ending values: -1.4119971951297252 -1.4119971951297217
2021-01-06 00:00:00 :     Ending values: 0.8244156348000389 0.8244156348000242
2021-01-07 00:00:00 :     Ending values: -1.3852369653049446 -1.3852369653049368
2021-01-08 00:00:00 :     Ending values: 2.419197502763868 2.419197502763848
2021-01-09 00:00:00 :     Ending values: 1.2857959235507017 1.2857959235507144
2021-01-12 00:00:00 :     Ending values: -1.445005171927405 -1.4450051719274049
2021-01-13 00:00:00 :     Ending values: -0.15902296291584503 -0.1590229629158446
2021-01-14 00:00:00 :     Ending values: 0.6753312945973498 0.6753312945973367
2021-01-15 00:00:00 :     Ending values: -0.5347424376661183 -0.5347424376661136
2021-01-16 00:00:00 :     Ending values: -0.7921107046285987 -0.7921107046286013
2021-01-20 00:00:00 :     Ending values: 1.4589880074392356 1.4589880074392436
2021-01-21 00:00:00 :     Ending values: 2.326095888246263 2.32609588824626

You can see that the values are almost exactly the same on all bars except the first one.

To my mind momentum calculated on self.History shows the right value.

Setting manual WarmUp
    self.SetWarmUp(RETURN_PERIOD + 1)
gives a near perfect match.

2021-01-01 00:00:00 :     Algorithm finished warming up.

2021-01-01 00:00:00 :     Ending values: 24.81340691994228 24.81340691994227
2021-01-05 00:00:00 :     Ending values: 20.09846769446864 20.09846769446866
2021-01-06 00:00:00 :     Ending values: 21.928520834152923 21.92852083415291
2021-01-07 00:00:00 :     Ending values: 18.67634798563535 18.676347985635356
2021-01-08 00:00:00 :     Ending values: 20.532073258109797 20.532073258109797

2021-02-24 00:00:00 :     Ending values: 14.425893287306161 14.425893287306145
2021-02-25 00:00:00 :     Ending values: 14.653224047104208 14.65322404710423
2021-02-26 00:00:00 :     Ending values: 9.792888934334282 9.792888934334298
2021-02-27 00:00:00 :     Ending values: 7.949116381614075 7.949116381614085
2021-03-02 00:00:00 :     Ending values: 11.54412363570464 11.54412363570465

This bring me to conclusion that
    self.EnableAutomaticIndicatorWarmUp = True
does not give enough bars for correct calculation some momentum indicators as MOMP, ROCP, LOGR , etc.

Hi Vladimir ,

This is an interesting problem that takes us to how RateOfChange (RateOfChange is the basis for ROC, ROCP, MOM, and MOMP) is handled in Lean:

where window argument/variable is a RollingWindow object.

We can see that when the window is ready (Samples > Size), it starts to use the MostRecentlyRemoved. It means that the momentum percent with a period of one, in fact, used 2 values, the input and the most recently removed item from the window of size of one. Otherwise, it will use the last available value which is the same as input and the result is zero.

Your main argument is that the historical data request requires RETURN_PERIOD+1:
To get momentum for 1 day, we need 2 bars of data (today and yesterday), one more bar than RETURN_PERIOD.
The same is true if you want a momentum of 21, 63, 126, 252 days.

You are totally right. So why are the results are different? Because the history request includes fill-forward data and the data point of Jan 1st is skewing the results.

Below we can see the original 160-period case starting from March 1st. where mom, mom0, and mom_3 match, but mom_2 doesn't, where:

Below we can see the original 10-period case starting from Jan 1st. where mom_2 matches the indicators in the beginning then mom_3 does after the fill-forward data is no longer in the history request.

Alexander Catarino,

I'm not talking about ROC, ROCP, MOM, MOMP and RateOfChange their basis.
They work correctly if there is enough data (RETURN_PERIOD + 1) at all times.
But about how their output is affected by:

     self.EnableAutomaticIndicatorWarmUp = True

Can you explain how AutomaticIndicatorWarmUp determines the size of the rolling window?
If it is equal or greater (RETURN_PERIOD + 1 in our case), then everything should be correct,
otherwise we need to fix the problem.

Here is backtest of "MOMP Comparison 3" with manual WarmUp
    self.SetWarmUp(RETURN_PERIOD + 1)
with a near perfect match of both indicators  that prove MOMP indicator work correctly.

P.S.
I have change the start date to 2021, 1, 6 to remove fill-forward effect you mention, you can choose
what ever you want but please leave RETURN_PERIOD = 126 and do not change any code related to
RET calculation for comparability

And here is the same code with
    self.EnableAutomaticIndicatorWarmUp = True

Hi Vladimir ,

I have explained the basis of RateOfChange to let QuantConnect Members know its logic and be confident in the calculations. I think it also helps us with the comparison against the alternative approach.

I have created a GitHub issue to address the warm-up issue:
Review RateOfChange IsReady And WarmUpPeriod #5491
because, in fact, it is not requesting sufficient data to warm them up.

The issue was 5 days ago but didn't reply to this thread. Sorry about the wait.