Andreas F Clenow Momentum

Hi Leigham,

I cloned your algorithm and changed the following parameters:

    self.SetStartDate(2006,01,01)  
    self.SetEndDate(2018,01,01)    
    self.SetCash(10000)        

I get below error when I backtest the algorithm:

Runtime Error: Python.Runtime.PythonException: ValueError : zero-size array to reduction operation maximum which has no identity
at Python.Runtime.PyObject.Invoke (Python.Runtime.PyTuple args, Python.Runtime.PyDict kw) [0x00033] in <59c711440fed482b8e57b026917e4706>:0
at Python.Runtime.PyObject.InvokeMethod (System.String name, Python.Runtime.PyTuple args, Python.Runtime.PyDict kw) [0x00007] in <59c711440fed482b8e57b026917e4706>:0
at Python.Runtime.PyObject.TryInvokeMember (System.Dynamic.InvokeMemberBinder binder, System.Object[] args, System.Object& result) [0x0003e] in <59c711440fed482b8e57b026917e4706>:0
at (wrapper dynamic-method) System.Object:CallSite.Target (System.Runtime.CompilerServices.Closure,System.Runtime.CompilerServices.CallSite,object,QuantConnect.Data.UniverseSelection.SecurityChanges)
at QuantConnect.AlgorithmFactory.Python.Wrappers.AlgorithmPythonWrapper.OnSecuritiesChanged (QuantConnect.Data.UniverseSelection.SecurityChanges changes) [0x0004f] in <7ab68f0d8e294fb99b0d6c17956078e8>:0
at QuantConnect.Lean.Engine.AlgorithmManager.Run (QuantConnect.Packets.AlgorithmNodePacket job, QuantConnect.Interfaces.IAlgorithm algorithm, QuantConnect.Lean.Engine.DataFeeds.IDataFeed feed, QuantConnect.Lean.Engine.TransactionHandlers.ITransactionHandler transactions, QuantConnect.Lean.Engine.Results.IResultHandler results, QuantConnect.Lean.Engine.RealTime.IRealTimeHandler realtime, QuantConnect.Lean.Engine.Server.ILeanManager leanManager, QuantConnect.Lean.Engine.Alpha.IAlphaHandler alphas, System.Threading.CancellationToken token) [0x00a96] in <34f68d5d9ef54d4e95213133d16a078b>:0

And this is the stacktrace:
 

[' File "../cache/main.py", line 106, in OnSecuritiesChanged
', ' File "../cache/main.py", line 137, in gapper
', ' File "/usr/local/lib/python2.7/dist-packages/numpy/core/fromnumeric.py", line 2272, in amax
out=out, **kwargs)
', ' File "/usr/local/lib/python2.7/dist-packages/numpy/core/_methods.py", line 26, in _amax
return umr_maximum(a, axis, None, out, keepdims)
']

The runtime error occurs on May 4, 2006 so you don't have to wait very long when you start the backtest.

Any ideas?

Thanks,

Jon
 

I haven't found a resolution to that bug as of yet, but I'm going to look into it, if you find the solution please make sure to let me know and I'll do the same :) 

That error sounds like a maximum operation is trying to reduce an axis with length zero inside the function gapper. On return, inside gapper, I noticed use of np.max. More importantly, I noticed History being used.

I knew right away it was a case of History not retuning the expected data for an asset; I often see this with dynamic universes. This was combined with use of np.diff, which returns Length-1 for an array, so enough(2) data points were required. I added some checks.

NOTE: I didn't review the trade logic. I'm not sure I returned the right thing in gapper when I force a return on a bad History. Code change in ###'s for review. Good luck!
 

Sample log of bad asset which returned under 2 close values:

Thanks for the help Derek TishlerI'm going to look over this tommorow :). 

It looks like Derek fixed the errors in May 2006 but like he said, not sure whether doing a "return" is the right thing to do.  Stumbled across another runtime error on May 22, 2008 and it's as follows:

My parameters were:

Using your same parameters and slightly changing the code to return a True statement, and it seems to be fine :). Please let me know if you can still clone the code and find other errors. To be clear the logic behind why I'm returning True when there isn't enough data is to determine whether or not the security gapped in the past 90 days because the strategy wants to avoid stocks if It can't know for certain whether or not it gapped. By returning True this line below will come out false. 

if self.Securities[stock].Price >self.moving_average(stock,100) and not self.gapper(stock,90) and cash >0 and not oo:

^^^ That ensures the return logic is doing 'the right thing' :p

As for the most recent error:

invalid value encountered in sqrt

^^

We simply won't encounter it now because the slope won't be calculated for securties that lack the data required calculate the slope. 

And we won't trade that security because we only want smoothly uptrending securities. Below is a backtest for the timeframe you stated you found the bug. 

Fixing my code to 'return 0' instead of nothing prevented a crash or other errors for for me. I went through and applied some other changes to the code to:
-apply fix to any use of History more generally(also ended up doing a period check)
-update how stocks_to_trade is managed to ensure addition AND removal of assets from universe
-added a leverage plot in order to track margin issues
-set realism setting to IB margin account
- used pandas/numpy where possible vs list comprehension to "unpack"
- divide by zero check in weight() to avoid error

Nice work Leigham Springer Sutton 

Its a shame these strategies take so long to backtest on QC. Following the Trend is next !

Thank you Lexx7 ,  I would love to code that strategy ableit I haven't bought that book as of yet if you'd like to tell me the precise buying/selling/ranking information I'll give it a go :)

Working on it Lexx =P.. thanks for your patience. 

Hi Leigham,

I have read the book and trying to simulate the same strategy. I also came across this post:

I believe you should select a new universe every time you rebalance your position and not in the init only. I could be wrong.

I would like to validate every function that is code. How did you debug them? Did you use a notebook for that?

Do you like to work together to see if we can get the same result as given in the book? Are you interested in reading the book about futures trading as well? Seems that this stock momentum strategy is derived from futures momentum trading. That is my second target to code. I can code in Python or C#.

J.

Thanks Jared - moreso my 'impatience' haha but I am happy with the support & growing community interest so far & will soon start my paid account once my strategies are fine tuned, keep up the good work !

Leigham, the rules are simple. For a portfolio of futures tickers (continuous), if the 10 day SMA crosses above the 100 day SMA then buy with ATR based position sizing similar to stocks on the move. I've attached a backtest that I started playing around with but I dont think anything works there but it may help as a skeleton. 

The position sizing formula should be : (risk factor x portfolio value)/(ATR100 x Point Value)

Hi,

I managed to implement the algorithm. I spent too much time thinking of keeping performance low. I put the securities updates in the the securitiesonchanged and not in the ondata as it won't run. I did put a small period back test of 1 year. Calculating for 500 stocks every wednesday based on the S&P500 with slope takes considerable time. I hope to show some basis principles like getting a stock 500, register indicators and rebalancing it on wednesdays. The portfolio risk part is to be done. Input is welcomed. This one calculates the slope every wednesday, I am not sure the Python version does, or what Andreas Clenow says about that. I will check the book.

I would like to get some input on how to reduce the risk of a sudden market drawndown and how to implement the initial sizing for the first 20 or 30 stocks to buy. With ATR it takes for low ATR a lot of stock and for high a lot of stock, but it does not take into account the position size of the overall portfolio which must consist more or less of 20-30 stocks. Maybe just program it that way and a maximum based on the ATR, or is it normal to take leverage on a low ATR, but that will call in margin.

If there are European traders, I would like to trade EUR as do not like to put all my money in the USD. Or does somebody know how to hedge this properly?

Currently I am papertrading this one and I want to go live trading it. So any flaws, please let me know.