Algo: Flock

Hi Warren,

Thanks for posting this.  My understanding is that using the daily data will give unrealistic results so I made an attempt to convert this to minute data and use a consolidator.  The results were drastically different and it was purchasing less stocks from the look of it.  I was getting numerous data errors when converting to minute and had to add some checks to make sure the securities were tradeable.  I'll look back at this tomorrow and see if I can't get better results.

 

Ignore the above, I found a very large silly error after I posted.

Ok, here is a version with minute data that I think is fairly correct.  It's getting a small number of errors due to data not being available for a symbol, I'm not sure why since I've excluded those symbols when selecting.

Thanks Cole.

I'll try and find some time to work on a minute version as well.

Why does consolidating minute res to daily res affect the overall performance? I might be missing the section in the Docs, but I thought if we were using a lower resolution e.g. Hour, Daily, LEAN was just consolidating Minute data to Hour/Daily data with OnData() as the handler. 

Does minute data account for slippage/fees in a way Hour or Daily isn't?

Hi AK. I think minute resolution uses bid ask data.

Here's a quick attempt at a minute resolution version. I had to reduce the universe size to lower the slippage problem. Drawdown is up and returns are down. Watch out trading this live as is, if a whole bunch of people try and place their orders at the exact same time there are going to be slippage problems. I think limit orders and a position builder might be the way to go. I'll try and write a better version shortly. 

I found the mistake in my version.  The Daily Consolidator doesn't kick in until 9:31 and I was running the rebalance at 9:31 as well.. I changed the rebalance to 9:32 and get similar results to Warren's version.

Thanks for getting back to me. Looks like I'll be refactoring some algos to use minute res then. 

I found a bug in my minute version, looks like it might have been slowing down the backtest. Here's another try.

Out of curiosity, had a quick look at Cole's implementation (preferring this version since I generally distrust Daily resolution for trade simulation).

Keyword summary: Momentum strategy (if I read the code correctly), curve fitted results (if not the worst I've seen).

My test 2016-19 failed miserably with -36% compounding annually. Markets do change, and I think the performance is thanks to the present regime. If one trades something like this is requires an off switch. Occam's razor would see if it can deliver better than random symbol choices at same frequency forward in 2022, though

Hi Petter,

I agree with the off switch. I don't think the results are entirely due to curve fitting given the number of trades made. The problem in my mind is how do you know when to shut it off given that the current market dynamics might be short lived. A simple stop loss on the equity of the account maybe? I've. found quite a bit of this type of short term, high trade count, high return stuff so I need to master the off switch part I think.

Consider the screenshot of one of my price action strategies below. The performance explodes over the last two years. I've seen this with a number of strategies. I'm not entirely sure what is causing it, or how long it is going to last. It's interesting though.

I've this behavior too. I think it mostly has to do with increased volatility. ^VIX on average has been higher since the XIV pop in 2018 and higher still after covid. Options trading has exploded in the last 2-3 years. See squeezemetrics.com/monitor/dix GEX for gamma exposure.

Greater volatility, faster price action, which means that for some algorithms, signals/insights are more frequent than in the past.

Found another bug in my minute version. Here's an update.

Dear Warren

I am a newbie in this field of trading. Hope to learn more. Your “flock” is definitely very interesting and enlightening. Hope you can continue to refine it. 

Also, am interested to know how to use pairs trading in Forex. If you have one to share, it be great.

thanks very much. also thanks to all who had contributed to this discussion. Look forward to more.

joolim

Hi

So I am also a newbie, trying to understand this Flock Algo.  I was learning Python and this is in C#, so my head is spinning.  I got plenty questions.  I cloned the code in above and put some comments in.  The comments with ‘I don’t know' or ‘idk’, mean that this is just a guess of what I think is happening.  If you like, please add more comments.  

My understanding is 

• that the code selects a group stocks
• it gets the difference between each stock and the group.
• I think it is then suppose to sell stocks having the largest positive difference and buy those with the largest negative difference to the group.  I don't yet understand how this logic works or even if it is working correctly.  If you could explain that great.
• it limits the number of stocks held on one day to TotalStocksToHold = 5. (how?)
• it only trades at market open at 9:31?
• There is no stoploss of takeprofit

Here is a brief explanation:

It‘s looping through and normalizing the prices for each stock by dividing each price by the average price.  This gives us normalized numbers around 1 so we can compare stocks with different prices.

It then compares each normalized stock prices to each other stock.  When it does that it calculates the difference between that stocks normalized prices and each other stock.  It sums up those differences for each stock.

The higher or lower this number is tells us if this stock’s returns are outside the norm for the group.  So then we select the top 5 that are the furthest away, higher or lower. Then we short those that are higher and go long on the lower.  This is a mean reversion algo because we expect these stocks to move back towards the average for the group.

I would download visual studio and study Linq/lambdas (the => operator) to help better understand.  It looks confusing at first but it‘s just easy to use for iterative tasks.  

Thank you for sharing your work Warren Harding. What it essentially does it buy low, sells high. Have you tried using ML algorithms in order to be more selective on shorting and longing the stocks, it could help to increase the win%? 

I'm trying to read the C code, so i can try this in python.  Can someone help my understanding of the logic in the psuedo-code below

# universe of 250 stocks
# get last close price for each symbol
# get mean of close price for each symbol over last 5 periods
# divide close price by mean of close price and do this for each symbol - (gives normalized prices, but normalized against the stock's 5 day average)
# short the top 5 and buy the bottom 5 

I'm confused on the logic or concept between: normalizedPrices1 and normalizedPrices2 or even just the difference between stockData1 and stockData2.  Are these shifted 1 period or something?