Share: Crypto Universe Trend Following w/EMA Crossovers

One necessary improvement comes to mind: address the issue where rebalancing causes winning positions to be decreased in size.

A simple example: if we are trading a max of 1 position, and set max exposure to 5% of total portfolio, then to stay balanced, the value of this position would need to be 5% of total portfolio value

However, this position might gain in value, and grow to make up 45% of total portfolio value.  In this case, our rebalancing process would sell much of it, to bring it back down to 5% of total portfolio value.
We don't want that.

Will need to add logic in the rebalancer method to address this. To keep winners winning, we could try excluding their gains from the allocation calculation. 

Here's a first attempt to rebalance more effectively.

In this version, the position size of a winner is not significantly affected by rebalancing. The weighting is adjusted to factor in the winner's gains. Now, allocations are weighed according to the initial cost basis of our portfolio (ie: we wont factor in gains of our current holdings).

Here's a link to the diff. The meaningful changes are in RebalanceHoldings() and SetWeightedHolding() 

This has higher drawdown but over 2x the CAGR, since we don't decrease the size of our winners.

Hey .ekz. ,

Thanks for sharing this, results look good but the drawdown seems to be distributed very abnormal, based on that it seems like there is some big deficiency somewhere in this algo. Mostly the drawdown seems to come from when crypto over all is performing very poorly, which makes sense but does raise a big red flag. Overall the equity curve here pretty much looks like BTC. One thing that would probably be good is a faster response to crypto trending down or some form of hedging since right now this algo is over exposed to crypto doing well.

I've attached a backtest and picture of the BTCUSD chart for the same time period to illustrate what I mean, I know this isn't a perfect example but it does help bring across my point;

Let me know if you already saw this problem and or had anything in mind on how to tackle it.

But again thank you for sharing this this work and I hope my comments were useful to you.

Thanks!

Thanks for the feedback, Mak K .

Firstly: Please go back and use the last version I shared. It addresses an inefficiency with rebalancing and should give you a more reasonable profit (and drawdown) distribution. 

Secondly: Yes, your overall equity curve can potentially mirror that of the strongest asset that was traded the most for that time period (in your case, BTC). For better diversification, try with a bigger basket of Cryptos, by uncommenting the relevant line in the code (the line with more test symbols). Also turn off  momentum weighting, so it uses equal weighting.

Thirdly: Hedging couldn't hurt. You could try adding short positions if you like. But perhaps with a different signal than EMA crossovers. Would love to hear suggestions you have.

Hey .ekz. ! had busy several days so haven't gotten the time to dive into the code, I did play around with some concepts - First regarding your note - 

That is, unless we can rationalize why it is better to use yesterdays prices to make decisions today

Good catch! I can't really rationalize why, and hence it was a bug leading to arbitrarily better results, and should be fixed.

Regarding your current published algorithm, I noticed several things - Sensitivity to fast/slow periods - This is also true to the last algorithms, I think that if we're using several crypto assets, we need to find a way to adapt the periods per crypto asset, probably based on volatility. In the following backtest just changing to 10 and 40 really impacted the results. 

Excessive trading - by removing the weekly rebalancing (maybe adding the check to not rebalancing if we already did at that week will result in similar results) reduced the number of trades and improved overall performance, also reducing the number of assets (to 3) traded had a similar effect. 

Universe selection - If I'm using the full list of assets - performance drops greatly. I think we should look into methods of filtering the assets into the top X assets, and only then apply the crossover only to the top assets. 

Order management - I've added logic to sort by largest weights and make their orders first, leaving the smaller weights to use what's left of the available cash, this reduced the number of invalid trades, and allowed me to use maxExposurePct of 100 (which resulted with favorable performance)

I've attached the results I was able to get to 

Good work Nitay Rabinovich, I love how it's taking shape! My feedback / followups

1) Making the strategy adaptive. 
Indeed, as you mentioned, the indicator period is very subjective to the asset and the current market regime. In the kalman thread, Vladimir  suggested that we adapt the Kalman filter period based on volatility measured by STDDev --see this specific comment. We could try the same with EMA period.

2) Volatility Adjusted Exits
On a related note, last week I implemented a version that uses an ATR-based trailing stop that seemed to perform better and exit earlier. It uses an activation threshold and implements the classic trend following principles of cutting losses early and letting profits run. Here is an image from a recent presentation I gave that describes it clearly. I will clean up the code and share here. 

3) Rebalancing 
I agree that the rebalancing logic can be smarter. When it is turned on it trades too much. I see you spotted see my comment in the code about ensuring we don't rebalance weekly, if rebalancing was recently triggered from a buy/sell. In addition, we could consider other factors for weighting, eg: market cap, volatility or even just a different period for momentum.

4) Universe Selection
Yes! This is a key missing component to trading universes effectively. Ideally i'd like to only trade the top ‘N’ market cap coins.  I know the QC team is working on universe selection, but till then we will have to use an external source. I'm still hunting for a Crypto data provider with an API that allows me to make the call below: If you find one, let me know!

GetTopCoinsByMarketCap(numberOfCoins, historicalDate) 

5) Order management
Thanks for making this change, sorting by weight. Definitely an improvement.  Ultimately we should switch to doing our own math to determine order quantity, and consider placing limit orders instead of market orders (my general rule of thumb for trading strategies, but may not be necessary for mid-term trend following)   

6) Max Exposure
“this reduced the number of invalid trades, and allowed me to use maxExposurePct of 100”
You are a braver man than I am, haha. I will probably go live with at most 50% :)

Great read / breakdown of the Turtle Trader trend following system, of which I have always been a big fan. 

He goes into detail on how they size each position size based on each asset's volatility, which I think, combined with volatility-adjusted exits, gives a complete adaptive system that will perform well.

That said, I will probably replace the volatility adjusted sizing with *noise* adjusted sizing, since volatility can also be a good thing, but noise can put a real damper on returns. I'd use the Kaufman effeciency ratio as the noise metric.

What do you all think? 🤔

And here is the article: 

http://raposa-website-production.herokuapp.com//blog/testing-turtle-trading-the-system-that-made-newbie-traders-millions/

When I try run the last two share algos here from .ekz and Nitay, they both throw the similar errors.  I just clone the code and run it and don't modify anything and get errors like

Runtime Error: Trying to retrieve an element from a collection using a key that does not exist in that collection throws a KeyError exception. To prevent the exception, ensure that the  key exist in the collection and/or that collection is not empty.
  at ExitPosition
    self.SelectedSymbolsAndWeights.pop(symbol)
  File "main.py" in main.py: line 296'

I go into the code and try correct the error, but to no avail.  On ‘.ekz’ I tried the following fix,  It fixed the problem but the algo jsut completed its run with no further trades.  Probably I'm not fixing the actual cause - any ideas?  I will look at it more tonight.

    def ExitPosition(self, symbol, exitMsg=""):
        profitPct   = round(self.Securities[symbol].Holdings.UnrealizedProfitPercent,2)
        self.Liquidate(symbol, tag=f"SELL {symbol.Value} ({profitPct}% profit) [{exitMsg}]")
        #check if the symbol is in the list before popping it
        if symbol in self.SelectedSymbolsAndWeights:
            self.SelectedSymbolsAndWeights.pop(symbol)
            ## TODO:
            ## Before go-live, wait until liquidation has happened before rebalancing
            ## Perhaps Call RebalanceHoldings after an order event has occured. 
            self.RebalanceHoldings()
        return

Hi

Well I the code is not working.  Weird that it worked previously for people and no longer does.  I put in a default value of ‘0’ or ‘None’ for the pop, and the code then works, but does no further trades.  The rebalanceing does not make sense to me, it seems to buy and sell the same symbol mulitiple times?  So very dissapointing - still good learning experience, though I am becoming more and more disillusioned with the whole setup here at QuantConnect.  All this work to try get a backtest working and never to get anything in production?  Whats the point? 

Hi .ekz

That link to the article, the site has taken the article down.  Any other description available?  Thanks

Hi Stephen,

Looks like raposa revamped their website. Google took me to an updated URL: 

On the code, At first glance it looks like it may have to do with recent changes in how we crypto is liquidated. It seems that Self.Liquidate is leaving trace amount of coins in the wallet. When this happens, we are technically still invested in those symbols, even though we have removed them from our ‘SymbolsAndWeights’ dictionary, where we track our holdings. See the attached backtest.

This misalignment of holdings (between our tracking and the portfolio) is causing the issue.  Alexandre Catarino please opine. 

I can take a crack at a fix later this week. 

[Deleted Duplicate]

It works if these two lines are commented out in your code .ekz  

        #if(symbol in self.SelectedSymbolsAndWeights.keys()):
       #    self.SelectedSymbolsAndWeights.pop(symbol)

.ekz.  I found that leaving trace amounts of crypto during trades is also an issue with other crypto trading platforms. Binance for example also provides a feature to convert these small amounts into BNB. 

To solve in issue in the algorithm I would determine if a crypto is invested if a meaningful amount is invested (e.g., >10 USDT):

self.Portfolio[symbol].Invested and self.Portfolio[symbol].Quantity * self.Securities[symbol].Price > self.minAmountToBeInvested

Thanks for the followups Lukas Eichler  and Jens Kutschera .

Jens: The issue with this is that we actually need to pop those elements from that dictionary when we exit the positions, otherwise they are factored into the rebalancing, when they shouldn't be. This would result in unexpected behaviour.

Lukas: This is a better work around, and can give more predictability but ideally we want to be fully divested, and have zero holding.

This issue seems to be happening because of the way LEAN calculates the quantity to be sold when we call Liquidate(). My hunch is that it has to do with the fee of the brokerage model, but I'm not certain. 

I believe the QC team is looking at this though, and we should hear from them soon. 

HI

Lukas Eichler  welll done - the code seems to work.  

.ekz.  the code has comments like 

## TODO: 
## For Go-Live, call OnNewPositionOpened only after 
## the order is actually filled

I guess these would have to be done by webhook notifications ?

Hi Newoptionz ,

Fortunately we don't need webhooks --there are order events that we can listen for.  Look at docs or search the forum for usage of OnOrderEvent(…)

This gets called anytime an order event occurs. Eg when it gets submitted or filled.

Hi Newoptionz , Jens Kutschera , Lukas Eichler :

I can confirm that Alex (@QC) actually did indeed file a Git issue for this.  He also included a description that explains what is happening with the fees.

You can subscribe to it for updates via the link below. Once the fix is merged, this should no longer be an issue.

Thanks Alexandre Catarino and team.

.ekz. Thanks for the update

I tried building a marketcap based universe selection that updates the symbol list every week and the results are way worse. I'll spend some time this week trying to figure out how to improve the performance on this.