I am sharing a few thoughts on risk management I had while developing trading algorithms for myself and my clients. Keep in mind my focus is mostly on Machine Learning based algorithms but these considerations are general enough apply also to all quants and even discretionary traders.
Most of the quants, or even traders, have the tendency to jump to the “sexy” part of the strategy definition, namely, using exotic datasets, crunching data via traditional analysis or Machine Learning to generate trading signals. Hours and resources are poured into squeezing the next 1% accuracy out of your signal generating model without acknowledging the elephant in the room: What happens when your signal is wrong?
This is not a hypothetical question, your model will be wrong at some point, maybe in a small way every day or in a big way during crashes. Therefore it is critical for you to adjust your trading framework accordingly.
I advocate for flipping the sequence of activities in trading algorithms’ development from:
- Analyze data and generate signals ideas
- Develop an algorithm to generate trading signals
- (Back)Test the trading signal
- Add a risk management layer
To:
- Define what is risk for you and how much you are willing to take
- Develop an infrastructure that targets the desired level of risk
- (Back)Test the risk infrastructure in the worst possible scenario (I like using a random model)
- Add a signal generation layer
While the concept of risk is very slippery (and its measurement even more), this change of perspective is certainly useful. You stop thinking that your signals are flawless and you start building contingency for when things will go against you.
To make things more practical you can find below a short algorithm that trades daily the SPY and that implements these concepts:
- A risk management framework: In this case I have used the Kelly Criterion which, together with the next day’s Up/Down prediction, defines the size of the position for that day.
- A signal generating model: Since the focus is on testing the risk management, I am using a DummyClassifier generating random 0/1 trading signals. I want to see how the algorithm behaves when a blindfolded monkey is on the steering wheel.
As you can see, the performance is by no means exceptional but, even with a totally random trading signal, the risk management still achieves reasonable returns and drawdown while squeezing out even a tiny alfa.
This is enlightening on the importance of this often overlooked component and my recommendation is to find your preferred risk management mechanism and test it under different conditions (tickers, asset classes, resolutions, …) with very bad or even random signals. In this way you can refine your own risk management layer so you can confidently release your real trading models.
Francesco
www.beawai.com
P.S. The considerations are inspired by several readings across Finance, Math and sometimes even Philosophy. I am sharing a few of these should you want to dig deeper:
Brandon Leonard
The longer I trade, the more this makes sense.
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Axist
This is really cool, and I could see being useful when combining with an In/Out or momentum based algo. Took a crack at trying to make this work for multiple tickers at once. While I am not getting an error, I think there is an issue with training multiple tickers? Curious if anyone's gotten it to function with a list:
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Jens.
Hi,
I really like the idea to use a random signal and I stumbled upon that myself. Thank you for your post and example. I figured that different monkeys (=different fixed seeds) get significantly different Sharpe Ratios. So I recommend employing a monkey gang to stress-test!
For me, risk management typically costs performance especially in the recent bull phase, but also makes the results more independent from the market. Do you have the same observation?
I will probably play around with your implementation to find out for myself, and post my assumptions later on.
Jens
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Francesco Baldisserri
Hi Axist,
You can definitely make it work with multiple tickers but the critical part is that, where you calculate the Kelly Criterion, you need to calculate the past win rate and returns for your trading strategy. I took a look at your code and the In/Out is used in the trading part but the Kelly Criterion is still being fed with the results of the Dummy Classifier which is never used to trade.
If you want to use the Kelly Criterion you need to calculate the signals for your In/Out strategy for the last X days and the related wins and returns.
Hope it helps!
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Francesco Baldisserri
Hi Jens. ,
You are totally right that a single backtest has limited information. I have made a few tweaks and added the parameter “seed” and “use_kelly” (0/1) so those parameters can be controlled and I ran a grid search over those.
Unfortunately I cannot share the grid search results like the backtests (feature request😊) but I am attaching one backtest with the new code, the screenshot of the grid search and the results in an excel (link).
Clearly this topic requires much more thought and analysis (the problem of induction has been debated for millenia) but the parameter “use_kelly” is positively correlated with the PSR, the Alfa and negatively correlated with Beta, Drawdown and Trades (to be fair it also has lower profit and annual returns, probably due to the limited trading during high risk situations).
PSR23%Sharpe Ratio-10%Net Profit-33%Drawdown-63%Total Trades-99%Alpha26%Compounding Annual Return-25%Beta-99%Annual Standard Deviation-99%
Thanks for the input and let me know if you have more insights!
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Spacetime
AlMoJo ,
Also, if you would like to explore more CBOE data sets, then you can use the Nasdaq Data Link provided below.
Correction in my last post: meant to say CBOE and not Quandl (Quandl was acquired by Nasdaq)
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Spacetime
AlMoJo , Just posting our QC CBOE Datasets link below for your reference if you need it.
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Vladimir
Spacetime,
There are only 6 trades for SHV in your version of the code.
Can you try changing in the code the VIX and VXV data retrieval in a way like in the attached sample.
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Spacetime
Hi Vladimir,
Hmm… not entirely sure why there were only 6 trades executed.
The above backtest which I have attached was taking a “bit” to complete for me and I was engaged with other work, so I ran the above model (shaerd by pangyuteng) with a start date of self.SetStartDate(2020, 11, 1) just to speed up things.
Hmm… I have extended the date range to start from self.SetStartDate(2019, 1, 1) and it does not have too many trades either. (backtest attached)
If you have noticed from the above example shared by pangyuteng , then it does not trade that often either. [ self.SetStartDate(2019, 1, 1) ]
But, the fixes which I have shared above does work as the error which was being received by AlMoJo is no longer occurring.
Hmm… maybe someone else might have a better idea on this, but I will try to look into this further. Hmm… maybe the LSTM configuration needs to be looked into or the volatility data resolution and other resolutions needs to be changed to minute time slice if possible.
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Vladimir
Spacetime,
I backtested cloned pangyuteng algorithm with only changes in my_custom_data
url_vix = "http://cache.quantconnect.com/alternative/cboe/vix.csv"
url_vxv = "http://cache.quantconnect.com/alternative/cboe/vix3m.csv"
It generated 13 SPY-SHY trades and metrics as your last one, but the performance is not comparable to pangyuteng's results.
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Spacetime
Vladimir,
Hmm… If you want to compare it with pangyuteng's results, then you have to provide an end date of self.SetEndDate(2020, 1, 1) because pangyuteng's model ends around that time. Please note, pangyuteng's post is time stamped somewhere in December 2019.
Try to run it with end date parameter and check to see if performance is comparable.
I have attached a backtest run starting from self.SetStartDate(2019, 1, 1) and ending at self.SetEndDate(2020, 1, 1) and it only traded once.
Hmm…
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Vladimir
Spacetime,
I definitely like the metrics from your last backtest, but my attempts to reproduce them for some reason were unsuccessful.
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
AlMoJo
Hello everybody,
Pangyuteng this is an amazing result. Jack Simonson I also think technical indicators would help a lot.
I tested Pangyuteng version on the 2022 crash and it is not so good at dodging it. I taught maybe going hourly timeframe instead of daily could help even if I know that trading more frequently is not really good.
Anyone has an idea on how to change the timeframe? I tried to pass all the .Daily to .Hourly but obviously it wasn't that simple.
Kind Regards
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Varad Kabade
Hi AlMoJo,
To change the timeframe of the entire algorithm we recommend adding the following to the Initialize method:
Note that the algorithm uses higher frequency bars to create lower-frequency bars. Therefore to use indicators or consolidators at the hourly resolution we need to have the universe at the minute, seconds, or hourly resolution.
Best,
Varad Kabade
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Will Berger INVESTOR
Hey Guys and Sherry,
Thanks for this incredible thread. So many interesting ideas!
Will
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
To unlock posting to the community forums please complete at least 30% of Boot Camp.
You can continue your Boot Camp training progress from the terminal. We hope to see you in the community soon!