Risk management and blindfolded monkeys

I am sharing a few thoughts on risk management I had while developing trading algorithms for myself and my clients. Keep in mind my focus is mostly on Machine Learning based algorithms but these considerations are general enough apply also to all quants and even discretionary traders.

Most of the quants, or even traders, have the tendency to jump to the “sexy” part of the strategy definition, namely, using exotic datasets, crunching data via traditional analysis or Machine Learning to generate trading signals. Hours and resources are poured into squeezing the next 1% accuracy out of your signal generating model without acknowledging the elephant in the room: What happens when your signal is wrong?
This is not a hypothetical question, your model will be wrong at some point, maybe in a small way every day or in a big way during crashes. Therefore it is critical for you to adjust your trading framework accordingly.

I advocate for flipping the sequence of activities in trading algorithms’ development from:

Analyze data and generate signals ideas
Develop an algorithm to generate trading signals
(Back)Test the trading signal
Add a risk management layer

To:

Define what is risk for you and how much you are willing to take
Develop an infrastructure that targets the desired level of risk
(Back)Test the risk infrastructure in the worst possible scenario (I like using a random model)
Add a signal generation layer

While the concept of risk is very slippery (and its measurement even more), this change of perspective is certainly useful. You stop thinking that your signals are flawless and you start building contingency for when things will go against you.

To make things more practical you can find below a short algorithm that trades daily the SPY and that implements these concepts:

A risk management framework: In this case I have used the Kelly Criterion which, together with the next day’s Up/Down prediction, defines the size of the position for that day.
A signal generating model: Since the focus is on testing the risk management, I am using a DummyClassifier generating random 0/1 trading signals. I want to see how the algorithm behaves when a blindfolded monkey is on the steering wheel.

As you can see, the performance is by no means exceptional but, even with a totally random trading signal, the risk management still achieves reasonable returns and drawdown while squeezing out even a tiny alfa.

This is enlightening on the importance of this often overlooked component and my recommendation is to find your preferred risk management mechanism and test it under different conditions (tickers, asset classes, resolutions, …) with very bad or even random signals. In this way you can refine your own risk management layer so you can confidently release your real trading models.

Francesco
www.beawai.com

P.S. The considerations are inspired by several readings across Finance, Math and sometimes even Philosophy. I am sharing a few of these should you want to dig deeper:

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.

This is really cool, and I could see being useful when combining with an In/Out or momentum based algo. Took a crack at trying to make this work for multiple tickers at once. While I am not getting an error, I think there is an issue with training multiple tickers? Curious if anyone's gotten it to function with a list:

Brandon Leonard INVESTOR

Invest Club | December 2022

The longer I trade, the more this makes sense.

Upvote

Axist INVESTOR

Axist Capital | December 2022

Jens. INVESTOR

Jens Kutschera | December 2022

Hi,

I really like the idea to use a random signal and I stumbled upon that myself. Thank you for your post and example. I figured that different monkeys (=different fixed seeds) get significantly different Sharpe Ratios. So I recommend employing a monkey gang to stress-test!

For me, risk management typically costs performance especially in the recent bull phase, but also makes the results more independent from the market. Do you have the same observation?

I will probably play around with your implementation to find out for myself, and post my assumptions later on.

Jens

Francesco Baldisserri INVESTOR

BeawAI | December 2022

Hi Axist,

You can definitely make it work with multiple tickers but the critical part is that, where you calculate the Kelly Criterion, you need to calculate the past win rate and returns for your trading strategy. I took a look at your code and the In/Out is used in the trading part but the Kelly Criterion is still being fed with the results of the Dummy Classifier which is never used to trade.

If you want to use the Kelly Criterion you need to calculate the signals for your In/Out strategy for the last X days and the related wins and returns.
Hope it helps!

Hi Jens. ,

You are totally right that a single backtest has limited information. I have made a few tweaks and added the parameter “seed” and “use_kelly” (0/1) so those parameters can be controlled and I ran a grid search over those.

Unfortunately I cannot share the grid search results like the backtests (feature request😊) but I am attaching one backtest with the new code, the screenshot of the grid search and the results in an excel (link).

Clearly this topic requires much more thought and analysis (the problem of induction has been debated for millenia) but the parameter “use_kelly” is positively correlated with the PSR, the Alfa and negatively correlated with Beta, Drawdown and Trades (to be fair it also has lower profit and annual returns, probably due to the limited trading during high risk situations).

PSR23%Sharpe Ratio-10%Net Profit-33%Drawdown-63%Total Trades-99%Alpha26%Compounding Annual Return-25%Beta-99%Annual Standard Deviation-99%

Thanks for the input and let me know if you have more insights!

Nathan Swenson INVESTOR

December 2020

Getting out of the Market early is more of a conservative move rather than aggressive such as going to war. I think the point of it is to take the least risk possible. The outer 1% of std deviation if a very high standard to meet, so I can understand why Peter made it that way. If you want multiple confirming signals, then you likely can't use 1% outliers.

The In and Out does very well with 3x leveraged funds because it is overly cautious, generally exiting the market too early, but safetly for the most part. So while you don't get all of the move, you could perhaps take greater risk for the shorter period you are in. The "jitter" from only 1 signal appears valid as the Out holding have done well, at least in the In sample data we've tested. That being said, it's difficult to watch this market zoom higher while sitting in Out holding since 10/6. In reality this is our first real "Out of Sample" data and it's not looking good so far but who knows what happens in the coming weeks. Everyone is predicting all time highs. We shall see.

Goldie Yalamanchi INVESTOR

Removing SHY from 2020 makes 2020 and possibly 2020+ trade normally and perform normally -- until such time as we can comment back in the SHY indicator.

If we are trying to "ace" the backtest then keeping SHY in always looks good until of course late 2020 when the algo stops trading in October.

But credit where credit is due... T Smith multiple signals (5 of 8) approach still did well with the Qual-Up universe approach from 2014-2018 as well. During those years by commenting out SHY in the original IN/OUT algo, the Qual-Up stocks didn't do well i.e. QQQ would have done well regardless during 2013-2018 because tech has been on a tear the whole past decard.

Vladimir INVESTOR

Here is updated DUAL MOMENTUM IN OUT v2.1

I have changed line 97 to:

prices = self.History(symbol, period + excl, Resolution.Daily).close

2 people upvoted this

EllaHamilton INVESTOR

Thx, nice one.

Guy Fleury INVESTOR

@Vladimir, I go with Nathan's explanation. The strategy goes to the sideline at the first sign of trouble. You do not want to wait for a consensus since you are already dealing with ETFs.

Trading QQQ is like trading a market average surrogate. It holds the same shares as the NASDAQ 100 index as everyone knows. The top ten holdings (AAPL, MSFT, AMZN, TSLA, FB, GOOGL, GOOG, NVDA, PYPL, ADBE) account for 55% of QQQ. It should be view as one of the easiest stock selection you can make. Playing QQQ tends to dampen overall volatility. While using TQQQ puts volatility back into play and at a higher level with an expected beta of 3.0x. Therefore, you are playing QQQ on steroids which evidently brings in higher risk. The reason for “extreme” caution even if there is a cost to it.

This is not a game where we will fix things after we lose. So, we should first play safe whatever the performance level we are at. We might need to compromise like playing this strategy at a higher level but with other strategies in order to reduce overall volatility and drawdowns, or only use part of the available capital (say 10 to 20% as if on riskier assets).

In my previous post, the point was made that you could drop some of the signal components (3 out of 4) and it would increase overall performance. Well, here is another point of interest: self.INI_WAIT_DAYS. I see its use as a way to reduce whipsaws around the moving average crossovers. The original code has it at 15 trading days. No one questioned this as it was a reasonable assumption since there are indeed a lot of whipsaws near those crossovers. Removing it, for instance, making self.INI_WAIT_DAYS = 0, dropped performance considerably and thereby justified its use.

In my version of the program, if you set it to zero, you get a 62.68% CAGR. If you keep it at 15, you have a 97.84% CAGR. If you set it to 10, you get about the same result (97.82%). However, if you set it below 5, something like 2 or 1, you improve the picture considerably. The economic reasoning is simple. The wait days operate on a high decay function: e^(-0,5t). It might also suggest that whipsaws fade away rather fast near the crossovers. Also, by reducing the wait days, you are increasing the number of days the strategy is fully invested.

The table below shows the evolution of the strategy where only the wait days are changed from 15 to 0. I think that the chart speaks for itself. Changing a single number in the program can have a tremendous long-term impact. Note that this is close to a 6000% improvement going from zero wait days to one. Nothing else in the program was changed for these tests.

Guy Fleury,

Looks like I saw a spreadsheet like above two months ago.
The only difference was Quantopian instead of Quantconnect.
But that not optimized strategy had a completely different decision-making structure:

-Consensus of individual signals.
-Far less degree of freedom.
-Three times fewer sources of information.
-Three times fewer variables.
-Static parameters.

Something like the one below.
In terms of total return, it exceeds the latest In_out_flex_v5 2020-12-16

BTW: What will be your decisions?

Frank Schikarski INVESTOR

Hi there,

some comments regarding the trigger for in or out:

if (extreme[self.SIGNALS + self.PAIR_LIST]).any():

There are several "regime scouts" that watch out if they observe danger. The beauty of this algo is, that danger is beeing memorized for some days.
They indicate danger, if they observe something extremely unusual, which is less likely than "np.nanpercentile(mom, 1, axis=0)" = lower 1% percentile of the data.
If there are "self.lookback = 252" observations, this roughly translates into less than 2.52 observations, or: the current signal observation is the worst or the second worst within the time period. This is a bit digital = not smooth.
This leads to the algo beeing very sensitive to not only the 1% but also the 252 observations which should not be the case. So we need more observations and then we can improve both the 1% treshold and the number of "regime scouts" theat we want to use in our algo.
As these are only "regime scouts", which only trigger our real algo logic e.g. "long 100 largest companies on Nasdaq" or anything else, I don't consider tuning these more or less independent scouts as overfitting.

What if we would (a) keep calculating daily returns for our signals, but (b) do this for every hour with a rolling 24-hours window? This should result in 24 times more observations = increase our resolution, allowing to optimize the 1%, the lookback period and increase the "any" until we get some redundancy from our scouts. Keep exploring ;)...

1 person upvoted this

Here is the updated DUAL MOMENTUM IN OUT v2.2
-Based on In_out_flex_v5_try.
-Used exponential like smoothing on line 116-121
-Line 120 is commented out.

@Vladimir, yes, as you say: "...the strategy had a completely different decision-making structure". You improved on that strategy design since... thanks.

Changing the number of wait days (self.INI_WAIT_DAYS) in the program is more like an administrative decision. The idea is not bad since we know there will be some whipsaws at crossover times. However, there was no need to wait more than one day or maybe two at the most.

It is not that surprising an observation. We want security, be decisive, and not be clobbered by added trading costs due to whipsaw after whipsaw for days after an exit, and yet, this says do wait but at most one day and probably no more.

Such a small decision with such an impact. You change a single number in the program from 0 to 1 and it increases performance by 5910%!

BukavuTrader INVESTOR

Ntale Lukama | December 2020

@Vladimir, Do you have one like that for FOREX?

BukavuTrader,

Do you have one like that for FOREX?

Not yet, but you can try yourself.

Some added notes. This trading strategy has shown that it could go quite far depending on some of its parameter settings, ETF selection, trading logic, and initial capital. Using 10k, 100k or 1 million is an administrative decision. The program will do its job either way since it is scalable (but up to a limit). It is a simple bond switcher based on QQQ, but it has interesting properties.

The max drawdown and overall volatility will be the same with either capital options. All you will be changing is the ongoing bet size. This will barely change the price at which a trade is executed. But will change the traded quantity. Increasing the capital ten-fold will increase the bet size 10-fold, and in turn, increase profits (losses) 10-fold. However, going for 10 million as initial capital will tend to make the strategy unfeasible since you might end up trading 175,000,000 shares of TQQQ on practically a weekly basis which is more than the average daily volume. So, there are practical limits to the strategy which will need to be addressed.

With 100k you can push the strategy beyond 1 B and with 1 M you can pass the 10 B mark. Almost incredible. However, this is achieved by taking on more risk, using 3x-leveraged ETFs which are also leveraged at 1.4x. Thereby pushing on the machine way beyond what the original design was. Of note, changing the wait days (self.INI_WAIT_DAYS) to 1 had a tremendous impact on overall performance, a real game-changer, and yet, just another administrative decision.

I have not touched risk reduction procedures yet. This comes at a later stage in my testing process. It is expected that by installing protective measures the overall performance will be reduced to some extent. But, I will know that after those measures are added. Meanwhile, I have other tests to make.

My version of this program is dealing with a 3.x leveraged QQQ surrogate (TQQQ). It is playing an index tracker but with 3.x the average market beta saying it swings more than QQQ which is itself an average market consensus equivalent.

In pushing further, the strategy reaches a performance plateau from which it starts breaking down. It does not blow up mind you. It simply trades less and less and thereby generates less and less suggesting not to go that far. But that should be expected. Knowing that the strategy has seen its own built-in structural limits, it is almost time to apply protective measures and scale it down to a more acceptable risk/reward level.

Here is my take. You PLAY the game for its long-term CAGR potential. Which trading methods will give you the highest return within your own trading constraints? Not somebody else's, but your own. We need to answer the question: will we accept 5% more on a temporary max drawdown for 5% more in CAGR? The decision has value and is based on the initial stake:

10k ∙ (1+0.30)^20 - 10k ∙ (1+0.25)^20 = 1,033,135

100k ∙ (1+0.30)^20 - 100k ∙ (1+0.25)^20 = 10,331,346

1M ∙ (1+0.30)^20 - 1M ∙ (1+0.25)^20 = 103,313,464

This should weigh in the evaluation of your acceptable risk/reward scenario. It can be a costly decision.

3 people upvoted this

Here is the updated DUAL MOMENTUM IN OUT v2.3

Based on In_out_flex_v5_disambiguate_v2.

Simone Pantaleoni INVESTOR

LesFlex Quantitative Nerd | December 2020

Great idea Vladimir! I was working on a similar update on the INOUT algo, but you anticipated me! :P

Have you also tried to decrease further the "decay" value for the SELF.WAIT_DAYS variable, reducing the waiting to increase sharpe and return? (guess probably yes, isn't it?)

Carsten INVESTOR

Carsten _ | December 2020

Vladimir as you requested.. :) was a bit trick, just happy to get it as a multi AlphaModel running. Its a super simplified version, but you can easily upgrade it. At the end it has much more lines than the normal version. It was quite trick to get the signal into the two AlphaModels. At the end I used ObjectStore. If someone has a simpler solution, with a global variable? please comment.

@Vladimir, I like the behavior and equity line of version 2.3. Remarkable, and great numbers. I will try to find some time to look at it since I think there are things I will learn in the process. Thanks for sharing.

Damiano Bolzoni INVESTOR

Guys, I really fell in love with this strategy (I actually started following the thread on Quantopian) and so ran some additional backtests taking into consideration several 5-year periods.

The strategy really shines during the 2008-2012 timeframe and then again in 2020. That's how it delivers 30% annual return. Take any other period of time and it will barely matches the returns of holding QQQ: I literally just finished a backtest between 1-1-2013 and 12-31-2019 and it's underperforming by nearly 10% overall.

If one substites QQQ and FDN with SP500 equivalents the same behavior can be observed, actually returns are even worse.

Am I the only one experiecing this?

Just tweeking the Waiting variable using a bigger decay, as suggested above to get slightly better returns and sharpe :)

Vladimir could you plese check again, should work now, the objektstore object was not created in the initialize, but it was yesterday on my disk as i was finding out how to impement it....

Jack Pizza INVESTOR

Sum Sum | December 2020

FYI to make this more robust these same arguments were brought up in the old QT thread.

Not sure if this is implemented in this or not.

There should be a 3rd option or ultimate out where it just goes into cash or adding gold as a 3rd / 4th asset to rotate into.

Given at some point in time stocks and bonds might breakdown together.

Platform

Radically Open-Source Algorithmic Trading Engine

Join Our Discord Channel

Quarterly Open-Source Trading Competition

Draft Discussions

Bookmarked Discussions

SEARCH DISCUSSIONS

367,000 Quants.

VOTE FOR UPCOMING FEATURES

JOIN OUR Community MAILING LIST

IN THIS RESEARCH

PARTICIPANTS

Discussion Awards

Actions

Join QuantConnect for Free