The In & Out Strategy - Continued from Quantopian

No, they do not, Chak 😊 You can use them on a normal distribution, yes, but they mean the same thing, no matter what the distribution is. For example the 1% percentile means the 1% left-most extreme observations … this is true in any distribution, not only a normal distribution.

https://numpy.org/doc/1.20/reference/generated/numpy.percentile.html

Moving forward, I'm suggesting that perhaps using np.percentile has some limitations to finding accurate outliers.

Hi Chak , now I don't know exactly what data you are talking about, I know how it works in & out with the percentile. But from what I understand from the discussion, taking a certain percentile level doesn't change the type of distribution. The random variable taken into consideration are equity returns, and it is the random variable itself that assumes a certain type of distribution, which usually in the financial markets tends to be a T-student. By placing yourself on the most extreme percentile, you are only observing what happens in the distribution queues, which is the least likely event.

Chak, thanks for the link. I am not sure where this suggests that percentiles assume an underlying normal distribution. They do not. We should not confuse a ‘normalized ranking’ (mentioned in the link) with a normal distribution. These are very different things. To determine percentiles we just rank order the observed values (e.g. historical returns) to determine the percentile of a given observation. What the algo uses this for is to check whether a given, current return is falling into the 1% most extreme (left hand side) observed returns. If it is, this creates an ‘out’ signal. The distribution of the historical returns does not need to be of any particular distribution type. This approach works with all kinds of distributions – the distribution simply does not matter for the approach to work. Hope this helps / clarifies.

I have read and followed this strategy since it was introduced on quantconnect 10 months ago. While my academic training lies in physics, neuroscience, statistics, and behavioral economics, I didn't really understand enough of the market's implicit undertones to suggest anything. Now, I believe I'm somewhat comfortable enough with this topic to begin to really talk about the limitation of each strategy's variation.

My point in bringing up the limitations of setting a hard rule, in this case 1%, is that it causes the strategy to exit out too late or too soon, and subsequently, miscalculates the suggested n number of wait days. When you take a combinatorial approach to the number of out signals, GLD>SLV, XLI>XLU, DBB>UUP, the chances of getting an incorrect signal multiplies. Sure, bonds go up when one/two of the signals are TRUE, but the market remains “bullish” as well. What one needs is a malleable cutoff point.

As a corollary example, the PE ratio of growth stocks post covid market crash is 25+, while precovid it's an entirely different number. Further, the PE ratio for a growth stock fluctuates across time and perhaps, in some cases, might not even be a leading indicator at all!

Thanks Peter Guenther! I agree with you on the effects of inflation expectations on the prices of SHY and TIP. I believe I mistakenly analysed the effects of inflation that has already happened on the prices of SHY and TIP. Thank you for the clarification:)

Regarding the use of the 1% threshold, I agree that this can be applied to any type of probability distributions. However, this ‘1% of all values’ do not necessarily, and I believe probably would never correspond directly to, a ‘99% confidence interval’ in the typical statistics definition unless the distribution was rectangular like I described earlier. I have attached a graph that I plotted on python to investigate the difference between using the 2.58 std as a threshold for the 99% confidence interval, and using just the bottom 1% as the threshold. The results are strikingly different.

On this note, I think I want to ask if the assumption that I have had about the use of the 1% threshold is right, which is that it is motivated by the requirement of 99% confidence that the observation is above the mean.

Chak About your point on the hard cutoff, while I agree that a 1% threshold, and any hardcoded number, would have a certain rigidity to them, they already do have some degree of malleability to them. A more rigid criteria would perhaps be a rule that is like ‘if day-to-day returns are below x, out-signal generated’. This is different from a rule like ‘if day-to-day returns are within the bottom 1% of returns in the most recent 187 days, out-signal generated'. The 1% threshold used in tandem with the moving window allows this signal-generating threshold to change based on the most recent ‘state’ of the market (defined in the algo as the most recent half a year? I think), and I see this as a form of malleability. Using your example of PE ratios, while selecting stocks based on a fixed PE would yield very different results pre and post covid, if we selected the top  / bottom 25 stocks by PE, we would not expect this stock selection to differ very much pre and post covid. the idea of using the most extreme 25 stocks is the same as using the most extreme 1% of returns, I think.

Of course, this does not address the point you made about indicators remaining as leading indicators in the long run. I think this would remain a rather difficult problem to solve. However, considering that there is a good deal of economics theory backing the algorithm (e.g. the use of diverging SHY and TIP returns to forecast inflation expectations and thus the impact on whatever ETF we are trading), I do not see the loss of predictive power of the indicators any time soon, if they have any currently. I myself am also still in the midst of evaluating this.

The stocks are sorted by returns. So, if you take 100 stocks sorted in this manner, the 1% will take the one stock at the end of the sample, the highest or lowest, no matter what the distribution may be. This last one, or first one, could be 5, even 10+ standard deviations from the mean (an outlier).

Nice work there again, Han Ruobin, and valid point, Guy Fleury!
The missing puzzle piece might be the following: We have to be careful that we do not equate a ‘99% confidence interval’ with a ‘99% confidence interval for a normally distributed variable’. The -2.58 std that you are comparing against in your chart is the cut-off value (1% bottom) for normally distributed variables. It is important to note that this cut-off value does not give us the 99% confidence interval cut-off for just any variable, but only for normally distributed variables. Now, stock returns are often not normally distributed, especially in small samples. When an underlying variable does not follow a normal distribution, then resampling and using the 1% percentile will give you an appropriate estimate for the 99% confidence level cut-off value. Therefore, the distances displayed in your chart are a result of the ‘99% confidence interval for a normally distributed variable’ not applying in this context, since the sampled returns are not normally distributed, so that using -2.58 sd creates an error/deviation. That is, the distances are not created by the 1% percentile deviating, but by the -2.58 sd deviating since it assumes a normal distribution that does not show neatly in the data.
Not sure if you have a background or interest in statistics, but to find an equivalent for this point, one example that comes to mind is the test for mediating effects in statistics (variable A affects variable B which affects variable C). Call the effects a (A on B) and b (B on C). The total mediation effect (A on C) can be calculated as a * b. This total effect does not follow a normal distribution (due to the multiplication), irrespective of the fact that the effect components (i.e., a and b) may be normally distributed. To test the statistical significance of this effect (or to create a confidence interval), we cannot use the logics we would use for normally distributed variables (e.g., t = effect / sd) since the assumption of a normal distribution does not hold. Instead, we can use bootstrapping (random draws from the original sample) to create observations of the total mediation effect in many different scenarios and, thereby, generate a distribution. To determine the ‘99% confidence interval’, we would look at the cut-off level provided by the bottom 1% percentile (similar to what the algo does) of the bootstrapped distribution. Again, we cannot use -2.58 sd in this context because the assumption of a normal distribution does not hold (as can be the case for real-life stock returns, especially in small samples). For more details, e.g. see the description and online tool for total mediating effects here: http://quantpsy.org/medmc/medmc.htm

Good reply regarding Chak’s note, reminding us that a 1% percentile is not a hard cut-off, since it is responsive to the actual data that is being sampled. I agree.

Trivia: 1% of 10 stocks is zero. 1% of 100 stocks is one. 1% of 199 stocks is one. 1% of 299 stocks is 2. If you trade less than 100 stocks, use a top or bottom function instead of the percentile function to at least have one stock selected. It is nonsensical to use the 1-percentile function if the number of stocks traded is less than 100...

We do not have fractional stocks. The number of stocks traded is an integer. There is no such thing as 0.6 Apple or any other stock for that matter. We have started to have fractional shares, but that applies only to the number of shares bought or sold on this integer stock.

The greater the number of stocks traded, the more the >|2.58σ| will contain some stocks. The more the portfolio has a large number of stocks, the more the distribution will approach a normal distribution, even if the distribution is not normal.

We cannot predict with any accuracy what will happen to 100 stocks tomorrow, and yet, we want to cut hairs to the 100th. All that is very fuzzy, and any approximation will not make that much of a difference on a 10-year trading strategy.

Thanks for sharing, Guy Fleury.
Maybe a thought to add, only a nuance really: The 1% percentile is used to find extreme returns within a given instrument (e.g., say, for the ETF tracking metals). That is, it is a time-series view (e.g., 100 historic returns) instead of a cross-sectional view (e.g., 100 stocks). The 1% percentile tracks whether a given current return is ‘extreme’, i.e. sits within the 1% percentile of the sampled returns (including the current return).

Your point is generally correct that we need to ensure that we have at least 100 returns in the sample, which is constantly fulfilled in the algo considering the lookback of 252 trading days. Now, it seems that the algo would still work if we forget to pay attention to this technical requirement and reduce the lookback period below 100 observations: numpy.nanpercentile() even gives us the most extreme observation when we specify a percentile of 1% and the sample includes fewer than 100 observations. If you have python at hand (e.g. via Research) and want to give it a try, the following code can be used:

import numpy as np
test = np.array(range(1,51))
np.nanpercentile(test, 1)

The test sample includes 50 observations, from 1 to 50. We request the 1% percentile. It gives us 1.49 as the output, meaning that the most extreme (smallest) observation (i.e., 1) would be categorized as ‘extreme’ since it is smaller than the percentile cut-off value.
Therefore, we would still be within our conceptual logic of checking for extreme returns/observations, although of course a case can be made that the sample size should be made more consistent with the percentile that we are checking for. 

Curious with all the work that has been done on these, how popular are these when it comes to live trading? Or combining this strategy with another.

My latest article: The Makings of a Stock Trading Strategy – PART I deals with the mathematical structure of stock trading strategies. It is related to the IN & OUT strategy, especially the upcoming PART II. 

The article is looking at stock portfolios from an equation point of view. From the expected endpoints, it tries to re-fabricate what could have been required to get there to then include these findings in the trade triggering mechanics to ensure a profitable outcome.

The objective is to gain control over the evolution of a trading strategy as well as increase performance.

The article is also in HTML format.

Hope some will find it interesting.

Here is PART II of The Makings of a Stock Trading Strategy

This HTML file can be very helpful to anyone designing automated short-term stock trading strategies. It has deep implications. It deals with correcting for long-term portfolio return degradation, on how to fix it, and even how to reverse it.

It builds on mathematical equations used in describing the outcome of our trading portfolios and shows how easy it is to improve on these designs with simple trade triggering techniques. As if saying that your trading procedures can be greatly improved just by requesting more and letting long-term compounding do its job.

The main idea is to stop long-term return degradation which tends to make trading strategies break down or fail going forward. Not only can we stop return degradation, but we can also easily reverse it to then make our strategies prosper and thrive instead of their overall return degrade and die. As always, it is only if you want to do it. But regardless, you should be aware of how to compensate for return decay, overcome it, and reverse its detrimental effects.

Take the time to play with the equations provided, adapt them to your own trading strategies. Study the limits of what you could do or can do. It will open portfolio building and portfolio reengineering to anyone wanting to make it big for whatever purpose. Evidently, there will be work to do, but it will be based on a solid foundation from which one can soar. And yet, you should find it so simple. Make the following trade imbalance hold: (1 + fW) ∙ (1 – fL) > 0.

Click tor the HTML version. 

The article is a little long, I agree. It is there to protect your portfolio, not mine. So, you be the judge.

Related Articles:

The Makings of a Stock Trading Strategy – PART I 

Your Automated Stock Trading Portfolio

On The Use of a Rebalancer, a Flipper, and a Flusher

Guy Fleury : thank you for sharing this. Seems like it will make for a great read.

Given the content / relevant of the article, I suggest sharing this in its own forum post. It will surely not get the best exposure, buried in this In&Out thread.  

This will likely reach, and benefit, more people in its own post. Further, we can have focused discussions around it on a dedicated thread. 

My reflections on the IN & OUT strategy.

I consider the IN & OUT strategy as operating on market noise, especially when rebalancing on a daily basis. We all know we cannot win a heads or tails process except, of course, by luck. So, why put so much emphasis on this strategy which under other trading procedures might be classified as rather ordinary. 

It operates as if a positive outlier or an unexpected sore thumb depending on the point of view. Who would risk so much on market noise in the first place?

The answer is: it is not totally market noise. As in any stochastic process equation, there is a drift part and a random-like component, which by itself does, in fact, have a zero expectancy. So, it is not from the stochastic part of the equation that will come the expected benefits, but from the drift component which is easy to get. The drift component of the equation is often defined as the underlying trend. 

Any trend-following strategy should capture this underlying trend. Not because it is prescient, but just because it participated in the game. Not surprisingly, the average long-term portfolio return tends to the market's long-term expected average: E[r_m]. 

That seems like taking the merit of designing such a trading system out of the equation. As if saying, you won, but it was not entirely your fault, you were buying in an upward market and were simply in it for the ride. Nothing more. You would still get the merits of participating and would collect the benefits of such a ride. That is perfectly OK. It was what you wanted in the first place: the profits, pretty please, the same here. I do not mind so much how or why I profit as long as (over the long term) my strategy can profit substantially from the game. And I suspect you would conclude to the same.

It remains that trading over market noise appears as an opportunity bet in the sense that you know why you got in a trade based on some reason, but that your gain is still connected to a partially random-like exit strategy. This is where you should take advantage of price variance. 

You know prices will swing all over the place, that you like it or not. It is what it is. So, your advantage would be to exit a trade while variance is also to your advantage (∆σdW > 0) even if this variance is the result of market noise. That is what is exploited in the IN & OUT strategy (all its versions, btw). 

Is this wrong? I do not think so. Why should it be? You entered the game to make some money. You determined that this type of daily rebalancing could provide an advantage, an edge, even if small since mostly dealing with the delta of the drift (∆rdt). 

The variation of the underlying trend from day to day is in the order of 0.04% over the long term. It is not much. It is 4 cents on a $100 dollar stock or 1 cent on a $25 dollar stock. But it is still there, it adds up day by day, and is given free of charge. All you have to do is participate while you identify the trend as going up. This trend has been mostly up since March 2009, or, with a longer perspective, for over 240 years.

Almost any trend-following strategy following this underlying trend could have profited during those past 12 years (in fact, all the IN & OUT variants did).

In a recent simulation of mine, I had the average win rate at 0.06% thereby catching part of the long-term drift as well as a little bit extra, the result of the trade mechanics associated with the rebalancing process itself. Note that the average trade duration was more than one day. It is like taking advantage of this stochastically moving statistical blob of fluctuating variance penny by penny.

Yet, I consider the trading strategy's trend declaration to be wrong most of the time (~85%). I also noted that the strategy by its structure had an overall safeguard measure, thereby becoming a desirable protective “feature”. 

This all-in at all times strategy was winning due to its repeated flushing which served as its universal stop-loss. The strategy would not accumulate falling knives even though it was intrinsically designed to do so (buying the dips and the dips and the dips). The flip-flopping would force the execution of the stop-losses before they could be considered troublesome or too detrimental to one's portfolio. And since the strategy dealt with 100 stocks, any one stock's downfall would be considered to have only a small impact on the overall portfolio.

You want to win? Understandably. Then know why you win. Know what your trading strategy is really doing. A simulation is just there to confirm your trade mechanics. If it didn't work on past data, you have your answer for going forward even if you do not know what the future might bring. However, if your trade mechanics give you an edge, exploit it.

Should the IN & OUT strategy warrant more investigation as to why it can be made even more profitable? I think so. How about you?

Thank you to all of the contributors who have made this forum post (and a couple of other related ones) an extremely educational read. I have played with a few variations of the In & Out algo, and it seems that there have been no trades since about April-May timeframe. Is that correct that the algo has been “out of the market” for the last 3-4 months?

Is there another thread that focuses on live trading that I may have overlooked? It would be great to hear about some live trading experience and results given that we've had about 10 months of real-world trading since the algo was first proposed by Mr. Guenther in October 2020.

Again, thank you all for the great work and especially the knowledge sharing in this forum.

Guys I read the whole discussion. Super interesting. My concern is that while the back tests look incredible, I suspect they are incredible because its overfit. 2008 to 2021 has been an incredible bull run. So if we just dodge couple of drawdowns, we can “beat” the market quite handily. However i suspect this might just be overfit rather than a signal. My main criticism is that as name suggests its an “in or out” strategy. So are inferring a lot just based on the signal. Its not a situation where we are probabilistically changing our weights given expectations of next day returns, but instead we completly rotate out. So this makes me think thats its very easy to overfit because its a very binary signal. If you get some of the big moves wrong you could lose a lot of money. So its not a question of slightly underperforming the markets, because the system is slightly overfit. Anyways I was curious to hear your reasons for why you believe that the system is not overfit.

Is “IN & OUT” overfitted or not? 

We all have “opinions” on that since we cannot “prove” if it is or not. We have no tools to effectively demonstrate either yes or no. 

The first question that should come to mind is: overfitted, compared to what? 

Maybe, here is a better question: is “IN & OUT” outperforming long-term market averages? Because, if it is not, who would want to play with it? It would have lost all its potential value since it would then be underperforming market averages.

I am of the “opinion” that “IN & OUT” is operating, for the most part, on the fringe, the fumes, of market noise. And that cannot be overfitted due to its random-like nature. You cannot consistently overfit randomness. Every day, the price variations are, structurally, surprises that are highly unpredictable. You could, nonetheless, take a bet on the long side based on the underlying long-term market uptrend. 

Any trading strategy should be biased toward its goal. Meaning that in an upward market, most trades should be biased to the long side. Should the upside last for 10 years or more, you still should favor, for the duration, the long side for your “average” trade. Moreover, that average trade should generate a profit, not because you anticipated a market rise (even if you should have) but just because you participated in some market upswing while it was going up.

The only prediction we can make going forward is that market noise will be there, that we like it or not. And it stops there. As for predicting tomorrow's prices, your guess is as good as mine, although I will continue to favor the upside. This is what the “IN & OUT” was designed to do over the last 12 years.

Regardless, the strategy still needs some added protection and will need to change its upward stance for when the market really turns down again. However, it is already quite sensitive to market fluctuations, to the point that its average universal “stop-loss” is relatively low (my average loss rate is below -0.1%). The trading signal goes down a fraction of a percent and its flusher kicks in which could explain why it is wrong so often. At least, it will catch all downtrends of significance. It is a rare strategy where being wrong often (~85%) becomes a positive protective measure... 

“IN & OUT” wins the game because it was wrong most of the time. That, in itself, is interesting. Note that its rebalancing procedures (flipper, switcher, and flusher) had a major structural role in making this strategy win over its simulation period. It won mostly due to its trade mechanics. And that is also quite interesting.