The In & Out Strategy - Continued from Quantopian

one way to check if this startegy is overfit would be run the backtest from 2000-2008 time period. We could simplify the code so that we can run the code from 2000-2008 and see how the backtest performs. What do you guys think?

Thanks for sharing your observation, Jake Rocket. I agree that that is quite a long time for being out of the market, although it's also indeed true that the in & outs are sometimes out of the market for several months. Anyway, I have looked into an In & Out version that exists the market less frequently. My approach was to try and find a better balance between the size of the returns sample and the critical cut-off percentile and at the same time, ideally, reduce the number of required signals. 

So, attached is the latest version of these efforts (v7) – work in progress. Some additional notes regarding this algo:
(1) It saves important state information (dcount and outday) to the ObjectStore which ensures recovery of its state in live trading when the algo is interrupted.
(2) It creates a chart called ‘Out return’ that helps to track the return delta generated by the decision to exit the market instead of stay in the market. This chart can be quite instructive in terms of whether the out decision indeed was a good one … or a lousy one.

As always: Looking forward to any comments and improved versions.

As a follow up to my prior post;
Since it's difficult to see in the backtest charts above, below is a picture of the Out return chart which some may find useful. The chart value is zero when we are in. As soon as we are out, the chart value starts to fluctuate according to the following difference: out portfolio return minus return if we had stayed in. In science they would call the latter component the counterfactual, which here is the return in the parallel-world scenario where we would have remained invested in the market instead of going into bonds. When we go back in, the Out return chart value jumps back to zero, giving us a clear cliff's edge (sometimes we may have to zoom in a little) indicating the eventual return differential (out return - counterfactual in return) of the out decision. If positive, the out decision was a good one. If negative, we would have been better off remaining invested in the market.

Thread anniversary

Happy first birthday to this thread, it’s been exactly one year since the post below. Much work has been done on the In & Out strategy and related in & out-type strategies.

Stock take (continued)

In terms of a stock take of the work, I had posted an overview and summaries earlier, so just a list of the previously discussed threads here:
Listed earlier
1. This thread
2. Amazing returns = superior stock selection strategy + superior in & out strategy (link).
3. Dual Momentum with Out Days (link).
4. Intersection of ROC comparison using OUT_DAY approach (link)
5. A very profitable version of IN and OUT, and why it is likely to fail in real life trading like its siblings (link).
6. ROC comparison Utilities and Industrials (link).

Additional new threads related to the discussion of timing market entries/exits
7. KEI based Strategy (link). Focuses on monthly and quarterly leading economic indicators for OECD and non-OECD countries (e.g., including consumer sentiment, prices, national accounts, etc.) to select/rotate into ETFs and asset classes as well as possibly timing market entry/exit. Fine-grained regimes are discussed that inform the investment allocation.
8. Simple stock bond strategy hedged using VIX options (link). Focuses on developing a strategy that buys VIX options according to the current VIX level, meaning that the latter provides an in/out signal for the investment into the former. Discusses the difficulty of predicting optimal entries/exits accurately.

In & Out: An evolution

1. Re-entry: The good old outday and waitdays parameters

Starting with the very first version I had posted, the In & Out (and the other in & out-type strategies for that matter) use the original concept of an exit day (outday) and a waiting period before re-entry (waitdays). Just eight days after posting the original In & Out, the above post was musing about alternative ways of implementing the re-entry process. However, the fundamental outday-waitdays mechanics have never been altered since.

2. Signal clutter

In addition, it has always been interesting to see how the signals cluster together when things really begin to go pear shaped. For instance, see the 2008/2009 downturn, or 2015/2016, or March 2020. So, there might be some informational value in the signal build-up. However, what never worked well for the In & Out is to ignore the first signal and wait for two or more signals (confirmation) before going out. One explanation is that the signals themselves are rare extreme signals (see the percentile approach) and therefore, in themselves, provide sufficient evidence for that something negative is going on in the market. See similar comments here and here. Therefore, the signal build-up has to be used differently, that is, not according to the concept of confirmation.

In & Out v8

The attached, new version of the In & Out tries to integrate the above ideas and observations. What the algo does can be illustrated using the above picture: The blue line tracks the in (1) and out (0) decisions, both of which are based on the black line. The black line is a smoothened average (using the 50 exponential moving average) of the eight signals in the previous picture. We go out when the black line is starting to increase, that is, when a new out signal comes in, thereby staying true to the original operationalization. The new part is that we go back in when the black line (consistently) declines, that is, approximately although not exactly at the inflection point. Due to this logic, the new algo version does not require the outday and waitdays parameters, which is a first for any of the in & out algos.

Benchmarking the In & Out v8 
At least for trading the QQQ ETF from Jan 1 2008 to now, it seems like the v8 is able to realize incremental returns compared to alternative algos.

A. Results using the v8

B. Results using the v7

C. Results using the flex_v5_disambiguate_v2

D. Results using the DistilledBear

Peter Guenther Can you explain what are we trying to achieve here? Looks like some odd delayed simple moving average calculation to me.

Thanks for joining the discussion, GEightyFour. The discussion history of the In & Out might be useful to understand the this component and its evolution more fully; it has been a key component of the very first algo version, although coded slightly differently at the beginning. 

Let me try and summarize the gist of the discussion and hopefully this will be useful: By the end of the day, the algo wants to identify extreme returns and use these as out signals. Using daily returns for this check is likely to be highly fickle since daily returns are fickle. So, we need a somewhat longer period for the returns calculation. Here I had to make a judgement call and I said: about three months back (20 trading days/month x 3 months = 60 trading days). However, using the price exactly 60 trading days back means that the denominator for the return calculation rests on a single price point. This could again introduce a certain fickleness and can be addressed by using an average price for the denominator i.e. using the price information surrounding the 60-days back price. I used +/- 5 trading days, i.e. 11 days including the 60-days back price. For these reasons, the formula you cite above uses .rolling(11, center=True).mean() (i.e. the 11 day average price) and .shift(60) (i.e. surrounding and including the 60-days back price) when you fill in the corresponding parameter specified in Initialize.

In the historic discussion, we had the question, why not use an average for the numerator as well. Answer: Because the most recent price contains the newest information and we want to react to new instead of lagged information.

Of course, when I say that I needed to make a ‘judgement call’, this indicates a parameter whose value has been optimally chosen to a certain degree (e.g., based on observation, trial and error etc.). We had some interesting conversations that you can discover in this thread's history regarding parameter optimization, out-of-sample robustness etc. A great contribution regarding testing the sensitivity of the 60 and 11 parameter settings can be found here. They seem to be close to optimal.

Hi Peter, would v8 suffer from the same “fail at live testing” fate as you wrote for the other algorithms?

sorry as that other author wrote for one of the versions of in / out. Think the biggest failing point will be the out being bonds. There will be a time when both bonds stocks will be correlated and it will probably totally blow up the strategy. 

For better research maybe a long / short one asset class or long / cash. 

Peter Guenther I appreciate your explanation Peter. Having the center=True parameter made that entire line a less intuitive read for me. So I experimented by setting it to false and by adjusting the shift to 55 (I left it as 60-5 in the attached backtest-still the same regardless) and I got the algorithm to match your v8 baseline.  

Good questions, Elsid Aliaj! Regarding the first one, I reckon you mean to what extent the code can recover from a Live Trading interruption. It is programmed so that it should be able to recover since the key parameter (visualized in the black line in the picture above) is continuously saved and loaded from the ObjectStore when the code is employed to Live Trading, if that makes sense. Specifically, see the handling of the self.signal_dens parameter. Not sure whether this answers your question or whether you were after something else.

Regarding the idea of going into cash instead of bonds (or a different asset combo), what we need then is a logic that helps us decide when to chose bonds vs cash when being out. Did you have something particular in mind here regarding how you would make that decision? For example, looking at the current environment, it might be an idea to base this decision on the inflation expectation level. If too high (e.g. the RINF ETF is above a certain median level for a certain timeframe) then go only partially in bonds (or not at all). Etc. So, to structure this a bit, what is needed in my view is at least two things:

1. an economic intuition. What factor should we be looking at to determine our allocation when being out and why (i.e., what is the underlying economic rationale)?

2. a measure. What measure (e.g. an ETF and how? returns? median? level or change?) should we be using to assess the factor?

To get things started, we could begin with the RINF measure and the median level logic (I have used that before for one of the earlier algo versions) and have a look whether something interesting pops up.


We can absolutely do this, GEightyFour, this should be mathematically equivalent. The only thing I would recommend is to keep in mind where the parameters came from, since we might forget over time that the 55/11 combo actually comes from the logic ‘three-months back (60 days) and a window of +/-5 days surrounding (and including) this shift’. 
I do see a certain backdrop for the center=True logic since you often see this in Finance research, especially concerning event studies for which abnormal returns are usually calculated for a window that is centered around an event and expressed like +/-3 days, or +/- 5 days, often written as [-3, +3] or [-5, +5]. On the other hand, I would say: whatever helps us personally to see the code more clearly should be our personal preference. So, I fully agree with you here.

Hi Peter,

I've been following the development of this algo with very high interest!

The use of moving average center=True vs False highly impacts the results/performance of the algo.  When the parameter center is equal to True, a centered (two-sided) moving average is used.  This type of moving average relies on future data (non-causal) and cannot/should not be used for real-time decision making.  When center is set to False, a one-sided moving average is calculated.  The latter relies on past and current data for the calculation (causal system).  

Thank you! 

Frank

Good to hear from you, Frank Abraham, and thanks for joining the discussion and sharing these thoughts. Generally I agree, however in this particular instance we have to consider two things: 
(1) QC's engine is designed so that future data can never be used, i.e. the look ahead bias that you described is technically ruled out. 
(2) The description concerning the center=True setting is referring to data that is completely in the past relative to the point in time when we evaluate the data. See the description above: it is 60 days in the past and then calculating the average price in the [-5, +5] window surrounding that 60-day shift. Or, in other words, the average is calculated using the prices from the period -65 to -55, i.e. the prices between 65 and 55 days ago. These are all historic prices relative to the point in time when we use them. So, no look ahead bias. Hope this clarifies.

Peter Guenther Think I mentioned some prior thoughts of having an ultimate 3rd or even 4th out, such as cash, and gold. But I have to re-remember is this strategy based on momentum? How is determining what is out / in? 

Or I guess some sort of range if outside this range ultimate out

If it's momentum based can just easily add those to the universe.

Thanks for the info, Elsid Aliaj. That's right, I definitely remember.

I reckon one could summarize that the In & Out's out signals are based on stress signs (i.e., extremely negative returns) in six markets (industrials, metal resources, natural resources, US dollar currency, bonds market, and the equity market itself) plus two asset pairs indicating safe haven moves (gold vs silver and utilities vs industrials). When the algo sees stress in one of these markets or pairs, it goes out of equity (QQQ) and into bonds (TLT).
Against this backdrop, the algo is not directly based on momentum. Let me muse about your range idea a bit and get back to you if something interesting comes to mind (and works). I will also have a look into the inflation idea discussed above (i.e., high inflation expectations → avoid bonds) and write a post about it if interesting findings come up.

Hi all! Just a small update: On https://github.com/QuantConnect/Lean/pull/5968 we shipped a new version of pythonNet that has shown a nice performance improvement, using as a benchmark the algorithm shared at https://www.quantconnect.com/forum/discussion/9597/the-in-amp-out-strategy-continued-from-quantopian/p4/comment-36557 it went from 47k data points per second to 79k data points per second.

I just started running INOUT_V8 live and I am seeing this error 

Runtime Error: NameError : name 'pickle' is not defined at SaveData self.ObjectStore.SaveBytes('OS_signal_dens' in main.py: line 188 Stack Trace: NameError : name 'pickle' is not defined at SaveData self.ObjectStore.SaveBytes('OS_signal_dens' in main.py: line 188

Has anyone seen or resolved it?

Thanks for sharing, Aalap Sharma. Much appreciated. I think this will save people a bit of a headache going into Live Mode: Pickle is a python module and a line was missing to import it. The error did not pop up in Backtesting, since the read and write of the algo's state info (here: the signal density history) is only happening in Live Mode.

Attached is the updated v8, now including the import in line 14 of the code.
Let me know if something else pops up, this is very useful information for all.

Thank you Peter Guenther !!! I will test this out 

Folks, I stumbled upon this:

and also this:

Isn't this “inspired” by what has been discussed on this thread for a year???

"Here is the list of funds used to generate signals for market state:

Invesco DB Base Metals Fund (DBB)
Invesco DB US Dollar Index Fund (UUP)
Consumer Discretionary Select Sector SPDR Fund (XLY)
Consumer Staples Select Sector SPDR Fund (XLP)
In our latest article we used the pair DBB-UUP as an absolute momentum indicator. In that application the return of DBB was compared with the return of the US dollar. In this application we add a new pair: XLY-XLP. The condition that must be satisfied for switching to risk-off allocation is that both pairs indicate a negative return. That happens when the return of DBB is smaller than that of UUP and the return of XLY is lower than the return of XLP over the relative strength evaluation period."

Did I miss something or this guy is selling the basics of IN&OUT as his idea??