The In & Out Strategy - Continued from Quantopian

Nice work Peter!  This new version especially excels with 3X funds.  I see 71% YoY return using TMF and TQQQ.

In_out_flex_v5_try

an implementation that uses increasing input prices (METL and NRES, above median) to mute the DEBT signal.
Excellent example of a curve fitting.

I found this thread a few days back and have spent several hours since then poring over the archived Quantopian thread and all 250+ replies in this thread. As someone new to quant finance coming from a software engineering background, the discussion alone has taught me so much. Thanks Peter Guenther for having this conversation open-sourced :) and I've learned a lot from Menno Dreischor's discussion about out-of-sample testing.

I'll be following this discussion and hope to contribute, though I'd want to start by validating the algorithm vs hacking returns assuming the alpha is valid. Kevin Davey's book has some interesting directions to take out-of-sample testing that might supplement the work Menno has already done. Cheers!

Austin: Welcome on board and good effort working your way through the archive and the thread here! Will be great to hear about your insights.

Additional musing about signal ambiguity
I reckon for the DEBT signal we were a bit too late in determining its ambiguity. So, I was wondering whether it might be possible to get ahead of the curve by directly musing about additional signal ambiguity. Below is a first brainstorming regarding the signals, their interpretation in the algo, and thoughts concerning possible ambiguity (i.e. that something that is interpreted as a bad thing may sometimes be a symptom of a positive development). The table below suggests three ambiguous signals, see the note "ambiguity, address".
For those looking for a slimmer algo, this might be an opportunity to try and drop these signals.
For others, this might be an opportunity to think about ways to disambiguate the signals.

Table. Signal ambiguity

Note: If your main concern is overfitting at this stage (and before), that is fine. Skip this post and focus on posts concerning testing and distilling signals. There will be posts coming up along these lines and we may achieve a certain level of convergence again later.

Second crack at disambiguating signals: the GOLD pair
We noted that gold can be an inflation hedge, such that the gold price increases when inflation expectations increase (see Table. “Signal ambiguity” above). However, inflation expectations tend to rise when the economy is expected to do well, making a rising gold price a symptom of a positive development in this scenario. This makes the GOLD pair signal ambiguous, since the algo interprets a rising gold price as a bad thing (= flight to safe haven = “market jitter”). The attached algo uses an ETF measuring inflation expectations (RINF, see line 45) to re-assess the GOLD pair signal (see lines 121-125). The logic: To rule out that the gold price increase is due to rising inflation expectations, we only react to the GOLD pair signal when changes in inflation expectations are below median.
Since RINF only started in Jan 2012, I had to catch this via a try ... except construction.
However, RINF could be manually constructed by comparing treasuries with inflation-protected treasuries. I have to explore that option a bit more and may give it a go later unless somebody else might be faster (?) ...

   Looks like it entered QQQ on 12/18 is this correct? Nathan Swenson  Peter Guenther

Yes, Elsid, entered QQQ on 12/18.

Yes that is a very very late entry into QQQ after such major move up.  I would just like to add that I have seen a lot of variations of the In/Out algo and the one change that really makes the algo react better without any significant increased drawdown (not using leveraged TQQQ, only QQQ or stocks universe) is to change the self.INI_WAIT_DAYS to 2 as pointed out by Guy Fleury 

self.INI_WAIT_DAYS = 2

This makes the algo in the March 2020 covid crash re-enter and stay in since about 4/6. And while it does get a little whip-sawed during the crash, the drawdown was only 15.5% I think one of the lowest drawdowns that I have seen in all the variations of the algo published here.  Also, it leaves in tact the SHY indicator as is.

Goldie, in that last run example you have SHY commented out.  

I agree that the entry for IN was far too late.  The algo needs to react more quickly.  There's no way I'm holding TQQQ this late in the move.  I am holding some regular equity funds and will follow algo with those. 

 Nathan Swenson thanks for picking that up.

Still, I have corrected that  (commented it back in) and it now stays in the market from Aug - Dec in the QQQ's... as long as 

self.INI_WAIT_DAYS = 2

I don't know what you guys are really trying to achieve, but when changing it to 2 days from 15, sharpe ratio plummets. CAGR drops and DD goes from 16.5 to about 22%

Wow....quite a bit of progress from Quanopian on this one. I am still getting my head around QC framework.

Can a kind soul care to explain below lines of code:

Goldie Yalamanchi , Nathan Swenson , and Elsid Aliaj: I tend to agree with Elsid and would not reduce the wait days too much. What I take away from Guy's table (reported elsewhere) is that the performance is relatively stable from 5-15 wait days, so there might be some scope to play around with reducing the wait days in this range. I would not go down to the 1 or 2 days, although the returns in the backtest look better. I would rather go to where we achieve some stability in the sense that results look relatively similar over a range of a parameter's values. I would also assume that by reducing the wait days we become more dependent on the return flips and that they provide precise signals in terms of pulling the wait days up when it is beneficial to do so. When these signals are less precise in the future compared to the backtest, we might be ending up with too few wait days. I will also try and run some tests to get a feel for what the new dependencies are ...

Performance from a full 2008 backtest actually goes down. And DD sky rockets to 24%. 

Maybe if you backtest just 2020 the numbers look better but the whole picture looks much worst. 

"What I take away from Guy's table (reported elsewhere) is that the performance is relatively stable from 5-15 wait days, so there might be some scope to play around with reducing the wait days in this range. I would not go down to the 1 or 2 days, although the returns in the backtest look better."

I agree, if the QC Optimizer wasn't pay-as-you-go I'd have run a couple of optimizations over the many parameters involved in this algo to look at stability and out-of-sample viability. As far as goals to measure against, parameter stability seems MUCH more important than backtest return % over many years. That is, as long as one's goal is to create a model that can describe the future, not soley describe the past. Jared mentioned in a different thread that QC is working to make the Optimizer feature work with nodes you're already paying for, and after that feature is pushed in ~10 days I'll run more tests and share my findings here...

Peter Guenther, if memory serves I remember you asking what market phenomonon this In/Out algo appears to be harnessing to capture such nice backtest results. Did the community ever come up with a market phenomonon this algo is exploiting, or is this experiment soley focused on the backtest?

In the mean time I'm working on a futures strategy, and since I've loved the discussion around this algo so much, I'm planning on sharing it if I can get it working / add value to the community :) these open sourced conversations are great!

Austin: Looking forward to your analysis results! This will be very interesting in terms of seeing the sensitivities and where there are any flimsy parts that need fixing, further development, or a more substantial overhaul. The futures version that you are working on also sounds very exciting and thanks for planning to share it! I am sure the community will love it and I fully subscribe to your positive note regarding having these open algo conversations, they are great.

Yes, I believe that you are right: currently, the source of the returns is not fully understood regarding its specific mechanics. I reckon a working hypothesis is that the algo signals pick up ‘jitter’ in the equity market that translates into a boost in bond prices. What was driving investors’ jitter often may not eventuate. Nevertheless, ‘riding the jitter wave’ in bonds vs remaining invested in equity seems to pays off.

Peter Gunther,

I'm sure you won't answer this, my last post on this thread, just as you haven't answered
many of my previous ones.
I showed you where are the flimsy parts that need fixing, further development,
or a more substantial overhaul.
In the United States, only one person has veto power.
You have given this right to any of the 8 sources who are not fully understood regarding its specific mechanics  and are therefore unreliable.

Happy curvefitting!