This thread is meant to continue the development of the In & Out strategy started on Quantopian. The first challenge for us will probalbly be to translate our ideas to QC code.
I'll start by attaching the version Bob Bob kindly translated on Vladimir's request.
Vladimir:
About your key error, did you also initialize UUP like this?
self.UUP = self.AddEquity('UUP', res).Symbol
Peter Guenther
Great investigative work, Damiano Bolzoni, thanks for sharing! Usually I would have said that it’s not unlikely that different people might come up with similar solutions. However, having read the Seeking Alpha posts by Toma Hentea, I tend to agree with you that we are likely looking at plagiarism here.
First, a small point: The three-month lookback period to calculate returns is exactly the one that I had chosen in the very first post of the In & Out strategy on Quantopian, and it has been the key lookback period of the algo since. So, it is definitely noteworthy that this market timing strategy on Seeking Alpha calculates returns over a three-month period.
Second, and this is very suspicious, the Seeking Alpha author uses “the pair DBB-UUP as an absolute momentum indicator” (i.e., for timing risk on vs off). In my opinion, you do not come up with this pair just like that. Why compare Metals (DBB) and the US dollar (UUP)? For us here in this thread, this pair is organically grown. To quickly recall: I had used DBB in the initial In & Out algo version; Tentor added UUP; Vladimir introduced an in & out algo based on pair comparisons (gold/silver and utilities/industrials; see Intersection of ROC comparison using OUT_DAY approach); and, finally, Dan added the DBB/UUP pair to create a super-algo, mixing the DBB and UUP components of the In & Out into the IoRcuODa. So, it's actually quite a long and intricate way to arrive at this pair and, without this evolution in mind, the pair might even be perceived as non-intuitive (see earlier posts asking for the economic rationale of this pair in particular and the difficulty to fully explain it).
Therefore, it seems that our thread here would at least have deserved a proper reference/mention in the Seeking Alpha article …
Damiano Bolzoni
Hi Peter Guenther didn't want to call it plagiarism and that's why I wrote “inspired” (within quotes). Not sure how much money this guy is making but he's on Seeking Alpha's market place, selling this strategy…
Peter Guenther
Love the “inspired” bit, Damiano Bolzoni!
My personal opinion (others may see it differently): I am absolutely open to someone using the In & Out elsewhere, e.g. to write about it or even to sell strategies based on the concept (if it’s a large-scale ETF, of course, a profit share for all discussion participants would be nice 😊). No problem at all. After all, I deliberately decided to discuss the strategy in a public space and the discussion was very valuable. Without it, I would still be sitting on a version probably pretty similar to the very first In & Out published on Quantopian; no pairs, no percentiles, and so on.
For me personally, the important bit really is the acknowledgement of the concept’s origin and, ideally, a link to our current discussion here on QuantConnect, so that value goes into both directions. Currently, the Seeking Alpha articles don’t bring us anything, it’s only the author generating income. We do not acquire any additional minds from the channel for our discussion, QuantConnect hosting our thread does not benefit from the traffic potential, etc. So, at the moment it’s a very one-sided exchange. And it does not have to be. The fix would actually be relatively simple: acknowledgement + link, why make it entirely one-sided?
Carsten
I was following both, the seeking Alpha and this channel (and the old Quantopian) and learned from the discussion…
Some suggestions which could help to improve the market strategy. Instead running and optimizing the whole strategy, it might be a good idea to measure the effectiveness of the single component. For example I found it very interesting that ALL pairs optimum is 3 month if you just use them single to decide to switch in-out, at least my finding.
Additionally I would be interesting how the momentum optimum was historically, that’s on my list.
An other interesting component would be the VIX,, absolut, relativ or using the term structure (spot VIX to 1-3 month out)
Next would be to feed different pairs into a HMM, just tried this with the VIX and Daly returns and got some decent result.
I would like to stress that all of these I only do in a small python script and if they work I run it in a full simulation (at the moment in zipline, as I need more time to realise it in lean)
I think it would increase the learning curve and awareness to test components singular instead to run them in a full simulation with a lot of other component.
What are your opinions?
Toma
@Peter Guenther I subscribe to your attitude and will act on it. Within a day, I will check all my articles on Seeking Alpha and see where I should add some recognition to those who came out with the ideas that inspired my work. That conversation will be reciprocally advantageous for Seeking Alpha and Quant Connect. Hopefully, Seeking Alpha gets some traffic and in return QuantConnect should get some exposure to the large Seeking Alpha community.
@DamianoBolzoni I do not “sell your strategy”. There is a lot more that I do for the community of Seeking Alpha subscribers. I will acknowledge the few ideas that came from Quantopian mostly, not from QuantConnect. My strategies are much more than what I got from Quantopian/QuantConnect.
Damiano Bolzoni
Toma it doesn’t really matter whether you do “more” for the Seeking Alpha subscribers.
According to plagiarism.org “to use (another's production) without crediting the source” is plagiarism. Even when you don’t earn a financial benefit. Even when you paraphrase it.
Please include sources when appropriate in your posts.
Thanks
Peter Guenther
Thanks for dropping by, Toma. Deal. That sounds like a solid action plan regarding the relevant existing Seeking Alpha articles. Also, good on you for the quick response. I could imagine many in a similar situation who would not have been aware or, worse, would have tried to sit it out. Therefore, the speedy reaction deserves credit. So, I would say: If you discover something of interest in this thread, keep on disseminating on Seeking Alpha, adding a reference + link where relevant. Fingers crossed, we get additional awareness for the concept and a few additional minds and ideas for our discussion here. I am sure, QuantConnect will appreciate the traffic potential as well. Also, if you have an interesting twist in one of your articles, or just want to illustrate an application, feel free to drop a link here and, I am sure, readers of this thread will be interested. (Same if you get a comment on one of your articles that could spur an interesting debate here.)
Thanks also to Damiano Bolzoni for bringing the parties together. The plagiarism definition is also a useful reminder, I didn’t know that they have a whole website on it :)
I reckon we can all shake hands on it, with hopefully new opportunities ahead.
Stephen David Bennett
This is a slightly tangential comment. I've spent about four months experimenting with both this and the similar Vladimir In/Out variant. I made a bunch of changes, tried lots of different things and run a lot of quant connect optimisations. I've pretty much came to the conclusion, other than with brute leverage, I personally can't really improve this in any honest way,
What I am plagued by though are the thoughts that I have not back tested these strategies through enough varied market conditions. We are effectively testing through a 20 year bull run with a few blips. A strategy that can ratchet its way up on a bull run, riding out the troughs in bonds. I would dearly love to see how this would have performed during the 1970s and 80s..
I'd kind of put these thoughts on the backburner but recently I re-read Naseem Telabs “Black Swan” which brought this to the fore again. I've been live trading a variant of this for a while and with each period of drawdown I think “is this the black swan!”
Has anyone created any external data sources which can provide proxies for these signals/etfs going back into previous decades? I started doing it a while ago looking but didn't get that far.
Chak
Stephen David Bennett a good start to improve these public baselines is to incorporate hour and minute timeframes to the Daily Resolutions.
Peter Guenther
Great comments, Stephen David Bennett, thanks for sharing!
I agree that testing the strategy before 2008 would be very interesting. Of course, the challenge that you also point out is that the ETFs providing the relevant signals are not available for most of the time before 2008. In theory, one could reconstruct the ETFs through appropriate universe selections (incl using Morningstar industry sector selectors) and bundle relevant firms in a ‘virtual ETF’. The returns of these virtual ETFs would then provide the relevant signals. In my view, we might not have to go as far back as the 70s or 80s since the world has been a very different place back then, e.g. in terms of market access and participants, central bank and government (crisis) philosophies and strategies, reaction speeds etc., although I do see your point regarding having a long time-series to draw more robust conclusions.
Regarding Taleb, we are still experimenting with this issue in the Amazing returns in & out thread. Here the challenge is that the ‘insurance’ we are taking out (e.g. via put options) to protect us from black swan events is quite expensive (esp. when holding it all the time), substantially grinding down the total return that one can realize. Part of the solution is that one only takes out the insurance selectively/not all the time. We have initial ideas regarding possible conditions (although, of course, this approach conflicts quite a bit with the definition of a black swan = unpredictable event). I reckon another important element which is not yet developed is about ‘taking money off the table’ when the insurance payoff would be high, since otherwise we end up constantly paying without ever realizing a benefit. Yet, this requires again a certain logic regarding when to cash the insurance in.
Manoj Agarwala
Another option to test it for period before 2008 is to use Futures and Mutual Fund prices. Here is link to some data going back to 1995 with this approach in mind but I never got around to fully testing it.
Manoj Agarwala
I love the v8 version but wondering how. the stock splits/dividends are handled here? My fear is that the consolidators used in this algo do not handle these events properly.
Also, I see that this algorithm uses 5 years of history but some of the ETFs do not have 5 years of history when you run it for 2008. I also noticed that the performance changes significantly if you change the 5 year period to 1 year and corresponding extreme percentile from 5 to 1.
Another concern I have is that the extreme percentile of 5 may not work well if we experience long drawn multi year down market.
Toma
@PeterGuenther It took me much longer to find the time to collect all the information I could about the timeline of changes I made in my strategies. The first time I used DBB and UUP as “canary assets” was in Mar. 02, 2021 in the article at the following link: https://seekingalpha.com/article/4410670-momentum-strategies-djia-stocks I initiated entering a reference as follows: “ The usage of a pair (DBB, UUP) as an indicator for market risk-off periods was introduced by Peter Guenther in a blog on Quantopian.com. A continuation of that blog is now available at the following link: https://www.quantconnect.com/forum/discussion/9597/the-in-amp-out-strategy-continued-from-quantopian/” Because of the delay in editing past posts by Seeking Alpha editors, the references will post only at the beginning of next week.
Meanwhile, I intend to start a new discussion thread here at QuantConnect to complement your thread with information about Momentum strategies published by many authors. Here is the list of authors I plan to mention in that blog: Yegadeesh and Titman (2002), Mebane Faber (2010), Gary Antonacci (2013), Frank Grossmann (2013), Asness, Moskowitz and Pedersen (2015), Keller and Keuning (2016), Yulia Malitskaia (2019), Michael Gettings (2019) and Scott Juds (2020).
Additionally, I will mention two of my articles published by Seeking Alpha. The first is an article in 2014 where I used a three-month look back period for momentum evaluation when most authors were still using a period of 12 months as proposed by Jegadeesh and Titman, period confirmed by Antonacci as optimal for a period of almost 100 years. In my second article (2019), I showed that for a broad class of assets, the optimal momentum period has changed. From 1988 to 2008 the optimal period was 12 months, but for 2008 to 2019, he optimal was 3 years. This is an example as argument for using the most recent past for optimization of strategy parameters as opposed to using very long past periods.
I hope that this new thread would reach many individuals who can contribute to further development of your strategies. Here I want to mention that the reason why I have never posted anything on Quantopian was the fact they were insisting on market neutral strategies that required shorting some assets. On your thread I had the impression that there was much effort to adapt the strategy for automatic intraday trading. All my strategies are long term investments with minimal trading. My strategies have been risk-on since May 2020. When DBB-UUP indicated risk-off I did not apply it because it was overridden by other conditions I use.
Finally, I would like to post a copy of my initial QuantConnect post as a blog on Seeking Alpha. There, some Seeking Alpha members will be exposed to information about your work.
Manoj Agarwala
Stephen David Bennett Wondering if you saw my post above with Link to alternative data that I have assembled for longer duration back testing. Data is from 1995. The data consists of futures and mutual funds that seem to be equivalent to the ETFs being used. https://www.dropbox.com/s/jhbv45w111szult/BackTestingData.csv?dl=0
Carsten
Hi
I tested the effect of the window lenght used for percentage change of the switching pairs. (pairs used to decide on In-OUT)
First test shows the difference between a window length of 2-800 days for the whole period from 2008 until today (one return for the whole period and variation of the window lenght)
Second test, I calculated the return with a rolling window of one year and than looked for the best performing window and its return.
Both test were done for different single pairs. IN-OUT was done with just the SPY. In the first test I switched between SPY and IEF and in the second test between SPY and Cash.
From the first test it looked that around 60 days give the best result/return for the whole period.
The second test, with a rolling window return and picking the optimal window, showed that the optimal window lenght ist ALL OVER THE PLACE….
And now? How should we decide which window length to use and to avoid fitting the window to the best historical result?
Please don't understand me wrong, I'm personally using this method for my investment and I really want to improve the method, but don't know how.
Here the result for the pair DBB/UUP. Upper graphic shows the optimal window lenght.
Following the notebook with the code:
Stephen David Bennett
Hi, Manoj Agarwala , apologies for not replying sooner. Yes I got that file, thanks so much for putting them together. I dissected all of the relevant instruments and created an external data source. I'm still having a bit of an issue getting it to run. I invested about 9 to 10 hours into it last week, hopefully I'll get some more time next week to see if I can fix the issue.
My issue is that I'm predominantly a C# developer. I've done that for 20 years. Python's probably a third language to me, so its slow going. Eventually, I did though at least get to be able to set breakpoints in local copy of lean in visual studio. Progress! Slow progress!
Ws2402
Carsten My guess would be (or maybe not) the window length is correlated to the overall volatility? hence the window length should be somehow adaptive to the market volatility?
Peter Guenther
Toma, thanks for sharing the edits and links. Looking forward to the ideas and articles you outlined above. The change over time concerning what is the optimal lookback period length is very interesting. Relatedly, the question you raise regarding whether to use only recent lookback periods vs very long periods is interesting and is likely to generate a robust debate. We had a bit of that debate in this thread too (e.g. one suggestion was that a strategy should ideally hold for extremely long periods, e.g. 100 years, to be considered valid etc.).
Peter Guenther
Manoj Agarwala, thanks for sharing the dataset that you have created, very nice work!
Regarding your points:
I never expected there to be an issue concerning dividends or splits. My understanding is that the price data is backward adjusted so that, for instance, stock splits are neutralized and do not affect the price signals that are used in the algo. So, my assumption is that, due to the adjustment, there is no effect/bias. Not sure whether you have observed something that made you believe that stock splits and dividends could have an effect (?).
The 5 years lookback, and data not being available before 2008, is a fair point. So, the algo backtest should actually be starting from 2013. I mean, the percentile approach still does what it’s supposed to do in the years before, i.e. it would be looking for the 5% extreme observations; however, of course, the sample of return observations would be smaller in the years before 2013 than in the following years.
Regarding sensitivities, that’s correct: If you change the 5-year lookback period to 1 year and the extreme percentile from 5 to 1, the performance changes substantially. It should get you closer to the v7 performance and probably quite a bit worse, since I removed a few of the signals compared to the v7.
Regarding multi-year down markets: You also raise a fair point here incl. how the algo could be made more robust for these scenarios. Theoretically, the algo would also generate out signals in down markets, since extreme price drops occur also when the prices are continuously declining (the decline is never fully smooth). However, what is true is that there would be times when the algo would go in. I cannot tell how lucky the algo would be in terms of capturing some of the upsides in a constantly declining market; there would definitely be opportunities to capture upsides even in declining markets since these are never only in decline, i.e. there is always a bit of back and forth.
Peter Guenther
Carsten, nice work, thanks for sharing! Here is my thinking: I fully understand the beauty of rolling optimizations runs to derive and test ‘would-be-optimal’ parameter values. However, in (semi)-efficient financial markets where these is ‘no free lunch’, I doubt that we would ever find one optimal parameter value, or a very tight range of optimal values for that purpose. If this one optimal value (or range) existed, then there would be free lunch, i.e. I would use this parameter setting and make optimal returns every single year (!). In my view, this is unlikely to ever exist in financial markets. Therefore, the optimal parameter values are all over the place, since they have to be in somewhat efficient financial markets. So, I think that the only thing we can really hope for is a parameter value that is actually quite sub-optimal when evaluated for each individual year, but is something of a valid average that still leaves us with a bit of excess return. I do not have the final answer, but the three-month lookback period might just be this sub-optimal average value that leaves us with an excess return (?).
Tentor Testivis
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
To unlock posting to the community forums please complete at least 30% of Boot Camp.
You can continue your Boot Camp training progress from the terminal. We hope to see you in the community soon!