The In & Out Strategy - Continued from Quantopian

I think I have a solution to this problem if I understand it correctly. Nathan Swenson Peter Guenther Please check out this code. So generally we will open the position only when the fresh signal is generated. I achieved it this way: long story short algo will open position after the second signal is generated. So we skip the first signal to avoid the situation open position in the middle or at the end of the cycle.

gpw radar: Thanks for the code! That looks good in terms of preventing 'between cycle' entry at the beginning when starting the algo. When live trading, I (= personal preference) probably would want to hold what the algo is holding right away, but I see the point that there might be an advantage of waiting for the next cycle; however only if we assume that the positive returns are sitting at the beginning of each cycle vs mid way or at the end.
Only one observation: I am not sure whether the gold holding is properly reset when being 'in' the market. I think it is defined at 0.33 in the Initialize dictionary and then in the final if statement (if self.be_in:) wt[self.GLD] is not set to zero, which I think should lead to a leveraged holding of being 100% in QQQ and 33% in gold i.e. at 133% investment. I could be wrong, so probably best to check the orders.

Peter Guenther Yes, I got your point. I just implemented a solution (probably) for the problem described by Nathan Swenson 

 Ah sorry for that I just copied someone's code and added logic about not enter into the market at the beginning as we don't know where we are in the terms of the cycle. And yes you are right about holding gold it is a bug in the code.

Peter, I also agree that it's a preference choice.  My thinking was I would manually sync by checking the last historical entry and see if the asset price was the same or lower as the algo entry.  If it is then enter.  My concern was mostly about the wait period and getting off ideal cycle timing.

Thank you for looking into this Gpw Radar!  I will review your edits.  I really need to get up to speed on Python so I can help you guys more.

Update (2nd): In & Out ‒ parsimonious trades edition
Just a quick update of the In & Out algo trading the QQQ (feel free to change the stock holding), addressing some issues:
(1) warm-period: Included a warm-up period, pulling historical data to calculate the relevant indicators (see lines 62-3; also see Nathan Swenson's earlier comments). I set this to 350 days which actually is more than we need. Better safe than sorry.
(2) being neither in nor out at algo start: The default of the previous algo versions was that we are 'in' when starting the algo. I have changed this by setting self.be_in to an arbitrary number (999), which is neither 1 (in) nor 0 (out) (see line 55).
(3) simplified setting of holdings: The earlier operationalization of the holdings weights (self.wt) was suboptimal since we defined weights in the Initialize part and then manually overwrote them in the two rebalancing parts, which can lead to unwanted errors (e.g. via overwriting using the wrong weights or forgetting to overwrite one of the holding parameters). So, I reformulated the codes (see lines 124 and 140). As a brief explanation: What we are basically doing here is we concatenate two dictionaries (matching stock tickers and corresponding weights): the 'out' holdings dictionary self.HLD_OUT = {self.BND1: .5, self.BND2: .5} and the 'in' holdings dictionary self.HLD_IN = {self.STKS: 1}. Python does a dictionary concatenation via the code {**dictA, **dictB}. However, depending on whether self.be_in says 'out' (lines 123ff) or 'in' (lines 139ff), we set all of the weights in one of the dictionaries to zero. For example, in the case that we want to be 'in', all of the self.HLD_OUT weights are set to zero via the code dict.fromkeys(self.HLD_OUT, 0) (see line 140).

Any coding errors, give me a shout or post an update.

Thanks Peter, I just re-added the GLD.

New to QuantConnect -- quick question -- I was thinking to put my money where my mouth is. Do you think an L-MICRO server could handle this algo? (maybe I should save that question for support but I was thinking to throw some money at the in/out and see how it works with real money)

Peter, it looks better.  Now if I set start date as 10/1/2020, I see the trade into bonds on 10/6/2020, but there is still the QQQ entry trade on 10/2/2020 that doesn't align with the longer term backtest.  In this case it's a good trade.  Perhaps it's fine as it is in sync as of 10/6.

Peter, looks like it"s using too much memory with the long warm up period if running live:

Ok, I don't believe the warm up is benefiting the algo.  When I comment it out, the trades are the same.  I'll just leave it out for now.  Perhaps the other edit you made is sufficient?

@Nathan Swenson: Good to know regarding live trading and data requirements. You could try and shorten the warm-up period. In addition, maybe someone from QuantConnect could advice regarding what's going on in terms of warm up in live trading.

Regarding the entry into the QQQ trade: The algo update does not include gwp rader's modification regarding waiting for the next cycle to enter. Technically, the algo is 'in' the market from 11 Sep 2020 to 6 Oct 2020, so when starting the algo on 2 Oct, we would directly go 'in' (since the algo is 'in' at that point in time, basically since 11 Sep). Then comes the swap to bonds on 6 Oct.

If you prefer to wait for the next cycle, gwp radar's code additions are relevant:

Guys, I have one concern as I got a different result when I run something like this (comparing to your code): 

        self.Schedule.On(
            self.DateRules.EveryDay(),
            self.TimeRules.AfterMarketOpen('SPY', 1),
            self.calculateSignal
        )
        self.Schedule.On(
            self.DateRules.EveryDay(),
            self.TimeRules.AfterMarketOpen('SPY', 120),
            self.rebalance_when_out_of_the_market
        )
        self.Schedule.On(
            self.DateRules.WeekEnd(),
            self.TimeRules.AfterMarketOpen('SPY', 121),
            self.rebalance_when_in_the_market
        )

In self.calculateSignal I just run the all calculation to get in/out paramter. Others two function are just to open/close trades.
Why am I get a different result? Here is my deduction (I can be wrong):
When your code sets two functions to fire 120 after the market open I believe the program runs it at the same time. So the function rebalance_when_out_of_the_market does not finish its calculation but the second function was fired and use the wrong in/out parameter. I will try modifiy Peter Guenther and will upload the result we can compare it.

gpw radar: Cool test, will be interesting! I would set the signal scheduling function to 119 mins after market open, since having it at 1 min can have an impact. I often find that the market behaves relatively 'erratic' in the first few minutes (up and down a lot), while, when we are 1-2 hours into the session, the picture often becomes a bit clearer in terms of where the journey is going to.

Peter Guenther Sure, but when you calculate the param in/out your algo every time use historical data am I right? So it does not matter when you start the calculation. I ran quick test and run calculation after 1min start and after 120min start and in both log the content of the history param. It was the same. And as far as I understand the code you only use the historical data. Anyway `set the signal scheduling function to 119 mins` is also fine in my opinion as you will calculate the signal and the function rebalance_when_in_the_market will be fire 1 minute later.

OK the test finished. It is a tiny improvment 1,733.59 % vs 1714.767%.
Interesting that drawdown also improved 13.5% vs 16.5%

Mateusz Pulka: Great tests, thanks for sharing! That's good to know that the algo performance is robust against enforcing a strict chronology regarding the 'out' trades and 'in' trades. Good stuff!

Goldie Yalamanchi: Thanks for joining the discussion! I reckon that the algo does not need huge capacities to run properly, due to the limited number of stocks/ETFs that are traded or used in the calculation and the small number of calculations compared with more complex stock selection strategies. Also, trading occurs only every 1.5 month, on average.

@all: for live trading, we are still searching for code that would wait for the confirmation that the sell orders are filled before submitting the buy orders (also see the discussion here: https://www.quantconnect.com/forum/discussion/9632/amazing-returns-superior-stock-selection-strategy-superior-in-amp-out-strategy/p1). This is to ensure that sufficient funds are available for the buy orders. If anyone has a suitable code snippet, that would be amazing :)

Peter Guenther I added the bit of code below to sort by weight low to high which should make sells happen first. Checked the output and looks like it.

self.wt = {k: v for k, v in sorted(self.wt.items(), key=lambda x: x[1])}

One more thing, I want to make this work for like a Roth IRA with no taxes, so I can contribute x more dollars every year and that should trigger a re-allocation somehow. Maybe it is not required by other people but I think I am going to work on that next.

@Lukáš Ceplický I think I can try to implement your idea to improve the strategy. How I can reach you to talk about the details?

100th comment
@all: Just to celebrate, this thread has 100 comments :) Great questions, ideas, and contributions from all, keep up the great work! Congrats to all on this milestone!

VIX fear gauge indicator
Lukáš Ceplický Intruiging idea there regarding using the VIX as a fear gauge. I have tried in the past but was not particularly successful with it (e.g., using certain levels such as 15, 20, 25, 30, 35 as cut offs). But I believe that it could just be a question of implementing it right.

Execute selling orders before buy orders are submitted
Goldie Yalamanchi: Nice sorting code to ensure that the selling orders are submitted before the buy orders. I am currently looking into whether there is a way to wait for a full confirmation that all the selling orders are filled before the buy orders are even submitted; as a precaution that funds are available in the event that the selling orders take some time to be filled. Possibly the QC team has a code snippet for us that might achieve this? (Shile Wen, Derek Melchin)

Peter Guenter: self.SetHoldings(sec, weight, True) will ensure there is enough margin