Hi All,
Inspired by the amazing series "From Research To Production" by Sherry Yang and Jack Simonson, I wanted to share with the community my implementation of a simple Long-Short Moving Average Crossover using the Algorithm Framework and a Research file with explanation of the Alpha logic.
There are already open-source implementations of this strategy of course, but the educational purpose of this algorithm is to show how seamlessly we can move from research to production with almost no changes in code. Also, I'm updating indicators "manually" using history calls on purpose, instead of using the QuantConnect Update method for indicators, as otherwise I couldn't replicate the logic in the research notebook.
ALGORITHM FRAMEWORK MODULES:
- Universe: Manual input of tickers.
- Alpha: Go Long when the Short Period Moving Average crosses above the Long Period Moving Average, and go Short when it crosses below.
- Portfolio: Equally Weighted portfolio (investing the same amounts in each security). It includes optional inputs for portfolio rebalancing.
- Execution: Immediate Execution with Market Orders (using a custom model Immediate 'Execution Model With Logs').
A couple of notes I'd like to share on these modules:
- The concept of Insights in the Alpha model. Normally and with few exceptions when using Minute data, the way I like to think about these insights is as a prediction that I update every trading bar (as opposed to giving a longer, uncertain period) for another extra bar until my model tells me that signal is no longer valid. As you can see, when we get a Long signal we emit InsightDirection.Up for one bar and then continue to update this prediction until we get a Short signal, and repeat the process. We use InsightDirection.Flat to stay in cash when there are no trading signals (in this algo that only happens from the launch of the backtest until it gets the first signal).
- In order to handle this way of sending Insights, I slightly modified the open-source code for the EqualWeightingPortfolioConstructionModel. This custom version will only rebalance the portfolio if:
- It's time to rebalance the portfolio. You can control the number of trading days between rebalances with the rebalancingParam input in the main.py script.
- OR there are Up/Down insights for securities that are not in the portfolio yet.
- OR there are changes in direction for actively invested securities.
- Finally, this project uses a custom implementation of the open-source Immediate Execution Model that includes many different logs of information about number of shares traded, average holding prices, profit and profit percent for both long and short positions. If you like that module you can simply plug it into any of your existing strategies that use market orders. This is a good example of the modularity of the Framework!
RESEARCH NOTEBOOK:
As mentioned above, this project includes a Research file with explanation of the Alpha logic. I tried to even 'replicate' the event-driven data flow of the Alpha module happening within the backtesting environment by looping through the dataframe of historical data in the research environment to update indicators and get trading signals, so we can see exactly what's happening behind the scenes of an algo. A few notes on this research notebook:
- The class SymbolData that stores the different indicator calculations for each symbol can be copied into the Alpha model with no changes in code.
- Then we're iterating through each row in the dataframe of historical data to update indicators and signals. You can see how this code is almost identical to the one found in the Alpha model. Of course, in the Alpha module the iterations through new data points happen behind the scenes, but this is pretty much what it's doing.
- At the end I'm just plotting the indicators and buy/sell signals so we can compare with the results from the backtesting plots. They should be the same.
- Play with different tickers, start/end dates and parameters for the moving averages on both the research notebook and the backtesting main.py script to check they both give the same plots!
OTHER FEATURES:
This project also includes some other features that might be of interest for beginners:
- Potting indicators in backtesting.
- Organizing code in a certain way with calculations per symbol stored in a separate Python class.
Merry Christmas!
Emilio
Sherry Yang
Hi Emilio,
Happy New Year, it's awesome to read such a clear and in-depth post! Many thanks for your contributions to the QC community.
Sherry
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Emilio Freire
Thanks for the message Sherry! And Happy New Year to the QC team!
Emilio
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
James Candan
I hadn't seen or thought of this use case for Research. THANK YOU!
Any chance you could update this? Currently, it throws an error:
--------------------------------------------------------------------------- FileNotFoundException Traceback (most recent call last) <ipython-input-6-b79d4cb63739> in <module> 3 AddReference("System") 4 AddReference("QuantConnect.Common") ----> 5 AddReference("QuantConnect.Jupyter") 6 AddReference("QuantConnect.Indicators") 7 from System import * FileNotFoundException: Unable to find assembly 'QuantConnect.Jupyter'. at Python.Runtime.CLRModule.AddReference(String name) in C:\Users\Colton\Desktop\Working\QuantConnect\Repos\pythonnet\src\runtime\moduleobject.cs:line 488
Is there a necessary replacement? I tried commenting out the Jupyter lines, but the plots never show.
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
James Candan
Never mind. I got it working. I set the dates too short before.
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
.ekz. INVESTOR
What a great post. Never would have spotted this if James Candan hadn't commented on this and bumped it up the feed.
Thanks for this solid community contribution, Emilio Freire Breaking it down as you did is very helpful.
( and thanks for the bump, James! )
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Emilio Freire
Thank you guys for the support!
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Spacetime
AlMoJo ,
Also, if you would like to explore more CBOE data sets, then you can use the Nasdaq Data Link provided below.
Correction in my last post: meant to say CBOE and not Quandl (Quandl was acquired by Nasdaq)
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Spacetime
AlMoJo , Just posting our QC CBOE Datasets link below for your reference if you need it.
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Vladimir
Spacetime,
There are only 6 trades for SHV in your version of the code.
Can you try changing in the code the VIX and VXV data retrieval in a way like in the attached sample.
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Spacetime
Hi Vladimir,
Hmm… not entirely sure why there were only 6 trades executed.
The above backtest which I have attached was taking a “bit” to complete for me and I was engaged with other work, so I ran the above model (shaerd by pangyuteng) with a start date of self.SetStartDate(2020, 11, 1) just to speed up things.
Hmm… I have extended the date range to start from self.SetStartDate(2019, 1, 1) and it does not have too many trades either. (backtest attached)
If you have noticed from the above example shared by pangyuteng , then it does not trade that often either. [ self.SetStartDate(2019, 1, 1) ]
But, the fixes which I have shared above does work as the error which was being received by AlMoJo is no longer occurring.
Hmm… maybe someone else might have a better idea on this, but I will try to look into this further. Hmm… maybe the LSTM configuration needs to be looked into or the volatility data resolution and other resolutions needs to be changed to minute time slice if possible.
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Vladimir
Spacetime,
I backtested cloned pangyuteng algorithm with only changes in my_custom_data
url_vix = "http://cache.quantconnect.com/alternative/cboe/vix.csv"
url_vxv = "http://cache.quantconnect.com/alternative/cboe/vix3m.csv"
It generated 13 SPY-SHY trades and metrics as your last one, but the performance is not comparable to pangyuteng's results.
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Spacetime
Vladimir,
Hmm… If you want to compare it with pangyuteng's results, then you have to provide an end date of self.SetEndDate(2020, 1, 1) because pangyuteng's model ends around that time. Please note, pangyuteng's post is time stamped somewhere in December 2019.
Try to run it with end date parameter and check to see if performance is comparable.
I have attached a backtest run starting from self.SetStartDate(2019, 1, 1) and ending at self.SetEndDate(2020, 1, 1) and it only traded once.
Hmm…
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Vladimir
Spacetime,
I definitely like the metrics from your last backtest, but my attempts to reproduce them for some reason were unsuccessful.
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
AlMoJo
Hello everybody,
Pangyuteng this is an amazing result. Jack Simonson I also think technical indicators would help a lot.
I tested Pangyuteng version on the 2022 crash and it is not so good at dodging it. I taught maybe going hourly timeframe instead of daily could help even if I know that trading more frequently is not really good.
Anyone has an idea on how to change the timeframe? I tried to pass all the .Daily to .Hourly but obviously it wasn't that simple.
Kind Regards
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Varad Kabade
Hi AlMoJo,
To change the timeframe of the entire algorithm we recommend adding the following to the Initialize method:
Note that the algorithm uses higher frequency bars to create lower-frequency bars. Therefore to use indicators or consolidators at the hourly resolution we need to have the universe at the minute, seconds, or hourly resolution.
Best,
Varad Kabade
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Will Berger INVESTOR
Hey Guys and Sherry,
Thanks for this incredible thread. So many interesting ideas!
Will
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
To unlock posting to the community forums please complete at least 30% of Boot Camp.
You can continue your Boot Camp training progress from the terminal. We hope to see you in the community soon!