Hi QC Community,
As Mercedes-Benz would say, some things are too good to not share. So here (attached) is a ML Crypto Algo template for you to add to your quant research tools. It was built with a lot of help from QC Support so everyone should benefit. (Thanks Louis Szeto, Varad Kabade & Vladimir). This is a very instructive all-in-one ML template on 1) how to structure your code to initialize & handle rolling windows and indicators in the QC Algorithm Framework, 2) how to add Indicators from the QC indicator suite, 3) creating indicators from indicators, 4) creating a dataframe for your feature set from the ‘rolling windows of’ indicators, and 5) preparing your feature dataset to pass to the Machine Learning model.
This example uses ‘BTCUSD’ crypto, but really, that can be easily switched out in order to experiment with any asset type. It is set up also in general to emit both long and short signals / insights, even though the Bitcoin cash market cannot be shorted. You can also easily plug in and try any of your favorite Machine Learning models (Neural Network, Ridge Classifier, SVC, Random Forest, XGBoost, etc). Enjoy and feel free to post your enhancements!
Sheikh
Varad Kabade
Hi Sheikh,
Thank you for sharing the above algorithm with the community. Moving forward, we performed hyper-parameter tuning on the last hidden layer to maximize the Sharpe ratio. We can see the results in the following image. The Optimizer tool can be extensively used to develop ML algorithms and even help in creating different types of ensemble models. Refer to the attached backtest.
Best,
Varad Kabade
Vladimir
Sheikh Pancham
Thanks for sharing.
Can someone explain why I get different results for the
same algorithm three times in a row?
1
2
3
Sheikh Pancham
Hi Vladimir - Neural Network algorithms are stochastic. This means they make use of randomness, such as initializing to random weights, and in turn the same network trained on the same data can produce different results. You can seed the random number generator so that you get the same results from the same neural network on the same data, every time using the following statement in GetMLModel():
self.MLModel = MLPClassifier(random state = 123, hidden_layer_sizes = (100, 166, 100), max_iter = 1000)
More on sklearn models can be found here if you would like to experiment with a few:
https://scikit-learn.org/stable/supervised_learning.html
Hi Varade - Thanks very much for extending the research on this ML algo with your optimization exercise. I'm taking a look and will get back shortly.
Spacetime
correction: random_state
Vladimir
I know that if I set the random seed to a specific number,
I can reproduce the backtest results.
self.MLModel = MLPClassifier(random_state = 123, hidden_layer_sizes = (100, 166, 100), max_iter = 1000)
Backtested 3 times with the same results.
Thank you Sheikh Pancham
Sheikh Pancham
(You're welcome, Vladimir. Thanks Spacetime)
Hi Varade, thanks again. I tried to follow what you did but instead on the hidden layer. The optimization has been hanging for hours so no results yet. I submitted a support ticket. Basically, I'm trying to optimize the scaling parameter, nn_scale, to calculate the optimal number of neurons in the hidden layer in the following calculation.:
Below is the code, maybe someone else will have better luck with the optimizer and also try it on two parameters.
Thanks!
Varad Kabade
Hi Sheikh,
We have used the above logic to optimize using the parameters:lookback period. We recommend using the O8-16 node for the optimation of algorithms involving ML models. You can find the results in the following image. Note that the optimation time is heavily dependent on the individual backtest time. Making the model lighter can speed things greatly. We used ten in the attached backtest instead of 100 neurons in the input and output layer. The average time for running a backtest was 13 minutes, and we ran a total of 4 backtests. Refer to the attached backtest.
Best,
Varad Kabade
Sheikh Pancham
Thanks Varade, very helpful as always, will take a look.
Axist
First, just want to say thanks for posting this algo Sheikh. Its quite a bit more indepth than the first random forest algo I worked with. Been watching this thread with interest.
I am trying to make this be able to trade multiple tickers. This is what I got so far:
The error I am getting:
During the algorithm initialization, the following exception has occurred: Trying to dynamically access a method that does not exist throws a TypeError exception. To prevent the exception, ensure each parameter type matches those required by the PublicKeyToken=7cec85d7bea7198e]]'>) method. Please checkout the API documentation.
at __init__
self.fast_volume_LWMA_indicator = self.algo.LWMA(self.symbols in main.py:line 150
TypeError : No method matches given arguments for LWMA: (<class 'dict'>, <class 'int'>, <class 'int'>, <class 'System.0, Culture=neutral, PublicKeyToken=7cec85d7bea7198e]]'>)
I feel like I got the part in the initialize correct, but I feel like I might need to do a for loop when it gets to this part:
Mislav Sagovac
I haven't looked at code at all, but would like for one explanation. In the backtest results, we can see the cumulative return is cca 9x. But in the same period BTC increases by 50x. If the algo only invests in BTC, than the benchmark should be BTC, and than the results are not that great. Just the opposite, the algo is underperforming buy and hold in BTC?
Sheikh Pancham
Hi Axist, you're welcome, glad you found the algo useful. You can just plug in the ML model you'd like to use such as Random Forest with the appropriate hyper-parameters. The algo is a work-in-progress and QC Support (Varad Kabade, Louis Szeto) have been generously giving their time to improve it.
The next steps for enhancement would definitely be what you are attempting: 1) adapt it to handle multiple symbols, 2) use daily data consolidated from minute data. I'm sure QC Support will get to it. In the meantime, you can look at what Louis Szeto did in the example here to handle multiple symbols.
Hi Mislav, yes you are correct. According to the Alpha Streams submission rules, the benchmark should be relevant to the asset type being traded. The algo though is just an ML template that is being updated for members to easily plug in their research ideas. Hopefully, it will become as general as possible. As is, it is not a winning algo. Community members are free to apply their own ML model, assets and indicators.
Sheikh
Carpediem911
Hi, many thanks for sharing the code! Great idea to put a kind of general ML approach.
Just wondering:
best wishes and many thanks!
Louis Szeto
Hi Carpediem911
Is there a reason for putting the training part not in the “train” method to have more time for traning?Also is it possible with sklearn to save the train models in objectstore?
Yes, you can save/read any data including sklearn models into/from objectstore by using JSON format (docs). For getting the model to be converted into a JSON file, you can find this stack overflow's thread as example. Basically you will need a dictionary to save 2 key-value pairs within it:
Then save it using our inbuilt self.ObjectStore.SaveJson method. To load it, after self.ObjectStore.ReadJson, you will need to call the model class and set its initial parameters and trained parameters.
Best,
Louis Szeto
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Sheikh Pancham
Hi Varade,
Updated ML algo code below. 1) Uses Ridge Classifier instead of Neural Network model, 2) Uses reversed order for feature set dataframe, and 3) Uses daily data consolidated from minute resolution data. When you have a moment can you pls check that I am updating the rolling windows and indicators (and indicator extended from indicator) correctly in this “daily consolidated” format.
Thanks / Sheikh
Varad Kabade
Hi Sheikh,
Thank you for sharing the above algorithm with the community. The algorithm doesn't fully warm up all of the indicators. For instance, slow_SMA_indicator has a Period of 60 days, but only lookback (30) days of history is used to warm up the indicators. The Update method should check if the indicator IsReady before adding it's value to the RollingWindows.
Best,
Varad Kabade
Sheikh Pancham
(Thanks Varad)
Hello QC Support,
With respect to saving & retrieving an ML model:
I trained an ML model locally in a Jupyter notebook and I want to be able to use it in a QC algo. I don't want to do any training in the QC algo. Also, I don't want to use object store as I hope to submit the algo to the Alpha marketplace so no importing. I want to extract the hyperparameters and fitted parameters from the fitted ML model and to just put that data in the algo itself and set that data in the ML model in the algo.
I tried the getting & setting steps from above (as stated by Louis Szeto) but am running into an error on the setting part:. Here is the code, please have a look when you get a moment:
Kindly have a look when you have a moment.
Thanks / Sheikh
Varad Kabade
Hi Sheikh,
Although it is possible to store ML models in JSON format sometimes, it can get very messy depending on the ML model used. An alternative and the easier way would be to use pickle or JobLib as both libraries are supported in QC. Please refer to this page for detailed information.
To save an ML model :
To load a saved ml model:
Best,
Varad Kabade
Sheikh Pancham
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
To unlock posting to the community forums please complete at least 30% of Boot Camp.
You can continue your Boot Camp training progress from the terminal. We hope to see you in the community soon!