Hello Community!
We're happy to share we've finally shipped cloud parameter optimization for parameter sensitivity testing. This feature lets you spin up a small cluster of servers for a few minutes to run up to 24 backtests in parallel. It joins all the results together in a nice view for you to explore the aggregate data and find areas of parameter "strength".
We recommend you use it to test your parameter robustness. If you find a sharp peak in returns on a single parameter value that's a sign it's been overfitted. If you see a wide range of parameter values that show a strong result it's reasonably safe to pick a parameter in that zone.
Parameters are pulled into your strategy from the GetParameter method. This takes the name of the parameter and returns the value at this step in time.
fast = self.GetParameter("ema-fast")
slow = self.GetParameter("ema-slow")
Check out the demonstration in this video below. We'll follow in the next weeks with documentation on how to use it and a tutorial video.
class EmaCrossParameterAlgorithm(QCAlgorithm):
def Initialize(self):
self.SetStartDate(2010,1,1)
self.SetEndDate(2020,1,1)
self.SetCash(100000)
self.slow = int(self.GetParameter("ema-slow"))
self.fast = int(self.GetParameter("ema-fast"))
self.symbol = self.AddEquity("QQQ", Resolution.Hour).Symbol
self.emaFast = self.EMA(self.symbol, self.fast)
self.emaSlow = self.EMA(self.symbol, self.slow)
def OnData(self, slice):
if self.emaFast.Current.Value > self.emaSlow.Current.Value:
self.SetHoldings(self.symbol, 1)
else:
self.SetHoldings(self.symbol, 0)
PS: It may be tempting to use this to hyper tune the backtest but resist the urge! It is generally meaningless to look at a single backtest with this analysis tool. Try always to pick strategies with meaning, and assign parameters meaningful values. E.g. Rainfall per day has a realistic range of values.
Pi..R
Awsome feature, thanks you very much!
Could you please post the equivalent example in C#?
I have tried the using the GetParameter method as follow, but it returns an error:
public int slow = GetParameter("ema-slow");
or
slow = (int)GetParameter("ema-slow");
Vncne
Hi Jared, do you think you could include in the upcoming documentation how to apply the optimization feature to parameters used/initialized during the universe selection?
Jared Broad
Thanks no problem Vncne, we'll make sure to include that.
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Jared Broad
Hey Pi..R ! C# has implicit syntax like this:
[Parameter("ema-fast")] public int FastPeriod = 100;
You can also use explicit accessors like Python:
var emaSlow = Convert.ToInt32(GetParameter("ema-slow"));
https://github.com/QuantConnect/Lean/blob/master/Algorithm.CSharp/ParameterizedAlgorithm.cs#L35The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Pi..R
Many thanks Jarod, it worked!
Dat En
Jared Broad
Derek mentioned in the thread below that parameter optimization will be available to run locally and I duly waited for the release. Have you now changed it to cloud only?
https://www.quantconnect.com/forum/discussion/5259/optimization-locally-in-lean-visual-studio/p1Jared Broad
It is in LEAN. As Derek notes in that comment, you could have followed along with the linked issue.
We don't have documentation for it yet but will get there soon. Otherwise set the start-up project to Optimization Launcher and edit the configuration appropriately.
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Harry Jones
That is some serious functionality there! Cheers Jared, I will give it a go.
Levi Freedman
Hey nice. Is there a function to skip a backtest? For example, if self.GetParameter("ema-slow") <= self.GetParameter("ema-fast")
Apollos Hill
im very impressed and can't wait to try this out!
Vladimir
I tried today and am happy how it works.
Some suggestions, if possible:
- Add a benchmark chart.
- switch to logarithmic scale and back to normal.
Emiliano Fraticelli
Is the maximum 2 parameters at once?
Derek Melchin
Hi Emiliano,
Yes, at the moment Optimization only support 2 parameters. We will improve the Optimization feature in the future and that will include more degrees of freedom.
Best,
Derek Melchin
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Emiliano Fraticelli
Including free, offline optimization? Derek Melchin
Jamie Lee
the columns in Optimzation result are off from the individual backtest results. some examples are as below, see my result output.
PSR -> Net Profit
Sharp Ratio -> Profit-Loss Ratio
Net Profit -> Loss Rate
Total Trades -> Total Fees
Derek Melchin
Hi Emiliano,
LEAN already supports local optimization. However, we don't currently have a tutorial for it.
Hi Jamie,
Thanks for reporting this. We've notified the UX team to have it resolved.
Best,
Derek Melchin
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Edinson Leandro Medina Alfonzo
Hi Community!
Recently, I have seen this video and also I have read all the comments about people who have tried Optimization. Did you know if this new feature has been added to Documentation? Where can I read more about it?
Grettings.
Derek Melchin
Hi Edinson,
There is not currently any documentation on the Optimizer. However, the wizard is very user-friendly so we recommend following the step-by-step GUI to learn how it works. We are in the process of updating our documentation and will be sure to include the new optimization feature in it.
Best,
Derek Melchin
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Lawrence tabak
Hi all,
Can we get an update on approx when Cloud Parameter Optimization will support more than 2 parameters? It really is a key feature.
Thanks!
Louis Szeto
Hi Lawrence
Sorry that right now we only support 2 parameters. Please try LEAN CLI for local optimization, which supports more than 2. We will extend this to the cloud in the future.
Best
Louis Szeto
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Jared Broad
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
To unlock posting to the community forums please complete at least 30% of Boot Camp.
You can continue your Boot Camp training progress from the terminal. We hope to see you in the community soon!