Share New Research 
I'd like to try a strategy using the z-score of a ratio of two assets, but it seems we don't have a built-in for this, and the built-in function for standard deviation will only accept a singular security. Anybody here happen to have an example of how to go about this one?
QUANTCONNECT COMMUNITY
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
QuantConnect™ 2024. All Rights Reserved
Robert Christian
I was able to get this working, but in a rather hacky way. I'm hoping someone will be kind enough to reply with a better solution than this.
Python is not my forte, and I'm not trained in data science, so probably this is not the best solution, and I wonder about the memory impact. All the example code I could find for rolling z-score, involved pandas, so I just imported it and used a pandas series object.
import pandas as pd import math asset_class = 'stocks' pairs_list = { 'stocks': [ ('CVX', 'XOM'), ('EGBN', 'FMBI') ], 'forex': [ ('EURUSD', 'GBPUSD') ] } (symbol1, symbol2) = pairs_list[asset_class][0] class MeanReversionResearch(QCAlgorithm): def Initialize(self): self.SetStartDate(2020, 12, 3) self.SetEndDate(2020, 12, 5) self.SetCash(100_000) self.ZScoreSeries = Series("Z Score", SeriesType.Line, 0) chart = Chart("Z Score") chart.AddSeries(self.ZScoreSeries) self.AddChart(chart) self.Ratios = pd.Series() if asset_class == 'stocks': self.AddEquity(symbol1, Resolution.Minute) self.AddEquity(symbol2, Resolution.Minute) elif asset_class == 'forex': self.AddForex(symbol1, Resolution.Minute) self.AddForex(symbol2, Resolution.Minute) def OnData(self, data): if data.ContainsKey(symbol1) and data.ContainsKey(symbol2): price1 = data[symbol1].Close price2 = data[symbol2].Close self.ZScore(price1 / price2) def ZScore(self, ratio): self.Debug(f'ratio: {ratio}') self.Ratios = self.Ratios.append(pd.Series([ratio])) x = self.Ratios.rolling(window = 1).mean() w = self.Ratios.rolling(window = 20) score = (x - w.mean()) / w.std() if score.size > 0: s = score.iat[-1] if math.isnan(s): return else: self.ZScoreSeries.AddPoint(self.Time, s)Â
Adam W
The method that Robert shared looks good, but has some computational overhead (via pandas.Series.rolling() which requires repeatedly computing cumulants on the past N observations). As N grows, the cost becomes more expensive - though the example limits the "window size" to 20 which alleviates this at the cost of precision of the sample estimates.
Another way would be to do this more incrementally like:
import math class MyAlgorithm(QCAlgorithm): def Initialize(self): self.AAPL_SPY_SpreadRatio = SpreadRatio('AAPL_SPY') def OnData(self, data): # Latest observation of AAPL/SPY AAPL_Close = data['AAPL'].Close SPY_Close = data['SPY'].Close AAPL_SPY_Ratio = AAPL_Close / SPY_Close # Update sample estimates of mean/variance self.AAPL_SPY_SpreadRatio.UpdateEstimates( AAPL_SPY_Ratio ) # Get the latest Z-score of the spread AAPL_SPY_ZScore = self.AAPL_SPY_SpreadRatio.GetZScore( AAPL_SPY_Ratio ) # Plot stuff self.Plot('Z Score of Ratios', 'AAPL/SPY', AAPL_SPY_ZScore) class SpreadRatio: def __init__(self, tickers): self.tickers = tickers self.N = 1 # Number of observations self.mean = 0 # Sample mean of observed x's self.var = 0 # Sample variance of observed x's self.squared_differences = 0 # Sum of squares of deviations of x from mean def UpdateEstimates(x): # Incrementally update sample estimates via Welford (1962) new_mean = self.mean + ( ( x - self.mean ) / self.N ) self.squared_differences += (x - self.mean) * (x - new_mean) new_var = self.squared_differences / self.N self.N += 1 self.mean = new_mean self.var = new_var def GetZScore(x): z_score = ( x - self.mean ) / math.sqrt(self.var) return z_scoreFor K symbols, you can add a for-loop then collect all the computed z-scores into a dict.
Robert Christian
This looks great, thank you! But I'm getting a division by zero error. It looks like new_mean will equal the first ratio passed in:
new_mean = 0 + ((x - 0) / 1)
# new_mean = x
squared_differences = 0 + (x - 0) * (x - x)
# squared_differences = 0
var = 0 / 1
# var = 0
z_score = (x - 0) / math.sqrt(0)
# x / 0: division by zero error
What is the correct way to fix this?
Thanks again.
Robert Christian
I tried making the division by square root conditional, and this resolved the error for me. I don't know if the math is right, but the chart looks right.
sqrt = math.sqrt(self.var) if sqrt == 0.0: return x - self.mean else: return (x - self.mean) / sqrtÂ
Adam W
Good catch - typed that up pretty quickly. It would throw an 0-division error during the first run since the variance is 0 (or if the spread ratio is exactly the same consecutively for whatever reason).
Many different ways to fix it, the conditional division looks good or adding some boolean flag like:
class MyAlgorithm(QCAlgorithm): def OnData(self, data): # if data.ContainsKey(etc) # Check if sufficient variance is observed if self.AAPL_SPY_SpreadRatio.isReady: AAPL_SPY_ZScore = self.AAPL_SPY_SpreadRatio.GetZScore( AAPL_SPY_Ratio ) class SpreadRatio: def __init__(self, tickers): self.tickers = tickers self.N = 1 # Number of observations self.mean = 0 # Sample mean of observed x's self.var = 0 # Sample variance of observed x's self.squared_differences = 0 # Sum of squares of deviations of x from mean self.isReady = False # Check if variance is non-zero def UpdateEstimates(self, x): # Incrementally update sample estimates via Welford (1962) new_mean = self.mean + ( ( x - self.mean ) / self.N ) self.squared_differences += (x - self.mean) * (x - new_mean) new_var = self.squared_differences / self.N self.N += 1 self.mean = new_mean self.var = new_var # Catch 0-variance case if math.sqrt(self.var) != 0.0: self.isReady = True def GetZScore(self, x): z_score = ( x - self.mean ) / math.sqrt(self.var) return z_scoreÂ
Derek Melchin
Hi Robert and Adam,
We can simplify this further by utilizing the SimpleMovingAverage and StandardDeviation indicators. If we update the indicators with the price ratio at each time step, then we can compute the z-score with
z_score = (ratio - self.sma.Current.Value) / self.std.Current.ValueSee the attached backtest for reference.
Best,
Derek Melchin
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Robert Christian
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
To unlock posting to the community forums please complete at least 30% of Boot Camp.
You can continue your Boot Camp training progress from the terminal. We hope to see you in the community soon!