Creating an indicator DataFrame inside of the QC Algorithm.

In the QuantConnect algorithm environment, you can't directly get a DataFrame from an indicator like you can in the Research environment. However, you can create a RollingWindow object to store historical indicator data, and then convert that to a DataFrame if needed.

Here's an example of how you can do this:

class MyAlgorithm(QCAlgorithm):
    def Initialize(self):
        self.symbol = self.AddEquity("SPY", Resolution.Daily).Symbol
        self.apo = AbsolutePriceOscillator(10, 20, MovingAverageType.Simple)
        self.window = RollingWindow[IndicatorDataPoint](252)  # Create a RollingWindow to store indicator data

    def OnData(self, data):
        if data.ContainsKey(self.symbol):
            self.apo.Update(self.Time, data[self.symbol].Close)
            if self.apo.IsReady:
                self.window.Add(IndicatorDataPoint(self.symbol, self.Time, self.apo.Current.Value))  # Add indicator data to the RollingWindow

    def OnEndOfAlgorithm(self):
        # Convert the RollingWindow to a DataFrame at the end of the algorithm
        df = pd.DataFrame([{'Time': x.EndTime, 'Value': x.Value} for x in self.window])
        self.Debug(str(df))

In this example, the OnEndOfAlgorithm method is used to convert the RollingWindow to a DataFrame when the algorithm finishes. You can adjust this to suit your needs.

Remember that the self.Indicator method is a helper method in the Research environment for plotting purposes. In the algorithm environment, you need to use the specific indicator constructor for each indicator. You can find more information about this in the indicator reference table in the documentation.

Also, note that the RollingWindow stores the most recent data at the front of the window (index 0), and older data is at higher indices. So self.window[0] gives the most recent data, and self.window[self.window.Count - 1] gives the oldest data in the window.

Sources:

• Forum Discussion: Historical indicators in algo environment
• Forum Discussion: Automatic Indicators and when they are calculated
• Docs: Absolute Price Oscillator

Hi Mia Alissi ,

This answer helped me a lot. I was wondering if you could help me with something that came up after I tried implementing what you explaied above:

I (think) I got the indicator rolling window going on nicely now:

# region imports
from AlgorithmImports import *
# endregion

class CrawlingBrownHyena(QCAlgorithm):

    def Initialize(self):
        self.SetStartDate(2014, 1, 1)
        self.SetEndDate(2014,1,10)
        self.SetCash(100000)
        self.SetBenchmark("SPY")
        self.spy = self.AddEquity("SPY", Resolution.Daily).Symbol
        self.apo = AbsolutePriceOscillator(10,20,MovingAverageType.Simple)
        self.apo_window = RollingWindow[IndicatorDataPoint](252)
        self.SetWarmup(252)
        self.Train(self.MyTrainingMethod)
        self.Train(self.DateRules.Every(DayOfWeek.Sunday), self.TimeRules.At(8 , 0), self.MyTrainingMethod)
        
    def OnData(self, data: Slice):
        if data.ContainsKey(self.spy):
            self.apo.Update(self.Time, data[self.spy].Close)
        if self.apo.IsReady:
            self.apo_window.Add(IndicatorDataPoint(self.spy, self.Time, self.apo.Current.Value))

    def MyTrainingMethod(self):
        self.Log(f'Start training at {self.Time}')
        history = self.History(self.spy, 252, Resolution.Daily).loc[self.spy]
        apo = pd.DataFrame([{'Time':x.EndTime,'Value':x.Value} for x in self.apo_window])
        df = pd.concat([history,apo],axis=1)
        self.Debug(str(df.tail(1)))
        self.Debug(apo.index)
        self.Debug(history.index)

However, what I am trying to do now is to concatenate the history dataframe of self.spy with the new dataframe I created with your help (apo). I tried creating a new DataFrame called df, but I noticed it is not merging it correctly.

Using the Debug statement I can clearly see that the index for the history dataframe is a DateTime Index while the index for the apo dataframe is a Range Index. I'm guessing this is what's not working.

Any tips on how I can correctly concatenate and get both the OHLC data and the indicator data in the same DataFrame?

The issue you're encountering is indeed due to the mismatch in the indices of the two dataframes. The history dataframe has a DateTime index, while the apo dataframe has a Range index.

To solve this, you can set the 'Time' column of the apo dataframe as its index before concatenating. Here's how you can do it:

apo.set_index('Time', inplace=True)
df = pd.concat([history, apo], axis=1)

This will set the 'Time' column as the index of the apo dataframe, which should allow it to concatenate correctly with the history dataframe.

Remember that pd.concat aligns dataframes along their indices. If the indices don't match, you'll get NaN values in the rows where it couldn't find a match.

Sources:

• Forum Discussion: Merge Prices and Indicators in One Single Dataframe in Research Environment
• Forum Discussion: Custom data for indicators