Capturing arbitrary data for indicator analysis after backtest?

To capture and analyze custom analytical data series during a backtest in QuantConnect, you can use the following methods and properties from the QCAlgorithm class:

1. Initialize Indicators: In your algorithm's initialize method, set up the indicators you want to track. Ensure they are registered with the appropriate symbols and resolutions using the register_indicator method.

2. Use the record Method: Utilize the record method within the on_data method to log the values of your indicators. This method allows you to plot a chart using a string series name and a decimal value. It is an alias for the plot method.

3. Create Custom Charts: Define custom charts and series in the initialize method to visualize the indicator data. Use the add_chart and add_series methods to set up these visualizations.

4. Log Indicator Values: Within the on_data method, continuously update the custom charts with the current values of your indicators using the plot method.

5. Store Data for Analysis: If you need to analyze the data post-backtest, consider using the Object Store to save the indicator values for later retrieval and analysis. You can use the object_store property to save and retrieve data.

Here's a basic example to illustrate these steps:

class CustomChartingAlgorithm(QCAlgorithm):
    def initialize(self):
        self.set_start_date(2022, 1, 1)
        self.set_end_date(2023, 1, 1)
        self.set_cash(100000)
        # Add an equity and register an indicator
        spy = self.add_equity("SPY", Resolution.DAILY).symbol
        self.fast_ma = self.sma(spy, 10, Resolution.DAILY)
        self.slow_ma = self.sma(spy, 50, Resolution.DAILY)
        # Create custom charts
        avg_cross = Chart("Average Cross")
        avg_cross.add_series(Series("FastMA", SeriesType.LINE, 0))
        avg_cross.add_series(Series("SlowMA", SeriesType.LINE, 0))
        self.add_chart(avg_cross)

    def on_data(self, slice):
        if not self.fast_ma.is_ready or not self.slow_ma.is_ready:
            return
        # Plot the indicator values
        self.plot("Average Cross", "FastMA", self.fast_ma.current.value)
        self.plot("Average Cross", "SlowMA", self.slow_ma.current.value)
        # Example of using Object Store to save data
        self.object_store.save("fast_ma_value", str(self.fast_ma.current.value))
        self.object_store.save("slow_ma_value", str(self.slow_ma.current.value))

This setup allows you to capture and visualize the performance of your indicators during a backtest. You can review the charts in the backtest results and use the Object Store to save data for further analysis.

Sources: Custom Charting Algorithm, Results - QuantConnect.com, Custom Indicator Algorithm, Debugging Tools - QuantConnect.com, Indicator Suite Algorithm

Thanks! I'll take a look at the examples. 

In fact, I was already using the plot() method, however the plot data was not appearing in the local backtest logs. 

Maybe the object store could be of use here? I hope it produces something under the local backtest, thanks!

This seems to have worked out, using an object store and a JSON dump of a Pandas dataframe from the indicator history. The data is then available locally under ‘<cloud_root>/storage/backtest/**/local/**'

   def on_end_of_algorithm(self) -> None:
        bb: BollingerBands = self.params[Const.INDICATOR]
        if bb.is_ready:
            symbol = self.params[Const.SYMBOL]
            bb_hist: IndicatorHistory = self.indicator_history(bb, symbol=symbol, period=bb.samples)
            bb_df: pd.DataFrame = bb_hist.data_frame
            cls = self.__class__.__name__
            name = self.name
            time = self.time.strftime("%Y%m%dT%H%M%S")
            key = f"backtest/{cls}/{name}/{time}/bb"
            
            json_path = self.object_store.get_file_path(key)
            bb_df.to_json(json_path)
        super().on_end_of_algorithm()

This uses a 'params' mapping created under ‘__init__’ to store the indicator and the primary symbol for the indicator when created under initialize()

It seems that the timestamp in the file path may be aligned to a historical timestamp in  backtest quote series.

Health, all!