Hi all,

I have been working on an implementation of the continuous futures prices for a universe of futures and in the process have found myself using "dirty hacks" so thought the community might help to achieve a better design.

My current implementation is the following (also in the attached backtest):

  1. Declare universe and store top-level contract names in a dictionary
  2. Inside OnData for each symbol add Daily consolidator for N most popular contracts, as advised here
  3. Inside DailyHandler, I create a pandas' Series object that contains Symbol, Close Price, Open Interest and Date and then append it to the `db` property of the algo class.
  4. A function scheduled to run daily then iterates through unique symbols in the `db` and for each symbol it creates a continuous price series.
    The particular methodology is not that important because at this step one can use any method. In this case, I use Perpetual Series based on Open Interest as described here. For each symbol I contruct 2 pivot tables, the first one with Open Interest (normalized to 1 for each row, as it is used as weights) and the second one with Close Price. Then I take element-wise product of the 2 tables and sum columns to get a Series of weighted average prices.
    All individual price series are then concatenated into a single data frame and written into algo's property `continuous_future`. 
  5. Now `self.continuous_future` can be used for longer term technical analysis.

There are several issues with that implementation:

  1. Price data is stored outside the `Slice` object, and not linked to a Symbol. This means that QC's rich toolkit, that includes Charting, Indicators and Insights cannot be used as it always requires Symbol object. I wonder if I create symbol manually through Symbol.Create method, can I then update data associated with this symbol, so that it is reflected in Slice later?
  2. This method generally is extremely slow. Not least because it uses pandas but I can't think of a cleaner way of creating the continuation logic without it. So any suggestion on how to speed it up would be great.
  3. Any other feedback is much appreciated.