Querying historical data timestamps

Hello All,

I have a universe selection algorithm already successfully stores OHLC (Open, High, Low, Close) data into rolling windows and prepopulates those windows with historical data. I would like add another rolling window that stores the timestamps associated with those OHLC entries, but I am not sure how query the timestamps from historical pandas dataframe. Can anyone offer some guidance? The last line of the block of code below is where I am trying to query the timestamp.

    def OnSecuritiesChanged(self, algorithm, changes):
       # clean up data for removed securities
       symbols = [ x.Symbol for x in changes.RemovedSecurities ]
       if len(symbols) > 0:
           for subscription in algorithm.SubscriptionManager.Subscriptions:
               if subscription.Symbol in symbols:
                   self.symbolDataBySymbol.pop(subscription.Symbol, None)
                   subscription.Consolidators.Clear()
               
       # initialize data for added securities
       
       addedSymbols = [ x.Symbol for x in changes.AddedSecurities if x.Symbol not in self.symbolDataBySymbol]
       if len(addedSymbols) == 0: return
       
       self.windowLength = max(self.strategyLookback + 90,self.volatilityLookback)
       history = algorithm.History(addedSymbols, self.windowLength, self.resolution)
       
       for symbol in addedSymbols:
           self.openWindows[symbol] = RollingWindow[float](self.windowLength)
           self.highWindows[symbol] = RollingWindow[float](self.windowLength)
           self.lowWindows[symbol] = RollingWindow[float](self.windowLength)
           self.closeWindows[symbol] = RollingWindow[float](self.windowLength)
           self.timestampWindows[symbol] = RollingWindow[float](self.windowLength)
           for tuple in history.loc[symbol].itertuples():
               self.openWindows[symbol].Add(tuple.open)
               self.highWindows[symbol].Add(tuple.high)
               self.lowWindows[symbol].Add(tuple.low)
               self.closeWindows[symbol].Add(tuple.close)
               self.timestampWindows[symbol].Add(tuple.time)
+ Expand
- Collapse

self.dataWindows = {}
history = algorithm.History(addedSymbols, self.windowLength, self.resolution)
for symbol in addedSymbols:
self.dataWindows[symbol] = RollingWindow[dict](self.windowLength)
# Manually warm up data windows
symbolHist = history.xs(symbol, level=0).tail(self.windowLength) # Slice DataFrame by symbol and get last few values
symbolHist = symbolHist.reset_index() # Add 'time' index as a column instead
open = symbolHist['open']
high = symbolHist['high']
#... etc
time = symbolHist['time']
for t in range(self.windowLength):
data = {
'open' : open[-self.windowLength+t]
'high' : high[-self.windowLength+t]
# ... etc
'time' : time[-self.windowLength+t]
}
self.dataWindows[symbol].Add( data )
# Using the data windows
self.dataWindows['SPY'][0] = # most recent dict of 'open','high','time' etc

+ Expand

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.

self.tradeBarWindow = RollingWindow[TradeBar](period)
history = self.History(symbol, period, resolution)
for bar in history.itertuples():
#creating a trade bar from rows of history dataframe
tradebar = TradeBar(bar.Index[1], self.symbol, bar.open, bar.high, bar.low, bar.close, bar.volume)
self.tradeBarWindow.Add(tradebar)

Adam W INVESTOR

HamCap | May 2022

The DataFrame returned by `.History` has MultiIndex (symbol, time), so you can just take the index of the second level. Also might be cleaner to just put everything into one RollingWindow, like

self.dataWindows = {}
history = algorithm.History(addedSymbols, self.windowLength, self.resolution)
for symbol in addedSymbols:
    self.dataWindows[symbol] = RollingWindow[dict](self.windowLength)
    
    # Manually warm up data windows
    symbolHist = history.xs(symbol, level=0).tail(self.windowLength)  # Slice DataFrame by symbol and get last few values
    
    symbolHist = symbolHist.reset_index()  # Add 'time' index as a column instead
    open = symbolHist['open']
    high = symbolHist['high']
    #... etc
    time = symbolHist['time']
    
    for t in range(self.windowLength):
        data = {
                'open' : open[-self.windowLength+t]
                'high' : high[-self.windowLength+t]
                # ... etc
                'time' : time[-self.windowLength+t]
                }
         self.dataWindows[symbol].Add( data )
         
# Using the data windows
self.dataWindows['SPY'][0] = # most recent dict of 'open','high','time' etc
+ Expand
- Collapse

Upvote

Varad Kabade INVESTOR

May 2022

Hi Garrett,

Thank you, Adam, for your answer.

We recommend using a single RollingWindow which stores TradeBar objects so it would contain OHCLV data along with the time stamp.

To create TradeBar objects from rows of the historical data refer to the following snippet:

self.tradeBarWindow = RollingWindow[TradeBar](period)
history = self.History(symbol, period, resolution)
for bar in history.itertuples():
#creating a trade bar from rows of history dataframe
    tradebar = TradeBar(bar.Index[1], self.symbol, bar.open, bar.high, bar.low, bar.close, bar.volume)
    self.tradeBarWindow.Add(tradebar)

Best,

Varad Kabade

Platform

Radically Open-Source Algorithmic Trading Engine

Join Our Discord Channel

Quarterly Open-Source Trading Competition

Draft Discussions

Bookmarked Discussions

SEARCH DISCUSSIONS

365,500 Quants.

VOTE FOR UPCOMING FEATURES

JOIN OUR Community MAILING LIST

IN THIS RESEARCH

PARTICIPANTS

Actions

Join QuantConnect for Free