Historical Data

History Responses

Introduction

When you make a history request, consider what data returns. Depending on how you request the data, history requests return a specific data type. The data that returns is in ascending order from oldest to newest. This order is necessary to use the data to warm up indicators.

DataFrames

To get a DataFrame from a history request, use Python.

The most popular return type is a DataFrame. To request a DataFrame of historical data, call the history method with at least one Symbol object. 

class DataFrameHistoryResponseAlgorithm(QCAlgorithm):

    def initialize(self) -> None:
        self.set_start_date(2014, 12, 20)
        # Get the Symbol objects of some assets.
        symbols = [self.add_equity(ticker).symbol for ticker in ['SPY', 'QQQ']]
        # Get 3 days of daily data for the assets.
        history = self.history(symbols, 3, Resolution.DAILY)
DataFrame of daily OHLCV values for two assets.

The structure of the DataFrame depends on the dataset. In most cases, there is a mulit-index that contains the Symbol and a timestamp. The timestamps in the DataFrame are based on the data time zone. Each row of the DataFrame represents the prices at a point in time. Each column of the DataFrame is a property of that data (for example, open, high, low, and close (OHLC)).

If you request a DataFrame, LEAN unpacks the data from Slice objects to populate the DataFrame. If you intend to use the data in the DataFrame to create TradeBar or QuoteBar objects, request that the history request returns the data type you need. Otherwise, LEAN will consume computational resources populating the DataFrame.

Slices

To request Slice objects of historical data, call the History method. If you pass a list of Symbol objects, it returns data for all the securities and datasets that the Symbol objects reference. 

// Get the latest 3 data points of some securities/datasets, packaged into Slice objects.
var history = History(symbols, 3);

If you don't pass any Symbol objects, it returns data for all the data subscriptions in your notebook.

To request Slice objects of historical data, call the history method without providing any Symbol objects. It returns data for all the data subscriptions in your notebook. 

// Get the latest 3 data points of all the securities/datasets in the notebook, packaged into Slice objects.
var history = History(3);
# Get the latest 3 data points of all the securities/datasets in the notebook, packaged into Slice objects.
history = self.history(3)

When your history request returns Slice objects, the Timetime properties of these objects are based on the notebook time zone, but the EndTimeend_time properties of the individual data objects (for example, TradeBar, QuoteBar, and Tick objects) are based on the data time zone. The EndTimeend_time is the end of the sampling period and when the data is actually available. For more information about time periods, see Periods.

Lists

When you request a list, LEAN avoids consuming computational resources populating a DataFrame. If you intend to use the data in the DataFrame to create objects like a TradeBar or QuoteBar, request that the history request returns the data type you need. To request a list of objects containing the historical data, call one of the History<Type>history[Type] methods. 

public class ListHistoryResponseAlgorithm : QCAlgorithm
{
    public override void Initialize()
    {
        SetStartDate(2024, 12, 1);
        // Get the Symbol of an asset.
        var symbol = AddEquity("SPY", Resolution.Daily).Symbol;
        // Create an indicator that is a function of historical bars.
        var indicator = new AverageTrueRange(21);
        // Get the latest TradeBar objects of the asset.
        var history = History<TradeBar>(symbol, indicator.WarmUpPeriod);
        // Update the indicator with the historical TradeBar objects.
        foreach (var bar in history)
        {
            indicator.Update(bar);
        }
    }
}
class ListHistoryResponseAlgorithm(QCAlgorithm):

    def initialize(self) -> None:
        self.set_start_date(2024, 12, 1)
        # Get the Symbol of an asset.
        symbol = self.add_equity('SPY', Resolution.DAILY).symbol
        # Create an indicator that is a function of historical bars.
        indicator = AverageTrueRange(21)
        # Get the latest TradeBar objects of the asset.
        history = self.history[TradeBar](symbol, indicator.warm_up_period)
        # Update the indicator with the historical TradeBar objects.
        for bar in history:
            indicator.update(bar)

The methods return different data types, depending on how you call it. The following table shows the return type for each input:

Input Generic <Type>[Type]Input Symbol TypeOutput Data Type
TradeBarSymbolList[TradeBar]List<TradeBar>
TradeBarList[Symbol]List<Symbol>List[TradeBars]List<TradeBars>
QuoteBarSymbolList[QuoteBar]List<QuoteBar>
QuoteBarList[Symbol]List<Symbol>List[QuoteBars]List<QuoteBars>
TickSymbolList[Tick]List<Tick>
TickList[Symbol]List<Symbol>List[Ticks]List<Ticks>
alternativeDataClass
(ex: CBOE)		SymbolList[alternativeDataClass]
(ex: List[CBOE])List<alternativeDataClass>
(ex: List<CBOE>)
alternativeDataClass
(ex: CBOE)		List[Symbol]List<Symbol>List[Dict[Symbol, alternativeDataClass]]
(ex: List[Dict[Symbol, CBOE]])List<Dictionary<Symbol, alternativeDataClass>>
(ex: List<Dictionary<Symbol, CBOE>>)
 
public class ListHistoryResponseAlgorithm : QCAlgorithm
{
    public override void Initialize()
    {
        SetStartDate(2024, 12, 1);
        // Get the Symbol of some assets.
        var spy = AddEquity("SPY").Symbol;
        var aapl = AddEquity("AAPL").Symbol;
        
        // Example: List<QuoteBars>
        foreach (var quoteBars in History<QuoteBar>(new[] { spy, aapl }, 3))
        {
            var t = quoteBars.Time;
            foreach (var kvp in quoteBars)
            {
                var symbol = kvp.Key;
                var quoteBar = kvp.Value;
                var bid = quoteBar.Bid;
            }
        }
        
        // Example: List<Ticks>
        foreach (var ticks in History(new[] { spy, aapl }, TimeSpan.FromDays(3), Resolution.Tick))
        {
            foreach (var kvp in ticks)
            {
                var symbol = kvp.Key;
                var tick = kvp.Value;
                var t = tick.Time;
                var price = tick.Price;
            }
        }
        
        // Example: List<Dictionary<Symbol, QuiverInsiderTrading>>
        var spyInsider = AddData<QuiverInsiderTrading>(spy).Symbol;
        var aaplInsider = AddData<QuiverInsiderTrading>(aapl).Symbol;
        foreach (var dataBySymbol in History<QuiverInsiderTrading>(new[] { spyInsider, aaplInsider }, 365, Resolution.Daily))
        {
            foreach (var kvp in dataBySymbol)
            {
                var datasetSymbol = kvp.Key;
                var insiderTrading = kvp.Value;
                var assetSymbol = datasetSymbol.Underlying;
                var t = insiderTrading.EndTime;
                var sharesTraded = insiderTrading.Shares;
            }
        }
    }
}
class ListHistoryResponseAlgorithm(QCAlgorithm):

    def initialize(self) -> None:
        self.set_start_date(2024, 12, 1)
        # Get the Symbol of some assets.
        spy = self.add_equity('SPY').symbol
        aapl = self.add_equity('AAPL').symbol
        
        # Example: List[QuoteBars]
        for quote_bars in self.history[QuoteBar]([spy, aapl], 3):
            t = quote_bars.time
            for symbol, quote_bar in quote_bars.items():
                bid = quote_bar.bid
        
        # Example: List[Ticks]
        for ticks in self.history[Tick]([spy, aapl], timedelta(3), Resolution.TICK):
            for symbol, tick in ticks.items():
                t = tick.time
                price = tick.price
              
        # Example: List[Dict[Symbol, QuiverInsiderTrading]]
        spy_insider = self.add_data(QuiverInsiderTrading, spy).symbol
        aapl_insider = self.add_data(QuiverInsiderTrading, aapl).symbol
        for data_by_symbol in self.history[QuiverInsiderTrading]([spy_insider, aapl_insider], 365, Resolution.DAILY):
            for dataset_symbol, insider_trading in data_by_symbol.items():
                asset_symbol = dataset_symbol.underlying
                t = insider_trading.end_time
                shares_traded = insider_trading.shares

When your history request returns a list of objects, the Timetime properties of these objects are based on the time zone, but the EndTimeend_time properties of the individual objects in the list are based on the data time zone. The EndTimeend_time is the end of the sampling period and when the data is actually available. For more information about time periods, see Periods.

Missing Data Points

There are several cases where you may get fewer data points from a history request than you expect.

Illiquid Securities

The default behavior for history requests is to fill-forward the data. However, if an asset had no trading activity during the time period you request historical data for, you won't get any data for that asset.

Requesting Data Before an Asset's IPO

All assets have a day when they first started trading. If your history request includes time before an asset's initial public offering (IPO) date, you won't get any data before the IPO date because the data doesn't exist.

Requesting Data Outside Market Hours

History requests for a trailing number of data samples return data based on the market hours of assets, so they can wrap across multiple trading days to gather the amount of data you request. In contrast, history requests for trailing time periods return data based on a rolling calendar time and history requests for specific date ranges return data for whatever range you request. In these two latter types of history requests, if the period of time is outside of the market hours of an asset, you won't get any data.

Data Issues

Data issues are generally a result of human error or from mistakes in the data collection process. Any QuantConnect member can report a data issue, which our Data Team works to quickly resolve. However, while the data is still missing, your history requests won't be able to return the data. To view the list of current data issues, log in to the Algorithm Lab and then, in the left navigation bar, click Datasets > Data Issues.

You can also see our Videos. You can also get in touch with us via Discord.

Did you find this page helpful?

Contribute to the documentation: