book
Checkout our new book! Hands on AI Trading with Python, QuantConnect, and AWS Learn More arrow

Historical Data

History Responses

Introduction

When you make a history request, consider what data returns. Depending on how you request the data, history requests return a specific data type. The data that returns is in ascending order from oldest to newest. This order is necessary to use the data to warm up indicators.

DataFrames

The most popular return type is a DataFrame. To request a DataFrame of historical data, call the history method with at least one Symbol object.

class DataFrameHistoryResponseAlgorithm(QCAlgorithm):

    def initialize(self) -> None:
        self.set_start_date(2014, 12, 20)
        # Get the Symbol objects of some assets.
        symbols = [self.add_equity(ticker).symbol for ticker in ['SPY', 'QQQ']]
        # Get 3 days of daily data for the assets.
        history = self.history(symbols, 3, Resolution.DAILY)
closehighlowopenvolume
symboltime
SPY2024-12-17 16:00:00604.29605.16602.89604.2238527534.0
2024-12-18 16:00:00586.28606.40585.89604.0080642184.0
2024-12-19 16:00:00586.10593.00585.85591.4370752398.0
QQQ2024-12-17 16:00:00535.80537.49534.13536.4125048673.0
2024-12-18 16:00:00516.47536.87515.01535.1147016560.0
2024-12-19 16:00:00514.17521.75513.83521.1142192908.0

The structure of the DataFrame depends on the dataset. In most cases, there is a mulit-index that contains the Symbol and a timestamp. The timestamps in the DataFrame are based on the data time zone. Each row of the DataFrame represents the prices at a point in time. Each column of the DataFrame is a property of that data (for example, open, high, low, and close (OHLC)).

If you intend to use the data in the DataFrame to create TradeBar or QuoteBar objects, request that the history request returns the data type you need. Otherwise, LEAN consumes unnecessary computational resources populating the DataFrame.

Slices

To request Slice objects of historical data, call the history method without providing any Symbol objects. It returns data for all the data subscriptions in your notebook.

Select Language:
# Get the latest 3 data points of all the securities/datasets in the notebook, packaged into Slice objects.
history = self.history(3)

When your history request returns Slice objects, the time properties of these objects are based on the algorithm time zone, but the end_time properties of the individual data objects (for example, TradeBar, QuoteBar, and Tick objects) are based on the data time zone. The end_time is the end of the sampling period and when the data is actually available. For more information about time periods, see Periods.

Lists

If you intend to use the data in the DataFrame to create objects like a TradeBar or QuoteBar, request that the history request returns the data type you need. Otherwise, LEAN consumes unnecessary computational resources populating the DataFrame. To request a list of objects containing the historical data, call one of the history[Type] methods.

Select Language:
class ListHistoryResponseAlgorithm(QCAlgorithm):

    def initialize(self) -> None:
        self.set_start_date(2024, 12, 1)
        # Get the Symbol of an asset.
        symbol = self.add_equity('SPY', Resolution.DAILY).symbol
        # Create an indicator that is a function of historical bars.
        indicator = AverageTrueRange(21)
        # Get the latest TradeBar objects of the asset.
        history = self.history[TradeBar](symbol, indicator.warm_up_period)
        # Update the indicator with the historical TradeBar objects.
        for bar in history:
            indicator.update(bar)

The methods return different data types, depending on how you call it. The following table shows the return type for each input:

Input Generic [Type]Input Symbol TypeOutput Data Type
TradeBarSymbolList[TradeBar]
TradeBarList[Symbol]List[TradeBars]
QuoteBarSymbolList[QuoteBar]
QuoteBarList[Symbol]List[QuoteBars]
TickSymbolList[Tick]
TickList[Symbol]List[Ticks]
alternativeDataClass
(ex: CBOE)
SymbolList[alternativeDataClass]
(ex: List[CBOE])
alternativeDataClass
(ex: CBOE)
List[Symbol]List[Dict[Symbol, alternativeDataClass]]
(ex: List[Dict[Symbol, CBOE]])
Select Language:
class ListHistoryResponseAlgorithm(QCAlgorithm):

    def initialize(self) -> None:
        self.set_start_date(2024, 12, 1)
        # Get the Symbol of some assets.
        spy = self.add_equity('SPY').symbol
        aapl = self.add_equity('AAPL').symbol
        
        # Example: List[QuoteBars]
        for quote_bars in self.history[QuoteBar]([spy, aapl], 3):
            t = quote_bars.time
            for symbol, quote_bar in quote_bars.items():
                bid = quote_bar.bid
        
        # Example: List[Ticks]
        for ticks in self.history[Tick]([spy, aapl], timedelta(3), Resolution.TICK):
            for symbol, tick in ticks.items():
                t = tick.time
                price = tick.price
              
        # Example: List[Dict[Symbol, QuiverInsiderTrading]]
        spy_insider = self.add_data(QuiverInsiderTrading, spy).symbol
        aapl_insider = self.add_data(QuiverInsiderTrading, aapl).symbol
        for data_by_symbol in self.history[QuiverInsiderTrading]([spy_insider, aapl_insider], 365, Resolution.DAILY):
            for dataset_symbol, insider_trading in data_by_symbol.items():
                asset_symbol = dataset_symbol.underlying
                t = insider_trading.end_time
                shares_traded = insider_trading.shares

When your history request returns a list of objects, the time properties of these objects are based on the time zone, but the end_time properties of the individual objects in the list are based on the data time zone. The end_time is the end of the sampling period and when the data is actually available. For more information about time periods, see Periods.

Missing Data Points

There are several cases where you may get fewer data points from a history request than you expect.

Illiquid Securities

The default behavior for history requests is to fill-forward the data. However, if an asset had no trading activity during the time period you request historical data for, you won't get any data for that asset.

Requesting Data Before an Asset's IPO

All assets have a day when they first started trading. If your history request includes time before an asset's initial public offering (IPO) date, you won't get any data before the IPO date because the data doesn't exist.

Requesting Data Outside Market Hours

History requests for a trailing number of data samples return data based on the market hours of assets, so they can wrap across multiple trading days to gather the amount of data you request. In contrast, history requests for trailing time periods return data based on a rolling calendar time and history requests for specific date ranges return data for whatever range you request. In these two latter types of history requests, if the period of time is outside of the market hours of an asset, you won't get any data.

Data Issues

Data issues are generally a result of human error or from mistakes in the data collection process. Any QuantConnect member can report a data issue, which our Data Team works to quickly resolve. However, while the data is still missing, your history requests won't be able to return the data. To view the list of current data issues, log in to the Algorithm Lab and then, in the left navigation bar, click Datasets > Data Issues.

You can also see our Videos. You can also get in touch with us via Discord.

Did you find this page helpful?

Contribute to the documentation: