Historical Data

History Requests

Introduction

There are two ways to request historical data in your algorithms: direct historical data requests and indirect algorithm warm up. You can use a direct historical data request at any time throughout your algorithm. It returns all of the data you request as a single object.

Trailing Data Samples

To get historical data for a trailing time period, call historyHistory method with an an integer and a Resolution. For example, if you pass an asset Symbol, 5, and Resolution.MINUTEResolution.Minute as the arguments, it returns the data of the asset during the most recent 5 minutes in the asset's market hours. These trailing minute bars can cross multiple trading days. 

public class ___Algorithm : QCAlgorithm
{
    public override void Initialize()
    {
        SetStartDate(2024, 12, 19);
        // Get the Symbol of an asset.
        var symbol = AddEquity("SPY").Symbol;
        // Get the minute-resolution TradeBar data of the asset over the trailing 5 minutes.
        var history = History<TradeBar>(symbol, 5, Resolution.Minute);
    }
}
class TrailingDataSamplesHistoryAlgorithm(QCAlgorithm):

    def initialize(self) -> None:
        self.set_start_date(2024, 12, 19)
        # Get the Symbol of an asset.
        symbol = self.add_equity('SPY').symbol
        # Get the minute-resolution TradeBar data of the asset over the trailing 5 minutes.
        history = self.history(TradeBar, symbol, 5, Resolution.MINUTE)
DataFrame of OHLCV data for SPY over the last 5 minutes.

If you don't pass a Resolution, it defaults to the resolution of the security subscription. The following example returns 3 days of data for QQQ and 3 minutes of data for SPY: 

public class TrailingDataSamplesForMulitpleAssetsHistoryAlgorithm : QCAlgorithm
{
    public override void Initialize()
    {
        SetStartDate(2024, 12, 20);
        // Add two assets with different resolutions.
        var spy = AddEquity("SPY", Resolution.Minute).Symbol;
        var qqq = AddEquity("QQQ", Resolution.Daily).Symbol;
        // Get the trailing 3 bars of data for each asset.
        var history = History<TradeBar>(new[] { spy, qqq }, 3);
    }
}
class TrailingDataSamplesForMulitpleAssetsHistoryAlgorithm(QCAlgorithm):

    def initialize(self) -> None:
        self.set_start_date(2024, 12, 20)
        # Add two assets with different resolutions.
        spy = self.add_equity('SPY', Resolution.MINUTE).symbol
        qqq = self.add_equity('QQQ', Resolution.DAILY).symbol
        # Get the trailing 3 bars of data for each asset.
        history = self.history(TradeBar, [spy, qqq], 3)
DataFrame of OHLCV data for two assets with different data resolutions.

If there is no data for the time period you request, the result has fewer samples. For instance, say an illiquid asset has no trading activity during the last 15 minutes of the trading day and you request the 10 most recent minute bars at market close, 4 PM Eastern Standard Time (ET). In this case, you won't get any data because LEAN will try to fetch data from 3:50 PM ET to 4 PM ET since the market was open during that time, but there were no trades for the asset. For more information about missing data points, see Missing Data Points.

Trailing Time Periods

To get historical data for a trailing time period, pass a timedeltaTimeSpan to the historyHistory method. The days of the timedeltaTimeSpan represent calendar days. 

public class TrailingTimePeriodHistoryAlgorithm : QCAlgorithm
{
    public override void Initialize()
    {
        SetStartDate(2024, 12, 19);
        // Get the Symbol of an asset.
        var symbol = AddEquity("SPY").Symbol;
        // Get the minute-resolution TradeBar data of the asset over the trailing 3 days.
        var history = History<TradeBar>(symbol, TimeSpan.FromDays(3), Resolution.Minute);
    }
}
class TrailingTimePeriodHistoryAlgorithm(QCAlgorithm):

    def initialize(self) -> None:
        self.set_start_date(2024, 12, 19)
        # Get the Symbol of an asset.
        symbol = self.add_equity('SPY').symbol
        # Get the minute-resolution TradeBar data of the asset over the trailing 3 days.
        history = self.history(TradeBar, symbol, timedelta(3), Resolution.MINUTE)
DataFrame of OHLCV data for SPY over the last 3 days.

If there is no data in the time period, the result is empty. For more information about missing data points, see Missing Data Points.

Date Ranges

To get historical data for a specific date range, call historyHistory method with start and end datetimeDateTime objects. The DateTimedatetime objects you provide are based in the notebook time zone. 

public class DateRangeHistoryAlgorithm : QCAlgorithm
{
    public override void Initialize()
    {
        SetStartDate(2024, 12, 1);
        // Get the Symbol of an asset.
        var symbol = AddEquity("SPY").Symbol;
        // Get the daily-resolution TradeBar data of the asset during 2020.
        var history = History<TradeBar>(symbol, new DateTime(2020, 1, 1), new DateTime(2021, 1, 1), Resolution.Daily);
    }
}
class DateRangeHistoryAlgorithm(QCAlgorithm):

    def initialize(self) -> None:
        self.set_start_date(2024, 12, 1)
        # Get the Symbol of an asset.
        symbol = self.add_equity('SPY').symbol
        # Get the daily-resolution TradeBar data of the asset during 2020.
        history = self.history(TradeBar, symbol, datetime(2020, 1, 1), datetime(2021, 1, 1), Resolution.DAILY)
DataFrame of daily OHLCV data for SPY during 2020.

If there is no data for the date range you request, the result is empty. For more information about missing data points, see Missing Data Points.

Flat Universe DataFrames

To get a DataFrame of historical data, use Python.

Most history requests return a flat DataFrame by default, where there is one column for each data point attribute. Universe history requests return a Series, where the values in the Series are lists of the universe data objects. For example, the following code snippet returns a Series where each value is a List[ETFConstituentUniverse]: 

class SeriesUniverseHistoryAlgorithm(QCAlgorithm):

    def initialize(self) -> None:
        self.set_start_date(2024, 12, 19)
        # Add an ETF constituents universe for SPY so you can get its historical data.
        universe = self.add_universe(self.universe.etf('SPY'))
        # Get the trailing 5 days of universe data.
        history = self.history(universe, 5, Resolution.DAILY)
Pandas series where each value is a list of ETFConstituentsUniverse objects.

To get the data into a DataFrame instead, set the flatten argument to True. In this case, the DataFrame has one column for each data point attribute.

# Get the trailing 5 days of universe data in DataFrame format
# so you can perform DataFrame wrangling operations.
history = self.history(universe, 5, Resolution.DAILY, flatten=True)
Pandas DataFrame with columns for the ETFConstituentsUniverse object attributes.

Default Values

The following table describes the assumptions of the History API:

ArgumentAssumption
resolutionLEAN guesses the resolution you request by looking at the securities you already have in your notebook. If you have a security subscription in your notebook with a matching Symbol, the history request uses the same resolution as the subscription. If you don't have a security subscription in your notebook with a matching Symbol, Resolution.MinuteResolution.MINUTE is the default.
<T>If you don't specify a type for the history request, TradeBar is the default. If the asset you request data for doesn't have TradeBar data, specify the QuoteBar type to receive history.

Additional Options

The Historyhistory method accepts the following additional arguments:

ArgumentData TypeDescriptionDefault Value
fillForwardfill_forwardbool?bool/NoneTypeTrue to fill forward missing data. Otherwise, false. If you don't provide a value, it uses the fill forward mode of the security subscription.nullNone
extendedMarketHoursextended_market_hoursbool?bool/NoneTypeTrue to include extended market hours data. Otherwise, false.nullNone
dataMappingModedata_mapping_modeDataMappingMode?DataMappingMode/NoneTypeThe contract mapping mode to use for the security history request.nullNone
dataNormalizationModedata_normalization_modeDataNormalizationMode?DataNormalizationMode/NoneTypeThe price scaling mode to use for US Equities or continuous Futures contracts. If you don't provide a value, it uses the data normalization mode of the security subscription.nullNone
contractDepthOffsetcontract_depth_offsetint?int/NoneTypeThe desired offset from the current front month for continuous Futures contracts.nullNone
 
future = qb.add_future(Futures.Indices.SP_500_E_MINI)
history = qb.history(
    symbols=[future.symbol], 
    start=qb.time - timedelta(days=15), 
    end=qb.time, 
    resolution=Resolution.MINUTE, 
    fill_forward=False, 
    extended_market_hours=False, 
    data_mapping_mode=DataMappingMode.LAST_TRADING_DAY, 
    data_normalization_mode=DataNormalizationMode.RAW, 
    contract_depth_offset=0
)
var future = qb.AddFuture(Futures.Indices.SP500EMini);
var history = qb.History(
    symbols: new[] {future.Symbol}, 
    start: qb.Time - TimeSpan.FromDays(15),
    end: qb.Time,
    resolution: Resolution.Minute,
    fillForward: false,
    extendedMarketHours: false,
    dataMappingMode: DataMappingMode.LastTradingDay,
    dataNormalizationMode: DataNormalizationMode.Raw,
    contractDepthOffset: 0);

Examples

The following examples demonstrate some common practices for trading using historical requests.

Example 1: Mean-Variance Portfolio

The following algorithm constructs a monthly rebalance mean-variance portfolio using the top 20 liquid equities. The position sizing can be optimized by 1-year historical daily return of the universe members.

public class HistoricalRequestAlgorithm : QCAlgorithm
{
    private Universe _universe;
    private IPortfolioOptimizer _optimizer;

    public override void Initialize()
    {
        SetStartDate(2020, 4, 1);
        SetEndDate(2020, 9, 30);
        
        // Monthly renewal of the top 20 liquid universe to trade popular stocks.
        UniverseSettings.Schedule.On(DateRules.MonthStart());
        _universe = AddUniverse(Universe.DollarVolume.Top(20));

        // Instantiate the optimizer to perform mean-variance optimization.
        // Mean-variance optimization will not consider a risk-free rate, so we use 0.
        _optimizer = new MaximumSharpeRatioPortfolioOptimizer(0, 1, 0);

        // Set a scheduled event to rebalance the portfolio at the start of every month.
        Schedule.On(
            DateRules.MonthStart(),
            TimeRules.At(9, 31),
            Rebalance
        );
    }

    private void Rebalance()
    {
        // Historical data request to get 1-year data for optimization.
        var symbols = _universe.Selected.ToList();
        var history = History<TradeBar>(symbols, 253, Resolution.Daily)
            .Where(x => symbols.All(y => x.ContainsKey(y)) && x.All(y => y.Value.Close > 0m))
            .ToList();
        // Daily return on the universe members to calculate the optimized weights.
        var returns = GetReturns(history, symbols);

        // Calculate the optimized weights.
        var weights = _optimizer.Optimize(returns);

        // Rebalance the portfolio according to the optimized weights.
        var targets = Enumerable.Range(0, weights.Length)
            .Select(i => new PortfolioTarget(symbols[i], Convert.ToDecimal(weights[i])))
            .ToList();
        SetHoldings(targets, liquidateExistingHoldings: true);
    }

    private double[,] GetReturns(List<DataDictionary<TradeBar>> history, List<Symbol> symbols)
    {
        // Create a 2d array of historical daily returns from historical price data.
        var returns = new double[history.Count, symbols.Count];
        for (int j = 0; j < symbols.Count; j++)
        {
            var lastPrice = 0m;
            for (int i = 0; i < history.Count; i++)
            {
                var current = history[i][symbols[j]].Close;
                if (i > 0)
                {
                    returns[i, j] = (double)((current - lastPrice) / lastPrice);
                }
                lastPrice = current;
            }
        }
        return returns;
    }
}
from Portfolio.MaximumSharpeRatioPortfolioOptimizer import MaximumSharpeRatioPortfolioOptimizer

class HistoricalRequestAlgorithm(QCAlgorithm):
    def initialize(self) -> None:
        self.set_start_date(2020, 4, 1)
        self.set_end_date(2020, 9, 30)
        
        # Monthly renewal of the top 20 liquid universe to trade popular stocks.
        self.universe_settings.schedule.on(self.date_rules.month_start())
        self._universe = self.add_universe(self.universe.dollar_volume.top(20))

        # Instantiate the optimizer to perform mean-variance optimization.
        # Mean-variance optimization will not consider a risk-free rate, so we use 0.
        self._optimizer = MaximumSharpeRatioPortfolioOptimizer(0.0, 1.0, 0.0)

        # Set a scheduled event to rebalance the portfolio at the start of every month.
        self.schedule.on(
            self.date_rules.month_start(), 
            self.time_rules.at(9, 31),
            self.rebalance
        )

    def rebalance(self) -> None:
        # Historical data request to get 1-year data for optimization.
        symbols = self._universe.selected
        history = self.history(symbols, 253, Resolution.DAILY).close.unstack(0).dropna()
        # Daily return on the universe members to calculate the optimized weights.
        returns = history.pct_change().dropna()

        # Calculate the optimized weights.
        weights = self._optimizer.optimize(returns)

        # Rebalance the portfolio according to the optimized weights.
        targets = [PortfolioTarget(symbol, size) for symbol, size in zip(symbols, weights)]
        self.set_holdings(targets, liquidate_existing_holdings=True)

Other Examples

For more examples, see the following algorithms:

Demonstration Algorithm
HistoryAlgorithm.py Python HistoryAlgorithm.cs C#

You can also see our Videos. You can also get in touch with us via Discord.

Did you find this page helpful?

Contribute to the documentation: