In This Page

Introduction

Key History Concepts

Request Data

Analyze Results

Data Formats

DataFrame Formats

Common Errors

Live Trading Considerations

Examples

Historical Data

History Requests

Introduction

There are two ways to request historical data in your algorithms: direct historical data requests and indirect algorithm warm up. You can use a direct historical data request at any time throughout your algorithm. It returns all of the data you request as a single object.

Key History Concepts

The historical data API has many different options to give you the greatest flexibility in how to apply it to your algorithm.

Time Period Options

You can request historical data based on a trailing number of bars, a trailing period of time, or a defined period of time. If you request data in a defined period of time, the DateTimedatetime objects you provide are based in the algorithm time zone.

Return Formats

Each asset class supports slightly different data formats. When you make a history request, consider what data returns. Depending on how you request the data, history requests return a specific data type. For example, if you don't provide Symbol objects, you get Slice objects that contain the entire universe.

The most popular return type is a DataFrame. If you request a DataFrame, LEAN unpacks the data from Slice objects to populate the DataFrame. If you intend to use the data in the DataFrame to create TradeBar or QuoteBar objects, request that the history request returns the data type you need. Otherwise, LEAN will waste computational resources populating the DataFrame.

Time Index

When your history request returns a DataFrame, the timestamps in the DataFrame are based on the data time zone. When your history request returns a TradeBars, QuoteBars, Ticks, or Slice object, the Timetime properties of these objects are based on the algorithm time zone, but the EndTimeend_time properties of the individual TradeBar, QuoteBar, and Tick objects are based on the data time zonedata time zone. The EndTimeend_time is the end of the sampling period and when the data is actually available. For daily US Equity data, this results in data points appearing on Saturday and skipping Monday.

Request Data

The simplest form of history request is for a known set of Symbol objects. This is common for fixed universes of securities or when you need to prepare new securities added to your algorithm. History requests return slightly different data depending on the overload you call. The data that returns is in ascending order from oldest to newest. This order is necessary to use the data to warm up indicators.

Single Symbol History Requests

To request history for a single asset, pass the asset Symbol to the Historyhistory method. The return type of the method call depends on the history request [Type]<Type>. The following table describes the return type of each request [Type]<Type>:

Request Type	Return Data Type
No argument	`DataFrameList<TradeBar>`
`TradeBar`	`List[TradeBars]List<TradeBar>`
`QuoteBar`	`List[QuoteBars]List<QuoteBar>`
`Tick`	`List[Ticks]List<Tick>`
`alternativeDataClass` (ex: `CBOE`)	`List[alternativeDataClass]` (ex: `List[CBOE]`)`List<alternativeDataClass>` (ex: `List<CBOE>`)

Each row of the DataFrame represents the prices at a point in time. Each column of the DataFrame is a property of that price data (for example, open, high, low, and close (OHLC)). If you request a DataFrame object and pass TradeBar as the first argument, the DataFrame that returns only contains the OHLC and volume columns. If you request a DataFrame object and pass QuoteBar as the first argument, the DataFrame that returns contains the OHLC of the bid and ask and it contains OHLC columns, which are the respective means of the bid and ask OHLC values. If you request a DataFrame and don't pass TradeBar or QuoteBar as the first arugment, the DataFrame that returns contains columns for all of the data that's available for the given resolution.

# EXAMPLE 1: Requesting By Bar Count: 5 bars at the security resolution:
vix_symbol = self.add_data(CBOE, "VIX", Resolution.DAILY).symbol
cboe_data = self.history[CBOE](vix_symbol, 5)

btc_symbol = self.add_crypto("BTCUSD", Resolution.MINUTE).symbol
trade_bars = self.history[TradeBar](btc_symbol, 5)
quote_bars = self.history[QuoteBar](btc_symbol, 5)
trade_bars_df = self.history(TradeBar, btc_symbol, 5)
quote_bars_df = self.history(QuoteBar, btc_symbol, 5)
df = self.history(btc_symbol, 5)   # Includes trade and quote data

// EXAMPLE 1: Requesting By Bar Count: 5 bars at the security resolution:
var vixSymbol = AddData<CBOE>("VIX", Resolution.Daily).Symbol;
var cboeData = History<CBOE>(vixSymbol, 5);

var btcSymbol = AddCrypto("BTCUSD", Resolution.Minute).Symbol;
var tradeBars = History<TradeBar>(btcSymbol, 5);
var quoteBars = History<QuoteBar>(btcSymbol, 5);
var tradeBars2 = History(btcSymbol, 5);

# EXAMPLE 2: Requesting By Bar Count: 5 bars with a specific resolution:
trade_bars = self.history[TradeBar](btc_symbol, 5, Resolution.DAILY)
quote_bars = self.history[QuoteBar](btc_symbol, 5, Resolution.MINUTE)
trade_bars_df = self.history(TradeBar, btc_symbol, 5, Resolution.MINUTE)
quote_bars_df = self.history(QuoteBar, btc_symbol, 5, Resolution.MINUTE)
df = self.history(btc_symbol, 5, Resolution.MINUTE)  # Includes trade and quote data

// EXAMPLE 2: Requesting By Bar Count: 5 bars with a specific resolution:
var tradeBars = History<TradeBar>(btcSymbol, 5, Resolution.Daily);
var quoteBars = History<QuoteBar>(btcSymbol, 5, Resolution.Minute);
var tradeBars2 = History(btcSymbol, 5, Resolution.Minute);

# EXAMPLE 3: Requesting By a Trailing Period: 3 days of data at the security resolution: 
eth_symbol = self.add_crypto('ETHUSD', Resolution.TICK).symbol
ticks = self.history[Tick](eth_symbol, timedelta(days=3))
ticks_df = self.history(eth_symbol, timedelta(days=3))

vix_data = self.history[CBOE](vix_symbol, timedelta(days=3)) 
trade_bars = self.history[TradeBar](btc_symbol, timedelta(days=3)) 
quote_bars = self.history[QuoteBar](btc_symbol, timedelta(days=3))
trade_bars_df = self.history(TradeBar, btc_symbol, timedelta(days=3))
quote_bars_df = self.history(QuoteBar, btc_symbol, timedelta(days=3))
df = self.history(btc_symbol, timedelta(days=3))  # Includes trade and quote data

// EXAMPLE 3: Requesting By a Trailing Period: 3 days of data at the security resolution:
var ethSymbol = AddCrypto("ETHUSD", Resolution.Tick).Symbol;
var ticks = History<Tick>(ethSymbol, TimeSpan.FromDays(3));

var cboeData = History<CBOE>(vixSymbol, TimeSpan.FromDays(3));
var tradeBars = History<TradeBar>(btcSymbol, TimeSpan.FromDays(3));
var quoteBars = History<QuoteBar>(btcSymbol, TimeSpan.FromDays(3));
var tradeBars2 = History(btcSymbol, TimeSpan.FromDays(3));

# EXAMPLE 4: Requesting By a Trailing Period: 3 days of data with a specific resolution: 
trade_bars = self.history[TradeBar](btc_symbol, timedelta(days=3), Resolution.DAILY) 
quote_bars = self.history[QuoteBar](btc_symbol, timedelta(days=3), Resolution.MINUTE)
ticks = self.history[Tick](eth_symbol, timedelta(days=3), Resolution.TICK)

trade_bars_df = self.history(TradeBar, btc_symbol, timedelta(days=3), Resolution.DAILY)
quote_bars_df = self.history(QuoteBar, btc_symbol, timedelta(days=3), Resolution.MINUTE)
ticks_df = self.history(eth_symbol, timedelta(days=3), Resolution.TICK)
df = self.history(btc_symbol, timedelta(days=3), Resolution.HOUR)  # Includes trade and quote data

# Important Note: Period history requests are relative to "now" algorithm time.

// EXAMPLE 4: Requesting By a Trailing Period: 3 days of data with a specific resolution:
var tradeBars = History<TradeBar>(btcSymbol, TimeSpan.FromDays(3), Resolution.Daily);
var quoteBars = History<QuoteBar>(btcSymbol, TimeSpan.FromDays(3), Resolution.Minute);
var ticks = History<Tick>(ethSymbol, TimeSpan.FromDays(3), Resolution.Tick);
var tradeBars2 = History(btcSymbol, TimeSpan.FromDays(3), Resolution.Minute);

# EXAMPLE 5: Requesting By a Defined Period: 3 days of data at the security resolution: 
start_time = datetime(2022, 1, 1)
end_time = datetime(2022, 1, 4)

vix_data = self.history[CBOE](vix_symbol, start_time, end_time) 
trade_bars = self.history[TradeBar](btc_symbol, start_time, end_time) 
quote_bars = self.history[QuoteBar](btc_symbol, start_time, end_time)
ticks = self.history[Tick](eth_symbol, start_time, end_time)

trade_bars_df = self.history(TradeBar, btc_symbol, start_time, end_time)
quote_bars_df = self.history(QuoteBar, btc_symbol, start_time, end_time)
ticks_df = self.history(Tick, eth_symbol, start_time, end_time)
df = self.history(btc_symbol, start_time, end_time)  # Includes trade and quote data

// EXAMPLE 5: Requesting By a Defined Period: 3 specific days of data at the security resolution:
var startTime = new DateTime(2022, 1, 1);
var endTime = new DateTime(2022, 1, 4);

var cboeData = History<CBOE>(vixSymbol, startTime, endTime);
var tradeBars = History<TradeBar>(btcSymbol, startTime, endTime);
var quoteBars = History<QuoteBar>(btcSymbol, startTime, endTime);
var ticks = History<Tick>(ethSymbol, startTime, endTime);
var tradeBars2 = History(btcSymbol, startTime, endTime);

# EXAMPLE 6: Requesting By a Defined Period: 3 days of data with a specific resolution: 
trade_bars = self.history[TradeBar](btc_symbol, start_time, end_time, Resolution.DAILY) 
quote_bars = self.history[QuoteBar](btc_symbol, start_time, end_time, Resolution.MINUTE)
ticks = self.history[Tick](eth_symbol, start_time, end_time, Resolution.TICK)

trade_bars_df = self.history(TradeBar, btc_symbol, start_time, end_time, Resolution.DAILY)
quote_bars_df = self.history(QuoteBar, btc_symbol, start_time, end_time, Resolution.MINUTE)
ticks_df = self.history(eth_symbol, start_time, end_time, Resolution.TICK)
df = self.history(btc_symbol, start_time, end_time, Resolution.HOUR)  # Includes trade and quote data

// EXAMPLE 6: Requesting By a Defined Period: 3 days of data with a specific resolution:
var tradeBars = History<TradeBar>(btcSymbol, startTime, endTime, Resolution.Daily);
var quoteBars = History<QuoteBar>(btcSymbol, startTime, endTime, Resolution.Minute);
var ticks = History<Tick>(ethSymbol, startTime, endTime, Resolution.Tick);
var tradeBars2 = History(btcSymbol, startTime, endTime, Resolution.Minute);

Multiple Symbol History Requests

To request history for multiple symbols at a time, pass an array of Symbol objects to the same API methods shown in the preceding section. The return type of the method call depends on the history request [Type]<Type>. The following table describes the return type of each request [Type]<Type>:

Request Type	Return Data Type
No argument	`DataFrameList<Slice>`
`TradeBar`	`List[TradeBars]List<TradeBars>`
`QuoteBar`	`List[QuoteBars]List<QuoteBars>`
`Tick`	`List[Ticks]List<Ticks>`
`alternativeDataClass` (ex: `CBOE`)	`List[Dict[Symbol, alternativeDataClass]]` (ex: `List[Dict[Symbol, CBOE]]`)`List<Dictionary<Symbol, alternativeDataClass>>` (ex: `List<Dictionary<Symbol, CBOE>>`)

The Slice return type provides a container that supports all data types. For example, a history request for Forex QuoteBars and Equity TradeBars has the Forex data under slices.QuoteBars and the Equity data under slices.Bars.

# EXAMPLE 7: Requesting By Bar Count for Multiple Symbols: 2 bars at the security resolution:
vix = self.add_data[CBOE]("VIX", Resolution.DAILY).symbol
v3m = self.add_data[CBOE]("VIX3M", Resolution.DAILY).symbol
cboe_data = self.history[CBOE]([vix, v3m], 2)

ibm = self.add_equity("IBM", Resolution.MINUTE).symbol
aapl = self.add_equity("AAPL", Resolution.MINUTE).symbol
trade_bars_list = self.history[TradeBar]([ibm, aapl], 2)
quote_bars_list = self.history[QuoteBar]([ibm, aapl], 2)

trade_bars_df = self.history(TradeBar, [ibm, aapl], 2)
quote_bars_df = self.history(QuoteBar, [ibm, aapl], 2)
df = self.history([ibm, aapl], 2)  # Includes trade and quote data

// EXAMPLE 7: Requesting By Bar Count for Multiple Symbols: 2 bars at the security resolution:
var vixSymbol = AddData<CBOE>("VIX", Resolution.Daily).Symbol;
var v3mSymbol = AddData<CBOE>("VIX3m", Resolution.Daily).Symbol;
var cboeData = History<CBOE>(new[] { vix, v3m }, 2);

var ibm = AddEquity("IBM", Resolution.Minute).Symbol;
var aapl = AddEquity("AAPL", Resolution.Minute).Symbol;
var tradeBarsList = History<TradeBar>(new[] { ibm, aapl }, 2);
var quoteBarsList = History<QuoteBar>(new[] { ibm, aapl }, 2);

# EXAMPLE 8: Requesting By Bar Count for Multiple Symbols: 5 bars with a specific resolution:
trade_bars_list = self.history[TradeBar]([ibm, aapl], 5, Resolution.DAILY)
quote_bars_list = self.history[QuoteBar]([ibm, aapl], 5, Resolution.MINUTE)

trade_bars_df = self.history(TradeBar, [ibm, aapl], 5, Resolution.DAILY)
quote_bars_df = self.history(QuoteBar, [ibm, aapl], 5, Resolution.MINUTE)
df = self.history([ibm, aapl], 5, Resolution.DAILY)  # Includes trade data only. No quote for daily equity data

// EXAMPLE 8: Requesting By Bar Count for Multiple Symbols: 5 bars with a specific resolution:
var tradeBarsList = History<TradeBar>(new[] { ibm, aapl }, 5, Resolution.Minute);
var quoteBarsList = History<QuoteBar>(new[] { ibm, aapl }, 5, Resolution.Minute);

# EXAMPLE 9: Requesting By Trailing Period: 3 days of data at the security resolution: 
ticks = self.history[Tick]([eth_symbol], timedelta(days=3))

trade_bars = self.history[TradeBar]([btc_symbol], timedelta(days=3)) 
quote_bars = self.history[QuoteBar]([btc_symbol], timedelta(days=3))
trade_bars_df = self.history(TradeBar, [btc_symbol], timedelta(days=3))
quote_bars_df = self.history(QuoteBar, [btc_symbol], timedelta(days=3))
df = self.history([btc_symbol], timedelta(days=3))  # Includes trade and quote data

// EXAMPLE 9: Requesting By Trailing Period: 3 days of data at the security resolution:
var ticks = History<Tick>(new[] {ethSymbol}, TimeSpan.FromDays(3));

var tradeBars = History<TradeBar>(new[] {btcSymbol}, TimeSpan.FromDays(3));
var quoteBars = History<QuoteBar>(new[] {btcSymbol}, TimeSpan.FromDays(3));
var tradeBars2 = History(new[] {btcSymbol}, TimeSpan.FromDays(3));

# EXAMPLE 10: Requesting By Defined Period: 3 days of data at the security resolution: 
trade_bars = self.history[TradeBar]([btc_symbol], start_time, end_time) 
quote_bars = self.history[QuoteBar]([btc_symbol], start_time, end_time)
ticks = self.history[Tick]([eth_symbol], start_time, end_time)
trade_bars_df = self.history(TradeBar, btc_symbol, start_time, end_time)
quote_bars_df = self.history(QuoteBar, btc_symbol, start_time, end_time)
ticks_df = self.history(Tick, eth_symbol, start_time, end_time)
df = self.history([btc_symbol], start_time, end_time)  # Includes trade and quote data

// EXAMPLE 10: Requesting By Defined Period: 3 days of data at the security resolution:
var tradeBars = History<TradeBar>(new[] {btcSymbol}, startTime, endTime);
var quoteBars = History<QuoteBar>(new[] {btcSymbol}, startTime, endTime);
var ticks = History<Tick>(new[] {ethSymbol}, startTime, endTime);
var tradeBars2 = History(new[] {btcSymbol}, startTime, endTime);

If you request data for multiple securities and you use the TickTICK request type, each Ticks object in the list of results only contains the last tick of each security for that particular timeslice.

All Symbol History Requests

You can request history for all active securities in your universe. The parameters are very similar to other history method calls, but the return type is an array of Slice objects. The Slice object holds all of the results in a sorted enumerable collection that you can iterate over with a loop.

# EXAMPLE 11: Requesting 5 bars for all securities at their respective resolution:

# Create subscriptions
self.add_equity("IBM", Resolution.DAILY)
self.add_equity("AAPL", Resolution.DAILY)

# Request history data and enumerate results
slices = self.history(5)
for s in slices:
    self.Log(str(s.time) + " AAPL:" + str(s.bars["AAPL"].close) + " IBM:" + str(s.bars["IBM"].close))

// EXAMPLE 11: Requesting 5 bars for all securities at their respective resolution:

// Set up the universe
AddEquity("IBM", Resolution.Daily);
AddEquity("AAPL", Resolution.Daily);

// Request history data and enumerate results:
var slices = History(5);
foreach (var s in slices) {
    var aaplClose = s.Bars["AAPL"].Close;
    var ibmClose = s.Bars["IBM"].Close;
    Log($"{s.Time} AAPL: {aaplClose} IBM: {ibmClose}");
}

# EXAMPLE 12: Requesting 5 minutes for all securities:

slices = self.history(timedelta(minutes=5), Resolution.MINUTE)
for s in slices:
    self.Log(str(s.time) + " AAPL:" + str(s.bars["AAPL"].close) + " IBM:" + str(s.bars["IBM"].close))

# timedelta history requests are relative to "now" in algorithm Time. If you request this data at 16:05, it returns an empty array because the market is closed.

// EXAMPLE 12: Requesting 24 hours of hourly data for all securities:

var slices = History(TimeSpan.FromHours(24), Resolution.Hour);
foreach (var s in slices) {
    var aaplClose = s.Bars["AAPL"].Close;
    var ibmClose = s.Bars["IBM"].Close;
    Log($"{s.Time} AAPL: {aaplClose} IBM: {ibmClose}");
}

// TimeSpan history requests are relative to "now" in algorithm Time.

Assumed Default Values

The following table describes the assumptions of the History API:

Argument	Assumption
Resolution	LEAN guesses the resolution you request by looking at the securities you already have in your algorithm. If you have a security subscription in your algorithm with a matching `Symbol`, the history request uses the same resolution as the subscription. If you don't have a security subscription in your algorithm with a matching `Symbol`, `Resolution.MinuteResolution.MINUTE` is the default.
Bar type	If you don't specify a type for the history request, `TradeBar` is the default. If the asset you request data for doesn't have `TradeBar` data, specify the `QuoteBar` type to receive history.

Additional Options

The Historyhistory method accepts the following additional arguments:

Argument	Data Type	Description	Default Value
`fillForwardfill_forward`	`bool?bool/NoneType`	True to fill forward missing data. Otherwise, false. If you don't provide a value, it uses the fill forward mode of the security subscription.	`nullNone`
`extendedMarketHoursextended_market_hours`	`bool?bool/NoneType`	True to include extended market hours data. Otherwise, false.	`nullNone`
`dataMappingModedata_mapping_mode`	`DataMappingMode?DataMappingMode/NoneType`	The contract mapping mode to use for the security history request.	`nullNone`
`dataNormalizationModedata_normalization_mode`	`DataNormalizationMode?DataNormalizationMode/NoneType`	The price scaling mode to use for US Equities or continuous Futures contracts. If you don't provide a value, it uses the data normalization mode of the security subscription.	`nullNone`
`contractDepthOffsetcontract_depth_offset`	`int?int/NoneType`	The desired offset from the current front month for continuous Futures contracts.	`nullNone`

self.future = self.add_future(Futures.Currencies.BTC)
history = self.history(
    tickers=[self.future.symbol], 
    start=self.time - timedelta(days=15), 
    end=self.time, 
    resolution=Resolution.MINUTE, 
    fill_forward=False, 
    extended_market_hours=False, 
    dataMappingMode=DataMappingMode.OPEN_INTEREST, 
    dataNormalizationMode=DataNormalizationMode.RAW, 
    contractDepthOffset=0)

var future = AddFuture(Futures.Currencies.BTC);
var history = History(
    symbols: new[] {future.Symbol}, 
    start: Time - TimeSpan.FromDays(15),
    end: Time,
    resolution: Resolution.Minute,
    fillForward: false,
    extendedMarketHours: false,
    dataMappingMode: DataMappingMode.OpenInterest,
    dataNormalizationMode: DataNormalizationMode.Raw,
    contractDepthOffset: 0);

Analyze Results

For most data types, the Historyhistory method returns a multi-index DataFrame where the first index is the Symbol. The data is then sorted in rows according to the second index, the EndTimeend_time of the data point. By learning a few helpful shortcuts, you can directly access the history values you need for your algorithm.

# Setup the universe:
eurusd = self.add_forex("EURUSD", Resolution.DAILY).symbol
nzdusd = self.add_forex("NZDUSD", Resolution.DAILY).symbol

# STEP 1:  Request a DataFrame:
df = self.history([eurusd, nzdusd], 3)



# STEP 2: Check if the DataFrame is empty and lock onto a Symbol index with the `loc[]` method.
#  You can access a security’s data in a DataFrame by using the Symbol or the string representation of the Symbol
if not df.empty:
    eurusd_quotebars = df.loc[eurusd] # or just "EURUSD" instead of Symbol



# STEP 3: Extract and manipulate a single column with the string column name
spread = eurusd_quotebars["askclose"] - eurusd_quotebars["bidclose"]

# Make sure to use the lowercase string column name.

To perform analysis between assets, you can unstack the DataFrame. The unstack method transforms each column into the Symbol values for one of the price-columns you select.

# UNSTACKING: Transform into columns

# Fetch multi-indexed history:
df = self.history([self.symbol("IBM"), self.symbol("AAPL")], 3)


# Transform using unstack:
df["close"].unstack(level=0)

# Make sure to use the lowercase string column name.

To reorganize the DataFrame so that the first index level is the EndTime and the second index level is the asset's Symbol, swap the index level and then sort the index.

history = self.history([eurusd, nzdusd], 3)
history.index = history.index.swaplevel(0, 1)
history = history.sort_index()

If you request historical data for derivatives, the DataFrame can have more index levels. For example, history requests for Equity Options return DataFrames with the following index levels:

Contract expiry
Contract strike price
Contract type (call or put)
Encoded contract Symbol
The EndTimeend_time of the data sample

In this case, you can remove the first three index levels to index the DataFrame with just the contract Symbol, similiar to how you would with non-derivative asset classes. To remove the first three index levels, call the droplevel method.

history.index = history.index.droplevel([0,1,2])

The History method returns an array of TradeBar, QuoteBar, or Slice objects, depending on how you call the method. To access the members of each of these objects, iterate over the objects in a for loop.

// Set up the universe
var spy = AddEquity("SPY").Symbol;
var aapl = AddEquity("AAPL").Symbol;

// Example 1: Iterate over TradeBar objects
var tradeBars = History(spy, 10);
foreach (var tradeBar in tradeBars)
{
    Debug($"{tradeBar.Time}: {tradeBar.Close}");
}

// Example 2: Iterate over QuoteBar objects
var quoteBars = History(aapl, 10);
foreach (var quoteBar in quoteBars)
{
     Debug($"{s.Time}: {quoteBar.Bid.Close} / {quoteBar.Ask.Close}");
}

// Example 3: Iterate over Slice objects
var slices = History(new[] {spy, aapl}, 10);
foreach (var slice in slices)
{
    Debug($"{slice.Time}: AAPL {slice.QuoteBars[aapl].Bid.Close}, SPY {slice[spy].Close}");
}

Data Formats

We provide a specific data formats for each asset class. To view the available data formats, see the Resolutions documentation of the following asset classes:

DataFrame Formats

To get the historical data in a DataFrame, use Python.

When you request historical data for a universe in pandas format, the default format bundles all of the daily data into a list of the universe objects. The following code block and image show an example of this Series format:

universe = self.add_universe(qb.universe.etf('SPY'))
history = self.history(universe, 2)

To flatten the data into a DataFrame so there is one column for each data point attribute in the universe objects, set the flatten argument to True.

history = self.history(universe, 2, flatten=True)

Common Errors

Errors can occur when you request historical data.

Empty Data Errors

If the history request returns an empty DataFrame and you try to slice it, it throws an exception. To avoid issues, check if the DataFrame contains data before slicing it.

df = self.history(symbol, 10).close    # raises exception if the request is empty

def get_safe_history_closes(self, symbols):
    if not symbols:
        self.log(f'No symbols')
        return  False, None
    df = self.history(symbols, 100, Resolution.DAILY)
    if df.empty:
        self.log(f'Empy history for {symbols}')
        return  False, None
     return True, df.close.unstack(0)

If you run algorithms on your local machine and history requests return no data, check if your data directory contains the data you request. To download datasets, see Download.

Numerical Precision Errors

Some factor files have INF split values, which indicate that the stock has so many splits that prices can't be calculated with correct numerical precision. To allow history requests with these symbols, we need to move the starting date forward when reading the data. If there are numerical precision errors in the factor files for a security in your history request, LEAN throws the following error:

"Warning: when performing history requests, the start date will be adjusted if there are numerical precision errors in the factor files."

Live Trading Considerations

In live trading, if you make a history request for minute data at noon and the history period covers the start of the previous day to the present moment, the data from the previous day will be backtest data. The data of the current day will be live data that we collected throughout the morning. If you make this history request in a backtest, you might get slightly different data for the current day because of post-processing from the data vendor.

Examples

The following examples demonstrate some common practices for trading using historical requests.

Example 1: Mean-Variance Portfolio

The following algorithm constructs a monthly rebalance mean-variance portfolio using the top 20 liquid equities. The position sizing can be optimized by 1-year historical daily return of the universe members.

public class HistoricalRequestAlgorithm : QCAlgorithm
{
    private Universe _universe;
    private IPortfolioOptimizer _optimizer;

    public override void Initialize()
    {
        SetStartDate(2020, 4, 1);
        SetEndDate(2020, 9, 30);
        
        // Monthly renewal of the top 20 liquid universe to trade popular stocks.
        UniverseSettings.Schedule.On(DateRules.MonthStart());
        _universe = AddUniverse(Universe.DollarVolume.Top(20));

        // Instantiate the optimizer to perform mean-variance optimization.
        // Mean-variance optimization will not consider a risk-free rate, so we use 0.
        _optimizer = new MaximumSharpeRatioPortfolioOptimizer(0, 1, 0);

        // Set a scheduled event to rebalance the portfolio at the start of every month.
        Schedule.On(
            DateRules.MonthStart(),
            TimeRules.At(9, 31),
            Rebalance
        );
    }

    private void Rebalance()
    {
        // Historical data request to get 1-year data for optimization.
        var symbols = _universe.Selected.ToList();
        var history = History<TradeBar>(symbols, 253, Resolution.Daily)
            .Where(x => symbols.All(y => x.ContainsKey(y)) && x.All(y => y.Value.Close > 0m))
            .ToList();
        // Daily return on the universe members to calculate the optimized weights.
        var returns = GetReturns(history, symbols);

        // Calculate the optimized weights.
        var weights = _optimizer.Optimize(returns);

        // Rebalance the portfolio according to the optimized weights.
        var targets = Enumerable.Range(0, weights.Length)
            .Select(i => new PortfolioTarget(symbols[i], Convert.ToDecimal(weights[i])))
            .ToList();
        SetHoldings(targets, liquidateExistingHoldings: true);
    }

    private double[,] GetReturns(List<DataDictionary<TradeBar>> history, List<Symbol> symbols)
    {
        // Create a 2d array of historical daily returns from historical price data.
        var returns = new double[history.Count, symbols.Count];
        for (int j = 0; j < symbols.Count; j++)
        {
            var lastPrice = 0m;
            for (int i = 0; i < history.Count; i++)
            {
                var current = history[i][symbols[j]].Close;
                if (i > 0)
                {
                    returns[i, j] = (double)((current - lastPrice) / lastPrice);
                }
                lastPrice = current;
            }
        }
        return returns;
    }
}

from Portfolio.MaximumSharpeRatioPortfolioOptimizer import MaximumSharpeRatioPortfolioOptimizer

class HistoricalRequestAlgorithm(QCAlgorithm):
    def initialize(self) -> None:
        self.set_start_date(2020, 4, 1)
        self.set_end_date(2020, 9, 30)
        
        # Monthly renewal of the top 20 liquid universe to trade popular stocks.
        self.universe_settings.schedule.on(self.date_rules.month_start())
        self._universe = self.add_universe(self.universe.dollar_volume.top(20))

        # Instantiate the optimizer to perform mean-variance optimization.
        # Mean-variance optimization will not consider a risk-free rate, so we use 0.
        self._optimizer = MaximumSharpeRatioPortfolioOptimizer(0.0, 1.0, 0.0)

        # Set a scheduled event to rebalance the portfolio at the start of every month.
        self.schedule.on(
            self.date_rules.month_start(), 
            self.time_rules.at(9, 31),
            self.rebalance
        )

    def rebalance(self) -> None:
        # Historical data request to get 1-year data for optimization.
        symbols = self._universe.selected
        history = self.history(symbols, 253, Resolution.DAILY).close.unstack(0).dropna()
        # Daily return on the universe members to calculate the optimized weights.
        returns = history.pct_change().dropna()

        # Calculate the optimized weights.
        weights = self._optimizer.optimize(returns)

        # Rebalance the portfolio according to the optimized weights.
        targets = [PortfolioTarget(symbol, size) for symbol, size in zip(symbols, weights)]
        self.set_holdings(targets, liquidate_existing_holdings=True)

Other Examples

For more examples, see the following algorithms:

Demonstration Algorithm

HistoryAlgorithm.py Python WarmupHistoryAlgorithm.py Python HistoryAlgorithm.cs C# WarmupHistoryAlgorithm.cs C#

You can also see our Videos. You can also get in touch with us via Discord.

Did you find this page helpful?

Contribute to the documentation:

SIGN IN

Browse

Cloud Platform

Writing Algorithms

▶Key Concepts

Initialization

▶Securities

▶Portfolio

▶Universes

▶Datasets

▶Importing Data

▶Consolidating Data

▶Historical Data

▶Trading and Orders

▶Reality Modeling

Scheduled Events

▶Indicators

Object Store

▶Optimization

▶Machine Learning

▶Algorithm Framework

Charting

Logging

▶Statistics

▶Live Trading

Strategy Library

API Reference

▶Migrations

Research Environment

Local Platform

LEAN CLI

LEAN Engine

Hello

Historical Data

History Requests

Introduction

Key History Concepts

Time Period Options

Return Formats

Time Index

Request Data

Single Symbol History Requests

Multiple Symbol History Requests

All Symbol History Requests

Assumed Default Values

Additional Options

Analyze Results

Data Formats

DataFrame Formats

Common Errors

Empty Data Errors

Numerical Precision Errors

Live Trading Considerations

Examples

Example 1: Mean-Variance Portfolio

Other Examples

▶
Key Concepts

▶
Securities

▶
Portfolio

▶
Universes

▶
Datasets

▶
Importing Data

▶
Consolidating Data

▶
Historical Data

▶
Trading and Orders

▶
Reality Modeling

▶
Indicators

▶
Optimization

▶
Machine Learning

▶
Algorithm Framework

▶
Statistics

▶
Live Trading

▶
Migrations