Historical Data

Warm Up Periods

Introduction

LEAN supports an automated fast-forward system called "Warm Up". It simulates winding back the date you deployed your algorithm by a specific time period. In backtests, it changes the start date of your algorithm to an earlier time. In live trading, it fetches data from before your deployment and fast forwards through your strategy up to the present day. Warm Up is a great way to prepare your algorithm and its indicators for trading. You can't place trades during the warm-up period because the data feed is replaying historical data for setting algorithm state.

Set Warm Up Periods

You can set a warm-up period based on a number of bars or a period of time.

Trailing Data Samples

To set a warm-up based on a trailing number of data samples, in your algorithm's Initializeinitialize method, call the SetWarmUpset_warm_up method with an integer argument. 

// Feed in 100 bars before start date
SetWarmUp(100);
# Feed in 100 bars before start date
self.set_warm_up(100)

LEAN calculates the start of the warm-up period for each of your security subscriptions by using the number of bars you specify, the resolution of each security, and the trading calendar of each security. After it calculates the start time of the warm-up period for each security, it sets the earliest start time as the start of the algorithm warm-up period. For instance, in the following example, the warm-up period consists of 100 minute resolution bars for SPY and as many second resolution bars for BTCUSD that occur from the start of the SPY warm-up period to the algorithm StartDatestart_date. 

public class WarmUpWithTrailingDataSamplesAlgorithm : QCAlgorithm
{
    public override void Initialize()
    {
        SetStartDate(2020, 1, 1);
        AddEquity("SPY", Resolution.Minute, fillForward: false);
        AddCrypto("BTCUSD", Resolution.Second, fillForward: false);
        SetWarmUp(100);
    }
}
class WarmUpWithTrailingDataSamplesAlgorithm(QCAlgorithm):

    def initialize(self) -> None:
        self.set_start_date(2020, 1, 1)
        self.add_equity("SPY", Resolution.MINUTE, fill_forward=False)
        self.add_crypto("BTCUSD", Resolution.SECOND, fill_forward=False)
        self.set_warm_up(100)

To use a specific resolution of data for the warm-up period, pass a resolution argument to the SetWarmUpset_warm_up method.

// Feed in data for 100 trading days of daily data before the start date
SetWarmUp(100, Resolution.Daily);
# Feed in data for 100 trading days before the start date
self.set_warm_up(100, Resolution.DAILY)

If you pass an integer and a resolution, the warm-up period consists of the number of bars and resolution you set, regardless of the resolution of your data subscriptions. In this case, LEAN calculates the start of the warm-up period for each of your security subscriptions just like it does when you only pass an integer argument. After it calculates the start time of the warm-up period for each security, it sets the earliest start time as the start of the algorithm warm-up period.

If you pass a resolution argument and you registered indicators or consolidators for automatic updates, the warm-up period resolution must be less than or equal to the lowest resolution of your automatic indicators and consolidators. For instance, in the following example, the indicators use minute resolution data, so you can set the warm-up period to use tick, second, or minute resolution data.

public class WarmUpWithTrailingDataSamplesAndResolutionAlgorithm : QCAlgorithm
{
    public override void Initialize()
    {
        SetStartDate(2020, 1, 1);
        var spy = AddEquity("SPY", Resolution.Minute, fillForward: false).Symbol;
        var btc = AddCrypto("BTCUSD", Resolution.Second, fillForward: false).Symbol;
        
        var spySma = SMA(spy, 10);
        var btcSMA = SMA(btc, 10, Resolution.Minute);

        SetWarmUp(100, Resolution.Minute);
    }
}
class WarmUpWithTrailingDataSamplesAndResolutionAlgorithm(QCAlgorithm):

    def initialize(self) -> None:
        self.set_start_date(2020, 1, 1)
        self._spy = self.add_equity("SPY", Resolution.MINUTE, fill_forward=False).symbol
        self._btc = self.add_crypto("BTCUSD", Resolution.SECOND, fill_forward=False).symbol
        
        self._spy_sma = self.sma(self._spy, 10)
        self._btc_sma = self.sma(self._btc, 10, Resolution.MINUTE)
        
        self.set_warm_up(100, Resolution.MINUTE)

Trailing Time Periods

To set a warm-up based on a trailing time period, in your algorithm's Initializeinitialize method, call the SetWarmUpset_warm_up method with a timedeltaTimeSpan. The warm-up period consists of that timedeltaTimeSpan before the start date and pipes in data that matches the resolution of your data subscriptions. 

// Wind time back 7 days from start
SetWarmUp(TimeSpan.FromDays(7));
# Wind time back 7 days from start
self.set_warm_up(timedelta(7))

To use a specific resolution of data for the warm-up period, pass a resolution argument to the SetWarmUpset_warm_up method.

// Feed in 100 days of daily data before the start date
SetWarmUp(TimeSpan.FromDays(100), Resolution.Daily);
# Feed in 100 days of daily data before the start date
self.set_warm_up(timedelta(days=100), Resolution.DAILY)

If you pass a timedeltaTimeSpan and a resolution, the warm-up period consists of the time period and resolution you set, regardless of the resolution of your data subscriptions.

Warm Up Status

You can't place trades during the warm-up period because the data feed is replaying historical data for setting algorithm state. To check if the algorithm is currently in a warm up period, use the IsWarmingUpis_warming_up property. 

public class WarmUpStatusAlgorithm : QCAlgorithm
{
    private ExponentialMovingAverage _ema;
    public override void Initialize()
    {
        SetStartDate(2020, 1, 1);
        var symbol = AddEquity("IBM", Resolution.Daily).Symbol;
        _ema = EMA(symbol, 100);
        SetWarmup(100);
    }

    public override void OnData(Slice data)
    {
        if (IsWarmingUp)
        {
            return;
        }
        Plot("EMA", "Is ready", _ema.IsReady ? 1 : 0);
    }
}
class WarmUpStatusAlgorithm(QCAlgorithm):

    def initialize(self) -> None:
        self.set_start_date(2020, 1, 1)
        symbol = self.add_equity('IBM', Resolution.DAILY).symbol
        self._ema = self.ema(symbol, 100)
        self.set_warmup(100)

    def on_data(self, data) -> None:
        if self.is_warming_up: 
            return
        self.plot('EMA', 'Is ready', int(self._ema.is_ready))

Event Handler

The OnWarmupFinishedon_warmup_finished event handler executes when the warm up period ends, even if you don't set a warm up period.

public override void OnWarmupFinished()
{
    // Done warming up
}

def on_warmup_finished(self) -> None:
    pass # Done warming up

Missing Data Points

There are several cases where you may get fewer data points from during warm-up than you expect.

Illiquid Securities

The default behavior for LEAN is to fill-forward the data. However, if an asset had no trading activity during the time period you request historical data for, you won't get any data for that asset.

Requesting Data Before an Asset's IPO

All assets have a day when they first started trading. If your warm-up period includes time before an asset's initial public offering (IPO) date, you won't get any data before the IPO date because the data doesn't exist.

Requesting Data Outside Market Hours

Warming up with a trailing number of data samples means the data is based on the market hours of assets, so they can wrap across multiple trading days to gather the amount of data you request. In contrast, warming up with a trailing time period provides means the data is based on a rolling calendar time. In this latter types of warm-up, if the period of time is outside of the market hours of an asset, you won't get any data.

Data Issues

Data issues are generally a result of human error or from mistakes in the data collection process. Any QuantConnect member can report a data issue, which our Data Team works to quickly resolve. However, while the data is still missing, your warm-up won't be able to provide the data. To view the list of current data issues, log in to the Algorithm Lab and then, in the left navigation bar, click Datasets > Data Issues.

Data Provider Limitations

When you run warm-up, consider the functionality of the data provider you use. For example, the QuantConnect data provider has quotas on the amount of data you can request and the Interactive Brokers data provider can be slow, so it can timeout if you run warm-up with too many assets. For more information about all the data providers, see their respective documentation in Cloud Platform, Local Platform, or the CLI.

Warm Up vs History Requests

Warm-up and history requests both provide historical data. The approach you take depends on your specific use case. A benefit to using warm-up is that it only takes one line of code to pump historical data into your algorithm, which processes universe selection and updates all the indicators. A consequence to using warm-up is that it may provide more historica data than you actually need since it provides data on all the subscriptions in your algorithm. If you have derivative assets in your algorithm, warm-up can be very slow to process.

Examples

The following examples demonstrate some common practices for warm up.

Example 1: Warm Up Indicator

The following algorithm trades mean-reversion according to signals from a Bollinger Bands indicator. It uses warm up so the indicator is ready by the start date of the algorithm. 

public class WarmUpPeriodsAlgorithm : QCAlgorithm
{
    private Symbol _spy;
    private BollingerBands _bbands = new(20, 2);

    public override void Initialize()
    {
        SetStartDate(2021, 1, 1);
        SetEndDate(2022, 1, 1);

        // Request SPY data for signal generation and trading.
        _spy = AddEquity("SPY", Resolution.Minute).Symbol;

        // The candlestick patterns are based on a daily consolidated trade bar.
        var consolidator = new TradeBarConsolidator(TimeSpan.FromDays(1));
        // Subscribe to update the indicators with the 1-day consolidator automatically.
        RegisterIndicator(_spy, _bbands, consolidator);
        // Add an event handler on candlestick indicators that are updated to trade the indicator.
        _bbands.Updated += OnUpdated;

        // Warm up the indicator and the SPY price data.
        SetWarmUp(20, Resolution.Daily);
    }

    private void OnUpdated(object sender, IndicatorDataPoint point)
    {
        var indicator = sender as BollingerBands;
        if (!indicator.IsReady) return;

        var holdings = Portfolio[_spy];

        // Trade mean-reversal of the Bollinger Band.
        if (holdings.Price > indicator.UpperBand && !holdings.IsShort)
        {
            SetHoldings(_spy, -0.5m);
        }
        else if (holdings.Price < indicator.LowerBand && !holdings.IsLong)
        {
            SetHoldings(_spy, 0.5m);
        }
    }
}
class WarmUpPeriodsAlgorithm(QCAlgorithm):
    bbands = BollingerBands(20, 2)

    def initialize(self) -> None:
        self.set_start_date(2021, 1, 1)
        self.set_end_date(2022, 1, 1)
        
        # Request SPY data for signal generation and trading.
        self.spy = self.add_equity("SPY", Resolution.MINUTE).symbol

        # The candlestick patterns are based on a daily consolidated trade bar.
        consolidator = TradeBarConsolidator(timedelta(1))
        # Subscribe to update the indicators with the 1-day consolidator automatically.
        self.register_indicator(self.spy, self.bbands, consolidator)
        # Add an event handler on candlestick indicators that are updated to trade the indicator.
        self.bbands.updated += self.on_updated

        # Warm up the indicator and the SPY price data.
        self.set_warm_up(20, Resolution.DAILY)

    def on_updated(self, sender: object, point: IndicatorDataPoint) -> None:
        if not sender.is_ready:
            return
        
        # Trade according to the updated indicator values.
        upper_band = sender.upper_band.current.value
        lower_band = sender.lower_band.current.value
        holdings = self.portfolio[self.spy]

        # Trade mean-reversal of the Bollinger Band.
        if holdings.price > upper_band and not holdings.is_short:
            self.set_holdings(self.spy, -0.5)
        if holdings.price < lower_band and not holdings.is_long:
            self.set_holdings(self.spy, 0.5)

Example 2: Warm Up Universe Selection

The following algorithm forms an equal-weighted portfolio with the 100 most liquid US Equities that are above their 60-day Exponential Moving Average (EMA). It uses the warm-up feature to prime the indicators in advance so it can start trading by the start date of the algorithm.

public class WarmUpPeriodsAlgorithm : QCAlgorithm
{
    private Universe _universe;
    private Dictionary<Symbol, ExponentialMovingAverage> _emaBySymbol = new();

    public override void Initialize()
    {
        SetStartDate(2021, 1, 1);
        SetEndDate(2021, 11, 1);
        SetSecurityInitializer(new BrokerageModelSecurityInitializer(BrokerageModel, new FuncSecuritySeeder(GetLastKnownPrices)));

        // Universe filtered by only the ones with prices above EMA, suggesting an uptrend.
        _universe = AddUniverse(SelectionByEma);

        // Rebalance weekly.
        Schedule.On(
            DateRules.WeekStart(),
            TimeRules.At(9, 31),
            Rebalance
        );

        SetWarmUp(60, Resolution.Daily);
    }

    private IEnumerable<Symbol> SelectionByEma(IEnumerable<Fundamental> fundamentals)
    {
        foreach (var f in fundamentals)
        {
            // Create an EMA indicator for the symbol if it is unavailable.
            if (!_emaBySymbol.TryGetValue(f.Symbol, out var ema))
            {
                _emaBySymbol[f.Symbol] = ema = new ExponentialMovingAverage(60);
            }
            // Update the indicator by the updated price data to select on the updated data.
            ema.Update(f.EndTime, f.AdjustedPrice);
        }

        // We don't add new securities during the warm-up process
        if (IsWarmingUp) return Universe.Unchanged;

        return fundamentals
            // Only trades the top 100 liquid stocks since their trend capitalizes quicker.
            .OrderByDescending(f => f.DollarVolume)
            // Select the ones with the current price above EMA to filter for the uptrend stocks.
            .Where(f => _emaBySymbol[f.Symbol].IsReady && _emaBySymbol[f.Symbol] > f.AdjustedPrice)
            .Take(100)
            .Select(f => f.Symbol);
    }

    private void Rebalance()
    {
        // Equally invest in uptrend stocks to dissipate capital risk evenly.
        var weight = 1m / _universe.Selected?.Count ?? 0m;
        if (weight == 0) return;
        var targets = _universe.Selected.Select(symbol => new PortfolioTarget(symbol, weight)).ToList();
        SetHoldings(targets);
    }

    public override void OnSecuritiesChanged(SecurityChanges changes)
    {
        // Exit position if the equity is below EMA since it becomes a downgoing trend.
        changes.RemovedSecurities.DoForEach(r => Liquidate(r.Symbol));
    }
}
class WarmUpPeriodsAlgorithm(QCAlgorithm):
    ema_by_symbol = {}

    def initialize(self) -> None:
        self.set_start_date(2021, 1, 1)
        self.set_end_date(2021, 11, 1)
        self.set_security_initializer(BrokerageModelSecurityInitializer(self.brokerage_model, FuncSecuritySeeder(self.get_last_known_prices)))

        # The universe is filtered by only the ones with prices above EMA, suggesting an uptrend.
        self._universe = self.add_universe(self.selection_by_ema)

        # Rebalance weekly.
        self.schedule.on(
            self.date_rules.week_start(),
            self.time_rules.at(9, 31),
            self.rebalance
        )

        self.set_warm_up(60, Resolution.DAILY)

    def selection_by_ema(self, fundamentals: List[Fundamental]) -> List[Symbol]:
        for f in fundamentals:
            # Create an EMA indicator for the symbol if it is not available.
            self.ema_by_symbol[f.symbol] = self.ema_by_symbol.get(f.symbol, ExponentialMovingAverage(60))
            # Update the indicator using the updated price data to select the updated data.
            self.ema_by_symbol[f.symbol].update(f.end_time, f.adjusted_price)

        # We don't add new securities during the warm-up process
        if self.is_warming_up:
            return Universe.UNCHANGED

        # Select the ones with the current price above EMA to filter for the uptrend stocks.
        def is_uptrend(f):
            ema = self.ema_by_symbol[f.symbol]
            return ema.is_ready and ema.current.value > f.adjusted_price
        
        # Only trades the top 100 liquid stocks since their trend capitalizes quicker.
        fundamentals = sorted([f for f in fundamentals if is_uptrend(f)],
            key=lambda x: x.dollar_volume, reverse=True)
                
        return [f.symbol for f in fundamentals[:100]]

    def rebalance(self) -> None:
        #  Equally invest in uptrend stocks to dissipate capital risk evenly.
        weight = 1 / len(self._universe.selected) if self._universe.selected else 0
        if weight == 0:
            return
        targets = [PortfolioTarget(symbol, weight) for symbol in self._universe.selected]
        self.set_holdings(targets)

    def on_securities_changed(self, changes: SecurityChanges) -> None:
        # Exit position if the equity is below EMA since it becomes a downgoing trend.
        [self.liquidate(r.symbol) for r in changes.removed_securities]

Other Examples

For more examples, see the following algorithms:

Demonstration Algorithms
WarmupAlgorithm.py Python WarmupAlgorithm.cs C#

You can also see our Videos. You can also get in touch with us via Discord.

Did you find this page helpful?

Contribute to the documentation: