Consolidating Data

The following code snippet demonstrates a simple consolidator that aggregates 1-minute bars into 10-minutes bars:

// Add some class members
private Symbol _symbol;
private TradeBarConsolidator _consolidator;

// In Initialize, subscribe to a security, create the consolidator, and register it for automatic updates
_symbol = AddEquity("SPY", Resolution.Minute).Symbol;
_consolidator = Consolidate(_symbol, TimeSpan.FromMinutes(10), ConsolidationHandler);

// Define the consolidation handler
void ConsolidationHandler(TradeBar consolidatedBar)
{
    
}

# In Initialize, subscribe to a security, create the consolidator, and register it for automatic updates
self._symbol = self.add_equity("SPY", Resolution.MINUTE).symbol
self._consolidator = self.consolidate(self._symbol, timedelta(minutes=10), self._consolidation_handler)

# Define the consolidation handler
def _consolidation_handler(self, consolidated_bar: TradeBar) -> None:
    pass

Consolidators usually produce output data that is the same format as the input data. They aggregate small TradeBar objects into a large TradeBar, small QuoteBar objects into a large QuoteBar, and Tick objects into either a TradeBar or QuoteBar.

Many asset classes in QuantConnect have data for both trades and quotes. By default, consolidators aggregate data into TradeBar objects. Forex data is the only exception, which consolidators aggregate into QuoteBar objects since Forex data doesn't have TradeBar objects.

There are many different types of consolidators you can create. The process to create and update the consolidator depends on the input data format, output data format, and the consolidation technique.

Time Period Consolidators

Time period consolidators aggregate data based on a period of time like a number of seconds, minutes, or days. If a time period consolidator aggregates data over multiple days, the multi-day aggregation cycle starts and repeats based on the time stamp of the first data point you use to update the consolidator.

The following table shows which consolidator type to use based on the data format of the input and output:

Calendar Consolidators

Calendar consolidators aggregate data based on specific periods of time like a weekly basis, quarterly basis, or any schedule with specific start and end times. In contrast to time period consolidators, the bars that calendar consolidators produce always end on the same schedule, regardless of the first data point the consolidator receives.

The following table shows which consolidator type to use based on the data format of the input and output:

Count Consolidators

Count consolidators aggregate data based on a number of bars or ticks. This type of consolidator aggregates n samples together, regardless of the time period the samples cover. Managing count consolidators is similiar to managing time period consolidators, except you create count consolidators with an integer argument instead of a time-based argument.

The following table shows which consolidator type to use based on the data format of the input and output:

Mixed-Mode Consolidators

Mixed-mode consolidators are a combination of count consolidators and time period consolidators. This type of consolidator aggregates n samples together or aggregates samples over a specific time period, whichever happens first.

The following table shows which consolidator type to use based on the data format of the input and output:

Renko Consolidators

Most Renko consolidators aggregate bars based on a fixed price movement. The RenkoConsolidator produces Renko bars by their traditional definition. In the case of a $1 bar size, the RenkoConsolidator produces bars that have a body spanning $1. The opening price of the first bar is set to the closest $1 multiple of the first trade. When the price moves by at least $1, the first bar closes. If a bar is a rising bar, the following bar closes when the price moves $1 above the closing price of the previous bar or $1 below the opening price of the previous bar. If a bar is a falling bar, the following bar closes when the price moves $1 below the closing price of the previous bar or $1 above the opening price of the previous bar. If the price jumps multiple dollars in a single tick, the RenkoConsolidator produces multiple $1 bars in a single time step.

The following table shows which consolidator type to use based on the data format of the input and output:

Classic Renko Consolidators

Most Renko consolidators aggregate bars based on a fixed price movement. The ClassicRenkoConsolidator produces a different type of Renko bars than the RenkoConsolidator. A ClassicRenkoConsolidator with a bar size of $1 produces a new bar that spans $1 every time an asset closes $1 away from the close of the previous bar. If the price jumps multiple dollars in a single tick, the ClassicRenkoConsolidator only produces one bar per time step where the open of each bar matches the close of the previous bar.

The following table shows which consolidator type to use based on the data format of the input and output:

Volume Renko Consolidators

Volume Renko consolidators aggregate bars based on a fixed trading volume. These types of bars are commonly known as volume bars. A VolumeRenkoConsolidator with a bar size of 10,000 produces a new bar every time 10,000 units of the security trades in the market. If the trading volume of a single time step exceeds two step sizes (i.e. >20,000), the VolumeRenkoConsolidator produces multiple bars in a single time step.

The following table shows which consolidator type to use based on the data format of the input and output:

Range Consolidators

Most Range consolidators aggregate bars based on a fixed price movement. It is very much like a Renko Bar, but instead of consolidating a fix movement from the opening price, a Range Bar would fix the range of the whole bar, i.e. High - Low. A new bar is formed when the range of the whole bar reached a preset value, thus it completely neglects the time effect and focuses on the variance/information carried by the price movement.

The following table shows which consolidator type to use based on the data format of the input and output:

Sequential Consolidators

Sequential consolidators wire two internal consolidators together such that the output of the first consolidator is the input to the second consolidator and the output of the second consolidator is the output of the sequential consolidator.

For more information about sequential consolidators, see Combining Consolidators.

The algorithm manager calls events in the following order:

1. Scheduled Events
2. Consolidation event handlers
3. OnDataon_data event handler

This event flow is important to note. For instance, if your consolidation handlers or OnDataon_data event handler appends data to a RollingWindow and you use that RollingWindow in your Scheduled Event, when the Scheduled Event executes, the RollingWindow won't contain the most recent data.

The consolidators are called in the order that they are updated. If you register them for automatic updates, they are updated in the order that you register them.

To plot consolidated bars, call the Plotplot method with a TradeBar argument in the consolidation handler.

// In Initialize
var equity = AddEquity("SPY");
Consolidate(equity.Symbol, TimeSpan.FromMinutes(10), ConsolidationHandler);

// Define the consolidation handler
void ConsolidationHandler(TradeBar consolidatedBar)
{
    Plot("<chartName>", "<seriesName>", consolidatedBar)
}

# In Initialize
equity = self.add_equity("SPY")
self.consolidate(equity.symbol, timedelta(minutes=10), self._consolidation_handler)

# Define the consolidation handler
def _consolidation_handler(self, consolidated_bar: TradeBar) -> None:
    self.plot("<chartName>", "<seriesName>", consolidated_bar)