Consolidating Data
Getting Started
Introduction
Consolidating data allows you to create bars of any length from smaller bars. Consolidation is commonly used to combine one-minute price bars into longer bars such as 10-20 minute bars. Consolidated bars are helpful because price movement over a longer period can sometimes contain less noise and bars of exotic length are less researched than standard bars, so they may present more opportunities to capture alpha.
To consolidate data, create a Consolidator
object and register it for data. The built-in consolidators make it easy to create consolidated bars without introducing bugs. In the following sections, we will introduce the different types of consolidators and show you how to shape data into any form.
Basic Usage
The following code snippet demonstrates a simple consolidator that aggregates 1-minute bars into 10-minutes bars:
// Add some class members private Symbol _symbol; private TradeBarConsolidator _consolidator; // In Initialize, subscribe to a security, create the consolidator, and register it for automatic updates _symbol = AddEquity("SPY", Resolution.Minute).Symbol; _consolidator = Consolidate(_symbol, TimeSpan.FromMinutes(10), ConsolidationHandler); // Define the consolidation handler void ConsolidationHandler(TradeBar consolidatedBar) { }
# In Initialize, subscribe to a security, create the consolidator, and register it for automatic updates self._symbol = self.add_equity("SPY", Resolution.MINUTE).symbol self._consolidator = self.consolidate(self._symbol, timedelta(minutes=10), self._consolidation_handler) # Define the consolidation handler def _consolidation_handler(self, consolidated_bar: TradeBar) -> None: pass
Data Shapes and Sizes
Consolidators usually produce output data that is the same format as the input data. They aggregate small TradeBar
objects into a large TradeBar
, small QuoteBar
objects into a large QuoteBar
, and Tick
objects into either a TradeBar
or QuoteBar
.
![Constructing a consolidated bar](https://cdn.quantconnect.com/docs/i/consolidators-small-to-large.png)
Many asset classes in QuantConnect have data for both trades and quotes. By default, consolidators aggregate data into TradeBar
objects. Forex data is the only exception, which consolidators aggregate into QuoteBar
objects since Forex data doesn't have TradeBar
objects.
Consolidator Types
There are many different types of consolidators you can create. The process to create and update the consolidator depends on the input data format, output data format, and the consolidation technique.
Time Period Consolidators
Time period consolidators aggregate data based on a period of time like a number of seconds, minutes, or days. If a time period consolidator aggregates data over multiple days, the multi-day aggregation cycle starts and repeats based on the time stamp of the first data point you use to update the consolidator.
The following table shows which consolidator type to use based on the data format of the input and output:
Input | Output | Class Type |
---|---|---|
TradeBar | TradeBar | TradeBarConsolidator |
QuoteBar | QuoteBar | QuoteBarConsolidator |
Tick | TradeBar | TickConsolidator |
Tick | QuoteBar | TickQuoteBarConsolidator |
Calendar Consolidators
Calendar consolidators aggregate data based on specific periods of time like a weekly basis, quarterly basis, or any schedule with specific start and end times. In contrast to time period consolidators, the bars that calendar consolidators produce always end on the same schedule, regardless of the first data point the consolidator receives.
The following table shows which consolidator type to use based on the data format of the input and output:
Input | Output | Class Type |
---|---|---|
TradeBar | TradeBar | TradeBarConsolidator |
QuoteBar | QuoteBar | QuoteBarConsolidator |
Tick | TradeBar | TickConsolidator |
Tick | QuoteBar | TickQuoteBarConsolidator |
Count Consolidators
Count consolidators aggregate data based on a number of bars or ticks. This type of consolidator aggregates n samples together, regardless of the time period the samples cover. Managing count consolidators is similiar to managing time period consolidators, except you create count consolidators with an integer argument instead of a time-based argument.
The following table shows which consolidator type to use based on the data format of the input and output:
Input | Output | Class Type |
---|---|---|
TradeBar | TradeBar | TradeBarConsolidator |
QuoteBar | QuoteBar | QuoteBarConsolidator |
Tick | TradeBar | TickConsolidator |
Tick | QuoteBar | TickQuoteBarConsolidator |
Mixed-Mode Consolidators
Mixed-mode consolidators are a combination of count consolidators and time period consolidators. This type of consolidator aggregates n samples together or aggregates samples over a specific time period, whichever happens first.
The following table shows which consolidator type to use based on the data format of the input and output:
Input | Output | Class Type |
---|---|---|
TradeBar | TradeBar | TradeBarConsolidator |
QuoteBar | QuoteBar | QuoteBarConsolidator |
Tick | TradeBar | TickConsolidator |
Tick | QuoteBar | TickQuoteBarConsolidator |
Renko Consolidators
Most Renko consolidators aggregate bars based on a fixed price movement. The RenkoConsolidator
produces Renko bars by their traditional definition.
In the case of a $1 bar size, the RenkoConsolidator
produces bars that have a body spanning $1.
The opening price of the first bar is set to the closest $1 multiple of the first trade.
When the price moves by at least $1, the first bar closes.
If a bar is a rising bar, the following bar closes when the price moves $1 above the closing price of the previous bar or $1 below the opening price of the previous bar.
If a bar is a falling bar, the following bar closes when the price moves $1 below the closing price of the previous bar or $1 above the opening price of the previous bar.
If the price jumps multiple dollars in a single tick, the RenkoConsolidator
produces multiple $1 bars in a single time step.
The following table shows which consolidator type to use based on the data format of the input and output:
Input | Output | Class Type |
---|---|---|
TradeBar | TradeBar | RenkoConsolidator |
QuoteBar | QuoteBar | RenkoConsolidator |
Tick | TradeBar | RenkoConsolidator |
Tick | QuoteBar | RenkoConsolidator |
Classic Renko Consolidators
Most Renko consolidators aggregate bars based on a fixed price movement. The ClassicRenkoConsolidator
produces a different type of Renko bars than the RenkoConsolidator
. A ClassicRenkoConsolidator
with a bar size of $1 produces a new bar that spans $1 every time an asset closes $1 away from the close of the previous bar. If the price jumps multiple dollars in a single tick, the ClassicRenkoConsolidator
only produces one bar per time step where the open of each bar matches the close of the previous bar.
The following table shows which consolidator type to use based on the data format of the input and output:
Input | Output | Class Type |
---|---|---|
TradeBar | TradeBar | ClassicRenkoConsolidator |
QuoteBar | QuoteBar | ClassicRenkoConsolidator |
Tick | TradeBar | ClassicRenkoConsolidator |
Tick | QuoteBar | ClassicRenkoConsolidator |
Volume Renko Consolidators
Volume Renko consolidators aggregate bars based on a fixed trading volume. These types of bars are commonly known as volume bars. A VolumeRenkoConsolidator
with a bar size of 10,000 produces a new bar every time 10,000 units of the security trades in the market. If the trading volume of a single time step exceeds two step sizes (i.e. >20,000), the VolumeRenkoConsolidator
produces multiple bars in a single time step.
The following table shows which consolidator type to use based on the data format of the input and output:
Input | Output | Class Type |
---|---|---|
TradeBar | TradeBar | VolumeRenkoConsolidator |
Tick | TradeBar | VolumeRenkoConsolidator |
Range Consolidators
Most Range consolidators aggregate bars based on a fixed price movement. It is very much like a Renko Bar, but instead of consolidating a fix movement from the opening price, a Range Bar would fix the range of the whole bar, i.e. High - Low. A new bar is formed when the range of the whole bar reached a preset value, thus it completely neglects the time effect and focuses on the variance/information carried by the price movement.
The following table shows which consolidator type to use based on the data format of the input and output:
Input | Output | Class Type |
---|---|---|
TradeBar | TradeBar | RangeConsolidator |
QuoteBar | QuoteBar | RangeConsolidator |
Tick | TradeBar | RangeConsolidator |
Tick | QuoteBar | RangeConsolidator |
Sequential Consolidators
Sequential consolidators wire two internal consolidators together such that the output of the first consolidator is the input to the second consolidator and the output of the second consolidator is the output of the sequential consolidator.
For more information about sequential consolidators, see Combining Consolidators.
Execution Sequence
The algorithm manager calls events in the following order:
- Scheduled Events
- Consolidation event handlers
OnData
on_data
event handler
This event flow is important to note. For instance, if your consolidation handlers or OnData
on_data
event handler appends data to a RollingWindow
and you use that RollingWindow
in your Scheduled Event, when the Scheduled Event executes, the RollingWindow
won't contain the most recent data.
The consolidators are called in the order that they are updated. If you register them for automatic updates, they are updated in the order that you register them.
Plot Consolidated Data
To plot consolidated bars, call the Plot
plot
method with a TradeBar
argument in the consolidation handler.
// In Initialize var equity = AddEquity("SPY"); Consolidate(equity.Symbol, TimeSpan.FromMinutes(10), ConsolidationHandler); // Define the consolidation handler void ConsolidationHandler(TradeBar consolidatedBar) { Plot("<chartName>", "<seriesName>", consolidatedBar) }
# In Initialize equity = self.add_equity("SPY") self.consolidate(equity.symbol, timedelta(minutes=10), self._consolidation_handler) # Define the consolidation handler def _consolidation_handler(self, consolidated_bar: TradeBar) -> None: self.plot("<chartName>", "<seriesName>", consolidated_bar)