book
Checkout our new book! Hands on AI Trading with Python, QuantConnect, and AWS Learn More arrow

Datasets

Futures

Introduction

This page explains how to request, manipulate, and visualize historical Futures data.

Create Subscriptions

Follow these steps to subscribe to a Future security:

  1. Create a QuantBook.
    2. Select Language: 
    qb = QuantBook()
  2. Call the add_future method with a ticker, resolution, and contract rollover settings.
    3. Select Language: 
    future = qb.add_future(Futures.Indices.SP_500_E_MINI, Resolution.MINUTE,
                data_normalization_mode = DataNormalizationMode.BACKWARDS_RATIO,
                data_mapping_mode = DataMappingMode.LAST_TRADING_DAY,
                contract_depth_offset = 0)

    To view the available tickers in the US Futures dataset, see Supported Assets.

    If you omit any of the arguments after the ticker, see the following table for their default values:

    ArgumentDefault Value
    resolutionResolution.MINUTE
    data_normalization_modeDataNormalizationMode.ADJUSTED
    data_mapping_modeDataMappingMode.OpenInterest
    contract_depth_offset0
  3. (Optional) Set a contract filter.
    4. Select Language: 
    future.set_filter(0, 90)

    If you don't call the set_filter method, the future_history method won't return historical data.

If you want historical data on individual contracts and their OpenInterest, follow these steps to subscribe to individual Future contracts:

  1. Call the GetFuturesContractList method with the underlying Future Symbol and a datetime.
    2. Select Language: 
    start_date = datetime(2021,12,20)
symbols = qb.future_chain_provider.get_future_contract_list(future.symbol, start_date)

    This method returns a list of Symbol objects that reference the Future contracts that were trading at the given time. If you set a contract filter with set_filter, it doesn't affect the results of get_future_contract_list.

  2. Select the Symbol of the FutureContract object(s) for which you want to get historical data.

    3. For example, select the Symbol of the contract with the closest expiry.

    Select Language: 
    contract_symbol = sorted(symbols, key=lambda s: s.id.date)[0]
  3. Call the add_future_contract method with an FutureContract Symbol and disable fill-forward.
    4. Select Language: 
    qb.add_future_contract(contract_symbol, fill_forward = False)

    Disable fill-forward because there are only a few OpenInterest data points per day.

Get Historical Data

You need a subscription before you can request historical data for Futures contracts. On the time dimension, you can request an amount of historical data based on a trailing number of bars, a trailing period of time, or a defined period of time. On the contract dimension, you can request historical data for a single contract, a subset of the contracts you created subscriptions for in your notebook, or all of the contracts in your notebook.

These history requests return the prices and open interest of the Option contracts. They don't provide the implied volatility or Greeks. To get the implied volaility and Greeks, call the option_chain method or create some indicators.

Before you request historical data, call the set_start_date method with a datetime to reduce the risk of look-ahead bias.

Select Language: 
qb.set_start_date(start_date)

If you call the set_start_date method, the date that you pass to the method is the latest date for which your history requests will return data.

Trailing Number of Bars

To get historical data for a number of trailing bars, call the history method with the contract Symbol object(s) and an integer.

Select Language: 
# DataFrame of trade and quote data
single_history_df = qb.history(contract_symbol, 10)
subset_history_df = qb.history([contract_symbol], 10)
all_history_df = qb.history(qb.securities.keys(), 10)

# DataFrame of trade data
single_history_trade_bar_df = qb.history(TradeBar, contract_symbol, 10)
subset_history_trade_bar_df = qb.history(TradeBar, [contract_symbol], 10)s
all_history_trade_bar_df = qb.history(TradeBar, qb.securities.keys(), 10)

# DataFrame of quote data
single_history_quote_bar_df = qb.history(QuoteBar, contract_symbol, 10)
subset_history_quote_bar_df = qb.history(QuoteBar, [contract_symbol], 10)
all_history_quote_bar_df = qb.history(QuoteBar, qb.securities.keys(), 10)

# DataFrame of open interest data
single_history_open_interest_df = qb.history(OpenInterest, contract_symbol, 400)
subset_history_open_interest_df = qb.history(OpenInterest, [contract_symbol], 400)
all_history_open_interest_df = qb.history(OpenInterest, qb.securities.keys(), 400)

# Slice objects
all_history_slice = qb.history(10)

# TradeBar objects
single_history_trade_bars = qb.history[TradeBar](contract_symbol, 10)
subset_history_trade_bars = qb.history[TradeBar]([contract_symbol], 10)
all_history_trade_bars = qb.history[TradeBar](qb.securities.keys(), 10)

# QuoteBar objects
single_history_quote_bars = qb.history[QuoteBar](contract_symbol, 10)
subset_history_quote_bars = qb.history[QuoteBar]([contract_symbol], 10)
all_history_quote_bars = qb.history[QuoteBar](qb.securities.keys(), 10)

# OpenInterest objects
single_history_open_interest = qb.history[OpenInterest](contract_symbol, 400)
subset_history_open_interest = qb.history[OpenInterest]([contract_symbol], 400)
all_history_open_interest = qb.history[OpenInterest](qb.securities.keys(), 400)

The preceding calls return the most recent bars, excluding periods of time when the exchange was closed.

To get historical data for the continous Futures contract, in the preceding history requests, replace contract_symbol with future.Symbol.

Trailing Period of Time

To get historical data for a trailing period of time, call the history method with the contract Symbol object(s) and a timedelta.

Select Language: 
# DataFrame of trade and quote data
single_history_df = qb.history(contract_symbol, timedelta(days=3))
subset_history_df = qb.history([contract_symbol], timedelta(days=3))
all_history_df = qb.history(qb.securities.keys(), timedelta(days=3))

# DataFrame of trade data
single_history_trade_bar_df = qb.history(TradeBar, contract_symbol, timedelta(days=3))
subset_history_trade_bar_df = qb.history(TradeBar, [contract_symbol], timedelta(days=3))
all_history_trade_bar_df = qb.history(TradeBar, qb.securities.keys(), timedelta(days=3))

# DataFrame of quote data
single_history_quote_bar_df = qb.history(QuoteBar, contract_symbol, timedelta(days=3))
subset_history_quote_bar_df = qb.history(QuoteBar, [contract_symbol], timedelta(days=3))
all_history_quote_bar_df = qb.history(QuoteBar, qb.securities.keys(), timedelta(days=3))

# DataFrame of open interest data
single_history_open_interest_df = qb.history(OpenInterest, contract_symbol, timedelta(days=3))
subset_history_open_interest_df = qb.history(OpenInterest, [contract_symbol], timedelta(days=3))
all_history_open_interest_df = qb.history(OpenInterest, qb.securities.keys(), timedelta(days=3))

# Slice objects
all_history_slice = qb.history(timedelta(days=3))

# TradeBar objects
single_history_trade_bars = qb.history[TradeBar](contract_symbol, timedelta(days=3))
subset_history_trade_bars = qb.history[TradeBar]([contract_symbol], timedelta(days=3))
all_history_trade_bars = qb.history[TradeBar](qb.securities.keys(), timedelta(days=3))

# QuoteBar objects
single_history_quote_bars = qb.history[QuoteBar](contract_symbol, timedelta(days=3), Resolution.MINUTE)
subset_history_quote_bars = qb.history[QuoteBar]([contract_symbol], timedelta(days=3), Resolution.MINUTE)
all_history_quote_bars = qb.history[QuoteBar](qb.securities.keys(), timedelta(days=3), Resolution.MINUTE) 

# Tick objects
single_history_ticks = qb.history[Tick](contract_symbol, timedelta(days=3), Resolution.TICK)
subset_history_ticks = qb.history[Tick]([contract_symbol], timedelta(days=3), Resolution.TICK)
all_history_ticks = qb.history[Tick](qb.securities.keys(), timedelta(days=3), Resolution.TICK)

# OpenInterest objects
single_history_open_interest = qb.history[OpenInterest](contract_symbol, timedelta(days=2))
subset_history_open_interest = qb.history[OpenInterest]([contract_symbol], timedelta(days=2))
all_history_open_interest = qb.history[OpenInterest](qb.securities.keys(), timedelta(days=2))

The preceding calls return the most recent bars, excluding periods of time when the exchange was closed.

To get historical data for the continous Futures contract, in the preceding history requests, replace contract_symbol with future.Symbol.

Defined Period of Time

To get historical data for individual Futures contracts during a specific period of time, call the history method with the Futures contract Symbol object(s), a start datetime, and an end datetime. The start and end times you provide are based in the notebook time zone.

Select Language: 
start_time = datetime(2021, 12, 1)
end_time = datetime(2021, 12, 31)

# DataFrame of trade and quote data
single_history_df = qb.history(contract_symbol, start_time, end_time)
subset_history_df = qb.history([contract_symbol], start_time, end_time)
all_history_df = qb.history(qb.securities.keys(), start_time, end_time)

# DataFrame of trade data
single_history_trade_bar_df = qb.history(TradeBar, contract_symbol, start_time, end_time)
subset_history_trade_bar_df = qb.history(TradeBar, [contract_symbol], start_time, end_time)
all_history_trade_bar_df = qb.history(TradeBar, qb.securities.keys(), start_time, end_time)

# DataFrame of quote data
single_history_quote_bar_df = qb.history(QuoteBar, contract_symbol, start_time, end_time)
subset_history_quote_bar_df = qb.history(QuoteBar, [contract_symbol], start_time, end_time)
all_history_quote_bar_df = qb.history(QuoteBar, qb.securities.keys(), start_time, end_time)

# DataFrame of open interest data
single_history_open_interest_df = qb.history(OpenInterest, contract_symbol, start_time, end_time)
subset_history_open_interest_df = qb.history(OpenInterest, [contract_symbol], start_time, end_time)
all_history_trade_open_interest_df = qb.history(OpenInterest, qb.securities.keys(), start_time, end_time)

# TradeBar objects
single_history_trade_bars = qb.history[TradeBar](contract_symbol, start_time, end_time)
subset_history_trade_bars = qb.history[TradeBar]([contract_symbol], start_time, end_time)
all_history_trade_bars = qb.history[TradeBar](qb.securities.keys(), start_time, end_time)

# QuoteBar objects
single_history_quote_bars = qb.history[QuoteBar](contract_symbol, start_time, end_time, Resolution.MINUTE)
subset_history_quote_bars = qb.history[QuoteBar]([contract_symbol], start_time, end_time, Resolution.MINUTE)
all_history_quote_bars = qb.history[QuoteBar](qb.securities.keys(), start_time, end_time, Resolution.MINUTE)

# Tick objects
single_history_ticks = qb.history[Tick](contract_symbol, start_time, end_time, Resolution.TICK)
subset_history_ticks = qb.history[Tick]([contract_symbol], start_time, end_time, Resolution.TICK)
all_history_ticks = qb.history[Tick](qb.securities.keys(), start_time, end_time, Resolution.TICK)

# OpenInterest objects
single_history_open_interest = qb.history[OpenInterest](contract_symbol, start_time, end_time)
subset_history_open_interest = qb.history[OpenInterest]([contract_symbol], start_time, end_time)
all_history_open_interest = qb.history[OpenInterest](qb.securities.keys(), start_time, end_time)

To get historical data for the continous Futures contract, in the preceding history requests, replace contract_symbol with future.Symbol.

To get historical data for all of the Futures contracts that pass your filter during a specific period of time, call the future_history method with the Symbol object of the continuous Future, a start datetime, and an end datetime.

Select Language: 
future_history = qb.future_history(future.Symbol, end_time-timedelta(days=2), end_time, Resolution.MINUTE, fill_forward=False, extended_market_hours=False)

The preceding calls return data that have a timestamp within the defined period of time.

Resolutions

The following table shows the available resolutions and data formats for Futures subscriptions:

ResolutionTradeBarQuoteBarTrade TickQuote Tick
TICKgreen checkgreen check
SECONDgreen checkgreen check
MINUTEgreen checkgreen check
HOURgreen checkgreen check
DAILYgreen checkgreen check

Markets

The following Market enumeration members are available for Futures:

Market Select Language:
    NYMEX
  • NYMEX: string
    • CBOT
  • CBOT: string
    • ICE
  • ICE: string
    • CFE
  • CFE: string
    • NSE - National Stock Exchange
  • INDIA: string
    • Comex
  • COMEX: string
    • CME
  • CME: string
    • EUREX
  • EUREX: string
    • Singapore Exchange
  • SGX: string
    • Hong Kong Exchange
  • HKFE: string
    • London International Financial Futures and Options Exchange
  • NYSELIFFE: string

Data Normalization

The data normalization mode doesn't affect data from history request for Futures contracts. If you change the data normalization mode, it won't change the outcome.

The following data normalization modes are available for continuous Futures contracts:

DataNormalizationMode Select Language:
    Eliminates price jumps between two consecutive contracts, adding a factor based on the difference of their prices. The first contract has the true price. Factor 0. (4)
  • FORWARD_PANAMA_CANAL: DataNormalizationMode
    • Eliminates price jumps between two consecutive contracts, adding a factor based on the difference of their prices. The last contract has the true price. Factor 0. (5)
  • BACKWARDS_PANAMA_CANAL: DataNormalizationMode
    • Eliminates price jumps between two consecutive contracts, multiplying the prices by their ratio. The last contract has the true price. Factor 1. (6)
  • BACKWARDS_RATIO: DataNormalizationMode

Wrangle Data

You need some historical data to perform wrangling operations. The process to manipulate the historical data depends on its data type. To display pandas objects, run a cell in a notebook with the pandas object as the last line. To display other data formats, call the print method.

DataFrame Objects

If your history request returns a DataFrame, the DataFrame has the following index levels:

  1. Contract expiry
  2. Encoded contract Symbol
  3. The end_time of the data sample

The columns of the DataFrame are the data properties. Depending on how you request data, the DataFrame may contain data for the continuous Futures contract. The continuous contract doesn't expire, so the default expiry date of December 30, 1899 doesn't have any practical meaning.

Historical data dataframe

To select the rows of the contract(s) that expire at a specific time, index the loc property of the DataFrame with the expiry time.

all_history_df.loc[datetime(2022, 3, 18, 13, 30)]
Historical data dataframe of selected date

If you remove the first index level, you can index the DataFrame with just the contract Symbol, similiar to how you would with non-derivative asset classes. To remove the first index level, call the droplevel method.

all_history_df.index = all_history_df.index.droplevel(0)
Historical data dataframe of dropped first column

To select the historical data of a single Futures contract, index the loc property of the DataFrame with the contract Symbol.

all_history_df.loc[contract_symbol]
DataFrame of one Futures

To select a column of the DataFrame, index it with the column name.

all_history_df.loc[contract_symbol]['close']
Series of close values

If you request historical data for multiple Futures contracts, you can transform the DataFrame so that it's a time series of close values for all of the Futures contracts. To transform the DataFrame, select the column you want to display for each Futures contract and then call the unstack method.

all_history_df['close'].unstack(level=0)

The DataFrame is transformed so that the column indices are the Symbol of each security and each row contains the close value.

Historical data dataframe with column indexed by symbols 
df[" close"]

Slice Objects

If the history method returns Slice objects, iterate through the Slice objects to get each one. The Slice objects may not have data for all of your Futures subscriptions. To avoid issues, check if the Slice contains data for your Futures contract before you index it with the Futures Symbol.

You can also iterate through each TradeBar and QuoteBar in the Slice.

Select Language: 
for slice in all_history_slice:
    for kvp in slice.bars:
        symbol = kvp.key
        trade_bar = kvp.value
    for kvp in slice.quote_bars:
        symbol = kvp.key
        quote_bar = kvp.value

TradeBar Objects

If the history method returns TradeBar objects, iterate through the TradeBar objects to get each one.

Select Language: 
for trade_bar in single_history_trade_bars:
    print(trade_bar)

If the history method returns TradeBars, iterate through the TradeBars to get the TradeBar of each Futures contract. The TradeBars may not have data for all of your Futures subscriptions. To avoid issues, check if the TradeBars object contains data for your security before you index it with the Futures Symbol.

Select Language: 
for trade_bars in all_history_trade_bars:
    if trade_bars.contains_key(contract_symbol):
        trade_bar = trade_bars[contract_symbol]

You can also iterate through each of the TradeBars.

Select Language: 
for trade_bars in all_history_trade_bars:
    for kvp in trade_bars:
        symbol = kvp.Key
        trade_bar = kvp.Value

QuoteBar Objects

If the history method returns QuoteBar objects, iterate through the QuoteBar objects to get each one.

Select Language: 
for quote_bar in single_history_quote_bars:
    print(quote_bar)

If the history method returns QuoteBars, iterate through the QuoteBars to get the QuoteBar of each Futures contract. The QuoteBars may not have data for all of your Futures subscriptions. To avoid issues, check if the QuoteBars object contains data for your security before you index it with the Futures Symbol.

Select Language: 
for quote_bars in all_history_quote_bars:
    if quote_bars.contains_key(contract_symbol):
        quote_bar = quote_bars[contract_symbol]

You can also iterate through each of the QuoteBars.

Select Language: 
for quote_bars in all_history_quote_bars:
    for kvp in quote_bars:
        symbol = kvp.key
        quote_bar = kvp.value

Tick Objects

If the history method returns TICK objects, iterate through the TICK objects to get each one.

Select Language: 
for tick in single_history_ticks:
    print(tick)

If the history method returns Ticks, iterate through the Ticks to get the TICK of each Futures contract. The Ticks may not have data for all of your Futures subscriptions. To avoid issues, check if the Ticks object contains data for your security before you index it with the Futures Symbol.

Select Language: 
for ticks in all_history_ticks:
    if ticks.contains_key(contract_symbol):
        ticks = ticks[contract_symbol]

You can also iterate through each of the Ticks.

Select Language: 
for ticks in all_history_ticks:
    for kvp in ticks:
        symbol = kvp.key
        tick = kvp.value

The Ticks objects only contain the last tick of each security for that particular timeslice

OpenInterest Objects

If the history method returns OpenInterest objects, iterate through the OpenInterest objects to get each one.

Select Language: 
for open_interest in single_history_open_interest:
    print(open_interest)

If the history method returns a dictionary of OpenInterest objects, iterate through the dictionary to get the OpenInterest of each Futures contract. The dictionary of OpenInterest objects may not have data for all of your Futures contract subscriptions. To avoid issues, check if the dictionary contains data for your contract before you index it with the Futures contract Symbol.

Select Language: 
for open_interest_dict in all_history_open_interest:
    if open_interest_dict.contains_key(contract_symbol):
        open_interest = open_interest_dict[contract_symbol]

You can also iterate through each of the OpenInterest dictionaries.

Select Language: 
for open_interest_dict in all_history_open_interest:
    for kvp in open_interest_dict:
        symbol = kvp.key
        open_interest = kvp.value

FutureHistory Objects

The future_history method returns a FutureHistory object. To get each slice in the FutureHistory object, iterate through it.

Select Language: 
for slice in future_history:
    for continuous_contract_symbol, chain in slice.futures_chains.items(): 
        for contract in chain:
            pass

To convert the FutureHistory object to a DataFrame that contains the trade and quote information of each contract, call the GetAllData method.

future_history.get_all_data()

To get the expiration dates of all the contracts in an FutureHistory object, call the GetExpiryDates method.

future_history.get_expiry_dates()

Plot Data

You need some historical Futures data to produce plots. You can use many of the supported plotting libraries to visualize data in various formats. For example, you can plot candlestick and line charts.

Candlestick Chart

Follow these steps to plot candlestick charts:

  1. Get some historical data.
    2. Select Language: 
    history = qb.history(contract_symbol, datetime(2021, 12, 1), datetime(2021, 12, 31), Resolution.DAILY)
  2. Drop the first two index levels.
    3. history.index = history.index.droplevel([0, 1])
  3. Import the plotly library.
    4. Select Language: 
    import plotly.graph_objects as go
  4. Create a Candlestick.
    5. Select Language: 
    candlestick = go.Candlestick(x=history.index,
                             open=history['open'],
                             high=history['high'],
                             low=history['low'],
                             close=history['close'])
  5. Create a Layout.
    6. Select Language: 
    layout = go.Layout(title=go.layout.Title(text=f'{contract_symbol.value} OHLC'),
                   xaxis_title='Date',
                   yaxis_title='Price',
                   xaxis_rangeslider_visible=False)
  6. Create the Figure.
    7. fig = go.Figure(data=[candlestick], layout=layout)
  7. Show the Figure.
    8. Select Language: 
    fig.show()

    Candlestick charts display the open, high, low, and close prices of the contract.

Candlestick plot of ES18H22 OHLC

Line Chart

Follow these steps to plot line charts using built-in methods:

  1. Get some historical data.
    2. Select Language: 
    history = qb.history(symbols, datetime(2021, 12, 1), datetime(2021, 12, 31), Resolution.DAILY)
  2. Drop the first index level.
    3. history.index = history.index.droplevel(0)
  3. Select data to plot.
    4. closing_prices = history['close'].unstack(level=0)
  4. Rename the columns to be the Symbol of each contract.
    5. closing_prices.columns = [Symbol.get_alias(SecurityIdentifier.parse(x)) for x in closing_prices.columns]
  5. Call the plot method on the pandas object.
    6. closing_prices.plot(title="Close", figsize=(15, 8))
  6. Show the plot.
    7. Select Language: 
    plt.show()

    Line charts display the value of the property you selected in a time series.

Line chart of close price of Future contracts

Examples

The following examples demonstrate some common practices for applying the Futures dataset.

Example 1: 5-Minute Candlestick Plot

The following example studies the candlestick pattern of the ES Future. To study the short term pattern, we consolidate the data into 5 minute bars and plot the 5-minute candlestick plot.

Select Language: 
# Import plotly library for plotting.
import plotly.graph_objects as go

# Create a QuantBook
qb = QuantBook()

# Request the continuous future historical data.
# Use raw normalization mode to fairly compare the actual bid and ask dollar volume.
future = qb.add_future(Futures.Indices.SP_500_E_MINI)
history = qb.history(future.symbol, 
    start=qb.time - timedelta(days=182), 
    end=qb.time, 
    resolution=Resolution.MINUTE, 
    fill_forward=True, 
    extended_market_hours=True, 
    data_mapping_mode=DataMappingMode.OPEN_INTEREST, 
    data_normalization_mode=DataNormalizationMode.BACKWARDS_RATIO, 
    contract_depth_offset=0)

# Drop level 0, 1 index (Symbol index) from the DataFrame
history = history.droplevel([0, 1])

# Select the required columns to obtain the 5-minute OHLC data.
history = history[["open", "high", "low", "close"]].resample("5T").agg({
    "open": "first",
    "high": "max",
    "low": "min",
    "close": "last"
})

# Crete the Candlestick chart using the 5-minute windows.
candlestick = go.Candlestick(x=history.index,
                             open=history['open'],
                             high=history['high'],
                             low=history['low'],
                             close=history['close'])
# Create a Layout as the plot settings.
layout = go.Layout(title=go.layout.Title(text=f'{future.symbol} OHLC'),
                   xaxis_title='Date',
                   yaxis_title='Price',
                   xaxis_rangeslider_visible=False)
# Create the Figure.
fig = go.Figure(data=[candlestick], layout=layout)
# Display the plot.
fig.show()

You can also see our Videos. You can also get in touch with us via Discord.

Did you find this page helpful?

Contribute to the documentation: