Datasets

Follow these steps to subscribe to an alternative dataset from the Dataset Market:

1. Load the required assembly files and data types.

#load "../Initialize.csx"
#load "../QuantConnect.csx"
#r "../Microsoft.Data.Analysis.dll"

using QuantConnect;
using QuantConnect.Data;
using QuantConnect.Algorithm;
using QuantConnect.Research;
using QuantConnect.DataSource;
using Microsoft.Data.Analysis;

2. Load the dynamic link library (DLL) of the dataset.

To load the DLL of any dataset, type:

#r "../QuantConnect.DataSource.<nameOfAlternativeDatasetClass>.dll"

For example, to load the DLL of the CBOE dataset, type:

#r "../QuantConnect.DataSource.CBOE.dll"

3. Create a QuantBook.

var qb = new QuantBook();

qb = QuantBook()

4. Call the AddDataadd_data method with the dataset class, a ticker, and a resolution and then save a reference to the alternative data Symbol.

var vix = qb.AddData<CBOE>("VIX", Resolution.Daily).Symbol;
var v3m = qb.AddData<CBOE>("VIX3M", Resolution.Daily).Symbol;

vix = qb.add_data(CBOE, "VIX", Resolution.DAILY).symbol
v3m = qb.add_data(CBOE, "VIX3M", Resolution.DAILY).symbol

To view the arguments that the AddDataadd_data method accepts for each dataset, see the dataset listing.

If you don't pass a resolution argument, the default resolution of the dataset is used by default. To view the supported resolutions and the default resolution of each dataset, see the dataset listing.

You need a subscription before you can request historical data for a dataset. On the time dimension, you can request an amount of historical data based on a trailing number of bars, a trailing period of time, or a defined period of time. On the dataset dimension, you can request historical data for a single dataset subscription, a subset of the dataset subscriptions you created in your notebook, or all of the dataset subscriptions in your notebook.

Trailing Number of Bars

To get historical data for a number of trailing bars, call the Historyhistory method with the Symbol object(s) and an integer.

// Slice objects
var singleHistorySlice = qb.History(vix, 10);
var subsetHistorySlice = qb.History(new[] {vix, v3m}, 10);
var allHistorySlice = qb.History(10);

// CBOE objects
var singleHistoryDataObjects = qb.History<CBOE>(vix, 10);
var subsetHistoryDataObjects = qb.History<CBOE>(new[] {vix, v3m}, 10);
var allHistoryDataObjects = qb.History<CBOE>(qb.Securities.Keys, 10);

# DataFrame
single_history_df = qb.History(vix, 10)
subset_history_df = qb.History([vix, v3m], 10)
all_history_df = qb.History(qb.Securities.Keys, 10)

# Slice objects
all_history_slice = qb.History(10)

# CBOE objects
single_history_data_objects = qb.History[CBOE](vix, 10)
subset_history_data_objects = qb.History[CBOE]([vix, v3m], 10)
all_history_data_objects = qb.History[CBOE](qb.Securities.Keys, 10)

The preceding calls return the most recent bars, excluding periods of time when the exchange was closed.

Trailing Period of Time

To get historical data for a trailing period of time, call the Historyhistory method with the Symbol object(s) and a TimeSpantimedelta.

// Slice objects
var singleHistorySlice = qb.History(vix, TimeSpan.FromDays(3));
var subsetHistorySlice = qb.History(new[] {vix, v3m}, TimeSpan.FromDays(3));
var allHistorySlice = qb.History(10);

// CBOE objects
var singleHistoryDataObjects = qb.History<CBOE>(vix, TimeSpan.FromDays(3));
var subsetHistoryDataObjects = qb.History<CBOE>(new[] {vix, v3m}, TimeSpan.FromDays(3));
var allHistoryDataObjects = qb.History<CBOE>(TimeSpan.FromDays(3));

# DataFrame
single_history_df = qb.History(vix, timedelta(days=3))
subset_history_df = qb.History([vix, v3m], timedelta(days=3))
all_history_df = qb.History(qb.Securities.Keys, timedelta(days=3))

# Slice objects
all_history_slice = qb.History(timedelta(days=3))

# CBOE objects
single_history_data_objects = qb.History[CBOE](vix, timedelta(days=3))
subset_history_data_objects = qb.History[CBOE]([vix, v3m], timedelta(days=3))
all_history_data_objects = qb.History[CBOE](qb.Securities.Keys, timedelta(days=3))

The preceding calls return the most recent bars or ticks, excluding periods of time when the exchange was closed.

Defined Period of Time

To get historical data for a specific period of time, call the Historyhistory method with the Symbol object(s), a start DateTimedatetime, and an end DateTimedatetime. The start and end times you provide are based in the notebook time zone.

var startTime = new DateTime(2021, 1, 1);
var endTime = new DateTime(2021, 3, 1);

// Slice objects
var singleHistorySlice = qb.History(vix, startTime, endTime);
var subsetHistorySlice = qb.History(new[] {vix, v3m}, startTime, endTime);
var allHistorySlice = qb.History(startTime, endTime);

// CBOE objects
var singleHistoryDataObjects = qb.History<CBOE>(vix, startTime, endTime);
var subsetHistoryDataObjects = qb.History<CBOE>(new[] {vix, v3m}, startTime, endTime);
var allHistoryDataObjects = qb.History<CBOE>(qb.Securities.Keys, startTime, endTime);

start_time = datetime(2021, 1, 1)
end_time = datetime(2021, 3, 1)

# DataFrame
single_history_df = qb.History(vix, start_time, end_time)
subset_history_df = qb.History([vix, v3m], start_time, end_time)
all_history_df = qb.History(qb.Securities.Keys, start_time, end_time)

# Slice objects
all_history_slice = qb.History(start_time, end_time)

# CBOE objects
single_history_data_objects = qb.History[CBOE](vix, start_time, end_time)
subset_history_data_objects = qb.History[CBOE]([vix, v3m], start_time, end_time)
all_history_data_objects = qb.History[CBOE](qb.Securities.Keys, start_time, end_time)

The preceding calls return the bars or ticks that have a timestamp within the defined period of time.

If you do not pass a resolution to the Historyhistory method, the Historyhistory method uses the resolution that the AddDataadd_data method used when you created the subscription.

You need some historical data to perform wrangling operations. The process to manipulate the historical data depends on its data type. To display pandas objects, run a cell in a notebook with the pandas object as the last line. To display other data formats, call the print method.

You need some historical data to perform wrangling operations. Use LINQ to wrangle the data and then call the Console.WriteLine method in a Jupyter Notebook to display the data. The process to manipulate the historical data depends on its data type.

DataFrame Objects

If the Historyhistory method returns a DataFrame, the first level of the DataFrame index is the encoded dataset Symbol and the second level is the EndTimeend_time of the data sample. The columns of the DataFrame are the data properties.

To select the historical data of a single dataset, index the loc property of the DataFrame with the dataset Symbol.

all_history_df.loc[vix]  # or all_history_df.loc['VIX']

To select a column of the DataFrame, index it with the column name.

all_history_df.loc[vix]['close']

If you request historical data for multiple tickers, you can transform the DataFrame so that it's a time series of close values for all of the tickers. To transform the DataFrame, select the column you want to display for each ticker and then call the unstack method.

all_history_df['close'].unstack(level=0)

The DataFrame is transformed so that the column indices are the Symbol of each ticker and each row contains the close value.

The historical data methods don't return DataFrame objects, but you can create one for efficient vectorized data wrangling.

using Microsoft.Data.Analysis; 

var columns = new DataFrameColumn[] {
    new PrimitiveDataFrameColumn("Time", history.Select(x => x[vix].EndTime)),
    new DecimalDataFrameColumn("VIX Open", history.Select(x => x[vix].Open)),
    new DecimalDataFrameColumn("VIX High", history.Select(x => x[vix].High)),
    new DecimalDataFrameColumn("VIX Low", history.Select(x => x[vix].Low)),
    new DecimalDataFrameColumn("VIX Close", history.Select(x => x[vix].Close))
};
var df = new DataFrame(columns);
df

To select a particular column of the DataFrame, index it with the column name.

df["VIX close"]

Slice Objects

If the Historyhistory method returns Slice objects, iterate through the Slice objects to get each one. The Slice objects may not have data for all of your dataset subscriptions. To avoid issues, check if the Slice contains data for your ticker before you index it with the dataset Symbol.

foreach (var slice in allHistorySlice) {
    if (slice.ContainsKey(vix))
    {
        var data = slice[vix];
    }
}

for slice in all_history_slice:
        if slice.contains_key(vix):
        data = slice[vix]

You need some historical alternative data to produce plots. You can use many of the supported plotting librariesPlot.NET package to visualize data in various formats. For example, you can plot candlestick and line charts.

Candlestick Chart

You can only create candlestick charts for alternative datasets that have open, high, low, and close properties.

Follow these steps to plot candlestick charts:

1. Get some historical data.

history = qb.history(vix, datetime(2021, 1, 1), datetime(2021, 2, 1)).loc[vix]

var history = qb.History<CBOE>(vix, new DateTime(2021, 12, 1), new DateTime(2021, 12, 31));

2. Import the plotlyPlot.NET library.

import plotly.graph_objects as go

#r "../Plotly.NET.dll"
using Plotly.NET;
using Plotly.NET.LayoutObjects;

3. Create a Candlestick.
   

candlestick = go.Candlestick(x=history.index,
                             open=history['open'],
                             high=history['high'],
                             low=history['low'],
                             close=history['close'])

var chart = Chart2D.Chart.Candlestick<decimal, decimal, decimal, decimal, DateTime, string>(
    history.Select(x => x.Open),
    history.Select(x => x.High),
    history.Select(x => x.Low),
    history.Select(x => x.Close),
    history.Select(x => x.EndTime)
);

4. Create a Layout.
   

layout = go.Layout(title=go.layout.Title(text='VIX from CBOE OHLC'),
                   xaxis_title='Date',
                   yaxis_title='Price',
                   xaxis_rangeslider_visible=False)

LinearAxis xAxis = new LinearAxis();
xAxis.SetValue("title", "Time");
LinearAxis yAxis = new LinearAxis();
yAxis.SetValue("title", "Price ($)");
Title title = Title.init($"{vix} OHLC");

Layout layout = new Layout();
layout.SetValue("xaxis", xAxis);
layout.SetValue("yaxis", yAxis);
layout.SetValue("title", title);

5. Create a Figure.
   

fig = go.Figure(data=[candlestick], layout=layout)

6. Assign the Layout to the chart.
   

chart.WithLayout(layout);

7. Show the Figure.
   

fig.show()

HTML(GenericChart.toChartHTML(chart))

Candlestick charts display the open, high, low, and close prices of the alternative data.

Line Chart

Follow these steps to plot line charts using built-in methodsPlotly.NET package:

1. Get some historical data.

history = qb.history([vix, v3m], datetime(2021, 1, 1), datetime(2021, 2, 1))

var history = qb.History<CBOE>(new [] {vix, v3x}, new DateTime(2021, 1, 1), new DateTime(2021, 2, 1));

2. Select the data to plot.

values = history['close'].unstack(0)

3. Call the plot method on the pandas object.

values.plot(title = 'Close', figsize=(15, 10))

4. Create Line charts.

var chart1 = Chart2D.Chart.Line<DateTime, decimal, string>(
    history.Select(x => x[vix].EndTime),
    history.Select(x => x[vix].Close),
    Name: $"{vix}"
);
var chart2 = Chart2D.Chart.Line<DateTime, decimal, string>(
    history.Select(x => x[v3m].EndTime),
    history.Select(x => x[v3m].Close),
    Name: $"{v3m}"
);

5. Create a Layout.
   

LinearAxis xAxis = new LinearAxis();
xAxis.SetValue("title", "Time");
LinearAxis yAxis = new LinearAxis();
yAxis.SetValue("title", "Price ($)");
Title title = Title.init($"{vix} & {v3m} Close Price");

Layout layout = new Layout();
layout.SetValue("xaxis", xAxis);
layout.SetValue("yaxis", yAxis);
layout.SetValue("title", title);

6. Combine the charts and assign the Layout to the chart.
   

var chart = Plotly.NET.Chart.Combine(new []{chart1, chart2});
chart.WithLayout(layout);

7. Show the plot.

plt.show()

HTML(GenericChart.toChartHTML(chart))

Line charts display the value of the property you selected in a time series.