Research Environment

To get all of the keys and values in the Object Store, iterate through the ObjectStoreobject_store property.

foreach (var kvp in qb.ObjectStore)
{
    var key = kvp.Key;
    var value = kvp.Value;
}

for kvp in qb.object_store:
    key = kvp.key
    value = kvp.value

To iterate through just the keys in the Object Store, iterate through the Keyskeys property.

foreach (var key in qb.ObjectStore.Keys)
{
    continue;
}

for key in qb.object_store.keys:
    continue

You need some data to store data in the Object Store.

Follow these steps to create some sample data:

1. Create a dictionary.

var dictSample = new Dictionary<string, int> { {"One", 1}, {"Two", 2}, {"Three", 3} };

2. Create a string.

var stringSample = "My string";

string_sample = "My string"

3. Create a Bytes object.

var bytesSample = Encoding.UTF8.GetBytes("My String");

bytes_sample = str.encode("My String")

4. Convert the dictionary to an XML-formatted object.

var xmlSample = new XElement("sample",
    dictSample.Select(kvp => new XElement(kvp.Key, kvp.Value)));
Console.WriteLine(xmlSample.ToString());

The Object Store saves objects under a key-value system. If you save objects in backtests, you can access them from the Research Environment.

If you run algorithms in QuantConnect Cloud, you need storage create permissions to save data in the Object Store.

If you don't have data to store, create some sample data.

You can save the following types of objects in the Object Store:

• Bytes objects
• string objects
• JSON objects
• XML-formatted objects

You can save Bytes and string objects in the Object Store.

Strings

To save a string object, call the Savesave or SaveStringsave_string method.

var saveSuccessful = qb.ObjectStore.Save($"{qb.ProjectId}/stringKey", stringSample);

save_successful = qb.object_store.save(f"{qb.project_id}/string_key", string_sample)

JSON

To save a JSON object, call the SaveJson<T> method. This method helps to serialize the data into JSON format.

var saveSuccessful = qb.ObjectStore.SaveJson<Dictionary<string, int>>($"{qb.ProjectId}/jsonKey", dictSample);

XML

To save an XML-formatted object, call the SaveXml<T> method.

var saveSuccessful = qb.ObjectStore.SaveXml<XElement>($"{qb.ProjectId}/xmlKey", xmlSample);

Bytes

To save a Bytes object (for example, zipped data), call the SaveBytessave_bytes method.

var saveSuccessful = qb.ObjectStore.SaveBytes($"{qb.ProjectId}/bytesKey", bytesSample)

var zippedDataSample = Compression.ZipBytes(Encoding.UTF8.GetBytes(stringSample), "data");
var saveSuccessful = qb.ObjectStore.SaveBytes($"{qb.ProjectId}/bytesKey.zip", zippedDataSample);

save_successful = qb.object_store.save_bytes(f"{qb.project_id}/bytes_key", bytes_sample)

zipped_data_sample = Compression.zip_bytes(bytes(string_sample, "utf-8"), "data")
zip_save_successful = qb.object_store.save_bytes(f"{qb.project_id}/bytesKey.zip", zipped_data_sample)

To read data from the Object Store, you need to provide the key you used to store the object.

You can load the following types of objects from the Object Store:

• Bytes objects
• string objects
• JSON objects
• XML-formatted objects

You can load Bytes and string objects from the Object Store.

Before you read data from the Object Store, check if the key exists.

if (qb.ObjectStore.ContainsKey(key))
{
    // Read data
}

if qb.object_store.contains_key(key):
    # Read data

Strings

To read a string object, call the Readread or ReadStringread_string method.

var stringData = qb.ObjectStore.Read($"{qb.ProjectId}/stringKey");

string_data = qb.object_store.read(f"{qb.project_id}/string_key")

JSON

To read a JSON object, call the ReadJson<T> method.

var jsonData = qb.ObjectStore.ReadJson<Dictionary<string, int>>($"{qb.ProjectId}/jsonKey");

XML

To read an XML-formatted object, call the ReadXml<T> method.

var xmlData = qb.ObjectStore.ReadXml<XElement>($"{qb.ProjectId}/xmlKey");

If you created the XML object from a dictionary, reconstruct the dictionary.

var dict = xmlData.Elements().ToDictionary(x => x.Name.LocalName, x => int.Parse(x.Value));

Bytes

To read a Bytes object, call the ReadBytesread_bytes method.

var bytesData = qb.ObjectStore.ReadBytes($"{qb.ProjectId}/bytesKey");

byte_data = qb.object_store.read_bytes(f"{qb.project_id}/bytes_key")

Delete objects in the Object Store to remove objects that you no longer need. If you use the Research Environment in QuantConnect Cloud, you need storage delete permissions to delete data from the Object Store.

To delete objects from the Object Store, call the Deletedelete method. Before you delete data, check if the key exists. If you try to delete an object with a key that doesn't exist in the Object Store, the method raises an exception.

if (qb.ObjectStore.ContainsKey(key))
{
    qb.ObjectStore.Delete(key);
}

if qb.object_store.contains_key(key):
    qb.object_store.delete(key)

To delete all of the content in the Object Store, iterate through all the stored data.

foreach (var kvp in qb.ObjectStore)
{
    qb.ObjectStore.Delete(kvp.Key);
}

for kvp in qb.object_store:
    qb.object_store.delete(kvp.key)

When you write to or read from the Object Store, the notebook caches the data. The cache speeds up the notebook execution because if you try to read the Object Store data again with the same key, it returns the cached data instead of downloading the data again. The cache speeds up execution, but it can cause problems if you are trying to share data between two nodes under the same Object Store key. For example, consider the following scenario:

1. You open project A and save data under the key 123.
2. You open project B and save new data under the same key 123.
3. In project A, you read the Object Store data under the key 123, expecting the data from project B, but you get the original data you saved in step #1 instead.

You get the data from step 1 instead of step 2 because the cache contains the data from step 1.

To clear the cache, call the Clear method.

qb.ObjectStore.Clear();

qb.object_store.clear()

To get the file path for a specific key in the Object Store, call the GetFilePathget_file_path method. If the key you pass to the method doesn't already exist in the Object Store, it's added to the Object Store.

var filePath = qb.ObjectStore.GetFilePath(key);

file_path = qb.object_store.get_file_path(key)

If you use the Research Environment locally, you can store as much data as your hardware will allow. If you use the Research Environment in QuantConnect Cloud, you must stay within your storage quota. If you need more storage space, edit your storage plan.

Follow these steps to create a DataFrame, save it into the Object Store, and load it from the Object Store:

1. Get some historical data.

var spy = qb.AddEquity("SPY").Symbol;
var history = qb.History(qb.Securities.Keys, 360, Resolution.Daily);

spy = qb.add_equity("SPY").symbol
df = qb.history(qb.securities.keys, 360, Resolution.DAILY)

2. Create a DataFrame.

using Microsoft.Data.Analysis; // 

var columns = new DataFrameColumn[] {
    new DateTimeDataFrameColumn("Time", history.Select(x => (DateTime)x[spy].EndTime)),
    new DecimalDataFrameColumn("SPY Open", history.Select(x => (decimal)x[spy].Open)),
    new DecimalDataFrameColumn("SPY High", history.Select(x => (decimal)x[spy].High)),
    new DecimalDataFrameColumn("SPY Low", history.Select(x => (decimal)x[spy].Low)),
    new DecimalDataFrameColumn("SPY Close", history.Select(x => (decimal)x[spy].Close))
};
var df = new DataFrame(columns);

3. Get the file path for a specific key in the Object Store.

var filePath = qb.ObjectStore.GetFilePath("df_to_csv");

file_path = qb.object_store.get_file_path("df_to_csv")

4. Call the SaveCsvto_csv method to save the DataFrame in the Object Store as a CSV file.

DataFrame.SaveCsv(df, filePath);    // File size: 26520 bytes

df.to_csv(file_path)   # File size: 32721 bytes

5. Call the LoadCsvread_csv method to load the CSV file from the Object Store.

var reread = DataFrame.LoadCsv(filePath);

reread = pd.read_csv(file_path)

pandas supports saving and loading DataFrame objects in the following additional formats:

• XML

file_path = qb.object_store.get_file_path("df_to_xml")
df.to_xml(file_path)   # File size: 87816 bytes
reread = pd.read_xml(file_path)

• JSON

file_path = qb.object_store.get_file_path("df_to_json")
df.to_json(file_path)   # File size: 125250 bytes
reread = pd.read_json(file_path)

• Parquet

file_path = qb.object_store.get_file_path("df_to_parquet")
df.to_parquet(file_path)   # File size: 23996 bytes
reread = pd.read_parquet(file_path)

• Pickle

file_path = qb.object_store.get_file_path("df_to_pickle")
df.to_pickle(file_path)   # File size: 19868 bytes
reread = pd.read_pickle(file_path)

You can use the Object Store to plot data from your backtests and live algorithm in the Research Environment. The following example demonstrates how to plot a Simple Moving Average indicator that's generated during a backtest.

1. Create a algorithm, add a data subscription, and add a simple moving average indicator.

public class ObjectStoreChartingAlgorithm : QCAlgorithm
{
    private SimpleMovingAverage _sma;
    private string _content;

    public override void Initialize()
    {
        AddEquity("SPY", Resolution.Minute);
        _sma = SMA("SPY", 22);
    }
}

class ObjectStoreChartingAlgorithm(QCAlgorithm):
    def initialize(self):
        self.add_equity("SPY")
    
        self.content = ''
        self._sma = self.sma("SPY", 22)

The algorithm will save _contentself.content to the Object Store.

2. Save the indicator data as string in _contentself.content.

public override void OnData(Slice data)
{
    _content += $"{_sma.Current.EndTime},{_sma}\n";
}

def on_data(self, data: Slice):
    self.plot('SMA', 'Value', self.sma.current.value)
    self.content += f'{self.sma.current.end_time},{self.sma.current.value}\n'

3. In the OnEndOfAlgorithm method, save the indicator data to the Object Store.

public override void OnEndOfAlgorithm()
{
    ObjectStore.Save("sma_values_csharp", _content);
}

def on_end_of_algorithm(self):
    self.object_store.save('sma_values_python', self.content)

4. Open the Research Environment and create a QuantBook.

// Execute the following command in first
#load "../Initialize.csx"

// Create a QuantBook object
#load "../QuantConnect.csx"
using QuantConnect;
using QuantConnect.Research;

var qb = new QuantBook();

qb = QuantBook()

5. Read the indicator data from the Object Store.

var content = qb.ObjectStore.Read("sma_values_csharp");

content = qb.object_store.read("sma_values_python")

The key you provide must be the same key you used to save the object.

6. Convert the data to a pandas object and create a chart.

data = {}
for line in content.split('\n'):
    csv = line.split(',')
    if len(csv) > 1:
        data[csv[0]] = float(csv[1])

series = pd.Series(data, index=data.keys())
series.plot()

7. Import the Plotly.NET and Plotly.NET.LayoutObjects packages.

#r "../Plotly.NET.dll"
using Plotly.NET;
using Plotly.NET.LayoutObjects;

8. Create the Layout object and set the title, xaxis, and yaxis properties.

var layout = new Layout();
layout.SetValue("title", Title.init("SMA"));

var xAxis = new LinearAxis();
xAxis.SetValue("title", "Time");
layout.SetValue("xaxis", xAxis);

var yAxis = new LinearAxis();
yAxis.SetValue("title", "SMA");
layout.SetValue("yaxis", yAxis);

9. Convert the data to a list of DateTime objects for the chart x-axis and a list of decimal objects for the chart y-axis, then create a Chart2D.Chart.Line object with the data.

var index = new List<DateTimee>();
var values = new List<decimal>();

foreach (var line in content.Split('\n'))
{
    var csv = line.Split(',');
    if (csv.Length > 1)
    {
        index.Add(Parse.DateTime(csv[0]));
        values.Add(decimal.Parse(csv[1]));
    }
}

var chart = Chart2D.Chart.Linee<DateTime, decimal, stringe>(index, values);

10. Apply the layout to the Line object and create the HTML object.

chart.WithLayout(layout);
var result = HTML(GenericChart.toChartHTML(chart));

public class ObjectStoreChartingAlgorithm : QCAlgorithm
{
    private SimpleMovingAverage _sma;
    private string _content;

    public override void Initialize()
    {
        SetStartDate(2023, 1, 1);   //Set Start Date
        SetCash(100000);            //Set Strategy Cash
        
        AddEquity("SPY", Resolution.Minute);
        // Create SMA indicator for referencing.
        _sma = SMA("SPY", 22);
    }

    public override void OnData(Slice data)
    {
        // Cache the indicator data point to save it.
        _content += $"{_sma.Current.EndTime},{_sma}\n";
    }

    public override void OnEndOfAlgorithm()
    {
        // Save the indicator values to object store for logging.
        ObjectStore.Save("sma_values_csharp", _content);
    }
}

class ObjectStoreChartingAlgorithm(QCAlgorithm):
    def initialize(self) -> None:
        self.set_start_date(2023, 1, 1)  # Set Start Date
        self.set_cash(100000)           # Set Strategy Cash
        self.add_equity("SPY", Resolution.MINUTE)
        
        self.content = ''
        # Create SMA indicator for referencing.
        self.sma = self.SMA("SPY", 22)
        
    def on_data(self, data: Slice) -> None:
        # Cache the indicator data point to save it.
        self.content += f'{self.sma.current.end_time},{self.sma.current.value}\n'

    def on_end_of_algorithm(self) -> None:
        # Save the indicator values to object store for logging.
        self.object_store.save('sma_values_python', self.content)