Dear Community,

Today we shipped an upgrade to the object store that should improve its performance, reliability, and test coverage. Behind the scenes, it has been moved from a cloud host to a local network dramatically improving the access times and removing some latency.

 

Global Feature Addition

Since the object store was under maintenance, we also made it a "global" object store for the organization. Frequently the community requested that objects from one project were accessible to others. Now you can designate one project a training project, and access the same object across all your organization and team members. 

 

Backward Compatibility

All old objects are still accessible, but you must adjust your keys to include the project-id to access them. For example; if you were using the following key:

self.ObjectStore.Read("my-key")

You need to prefix it with your project id to access the old object.

self.ObjectStore.Read("123456/my-key")

You can find the project id in the URL of your browser when you open a project: e.g., quantconnect.com/project/12345

 

Cloning Projects

If you have two projects that use the same key, they will now overwrite the same object. It's important to update keys if you need to preserve the objects. You might want to use directory prefixes for example to ensure this isolation, or include the project id in the keys:

self.ObjectStore.Read("my-key") ->
self.ObjectStore.Read("12345/my-key")

 

Future Plans and Vision

Our plan over the next few months is to create a view for uploading and browsing the organization objects. This would allow you to see what is there and delete objects as needed.

As the objects are now local to your backtest, it allows them to be data for your strategy. We want to enable custom data types powered by the object store data. This would result in 10-100x speed-ups for custom data strategies that currently serve files from Dropbox/FTP. For example: 

AddData(EuropeanTickData, "ABC", Resolution.Tick)
...
class EuropeanTickData:
    GetSource():
         return ObjectStore("johns-files/tickdata/ABC/2022_08_01.zip")

 

If you have any feedback or issues, please let us know. We'll be updating the official documentation over the next 24 hours. Once it's updated you can find more information here: 

Sincerely,

Jared