Datasets

The IB data provider sources data directly from IB. For more information about the data source, see the Trader Workstation API documentation.

If you use the IB data provider, IB only provides the security price data. We provide the following auxiliary datasets from the Dataset Market:

• US Equity Security Master
• US Futures Security Master
• Universe selection datasets
• Non-streaming alternative datasets

Universe selection is available with the IB data provider.

UniverseSettings.Asynchronous = true;
AddUniverse(FundamentalUniverseSelection);

self.universe_settings.asynchronous = True
self.add_universe(self.fundamental_universe_selection)

The universe selection data comes from our Dataset Market, not the TWS market scanners. Universe selection with the IB data provider occurs around 6-7 AM Eastern Time (ET) on Tuesday to Friday and at 2 AM ET on Sunday. Universe selection data isn't available when the IB servers are closed. To check the IB server status, see the Current System Status page on the IB website.

The IB data provider can stream data for up to 100 assets by default, but IB may let you stream more than 100 assets based on your commissions and equity value. For more information about quotas from IB, see the Market Data Pricing Overview page on the IB website. If your algorithm adds more than the your quota, LEAN logs an error message from IB. To increase the quota, purchase a Quote Booster from IB.

The data is a summarized snapshot of the trades and quotes at roughly 300 milliseconds per snapshot.

Third-party data providers support most alternative datasets, except data that streams real-time intraday data. Streaming datasets, like the Tiingo News Feed and Benzinga News Feed, require the QuantConnect data provider. The hybrid QuantConnect-IB data provider supports streaming datasets.

When you deploy a live algorithm with the IB brokerage, you can use a third-party data provider, the IB data provider, or both. If you use multiple data providers, the order you select them in defines their order of precedence in Lean. For example, if you set QC as the first provider and IB as the second provider, Lean only uses the IB data provider for securities that aren't available from the QC data provider. This configuration makes it possible to use our data provider for Equity universe selection and then place Options trades on the securities in the universe. If you use a third-party data provider, the assets that you subscribe to don't contribute to the IB data limit.

If you get historical data from IB through a history request or a warm-up period, the historical data has the following characteristics:

• Second resolution data is limited to six months of history.
• The historical data excludes delisted Equities and expired Options.
• The historical data excludes expired Futures after two years.

The following quotas are in place for tick and second resolution historical data:

• You can have up to 50 simultaneous requests.
• You can make up to 60 requests within any 10-minute period.

In the preceding quotas, TradeBar and QuoteBar data count as separate requests. For example, if you request TradeBar and QuoteBar data for SPY, it counts as two requests.

For more information about historical data from IB, see Historical Data Limitations in the IB documentation.

To use IB data in your algorithms, subscribe to IB market data. We support all of the IB data subscriptions that are related to the securities and markets we support. Members usually subscribe to the following IB market data:

• US Securities Snapshot and Futures Value Bundle
• US Equity and Options Add-On Streaming Bundle
• CFE Enhanced Top of Book (L1 for VIX Futures)
• CME S&P Indexes (L1 for SPX and NDX)
• CBOE Streaming Market Indexes (L1 for VIX Index)

To see the latest prices, check the Market Data Pricing Overview page on the IB website. IB can take up to 24 hours to process subscription requests. So after you subscribe to data, you need to wait 24 hours before you can use it in your algorithms. When you subscribe to data, IB only assigns your data subscription to one of your accounts. If you want to assign the subscription to a different account, for example, a paper trading account instead of a live trading account, then contact IB.