Share New Research 
Hi! Two questions:
1. Can we paper trade locally with LEAN and IB using our own data sources?
From my understanding, when paper trading with IB, we can either choose an IB or QC datafeed. Is it possible to use our own custom datafeeds instead? I'm running LEAN locally and am using my own data obtained from some vendors. Backtesting works fine, but once I try to advance my strategy to paper trading I don't seem to be able to access the data on my local machine anymore.
Is there anything I can do to update the config file? I tried changing the `data-feed-handler` to `FileSystemDataFeed` as seen below which I thought may fix it but unfortunately not.
// defines the 'live-interactive' environment
"live-interactive": {
"live-mode": true,
// real brokerage implementations require the BrokerageTransactionHandler
"live-mode-brokerage": "InteractiveBrokersBrokerage",
"setup-handler": "QuantConnect.Lean.Engine.Setup.BrokerageSetupHandler",
"result-handler": "QuantConnect.Lean.Engine.Results.LiveTradingResultHandler",
// "data-feed-handler": "QuantConnect.Lean.Engine.DataFeeds.LiveTradingDataFeed",
"data-feed-handler": "QuantConnect.Lean.Engine.DataFeeds.FileSystemDataFeed",
"data-queue-handler": "QuantConnect.Brokerages.InteractiveBrokers.InteractiveBrokersBrokerage",
"real-time-handler": "QuantConnect.Lean.Engine.RealTime.BacktestingRealTimeHandler",
"transaction-handler": "QuantConnect.Lean.Engine.TransactionHandlers.BrokerageTransactionHandler",
"history-provider": "BrokerageHistoryProvider"
}2. Can we use Universe Selection when paper trading with IB?
I read that Universe Selection is not available with IB, but from my understanding (reading the forums and docs) that's only if we use IB's data feeds.
Thanks!
Shile Wen
Hi Serena,
Users can paper trade using IB and LEAN using custom data sources. Please see our documentation on how to use custom data feeds here.
Users can also write their data in LEAN format by replacing the
"data-folder":value in the config to use their data instead.
Universe Selection with IB is possible through our CoarseUniverseGenerator.
Best,
Shile Wen
Serena M McDonnell
Hi Shile,
Thanks for your reply!
I actually am not using custom data sources as described in that link, but I've saved my own data from some vendors to the required `Lean/Data` folder. So this custom data is daily tick data, as well as coarse and fine fundamental data. It all works fine when I backetest, but not when using the `live-interactive` mode. That's why I set the `data-feed-handler` to the `FileSystemDataFeed` in an effort to get LEAN to read data from my local machine rather than pull from IB's datafeed. Essentially I would like to use my data locally, with universe selection, to create trading rules. Then I would like my orders to be executed via IB for paper trading.
Could you clarify the role of the CoarseUniverseGenerator? I'm hoping that since I'm using my own fundmantal data that I can still use both coarse and fine universe selection functions. As mentioned, my understanding is that universe selection is not available when using IB's data feeds, but then shouldn't it work if I'm using my own data, especially since it worked when backtesting?
Alexandre Catarino
Hi Serena M McDonnell ,
Sorry about the wait.
For live trading, we cannot use `FileSystemDataFeed` because the data on disk is past data, and the way the data is enqueued is different: it won't read the files every minute/second for new data. We need to use either `InteractiveBrokersBrokerage` or `IQFeedDataQueueHandler` or your implementation of `IDataQueueHandler`. For historical data requests, we can use the `SubscriptionDataReaderHistoryProvider`, but it won't work as well as our cloud implementation.
The role of the CoarseUniverseGenerator is to generate the Coarse Universe Selection data that will be used to create the data that will be using in the Universe Selection functions. The code to handle it in live mode is proprietary (not available in open-source) so you will need to implement it.
Serena M McDonnell
Hi Alexandre, thanks for your reply!
To confirm, the only way to use my own data feeds (for daily data) is to write my own implementation of `IDataQueueHandler`?
Jared Broad
It's not the only way, there many data feeds open-sourced and in LEAN. But if you'd like to do something customized, you'll need to implement an IDataQueueHandler to feed the live system data. There you can pump in your custom live source. Do you have a data source for coarse universe selection?
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Serena M McDonnell
Hi Jared!
If that's not the only way, can you share examples of other ways to use my own data so I can get a better idea? To re-iterate, I have a daily data feed from a vendor, so yes I have a data source for coarse universe selection. It works fine when I backtest but not when I paper trade. I'd like to use that daily data to then create orders, and execute those orders via paper trading. So the steps would roughly be:
1. Universe selection with custom data feed (this is NOT live/streamed data but would include data up until the day of trading)
2. Create insights based on this custom historical data feed
3. Execute the orders based on insights in paper trading via Interactive Brokers
Another way to phrase it: today is June 25, 2020. I'd like to paper trade a strategy that looks at historical data, prior to June 25, sourced from a vendor. The data is stored on my local machine. If certain conditions in the historical data are met, I would like my order to be executed and be displayed in my paper trading account with Interactive Brokers.
Jared Broad
Can you please share the vendor's name we'll check their API to see the pathway.
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Paula Ten
Hi Serena,
I hope this message reaches you well, and you might be able to assist me in some way. I'm currently trying to implement my QuantConnect algorithm in Live mode with my InteractiveBrokers data subscriptions (specifically for European futures markets). The response I've received when inquiring is that I need to implement an 'IDataQueueHandler' but unfortunately, I can't find any examples of implementation in Python anywhere. I'm new to this, and I would greatly appreciate any guidance or assistance you can provide.
Thank you for your time!
Paula.
Serena M McDonnell
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
To unlock posting to the community forums please complete at least 30% of Boot Camp.
You can continue your Boot Camp training progress from the terminal. We hope to see you in the community soon!