Share New Research 
I'm thinking about using IQFeed for market data with IB using the " live-interactive-iqfeed " environment
Quick question. Do i have to start IQFeed prior to starting LEAN or will the LEAN engine start the IQFeed process?
Thanks
QUANTCONNECT COMMUNITY
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
QuantConnect™ 2024. All Rights Reserved
ScalpTrader
When i start up LEAN with the live-interactive-iqfeed environment variable IB gateway launches and then the IQConnect process is launched. But the login fails in IQConnect. I know that the username and password is correct as i'm able to use the feed with the IQFeed sample applications. Any idea why it would fail when using LEAN?
ScalpTrader
I'm able to get a bit further by starting the IQConnect program before launching LEAN. If that is done, LEAN connects to the admin port but then there's another failure
20180807 14:04:19.313 Trace:: IQFeed.Connect(): Connecting to admin...
20180807 14:04:20.341 Trace:: Loading IQFeed futures symbol map file...
20180807 14:04:20.344 Trace:: Found up-to-date IQFeed symbol universe file in local cache. Loading it...
20180807 14:04:33.997 ERROR:: IQFeed.Connect(): Error Connecting to IQFeed: Object reference not set to an instance of an object.
ScalpTrader
Does anyone know what are the correct values for these fields are in the config for IQFeed? I believe that productName needs to be valid for an application to attach to IQConnect
// iqfeed configuration
"iqfeed-productName": "",
"iqfeed-version": "1.0",
ScalpTrader
Does one need to be registered as a developer with IQFeed to get the product name?
Jared Broad
was the null exception on?
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
ScalpTrader
Thanks Jared.
The null exception seems to be coming from the call to;
_symbolUniverse = new IQFeedDataQueueUniverseProvider()
Jared Broad
Please post the full stack trace and error message
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
ScalpTrader
QuantConnect.ToolBox.IQFeed.IQFeedDataQueueUniverseProvider.LoadSymbols() in Lean\\ToolBox\\IQFeed\\IQFeedDataQueueUniverseProvider.cs:line 361 QuantConnect.ToolBox.IQFeed.IQFeedDataQueueUniverseProvider..ctor() in Lean\\ToolBox\\IQFeed\\IQFeedDataQueueUniverseProvider.cs:line 90 QuantConnect.ToolBox.IQFeed.IQFeedDataQueueHandler.Connect() Lean\\ToolBox\\IQFeed\\IQFeedDataQueueHandler.cs:line 227ScalpTrader
Just as a note i tested the IQFeed independantly on LEAN and it seems to work okay, so feed it good.
ScalpTrader
Jared Broad I will try to debug a bit if i get a chance on my end
ScalpTrader
I was able to debug a bit and found the initial problem. There's a ticker that causes the method ParseFutureTicker to fail. The ticker symbol is TST$Y
If i skip that symbol i no longer get the 'Object reference not set to an instance of an object' error
ScalpTrader
But after that error i see another error during Engine.Run()
20180809 19:51:52.500 ERROR:: Engine.Run(): During the algorithm initialization, the following exception has occurred: System.InvalidOperationException: Sequence contains no elements at System.Linq.Enumerable.First[TSource](IEnumerable`1 source) at QuantConnect.ToolBox.IQFeed.IQFeedDataQueueUniverseProvider.LookupSymbols(String lookupName, SecurityType securityType, String securityCurrency, String securityExchange) in Lean\ToolBox\IQFeed\IQFeedDataQueueUniverseProvider.cs:line 148 at QuantConnect.ToolBox.IQFeed.IQFeedDataQueueHandler.LookupSymbols(String lookupName, SecurityType securityType, String securityCurrency, String securityExchange) in Lean\ToolBox\IQFeed\IQFeedDataQueueHandler.cs:line 522 at QuantConnect.Lean.Engine.DataFeeds.Enumerators.Factories.FuturesChainUniverseSubscriptionEnumeratorFactory.CreateEnumerator(SubscriptionRequest request, IDataProvider dataProvider) in Lean\Engine\DataFeeds\Enumerators\Factories\FuturesChainUniverseSubscriptionEnumeratorFactory.cs:line 75 at QuantConnect.Lean.Engine.DataFeeds.LiveTradingDataFeed.CreateUniverseSubscription(SubscriptionRequest request) in Lean\Engine\DataFeeds\LiveTradingDataFeed.cs:line 692 at QuantConnect.Lean.Engine.DataFeeds.LiveTradingDataFeed.AddSubscription(SubscriptionRequest request) in Lean\Engine\DataFeeds\LiveTradingDataFeed.cs:line 171 at QuantConnect.Lean.Engine.DataFeeds.LiveTradingDataFeed.<>c__DisplayClass23_0.<Initialize>b__2(Object sender, NotifyCollectionChangedEventArgs args) in Lean\Engine\DataFeeds\LiveTradingDataFeed.cs:line 140 at QuantConnect.Securities.UniverseManager.OnCollectionChanged(NotifyCollectionChangedEventArgs e) in Lean\Common\Securities\UniverseManager.cs:line 263 at QuantConnect.Securities.UniverseManager.Add(Symbol key, Universe universe) in Lean\Common\Securities\UniverseManager.cs:line 171 at QuantConnect.Algorithm.QCAlgorithm.OnEndOfTimeStep() in Lean\Algorithm\QCAlgorithm.Universe.cs:line 138 at QuantConnect.Algorithm.QCAlgorithm.PostInitialize() in Lean\Algorithm\QCAlgorithm.cs:line 548 at QuantConnect.Lean.Engine.Setup.BrokerageSetupHandler.<>c__DisplayClass25_0.<Setup>b__1() in Lean\Engine\Setup\BrokerageSetupHandler.cs:line 219Sequence contains no elementsScalpTrader
I did a bit of debugging on this error. It seems that the LookupSymbols method in IQFeedDataQueueUniverseProvider.cs cannot find a symbol in the _symbolUniverse and the result variable is null.
Later result is accessed and this causes an exception.
The code that fails is the call var symbolData = ... below
public IEnumerable<Symbol> LookupSymbols ( string lookupName, SecurityType securityType, string securityCurrency = null, string securityExchange = null ) { Func<Symbol, string> lookupFunc; // for option, futures contract we search the underlying if ( securityType == SecurityType.Option || securityType == SecurityType.Future ) { lookupFunc = symbol => symbol.HasUnderlying ? symbol.Underlying.Value : string.Empty; } else { lookupFunc = symbol => symbol.Value; } var result = _symbolUniverse.Where ( x => lookupFunc ( x.Symbol ) == lookupName && x.Symbol.ID.SecurityType == securityType && ( securityCurrency == null || x.SecurityCurrency == securityCurrency ) && ( securityExchange == null || x.SecurityExchange == securityExchange ) ) .ToList(); bool onDemandRequests = result.All ( symbolData => !symbolData.IsDataLoaded() ); if ( onDemandRequests ) { var exchanges = securityType == SecurityType.Future ? _futuresExchanges.Values.Reverse().ToArray() : new string[] { }; // sorting list of available contracts by exchange priority, taking the top 1 var symbolData = result .OrderByDescending ( e => Array.IndexOf ( exchanges, e ) ) .First();ScalpTrader
The symbol passed into this method as variable lookupName was 'CL' as i'm running a Crude oil futures algo.
ScalpTrader
I'm attaching a copy of my test algorithm here. It subscribes to three futures contracts and just simply prints out the ticks
using System; using QuantConnect.Data; using QuantConnect.Securities; namespace QuantConnect.Algorithm.CSharp { public class MarketSpeedAlgorithm : QCAlgorithm { private const string RootWTI = Futures.Energies.CrudeOilWTI; private const string RootES = Futures.Indices.SP500EMini; private const string RootNQ = Futures.Indices.NASDAQ100EMini; public override void Initialize() { SetEndDate ( 2018, 08, 13 ); var futureWTI_Tick = AddFuture ( RootWTI, Resolution.Tick ); futureWTI_Tick.SetFilter ( TimeSpan.Zero, TimeSpan.FromDays ( 30 ) ); var futureES_Tick = AddFuture ( RootES, Resolution.Tick ); futureES_Tick.SetFilter ( TimeSpan.Zero, TimeSpan.FromDays ( 182 ) ); var futureNQ_Tick = AddFuture ( RootNQ, Resolution.Tick ); futureNQ_Tick.SetFilter ( TimeSpan.Zero, TimeSpan.FromDays ( 182 ) ); } public override void OnData ( Slice slice ) { foreach ( var chain in slice.FutureChains ) { foreach ( var contract in chain.Value ) { var bid = contract.BidPrice; var ask = contract.AskPrice; var last = contract.LastPrice; var volume = contract.Volume; Log ( String.Format ( "{0},Bid={1} Ask={2} Last={3} Volume={4}", contract.Symbol.Value, bid, ask, last, volume ) ); } } } } } // namespaceScalpTrader
This algorithm works fine when i have my setting as "environment": "live-interactive" but i get the failure mentioned above when using "environment": "live-interactive-iqfeed"
I'd really like to get the IQFeed working but it's not obvious why the code fails with IQFeed. Any help appreciated.
ScalpTrader
I'm using LEAN master branch and the latest version of the IQFeed client ( 6.0.0.5 )
Jared Broad
Thanks ScalpTrader we'll look into it and report back what we find. Just a hunch - it might not be able to find the universe of future contracts through IQFeed.
You can prove this by putting a breakpoint: "IQFeedDataQueueUniverseProvider.cs" > Line 223 and making sure its downloading and parsing the symbols you're trying to subscribe.
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
ScalpTrader
Looks like it's going into that method and successfully pulls the symbols from dtniq. I'm attaching log upto the failure point
20180814 13:04:22.516 Trace:: InteractiveBrokersBrokerage.ResetHandler(): thread started. 20180814 13:04:22.530 Trace:: IQFeed.Connect(): Launching client... 20180814 13:04:28.605 Trace:: IQFeed.Connect(): Connecting to admin... 20180814 13:04:35.892 Trace:: Loading IQFeed futures symbol map file... 20180814 13:04:39.531 Trace:: Loading and unzipping IQFeed symbol universe file (http://www.dtniq.com/product/mktsymbols_v2.zip)... 20180814 13:05:17.953 Trace:: ParseFutureTicker failed for ticker TST$Y 20180814 13:05:19.985 Trace:: Finished loading IQFeed symbol universe file. 20180814 13:05:20.027 Trace:: IQFeed.Connect(): Connecting to L1 data... 20180814 13:05:20.032 Trace:: IQFeed.Connect(): Connecting to Historical data... 20180814 13:05:20.033 Trace:: Config.Get(): Configuration key not found. Key: data-directory - Using default value: ../../../Data 20180814 13:05:20.033 Trace:: Config.GetValue(): max-history-minutes - Using default value: 5 20180814 13:05:20.041 Trace:: BaseDataExchange(DataQueueExchange) Starting... 20180814 13:05:20.041 Trace:: BaseDataExchange(CustomDataExchange) Starting... 20180814 13:05:20.049 Trace:: BrokerageSetupHandler.Setup(): Initializing algorithm... 20180814 13:05:20.049 Trace:: LiveTradingResultHandler.SendStatusUpdate(): Initializing Initializing algorithm... 20180814 13:05:20.054 Trace:: Config.GetValue(): send-via-api - Using default value: False 20180814 13:05:20.058 Trace:: Config.Get(): Configuration key not found. Key: security-data-feeds - Using default value: 20180814 13:05:20.148 Trace:: LiveTradingDataFeed.CreateUniverseSubscription(): Creating futures chain universe: /CL 20180814 13:05:21.051 Trace:: LiveTradingResultHandler.SendStatusUpdate(): LoggingIn Logging into brokerage... 20180814 13:05:21.051 Trace:: BrokerageSetupHandler.Setup(): Connecting to brokerage... 20180814 13:05:21.056 Trace:: InteractiveBrokersBrokerage.Connect(): Attempting to connect (1/5) ... 20180814 13:05:21.123 Trace:: IB message processing thread started: #24 20180814 13:05:21.125 Trace:: InteractiveBrokersBrokerage.HandleNextValidID(): 1 20180814 13:05:21.125 Trace:: IB next valid id received. 20180814 13:05:21.135 Trace:: InteractiveBrokersBrokerage.HandleError(): RequestId: -1 ErrorCode: 2104 - Market data farm connection is OK:usfuture 20180814 13:05:21.137 Trace:: Brokerage.OnMessage(): Warning - Code: 2104 - Market data farm connection is OK:usfuture 20180814 13:05:23.643 Trace:: BrokerageSetupHandler.Setup(): Fetching cash balance from brokerage... 20180814 13:05:23.645 Trace:: InteractiveBrokersBrokerage.CheckIbGateway(): start 20180814 13:05:23.755 Trace:: InteractiveBrokersBrokerage.CheckIbGateway(): end 20180814 13:05:23.764 Trace:: BrokerageSetupHandler.Setup(): Fetching open orders from brokerage... 20180814 13:05:23.769 Trace:: InteractiveBrokersBrokerage.HandleOpenOrderEnd() 20180814 13:05:23.770 Trace:: BrokerageSetupHandler.Setup(): Fetching holdings from brokerage... 20180814 13:05:23.771 Trace:: InteractiveBrokersBrokerage.CheckIbGateway(): start 20180814 13:05:23.851 Trace:: InteractiveBrokersBrokerage.CheckIbGateway(): end 20180814 13:05:23.862 ERROR:: Engine.Run(): During the algorithm initialization, the following exception has occurred: System.InvalidOperationException: Sequence contains no elements at System.Linq.Enumerable.First[TSource](IEnumerable`1 source)ScalpTrader
Did a bit more debugging on this.
Exception happens in this method
public IEnumerable<Symbol> LookupSymbols(string lookupName, SecurityType securityType, string securityCurrency = null, string securityExchange = null) { return _symbolUniverse.LookupSymbols(lookupName, securityType, securityCurrency, securityExchange); }_symbolUniverse was null
The other variables passed looked correct ( lookupName = "CL", securityType = Future )
ScalpTrader
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
To unlock posting to the community forums please complete at least 30% of Boot Camp.
You can continue your Boot Camp training progress from the terminal. We hope to see you in the community soon!