Behavior of OnData processing computationally intensive calculation

Hi,

Few questions:

I have added multiple symbols at minute resolution, the OnData function will be called per minute? Or as and when data arrives for any of the symbols? For LiveMode especially. I am seeing more onData being triggered more than once during a minute.
If my OnData function takes more 1 minute to complete, and a new OnData event is triggered, what will the behavior be?

I am currently running my main trading logics inside scheduled event, but would like the system to be more “real-time”, thus wanted to shift the logic into OnData. But am worried that the computation time is too long and the next OnData event is triggered before the earlier one complete.

3. What if I set the data resolution to tick? Is it advisable? I am good if the ticks which come in during the OnData's processing are ignored. As long as there isn't concurrency issues etc :D

Thks!

Hi pranava ,

Sorry about the wait.

If we added multiple securities to the algorithm, their data is synchronized and delivered at OnData in both backtesting and live mode. You have seen OnData being triggered more than once per minute in the live mode because of a latency issue with QuantConnect data feed that we have been working on. At the moment, if we subscribe to 1000 securities, the latency is about 300ms. In principle, the OnData method should rarely be triggered more than once per minute now.

In case OnData takes more ran one minute to run, it will skip the data. For example, if OnData took 3 minutes to run from 10:00 to 10:03, it won't see data from 10:01 and 10:02. We recommend using Scheduled Events for this case. We don't advise against using tick resolution data.

Since it's not a matter of computational power, running Lean locally won't address these problems. Additionally, QuantConnect live data feed is only available in QuantConnect Cloud.

So, if you want to run Lean locally, you will need to get an alternative live data feed. You can use the data feed provided by either Interactive Brokers or Tradier if the algorithm is trading with these brokerages. Alternatively, you can use IQFeed. Assuming that the algorithm will need to make historical data requests, you will need to acquire historical data. The Security Master provides splits, dividends, and ticker change data. The US Equity market data is bought separately, but Security Master is required.

Best regards,
Alex

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.

Hi,

If i were to run lean locally, with a more power workstation, I guess it would reduce the problem.

How much would the data cost? I am seeing the Security Master costing $600 a year. Does it provide minutes data or just daily granularity?

Thanks!

Platform

Radically Open-Source Algorithmic Trading Engine

Join Our Discord Channel

Quarterly Open-Source Trading Competition

Draft Discussions

Bookmarked Discussions

SEARCH DISCUSSIONS

369,200 Quants.

VOTE FOR UPCOMING FEATURES

JOIN OUR Community MAILING LIST

IN THIS RESEARCH

PARTICIPANTS

Actions

Join QuantConnect for Free