Share New Research 
Hi,
I have switched from Second to Tick resolution to make the algo ready for IB's live data feed. Now everything stopped working because in QC's tick data the last record's price does not match with the Close price of that particular second.
I vaguely remember reading under a thread that Jared was saying QC's tick data was not time sequenced. However, IB claims that their data feed is aggregated every 200-300 ms and is ordered correctly by time. Is there any way for me to back test my algo with tick data but get a proper time sequence so that it will be similar to IB's live feed? I don't mind if it's every 200, 300, or 500 ms as long as it's sequential.
Thanks!
Petter Hansson
The tick data in QC should be sequenced correctly (at least, that's why Jared said in a mail a little while ago) in a single instrument that is. Otherwise, what would be the point? This is a strong assumption I've made and if incorrect it will break my code, so I'm very curious about your results.
In backtests, QC tick data is timestamped to closest second though, meaning you will get one second worth of ticks at once (if any). This is different to live where any queued up ticks are sent to OnData ASAP. There may be a difference in how the timestamp is calculed though: For instance, it could be rounded or it could be truncated to closest second. If this is different from how second bars are producted at the source then it could lead to different results.
William Patterson
I may be wrong, but isn't tick data organized by purchase not by time. For instance during some illiquid markets at illiquid times a stock might not be traded for 5 minutes, and while 5 minute bars would have passed, no tick bars have happened during that period?
Petter Hansson
My understanding is you won't get any bars in OnData if there aren't any trades at all.
Patrick Star
Well let's talk with examples. Let's say I'm looking at ABC's prices starting at 2017-07-28 10:00:00.000.
QC's tick data comes as:
time price quantity
10:00:00.000 28.14 200
10:00:00.000 28.13 800
10:00:00.000 28.13 1200
10:00:00.000 28.12 200
10:00:00.000 28.14 400
10:00:00.000 28.12 200
10:00:01.000 28.13 500 // next second
10:00:01.000 28.14 300
10:00:01.000 28.14 400
10:00:02.000 28.15 900 // next second
And the second data for the same stock, same time span comes as:
time close volume
10:00:00.000 28.11 4800
10:00:01.000 28.14 1600
If the tick data was in proper order, the last tick during 10:00:00 would be $28.12 and therefore the close price on that second should be $28.12 but as you see the close price on that second is 28.11.
Patrick Star
Petter, you are right, If you don't have data for a (or more) second(s) you won't get a call back to OnData which is actually fine (at least for my algo) but again if the tick data is not sequential, the indicators and calculations will become inaccurate (again it may be only for my algo and not others).
Jared Broad
Tick data is sequential ordered in the list in time. Bars are built with a filtered subset of ticks,
Duplicate log messages are ignored so likely the logging of the ticks above ignored some of the ticks --
Tick data doesn't fill forward as there is no time period for a tick; its a spot value.
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Patrick Star
Thanks Jared
So you said duplicate logs are ignored but if there was a change in price, it wouldn't be duplicate, right? So the last Tick at 10:00:00 should have a price of $28.12 but the Second data is showing a close price of $28.11 for the same time period.
Note: these are not real numbers but basically the same logic as I see in the tests. If you want I can send you the symbol and date/time so you can take a look at the database directly.
This issue was not only on one record. I found other discrepencies with high/low numbers too. But again it could be me not understanding the aggregation process correctly.
Patrick Star
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
To unlock posting to the community forums please complete at least 30% of Boot Camp.
You can continue your Boot Camp training progress from the terminal. We hope to see you in the community soon!