Share New Research 
Hi,
I have switched from Second to Tick resolution to make the algo ready for IB's live data feed. Now everything stopped working because in QC's tick data the last record's price does not match with the Close price of that particular second.
I vaguely remember reading under a thread that Jared was saying QC's tick data was not time sequenced. However, IB claims that their data feed is aggregated every 200-300 ms and is ordered correctly by time. Is there any way for me to back test my algo with tick data but get a proper time sequence so that it will be similar to IB's live feed? I don't mind if it's every 200, 300, or 500 ms as long as it's sequential.
Thanks!
Petter Hansson
The tick data in QC should be sequenced correctly (at least, that's why Jared said in a mail a little while ago) in a single instrument that is. Otherwise, what would be the point? This is a strong assumption I've made and if incorrect it will break my code, so I'm very curious about your results.
In backtests, QC tick data is timestamped to closest second though, meaning you will get one second worth of ticks at once (if any). This is different to live where any queued up ticks are sent to OnData ASAP. There may be a difference in how the timestamp is calculed though: For instance, it could be rounded or it could be truncated to closest second. If this is different from how second bars are producted at the source then it could lead to different results.
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
William Patterson
I may be wrong, but isn't tick data organized by purchase not by time. For instance during some illiquid markets at illiquid times a stock might not be traded for 5 minutes, and while 5 minute bars would have passed, no tick bars have happened during that period?
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Petter Hansson
My understanding is you won't get any bars in OnData if there aren't any trades at all.
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Patrick Star
Well let's talk with examples. Let's say I'm looking at ABC's prices starting at 2017-07-28 10:00:00.000.
QC's tick data comes as:
time price quantity
10:00:00.000 28.14 200
10:00:00.000 28.13 800
10:00:00.000 28.13 1200
10:00:00.000 28.12 200
10:00:00.000 28.14 400
10:00:00.000 28.12 200
10:00:01.000 28.13 500 // next second
10:00:01.000 28.14 300
10:00:01.000 28.14 400
10:00:02.000 28.15 900 // next second
And the second data for the same stock, same time span comes as:
time close volume
10:00:00.000 28.11 4800
10:00:01.000 28.14 1600
If the tick data was in proper order, the last tick during 10:00:00 would be $28.12 and therefore the close price on that second should be $28.12 but as you see the close price on that second is 28.11.
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Patrick Star
Petter, you are right, If you don't have data for a (or more) second(s) you won't get a call back to OnData which is actually fine (at least for my algo) but again if the tick data is not sequential, the indicators and calculations will become inaccurate (again it may be only for my algo and not others).
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Jared Broad
Tick data is sequential ordered in the list in time. Bars are built with a filtered subset of ticks,
Duplicate log messages are ignored so likely the logging of the ticks above ignored some of the ticks --
Tick data doesn't fill forward as there is no time period for a tick; its a spot value.
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Patrick Star
Thanks Jared
So you said duplicate logs are ignored but if there was a change in price, it wouldn't be duplicate, right? So the last Tick at 10:00:00 should have a price of $28.12 but the Second data is showing a close price of $28.11 for the same time period.
Note: these are not real numbers but basically the same logic as I see in the tests. If you want I can send you the symbol and date/time so you can take a look at the database directly.
This issue was not only on one record. I found other discrepencies with high/low numbers too. But again it could be me not understanding the aggregation process correctly.
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Robert Peterson
Another idea for VIX strategy, low DD, high performance..
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
GABRIEL PIQUE ROCA
Hello, I'm tryng to backtest alex meuci strategy until february 2018, because of termination of XIV and I can't. Runtime Error: A data subscription for type 'PythonQuandl' was not found. Also I want to test this strategy but only with VXX. short VXX under 0.95 and long VXX above 1.05. How I should write these code.
Thanks
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Alex Muci
Just tried to re-run the first algo I posted above (VIX by RSI) - by limiting the backtesting to end of Jan 2018 (i.e. self.SetEndDate(2018,1,20)) - and it did work as before.
Gabriel, why do not you try and use long positions in ZIV (which shorts mid-curve VIX futures, rather than shorting VXX) for a safer and, potentially, easier way to hedge vol spikes with either front VIX futures (now available) or VXX calls?
If I have time will try to post such an example later.
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
GABRIEL PIQUE ROCA
Thanks for the advice, but I want to backtest the system with short VXX because I already have wich ratios VIX/VXX are optimal. How could I change the code to allow short VXX and long VXX. I'm still learning the documentation. Thanks anyway for the advice. Later I will try with ZIV.
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
GABRIEL PIQUE ROCA
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Pangyuteng
RIP XIV and the 1+ billions that vaporated during the liquidation event.
That being said, I am reviving this thread. ;) added my fix on data retrival below (may still have bugs!), some minor refactoring, and lastly, switched XIV to SVXY.
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Pangyuteng
adding another version below. Summary of chages listed below:
+ as suggested by Alex, added "momentum" term based on increasing/decreassing z score of the vix/vxv ratio.
+ logic for long/short volatility is based on if vix/vxv is trending towards contago/backwardation based on "momentum".
+ provided 2 kinds of allocation style - aggressive and conservative.
Backtest performance of the 2 kinds of allocation are listed below (timeframe: 2012 to present).
# aggressive mode: PSR 41%, win/loss rate 59/41, max drawdown 49%, return 1573%
# conservative mode: PSR 34%, win/loss rate 57/43, max drawdown 18%, return 246% (displayed below)
Happy holidays.
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Andrés M
Ted that great coding Ted, thanks for sharing! ... got a good laugh looking at the code (braveheart vs babies with 30 year mortgage LOL)
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Sunny Nagam
Still new to this but learned the hard way to use the Open and not Close from the custom data imported. Lookahead bias is quite powerful! See Open vs Close performance below:
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Sunny Nagam
Tried combining momentum indicators along with the vix/vix3m and vix/vix9d ratios as a mesure of contango/backwardation (I've tried the futures term structure ratios but Quantconnect's futures system makes me want to pull my hair out). I use indicators on SPY since I belive it's the "underlying" and more represtative of the underlying market that ultimately affects VIX values, and technical analysis on a actual ETF makes more sense to me.
I used the SPY crossing sma as an exit signal for short volatility, and SPY macd as an exit signal for long volatility. I find that macd tends to "want" to switch every once in a while even when growth is sustained (but not gaining momentum) and provides too many false positives as an exit signal for short vol since the majority of the time the market trends upwards gradually, so I used sma as a kind of trailing exit. However for long vol I find that once the party is over you want to get out quickly before the spike crashes, so I attempt to use SPY macd as measure of momentum and an exit to get out and minimze losses.
Additionally, usually when exiting a long vol position, much of the time the market is recovering even if temporarily, however due to the lingering backwardation this recovery is not always reflected in a short vol ETP such as SVXY, so I take advantage of this period by staying in TQQQ.
If anyone sees something I missed or could improvme, or has thoughts to add it would be much appreciated.
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Jared Broad
I think you should add 1 day to the Time or set the Period = timedelta(days=1). You might be getting some lookahead bias with the custom data imports. I walk through importing custom data here.
https://www.youtube.com/watch?v=VCf9e0S4rDgThe material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
To unlock posting to the community forums please complete at least 30% of Boot Camp.
You can continue your Boot Camp training progress from the terminal. We hope to see you in the community soon!