Share New Research 
Hi All!
We're excited to share that coming soon there will be L1 quote data automatically included in backtests for US equity stocks. We are also updating the historical and live data feed to be based on the SIP; the aggregated tick feed from all the exchanges.
We strive to make our backtests as close as possible to live trading, but this modeling is difficult without the full volume and a realistic spread. With the new data sources, it should be a significant advance in the realism of the backtests. The new source would be identical to paper trading, allowing your live paper trading to be more realistic than ever.
If you are interested in the engineering please feel free to subscribe to follow along. Once the code is ready and data in place we'll merge the pull-request and hot-swap the data over. There should be no downtime. I will also comment on this discussion thread once the merge is complete.
Important Note #1: Because the backtests will now have spread; your backtest performance will likely be worse than it was before! I understand this will be upsetting but please see it as a more realistic reflection of the model. It means if you buy and sell in the same second, it will constantly lose the spread between the equity bid and ask. This cost will be significant if you are trading quickly or on low liquidity assets.
Important Note #2: Quote data is approximately 3x the amount of data. Although the processing speed of LEAN will get faster in "data points per second" -- the absolute wall-clock time for the backtest will increase. We've got 2-3 massive engineering leaps up our sleeves which should offset that delay by running 300% faster... there will be 1-2 weeks where the backtest will be slower while we install those new engineering advances.
We expect this upgrade to happen in the next week. Thank you for supporting QuantConnect! Here's to an awesome 2020!
Best Regards,
Jared
Pangyuteng
Very exciting and wonderful update guys!
The more data the merrier. Much preferred to be upset now then later!!
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Arthur Asenheimer INVESTOR
I totally agree with Ted. You guys are doing a great job!
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Jared Broad
Thanks Arthur! =)
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Jordan Leigh
Very interested to use this new data.
How will this effect the alpha markets existing algos? It's likley that algos that were previous thought to be suitable may no longer be?
I guess having an overall, higher quality/realistic algo offering would amount to a better QC reputation amongst funds.
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Dirk bothof
Looking forward to the performance impact on my algo's, exciting! Short side question relating to this: what will be the impact of including this data on the Alpha slippage model, is it still relevant or will this be depricated?
best,
Dirk
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Jared Broad
The alphas in the market will be stopped and redeployed (with their history intact but a mark on curve noting the change in the data source). If they continue to perform well with the new slippage factored in they'll be left to run.
Separately we're building in an alpha-system-wide, automatic "stop" mechanism for alphas who no longer perform according to backtest. This should stop any which fail due to the new slippage.
The AlphaStream slippage model will be reduced but not eliminated. Even after accounting for the bid-ask spread there is still slippage on the filling of a large order. This slippage should be made proportional to the size of the trade.
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Petter Hansson
Fantastic news. Overly optimistic fills was really one of the main time killers when starting out (before knowing better). Not to mention all other potential uses. :-)
Will it be possible to find e.g. average spread in universe selection to e.g. filter out securities with too wide spreads?
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Jared Broad
Cool idea Petter; once installed we can start exploring expanding coarse universe to factor in spreads.
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Laurent Crouzet
Hey Jared Broad
That is a very good enhancement! It is so important (at time for Alpha's authors and for QuantConnect!) to prove that the Alpha on the marketplace are not profitting from any data anomally to make the QC Alpha Streams marketplace the reference!
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Soumya sen
Looking forward to it
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Dirk bothof
Hi Jared! Any news on the progress, or an ETA, I'm so freaking excited to see what it does to my backtests : )
Best,
Dirk
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Jared Broad
=D Thanks Dirk getting there. We hit a roadblock in the final deployment of the data and needed to rebuild the entire 40TB data set. It's now processed and we're re-running the production update process to ensure it is updated each day, on time for the universe selection in live trading.
We're optimistic it'll be ready this week.
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Laurent Crouzet
Hey Jared!
Hope you are well, and that this improvement will soon be effective!
Do you know when we will be able to try and test this L1 Equity Quote Data?
Best regards, and keep up the good work!
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Jared Broad
Hey Laurent! Getting there! We've green-lit the pull request and are not making the tests to ensure the redundant data processing is online from day-1 as well. We currently have full redundancy in the data stack and would like to keep it that way.
-----
One interesting change was the new coarse universe data dollar volume ranks change due to the new rules of what ticks will be included in the final volume math. Previously FINRA ticks were excluded from bars as they are not tradable (FINRA feed is post-trade reporting). This generated thousands of support tickets and forum questions from people wondering why we didn't match Yahoo.
We've opted to include this volume now to quiet these questions; in the hope that the more complete representation of volumes will improve those strategies sensitive to dollar-volume rankings; and technical indicators that require trade volume.
Since the new data starts in 2007 we'll also go back and reprocess all the 1998-2007 data with these new tick rules to match the SIP feed data. This will ensure consistent ranking behavior from 1998-present.
Important Note #3: Dollar volume rankings of stocks will change with this new data. The analysis of the rank changes shows smaller ETFs (e.g. Brazilian Stock Market ETF) which previously had high ranks are now displaced by stock counterparts. We can see the deviation of the two data sources from last week's coarse universe within the plots below:
For a single day the differences are more obvious:
Which if you look at in detail is mostly movements of obscure assets out of the top 100, and popular assets into the top 100. We think this is a good change and although a breaking change, makes the data better overall.
The further you go back in time the differences become less apparent. Presumably due to lower dark-pool volume which is reported to FINRA; this is the same scatter from 2008:
-----------
Ultimately the most important thing is that our backtesting matches live-trading. So once the new live SIP feed is installed we'll also have the FINRA dark pool data there for algorithm consumption! We'll email all the impacted live users and recommend they re-backtest their strategies to confirm the coarse selection process still run as they expect or to take their strategies offline for review.
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Dave Mueller
Hi Jared: I'm super excited about this addition. Is this data in full effect for backtesting and live trading now? If not, how far out is it expected to be?
Again, super exciting stuff! Thank you and best regards
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Jared Broad
Thanks David we're excited! The backtesting can be this week. The live feed paperwork is signed and the prototype live feed is in place. We're doing load tests on the live data feed now to ensure the new increases in tick volume won't overwhelm the running algorithms. I'd ideally like to hold off deploying the backtesting data until we can swap the live feed at the same time. This might make it after market close Friday.
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Jared Broad
Hello All! This was deployed today! Hopefully, you start seeing more accurate backtests =) If you're trading very liquid assets you won't notice much of a difference, but illiquid smaller assets will see a spread between the buy and sell price they achieve.
If you see anything you think is a bug to support@quantconnect.com ideally with a code snippet so we can repeat the issue.
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Pierre Vidal
Hi Jared,
This is good news, well done to the team!
I tried one simple backtest and my result actually got somewhat *better*. I am not complaining... unless this reveals a deficiency. Any idea what could cause results to improve (perhaps other changes in the same release)?
Also, I am attempting other backtests now but they get queued for a very long time. Last one queued for 29 minutes and counting. Could this be due to many people re-submitting their backtests, or is it just a problem on my side?
Thanks !
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Jared Broad
Sorry for the false report -- we discovered a number of issues and had to immediately revert back to the old data and old LEAN code. We're fixing the issues in both the data and the quote modeling and will try again at deploying at the end of the day.
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Dave Mueller
Hey Jared: Just wondering about the status of the redeployment. Is it back up and running or still needing a little more time to fix up? I'm excited to play around with it =)
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Jared Broad
Hey Dave, we have finished the code changes required to maintain backward compatibility but didn't want to deploy it on the weekend. We've now got the data, code, and new tests in place and will re-deploy Monday-AM.
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Dave Mueller
Awesome! I'm going to check it out tonight =)
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Jared Broad
Sorry Dave still delayed; please subscribe to notifications and I'll update this thread as soon as its pushed. We found differences in the ticker symbology and are mapping the tickers back to the current QC format so the data should be nearly identical and not require changes to your code.
It'll likely be 1 day of engineering, 2 days of data processing and then 1 day of retesting. ETA Thursday-Friday.
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Arthur Asenheimer INVESTOR
Take all the time you need. Stability and reliability is more important than a quick implementation.
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Grigori Gustin
Hi Jared/QC team,
I have noticed that there is access to ask/bid prices in research notebooks for second resolution.
However, I could not access to tick data - is there any work still in progress or the problem is in my code?
Also, is ask/bid data is already available during backtesting?
Thanks, Grigori.
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Jared Broad
Hey Grigori! Yes, it was deployed quietly yesterday.
Tick data is also available but it is not the same format as QuoteBars, each Tick has a TickType attached to it. The value of TickType.Quote marks ticks which are bids/asks. These are presented as a list for a specific moment of time.
Our tick data for Equities is now also timestamped to the millisecond! Previously it was rounded down to the previous second
--.
Live Trading does not have the new quote feed yet -- we're working on that and it'll be installed in the next week or two. We're literally waiting on physical delivery of some 10GB fiber modules and with the virus, everything is moving slowly.
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Grigori Gustin
> Tick data is also available but it is not the same format as QuoteBars, each Tick has a TickType attached to it.
Trying to access Resolution.Tick through qb.History() call gives me the following exception:
"Exception : cannot handle a non-unique multi-index!"
Still did not figure out the root cause, the date range seems valid (couple of hours) and it worked before.
Could you please help me on this?
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Jared Broad
No problem Grigori but please create a new thread with a sample attached of what you're trying to do or send in a ticket for private support.
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Hugh Todd
I've been testing the quoteBars in slice with some low volume ETFs I follow and the availability of bid/ask improves the information content gathered from these price series. Thanks for the continuing improvements to the site. When do you see this data being available for live trading? Thanks, Hugh
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Jared Broad
Thanks Hugh;
All going well - we'll do the installation of the new live feed this weekend. We've wired up a brand new rack and are just waiting on a cross-connect cable install for the new data feed. Old live algorithms will need to be stopped and redeployed to move to the new infrastructure.
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Laurent Crouzet
Jared Broad I really think it will be an important and a very useful development for QC and the QC's community that all algorithms (those that are working and the new ones) use the new bid/ask structure!
If I understand well, that would also imply that all currently validated Alphas (those on the marketplace) will be stopped and redeployed, right?
Does it also mean that it would deplete the portfolio (cancellation of all insights) of these Alphas? Or as long as everything goes smoothly during the week-end (as no market is open), the Alphas will still keep their current investments?
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Hugh Todd
Thanks Jared. We appreciate the efforts you all are putting into this.
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Jared Broad
Pleasure Hugh!
Laurent: All QuantConnect backtest algorithms currently use spread, and soon live trading will as well. Once the new live feed is installed Alphas will need to be stopped and redeployed. An Alpha should reinitialize its state in the event of a shutdown, so it should resume their previous positions. When we eventually restart them we'll do it after market close; before universe selection; on Friday. This should minimize the impact of most of the alphas. We'll mark on the charts when the alpha was redeployed with the spread data.
A nice side effect as well -- our new live history server should be 10x faster =) Not quite sure how much faster yet as it's a new approach which should make things close to backtesting speeds.
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Laurent Crouzet
Ok, thank you Jared. Hopefully, everything will be as smooth as possible!
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Davide Carbone
Hi Jared,
do you have an estimated release date for the L1 Equity Quote in live mode?
Thanks
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Cary Cocke
Words cannot express my thanx and praises enough!
OUTSTANDING WORK......this is why I love QC
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Arthur Asenheimer INVESTOR
Now that L1 Equity Quotes are available for backtesting, the next step would be a good market impact model (slippage model).
For example, when a Buy Market Order is submitted, I have to take into account not only the spread, but also the available volume at the next best bid price offer.
So if you want to buy 1,000 shares of XYZ and the current bid price is $ 100.00, but only 100 shares are offered at that price, the order will not be executed in full at $ 100.00, but will reach the next level in the order book and execute the next orders at $ 100.01 and so on.
As a result, the average execution price wil be higher than current backtesting suggests.
If you want to display it 100% realistically, you would need the complete history with the highest granularity (Tick) of the entire order book (Full DOM = Full Depth of Market, i.e. Level 2 Data). You can imagine that this would pose completely new challenges to the hardware performance as the data volumes would be enormous. Therefore it makes sense to work with models that are very close to reality.
Quantopian offers various slippage models for this. I personally prefer to talk about market impact models as this is also common practice in the scientific literature.
As you can see in the documentation here, the Alpha Streams Brokerage Model uses a constant Slippage model.
So my question is: Since we now have L1 Equity Quote Data, I wonder if it wouldn't be possible to create better/more realistic Slippage Models based on that data?
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Arthur Asenheimer INVESTOR
Edit: Edit: A good approach would be the square-root formula for market impact, which we can use now where L1 Quote Data and thus the Spreads are available.
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Derek Melchin
Hi Arthur,
Interesting suggestion. Users will find our documentation on creating custom slippage models useful when implementing this.
To go forward with this, I recommend starting a new forum thread where alternative slippage models can be discussed. Such a discussion is out of the scope of this thread.
Best,
Derek Melchin
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Hugh Todd
Is this available for live trading yet? I'm seeing errors when I try to reference quote bars during live trading with QC data and don't know if it is my model or the state of development.
Thanks, Hugh
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Shile Wen
Hi Hugh,
The L1 data is not yet ready for live trading, and we are working on it. We hope to get it out to the users soon.
Best,
Shile Wen
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Laurent Crouzet
Shile Wen, do you know when the L1 should be ready for live trading?
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Gabriel Gal
Will this update allow us to add equities using CUSIPs? Will there be documentation?
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Shile Wen
Hi Gabriel,
Since CUSIP numbers are proprietary (thread), we won't be adding support for CUSIP. Instead, we use our own security identifiers. To read more about our security identifiers, please see our docs.
Best,
Shile Wen
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Anthony FJ Garner
I hope you will not mind my suggestion that the introduction of the millisecond bid/ask equity data needs far wider and better documentation. I recently bothered Support because I was using minute data and could not for the life of me work out where the extraordinary fills were coming from on an illiquid stock.
I had assumed back testing fills were still being done on minute bars but embarrasingly I was way out of date.
If you don't mind my saying so I would much appreciate a few sample algos as you have provided for many other aspects of the Lean Engine.
Or do such examples already exist? Have I overlooked them?
Also I would appreciate an update on whether L1 B/O data is now being provided for live trading?
Please forgive my ignorance, but if your live account is with IB (who of course themselves provide data) will the live data still nonetheless be provided by Quantquote?
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Shile Wen
Hi Anthony,
L1 data is not yet ready for live trading. As for the data part, the data can either be sourced from the brokerage or QuantConnect (docs). As an additional note, we now use AlgoSeek instead of QuantQuote for equity data.
Best,
Shile Wen
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Charles Naccio
Hey Shile Wen in the meantime, is it possible to supplement L1 data in live trading via a custom data provider? i.e. If we have a Polygon.io subscription, and have access to real-time L1 quote data, can we use that for live trading until QC has native support for it? Thanks in advance!
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Shile Wen
Hi Charles,
We have Polygon support in our Lean engine, however, it's not currently supported on the web QC platform, so you could run Lean locally.
Best,
Shile Wen
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Charles Naccio
Fair enough; thanks for the confirmation!
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Arthur Asenheimer INVESTOR
Hi Shile,
just noticed you've switched to AlgoSeek as your equity data provider. In this respect I've a few questions.
Can you tell us more about the reasons why QC switched to AlgoSeek for equity data ?
Have there been a reconciliation project to scan the differences between both data providers ?
And to what extent does it affect backtest results ?
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Jared Broad
Shipped here:
https://www.quantconnect.com/forum/discussion/9842/shipped-l1-equity-quote-data-instant-universe-selection/p1The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
To unlock posting to the community forums please complete at least 30% of Boot Camp.
You can continue your Boot Camp training progress from the terminal. We hope to see you in the community soon!