Share New Research 
Hello All!
After a mammoth 11 months of engineering, we've finished the deployment of L1 data to production. It has been in backtesting for approximately 7 months, but we encountered some pretty difficult engineering challenges with getting this through to live-trading for the community at this scale.
L1 equity data is about 10x the trade-only live feed; multiplied by the sheer number of community strategies we hit bandwidth and physical limitations of hardware we were operating. This required new racks of equipment and physically upgrading switches and routers to handle the new traffic. We had some brand new engineering leaps on the distribution of the data. Fun parts of the project included:
- Upgrading the live racks to 10G fiber internet connections.
- Hitting the limits of RAM bandwidth and having to optimize the code to shave off the milliseconds.
- Tracking and setting the perfect exchange opening-auction market ticks to all our aggregated bars.
This upgrade brings:
- Instant universe selection on deploying your algorithm. You no longer need to wait for the next day to do a universe selection. This is a change of behavior!
- Spread in live trading. New alphas and paper trading fill will be filled on the quote spread just in backtesting.
- New live servers; more modern CPUs, RAM, and more efficient live code giving meaning you get more of your allocated node capacity.
- New crypto 6+ character ticker changes. Crypto users will need to swap a few tickers in their backtesting and live trading, effective immediately. (ALO -> ALGO, NIO -> ANIO, BSV -> BCHSV, DSH -> DASH, IOT -> IOTA, LIK -> LINK, LOM -> LOOM, MNA -> MANA, GOT -> PKGO, QTM -> QTUM, UST -> USDT, YYW -> YOYOW)
One thing to note -- previously even if you subscribed to minute data, your algorithm charts would update every second. We had to drop this functionality for efficiency but now your chart will update at least every minute even with daily data for your algorithms. We're fairly confident the technology is stable as it has been deployed for more than 2 months to hundreds of alphas working out edge cases and issues. If you have issues please let us know via a support ticket. We're watching this very closely over the next few days.
Best
Team @ QuantConnect
Tristan F
Great work QC team! We appreciate your continued commitment and support to the open-source quantitative investment community.
Laurent Crouzet
Great news! That is a major milestone for live results of Alpha Streams!
Emilio Freire
Fantastic news! Big THANK YOU to the team for all the hard work!
Flame
Amazing news! Thank you very much team! Does this mean we will be able to get live CBOE index prices?
Spacetime
It is a remarkable achievement indeed.
I appreciate the team's time and efforts given to this project.
Jared Broad
Flame - yes give us the rest of the week for testing, but by early next week, we should be able to support all alternative data sources in live trading.
Thanks all for your support and patience =)
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Charles Naccio
Another reminder of why you don't want to build your own stack :-)
Hector Barrio
Nice improvements, the instant universe selection brings and improvement in the quality of life. Is it a possibility that the changes have knocked out the universes in live trading though? I was expecting a new universe on friday and nothing came through, I have restarted the algorithm right now, middle of trading day, and it has resolved the universe inmediately.
Could the contents of the active universe be added in the bottom panel of live trading (where holdings, orders, insights...)? Similar to the holdings and all holdings panels. It could save some logging effort.
Erol Aspromatis
Thanks for making these investments in data and hardware to enable us to build better algorithms and compete more effectively as individual traders.
Jared Broad
Hector Barrio re: Universe on Charts. Nice idea, we're pretty swamped for the next couple of weeks with optimization but we should have some bandwidth in December to sketch it up or at least address the underlying question of "universe constituents".
The universes are quite isolated from this technology and it was only deployed/touched Monday; if you have other issues please submit a support ticket for that live deployment we can look into it there. Keep in mind Friday's universe arrives Saturday morning.
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Ritaelyn
Great work! Does the L1 data include ETF quotes or is it just stocks for the time being?
If it doesn't include L1 for ETFs, are there any plans to add such data? If you do have ETF L1 data, would you also be able to add the iNAV value (the Indicative Value that is updated every 15 seconds and diseminated of an ETF's current NAV.) I would really look forward to this data next. :)
Derek Melchin
Hi Ritaelyn,
Yes, L1 data includes ETFs.
We've created a GitHub Issue to have the INAV data source added. Track our progress here.
Best,
Derek Melchin
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Ritaelyn
Sweet! Thanks Derek!
Jared Broad
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
To unlock posting to the community forums please complete at least 30% of Boot Camp.
You can continue your Boot Camp training progress from the terminal. We hope to see you in the community soon!