Simulation price is different with price on Google and Yahoo finance

Loading [MathJax]/jax/output/HTML-CSS/config.js

QuantConnect Community Discussions

QUANTCONNECT COMMUNITY

Join Our Discord Channel

Join QuantConnect's Discord server for real-time support, where a vibrant community of traders and developers awaits to help you with any of your QuantConnect needs.

Draft Discussions

Bookmarked Discussions

Simulation price is different with price on Google and Yahoo finance

Last comment by - April 2017

Simulation price is different with price on Google and Yahoo finance

Price discrepancies between simulation and Google/Yahoo finance caused trade to occur at lower price.

Share New Research

Start New Discussion Sign up

SEARCH DISCUSSIONS

361,700 Quants.

Become a Quant

VOTE FOR UPCOMING FEATURES

Share your input and vote on our future direction.

LEAN Roadmap

JOIN OUR Community MAILING LIST

Create an account on QuantConnect for the latest community delivered to your inbox.

Simulation price is different with price on Google and Yahoo finance

Qingpeng Niu | January 2016

2010-07-23 TLT price is day high 100.97 and day low 99.67 on Google or Yahoo.

How can system give 90.45 price to trade in?

2010-09-30 09:31:00 TLT $94.55092451 Market -2,403 Short Filled

2010-07-23 09:31:00 TLT $90.4501584 Market 2,403 Long Filled

Author

Qingpeng Niu

January 2016

Upvote

Jared Broad INVESTOR

QuantConnect | January 2016

Welcome @Qingpeng, we use split and dividend adjusted data by default. Yahoo and Google apply the dividends separately. You can control this with the SetDataNormalization() controls -- see more here: https://www.quantconnect.com/forum/discussion/comment/2382

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.

Upvote

Qingpeng Niu INVESTOR

January 2016

Thanks Jared! If i want the price is the same with the one at Google or Yahoo, i should use which enum in DataNormalizationMode? DataNormalizationMode.TotalReturn? Where can i find more documentation about this API? I am new to this platform. If i use the same price with Google or Yahoo, the dividend is still paid in simulation or not? AddSecurity(SecurityType.Equity, symbol, Resolution.Minute); Securities[symbol].SetDataNormalizationMode(DataNormalizationMode.TotalReturn);

Upvote

Qingpeng Niu INVESTOR

January 2016

Thanks Jared. I find you post some times ago and it is very helpful. https://www.quantconnect.com/forum/discussion/508/update-dividends-splits-and-custom-price-normalization

Upvote

Levitikon INVESTOR

January 2016

I'm also seeing a difference between the closing prices I'm getting while backtesting vs the data I pulled from Tradier API & Google API. I've tried all the normalization modes and they don't seem to have an affect on the closing price. Also tried switching between FillDataForward with no affect. It's usually very small differences, about 1-4/100th difference but it's enough to throw off the outcome of an algorithm compared to my local environment. For example: TVIX 12/3/2015 Time......BackTest | Tradier (5 Minute) 1:51pm | 6.4100 | 1:52pm | 6.4200 | 1:53pm | 6.4000 | 1:54pm | 6.4000 | 1:55pm | 6.4100 | 6.44 Most of the time it's differences in rounding, such as 6.11 vs 6.10, but on occasion they get a few digits off like above. Any thoughts?

Upvote

Jared Broad INVESTOR

QuantConnect | January 2016

Sorry its impossible to say - we don't know how Google or Tradier process their data, or where it comes from. Some public facing websites only display BATS exchange data which is much cheaper but is only 1/10th the volume of the primary exchanges. For backtesting we build our bars from ticks, taken from all the exchanges. We apply a filter on those ticks for the bars. Advanced users who request ticks can do their own filtering.


        public override bool PassesFilter(Tick tick)
        {
            // filter out bad ticks for equities
            if (tick.Suspicious) return false;
            if (tick.SaleCondition == "W") return false;
            switch (tick.Exchange)
            {
                case "W": //Chicago Options
                    return false;
                case "u": //OTC
                    return false;
                case "U": //OTC Bulletin
                    return false;
                case "D": //FINRA ADF
                    return false;
            }
            return true;
        }

In live trading our live data source has the same high fidelity tick stream, and we build those bars on the fly.

Upvote

Levitikon INVESTOR

January 2016

Interesting.. I tried using the TickConsolidator->5min tradebars and applying your filtering. I get the exact sames results as before, which is good, I reached parity. I then varied combinations of exchanges where I started to see the prices start to change a little. I wasn't able to find the right combination to yield the results I'm getting from Tradier so i'm working with them to see if I can find out how they're building their data. Hopefully I can find the right combo. It's very important I get this. I've local scripts that spend hours calibrating combinations of configurations for each security that I plug into QC algo. When my local data differs from QC data, it's just a fail. In the meantime, could any suggest an API I could use that would yield the EXACT same results I get while back testing in QC? And I mean always precisely the exact same down to the 1/10000 of a cent. I know QuantQuote is recommended, but they took a week+ to get purchased data dump last year and I'm not that patient. I'd rather have access to an API where I can get what I want when I want and feed it into a local db. Any suggestions? Paid APIs are cool so long as it's reasonably priced.

Upvote

Levitikon INVESTOR

January 2016

@Jared, thanks for the insight into how you build your data. That was very very helpful. I ended up abandoning Tradier API in favor of ActiveTick (AT) API which already seems to be much more inline with QC data. After closely comparing AT's tick data against QC's minute data, I've concluded you are indeed filtering your tick data as described in your filtering method. So I have two options to synchronize my local db with QC. 1. Rebuild my local db for 1 minute resolution from AT's consolidated tick data follow your filtering rules (probably have two fields for close, Close and QCClose so I could use either) 2. Or rebuild my local db with AT's provided 1 minute resolution & use the TickConsolidator in my QC Algo and use data from all exchanges (including W u U D) because it seems AT doesn't do any filtering. Both will yield me identical results local vs backtest. But I really have to wonder. Why are you filtering out those exchanges? Is there a good reason not to include ticks from them? It seems D was the last Tick of a minute more often than others, so I just don't know which strategy is better.

Upvote

Jared Broad INVESTOR

QuantConnect | January 2016

Hmm tricky questions @Levi. More fundamentally rather than what data to use; I think it comes down to what you can execute on and I'm not sure its practical to build a strategy which is sensitive to tick by tick variations without having co-located infrastructure and a brokerage which will let you execute on a tick by tick basis. These are available (Equinox Colo + Lime Brokerage) but will cost you roughly $1500/mo for equinox + $3000/mo for lime. Our data-feed is tick by tick, but the execution is through your brokerage (IB/Tradier/Oanda/FXCM) -- so if your brokerage is slow or a market maker there is no guarantee you'll fill at the quote price. Tradier for example patches their trades through to Knight which has to match the NBBO (national best bid or offer) more than 90% of the time. This has very little to do with the trades/quotes you'll receive from any datafeed. What are your holding periods on average? Is it ballpark 1 second or 1 minute? Typically we say QC.com is suitable for holdings periods down to 20-30 seconds on volatile tickers. If you write your own brokerage connections and self-host LEAN in NY4/Equinox you could probably get it down to milliseconds. We filter out those ticks as the majority of the time they have bad or delayed ticks, which cause spikes and are unrealistic price you can never fill at. The majority of "TradeBar" users don't care about those ticks but care about prices they can achieve. We provide the tick resolution data stream if you want to do something more advanced.

Upvote

Sai P INVESTOR

April 2017

Hello,

I am new here and was testing QC backtest data and have noticed the following. I will not post long list of prices here but point out couple.

TSLA

Trade Date: 03/28/2017

5 Minute TradeBar: Time 8:55 has a High of 278.35 on IB, TDAmeritrade etc.

When the Same Bar, consolidated via code using Tick Resolution on Security and using TickConsolidator, to form 5 MinuteBar in QC

I am getting a price of 278.38.

I have noticed so many of these differences to loose my confidence at this point.

I have changed the resolution from minutes, seconds to Tick. This helped a little bit to more accurate.

How do we handle this in live? Use broker feed fo back testing to be more accurate? Please suggest for best way to move forward.

Upvote

1 person upvoted this

Sai P INVESTOR

April 2017

Time 9:55 EST sorry. I am in CST.

Upvote

Jared Broad INVESTOR

QuantConnect | April 2017

Sai; please read the thread above. Its laid out clearly there.

Ultimately we build our data from ticks which is much more accurate than Yahoo or Google's daily sources. In live trading you can't filter out ticks after the fact - you don't know what will be marked as invalid.

Ticks arrive late, or are invalid prices far from market price. When you request tick data we expect you to understand that. If you use second or minute bars we pre-filter the common causes of price error as shown in above discussion.

Also note the terms and conditions prohibit logging price data due to data vendor restrictions. If you'd like to download the raw data you can purchase it from QuantQuote.com

Upvote

Author:

January 2016

Platform

Radically Open-Source Algorithmic Trading Engine

Join Our Discord Channel

Quarterly Open-Source Trading Competition

Draft Discussions

Bookmarked Discussions

SEARCH DISCUSSIONS

361,700 Quants.

VOTE FOR UPCOMING FEATURES

JOIN OUR Community MAILING LIST

IN THIS RESEARCH

PARTICIPANTS

Actions

Join QuantConnect for Free

Platform

SIGN IN

Radically Open-Source Algorithmic Trading Engine

Join Our Discord Channel

Quarterly Open-Source Trading Competition

Draft Discussions

Bookmarked Discussions

SEARCH DISCUSSIONS

361,700 Quants.

VOTE FOR UPCOMING FEATURES

JOIN OUR Community MAILING LIST

IN THIS RESEARCH

PARTICIPANTS

SHARE RESEARCH

SHARE DISCUSSION

SHARE ARTICLE

SHARE

Actions

Join QuantConnect for Free