Share New Research 
Hi, It appears that there is a significant issue with the stock filtering backtests in QC. I have noticed that the algorithm used to filter stocks produced different results when run last month compared to when it was run today. (I copied it from the Log which was ran last month.)
2024-07-02 00:00:00 :['CONN', 'NCSM', 'MESA', 'IRBT', 'SDIG', 'GOCO', 'HOOK', 'NUTX', 'DTIL', 'BMTX']
But when I ran the same Algorithm today, it gave me a different result.
2024-07-02 00:00:00 :['CELU', 'ALGS', 'LPSN', 'PRPL', 'MESA', 'OPI', 'SDIG', 'IRBT', 'GOCO', 'HOOK']
Especially considering that certain stocks, such as ‘CONN’ has gone bankrupt and is to be delisted, price going to 0. But the same Algorithm ran today didn't reflect this, and just ignores ‘CONN’. It's not just for this month.
▽Log from backtest ran last month.
2024-02-02 00:00:00 :['VBIV', 'PLBY', 'CONN', 'VERI', 'SDIG', 'GOCO', 'EHTH', 'TSHA', 'STRO', 'SGMA']2024-03-02 00:00:00 :['PLBY', 'BBGI', 'CONN', 'SDIG', 'VERI', 'GOCO', 'STHO', 'GIFI', 'STRO', 'SGMA']
▽Log from backtest ran today.
2024-02-02 00:00:00 :['CELU', 'PLBY', 'VERI', 'SDIG', 'GOCO', 'TSHA', 'EHTH', 'STRO', 'SGMA', 'GIFI']2024-03-02 00:00:00 :['PLBY', 'BBGI', 'CONN', 'SDIG', 'VERI', 'GOCO', 'STHO', 'GIFI', 'STRO', 'SGMA']Unless the engine/data has lookahead bias the same code should be returning the same result.
I am using filters such as
if (x.HasFundamentalData)
and x.Market == 'usa'
and x.Price > 0.1
and x.DollarVolume>1000
if x.MarketCap > 0
and (x.CompanyReference.IndustryTemplateCode!="B")
and (x.CompanyReference.IndustryTemplateCode!="I")and some other financial filters too. Looking at a similar feed from 2022, I guess datas related to company's basic info is the problem. It is clear that they are somehow bypassing the lookahead bias prevention methods, probably because these datas are using data of today.
This problem makes all the research in equity factor analysis untrustable, which is very disappointing.
Survivorship Bias issues in universe selection (equities) by Tom M - QuantConnect.com
Mia Alissi
Hi Lance,
The issue you're experiencing with the stock filtering backtests in QuantConnect could be related to survivorship bias and the handling of fundamental data. When stocks go bankrupt or are delisted, they may not appear in historical data, leading to discrepancies in backtest results. This is a common challenge when dealing with historical stock data, as the data might not accurately reflect the universe of stocks available at a past date.
QuantConnect's data might include survivorship bias, where only currently active stocks are considered, ignoring those that have been delisted or gone bankrupt. This can lead to different results when running the same algorithm at different times, as the universe of stocks changes.
To address this, you might want to ensure that your data source includes delisted stocks or use a data provider that accounts for survivorship bias. Additionally, reviewing your algorithm's logic to ensure it handles such cases explicitly can help mitigate these discrepancies.
If your algorithm relies heavily on fundamental data, ensure that the data is consistent and reflects the historical context accurately. This might involve using a more robust data source or adjusting your algorithm to account for potential biases in the data.
I hope this helps clarify the issue you're facing.
Sources: Survivorship Bias issues in universe selection (equities) by Tom M - QuantConnect.com, Stock Selection Based on Fundamental Factors Issues by Tom Penrose - QuantConnect.com, Momentum in Mutual Fund Returns by Derek Melchin - QuantConnect.com, Tickers Missing from Paper Trading Data Source on QC by Bolt.Investments - QuantConnect.com, Different Backtest Results When Adding Securities to Universe by AK M - QuantConnect.com, Survivorship Bias in Fundamental Data by Anderson Lidz - QuantConnect.com, Problems Creating Universe: How Much Data Can I Handle by Jack Bueller - QuantConnect.com, 5-10-20 Year Performance in Universe by Rakesh Gopal - QuantConnect.com
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Lance je
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
To unlock posting to the community forums please complete at least 30% of Boot Camp.
You can continue your Boot Camp training progress from the terminal. We hope to see you in the community soon!