Share New Research 
The history request in question is a minute-resolution 1-bar request for QQQ on 2023/2/15. A few puzzling observations:
- If I request 1 bar ~a minute after open on 2/15, I get no results. But if I request 2 bars, I do get a list of 2 bars (the last bar of 2/14 and the first bar of 2/15). [see backtest 1 Logs]
- For 2/16, the request for 2 bars only gets the first bar of 2/16, which is inconsistent with what I would've expected seeing the 2-bar-request-results from 2/15. [see backtest 1 Logs]
- If I schedule an after-market-open event and start the backtest on 2/15, the 2/15 event executes at 10am. But if I start the backtest on 2/14, the 2/15 event executes at 9:30am. [see backtests 1&2 Logs]
Would anyone be able to shed some light on this? Thank you!
Backtest 1 Log:
2023-02-15 00:00:00 : Launching analysis for 85d18005dd227a8f543e0e97416eb766 with LEAN Engine v2.5.0.0.16001
2023-02-15 10:00:00 : Requesting 1 bar:
2023-02-15 10:00:00 : []
2023-02-15 10:00:00 : Requesting 2 bars:
2023-02-15 10:00:00 : [<QuantConnect.Data.Market.TradeBar object at 0x7f6b4fb96c40>, <QuantConnect.Data.Market.TradeBar object at 0x7f6b4fb96c80>]
2023-02-15 10:00:00 : list comprehension on 2 bars:
2023-02-15 10:00:00 : [datetime.datetime(2023, 2, 14, 15, 59), datetime.datetime(2023, 2, 15, 9, 30)]
2023-02-16 09:32:00 : Requesting 1 bar:
2023-02-16 09:32:00 : [<QuantConnect.Data.Market.TradeBar object at 0x7f6b4f9e1a00>]
2023-02-16 09:32:00 : Requesting 2 bars:
2023-02-16 09:32:00 : [<QuantConnect.Data.Market.TradeBar object at 0x7f6b4f9e1a40>]
2023-02-16 09:32:00 : list comprehension on 2 bars:
2023-02-16 09:32:00 : [datetime.datetime(2023, 2, 16, 9, 30)]
2023-02-16 16:00:00 : Algorithm Id:(85d18005dd227a8f543e0e97416eb766) completed in 0.44 seconds at 2k data points per second. Processing total of 806 data points.Backtest 2 Log:
2023-02-14 00:00:00 : Launching analysis for 8a26700719c997759af0743b30ea6893 with LEAN Engine v2.5.0.0.16001
2023-02-14 09:32:00 : Requesting 1 bar:
2023-02-14 09:32:00 : [<QuantConnect.Data.Market.TradeBar object at 0x7f23bc581240>]
2023-02-14 09:32:00 : Requesting 2 bars:
2023-02-14 09:32:00 : [<QuantConnect.Data.Market.TradeBar object at 0x7f23bbe5efc0>, <QuantConnect.Data.Market.TradeBar object at 0x7f23bbe60040>]
2023-02-14 09:32:00 : list comprehension on 2 bars:
2023-02-14 09:32:00 : [datetime.datetime(2023, 2, 13, 15, 59), datetime.datetime(2023, 2, 14, 9, 30)]
2023-02-15 09:32:00 : Requesting 1 bar:
2023-02-15 09:32:00 : []
2023-02-15 09:32:00 : Requesting 2 bars:
2023-02-15 09:32:00 : [<QuantConnect.Data.Market.TradeBar object at 0x7f24230ec1c0>, <QuantConnect.Data.Market.TradeBar object at 0x7f24230f5bc0>]
2023-02-15 09:32:00 : list comprehension on 2 bars:
2023-02-15 09:32:00 : [datetime.datetime(2023, 2, 14, 15, 59), datetime.datetime(2023, 2, 15, 9, 30)]
2023-02-16 09:32:00 : Requesting 1 bar:
2023-02-16 09:32:00 : [<QuantConnect.Data.Market.TradeBar object at 0x7f24230cf100>]
2023-02-16 09:32:00 : Requesting 2 bars:
2023-02-16 09:32:00 : [<QuantConnect.Data.Market.TradeBar object at 0x7f24230cf2c0>]
2023-02-16 09:32:00 : list comprehension on 2 bars:
2023-02-16 09:32:00 : [datetime.datetime(2023, 2, 16, 9, 30)]
2023-02-16 16:00:00 : Algorithm Id:(8a26700719c997759af0743b30ea6893) completed in 0.45 seconds at 5k data points per second. Processing total of 2,376 data points.Code:
# region imports
from AlgorithmImports import *
# endregion
class VirtualGreenCoyote(QCAlgorithm):
def Initialize(self):
# Backtest 1
self.SetStartDate(2023, 2, 15)
# Backtest 2
# self.SetStartDate(2023, 2, 14)
self.SetEndDate(2023, 2, 16)
self.qqq = self.AddEquity('QQQ', Resolution.Minute)
self.Schedule.On(self.DateRules.EveryDay(self.qqq.Symbol), self.TimeRules.AfterMarketOpen(self.qqq.Symbol, 1.01), self.get_hist)
def OnData(self, data: Slice):
pass
def get_hist(self):
self.Log('Requesting 1 bar: ')
self.Log(list(self.History[TradeBar](self.qqq.Symbol, 1, Resolution.Minute, extendedMarketHours = False)))
self.Log('Requesting 2 bars: ')
l2 = list(self.History[TradeBar](self.qqq.Symbol, 2, Resolution.Minute, extendedMarketHours = False))
self.Log(l2)
self.Log('list comprehension on 2 bars: ')
self.Log([b.Time for b in l2])
Mia Alissi
Based on the information provided in the extracts, it seems like there might be some confusion about how the
Historymethod works in QuantConnect.The
Historymethod in QuantConnect is used to retrieve historical data. The number of bars returned by theHistorymethod depends on the resolution and the trading hours of the asset. For instance, if you request 1000 second bars, it will go back 1000 seconds from the current time. But that does not guarantee you will get 1000 second bars. It will only be the case if the asset traded at least once every second over that 1000 seconds period.In your case, when you request 1 bar a minute after the market opens, it's possible that the bar for that minute hasn't been completed yet, hence no results are returned. When you request 2 bars, you get the last bar of the previous day and the first bar of the current day.
As for the issue with the scheduled event, the
Schedule.Onmethod in QuantConnect schedules an event to fire at a specific time of day. The time the event is triggered depends on the start date of the backtest. If you start the backtest on the same day as the scheduled event, the event might be triggered later due to the time it takes to initialize the algorithm and load the necessary data.Please note that these are just possible explanations based on the information provided in the extracts. For a more accurate diagnosis of the issue, you might want to post this question on the QuantConnect Forum where the community and the QuantConnect team can provide more specific help.
Here are some relevant links from the extracts for further reading:
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Robert Lee
First of all, thank you for the comment. The links you provided were extremely helpful.
I have still yet to convince myself that I fully understand the issue--some thoughts below:
Number of bars/timing
Scheduled Events
Again, thank you for your help, I would love to hear more of your/everyone else's thoughts!
Mia Alissi
Based on the information provided in the extracts, it seems like there might be some specific behaviors of QuantConnect's engine that could be causing the issues you're observing.
Number of bars/timing
The
Historymethod retrieves historical data based on the time, not the number of bars. This means that if you request 1 bar a minute after the market opens, it's possible that the bar for that minute hasn't been completed yet, hence no results are returned. When you request 2 bars, you get the last bar of the previous day and the first bar of the current day. This behavior is consistent with the information provided in the forum discussion you mentioned.The fact that the 1-bar request and the 2-bar requests were executed at the exact same time but returned different results could be due to the way the
Historymethod works. It's possible that the 2-bar request was able to retrieve the first bar of the current day because it also requested the last bar of the previous day, which might have triggered the data update for the current day.The anomaly on 2/15 could be due to a variety of factors, such as data availability, data updates, or the specific timing of the requests. Without more information, it's hard to pinpoint the exact cause.
Scheduled Events
In live trading, Scheduled Events execute in a parallel thread based on a real-time clock. If you set a Scheduled Event to fire at 10:00 AM, it executes at exactly 10:00 AM. In backtesting, Scheduled Events are part of the main algorithm manager loop, so they may not execute exactly when you set them. This is mentioned in the documentation.
The fact that the market open event was executed at 10:00 sharp could be due to the way the backtesting engine works. It's possible that the engine was busy processing other tasks and didn't get to the scheduled event until 10:00.
The issue being specific to 2/15 could again be due to a variety of factors, such as the specific timing of the event, the state of the backtesting engine at that time, or other factors specific to that day.
Again, these are just possible explanations based on the information provided in the extracts. For a more accurate diagnosis of the issue, you might want to post this question on the QuantConnect Forum where the community and the QuantConnect team can provide more specific help.
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Robert Lee
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
To unlock posting to the community forums please complete at least 30% of Boot Camp.
You can continue your Boot Camp training progress from the terminal. We hope to see you in the community soon!