Debug logs not visible in live section

Based on the code you provided, it seems like you are trying to log the first half-hour return at 10:05 NY time. However, it's important to note that the self.Schedule.On method schedules an event to fire at a specific time of day. In your case, it's set to fire at 10:05.

The BuyOnOpen method is called at this time, and it checks if the rolling window is ready. If it is, it calculates the return and logs it. If the rolling window is not ready, it simply returns and does nothing.

If you are not seeing any logs at 10:05, it could be because the rolling window is not ready at that time. This could happen if there are not enough data points to fill the window.

You can add a debug statement to check if the rolling window is ready at the time of the scheduled event:

def BuyOnOpen(self):
    if not self.rollingWindow.IsReady:
        self.Debug("Rolling window is not ready")
        return
    self.first_half_hour_return = self.CalculateReturn(self.rollingWindow)
    time = self.rollingWindow[0].Time
    timePrev = self.rollingWindow[1].EndTime
    self.Debug(f"First Half-Hour Return: {self.first_half_hour_return}")

This will log a message if the rolling window is not ready, which can help you diagnose the issue.

Also, ensure that your algorithm is running during the market hours of the asset you are trading. If the market is not open at 10:05, the scheduled event might not fire as expected.

If you're still having issues, please provide more details about your algorithm, such as the asset you are trading and the specific dates and times you are running the algorithm.

Oh it seems, that in backtesting I get the first debug the day after the deployement. So I guess it will be the same in the live section and I will get the logs as of tomorrow since I deployed today, but do you know why it takes a day to be ready?

Hi Mia, thanks for the reply. It should be open, I am trading QQQ, so it should 35 minutes after the opening, right?
It seems the behavior is consistent with backtesting (if tomorrow I get the log), so I guess the windows was not ready. But is it expected?

The delay you're experiencing might be due to the warm-up period required for your rolling window to be ready. In your code, you've defined a rolling window of size 2. This means that it needs two data points before it's considered "ready".

In backtesting, events are triggered as fast as possible, whereas in live trading, events are triggered in real-time. Therefore, if you're using minute resolution data, it would take two minutes after the start of your algorithm for the rolling window to be ready.

However, if your algorithm starts after the market opens, it might not receive data immediately. For example, if your algorithm starts at 10:00 but the market opens at 09:30, the first data point your algorithm receives would be for the 10:00 bar, not the 09:30 bar. This means that your rolling window wouldn't be ready until 10:01 at the earliest.

In live trading, the algorithm portfolio is loaded from the brokerage holdings, including open orders. Your algorithm should not assume it starts from an empty portfolio as you may have holdings from previous deployments. There is no automatic state management of live strategies in QuantConnect. We recommend algorithms reconstruct state or indicator needs using the WarmUp and History methods. Once the algorithm has warmed up some objects you can use the Object Store to save the data for the next algorithm restart.

If you need your rolling window to be ready immediately at the start of your algorithm, you could use the self.History method to pre-fill your rolling window with historical data. This would allow your rolling window to be ready as soon as your algorithm starts.

Here's an example of how you might do this:

history = self.History(self.ticker, 2, Resolution.Minute)
for bar in history.itertuples():
    self.rollingWindow.Add(bar)

This code retrieves the last two minutes of historical data for your ticker and adds it to your rolling window. Now, your rolling window should be ready immediately at the start of your algorithm.

Please note that there is a delay in when new live data is available for backtesting. It's normally available after 24-48 hours. If you need to closely monitor new data, use live paper trading.

Sources:

• QuantConnect Documentation: Key Concepts
• QuantConnect Documentation: Misconceptions
• QuantConnect Forum Discussion: Guide to general tips to improve backtest speeds
• QuantConnect Forum Discussion: When should I launch my live-trading algorithm?

Yes, you're correct. If you're trading QQQ, the market opens at 9:30 AM ET, so 10:05 AM ET would be 35 minutes after the market opens.

As for the rolling window not being ready, it's indeed expected if it doesn't have enough data points yet. In your case, you've defined a rolling window of size 2, which means it needs two data points before it's considered "ready".

If you're using minute resolution data, it would take two minutes after the start of your algorithm for the rolling window to be ready. However, if your algorithm starts after the market opens, it might not receive data immediately.

For example, if your algorithm starts at 10:00 but the market opens at 09:30, the first data point your algorithm receives would be for the 10:00 bar, not the 09:30 bar. This means that your rolling window wouldn't be ready until 10:01 at the earliest.

If you need your rolling window to be ready immediately at the start of your algorithm, you could use the self.History method to pre-fill your rolling window with historical data. This would allow your rolling window to be ready as soon as your algorithm starts.

Here's an example of how you might do this:

history = self.History(self.ticker, 2, Resolution.Minute)
for bar in history.itertuples():
    self.rollingWindow.Add(bar)

This code retrieves the last two minutes of historical data for your ticker and adds it to your rolling window. Now, your rolling window should be ready immediately at the start of your algorithm.

Please note that there is a delay in when new live data is available for backtesting. It's normally available after 24-48 hours. If you need to closely monitor new data, use live paper trading.

thanks a lot for your quick reply and clarifications, I deployed the algo few hours before the market opened and I am currently using IBKR paper account. So it should be enough time for the windows to warm up? Anyway I will check tomorrow if I get the log (at least it will be consistent with backtesting) and let you know if I have more doubts :)

You're welcome, Francesco! Yes, if you deployed the algorithm a few hours before the market opened, it should have been enough time for the rolling window to warm up, assuming that the data was available.

Remember, in live trading, the algorithm portfolio is loaded from the brokerage holdings, including open orders. Your algorithm should not assume it starts from an empty portfolio as you may have holdings from previous deployments.

If you're using an Interactive Brokers paper account, you should be aware that there might be a delay in when new live data is available for backtesting. It's normally available after 24-48 hours. If you need to closely monitor new data, use live paper trading.

Please do check tomorrow and let us know if you have more doubts. We're here to help!

Sources:

• QuantConnect Documentation: Key Concepts
• QuantConnect Documentation: Misconceptions
• QuantConnect Forum Discussion: Guide to general tips to improve backtest speeds
• QuantConnect Forum Discussion: When should I launch my live-trading algorithm?

Hi Mia, thank you. I will let you know. Regarding the live account, I understand that I need to be careful for existing open positions (I would say I need to close any position on QQQ if I am deploying an algo trading that security).

Regarding the cash in my IBKR live account, as I understood the default cash considered by the algo is USD, right?

So I assuming I have 100k euro and 50k USD in my IBKR account as cash, and I set up the algo as setholdings("QQQ", 1). The algo will invest the 50k USD in my account in QQQ and disregard the euros, is that correct?

Yes, you're correct. By default, QuantConnect's LEAN engine considers the base currency to be USD. When you use the SetHoldings method, it uses the available cash in the base currency to make the purchase.

So in your case, if you have 100k euro and 50k USD in your IBKR account, and you call SetHoldings("QQQ", 1), the algorithm will invest the 50k USD in QQQ and disregard the euros.

However, it's important to note that LEAN is designed to be a multi-currency platform. It can trade Forex, Cryptocurrencies, and other assets that are quoted in other currencies. The portfolio manages your currencies in its cashbook, which models the cash as a ledger of transactions. When you buy assets, LEAN uses the currencies in your cashbook to purchase the asset and pay the transaction fees.

If you want to trade assets quoted in euros or other currencies, you can add those currencies to your cashbook. For more information about the cashbook, see Cashbook.

Sources:

• QuantConnect Documentation: Multi-Asset Modeling
• QuantConnect Forum Discussion: Hacking LEAN to use EUR instead of USD with IBKR

Hi Mia, the day after it worked like for backtesting, I think the reason is that I am defining a rolling window of 2 bars of 30 minutes :

self.ticker = self.AddEquity(self.asset, Resolution.Minute).Symbol

        self.rollingWindow = RollingWindow[TradeBar](2)

        self.Consolidate(self.ticker, timedelta(minutes=30), self.CustomBarHandler)

and since the event starts 35 minutes after the market open I would have only one 30 minutes bar so the rolling window won't be ready. That's why I need to wait 1 more day.

I could solve it by adding the history as you suggested by your history code example will add 2 bars of 1 minute right? 
 

But the rolling window should contains the last 2 bars of 30 minutes, am I right? can I add 30 minutes bars to warm up the code?

Hey Francesco,

You have the option to utilize the WarmUp feature provided by QuantConnect to initialize and warm up the rolling window, ditching using the History method.

I have attached a sample backtest which might give you a starter to build things on.

Output:
Date: 2024-01-02 10:05:00 | First Half-Hour Return: 0.0009107468123863427
Date: 2024-01-03 10:05:00 | First Half-Hour Return: -0.00018926080138426116
Date: 2024-01-04 10:05:00 | First Half-Hour Return: -0.00030273007485692727
Date: 2024-01-05 10:05:00 | First Half-Hour Return: 0.0008780113043955762