Missing data under backtest for an FX currency pair

Here's the revised version of the code with corrections and improvements:

# region imports
from datetime import timedelta
from enum import StrEnum
import pandas as pd
from typing import TYPE_CHECKING, Any
from AlgorithmImports import (
    QCAlgorithm, Resolution, Slice, Market, TimeSpan,
    MovingAverageType, TradeBarConsolidator, SubscriptionManager,
    TradeBar, BollingerBands, BaseDataConsolidator
)
import numpy as np
# endregion

SYMBOL: str = "NZDUSD"
TEST_ORDER_SIZE: int = 100
BB_PERIOD: int = 15
BB_K = 1.5
BB_MA = MovingAverageType.WILDERS
BB_RESOL = Resolution.MINUTE

class Const(StrEnum):
    SYMBOL = "symbol"
    WARMUP = "warmup"
    INDICATOR = "indicator"

class LeanTest00(QCAlgorithm):
    initialized: bool = False

    if TYPE_CHECKING:
        params: dict[str, Any]

    def __init__(self) -> None:
        super().__init__()
        self.params = dict()

    def initialize(self):
        self.set_start_date(2024, 6, 1)  # Set Start Date
        self.set_end_date(2024, 8, 31)  # Set End Date
        self.set_cash(2000)  # Set Strategy Cash

        fx = self.add_forex(SYMBOL, resolution=Resolution.MINUTE,
                            market=Market.OANDA, leverage=50)
        symbol = fx.symbol
        self.log(f"-- symbol {symbol}")

        # Create a 15-minute consolidator
        consolidator = TradeBarConsolidator(timedelta(minutes=15))
        consolidator.data_consolidated += self.on_15m
        self.subscription_manager.add_consolidator(symbol, consolidator)
        self.log(f"-- Added consolidator {consolidator}")

        # Initialize Bollinger Bands indicator
        bb = self.bb(symbol, BB_PERIOD, BB_K, BB_MA, BB_RESOL)
        self.params[Const.INDICATOR] = bb
        self.params[Const.SYMBOL] = symbol

    def on_data(self, data: Slice):
        symbol = self.params[Const.SYMBOL]
        bar: TradeBar = data.bars.get(symbol)
        bb: BollingerBands = self.params[Const.INDICATOR]
        if bar:
            bb.update(bar.end_time, np.mean([bar.high, bar.low, bar.close, bar.close]))
        if bb.is_ready:
            self.plot("BollingerBands", "bb", bb.current.value)
            self.plot("BollingerBands", "standard_deviation", bb.standard_deviation.current.value)
            self.plot("BollingerBands", "middle_band", bb.middle_band.current.value)
            self.plot("BollingerBands", "upper_band", bb.upper_band.current.value)
            self.plot("BollingerBands", "lower_band", bb.lower_band.current.value)
            self.plot("BollingerBands", "band_width", bb.band_width.current.value)
            self.plot("BollingerBands", "percent_b", bb.percent_b.current.value)
            self.plot("BollingerBands", "price", bb.price.current.value)

    def on_15m(self, sender: BaseDataConsolidator, bar: TradeBar):
        # Event handler for M15 consolidator
        if self.is_warming_up:
            self.log(f"Consolidator [warmup] {bar.time}")
            return

        self.log(f"Consolidator {bar.time}")

        bb: BollingerBands = self.params[Const.INDICATOR]
        if bb.is_ready:
            price = np.mean([bar.high, bar.low, bar.close, bar.close])
            symbol = self.params[Const.SYMBOL]
            if price < bb.lower_band.current.value:
                self.sell(symbol, TEST_ORDER_SIZE)
            elif price > bb.upper_band.current.value:
                self.liquidate(symbol)

Key Corrections and Improvements:

1. Consolidator Setup: The TradeBarConsolidator is correctly initialized with a timedelta for 15 minutes, and the event handler on_15m is properly assigned.
2. Bollinger Bands Update: The update method of the BollingerBands indicator is used with the correct parameters: bar.end_time and the average price.
3. Logging and Debugging: Improved logging for better traceability during backtests.

Next Steps:

• Data Availability: Ensure that the forex data for "NZDUSD" is available for the specified date range. If not, adjust the date range to match available data.
• Backtest Locally and in Cloud: Run the backtest both locally and in the QuantConnect cloud to compare results and use logs to diagnose any differences in behavior or data availability.

Sources: Momentum Effect In Commodities Futures, Gradient Boosting Model, Momentum And State Of Market Filters, Indicator Warmup Algorithm, Bybit Crypto Futures Regression Algorithm

I've published a gist with the same main.py code, formatted for the content presentation at GitHub. I'm not certain if the backtest error could be located in the code in itself, though. The error may seem to be in how the backtest is being run?

I'm running the backtest under the local docker image, just like so:

lean backtest --verbose bollinger00

I'm not certain of how the missing data was supposed to be made available for the backtest. Thanks for any help here

When running this with forex data, it seems that I've needed to use a QuoteBarConsolidator. I've updated the code for this and the patches above. I'm still seeing the same error messages.

I believe NZDUSD data should be available for some days in this duration? I'd assumed that the backtest system would just skip any weekends and holidays when trying to draw data. 

From all of the error messages that I'm seeing like the excerpt above, it seems like the backtest is being run with no data?

I've tried running this under the cloud. It ran without error there, but I was unable to debug how it's failing in a functional sense.

I'm trying to debug the functional aspects with the local backtest. except that the local backtest isn't running at all.

I've updated the main.py code to use start and end days on weekdays. I'm still seeing the same series of errors for missing data files in the backtest system, for every single day in the duration of start to end days.

It was my hope to get familiar with quantconnect under the free tier, before any commitment to a subscription. I understand that the OANDA and QuantConnect data providers would need some form of a subscription. 

Could there be some unpublished limits on the data available for testing in the free tier? 

After reviewing the files under <cloud_root>/data/forex/oanda/minute/nzdusd/  I've updated the start, end dates to the following:

• self.set_start_date(2014, 5, 1)  # Set Start Date 
• self.set_start_date(2014, 5, 14)  # Set Start Date 

This much seems to have worked out. The backtest is running, presently, and it's running on into 2015 right now. I'll try to figure out that part, next. 

Thanks for the quick advice!

lol I'd used two set_start_date and no set_end_date. 

The backtest is running pretty quickly though. ngl this is wildly faster than under metatrader backtesting