Read in local forex data

Hello all,

I am trying to read in local forex data from .csv files. The files are located under ./Data/own. To do this, I have created a class in Python that implements the GerSource() and Reader() methods of the PythonData base class. After processing the method Initialize() in main, the following error message appears.

main.py

from AlgorithmImports import *
from dataReader import MyDataReader
class RetrospectiveOrangeBat(QCAlgorithm):
    def Initialize(self):
        self.SetStartDate(2013, 10, 7)  # Set Start Date
        self.SetEndDate(2013, 10, 11)  # Set End Date
        self.SetCash(100000)  # Set Strategy Cash
        self.pair = self.AddData(MyDataReader, 'EURUSD', Resolution.Daily, leverage=20).Symbol
    def OnData(self, data: Slice):
        if not self.Portfolio.Invested:
            self.MarketOrder(self.pair, 10000)

datareader.py

from AlgorithmImports import *
from pathlib import Path
from datetime import datetime, timedelta
class MyDataReader(PythonData):
    def GetSource(self, config : SubscriptionDataConfig, date : datetime, isLiveMode : bool) -> SubscriptionDataSource:
        fileName = '2013.csv'
        path2file = Path(Globals.DataFolder, 'own', fileName)
        return SubscriptionDataSource(path2file.as_posix(), SubscriptionTransportMedium.LocalFile)
    def Reader(self, config : SubscriptionDataConfig, line : str, date : datetime, isLiveMode : bool) -> BaseData:
        print(line)
        bar = TradeBar()
        data = line.split(',')
        bar.Time = datetime.strptime(data[0], "%Y-%m-%d %H:%M:%S")
        bar.EndTime = bar.Time + timedelta(days=1) - timedelta(milliseconds=1)
        bar.Value = data[4]
        bar.Open = float(data[1])
        bar.High = float(data[2])
        bar.Low = float(data[3])
        bar.Close = float(data[4])
        bar.Volume = 0.0
        return bar
+ Expand
- Collapse

20231024 13:27:30.940 ERROR:: <>c__DisplayClass1_0.b__0(): Subscription worker task exception EURUSD,#0,EURUSD,Daily,PythonData,Trade,Adjusted,OpenInterest. Microsoft.CSharp.RuntimeBinder.RuntimeBinderException: Cannot perform runtime binding on a null reference at CallSite.Target(Closure , CallSite , Object , SubscriptionDataConfig , DateTime , Boolean ) at System.Dynamic.UpdateDelegates.UpdateAndExecute4[T0,T1,T2,T3,TRet](CallSite site, T0 arg0, T1 arg1, T2 arg2, T3 arg3) at QuantConnect.Python.PythonData.GetSource(SubscriptionDataConfig config, DateTime date, Boolean isLiveMode) in /LeanCloud/CI.Builder/bin/Debug/src/QuantConnect/Lean/Common/Python/PythonData.cs:line 99 at QuantConnect.Lean.Engine.DataFeeds.SubscriptionDataReader.UpdateDataEnumerator(Boolean endOfEnumerator) in /LeanCloud/CI.Builder/bin/Debug/src/QuantConnect/Lean/Engine/DataFeeds/SubscriptionDataReader.cs:line 413 at QuantConnect.Lean.Engine.DataFeeds.SubscriptionDataReader.Initialize() in /LeanCloud/CI.Builder/bin/Debug/src/QuantConnect/Lean/Engine/DataFeeds/SubscriptionDataReader.cs:line 251 at QuantConnect.Lean.Engine.DataFeeds.SubscriptionDataReader.MoveNext() in /LeanCloud/CI.Builder/bin/Debug/src/QuantConnect/Lean/Engine/DataFeeds/SubscriptionDataReader.cs:line 269 at QuantConnect.Lean.Engine.DataFeeds.Enumerators.SynchronizingEnumerator`1.GetBruteForceMethod(IEnumerator`1[] enumerators)+MoveNext() in /LeanCloud/CI.Builder/bin/Debug/src/QuantConnect/Lean/Engine/DataFeeds/Enumerators/SynchronizingEnumerator.cs:line 143 at QuantConnect.Lean.Engine.DataFeeds.Enumerators.SynchronizingEnumerator`1.MoveNext() in /LeanCloud/CI.Builder/bin/Debug/src/QuantConnect/Lean/Engine/DataFeeds/Enumerators/SynchronizingEnumerator.cs:line 91 at QuantConnect.Lean.Engine.DataFeeds.Enumerators.SubscriptionFilterEnumerator.MoveNext() in /LeanCloud/CI.Builder/bin/Debug/src/QuantConnect/Lean/Engine/DataFeeds/Enumerators/SubscriptionFilterEnumerator.cs:line 123 at QuantConnect.Lean.Engine.DataFeeds.SubscriptionUtils.<>c__DisplayClass1_0.b__0(Int32 workBatchSize) in /LeanCloud/CI.Builder/bin/Debug/src/QuantConnect/Lean/Engine/DataFeeds/SubscriptionUtils.cs:line 93

I am using the latest docker image from lean cli. Do you have any ideas why this error message appears?

def GetSource(self, config : SubscriptionDataConfig, date : datetime, isLiveMode : bool) -> SubscriptionDataSource:
fileName = '2013.csv'
path2file = os.path.join(Globals.DataFolder, 'own', fileName)
return SubscriptionDataSource(path2file, SubscriptionTransportMedium.LocalFile)

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.

Mia Alissi INVESTOR

October 2023

From the error message, it seems like there is a null reference error when trying to bind at runtime. This could be due to the GetSource method in your MyDataReader class.

In your GetSource method, you are using the Path module to construct the path to your data file. However, according to the QuantConnect documentation, you should avoid using backward slashes as path separators as they don't work. You should only use forward slashes.

Here is how you should modify your GetSource method:

def GetSource(self, config : SubscriptionDataConfig, date : datetime, isLiveMode : bool) -> SubscriptionDataSource:
    fileName = '2013.csv'
    path2file = os.path.join(Globals.DataFolder, 'own', fileName)
    return SubscriptionDataSource(path2file, SubscriptionTransportMedium.LocalFile)

Please note that you need to import the os module at the beginning of your script:

import os

Also, ensure that the file '2013.csv' is located in the correct directory and that the directory is accessible.

If the problem persists, it might be due to the way you are reading the data in the Reader method. Make sure that the data is correctly formatted and matches the way you are trying to read it.

Upvote

Platform

Radically Open-Source Algorithmic Trading Engine

Join Our Discord Channel

Quarterly Open-Source Trading Competition

Draft Discussions

Bookmarked Discussions

SEARCH DISCUSSIONS

360,100 Quants.

VOTE FOR UPCOMING FEATURES

JOIN OUR Community MAILING LIST

IN THIS RESEARCH

PARTICIPANTS

Actions

Join QuantConnect for Free