I am trying to write a custom data import code in python. I am using this algo as an example: https://raw.githubusercontent.com/QuantConnect/Lean/master/Algorithm.Python/CustomDataBitcoinAlgorithm.py

Every time I use "SubscriptionDataSource" with RemoteFIle paramter I get the following error "Protocol Error"

Backtest Handled Error: Error downloading custom data source file, skipped: RemoteFile: Csv https://www.quandl.com/api/v3/datasets/BCHARTS/BITSTAMPUSD.csv?order=asc Error: Error: ProtocolError (Open Stacktrace)

Here is the GetSource code
 

class Bitcoin(PythonData): '''Custom Data Type: Bitcoin data from Quandl - http://www.quandl.com/help/api-for-bitcoin-data''' def GetSource(self, config, date, isLiveMode): if isLiveMode: return SubscriptionDataSource("https://www.bitstamp.net/api/ticker/", SubscriptionTransportMedium.Rest); #return "http://my-ftp-server.com/futures-data-" + date.ToString("Ymd") + ".zip"; # OR simply return a fixed small data file. Large files will slow down your backtest return SubscriptionDataSource("https://www.quandl.com/api/v3/datasets/BCHARTS/BITSTAMPUSD.csv?order=asc", SubscriptionTransportMedium.RemoteFile); def Reader(self, config, line, date, isLiveMode): coin = Bitcoin() coin.Symbol = config.Symbol if isLiveMode: # Example Line Format: # {"high": "441.00", "last": "421.86", "timestamp": "1411606877", "bid": "421.96", "vwap": "428.58", "volume": "14120.40683975", "low": "418.83", "ask": "421.99"} try: liveBTC = json.loads(line) # If value is zero, return None value = liveBTC["last"] if value == 0: return None coin.Time = datetime.now() coin.Value = value coin["Open"] = float(liveBTC["open"]) coin["High"] = float(liveBTC["high"]) coin["Low"] = float(liveBTC["low"]) coin["Close"] = float(liveBTC["last"]) coin["Ask"] = float(liveBTC["ask"]) coin["Bid"] = float(liveBTC["bid"]) coin["VolumeBTC"] = float(liveBTC["volume"]) coin["WeightedPrice"] = float(liveBTC["vwap"]) return coin except ValueError: # Do nothing, possible error in json decoding return None



I've tried number of other datasources and they all return this cryptic error message.

I've also tried the example shown in documentation, which doesn't seem to work either, even when source URL is replaced with a URL that contains actual data still "ProtocolError" is thrown.

class Weather(PythonData): ''' Weather based rebalancing''' def GetSource(self, config, date, isLive): source = "https://www.wunderground.com/history/airport/{0}/{1}/1/1/CustomHistory.html?dayend=31&monthend=12&yearend={1}&format=1".format(config.Symbol, date.year); return SubscriptionDataSource(source, SubscriptionTransportMedium.RemoteFile); def Reader(self, config, line, date, isLive): # If first character is not digit, pass if not (line.strip() and line[0].isdigit()): return None data = line.split(',') weather = Weather() weather.Symbol = config.Symbol weather.Time = datetime.strptime(data[0], '%Y-%m-%d') + timedelta(hours=20) # Make sure we only get this data AFTER trading day - don't want forward bias. weather.Value = decimal.Decimal(data[2]) weather["MaxC"] = float(data[1]) weather["MinC"] = float(data[3]) return weather



Can anyone shed some light on this ?

Author

Konrad S

May 2019