Receivign an error trying to run algorithm in LEAN.
Here's the error. Error: Unable to cast object of type 'System.RuntimeType' to type 'QuantConnect.Data.BaseData'. And output:
20200928 17:26:26.194 ERROR:: Error invoking IGLN.IGLN data reader. Line: 2011-04-12,29.27,29.27,29.27,29.27,1 Error: Unable to cast object of type 'System.RuntimeType' to type 'QuantConnect.Data.BaseData'.
at Python.Runtime.PyObject.As[T]()
at QuantConnect.Extensions.GetAndDispose[T](PyObject instance) in C:\Users\cadmin\Lean-master\Lean-master\Common\Extensions.cs:line 425
at QuantConnect.Python.PythonData.Reader(SubscriptionDataConfig config, String line, DateTime date, Boolean isLiveMode) in C:\Users\cadmin\Lean-master\Lean-master\Common\Python\PythonData.cs:line 89
at QuantConnect.Lean.Engine.DataFeeds.TextSubscriptionDataSourceReader.<Read>d__17.MoveNext() in C:\Users\cadmin\Lean-master\Lean-master\Engine\DataFeeds\TextSubscriptionDataSourceReader.cs:line 146
And here is the script.
I thought it may be the Float but seems not, or the dataframe? How to fix? Thanks!
from clr import AddReference
AddReference("System")
AddReference("QuantConnect.Algorithm")
AddReference("QuantConnect.Common")
from System import *
from QuantConnect import *
from QuantConnect.Algorithm import *
from QuantConnect.Securities.Equity import EquityExchange
from QuantConnect.Orders import *
import math
import json
from datetime import datetime, timedelta
import numpy as np
import pandas as pd
from scipy.stats import norm
from QuantConnect.Python import PythonData # custom data
from QuantConnect.Data import SubscriptionDataSource
# Risk Premia RW algorithm
class RPRWAlgorithm(QCAlgorithm):
def Initialize(self):
# Initial settings
self.SetStartDate(2017, 1, 1)
self.SetEndDate(2019, 12, 31)
self.SetCash(30000)
self.MarketAsset = "SPY"
self.WarmupTime = 3
self.Window = 300
#parameters
self.vol_lookback = 90
self.corr_lookback = 120
self.formation_periods = np.array([3, 6, 9, 12])*22
self.z_score_cutoff = 0
self.momo_multiplier = 0.1
self.GrowthSymbols = [self.AddData(IGLN, "IGLN", Resolution.Daily).Symbol, # iShares Phys Metals PLC
self.AddData(IDTL, "IDTL", Resolution.Daily).Symbol, # iShares $ Treasury Bond 20+yr UCITS ETF
self.AddData(VDNR, "VDNR", Resolution.Daily).Symbol, # Vanguard FTSE North America UCITS ETF
self.AddData(VWO, "VWO", Resolution.Daily).Symbol, # iShares MSCI Emerging Markets Indx
self.AddData(TLT, "TLT", Resolution.Daily).Symbol, # iShares 20+ Year Treasury Bond ETF
self.AddData(VNQI, "VNQI", Resolution.Daily).Symbol, # VANGUARD INTL E/GLB EX-US RL EST IX
self.AddData(VTI, "VTI", Resolution.Daily).Symbol, # Vanguard Total Stock Market ETF
self.AddData(SPY, "SPY", Resolution.Daily).Symbol,
self.AddData(EMB, "EMB", Resolution.Daily).Symbol, # iShares J.P. Morgan USD Emerging Markets Bond ETF
self.AddData(PUTW, "PUTW", Resolution.Daily).Symbol] # WisdomTree CBOE S&P 500 PutWrite Strategy Fund
#"VEA", # VEA - Vanguard FTSE Developed Markets
#"UST", # ProShares Ultra 7-10 Year Treasury
#"GBTC", #BTC
# these are the safety symbols we go to when things are looking bad for growth
# this part is not supposed to work
# I don't know how to open these assets
#self.SafetySymbols = "PUTW", # WisdomTree CBOE S&P 500 PutWrite Strategy Fund
# "EMB"] # iShares J.P. Morgan USD Emerging Markets Bond ETF
#self.ticker = "PUTW"
#self.symbol = self.AddData(TiingoDailyData, self.ticker, Resolution.Daily).Symbol
#self.AddEquity("SPY", Resolution.Daily)
#if self.LiveMode:
# self.Debug("Trading Live!")
#self.SafetySymbols = [self.AddData(EMB, "EMB", Resolution.Daily).Symbol, # iShares J.P. Morgan USD Emerging Markets Bond ETF
#self.AddData(PUTW, "PUTW", Resolution.Daily).Symbol] # WisdomTree CBOE S&P 500 PutWrite Strategy Fund
# all symbols set
# self.AllSymbols = list(set(self.GrowthSymbols) | set(self.SafetySymbols))
self.TickerVolTarget = 0.05
self.VolLookback = 60
self.MaxLeverage = 1
self.SetWarmUp(self.VolLookback)
# open equity symbols
for symbol in self.GrowthSymbols:
self.AddEquity(symbol, Resolution.Daily)
# this doesn't do anything at the moment. We need to work out how to properly handles these assets
#for symbol in self.SafetySymbols:
# self.AddOption(symbol, Resolution.Daily)
# wait for warming up
self.SetWarmUp(self.WarmupTime)
# schedule the trading function
self.Schedule.On(self.DateRules.EveryDay(self.MarketAsset), self.TimeRules.AfterMarketOpen(self.MarketAsset, 120), Action(self.RebalanceAndTrade))
# schedule the Portfolio Statistics
self.Schedule.On(self.DateRules.EveryDay(self.MarketAsset), self.TimeRules.AfterMarketOpen(self.MarketAsset, 10), Action(self.Perfomance))
def OnEndOfDay(self, symbol):
self.Plot(str(symbol),'EOD',self.Securities[symbol].Price)
#def OnData(self, data):
# pass
# if self.LiveMode: self.Debug("Running algorithm!!")
def OnData(self, slice):
#if self.LiveMode: self.Debug("Running algorithm!!")
# Make sure all the data we need is in place
if self.IsWarmingUp: return
# if not slice.ContainsKey("PUTW"):
# self.Debug("PUTW not found!!")
# return
# if not slice.ContainsKey("EMB"):
# self.Debug("EMB not found!!")
# return
#if self.LiveMode: self.Debug("Warm Up Complete Deciding..")
def Perfomance(self):
slices = self.History(self.GrowthSymbols, self.Window, Resolution.Daily)
slices_df = pd.pivot_table(slices, values = 'close', index='time', columns = 'symbol').reset_index()
slices_df = slices_df.drop(columns=['time'])
#slices_df.columns = [SymbolCache.GetTicker(x) for x in slices_df.columns]
returns = slices_df.pct_change()
# trading function
def RebalanceAndTrade(self):
slices = self.History(self.GrowthSymbols, self.Window, Resolution.Daily)
slices_df = pd.pivot_table(slices, values = 'close', index='time', columns = 'symbol').reset_index()
slices_df = slices_df.drop(columns=['time'])
#slices_df.columns = [SymbolCache.GetTicker(x) for x in slices_df.columns]
returns = slices_df.pct_change()
if self.IsWarmingUp: return
#if self.Time.day != 6: return
# creating the pandas DataFrame
'''
slices = self.History(self.AllSymbols, self.Window, Resolution.Daily)
slices_df = pd.pivot_table(slices, values = 'close', index='time', columns = 'symbol').reset_index()
slices_df = slices_df.drop(columns=['time'])
slices_df.columns = [SymbolCache.GetTicker(x) for x in slices_df.columns]
returns = slices_df.pct_change()
'''
# for debugging
#self.Debug(self.Time)
#self.Debug(returns.shape)
# weights calculation
vol_weights = self.get_srp_weights(returns, self.vol_lookback)
cor_adjust = self.get_cor_adjustments(returns, self.corr_lookback)
cor_adjust_weights = self.adjust_weights(vol_weights, cor_adjust, shrinkage=1)
momo_adjusted_weights = self.get_momo_adjusted_weights(returns, cor_adjust_weights, self.formation_periods, self.z_score_cutoff, self.momo_multiplier)
# the following should contain asset EMB instead of EEM
capped_weights = self.cap_allocation_and_rescale(momo_adjusted_weights, ticker="EMB", cap=0.15)
# the following should VTI and PUTW but I don't know how to handle yet
final_weights = self.split_allocation(capped_weights, "VTI", "PUTW", ratio=0.5)
self.Debug(final_weights.shape)
self.Debug(self.Time)
self.Debug(final_weights)
# allocating assets
for i in range(len(final_weights)):
self.Log("{} : asset {}, allocating {}".format(self.Time, slices_df.columns[i], final_weights[i]))
self.SetHoldings(slices_df.columns[i], final_weights[i])
def get_srp_weights(self, returns, vol_lookback):
"""
returns current srp werights given a pandas DataFrame of returns and a vol_lookback period
"""
n_assets = len(returns.columns)
vols = returns.iloc[-vol_lookback:, :].apply(lambda x: np.std(x)*np.sqrt(252), axis=0)
raw_weights = 1/vols
weights = raw_weights/np.sum(raw_weights)
return weights
def get_cor_adjustments(self, returns, corr_lookback):
"""
returns current correlation adjustments given a pandas DataFrame of returns and a corr_lookback period
"""
cor = returns.iloc[-corr_lookback:, :].corr()
pairwise_ave_cor = cor.mean(axis=1)
zscore_pairwise_ave_cor = (pairwise_ave_cor - pairwise_ave_cor.mean())/pairwise_ave_cor.std()
gauss_scale = 1 - norm.cdf(zscore_pairwise_ave_cor, 0, 1)
raw_adjustments = gauss_scale/gauss_scale.sum()
norm_adjustments = raw_adjustments - 1./len(returns.columns)
return norm_adjustments
def adjust_weights(self, vol_weights, corr_adjustments, shrinkage):
raw_weights = vol_weights * (1 +corr_adjustments * shrinkage)
adj_weights = raw_weights/raw_weights.sum()
return adj_weights
def get_momo_adjustments(self, returns, formation_period):
"""
returns current cross-sectional zscore of total return momentum
given a pandas DataFrame of returns and formation_period
"""
synth_prices = (returns+1).cumprod()
roc = (synth_prices.iloc[-1,:]/synth_prices.iloc[-formation_period-1,:]-1)
momo_adjustments = (roc - roc.mean())/roc.std()
return momo_adjustments
def get_sma_slope_adjustments(self, returns, formation_period):
"""
returns current cross-sectional zscore of slope of moving average
given a pandes DataFrame of returns and a formation_period
"""
synth_prices = (returns+1).cumprod()
sma = synth_prices.iloc[-formation_period-1:,:].rolling(formation_period).mean()
sma_slope = (sma.iloc[-1,:]/sma.iloc[-2,:])-1
momo_adjustments = (sma_slope - sma_slope.mean())/sma_slope.std()
return momo_adjustments
def adjust_momo_weights(self, base_weights, momo_adjustments, z_score_cutoff, multiplier):
raw_weights = base_weights * (1 + ((momo_adjustments >= z_score_cutoff) * multiplier))
adj_weights = raw_weights/raw_weights.sum()
return adj_weights
def get_momo_adjusted_weights(self, returns, base_weights, formation_periods, z_score_cutoff, multiplier):
"""
returns current momentum-adjusted weights given a pandes DataFrame of returns and a formation_period
"""
momo_weights = base_weights
for period in formation_periods :
momo_adjustments = self.get_momo_adjustments(returns, period)
momo_weights = self.adjust_momo_weights(momo_weights, momo_adjustments, z_score_cutoff, multiplier)
for period in formation_periods :
momo_adjustments = self.get_sma_slope_adjustments(returns, period)
momo_weights = self.adjust_momo_weights(momo_weights, momo_adjustments, z_score_cutoff, multiplier)
return momo_weights
def cap_allocation_and_rescale(self, weights, ticker, cap=0.15):
"""
cap the allocation into ticker and rescale remaining weights
"""
if weights[ticker] > cap:
weights = (1-cap)*weights.drop(ticker)/weights.drop(ticker).sum()
weights[ticker] = cap
return weights
def split_allocation(self, weights, ticker, split_ticker, ratio=0.5):
"""
split the allocation into ticker into ticker and split_ticker according to ratio
"""
weights[split_ticker] = (1-ratio)*weights[ticker]
weights[ticker] = ratio*weights[ticker]
#global tradeable_universe
#if split_ticker not in tradeable_universe:
# tradeable_universe.append(split_ticker)
return weights
class IGLN(PythonData):
'''IGLN Custom Data Class'''
def GetSource(self, config, date, datafeed):
#return SubscriptionDataSource(source, SubscriptionTransportMedium.Rest);
return SubscriptionDataSource("https://www.dropbox.com/s/s9a65ecegg8kvu0/IGLN.csv?dl=1", SubscriptionTransportMedium.RemoteFile)
def Reader(self, config, line, date, datafeed):
if not (line.strip() and line[0].isdigit()): return None
# New GoldPhys object
igln1 = IGLN()
igln1.Symbol = config.Symbol
try:
# Example File Format:
# Date, Open High Low Close Volume
# 2011-09-13 7792.9 7799.9 7722.65 7748.7 116534670
data = line.split(',')
igln1.Time = datetime.strptime(data[0], "%Y-%m-%d")
igln1.Value = data[4]
igln1["Open"] = float(data[1])
igln1["High"] = float(data[2])
igln1["Low"] = float(data[3])
igln1["Close"] = float(data[4])
except ValueError:
# Do nothing
return None
return IGLN
class IDTL(PythonData):
'''IDTL Custom Data Class'''
def GetSource(self, config, date, datafeed):
return SubscriptionDataSource("https://www.dropbox.com/s/ac9sc2e6px754k5/IDTL.csv?dl=1", SubscriptionTransportMedium.RemoteFile)
def Reader(self, config, line, date, datafeed):
if not (line.strip() and line[0].isdigit()): return None
# New Treas20 object
idtl1 = IDTL()
idtl1.Symbol = config.Symbol
try:
# Example File Format:
# Date, Open High Low Close Volume
# 2011-09-13 7792.9 7799.9 7722.65 7748.7 116534670
data = line.split(',')
idtl1.Time = datetime.strptime(data[0], "%Y-%m-%d")
idtl1.Value = data[4]
idtl1["Open"] = float(data[1])
idtl1["High"] = float(data[2])
idtl1["Low"] = float(data[3])
idtl1["Close"] = float(data[4])
except ValueError:
# Do nothing
return None
return IDTL
class VDNR(PythonData):
'''VDNR Custom Data Class'''
def GetSource(self, config, date, datafeed):
return SubscriptionDataSource("https://www.dropbox.com/s/pqwv2psx3qeysl1/VDNR.csv?dl=1", SubscriptionTransportMedium.RemoteFile)
def Reader(self, config, line, date, datafeed):
if not (line.strip() and line[0].isdigit()): return None
# New VanSPY object
vdnr1 = VDNR()
vdnr1.Symbol = config.Symbol
try:
# Example File Format:
# Date, Open High Low Close Volume
# 2011-09-13 7792.9 7799.9 7722.65 7748.7 116534670
data = line.split(',')
vdnr1.Time = datetime.strptime(data[0], "%Y-%m-%d")
vdnr1.Value = data[4]
vdnr1["Open"] = float(data[1])
vdnr1["High"] = float(data[2])
vdnr1["Low"] = float(data[3])
vdnr1["Close"] = float(data[4])
except ValueError:
# Do nothing
return None
return vdnr1
class VWO(PythonData):
'''VWO Custom Data Class'''
def GetSource(self, config, date, datafeed):
return SubscriptionDataSource("https://www.dropbox.com/s/tzk1e1h09a5etq4/VWO.csv?dl=1", SubscriptionTransportMedium.RemoteFile)
def Reader(self, config, line, date, datafeed):
if not (line.strip() and line[0].isdigit()): return None
# New VanSPY object
vwo1 = VWO()
vwo1.Symbol = config.Symbol
try:
# Example File Format:
# Date, Open High Low Close Volume
# 2011-09-13 7792.9 7799.9 7722.65 7748.7 116534670
data = line.split(',')
vwo1.Time = datetime.strptime(data[0], "%Y-%m-%d")
vwo1.Value = data[4]
vwo1["Open"] = float(data[1])
vwo1["High"] = float(data[2])
vwo1["Low"] = float(data[3])
vwo1["Close"] = float(data[4])
except ValueError:
# Do nothing
return None
return vwo1
class TLT(PythonData):
'''TLT Custom Data Class'''
def GetSource(self, config, date, datafeed):
return SubscriptionDataSource("https://www.dropbox.com/s/8shi6ow4be3dzz9/TLT.csv?dl=1", SubscriptionTransportMedium.RemoteFile)
def Reader(self, config, line, date, datafeed):
if not (line.strip() and line[0].isdigit()): return None
# New VanSPY object
tlt1 = TLT()
tlt1.Symbol = config.Symbol
try:
# Example File Format:
# Date, Open High Low Close Volume
# 2011-09-13 7792.9 7799.9 7722.65 7748.7 116534670
data = line.split(',')
tlt1.Time = datetime.strptime(data[0], "%Y-%m-%d")
tlt1.Value = data[4]
tlt1["Open"] = float(data[1])
tlt1["High"] = float(data[2])
tlt1["Low"] = float(data[3])
tlt1["Close"] = float(data[4])
except ValueError:
# Do nothing
return None
return tlt1
class VNQI(PythonData):
'''VNQI Custom Data Class'''
def GetSource(self, config, date, datafeed):
return SubscriptionDataSource("https://www.dropbox.com/s/0b96784rc8b8zh3/VNQI.csv?dl=1", SubscriptionTransportMedium.RemoteFile)
def Reader(self, config, line, date, datafeed):
if not (line.strip() and line[0].isdigit()): return None
# New VanSPY object
vnqi1 = VNQI()
vnqi1.Symbol = config.Symbol
try:
# Example File Format:
# Date, Open High Low Close Volume
# 2011-09-13 7792.9 7799.9 7722.65 7748.7 116534670
data = line.split(',')
vnqi1.Time = datetime.strptime(data[0], "%Y-%m-%d")
vnqi1.Value = data[4]
vnqi1["Open"] = float(data[1])
vnqi1["High"] = float(data[2])
vnqi1["Low"] = float(data[3])
vnqi1["Close"] = float(data[4])
except ValueError:
# Do nothing
return None
return vnqi1
class PUTW(PythonData):
'''PUTW Custom Data Class'''
def GetSource(self, config, date, datafeed):
return SubscriptionDataSource("https://www.dropbox.com/s/ovqzyz5l91524f8/PUTW.csv?dl=1", SubscriptionTransportMedium.RemoteFile)
def Reader(self, config, line, date, datafeed):
if not (line.strip() and line[0].isdigit()): return None
# New VanSPY object
putw1 = PUTW()
putw1.Symbol = config.Symbol
try:
# Example File Format:
# Date, Open High Low Close Volume
# 2011-09-13 7792.9 7799.9 7722.65 7748.7 116534670
data = line.split(',')
putw1.Time = datetime.strptime(data[0], "%Y-%m-%d")
putw1.Value = data[4]
putw1["Open"] = float(data[1])
putw1["High"] = float(data[2])
putw1["Low"] = float(data[3])
putw1["Close"] = float(data[4])
except ValueError:
# Do nothing
return None
return putw1
class EMB(PythonData):
'''EMB Custom Data Class'''
def GetSource(self, config, date, datafeed):
return SubscriptionDataSource("https://www.dropbox.com/s/o4qjcj586woc7m1/EMB.csv?dl=1", SubscriptionTransportMedium.RemoteFile)
def Reader(self, config, line, date, datafeed):
if not (line.strip() and line[0].isdigit()): return None
# New VanSPY object
emb1 = EMB()
emb1.Symbol = config.Symbol
try:
# Example File Format:
# Date, Open High Low Close Volume
# 2011-09-13 7792.9 7799.9 7722.65 7748.7 116534670
data = line.split(',')
emb1.Time = datetime.strptime(data[0], "%Y-%m-%d")
emb1.Value = data[4]
emb1["Open"] = float(data[1])
emb1["High"] = float(data[2])
emb1["Low"] = float(data[3])
emb1["Close"] = float(data[4])
except ValueError:
# Do nothing
return None
return emb1
class VTI(PythonData):
'''VTI Custom Data Class'''
def GetSource(self, config, date, datafeed):
return SubscriptionDataSource("https://www.dropbox.com/s/mg222g0ykrhk6ul/VTI.csv?dl=1", SubscriptionTransportMedium.RemoteFile)
def Reader(self, config, line, date, datafeed):
if not (line.strip() and line[0].isdigit()): return None
# New VanSPY object
vti1 = VTI()
vti1.Symbol = config.Symbol
try:
# Example File Format:
# Date, Open High Low Close Volume
# 2011-09-13 7792.9 7799.9 7722.65 7748.7 116534670
data = line.split(',')
vti1.Time = datetime.strptime(data[0], "%Y-%m-%d")
vti1.Value = data[4]
vti1["Open"] = float(data[1])
vti1["High"] = float(data[2])
vti1["Low"] = float(data[3])
vti1["Close"] = float(data[4])
except ValueError:
# Do nothing
return None
return vti1
class SPY(PythonData):
'''SPY Custom Data Class'''
def GetSource(self, config, date, datafeed):
return SubscriptionDataSource("https://www.dropbox.com/s/5e9eon88l91nxge/SPY.csv?dl=1", SubscriptionTransportMedium.RemoteFile)
def Reader(self, config, line, date, datafeed):
if not (line.strip() and line[0].isdigit()): return None
# New VanSPY object
spy1 = SPY()
spy1.Symbol = config.Symbol
try:
# Example File Format:
# Date, Open High Low Close Volume
# 2011-09-13 7792.9 7799.9 7722.65 7748.7 116534670
data = line.split(',')
spy1.Time = datetime.strptime(data[0], "%Y-%m-%d")
spy1.Value = data[4]
spy1["Open"] = float(data[1])
spy1["High"] = float(data[2])
spy1["Low"] = float(data[3])
spy1["Close"] = float(data[4])
except ValueError:
# Do nothing
return None
return spy1
Shile Wen
Hi AM H,
This error is caused because we are using the wrong date format. The algorithm uses %Y-%m-%d, but the CSV date format is %Y/%m/%d.
Best,
Shile Wen
AM H
Ah, I see! Though I'm having a lot of difficulty formatting. As MSDOS csv file after formatting the date in Excel to the correct format, it can be seen that the format is correct when opening the file as text or in other text readers. Re-opening the CSV file in Excel is imediately back to the / / format!!!
I found a fix for this by altering the system format on my Windows machine, which in turn showed the correct format as I had set it in the csv files.
However, going to all this trouble hasn't solved the error which remains.
I then thought the 'Time' reference was wrong, renamed all columns to Time instead of Date. Same errors!
AM H
Turns out there were so many other issues I lost count. BUT it's up and running now!
Thanks for your help!
AM H
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
To unlock posting to the community forums please complete at least 30% of Boot Camp.
You can continue your Boot Camp training progress from the terminal. We hope to see you in the community soon!