Share New Research 
Hi everyone!
Thanks for the great platform to backtest and trade with!
I'm working on a new strategy that uses a different universe of stocks everyday pulled from a dropbox file.
My algo needs to calculate the 90 day rolling moving standard deviation for the close to open returns. Therefore I have to set the start date of the algo earlier than my inended start date because I can't use warm ups with universes. That's fine and working.
However, I can't figure out how to access the close prices from the multi-index dataframe I get back. I'm sure it's something silly but I'm stuck.
Here's a snippet from a function called by the `AfterMarketOpen` time rule:
self.Debug('security.Symbol: {}'.format(security.Symbol))
# Get a multi index dataframe
history_midf = self.History(security.Symbol, 90, Resolution.Daily)
self.Debug('{}'.format(history_midf))
# Select the security close and get a single index dataframe
close_df = history_midf.loc[security.Symbol]['close']
self.Debug('{}'.format(close_df))
I get this error though:
Runtime Error: In Scheduled Event 'SPY: EveryDay: SPY: 1 min after MarketOpen', TypeError : object is not callable TypeError : object is not callable (Open Stacktrace)
1639 | 15:20:53:
Launching analysis for ef718efee8a2329c161567f513e46cfb with LEAN Engine v2.4.0.0.4601
1640 | 15:20:53:
2017-07-07 00:00:00 Universe Changes: SecurityChanges: Added: SPY R735QTJ8XC9X
1641 | 15:20:53:
Backtest deployed in 16.057 seconds
1642 | 15:20:54:
2018-01-04 00:00:00 Universe Changes: SecurityChanges: Added: CMC R735QTJ8XC9X,NEOG R735QTJ8XC9X,UNF R735QTJ8XC9X
1643 | 15:20:55:
Runtime Error: In Scheduled Event 'SPY: EveryDay: SPY: 1 min after MarketOpen', TypeError : object is not callable TypeError : object is not callable (Open Stacktrace)
1644 | 15:21:05:
Algorithm Id:(ef718efee8a2329c161567f513e46cfb) completed in 12.19 seconds at 4k data points per second. Processing total of 48,668 data points.
1645 | 15:21:05:
security.Symbol: CMC R735QTJ8XC9X
1647 | 15:21:05:
open ... volume
symbol time ...
CMC R735QTJ8XC9X 2017-08-26 17.341478 ... 507806.0
...
The security I'm looking at in this example is CMC but security.Symbol is this weird thing:
CMC R735QTJ8XC9X
The error is definitely generated by this line:
close_df = history_midf.loc[security.Symbol]['close']
Any ideas?
Hugh Todd
I'm far from an expert on this, but have several backtests that load symbols from Dropbox using AddUniverse in a Python environment. If I understand it correctly symbol strings and symbol objects should be treated the same, but this doesn't seem to be the case when symbols are added with AddUniverse. I would try calling history with security.Symbol.Value and see if that works.
No guarantees.
Brett Elliot
Hi Hugh. Thanks for the reply. I also found references to `security.Symbol.Value`. But when I used that I get this error:
Cannot create history for the given ticker. Either explicitly use a symbol object to make the history request or ensure the symbol has been added using the AddSecurity() method before making the history request.
Now, it does suggest to use the symbol object (which I did and got the above error) or call AddSecurity before the call to history. Doing that gets me another error message:
TypeError : unindexable object TypeError : unindexable object (Open Stacktrace)
So, i'm SOL either way.
Jared Broad
Hi Brett; you should use security.Symbol not .Value -- the value is the ticker string but you need to use the actual object.
Note the missing brackets on your API call:
self.History( [ security.Symbol ] , 90, Resolution.Daily)The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Brett Elliot
Hi Jared. Thanks for the reply. I still get the error. Let me share the code. You have to uncomment the lines below so that the error occurs because I can't share a backtest with a runtime error:
### Uncomment these lines to see the error: # Select the security close and get a single index dataframe # close_df = history_midf[security.Symbol]['close'] # self.Debug('{}'.format(close_df))Jared Broad
There are some great docs here on history along with many examples.
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Hugh Todd
Brett,
Sorry about sending you in the wrong direction with my comment yesterday.
I'm not sure if you've solved your latest error but this one works. I spent way too much time on this, but like I said yesterday I have several projects using similar structures and wanted to figure it out. As Jared said, you must call self.History using security.Symbol, but it turns out that to reference the data in the df you must use str(security.Symbol). This makes no sense to me, but it works. The pertinent code is this:
sym = str(security.Symbol) close_df = history_df.loc[sym]['close'] # select date & closing prices for one symsecurity.Symbol does not work, and security.Symbol.Value does not work, but str(security.Symbol) does work. See lines 127 - 133 of the attached backtest.
This may be a bug, because it is not consistent with any example code I've seen.
Hope it helps you out.
Brett Elliot
Hugh Todd
Thanks! That works! I really appreciate it!
Alexandre Catarino
In Lean/QuantConnect, string is implictly converted into Symbol object (and vice-versa). It means that in any method or dictionary, we can use string where Symbol object is required. For instance in a History call:
spy_symbol = self.AddEquity('SPY').Symbol # Last 10 SPY bars with minute resolution df = self.History([spy_symbol], 10) # or df = self.History(['SPY'], 10)Unfortunatelly implicit conversion is not supported by pandas DataFrame:
spy_df = df['SPY'] # works spy_df = df[spy_symbol] # type error spy_df = df[str(spy_symbol)] # worksBrett Elliot
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
To unlock posting to the community forums please complete at least 30% of Boot Camp.
You can continue your Boot Camp training progress from the terminal. We hope to see you in the community soon!