Using QuantBook inside QCAlgorithm to get historical fundamental data?

Hi Karthik,

There shouldn't arise any framework issues from instantiating a QuantBook in the backtesting environment, however, it's not what the QuantBook class is intended for. We are adding a fundamental history method to the backtester. Subscribe to this GitHub Issue to be notified of our progress.

Best,
Derek Melchin

Pretty slick.  I was running a backtest algorithm to collect initial data on a fundamental universe and then storing that into an ObjectStore and then running a script in the research environment that would read in the ObjectStore, get the needed historical fundamental data and make buy/sell decisions and then store those decisions in another ObjectStore and then run a second backtest algorithm that processed the trades.  The research environment is much easier to use to get fundamental historical data and play around with it.  It will be nice when the backtest environment can do historical fundamental data calls, but this is a slick solution until then! Thanks, Karthik Kailash

As of some point in the last week, this isn't working anymore.  Cloning and running the exact algorithm in the original post gets an ugly error.

All versions of my algorithms that use this also stopped working with similar errors. Even copy-pasting previously successful backtests throws errors. Derek Melchin , any ideas how we can approach fixing this while the github issue you linked is still in progress?

Runtime Error: Exception : QuantBook.Main(): System.NullReferenceException: Object reference not set to an instance of an object.
   at QuantConnect.Cloud.Server.CloudDataPermissionManager.GetResolution(Resolution preferredResolution) in /LeanCloud/CI.Builder/bin/Debug/src/QuantConnect/LeanCloud/Cloud.LeanManagement/CloudDataPermissionManager.cs:line 81
   at QuantConnect.Lean.Engine.DataFeeds.UniverseSelection..ctor(IAlgorithm algorithm, ISecurityService securityService, IDataPermissionManager dataPermissionManager, IDataProvider dataProvider, Resolution internalConfigResolution) in /LeanCloud/CI.Builder/bin/Debug/src/QuantConnect/Lean/Engine/DataFeeds/UniverseSelection.cs:line 69
   at QuantConnect.Research.QuantBook..ctor() in /LeanCloud/CI.Builder/bin/Debug/src/QuantConnect/Lean/Research/QuantBook.cs:line 158
   at QuantConnect.Research.QuantBook..ctor() in /LeanCloud/CI.Builder/bin/Debug/src/QuantConnect/Lean/Research/QuantBook.cs:line 189
  at CoarseFilter
    qb = QuantBook()
===
   at Python.Runtime.Dispatcher.TrueDispatch(Object[] args)
   at Python.Runtime.Dispatcher.Dispatch(Object[] args)
   at __System_Func`2\[\[System_Collections_Generic_IEnumerable`1\[\[QuantConnect_Data_UniverseSelection_CoarseFundamental\ in main.py:line 17
 Exception : QuantBook.Main(): System.NullReferenceException: Object reference not set to an instance of an object.
   at QuantConnect.Cloud.Server.CloudDataPermissionManager.GetResolution(Resolution preferredResolution) in /LeanCloud/CI.Builder/bin/Debug/src/QuantConnect/LeanCloud/Cloud.LeanManagement/CloudDataPermissionManager.cs:line 81
   at QuantConnect.Lean.Engine.DataFeeds.UniverseSelection..ctor(IAlgorithm algorithm, ISecurityService securityService, IDataPermissionManager dataPermissionManager, IDataProvider dataProvider, Resolution internalConfigResolution) in /LeanCloud/CI.Builder/bin/Debug/src/QuantConnect/Lean/Engine/DataFeeds/UniverseSelection.cs:line 69
   at QuantConnect.Research.QuantBook..ctor() in /LeanCloud/CI.Builder/bin/Debug/src/QuantConnect/Lean/Research/QuantBook.cs:line 158
   at QuantConnect.Research.QuantBook..ctor() in /LeanCloud/CI.Builder/bin/Debug/src/QuantConnect/Lean/Research/QuantBook.cs:line 189

A little bit more information to help debugging; this link in the docs says to replace qb = Quantbook() with qb = self. Doing this avoids the error.

However, it also says “GetFundamental in research does not have any counterpart in backtesting”. This is certainly a recent change since GetFundamental worked up until last week in the backtester. 

Why remove this? This was a perfect workaround while we don't have fundamental history. 

Hi Big mak

We've tested the backtest attached in the original post, which did not result in any error. However, we would not recommend using this method to obtain Fundamental history as this method received datetime object that could be a future time of the backtest's time. You might easily fall into the trap of forward bias! Similar reason on why "GetFundamental in research does not have any counterpart in backtesting". You could easily create your own fundamental history by pandas dataframe for backtesting.

As for research purposes, the .GetFundamental method is also working. Please find the embedded backtest and research notebook as references.

Best,
Louis Szeto

Louis,

I just checked the OP algorithm and my own algorithms - it's all working now. No idea what happened a few days ago but it was definitely not working for some time, producing the error I linked above. I understand the concerns on the forward looking bias: I worked extremely hard to avoid this by creating my own wrapper. Check out my thread here.