Hi,
I would like to ask for advise on how Lean developers debug Python code that depends on the CLR.
For example, PR #4944 is a fix for this issue in the Research notebook when run locally using Docker:
I am wondering what were the tools and workflow used to identify the root cause of the issue.
As a Python developer, my starting point would be the stack trace. But this exception did not print any. The next thing I'd consider is looking at the source code of the line that threw the exception:
but this is no help at all.
So I'm really curious to know how the developers at QuantConnect go about solving this sort of problem.
I'm new to C# and not very familiar with the tooling around it so feel free to direct me to resources that can help newcomers understand how the project is structured.
Jared Broad
Mind sharing something you're trying to debug? Most of the time there is a stack-trace especially if using Try-Catch. If its a runtime exception without a try-catch there is an ugly hybrid C#-Python stack but its decipherable. The python code is running in the python runtime; so normally comes with all the normal python information but if you give us more information we can see what your case might be.
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Aaron Janeiro Stone
In general, note that for a imported dll (e.g., QuantConnect.Algorithm), the "underlying" code used to generate the module is present in the C# source at github.
Thus, when debugging the underlying code for a dll in the context of Python (e.g., to create a regression test for Algorithm.Python, I usually look for the imported element and place a breakpoint there.
As an example, say I am interested in a value called during QCAlgorithm.OnData. I would place my breakpoint in the QCAlgorithm.cs source within the OnData method and then be able to look at values which exist in the environment at that point (say, the value of some list I specified in my self.Initialize step, "self.ListOfSymbols", which updates in the self.OnData step). I usually use Rider when doing this kind of development but Visual Studio would work as well (or vs code although I find it to be clunkier).
Moeh12
I had attached images to show what I'm trying to debug but not sure why it did not get rendered. I will try putting code snippets instead.
The thing I am trying to debug is exactly the same as this bug report. I know it's kinda solved already but I'd like to learn how to figure it out.
This is the notebook code (to be run inside the research container):
%matplotlib inline
# Imports
from clr import AddReference
AddReference("System")
AddReference("QuantConnect.Common")
AddReference("QuantConnect.Research")
AddReference("QuantConnect.Indicators")
from System import *
from QuantConnect import *
from QuantConnect.Data.Custom import *
from QuantConnect.Data.Market import TradeBar, QuoteBar
from QuantConnect.Research import *
from QuantConnect.Indicators import *
from datetime import datetime, timedelta
import matplotlib.pyplot as plt
import pandas as pd
# Create an instance
qb = QuantBook()
# Select asset data
spy = qb.AddEquity("SPY")
spy = qb.AddEquity("SPY")
# Gets historical data from the subscribed assets, the last 360 datapoints with daily resolution
h1 = qb.History(qb.Securities.Keys, 360, Resolution.Daily)
# Example with BB, it is a datapoint indicator
# Define the indicator
bb = BollingerBands(30, 2)
# Gets historical data of indicator
bbdf = qb.Indicator(bb, "SPY", 360, Resolution.Daily)
# drop undesired fields
bbdf = bbdf.drop('standarddeviation', 1)
# Plot
bbdf.plot()
This is the entire stack trace:
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-1-f0e5db2bb4bf> in <module>
32
33 # Gets historical data of indicator
---> 34 bbdf = qb.Indicator(bb, "SPY", 360, Resolution.Daily)
35
36 # drop undesired fields
ValueError: cannot reindex from a duplicate axis
Aaron Janeiro Stone Thanks for the tips. I'll try to get the workflow set up in Visual Studio and/or VS Code.
Moeh12
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
To unlock posting to the community forums please complete at least 30% of Boot Camp.
You can continue your Boot Camp training progress from the terminal. We hope to see you in the community soon!