I am still climbing the learning curve here, and I learnt a lot from the LEAN source code. However, it is not clear to me what is the preferred approach I would have to take in my little research project.
I am more fluid with C#(/Java) then Python, but I only need Python for some machine learning bits (which I will implement through Tensorflow/Keras). What would be the primary route to take here?
Start from C#, encapsulate and wrap all ML bits in as few python classes as possible, and call these from compiled code, or just forego C#, and do everything in python?
Things to consider for me are ease of development & debugging, execution speed (compiled code vs interpreted code), reduce the footprint of the LEAN docker container (15GB out of the box, which is hefty), potentially be able to extend LEAN with other building blocks like a REST interface, or through class inheritance, database connectivity,.... and lastly avoid the version dependency mess with all the python packages that are part of LEAN (i.e. I try to build a docker image using the Dockerfiles provided in the distribution, but that is a road full of bumps as there dependency issues)
Is the Python .NET integration "fluid", or are there any things/pitfalls to know in advance that might impact the decision to take?
Jared Broad
Hi Karel, welcome to QuantConnect!
Can you use C# machine learning libraries? Given your skillset that would be the best for your case IMHO.
Embedding is not great. If possible I'd recommend a pure python version.
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
Karel Goderis
mmh.. My understanding of machine learning is quite language-agnostic, so the short answer is no, but I was just looking into Tensorflow.NET as a nice alternative to use python's tensorflow, esp. given that the syntax is very similar and so algo's can be used without too much modification. If that would work, then we are one step closer to being C# only.
Karel Goderis
Jared Broad It is not a walk in the park. Pffff ;-)
As a test, I just added the Tensorflow.NET nuget to the Algorith.CSharp project, then added a simple
using static Tensorflow.Binding;
to BasicTemplateFrameworkAlgorithm.cs, and all I get is
20201115 18:02:36.874 ERROR:: WorkerThread.<.ctor>b__7_0(): WorkerThread(): exception thrown when running task System.TypeInitializationException: The type initializer for 'Tensorflow.Binding' threw an exception. ---> System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.DllNotFoundException: tensorflow assembly:<unknown assembly> type:<unknown type> member:(null)
at (wrapper managed-to-native) Tensorflow.c_api.TF_NewStatus()
at Tensorflow.Status..ctor () [0x00006] in <f58589f3e4134776b3e4cecbf8fb8b86>:0
at Tensorflow.tensorflow..ctor () [0x0011c] in <f58589f3e4134776b3e4cecbf8fb8b86>:0
Karel Goderis
Nevermind. Forgot to install the macOS C dylib for Tensorflow. Just pasting
https://www.tensorflow.org/install/lang_chere in case someone else is attempting to do the same thing
Karel Goderis
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
To unlock posting to the community forums please complete at least 30% of Boot Camp.
You can continue your Boot Camp training progress from the terminal. We hope to see you in the community soon!