Not possible to build Lean:algorithm docker container

Hey Don Q,

Have tried running Lean for both C# and Python? It may be possible that some files are missing because Lean was not compiled. (refer to this thread)

Please also change

COPY ./Launcher/bin/Release /root/Lean/Launcher/bin/Release

to

COPY ./Launcher/bin/Debug /root/Lean/Launcher/bin/Debug

Let us know if that works or if there are any further issues.

Best
Rahul

Hi Rahul,

What do you mean by "running Lean for both C# and Python?"

I couldn't find any tutorlial or just simple file with steps to follow.

I followed this steps on ubuntu 18.04

1) Installed docker-ce

2) Cloned Lean and build lean:foundation package which os OK. When I cloned lean there is no folder 

When I red this artickel I understood you can run lean directly from container so no compilation is needed "OS Setup: We provide a docker file so you can setup the OS instantly by following along with the docker setup - or running just running LEAN in the docker container we provide."

Hey Don Q,

Lean still needs to be compiled to be able to run. The dockerfile Lean foundation sets up the enviroment on which Lean runs. We still need copy a compiled version of Lean into that enviroment. This is the Dockerfile and this is the DockerfileLeanFoundation.

Best
Rahul

The link in the Dockerfile above doesn't work. I have my own rudimentary container running. I'm using C#. Compiling locally and then mounting/copying running inside of Docker defeats the purpose of using Docker in the first place... I need to look at the command line arguments to be able to pass in the config file instead of it getting build in, but for now... if you create a directory with your algorithm file, a copy of the `QuantConnect.Algorithm.CSharp.csproj`, renamed to "proj.csproj", and your file with your algorithm file in the "includes" section, and then the config.json file, and then use this for the Dockerfile.

For example, my algo test file is called "NabeelTestAlgo.cs", open the proj.csproj file, and add this where the other `<Compile Include="..." />` files are:

Then the Dockerfile:

Then build and run:

This takes advantage of Dockers stage caching, and the csproj editing is required of the dumb way includes are treated by the csproj files. If QuantConnect fixes the build so it can use the .NET Core format, then this hack of having to create a copy of the csproj file will work.

If you use docker-compose, you can mount the Data directories, etc, so that it won't store them in the container.

Right now, I have a solution (I use Rider) which just pulls in the QC libraries from nuget, instead of the full Lean build... which is a pain in the butt to rebase, etc when there are changes. This container is also much more lightweight than the one supplied.

Here, this works, runs a backtest:

FROM mono:6.8.0.96

WORKDIR /opt/Lean

RUN apt-get update && apt-get -y install git nuget
RUN git clone https://github.com/QuantConnect/Lean.git /opt/Lean
RUN nuget restore QuantConnect.Lean.sln

COPY config.json Launcher/

COPY proj.csproj Algorithm.CSharp/QuantConnect.Algorithm.CSharp.csproj
COPY *.cs Algorithm.CSharp/

RUN msbuild QuantConnect.Lean.sln /p:RunTest=false

WORKDIR /opt/Lean/Launcher/bin/Debug/

CMD ["mono", "QuantConnect.Lean.Launcher.exe", "--data-folder", "/opt/Lean/Data", "--config", "/opt/Lean/Launcher/config.json"]

When running the container, you should mount the data and results folders. I use docker-compose, in which I create a local volume and then Docker mounts it.

If there's interest, I can create a repo with a full example and README. This is really lightweight and you don't have to compile anything locally or even have the Lean repo, and should be completely repeatable. 

Created a repo with an example:

Great idea thanks Nabeel, we'll update the dockerfile image to include the build of LEAN. It is currently focused on providing the right runtime environment to facilitate local python debugging. 

Thanks, Jared. I posted a detailed comment on the GitHub issue. It's a little more complicated than that if you have custom source files in C# (might be OK for Python), but I have a request which (I think) should be feasible to enable Docker builds and not have to mess with the csproj file.

Hi Nabeel, I also posted a detailed issue there =) I'll see what we can do this week. 

Got it, I posted to here: 

Hi Nabeel ,

How does your containerized algorithm receive market data?  I'm assuming your container no longer gets market data feed from QuantConnect.  Do you have to have a separate market data subscription from Interactive Brokers for example?

Thanks!

Hi Jon Quant ,

If you select the "live-interactive" environment:

the containerized algorithm should receive market data from Interactive Brokers.