I did the following:
$ lean create-project --language python "A Test Python Project"
Successfully created Python project 'A Test Python Project'
$ lean backtest "A Test Python Project"
Error: 400 Client Error for http+docker://localhost/v1.41/containers/create: Bad Request ("invalid mount config for type
"bind": bind source path does not exist: /var/folders/77/sdf7kt8d18ngmv2hq8485_180000gn/T/tmpv7qlygm0/config.json")
$ ls /var/folders/77/sdf7kt8d18ngmv2hq8485_180000gn/T/tmpv7qlygm0/config.json
/var/folders/77/sdf7kt8d18ngmv2hq8485_180000gn/T/tmpv7qlygm0/config.json
The backtest command fails saying a source path does not exist, but if I check that path with "ls" then it's clear it's there. This is on an Intel Mac running Docker desktop 3.3.1, with docker engine 20.10.5. I did "pip install lean" on a freshly created venv in Python 3.8.0.
Thanks for your suggestions!
Jasper van Merle
Hi Jesse,
This sounds like some sort of permission issue. Can you verify Docker has access to /var/folders by opening Docker and going to Settings > Resources > File sharing? It should look like this.
Jesse Pangburn
Hi Jasper,
I agree, that was the first thing I tried too (I didn't have that directory in the list before this). I even rebooted the computer when it didn't work, and it still has the same result.
https://imgur.com/3xiAFNcI also tried --verbose, and it prints the error stacktrace but can't see the docker command it's trying so not sure what else could be going wrong. I checked the settings.json to make sure it's not corrupted in there or something and it shows:
"filesharingDirectories" : [
"\/Users\/jpangburn\/Documents\/amazon_aws\/aws_sam_projects",
"\/Users\/jpangburn\/Documents\/dockervolumes",
"\/tmp",
"\/var\/folders"
],
So it appears like that setting is correct. But no idea what else it could be. Thanks!
Jasper van Merle
Hi Jesse,
That's weird, I'm not able to reproduce this error and don't really know what's going wrong here. Could you try giving Docker full disk access in macOS settings > Security & Privacy > Privacy? Looking at a similar issue on GitHub that may be a potential fix.
Jesse Pangburn
Hi Jasper,
Ah great idea! I had looked at that similar issue as well, but didn't read that last comment. Anyway, I tried it and still didn't work, even with rebooting the computer. Still thinking it's a permission problem, I edited the temp_manager.py to make the temp directory under /tmp
# line 31 path = Path(tempfile.mkdtemp(dir='/tmp'))
But it had the same problem: Error: 400 Client Error for http+docker://localhost/v1.41/containers/create: Bad Request ("invalid mount config for type "bind": bind source path does not exist: /tmp/tmpdcjxmfgc/config.json")
Pretty bizarre as /tmp has been allowed under Docker forever and every user should have access to it. For kicks I made a ~/tmp directory under my user account, and tried that.
path = Path(tempfile.mkdtemp(dir='/Users/jpangburn/tmp'))
This seems to have gotten past that error and found a new one:
Error: 500 Server Error for http+docker://localhost/v1.41/containers/ab48c697a672f5a8afa9da29f9e0f119054cf00ee52c67f1c51667516bc97a20/start: Internal Server Error ("b'Mounts denied: \nThe path
/Users/jpangburn/Documents/vsc_workspaces/quantconnect_test/data is not shared from the host and is not known to Docker.\nYou can configure shared paths from Docker -> Preferences... -> Resources -> File Sharing.\nSee
https://docs.docker.com/docker-for-mac for more info.'")
It appears it's trying to bind a directory underneath of my project dir "/Users/jpangburn/Documents/vsc_workspaces/quantconnect_test". I guess it makes sense since a backtest would need to access that data subdirectory. But the setup guide said nothing about making this directory accessible to Docker's file sharing. I guess in your Docker file sharing screenshot, you have /Users, /Volumes, and /private so that's why you didn't need to explicitly add this one.
I tried adding those 3 directories to my Docker file sharing, then it ran the backtest just fine. I tried it again after removing Full Disk Access permission, and it still ran fine. So seems like one or more of those shares is critical. Unfortunately it also gets rid of the security you get by only allowing a small subset of directories on your machine to be accessed by Docker shared volumes, which is why they have that file sharing config in the first place.
Digging further, it seems you need /private/var/folders rather than /var/folders because /var is really mapped to /private/var. So in summary, for the paranoid who are still reading and who don't want to add /Users, /Volumes, and /private to their shared Docker list, you can add:
/private/var/folders
/Users/<your username>/path/to/project_parent (e.g. /Users/jpangburn/Documents/vsc_workspaces/quantconnect_test where I have my QuantConnect projects under this dir)
Thanks for all your help Jasper, you led me right to this solution! Really excited for local debugging :-) The online thing is good, but the debugging is tough on it when you're new and don't have a good feel for what objects have what properties. The local debugger should make that a lot easier.
Jesse Pangburn
TLDR; for the easiest (but compromises on security) solution add the following to your Docker file sharing:
/private /Users /Volumes
For the paranoid, add:
/private/var/folders /Users/<your username>/path/to/quantconnect (should have the /data child directory and your project directories as children)
Jesse Pangburn
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
To unlock posting to the community forums please complete at least 30% of Boot Camp.
You can continue your Boot Camp training progress from the terminal. We hope to see you in the community soon!