First of all, when I import a library that I created (including the torch imports and architecture), I start getting the following error: "AttributeError: Oanda". I have seen that other people also experienced this error due to some imports:
https://www.quantconnect.com/forum/discussion/4742/addforex-market-oanda/p1/comment-13833Apart from that, I would like to know how I can download and load an external PyTorch model. I have downloaded the file with Python requests but wasn't able to load the model with the same name that I have utilized for downloading. Then, I have tried self.Download, which is existing in some examples, but getting "File name too long" error while trying to load that into a PyTorch model. I guess that the variable to which I assigned self.Download output doesn't contain the filename. Hence, my question is how I can download an external file from Dropbox and then use that for loading to a PyTorch model as follows:
file = self.Download("https://www.dropbox.com/.../best_model.pt?dl=0")
model.load_state_dict(torch.load(file))
model.eval()
Kamer Ali Yüksel
Note: I also basically need the same for loading a pretrained and pickled class from sklearn.preprocessing.
Gabriel Moncarz
Hi Kamer. Attached you have a sample generic code about how you can read a pickle file stored on Dropbox. The pickle sample file I created just contains a dictionary, as it is an example. You could add more complex objects there.
Regarding PyTorch, I have no experience at all with that library. Nevertheless, I suggest you to train your model with exactly the same version that QuantConnect is using. You could check the version on DockerfileLeanFoundation.
https://github.com/QuantConnect/Lean/blob/master/DockerfileLeanFoundationExactly the same applies to your sklearn.preprocessing classes. Now, I am not sure if there's limit on the size of file you can get with requests.
Kamer Ali Yüksel
Thank you very much Gabriel, and have a great weekend!!! Your solution did not work neither unfortunately but at least gave me a proposal (through io.BytesIO) that I can investigate further.
AttributeError : 'bytes' object has no attribute 'seek'. You can only torch.load from a file that is seekable. Please pre-load the data into a buffer like io.BytesIO and try to load from it instead.
Kamer Ali Yüksel
Ok, I managed to solve the issue as follows: model.load_state_dict(torch.load(io.BytesIO(req.content)))
But it gave an UnpicklingError, probably due to older PyTorch version as you have guessed in advance!
Kamer Ali Yüksel
Lean actually has PyTorch 1.0.0, that I am also using, please let me know in case of ideas on UnpicklingError.
Kamer Ali Yüksel
I saved the model with Python 3.6.8 and PyTorch 1.0.1.post2 (very close versions to what exist in the Lean).
Kamer Ali Yüksel
Resolved. I found the solution by developing an API for my PyTorch model and interacting with that, rather than trying to download and load the PyTorch model to the platform.
Kamer Ali Yüksel
API version shared in the following issue, I can also share the Flask server code for the model if required by anybody:
https://www.quantconnect.com/forum/discussion/5428/converting-portfolio-balancing-classic-algorithm-to-a-framework-algorithm/p1Kamer Ali Yüksel
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
To unlock posting to the community forums please complete at least 30% of Boot Camp.
You can continue your Boot Camp training progress from the terminal. We hope to see you in the community soon!