Share New Research 
Hi,
I'm struggling trying to import a TensorFlow model that has been trained on my own server.
I am very experienced with TensorFlow and have been using QC for some time also. It looks like it might be an issue with QC's TF/Keras H5 reader from what I can see. Anyone have any ideas?
def Initialize(self):
self.SetStartDate(2020, 9, 19) # Set Start Date
self.SetCash(100000) # Set Strategy Cash
self.AddEquity("SPY", Resolution.Minute)
self.model = self.Download('model.h5')
self.ai_model = tf.keras.models.load_model(self.model)During the algorithm initialization, the following exception has occurred: UnicodeEncodeError : 'utf-8' codec can't encode characters in position 423-424: surrogates not allowed
at isfile
st = os.stat(path)
File "site-packages/h5py/_hl/base.py" in genericpath.py: line 29
UnicodeEncodeError : 'utf-8' codec can't encode characters in position 423-424: surrogates not allowed
John Rewolinski
I forgot to note that I have tested this model file in 2 other TensorFlow environments (my desktop, and AWS Sagemaker) and got no errors on either platform.
Fred Painchaud
Hi John,
My guess is that you need to put your model file somewhere on the Internet (like Dropbox) so it can be downloaded by the web environment.
By doing self.Download('model.h5'), it is like your model is already local to the web environment… you need a URL…
Fred
John Rewolinski
Hi Fred,
I am downloading it from S3. I replaced my actual link with ‘model.h5’ because I did not want to share my private S3 bucket with the world. Additionally, the error indicates it's receiving the file……and that it can't read a character correctly due to UTF-8/16 encoding issues which are common with virtual environments.
I am more or less nicely saying I think there's an error with the Keras package installed on QuantConnect.
John Rewolinski
I have done a little more research on the issue. I tried decoding then re-encoding the entire model file in ascii as a work around.
It looks like the 1.13.1 version of Tensorflow is angry about the “/x0” null characters in the .H5 File.
Now we get this error:
I have tried both .h5 files and .pb files at this point and getting the same error here and not on other TF/Keras environments.
Varad Kabade
Hi John
Sorry for the inconvenience caused; We have created a GH issue. Subscribe to the following for updates:
https://github.com/QuantConnect/Lean/issues/6205
Alternatively, we can train the model in the research environment, store the weights using ObjectStore and retrieve them inside the backtest. Refer to the following docs and thread for more information regarding saving and loading Ml models with ObjectStore.
Best,
Varad Kabade
John Rewolinski
Varad,
Thanks for your response. I found a workaround to the issue by doing the following:
This fix is however complicated by the fact that TensorFlow saves models as “model.tar.gz” in SageMaker. It appears the “Tarfile.open” function is prohibited for some strange reason on QuantConnect
John Rewolinski
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
To unlock posting to the community forums please complete at least 30% of Boot Camp.
You can continue your Boot Camp training progress from the terminal. We hope to see you in the community soon!