Hello, 

I am trying to import a trained model in PyTorch. It consists of a tokenizer and another model that is very large. However, I started with the tokenizer using the following: -

 

  1. tokenizer = qb.Download("DropBox URL")
  1. import torch
  2. tokenizer_model = torch.load(tokenizer)

 

But I ended with the following error:-

  1. ValueError                                Traceback (most recent call last)
  2. <ipython-input-28-acfbb532fa2a> in <module>
  3.       1 import torch
  4. ----> 2 tokenizer_model = torch.load(tokenizer)
  6. /opt/miniconda3/lib/python3.6/site-packages/torch/serialization.py in load(f, map_location, pickle_module, **pickle_load_args)
  7.     582         pickle_load_args['encoding'] = 'utf-8'
  8.     583 
  9. --> 584     with _open_file_like(f, 'rb') as opened_file:
  10.     585         if _is_zipfile(opened_file):
  11.     586             with _open_zipfile_reader(f) as opened_zipfile:
  13. /opt/miniconda3/lib/python3.6/site-packages/torch/serialization.py in _open_file_like(name_or_buffer, mode)
  14.     232 def _open_file_like(name_or_buffer, mode):
  15.     233     if _is_path(name_or_buffer):
  16. --> 234         return _open_file(name_or_buffer, mode)
  17.     235     else:
  18.     236         if 'w' in mode:
  20. /opt/miniconda3/lib/python3.6/site-packages/torch/serialization.py in __init__(self, name, mode)
  21.     213 class _open_file(_opener):
  22.     214     def __init__(self, name, mode):
  23. --> 215         super(_open_file, self).__init__(open(name, mode))
  24.     216 
  25.     217     def __exit__(self, *args):
  27. ValueError: embedded null byte
+ Expand

I found an old thread, but unfortunately the author did not share the exact solution he found.

Author

Adham Al-Harazi

June 2022