Unable to load the tokenizer
I need to serve the mixtral 8x22b through vLLM but the tokenizer is not getting loaded properly getting the error as:
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained('/usr/src/app/mixtral8x22b/mixtral8x22b')
Traceback (most recent call last):
File "", line 1, in
File "/usr/local/lib/python3.12/dist-packages/transformers/models/auto/tokenization_auto.py", line 844, in from_pretrained
tokenizer_config = get_tokenizer_config(pretrained_model_name_or_path, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/transformers/models/auto/tokenization_auto.py", line 698, in get_tokenizer_config
result = json.load(reader)
^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/json/init.py", line 293, in load
return loads(fp.read(),
^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/json/init.py", line 346, in loads
return _default_decoder.decode(s)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)