fix fast tokenizer for transformers >= "from_slow": false, f340f48 verified Zymrael commited on Mar 8, 2024
Update tokenizer class in tokenizer_config,json to llama tokenizer. f2b1452 pragaash commited on Dec 7, 2023