jamesdborin
/

InternVL2-2B

Image-Text-to-Text

feature-extraction

Model card Files Files and versions Community

jamesdborin commited on 9 days ago

Commit

08a3b34

·

verified ·

1 Parent(s): 2a582df

Update tokenizer_config.json

Files changed (1) hide show

tokenizer_config.json +1 -1

tokenizer_config.json CHANGED Viewed

@@ -165,7 +165,7 @@
   "auto_map": {
     "AutoTokenizer": [
       "tokenization_internlm2.InternLM2Tokenizer",
-      null
     ]
   },
   "bos_token": "<s>",

   "auto_map": {
     "AutoTokenizer": [
       "tokenization_internlm2.InternLM2Tokenizer",
+      "tokenization_internlm2_fast.InternLM2TokenizerFast"
     ]
   },
   "bos_token": "<s>",