KE-Blender / special_tokens_map.json
sunminyu's picture
Upload special_tokens_map.json
f750d35
raw
history blame
164 Bytes
{"bos_token": "__start__", "eos_token": "__end__", "unk_token": "__unk__", "pad_token": "__null__", "additional_special_tokens": ["__defi__", "__hype__", "[MASK]"]}