LING575-WIN21-Reorder / added_tokens.json
junyinc's picture
Upload tokenizer
89548a3
raw
history blame contribute delete
413 Bytes
{
"<S0>": 50268,
"<S10>": 50278,
"<S11>": 50279,
"<S12>": 50280,
"<S13>": 50281,
"<S14>": 50282,
"<S15>": 50283,
"<S16>": 50284,
"<S17>": 50285,
"<S18>": 50286,
"<S19>": 50287,
"<S1>": 50269,
"<S2>": 50270,
"<S3>": 50271,
"<S4>": 50272,
"<S5>": 50273,
"<S6>": 50274,
"<S7>": 50275,
"<S8>": 50276,
"<S9>": 50277,
"<eos>": 50267,
"[orig]": 50266,
"[shuffled]": 50265
}