About the encoding type of tokenizer vocab
#1
by
isllmsmart
- opened
Thanks for your work, but I can't find the right decoding type of the tokenizer vocab, e.g. there is no Chinese in it. I'm looking for your answer.
isllmsmart
changed discussion status to
closed