Question of vocab.json

by jinyong - opened Sep 13, 2022

Sep 13, 2022

Why is the vocab like "Ġlast" instead of "last"？

Owner Sep 13, 2022

based on what i saw, tokenizers of BERT, BART module produce that

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment