Spaces:

dayannex
/

anonimizador

Sleeping

dayannex commited on Jul 5, 2024

Commit

befc23c

1 Parent(s): 0c893dd

app modified dataset reorden token

Files changed (1) hide show

app.py CHANGED Viewed

@@ -449,7 +449,7 @@ class ModeloDataset:
         for token in tokens:
             if pre_tokens[i]=='O' or 'MISC' in pre_tokens[i]:
-              new_labels.append(' ' +token.replace('▁',''))
             else:
                new_labels.append(' ' + pre_tokens[i])
             i=i+1

         for token in tokens:
             if pre_tokens[i]=='O' or 'MISC' in pre_tokens[i]:
+              new_labels.append(' ' +token.replace('▁','').replace('Ġ',''))
             else:
                new_labels.append(' ' + pre_tokens[i])
             i=i+1