File size: 1,495 Bytes
e80ee52 647c107 e80ee52 647c107 5569b60 647c107 e80ee52 5569b60 e80ee52 5569b60 e80ee52 5569b60 e80ee52 5569b60 e80ee52 5569b60 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 |
---
language:
- es
tags:
- masked-lm
license: gpl-3.0
datasets:
- jorgeortizfuentes/spanish_books
- jorgeortizfuentes/small-chilean-spanish-corpus
pipeline_tag: fill-mask
---
# Tulio
Tulio is a BERT model trained with Chilean Spanish. This model is a fine-tuned version of [dccuchile/bert-base-spanish-wwm-cased](https://huggingface.co/dccuchile/bert-base-spanish-wwm-cased) on Spanish Books and Small Chilean Spanish Corpus.
## Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 20
- eval_batch_size: 20
- seed: 42
- distributed_type: multi-GPU
- num_devices: 2
- total_train_batch_size: 20
- total_eval_batch_size: 20
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 2.0
## Acknowledgments
We are grateful for the servers provided by the Computer Science Department of the University of Chile and the ReLeLa (Representations for Learning and Language) study group for the training of the model.
## License Disclaimer
The license gpl-3.0 best describes our intentions for our work. However we are not sure that all the datasets used to train the model have licenses compatible with gpl-3.0. Please use at your own discretion and verify that the licenses of the original text resources match your needs.
## Limitations
The training dataset was not censored in any way. Therefore, the model may contain unwanted ideological representations. Use with caution.
|