metadata
license: mit
base_model: facebook/w2v-bert-2.0
tags:
- generated_from_trainer
metrics:
- wer
model-index:
- name: malayalam_combined_
results: []
malayalam_combined_
This model is a fine-tuned version of facebook/w2v-bert-2.0 on the None dataset. It achieves the following results on the evaluation set:
- Loss: 0.4712
- Wer: 0.4649
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 50
- num_epochs: 25
- mixed_precision_training: Native AMP
Training results
Training Loss | Epoch | Step | Validation Loss | Wer |
---|---|---|---|---|
0.8201 | 0.2031 | 500 | 0.8317 | 0.6757 |
0.7455 | 0.4063 | 1000 | 0.7271 | 0.6119 |
0.6928 | 0.6094 | 1500 | 0.6823 | 0.6083 |
0.6497 | 0.8125 | 2000 | 0.6775 | 0.5955 |
0.5968 | 1.0156 | 2500 | 0.6554 | 0.5837 |
0.594 | 1.2188 | 3000 | 0.6127 | 0.5772 |
0.558 | 1.4219 | 3500 | 0.6149 | 0.5574 |
0.5906 | 1.6250 | 4000 | 0.5856 | 0.5485 |
0.5846 | 1.8282 | 4500 | 0.5894 | 0.5413 |
0.5504 | 2.0313 | 5000 | 0.5571 | 0.5310 |
0.5059 | 2.2344 | 5500 | 0.5735 | 0.5542 |
0.5019 | 2.4375 | 6000 | 0.5555 | 0.5278 |
0.5274 | 2.6407 | 6500 | 0.5592 | 0.5111 |
0.5113 | 2.8438 | 7000 | 0.5391 | 0.5318 |
0.4624 | 3.0469 | 7500 | 0.5191 | 0.5189 |
0.4696 | 3.2501 | 8000 | 0.5312 | 0.5072 |
0.4866 | 3.4532 | 8500 | 0.5208 | 0.5167 |
0.4817 | 3.6563 | 9000 | 0.5141 | 0.4966 |
0.5053 | 3.8594 | 9500 | 0.5104 | 0.5010 |
0.4541 | 4.0626 | 10000 | 0.5247 | 0.5130 |
0.4761 | 4.2657 | 10500 | 0.5287 | 0.5157 |
0.4385 | 4.4688 | 11000 | 0.5196 | 0.5018 |
0.4698 | 4.6719 | 11500 | 0.5286 | 0.5120 |
0.4631 | 4.8751 | 12000 | 0.5045 | 0.4957 |
0.4269 | 5.0782 | 12500 | 0.5102 | 0.5018 |
0.4253 | 5.2813 | 13000 | 0.5085 | 0.4949 |
0.425 | 5.4845 | 13500 | 0.5209 | 0.4894 |
0.4391 | 5.6876 | 14000 | 0.5037 | 0.4900 |
0.4206 | 5.8907 | 14500 | 0.5265 | 0.4802 |
0.4087 | 6.0938 | 15000 | 0.5044 | 0.4829 |
0.4112 | 6.2970 | 15500 | 0.4962 | 0.4860 |
0.3864 | 6.5001 | 16000 | 0.4823 | 0.4777 |
0.4403 | 6.7032 | 16500 | 0.4898 | 0.4808 |
0.3942 | 6.9064 | 17000 | 0.4821 | 0.4808 |
0.386 | 7.1095 | 17500 | 0.4804 | 0.4789 |
0.3752 | 7.3126 | 18000 | 0.4735 | 0.4662 |
0.3863 | 7.5157 | 18500 | 0.4680 | 0.4662 |
0.3767 | 7.7189 | 19000 | 0.4692 | 0.4610 |
0.3942 | 7.9220 | 19500 | 0.4700 | 0.4721 |
0.3502 | 8.1251 | 20000 | 0.4759 | 0.4742 |
0.3504 | 8.3283 | 20500 | 0.4702 | 0.4653 |
0.3594 | 8.5314 | 21000 | 0.4712 | 0.4649 |
Framework versions
- Transformers 4.43.0.dev0
- Pytorch 1.14.0a0+44dac51
- Datasets 2.16.1
- Tokenizers 0.19.1