metadata

license: mit
base_model: facebook/w2v-bert-2.0
tags:
  - generated_from_trainer
metrics:
  - wer
model-index:
  - name: malayalam_combined_
    results: []

malayalam_combined_

This model is a fine-tuned version of facebook/w2v-bert-2.0 on the None dataset. It achieves the following results on the evaluation set:

Loss: 0.5153
Wer: 0.5077

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5e-05
train_batch_size: 16
eval_batch_size: 16
seed: 42
gradient_accumulation_steps: 2
total_train_batch_size: 32
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
lr_scheduler_warmup_steps: 50
num_epochs: 25
mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
0.8243	0.2031	500	0.8413	0.6658
0.7336	0.4063	1000	0.7351	0.6251
0.6824	0.6094	1500	0.6786	0.5956
0.6489	0.8125	2000	0.6836	0.6075
0.585	1.0156	2500	0.6295	0.5864
0.5917	1.2188	3000	0.6166	0.5579
0.56	1.4219	3500	0.6006	0.5646
0.5736	1.6250	4000	0.6268	0.5643
0.5821	1.8282	4500	0.6216	0.5786
0.5505	2.0313	5000	0.5705	0.5379
0.5065	2.2344	5500	0.5864	0.5460
0.5004	2.4375	6000	0.5555	0.5259
0.5327	2.6407	6500	0.5539	0.5255
0.5148	2.8438	7000	0.5584	0.5457
0.4751	3.0469	7500	0.5389	0.5208
0.4779	3.2501	8000	0.5284	0.5102
0.4874	3.4532	8500	0.5300	0.5084
0.4955	3.6563	9000	0.5248	0.5125
0.4961	3.8594	9500	0.5116	0.5061
0.4449	4.0626	10000	0.5257	0.5122
0.48	4.2657	10500	0.5254	0.5046
0.4513	4.4688	11000	0.5364	0.5232
0.4698	4.6719	11500	0.5293	0.5106
0.4674	4.8751	12000	0.5153	0.5077

Framework versions

Transformers 4.43.0.dev0
Pytorch 1.14.0a0+44dac51
Datasets 2.16.1
Tokenizers 0.19.1