wav2vec2-xls-r-1b-scandinavian-E4-100h-30-epochs-20250124

This model is a fine-tuned version of facebook/wav2vec2-xls-r-1b on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: nan
• Wer: 100.0
• Cer: 100.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 32
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_steps: 6000
• num_epochs: 30
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
1.267	0.7819	1000	inf	47.6367	13.5610
0.5584	1.5637	2000	inf	26.4059	7.2903
0.4211	2.3456	3000	inf	23.4538	6.4409
0.5473	3.1274	4000	inf	21.6233	6.0101
0.4631	3.9093	5000	inf	21.4830	5.9161
0.2869	4.6912	6000	inf	21.6286	6.0356
0.4572	5.4730	7000	inf	22.4768	6.2442
0.2378	6.2549	8000	inf	21.7372	6.1153
0.3479	7.0367	9000	inf	22.6372	6.2812
0.368	7.8186	10000	inf	23.1700	6.3923
0.5023	8.6005	11000	inf	22.2996	6.2065
0.4695	9.3823	12000	inf	21.2941	5.9487
0.0	10.1642	13000	nan	100.0	100.0

Framework versions

• Transformers 4.48.1
• Pytorch 2.5.1+cu124
• Datasets 3.2.0
• Tokenizers 0.21.0