---
library_name: transformers
license: apache-2.0
base_model: facebook/wav2vec2-xls-r-1b
tags:
- generated_from_trainer
metrics:
- wer
model-index:
- name: wav2vec2-xls-r-1b-scandinavian-E3-100h-30-epochs-20250123
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# wav2vec2-xls-r-1b-scandinavian-E3-100h-30-epochs-20250123

This model is a fine-tuned version of [facebook/wav2vec2-xls-r-1b](https://huggingface.co/facebook/wav2vec2-xls-r-1b) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 0.2819
- Wer: 21.6559
- Cer: 5.6618

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 32
- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: cosine
- lr_scheduler_warmup_steps: 6000
- num_epochs: 30
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch   | Step  | Validation Loss | Wer     | Cer     |
|:-------------:|:-------:|:-----:|:---------------:|:-------:|:-------:|
| 1.0778        | 0.6307  | 1000  | 0.6403          | 50.3070 | 15.8896 |
| 0.5579        | 1.2611  | 2000  | 0.3271          | 33.1503 | 9.3525  |
| 0.4943        | 1.8918  | 3000  | 0.2798          | 29.3342 | 8.2452  |
| 0.364         | 2.5222  | 4000  | 0.2706          | 29.1800 | 8.1400  |
| 0.3936        | 3.1526  | 5000  | 0.2704          | 29.8631 | 8.2974  |
| 0.3917        | 3.7833  | 6000  | 0.2831          | 30.0634 | 8.6321  |
| 0.401         | 4.4137  | 7000  | 0.2827          | 30.6442 | 8.7321  |
| 0.3556        | 5.0442  | 8000  | 0.2810          | 29.5504 | 8.3789  |
| 0.3758        | 5.6749  | 9000  | 0.2649          | 29.0661 | 8.2426  |
| 0.2998        | 6.3053  | 10000 | 0.2632          | 29.1123 | 8.1817  |
| 0.309         | 6.9360  | 11000 | 0.2585          | 28.3773 | 7.9461  |
| 0.327         | 7.5664  | 12000 | 0.2638          | 29.0359 | 8.1756  |
| 0.3288        | 8.1968  | 13000 | 0.2584          | 28.9768 | 8.0346  |
| 0.3118        | 8.8275  | 14000 | 0.2536          | 28.0905 | 7.8639  |
| 0.2333        | 9.4579  | 15000 | 0.2543          | 27.0745 | 7.5489  |
| 0.2406        | 10.0883 | 16000 | 0.2584          | 27.0399 | 7.6065  |
| 0.2323        | 10.7190 | 17000 | 0.2525          | 26.9203 | 7.4725  |
| 0.2726        | 11.3494 | 18000 | 0.2438          | 26.8165 | 7.4168  |
| 0.2361        | 11.9801 | 19000 | 0.2442          | 26.4202 | 7.3170  |
| 0.2217        | 12.6105 | 20000 | 0.2416          | 25.8078 | 7.0961  |
| 0.2006        | 13.2409 | 21000 | 0.2481          | 25.5728 | 7.0706  |
| 0.1744        | 13.8716 | 22000 | 0.2375          | 25.5901 | 6.9779  |
| 0.1803        | 14.5020 | 23000 | 0.2407          | 25.4806 | 6.9786  |
| 0.1835        | 15.1325 | 24000 | 0.2390          | 25.3941 | 6.9559  |
| 0.1835        | 15.7632 | 25000 | 0.2398          | 24.9474 | 6.8212  |
| 0.1356        | 16.3936 | 26000 | 0.2371          | 24.1332 | 6.5554  |
| 0.1275        | 17.0240 | 27000 | 0.2414          | 24.2600 | 6.5575  |
| 0.1354        | 17.6547 | 28000 | 0.2421          | 23.9026 | 6.4959  |
| 0.1377        | 18.2851 | 29000 | 0.2541          | 23.6561 | 6.3947  |
| 0.1245        | 18.9158 | 30000 | 0.2326          | 23.4256 | 6.3225  |
| 0.1146        | 19.5462 | 31000 | 0.2477          | 22.9918 | 6.1989  |
| 0.0842        | 20.1766 | 32000 | 0.2480          | 22.9082 | 6.1661  |
| 0.0822        | 20.8073 | 33000 | 0.2508          | 22.6992 | 6.0991  |
| 0.0813        | 21.4377 | 34000 | 0.2546          | 22.6776 | 6.0490  |
| 0.0876        | 22.0681 | 35000 | 0.2610          | 22.4600 | 5.9675  |
| 0.0906        | 22.6988 | 36000 | 0.2623          | 22.3274 | 5.9101  |
| 0.0755        | 23.3292 | 37000 | 0.2632          | 22.3750 | 5.8953  |
| 0.0734        | 23.9599 | 38000 | 0.2637          | 22.0378 | 5.8312  |
| 0.0665        | 24.5904 | 39000 | 0.2690          | 21.9455 | 5.7953  |
| 0.0628        | 25.2208 | 40000 | 0.2816          | 21.8274 | 5.7466  |
| 0.068         | 25.8515 | 41000 | 0.2767          | 21.8245 | 5.7178  |
| 0.0637        | 26.4819 | 42000 | 0.2801          | 21.7755 | 5.7103  |
| 0.0535        | 27.1123 | 43000 | 0.2787          | 21.7423 | 5.6853  |
| 0.0523        | 27.7430 | 44000 | 0.2833          | 21.6746 | 5.6771  |
| 0.0493        | 28.3734 | 45000 | 0.2822          | 21.6587 | 5.6658  |
| 0.0576        | 29.0038 | 46000 | 0.2805          | 21.6414 | 5.6632  |
| 0.0559        | 29.6345 | 47000 | 0.2819          | 21.6559 | 5.6618  |


### Framework versions

- Transformers 4.48.1
- Pytorch 2.5.1+cu124
- Datasets 3.2.0
- Tokenizers 0.21.0