|
--- |
|
pipeline_tag: automatic-speech-recognition |
|
library_name: transformers |
|
tags: |
|
- wav2vec2 |
|
- speech-recognition |
|
- english-phoneme-recognition |
|
--- |
|
|
|
# Wav2Vec2-Large-Robust ETRI Korean-English Pronunciation Model |
|
|
|
This repository contains a fine-tuned Wav2Vec2-Large-Robust model for phoneme recognition tasks. The model was trained and evaluated on our in-house dataset, English pronunciations of Korean learners made with ETRI. |
|
|
|
## Data Information |
|
- **Dataset Name**: ETRI English Pronunciation of Korean Learners |
|
- **Train Data**: 14,305 samples |
|
- **Valid Data**: 1,590 samples |
|
- **Test Data**: 3,974 samples |
|
|
|
## Training Procedure |
|
The model was fine-tuned for phoneme recognition using the Hugging Face `transformers` library. Below are the training steps: |
|
1. Data preprocessing to align audio with phoneme labels. |
|
2. Wav2Vec2-Large-Robust model fine-tuning with CTC loss. |
|
3. Evaluation on validation and test sets. |
|
|
|
### Training Hyperparameters |
|
- **Epochs**: 50 |
|
- **Learning Rate**: 0.0001 |
|
- **Warmup Ratio**: 0.1 |
|
- **Scheduler**: Linear |
|
- **Batch Size**: 8 |
|
- **Loss Reduction**: Mean |
|
- **Feature Extractor Freeze**: Enabled |
|
|
|
## Training Results |
|
The following metrics were achieved during training: |
|
- **Final Training Loss**: 0.2527 |
|
- **Validation Loss**: 0.4532 |
|
- **Word Error Rate (WER) on Validation Set**: 0.1617 |
|
|
|
## Test Results |
|
The model was evaluated on the test dataset with the following performance: |
|
- **Word Error Rate (WER)**: 0.1223 |
|
|
|
## Phoneme Data Example |
|
Below is an example of how the dataset is structured for phoneme recognition tasks: |
|
|
|
**Sample 1:** |
|
- **Provided Sentence**: The one with the ribbon on its head |
|
- **Correct Korean English Phonemes**: dh ah w ah n w ih dh ax r ih b ah n ao n ih t s hh eh dd |
|
- **Predicted Phonemes**: d ah w ah n w ih dh ah r ih b ah n ao n ih ts hh eh dd |
|
|
|
## Training Logs |
|
TensorBoard logs are available for detailed training analysis: |
|
- `events.out.tfevents.1732529747.oem-WS-C621E-SAGE-Series.2265579.0` |
|
- `events.out.tfevents.1732573537.oem-WS-C621E-SAGE-Series.2265579.1` |
|
|
|
Use the following command to visualize logs: |
|
```bash |
|
tensorboard --logdir=./logs/ |
