# 🎯 Phonemic Transcription Leaderboard

Welcome to the Phonemic Transcription Leaderboard! This simple leaderboard helps you track and compare the performance of different speech-to-phoneme model. Feel free to use it for your own hugging face leaderboards! 

![leaderboard](img/leaderboard.png)

## ✨ Features

* 📊 Interactive leaderboard with real-time sorting
* 🔄 Easy model submission system
* 📈 Automatic evaluation of submitted models 
* 📱 Responsive design that works on all devices

## 🎯 What This Project Does

This leaderboard tracks two key metrics for phonemic transcription models:


* **PER (Phoneme Error Rate)**: How accurately your model converts speech to phonemes
* **PWED (Phoneme Weighted Edit Distance)**: A more nuanced metric that considers phonemic features

Read more about evaluations on our [blog](https://www.koellabs.com/blog/phonemic-transcription-metrics)

Models are evaluated on the TIMIT speech corpus, a gold standard in speech recognition research.

## 🚀 Getting Started

### Prerequisites

* Python 3.10
* Git
* A love for speech recognition! 🎤

### Quick Installation

1. Clone this repository:

```bash
git clone [your-repo-url]
cd phonemic-leaderboard
```

2. Set up your environment:

```bash
# Create a virtual environment with Python 3.10
python3.10 -m venv venv

# Activate the virtual environment
source venv/bin/activate

# Install the required dependencies
pip install -r requirements.txt
```

3. Launch the leaderboard:

```bash
# Run the application
uvicorn app:app --host 0.0.0.0 --port 7860
```

4. Visit `http://localhost:7860` in your browser and see the magic! ✨

## 🎮 Using the Leaderboard

### Submitting a Model

1. Go to the "Submit Model" tab
2. Enter your model details:
   * Model name (e.g., "wav2vec2-phoneme-wizard")
   * Submission name (e.g., "MyAwesomeModel v1.0")
   * GitHub/Kaggle/HuggingFace URL (optional)
3. Click Submit and watch your model climb the ranks! 🚀

### Checking Model Status

1. Navigate to the "Model Status" tab
2. Enter your model name or task ID
3. Get real-time updates on your model's evaluation progress

## 📊 Understanding the Results

The leaderboard shows:

* Model names and submission details
* PER and PWED scores (lower is better!)
* Links to model repositories
* Submission dates

Sort by either metric to see who's leading the pack!

## 🛠️ Technical Details

* Built with Gradio for a smooth UI experience
* Runs on a basic compute plan (16GB RAM, 2vCPUs) for easy reproducibility
* Evaluation can take several hours - perfect time to grab a coffee ☕

## 🤝 Contributing

Want to make this leaderboard even better? We'd love your help! Here are some ways you can contribute:

* Add new evaluation metrics
* Improve the UI design
* Enhance documentation
* Submit bug fixes
* Add new features

## 📝 License

This project is licensed under the MIT License - see the LICENSE file for details.

## 🌟 Acknowledgments

* Thanks to the TIMIT speech corpus for providing evaluation data
* Shoutout to the [panphon library](https://github.com/dmort27/panphon) for PWED calculations
* Built with love by Koel Labs 💙

## 🆘 Need Help?

Got questions? Found a bug? Want to contribute? Open an issue or reach out to us! We're here to help make speech recognition evaluation fun and accessible for everyone!

Remember: Every great model deserves its moment to shine! 🌟

---

Happy Transcribing! 🎤✨