test set specificaiton

README-github.md

@@ -0,0 +1,124 @@
+# 🎯 Phonemic Transcription Leaderboard
+
+Welcome to the Phonemic Transcription Leaderboard! This simple leaderboard helps you track and compare the performance of different speech-to-phoneme model. Feel free to use it for your own hugging face leaderboards! 
+
+![leaderboard](img/leaderboard.png)
+
+## ✨ Features
+
+* 📊 Interactive leaderboard with real-time sorting
+* 🔄 Easy model submission system
+* 📈 Automatic evaluation of submitted models 
+* 📱 Responsive design that works on all devices
+
+## 🎯 What This Project Does
+
+This leaderboard tracks two key metrics for phonemic transcription models:
+
+* **PER (Phoneme Error Rate)**: How accurately your model converts speech to phonemes
+* **PWED (Phoneme Weighted Edit Distance)**: A more nuanced metric that considers phonemic features
+
+Models are evaluated on the TIMIT speech corpus, a gold standard in speech recognition research.
+
+## 🚀 Getting Started
+
+### Prerequisites
+
+* Python 3.10
+* Git
+* A love for speech recognition! 🎤
+
+### Quick Installation
+
+1. Clone this repository:
+
+```bash
+git clone [your-repo-url]
+cd phonemic-leaderboard
+```
+
+2. Set up your environment:
+
+```bash
+# Create a virtual environment with Python 3.10
+python3.10 -m venv venv
+
+# Activate the virtual environment
+source venv/bin/activate
+
+# Install the required dependencies
+pip install -r requirements.txt
+```
+
+3. Launch the leaderboard:
+
+```bash
+# Run the application
+uvicorn app:app --host 0.0.0.0 --port 7860
+```
+
+4. Visit `http://localhost:7860` in your browser and see the magic! ✨
+
+## 🎮 Using the Leaderboard
+
+### Submitting a Model
+
+1. Go to the "Submit Model" tab
+2. Enter your model details:
+   * Model name (e.g., "wav2vec2-phoneme-wizard")
+   * Submission name (e.g., "MyAwesomeModel v1.0")
+   * GitHub/Kaggle/HuggingFace URL (optional)
+3. Click Submit and watch your model climb the ranks! 🚀
+
+### Checking Model Status
+
+1. Navigate to the "Model Status" tab
+2. Enter your model name or task ID
+3. Get real-time updates on your model's evaluation progress
+
+## 📊 Understanding the Results
+
+The leaderboard shows:
+
+* Model names and submission details
+* PER and PWED scores (lower is better!)
+* Links to model repositories
+* Submission dates
+
+Sort by either metric to see who's leading the pack!
+
+## 🛠️ Technical Details
+
+* Built with Gradio for a smooth UI experience
+* Runs on a basic compute plan (16GB RAM, 2vCPUs) for easy reproducibility
+* Evaluation can take several hours - perfect time to grab a coffee ☕
+
+## 🤝 Contributing
+
+Want to make this leaderboard even better? We'd love your help! Here are some ways you can contribute:
+
+* Add new evaluation metrics
+* Improve the UI design
+* Enhance documentation
+* Submit bug fixes
+* Add new features
+
+## 📝 License
+
+This project is licensed under the MIT License - see the LICENSE file for details.
+
+## 🌟 Acknowledgments
+
+* Thanks to the TIMIT speech corpus for providing evaluation data
+* Shoutout to the [panphon library](https://github.com/dmort27/panphon) for PWED calculations
+* Built with love by Koel Labs 💙
+
+## 🆘 Need Help?
+
+Got questions? Found a bug? Want to contribute? Open an issue or reach out to us! We're here to help make speech recognition evaluation fun and accessible for everyone!
+
+Remember: Every great model deserves its moment to shine! 🌟
+
+---
+
+Happy Transcribing! 🎤✨

main.py

@@ -298,7 +298,7 @@ class StorageManager:
 class EvaluationRequest(BaseModel):
     """Request model for TIMIT evaluation"""
     transcription_model: str
-    subset: str = "test"
+    subset: str = "timit-test"
     max_samples: Optional[int] = None
     submission_name: str
     github_url: Optional[str] = None

queue/leaderboard.json

@@ -5,7 +5,7 @@
         "model": "facebook/wav2vec2-lv-60-espeak-cv-ft",
         "average_per": 0.33667301260691423,
         "average_pwed": 0.1276725657099669,
-        "subset": "test",
+        "subset": "timit-test",
         "github_url": "https://github.com/facebookresearch/fairseq/blob/main/examples/wav2vec/README.md",
         "submission_date": "2024-12-05T07:32:06.850230"
     },
@@ -15,7 +15,7 @@
         "model": "KoelLabs/xlsr-timit-b0",
         "average_per": 0.12572285528714347,
         "average_pwed": 0.06476636812791145,
-        "subset": "test",
+        "subset": "timit-test",
         "github_url": "https://github.com/KoelLabs/",
         "submission_date": "2024-12-05T08:25:24.982477"
     },
@@ -25,7 +25,7 @@
         "model": "speech31/wav2vec2-large-TIMIT-IPA",
         "average_per": 0.4415425496841929,
         "average_pwed": 0.18625930002594002,
-        "subset": "test",
+        "subset": "timit-test",
         "github_url": "https://huggingface.co/speech31/wav2vec2-large-TIMIT-IPA2",
         "submission_date": "2024-12-05T09:36:14.570315"
     },
@@ -35,7 +35,7 @@
         "model": "Jubliano/wav2vec2-large-xls-r-300m-ipa-INTERNATIONAL1.5",
         "average_per": 0.6318471187460027,
         "average_pwed": 0.222932144739126,
-        "subset": "test",
+        "subset": "timit-test",
         "github_url": "https://huggingface.co/Jubliano/wav2vec2-large-xls-r-300m-ipa-INTERNATIONAL1.5WithoutSpaces/tree/d5312009d8e620b183c334dfdd9ffc6b4f06f8c1",
         "submission_date": "2024-12-05T10:17:21.334530"
     },
@@ -45,7 +45,7 @@
         "model": "facebook/wav2vec2-xlsr-53-espeak-cv-ft",
         "average_per": 0.348845592557092,
         "average_pwed": 0.1386742019529415,
-        "subset": "test",
+        "subset": "timit-test",
         "github_url": "https://github.com/facebookresearch/fairseq/blob/main/examples/wav2vec/README.md",
         "submission_date": "2024-12-05T10:34:26.157054"
     },
@@ -55,7 +55,7 @@
         "model": "ginic/hyperparam_tuning_1_wav2vec2-large-xlsr-buckeye-ipa",
         "average_per": 0.2766466385175833,
         "average_pwed": 0.10410683992600853,
-        "subset": "test",
+        "subset": "timit-test",
         "github_url": "https://huggingface.co/ginic/vary_individuals_old_only_1_wav2vec2-large-xlsr-buckeye-ipa",
         "submission_date": "2024-12-05T11:06:07.984825"
     },
@@ -65,7 +65,7 @@
         "model": "KoelLabs/xlsr-timit-a0",
         "average_per": 0.24242141955346685,
         "average_pwed": 0.17395311976938,
-        "subset": "test",
+        "subset": "timit-test",
         "github_url": "https://github.com/KoelLabs/ML/",
         "submission_date": "2024-12-12T16:07:25.391145"
     },
@@ -75,7 +75,7 @@
         "model": "snu-nia-12/wav2vec2-large_nia12_phone-ipa_english",
         "average_per": 0.4847029843149011,
         "average_pwed": 0.2072006544586948,
-        "subset": "test",
+        "subset": "timit-test",
         "github_url": null,
         "submission_date": "2024-12-18T22:01:20.855881"
     },
@@ -85,7 +85,7 @@
         "model": "vitouphy/wav2vec2-xls-r-300m-timit-phoneme",
         "average_per": 0.2561961414705681,
         "average_pwed": 0.1378394393452702,
-        "subset": "test",
+        "subset": "timit-test",
         "github_url": "https://www.kaggle.com/code/vitouphy/phoneme-recognition-with-wav2vec2",
         "submission_date": "2024-12-18T22:50:59.627338"
     },
@@ -95,7 +95,7 @@
         "model": "ctaguchi/wav2vec2-large-xlsr-japlmthufielta-ipa-plus-2000",
         "average_per": 0.6479484324708775,
         "average_pwed": 0.18710002665151734,
-        "subset": "test",
+        "subset": "timit-test",
         "github_url": "https://huggingface.co/ctaguchi/wav2vec2-large-xlsr-japlmthufielta-ipa1000-ns",
         "submission_date": "2024-12-18T23:29:27.322286"
     },
@@ -105,7 +105,7 @@
         "model": "ctaguchi/wav2vec2-large-xlsr-japlmthufielta-ipa1000-ns",
         "average_per": 0.6417205190285036,
         "average_pwed": 0.19048963968896404,
-        "subset": "test",
+        "subset": "timit-test",
         "github_url": "https://huggingface.co/ctaguchi/wav2vec2-large-xlsr-japlmthufielta-ipa1000-ns",
         "submission_date": "2024-12-19T07:41:18.135985"
     },
@@ -115,7 +115,7 @@
         "model": "ginic/gender_split_70_female_4_wav2vec2-large-xlsr-buckeye-ipa",
         "average_per": 0.2810165988557621,
         "average_pwed": 0.10703377161801164,
-        "subset": "test",
+        "subset": "timit-test",
         "github_url": "https://github.com/ginic/multipa/tree/buckeye_experiments",
         "submission_date": "2024-12-20T13:45:52.010575"
     },
@@ -125,7 +125,7 @@
         "model": "Jubliano/wav2vec2-large-xls-r-300m-ipa-INTERNATIONAL1.9.2WithoutSpaces",
         "average_per": 0.9537775908999574,
         "average_pwed": 0.9351204819224959,
-        "subset": "test",
+        "subset": "timit-test",
         "github_url": "https://huggingface.co/Jubliano/wav2vec2-large-xls-r-300m-ipa-INTERNATIONAL1.5WithoutSpaces",
         "submission_date": "2024-12-20T14:21:32.293694"
     },
@@ -135,7 +135,7 @@
         "model": "Jubliano/wav2vec2-large-xls-r-300m-ipa-nl",
         "average_per": 0.9887075544197294,
         "average_pwed": 0.9692486915717254,
-        "subset": "test",
+        "subset": "timit-test",
         "github_url": "https://huggingface.co/Jubliano/wav2vec2-large-xls-r-300m-ipa-nl1.1",
         "submission_date": "2024-12-20T15:40:51.632895"
     },
@@ -145,7 +145,7 @@
         "model": "speech31/wavlm-large-english-ipa",
         "average_per": 0.3694017596969614,
         "average_pwed": 0.1356824900612308,
-        "subset": "test",
+        "subset": "timit-test",
         "github_url": "https://huggingface.co/speech31/wavlm-large-english-ipa",
         "submission_date": "2024-12-20T16:26:47.982209"
     },
@@ -155,7 +155,7 @@
         "model": "speech31/XLS-R-300m-english-ipa",
         "average_per": 0.36382554692045954,
         "average_pwed": 0.1299702312124616,
-        "subset": "test",
+        "subset": "timit-test",
         "github_url": "https://huggingface.co/speech31/XLS-R-300m-english-ipa",
         "submission_date": "2024-12-20T16:47:54.826509"
     },
@@ -165,7 +165,7 @@
         "model": "speech31/wav2vec2-large-english-TIMIT-phoneme_v3",
         "average_per": 0.44563344149564776,
         "average_pwed": 0.18844914029048124,
-        "subset": "test",
+        "subset": "timit-test",
         "github_url": "https://huggingface.co/speech31/wav2vec2-large-english-TIMIT-phoneme_v3",
         "submission_date": "2024-12-20T17:05:35.213738"
     },
@@ -175,7 +175,7 @@
         "model": "speech31/wav2vec2-large-TIMIT-IPA2",
         "average_per": 0.4847029843149011,
         "average_pwed": 0.2072006544586948,
-        "subset": "test",
+        "subset": "timit-test",
         "github_url": "https://huggingface.co/speech31/wav2vec2-large-TIMIT-IPA2",
         "submission_date": "2024-12-20T22:50:50.645178"
     },
@@ -185,7 +185,7 @@
         "model": "ginic/vary_individuals_young_only_3_wav2vec2-large-xlsr-buckeye-ipa",
         "average_per": 0.2807914104790719,
         "average_pwed": 0.10494355278037441,
-        "subset": "test",
+        "subset": "timit-test",
         "github_url": "https://huggingface.co/ginic/vary_individuals_young_only_3_wav2vec2-large-xlsr-buckeye-ipa",
         "submission_date": "2024-12-21T01:31:04.862397"
     }

queue/results.json

@@ -2,7 +2,7 @@
     {
         "task_id": "721b4c64-a825-42d3-bb0a-bdff9ee1ed0f",
         "model": "facebook/wav2vec2-lv-60-espeak-cv-ft",
-        "subset": "test",
+        "subset": "timit-test",
         "num_files": 1680,
         "average_per": 0.33667301260691423,
         "average_pwed": 0.1276725657099669,
@@ -48,7 +48,7 @@
     {
         "task_id": "d6fe0956-b5b4-4105-835e-8dee1872ee4d",
         "model": "KoelLabs/xlsr-timit-b0",
-        "subset": "test",
+        "subset": "timit-test",
         "num_files": 1680,
         "average_per": 0.12572285528714347,
         "average_pwed": 0.06476636812791145,
@@ -94,7 +94,7 @@
     {
         "task_id": "dbf4642a-fb13-402c-8a74-cc41fc4be599",
         "model": "speech31/wav2vec2-large-TIMIT-IPA",
-        "subset": "test",
+        "subset": "timit-test",
         "num_files": 1680,
         "average_per": 0.4415425496841929,
         "average_pwed": 0.18625930002594002,
@@ -140,7 +140,7 @@
     {
         "task_id": "912449a4-d7ed-4af4-b5be-5c2c57ec09ff",
         "model": "Jubliano/wav2vec2-large-xls-r-300m-ipa-INTERNATIONAL1.5",
-        "subset": "test",
+        "subset": "timit-test",
         "num_files": 1680,
         "average_per": 0.6318471187460027,
         "average_pwed": 0.222932144739126,
@@ -186,7 +186,7 @@
     {
         "task_id": "c79df17e-2bb2-4253-ae26-f7cc6ab21265",
         "model": "facebook/wav2vec2-xlsr-53-espeak-cv-ft",
-        "subset": "test",
+        "subset": "timit-test",
         "num_files": 1680,
         "average_per": 0.348845592557092,
         "average_pwed": 0.1386742019529415,
@@ -232,7 +232,7 @@
     {
         "task_id": "f36060e6-a746-44dc-a527-54995b270053",
         "model": "ginic/hyperparam_tuning_1_wav2vec2-large-xlsr-buckeye-ipa",
-        "subset": "test",
+        "subset": "timit-test",
         "num_files": 1680,
         "average_per": 0.2766466385175833,
         "average_pwed": 0.10410683992600853,
@@ -278,7 +278,7 @@
     {
         "task_id": "47d56349-8111-4bda-a47f-e007dbedd36d",
         "model": "KoelLabs/xlsr-timit-a0",
-        "subset": "test",
+        "subset": "timit-test",
         "num_files": 1680,
         "average_per": 0.24242141955346685,
         "average_pwed": 0.17395311976938,
@@ -324,7 +324,7 @@
     {
         "task_id": "51dd5735-63bd-4fe5-a588-c0fc079076e0",
         "model": "KoelLabs/xlsr-timit-a0",
-        "subset": "test",
+        "subset": "timit-test",
         "num_files": 1680,
         "average_per": 0.24242141955346685,
         "average_pwed": 0.17395311976938,
@@ -370,7 +370,7 @@
     {
         "task_id": "2e592612-ca38-4afb-a6a0-3c870b288960",
         "model": "snu-nia-12/wav2vec2-large_nia12_phone-ipa_english",
-        "subset": "test",
+        "subset": "timit-test",
         "num_files": 1680,
         "average_per": 0.4847029843149011,
         "average_pwed": 0.2072006544586948,
@@ -416,7 +416,7 @@
     {
         "task_id": "d38e65ce-75b5-4dbf-8ade-bff6a5803790",
         "model": "vitouphy/wav2vec2-xls-r-300m-timit-phoneme",
-        "subset": "test",
+        "subset": "timit-test",
         "num_files": 1680,
         "average_per": 0.2561961414705681,
         "average_pwed": 0.1378394393452702,
@@ -462,7 +462,7 @@
     {
         "task_id": "2839c0c6-8f3b-426e-9eb7-04b6e133dc47",
         "model": "ctaguchi/wav2vec2-large-xlsr-japlmthufielta-ipa-plus-2000",
-        "subset": "test",
+        "subset": "timit-test",
         "num_files": 1680,
         "average_per": 0.6479484324708775,
         "average_pwed": 0.18710002665151734,
@@ -508,7 +508,7 @@
     {
         "task_id": "59afc37a-0072-44dd-a02a-0cf47d89c120",
         "model": "ctaguchi/wav2vec2-large-xlsr-japlmthufielta-ipa1000-ns",
-        "subset": "test",
+        "subset": "timit-test",
         "num_files": 1680,
         "average_per": 0.6417205190285036,
         "average_pwed": 0.19048963968896404,
@@ -554,7 +554,7 @@
     {
         "task_id": "5517f6b2-6a76-4a2d-a6ce-33446f390c3b",
         "model": "ginic/gender_split_70_female_4_wav2vec2-large-xlsr-buckeye-ipa",
-        "subset": "test",
+        "subset": "timit-test",
         "num_files": 1680,
         "average_per": 0.2810165988557621,
         "average_pwed": 0.10703377161801164,
@@ -600,7 +600,7 @@
     {
         "task_id": "c2139f96-e79e-4f25-a525-aa039f65555f",
         "model": "Jubliano/wav2vec2-large-xls-r-300m-ipa-INTERNATIONAL1.9.2WithoutSpaces",
-        "subset": "test",
+        "subset": "timit-test",
         "num_files": 1680,
         "average_per": 0.9537775908999574,
         "average_pwed": 0.9351204819224959,
@@ -646,7 +646,7 @@
     {
         "task_id": "d146f1f1-6e6e-4b28-9420-c652ae9a1002",
         "model": "Jubliano/wav2vec2-large-xls-r-300m-ipa-nl",
-        "subset": "test",
+        "subset": "timit-test",
         "num_files": 1680,
         "average_per": 0.9887075544197294,
         "average_pwed": 0.9692486915717254,
@@ -692,7 +692,7 @@
     {
         "task_id": "265c5859-e7ba-492d-a6c9-45733dc17c99",
         "model": "Jubliano/wav2vec2-large-xls-r-300m-ipa-nl",
-        "subset": "test",
+        "subset": "timit-test",
         "num_files": 1680,
         "average_per": 0.9887075544197294,
         "average_pwed": 0.9692486915717254,
@@ -738,7 +738,7 @@
     {
         "task_id": "e297dfde-95e5-462b-a6e5-8fa43bc30bc0",
         "model": "speech31/wavlm-large-english-ipa",
-        "subset": "test",
+        "subset": "timit-test",
         "num_files": 1680,
         "average_per": 0.3694017596969614,
         "average_pwed": 0.1356824900612308,
@@ -784,7 +784,7 @@
     {
         "task_id": "efe95f71-05e3-485d-8e0c-1823a3037cf4",
         "model": "speech31/wavlm-large-english-ipa",
-        "subset": "test",
+        "subset": "timit-test",
         "num_files": 1680,
         "average_per": 0.3694017596969614,
         "average_pwed": 0.1356824900612308,
@@ -830,7 +830,7 @@
     {
         "task_id": "4b2ae2fc-fe2f-4f8b-9e8f-25c0bae13c0d",
         "model": "speech31/XLS-R-300m-english-ipa",
-        "subset": "test",
+        "subset": "timit-test",
         "num_files": 1680,
         "average_per": 0.36382554692045954,
         "average_pwed": 0.1299702312124616,
@@ -876,7 +876,7 @@
     {
         "task_id": "33d387c0-703c-415d-b8e2-81cea87a2146",
         "model": "speech31/wav2vec2-large-english-TIMIT-phoneme_v3",
-        "subset": "test",
+        "subset": "timit-test",
         "num_files": 1680,
         "average_per": 0.44563344149564776,
         "average_pwed": 0.18844914029048124,
@@ -922,7 +922,7 @@
     {
         "task_id": "c89bcefc-3884-435a-a54c-24297fe6f041",
         "model": "speech31/wav2vec2-large-TIMIT-IPA2",
-        "subset": "test",
+        "subset": "timit-test",
         "num_files": 1680,
         "average_per": 0.4847029843149011,
         "average_pwed": 0.2072006544586948,
@@ -968,7 +968,7 @@
     {
         "task_id": "81fa94f8-94ae-4601-952c-24abaddaf691",
         "model": "ginic/vary_individuals_young_only_3_wav2vec2-large-xlsr-buckeye-ipa",
-        "subset": "test",
+        "subset": "timit-test",
         "num_files": 1680,
         "average_per": 0.2807914104790719,
         "average_pwed": 0.10494355278037441,

queue/tasks.json

@@ -2,7 +2,7 @@
     {
         "id": "721b4c64-a825-42d3-bb0a-bdff9ee1ed0f",
         "model": "facebook/wav2vec2-lv-60-espeak-cv-ft",
-        "subset": "test",
+        "subset": "timit-test",
         "submission_name": "facebook espeak",
         "github_url": "https://github.com/facebookresearch/fairseq/blob/main/examples/wav2vec/README.md",
         "status": "completed",
@@ -11,7 +11,7 @@
     {
         "id": "d6fe0956-b5b4-4105-835e-8dee1872ee4d",
         "model": "KoelLabs/xlsr-timit-b0",
-        "subset": "test",
+        "subset": "timit-test",
         "submission_name": "english phoneme model",
         "github_url": "https://github.com/KoelLabs/",
         "status": "completed",
@@ -20,7 +20,7 @@
     {
         "id": "dbf4642a-fb13-402c-8a74-cc41fc4be599",
         "model": "speech31/wav2vec2-large-TIMIT-IPA",
-        "subset": "test",
+        "subset": "timit-test",
         "submission_name": "speech 31 model",
         "github_url": "https://huggingface.co/speech31/wav2vec2-large-TIMIT-IPA2",
         "status": "completed",
@@ -29,7 +29,7 @@
     {
         "id": "4e3b80be-b255-47f2-b4ae-18a12e232e8a",
         "model": "Jubliano/wav2vec2-large-xls-r-300m-ipa-INTERNATIONAL1.5",
-        "subset": "test",
+        "subset": "timit-test",
         "submission_name": "Jubliano model",
         "github_url": "https://huggingface.co/Jubliano/wav2vec2-large-xls-r-300m-ipa-INTERNATIONAL1.5WithoutSpaces/tree/d5312009d8e620b183c334dfdd9ffc6b4f06f8c1",
         "status": "processing",
@@ -38,7 +38,7 @@
     {
         "id": "912449a4-d7ed-4af4-b5be-5c2c57ec09ff",
         "model": "Jubliano/wav2vec2-large-xls-r-300m-ipa-INTERNATIONAL1.5",
-        "subset": "test",
+        "subset": "timit-test",
         "submission_name": "jubiliano model wav2vec2",
         "github_url": "https://huggingface.co/Jubliano/wav2vec2-large-xls-r-300m-ipa-INTERNATIONAL1.5WithoutSpaces/tree/d5312009d8e620b183c334dfdd9ffc6b4f06f8c1",
         "status": "completed",
@@ -47,7 +47,7 @@
     {
         "id": "c79df17e-2bb2-4253-ae26-f7cc6ab21265",
         "model": "facebook/wav2vec2-xlsr-53-espeak-cv-ft",
-        "subset": "test",
+        "subset": "timit-test",
         "submission_name": "xlsr 53 model",
         "github_url": "https://github.com/facebookresearch/fairseq/blob/main/examples/wav2vec/README.md",
         "status": "completed",
@@ -56,7 +56,7 @@
     {
         "id": "f36060e6-a746-44dc-a527-54995b270053",
         "model": "ginic/hyperparam_tuning_1_wav2vec2-large-xlsr-buckeye-ipa",
-        "subset": "test",
+        "subset": "timit-test",
         "submission_name": "ginic model wav2vec2 finetuned on buckeye",
         "github_url": "https://huggingface.co/ginic/vary_individuals_old_only_1_wav2vec2-large-xlsr-buckeye-ipa",
         "status": "completed",
@@ -65,7 +65,7 @@
     {
         "id": "abf6c247-9faf-46ef-b0fa-25f2669da922",
         "model": "KoelLabs/xlsr-timit-a0",
-        "subset": "test",
+        "subset": "timit-test",
         "submission_name": "Koel Labs early version of finetuned model ",
         "github_url": "https://github.com/KoelLabs/ML",
         "status": "processing",
@@ -74,7 +74,7 @@
     {
         "id": "47d56349-8111-4bda-a47f-e007dbedd36d",
         "model": "KoelLabs/xlsr-timit-a0",
-        "subset": "test",
+        "subset": "timit-test",
         "submission_name": "koel labs initial ",
         "github_url": "https://github.com/KoelLabs/ML/",
         "status": "completed",
@@ -83,7 +83,7 @@
     {
         "id": "51dd5735-63bd-4fe5-a588-c0fc079076e0",
         "model": "KoelLabs/xlsr-timit-a0",
-        "subset": "test",
+        "subset": "timit-test",
         "submission_name": "koel labs initial ",
         "github_url": "https://github.com/KoelLabs/ML/",
         "status": "completed",
@@ -92,7 +92,7 @@
     {
         "id": "2e592612-ca38-4afb-a6a0-3c870b288960",
         "model": "snu-nia-12/wav2vec2-large_nia12_phone-ipa_english",
-        "subset": "test",
+        "subset": "timit-test",
         "submission_name": "wav2vec2 ipa eng ",
         "github_url": "",
         "status": "completed",
@@ -101,7 +101,7 @@
     {
         "id": "ac4cbe86-4dbe-4929-8f76-4d2052e0acf1",
         "model": "vitouphy/wav2vec2-xls-r-300m-timit-phoneme",
-        "subset": "test",
+        "subset": "timit-test",
         "submission_name": "fine-tuned version of facebook/wav2vec2-xls-r-300m on the Timit dataset",
         "github_url": "https://www.kaggle.com/code/vitouphy/phoneme-recognition-with-wav2vec2",
         "status": "processing",
@@ -110,7 +110,7 @@
     {
         "id": "d38e65ce-75b5-4dbf-8ade-bff6a5803790",
         "model": "vitouphy/wav2vec2-xls-r-300m-timit-phoneme",
-        "subset": "test",
+        "subset": "timit-test",
         "submission_name": "fine-tuned version of facebook/wav2vec2-xls-r-300m on the Timit dataset",
         "github_url": "https://www.kaggle.com/code/vitouphy/phoneme-recognition-with-wav2vec2",
         "status": "completed",
@@ -119,7 +119,7 @@
     {
         "id": "2839c0c6-8f3b-426e-9eb7-04b6e133dc47",
         "model": "ctaguchi/wav2vec2-large-xlsr-japlmthufielta-ipa-plus-2000",
-        "subset": "test",
+        "subset": "timit-test",
         "submission_name": "wav2vec2 model",
         "github_url": "https://huggingface.co/ctaguchi/wav2vec2-large-xlsr-japlmthufielta-ipa1000-ns",
         "status": "completed",
@@ -128,7 +128,7 @@
     {
         "id": "59afc37a-0072-44dd-a02a-0cf47d89c120",
         "model": "ctaguchi/wav2vec2-large-xlsr-japlmthufielta-ipa1000-ns",
-        "subset": "test",
+        "subset": "timit-test",
         "submission_name": "wav2vec2 non-english transcription",
         "github_url": "https://huggingface.co/ctaguchi/wav2vec2-large-xlsr-japlmthufielta-ipa1000-ns",
         "status": "completed",
@@ -137,7 +137,7 @@
     {
         "id": "e57eda9d-7a1d-4b41-9d47-a3d3839cac8b",
         "model": "ginic/gender_split_70_female_4_wav2vec2-large-xlsr-buckeye-ipa",
-        "subset": "test",
+        "subset": "timit-test",
         "submission_name": "phonetic transcription with the Buckeye corpus, from xlsr-53 model ",
         "github_url": "https://github.com/ginic/multipa/tree/buckeye_experiments",
         "status": "failed",
@@ -147,7 +147,7 @@
     {
         "id": "5517f6b2-6a76-4a2d-a6ce-33446f390c3b",
         "model": "ginic/gender_split_70_female_4_wav2vec2-large-xlsr-buckeye-ipa",
-        "subset": "test",
+        "subset": "timit-test",
         "submission_name": "phonetic transcription with the Buckeye corpus, from xlsr-53 model",
         "github_url": "https://github.com/ginic/multipa/tree/buckeye_experiments",
         "status": "completed",
@@ -156,7 +156,7 @@
     {
         "id": "c2139f96-e79e-4f25-a525-aa039f65555f",
         "model": "Jubliano/wav2vec2-large-xls-r-300m-ipa-INTERNATIONAL1.9.2WithoutSpaces",
-        "subset": "test",
+        "subset": "timit-test",
         "submission_name": "phonetic transcription",
         "github_url": "https://huggingface.co/Jubliano/wav2vec2-large-xls-r-300m-ipa-INTERNATIONAL1.5WithoutSpaces",
         "status": "completed",
@@ -165,7 +165,7 @@
     {
         "id": "d146f1f1-6e6e-4b28-9420-c652ae9a1002",
         "model": "Jubliano/wav2vec2-large-xls-r-300m-ipa-nl",
-        "subset": "test",
+        "subset": "timit-test",
         "submission_name": "Jubliano xlsr model",
         "github_url": "https://huggingface.co/Jubliano/wav2vec2-large-xls-r-300m-ipa-nl1.1",
         "status": "completed",
@@ -174,7 +174,7 @@
     {
         "id": "265c5859-e7ba-492d-a6c9-45733dc17c99",
         "model": "Jubliano/wav2vec2-large-xls-r-300m-ipa-nl",
-        "subset": "test",
+        "subset": "timit-test",
         "submission_name": "Jubliano xlsr model",
         "github_url": "https://huggingface.co/Jubliano/wav2vec2-large-xls-r-300m-ipa-nl1.1",
         "status": "completed",
@@ -183,7 +183,7 @@
     {
         "id": "e297dfde-95e5-462b-a6e5-8fa43bc30bc0",
         "model": "speech31/wavlm-large-english-ipa",
-        "subset": "test",
+        "subset": "timit-test",
         "submission_name": "speech31 phoneme transcription english",
         "github_url": "https://huggingface.co/speech31/wavlm-large-english-ipa",
         "status": "completed",
@@ -192,7 +192,7 @@
     {
         "id": "efe95f71-05e3-485d-8e0c-1823a3037cf4",
         "model": "speech31/wavlm-large-english-ipa",
-        "subset": "test",
+        "subset": "timit-test",
         "submission_name": "speech31 phoneme transcription english",
         "github_url": "https://huggingface.co/speech31/wavlm-large-english-ipa",
         "status": "completed",
@@ -201,7 +201,7 @@
     {
         "id": "4b2ae2fc-fe2f-4f8b-9e8f-25c0bae13c0d",
         "model": "speech31/XLS-R-300m-english-ipa",
-        "subset": "test",
+        "subset": "timit-test",
         "submission_name": "speech31 xlsr model",
         "github_url": "https://huggingface.co/speech31/XLS-R-300m-english-ipa",
         "status": "completed",
@@ -210,36 +210,16 @@
     {
         "id": "33d387c0-703c-415d-b8e2-81cea87a2146",
         "model": "speech31/wav2vec2-large-english-TIMIT-phoneme_v3",
-        "subset": "test",
+        "subset": "timit-test",
         "submission_name": "model is a fine-tuned version of facebook/wav2vec2-large on the TIMIT dataset",
         "github_url": "https://huggingface.co/speech31/wav2vec2-large-english-TIMIT-phoneme_v3",
         "status": "completed",
         "submitted_at": "2024-12-20T16:52:07.883839"
     },
-    {
-        "id": "03e4e265-dc1c-4052-88bd-4ef481938d9d",
-        "model": "speech31/wav2vec2-large-TIMIT-IPA2\"",
-        "subset": "test",
-        "submission_name": "fine-tuned version of facebook/wav2vec2-large on the None dataset",
-        "github_url": "https://huggingface.co/speech31/wav2vec2-large-TIMIT-IPA2",
-        "status": "failed",
-        "submitted_at": "2024-12-20T21:54:21.539246",
-        "error": "Evaluation failed: Incorrect path_or_model_id: 'speech31/wav2vec2-large-TIMIT-IPA2\"'. Please provide either the path to a local folder or the repo_id of a model on the Hub."
-    },
-    {
-        "id": "4575245b-ae3c-4969-adef-fd07a58560b9",
-        "model": "speech31/wav2vec2-large-TIMIT-IPA2\"",
-        "subset": "test",
-        "submission_name": "fine-tuned version of facebook/wav2vec2-large on the None dataset",
-        "github_url": "https://huggingface.co/speech31/wav2vec2-large-TIMIT-IPA2",
-        "status": "failed",
-        "submitted_at": "2024-12-20T21:54:25.670276",
-        "error": "Evaluation failed: Incorrect path_or_model_id: 'speech31/wav2vec2-large-TIMIT-IPA2\"'. Please provide either the path to a local folder or the repo_id of a model on the Hub."
-    },
     {
         "id": "c89bcefc-3884-435a-a54c-24297fe6f041",
         "model": "speech31/wav2vec2-large-TIMIT-IPA2",
-        "subset": "test",
+        "subset": "timit-test",
         "submission_name": "fine-tuned version of facebook/wav2vec2-large on the None dataset",
         "github_url": "https://huggingface.co/speech31/wav2vec2-large-TIMIT-IPA2",
         "status": "completed",
@@ -248,7 +228,7 @@
     {
         "id": "81fa94f8-94ae-4601-952c-24abaddaf691",
         "model": "ginic/vary_individuals_young_only_3_wav2vec2-large-xlsr-buckeye-ipa",
-        "subset": "test",
+        "subset": "timit-test",
         "submission_name": "ginic model, facebook/wav2vec2-large-xlsr-53 fine tuned",
         "github_url": "https://huggingface.co/ginic/vary_individuals_young_only_3_wav2vec2-large-xlsr-buckeye-ipa",
         "status": "completed",