nomnoos37/stt-turbo-1227-v1.1-peft-eng-1k-1e-3-rank64-reduced

Whisper Small ko

This model is a fine-tuned version of openai/whisper-large-v3-turbo on the custom dataset. It achieves the following results on the evaluation set:

• Loss: 0.0681

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.001
• train_batch_size: 64
• eval_batch_size: 256
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 200
• training_steps: 1000
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
0.7511	0.0641	10	1.4828
0.6562	0.1282	20	1.2493
0.3357	0.1923	30	0.8826
0.1966	0.2564	40	0.7550
0.1566	0.3205	50	0.6223
0.1253	0.3846	60	0.5636
0.104	0.4487	70	0.5163
0.0979	0.5128	80	0.4967
0.0913	0.5769	90	0.4598
0.107	0.6410	100	0.4358
0.1104	0.7051	110	0.4337
0.0819	0.7692	120	0.4072
0.0892	0.8333	130	0.3902
0.0954	0.8974	140	0.3555
0.0789	0.9615	150	0.3516
0.0811	1.0256	160	0.3237
0.0822	1.0897	170	0.2995
0.0747	1.1538	180	0.3195
0.0838	1.2179	190	0.3436
0.0587	1.2821	200	0.3178
0.0675	1.3462	210	0.2981
0.062	1.4103	220	0.2745
0.0662	1.4744	230	0.2650
0.0639	1.5385	240	0.2802
0.0745	1.6026	250	0.3079
0.0632	1.6667	260	0.2631
0.0724	1.7308	270	0.2406
0.0728	1.7949	280	0.2496
0.0601	1.8590	290	0.2328
0.0653	1.9231	300	0.2230
0.0668	1.9872	310	0.2076
0.0606	2.0513	320	0.1849
0.0543	2.1154	330	0.1851
0.0493	2.1795	340	0.1697
0.047	2.2436	350	0.1630
0.0451	2.3077	360	0.1647
0.0403	2.3718	370	0.1634
0.0477	2.4359	380	0.1530
0.0491	2.5	390	0.1517
0.0444	2.5641	400	0.1464
0.0399	2.6282	410	0.1421
0.0622	2.6923	420	0.1690
0.0509	2.7564	430	0.1800
0.0447	2.8205	440	0.1773
0.0496	2.8846	450	0.1765
0.049	2.9487	460	0.1721
0.0388	3.0128	470	0.1666
0.0667	3.0769	480	0.1477
0.038	3.1410	490	0.1378
0.0363	3.2051	500	0.1376
0.0264	3.2692	510	0.1289
0.032	3.3333	520	0.1339
0.0324	3.3974	530	0.1226
0.034	3.4615	540	0.1196
0.0257	3.5256	550	0.1092
0.0355	3.5897	560	0.1054
0.0256	3.6538	570	0.1096
0.033	3.7179	580	0.1063
0.0337	3.7821	590	0.1121
0.0324	3.8462	600	0.1065
0.0327	3.9103	610	0.1075
0.0314	3.9744	620	0.1042
0.0407	4.0385	630	0.0929
0.0185	4.1026	640	0.1295
0.0215	4.1667	650	0.1258
0.0198	4.2308	660	0.1269
0.0204	4.2949	670	0.1243
0.019	4.3590	680	0.1252
0.0236	4.4231	690	0.1229
0.0263	4.4872	700	0.1196
0.0232	4.5513	710	0.1184
0.0189	4.6154	720	0.1171
0.0187	4.6795	730	0.1155
0.0199	4.7436	740	0.1114
0.026	4.8077	750	0.1105
0.0193	4.8718	760	0.1112
0.0174	4.9359	770	0.1116
0.0205	5.0	780	0.1117
0.0132	5.0641	790	0.1101
0.0125	5.1282	800	0.1100
0.0142	5.1923	810	0.1098
0.0555	5.2564	820	0.0805
0.0138	5.3205	830	0.0787
0.0193	5.3846	840	0.0748
0.0141	5.4487	850	0.0731
0.0158	5.5128	860	0.0724
0.0133	5.5769	870	0.0722
0.0166	5.6410	880	0.0716
0.0125	5.7051	890	0.0706
0.0133	5.7692	900	0.0701
0.0209	5.8333	910	0.0700
0.0144	5.8974	920	0.0696
0.0111	5.9615	930	0.0697
0.0109	6.0256	940	0.0691
0.0131	6.0897	950	0.0688
0.0174	6.1538	960	0.0685
0.0094	6.2179	970	0.0684
0.0128	6.2821	980	0.0682
0.011	6.3462	990	0.0681
0.0112	6.4103	1000	0.0681

Framework versions

• PEFT 0.14.0
• Transformers 4.47.1
• Pytorch 2.5.1+cu124
• Datasets 3.2.0
• Tokenizers 0.21.0