swin-transformer-class

This model is a fine-tuned version of microsoft/swinv2-base-patch4-window16-256 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.2549
• Accuracy: 0.4953
• F1: 0.4547

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 64
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 150

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1
2.1381	0.9748	29	2.1103	0.2594	0.1420
1.9462	1.9832	59	1.8963	0.2783	0.1481
1.7299	2.9916	89	1.6978	0.3066	0.2504
1.6406	4.0	119	1.5954	0.3585	0.3221
1.5067	4.9748	148	1.5339	0.3915	0.3527
1.4566	5.9832	178	1.4972	0.4151	0.3769
1.4487	6.9916	208	1.4635	0.4387	0.3369
1.4335	8.0	238	1.4377	0.4481	0.3958
1.3974	8.9748	267	1.4213	0.4623	0.4066
1.3542	9.9832	297	1.4004	0.4575	0.4090
1.2964	10.9916	327	1.3880	0.4434	0.3832
1.3073	12.0	357	1.3716	0.4906	0.4449
1.3256	12.9748	386	1.3664	0.4528	0.4175
1.2867	13.9832	416	1.3622	0.4434	0.4033
1.3096	14.9916	446	1.3418	0.4764	0.4281
1.3012	16.0	476	1.3321	0.4528	0.4161
1.3086	16.9748	505	1.3248	0.4481	0.3578
1.2646	17.9832	535	1.3164	0.4717	0.4269
1.2647	18.9916	565	1.3140	0.4811	0.4394
1.2673	20.0	595	1.3073	0.4670	0.4311
1.2649	20.9748	624	1.2999	0.4906	0.4319
1.2721	21.9832	654	1.3007	0.4764	0.4236
1.317	22.9916	684	1.2982	0.4670	0.4167
1.2397	24.0	714	1.3031	0.4623	0.4115
1.209	24.9748	743	1.3075	0.4811	0.4379
1.1994	25.9832	773	1.3091	0.4245	0.3765
1.2695	26.9916	803	1.3017	0.4717	0.4362
1.2167	28.0	833	1.2986	0.4575	0.4153
1.234	28.9748	862	1.3082	0.4292	0.3773
1.2726	29.9832	892	1.3003	0.4670	0.4238
1.207	30.9916	922	1.2964	0.4670	0.4260
1.1534	32.0	952	1.3059	0.4292	0.3727
1.2477	32.9748	981	1.2924	0.4858	0.4397
1.2202	33.9832	1011	1.2924	0.4623	0.3850
1.2248	34.9916	1041	1.2969	0.4434	0.3680
1.1775	36.0	1071	1.2848	0.4953	0.4485
1.2401	36.9748	1100	1.2887	0.4575	0.4214
1.2311	37.9832	1130	1.2838	0.4858	0.4420
1.2143	38.9916	1160	1.2846	0.4906	0.4354
1.1548	40.0	1190	1.2828	0.4481	0.4057
1.1405	40.9748	1219	1.2878	0.4717	0.4356
1.1957	41.9832	1249	1.2839	0.4528	0.4063
1.211	42.9916	1279	1.2853	0.4670	0.4097
1.1849	44.0	1309	1.2779	0.4811	0.4360
1.1466	44.9748	1338	1.2765	0.4764	0.4341
1.1386	45.9832	1368	1.2836	0.4623	0.4184
1.2258	46.9916	1398	1.2718	0.4717	0.4293
1.2139	48.0	1428	1.2695	0.4906	0.4409
1.1938	48.9748	1457	1.2737	0.4764	0.4385
1.2171	49.9832	1487	1.2709	0.4670	0.4189
1.1804	50.9916	1517	1.2657	0.4764	0.4327
1.143	52.0	1547	1.2701	0.4764	0.4345
1.1723	52.9748	1576	1.2783	0.4717	0.4152
1.1454	53.9832	1606	1.2670	0.5047	0.4496
1.1957	54.9916	1636	1.2709	0.4670	0.4211
1.2383	56.0	1666	1.2752	0.4670	0.4136
1.1935	56.9748	1695	1.2670	0.4623	0.4201
1.159	57.9832	1725	1.2696	0.4717	0.4199
1.2267	58.9916	1755	1.2676	0.4858	0.4404
1.2047	60.0	1785	1.2659	0.4764	0.4336
1.1168	60.9748	1814	1.2680	0.4953	0.4466
1.2396	61.9832	1844	1.2741	0.4481	0.4045
1.1193	62.9916	1874	1.2791	0.4623	0.4184
1.1587	64.0	1904	1.2657	0.4858	0.4369
1.1492	64.9748	1933	1.2736	0.4717	0.4367
1.1303	65.9832	1963	1.2683	0.4811	0.4300
1.1672	66.9916	1993	1.2683	0.4953	0.4494
1.2035	68.0	2023	1.2667	0.4811	0.4447
1.1494	68.9748	2052	1.2645	0.4858	0.4476
1.1537	69.9832	2082	1.2714	0.4811	0.4434
1.18	70.9916	2112	1.2701	0.4811	0.4344
1.1386	72.0	2142	1.2688	0.4858	0.4440
1.1757	72.9748	2171	1.2694	0.4906	0.4514
1.1335	73.9832	2201	1.2712	0.4858	0.4419
1.1669	74.9916	2231	1.2701	0.5094	0.4651
1.1862	76.0	2261	1.2684	0.4764	0.4316
1.1695	76.9748	2290	1.2642	0.4906	0.4509
1.1317	77.9832	2320	1.2687	0.4811	0.4391
1.2023	78.9916	2350	1.2647	0.5	0.4579
1.1603	80.0	2380	1.2650	0.5	0.4596
1.1461	80.9748	2409	1.2623	0.4811	0.4396
1.1356	81.9832	2439	1.2621	0.4953	0.4449
1.1646	82.9916	2469	1.2713	0.4953	0.4526
1.152	84.0	2499	1.2661	0.5047	0.4632
1.0999	84.9748	2528	1.2685	0.5047	0.4576
1.1749	85.9832	2558	1.2716	0.4858	0.4459
1.1823	86.9916	2588	1.2624	0.4906	0.4441
1.1736	88.0	2618	1.2650	0.4811	0.4377
1.1565	88.9748	2647	1.2667	0.4670	0.4226
1.1565	89.9832	2677	1.2667	0.4953	0.4453
1.192	90.9916	2707	1.2634	0.5047	0.4635
1.1271	92.0	2737	1.2639	0.4764	0.4303
1.19	92.9748	2766	1.2631	0.4858	0.4412
1.1866	93.9832	2796	1.2616	0.4953	0.4555
1.0829	94.9916	2826	1.2586	0.4953	0.4522
1.1692	96.0	2856	1.2608	0.4906	0.4497
1.1503	96.9748	2885	1.2607	0.4953	0.4551
1.1263	97.9832	2915	1.2577	0.4953	0.4543
1.2199	98.9916	2945	1.2570	0.5047	0.4601
1.1347	100.0	2975	1.2555	0.4953	0.4503
1.1583	100.9748	3004	1.2557	0.5	0.4592
1.1697	101.9832	3034	1.2578	0.4858	0.4467
1.1918	102.9916	3064	1.2572	0.5047	0.4598
1.1959	104.0	3094	1.2563	0.5094	0.4649
1.2032	104.9748	3123	1.2551	0.4906	0.4480
1.2031	105.9832	3153	1.2552	0.4906	0.4491
1.1565	106.9916	3183	1.2544	0.5142	0.4668
1.1703	108.0	3213	1.2570	0.5	0.4598
1.2085	108.9748	3242	1.2550	0.5094	0.4639
1.1641	109.9832	3272	1.2578	0.4953	0.4551
1.1846	110.9916	3302	1.2579	0.4906	0.4510
1.1989	112.0	3332	1.2560	0.5	0.4579
1.111	112.9748	3361	1.2561	0.4953	0.4545
1.1703	113.9832	3391	1.2561	0.5047	0.4567
1.165	114.9916	3421	1.2567	0.5	0.4480
1.1295	116.0	3451	1.2582	0.4953	0.4475
1.1084	116.9748	3480	1.2574	0.5	0.4571
1.1577	117.9832	3510	1.2573	0.5047	0.4617
1.156	118.9916	3540	1.2565	0.4953	0.4559
1.1491	120.0	3570	1.2564	0.5	0.4573
1.1396	120.9748	3599	1.2572	0.5	0.4534
1.1545	121.9832	3629	1.2565	0.5	0.4604
1.1796	122.9916	3659	1.2563	0.5	0.4593
1.2012	124.0	3689	1.2559	0.4858	0.4454
1.1396	124.9748	3718	1.2567	0.4953	0.4555
1.1999	125.9832	3748	1.2558	0.4858	0.4450
1.1524	126.9916	3778	1.2569	0.4953	0.4554
1.2299	128.0	3808	1.2560	0.4953	0.4525
1.1548	128.9748	3837	1.2553	0.4764	0.4375
1.1869	129.9832	3867	1.2554	0.4811	0.4426
1.1891	130.9916	3897	1.2555	0.4811	0.4423
1.1353	132.0	3927	1.2565	0.4953	0.4554
1.1717	132.9748	3956	1.2569	0.5047	0.4643
1.1536	133.9832	3986	1.2556	0.5	0.4574
1.1667	134.9916	4016	1.2555	0.5	0.4594
1.1633	136.0	4046	1.2550	0.4953	0.4551
1.1646	136.9748	4075	1.2539	0.4858	0.4457
1.1618	137.9832	4105	1.2540	0.5047	0.4594
1.1581	138.9916	4135	1.2545	0.4858	0.4460
1.117	140.0	4165	1.2549	0.4858	0.4457
1.184	140.9748	4194	1.2552	0.4906	0.4504
1.1323	141.9832	4224	1.2553	0.4906	0.4504
1.1219	142.9916	4254	1.2550	0.4953	0.4547
1.1478	144.0	4284	1.2550	0.4953	0.4547
1.1177	144.9748	4313	1.2550	0.4953	0.4547
1.1326	145.9832	4343	1.2549	0.4953	0.4547
1.1392	146.2185	4350	1.2549	0.4953	0.4547

Framework versions

• Transformers 4.45.2
• Pytorch 2.4.0+cu121
• Datasets 3.1.0
• Tokenizers 0.20.1