flanT5_base_Fact_updates

This model is a fine-tuned version of google/flan-t5-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.3672
• Accuracy: 0.8228
• Weighted F1: 0.8225
• Micro F1: 0.8228
• Macro F1: 0.8193

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 1
• eval_batch_size: 1
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 5

Training results

Training Loss	Epoch	Step	Accuracy	F1	Validation Loss	Macro F1	Micro F1	Precision	Recall	Weighted F1	Weighted Precision	Weighted Recall
1.0427	0.0282	200	0.6400	[0.7486209613869188, 0.3658051689860835]	0.8832	0.5572	0.6400	[0.6168831168831169, 0.7931034482758621]	[0.9519038076152304, 0.23772609819121446]	0.5814	0.6939	0.6400
1.208	0.0564	400	0.6907	[0.7697478991596639, 0.5292096219931272]	1.8381	0.6495	0.6907	[0.662807525325615, 0.7897435897435897]	[0.9178356713426854, 0.3979328165374677]	0.6647	0.7183	0.6907
1.6327	0.0847	600	0.6862	[0.765993265993266, 0.523972602739726]	1.3924	0.6450	0.6862	[0.660377358490566, 0.7766497461928934]	[0.9118236472945892, 0.3953488372093023]	0.6603	0.7112	0.6862
1.3962	0.1129	800	0.6885	[0.7688442211055276, 0.5224913494809689]	1.7990	0.6457	0.6885	[0.660431654676259, 0.7905759162303665]	[0.9198396793587175, 0.39018087855297157]	0.6612	0.7173	0.6885
1.3163	0.1411	1000	0.6862	[0.765993265993266, 0.523972602739726]	1.5711	0.6450	0.6862	[0.660377358490566, 0.7766497461928934]	[0.9118236472945892, 0.3953488372093023]	0.6603	0.7112	0.6862
1.1548	0.1693	1200	0.7043	[0.7495219885277247, 0.6391184573002755]	1.6964	0.6943	0.7043	[0.716636197440585, 0.6843657817109144]	[0.7855711422845691, 0.599483204134367]	0.7013	0.7025	0.7043
1.0917	0.1976	1400	0.7122	[0.7442326980942828, 0.6709677419354839]	1.6431	0.7076	0.7122	[0.7449799196787149, 0.6701030927835051]	[0.7434869739478958, 0.6718346253229974]	0.7122	0.7123	0.7122
1.2556	0.2258	1600	0.6941	[0.775103734439834, 0.5220458553791887]	1.4097	0.6486	0.6941	[0.6614730878186968, 0.8222222222222222]	[0.935871743486974, 0.38242894056847543]	0.6646	0.7317	0.6941
1.1785	0.2540	1800	0.6907	[0.7677966101694915, 0.5371621621621622]	1.1850	0.6525	0.6907	[0.6651982378854625, 0.775609756097561]	[0.9078156312625251, 0.4108527131782946]	0.6671	0.7134	0.6907
1.2531	0.2822	2000	0.6907	[0.7724252491694352, 0.5176056338028169]	1.1955	0.6450	0.6907	[0.6595744680851063, 0.8121546961325967]	[0.9318637274549099, 0.3798449612403101]	0.6611	0.7262	0.6907
0.9859	0.3105	2200	0.7122	[0.7780678851174935, 0.5906902086677368]	1.3978	0.6844	0.7122	[0.6876923076923077, 0.7796610169491526]	[0.8957915831663327, 0.4754521963824289]	0.6962	0.7279	0.7122
1.0743	0.3387	2400	0.7156	[0.7769911504424779, 0.6074766355140186]	1.3271	0.6922	0.7156	[0.6957210776545166, 0.7647058823529411]	[0.8797595190380761, 0.5038759689922481]	0.7029	0.7259	0.7156
1.1299	0.3669	2600	0.7178	[0.7807017543859649, 0.6044303797468354]	1.1492	0.6926	0.7178	[0.6942277691107644, 0.7795918367346939]	[0.8917835671342685, 0.4935400516795866]	0.7037	0.7315	0.7178
0.978	0.3951	2800	0.7427	[0.7760314341846758, 0.6976127320954907]	1.2433	0.7368	0.7427	[0.7610789980732178, 0.7166212534059946]	[0.7915831663326653, 0.6795865633074936]	0.7418	0.7417	0.7427
0.9792	0.4234	3000	0.7336	[0.7773584905660378, 0.6685393258426966]	1.1582	0.7229	0.7336	[0.7344028520499108, 0.7323076923076923]	[0.8256513026052105, 0.6149870801033591]	0.7298	0.7335	0.7336
0.8484	0.4516	3200	0.7099	[0.7849372384937239, 0.5545927209705372]	1.2349	0.6698	0.7099	[0.6738505747126436, 0.8421052631578947]	[0.9398797595190381, 0.4134366925064599]	0.6843	0.7473	0.7099
1.0107	0.4798	3400	0.7325	[0.7603640040444893, 0.6973180076628352]	1.1968	0.7288	0.7325	[0.7673469387755102, 0.6893939393939394]	[0.7535070140280561, 0.7054263565891473]	0.7328	0.7333	0.7325
1.0096	0.5080	3600	0.6885	[0.775609756097561, 0.4907749077490775]	1.0334	0.6332	0.6885	[0.652530779753762, 0.8580645161290322]	[0.9559118236472945, 0.34366925064599485]	0.6512	0.7423	0.6885
0.9465	0.5363	3800	0.7302	[0.7460148777895855, 0.7123947051744886]	1.1272	0.7292	0.7302	[0.7941176470588235, 0.6666666666666666]	[0.7034068136272545, 0.7648578811369509]	0.7313	0.7384	0.7302
0.9628	0.5645	4000	0.7427	[0.7849056603773585, 0.6797752808988764]	1.0353	0.7323	0.7427	[0.7415329768270945, 0.7446153846153846]	[0.8336673346693386, 0.6253229974160207]	0.7390	0.7429	0.7427
0.8945	0.5927	4200	0.7438	[0.7930720145852325, 0.6637037037037037]	1.1590	0.7284	0.7438	[0.7274247491638796, 0.7777777777777778]	[0.8717434869739479, 0.5788113695090439]	0.7366	0.7494	0.7438
1.0332	0.6209	4400	0.7483	[0.793709528214616, 0.6772793053545586]	1.1790	0.7355	0.7483	[0.7371134020618557, 0.7697368421052632]	[0.8597194388777555, 0.6046511627906976]	0.7429	0.7514	0.7483
0.9625	0.6492	4600	0.7641	[0.7992315081652257, 0.7140902872777017]	1.2291	0.7567	0.7641	[0.7675276752767528, 0.7587209302325582]	[0.8336673346693386, 0.6744186046511628]	0.7620	0.7637	0.7641
1.0516	0.6774	4800	0.7562	[0.8039927404718693, 0.6776119402985075]	1.0852	0.7408	0.7562	[0.7346600331674958, 0.8021201413427562]	[0.8877755511022044, 0.58656330749354]	0.7488	0.7641	0.7562
1.044	0.7056	5000	0.7675	[0.8015414258188824, 0.7193460490463215]	1.0155	0.7604	0.7675	[0.7717996289424861, 0.760806916426513]	[0.8336673346693386, 0.6821705426356589]	0.7656	0.7670	0.7675
0.8673	0.7338	5200	1.0519	0.7675	0.7663	0.7675	0.7615
0.9855	0.7621	5400	0.7501	0.7777	0.7708	0.7777	0.7635
0.863	0.7903	5600	0.9434	0.7720	0.7685	0.7720	0.7624
1.0366	0.8185	5800	1.0371	0.7122	0.6780	0.7122	0.6615
0.8824	0.8467	6000	1.0265	0.7619	0.7612	0.7619	0.7567
0.8301	0.8750	6200	1.0980	0.7765	0.7744	0.7765	0.7692
0.9856	0.9032	6400	0.7633	0.7867	0.7825	0.7867	0.7765
0.8429	0.9314	6600	0.9456	0.7743	0.7670	0.7743	0.7594
1.0278	0.9596	6800	0.9540	0.7551	0.7528	0.7551	0.7471
0.9641	0.9879	7000	0.8433	0.7878	0.7831	0.7878	0.7769
0.694	1.0161	7200	1.2788	0.7472	0.7395	0.7472	0.7312
0.7612	1.0443	7400	0.8530	0.8047	0.7982	0.8047	0.7915
0.8366	1.0725	7600	0.8560	0.7991	0.7960	0.7991	0.7906
0.7097	1.1008	7800	0.9766	0.7935	0.7906	0.7935	0.7853
0.9577	1.1290	8000	0.7627	0.7980	0.7966	0.7980	0.7922
0.8404	1.1572	8200	0.8350	0.7968	0.7938	0.7968	0.7885
0.7927	1.1854	8400	0.8045	0.7946	0.7935	0.7946	0.7893
0.8905	1.2137	8600	0.7522	0.8093	0.8058	0.8093	0.8006
0.6972	1.2419	8800	0.9325	0.7968	0.7907	0.7968	0.7840
0.8006	1.2701	9000	0.8882	0.7957	0.7935	0.7957	0.7886
0.6982	1.2983	9200	0.8838	0.8002	0.7959	0.8002	0.7901
0.7914	1.3266	9400	0.7716	0.8059	0.8023	0.8059	0.7969
0.7378	1.3548	9600	0.9446	0.7968	0.7941	0.7968	0.7889
0.9703	1.3830	9800	0.8472	0.7935	0.7873	0.7935	0.7805
0.7271	1.4112	10000	0.9727	0.7991	0.7967	0.7991	0.7918
0.78	1.4395	10200	0.8350	0.7901	0.7827	0.7901	0.7755
0.8381	1.4677	10400	0.8018	0.8036	0.8003	0.8036	0.7950
0.8608	1.4959	10600	0.8652	0.7923	0.7906	0.7923	0.7859
0.688	1.5241	10800	0.9063	0.8036	0.8019	0.8036	0.7975
0.873	1.5524	11000	0.8821	0.8002	0.7972	0.8002	0.7919
0.9033	1.5806	11200	0.8250	0.7991	0.7944	0.7991	0.7883
0.8112	1.6088	11400	0.7788	0.8025	0.7976	0.8025	0.7915
0.6878	1.6370	11600	0.9670	0.8014	0.7970	0.8014	0.7911
0.7777	1.6653	11800	0.9020	0.7968	0.7935	0.7968	0.7881
0.9171	1.6935	12000	0.8787	0.7946	0.7923	0.7946	0.7873
0.8384	1.7217	12200	0.9244	0.7754	0.7752	0.7754	0.7714
0.7736	1.7499	12400	0.8608	0.7980	0.7961	0.7980	0.7914
0.8294	1.7782	12600	0.8227	0.8025	0.8005	0.8025	0.7959
0.8057	1.8064	12800	0.8276	0.7889	0.7883	0.7889	0.7844
0.7318	1.8346	13000	0.8662	0.7867	0.7870	0.7867	0.7840
0.7388	1.8628	13200	0.8807	0.7957	0.7916	0.7957	0.7858
0.7054	1.8911	13400	0.8478	0.8070	0.8045	0.8070	0.7997
0.875	1.9193	13600	0.7832	0.7980	0.7937	0.7980	0.7879
0.692	1.9475	13800	0.9485	0.7968	0.7961	0.7968	0.7921
1.0122	1.9757	14000	0.6915	0.8059	0.8010	0.8059	0.7950
0.613	2.0040	14200	0.8384	0.8115	0.8090	0.8115	0.8042
0.5873	2.0322	14400	0.9702	0.7912	0.7900	0.7912	0.7856
0.5487	2.0604	14600	1.0145	0.8014	0.7986	0.8014	0.7936
0.7213	2.0886	14800	0.8915	0.8070	0.8034	0.8070	0.7980
0.678	2.1169	15000	0.8858	0.8070	0.8045	0.8070	0.7997
0.4698	2.1451	15200	1.1199	0.7923	0.7925	0.7923	0.7894
0.7592	2.1733	15400	0.9553	0.8104	0.8070	0.8104	0.8018
0.6606	2.2015	15600	1.0627	0.7901	0.7875	0.7901	0.7822
0.6252	2.2297	15800	1.0067	0.7923	0.7926	0.7923	0.7895
0.6854	2.2580	16000	0.8288	0.8059	0.8031	0.8059	0.7981
0.7025	2.2862	16200	0.9184	0.8014	0.7985	0.8014	0.7934
0.6804	2.3144	16400	0.9403	0.7957	0.7926	0.7957	0.7872
0.66	2.3426	16600	0.9250	0.8014	0.7995	0.8014	0.7948
0.751	2.3709	16800	0.9244	0.8081	0.8074	0.8081	0.8036
0.6358	2.3991	17000	0.8830	0.8025	0.7996	0.8025	0.7945
0.7491	2.4273	17200	0.8206	0.8093	0.8048	0.8093	0.7991
0.641	2.4555	17400	0.8518	0.8149	0.8110	0.8149	0.8056
0.7503	2.4838	17600	0.9473	0.8081	0.8045	0.8081	0.7991
0.6213	2.5120	17800	0.9772	0.8014	0.7995	0.8014	0.7948
0.5634	2.5402	18000	1.0702	0.8002	0.7989	0.8002	0.7946
0.6771	2.5684	18200	0.9751	0.7980	0.7937	0.7980	0.7879
0.5203	2.5967	18400	0.9834	0.8126	0.8110	0.8126	0.8068
0.6773	2.6249	18600	0.8943	0.8126	0.8105	0.8126	0.8060
0.7521	2.6531	18800	0.8472	0.8138	0.8121	0.8138	0.8079
0.645	2.6813	19000	0.8852	0.8172	0.8165	0.8172	0.8130
0.5444	2.7096	19200	0.9600	0.8081	0.8055	0.8081	0.8006
0.7671	2.7378	19400	0.8624	0.8036	0.8018	0.8036	0.7973
0.7422	2.7660	19600	0.9530	0.8070	0.8042	0.8070	0.7992
0.5092	2.7942	19800	1.0464	0.8183	0.8169	0.8183	0.8128
0.6901	2.8225	20000	0.9704	0.8183	0.8160	0.8183	0.8114
0.6683	2.8507	20200	0.9721	0.8149	0.8137	0.8149	0.8097
0.5067	2.8789	20400	1.0650	0.8104	0.8070	0.8104	0.8018
0.7208	2.9071	20600	0.9588	0.8104	0.8062	0.8104	0.8006
0.5991	2.9354	20800	0.9024	0.8205	0.8178	0.8205	0.8131
0.6839	2.9636	21000	0.9077	0.8093	0.8087	0.8093	0.8051
0.652	2.9918	21200	0.9307	0.8172	0.8161	0.8172	0.8123
0.5023	3.0200	21400	0.9503	0.8262	0.8245	0.8262	0.8205
0.4241	3.0483	21600	1.0769	0.8126	0.8118	0.8126	0.8081
0.5699	3.0765	21800	1.1059	0.8059	0.8055	0.8059	0.8020
0.5996	3.1047	22000	0.9478	0.8149	0.8143	0.8149	0.8108
0.4193	3.1329	22200	1.0867	0.8093	0.8080	0.8093	0.8039
0.4676	3.1612	22400	1.1191	0.8194	0.8184	0.8194	0.8147
0.448	3.1894	22600	1.1390	0.8172	0.8151	0.8172	0.8107
0.4389	3.2176	22800	1.1672	0.8126	0.8117	0.8126	0.8078
0.4062	3.2458	23000	1.1531	0.8205	0.8181	0.8205	0.8136
0.5454	3.2741	23200	1.1332	0.8284	0.8268	0.8284	0.8228
0.605	3.3023	23400	1.0768	0.8172	0.8147	0.8172	0.8100
0.7416	3.3305	23600	0.9257	0.8172	0.8154	0.8172	0.8112
0.3946	3.3587	23800	1.0511	0.8172	0.8144	0.8172	0.8096
0.6225	3.3870	24000	1.1009	0.8138	0.8130	0.8138	0.8093
0.4774	3.4152	24200	1.0590	0.8239	0.8217	0.8239	0.8174
0.3623	3.4434	24400	1.1219	0.8228	0.8212	0.8228	0.8172
0.5161	3.4716	24600	1.0997	0.8273	0.8265	0.8273	0.8231
0.4854	3.4999	24800	0.9759	0.8284	0.8264	0.8284	0.8222
0.5082	3.5281	25000	1.0789	0.8251	0.8228	0.8251	0.8185
0.5719	3.5563	25200	1.0121	0.8149	0.8108	0.8149	0.8054
0.475	3.5845	25400	1.0661	0.8217	0.8202	0.8217	0.8161
0.4861	3.6128	25600	1.0291	0.8228	0.8213	0.8228	0.8174
0.3472	3.6410	25800	1.0936	0.8251	0.8235	0.8251	0.8195
0.5	3.6692	26000	1.0842	0.8217	0.8202	0.8217	0.8163
0.4416	3.6974	26200	1.0122	0.8205	0.8187	0.8205	0.8144
0.5956	3.7257	26400	1.0498	0.8172	0.8163	0.8172	0.8126
0.6085	3.7539	26600	1.0169	0.8217	0.8207	0.8217	0.8170
0.4876	3.7821	26800	1.0093	0.8217	0.8206	0.8217	0.8168
0.3774	3.8103	27000	1.1292	0.8126	0.8122	0.8126	0.8088
0.48	3.8386	27200	1.0578	0.8251	0.8240	0.8251	0.8202
0.4071	3.8668	27400	1.0626	0.8273	0.8256	0.8273	0.8216
0.3274	3.8950	27600	1.1553	0.8205	0.8193	0.8205	0.8155
0.4566	3.9232	27800	1.0869	0.8341	0.8327	0.8341	0.8290
0.5898	3.9515	28000	1.1071	0.8183	0.8178	0.8183	0.8145
0.4661	3.9797	28200	1.2260	0.8183	0.8178	0.8183	0.8143
0.5611	4.0079	28400	1.0554	0.8284	0.8264	0.8284	0.8222
0.3045	4.0361	28600	1.0761	0.8284	0.8276	0.8284	0.8240
0.2848	4.0644	28800	1.3040	0.8138	0.8128	0.8138	0.8089
0.3582	4.0926	29000	1.1999	0.8239	0.8233	0.8239	0.8198
0.1791	4.1208	29200	1.3165	0.8262	0.8249	0.8262	0.8211
0.3963	4.1490	29400	1.3130	0.8160	0.8156	0.8160	0.8123
0.2422	4.1773	29600	1.2456	0.8352	0.8335	0.8352	0.8297
0.4922	4.2055	29800	1.1898	0.8307	0.8294	0.8307	0.8257
0.3682	4.2337	30000	1.1140	0.8318	0.8308	0.8318	0.8272
0.2968	4.2619	30200	1.1722	0.8307	0.8295	0.8307	0.8258
0.5083	4.2901	30400	1.1706	0.8228	0.8225	0.8228	0.8194
0.3826	4.3184	30600	1.2276	0.8194	0.8191	0.8194	0.8158
0.5348	4.3466	30800	1.2370	0.8228	0.8225	0.8228	0.8193
0.4252	4.3748	31000	1.1660	0.8307	0.8296	0.8307	0.8260
0.4772	4.4030	31200	1.1557	0.8318	0.8306	0.8318	0.8269
0.2945	4.4313	31400	1.1913	0.8307	0.8293	0.8307	0.8256
0.3712	4.4595	31600	1.1258	0.8352	0.8336	0.8352	0.8298
0.2715	4.4877	31800	1.1424	0.8330	0.8317	0.8330	0.8280
0.2835	4.5159	32000	1.1789	0.8318	0.8304	0.8318	0.8267
0.3025	4.5442	32200	1.1726	0.8284	0.8276	0.8284	0.8240
0.3259	4.5724	32400	1.2833	0.8205	0.8200	0.8205	0.8165
0.5595	4.6006	32600	1.2895	0.8172	0.8170	0.8172	0.8140
0.3064	4.6288	32800	1.2627	0.8296	0.8288	0.8296	0.8254
0.3127	4.6571	33000	1.2772	0.8307	0.8302	0.8307	0.8270
0.2227	4.6853	33200	1.3326	0.8307	0.8302	0.8307	0.8270
0.1252	4.7135	33400	1.3394	0.8330	0.8320	0.8330	0.8286
0.1683	4.7417	33600	1.3572	0.8341	0.8331	0.8341	0.8296
0.4833	4.7700	33800	1.3697	0.8217	0.8213	0.8217	0.8181
0.3474	4.7982	34000	1.3793	0.8205	0.8203	0.8205	0.8171
0.358	4.8264	34200	1.3703	0.8228	0.8225	0.8228	0.8194
0.1538	4.8546	34400	1.3833	0.8217	0.8213	0.8217	0.8181
0.3807	4.8829	34600	1.3954	0.8217	0.8213	0.8217	0.8181
0.3915	4.9111	34800	1.3835	0.8217	0.8213	0.8217	0.8181
0.4989	4.9393	35000	1.3631	0.8228	0.8225	0.8228	0.8193
0.3053	4.9675	35200	1.3674	0.8228	0.8225	0.8228	0.8193
0.3438	4.9958	35400	1.3672	0.8228	0.8225	0.8228	0.8193

Framework versions

• Transformers 4.41.2
• Pytorch 2.3.0+cu121
• Datasets 2.19.1
• Tokenizers 0.19.1