layoutlm-funsd

This model is a fine-tuned version of microsoft/layoutlm-base-uncased on the funsd dataset. It achieves the following results on the evaluation set:

• Loss: 0.6795
• Answer: {'precision': 0.7247807017543859, 'recall': 0.8170580964153276, 'f1': 0.768158047646717, 'number': 809}
• Header: {'precision': 0.3208955223880597, 'recall': 0.36134453781512604, 'f1': 0.33992094861660077, 'number': 119}
• Question: {'precision': 0.7793721973094171, 'recall': 0.815962441314554, 'f1': 0.7972477064220184, 'number': 1065}
• Overall Precision: 0.7279
• Overall Recall: 0.7893
• Overall F1: 0.7573
• Overall Accuracy: 0.8097

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 15
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Answer	Header	Question	Overall Precision	Overall Recall	Overall F1	Overall Accuracy
1.7157	1.0	10	1.4996	{'precision': 0.07431693989071038, 'recall': 0.08405438813349815, 'f1': 0.0788863109048724, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.22849213691026826, 'recall': 0.231924882629108, 'f1': 0.23019571295433364, 'number': 1065}	0.1578	0.1581	0.1579	0.4464
1.3388	2.0	20	1.1680	{'precision': 0.2980891719745223, 'recall': 0.2892459826946848, 'f1': 0.2936010037641154, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.49192245557350567, 'recall': 0.571830985915493, 'f1': 0.5288753799392097, 'number': 1065}	0.4167	0.4230	0.4198	0.6170
1.0157	3.0	30	0.8917	{'precision': 0.5579302587176603, 'recall': 0.6131025957972805, 'f1': 0.5842167255594817, 'number': 809}	{'precision': 0.13513513513513514, 'recall': 0.04201680672268908, 'f1': 0.06410256410256411, 'number': 119}	{'precision': 0.5931528662420382, 'recall': 0.6995305164319249, 'f1': 0.6419646704006894, 'number': 1065}	0.5710	0.6252	0.5969	0.7325
0.7726	4.0	40	0.7448	{'precision': 0.6359875904860393, 'recall': 0.7601977750309024, 'f1': 0.6925675675675675, 'number': 809}	{'precision': 0.29850746268656714, 'recall': 0.16806722689075632, 'f1': 0.21505376344086022, 'number': 119}	{'precision': 0.691696113074205, 'recall': 0.7352112676056338, 'f1': 0.7127901684114702, 'number': 1065}	0.6547	0.7115	0.6819	0.7725
0.6355	5.0	50	0.6844	{'precision': 0.6714129244249726, 'recall': 0.757725587144623, 'f1': 0.7119628339140535, 'number': 809}	{'precision': 0.3170731707317073, 'recall': 0.2184873949579832, 'f1': 0.25870646766169153, 'number': 119}	{'precision': 0.705, 'recall': 0.7943661971830986, 'f1': 0.7470198675496689, 'number': 1065}	0.6765	0.7451	0.7092	0.7944
0.5353	6.0	60	0.6699	{'precision': 0.6676860346585117, 'recall': 0.8096415327564895, 'f1': 0.7318435754189945, 'number': 809}	{'precision': 0.3068181818181818, 'recall': 0.226890756302521, 'f1': 0.2608695652173913, 'number': 119}	{'precision': 0.7171717171717171, 'recall': 0.8, 'f1': 0.7563249001331558, 'number': 1065}	0.6797	0.7697	0.7219	0.7951
0.4614	7.0	70	0.6517	{'precision': 0.7006507592190889, 'recall': 0.7985166872682324, 'f1': 0.7463893703061815, 'number': 809}	{'precision': 0.26495726495726496, 'recall': 0.2605042016806723, 'f1': 0.2627118644067797, 'number': 119}	{'precision': 0.7355442176870748, 'recall': 0.812206572769953, 'f1': 0.7719767960731816, 'number': 1065}	0.6962	0.7737	0.7329	0.8045
0.4076	8.0	80	0.6567	{'precision': 0.7194719471947195, 'recall': 0.8084054388133498, 'f1': 0.761350407450524, 'number': 809}	{'precision': 0.2713178294573643, 'recall': 0.29411764705882354, 'f1': 0.28225806451612906, 'number': 119}	{'precision': 0.7508561643835616, 'recall': 0.8234741784037559, 'f1': 0.7854903716972683, 'number': 1065}	0.7099	0.7858	0.7459	0.8056
0.3664	9.0	90	0.6529	{'precision': 0.7176724137931034, 'recall': 0.823238566131026, 'f1': 0.7668393782383419, 'number': 809}	{'precision': 0.265625, 'recall': 0.2857142857142857, 'f1': 0.27530364372469635, 'number': 119}	{'precision': 0.7686768676867687, 'recall': 0.8018779342723005, 'f1': 0.7849264705882354, 'number': 1065}	0.7171	0.7797	0.7471	0.8098
0.3515	10.0	100	0.6537	{'precision': 0.7129032258064516, 'recall': 0.8195302843016069, 'f1': 0.7625071880391028, 'number': 809}	{'precision': 0.32075471698113206, 'recall': 0.2857142857142857, 'f1': 0.30222222222222217, 'number': 119}	{'precision': 0.7788632326820604, 'recall': 0.8234741784037559, 'f1': 0.8005476951163851, 'number': 1065}	0.7280	0.7898	0.7576	0.8157
0.3077	11.0	110	0.6694	{'precision': 0.7241379310344828, 'recall': 0.8046971569839307, 'f1': 0.7622950819672132, 'number': 809}	{'precision': 0.302158273381295, 'recall': 0.35294117647058826, 'f1': 0.3255813953488373, 'number': 119}	{'precision': 0.7599653379549394, 'recall': 0.8234741784037559, 'f1': 0.790446146913024, 'number': 1065}	0.7162	0.7878	0.7503	0.8078
0.2941	12.0	120	0.6687	{'precision': 0.7135076252723311, 'recall': 0.8096415327564895, 'f1': 0.7585408222350897, 'number': 809}	{'precision': 0.3053435114503817, 'recall': 0.33613445378151263, 'f1': 0.32000000000000006, 'number': 119}	{'precision': 0.7793721973094171, 'recall': 0.815962441314554, 'f1': 0.7972477064220184, 'number': 1065}	0.7227	0.7847	0.7525	0.8118
0.2726	13.0	130	0.6769	{'precision': 0.720348204570185, 'recall': 0.8182941903584673, 'f1': 0.7662037037037037, 'number': 809}	{'precision': 0.33064516129032256, 'recall': 0.3445378151260504, 'f1': 0.33744855967078186, 'number': 119}	{'precision': 0.7713280562884784, 'recall': 0.8234741784037559, 'f1': 0.7965485921889193, 'number': 1065}	0.7248	0.7928	0.7572	0.8095
0.2575	14.0	140	0.6821	{'precision': 0.7238723872387238, 'recall': 0.8133498145859085, 'f1': 0.7660069848661233, 'number': 809}	{'precision': 0.3333333333333333, 'recall': 0.3697478991596639, 'f1': 0.350597609561753, 'number': 119}	{'precision': 0.7810545129579982, 'recall': 0.8206572769953052, 'f1': 0.8003663003663004, 'number': 1065}	0.7296	0.7908	0.7590	0.8095
0.2563	15.0	150	0.6795	{'precision': 0.7247807017543859, 'recall': 0.8170580964153276, 'f1': 0.768158047646717, 'number': 809}	{'precision': 0.3208955223880597, 'recall': 0.36134453781512604, 'f1': 0.33992094861660077, 'number': 119}	{'precision': 0.7793721973094171, 'recall': 0.815962441314554, 'f1': 0.7972477064220184, 'number': 1065}	0.7279	0.7893	0.7573	0.8097

Framework versions

• Transformers 4.47.0
• Pytorch 2.5.1+cpu
• Datasets 3.2.0
• Tokenizers 0.21.0