layoutlm-funsd

This model is a fine-tuned version of microsoft/layoutlm-base-uncased on the funsd dataset. It achieves the following results on the evaluation set:

• Loss: 0.6932
• Answer: {'precision': 0.7073434125269978, 'recall': 0.8096415327564895, 'f1': 0.755043227665706, 'number': 809}
• Header: {'precision': 0.25925925925925924, 'recall': 0.23529411764705882, 'f1': 0.24669603524229072, 'number': 119}
• Question: {'precision': 0.7767158992180713, 'recall': 0.8394366197183099, 'f1': 0.8068592057761733, 'number': 1065}
• Overall Precision: 0.7217
• Overall Recall: 0.7913
• Overall F1: 0.7549
• Overall Accuracy: 0.8047

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 32
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 20
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Answer	Header	Question	Overall Precision	Overall Recall	Overall F1	Overall Accuracy
1.8958	1.0	5	1.7117	{'precision': 0.015232292460015232, 'recall': 0.024721878862793572, 'f1': 0.01885014137606032, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.11567164179104478, 'recall': 0.11643192488262911, 'f1': 0.11605053813757606, 'number': 1065}	0.0601	0.0723	0.0656	0.3457
1.632	2.0	10	1.4951	{'precision': 0.027700831024930747, 'recall': 0.037082818294190356, 'f1': 0.03171247357293869, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.3054968287526427, 'recall': 0.27136150234741785, 'f1': 0.2874191944306315, 'number': 1065}	0.1571	0.1601	0.1586	0.4391
1.4434	3.0	15	1.2975	{'precision': 0.1747173689619733, 'recall': 0.21013597033374537, 'f1': 0.19079685746352415, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.39224704336399474, 'recall': 0.5605633802816902, 'f1': 0.46153846153846156, 'number': 1065}	0.3074	0.3848	0.3418	0.5806
1.2417	4.0	20	1.1193	{'precision': 0.3728470111448835, 'recall': 0.45488257107540175, 'f1': 0.40979955456570155, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.5278207109737248, 'recall': 0.6413145539906103, 'f1': 0.5790589232725731, 'number': 1065}	0.4608	0.5273	0.4918	0.6383
1.0407	5.0	25	0.9770	{'precision': 0.48043478260869565, 'recall': 0.546353522867738, 'f1': 0.5112781954887218, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.5961698584512906, 'recall': 0.672300469483568, 'f1': 0.6319505736981466, 'number': 1065}	0.5447	0.5810	0.5623	0.6890
0.8977	6.0	30	0.8662	{'precision': 0.559375, 'recall': 0.6637824474660075, 'f1': 0.6071226681741096, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.6157517899761337, 'recall': 0.7267605633802817, 'f1': 0.6666666666666667, 'number': 1065}	0.5845	0.6578	0.6190	0.7313
0.787	7.0	35	0.8246	{'precision': 0.6031589338598223, 'recall': 0.7552533992583437, 'f1': 0.6706915477497257, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.6820194862710364, 'recall': 0.7230046948356808, 'f1': 0.7019143117593437, 'number': 1065}	0.6332	0.6929	0.6617	0.7434
0.6913	8.0	40	0.7630	{'precision': 0.6112820512820513, 'recall': 0.7367119901112484, 'f1': 0.6681614349775784, 'number': 809}	{'precision': 0.02702702702702703, 'recall': 0.01680672268907563, 'f1': 0.02072538860103627, 'number': 119}	{'precision': 0.6604795050270689, 'recall': 0.8018779342723005, 'f1': 0.7243426632739611, 'number': 1065}	0.6200	0.7285	0.6699	0.7753
0.6196	9.0	45	0.7427	{'precision': 0.629145728643216, 'recall': 0.7737948084054388, 'f1': 0.6940133037694012, 'number': 809}	{'precision': 0.08108108108108109, 'recall': 0.05042016806722689, 'f1': 0.062176165803108814, 'number': 119}	{'precision': 0.7155247181266262, 'recall': 0.7746478873239436, 'f1': 0.7439134355275022, 'number': 1065}	0.6557	0.7311	0.6913	0.7750
0.57	10.0	50	0.7250	{'precision': 0.6613588110403397, 'recall': 0.7700865265760197, 'f1': 0.7115933752141633, 'number': 809}	{'precision': 0.14285714285714285, 'recall': 0.09243697478991597, 'f1': 0.11224489795918367, 'number': 119}	{'precision': 0.6997538966365874, 'recall': 0.8009389671361502, 'f1': 0.7469352014010507, 'number': 1065}	0.6644	0.7461	0.7029	0.7852
0.5314	11.0	55	0.7002	{'precision': 0.6476578411405295, 'recall': 0.7861557478368356, 'f1': 0.7102177554438861, 'number': 809}	{'precision': 0.16216216216216217, 'recall': 0.10084033613445378, 'f1': 0.12435233160621763, 'number': 119}	{'precision': 0.7123745819397993, 'recall': 0.8, 'f1': 0.7536488279522335, 'number': 1065}	0.6661	0.7526	0.7067	0.7901
0.4856	12.0	60	0.6901	{'precision': 0.682454251883746, 'recall': 0.7836835599505563, 'f1': 0.7295742232451092, 'number': 809}	{'precision': 0.2717391304347826, 'recall': 0.21008403361344538, 'f1': 0.23696682464454974, 'number': 119}	{'precision': 0.7309602649006622, 'recall': 0.8291079812206573, 'f1': 0.7769467663880335, 'number': 1065}	0.6918	0.7737	0.7305	0.8001
0.4486	13.0	65	0.6915	{'precision': 0.6917211328976035, 'recall': 0.7849196538936959, 'f1': 0.7353792704111175, 'number': 809}	{'precision': 0.25806451612903225, 'recall': 0.20168067226890757, 'f1': 0.22641509433962265, 'number': 119}	{'precision': 0.7395659432387313, 'recall': 0.831924882629108, 'f1': 0.7830313742819267, 'number': 1065}	0.6994	0.7752	0.7354	0.8011
0.4177	14.0	70	0.6835	{'precision': 0.6862326574172892, 'recall': 0.7948084054388134, 'f1': 0.736540664375716, 'number': 809}	{'precision': 0.24731182795698925, 'recall': 0.19327731092436976, 'f1': 0.2169811320754717, 'number': 119}	{'precision': 0.7416666666666667, 'recall': 0.8356807511737089, 'f1': 0.7858719646799116, 'number': 1065}	0.6978	0.7807	0.7369	0.8026
0.3927	15.0	75	0.6891	{'precision': 0.7016393442622951, 'recall': 0.7935723114956736, 'f1': 0.7447795823665894, 'number': 809}	{'precision': 0.2358490566037736, 'recall': 0.21008403361344538, 'f1': 0.22222222222222224, 'number': 119}	{'precision': 0.7632478632478632, 'recall': 0.8384976525821596, 'f1': 0.7991051454138701, 'number': 1065}	0.7120	0.7827	0.7457	0.8038
0.3743	16.0	80	0.6904	{'precision': 0.7064622124863089, 'recall': 0.7972805933250927, 'f1': 0.7491289198606274, 'number': 809}	{'precision': 0.27450980392156865, 'recall': 0.23529411764705882, 'f1': 0.2533936651583711, 'number': 119}	{'precision': 0.7675814751286449, 'recall': 0.8403755868544601, 'f1': 0.8023307933662035, 'number': 1065}	0.7189	0.7868	0.7513	0.8034
0.3683	17.0	85	0.6928	{'precision': 0.7037837837837838, 'recall': 0.8046971569839307, 'f1': 0.7508650519031143, 'number': 809}	{'precision': 0.2545454545454545, 'recall': 0.23529411764705882, 'f1': 0.2445414847161572, 'number': 119}	{'precision': 0.7735191637630662, 'recall': 0.8338028169014085, 'f1': 0.8025305015815634, 'number': 1065}	0.7178	0.7863	0.7505	0.8035
0.3553	18.0	90	0.6940	{'precision': 0.7045454545454546, 'recall': 0.8046971569839307, 'f1': 0.7512983266012695, 'number': 809}	{'precision': 0.25925925925925924, 'recall': 0.23529411764705882, 'f1': 0.24669603524229072, 'number': 119}	{'precision': 0.7739130434782608, 'recall': 0.8356807511737089, 'f1': 0.8036117381489841, 'number': 1065}	0.7191	0.7873	0.7516	0.8035
0.3471	19.0	95	0.6932	{'precision': 0.7071583514099783, 'recall': 0.8059332509270705, 'f1': 0.753321779318313, 'number': 809}	{'precision': 0.2545454545454545, 'recall': 0.23529411764705882, 'f1': 0.2445414847161572, 'number': 119}	{'precision': 0.7746967071057193, 'recall': 0.8394366197183099, 'f1': 0.8057683641279857, 'number': 1065}	0.7200	0.7898	0.7533	0.8043
0.3386	20.0	100	0.6932	{'precision': 0.7073434125269978, 'recall': 0.8096415327564895, 'f1': 0.755043227665706, 'number': 809}	{'precision': 0.25925925925925924, 'recall': 0.23529411764705882, 'f1': 0.24669603524229072, 'number': 119}	{'precision': 0.7767158992180713, 'recall': 0.8394366197183099, 'f1': 0.8068592057761733, 'number': 1065}	0.7217	0.7913	0.7549	0.8047

Framework versions

• Transformers 4.45.1
• Pytorch 2.4.0
• Datasets 3.0.1
• Tokenizers 0.20.0