|
--- |
|
license: mit |
|
base_model: microsoft/layoutlm-base-uncased |
|
tags: |
|
- generated_from_trainer |
|
datasets: |
|
- funsd |
|
model-index: |
|
- name: layoutlm-funsd |
|
results: [] |
|
--- |
|
|
|
<!-- This model card has been generated automatically according to the information the Trainer had access to. You |
|
should probably proofread and complete it, then remove this comment. --> |
|
|
|
# layoutlm-funsd |
|
|
|
This model is a fine-tuned version of [microsoft/layoutlm-base-uncased](https://huggingface.co/microsoft/layoutlm-base-uncased) on the funsd dataset. |
|
It achieves the following results on the evaluation set: |
|
- Loss: 0.7279 |
|
- Answer: {'precision': 0.706858407079646, 'recall': 0.7898640296662547, 'f1': 0.7460595446584939, 'number': 809} |
|
- Header: {'precision': 0.312, 'recall': 0.3277310924369748, 'f1': 0.31967213114754095, 'number': 119} |
|
- Question: {'precision': 0.7741935483870968, 'recall': 0.8338028169014085, 'f1': 0.8028933092224232, 'number': 1065} |
|
- Overall Precision: 0.7197 |
|
- Overall Recall: 0.7858 |
|
- Overall F1: 0.7513 |
|
- Overall Accuracy: 0.8034 |
|
|
|
## Model description |
|
|
|
More information needed |
|
|
|
## Intended uses & limitations |
|
|
|
More information needed |
|
|
|
## Training and evaluation data |
|
|
|
More information needed |
|
|
|
## Training procedure |
|
|
|
### Training hyperparameters |
|
|
|
The following hyperparameters were used during training: |
|
- learning_rate: 3e-05 |
|
- train_batch_size: 16 |
|
- eval_batch_size: 8 |
|
- seed: 42 |
|
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 |
|
- lr_scheduler_type: linear |
|
- num_epochs: 15 |
|
- mixed_precision_training: Native AMP |
|
|
|
### Training results |
|
|
|
| Training Loss | Epoch | Step | Validation Loss | Answer | Header | Question | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy | |
|
|:-------------:|:-----:|:----:|:---------------:|:-----------------------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------:|:-----------------:|:--------------:|:----------:|:----------------:| |
|
| 1.7555 | 1.0 | 10 | 1.5320 | {'precision': 0.02317596566523605, 'recall': 0.03337453646477132, 'f1': 0.02735562310030395, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.11706783369803063, 'recall': 0.10046948356807511, 'f1': 0.10813542193026779, 'number': 1065} | 0.0645 | 0.0672 | 0.0658 | 0.3975 | |
|
| 1.3972 | 2.0 | 20 | 1.1941 | {'precision': 0.2606060606060606, 'recall': 0.2657601977750309, 'f1': 0.2631578947368421, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.4398805078416729, 'recall': 0.5530516431924882, 'f1': 0.49001663893510816, 'number': 1065} | 0.3715 | 0.4034 | 0.3868 | 0.6107 | |
|
| 1.0502 | 3.0 | 30 | 0.9315 | {'precision': 0.5343347639484979, 'recall': 0.61557478368356, 'f1': 0.5720850086157381, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.5706304868316041, 'recall': 0.6713615023474179, 'f1': 0.6169111302847283, 'number': 1065} | 0.5484 | 0.6086 | 0.5769 | 0.7259 | |
|
| 0.8107 | 4.0 | 40 | 0.7975 | {'precision': 0.6146288209606987, 'recall': 0.695920889987639, 'f1': 0.6527536231884058, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.6344605475040258, 'recall': 0.739906103286385, 'f1': 0.683138274815778, 'number': 1065} | 0.6124 | 0.6779 | 0.6435 | 0.7593 | |
|
| 0.6553 | 5.0 | 50 | 0.7487 | {'precision': 0.6507760532150776, 'recall': 0.7255871446229913, 'f1': 0.6861484511981296, 'number': 809} | {'precision': 0.12, 'recall': 0.07563025210084033, 'f1': 0.09278350515463916, 'number': 119} | {'precision': 0.6690085870413739, 'recall': 0.8046948356807512, 'f1': 0.7306052855924979, 'number': 1065} | 0.6435 | 0.7291 | 0.6836 | 0.7719 | |
|
| 0.5642 | 6.0 | 60 | 0.7147 | {'precision': 0.6557203389830508, 'recall': 0.765142150803461, 'f1': 0.7062179121505989, 'number': 809} | {'precision': 0.20833333333333334, 'recall': 0.12605042016806722, 'f1': 0.15706806282722513, 'number': 119} | {'precision': 0.7058333333333333, 'recall': 0.7953051643192488, 'f1': 0.7479028697571745, 'number': 1065} | 0.6683 | 0.7431 | 0.7037 | 0.7847 | |
|
| 0.4833 | 7.0 | 70 | 0.6895 | {'precision': 0.6919691969196919, 'recall': 0.7775030902348579, 'f1': 0.7322467986030267, 'number': 809} | {'precision': 0.25742574257425743, 'recall': 0.2184873949579832, 'f1': 0.23636363636363636, 'number': 119} | {'precision': 0.7268041237113402, 'recall': 0.7943661971830986, 'f1': 0.7590847913862719, 'number': 1065} | 0.6904 | 0.7531 | 0.7204 | 0.7920 | |
|
| 0.4356 | 8.0 | 80 | 0.6896 | {'precision': 0.6869109947643979, 'recall': 0.8108776266996292, 'f1': 0.7437641723356009, 'number': 809} | {'precision': 0.27884615384615385, 'recall': 0.24369747899159663, 'f1': 0.2600896860986547, 'number': 119} | {'precision': 0.730185497470489, 'recall': 0.8131455399061033, 'f1': 0.7694358063083073, 'number': 1065} | 0.6909 | 0.7782 | 0.7319 | 0.7919 | |
|
| 0.3874 | 9.0 | 90 | 0.7000 | {'precision': 0.7158836689038032, 'recall': 0.7911001236093943, 'f1': 0.7516147974163241, 'number': 809} | {'precision': 0.2818181818181818, 'recall': 0.2605042016806723, 'f1': 0.27074235807860264, 'number': 119} | {'precision': 0.7395388556789069, 'recall': 0.8131455399061033, 'f1': 0.7745974955277279, 'number': 1065} | 0.7067 | 0.7712 | 0.7375 | 0.7957 | |
|
| 0.3771 | 10.0 | 100 | 0.7200 | {'precision': 0.7097142857142857, 'recall': 0.7676143386897404, 'f1': 0.7375296912114014, 'number': 809} | {'precision': 0.272, 'recall': 0.2857142857142857, 'f1': 0.27868852459016397, 'number': 119} | {'precision': 0.7438715131022823, 'recall': 0.8262910798122066, 'f1': 0.7829181494661922, 'number': 1065} | 0.7032 | 0.7702 | 0.7352 | 0.7886 | |
|
| 0.3231 | 11.0 | 110 | 0.7101 | {'precision': 0.7115177610333692, 'recall': 0.8170580964153276, 'f1': 0.760644418872267, 'number': 809} | {'precision': 0.29133858267716534, 'recall': 0.31092436974789917, 'f1': 0.3008130081300813, 'number': 119} | {'precision': 0.7672188317349607, 'recall': 0.8262910798122066, 'f1': 0.7956600361663653, 'number': 1065} | 0.7163 | 0.7918 | 0.7521 | 0.8000 | |
|
| 0.3026 | 12.0 | 120 | 0.7186 | {'precision': 0.7138009049773756, 'recall': 0.7799752781211372, 'f1': 0.7454223272297696, 'number': 809} | {'precision': 0.325, 'recall': 0.3277310924369748, 'f1': 0.3263598326359833, 'number': 119} | {'precision': 0.7658833768494343, 'recall': 0.8262910798122066, 'f1': 0.7949412827461607, 'number': 1065} | 0.7199 | 0.7777 | 0.7477 | 0.7984 | |
|
| 0.2876 | 13.0 | 130 | 0.7213 | {'precision': 0.7156862745098039, 'recall': 0.8121137206427689, 'f1': 0.7608569774174871, 'number': 809} | {'precision': 0.325, 'recall': 0.3277310924369748, 'f1': 0.3263598326359833, 'number': 119} | {'precision': 0.7737676056338029, 'recall': 0.8253521126760563, 'f1': 0.7987278509768287, 'number': 1065} | 0.7245 | 0.7903 | 0.7559 | 0.8025 | |
|
| 0.2709 | 14.0 | 140 | 0.7269 | {'precision': 0.7147613762486127, 'recall': 0.796044499381953, 'f1': 0.7532163742690058, 'number': 809} | {'precision': 0.30158730158730157, 'recall': 0.31932773109243695, 'f1': 0.310204081632653, 'number': 119} | {'precision': 0.7709059233449478, 'recall': 0.8309859154929577, 'f1': 0.7998192498870312, 'number': 1065} | 0.7205 | 0.7863 | 0.7519 | 0.8033 | |
|
| 0.2745 | 15.0 | 150 | 0.7279 | {'precision': 0.706858407079646, 'recall': 0.7898640296662547, 'f1': 0.7460595446584939, 'number': 809} | {'precision': 0.312, 'recall': 0.3277310924369748, 'f1': 0.31967213114754095, 'number': 119} | {'precision': 0.7741935483870968, 'recall': 0.8338028169014085, 'f1': 0.8028933092224232, 'number': 1065} | 0.7197 | 0.7858 | 0.7513 | 0.8034 | |
|
|
|
|
|
### Framework versions |
|
|
|
- Transformers 4.42.4 |
|
- Pytorch 2.4.0+cu121 |
|
- Datasets 2.21.0 |
|
- Tokenizers 0.19.1 |
|
|
