File size: 9,321 Bytes
a2a1a5e |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 |
---
license: mit
base_model: microsoft/layoutlm-base-uncased
tags:
- generated_from_trainer
datasets:
- funsd
model-index:
- name: layoutlm-funsd
results: []
---
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->
# layoutlm-funsd
This model is a fine-tuned version of [microsoft/layoutlm-base-uncased](https://huggingface.co/microsoft/layoutlm-base-uncased) on the funsd dataset.
It achieves the following results on the evaluation set:
- Loss: 0.7279
- Answer: {'precision': 0.706858407079646, 'recall': 0.7898640296662547, 'f1': 0.7460595446584939, 'number': 809}
- Header: {'precision': 0.312, 'recall': 0.3277310924369748, 'f1': 0.31967213114754095, 'number': 119}
- Question: {'precision': 0.7741935483870968, 'recall': 0.8338028169014085, 'f1': 0.8028933092224232, 'number': 1065}
- Overall Precision: 0.7197
- Overall Recall: 0.7858
- Overall F1: 0.7513
- Overall Accuracy: 0.8034
## Model description
More information needed
## Intended uses & limitations
More information needed
## Training and evaluation data
More information needed
## Training procedure
### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 3e-05
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 15
- mixed_precision_training: Native AMP
### Training results
| Training Loss | Epoch | Step | Validation Loss | Answer | Header | Question | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:-----------------------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------:|:-----------------:|:--------------:|:----------:|:----------------:|
| 1.7555 | 1.0 | 10 | 1.5320 | {'precision': 0.02317596566523605, 'recall': 0.03337453646477132, 'f1': 0.02735562310030395, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.11706783369803063, 'recall': 0.10046948356807511, 'f1': 0.10813542193026779, 'number': 1065} | 0.0645 | 0.0672 | 0.0658 | 0.3975 |
| 1.3972 | 2.0 | 20 | 1.1941 | {'precision': 0.2606060606060606, 'recall': 0.2657601977750309, 'f1': 0.2631578947368421, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.4398805078416729, 'recall': 0.5530516431924882, 'f1': 0.49001663893510816, 'number': 1065} | 0.3715 | 0.4034 | 0.3868 | 0.6107 |
| 1.0502 | 3.0 | 30 | 0.9315 | {'precision': 0.5343347639484979, 'recall': 0.61557478368356, 'f1': 0.5720850086157381, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.5706304868316041, 'recall': 0.6713615023474179, 'f1': 0.6169111302847283, 'number': 1065} | 0.5484 | 0.6086 | 0.5769 | 0.7259 |
| 0.8107 | 4.0 | 40 | 0.7975 | {'precision': 0.6146288209606987, 'recall': 0.695920889987639, 'f1': 0.6527536231884058, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.6344605475040258, 'recall': 0.739906103286385, 'f1': 0.683138274815778, 'number': 1065} | 0.6124 | 0.6779 | 0.6435 | 0.7593 |
| 0.6553 | 5.0 | 50 | 0.7487 | {'precision': 0.6507760532150776, 'recall': 0.7255871446229913, 'f1': 0.6861484511981296, 'number': 809} | {'precision': 0.12, 'recall': 0.07563025210084033, 'f1': 0.09278350515463916, 'number': 119} | {'precision': 0.6690085870413739, 'recall': 0.8046948356807512, 'f1': 0.7306052855924979, 'number': 1065} | 0.6435 | 0.7291 | 0.6836 | 0.7719 |
| 0.5642 | 6.0 | 60 | 0.7147 | {'precision': 0.6557203389830508, 'recall': 0.765142150803461, 'f1': 0.7062179121505989, 'number': 809} | {'precision': 0.20833333333333334, 'recall': 0.12605042016806722, 'f1': 0.15706806282722513, 'number': 119} | {'precision': 0.7058333333333333, 'recall': 0.7953051643192488, 'f1': 0.7479028697571745, 'number': 1065} | 0.6683 | 0.7431 | 0.7037 | 0.7847 |
| 0.4833 | 7.0 | 70 | 0.6895 | {'precision': 0.6919691969196919, 'recall': 0.7775030902348579, 'f1': 0.7322467986030267, 'number': 809} | {'precision': 0.25742574257425743, 'recall': 0.2184873949579832, 'f1': 0.23636363636363636, 'number': 119} | {'precision': 0.7268041237113402, 'recall': 0.7943661971830986, 'f1': 0.7590847913862719, 'number': 1065} | 0.6904 | 0.7531 | 0.7204 | 0.7920 |
| 0.4356 | 8.0 | 80 | 0.6896 | {'precision': 0.6869109947643979, 'recall': 0.8108776266996292, 'f1': 0.7437641723356009, 'number': 809} | {'precision': 0.27884615384615385, 'recall': 0.24369747899159663, 'f1': 0.2600896860986547, 'number': 119} | {'precision': 0.730185497470489, 'recall': 0.8131455399061033, 'f1': 0.7694358063083073, 'number': 1065} | 0.6909 | 0.7782 | 0.7319 | 0.7919 |
| 0.3874 | 9.0 | 90 | 0.7000 | {'precision': 0.7158836689038032, 'recall': 0.7911001236093943, 'f1': 0.7516147974163241, 'number': 809} | {'precision': 0.2818181818181818, 'recall': 0.2605042016806723, 'f1': 0.27074235807860264, 'number': 119} | {'precision': 0.7395388556789069, 'recall': 0.8131455399061033, 'f1': 0.7745974955277279, 'number': 1065} | 0.7067 | 0.7712 | 0.7375 | 0.7957 |
| 0.3771 | 10.0 | 100 | 0.7200 | {'precision': 0.7097142857142857, 'recall': 0.7676143386897404, 'f1': 0.7375296912114014, 'number': 809} | {'precision': 0.272, 'recall': 0.2857142857142857, 'f1': 0.27868852459016397, 'number': 119} | {'precision': 0.7438715131022823, 'recall': 0.8262910798122066, 'f1': 0.7829181494661922, 'number': 1065} | 0.7032 | 0.7702 | 0.7352 | 0.7886 |
| 0.3231 | 11.0 | 110 | 0.7101 | {'precision': 0.7115177610333692, 'recall': 0.8170580964153276, 'f1': 0.760644418872267, 'number': 809} | {'precision': 0.29133858267716534, 'recall': 0.31092436974789917, 'f1': 0.3008130081300813, 'number': 119} | {'precision': 0.7672188317349607, 'recall': 0.8262910798122066, 'f1': 0.7956600361663653, 'number': 1065} | 0.7163 | 0.7918 | 0.7521 | 0.8000 |
| 0.3026 | 12.0 | 120 | 0.7186 | {'precision': 0.7138009049773756, 'recall': 0.7799752781211372, 'f1': 0.7454223272297696, 'number': 809} | {'precision': 0.325, 'recall': 0.3277310924369748, 'f1': 0.3263598326359833, 'number': 119} | {'precision': 0.7658833768494343, 'recall': 0.8262910798122066, 'f1': 0.7949412827461607, 'number': 1065} | 0.7199 | 0.7777 | 0.7477 | 0.7984 |
| 0.2876 | 13.0 | 130 | 0.7213 | {'precision': 0.7156862745098039, 'recall': 0.8121137206427689, 'f1': 0.7608569774174871, 'number': 809} | {'precision': 0.325, 'recall': 0.3277310924369748, 'f1': 0.3263598326359833, 'number': 119} | {'precision': 0.7737676056338029, 'recall': 0.8253521126760563, 'f1': 0.7987278509768287, 'number': 1065} | 0.7245 | 0.7903 | 0.7559 | 0.8025 |
| 0.2709 | 14.0 | 140 | 0.7269 | {'precision': 0.7147613762486127, 'recall': 0.796044499381953, 'f1': 0.7532163742690058, 'number': 809} | {'precision': 0.30158730158730157, 'recall': 0.31932773109243695, 'f1': 0.310204081632653, 'number': 119} | {'precision': 0.7709059233449478, 'recall': 0.8309859154929577, 'f1': 0.7998192498870312, 'number': 1065} | 0.7205 | 0.7863 | 0.7519 | 0.8033 |
| 0.2745 | 15.0 | 150 | 0.7279 | {'precision': 0.706858407079646, 'recall': 0.7898640296662547, 'f1': 0.7460595446584939, 'number': 809} | {'precision': 0.312, 'recall': 0.3277310924369748, 'f1': 0.31967213114754095, 'number': 119} | {'precision': 0.7741935483870968, 'recall': 0.8338028169014085, 'f1': 0.8028933092224232, 'number': 1065} | 0.7197 | 0.7858 | 0.7513 | 0.8034 |
### Framework versions
- Transformers 4.42.4
- Pytorch 2.4.0+cu121
- Datasets 2.21.0
- Tokenizers 0.19.1
|