Training complete

README.md

@@ -0,0 +1,95 @@
+---
+tags:
+- summarization
+- generated_from_trainer
+metrics:
+- rouge
+model-index:
+- name: exp2-led-risalah_data_v6
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="200" height="32"/>](https://wandb.ai/silmiaulia/huggingface/runs/7pt54hkh)
+# exp2-led-risalah_data_v6
+
+This model was trained from scratch on an unknown dataset.
+It achieves the following results on the evaluation set:
+- Loss: 1.7971
+- Rouge1: 35.2105
+- Rouge2: 14.2825
+- Rougel: 18.7356
+- Rougelsum: 33.8518
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 2e-05
+- train_batch_size: 1
+- eval_batch_size: 1
+- seed: 42
+- gradient_accumulation_steps: 4
+- total_train_batch_size: 4
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: linear
+- lr_scheduler_warmup_steps: 200
+- num_epochs: 30
+- mixed_precision_training: Native AMP
+
+### Training results
+
+| Training Loss | Epoch | Step | Validation Loss | Rouge1  | Rouge2  | Rougel  | Rougelsum |
+|:-------------:|:-----:|:----:|:---------------:|:-------:|:-------:|:-------:|:---------:|
+| 3.6579        | 1.0   | 20   | 2.9715          | 15.0902 | 2.4602  | 8.8016  | 14.54     |
+| 3.8322        | 2.0   | 40   | 2.6400          | 19.8395 | 3.3421  | 10.0041 | 18.8629   |
+| 3.3928        | 3.0   | 60   | 2.4780          | 24.1438 | 4.9177  | 11.8553 | 23.0439   |
+| 3.1159        | 4.0   | 80   | 2.3336          | 26.1339 | 5.4015  | 12.1066 | 24.5407   |
+| 2.8469        | 5.0   | 100  | 2.2554          | 25.3388 | 5.6665  | 12.0706 | 24.1799   |
+| 2.6486        | 6.0   | 120  | 2.1842          | 33.8164 | 9.2363  | 15.7673 | 31.606    |
+| 2.5429        | 7.0   | 140  | 2.1322          | 32.5361 | 8.5141  | 15.3201 | 30.935    |
+| 2.3159        | 8.0   | 160  | 2.0631          | 32.3657 | 9.171   | 15.1179 | 30.9634   |
+| 2.1821        | 10.0  | 200  | 1.9358          | 33.1626 | 10.8072 | 16.5887 | 31.0652   |
+| 2.2141        | 11.0  | 220  | 1.9274          | 36.3525 | 13.5885 | 18.4941 | 34.9263   |
+| 2.1213        | 12.0  | 240  | 1.9033          | 34.4359 | 11.4335 | 17.8322 | 32.5781   |
+| 1.9791        | 13.0  | 260  | 1.8914          | 37.0733 | 14.2739 | 18.9338 | 35.5985   |
+| 1.9504        | 14.0  | 280  | 1.8642          | 34.7529 | 13.0325 | 18.1055 | 33.257    |
+| 1.9848        | 15.0  | 300  | 1.8641          | 35.9266 | 13.4528 | 18.459  | 34.0294   |
+| 1.845         | 16.0  | 320  | 1.8507          | 37.7424 | 15.2488 | 18.993  | 35.4955   |
+| 1.8049        | 17.0  | 340  | 1.8390          | 36.5023 | 13.6069 | 18.4956 | 34.883    |
+| 1.8158        | 18.0  | 360  | 1.8393          | 34.4722 | 13.6438 | 18.1636 | 32.4511   |
+| 1.8541        | 19.0  | 380  | 1.8395          | 37.0215 | 14.3221 | 19.6743 | 35.3083   |
+| 1.7967        | 20.0  | 400  | 1.8403          | 36.3048 | 13.3475 | 19.9887 | 34.6884   |
+| 1.7285        | 21.0  | 420  | 1.8394          | 36.4051 | 14.3198 | 19.4997 | 34.9803   |
+| 1.7303        | 22.0  | 440  | 1.8287          | 36.1003 | 14.166  | 17.8619 | 34.3505   |
+| 1.6976        | 23.0  | 460  | 1.8040          | 34.3036 | 12.8173 | 18.6643 | 32.6019   |
+| 1.6916        | 24.0  | 480  | 1.7963          | 34.7753 | 14.0332 | 18.923  | 33.3743   |
+| 1.6872        | 25.0  | 500  | 1.8073          | 37.0718 | 14.6821 | 20.1188 | 35.7824   |
+| 1.6979        | 26.0  | 520  | 1.8340          | 37.1726 | 15.1384 | 20.2153 | 36.3188   |
+| 1.6867        | 27.0  | 540  | 1.8000          | 37.2831 | 14.2806 | 19.1448 | 36.1598   |
+| 1.6959        | 28.0  | 560  | 1.7886          | 34.8414 | 13.5902 | 18.5803 | 33.5383   |
+| 1.7546        | 29.0  | 580  | 1.8068          | 37.6551 | 16.1055 | 20.2492 | 36.1177   |
+| 1.632         | 30.0  | 600  | 1.7971          | 35.2105 | 14.2825 | 18.7356 | 33.8518   |
+
+
+### Framework versions
+
+- Transformers 4.42.3
+- Pytorch 2.1.2
+- Datasets 2.20.0
+- Tokenizers 0.19.1

generation_config.json

@@ -0,0 +1,14 @@
+{
+  "bos_token_id": 0,
+  "decoder_start_token_id": 2,
+  "early_stopping": true,
+  "eos_token_id": 2,
+  "length_penalty": 2.0,
+  "max_length": 1024,
+  "min_length": 128,
+  "no_repeat_ngram_size": 3,
+  "num_beams": 2,
+  "pad_token_id": 1,
+  "transformers_version": "4.42.3",
+  "use_cache": false
+}

runs/Jul26_03-38-16_be52605a26a8/events.out.tfevents.1721965102.be52605a26a8.34.0

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:bbd7c872f5a7a200393cb153fe65641cb1ff94b2ad262cee0452bc4eac61d50a
-size 14903
+oid sha256:4dc202bf2562238892430d0e475739cdaa3f62b601919ead8feb3a7cfa7e5c0d
+size 15731

runs/Jul26_03-38-16_be52605a26a8/events.out.tfevents.1721980259.be52605a26a8.34.1

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5ffca5189886e3cd3769a946de4a9aa79652dffb589f90e15ca78324f38ebe5c
+size 562