---
license: apache-2.0
base_model: google/flan-t5-small
tags:
- generated_from_trainer
metrics:
- rouge
model-index:
- name: fine-tuned-flan-t5-20-epochs-wang-lab
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# fine-tuned-flan-t5-20-epochs-wang-lab

This model is a fine-tuned version of [google/flan-t5-small](https://huggingface.co/google/flan-t5-small) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 0.7769
- Rouge1: 0.2685
- Rouge2: 0.0913
- Rougel: 0.2316
- Rougelsum: 0.2318
- Gen Len: 19.51

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 4
- eval_batch_size: 4
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 20

### Training results

| Training Loss | Epoch | Step | Validation Loss | Rouge1 | Rouge2 | Rougel | Rougelsum | Gen Len |
|:-------------:|:-----:|:----:|:---------------:|:------:|:------:|:------:|:---------:|:-------:|
| No log        | 1.0   | 301  | 0.8814          | 0.2192 | 0.0734 | 0.1907 | 0.1911    | 19.4    |
| 1.4237        | 2.0   | 602  | 0.8212          | 0.2283 | 0.0703 | 0.1962 | 0.1977    | 19.39   |
| 1.4237        | 3.0   | 903  | 0.7864          | 0.2324 | 0.0656 | 0.1989 | 0.201     | 19.35   |
| 0.8542        | 4.0   | 1204 | 0.7753          | 0.2489 | 0.0789 | 0.2142 | 0.2156    | 19.27   |
| 0.7603        | 5.0   | 1505 | 0.7662          | 0.2458 | 0.0749 | 0.2119 | 0.2131    | 19.29   |
| 0.7603        | 6.0   | 1806 | 0.7608          | 0.2512 | 0.0896 | 0.2203 | 0.2217    | 19.48   |
| 0.699         | 7.0   | 2107 | 0.7611          | 0.259  | 0.0836 | 0.23   | 0.2318    | 19.52   |
| 0.699         | 8.0   | 2408 | 0.7566          | 0.264  | 0.087  | 0.2304 | 0.2312    | 19.43   |
| 0.646         | 9.0   | 2709 | 0.7558          | 0.2815 | 0.0947 | 0.2447 | 0.2453    | 19.43   |
| 0.6106        | 10.0  | 3010 | 0.7585          | 0.2692 | 0.0958 | 0.2377 | 0.2374    | 19.5    |
| 0.6106        | 11.0  | 3311 | 0.7611          | 0.2635 | 0.0997 | 0.2283 | 0.2278    | 19.5    |
| 0.5944        | 12.0  | 3612 | 0.7639          | 0.2785 | 0.0928 | 0.24   | 0.2403    | 19.46   |
| 0.5944        | 13.0  | 3913 | 0.7612          | 0.2648 | 0.0886 | 0.231  | 0.2318    | 19.38   |
| 0.5671        | 14.0  | 4214 | 0.7636          | 0.2786 | 0.0934 | 0.2431 | 0.2428    | 19.48   |
| 0.5387        | 15.0  | 4515 | 0.7656          | 0.2834 | 0.0938 | 0.2442 | 0.2432    | 19.47   |
| 0.5387        | 16.0  | 4816 | 0.7717          | 0.278  | 0.0927 | 0.2408 | 0.2403    | 19.48   |
| 0.5349        | 17.0  | 5117 | 0.7725          | 0.2733 | 0.0912 | 0.2367 | 0.2368    | 19.47   |
| 0.5349        | 18.0  | 5418 | 0.7761          | 0.2699 | 0.087  | 0.2339 | 0.2332    | 19.5    |
| 0.5182        | 19.0  | 5719 | 0.7786          | 0.2694 | 0.0874 | 0.2333 | 0.2336    | 19.5    |
| 0.5141        | 20.0  | 6020 | 0.7769          | 0.2685 | 0.0913 | 0.2316 | 0.2318    | 19.51   |


### Framework versions

- Transformers 4.35.2
- Pytorch 2.1.0+cu121
- Datasets 2.16.1
- Tokenizers 0.15.0