flan-t5-base-insight2

This model is a fine-tuned version of google/flan-t5-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.9687
• Rouge1: 24.6641
• Rouge2: 19.0841
• Rougel: 23.6027
• Rougelsum: 24.2082
• Gen Len: 19.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 1
• eval_batch_size: 1
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
1.5604	1.0	172	1.1982	23.0727	15.1292	21.7613	22.3596	19.0
1.2717	2.0	344	1.0511	24.16	18.1468	22.8935	23.6175	19.0
1.1373	3.0	516	1.0002	24.5212	18.789	23.4331	24.0315	19.0
1.0895	4.0	688	0.9699	24.7531	19.3582	23.694	24.3304	19.0
1.0176	5.0	860	0.9687	24.6641	19.0841	23.6027	24.2082	19.0

Framework versions

• Transformers 4.36.2
• Pytorch 2.1.0+cu121
• Datasets 2.16.0
• Tokenizers 0.15.0