t5-small-finetuned-amazon-en

This model is a fine-tuned version of google-t5/t5-small on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 2.6372
• Rouge1: 23.1675
• Rouge2: 13.593
• Rougel: 22.2169
• Rougelsum: 22.3275

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5.6e-05
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 8

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum
2.8706	1.0	79	2.7303	22.6649	13.1507	21.6996	21.8226
2.8591	2.0	158	2.6905	22.819	13.2646	21.8821	21.9906
2.8214	3.0	237	2.6750	22.8648	13.3025	21.931	22.0772
2.7875	4.0	316	2.6564	22.8115	13.2944	21.874	22.0061
2.779	5.0	395	2.6451	23.0489	13.5043	22.0878	22.2228
2.763	6.0	474	2.6435	23.0105	13.4214	22.0635	22.1647
2.7413	7.0	553	2.6385	23.1595	13.6131	22.2155	22.3352
2.7443	8.0	632	2.6372	23.1675	13.593	22.2169	22.3275

Framework versions

• Transformers 4.48.0
• Pytorch 2.1.0+cu118
• Datasets 3.2.0
• Tokenizers 0.21.0