Output_llama2_70-15-15

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.6250
• Balanced Accuracy: 0.6326
• Accuracy: 0.6282

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Balanced Accuracy	Accuracy
No log	1.0	46	0.7111	0.5764	0.5641
No log	2.0	92	0.7043	0.5656	0.5577
No log	3.0	138	0.6619	0.5142	0.5192
No log	4.0	184	0.7013	0.5595	0.5513
No log	5.0	230	0.6493	0.5620	0.5577
No log	6.0	276	0.6496	0.5671	0.5641
No log	7.0	322	0.6466	0.5798	0.5769
No log	8.0	368	0.6748	0.5527	0.5513
No log	9.0	414	0.6551	0.5692	0.5705
No log	10.0	460	0.6205	0.6063	0.5833
0.6541	11.0	506	0.6537	0.6020	0.6026
0.6541	12.0	552	0.6379	0.6167	0.6154
0.6541	13.0	598	0.6243	0.6107	0.6026
0.6541	14.0	644	0.6248	0.6074	0.6026
0.6541	15.0	690	0.6172	0.6370	0.6218
0.6541	16.0	736	0.6237	0.6202	0.6154
0.6541	17.0	782	0.6308	0.6230	0.6218
0.6541	18.0	828	0.6179	0.6319	0.6218
0.6541	19.0	874	0.6252	0.6326	0.6282
0.6541	20.0	920	0.6250	0.6326	0.6282

Framework versions

• PEFT 0.10.0
• Transformers 4.41.2
• Pytorch 2.3.1+cu121
• Datasets 3.2.0
• Tokenizers 0.19.1