Hugging Face's logo

imdatta0
/
Qwen2-7B_pct_ortho

PEFT
Safetensors
unsloth
Generated from Trainer
Model card Files Community
Qwen2-7B_pct_ortho / README.md
imdatta0's picture
imdatta0
End of training
c0152da verified
preview code
|
raw
history blame
3.9 kB
metadata
base_model: unsloth/qwen2-7b-bnb-4bit
library_name: peft
license: apache-2.0
tags:
  - unsloth
  - generated_from_trainer
model-index:
  - name: Qwen2-7B_pct_ortho
    results: []

Qwen2-7B_pct_ortho

This model is a fine-tuned version of unsloth/qwen2-7b-bnb-4bit on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 2.1027

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0003
  • train_batch_size: 8
  • eval_batch_size: 8
  • seed: 42
  • gradient_accumulation_steps: 8
  • total_train_batch_size: 64
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: cosine
  • lr_scheduler_warmup_ratio: 0.02
  • num_epochs: 1

Training results

Training Loss Epoch Step Validation Loss
2.1596 0.0206 8 2.0832
2.095 0.0412 16 2.0465
2.1302 0.0618 24 2.0538
2.0886 0.0824 32 2.0696
2.1386 0.1031 40 2.0840
2.141 0.1237 48 2.0979
2.1451 0.1443 56 2.0972
2.1323 0.1649 64 2.1059
2.152 0.1855 72 2.1132
2.2215 0.2061 80 2.1120
2.187 0.2267 88 2.1149
2.1712 0.2473 96 2.1171
2.2009 0.2680 104 2.1281
2.1177 0.2886 112 2.1351
2.1735 0.3092 120 2.1326
2.1785 0.3298 128 2.1293
2.1826 0.3504 136 2.1398
2.1799 0.3710 144 2.1419
2.1817 0.3916 152 2.1564
2.2199 0.4122 160 2.1452
2.2533 0.4329 168 2.1420
2.1606 0.4535 176 2.1434
2.1773 0.4741 184 2.1394
2.2177 0.4947 192 2.1369
2.1614 0.5153 200 2.1360
2.2003 0.5359 208 2.1389
2.2389 0.5565 216 2.1424
2.1515 0.5771 224 2.1329
2.16 0.5977 232 2.1388
2.1584 0.6184 240 2.1229
2.1446 0.6390 248 2.1275
2.19 0.6596 256 2.1256
2.1612 0.6802 264 2.1182
2.2218 0.7008 272 2.1202
2.1112 0.7214 280 2.1163
2.2125 0.7420 288 2.1118
2.1432 0.7626 296 2.1115
2.1537 0.7833 304 2.1066
2.1391 0.8039 312 2.1069
2.1831 0.8245 320 2.1115
2.1835 0.8451 328 2.1099
2.1461 0.8657 336 2.1126
2.183 0.8863 344 2.1060
2.1507 0.9069 352 2.1017
2.216 0.9275 360 2.1033
2.1464 0.9481 368 2.1032
2.1925 0.9688 376 2.1028
2.1514 0.9894 384 2.1027

Framework versions

  • PEFT 0.12.0
  • Transformers 4.44.0
  • Pytorch 2.4.0+cu121
  • Datasets 2.20.0
  • Tokenizers 0.19.1