---
library_name: peft
license: other
base_model: unsloth/Llama-3.2-3B-Instruct
tags:
- llama-factory
- lora
- unsloth
- generated_from_trainer
model-index:
- name: llm3br256
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# llm3br256

This model is a fine-tuned version of [meta-llama/Llama-3.2-3B-Instruct](https://huggingface.co/meta-llama/Llama-3.2-3B-Instruct) on the reliance dataset.
It achieves the following results on the evaluation set:
- Loss: 0.0014

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 4
- eval_batch_size: 4
- seed: 42
- gradient_accumulation_steps: 8
- total_train_batch_size: 32
- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: cosine
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 25.0

### Training results

| Training Loss | Epoch   | Step | Validation Loss |
|:-------------:|:-------:|:----:|:---------------:|
| 0.0197        | 0.2024  | 25   | 0.0177          |
| 0.0118        | 0.4049  | 50   | 0.0114          |
| 0.0103        | 0.6073  | 75   | 0.0080          |
| 0.0091        | 0.8097  | 100  | 0.0072          |
| 0.003         | 1.0121  | 125  | 0.0060          |
| 0.0036        | 1.2146  | 150  | 0.0053          |
| 0.0052        | 1.4170  | 175  | 0.0049          |
| 0.0029        | 1.6194  | 200  | 0.0042          |
| 0.0017        | 1.8219  | 225  | 0.0039          |
| 0.0022        | 2.0243  | 250  | 0.0035          |
| 0.0027        | 2.2267  | 275  | 0.0032          |
| 0.0017        | 2.4291  | 300  | 0.0030          |
| 0.0012        | 2.6316  | 325  | 0.0028          |
| 0.002         | 2.8340  | 350  | 0.0025          |
| 0.0008        | 3.0364  | 375  | 0.0026          |
| 0.0012        | 3.2389  | 400  | 0.0025          |
| 0.0013        | 3.4413  | 425  | 0.0021          |
| 0.0008        | 3.6437  | 450  | 0.0019          |
| 0.0014        | 3.8462  | 475  | 0.0020          |
| 0.0009        | 4.0486  | 500  | 0.0020          |
| 0.0006        | 4.2510  | 525  | 0.0019          |
| 0.0005        | 4.4534  | 550  | 0.0018          |
| 0.0007        | 4.6559  | 575  | 0.0017          |
| 0.0004        | 4.8583  | 600  | 0.0016          |
| 0.0004        | 5.0607  | 625  | 0.0016          |
| 0.0004        | 5.2632  | 650  | 0.0015          |
| 0.0004        | 5.4656  | 675  | 0.0016          |
| 0.0005        | 5.6680  | 700  | 0.0015          |
| 0.0003        | 5.8704  | 725  | 0.0015          |
| 0.0006        | 6.0729  | 750  | 0.0016          |
| 0.0003        | 6.2753  | 775  | 0.0015          |
| 0.0003        | 6.4777  | 800  | 0.0015          |
| 0.0003        | 6.6802  | 825  | 0.0015          |
| 0.0004        | 6.8826  | 850  | 0.0014          |
| 0.0003        | 7.0850  | 875  | 0.0015          |
| 0.0004        | 7.2874  | 900  | 0.0014          |
| 0.0004        | 7.4899  | 925  | 0.0015          |
| 0.0004        | 7.6923  | 950  | 0.0014          |
| 0.0005        | 7.8947  | 975  | 0.0014          |
| 0.0003        | 8.0972  | 1000 | 0.0014          |
| 0.0003        | 8.2996  | 1025 | 0.0014          |
| 0.0003        | 8.5020  | 1050 | 0.0016          |
| 0.0003        | 8.7045  | 1075 | 0.0014          |
| 0.0003        | 8.9069  | 1100 | 0.0015          |
| 0.0003        | 9.1093  | 1125 | 0.0014          |
| 0.0004        | 9.3117  | 1150 | 0.0016          |
| 0.0002        | 9.5142  | 1175 | 0.0014          |
| 0.0005        | 9.7166  | 1200 | 0.0014          |
| 0.0002        | 9.9190  | 1225 | 0.0014          |
| 0.0002        | 10.1215 | 1250 | 0.0014          |
| 0.0002        | 10.3239 | 1275 | 0.0014          |
| 0.0002        | 10.5263 | 1300 | 0.0014          |
| 0.0002        | 10.7287 | 1325 | 0.0015          |


### Framework versions

- PEFT 0.12.0
- Transformers 4.46.1
- Pytorch 2.4.0+cu121
- Datasets 3.1.0
- Tokenizers 0.20.3