File size: 2,076 Bytes
9c2d1ec 0937a29 936b4d8 f717efc 936b4d8 f717efc 936b4d8 f717efc 936b4d8 f717efc 936b4d8 f717efc 936b4d8 f717efc 936b4d8 f717efc 936b4d8 f717efc 936b4d8 f717efc 1876617 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 |
---
license: apache-2.0
datasets:
- Salesforce/xlam-function-calling-60k
language:
- en
base_model:
- meta-llama/Llama-3.2-1B
pipeline_tag: text-generation
---
# llama3.2-1B-Function-calling
**⚠️ Important: This model is still under development and has not been fully fine-tuned. It is not yet suitable for use in production and should be treated as a work-in-progress. The results and performance metrics shared here are preliminary and subject to change.**
## Model description
This model was trained from scratch on an unknown dataset and is intended for function-calling tasks. As it is still in early stages, further development is required to optimize its performance.
## Intended uses & limitations
Currently, this model is not fully trained or optimized for any specific task. It is intended to handle function-calling tasks but should not be used in production until more comprehensive fine-tuning and evaluation are completed.
## Training and evaluation data
More information is needed regarding the dataset used for training. The model has not yet been fully evaluated, and additional testing is required to confirm its capabilities.
## Training procedure
### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 1
- eval_batch_size: 1
- seed: 42
- gradient_accumulation_steps: 32
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 3
- mixed_precision_training: Native AMP
### Training results
| Training Loss | Epoch | Step | Validation Loss |
|:-------------:|:------:|:----:|:---------------:|
| 0.3083 | 0.9997 | 1687 | 0.3622 |
| 0.202 | 2.0 | 3375 | 0.2844 |
| 0.1655 | 2.9997 | 5061 | 0.1491 |
These results are preliminary, and further training will be necessary to refine the model's performance.
## Framework versions
- Transformers 4.45.2
- Pytorch 2.4.1+cu121
- Datasets 3.0.1
- Tokenizers 0.20.0 |