|
--- |
|
license: apache-2.0 |
|
--- |
|
|
|
# Model summary |
|
|
|
This is CodeLlama model fine-tuned on Kotlin Exercices dataset. |
|
|
|
# Training setup |
|
|
|
The model was trained on one A100 GPU with following hyperparameters: |
|
|
|
| **Hyperparameter** | **Value** | |
|
|:---------------------------:|:----------------------------------------:| |
|
| `warmup` | 10% | |
|
| `max_lr` | 1e-4 | |
|
| `scheduler` | linear | |
|
| `total_batch_size` | 256 (~130K tokens per step) | |
|
|
|
|
|
# Fine-tuning data |
|
|
|
For this model we used 15K exmaples of Kotlin Exercices dataset. For more information about the dataset follow th link. |
|
|
|
# Evaluation |
|
|
|
To evaluate we used Kotlin Humaneval (more infromation here) |
|
|
|
Fine-tuned model: |
|
|
|
**Kotlin Humaneval: 42.24** |
|
**Kotlin Compleation: 0.344** |
|
|
|
Base model: |
|
|
|
**Kotlin Humaneval: 26.89** |
|
**Kotlin Compleation: 0.388** |
