--- license: other base_model: microsoft/phi-1_5 tags: - generated_from_trainer model-index: - name: phi-sft-outC results: [] --- [Built with Axolotl](https://github.com/OpenAccess-AI-Collective/axolotl) # phi-sft-outC This model is a fine-tuned version of [microsoft/phi-1_5](https://huggingface.co/microsoft/phi-1_5) on the None dataset. It achieves the following results on the evaluation set: - Loss: 0.7401 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 3e-05 - train_batch_size: 4 - eval_batch_size: 4 - seed: 42 - gradient_accumulation_steps: 4 - total_train_batch_size: 16 - optimizer: Adam with betas=(0.9,0.95) and epsilon=1e-05 - lr_scheduler_type: cosine - lr_scheduler_warmup_steps: 100 - num_epochs: 4 ### Training results | Training Loss | Epoch | Step | Validation Loss | |:-------------:|:-----:|:----:|:---------------:| | 0.8712 | 0.01 | 1 | 0.9999 | | 1.0178 | 0.21 | 30 | 0.9624 | | 1.0656 | 0.41 | 60 | 0.8691 | | 0.8251 | 0.62 | 90 | 0.8188 | | 0.8962 | 0.82 | 120 | 0.7913 | | 0.9011 | 1.03 | 150 | 0.7756 | | 0.8312 | 1.23 | 180 | 0.7655 | | 0.7654 | 1.44 | 210 | 0.7585 | | 1.0281 | 1.64 | 240 | 0.7522 | | 0.8819 | 1.85 | 270 | 0.7480 | | 0.8851 | 2.05 | 300 | 0.7453 | | 0.8821 | 2.26 | 330 | 0.7434 | | 0.7589 | 2.47 | 360 | 0.7423 | | 0.7541 | 2.67 | 390 | 0.7411 | | 0.9416 | 2.88 | 420 | 0.7406 | | 0.8544 | 3.08 | 450 | 0.7403 | | 0.8682 | 3.29 | 480 | 0.7402 | | 0.8589 | 3.49 | 510 | 0.7401 | | 0.7781 | 3.7 | 540 | 0.7402 | | 0.8715 | 3.9 | 570 | 0.7401 | ### Framework versions - Transformers 4.34.0.dev0 - Pytorch 2.0.0+cu118 - Datasets 2.14.5 - Tokenizers 0.14.0