End of training
Browse files- README.md +16 -1
- adapter_model.bin +3 -0
README.md
CHANGED
@@ -2,6 +2,7 @@
|
|
2 |
license: apache-2.0
|
3 |
library_name: peft
|
4 |
tags:
|
|
|
5 |
- generated_from_trainer
|
6 |
base_model: mistralai/Mixtral-8x7B-Instruct-v0.1
|
7 |
model-index:
|
@@ -104,7 +105,9 @@ fsdp_config:
|
|
104 |
|
105 |
# mixtral-fc-w-resp-new-format-4e-no-negative-new
|
106 |
|
107 |
-
This model is a fine-tuned version of [mistralai/Mixtral-8x7B-Instruct-v0.1](https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1) on
|
|
|
|
|
108 |
|
109 |
## Model description
|
110 |
|
@@ -150,6 +153,18 @@ The following hyperparameters were used during training:
|
|
150 |
- lr_scheduler_warmup_steps: 10
|
151 |
- num_epochs: 4
|
152 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
153 |
### Framework versions
|
154 |
|
155 |
- PEFT 0.7.0
|
|
|
2 |
license: apache-2.0
|
3 |
library_name: peft
|
4 |
tags:
|
5 |
+
- axolotl
|
6 |
- generated_from_trainer
|
7 |
base_model: mistralai/Mixtral-8x7B-Instruct-v0.1
|
8 |
model-index:
|
|
|
105 |
|
106 |
# mixtral-fc-w-resp-new-format-4e-no-negative-new
|
107 |
|
108 |
+
This model is a fine-tuned version of [mistralai/Mixtral-8x7B-Instruct-v0.1](https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1) on the None dataset.
|
109 |
+
It achieves the following results on the evaluation set:
|
110 |
+
- Loss: 0.0848
|
111 |
|
112 |
## Model description
|
113 |
|
|
|
153 |
- lr_scheduler_warmup_steps: 10
|
154 |
- num_epochs: 4
|
155 |
|
156 |
+
### Training results
|
157 |
+
|
158 |
+
| Training Loss | Epoch | Step | Validation Loss |
|
159 |
+
|:-------------:|:-----:|:----:|:---------------:|
|
160 |
+
| 2.1636 | 0.0 | 1 | 2.0806 |
|
161 |
+
| 0.0914 | 0.8 | 328 | 0.0907 |
|
162 |
+
| 0.0836 | 1.58 | 656 | 0.0874 |
|
163 |
+
| 0.073 | 2.36 | 984 | 0.0851 |
|
164 |
+
| 0.0663 | 3.15 | 1312 | 0.0856 |
|
165 |
+
| 0.071 | 3.95 | 1640 | 0.0848 |
|
166 |
+
|
167 |
+
|
168 |
### Framework versions
|
169 |
|
170 |
- PEFT 0.7.0
|
adapter_model.bin
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:4b7f32cee0aee285f40a9601f75ddc1175e57def005b74d54d8aedda61706b67
|
3 |
+
size 109144269
|