dyang415 commited on
Commit
12ca97b
·
verified ·
1 Parent(s): a4396dc

End of training

Browse files
Files changed (2) hide show
  1. README.md +16 -1
  2. adapter_model.bin +3 -0
README.md CHANGED
@@ -2,6 +2,7 @@
2
  license: apache-2.0
3
  library_name: peft
4
  tags:
 
5
  - generated_from_trainer
6
  base_model: mistralai/Mixtral-8x7B-Instruct-v0.1
7
  model-index:
@@ -104,7 +105,9 @@ fsdp_config:
104
 
105
  # mixtral-fc-w-resp-new-format-4e-no-negative-new
106
 
107
- This model is a fine-tuned version of [mistralai/Mixtral-8x7B-Instruct-v0.1](https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1) on an unknown dataset.
 
 
108
 
109
  ## Model description
110
 
@@ -150,6 +153,18 @@ The following hyperparameters were used during training:
150
  - lr_scheduler_warmup_steps: 10
151
  - num_epochs: 4
152
 
 
 
 
 
 
 
 
 
 
 
 
 
153
  ### Framework versions
154
 
155
  - PEFT 0.7.0
 
2
  license: apache-2.0
3
  library_name: peft
4
  tags:
5
+ - axolotl
6
  - generated_from_trainer
7
  base_model: mistralai/Mixtral-8x7B-Instruct-v0.1
8
  model-index:
 
105
 
106
  # mixtral-fc-w-resp-new-format-4e-no-negative-new
107
 
108
+ This model is a fine-tuned version of [mistralai/Mixtral-8x7B-Instruct-v0.1](https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1) on the None dataset.
109
+ It achieves the following results on the evaluation set:
110
+ - Loss: 0.0848
111
 
112
  ## Model description
113
 
 
153
  - lr_scheduler_warmup_steps: 10
154
  - num_epochs: 4
155
 
156
+ ### Training results
157
+
158
+ | Training Loss | Epoch | Step | Validation Loss |
159
+ |:-------------:|:-----:|:----:|:---------------:|
160
+ | 2.1636 | 0.0 | 1 | 2.0806 |
161
+ | 0.0914 | 0.8 | 328 | 0.0907 |
162
+ | 0.0836 | 1.58 | 656 | 0.0874 |
163
+ | 0.073 | 2.36 | 984 | 0.0851 |
164
+ | 0.0663 | 3.15 | 1312 | 0.0856 |
165
+ | 0.071 | 3.95 | 1640 | 0.0848 |
166
+
167
+
168
  ### Framework versions
169
 
170
  - PEFT 0.7.0
adapter_model.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4b7f32cee0aee285f40a9601f75ddc1175e57def005b74d54d8aedda61706b67
3
+ size 109144269