FatCat87 commited on
Commit
793e0bf
·
verified ·
1 Parent(s): 1225d47

End of training

Browse files
Files changed (2) hide show
  1. README.md +21 -23
  2. adapter_model.bin +2 -2
README.md CHANGED
@@ -4,9 +4,9 @@ library_name: peft
4
  tags:
5
  - axolotl
6
  - generated_from_trainer
7
- base_model: JackFram/llama-160m
8
  model-index:
9
- - name: 1b054327-a6b9-45e0-98ef-2d5fb503dfb8
10
  results: []
11
  ---
12
 
@@ -19,19 +19,19 @@ should probably proofread and complete it, then remove this comment. -->
19
  axolotl version: `0.4.1`
20
  ```yaml
21
  adapter: lora
22
- base_model: JackFram/llama-160m
23
  bf16: auto
24
  datasets:
25
  - data_files:
26
- - ab80a554070cdc53_train_data.json
27
  ds_type: json
28
  format: custom
29
- path: ab80a554070cdc53_train_data.json
30
  type:
31
  field: null
32
- field_input: rejected
33
- field_instruction: prompt
34
- field_output: chosen
35
  field_system: null
36
  format: null
37
  no_input_format: null
@@ -51,7 +51,7 @@ fsdp_config: null
51
  gradient_accumulation_steps: 4
52
  gradient_checkpointing: true
53
  group_by_length: false
54
- hub_model_id: FatCat87/1b054327-a6b9-45e0-98ef-2d5fb503dfb8
55
  learning_rate: 0.0002
56
  load_in_4bit: false
57
  load_in_8bit: true
@@ -73,8 +73,7 @@ sample_packing: true
73
  saves_per_epoch: 1
74
  seed: 701
75
  sequence_len: 4096
76
- special_tokens:
77
- pad_token: </s>
78
  strict: false
79
  tf32: false
80
  tokenizer_type: AutoTokenizer
@@ -83,9 +82,9 @@ val_set_size: 0.1
83
  wandb_entity: fatcat87-taopanda
84
  wandb_log_model: null
85
  wandb_mode: online
86
- wandb_name: 1b054327-a6b9-45e0-98ef-2d5fb503dfb8
87
  wandb_project: subnet56
88
- wandb_runid: 1b054327-a6b9-45e0-98ef-2d5fb503dfb8
89
  wandb_watch: null
90
  warmup_ratio: 0.05
91
  weight_decay: 0.0
@@ -95,12 +94,12 @@ xformers_attention: null
95
 
96
  </details><br>
97
 
98
- [<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="200" height="32"/>](https://wandb.ai/fatcat87-taopanda/subnet56/runs/xhyl66qj)
99
- # 1b054327-a6b9-45e0-98ef-2d5fb503dfb8
100
 
101
- This model is a fine-tuned version of [JackFram/llama-160m](https://huggingface.co/JackFram/llama-160m) on the None dataset.
102
  It achieves the following results on the evaluation set:
103
- - Loss: 2.7122
104
 
105
  ## Model description
106
 
@@ -130,18 +129,17 @@ The following hyperparameters were used during training:
130
  - total_eval_batch_size: 4
131
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
132
  - lr_scheduler_type: cosine
133
- - lr_scheduler_warmup_steps: 2
134
  - num_epochs: 1
135
 
136
  ### Training results
137
 
138
  | Training Loss | Epoch | Step | Validation Loss |
139
  |:-------------:|:------:|:----:|:---------------:|
140
- | 3.7539 | 0.0308 | 1 | 3.8105 |
141
- | 3.4309 | 0.2462 | 8 | 3.3958 |
142
- | 3.0223 | 0.4923 | 16 | 2.9836 |
143
- | 2.7418 | 0.7385 | 24 | 2.7555 |
144
- | 2.6571 | 0.9846 | 32 | 2.7122 |
145
 
146
 
147
  ### Framework versions
 
4
  tags:
5
  - axolotl
6
  - generated_from_trainer
7
+ base_model: unsloth/Qwen2.5-0.5B-Instruct
8
  model-index:
9
+ - name: f2c37cd7-d8ff-4753-9fa2-fd45b186c216
10
  results: []
11
  ---
12
 
 
19
  axolotl version: `0.4.1`
20
  ```yaml
21
  adapter: lora
22
+ base_model: unsloth/Qwen2.5-0.5B-Instruct
23
  bf16: auto
24
  datasets:
25
  - data_files:
26
+ - 8a7bfa6a070007c1_train_data.json
27
  ds_type: json
28
  format: custom
29
+ path: 8a7bfa6a070007c1_train_data.json
30
  type:
31
  field: null
32
+ field_input: null
33
+ field_instruction: question
34
+ field_output: answer
35
  field_system: null
36
  format: null
37
  no_input_format: null
 
51
  gradient_accumulation_steps: 4
52
  gradient_checkpointing: true
53
  group_by_length: false
54
+ hub_model_id: FatCat87/f2c37cd7-d8ff-4753-9fa2-fd45b186c216
55
  learning_rate: 0.0002
56
  load_in_4bit: false
57
  load_in_8bit: true
 
73
  saves_per_epoch: 1
74
  seed: 701
75
  sequence_len: 4096
76
+ special_tokens: null
 
77
  strict: false
78
  tf32: false
79
  tokenizer_type: AutoTokenizer
 
82
  wandb_entity: fatcat87-taopanda
83
  wandb_log_model: null
84
  wandb_mode: online
85
+ wandb_name: f2c37cd7-d8ff-4753-9fa2-fd45b186c216
86
  wandb_project: subnet56
87
+ wandb_runid: f2c37cd7-d8ff-4753-9fa2-fd45b186c216
88
  wandb_watch: null
89
  warmup_ratio: 0.05
90
  weight_decay: 0.0
 
94
 
95
  </details><br>
96
 
97
+ [<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="200" height="32"/>](https://wandb.ai/fatcat87-taopanda/subnet56/runs/lrmaou1e)
98
+ # f2c37cd7-d8ff-4753-9fa2-fd45b186c216
99
 
100
+ This model is a fine-tuned version of [unsloth/Qwen2.5-0.5B-Instruct](https://huggingface.co/unsloth/Qwen2.5-0.5B-Instruct) on the None dataset.
101
  It achieves the following results on the evaluation set:
102
+ - Loss: 1.2166
103
 
104
  ## Model description
105
 
 
129
  - total_eval_batch_size: 4
130
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
131
  - lr_scheduler_type: cosine
132
+ - lr_scheduler_warmup_steps: 4
133
  - num_epochs: 1
134
 
135
  ### Training results
136
 
137
  | Training Loss | Epoch | Step | Validation Loss |
138
  |:-------------:|:------:|:----:|:---------------:|
139
+ | 2.1114 | 0.0103 | 1 | 2.1366 |
140
+ | 1.4492 | 0.2577 | 25 | 1.4891 |
141
+ | 1.2531 | 0.5155 | 50 | 1.2947 |
142
+ | 1.2349 | 0.7732 | 75 | 1.2166 |
 
143
 
144
 
145
  ### Framework versions
adapter_model.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:0b3f9767b9eea35f1aeddfa05d91df06dde31b60453f2816965bdb3948966b9f
3
- size 27192298
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:17fafc8aac5daab6d28871a91efcc7e449fad0523da85cd7ab1b29ab5192d64b
3
+ size 70506570