rombodawg commited on
Commit
1001345
·
verified ·
1 Parent(s): 83708db

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +17 -42
README.md CHANGED
@@ -1,42 +1,17 @@
1
- ---
2
- base_model: []
3
- library_name: transformers
4
- tags:
5
- - mergekit
6
- - merge
7
-
8
- ---
9
- # Rombo-LLM-V2.5-qwen-2.5-0.5b
10
-
11
- This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
12
-
13
- ## Merge Details
14
- ### Merge Method
15
-
16
- This model was merged using the [TIES](https://arxiv.org/abs/2306.01708) merge method using E:\Open_source_ai_chatbot\OOBA-13\text-generation-webui-main\models\unsloth_Qwen2.5-0.5B as a base.
17
-
18
- ### Models Merged
19
-
20
- The following models were included in the merge:
21
- * E:\Open_source_ai_chatbot\OOBA-13\text-generation-webui-main\models\unsloth_Qwen2.5-0.5B-Instruct
22
-
23
- ### Configuration
24
-
25
- The following YAML configuration was used to produce this model:
26
-
27
- ```yaml
28
- models:
29
- - model: E:\Open_source_ai_chatbot\OOBA-13\text-generation-webui-main\models\unsloth_Qwen2.5-0.5B-Instruct
30
- parameters:
31
- weight: 1
32
- density: 1
33
- merge_method: ties
34
- base_model: E:\Open_source_ai_chatbot\OOBA-13\text-generation-webui-main\models\unsloth_Qwen2.5-0.5B
35
- parameters:
36
- weight: 1
37
- density: 1
38
- normalize: true
39
- int8_mask: false
40
- dtype: bfloat16
41
-
42
- ```
 
1
+ ---
2
+ library_name: transformers
3
+ base_model:
4
+ - Qwen/Qwen2.5-0.5B-Instruct
5
+ license: apache-2.0
6
+ ---
7
+ # Rombos-LLM-V2.5-Qwen-0.5b
8
+
9
+ ![image/jpeg](https://cdn-uploads.huggingface.co/production/uploads/642cc1c253e76b4c2286c58e/oL_yvvRsWj2C4niGgkT2A.jpeg)
10
+
11
+ Rombos-LLM-V2.5-Qwen-0.5b is a continues finetuned version of Qwen2.5-0.5B. I noticed recently that the Qwen team did not learn from my methods of continuous finetuning, the great benefits, and no downsides of it. So I took it upon myself to merge the instruct model with the base model myself using the *Ties* merge method
12
+
13
+ This version of the model shows higher performance than the original instruct and base models.
14
+
15
+ Quants:
16
+
17
+ GGUF: https://huggingface.co/bartowski/Replete-LLM-V2.5-Qwen-0.5b-GGUF