Update README.md
Browse files
README.md
CHANGED
@@ -113,70 +113,8 @@ model-index:
|
|
113 |
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=ABX-AI/Quantum-Citrus-9B
|
114 |
name: Open LLM Leaderboard
|
115 |
---
|
116 |
-
|
117 |
|
118 |
-
|
119 |
-
|
120 |
-
|
121 |
-
|
122 |
-
**Importance Matrix**, at least based on my testing, has shown to improve the output and performance of "IQ"-type quantizations, where the compression becomes quite heavy.
|
123 |
-
The **Imatrix** performs a calibration, using a provided dataset. Testing has shown that semi-randomized data can help perserve more important segments as the compression is applied.
|
124 |
-
|
125 |
-
Related discussions in Github:
|
126 |
-
[[1]](https://github.com/ggerganov/llama.cpp/discussions/5006) [[2]](https://github.com/ggerganov/llama.cpp/discussions/5263#discussioncomment-8395384)
|
127 |
-
|
128 |
-
The imatrix.txt file that I used contains general, semi-random data, with some custom kink.
|
129 |
-
|
130 |
-
|
131 |
-
# Quantum-Citrus-9B
|
132 |
-
|
133 |
-
This merge is another attempt at making and intelligent, refined and unaligned model.
|
134 |
-
|
135 |
-
Based on my tests so far, it has accomplished the goals, and I am continuing to experiment with my interactions with it.
|
136 |
-
|
137 |
-
It includes previous merges of Starling, Cerebrum, LemonadeRP, InfinityRP, and deep down has a base of layla v0.1, as I am not that happy with the result form using v0.2.
|
138 |
-
|
139 |
-
The model is intended for fictional storytelling and roleplaying and may not be intended for all audences.
|
140 |
-
|
141 |
-
## Merge Details
|
142 |
-
|
143 |
-
This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
|
144 |
-
|
145 |
-
### Merge Method
|
146 |
-
|
147 |
-
This model was merged using the passthrough merge method.
|
148 |
-
|
149 |
-
### Models Merged
|
150 |
-
|
151 |
-
The following models were included in the merge:
|
152 |
-
* [ABX-AI/Starfinite-Laymospice-v2-7B](https://huggingface.co/ABX-AI/Starfinite-Laymospice-v2-7B)
|
153 |
-
* [ABX-AI/Cerebral-Infinity-7B](https://huggingface.co/ABX-AI/Cerebral-Infinity-7B)
|
154 |
-
|
155 |
-
### Configuration
|
156 |
-
|
157 |
-
The following YAML configuration was used to produce this model:
|
158 |
-
|
159 |
-
```yaml
|
160 |
-
slices:
|
161 |
-
- sources:
|
162 |
-
- model: ABX-AI/Cerebral-Infinity-7B
|
163 |
-
layer_range: [0, 20]
|
164 |
-
- sources:
|
165 |
-
- model: ABX-AI/Starfinite-Laymospice-v2-7B
|
166 |
-
layer_range: [12, 32]
|
167 |
-
merge_method: passthrough
|
168 |
-
dtype: float16
|
169 |
-
```
|
170 |
-
|
171 |
-
# [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
|
172 |
-
Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_ABX-AI__Quantum-Citrus-9B)
|
173 |
-
|
174 |
-
| Metric |Value|
|
175 |
-
|---------------------------------|----:|
|
176 |
-
|Avg. |66.74|
|
177 |
-
|AI2 Reasoning Challenge (25-Shot)|65.19|
|
178 |
-
|HellaSwag (10-Shot) |84.75|
|
179 |
-
|MMLU (5-Shot) |64.58|
|
180 |
-
|TruthfulQA (0-shot) |55.96|
|
181 |
-
|Winogrande (5-shot) |79.40|
|
182 |
-
|GSM8k (5-shot) |50.57|
|
|
|
113 |
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=ABX-AI/Quantum-Citrus-9B
|
114 |
name: Open LLM Leaderboard
|
115 |
---
|
116 |
+
## Note:
|
117 |
|
118 |
+
This repo hosts only a Q5_K_S iMatrix of [Quantum Citrus 9B](https://huggingface.co/ABX-AI/Quantum-Citrus-9B).
|
119 |
+
GGUF quant is from [ABX-AI/Quantum-Citrus-9B-GGUF-IQ-Imatrix](https://huggingface.co/ABX-AI/Quantum-Citrus-9B-GGUF-IQ-Imatrix).
|
120 |
+
The additional files in this GGUF repo is for personal usage using Text Gen Webui with llamacpp_hf.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|