Create README.md
Browse files
README.md
ADDED
|
@@ -0,0 +1,30 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
language:
|
| 3 |
+
- en
|
| 4 |
+
---
|
| 5 |
+
## Information
|
| 6 |
+
|
| 7 |
+
This is a Exl2 quantized version of [MN-12B-Starcannon-v1](https://huggingface.co/aetherwiing/MN-12B-Starcannon-v1)
|
| 8 |
+
|
| 9 |
+
Please refer to the original creator for more information.
|
| 10 |
+
|
| 11 |
+
Calibration dataset: Exl2 default
|
| 12 |
+
|
| 13 |
+
## Branches:
|
| 14 |
+
|
| 15 |
+
- main: Measurement files
|
| 16 |
+
- 4bpw: 4 bits per weight
|
| 17 |
+
- 5bpw: 5 bits per weight
|
| 18 |
+
- 6bpw: 6 bits per weight
|
| 19 |
+
|
| 20 |
+
## Notes
|
| 21 |
+
|
| 22 |
+
- 6bpw is recommended for the best quality to vram usage ratio (assuming you have enough vram).
|
| 23 |
+
- Quants greater than 6bpw will not be created because there is no improvement in using them. If you really want them, ask someone else or make them yourself.
|
| 24 |
+
|
| 25 |
+
## Donate?
|
| 26 |
+
|
| 27 |
+
All my infrastructure and cloud expenses are paid out of pocket. If you'd like to donate, you can do so here: https://ko-fi.com/kingbri
|
| 28 |
+
|
| 29 |
+
You should not feel obligated to donate, but if you do, I'd appreciate it.
|
| 30 |
+
---
|