Commit
·
d67a66e
1
Parent(s):
1c0ff80
Create README.md
Browse files
README.md
ADDED
|
@@ -0,0 +1,17 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
inference: false
|
| 3 |
+
language:
|
| 4 |
+
- en
|
| 5 |
+
pipeline_tag: text-generation
|
| 6 |
+
tags:
|
| 7 |
+
- llama
|
| 8 |
+
- llama-2
|
| 9 |
+
---
|
| 10 |
+
# lzlv-limarpv3-l2-70b-exl2
|
| 11 |
+
|
| 12 |
+
Exllama v2 quant of [Doctor-Shotgun/lzlv-limarpv3-l2-70b](https://huggingface.co/Doctor-Shotgun/lzlv-limarpv3-l2-70b)
|
| 13 |
+
|
| 14 |
+
Branches:
|
| 15 |
+
- main: measurement.json calculated at 2048 token calibration rows on PIPPA
|
| 16 |
+
- 5.0bpw-h6: 5 decoder bits per weight, 6 head bits
|
| 17 |
+
- ideal for 2x 24gb GPUs at 8192 context, or 1x 48gb GPU at 8192 context with CFG cache
|