Hugging Face's logo

ZeroWw
/
microsoft_WizardLM-2-7B-GGUF

GGUF
English
Inference Endpoints
conversational
Model card Files Community
ZeroWw commited on
Commit
aeddb2b
·
verified ·
1 Parent(s): c693ce2

Upload folder using huggingface_hub

Browse files
Files changed (7) hide show
  1. .gitattributes +1 -0
  2. README.md +9 -6
  3. microsoft_WizardLM-2-7B.f16.gguf +2 -2
  4. microsoft_WizardLM-2-7B.q5_k.gguf +2 -2
  5. microsoft_WizardLM-2-7B.q6_k.gguf +2 -2
  6. microsoft_WizardLM-2-7B.q8_0.gguf +2 -2
  7. microsoft_WizardLM-2-7B.q8_p.gguf +3 -0
.gitattributes CHANGED
@@ -37,3 +37,4 @@ microsoft_WizardLM-2-7B.f16.gguf filter=lfs diff=lfs merge=lfs -text
37
  microsoft_WizardLM-2-7B.q5_k.gguf filter=lfs diff=lfs merge=lfs -text
38
  microsoft_WizardLM-2-7B.q6_k.gguf filter=lfs diff=lfs merge=lfs -text
39
  microsoft_WizardLM-2-7B.q8_0.gguf filter=lfs diff=lfs merge=lfs -text
 
 
37
  microsoft_WizardLM-2-7B.q5_k.gguf filter=lfs diff=lfs merge=lfs -text
38
  microsoft_WizardLM-2-7B.q6_k.gguf filter=lfs diff=lfs merge=lfs -text
39
  microsoft_WizardLM-2-7B.q8_0.gguf filter=lfs diff=lfs merge=lfs -text
40
+ microsoft_WizardLM-2-7B.q8_p.gguf filter=lfs diff=lfs merge=lfs -text
README.md CHANGED
@@ -1,13 +1,16 @@
 
1
  ---
2
  license: mit
3
  language:
4
  - en
5
  ---
6
 
7
- My own (ZeroWw) quantizations.
8
- output and embed tensors quantized to f16.
9
- all other tensors quantized to q5_k or q6_k.
 
 
 
 
10
 
11
- Result:
12
- both f16.q6 and f16.q5 are smaller than q8_0 standard quantization
13
- and they perform as well as the pure f16.
 
1
+
2
  ---
3
  license: mit
4
  language:
5
  - en
6
  ---
7
 
8
+ My own (ZeroWw) quantizations.
9
+ output and embed tensors quantized to f16.
10
+ all other tensors quantized to q5_k or q6_k.
11
+
12
+ Result:
13
+ both f16.q6 and f16.q5 are smaller than q8_0 standard quantization
14
+ and they perform as well as the pure f16.
15
 
16
+ Updated on: Sun Jul 21, 08:30:21
 
 
microsoft_WizardLM-2-7B.f16.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:c81a8912fa7d7c616ccd8b66bd178937a0eb232cbc2c18b14cfaab90203aa515
3
- size 14484732352
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5d916de924790382c29e237304a7a0b93f3b557f7633d97829f258c8984251ca
3
+ size 14484732416
microsoft_WizardLM-2-7B.q5_k.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:ffa2f8360a706820a7aad935709e5eaf06c658420f7b5552e43d103b43cb3c79
3
- size 5458065856
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:36df2e361c490fee1a88ac167a2ef43b2f2122d63d2916539019a04aa2772344
3
+ size 5458065920
microsoft_WizardLM-2-7B.q6_k.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:de8125f83978a7704ba3adcf4b1815c944d68e0dd1ac8c78365e5184107e5ce4
3
- size 6251313600
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:618ef9ff9929f8f3464ad457cda6c7bf9e5735991514eff209cbfa03bf416df9
3
+ size 6251313664
microsoft_WizardLM-2-7B.q8_0.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:f99bbb16a311ad4a850718cdf35ea8fad79abd137d74adcd80c29d6c00279f60
3
- size 7941618112
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7019c0a6edde7f97216dab7ca8e5c22b2efddeea05fd20d0d625546c9f1102cc
3
+ size 7941618176
microsoft_WizardLM-2-7B.q8_p.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7b86a6fb31aba07ac377e47f50d7c19f066b33531e5b8bd94688ee3daf4b36cc
3
+ size 7695858176