Hugging Face's logo

Nexesenex
/
cloudyu_Mixtral_34Bx2_MoE_60B-iMat.GGUF

GGUF
Inference Endpoints
Model card Files Community
Nexesenex commited on
Commit
a8a5a55
·
verified ·
1 Parent(s): 9d936bd

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +13 -1
README.md CHANGED
@@ -2,4 +2,16 @@ Quants for the following model : https://huggingface.co/cloudyu/Mixtral_34Bx2_Mo
2
 
3
  I'm not satisfied with them, though. Their size is weird.
4
 
5
- For now, prefer the quants of The Bloke : https://huggingface.co/TheBloke/Mixtral_34Bx2_MoE_60B-GGUF
 
 
 
 
 
 
 
 
 
 
 
 
 
2
 
3
  I'm not satisfied with them, though. Their size is weird.
4
 
5
+ For now, prefer the quants of The Bloke : https://huggingface.co/TheBloke/Mixtral_34Bx2_MoE_60B-GGUF
6
+
7
+ Bench of a Q3_K_M from TheBloke :
8
+
9
+ - mixtral_34bx2_moe_60b.Q3_K_M.gguf,-,Hellaswag,84.5,84.25,400,2024-01-27 00:00:00,,70b,Llama_2,4096,,,GGUF,,,
10
+ - mixtral_34bx2_moe_60b.Q3_K_M.gguf,-,Hellaswag,,,1000,2024-01-27 00:00:00,,70b,Llama_2,4096,,,GGUF,,,
11
+ - mixtral_34bx2_moe_60b.Q3_K_M.gguf,-,Arc-Challenge,61.53846154,,299,2024-01-27 05:40:00,,01.3b,Llama_2,4096,,,GGUF,,,
12
+ - mixtral_34bx2_moe_60b.Q3_K_M.gguf,-,Arc-Easy,82.28070175,,570,2024-01-27 05:40:00,,01.3b,Llama_2,4096,,,GGUF,,,
13
+ - mixtral_34bx2_moe_60b.Q3_K_M.gguf,-,MMLU,40.89456869,,313,2024-01-27 05:40:00,,01.3b,Llama_2,4096,,,GGUF,,,
14
+ - mixtral_34bx2_moe_60b.Q3_K_M.gguf,-,Thruthful-QA,42.35006120,,817,2024-01-27 05:40:00,,01.3b,Llama_2,4096,,,GGUF,,,
15
+ - mixtral_34bx2_moe_60b.Q3_K_M.gguf,-,Winogrande,79.0845,,1267,2024-01-27 05:40:00,,01.3b,Llama_2,4096,,,GGUF,,,
16
+ - mixtral_34bx2_moe_60b.Q3_K_M.gguf,-,wikitext,5.3715,512,512,2024-01-27 00:00:00,,70b,Llama_2,4096,,,GGUF,,,
17
+ - mixtral_34bx2_moe_60b.Q3_K_M.gguf,-,wikitext,5.1792,4096,4096,2024-01-27 00:00:00,,70b,Llama_2,4096,,,GGUF,,,24