Hugging Face's logo

etemiz
/
Llama-3.1-405B-Inst-GGUF

GGUF
Inference Endpoints
imatrix
conversational
Model card Files Community
1
etemiz commited on
Commit
8bd05ec
·
verified ·
1 Parent(s): 91024dd

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +5 -3
README.md CHANGED
@@ -2,13 +2,15 @@
2
  license: llama3.1
3
  ---
4
 
5
- Requants of BF16 of
6
  https://huggingface.co/nisten/meta-405b-instruct-cpu-optimized-gguf/
7
 
8
- Which is converted from
9
  https://huggingface.co/meta-llama/Meta-Llama-3.1-405B-Instruct
10
 
11
 
12
  llama.cpp version b3459
13
 
14
- imatrix file https://huggingface.co/nisten/meta-405b-instruct-cpu-optimized-gguf/blob/main/405imatrix.dat
 
 
 
2
  license: llama3.1
3
  ---
4
 
5
+ Quantization from BF16 here:
6
  https://huggingface.co/nisten/meta-405b-instruct-cpu-optimized-gguf/
7
 
8
+ which is converted from Llama 3.1 405B:
9
  https://huggingface.co/meta-llama/Meta-Llama-3.1-405B-Instruct
10
 
11
 
12
  llama.cpp version b3459
13
 
14
+ imatrix file https://huggingface.co/nisten/meta-405b-instruct-cpu-optimized-gguf/blob/main/405imatrix.dat
15
+
16
+ Lmk if you need bigger quants.