---
license: llama3.1
tags:
- gguf
- llama3
pipeline_tag: text-generation
datasets:
- froggeric/imatrix
language:
- en
library_name: ggml
---

# Meta-Llama-3.1-405B-Instruct-GGUF

![image/png](https://cdn-uploads.huggingface.co/production/uploads/6604e5b21eb292d6df393365/o7DiWuILyzaPLh4Ne1JKr.png)

Low bit quantizations of Meta's Llama 3.1 405B Instruct model. Quantized from ollama q4_0 GGUF.

Quantized with llama.cpp [b3449](https://github.com/ggerganov/llama.cpp/releases/tag/b3449)

**Quants:**
- Q2_K
- (imatrix)
- Q3_K_M
- Q3_K_S
- Q3_K_L
- Q4_K_M
- Q4_0
- Q4_K_S

For higher quality quantizations (q4+), please refer to [nisten/meta-405b-instruct-cpu-optimized-gguf](https://huggingface.co/nisten/meta-405b-instruct-cpu-optimized-gguf).

## imatrix

Generated from Q2_K quant.

imatrix calibration data: `groups_merged.txt`