3-bit OmniQuant quantized version of FuseChat-Llama-3.1-8B-Instruct for inference with the Private LLM app.

Inference Examples

Inference API (serverless) does not yet support mlc-llm models for this pipeline type.

Model tree for numen-tech/FuseChat-Llama-3.1-8B-Instruct-w3a16g40sym

Base model

Quantized

(10)

this model