Edit Models filters

Inference status

Misc

compressed-tensors

Inference Endpoints

AutoTrain Compatible

text-generation-inference

8-bit precision

Misc with no match

4-bit precision

text-embeddings-inference

Carbon Emissions

Mixture of Experts

Models

546

Full-text search

Active filters: compressed-tensors

neuralmagic/Llama-3.1-Nemotron-70B-Instruct-HF-FP8-dynamic

Text Generation • Updated Oct 17, 2024 • 1.62k • 14

nm-testing/Qwen2.5-Coder-32B-Instruct-FP8-dynamic

Updated Nov 15, 2024 • 249 • 1

neuralmagic/Sparse-Llama-3.1-8B-gsm8k-2of4-FP8-dynamic

Text Generation • Updated 17 days ago • 109 • 1

nm-testing/Qwen2.5-32B-Instruct-FP8-dynamic

Updated Nov 15, 2024 • 250 • 1

ConfidentialMind/arcee-ai-SuperNova-Medius-CM-w4a16

Updated Nov 20, 2024 • 3.41k • 1

neuralmagic/Sparse-Llama-3.1-8B-ultrachat_200k-2of4-FP8-dynamic

Text Generation • Updated 17 days ago • 51 • 1

nm-testing/pixtral-12b-W8A8

Updated Nov 19, 2024 • 186 • 2

neuralmagic/Sparse-Llama-3.1-8B-ultrachat_200k-2of4-quantized.w4a16

Text Generation • Updated 17 days ago • 218 • 3

alpindale/Mistral-Large-Instruct-2411-FP8

Updated 8 days ago • 123 • 1

janhq/llama3-s-instruct-v0.4-vllm-fp8

Updated Nov 26, 2024 • 9 • 1

qeternity/AIDC-AI_Marco-o1-GPTQ-INT4

Updated Nov 27, 2024 • 14 • 2

reinforce20001/SakuraLLM.Sakura-14B-Qwen2.5-v1.0-W8A8-Int8

Updated Nov 28, 2024 • 52 • 1

pL-Community/SauerkrautLM-v2-14b-DPO-FP8

Text Generation • Updated Dec 4, 2024 • 66 • 1

nm-testing/Llama-3.3-70B-Instruct-FP8-dynamic

Updated about 1 month ago • 24.5k • 7

CalamitousFelicitousness/Llama-3.3-70B-Instruct-W8A8-INT8

Updated about 1 month ago • 367 • 3

reinforce20001/SakuraLLM.Sakura-7B-Qwen2.5-v1.0-W8A8-Int8

Updated 29 days ago • 67 • 1

Infermatic/72B-Qwen2.5-Kunou-v1-FP8-Dynamic

Text Generation • Updated 27 days ago • 1.17k • 1

ibm-fms/Bamba-9B-1.8T-fp8

Text Generation • Updated 18 days ago • 159 • 1

ibm-fms/Bamba-9B-2T-fp8

Text Generation • Updated 18 days ago • 144 • 2

ibm-fms/Bamba-9B-fp8

Text Generation • Updated 18 days ago • 116 • 2

cortecs/EuroLLM-9B-Instruct-FP8-Dynamic

Updated 24 days ago • 113 • 1

BigHuggyD/TheDrummer_Anubis-70B-v1-FP8-Dynamic

Updated 14 days ago • 90 • 1

BigHuggyD/gghfez_Writer-Large-2411-v2.1-FP8-Dynamic

Updated 13 days ago • 165 • 1

CalamitousFelicitousness/Evathene-v1.0-FP8-Dynamic

Text Generation • Updated Nov 19, 2024 • 23 • 2

nm-testing/tinyllama-one-shot-static-quant-test-compressed

Text Generation • Updated Oct 9, 2024 • 13

nm-testing/tinyllama-one-shot-dynamic-test

Text Generation • Updated Oct 9, 2024 • 25

nm-testing/tinyllama-one-shot-w4a16-group-packed

Text Generation • Updated Oct 10, 2024 • 32

nm-testing/tinyllama-one-shot-w4a16-channel-compressed

Text Generation • Updated Oct 9, 2024 • 24

nm-testing/tinyllama-one-shot-w4a16-channel-packed

Text Generation • Updated Oct 9, 2024 • 25

nm-testing/llama7b-one-shot-2_4-w4a16-packed

Text Generation • Updated Oct 9, 2024 • 33