16 6 5

Robert Shaw

robertgshaw2

rsnm2

AI & ML interests

None yet

Recent Activity

new activity 17 days ago

nm-testing/pixtral-12b-w4a16-actorder-group:What is an actorder group and what are the advantages of running this in vLLM?

new activity about 1 month ago

neuralmagic/Sparse-Llama-3.1-8B-2of4:Can I apply a LoRA?

new activity about 1 month ago

nm-testing/Llama-3.3-70B-Instruct-FP8-dynamic:Nice model, any info on scripts used to quantize?

View all activity

Organizations

robertgshaw2's activity

New activity in nm-testing/pixtral-12b-w4a16-actorder-group 17 days ago

What is an actorder group and what are the advantages of running this in vLLM?

#1 opened 17 days ago by

nickandbro

New activity in neuralmagic/Sparse-Llama-3.1-8B-2of4 about 1 month ago

Can I apply a LoRA?

#1 opened about 1 month ago by

RonanMcGovern

New activity in nm-testing/Llama-3.3-70B-Instruct-FP8-dynamic about 1 month ago

Nice model, any info on scripts used to quantize?

#1 opened about 1 month ago by

RonanMcGovern

updated a model about 1 month ago

nm-testing/llama-3-fp8-2of4-dynamic-uncompressed

Updated Dec 8, 2024 • 2

upvoted a paper 3 months ago

"Give Me BF16 or Give Me Death"? Accuracy-Performance Trade-Offs in LLM Quantization

Paper • 2411.02355 • Published Nov 4, 2024 • 47

New activity in neuralmagic/Meta-Llama-3-8B-Instruct-FP8 3 months ago

How to download the model with transformer library

#6 opened 3 months ago by

Rick10

New activity in mistralai/Pixtral-12B-2409 3 months ago

Update README.md

#25 opened 3 months ago by

robertgshaw2

updated a model 3 months ago

robertgshaw2/llama-3-act-order

Updated Oct 9, 2024 • 1

upvoted a collection 3 months ago

Llama-3.1 Quantization

Collection

Neural Magic quantized Llama-3.1 models • 22 items • Updated Nov 22, 2024 • 43

New activity in neuralmagic/Meta-Llama-3.1-70B-Instruct-FP8 4 months ago

Issue running on vLLM using FP8

#3 opened 4 months ago by

ffleandro

updated a model 4 months ago

nm-testing/pixtral-fp8-test

Updated Oct 2, 2024

New activity in neuralmagic/Meta-Llama-3.1-70B-Instruct-quantized.w8a8 5 months ago

vllm says the requested model does not exist

#1 opened 5 months ago by

shivams101

New activity in neuralmagic/Meta-Llama-3.1-405B-Instruct-quantized.w4a16 5 months ago

Storage format differs from other w4a16 models

#2 opened 5 months ago by

timdettmers

New activity in neuralmagic/Meta-Llama-3.1-8B-Instruct-quantized.w8a16 5 months ago

Model weights are not loaded

#3 opened 5 months ago by

MarvelousMouse

updated 2 models 6 months ago

nm-testing/Meta-Llama-3-70B-Instruct-FBGEMM-nonuniform

Text Generation • Updated Jul 20, 2024 • 92 • 1

nm-testing/Meta-Llama-3-8B-Instruct-FBGEMM-nonuniform

Text Generation • Updated Jul 20, 2024 • 35

New activity in neuralmagic/Mistral-Nemo-Instruct-2407-FP8 6 months ago

Can not be inferenced with vllm openai server

#1 opened 6 months ago by

jjqsdq

updated 3 models 6 months ago