Hugging Face's logo
Robert Shaw's picture
16 6 5

Robert Shaw

robertgshaw2
joennlae's profile picture brwoodside's profile picture shubhrapandit's profile picture
·

AI & ML interests

None yet

Recent Activity

new activity 2 days ago
nm-testing/pixtral-12b-w4a16-actorder-group:What is an actorder group and what are the advantages of running this in vLLM?
new activity 24 days ago
neuralmagic/Sparse-Llama-3.1-8B-2of4:Can I apply a LoRA?
new activity 24 days ago
nm-testing/Llama-3.3-70B-Instruct-FP8-dynamic:Nice model, any info on scripts used to quantize?
View all activity

Organizations

HeavyAI's profile picture Neural Magic's profile picture Revel Labs's profile picture NM Testing's profile picture

robertgshaw2's activity

New activity in nm-testing/pixtral-12b-w4a16-actorder-group 2 days ago

What is an actorder group and what are the advantages of running this in vLLM?

4
#1 opened 2 days ago by
nickandbro
New activity in neuralmagic/Sparse-Llama-3.1-8B-2of4 24 days ago

Can I apply a LoRA?

2
#1 opened 24 days ago by
RonanMcGovern
New activity in nm-testing/Llama-3.3-70B-Instruct-FP8-dynamic 24 days ago

Nice model, any info on scripts used to quantize?

1
#1 opened 24 days ago by
RonanMcGovern
New activity in neuralmagic/Meta-Llama-3-8B-Instruct-FP8 2 months ago

How to download the model with transformer library

5
#6 opened 3 months ago by
Rick10
New activity in mistralai/Pixtral-12B-2409 3 months ago

Update README.md

3
#25 opened 3 months ago by
robertgshaw2
New activity in neuralmagic/Meta-Llama-3.1-70B-Instruct-FP8 3 months ago

Issue running on vLLM using FP8

2
#3 opened 3 months ago by
ffleandro
New activity in neuralmagic/Meta-Llama-3.1-70B-Instruct-quantized.w8a8 4 months ago

vllm says the requested model does not exist

2
#1 opened 4 months ago by
shivams101
New activity in neuralmagic/Meta-Llama-3.1-405B-Instruct-quantized.w4a16 4 months ago

Storage format differs from other w4a16 models

2
#2 opened 4 months ago by
timdettmers
New activity in neuralmagic/Meta-Llama-3.1-8B-Instruct-quantized.w8a16 4 months ago

Model weights are not loaded

4
#3 opened 4 months ago by
MarvelousMouse
New activity in neuralmagic/Mistral-Nemo-Instruct-2407-FP8 6 months ago

Can not be inferenced with vllm openai server

1
#1 opened 6 months ago by
jjqsdq
New activity in neuralmagic/Meta-Llama-3-70B-Instruct-quantized.w4a16 6 months ago

Code example request with vllm

2
#1 opened 6 months ago by
ShiningJazz
New activity in neuralmagic/Mistral-7B-Instruct-v0.3-GPTQ-4bit 6 months ago

4bit quantisation does not reduce vram usage.

1
#2 opened 6 months ago by
fu-man
New activity in neuralmagic/Meta-Llama-3-70B-Instruct-FP8 6 months ago

How to run Meta-Llama-3-70B-Instruct-FP8 using several devices?

5
#3 opened 6 months ago by
Fertel
New activity in open-llm-leaderboard/open_llm_leaderboard 6 months ago

Reproduction

2
#792 opened 6 months ago by
robertgshaw2
New activity in neuralmagic/Meta-Llama-3-8B-Instruct-FP8 8 months ago

Fails to run with nm-vllm

1
#1 opened 8 months ago by
clintonruairi
New activity in neuralmagic/Llama-2-7b-ultrachat200k-pruned_50 9 months ago

Update chart template
#2 opened 9 months ago by
robertgshaw2

Update chart template
#2 opened 9 months ago by
robertgshaw2