vLLM help pls :(
1
#6 opened about 5 hours ago
by
fsaudm
How much cuda memory is needed to run this model?
2
#5 opened 7 days ago
by
JohnnyBoyzzz
Any chance of an int4 or quantised version?
2
#3 opened 8 days ago
by
smcleod