Hugging Face's logo
abubakar's picture
14 6 3

abubakar

stormchaser
kargaranamir's profile picture Mi6paulino's profile picture 21world's profile picture
·

AI & ML interests

None yet

Recent Activity

reacted to bartowski's post with 👀 22 days ago
Looks like Q4_0_N_M file types are going away Before you panic, there's a new "preferred" method which is online (I prefer the term on-the-fly) repacking, so if you download Q4_0 and your setup can benefit from repacking the weights into interleaved rows (what Q4_0_4_4 was doing), it will do that automatically and give you similar performance (minor losses I think due to using intrinsics instead of assembly, but intrinsics are more maintainable) You can see the reference PR here: https://github.com/ggerganov/llama.cpp/pull/10446 So if you update your llama.cpp past that point, you won't be able to run Q4_0_4_4 (unless they add backwards compatibility back), but Q4_0 should be the same speeds (though it may currently be bugged on some platforms) As such, I'll stop making those newer model formats soon, probably end of this week unless something changes, but you should be safe to download and Q4_0 quants and use those ! Also IQ4_NL supports repacking though not in as many shapes yet, but should get a respectable speed up on ARM chips, PR for that can be found here: https://github.com/ggerganov/llama.cpp/pull/10541 Remember, these are not meant for Apple silicon since those use the GPU and don't benefit from the repacking of weights
reacted to bartowski's post with 👍 22 days ago
Looks like Q4_0_N_M file types are going away Before you panic, there's a new "preferred" method which is online (I prefer the term on-the-fly) repacking, so if you download Q4_0 and your setup can benefit from repacking the weights into interleaved rows (what Q4_0_4_4 was doing), it will do that automatically and give you similar performance (minor losses I think due to using intrinsics instead of assembly, but intrinsics are more maintainable) You can see the reference PR here: https://github.com/ggerganov/llama.cpp/pull/10446 So if you update your llama.cpp past that point, you won't be able to run Q4_0_4_4 (unless they add backwards compatibility back), but Q4_0 should be the same speeds (though it may currently be bugged on some platforms) As such, I'll stop making those newer model formats soon, probably end of this week unless something changes, but you should be safe to download and Q4_0 quants and use those ! Also IQ4_NL supports repacking though not in as many shapes yet, but should get a respectable speed up on ARM chips, PR for that can be found here: https://github.com/ggerganov/llama.cpp/pull/10541 Remember, these are not meant for Apple silicon since those use the GPU and don't benefit from the repacking of weights
updated a collection about 1 month ago
papers
View all activity

Organizations

None yet

stormchaser's activity

commented a paper about 2 months ago
New activity in KingNish/Realtime-FLUX 3 months ago

Красивая девушка и молодой юноша. Свидание в отеле. Кровать.

1
#9 opened 3 months ago by
Asanych
New activity in YanweiLi/MGM-2B 9 months ago

can i get gguf files?
#1 opened 9 months ago by
stormchaser
New activity in stabilityai/stable-code-3b 11 months ago

is it trained on 1.3t or 4t tokens?

2
#8 opened 12 months ago by
stormchaser
New activity in afrideva/MiniChat-3B-GGUF about 1 year ago

question about command line to convert

1
#1 opened about 1 year ago by
stormchaser
New activity in Felladrin/TinyMistral-248M-Chat-v2 about 1 year ago

whats this for?

5
#1 opened about 1 year ago by
stormchaser
New activity in RachidAR/Marx-3b-v3-gguf about 1 year ago

not working

2
#1 opened about 1 year ago by
stormchaser
New activity in TheBloke/sqlcoder-GGUF over 1 year ago

error loading model: invalid character
#4 opened over 1 year ago by
stormchaser
New activity in TheBloke/Falcon-7B-Instruct-GGML over 1 year ago

gguf version

1
#5 opened over 1 year ago by
stormchaser
New activity in smallcloudai/Refact-1_6B-fim over 1 year ago

gguf format?

3
#4 opened over 1 year ago by
stormchaser
New activity in stabilityai/stablecode-completion-alpha-3b over 1 year ago

will it work with llama.cpp?

2
#4 opened over 1 year ago by
stormchaser

quantized models?
#5 opened over 1 year ago by
stormchaser
New activity in Open-Orca/OpenOrcaxOpenChat-Preview2-13B over 1 year ago

Good model, but still struggle with riddles

4
#2 opened over 1 year ago by
gt332a
New activity in bigscience/bloom over 2 years ago

Can Bloom-176B really be evaluated on normal hardware at a rate of 3 minutes per token?

30
#87 opened over 2 years ago by
Philimipp

Can Bloom-176B really be evaluated on normal hardware at a rate of 3 minutes per token?

30
#87 opened over 2 years ago by
Philimipp