Hugging Face's logo

brucethemoose
/
Yi-34B-200K-RPMerge-iMat.GGUF

Transformers
GGUF
English
mergekit
Merge
Yi
Inference Endpoints
Model card Files Community
1

See the main model card: https://huggingface.co/brucethemoose/Yi-34B-200K-RPMerge

Experimental imatrix quantizations, at 16K context, using a cocktail of data taken from the exllamav2 repo, kalomaze's random token tests and some stories.

Consider them experimental! But they may be better than the equivalent non-matrix GGUFs. Or they may not!

Downloads last month
13
GGUF
Model size
34.4B params
Architecture
llama

2-bit

View +7 files
Inference Providers NEW
This model is not currently available via any of the supported third-party Inference Providers, and HF Inference API was unable to determine this model’s pipeline type.

Collection including brucethemoose/Yi-34B-200K-RPMerge-iMat.GGUF