Kooten
/

BagelMIsteryTour-v2-8x7B-Imatrix-GGUF

Inference Endpoints

Model card Files Files and versions Community

Kooten commited on Feb 7, 2024

Commit

c1d0506

·

verified ·

1 Parent(s): 4283392

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -19,7 +19,7 @@ Imatrix GGUF quant of [ycros/BagelMIsteryTour-v2-8x7B](https://huggingface.co/yc
 EXL2: [5bpw](https://huggingface.co/Kooten/BagelMIsteryTour-v2-8x7B-5bpw-exl2), [3.5bpw](https://huggingface.co/Kooten/BagelMIsteryTour-v2-8x7B-3.5bpw-exl2)
-GGUF:
 ## Prompt format: Alpaca
 It is noted to also work with mistral

 EXL2: [5bpw](https://huggingface.co/Kooten/BagelMIsteryTour-v2-8x7B-5bpw-exl2), [3.5bpw](https://huggingface.co/Kooten/BagelMIsteryTour-v2-8x7B-3.5bpw-exl2)
+[GGUF](https://huggingface.co/Kooten/BagelMIsteryTour-v2-8x7B-Imatrix-GGUF): [IQ3_XXS](https://huggingface.co/Kooten/BagelMIsteryTour-v2-8x7B-Imatrix-GGUF/blob/main/BagelMIsteryTour-v2-8x7B-IQ3_XXS.gguf), [IQ2_XS](https://huggingface.co/Kooten/BagelMIsteryTour-v2-8x7B-Imatrix-GGUF/blob/main/BagelMIsteryTour-v2-8x7B-IQ2_XS.gguf), [IQ2_XXS](https://huggingface.co/Kooten/BagelMIsteryTour-v2-8x7B-Imatrix-GGUF/blob/main/BagelMIsteryTour-v2-8x7B-IQ2_XXS.gguf)
 ## Prompt format: Alpaca
 It is noted to also work with mistral