Update README.md

README.md

@@ -3,12 +3,41 @@ license: apache-2.0
 language:
   - en
 pipeline_tag: text-generation
-inference: true
+inference: false
 tags:
   - pytorch
   - mistral
   - finetuned
 ---
+# Mistral 7B - Holodeck exl2
+
+Original model: [Mistral-7B-Holodeck-1](https://huggingface.co/KoboldAI/Mistral-7B-Holodeck-1)  
+Model creator: [KoboldAI](https://huggingface.co/KoboldAI)  
+
+## Quants
+[4bpw-h6 (main)](https://huggingface.co/cgus/Mistral-7B-Holodeck-1-exl2/tree/main)  
+[4.25bpw-h6](https://huggingface.co/cgus/Mistral-7B-Holodeck-1-exl2/tree/4.25bpw-h6)  
+[4.65bpw-h6](https://huggingface.co/cgus/Mistral-7B-Holodeck-1-exl2/tree/4.65bpw-h6)  
+[5bpw-h6](https://huggingface.co/cgus/Mistral-7B-Holodeck-1-exl2/tree/5bpw-h6)  
+[6bpw-h6](https://huggingface.co/cgus/Mistral-7B-Holodeck-1-exl2/tree/6bpw-h6)  
+[8bpw-h8](https://huggingface.co/cgus/Mistral-7B-Holodeck-1-exl2/tree/8bpw-h8)  
+
+## Quantization notes
+
+Made with exllamav2 0.0.15 with the default dataset.
+
+## How to run
+
+This quantization method uses GPU and requires Exllamav2 loader which can be found in following applications:
+
+[Text Generation Webui](https://github.com/oobabooga/text-generation-webui)  
+
+[KoboldAI](https://github.com/henk717/KoboldAI)  
+
+[ExUI](https://github.com/turboderp/exui)  
+
+# Original card
+
 # Mistral 7B - Holodeck
 ## Model Description
 Mistral 7B-Holodeck is a finetune created using Mistral's 7B model.