Update README.md

README.md

@@ -1,3 +1,21 @@
----
-license: apache-2.0
----
+---
+license: apache-2.0
+---
+# Pure quantizations of `Mistral-7B-Instruct-v0.3` for [mistral.java](https://github.com/mukel/mistral.java).
+
+In the wild, Q8_0 quantizations are fine, but Q4_0 quantizations are rarely pure e.g. the output.weights tensor is quantized with Q6_K, instead of Q4_0.  
+A pure Q4_0 quantization can be generated from a high precision (F32, F16, BFLOAT16) .gguf source with the quantize utility from llama.cpp as follows:
+
+```
+./quantize --pure ./Mistral-7B-Instruct-v0.3-F32.gguf ./Mistral-7B-Instruct-v0.3-Q4_0.gguf Q4_0
+```
+
+Original model: [https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.3](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.3)
+
+****Note that this model does not support a System prompt.**
+
+The Mistral-7B-Instruct-v0.3 Large Language Model (LLM) is an instruct fine-tuned version of the Mistral-7B-v0.3.  
+Mistral-7B-v0.3 has the following changes compared to Mistral-7B-v0.2  
+ - Extended vocabulary to 32768
+ - Supports v3 Tokenizer
+ - Supports function calling