Upload README.md

README.md

@@ -0,0 +1,76 @@
+```markdown
+# UlizaLlama_Q4_K_M-gguf 4-bit Quantized Bilingual Language Model
+
+## Overview
+
+UlizaLlama_Q4_K_M-gguf is a 4-bit quantized version of the UlizaLlama model, a 7B parameter language model fine-tuned for Swahili and English. This quantized model offers the same bilingual capabilities as the original UlizaLlama but with significantly reduced model size and improved inference speed, making it ideal for deployment in resource-constrained environments.
+
+### Key Features
+
+- **Bilingual Proficiency**: Excels in both Swahili and English, with a focus on instructional tasks.
+- **4-bit Quantization**: Utilizes the QQUF (Quantized QUarter Float) format for a 75% reduction in model size.
+- **Efficient Inference**: Faster processing and lower memory footprint compared to the full-precision model.
+- **Versatile Applications**: Suitable for question-answering, chat assistants, and various domain-specific tasks.
+
+## Model Details
+
+- **Original Model**: UlizaLlama (7B parameters)
+- **Base Model**: Jacaranda/kiswallama-pretrained (derived from Meta/Llama2)
+- **Quantization Method**: 4-bit QQUF
+- **Languages**: Swahili and English
+- **License**: CC BY-NC-SA 4.0 DEED
+
+## Installation
+
+To use UlizaLlama-QQUF, you'll need a library that supports 4-bit quantized models. We recommend using the `bitsandbytes` library:
+
+```bash
+pip install bitsandbytes
+pip install transformers
+```
+
+## Usage
+
+Here's a simple example of how to load and use de-coder/UlizaLlama_Q4_K_M-gguf
+
+```python
+from transformers import AutoTokenizer, AutoModelForCausalLM
+import bitsandbytes as bnb
+
+# Load the quantized model
+model = AutoModelForCausalLM.from_pretrained("de-coder/UlizaLlama_Q4_K_M-gguf", 
+                                            device_map="auto", 
+                                            trust_remote_code=True)
+tokenizer = AutoTokenizer.from_pretrained("de-coder/UlizaLlama_Q4_K_M-gguf")
+
+# Example usage
+prompt = "Niambie kuhusu historia ya Kilimanjaro."
+input_ids = tokenizer(prompt, return_tensors="pt").input_ids
+output = model.generate(input_ids, max_length=100)
+print(tokenizer.decode(output[0], skip_special_tokens=True))
+```
+
+## Performance and Trade-offs
+
+UlizaLlama-QQUF offers substantial improvements in model size and inference speed. However, there might be a slight degradation in performance compared to the full-precision model. We encourage users to benchmark the model on their specific tasks to understand these trade-offs.
+
+## Use Cases
+
+1. Chatbots for healthcare, agriculture, education, and more.
+2. Language learning applications.
+3. Information services in Swahili-speaking regions.
+4. Edge devices and mobile applications.
+
+## Citation and Acknowledgments
+
+If you use UlizaLlama_Q4_K_M-gguf in your work, please cite:
+
+```bibtex
+@misc{mwongela2023ulizallama,
+  title={UlizaLlama: A Bilingual Language Model for Swahili and English},
+  author={Kelvin Githu(de-coder)},
+  year={2024},
+  publisher={Kelvin Githu},
+  howpublished={\url{https://huggingface.co/de-coder/UlizaLlama_Q4_K_M-gguf}},
+}
+```