Update README.md

README.md

@@ -1,10 +1,89 @@
----
-library_name: transformers
-license: apache-2.0
-datasets:
-- HarryRoy/ninja-redact-2-large
-language:
-- en
----
-
-Feel free to modify any part of the model card to better suit your needs!
+
+
+# Model Card: Ninja-Masker-2-PII-Redaction
+
+## Model Overview
+
+**Model Name:** Ninja-Masker-2-PII-Redaction  
+**Model Type:** Language Model for PII Redaction  
+**License:** Apache 2.0  
+**Model Creator:** King Harry (Roy)  
+**Model Repository:** [Hugging Face Hub - Ninja-Masker-2-PII-Redaction](https://huggingface.co/King-Harry/Ninja-Masker-2-PII-Redaction)  
+
+### Model Description
+
+Ninja-Masker-2-PII-Redaction is a fine-tuned language model designed to identify and redact Personally Identifiable Information (PII) from text data. The model is based on the Meta-Llama-3.1-8B architecture and has been fine-tuned on a dataset of over 30,000 input-output pairs to perform accurate PII masking using a set of predefined tags.
+
+### Preprocessing
+
+The training data was formatted using a specific Alpaca-style prompt structure. Each prompt was paired with an instruction and input context, and the model was trained to generate the appropriate redacted output. The model was trained on a variety of PII types, including but not limited to names, email addresses, phone numbers, and credit card information.
+
+### Quantization and Optimization
+
+To optimize performance and reduce memory usage, the model was fine-tuned using 4-bit quantization. Additional optimizations included the use of Flash Attention (Xformers) and gradient checkpointing, which allowed for efficient training and inference.
+
+### Training Details
+
+- **Dataset:** HarryRoy/Ninja-Redact-2-large (Custom PII redaction dataset)
+- **Training Environment:** Google Colab, NVIDIA A100 GPU
+- **Training Framework:** PyTorch with Hugging Face Transformers, Unsloth
+- **Training Configuration:**
+  - Max sequence length: 2048 tokens
+  - Batch size: 8
+  - Gradient accumulation steps: 4
+  - Learning rate: 1e-5
+  - Epochs: 1 (500 steps)
+  - Optimizer: AdamW 8-bit
+
+### Model Performance
+
+The model was evaluated based on its ability to accurately redact PII from text while maintaining the original context and meaning. The fine-tuning process resulted in a model that effectively identifies and replaces PII with the appropriate tags in various text scenarios.
+
+### Use Cases
+
+- **Data Anonymization:** Useful for redacting PII in datasets before sharing or analysis.
+- **Email and Document Redaction:** Can be integrated into email processing systems or document management workflows to automatically redact sensitive information.
+- **Customer Support:** Enhances customer support systems by ensuring PII is automatically redacted in customer communications.
+
+### Limitations
+
+- **Tag Set:** The model relies on a predefined set of tags for redaction. It may not recognize PII types outside of this set.
+- **Context Dependence:** While the model performs well in most scenarios, its accuracy may decrease with highly complex or ambiguous input contexts.
+- **Inference Speed:** Depending on the hardware, the model's inference speed may vary, especially for long sequences.
+
+### Ethical Considerations
+
+The model is designed for responsible data management, ensuring that sensitive information is properly anonymized. However, users should be aware of the limitations and should not rely solely on automated redaction for highly sensitive data.
+
+### How to Use
+
+To use this model, you can load it from the Hugging Face Hub and integrate it into your Python or API-based applications. Below is an example of how to load and use the model:
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model_name = "King-Harry/Ninja-Masker-2-PII-Redaction"
+model = AutoModelForCausalLM.from_pretrained(model_name)
+tokenizer = AutoTokenizer.from_pretrained(model_name)
+
+input_text = "Write an email to Kendra Harvey at [email protected] summarizing the key findings from a recent cognitive therapy conference."
+inputs = tokenizer(input_text, return_tensors="pt")
+outputs = model.generate(**inputs, max_new_tokens=64)
+
+redacted_text = tokenizer.decode(outputs[0], skip_special_tokens=True)
+print(redacted_text)
+```
+
+### Citation
+
+If you use this model, please consider citing the model repository:
+
+```bibtex
+@misc{ninja_masker_2024,
+  author = {King Harry (Roy)},
+  title = {Ninja-Masker-2-PII-Redaction},
+  year = {2024},
+  url = {https://huggingface.co/King-Harry/Ninja-Masker-2-PII-Redaction},
+}
+```
+