Update README.md

README.md

@@ -11,12 +11,72 @@ tags:
 - trl
 ---
 
-# Uploaded  model
+## Inference
 
-- **Developed by:** ImranzamanML
-- **License:** apache-2.0
-- **Finetuned from model :** unsloth/mistral-7b-v0.3-bnb-4bit
+```python
+from unsloth import FastLanguageModel
+model, tokenizer = FastLanguageModel.from_pretrained(
+model_name = "ImranzamanML/7B_finetuned_Mistral",
+max_seq_length = 5020,
+dtype = None,
+load_in_4bit = True)
+```
 
-This mistral model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
+## Prompt to use for model answer
+
+```python
+data_prompt = """Analyze the provided text from a mental health perspective. Identify any indicators of emotional distress, coping mechanisms, or psychological well-being. Highlight any potential concerns or positive aspects related to mental health, and provide a brief explanation for each observation.
+
+### Input:
+{}
+
+### Response:
+{}"""
+
+EOS_TOKEN = tokenizer.eos_token
+def formatting_prompt(examples):
+    inputs       = examples["Context"]
+    outputs      = examples["Response"]
+    texts = []
+    for input_, output in zip(inputs, outputs):
+        text = data_prompt.format(input_, output) + EOS_TOKEN
+        texts.append(text)
+    return { "text" : texts, }
+```
+
+    
+text="I'm going through some things with my feelings and myself. I barely sleep and I do nothing but think about how I'm worthless and how I shouldn't be here. I've never tried or contemplated suicide. I've always wanted to fix my issues, but I never get around to it. How can I change my feeling of being worthless to everyone?"
+
+<div style="background-color: #f2f2f2; border-left: 5px solid #4CAF50; padding: 15px; margin: 20px 0;">
+    <strong>Note:</strong> Lets use the fine-tuned model for inference in order to generate responses based on mental health-related prompts !
+</div>
+
+<h3 style="color: #388e3c; font-family: Arial, sans-serif;">Here is some keys to note:</h3>
+
+<ol style="margin-left: 20px;">
+        <p>The <code>model = FastLanguageModel.for_inference(model)</code> configures the model specifically for inference, optimizing its performance for generating responses.</p>
+    </li>
+        <p>The input text is tokenized using the <code>tokenizer</code>, it convert the text into a format that model can process. We are using <code>data_prompt</code> to format the input text, while the response placeholder is left empty to get response from model. The <code>return_tensors = "pt"</code> parameter specifies that the output should be in PyTorch tensors, which are then moved to the GPU using <code>.to("cuda")</code> for faster processing.</p>
+    </li>
+        <p>The <code>model.generate</code> method generating response based on the tokenized inputs. The parameters <code>max_new_tokens = 5020</code> and <code>use_cache = True</code> ensure that the model can produce long and coherent responses efficiently by utilizing cached computation from previous layers.</p>
+    </li>
+</ol>
+
+```python
+model = FastLanguageModel.for_inference(model)
+inputs = tokenizer(
+[
+    data_prompt.format(
+        #instructions
+        text,
+        #answer
+        "",
+    )
+], return_tensors = "pt").to("cuda")
+
+outputs = model.generate(**inputs, max_new_tokens = 5020, use_cache = True)
+answer=tokenizer.batch_decode(outputs)
+answer = answer[0].split("### Response:")[-1]
+print("Answer of the question is:", answer)
+```
 
-[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)