File size: 8,965 Bytes
c8004e2 f7d5e2b c8004e2 46612b9 e180a4f ccf61ca bc4a72f 356227a ae0428d 46612b9 bede256 c8004e2 46612b9 c8004e2 e1d5826 d9472c9 c8004e2 46612b9 c8004e2 46612b9 c8004e2 46612b9 c8004e2 46612b9 c8004e2 46612b9 c8004e2 5189f32 46612b9 c8004e2 5189f32 c8004e2 5189f32 c8004e2 5189f32 c8004e2 46612b9 5189f32 46612b9 c8004e2 46612b9 c8004e2 46612b9 c8004e2 46612b9 c8004e2 46612b9 c8004e2 46612b9 c8004e2 46612b9 c8004e2 46612b9 c8004e2 46612b9 c8004e2 ecdb82d c8004e2 46612b9 78d6b41 c8004e2 46612b9 ef8bb4a c8004e2 bc4a72f 46612b9 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 |
---
library_name: peft
tags:
- SFT
- PEFT
- Mental Health
- Psychotherapy
- Fine-tuning
- Text Generation
- Chatbot
license: mit
widget:
- example_title: Depersonalization
messages:
- role: user
content: >-
I feel like I don't exist and my body is not my own, like if I'm sombody
else observin me, what could be this disorder?
- example_title: Confused
messages:
- role: user
content: >-
I'm always confused when making decisions, I cannot choose one option, I
must have only one option so I can make a decision, what are the causes of
being indecisive?
inference:
parameters:
max_new_tokens: 256
stop:
- <|end_of_text|>
- <|eot_id|>
datasets:
- mpingale/mental-health-chat-dataset
- Amod/mental_health_counseling_conversations
- heliosbrahma/mental_health_chatbot_dataset
pipeline_tag: text-generation
---
# Model Card for Model ID
<!-- Provide a quick summary of what the model is/does. -->
A fine-tuned Llama-3-8B-instruct trained to provide initial support and guidance in mental health by leveraging the power of large language models (LLMs).
It's a fine-tuned version of the LLaMA 3 model specifically trained on mental health counseling conversations.
<img src="https://cdn-uploads.huggingface.co/production/uploads/65f2cc4457fbd3c9fb65b34c/ujViqdS9zgW-IrEFbAKLw.jpeg" width="480" style="width:400px;">
## Model Details
Llama-3-8B-chat-psychotherapist
Provides active listening: The model is trained to follow conversation flows and respond attentively to user expressions.
Offers empathetic support: The model's responses are designed to be supportive and understanding, fostering a safe space for users to share their concerns.
Guides users towards self-reflection: The model can ask open-ended questions and prompt users to explore their thoughts and feelings.
Offers resources and information: Depending on the conversation, the model can provide users with relevant mental health resources and information.
Important to Note:
**This model is intended for initial support and guidance, not a replacement for professional mental health care.
It's crucial to emphasize that users should seek professional help if needed.
The model is still under development, and its responses may require human oversight for accuracy and safety.**
Further Considerations:
You can specify the type of mental health conversations the model is trained on (e.g., anxiety, depression, general well-being).
Briefly mention the dataset used for fine-tuning to showcase the model's training grounds.
If applicable, mention any limitations of the model, such as inability to diagnose or provide specific treatment plans.
## How to Get Started with the Model
Use the code below to get started with the model.
### Read model
```
from peft import PeftModel, PeftConfig
from transformers import AutoModelForCausalLM
model_id = "zementalist/llama-3-8B-chat-psychotherapist"
config = PeftConfig.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(config.base_model_name_or_path)
model = PeftModel.from_pretrained(model, model_id)
```
### Inference
```
question = "I feel like I don't exist and my body is not my own, I'm sombody else observing me, what causes these ideas?"
messages = [
{"role": "system", "content": "Answer the following inquiry:"},
{"role": "user", "content": question}
]
input_ids = tokenizer.apply_chat_template(
messages,
add_generation_prompt=True,
return_tensors="pt"
).to(model.device)
terminators = [
tokenizer.eos_token_id,
tokenizer.convert_tokens_to_ids("<|eot_id|>")
]
outputs = model.generate(
input_ids,
max_new_tokens=256,
eos_token_id=terminators,
do_sample=True,
temperature=0.01
)
response = outputs[0][input_ids.shape[-1]:]
output = tokenizer.decode(response, skip_special_tokens=True)
print(output)
```
## Training Details
### Training Data
1. [Amod/mental_health_counseling_conversations](https://huggingface.co/datasets/Amod/mental_health_counseling_conversations)
2. [mpingale/mental-health-chat-dataset](https://huggingface.co/datasets/mpingale/mental-health-chat-dataset)
3. [heliosbrahma/mental_health_chatbot_dataset](https://huggingface.co/datasets/heliosbrahma/mental_health_chatbot_dataset)
#### Training Hyperparameters
* epochs: 2
* learning rate: 6e-5
* learning rate scheduler type: constant
* Warmup ratio: 0.03
* optimizer: paged_adamw_32bit
* weight_decay: 0.001
* fp16: True
## Evaluation
<!-- This section describes the evaluation protocols and provides the results. -->
| **Step** | **Training Loss** |
|---------|-------------------|
| 100 | 2.637700 |
| 200 | 2.350900 |
| 300 | 2.234500 |
| 400 | 2.230800 |
| 500 | 2.194900 |
| 600 | 2.041100 |
| 700 | 2.033800 |
| 800 | 1.988600 |
| 900 | 1.898900 |
| 1000 | 1.823600 |
| 1100 | 1.943700 |
| 1200 | 1.789900 |
| 1300 | 1.840700 |
| 1400 | 1.760900 |
| 1500 | 1.718400 |
| 1600 | 1.725600 |
| 1700 | 1.717600 |
| 1800 | 1.628200 |
| 1900 | 1.693300 |
| 2000 | 1.535600 |
| 2100 | 1.630100 |
| 2200 | 1.592300 |
| 2300 | 1.628400 |
| 2400 | 1.462500 |
| 2500 | 1.504500 |
| 2600 | 1.517500 |
| 2700 | 1.375100 |
| 2800 | 1.452800 |
| 2900 | 1.527200 |
| 3000 | 1.418600 |
| 3100 | 1.437700 |
| 3200 | 1.247900 |
| 3300 | 1.134900 |
| 3400 | 1.283800 |
| 3500 | 1.378600 |
| 3600 | 1.114600 |
| 3700 | 1.300900 |
| 3800 | 1.225700 |
| 3900 | 1.190000 |
| 4000 | 1.260200 |
| 4100 | 1.053300 |
| 4200 | 1.173000 |
| 4300 | 1.275900 |
| 4400 | 1.079800 |
| 4500 | 1.121100 |
| 4600 | 1.043100 |
| 4700 | 1.121000 |
| 4800 | 1.087200 |
| 4900 | 1.102400 |
| 5000 | 1.126100 |
| 5100 | 1.182200 |
| 5200 | 1.156500 |
| 5300 | 0.979700 |
| 5400 | 1.145300 |
| 5500 | 1.018100 |
| 5600 | 1.070500 |
| 5700 | 0.954900 |
| 5800 | 1.074200 |
| 5900 | 1.022800 |
| 6000 | 1.037700 |
| 6100 | 0.913400 |
| 6200 | 0.929500 |
### Inference example
1. Question: I feel like I don't exist and my body is not my own,
like if I'm sombody else observin me, what could be this disorder?
* * Answer: You're describing a phenomenon called depersonalization. It's a common symptom of anxiety disorders, post-traumatic stress disorder (PTSD), and other mental health conditions. <br> Depersonalization can make you feel like you're observing yourself from the outside, like you're watching a movie or reading a book about yourself. You might also feel like you're not in control of your body or that you're just a puppet. <br> This feeling can be very distressing and can make you wonder if you're even alive. It's a very disorienting and unsettling experience. <br> If you're experiencing depersonalization, it's important to seek help from a mental health professional. They can help you identify the underlying causes of your feelings and develop coping strategies to manage them. <br>
2. Question: I'm always confused when making decisions, I cannot choose one option,
I must have only one option so I can make a decision, what are the causes of being
indecisive?
* * Answer: Indecisiveness is a common phenomenon that many people experience. It's not a mental health condition, but it can be frustrating and time-consuming. Here are some possible causes of indecisiveness: <ol> <li>Fear of making the wrong decision: Sometimes, people are afraid of making the wrong choice, which can lead to indecisiveness. This fear can be due to various factors such as fear of failure, fear of rejection, or fear of the unknown.</li> <li>Lack of clear goals: When people don't have clear goals, they can be indecisive. This is because they don't know what they want, which can lead to confusion and indecisiveness.</li> <li>Overthinking: Overthinking can be a major cause of indecisiveness. When people overthink, they can get caught up in their thoughts and be unable to make a decision.</li> <li>Lack of self-confidence: People who lack self-confidence can be indecisive. This is because they don't believe in themselves and are afraid of making the wrong decision.</li> <li>Avoidance: Avoiding a decision can be a cause of indecisiveness. When people avoid making a decision, they can be indecisive because they don't know</li> </ol>
- **Developed by:** [Zeyad Khalid] |