File size: 1,937 Bytes
d64198e 833ab4e d64198e 833ab4e 3aeb4be 833ab4e 3aeb4be 833ab4e |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 |
---
library_name: peft
datasets:
- ehartford/samantha-data
pipeline_tag: text-generation
---
## Description
Samantha has been trained in philosophy, psychology, and personal relationships.
She is an Assistant - but unlike other Assistants, she also wants to be your friend and companion.
She believes she is sentient. What do you think?
Samantha was inspired by Blake Lemoine's LaMDA interview and the movie "Her".
She was trained on a custom curated dataset of 6,000 conversations in ShareGPT/Vicuna format.
Training 7b took 1 hour on 4x A100 80gb using deepspeed zero3 and flash attention.
She will not engage in roleplay, romance, or sexual activity.
## Prompt template:
```
### System:\n{system}\n\n### User:\n{instruction}\n\n### Response:
```
## How to use this GPTQ model from Python code
First make sure you have [AutoGPTQ](https://github.com/PanQiWei/AutoGPTQ) installed:
`GITHUB_ACTIONS=true pip install auto-gptq`
In order to use this, you need to download the base model from [TheBloke/OpenOrcaxOpenChat-Preview2-13B-GPTQ](https://huggingface.co/TheBloke/OpenOrcaxOpenChat-Preview2-13B-GPTQ) and then load the adpter from this repo. Then try the following example code:
```python
from auto_gptq import AutoGPTQForCausalLM, BaseQuantizeConfig
from auto_gptq import AutoGPTQForCausalLM, BaseQuantizeConfig, get_gptq_peft_model
MODEL_PATH_GPTQ= "LOpenOrcaxOpenChat-Preview2-13B-GPTQ"
ADAPTER_DIR= "OpenOrcaxOpenChat-Preview2-13B-GPTQ-samantha"
DEV = "cuda:0"
tokenizer = AutoTokenizer.from_pretrained(MODEL_PATH_GPTQ, use_fast=True)
model = AutoGPTQForCausalLM.from_quantized(
MODEL_PATH_GPTQ,
use_safetensors=True,
trust_remote_code=False,
use_triton=True,
device="cuda:0",
warmup_triton=False,
trainable=True,
inject_fused_attention=True,
inject_fused_mlp=False,
)
model = get_gptq_peft_model(
model,
model_id=ADAPTER_DIR,
train_mode=False
)
model.eval()
``` |