|
--- |
|
license: apache-2.0 |
|
language: |
|
- en |
|
tags: |
|
- control-vector |
|
- creative-writing |
|
base_model: |
|
- gghfez/WizardLM-2-22b-RP |
|
--- |
|
|
|
## gghfez/WizardLM-2-22B-RP-control-vectors |
|
|
|
Creative Writing control-vectors for [gghfez/WizardLM-2-22B-RP](https://huggingface.co/WizardLM-2-22B-RP) |
|
|
|
## Usage example |
|
|
|
### llama.cpp |
|
```sh |
|
llama-server --model wizardlm-2-22b-rp-q6_k.gguf [other CLI arguments] \ |
|
--control-vector WizardLM-2-22B-RP-character_focus__debias.gguf \ |
|
--control-vector WizardLM-2-22B-RP-character_focus__narration.gguf |
|
``` |
|
|
|
### Exllamav2 |
|
|
|
Clone this repo into the same directory as the exl2 model but with -vectors appended: |
|
`huggingface-cli download gghfez/WizardLM-2-22B-RP-control-vectors --local-dir WizardLM-2-22B-RP-exl2-vectors |
|
|
|
So you should have something like |
|
/models/WizardLM-2-22B-RP-exl2 |
|
/models/WizardLM-2-22B-RP-exl2-vectors |
|
|
|
Follow instructions the instructions below for TabbyAPI or Exui: |
|
|
|
#### TabbyAPI Fork |
|
|
|
1. `git clone https://github.com/gapeleon/tabbyAPI-control-vectors` |
|
2. `cd tabbyAPI-control-vectors` |
|
3. ./start.sh #let it build the venv, etc then exit |
|
4. `source venv/bin/activate` #to enter the venv |
|
5. `pip install gguf` #to read the vectors |
|
6. `cp config_sample_control-vectors.yml config.yml` |
|
7. set the model_dir and model_name in config.yml |
|
8. tweak the control-vectors how you want them |
|
9. ./start.sh |
|
|
|
#### Exui Fork |
|
|
|
|
|
1. `git clone https://github.com/gapeleon/exui-control-vectors` |
|
2. Set it up / install it the same as regular exui |
|
3. `pip install gguf` if you don't have it already |
|
4. Start exui `python server.py` |
|
5. Configure the model as usual. |
|
6. Tick "Enable control vectors" and wait for them to appear in the UI |
|
7. Enable the vectors + direction + scaling you want, then hit "Load Model" |
|
|
|
|
|
Based off Version v3.0 of [creative-writing-control-vectors-v3.0](https://huggingface.co/jukofyork/creative-writing-control-vectors-v3.0/). |
|
|
|
See here for more info about control-vectors [creative-writing-control-vectors-v3.0](https://huggingface.co/jukofyork/creative-writing-control-vectors-v3.0/) |
