File size: 2,046 Bytes
9dc92b3 cad13af 9dc92b3 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 |
---
license: apache-2.0
language:
- en
tags:
- control-vector
- creative-writing
base_model:
- gghfez/WizardLM-2-22b-RP
---
## gghfez/WizardLM-2-22B-RP-control-vectors
Creative Writing control-vectors for [gghfez/WizardLM-2-22B-RP](https://huggingface.co/WizardLM-2-22B-RP)
## Usage example
### llama.cpp
```sh
llama-server --model wizardlm-2-22b-rp-q6_k.gguf [other CLI arguments] \
--control-vector WizardLM-2-22B-RP-character_focus__debias.gguf \
--control-vector WizardLM-2-22B-RP-character_focus__narration.gguf
```
### Exllamav2
Clone this repo into the same directory as the exl2 model but with -vectors appended:
`huggingface-cli download gghfez/WizardLM-2-22B-RP-control-vectors --local-dir WizardLM-2-22B-RP-exl2-vectors
So you should have something like
/models/WizardLM-2-22B-RP-exl2
/models/WizardLM-2-22B-RP-exl2-vectors
Follow instructions the instructions below for TabbyAPI or Exui:
#### TabbyAPI Fork
1. `git clone https://github.com/gapeleon/tabbyAPI-control-vectors`
2. `cd tabbyAPI-control-vectors`
3. ./start.sh #let it build the venv, etc then exit
4. `source venv/bin/activate` #to enter the venv
5. `pip install gguf` #to read the vectors
6. `cp config_sample_control-vectors.yml config.yml`
7. set the model_dir and model_name in config.yml
8. tweak the control-vectors how you want them
9. ./start.sh
#### Exui Fork
1. `git clone https://github.com/gapeleon/exui-control-vectors`
2. Set it up / install it the same as regular exui
3. `pip install gguf` if you don't have it already
4. Start exui `python server.py`
5. Configure the model as usual.
6. Tick "Enable control vectors" and wait for them to appear in the UI
7. Enable the vectors + direction + scaling you want, then hit "Load Model"
Based off Version v3.0 of [creative-writing-control-vectors-v3.0](https://huggingface.co/jukofyork/creative-writing-control-vectors-v3.0/).
See here for more info about control-vectors [creative-writing-control-vectors-v3.0](https://huggingface.co/jukofyork/creative-writing-control-vectors-v3.0/) |