README.md

---
license: cc-by-nc-4.0
tags:
- not-for-all-audiences
- nsfw
- mistral
- pretrained
---

Mistral-7B and airoboros-mistral2.2-7b glued together and finetuned with qlora of Pippa and LimaRPv3 dataset.

<!-- description start -->
## Description

This repo contains quantized files of Mistral-11B-Airoboros-RP-v1.

<!-- description end -->
<!-- description start -->
## Model used

- [Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1/)
- [airoboros-mistral2.2-7b](https://huggingface.co/teknium/airoboros-mistral2.2-7b/)
- PIPPA dataset 11B qlora
- LimaRPv3 dataset 11B qlora

<!-- description end -->
<!-- prompt-template start -->
## Prompt template: Alpaca or default

```
Below is an instruction that describes a task. Write a response that appropriately completes the request.

### Instruction:
{prompt}

### Response:

```

```
USER: <prompt>
ASSISTANT:
```

## The secret sauce

```
slices:
  - sources:
    - model: mistralai/Mistral-7B-v0.1
      layer_range: [0, 24]
  - sources:
    - model: teknium/airoboros-mistral2.2-7b
      layer_range: [8, 32]
merge_method: passthrough
dtype: float16
```


Special thanks to Sushi.

If you want to support me, you can [here](https://ko-fi.com/undiai).