Update README.md
Browse files
README.md
CHANGED
@@ -1,3 +1,42 @@
|
|
1 |
---
|
2 |
license: llama2
|
|
|
|
|
|
|
|
|
|
|
3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
---
|
2 |
license: llama2
|
3 |
+
pipeline_tag: text-generation
|
4 |
+
tags:
|
5 |
+
- llama
|
6 |
+
- llama2
|
7 |
+
- not-for-all-audiences
|
8 |
---
|
9 |
+
# Introduction
|
10 |
+
|
11 |
+
Astramix is a merge of various Llama-2-7b finetune models, using [ties-merge method](https://github.com/cg123/ties-merge), thanks for [Chargoddard](https://huggingface.co/chargoddard).
|
12 |
+
Subsequently, lora merge script was used, created by [zarakiquemparte](https://github.com/zarakiquemparte/zaraki-tools) ([link](https://huggingface.co/zarakiquemparte) to the Hugging Face profile.)
|
13 |
+
|
14 |
+
Approximate rating for this model, noticed in short-time use:
|
15 |
+
* Great roleplay capabilities, limited only by model parameters.
|
16 |
+
* Poor quality reasoning, because of the model limitations.
|
17 |
+
* Almost censorship absence
|
18 |
+
|
19 |
+
Feel free to test the model.
|
20 |
+
|
21 |
+
Following base model used for merge: [Llama-2-7B-fp16](https://huggingface.co/TheBloke/Llama-2-7B-fp16)
|
22 |
+
|
23 |
+
#### List of models used for merge:
|
24 |
+
* [Nous-Hermes-llama-2-7b](https://huggingface.co/NousResearch/Nous-Hermes-llama-2-7b)
|
25 |
+
* [airoboros-l2-7b-2.1](https://huggingface.co/jondurbin/airoboros-l2-7b-2.1)
|
26 |
+
* [orca_mini_v3_7b ](https://huggingface.co/psmathur/orca_mini_v3_7b)
|
27 |
+
* [Platypus2-7B](https://huggingface.co/garage-bAInd/Platypus2-7B)
|
28 |
+
* [Tulpar-7b-v0](https://huggingface.co/HyperbeeAI/Tulpar-7b-v0)
|
29 |
+
|
30 |
+
Then, two LoRAs was merged into basemix model, using script specified above:
|
31 |
+
* [limarp-llama2-v2](https://huggingface.co/lemonilia/limarp-llama2-v2) (Licensed under AGPLv3)
|
32 |
+
* [airoboros-lmoe-7b-2.1](https://huggingface.co/jondurbin/airoboros-lmoe-7b-2.1) (Utilizing creative version)
|
33 |
+
|
34 |
+
#### I suggest using Alpaca instruct format:
|
35 |
+
```
|
36 |
+
### Instruction:
|
37 |
+
(your instruct prompt is here)
|
38 |
+
### Response: {prompt}
|
39 |
+
```
|
40 |
+
## Limitations and risks
|
41 |
+
Llama2 is licensed under LLama 2 Community License, various finetunes or (Q)LoRAs has appropriate licenses depending on used datasets in finetuning or training Low-Rank Adaptations.
|
42 |
+
This mix can generate heavily biased output, which aren't suitable for minors or common audience.
|