OwenArli commited on
Commit
be826a7
·
verified ·
1 Parent(s): 4d26a2f

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +58 -3
README.md CHANGED
@@ -1,3 +1,58 @@
1
- ---
2
- license: llama3.1
3
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: llama3.1
3
+ ---
4
+
5
+ # Llama-3.1-8B-ArliAI-RPMax-v1.3
6
+
7
+ =====================================
8
+
9
+ ## RPMax Series Overview
10
+
11
+ v1.1 = [2B](https://huggingface.co/ArliAI/Gemma-2-2B-ArliAI-RPMax-v1.1) | [3.8B](https://huggingface.co/ArliAI/Phi-3.5-mini-3.8B-ArliAI-RPMax-v1.1) | [8B](https://huggingface.co/ArliAI/Llama-3.1-8B-ArliAI-RPMax-v1.1) | [9B](https://huggingface.co/ArliAI/Gemma-2-9B-ArliAI-RPMax-v1.1) | [12B](https://huggingface.co/ArliAI/Mistral-Nemo-12B-ArliAI-RPMax-v1.1) | [20B](https://huggingface.co/ArliAI/InternLM2_5-20B-ArliAI-RPMax-v1.1) | [22B](https://huggingface.co/ArliAI/Mistral-Small-22B-ArliAI-RPMax-v1.1) | [70B](https://huggingface.co/ArliAI/Llama-3.1-70B-ArliAI-RPMax-v1.1)
12
+
13
+ v1.2 = [8B](https://huggingface.co/ArliAI/Llama-3.1-8B-ArliAI-RPMax-v1.2) | [12B](https://huggingface.co/ArliAI/Mistral-Nemo-12B-ArliAI-RPMax-v1.2) | [70B](https://huggingface.co/ArliAI/Llama-3.1-70B-ArliAI-RPMax-v1.2)
14
+
15
+ v1.3 = [8B](https://huggingface.co/ArliAI/Llama-3.1-8B-ArliAI-RPMax-v1.3) | [32B](https://huggingface.co/ArliAI/Qwen2.5-32B-ArliAI-RPMax-v1.3)
16
+
17
+ RPMax is a series of models that are trained on a diverse set of curated creative writing and RP datasets with a focus on variety and deduplication. This model is designed to be highly creative and non-repetitive by making sure no two entries in the dataset have repeated characters or situations, which makes sure the model does not latch on to a certain personality and be capable of understanding and acting appropriately to any characters or situations.
18
+
19
+ Many RPMax users mentioned that these models does not feel like any other RP models, having a different writing style and generally doesn't feel in-bred.
20
+
21
+ You can access the model at https://arliai.com and we also have a models ranking page at https://www.arliai.com/models-ranking
22
+
23
+ Ask questions in our new Discord Server https://discord.com/invite/t75KbPgwhk or on our subreddit https://www.reddit.com/r/ArliAI/
24
+
25
+ ## Model Description
26
+
27
+ Llama-3.1-8B-ArliAI-RPMax-v1.3 is a variant made from the Llama-3.1-8B-Instruct model.
28
+
29
+ Let us know what you think of the model! The different parameter versions are based on different models, so they might all behave slightly differently in their own way.
30
+
31
+ v1.3 updated models are trained with updated software and configs such as the updated transformers library that fixes the gradient checkpointing bug which should help the model learn better.
32
+ This version also uses RSLORA+ for training which helps the model learn even better.
33
+
34
+ ### Specs
35
+
36
+ * **Context Length**: 128K
37
+ * **Parameters**: 8B
38
+
39
+ ### Training Details
40
+
41
+ * **Sequence Length**: 8192
42
+ * **Training Duration**: Approximately 10 hours on 2x3090Ti
43
+ * **Epochs**: 1 epoch training for minimized repetition sickness
44
+ * **RS-QLORA+**: 64-rank 64-alpha, resulting in ~2% trainable weights
45
+ * **Learning Rate**: 0.00001
46
+ * **Gradient accumulation**: Very low 32 for better learning.
47
+
48
+ ## Quantization
49
+
50
+ The model is available in quantized formats:
51
+
52
+ * **FP16**: https://huggingface.co/ArliAI/Llama-3.1-8B-ArliAI-RPMax-v1.3
53
+ * **GGUF**: https://huggingface.co/ArliAI/Llama-3.1-8B-ArliAI-RPMax-v1.3-GGUF
54
+
55
+ ## Suggested Prompt Format
56
+
57
+ Meta Llama 3 Instruct Format
58
+