docs: update model card with presets
Browse files
README.md
CHANGED
@@ -29,6 +29,7 @@ Iridium is a 72B parameter language model created through a merge of Qwen2-72B-I
|
|
29 |
- Models: Qwen2-72B-Instruct (base), calme2.1-72b, magnum-72b-v1
|
30 |
- Merged layers: 80
|
31 |
- Total tensors: 1,043
|
|
|
32 |
|
33 |
### Tensor Distribution
|
34 |
- Attention layers: 560 files
|
@@ -55,6 +56,19 @@ tokenizer = AutoTokenizer.from_pretrained("leafspark/Iridium-72B-v0.1")
|
|
55 |
|
56 |
Find them here: [leafspark/Iridium-72B-v0.1-GGUF](https://huggingface.co/leafspark/Iridium-72B-v0.1-GGUF)
|
57 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
58 |
### Hardware Requirements
|
59 |
-
-
|
60 |
-
- ~140GB VRAM
|
|
|
29 |
- Models: Qwen2-72B-Instruct (base), calme2.1-72b, magnum-72b-v1
|
30 |
- Merged layers: 80
|
31 |
- Total tensors: 1,043
|
32 |
+
- Context length: 128k
|
33 |
|
34 |
### Tensor Distribution
|
35 |
- Attention layers: 560 files
|
|
|
56 |
|
57 |
Find them here: [leafspark/Iridium-72B-v0.1-GGUF](https://huggingface.co/leafspark/Iridium-72B-v0.1-GGUF)
|
58 |
|
59 |
+
### Optimal Sampling Parameters
|
60 |
+
|
61 |
+
I found these to work well:
|
62 |
+
```json
|
63 |
+
{
|
64 |
+
"temperature": 1
|
65 |
+
"min_p": 0.08
|
66 |
+
"top_p": 1
|
67 |
+
"top_k": 40
|
68 |
+
"repetition_penalty": 1
|
69 |
+
}
|
70 |
+
```
|
71 |
+
|
72 |
### Hardware Requirements
|
73 |
+
- At least 135GB of free space
|
74 |
+
- ~140GB VRAM/RAM
|