Update README.md
Browse files
README.md
CHANGED
@@ -27,21 +27,21 @@ NexoNimbus-MoE-2x7B is the 10th best-performing 13B LLM on the Open LLM Leaderbo
|
|
27 |
|
28 |
| Task |Version| Metric |Value| |Stderr|
|
29 |
|-------------|------:|--------|----:|---|-----:|
|
30 |
-
|arc_challenge| 0|acc |
|
31 |
-
| | |acc_norm|
|
32 |
-
|hellaswag | 0|acc |
|
33 |
-
| | |acc_norm|
|
34 |
-
|gsm8k | 0|acc |
|
35 |
-
|winogrande | 0|acc |
|
36 |
-
|mmlu | 0|acc |64.
|
37 |
|
38 |
-
Average:
|
39 |
|
40 |
### TruthfulQA
|
41 |
| Task |Version|Metric|Value| |Stderr|
|
42 |
|-------------|------:|------|----:|---|-----:|
|
43 |
-
|truthfulqa_mc| 1|mc1 |
|
44 |
-
| | |mc2 |
|
45 |
|
46 |
|
47 |
## 🧩 Configuration
|
@@ -93,7 +93,7 @@ experts:
|
|
93 |
|
94 |
## 💻 Usage
|
95 |
|
96 |
-
Here's a [Colab notebook](https://colab.research.google.com/drive/
|
97 |
|
98 |
```python
|
99 |
!pip install -qU transformers bitsandbytes accelerate
|
@@ -111,7 +111,7 @@ pipeline = transformers.pipeline(
|
|
111 |
model_kwargs={"torch_dtype": torch.float16, "load_in_4bit": True},
|
112 |
)
|
113 |
|
114 |
-
messages = [{"role": "user", "content": "Explain what
|
115 |
prompt = pipeline.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
|
116 |
outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
|
117 |
print(outputs[0]["generated_text"])
|
|
|
27 |
|
28 |
| Task |Version| Metric |Value| |Stderr|
|
29 |
|-------------|------:|--------|----:|---|-----:|
|
30 |
+
|arc_challenge| 0|acc |62.28|± | 1.41|
|
31 |
+
| | |acc_norm|66.80|± | 1.37|
|
32 |
+
|hellaswag | 0|acc |66.83|± | 0.46|
|
33 |
+
| | |acc_norm|85.66|± | 0.34|
|
34 |
+
|gsm8k | 0|acc |53.52|± | 1.37|
|
35 |
+
|winogrande | 0|acc |81.53|± | 1.09|
|
36 |
+
|mmlu | 0|acc |64.51|± | 1.00|
|
37 |
|
38 |
+
Average: 67.51%
|
39 |
|
40 |
### TruthfulQA
|
41 |
| Task |Version|Metric|Value| |Stderr|
|
42 |
|-------------|------:|------|----:|---|-----:|
|
43 |
+
|truthfulqa_mc| 1|mc1 |35.98|± | 1.68|
|
44 |
+
| | |mc2 |53.05|± | 1.53|
|
45 |
|
46 |
|
47 |
## 🧩 Configuration
|
|
|
93 |
|
94 |
## 💻 Usage
|
95 |
|
96 |
+
Here's a [Colab notebook](https://colab.research.google.com/drive/1B1Q7vO95cDkEJbKIPhOWr6exB9-Q_lr-?usp=sharing) to run NexoNimbus-MoE-2x7B in 4-bit precision on a free T4 GPU.
|
97 |
|
98 |
```python
|
99 |
!pip install -qU transformers bitsandbytes accelerate
|
|
|
111 |
model_kwargs={"torch_dtype": torch.float16, "load_in_4bit": True},
|
112 |
)
|
113 |
|
114 |
+
messages = [{"role": "user", "content": "Explain what is machine learning."}]
|
115 |
prompt = pipeline.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
|
116 |
outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
|
117 |
print(outputs[0]["generated_text"])
|