Update README.md
Browse files
README.md
CHANGED
@@ -17,6 +17,26 @@ datasets:
|
|
17 |
- HuggingFaceH4/ultrachat_200k
|
18 |
- liuhaotian/LLaVA-Instruct-150K
|
19 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
20 |
# Model Card for Mixtral-8x7B
|
21 |
|
22 |
The Mixtral-8x7B Large Language Model (LLM) is a pretrained generative Sparse Mixture of Experts. The Mixtral-8x7B outperforms Llama 2 70B on most benchmarks we tested.
|
|
|
17 |
- HuggingFaceH4/ultrachat_200k
|
18 |
- liuhaotian/LLaVA-Instruct-150K
|
19 |
---
|
20 |
+
|
21 |
+
|
22 |
+
## Use below code to develop
|
23 |
+
|
24 |
+
```py
|
25 |
+
# pip install accelerate transformers torch
|
26 |
+
|
27 |
+
import torch
|
28 |
+
from transformers import pipeline
|
29 |
+
pipe = pipeline("text-generation", model="vicky4s4s/Mistral-instruct-46B", torch_dtype=torch.bfloat16, device_map="auto")
|
30 |
+
|
31 |
+
messages = [
|
32 |
+
|
33 |
+
{"role": "user", "content": "what is your name?"},
|
34 |
+
]
|
35 |
+
prompt = pipe.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
|
36 |
+
outputs = pipe(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
|
37 |
+
print(outputs[0]["generated_text"])
|
38 |
+
|
39 |
+
```
|
40 |
# Model Card for Mixtral-8x7B
|
41 |
|
42 |
The Mixtral-8x7B Large Language Model (LLM) is a pretrained generative Sparse Mixture of Experts. The Mixtral-8x7B outperforms Llama 2 70B on most benchmarks we tested.
|