vicky4s4s commited on
Commit
eecd5f1
·
verified ·
1 Parent(s): b8139c9

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +20 -0
README.md CHANGED
@@ -17,6 +17,26 @@ datasets:
17
  - HuggingFaceH4/ultrachat_200k
18
  - liuhaotian/LLaVA-Instruct-150K
19
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
20
  # Model Card for Mixtral-8x7B
21
 
22
  The Mixtral-8x7B Large Language Model (LLM) is a pretrained generative Sparse Mixture of Experts. The Mixtral-8x7B outperforms Llama 2 70B on most benchmarks we tested.
 
17
  - HuggingFaceH4/ultrachat_200k
18
  - liuhaotian/LLaVA-Instruct-150K
19
  ---
20
+
21
+
22
+ ## Use below code to develop
23
+
24
+ ```py
25
+ # pip install accelerate transformers torch
26
+
27
+ import torch
28
+ from transformers import pipeline
29
+ pipe = pipeline("text-generation", model="vicky4s4s/Mistral-instruct-46B", torch_dtype=torch.bfloat16, device_map="auto")
30
+
31
+ messages = [
32
+
33
+ {"role": "user", "content": "what is your name?"},
34
+ ]
35
+ prompt = pipe.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
36
+ outputs = pipe(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
37
+ print(outputs[0]["generated_text"])
38
+
39
+ ```
40
  # Model Card for Mixtral-8x7B
41
 
42
  The Mixtral-8x7B Large Language Model (LLM) is a pretrained generative Sparse Mixture of Experts. The Mixtral-8x7B outperforms Llama 2 70B on most benchmarks we tested.