wassemgtk nbroad HF staff commited on
Commit
22c5896
·
1 Parent(s): f4592e8

add example for text-generation-inference (#3)

Browse files

- add example for text-generation-inference (75567ca2bc15f3012f7380032ed1229345223c5e)


Co-authored-by: Nicholas Broad <[email protected]>

Files changed (1) hide show
  1. README.md +9 -0
README.md CHANGED
@@ -84,6 +84,15 @@ clean_output = output_text.split("### Response:")[1].strip()
84
  print(clean_output)
85
  ```
86
 
 
 
 
 
 
 
 
 
 
87
 
88
  ### Limitations and Biases
89
 
 
84
  print(clean_output)
85
  ```
86
 
87
+ It can also be used with text-generation-inference
88
+
89
+ ```sh
90
+ model=Writer/InstructPalmyra-20b
91
+ volume=$PWD/data
92
+
93
+ docker run --gpus all --shm-size 1g -p 8080:80 -v $volume:/data ghcr.io/huggingface/text-generation-inference --model-id $model
94
+ ```
95
+
96
 
97
  ### Limitations and Biases
98