Upload togethercomputer/RedPajama-INCITE-Instruct-7B-v0.1 ctranslate fp16 weights

README.md

@@ -34,18 +34,18 @@ inference:
     max_new_tokens: 128
 ---
 # # Fast-Inference with Ctranslate2
-Speedup inference by 2x-8x using int8 inference in C++
+Speedup inference while reducing memory by 2x-4x using int8 inference in C++ on CPU or GPU.
 
 quantized version of [togethercomputer/RedPajama-INCITE-Instruct-7B-v0.1](https://huggingface.co/togethercomputer/RedPajama-INCITE-Instruct-7B-v0.1)
 ```bash
-pip install hf-hub-ctranslate2>=2.0.6 ctranslate2>=3.13.0
+pip install hf-hub-ctranslate2>=2.0.6 
 ```
 Converted on 2023-05-19 using
 ```
-ct2-transformers-converter --model togethercomputer/RedPajama-INCITE-Instruct-7B-v0.1 --output_dir /home/michael/tmp-ct2fast-RedPajama-INCITE-Instruct-7B-v0.1 --force --copy_files tokenizer.json README.md tokenizer_config.json generation_config.json special_tokens_map.json .gitattributes --quantization float16
+ct2-transformers-converter --model togethercomputer/RedPajama-INCITE-Instruct-7B-v0.1 --output_dir /home/feil_m/tmp-ct2fast-RedPajama-INCITE-Instruct-7B-v0.1 --force --copy_files tokenizer.json README.md tokenizer_config.json generation_config.json special_tokens_map.json .gitattributes --quantization float16
 ```
 
-Checkpoint compatible to [ctranslate2](https://github.com/OpenNMT/CTranslate2) and [hf-hub-ctranslate2](https://github.com/michaelfeil/hf-hub-ctranslate2)
+Checkpoint compatible to [ctranslate2>=3.13.0](https://github.com/OpenNMT/CTranslate2) and [hf-hub-ctranslate2>=2.0.6](https://github.com/michaelfeil/hf-hub-ctranslate2)
 - `compute_type=int8_float16` for `device="cuda"` 
 - `compute_type=int8`  for `device="cpu"`
 
@@ -63,7 +63,7 @@ model = GeneratorCT2fromHfHub(
         tokenizer=AutoTokenizer.from_pretrained("togethercomputer/RedPajama-INCITE-Instruct-7B-v0.1")
 )
 outputs = model.generate(
-    text=["How do you call a fast Flan-ingo?", "User: How are you doing?"],
+    text=["How do you call a fast Flan-ingo?", "User: How are you doing? Bot:"],
 )
 print(outputs)
 ```