Upload EleutherAI/pythia-2.8b ctranslate fp16 weights

README.md

@@ -13,18 +13,18 @@ datasets:
 - the_pile
 ---
 # # Fast-Inference with Ctranslate2
-Speedup inference by 2x-8x using int8 inference in C++
+Speedup inference while reducing memory by 2x-4x using int8 inference in C++ on CPU or GPU.
 
 quantized version of [EleutherAI/pythia-2.8b](https://huggingface.co/EleutherAI/pythia-2.8b)
 ```bash
-pip install hf-hub-ctranslate2>=2.0.6 ctranslate2>=3.13.0
+pip install hf-hub-ctranslate2>=2.0.6 
 ```
 Converted on 2023-05-19 using
 ```
-ct2-transformers-converter --model EleutherAI/pythia-2.8b --output_dir /home/michael/tmp-ct2fast-pythia-2.8b --force --copy_files tokenizer.json README.md tokenizer_config.json special_tokens_map.json .gitattributes --quantization float16
+ct2-transformers-converter --model EleutherAI/pythia-2.8b --output_dir /home/feil_m/tmp-ct2fast-pythia-2.8b --force --copy_files tokenizer.json README.md tokenizer_config.json special_tokens_map.json .gitattributes --quantization float16
 ```
 
-Checkpoint compatible to [ctranslate2](https://github.com/OpenNMT/CTranslate2) and [hf-hub-ctranslate2](https://github.com/michaelfeil/hf-hub-ctranslate2)
+Checkpoint compatible to [ctranslate2>=3.13.0](https://github.com/OpenNMT/CTranslate2) and [hf-hub-ctranslate2>=2.0.6](https://github.com/michaelfeil/hf-hub-ctranslate2)
 - `compute_type=int8_float16` for `device="cuda"` 
 - `compute_type=int8`  for `device="cpu"`
 
@@ -42,7 +42,7 @@ model = GeneratorCT2fromHfHub(
         tokenizer=AutoTokenizer.from_pretrained("EleutherAI/pythia-2.8b")
 )
 outputs = model.generate(
-    text=["How do you call a fast Flan-ingo?", "User: How are you doing?"],
+    text=["How do you call a fast Flan-ingo?", "User: How are you doing? Bot:"],
 )
 print(outputs)
 ```