Update README_zh-CN.md

README_zh-CN.md

@@ -1,14 +1,12 @@
----
-license: gpl-3.0
----
 # **NanoTranslator-S**
 
+[English](README.md) | 简体中文
+
 ## Introduction
 
 这是 NanoTranslator 的 Small 型号，目前仅支持**英译中**。仓库中同时提供了 ONNX 版本的模型。
 
 
-
 | Size | Params. |  V.  |  H.  |  I.  |  L.  | Att. H. | KV H. | Tie Emb. |
 | :--: | :-----: | :--: | :--: | :--: | :--: | :-----: | :---: | :------: |
 |  XL  |  50 M   | 8000 | 320  | 1792 |  24  |   16    |   4   |   True   |
@@ -28,7 +26,89 @@ license: gpl-3.0
 
 ## How to use
 
-### Normal
+Prompt 格式如下：
+
+```
+<|im_start|> {English Text} <|endoftext|>
+```
+
+### Directly using transformers
+
+```python
+import torch
+from transformers import AutoTokenizer, AutoModelForCausalLM
+
+model_path = 'Mxode/NanoTranslator-S'
+
+tokenizer = AutoTokenizer.from_pretrained(model_path)
+model = AutoModelForCausalLM.from_pretrained(model_path)
+
+def translate(text: str, model, **kwargs):
+    generation_args = dict(
+        max_new_tokens = kwargs.pop("max_new_tokens", 512),
+        do_sample = kwargs.pop("do_sample", True),
+        temperature = kwargs.pop("temperature", 0.55),
+        top_p = kwargs.pop("top_p", 0.8),
+        top_k = kwargs.pop("top_k", 40),
+        **kwargs
+    )
+
+    prompt = "<|im_start|>" + text + "<|endoftext|>"
+    model_inputs = tokenizer([prompt], return_tensors="pt").to(model.device)
+
+    generated_ids = model.generate(model_inputs.input_ids, **generation_args)
+    generated_ids = [
+        output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
+    ]
+
+    response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
+    return response
+
+text = "I love to watch my favorite TV series."
+
+response = translate(text, model, max_new_tokens=64, do_sample=False)
+print(response)
+```
 
 
 ### ONNX
+
+根据实际测试，使用 ONNX 模型推理会比直接使用 transformers 推理要**快 2～10 倍**。
+
+如果希望使用 ONNX 模型，那么你需要手动切换到 [onnx 分支](https://huggingface.co/Mxode/NanoTranslator-S/tree/onnx)并从本地加载。
+
+参考文档：
+
+- [Export to ONNX](https://huggingface.co/docs/transformers/serialization)
+- [Inference pipelines with the ONNX Runtime accelerator](https://huggingface.co/docs/optimum/main/en/onnxruntime/usage_guides/pipelines)
+
+**Using ORTModelForCausalLM**
+
+```python
+from optimum.onnxruntime import ORTModelForCausalLM
+from transformers import AutoTokenizer
+
+model_path = "your/folder/to/onnx_model"
+
+ort_model = ORTModelForCausalLM.from_pretrained(model_path)
+tokenizer = AutoTokenizer.from_pretrained(model_path)
+
+text = "I love to watch my favorite TV series."
+
+response = translate(text, ort_model, max_new_tokens=64, do_sample=False)
+print(response)
+```
+
+**Using pipeline**
+
+```python
+from optimum.pipelines import pipeline
+
+model_path = "your/folder/to/onnx_model"
+pipe = pipeline("text-generation", model=model_path, accelerator="ort")
+
+text = "I love to watch my favorite TV series."
+
+response = pipe(text, max_new_tokens=64, do_sample=False)
+response
+```