Update README.md

README.md

@@ -6,7 +6,7 @@ license: cc-by-nc-nd-3.0
 ## Introduction
 We release a state-of-the-art instruct model of its class, **SFR-Iterative-DPO-LLaMA-3-8B-R**.
 On all three widely-used instruct model benchmarks: **Alpaca-Eval-V2**, **MT-Bench**, **Chat-Arena-Hard**, our model outperforms all models of similar size (e.g., LLaMA-3-8B-it), most large open-sourced models (e.g., Mixtral-8x7B-it),
-and strong proprietary models (e.g., GPT-3.5-turbo-0613). The model is trained with open-sourced datasets without any additional human- or GPT4-labeling.
+and strong proprietary models (e.g., GPT-3.5-turbo-0613). The model is trained with open-sourced datasets without any additional human-/GPT4-labeling.
 
 ## Model Releases
 - [SFT model](https://huggingface.co/Salesforce/SFR-SFT-LLaMA-3-8B-R)
@@ -18,7 +18,9 @@ and strong proprietary models (e.g., GPT-3.5-turbo-0613). The model is trained w
 - [Prompt collection for RLHF training]()
 
 ## Training methods
-The key to our training is iterative RLHF.
+We have developed a simple and efficient online RLHF recipe for LLM instruct training. Our recipe is DPO-based and thus much cheaper and simpler to train and tune compared to PPO-based approaches.
+Unlike widely-used offline DPO, the online component of our approach effectively mitigates distribution shifts during policy optimization.
+For a detailed exposition, please refer to our accompanying technical report.
 
 
 ## Chat Benchmarks