IvanHU commited on
Commit
125cb7c
1 Parent(s): 1920b0e

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +8 -4
README.md CHANGED
@@ -174,7 +174,7 @@ The intermediate stage checkpoints are released in <a href="https://huggingface.
174
 
175
  <details><summary>3. Optimizer States Before Annealing</summary>
176
 
177
- Optimizer states before annealing will be released in a future update.
178
  </details>
179
 
180
 
@@ -213,11 +213,11 @@ Intermediate optimizer states will be released in a future update.
213
 
214
  ### What you can do with these pre-training resources
215
 
216
- 1. **Pre-train** your own LLM. You can use our data and curriculum to train a model that's just as powerful as YuLan-Mini.
217
- 2. Perform your own **learning rate annealing**. During the annealing phase, YuLan-Mini's learning ability is at its peak. You can resume training from the checkpoint before annealing and use your own dataset for learning rate annealing.
218
  3. **Fine-tune** the Instruct version of the LLM. You can use the YuLan-Mini base model to train your own Instruct version.
219
  4. **Training dynamics** research. You can use YuLan-Mini's intermediate checkpoints to explore internal changes during the pre-training process.
220
- 5. **Synthesize** your own data. You can use YuLan-Mini's data pipeline to clean and generate your own dataset.
221
 
222
  ---
223
 
@@ -255,6 +255,10 @@ python -m sglang.launch_server --model-path yulan-team/YuLan-Mini --port 30000 -
255
 
256
  ---
257
 
 
 
 
 
258
  ## License
259
 
260
  - The code in this repository is released under the [MIT License](./LICENSE).
 
174
 
175
  <details><summary>3. Optimizer States Before Annealing</summary>
176
 
177
+ <a href="https://huggingface.co/yulan-team/YuLan-Mini-Before-Annealing">YuLan-Mini-Before-Annealing</a>
178
  </details>
179
 
180
 
 
213
 
214
  ### What you can do with these pre-training resources
215
 
216
+ 1. **Pre-train** your own LLM. You can use [our data](https://huggingface.co/yulan-team/YuLan-Mini-Datasets) and curriculum to train a model that's just as powerful as YuLan-Mini.
217
+ 2. Perform your own **learning rate annealing**. During the annealing phase, YuLan-Mini's learning ability is at its peak. You can resume training from [the checkpoint before annealing](https://huggingface.co/yulan-team/YuLan-Mini-Before-Annealing) and use your own dataset for learning rate annealing.
218
  3. **Fine-tune** the Instruct version of the LLM. You can use the YuLan-Mini base model to train your own Instruct version.
219
  4. **Training dynamics** research. You can use YuLan-Mini's intermediate checkpoints to explore internal changes during the pre-training process.
220
+ 5. **Synthesize** your own data. You can use YuLan-Mini's [data pipeline](https://github.com/RUC-GSAI/YuLan-Mini) to clean and generate your own dataset.
221
 
222
  ---
223
 
 
255
 
256
  ---
257
 
258
+ ## The Team
259
+
260
+ YuLan-Mini is developed and maintained by [AI Box, Renmin University of China](http://aibox.ruc.edu.cn/).
261
+
262
  ## License
263
 
264
  - The code in this repository is released under the [MIT License](./LICENSE).