Update README.md
Browse files
README.md
CHANGED
|
@@ -36,7 +36,7 @@ Good at solving NLU tasks, the largest Chinese BERT (39B) currently.
|
|
| 36 |
|
| 37 |
Erlangshen-MegatronBert-3.9B-Chinese是一个比[Erlangshen-MegatronBert-1.3B](https://huggingface.co/IDEA-CCNL/Erlangshen-MegatronBert-1.3B)拥有更多参数的版本（39亿）。我们遵循原来的预训练方式在悟道数据集（300G版本）上进行预训练。具体地，我们在预训练阶段中使用了封神框架大概花费了64张A100（40G）约30天。
|
| 38 |
|
| 39 |
-
Erlangshen-MegatronBert-3.9B-Chinese (3.9B) is a larger version of [Erlangshen-MegatronBert-1.3B](https://huggingface.co/IDEA-CCNL/Erlangshen-MegatronBert-1.3B). By following the original instructions, we apply WuDao Corpora (300 GB version) as the pretraining dataset. Specifically, we use the [fengshen framework](https://github.com/IDEA-CCNL/Fengshenbang-LM/tree/main/fengshen) in the pre-training phase which cost about 30 days with 64 A100 GPUs.
|
| 40 |
|
| 41 |
### 更多信息 More Information
|
| 42 |
|
|
|
|
| 36 |
|
| 37 |
Erlangshen-MegatronBert-3.9B-Chinese是一个比[Erlangshen-MegatronBert-1.3B](https://huggingface.co/IDEA-CCNL/Erlangshen-MegatronBert-1.3B)拥有更多参数的版本（39亿）。我们遵循原来的预训练方式在悟道数据集（300G版本）上进行预训练。具体地，我们在预训练阶段中使用了封神框架大概花费了64张A100（40G）约30天。
|
| 38 |
|
| 39 |
+
Erlangshen-MegatronBert-3.9B-Chinese (3.9B) is a larger version of [Erlangshen-MegatronBert-1.3B](https://huggingface.co/IDEA-CCNL/Erlangshen-MegatronBert-1.3B). By following the original instructions, we apply WuDao Corpora (300 GB version) as the pretraining dataset. Specifically, we use the [fengshen framework](https://github.com/IDEA-CCNL/Fengshenbang-LM/tree/main/fengshen) in the pre-training phase which cost about 30 days with 64 A100 (40G) GPUs.
|
| 40 |
|
| 41 |
### 更多信息 More Information
|
| 42 |
|