Commit
·
d15205c
1
Parent(s):
f97aa97
Update README.md
Browse files
README.md
CHANGED
|
@@ -45,7 +45,9 @@ This official repo introduces the TransNormerLLM model, featuring its open-sourc
|
|
| 45 |
- **TransNormerLLM1** incorporates Simple GLU in its channel mixer, GLA in the token mixer, and SRMSNorm for normalization. In this model, the channel and token mixers function sequentially in a pipeline arrangement.
|
| 46 |
- **TransNormerLLM2** also utilizes Simple GLU in the channel mixer, GLA in the token mixer, and SRMSNorm for normalization. However, in this version, the channel and token mixers operate concurrently, in parallel.
|
| 47 |
|
| 48 |
-
|
|
|
|
|
|
|
| 49 |
|
| 50 |
# Released Weights
|
| 51 |
|
|
@@ -186,4 +188,9 @@ If you wish to cite our work, please use the following reference:
|
|
| 186 |
journal={arXiv preprint arXiv:2307.14995},
|
| 187 |
year={2023}
|
| 188 |
}
|
| 189 |
-
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 45 |
- **TransNormerLLM1** incorporates Simple GLU in its channel mixer, GLA in the token mixer, and SRMSNorm for normalization. In this model, the channel and token mixers function sequentially in a pipeline arrangement.
|
| 46 |
- **TransNormerLLM2** also utilizes Simple GLU in the channel mixer, GLA in the token mixer, and SRMSNorm for normalization. However, in this version, the channel and token mixers operate concurrently, in parallel.
|
| 47 |
|
| 48 |
+
<p align="left">
|
| 49 |
+
<img src="./images/TransNormer1-2Arch.jpg" width="45%" />
|
| 50 |
+
</p>
|
| 51 |
|
| 52 |
# Released Weights
|
| 53 |
|
|
|
|
| 188 |
journal={arXiv preprint arXiv:2307.14995},
|
| 189 |
year={2023}
|
| 190 |
}
|
| 191 |
+
```
|
| 192 |
+
|
| 193 |
+
<p align="center">
|
| 194 |
+
<img src="./images/lightning-leopard.jpg" width="33%" />
|
| 195 |
+
- OpenNLPLab @2024 -
|
| 196 |
+
</p>
|