dumitrescustefan
commited on
Commit
·
4194ec1
1
Parent(s):
8b1fdb1
Update README.md
Browse files
README.md
CHANGED
@@ -10,7 +10,7 @@ This is a pretrained-from-scratch [t5x](https://github.com/google-research/t5x)
|
|
10 |
|
11 |
Training was performed on a clean 80GB Romanian text corpus for 4M steps with these [scripts](https://github.com/dumitrescustefan/t5x_models). The model was trained with an encoder sequence length of 512 and a decoder sequence length of 256.
|
12 |
|
13 |
-
|
14 |
|
15 |
### How to load a t5x model
|
16 |
|
|
|
10 |
|
11 |
Training was performed on a clean 80GB Romanian text corpus for 4M steps with these [scripts](https://github.com/dumitrescustefan/t5x_models). The model was trained with an encoder sequence length of 512 and a decoder sequence length of 256.
|
12 |
|
13 |
+
**!! IMPORTANT !!** This model was pretrained on the span corruption MLM task, meaning this model is **not usable** in any downstream task **without finetuning** first!
|
14 |
|
15 |
### How to load a t5x model
|
16 |
|