Update README.md
Browse files
README.md
CHANGED
@@ -131,13 +131,13 @@ This example demonstrates how to load the model and tokenizer, prepare input, ge
|
|
131 |
|
132 |
## Training Details
|
133 |
|
134 |
-
### Training Data
|
135 |
|
136 |
<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
|
137 |
|
138 |
Teuken-7B-base-v0.4 was pre-trained on 4 trillion tokens of data from publicly available sources.
|
139 |
-
|
140 |
The pretraining data has a cutoff of September 2023.
|
|
|
141 |
|
142 |
For composing the final instruction-tuning dataset termed "Honey", we first include all German examples. We aim to include roughly the same amount of English examples, as we have German examples:
|
143 |
1. Add all multi-turn examples
|
|
|
131 |
|
132 |
## Training Details
|
133 |
|
134 |
+
### Pre-Training Data
|
135 |
|
136 |
<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
|
137 |
|
138 |
Teuken-7B-base-v0.4 was pre-trained on 4 trillion tokens of data from publicly available sources.
|
|
|
139 |
The pretraining data has a cutoff of September 2023.
|
140 |
+
More information are available in our [preprint](http://arxiv.org/abs/2410.08800).
|
141 |
|
142 |
For composing the final instruction-tuning dataset termed "Honey", we first include all German examples. We aim to include roughly the same amount of English examples, as we have German examples:
|
143 |
1. Add all multi-turn examples
|