Text Generation
Transformers
Safetensors
llama
text-generation-inference
Inference Endpoints
mfromm commited on
Commit
4acfc1e
·
verified ·
1 Parent(s): d4e0be4

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -2
README.md CHANGED
@@ -131,13 +131,13 @@ This example demonstrates how to load the model and tokenizer, prepare input, ge
131
 
132
  ## Training Details
133
 
134
- ### Training Data
135
 
136
  <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
137
 
138
  Teuken-7B-base-v0.4 was pre-trained on 4 trillion tokens of data from publicly available sources.
139
-
140
  The pretraining data has a cutoff of September 2023.
 
141
 
142
  For composing the final instruction-tuning dataset termed "Honey", we first include all German examples. We aim to include roughly the same amount of English examples, as we have German examples:
143
  1. Add all multi-turn examples
 
131
 
132
  ## Training Details
133
 
134
+ ### Pre-Training Data
135
 
136
  <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
137
 
138
  Teuken-7B-base-v0.4 was pre-trained on 4 trillion tokens of data from publicly available sources.
 
139
  The pretraining data has a cutoff of September 2023.
140
+ More information are available in our [preprint](http://arxiv.org/abs/2410.08800).
141
 
142
  For composing the final instruction-tuning dataset termed "Honey", we first include all German examples. We aim to include roughly the same amount of English examples, as we have German examples:
143
  1. Add all multi-turn examples