taidopurason
commited on
Added paper and citation info
Browse files
README.md
CHANGED
@@ -6,4 +6,20 @@ pipeline_tag: text-generation
|
|
6 |
---
|
7 |
# LLammas-base 🐑
|
8 |
|
9 |
-
Llama-2-7B with continued pre-training of 5B tokens of CulturaX (
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
6 |
---
|
7 |
# LLammas-base 🐑
|
8 |
|
9 |
+
Llama-2-7B with continued pre-training of 5B tokens of CulturaX (75% Estonian, 25% English documents).
|
10 |
+
|
11 |
+
This model is also instruction-tuned resulting in [Llammas](https://huggingface.co/tartuNLP/Llammas).
|
12 |
+
|
13 |
+
More details in our [paper](https://arxiv.org/abs/2404.04042).
|
14 |
+
|
15 |
+
### Citation
|
16 |
+
```
|
17 |
+
@misc{kuulmets2024teaching,
|
18 |
+
title={Teaching Llama a New Language Through Cross-Lingual Knowledge Transfer},
|
19 |
+
author={Hele-Andra Kuulmets and Taido Purason and Agnes Luhtaru and Mark Fishel},
|
20 |
+
year={2024},
|
21 |
+
eprint={2404.04042},
|
22 |
+
archivePrefix={arXiv},
|
23 |
+
primaryClass={cs.CL}
|
24 |
+
}
|
25 |
+
```
|