Added paper and citation info
Browse files
README.md
CHANGED
|
@@ -6,4 +6,20 @@ pipeline_tag: text-generation
|
|
| 6 |
---
|
| 7 |
# LLammas-base 🐑
|
| 8 |
|
| 9 |
-
Llama-2-7B with continued pre-training of 5B tokens of CulturaX (
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 6 |
---
|
| 7 |
# LLammas-base 🐑
|
| 8 |
|
| 9 |
+
Llama-2-7B with continued pre-training of 5B tokens of CulturaX (75% Estonian, 25% English documents).
|
| 10 |
+
|
| 11 |
+
This model is also instruction-tuned resulting in [Llammas](https://huggingface.co/tartuNLP/Llammas).
|
| 12 |
+
|
| 13 |
+
More details in our [paper](https://arxiv.org/abs/2404.04042).
|
| 14 |
+
|
| 15 |
+
### Citation
|
| 16 |
+
```
|
| 17 |
+
@misc{kuulmets2024teaching,
|
| 18 |
+
title={Teaching Llama a New Language Through Cross-Lingual Knowledge Transfer},
|
| 19 |
+
author={Hele-Andra Kuulmets and Taido Purason and Agnes Luhtaru and Mark Fishel},
|
| 20 |
+
year={2024},
|
| 21 |
+
eprint={2404.04042},
|
| 22 |
+
archivePrefix={arXiv},
|
| 23 |
+
primaryClass={cs.CL}
|
| 24 |
+
}
|
| 25 |
+
```
|