File size: 600 Bytes
66441f9 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 |
---
datasets:
- Yukang/LongAlpaca-16k-length
language:
- en
pipeline_tag: text-generation
tags:
- facebook
- meta
- llama
- llama-3
- GGUF
license: other
license_name: llama3
license_link: LICENSE
---
# GGUFs of Llama-3-8B-16K
GGUF conversion and quantization of https://huggingface.co/mattshumer/Llama-3-8B-16K
Done with Maxime Labonne's AutoGGUF
## Orginal model card
This is an extended (16K) context version of LLaMA 3 8B (base, not instruct). Trained for five hours on 8x A6000 GPUs, using the `Yukang/LongAlpaca-16k-length` dataset.
`rope_theta` was set to `1000000.0`. Trained with Axolotl. |