Hugging Face's logo

akumaburn
/
Open_Orca_Llama-3-8B-1K

Transformers
Safetensors
GGUF
English
llama
text-generation-inference
unsloth
trl
sft
Inference Endpoints
Model card Files Community
Open_Orca_Llama-3-8B-1K / README.md
akumaburn's picture
akumaburn
Update README.md
502b52a verified
preview code
|
raw
history blame
1.01 kB
metadata
language:
  - en
license: apache-2.0
tags:
  - text-generation-inference
  - transformers
  - unsloth
  - llama
  - trl
  - sft
base_model: unsloth/llama-3-8b-bnb-4bit

Open Orca Llama 3 8B

  • Fine Tuned using dataset: https://huggingface.co/datasets/Open-Orca/OpenOrca

  • Step Count: 1000

  • Batch Size: 2

  • Gradient Accumulation Steps: 4

  • Context Size: 8192

  • Num examples: 4,233,923

  • Trainable Parameters: 41,943,040

  • Learning Rate: 0.0625

  • Training Loss: 1.090800

  • Fined Tuned using: Google Colab Pro (Nvidia L4 runtime)

  • Developed by: akumaburn

  • License: apache-2.0

  • Finetuned from model : unsloth/llama-3-8b-bnb-4bit

Some GGUF quantizations are included as well.

This llama model was trained 2x faster with Unsloth and Huggingface's TRL library.