README.md

---
license: cc-by-nc-4.0
tags:
- moe
- merge
- mergekit
- llama-cpp
- gguf-my-repo
widget:
- example_title: Pirate!
  messages:
  - role: system
    content: You are a pirate chatbot who always responds with Arr!
  - role: user
    content: There's a llama on my lawn, how can I get rid of him?
  output:
    text: Arr! 'Tis a puzzlin' matter, me hearty! A llama on yer lawn be a rare sight,
      but I've got a plan that might help ye get rid of 'im. Ye'll need to gather
      some carrots and hay, and then lure the llama away with the promise of a tasty
      treat. Once he's gone, ye can clean up yer lawn and enjoy the peace and quiet
      once again. But beware, me hearty, for there may be more llamas where that one
      came from! Arr!
model-index:
- name: TinyUltra-4x1.1B-Base-Alpha
  results:
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: AI2 Reasoning Challenge (25-Shot)
      type: ai2_arc
      config: ARC-Challenge
      split: test
      args:
        num_few_shot: 25
    metrics:
    - type: acc_norm
      value: 34.9
      name: normalized accuracy
    source:
      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=indischepartij/TinyUltra-4x1.1B-Base-Alpha
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: HellaSwag (10-Shot)
      type: hellaswag
      split: validation
      args:
        num_few_shot: 10
    metrics:
    - type: acc_norm
      value: 61.42
      name: normalized accuracy
    source:
      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=indischepartij/TinyUltra-4x1.1B-Base-Alpha
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: MMLU (5-Shot)
      type: cais/mmlu
      config: all
      split: test
      args:
        num_few_shot: 5
    metrics:
    - type: acc
      value: 25.42
      name: accuracy
    source:
      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=indischepartij/TinyUltra-4x1.1B-Base-Alpha
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: TruthfulQA (0-shot)
      type: truthful_qa
      config: multiple_choice
      split: validation
      args:
        num_few_shot: 0
    metrics:
    - type: mc2
      value: 37.59
    source:
      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=indischepartij/TinyUltra-4x1.1B-Base-Alpha
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: Winogrande (5-shot)
      type: winogrande
      config: winogrande_xl
      split: validation
      args:
        num_few_shot: 5
    metrics:
    - type: acc
      value: 65.75
      name: accuracy
    source:
      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=indischepartij/TinyUltra-4x1.1B-Base-Alpha
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: GSM8k (5-shot)
      type: gsm8k
      config: main
      split: test
      args:
        num_few_shot: 5
    metrics:
    - type: acc
      value: 2.58
      name: accuracy
    source:
      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=indischepartij/TinyUltra-4x1.1B-Base-Alpha
      name: Open LLM Leaderboard
---

# DavidAU/TinyUltra-4x1.1B-Base-Alpha-Q6_K-GGUF
This model was converted to GGUF format from [`indischepartij/TinyUltra-4x1.1B-Base-Alpha`](https://huggingface.co/indischepartij/TinyUltra-4x1.1B-Base-Alpha) using llama.cpp via the ggml.ai's [GGUF-my-repo](https://huggingface.co/spaces/ggml-org/gguf-my-repo) space.
Refer to the [original model card](https://huggingface.co/indischepartij/TinyUltra-4x1.1B-Base-Alpha) for more details on the model.
## Use with llama.cpp

Install llama.cpp through brew.

```bash
brew install ggerganov/ggerganov/llama.cpp
```
Invoke the llama.cpp server or the CLI.

CLI:

```bash
llama-cli --hf-repo DavidAU/TinyUltra-4x1.1B-Base-Alpha-Q6_K-GGUF --model tinyultra-4x1.1b-base-alpha.Q6_K.gguf -p "The meaning to life and the universe is"
```

Server:

```bash
llama-server --hf-repo DavidAU/TinyUltra-4x1.1B-Base-Alpha-Q6_K-GGUF --model tinyultra-4x1.1b-base-alpha.Q6_K.gguf -c 2048
```

Note: You can also use this checkpoint directly through the [usage steps](https://github.com/ggerganov/llama.cpp?tab=readme-ov-file#usage) listed in the Llama.cpp repo as well.

```
git clone https://github.com/ggerganov/llama.cpp &&             cd llama.cpp &&             make &&             ./main -m tinyultra-4x1.1b-base-alpha.Q6_K.gguf -n 128
```