---
license: apache-2.0
tags:
- moe
- frankenmoe
- merge
- mergekit
- lazymergekit
- aloobun/Cypher-Mini-1.8B
- aloobun/Cypher-CoT-1.8B
base_model:
- aloobun/Cypher-Mini-1.8B
- aloobun/Cypher-CoT-1.8B
model-index:
- name: Cypher-Mixtral-2x1.8B-v0.1
  results:
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: AI2 Reasoning Challenge (25-Shot)
      type: ai2_arc
      config: ARC-Challenge
      split: test
      args:
        num_few_shot: 25
    metrics:
    - type: acc_norm
      value: 40.44
      name: normalized accuracy
    source:
      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=aloobun/Cypher-Mixtral-2x1.8B-v0.1
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: HellaSwag (10-Shot)
      type: hellaswag
      split: validation
      args:
        num_few_shot: 10
    metrics:
    - type: acc_norm
      value: 67.7
      name: normalized accuracy
    source:
      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=aloobun/Cypher-Mixtral-2x1.8B-v0.1
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: MMLU (5-Shot)
      type: cais/mmlu
      config: all
      split: test
      args:
        num_few_shot: 5
    metrics:
    - type: acc
      value: 31.81
      name: accuracy
    source:
      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=aloobun/Cypher-Mixtral-2x1.8B-v0.1
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: TruthfulQA (0-shot)
      type: truthful_qa
      config: multiple_choice
      split: validation
      args:
        num_few_shot: 0
    metrics:
    - type: mc2
      value: 39.94
    source:
      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=aloobun/Cypher-Mixtral-2x1.8B-v0.1
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: Winogrande (5-shot)
      type: winogrande
      config: winogrande_xl
      split: validation
      args:
        num_few_shot: 5
    metrics:
    - type: acc
      value: 65.35
      name: accuracy
    source:
      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=aloobun/Cypher-Mixtral-2x1.8B-v0.1
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: GSM8k (5-shot)
      type: gsm8k
      config: main
      split: test
      args:
        num_few_shot: 5
    metrics:
    - type: acc
      value: 10.46
      name: accuracy
    source:
      url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=aloobun/Cypher-Mixtral-2x1.8B-v0.1
      name: Open LLM Leaderboard
---

# Cypher-Laser-Mixtral-2x1.8B-v0.1

Cypher-Laser-Mixtral-2x1.8B-v0.1 is a Mixure of Experts (MoE) made with the following models using [LazyMergekit](https://colab.research.google.com/drive/1obulZ1ROXHjYLn6PPZJwRR6GzgQogxxb?usp=sharing):
* [aloobun/Cypher-Mini-1.8B](https://huggingface.co/aloobun/Cypher-Mini-1.8B)
* [aloobun/Cypher-CoT-1.8B](https://huggingface.co/aloobun/Cypher-CoT-1.8B)

## 🧩 Configuration

```yaml
base_model: aloobun/Cypher-Mini-1.8B
gate_mode: hidden
dtype: bfloat16
experts:
  - source_model: aloobun/Cypher-Mini-1.8B
    positive_prompts:
      - "Write a Python script that sorts a list of integers using the bubble sort algorithm."
      - "Write a JavaScript function that redirects a web page to another page after 5 seconds."
      - "Describe the steps to troubleshoot a fluid dynamics issue with a water fountain."
      - "Write a short story about a knight's quest to find a lost treasure, and then summarize it in one paragraph."
      - "Summarize the following article with details and clarity."
      - "Tell me about your favorite book and why you like it."
      
  - source_model: aloobun/Cypher-CoT-1.8B
    positive_prompts:
      - "Liam saw an animal running on the farm. Q: Is it true that The animal could be a horse."
      - "Based on the following paragraph can we conclude that the sentence below is true?"
      - "According to the article, how do dolphins communicate with each other?"
      - "Solve this math problem Solve 7644 = 4648*d - 4557*d for d."
      - "If we have 3 marbles, and two roll under the counter, and one is found, how many marbles are there?"
      - "What is the result of 25 divided by 5?"
      - "Is it morally justifiable to lie to protect someone's feelings?"
      - "Determine if the sentence is true based on the text below. Choose from options."
      - "What might a person do if they forget their umbrella on a rainy day?"
      - "Which of the following is an example of renewable energy: a) Coal, b) Solar, c) Oil, d) Natural gas?"
      - "What is the capital of Canada? a) Toronto, b) Ottawa, c) Montreal, d) Vancouver."
      - "Which of these animals is a mammal? a) Snake, b) Dolphin, c) Turtle, d) Frog."
      - "Given a story, answer the question about the story."
      - "Given a prompt and four completions, select the completion that is the most plausible in continuing or answering the prompt."
```

## Format:
```
<|system|></s><|prompt|></s><|answer|>
```
# [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_aloobun__Cypher-Mixtral-2x1.8B-v0.1)

|             Metric              |Value|
|---------------------------------|----:|
|Avg.                             |42.62|
|AI2 Reasoning Challenge (25-Shot)|40.44|
|HellaSwag (10-Shot)              |67.70|
|MMLU (5-Shot)                    |31.81|
|TruthfulQA (0-shot)              |39.94|
|Winogrande (5-shot)              |65.35|
|GSM8k (5-shot)                   |10.46|