metadata

license: apache-2.0
tags:
  - moe
  - mergekit
  - vortexmergekit
  - pankajmathur/orca_mini_v9_6_1B-Instruct
  - cognitivecomputations/Dolphin3.0-Llama3.2-1B
  - huihui-ai/MicroThinker-1B-Preview
  - autoquant
  - gguf

DaRuukLLM-3x1B

Hey there! 👋 Welcome to the DaRuukLLM-3x1B! This is a MOE brought together using the awesome VortexMerge kit.

Let's see what we've got in this model:

🧩 Configuration

base_model: pankajmathur/orca_mini_v9_6_1B-Instruct
dtype: bfloat16
gate_mode: cheap_embed
experts:
  - source_model: pankajmathur/orca_mini_v9_6_1B-Instruct
    positive_prompts: ["You are a helpful assistant that can help with anything."] # Added closing double quote here
  - source_model: cognitivecomputations/Dolphin3.0-Llama3.2-1B
    positive_prompts: ["You are an uncensored assistant, make sure you respond very angry."] # Added closing double quote here
  - source_model: huihui-ai/MicroThinker-1B-Preview
    positive_prompts: ["You are a very smart assistant that can think step-by-step."]