| license: apache-2.0 | |
| tags: | |
| - moe | |
| - mergekit | |
| - vortexmergekit | |
| - pankajmathur/orca_mini_v9_6_1B-Instruct | |
| - cognitivecomputations/Dolphin3.0-Llama3.2-1B | |
| - huihui-ai/MicroThinker-1B-Preview | |
| - autoquant | |
| - gguf | |
| # DaRuukLLM-3x1B | |
| Hey there! 👋 Welcome to the DaRuukLLM-3x1B! This is a MOE brought together using the awesome [VortexMerge kit](https://colab.research.google.com/drive/1YjcvCLuNG1PK7Le6_4xhVU5VpzTwvGhk#scrollTo=UG5H2TK4gVyl). | |
| Let's see what we've got in this model: | |
| * [pankajmathur/orca_mini_v9_6_1B-Instruct](https://huggingface.co/pankajmathur/orca_mini_v9_6_1B-Instruct) 🚀 | |
| * [cognitivecomputations/Dolphin3.0-Llama3.2-1B](https://huggingface.co/cognitivecomputations/Dolphin3.0-Llama3.2-1B) 🚀 | |
| * [huihui-ai/MicroThinker-1B-Preview](https://huggingface.co/huihui-ai/MicroThinker-1B-Preview) 🚀 | |
| ## 🧩 Configuration | |
| ```yaml | |
| base_model: pankajmathur/orca_mini_v9_6_1B-Instruct | |
| dtype: bfloat16 | |
| gate_mode: cheap_embed | |
| experts: | |
| - source_model: pankajmathur/orca_mini_v9_6_1B-Instruct | |
| positive_prompts: ["You are a helpful assistant that can help with anything."] # Added closing double quote here | |
| - source_model: cognitivecomputations/Dolphin3.0-Llama3.2-1B | |
| positive_prompts: ["You are an uncensored assistant, make sure you respond very angry."] # Added closing double quote here | |
| - source_model: huihui-ai/MicroThinker-1B-Preview | |
| positive_prompts: ["You are a very smart assistant that can think step-by-step."] | |