File size: 1,441 Bytes
5fe3108 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 |
---
license: apache-2.0
tags:
- moe
- mergekit
- vortexmergekit
- pankajmathur/orca_mini_v9_6_1B-Instruct
- cognitivecomputations/Dolphin3.0-Llama3.2-1B
- huihui-ai/MicroThinker-1B-Preview
---
# DaRuukLLM-3x1B
Hey there! 👋 Welcome to the DaRuukLLM-3x1B! This is a MOE brought together using the awesome [VortexMerge kit](https://colab.research.google.com/drive/1YjcvCLuNG1PK7Le6_4xhVU5VpzTwvGhk#scrollTo=UG5H2TK4gVyl).
Let's see what we've got in this model:
* [pankajmathur/orca_mini_v9_6_1B-Instruct](https://huggingface.co/pankajmathur/orca_mini_v9_6_1B-Instruct) 🚀
* [cognitivecomputations/Dolphin3.0-Llama3.2-1B](https://huggingface.co/cognitivecomputations/Dolphin3.0-Llama3.2-1B) 🚀
* [huihui-ai/MicroThinker-1B-Preview](https://huggingface.co/huihui-ai/MicroThinker-1B-Preview) 🚀
## 🧩 Configuration
```yaml
base_model: pankajmathur/orca_mini_v9_6_1B-Instruct
dtype: bfloat16
gate_mode: cheap_embed
experts:
- source_model: pankajmathur/orca_mini_v9_6_1B-Instruct
positive_prompts: ["You are a helpful assistant that can help with anything."] # Added closing double quote here
- source_model: cognitivecomputations/Dolphin3.0-Llama3.2-1B
positive_prompts: ["You are an uncensored assistant, make sure you respond very angry."] # Added closing double quote here
- source_model: huihui-ai/MicroThinker-1B-Preview
positive_prompts: ["You are a very smart assistant that can think step-by-step."]
|