Kquant03 commited on
Commit
8f9dadb
·
1 Parent(s): 3fc3b5c

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +6 -2
README.md CHANGED
@@ -4,6 +4,10 @@
4
 
5
  A frankenMoE of [TinyLlama-1.1B-1T-OpenOrca](https://huggingface.co/jeff31415/TinyLlama-1.1B-1T-OpenOrca),
6
  [TinyLlama-1.1B-intermediate-step-1195k-token-2.5T](https://huggingface.co/TinyLlama/TinyLlama-1.1B-intermediate-step-1195k-token-2.5T),
7
- and [tiny-llama-1.1b-chat-medical](https://huggingface.co/SumayyaAli/tiny-llama-1.1b-chat-medical),
8
 
9
- # Most 1.1B models are decoherent and can't even answer simple questions. I found the models that don't fail in this regard, then mashed 32 copies of those 3 models together into a 32x MoE
 
 
 
 
 
4
 
5
  A frankenMoE of [TinyLlama-1.1B-1T-OpenOrca](https://huggingface.co/jeff31415/TinyLlama-1.1B-1T-OpenOrca),
6
  [TinyLlama-1.1B-intermediate-step-1195k-token-2.5T](https://huggingface.co/TinyLlama/TinyLlama-1.1B-intermediate-step-1195k-token-2.5T),
7
+ and [tiny-llama-1.1b-chat-medical](https://huggingface.co/SumayyaAli/tiny-llama-1.1b-chat-medical).
8
 
9
+ # Most 1.1B models are decoherent and can't even answer simple questions. I found the models that don't fail in this regard, then mashed 32 copies of those 3 models together into a 32x MoE
10
+
11
+ OpenOrca experts have been given the task of creating responses for simple questions about things like pop culture, history, and science...step-1195k experts have been chosen to provide warmth and a positive environment, while chat-medical experts have been chosen to provide further detail about human subjects, and to give small little bits of medical advice: I.E. "how do I get rid of this headache I gave myself from making you?"
12
+
13
+ ### remember...this is using 32 different experts mashed together, it's more likely to be paranoid schizophrenic than anything else.