Update README.md
Browse files
README.md
CHANGED
@@ -13,9 +13,9 @@ license: cc-by-nc-4.0
|
|
13 |
|
14 |
![image/png](https://cdn-uploads.huggingface.co/production/uploads/65b19c1b098c85365af5a83e/SpNOAI3VKUsWegChuQHTk.png)
|
15 |
|
16 |
-
Badger Writer is a *normalized fourier task addition* of maldv/badger-lambda-llama-3-8b and maldv/llama-3-fantasy-writer-8b.
|
17 |
|
18 |
-
I also used the first and last layer directly from dreamgen-preview/opus-v1.2-llama-3-8b-instruct-run3.5-epoch2.5 due to the obvious advantages. I didn't train either the lm_head or
|
19 |
|
20 |
Rep-pen 1.1 ; Min-p 0.01 ; Temp 0.7 ; Dynatemp 0.4 ; 32k context ; llama 3 instruct template
|
21 |
|
|
|
13 |
|
14 |
![image/png](https://cdn-uploads.huggingface.co/production/uploads/65b19c1b098c85365af5a83e/SpNOAI3VKUsWegChuQHTk.png)
|
15 |
|
16 |
+
Badger Writer is a *normalized fourier task addition* of [maldv/badger-lambda-llama-3-8b](/maldv/badger-lambda-llama-3-8b) and [maldv/llama-3-fantasy-writer-8b](/maldv/llama-3-fantasy-writer-8b).
|
17 |
|
18 |
+
I also used the first and last layer directly from [dreamgen-preview/opus-v1.2-llama-3-8b-instruct-run3.5-epoch2.5](/dreamgen-preview/opus-v1.2-llama-3-8b-instruct-run3.5-epoch2.5) due to the obvious advantages. I didn't train either the lm_head or embed_tokens layers on the fantasy-writer, but opus is part of lambda ; so they all fit nicely together.
|
19 |
|
20 |
Rep-pen 1.1 ; Min-p 0.01 ; Temp 0.7 ; Dynatemp 0.4 ; 32k context ; llama 3 instruct template
|
21 |
|