Update README.md
Browse files
README.md
CHANGED
@@ -3,7 +3,9 @@ tags:
|
|
3 |
- llama
|
4 |
- merge
|
5 |
---
|
6 |
-
|
|
|
|
|
7 |
Merged tensor: model.layers.0.self_attn.q_proj.weight with similarity 0.9727
|
8 |
Merged tensor: model.layers.0.self_attn.k_proj.weight with similarity 0.9961
|
9 |
Merged tensor: model.layers.0.self_attn.v_proj.weight with similarity 0.9937
|
@@ -291,4 +293,5 @@ Merged tensor: model.layers.31.mlp.gate_proj.weight with similarity 0.8525
|
|
291 |
Merged tensor: model.layers.31.mlp.up_proj.weight with similarity 0.8320
|
292 |
Merged tensor: model.layers.31.mlp.down_proj.weight with similarity 0.8535
|
293 |
Merged tensor: model.layers.31.input_layernorm.weight with similarity 1.0000
|
294 |
-
Merged tensor: model.layers.31.post_attention_layernorm.weight with similarity 1.0000
|
|
|
|
3 |
- llama
|
4 |
- merge
|
5 |
---
|
6 |
+
|
7 |
+
```
|
8 |
+
Non-Linear Cosine Similarity Merge z_value: 0.0001 this will take a while...
|
9 |
Merged tensor: model.layers.0.self_attn.q_proj.weight with similarity 0.9727
|
10 |
Merged tensor: model.layers.0.self_attn.k_proj.weight with similarity 0.9961
|
11 |
Merged tensor: model.layers.0.self_attn.v_proj.weight with similarity 0.9937
|
|
|
293 |
Merged tensor: model.layers.31.mlp.up_proj.weight with similarity 0.8320
|
294 |
Merged tensor: model.layers.31.mlp.down_proj.weight with similarity 0.8535
|
295 |
Merged tensor: model.layers.31.input_layernorm.weight with similarity 1.0000
|
296 |
+
Merged tensor: model.layers.31.post_attention_layernorm.weight with similarity 1.0000
|
297 |
+
```
|