v000000 commited on
Commit
827aaa8
·
verified ·
1 Parent(s): d5c1f65

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +5 -2
README.md CHANGED
@@ -3,7 +3,9 @@ tags:
3
  - llama
4
  - merge
5
  ---
6
- z_value: 0.0001 this will take a while...
 
 
7
  Merged tensor: model.layers.0.self_attn.q_proj.weight with similarity 0.9727
8
  Merged tensor: model.layers.0.self_attn.k_proj.weight with similarity 0.9961
9
  Merged tensor: model.layers.0.self_attn.v_proj.weight with similarity 0.9937
@@ -291,4 +293,5 @@ Merged tensor: model.layers.31.mlp.gate_proj.weight with similarity 0.8525
291
  Merged tensor: model.layers.31.mlp.up_proj.weight with similarity 0.8320
292
  Merged tensor: model.layers.31.mlp.down_proj.weight with similarity 0.8535
293
  Merged tensor: model.layers.31.input_layernorm.weight with similarity 1.0000
294
- Merged tensor: model.layers.31.post_attention_layernorm.weight with similarity 1.0000
 
 
3
  - llama
4
  - merge
5
  ---
6
+
7
+ ```
8
+ Non-Linear Cosine Similarity Merge z_value: 0.0001 this will take a while...
9
  Merged tensor: model.layers.0.self_attn.q_proj.weight with similarity 0.9727
10
  Merged tensor: model.layers.0.self_attn.k_proj.weight with similarity 0.9961
11
  Merged tensor: model.layers.0.self_attn.v_proj.weight with similarity 0.9937
 
293
  Merged tensor: model.layers.31.mlp.up_proj.weight with similarity 0.8320
294
  Merged tensor: model.layers.31.mlp.down_proj.weight with similarity 0.8535
295
  Merged tensor: model.layers.31.input_layernorm.weight with similarity 1.0000
296
+ Merged tensor: model.layers.31.post_attention_layernorm.weight with similarity 1.0000
297
+ ```