ZeroXClem commited on
Commit
23992e1
·
verified ·
1 Parent(s): 71d7a4f

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +48 -1
README.md CHANGED
@@ -50,13 +50,60 @@ parameters:
50
  dtype: bfloat16
51
  ```
52
 
53
- ### Key Parameters
54
 
55
  - **Self-Attention Filtering** (`self_attn`): Controls the blending extent across self-attention layers, allowing for a dynamic mix between the two source models.
56
  - **MLP Filtering** (`mlp`): Adjusts the balance within the Multi-Layer Perceptrons, fine-tuning the model’s neural network layers for optimal performance.
57
  - **Global Weight (`t.value`)**: Sets a general interpolation factor for all unspecified layers, ensuring an equal contribution from both models.
58
  - **Data Type (`dtype`)**: Utilizes `bfloat16` to maintain computational efficiency while preserving high precision.
59
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
60
  ## 🎯 Use Case & Applications
61
 
62
  **Qwen-2.5-Aether-SlerpFusion-7B** excels in scenarios that require both robust language understanding and specialized task performance. This merged model is ideal for:
 
50
  dtype: bfloat16
51
  ```
52
 
53
+ ### 🔑 Key Parameters
54
 
55
  - **Self-Attention Filtering** (`self_attn`): Controls the blending extent across self-attention layers, allowing for a dynamic mix between the two source models.
56
  - **MLP Filtering** (`mlp`): Adjusts the balance within the Multi-Layer Perceptrons, fine-tuning the model’s neural network layers for optimal performance.
57
  - **Global Weight (`t.value`)**: Sets a general interpolation factor for all unspecified layers, ensuring an equal contribution from both models.
58
  - **Data Type (`dtype`)**: Utilizes `bfloat16` to maintain computational efficiency while preserving high precision.
59
 
60
+ ### 🗣️ Inference
61
+
62
+ Below is an example of how to load and use the model for text generation:
63
+
64
+ ```python
65
+ from transformers import AutoTokenizer, AutoModelForCausalLM, pipeline
66
+ import torch
67
+
68
+ # Define the model name
69
+ model_name = "ZeroXClem/Qwen-2.5-Aether-SlerpFusion-7B"
70
+
71
+ # Load the tokenizer
72
+ tokenizer = AutoTokenizer.from_pretrained(model_name)
73
+
74
+ # Load the model
75
+ model = AutoModelForCausalLM.from_pretrained(
76
+ model_name,
77
+ torch_dtype=torch.bfloat16,
78
+ device_map="auto"
79
+ )
80
+
81
+ # Initialize the pipeline
82
+ text_generator = pipeline(
83
+ "text-generation",
84
+ model=model,
85
+ tokenizer=tokenizer,
86
+ torch_dtype=torch.bfloat16,
87
+ device_map="auto"
88
+ )
89
+
90
+ # Define the input prompt
91
+ prompt = "Explain the significance of artificial intelligence in modern healthcare."
92
+
93
+ # Generate the output
94
+ outputs = text_generator(
95
+ prompt,
96
+ max_new_tokens=150,
97
+ do_sample=True,
98
+ temperature=0.7,
99
+ top_k=50,
100
+ top_p=0.95
101
+ )
102
+
103
+ # Print the generated text
104
+ print(outputs[0]["generated_text"])
105
+ ```
106
+
107
  ## 🎯 Use Case & Applications
108
 
109
  **Qwen-2.5-Aether-SlerpFusion-7B** excels in scenarios that require both robust language understanding and specialized task performance. This merged model is ideal for: