README.md

---
license: apache-2.0
tags:
- merge
- mergekit
- lazymergekit
- bfloat16
- text-generation-inference
- model_stock
- crypto
- finance
- llama
language:
- en
base_model:
- Chainbase-Labs/Theia-Llama-3.1-8B-v1
- EpistemeAI/Fireball-Meta-Llama-3.2-8B-Instruct-agent-003-128k-code-DPO
- mukaj/Llama-3.1-Hawkish-8B
pipeline_tag: text-generation
library_name: transformers
---

# ZeroXClem/LLama3.1-Hawkish-Theia-Fireball-8B

**ZeroXClem/LLama3.1-Hawkish-Theia-Fireball-8B** is an advanced language model meticulously crafted by merging three pre-trained models using the powerful [mergekit](https://github.com/cg123/mergekit) framework. This fusion leverages the **Model Stock** merge method to combine the specialized capabilities of **Theia-Llama**, **Fireball-Meta-Llama**, and **Llama-Hawkish**. The resulting model excels in creative text generation, technical instruction following, financial reasoning, and dynamic conversational interactions.

## 🚀 Merged Models

This model merge incorporates the following:

- [**Chainbase-Labs/Theia-Llama-3.1-8B-v1**](https://huggingface.co/Chainbase-Labs/Theia-Llama-3.1-8B-v1): Specializes in cryptocurrency-oriented knowledge, enhancing the model's ability to generate and comprehend crypto-related content with high accuracy and depth.

- [**EpistemeAI/Fireball-Meta-Llama-3.2-8B-Instruct-agent-003-128k-code-DPO**](https://huggingface.co/EpistemeAI/Fireball-Meta-Llama-3.2-8B-Instruct-agent-003-128k-code-DPO): Focuses on instruction-following and coding capabilities, improving the model's performance in understanding and executing user commands, as well as generating executable code snippets.

- [**mukaj/Llama-3.1-Hawkish-8B**](https://huggingface.co/mukaj/Llama-3.1-Hawkish-8B): Enhances financial reasoning and mathematical precision, enabling the model to handle complex financial analyses, economic discussions, and quantitative problem-solving with high proficiency.

## 🧩 Merge Configuration

The configuration below outlines how the models are merged using the **Model Stock** method. This approach ensures a balanced and effective integration of the unique strengths from each source model.

```yaml
# Merge configuration for ZeroXClem/LLama3.1-Hawkish-Theia-Fireball-8B using Model Stock

models:
  - model: Chainbase-Labs/Theia-Llama-3.1-8B-v1
  - model: EpistemeAI/Fireball-Meta-Llama-3.2-8B-Instruct-agent-003-128k-code-DPO
  - model: mukaj/Llama-3.1-Hawkish-8B
merge_method: model_stock
base_model: mukaj/Llama-3.1-Hawkish-8B
normalize: false
int8_mask: true
dtype: bfloat16
```

### Key Parameters

- **Merge Method (`merge_method`):** Utilizes the **Model Stock** method, as described in [Model Stock](https://arxiv.org/abs/2403.19522), to effectively combine multiple models by leveraging their strengths.

- **Models (`models`):** Specifies the list of models to be merged:
  - **Chainbase-Labs/Theia-Llama-3.1-8B-v1:** Enhances cryptocurrency-oriented knowledge and content generation.
  - **EpistemeAI/Fireball-Meta-Llama-3.2-8B-Instruct-agent-003-128k-code-DPO:** Improves instruction-following and coding capabilities.
  - **mukaj/Llama-3.1-Hawkish-8B:** Enhances financial reasoning and mathematical precision.

- **Base Model (`base_model`):** Defines the foundational model for the merge, which is **mukaj/Llama-3.1-Hawkish-8B** in this case.

- **Normalization (`normalize`):** Set to `false` to retain the original scaling of the model weights during the merge.

- **INT8 Mask (`int8_mask`):** Enabled (`true`) to apply INT8 quantization masking, optimizing the model for efficient inference without significant loss in precision.

- **Data Type (`dtype`):** Uses `bfloat16` to maintain computational efficiency while ensuring high precision.

## 🏆 Performance Highlights

- **Cryptocurrency Knowledge:** Enhanced ability to generate and comprehend crypto-related content, making the model highly effective for blockchain discussions, crypto market analysis, and related queries.

- **Instruction Following and Coding:** Improved performance in understanding and executing user instructions, as well as generating accurate and executable code snippets, suitable for coding assistance and technical support.

- **Financial Reasoning and Mathematical Precision:** Advanced capabilities in handling complex financial analyses, economic discussions, and quantitative problem-solving, making the model ideal for financial modeling, investment analysis, and educational purposes.

- **Smooth Weight Blending:** Utilization of the Model Stock method ensures a harmonious integration of different model attributes, resulting in balanced performance across various specialized tasks.

- **Optimized Inference:** INT8 masking and `bfloat16` data type contribute to efficient computation, enabling faster response times without compromising quality.

## 🎯 Use Case & Applications

**ZeroXClem/LLama3.1-Hawkish-Theia-Fireball-8B** is designed to excel in environments that demand a combination of creative generation, technical instruction following, financial reasoning, and dynamic conversational interactions. Ideal applications include:

- **Cryptocurrency Analysis and Reporting:** Generating detailed reports, analyses, and summaries related to blockchain projects, crypto markets, and financial technologies.

- **Coding Assistance and Technical Support:** Providing accurate and executable code snippets, debugging assistance, and technical explanations for developers and technical professionals.

- **Financial Modeling and Investment Analysis:** Assisting financial analysts and investors in creating models, performing economic analyses, and making informed investment decisions through precise calculations and reasoning.

- **Educational Tools and Tutoring Systems:** Offering detailed explanations, answering complex questions, and assisting in educational content creation across subjects like finance, economics, and mathematics.

- **Interactive Conversational Agents:** Powering chatbots and virtual assistants with specialized knowledge in cryptocurrency, finance, and technical domains, enhancing user interactions and support.

- **Content Generation for Finance and Tech Blogs:** Creating high-quality, contextually relevant content for blogs, articles, and marketing materials focused on finance, technology, and cryptocurrency.

## 📝 Usage

To utilize **ZeroXClem/LLama3.1-Hawkish-Theia-Fireball-8B**, follow the steps below:

### Installation

First, install the necessary libraries:

```bash
pip install -qU transformers accelerate
```

### Example Code

Below is an example of how to load and use the model for text generation:

```python
from transformers import AutoTokenizer, AutoModelForCausalLM, pipeline
import torch

# Define the model name
model_name = "ZeroXClem/LLama3.1-Hawkish-Theia-Fireball-8B"

# Load the tokenizer
tokenizer = AutoTokenizer.from_pretrained(model_name)

# Load the model
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

# Initialize the pipeline
text_generator = pipeline(
    "text-generation",
    model=model,
    tokenizer=tokenizer,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

# Define the input prompt
prompt = "Explain the impact of decentralized finance on traditional banking systems."

# Generate the output
outputs = text_generator(
    prompt,
    max_new_tokens=150,
    do_sample=True,
    temperature=0.7,
    top_k=50,
    top_p=0.95
)

# Print the generated text
print(outputs[0]["generated_text"])
```

### Notes

- **Fine-Tuning:** This merged model may require fine-tuning to optimize performance for specific applications or domains, especially in highly specialized fields like cryptocurrency and finance.

- **Resource Requirements:** Ensure that your environment has sufficient computational resources, especially GPU-enabled hardware, to handle the model efficiently during inference.

- **Customization:** Users can adjust parameters such as `temperature`, `top_k`, and `top_p` to control the creativity and diversity of the generated text, tailoring the model's output to specific needs.


## 📜 License

This model is open-sourced under the **Apache-2.0 License**.

## 💡 Tags

- `merge`
- `mergekit`
- `model_stock`
- `Llama`
- `Hawkish`
- `Theia`
- `Fireball`
- `ZeroXClem/LLama3.1-Hawkish-Theia-Fireball-8B`
- `Chainbase-Labs/Theia-Llama-3.1-8B-v1`
- `EpistemeAI/Fireball-Meta-Llama-3.2-8B-Instruct-agent-003-128k-code-DPO`
- `mukaj/Llama-3.1-Hawkish-8B`