README.md

---
library_name: transformers
language:
- es
base_model:
- Qwen/Qwen2.5-7B-Instruct
pipeline_tag: text-generation
license: other
license_name: rigochat-nc
license_link: https://huggingface.co/IIC/RigoChat-7b-v2/blob/main/LICENSE
tags:
- chat
---

# Model Card for RigoChat-7b-v2

RigoChat-7b-v2 is a Qwen-2.5-based model specifically designed to provide accurate responses from Spanish queries. Specifically, is based on the [Qwen/Qwen2.5-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct) model and has been fine-tuned with Direct Preference Optimization ([DPO](https://arxiv.org/pdf/2305.18290)) for improved performance in Spanish language.

This model is licensed for **non-commercial use**. If you want to use it commercially, please contact us or use it through the service we offer from the [AWS Marketplace](https://aws.amazon.com/marketplace/pp/prodview-3j6o63v6c3pq2?sr=0-2&ref_=beagle&applicationId=AWSMPContessa).You can find a tutorial on how to use it from that infrastructure at the [following link](https://github.com/iiconocimiento/iic-aws/blob/main/notebooks/rigochat-7b/rigochat-7b-Model.ipynb).

## Model Details

### Model Description

This model is the second version of RigoChat, a family of Large Language Models (LLMs) designed to solve typical NLP tasks with Spanish instructions such as: Tool Use, Summarization, Math, Code, Abstractive-QA, etc. Like [Qwen/Qwen2.5-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct), this model has no specific use case and can be applied to a wide range of tasks. Indeed, it offers a slight improvement for generalist tasks in Spanish, particularly in RAG (Retriever Augmented Generation) systems with Spanish databases, as its training focused on resolving questions about contexts to prevent hallucinations and ensure safe responses.

Key benefits of this model include:

- Improved performance on generalist tasks in Spanish.
- Enhanced safety and reduced hallucinations in RAG systems with Spanish texts.
- Possibility of using it in different hardware requirements, especially those with reduced computational capacity. For more information on how to use RigoChat-7b-v2 on reduced hardware, see [IIC/RigoChat-7b-v2-GGUF](https://huggingface.co/IIC/RigoChat-7b-v2-GGUF).

Remarkably, this model was trained on a single A100 GPU with limited computational resources, yet achieved its current state in a relatively short time (8.5 hours). This feat was made possible by leveraging a high-quality dataset and employing advanced techniques such as [LoRA](https://arxiv.org/pdf/2106.09685) to optimize memory usage. Further details on the training process can be found below.

- **Developed by:** Instituto de Ingeniería del Conocimiento (IIC).
- **Model type:** Generative Fine-tuned Transformer.
- **Language(s) (NLP):** Spanish (BCP-47 es).
- **License:** RIGOCHAT NON-COMMERCIAL.
- **Arquitecture:** We use [Qwen's](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct) architecture without modifications.
- **Finetuned from model:** [Qwen/Qwen2.5-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct).

### Model Sources

- **Paper:** **Coming soon**.

## Uses

### Direct Use

You can use and deploy RigoChat-v2 for commercial purposes through a model package from AWS Marketplace. You can check the instructions inside the [following notebook](https://github.com/iiconocimiento/iic-aws/tree/main/notebooks/rigochat-v2).

### Out-of-Scope Use

This language model has been adapted for general natural language processing tasks in Spanish and specific use cases such as RAG. However, there are several cases where the model should not be used due to its technical and ethical limitations:

- **Illegal Activities:** The model should not be used to generate content related to illegal activities, such as creating malicious software, fraud, incitement to crime, or any illegal material.
- **Harmful or Dangerous Content:** It should not be used to generate hate speech, violence, harassment, or any content that promotes discrimination, violence, or abuse.

### Bias, Risks, and Limitations

Although this model has been trained to understand and generate text in Spanish, there are several risks, biases, and limitations that users should be aware of:

- **Biases:** The model may reflect biases present in the training data. These biases could be related to gender, race, social class, sexual orientation, among others, and may generate responses that perpetuate stereotypes or discrimination.
- **Accuracy and Reliability:** While the model generates coherent and useful text in many contexts, it may not always be 100% accurate or reliable, especially in technical, scientific, or legal matters where high certainty is required.
- **Limited or Outdated Knowledge:** The model is not trained with information beyond its training cutoff date. Therefore, it may not reflect recent events, research, or advancements.

### Recommendations

We recommend using this model as a general chatbot or within applications designed for specific tasks, such as SQL queries, RAG systems, or as an autonomous agent to facilitate the use of tools.

## How to Get Started with the Model

As [Qwen/Qwen2.5-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct) or any model that uses the `chatml` template.

### To load the model and tokenizer

```python
from transformers import (
    AutoModelForCausalLM,
    AutoTokenizer,
)
import torch

model_name = "IIC/RigoChat-7b-v2"

model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype=torch.bfloat16,
    device_map="cuda",
    trust_remote_code=True,
)
tokenizer = AutoTokenizer.from_pretrained(
    model_name,
    trust_remote_code=True,
)
```

### Sample generation

```python
messages = [
    {"role": "user", "content": "¿Cómo puedo transformar un diccionario de listas en una lista de diccionarios, y viceversa, en Python sin utilizar bucles for?"}
]

text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)
model_inputs = tokenizer([text], return_tensors="pt").to(model.device)

generated_ids = model.generate(
    **model_inputs,
    max_new_tokens=1024,
)
generated_ids = [
    output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
]

response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
```

For a better experience, we recommend to use [the following generation parameters](https://huggingface.co/IIC/RigoChat-7b-v2/blob/main/generation_config.json).

### Tool Use

```python
def get_current_weather(location: str, date: str) -> float:
    """
    Obtener datos del tiempo de una localización.
    
    Args:
        location: La locaización, con el siguiente formato: "Ciudad, País."
        date: La fecha, en el formato AAAA-MM-DD.
    Returns:
        El tiempo en dicha localización.
    """
    return {"temperatura": 22, "cielo": "nublado", "probabilidad de lluvias": "60%"}

messages = [
  {"role": "user", "content": "Este fin de semana quiero visitar Madrid, y no se qué ropa llevarme. ¿Podrías decirme qué tal va a hacer? Es el puente del 6 de diciembre de 2024."}
]

text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    tools=[get_current_weather],
    add_generation_prompt=True
)
model_inputs = tokenizer([text], return_tensors="pt").to(model.device)

generated_ids = model.generate(
    **model_inputs,
    max_new_tokens=1024
)
generated_ids = [
    output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
]

response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
```

Check the [tool use documentation](https://huggingface.co/docs/transformers/main/chat_templating#advanced-tool-use--function-calling) from HuggingFace for more information.

## Training Details

### Training Data

A combination of both public and private datasets designed in the IIC. The dataset consists of 21975 conversations in Spanish, with the format `chatml` and has the same structure as the [Anthropic/hh-rlhf dataset](https://huggingface.co/datasets/Anthropic/hh-rlhf). Each conversation has two variants: `chosen` and `rejected`, and only differs the last answer of the assistant. The last answer in the `chosen` variant is considered a better answer than the one in the `rejected` variant. Different techniques have been used to generate the dataset, which we explain in depth in the research (**coming soon**).

### Training Procedure

We use the [Transformer Reinforcement Learning](https://huggingface.co/docs/trl/index) (TRL) library. Specifically, we have applied [the script they have published](https://github.com/huggingface/trl/blob/main/examples/scripts/dpo.py) as an example for using DPO to the dataset we have generated.

#### Training Hyperparameters


<details><summary>Details</summary>
<p>
  
```python
LORA_CONFIG = {
    "r": 64,
    "lora_alpha": 16,
    "lora_dropout": 0.1,
    "bias": "none",
    "task_type": "CAUSAL_LM",
    "target_modules": [
        "q_proj",
        "k_proj",
        "v_proj",
        "o_proj",
        "up_proj",
        "gate_proj",
        "down_proj",
    ],
    "use_rslora": True,
}

DPO_CONFIG = {
    "num_train_epochs": 2,
    "logging_steps": 25,
    "eval_steps": 500,
    "save_steps": 100,
    "save_total_limit": 5,
    "per_device_train_batch_size": 1,
    "per_device_eval_batch_size": 1,
    "gradient_accumulation_steps": 16,
    "learning_rate": 5e-6,
    "max_length": 8192, # max length in the history chat + latest assistant response.
    "max_prompt_length": 6656, # max length in the history chat: user-assistant-...-assistant-user.
    "gradient_checkpointing": True,
    "weight_decay": 0.001,
    "optim": "rmsprop",
    "evaluation_strategy": "steps",
    "lr_scheduler_type": "cosine",
    "bf16": True,
}
```

</p>
</details>

#### Speeds, Sizes, Times

Below are some useful parameters showing the results of the latest training logs.

```python
latest_logs = {'loss': 0.3716, 'grad_norm': 4.989994049072266, 'learning_rate': 1.0380020311950844e-10, 'rewards/chosen': 0.534086287021637, 'rewards/rejected': -0.6236276030540466, 'rewards/accuracies': 0.8899999856948853, 'rewards/margins': 1.1577140092849731, 'logps/rejected': -218.88198852539062, 'logps/chosen': -250.0700225830078, 'logits/rejected': -1.6214849948883057, 'logits/chosen': -1.9585875272750854, 'epoch': 1.99}

final_training_results = {'train_runtime': 30825.7138, 'train_samples_per_second': 1.432, 'train_steps_per_second': 0.089, 'train_loss': 0.483570138469306, 'epoch': 2.0}
```

As can be seen in the time used, in eight and a half hours we have managed to improve a state-of-the-art model, with very little hardware, in tasks adapted to Spanish. This can be seen in more detail in the following sections.

## Evaluation

### Testing Data, Factors & Metrics

#### Testing Data


To assess the performance of Large Language Models (LLMs), we have developed and utilized several high-quality corpora tailored to specific evaluation needs:

1. [IIC/AQuAS](https://huggingface.co/datasets/IIC/AQuAS): A manually curated corpus created by two computational linguists to evaluate language models in the task of Abstractive Question Answering in Spanish. It includes examples from domains such as finance, insurance, healthcare, law, and music.

2. [IIC/RagQuAS](https://huggingface.co/datasets/IIC/RagQuAS). Another manually curated corpus developed by the same linguists to evaluate full RAG systems and language models in Abstractive Question Answering tasks in Spanish. This corpus spans a wide range of domains, including hobbies, linguistics, pets, health, astronomy, customer service, cars, daily life, documentation, energy, skiing, fraud, gastronomy, languages, games, nail care, music, skating, first aid, recipes, recycling, complaints, insurance, tennis, transportation, tourism, veterinary, travel, and yoga.

3. **CAM:** Designed for all CAM tasks, this corpus consists of frequently asked questions (FAQs) sourced from consumer-related topics on the websites of the Comunidad de Madrid. The questions are categorized into three levels of degradation—E1, E2, and E3—intended to measure the LLMs’ ability to understand and effectively respond to poorly formulated queries caused by spelling errors, varying levels of colloquialism, and similar issues. This task also falls under the Abstractive Question Answering category.

4. **Shops:** A multi-turn conversational corpus centered on policies from various clothing companies. The task involves Multi-turn Abstractive Question Answering.

5. **Insurance:** Another multi-turn conversational corpus, this one focuses on policies from various insurance companies. It also involves Multi-turn Abstractive Question Answering.

Each corpus includes the following columns: question, answer, and context(s) containing relevant information from which the model can derive the answer. In multi-turn tasks, a chat history is also provided.

The scoring process for LLMs involves measuring the similarity between the original answer and the one generated by the model. All corpora are private except for AQuAS and RagQuAS, which are publicly available and can serve as examples of the structure and content of the others.

#### Factors

These evaluations are very specific and do not encompass all the general scenarios to which the model could be exposed, since all evaluations are focused on solving tasks for RAG in very specific domains.

#### Metrics

The evaluation is based on using [Llama-3.1-8B-Instruct](https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct) to score the answers.

### Results

| **Model**                  | **Average** | **AQuAS** | **RagQuAS** | **CAM**  | **CAM_E1** | **CAM_E2** | **CAM_E3** | **Shops** | **Insurance** |
|----------------------------|-------------|-----------|-------------|----------|------------|------------|------------|-----------|---------------|
| **RigoChat-7b-v2**         | **79.01**   | 82.06     | 77.91       | **78.91**| **79.27**  | 76.55      | 75.27      | **81.05** | **81.04**     |
| GPT-4o                     | 78.26       | **85.23** | 77.91       | 78.00    | 74.91      | 73.45      | **77.09**  | 78.60     | 80.89         |
| stablelm-2-12b-chat        | 77.74       | 78.88     | 78.21       | 77.82    | 78.73      | **77.27**  | 74.73      | 77.03     | 79.26         |
| Mistral-Small-Instruct-2409| 77.29       | 80.56     | 78.81       | 77.82    | 75.82      | 73.27      | 73.45      | 78.25     | 80.36         |
| Qwen2.5-7B-Instruct        | 77.17       | 80.93     | 77.41       | 77.82    | 75.09      | 75.45      | 72.91      | 78.08     | 79.67         |
| Meta-Llama-3.1-8B-Instruct | 76.55       | 81.87     | 80.50       | 72.91    | 73.45      | 75.45      | 71.64      | 77.73     | 78.88         |
| GPT-4o-mini                | 76.48       | 82.80     | 75.82       | 76.36    | 74.36      | 72.36      | 71.82      | 78.25     | 80.08         |
| Phi-3.5-mini-instruct      | 76.38       | 81.68     | **81.09**   | 75.82    | 74.73      | 71.45      | 70.36      | 77.43     | 78.45         |
| gemma-2-9b-it              | 75.80       | 82.80     | 78.11       | 72.91    | 73.45      | 71.09      | 71.27      | 77.08     | 79.72         |
| Ministral-8B-Instruct-2410 | 75.19       | 79.63     | 77.31       | 76.00    | 73.45      | 72.36      | 70.18      | 76.44     | 76.14         |
| GPT-3.5-turbo-0125         | 74.78       | 80.93     | 73.53       | 76.73    | 72.55      | 72.18      | 69.09      | 75.63     | 77.64         |
| Llama-2-7b-chat-hf         | 71.18       | 67.10     | 77.31       | 71.45    | 70.36      | 70.73      | 68.55      | 72.07     | 71.90         |
| granite-3.0-8b-instruct    | 71.08       | 73.08     | 72.44       | 72.36    | 71.82      | 69.09      | 66.18      | 69.97     | 73.73         |
| RigoChat-7b-v1             | 62.13       | 72.34     | 67.46       | 61.27    | 59.45      | 57.45      | 57.64      | 62.10     | 59.34         |
| salamandra-7b-instruct     | 61.96       | 63.74     | 60.70       | 64.91    | 63.27      | 62.36      | 60.55      | 59.94     | 60.23         |


#### Summary

RigoChat-7b-v2 manages to significantly improve performance compared to Qwen-2.5 in the tasks for which it has been indirectly designed. On the other hand, it manages to outperform most state-of-the-art models in these tasks, demonstrating that with few resources LLMs can be aligned for specific use cases.

## Environmental Impact

<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->

Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).

- **Hardware Type:** [More Information Needed]
- **Hours used:** [More Information Needed]
- **Cloud Provider:** [More Information Needed]
- **Compute Region:** [More Information Needed]
- **Carbon Emitted:** [More Information Needed]

### Model Architecture and Objective

[More Information Needed]

### Compute Infrastructure

[More Information Needed]

#### Hardware

GPU NVIDIA A100 with Tensor Core.

#### Software

`Ubuntu 22.04.5 LTS` and the following requirements:

```
python=3.11
flash_attn>=2.5.8
datasets
numpy
trl
peft
huggingface_hub
```

## Citation

```
@misc {instituto_de_ingeniería_del_conocimiento_2025,
	author       = { {Instituto de Ingeniería del Conocimiento} },
	title        = { RigoChat-7b-v2 },
	year         = 2025,
	url          = { https://huggingface.co/IIC/RigoChat-7b-v2 },
	doi          = { 10.57967/hf/4158 },
	publisher    = { Hugging Face }
}
```

## Disclaimer

Please note that the model may contain biases or other undesired distortions based on the training data. When third parties implement systems or provide services based on this model, or use the model themselves, they are responsible for mitigating associated risks and ensuring compliance with applicable regulations, including those governing the use of artificial intelligence. The IIC, as the owner and creator of the model, shall not be held liable for any outcomes resulting from third-party use.

## Model Card Contact

- [[email protected]]([email protected]).

## Authorship Notice

### Additional Information

- **Author**: Instituto de Ingeniería del Conocimiento (IIC).
- **Contact**: For more information, please email [[email protected]]([email protected]).
- **Copyright**: Copyright(c) 2024 ADIC.