Triangle104/Dria-Agent-a-3B-Q5_K_M-GGUF

This model was converted to GGUF format from driaforall/Dria-Agent-a-3B using llama.cpp via the ggml.ai's GGUF-my-repo space. Refer to the original model card for more details on the model.

---

Model details:

Dria-Agent-α are series of large language models trained on top of the Qwen2.5-Coder series, specifically on top of the Qwen/Qwen2.5-Coder-3B-Instruct and Qwen/Qwen2.5-Coder-7B-Instruct models to be used in agentic applications. These models are the first instalment of agent-focused LLMs (hence the α in the naming) we hope to improve with better and more elaborate techniques in subsequent releases.

Dria-Agent-α employs Pythonic function calling, which is LLMs using blocks of Python code to interact with provided tools and output actions. This method was inspired by many previous work, including but not limited to DynaSaur, RLEF, ADAS and CAMEL. This way of function calling has a few advantages over traditional JSON-based function calling methods:

One-shot Parallel Multiple Function Calls: The model can can utilise many synchronous processes in one chat turn to arrive to a solution, which would require other function calling models multiple turns of conversation.
Free-form Reasoning and Actions: The model provides reasoning traces freely in natural language and the actions in between ```python ``` blocks, as it already tends to do without special prompting or tuning. This tries to mitigate the possible performance loss caused by imposing specific formats on LLM outputs discussed in Let Me Speak Freely?
On-the-fly Complex Solution Generation: The solution provided by the model is essentially a Python program with the exclusion of some "risky" builtins like exec, eval and compile (see full list in Quickstart below). This enables the model to implement custom complex logic with conditionals and synchronous pipelines (using the output of one function in the next function's arguments) which would not be possible with the current JSON-based function calling methods (as far as we know).

Quickstart

import json from typing import Any, Dict, List from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "driaforall/Dria-Agent-a-3B" model = AutoModelForCausalLM.from_pretrained( model_name, device_map="auto", torch_dtype="auto", trust_remote_code=True ) tokenizer = AutoTokenizer.from_pretrained(model_name)

Please use our provided prompt for best performance

SYSTEM_PROMPT = """ You are an expert AI assistant that specializes in providing Python code to solve the task/problem at hand provided by the user.

You can use Python code freely, including the following available functions:

<|functions_schema|> {{functions_schema}} <|end_functions_schema|>

The following dangerous builtins are restricted for security:

• exec
• eval
• execfile
• compile
• importlib
• input
• exit

Think step by step and provide your reasoning, outside of the function calls. You can write Python code and use the available functions. Provide all your python code in a SINGLE markdown code block like the following:

result = example_function(arg1, "string")
result2 = example_function2(result, arg2)

DO NOT use print() statements AT ALL. Avoid mutating variables whenever possible. """.strip()

get_sample_data = """ def check_availability(day: str, start_time: str, end_time: str) -> bool: """ Check if a time slot is available on a given day.

Args:
- day: The day to check in YYYY-MM-DD format
- start_time: Start time in HH:MM format
- end_time: End time in HH:MM format

Returns:
- True if slot is available, False otherwise
\"\"\"
pass

def make_appointment(day: str, start_time: str, end_time: str) -> dict: """ Make an appointment for a given time slot.

Args:
- day: The day to make appointment in YYYY-MM-DD format
- start_time: Start time in HH:MM format
- end_time: End time in HH:MM format
- title: The title of the appointment

Returns:
- A dictionary with the appointment details and if it's made or not.
    dict keys:
        - day (str): The day the appointment is on, in YYYY-MM-DD format
        - start_time (str): Start time in HH:MM format
        - end_time (str): End time in HH:MM format
        - appointment_made (bool): Whether the appointment is successfully made or not. 
\"\"\"
pass

def add_to_reminders(reminder_text: str) -> bool: """ Add a text to reminders.

Args: 
- reminder_text: The text to add to reminders

Returns:
- Whether the reminder was successfully created or not.
\"\"\"
pass

"""

Helper function to create the system prompt for our model

def format_prompt(tools: str): return SYSTEM_PROMPT.format(functions_schema=tools)

system_prompt = SYSTEM_PROMPT.replace("{{functions_schema}}", get_sample_data)

USER_QUERY = """ Can you check if I have tomorrow 10:00-12:00 available and make an appointment for a meeting with my thesis supervisor if so? If you made the appointment, please add it to my reminders. """

messages = [ {"role": "system", "content": system_prompt}, {"role": "user", "content": USER_QUERY}, ]

text = tokenizer.apply_chat_template( messages, tokenize=False, add_generation_prompt=True ) model_inputs = tokenizer([text], return_tensors="pt").to(model.device)

generated_ids = model.generate( **model_inputs, max_new_tokens=2048 ) generated_ids = [ output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids) ]

response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0] print(response)

The output should be something like:

Get today's date and calculate tomorrow's date

from datetime import datetime, timedelta today = datetime.now() tomorrow = (today + timedelta(days=1)).strftime("%Y-%m-%d")

Define the time slots

start_time = "10:00" end_time = "12:00"

Check availability first

is_available = check_availability(tomorrow, start_time, end_time)

Only proceed with making the appointment if it's available

appointment_result = ( make_appointment( day=tomorrow, start_time=start_time, end_time=end_time, title="Meeting with Thesis Supervisor" ) if is_available else {"appointment_made": False} )

Add to reminders only if the appointment was made

if appointment_result["appointment_made"]: add_to_reminders("Meeting with Thesis Supervisor scheduled for 10:00 AM tomorrow")

This code will:
1. Calculate tomorrow's date in YYYY-MM-DD format
2. Check if the 10:00-12:00 slot is available
3. If available, make the appointment with the specified details
4. If the appointment is successfully made, add a reminder to the system

The code handles all error cases implicitly through the boolean returns of the functions. If any step fails, the subsequent steps won't execute, preventing partial or invalid appointments.

Evaluation & Performance

We evaluate the model on the following benchmarks:

    Berkeley Function Calling Leaderboard (BFCL)
    MMLU-Pro
    Dria-Pythonic-Agent-Benchmark (DPAB): The benchmark we curated with a synthetic data generation +model-based validation + filtering and manual selection to evaluate LLMs on their Pythonic function calling ability, spanning multiple scenarios and tasks. More detailed information about the benchmark and the Github repo will be released soon.

---

## Use with llama.cpp
Install llama.cpp through brew (works on Mac and Linux)

```bash
brew install llama.cpp

Invoke the llama.cpp server or the CLI.

CLI:

llama-cli --hf-repo Triangle104/Dria-Agent-a-3B-Q5_K_M-GGUF --hf-file dria-agent-a-3b-q5_k_m.gguf -p "The meaning to life and the universe is"

Server:

llama-server --hf-repo Triangle104/Dria-Agent-a-3B-Q5_K_M-GGUF --hf-file dria-agent-a-3b-q5_k_m.gguf -c 2048

Note: You can also use this checkpoint directly through the usage steps listed in the Llama.cpp repo as well.

Step 1: Clone llama.cpp from GitHub.

git clone https://github.com/ggerganov/llama.cpp

Step 2: Move into the llama.cpp folder and build it with LLAMA_CURL=1 flag along with other hardware-specific flags (for ex: LLAMA_CUDA=1 for Nvidia GPUs on Linux).

cd llama.cpp && LLAMA_CURL=1 make

Step 3: Run inference through the main binary.

./llama-cli --hf-repo Triangle104/Dria-Agent-a-3B-Q5_K_M-GGUF --hf-file dria-agent-a-3b-q5_k_m.gguf -p "The meaning to life and the universe is"

or

./llama-server --hf-repo Triangle104/Dria-Agent-a-3B-Q5_K_M-GGUF --hf-file dria-agent-a-3b-q5_k_m.gguf -c 2048