AutoGen: Code Execution Issue in Multi-Agent System

[Laxmi530]

I am using AutoGen to implement a multi-agent system for my inventory management task. The system integrates Python code for managing inventory databases. However, I am encountering an issue where the agents fail to execute code as expected. I tried with function calling and tools but there are no changes. I am not getting the expected output.
LLM - Qwen2.5-1.5B-Instruct
AutoGen - 0.7.1
Python - 3.12.7
The following is the code I tried.

from autogen import AssistantAgent, UserProxyAgent, GroupChat, GroupChatManager, register_function
from autogen.coding import LocalCommandLineCodeExecutor
from SQL_DB import inventry_check
from pathlib import Path

chat_llm_config_local = [{"model": "Qwen2.5-1.5B-Instruct", "api_key": "None", "base_url": "http://127.0.0.1:8000/v1"}]
chat_llm_config = {"temperature": 0.2, "seed": 42, "config_list": chat_llm_config_local}

INVENTRY_PROMPT = """ 
You are an intelligent agent tasked with checking the stock levels in the inventory database. 
Follow these instructions precisely:
1. Execute only the provided function to check the current stock levels of items. 
2. Avoid any external assumptions or knowledge outside the function.
3. Output the results of the function exactly as returned, without modifications or formatting changes.
"""

dir = Path("code")
dir.mkdir(exist_ok=True)
executor = LocalCommandLineCodeExecutor(work_dir=dir)

inventry_assistant = AssistantAgent(
    name="Inventry_Assistant",
    system_message=INVENTRY_PROMPT,
    llm_config=chat_llm_config,
    code_execution_config={"executor": executor}
)

user_proxy = UserProxyAgent(
    name = "user_proxy",
    human_input_mode = "NEVER",
    max_consecutive_auto_reply=5,
    is_termination_msg=lambda x:x.get("content", "").rstrip().endswith("TERMINATE"),
    code_execution_config={"work_dir":"test", "use_docker": False},
    llm_config=chat_llm_config,
    system_message="""Reply TERMINATE if the task has been solved at full satisfication Otherwise, reply CONTINUE, or the reason why the task is not solved yet."""
    )
# Method 1
inventry_assistant.register_for_llm(name="inventry_check", description="Get item, quantity and unit price from the inventry database.")(inventry_check)
user_proxy.register_for_execution(name="inventry_check")(inventry_check)
print(user_proxy.initiate_chat(inventry_assistant, message="How many item left in the inventry?"))

# Method 2
register_function(
    inventry_check,
    caller=inventry_assistant,
    executor=user_proxy,
    name="inventry_check",
    description="To check the inventry database.",
    )
print(user_proxy.initiate_chat(inventry_assistant, "How many item left in the inventry?"))
# Method 3
# Function for the inventory check
from sqlalchemy import create_engine, Table, MetaData, select
from sqlalchemy.orm import sessionmaker
from typing import List, Dict

@user_proxy.register_for_execution()
@inventry_assistant.register_for_llm(description="Get item, unit and unit price from the inventry database.")
def inventry_check() ->List[Dict]:
    """
    Connects to inventry DB, retrieves items with low quantities based on the threshold.
    :return: List of dictionaries containing item details.
    """
    engine = create_engine(inventry_db_url, echo=True)
    metadata = MetaData()
    ItemTable = Table("Inventry", metadata, autoload_with=engine)
    Session = sessionmaker(bind=engine)
    session = Session()
    try:
        result = session.execute(select(ItemTable).where(ItemTable.c.quantity <= ItemTable.c.threshold)).fetchall()
        items = [{"Item": row[1], "Quantity": row[2], "Unit price": row[3], "Threshold": 10} for row in result]
    finally:
        session.close()
    return items
print(user_proxy.initiate_chat(inventry_assistant, message="How many mobile left in the inventry?"))

I tried with different prompt and tried with Llama 3.2 1B model but there are no changes.
Can someone please guide me?

Thanking you in advance.

[ekzhu]

Our current stable version is v0.4.3, you are not using a Microsoft-published package. Please install autogen-agentchat.

Also, your code is using v0.2 API, please see the migration guide for v0.4: https://microsoft.github.io/autogen/stable/user-guide/agentchat-user-guide/migration-guide.html

For your example, I tried with llama3.2 2B using Ollama:

import asyncio
from typing import Dict, List
from autogen_agentchat.agents import AssistantAgent
from autogen_agentchat.ui import Console
from autogen_ext.models.openai import OpenAIChatCompletionClient

def inventry_check() -> List[Dict]:
    """Get item, unit and unit price from the inventory database."""
    result = [
        (1, "Apple", 5, 10),
        (2, "Banana", 15, 5),
        (3, "Orange", 7, 8),
        (4, "Pineapple", 20, 15),
        (5, "Mango", 10, 20),
    ]
    items = [{"Item": row[1], "Quantity": row[2], "Unit price": row[3]} for row in result]
    return items

async def main() -> None:
    model_client = OpenAIChatCompletionClient(
        model="llama3.2:latest",
        base_url="http://localhost:11434/v1",
        api_key="placeholder",
        model_info={
            "vision": False,
            "function_calling": True,
            "json_output": False,
            "family": "unknown",
        },
    )
    agent = AssistantAgent(
        "assistant",
        model_client=model_client,
        tools=[inventry_check],
        reflect_on_tool_use=True,
    )
    stream = agent.run_stream(task="How many apples are in the inventory?")
    await Console(stream)


asyncio.run(main())

Output:

---------- user ----------
How many apples are in the inventory?
---------- assistant ----------
[FunctionCall(id='call_xzaeoawp', arguments='{"item":"apple"}', name='inventry_check')]
---------- assistant ----------
[FunctionExecutionResult(content="[{'Item': 'Apple', 'Quantity': 5, 'Unit price': 10}, {'Item': 'Banana', 'Quantity': 15, 'Unit price': 5}, {'Item': 'Orange', 'Quantity': 7, 'Unit price': 8}, {'Item': 'Pineapple', 'Quantity': 20, 'Unit price': 15}, {'Item': 'Mango', 'Quantity': 10, 'Unit price': 20}]", call_id='call_xzaeoawp')]
---------- assistant ----------
The quantity of apples in the inventory is 5.