Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable ToolResult Type in Core/AgentChat #4725

Open
victordibia opened this issue Dec 16, 2024 · 1 comment
Open

Enable ToolResult Type in Core/AgentChat #4725

victordibia opened this issue Dec 16, 2024 · 1 comment

Comments

@victordibia
Copy link
Collaborator

Currently, in Core/AgentChat the result of tool execution is mostly cast to a string ...
It would be good to standardize on some type e.g,

  • content
  • attachments
  • mime
  • byte ..

Several benefits

  • We can cast strucutred responses into messages that other agents/teams can "see" .e.g, inform them of artifacts created from tool call.
  • Interfaces like AGS can display such artifacts

How

  • Add toolresult type
  • Update core infra for tool call execution to check for structured output and honor them
  • Update things like AssistantAgent to parse structured output into inner_messages etc?
@ekzhu
Copy link
Collaborator

ekzhu commented Jan 23, 2025

Yes, this should be added, similar to the structured message type #5131. Though I would consider instead of going for mime type we can stay in Python land for now.

When we type hint a function tool, the return type can be a BaseModel. Thus it is possible to capture the result as part of the structured tool result message, with type information embedded in the message itself.

@jackgerrits some ideas on this? This will enable many scenarios.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants