Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support base64 images in Function Tool Call Responses #5250

Open
BenConstable9 opened this issue Jan 29, 2025 · 1 comment
Open

Support base64 images in Function Tool Call Responses #5250

BenConstable9 opened this issue Jan 29, 2025 · 1 comment
Labels
awaiting-op-response Issue or pr has been triaged or responded to and is now awaiting a reply from the original poster
Milestone

Comments

@BenConstable9
Copy link

What feature would you like to be added?

Support for multi-modal messages in a Function Tool Call response similarly to the MultiModal input message.

Why is this needed?

It allows agents to call tools that will return images e.g. part of a multi-modal rag scenario

@victordibia
Copy link
Collaborator

Hey @BenConstable9 ,

Is this possibly related to #4725 ?
Any chance you are interested in contributing to this? We can help along the way.

In the mean time, it will be a huge help if you can describe the scenario you have in mind in detail. E.g, are you using core api vs agentchat? Can you give a general example of the the code/api you expect, how you would consume the multimodal message (or other message types etc).

@victordibia victordibia added awaiting-op-response Issue or pr has been triaged or responded to and is now awaiting a reply from the original poster and removed needs-triage labels Jan 29, 2025
@ekzhu ekzhu added this to the python-0.4.x milestone Jan 29, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
awaiting-op-response Issue or pr has been triaged or responded to and is now awaiting a reply from the original poster
Projects
None yet
Development

No branches or pull requests

3 participants