feat: Support binary response in tool return

[Wh1isper]

I was thinking that it seems we could use BinaryContent as the return of the tool call as a way to support the tool return part returning images, audio, etc.

This is a draft and I may not have time to refine it lately. Feel free to cherry-pick or discuss further.

[Wh1isper]

Seems works:

agent = Agent(
    infer_model("bedrock:us.anthropic.claude-3-7-sonnet-20250219-v1:0"),
    system_prompt="You are a helpful assistant.",
)


@agent.tool_plain
def read_image(file_path: str) -> BinaryContent:
    """Read an image file."""
    return BinaryContent(
        data=open(file_path, "rb").read(),
        media_type="image/png",
    )


print(
    agent.run_sync(
        "Tell me what this image is about: /the/image/path"
    )
)

[Wh1isper]

@Kludex What do you think, with model multimodal capabilities, model-centric applications often require the use of tools to read files, they may be images, audio, documents.

As far as I know, claude code's view tool provides the ability to read images.

[Wh1isper]

I found that anthropic supports url as well, so maybe ImageUrl (DocumentUrl...) that could be supported as well.

[TheFirstMe]

I also need this feature badly. I am forced to move away from Pydantic AI because of this limitation :(