feat: Streaming w/ Tools

[ksylvest]

Right now streaming + tools isn't supported. The API needs to be adjusted to support.

[artob]

This presumably meaning the https://sdk.vercel.ai/docs/ai-sdk-ui/stream-protocol protocol?

[ksylvest]

@artob I'm not sure. Currently all the LLM providers support streaming via SSE:

https://github.com/ksylvest/omniai/blob/main/lib/omniai/chat/stream.rb

The challenging bit is how tool-call messages are chunked. For example, OpenAI sends through through like this:

[{"index": 0, "id": "call_DdmO9pD3xa9XTPNJ32zg2hcA", "function": {"arguments": "", "name": "get_weather"}, "type": "function"}]
[{"index": 0, "id": null, "function": {"arguments": "{\"", "name": null}, "type": null}]
[{"index": 0, "id": null, "function": {"arguments": "location", "name": null}, "type": null}]
[{"index": 0, "id": null, "function": {"arguments": "\":\"", "name": null}, "type": null}]
[{"index": 0, "id": null, "function": {"arguments": "Paris", "name": null}, "type": null}]
[{"index": 0, "id": null, "function": {"arguments": ",", "name": null}, "type": null}]
[{"index": 0, "id": null, "function": {"arguments": " France", "name": null}, "type": null}]
[{"index": 0, "id": null, "function": {"arguments": "\"}", "name": null}, "type": null}]

I think we'd need some basic understanding that a chunk is for a tool call - then not yield it back immediately, but collect the entire tool call and perform it.