Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: handle non-string function arguments in tool calls and add corresponding warnings #5260

Merged
merged 3 commits into from
Jan 30, 2025

Conversation

ekzhu
Copy link
Collaborator

@ekzhu ekzhu commented Jan 30, 2025

Resolves #5257

Copy link

codecov bot commented Jan 30, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 70.42%. Comparing base (fff201f) to head (deaa3f1).
Report is 2 commits behind head on main.

Additional details and impacted files
@@             Coverage Diff             @@
##             main    #5260       +/-   ##
===========================================
- Coverage   83.70%   70.42%   -13.29%     
===========================================
  Files          60      179      +119     
  Lines        2817    11625     +8808     
===========================================
+ Hits         2358     8187     +5829     
- Misses        459     3438     +2979     
Flag Coverage Δ
unittests 70.42% <100.00%> (-13.29%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@lspinheiro
Copy link
Collaborator

lspinheiro commented Jan 30, 2025

I found it quite weird how the openai SDK was validating it and converting to dict when Function is a pydantic model but then I realised they have a custom implementation of pydantic BaseModel at openai._models.py.

It could be a bug in their SDK that maybe we should report. Based on the definition and data validation the function should never return a dict.

class Function(BaseModel):
    arguments: str
    """
    The arguments to call the function with, as generated by the model in JSON
    format. Note that the model does not always generate valid JSON, and may
    hallucinate parameters not defined by your function schema. Validate the
    arguments in your code before calling your function.
    """

    name: str
    """The name of the function to call."""

@ekzhu
Copy link
Collaborator Author

ekzhu commented Jan 30, 2025

Yes. Would you like to report it? Though fixing that will immediately break hugging face users.

@ekzhu ekzhu enabled auto-merge (squash) January 30, 2025 16:44
@ekzhu ekzhu merged commit 44db2cc into main Jan 30, 2025
64 checks passed
@ekzhu ekzhu deleted the ekzhu-fix-non-string-args branch January 30, 2025 16:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

add support for some models whose tool call arguments is not a json object
2 participants