Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add FallbackModel support #894

Merged
merged 37 commits into from
Feb 25, 2025
Merged

Add FallbackModel support #894

merged 37 commits into from
Feb 25, 2025

Conversation

sydney-runkle
Copy link
Contributor

@sydney-runkle sydney-runkle commented Feb 11, 2025

This PR adds a FallbackModel to the models folder that allows for delegation of a task to a new model if the current model fails with a 4xx or 5xx response. See the docs for a detailed example.

Easiest to review by toggling the "hide whitespace" button.

Fix #516

This is one of the items on our roadmap as well.

In a follow up PR:

  • Rename ModelStatusError and rethink our error structure internally (before the next release)
  • Consider errors with no status code (connection errors)

@github-actions github-actions bot temporarily deployed to deploy-preview February 11, 2025 15:19 Inactive
@github-actions github-actions bot temporarily deployed to deploy-preview February 11, 2025 16:12 Inactive
@github-actions github-actions bot temporarily deployed to deploy-preview February 11, 2025 18:21 Inactive
@github-actions github-actions bot temporarily deployed to deploy-preview February 11, 2025 18:25 Inactive
@github-actions github-actions bot temporarily deployed to deploy-preview February 11, 2025 18:43 Inactive
@github-actions github-actions bot temporarily deployed to deploy-preview February 11, 2025 19:10 Inactive
@github-actions github-actions bot temporarily deployed to deploy-preview February 11, 2025 22:12 Inactive
@github-actions github-actions bot temporarily deployed to deploy-preview February 11, 2025 22:40 Inactive
@github-actions github-actions bot temporarily deployed to deploy-preview February 12, 2025 15:32 Inactive
@github-actions github-actions bot temporarily deployed to deploy-preview February 13, 2025 15:41 Inactive
@github-actions github-actions bot temporarily deployed to deploy-preview February 13, 2025 16:37 Inactive
@github-actions github-actions bot temporarily deployed to deploy-preview February 14, 2025 15:50 Inactive
Copy link
Contributor

@dmontagu dmontagu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great. Let's confirm the new exception names with Samuel tomorrow morning then merge.

@github-actions github-actions bot temporarily deployed to deploy-preview February 18, 2025 16:18 Inactive
@sydney-runkle
Copy link
Contributor Author

Thanks @dmontagu for picking up the rest of the changes here :)

@Kludex
Copy link
Member

Kludex commented Feb 25, 2025

This PR is mine now.

@Kludex Kludex self-assigned this Feb 25, 2025
Copy link

github-actions bot commented Feb 25, 2025

Docs Preview

commit: da98197
Preview URL: https://23b1b62d-pydantic-ai-previews.pydantic.workers.dev

@Kludex Kludex changed the title FallbackModel support Add FallbackModel support Feb 25, 2025
@Kludex Kludex requested a review from dmontagu February 25, 2025 11:38
Copy link
Member

@Kludex Kludex left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. We should be able to pass a callable that determines if we should fallback to the next model, or a tuple of exception types.
  2. We need to subclass ExceptionGroup.

@dmontagu dmontagu force-pushed the fallback-model-updated branch from ba592e2 to f5d2a5c Compare February 25, 2025 17:09
@Kludex Kludex merged commit 2f2713b into main Feb 25, 2025
17 checks passed
@Kludex Kludex deleted the fallback-model-updated branch February 25, 2025 21:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Fallback Model
5 participants