Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FT] LiteLLM concurrency parameters hard-coded #567

Open
lhl opened this issue Feb 16, 2025 · 0 comments
Open

[FT] LiteLLM concurrency parameters hard-coded #567

lhl opened this issue Feb 16, 2025 · 0 comments
Labels
feature request New feature/request

Comments

@lhl
Copy link

lhl commented Feb 16, 2025

I noticed that for LiteLLM the concurrency is hard-coded for an arbitrary number (20). Well, a lot of the request behavior is hard coded, but the concurrency is the one that is probably least likely to work well as a one size fits all.

If everything is going through the YAML file, maybe it can be expanded, although the YAML file is another ball of wax (makes automated runs much harder when you can't just pass in parameters via a call)

@lhl lhl added the feature request New feature/request label Feb 16, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature request New feature/request
Projects
None yet
Development

No branches or pull requests

1 participant