You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I noticed that for LiteLLM the concurrency is hard-coded for an arbitrary number (20). Well, a lot of the request behavior is hard coded, but the concurrency is the one that is probably least likely to work well as a one size fits all.
If everything is going through the YAML file, maybe it can be expanded, although the YAML file is another ball of wax (makes automated runs much harder when you can't just pass in parameters via a call)
The text was updated successfully, but these errors were encountered:
I noticed that for LiteLLM the concurrency is hard-coded for an arbitrary number (20). Well, a lot of the request behavior is hard coded, but the concurrency is the one that is probably least likely to work well as a one size fits all.
If everything is going through the YAML file, maybe it can be expanded, although the YAML file is another ball of wax (makes automated runs much harder when you can't just pass in parameters via a call)
The text was updated successfully, but these errors were encountered: