[Bug]: Enable Local endpoints on another network

[kevin-support-bot]

All-Hands-AI#7312 Issue

---

@metalshanked, Would you check whether this litellm script is working?

[metalshanked]

Sure, this seems to work

ModelResponse(id='chatcmpl-edee5eb2-d579-4a1c-8fd8-4a4fb2cf7ad3', created=1742301578, model='ollama/ ...

I also use the other option of directly using Litellm endpoingts with an OpenAI client

import openai
client = openai.OpenAI(
    api_key="my_key",
    base_url="http://mylitellmlocalserver" # LiteLLM Proxy is OpenAI compatible, Read More: https://docs.litellm.ai/docs/proxy/user_keys
)

response = client.chat.completions.create(
    model="ollama", # model to send to the proxy
    messages = [
        {
            "role": "user",
            "content": "this is a test request, write a short poem"
        }
    ]
)

print(response)

[SmartManoj]

Does this work in the docker "openhands app" container's terminal?