diff --git a/docs/source/serving/openai_compatible_server.md b/docs/source/serving/openai_compatible_server.md index 0880a4530d8..a6ec05f45b6 100644 --- a/docs/source/serving/openai_compatible_server.md +++ b/docs/source/serving/openai_compatible_server.md @@ -29,6 +29,11 @@ completion = client.chat.completions.create( print(completion.choices[0].message) ``` +:::{tip} +vLLM supports some parameters that are not supported by OpenAI, `top_k` for example. +You can pass these parameters to vLLM using the OpenAI client in the `extra_body` parameter of your requests, i.e. `extra_body={"top_k": 50}` for `top_k`. +::: + ## Supported APIs We currently support the following OpenAI APIs: