Synchronize vLLM flags to support cross-node inference

[IT-Forrest]

synchronize 12 vLLM flags to non-driver workers in Ray executor

FIX "not warmed-up" bucket issue in cross-node vLLM inference.

Root cause: the issue is caused by not synchronizing the 12 vLLM flags to all the non-driver workers within the Ray cluster

[michalkuligowski]

Please add hpu specific to existing platform specific list in line 344

vllm-fork/vllm/executor/ray_distributed_executor.py

Line 344 in 24480c3

for name in [

[IT-Forrest]

Hi @michalkuligowski, thanks for the comment. Let me know if I understand you correctly, do you want me (1) to add platform-specific comments in the commit, such as if current_platform.is_hpu() or (2) to put the fix from line 201 to line 344?

• As for (1), this change is not HPU platform-specific, but needed for the else branch of if current_platform.ray_device_key == "GPU": at line 191. Therefore, no comment is necessary

• As for (2), this fix should be done while the driver worker flow is creating the non-driver workers. I have tried to put the change to line 344, however, those 12 flags are not propagated to the cross node. That is the reason to keep the commit at line 191

Let me know if I get your point or provide me further feedback.

[michalkuligowski]

I am basing my previous comment on "Set environment variables for the driver and workers." in line 335, and then structure that contains env vars is used to call method "update_environment_variables" in line 357. Does that call has no effect? Maybe need to overraide base class method -

vllm-fork/vllm/worker/worker_base.py

Line 531 in 02d1110

def update_environment_variables(self, envs_list: List[Dict[str,

?