Distributed training with mutliple pods, with multi-gpu in each pod

[githubthunder]

I want to execute distributed training in a Kubernetes environment using the command "kubectl apply -f train.yaml".

Which version of Kubeflow supports the torchrun command for distributed training across multiple PODs, with multiple GPUs in each POD?

Please provide a working example, including sample code and YAML files, with a focus on how to write the YAML file.

Thank you very much!