Validate K8s 1.27 in-place Pod spec.resources update alpha feature

[jonathan-innis]

Tell us about your request

Karpenter currently does tracking of resources requests, specifically in the context of understanding how many resources have been consumed on the inflight or real node and how many resources remain.

K8s 1.27 introduces an alpha feature that enables users to make in-place updates to pod resource requests. Alongside this change, since the spec.resources is now mutable after creation, the Pod introduces a new status.allocatedResources field which indicates the resources that are currently allocated to the pod. The fields spec.resources and status.allocatedResources should always match unless the field has recently changed and the pod is in a transient state.

Since we do inflight and real tracking of node requests (as well as scheduling simulations that rely on these requests), we should validate if our handling of requests currently is still valid with this new semantic.

Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?

N/A

Are you currently working around this issue?

N/A

Additional Context

No response

Attachments

No response

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

[tzneal]

We should use the PodRequests function at https://github.com/kubernetes/kubernetes/blob/7024beeeeb1f2e4cde93805a137cd7ad92fec466/pkg/api/v1/resource/helpers.go#L50 to calculate requests. This should work for Karpenter for both this feature and sidecar containers.

[Bryce-Soghigian]

/assign Bryce-Soghigian

[Bryce-Soghigian]

Figured I would mention this here, It was brought to my attention that this feature will remain in Alpha for 1.30 upstream.

The InPlaceVerticalAutoscaling feature has been dropped by the primary maintainer, and Node Sig is mentioning it needs to be completely re-worked due to a host of bugs in how kubelet percieves state with this new model.

See the following issue to monitor the progress:
kubernetes/kubernetes#116971