fix: update netlink dep to latest commit

[vadasambar]

• this fixes interrupted sys call errors in the plugin

What type of PR is this?
bug

Which issue does this PR fix?:

#3196

What does this PR do / Why do we need it?:
This PR fixes the frequent interrupted sys call errors like these:

Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox \"xxxxxxxx\": plugin type=\"aws-cni\" name=\"aws-cni\" failed (add): add command: failed to setup network: SetupPodNetwork: failed to setup veth pair: failed to setup veth network: setup NS network: failed while waiting for v6 addresses to be stable: could not list addresses: interrupted system call

Testing done on this change:

Will this PR introduce any new dependencies?:

No

Will this break upgrades or downgrades? Has updating a running cluster been tested?:
No

Does this change require updates to the CNI daemonset config files to work?:

No

Does this PR introduce any user-facing change?:

fix: interrupted system call error

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

[vadasambar]

v1.3.1 hasn't been released yet. This uses the latest commit at the time of creating this PR (until we get v1.3.1 release of netlink): vishvananda/netlink@0e7078e

[vadasambar]

We could handle the error in the code here:

amazon-vpc-cni-k8s/pkg/utils/cniutils/cni_utils.go

Lines 55 to 58 in 8dd2a5a

	addrs, err := netLink.AddrList(link, netlink.FAMILY_V6)
	if err != nil {
	return fmt.Errorf("could not list addresses: %v", err)
	}

with something like:

		if errors.Is(err, unix.EINTR) || err != nil {
			return fmt.Errorf("could not list addresses: %v", err)
		}

but we might end up ignoring unix.EINTR's which really matter.

P.S.: updated code link

[haouc]

@vadasambar thanks for updating this. Did you get chance to test? If you did, can you comment with the reprod you had in the overview?

[vadasambar]

@vadasambar thanks for updating this. Did you get chance to test? If you did, can you comment with the reprod you had in the overview?

This PR is only updating a dependency. If the tests are passing (for this repo), even if the issue is not fixed (I think the issue will be fixed), we would be in a better state.

The only problem I see is using a master/trunk version of a dependency. Provided upstream dependency is doing its due diligence, we should be fine? Let me know what you think.

[vadasambar]

@haouc ^

[yash97]

I don't see the urgency to use package commit while upgrading dependency. We can update package when it is officially released. Even with this update we only get benefit of error categorization. But we either we retry silently using this new update which is not officially released yet or we let kubelet retry on failure. It doesn't seem worth the risk of importing package with a tag with no official release as of now.

[vadasambar]

I don't see the urgency to use package commit while upgrading dependency. We can update package when it is officially released. Even with this update we only get benefit of error categorization.

We could wait for the upstream to update the dependency but we don't know when that would happen. The fix was merged in Sep 2024.

Even with this update we only get benefit of error categorization. But we either we retry silently using this new update which is not officially released yet or we let kubelet retry on failure. It doesn't seem worth the risk of importing package with a tag with no official release as of now.

We are throwing an error in the CNI (now) and letting kubelet retry vs using the result from call (before) even if the system call was interrupted (and then keep on retrying until the result contains the the value we were looking for). If we let kubelet retry, it's going to add to the pod startup time.

P.S.: the problem is only going to get worse with increased pod churn or larger clusters since the probability of sys call getting interrupted is only going to go up.

I guess we have 3 options now:

1. Wait for upstream to release the fix (no timeline on this; last releases: 2024, 2022, 2020)
2. Use the upstream's trunk version (comes with risk of using an unreleased version of a dependency)
3. Handle the error in CNI (need to be careful with dropping sys call interrupted errors; might need to replicate upstream's fix until there's a new release, something like this and this)