KubeOne mirroring with harbor is not usable

[ix-containerservices]

Description of the feature you would like to add / User story

As a user
I would like the option to add additional parameters to the mirroring config of containerd with KubeOne
in order to make the mirroring work with harbors proxy cache projects.

Solution details

• An option to add additional parameters to the mirroring section of containerd e.g. override_path = true

Alternative approaches

Since the workers are a dynamic ressource we can't add this config to the vm template. KubeOne/machine-controller would overwrite the config we have there in place. So an alternative approach would be that we can deactivate the containerd config overwrite through KubeOne/machine-controller.

Use cases

Harbors proxy cache projects have a different implementation than containerd. So we have to set the "/v2/" in the url which is possible right now. But we also need to set the parameter: 'override_path = true' for the mirroring to properly work. Otherwise we can't use the mirroring with harbor.

Additional information

When the config doesn't look like this:
[plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io"]
endpoint = ["https://registry/v2/project"]
override_path = true

We get this error when we try to pull images from e.g. Dockerhub:
Failed to pull image "nicolaka/netshoot": rpc error: code = NotFound desc = failed to pull and unpack image "docker.io/nicolaka/netshoot:latest": failed to unpack image on snapshotter overlayfs: unexpected media type text/html for sha256:ecd8a6c12adb298a513c0aa9ceba8d2c93263437d77b123c5ccfd97abaa49abb: not found
goharbor/harbor#18530 (comment)

[csengerszabo]

/label customer-request
Internal reference: 7864

[csengerszabo]

We need to support this in OSM first, as it also needs to be propagated to worker nodes as well. We have a mechanism in place to propagate such features via MachineDeployment annotations.