Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ability to customize the dind image in gha-runner-scale-set #3709

Open
victor-cetinic opened this issue Aug 15, 2024 · 3 comments
Open

Ability to customize the dind image in gha-runner-scale-set #3709

victor-cetinic opened this issue Aug 15, 2024 · 3 comments
Labels
community Community contribution enhancement New feature or request needs triage Requires review from the maintainers

Comments

@victor-cetinic
Copy link

What would you like added?

I have been trying to customise the gha-runner-scale-set so that i can set the docker registry mirror globally for all the runners using the Docker DIND approach.

Most instructions point to modifying the registry settings of the docker service by adding a command line like :
--registry-mirror=https://mirror.gcr.io
(further options - https://cloud.google.com/artifact-registry/docs/pull-cached-dockerhub-images )

The issue im facing is that these need to be configured on the dind container, and not the runner container., and when i created a template like this


template:
  spec: 
    nodeSelector:
      cloud.google.com/gke-preemptible: "true"
    initContainers:
      - name: dockerconfigwriter
        image: alpine
        command:
          - sh
          - -c
          - |
            echo "{\"registry-mirrors\": [ \"https://mirror.gcr.io\"] }" > /home/runner/.docker/daemon.json
        volumeMounts:
          - name: docker-config-volume
            mountPath: /home/runner/.docker
    containers:
      - name: runner
        command:
          - /home/runner/run.sh
        image:
          ghcr.io/actions/actions-runner:latest
        env:
          - name: DOCKER_HOST
            value: unix:///var/run/docker.sock
          - name: RUNNER_WAIT_FOR_DOCKER_IN_SECONDS
            value: \"120\"
        resources:
          limits:
            cpu: 6
            memory: 5Gi
        volumeMounts:
          - name: work
            mountPath: /home/runner/_work
          - name: dind-sock
            mountPath: /var/run
          - name: docker-config-volume
            mountPath: /etc/docker
      - name: dind
        image: docker:dind
        args:
          - dockerd
          - --host=unix:///var/run/docker.sock
          - --group=$(DOCKER_GROUP_GID)
          - --registry-mirror=https://mirror.gcr.io
        env:
          - name: DOCKER_GROUP_GID
            value: \"123\"
        securityContext:
          privileged: true
        volumeMounts:
          - name: work
            mountPath: /home/runner/_work
          - name: dind-sock
            mountPath: /var/run
          - name: dind-externals
            mountPath: /home/runner/externals
          - name: docker-config-volume
            mountPath: /etc/docker
    volumes:
      - name: docker-config-volume
        emptyDir: { }

I expected to be able to customise the dind container, however, it does not get included in the template, it simply uses the default pre-defined in the template, so none of these customisations are considered in the final template, consequently , the settings are not applied.

Output from dry run

Below you can see the final template, that contains the customisations i made to the runner, however, the dind container is ignored so my mirror registry settings are not applied.

  containers:
      - name: runner
        
        command: 
          - /home/runner/run.sh
        image: 
          ghcr.io/actions/actions-runner:latest
        resources: 
          limits:
            cpu: 6
            memory: 5Gi
        env:
          - 
            name: DOCKER_HOST
            value: unix:///var/run/docker.sock
          - 
            name: RUNNER_WAIT_FOR_DOCKER_IN_SECONDS
            value: \"120f\"
        volumeMounts:
          - 
            mountPath: /home/runner/_work
            name: work
          - 
            mountPath: /var/run
            name: dind-sock
          - 
            mountPath: /etc/docker
            name: docker-config-volume
      - name: dind
        image: docker:dind
        args:
          - dockerd
          - --host=unix:///var/run/docker.sock
          - --group=$(DOCKER_GROUP_GID)
        env:
          - name: DOCKER_GROUP_GID
            value: "123"
        securityContext:
          privileged: true
        volumeMounts:
          - name: work
            mountPath: /home/runner/_work
          - name: dind-sock
            mountPath: /var/run
          - name: dind-externals
            mountPath: /home/runner/externals

Why is this needed?

The issue for me is I hit limits going to docker registry, and having a single way to do this for all runners, makes more sense and would apply for all rather than each workflow having to do something

Additional context

Add any other context or screenshots about the feature request here.

@victor-cetinic victor-cetinic added community Community contribution enhancement New feature or request needs triage Requires review from the maintainers labels Aug 15, 2024
Copy link
Contributor

Hello! Thank you for filing an issue.

The maintainers will triage your issue shortly.

In the meantime, please take a look at the troubleshooting guide for bug reports.

If this is a feature request, please review our contribution guidelines.

@Alpenovsky
Copy link

Alpenovsky commented Sep 6, 2024

I had the same problem. I use FluxCD to create Helm release and I used the postRenderers functionality to overwrite (as a workaround) the dind image or dind args:

apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:
  name: dev-runners
spec:
  releaseName: dev-runners
  chartRef:
    kind: OCIRepository
    name: dev-runners
  postRenderers:
    - kustomize:
        patches:
          - target:
              kind: AutoscalingRunnerSet
              name: dev-runners
            patch: |
              - op: replace
                path: /spec/template/spec/containers/1/image
                value: 24.0.9-dind
[...]

@orbi-ci-bot
Copy link

+1 We are using RunnerDeployments. Can we get this option on there as well. We have reached our pull limit with Docker hub with no way to update the image.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
community Community contribution enhancement New feature or request needs triage Requires review from the maintainers
Projects
None yet
Development

No branches or pull requests

3 participants