git-mirror

git-mirror is an app to periodically mirror (bare clones) remote repositories locally. It supports multiple mirrored checked out worktrees on different references and it can also mirror multiple repositories.

git-mirror can be used as golang library in your code. docs
The mirror is created with --mirror=fetch hence everything in refs/* on the remote will be directly mirrored into refs/* in the local repository.

The implementation borrows heavily from kubernetes/git-sync. If you want to sync single repository on one reference then you are probably better off with kubernetes/git-sync, as it provides a lot more customisation. git-mirror should be used if multiple mirrored repositories with multiple checked out branches (worktrees) is required.

Usage

Usage:
    git-mirror [global options]

GLOBAL OPTIONS:
    -log-level value          (default: 'info') Log level [$LOG_LEVEL]
    -config value             (default: '/etc/git-mirror/config.yaml') Absolute path to the config file. [$GIT_MIRROR_CONFIG]
    -watch-config value       (default: true) watch config for changes and reload when changes encountered. [$GIT_MIRROR_WATCH_CONFIG]
    -http-bind-address value  (default: ':9001') The address the web server binds to. [$GIT_MIRROR_HTTP_BIND]

Config

configuration file contains default parameters and list of repositories. each repository also contains list of worktrees to checkout. defaults fields values are used if repository parameters are not specified.

defaults:
  # root is the absolute path to the root dir where all repositories directories
  # will be created all repos worktree links will be created here if not
  # specified in repo config (default: '/tmp/git-mirror')
  root: /tmp/git-mirror

  # interval is time duration for how long to wait between mirrors. (default: '30s')
  interval: 30s

  # mirrorTimeout represents the total time allowed for the complete mirror loop (default: '2m')
  mirror_timeout: 2m

  # git_gc garbage collection string. valid values are
  # 'auto', 'always', 'aggressive' or 'off' (default: 'always')
  git_gc: always

  # auth config to fetch remote repos
  auth:
    # path to the ssh key & known hosts used to fetch remote
    ssh_key_path: /etc/git-secret/ssh
    ssh_known_hosts_path: /etc/git-secret/known_hosts
repositories:
  - remote: https://github.com/utilitywarehouse/git-mirror # required
    # remote is the git URL of the remote repository to mirror.
    # supported urls are '[email protected]:org/repo.git','ssh://[email protected]/org/repo.git'
    # or 'https://host.xz/org/repo.git'. '.git' suffix is optional

    # following fields are optional.
    # if these fields are not specified values from defaults section will be used
    root: /some/other/location
    interval: 1m
    mirror_timeout: 5m
    git_gc: always
    auth:
      ssh_key_path: /some/other/location
      ssh_known_hosts_path: /some/other/location
    worktrees:
      - link: alerts # required
        # link is the path at which to create a symlink to the worktree dir
        # if path is not absolute it will be created under repository root

        ref: main
        # ref represents the git reference of the worktree branch, tags or hash
        # are supported. default is HEAD

        pathspec: example
        # pathspec of the dirs to checkout if required, optional if omitted
        # whole repo will be checked out

App can load changes in config without restart. At repository level only adding and removing repository is supported. changes in interval, timeout and auth will require an app restart. At worktree level apart from adding or removing, changes in existing worktree's link, ref and pathspec is supported.