Skip to content

A Kubectl plugin to allow to compare a known valid reference configuration and a set of specific cluster configuration CRs.

License

Notifications You must be signed in to change notification settings

openshift/kube-compare

Folders and files

NameName
Last commit message
Last commit date
Sep 19, 2024
Jan 30, 2025
Jan 7, 2025
Feb 14, 2025
Feb 14, 2025
Feb 13, 2025
Feb 17, 2025
Dec 6, 2024
May 23, 2024
Jun 24, 2024
Oct 25, 2024
Apr 25, 2024
Aug 14, 2024
Dec 6, 2024
Apr 4, 2024
Jan 7, 2025
Jan 8, 2025
Nov 11, 2024
Feb 17, 2025
Feb 17, 2025
May 28, 2024

Repository files navigation

kube-compare

Main branch test status

This repository is primarily home to a kubectl plugin called cluster-compare which allows the user to compare an individual manifest or entire running cluster to a reference and find the differences. Expected differences are ignored as are cluster-managed fields, highlighting only differences that are of interest.

cluster-compare is intended for administrators, architects, support engineers, and others to quickly check that a configuration is as-expected. For a more detailed description of the purpose and approach of the tool please read the proposal document.

Install

This plugin can be installed with the Krew plugin manager:

kubectl krew install cluster-compare

Alternatively, build the code locally and install it in /usr/local/bin:

make build
sudo make install

Run

A reference configuration is required in order to run. A reference configuration is a directory containing a metadata.yaml and one or more templates.

Compare a known valid reference configuration with a live cluster:

kubectl cluster-compare -r ./reference/metadata.yaml

Compare a known valid reference configuration with a local set of CRs:

kubectl cluster-compare -r ./reference/metadata.yaml -f ./crsdir -R

Compare a known valid reference configuration with a live cluster and with a user config:

kubectl cluster-compare -r ./reference/metadata.yaml -c ./user_config

Run a known valid reference configuration with an oc must-gather output:

kubectl cluster-compare -r ./reference/metadata.yaml -f "must-gather*/*/cluster-scoped-resources","must-gather*/*/namespaces" -R

Run kubectl cluster-compare --help for a more extensive usage description.

Output

The tool outputs a diff for each comparison made, and a final summary.

Each comparison is surrounded by a line of *. The comparison identifies the cluster manifest and reference file being compared and a diff:

**********************************

Cluster CR: apps/v1_Deployment_kubernetes-dashboard_kubernetes-dashboard
Reference File: deploymentDashboard.yaml
Diff Output: diff -u -N /tmp/MERGED-4218954955/apps-v1_deployment_kubernetes-dashboard_kubernetes-dashboard /tmp/LIVE-168878603/apps-v1_deployment_kubernetes-dashboard_kubernetes-dashboard
--- /tmp/MERGED-4218954955/apps-v1_deployment_kubernetes-dashboard_kubernetes-dashboard 2024-07-02 09:18:04.314476186 -0400
+++ /tmp/LIVE-168878603/apps-v1_deployment_kubernetes-dashboard_kubernetes-dashboard    2024-07-02 09:18:04.314476186 -0400
@@ -14,7 +14,7 @@
   template:
     metadata:
       labels:
-        k8s-app: kubernetes-dashboard
+        k8s-app: kubernetes-dashboard-diff
     spec:
       containers:
       - args:

**********************************

The output ends with a summary of all comparisons made, which lists differences, and highlights if references were not found for any cluster manifests.

Summary
CRs with diffs: 1
CRs in reference missing from the cluster: 1
ExamplePart:
  Dashboard:
  - deploymentMetrics.yaml
No CRs are unmatched to reference CRs

Metadata.yaml

At the basic level, metadata.yaml lays out a reference configuration in Parts, each containing Components and defines the templates and comparison rules.

The following example describes an one-part guestbook app, with redis and frontend components. The templates it references are stored in the same directory as metadata.yaml.

parts:
  - name: guestbook
    components:
      - name: redis
        type: Required # mark the Component as "Required" or "Optional"
        requiredTemplates: # absence from cluster manifests is considered a diff
          - path: redis-master-deployment.yaml
            config:
               fieldsToOmitRefs: # default and deployments
                  - default
                  - deployments
          - path: redis-master-service.yaml
        optionalTemplates: # will not be reported if missing from cluster manifests
          - path: redis-replica-deployment.yaml
            config:
               fieldsToOmitRefs: # Only deployments
                  - deployments
          - path: redis-replica-service.yaml
      - name: frontend
        type: Required
        requiredTemplates:
          - path: frontend-deployment.yaml
            config:
               fieldsToOmitRefs: # Only deployments
                  - deployments
          - path: frontend-service.yaml
            config:
               fieldsToOmitRefs:
                  - cluster-compare-built-in  # The built in default values, used if no defaultOmitRef is provided

fieldsToOmit:
   defaultOmitRef: default
   items:
      deployments:
         - pathToKey: spec.selector.matchLabels.k8s-app # remove spec.selector.matchLabels.k8s-app before diff
         - pathToKey: metadata.labels.k8s-app
         - pathToKey: spec.template.metadata.labels.k8s-app
      default:
         - pathToKey: a.custom.default."k8s.io" # Keys containing dots should be quoted

metadata.yaml supports several other advanced behaviours:

  • Declaring predefined fragments using templateFunctionFiles that can be used in multiple resource templates.
  • Globally ignoring specific fields by yaml path with fieldsToOmit.
  • Ignoring, per template, fields that are not defined in the reference template with ignore-unspecified-fields.

See the included test cases for more examples of reference configs. For a complete explanation of metadata.yaml please see Building a Reference Config.

Further Resources

User Guide

Building a Reference Config

Developer Intro

Build Image

About

A Kubectl plugin to allow to compare a known valid reference configuration and a set of specific cluster configuration CRs.

Resources

License

Security policy

Stars

Watchers

Forks

Packages

No packages published