Skip to content

Incosistnet tabs or new line characters are displayd when checking v1beta2 conditions of cluster resources #12031

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
Karthik-K-N opened this issue Mar 26, 2025 · 9 comments
Open

Incosistnet tabs or new line characters are displayd when checking v1beta2 conditions of cluster resources #12031

Karthik-K-N opened this issue Mar 26, 2025 · 9 comments
Labels
kind/bug Categorizes issue or PR as related to a bug. needs-priority Indicates an issue lacks a `priority/foo` label and requires one. needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one.

Comments

@Karthik-K-N
Copy link
Contributor

What steps did you take and what happened?

While creating a IBMPowerVSCluster, Observed some inconsistency in formatting of conditions and its hard to understand

kubectl describe cluster

v1beta2:
    Conditions:
      Last Transition Time:  2025-03-26T05:09:27Z
      Message:               * RemoteConnectionProbe: Remote connection probe failed
* ControlPlaneAvailable: Control plane not yet initialized
* WorkersAvailable:
  * MachineDeployment karthikkn-capi-powervs-md-0: 0 available replicas, at least 1 required (spec.strategy.rollout.maxUnavailable is 0, spec.replicas is 1)
      Observed Generation:   2
      Reason:                NotAvailable
      Status:                False
      Type:                  Available
      Last Transition Time:  2025-03-26T05:09:27Z
      Message:               Remote connection probe failed
      Observed Generation:   2
      Reason:                ProbeFailed
      Status:                False
      Type:                  RemoteConnectionProbe
      Last Transition Time:  2025-03-26T05:19:50Z
      Message:
      Observed Generation:   2
      Reason:                Ready
      Status:                True
      Type:                  InfrastructureReady
      Last Transition Time:  2025-03-26T05:09:27Z
      Message:               Control plane not yet initialized
      Observed Generation:   2
      Reason:                NotInitialized
      Status:                False
      Type:                  ControlPlaneInitialized
      Last Transition Time:  2025-03-26T05:09:27Z
      Message:               Control plane not yet initialized
      Observed Generation:   2
      Reason:                NotAvailable
      Status:                False
      Type:                  ControlPlaneAvailable
      Last Transition Time:  2025-03-26T05:09:27Z
      Message:               * MachineDeployment karthikkn-capi-powervs-md-0: 0 available replicas, at least 1 required (spec.strategy.rollout.maxUnavailable is 0, spec.replicas is 1)
      Observed Generation:   2
      Reason:                NotAvailable
      Status:                False
      Type:                  WorkersAvailable
      Last Transition Time:  2025-03-26T05:09:27Z
      Message:
      Observed Generation:   2
      Reason:                NotRollingOut
      Status:                False
      Type:                  RollingOut
      Last Transition Time:  2025-03-26T05:09:27Z
      Message:
      Observed Generation:   2
      Reason:                NotRemediating
      Status:                False
      Type:                  Remediating
      Last Transition Time:  2025-03-26T05:09:27Z
      Message:
      Observed Generation:   2
      Reason:                NotScalingDown
      Status:                False
      Type:                  ScalingDown
      Last Transition Time:  2025-03-26T05:09:27Z
      Message:               * KubeadmControlPlane karthikkn-capi-powervs-control-plane:
  * Scaling up from 1 to 3 replicas is blocked because:
    * waiting for control plane components to become healthy
    * waiting for etcd cluster to become healthy
      Observed Generation:   2
      Reason:                ScalingUp
      Status:                True
      Type:                  ScalingUp
      Last Transition Time:  2025-03-26T05:19:51Z
      Message:               * Machine karthikkn-capi-powervs-control-plane-pbxld:
  * InfrastructureReady:
    * IBMPowerVSMachine karthikkn-capi-powervs-control-plane-pbxld:
  * NodeHealthy: Waiting for Cluster control plane to be initialized
  * Control plane components: Waiting for Cluster control plane to be initialized
  * EtcdMemberHealthy: Waiting for Cluster control plane to be initialized
      Observed Generation:   2
      Reason:                NotReady
      Status:                False
      Type:                  ControlPlaneMachinesReady
      Last Transition Time:  2025-03-26T05:19:57Z
      Message:               * Machine karthikkn-capi-powervs-md-0-4bt5m-l65ms:
  * BootstrapConfigReady:
    * DataSecretAvailable: Waiting for Cluster control plane to be initialized
    * CertificatesAvailable: Condition not yet reported
  * InfrastructureReady:
    * IBMPowerVSMachine karthikkn-capi-powervs-md-0-4bt5m-l65ms:
  * NodeHealthy: Waiting for Cluster control plane to be initialized
      Observed Generation:   2
      Reason:                NotReady
      Status:                False
      Type:                  WorkerMachinesReady
      Last Transition Time:  2025-03-26T05:09:27Z
      Message:
      Observed Generation:   2
      Reason:                UpToDate
      Status:                True
      Type:                  ControlPlaneMachinesUpToDate
      Last Transition Time:  2025-03-26T05:09:27Z
      Message:
      Observed Generation:   2
      Reason:                UpToDate
      Status:                True
      Type:                  WorkerMachinesUpToDate
      Last Transition Time:  2025-03-26T05:09:27Z
      Message:
      Observed Generation:   2
      Reason:                NotPaused
      Status:                False
      Type:                  Paused
      Last Transition Time:  2025-03-26T05:09:27Z
      Message:
      Observed Generation:   2
      Reason:                NotDeleting
      Status:                False
      Type:                  Deleting
    Control Plane:
      Available Replicas:   0
      Desired Replicas:     3
      Ready Replicas:       0
      Replicas:             1
      Up To Date Replicas:  1
    Workers:
      Available Replicas:   0
      Desired Replicas:     1
      Ready Replicas:       0
      Replicas:             1
      Up To Date Replicas:  1


kubectl get cluster -o yaml

    v1beta2:
      conditions:
      - lastTransitionTime: "2025-03-26T05:09:27Z"
        message: |-
          * RemoteConnectionProbe: Remote connection probe failed
          * ControlPlaneAvailable: Control plane not yet initialized
          * WorkersAvailable:
            * MachineDeployment karthikkn-capi-powervs-md-0: 0 available replicas, at least 1 required (spec.strategy.rollout.maxUnavailable is 0, spec.replicas is 1)
        observedGeneration: 2
        reason: NotAvailable
        status: "False"
        type: Available
      - lastTransitionTime: "2025-03-26T05:09:27Z"
        message: Remote connection probe failed
        observedGeneration: 2
        reason: ProbeFailed
        status: "False"
        type: RemoteConnectionProbe
      - lastTransitionTime: "2025-03-26T05:19:50Z"
        message: ""
        observedGeneration: 2
        reason: Ready
        status: "True"
        type: InfrastructureReady
      - lastTransitionTime: "2025-03-26T05:09:27Z"
        message: Control plane not yet initialized
        observedGeneration: 2
        reason: NotInitialized
        status: "False"
        type: ControlPlaneInitialized
      - lastTransitionTime: "2025-03-26T05:09:27Z"
        message: Control plane not yet initialized
        observedGeneration: 2
        reason: NotAvailable
        status: "False"
        type: ControlPlaneAvailable
      - lastTransitionTime: "2025-03-26T05:09:27Z"
        message: '* MachineDeployment karthikkn-capi-powervs-md-0: 0 available replicas,
          at least 1 required (spec.strategy.rollout.maxUnavailable is 0, spec.replicas
          is 1)'
        observedGeneration: 2
        reason: NotAvailable
        status: "False"
        type: WorkersAvailable
      - lastTransitionTime: "2025-03-26T05:09:27Z"
        message: ""
        observedGeneration: 2
        reason: NotRollingOut
        status: "False"
        type: RollingOut
      - lastTransitionTime: "2025-03-26T05:09:27Z"
        message: ""
        observedGeneration: 2
        reason: NotRemediating
        status: "False"
        type: Remediating
      - lastTransitionTime: "2025-03-26T05:09:27Z"
        message: ""
        observedGeneration: 2
        reason: NotScalingDown
        status: "False"
        type: ScalingDown
      - lastTransitionTime: "2025-03-26T05:09:27Z"
        message: |-
          * KubeadmControlPlane karthikkn-capi-powervs-control-plane:
            * Scaling up from 1 to 3 replicas is blocked because:
              * waiting for control plane components to become healthy
              * waiting for etcd cluster to become healthy
        observedGeneration: 2
        reason: ScalingUp
        status: "True"
        type: ScalingUp
      - lastTransitionTime: "2025-03-26T05:19:51Z"
        message: "* Machine karthikkn-capi-powervs-control-plane-pbxld:\n  * InfrastructureReady:\n
          \   * IBMPowerVSMachine karthikkn-capi-powervs-control-plane-pbxld: \n  *
          NodeHealthy: Waiting for Cluster control plane to be initialized\n  * Control
          plane components: Waiting for Cluster control plane to be initialized\n
          \ * EtcdMemberHealthy: Waiting for Cluster control plane to be initialized"
        observedGeneration: 2
        reason: NotReady
        status: "False"
        type: ControlPlaneMachinesReady
      - lastTransitionTime: "2025-03-26T05:19:57Z"
        message: "* Machine karthikkn-capi-powervs-md-0-4bt5m-l65ms:\n  * BootstrapConfigReady:\n
          \   * DataSecretAvailable: Waiting for Cluster control plane to be initialized\n
          \   * CertificatesAvailable: Condition not yet reported\n  * InfrastructureReady:\n
          \   * IBMPowerVSMachine karthikkn-capi-powervs-md-0-4bt5m-l65ms: \n  * NodeHealthy:
          Waiting for Cluster control plane to be initialized"
        observedGeneration: 2
        reason: NotReady
        status: "False"
        type: WorkerMachinesReady
      - lastTransitionTime: "2025-03-26T05:09:27Z"
        message: ""
        observedGeneration: 2
        reason: UpToDate
        status: "True"
        type: ControlPlaneMachinesUpToDate
      - lastTransitionTime: "2025-03-26T05:09:27Z"
        message: ""
        observedGeneration: 2
        reason: UpToDate
        status: "True"
        type: WorkerMachinesUpToDate
      - lastTransitionTime: "2025-03-26T05:09:27Z"
        message: ""
        observedGeneration: 2
        reason: NotPaused
        status: "False"
        type: Paused
      - lastTransitionTime: "2025-03-26T05:09:27Z"
        message: ""
        observedGeneration: 2
        reason: NotDeleting
        status: "False"
        type: Deleting
      controlPlane:
        availableReplicas: 0
        desiredReplicas: 3
        readyReplicas: 0
        replicas: 1
        upToDateReplicas: 1
      workers:
        availableReplicas: 0
        desiredReplicas: 1
        readyReplicas: 0
        replicas: 1
        upToDateReplicas: 1

What did you expect to happen?

Properly indent the condition messages

Cluster API version

Used tilt with latest main branch on 26th March.

Kubernetes version

Client Version: v1.31.2
Kustomize Version: v5.4.2
Server Version: v1.31.0

Anything else you would like to add?

Also observed some changes in IBMPowerVSMachine resource where a Message filed is randomly generated
for v1beta2 which I feel is confusing. Not sure if I missed some configurations while setting.

kubectl describe ibmpowervsmachine 

Conditions:
    Last Transition Time:  2025-03-26T05:19:52Z
    Reason:                WaitingForIBMPowerVSImage
    Severity:              Info
    Status:                False
    Type:                  Ready
    Last Transition Time:  2025-03-26T05:19:52Z
    Reason:                WaitingForIBMPowerVSImage
    Severity:              Info
    Status:                False
    Type:                  InstanceReady
  v1beta2:
    Conditions:
      Last Transition Time:  2025-03-26T05:19:52Z
      Message:               * IBMPowerVSMachine karthikkn-capi-powervs-control-plane-pbxld:
      Observed Generation:   1
      Reason:                NotReady
      Status:                False
      Type:                  Ready
      Last Transition Time:  2025-03-26T05:19:52Z
      Message:
      Observed Generation:   1
      Reason:                WaitingForIBMPowerVSImage
      Status:                False
      Type:                  InstanceReady
      Last Transition Time:  2025-03-26T05:19:51Z
      Message:
      Observed Generation:   1
      Reason:                NotPaused
      Status:                False
      Type:                  Paused

Label(s) to be applied

/kind bug
One or more /area label. See https://github.com/kubernetes-sigs/cluster-api/labels?q=area for the list of labels.
@k8s-ci-robot k8s-ci-robot added kind/bug Categorizes issue or PR as related to a bug. needs-priority Indicates an issue lacks a `priority/foo` label and requires one. needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. labels Mar 26, 2025
@k8s-ci-robot
Copy link
Contributor

This issue is currently awaiting triage.

If CAPI contributors determine this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@Karthik-K-N
Copy link
Contributor Author

When I check the conditions I see

- lastTransitionTime: "2025-03-26T05:09:27Z"
        message: |-
          * KubeadmControlPlane karthikkn-capi-powervs-control-plane:
            * Scaling up from 1 to 3 replicas is blocked because:
              * waiting for control plane components to become healthy
              * waiting for etcd cluster to become healthy
        observedGeneration: 2
        reason: ScalingUp
        status: "True"
        type: ScalingUp

Properly displayed where as

- lastTransitionTime: "2025-03-26T05:37:53Z"
        message: "* Machine karthikkn-capi-powervs-control-plane-pbxld:\n  * InfrastructureReady:\n
          \   * IBMPowerVSMachine karthikkn-capi-powervs-control-plane-pbxld: \n  *
          NodeHealthy: Waiting for Cluster control plane to be initialized\n  * Control
          plane components: Waiting for Cluster control plane to be initialized\n
          \ * EtcdMemberHealthy: Waiting for Cluster control plane to be initialized"
        observedGeneration: 2
        reason: ReadyUnknown
        status: Unknown
        type: ControlPlaneMachinesReady

Only difference I see the multi line character (|-) infront of message

@Karthik-K-N
Copy link
Contributor Author

/cc @sbueringer @chrischdi

@Karthik-K-N
Copy link
Contributor Author

Also I feel the bullet points under Message fields should be properly indented

Image

@chrischdi
Copy link
Member

Regarding kubectl describe:

kubectl describe poorly formats/outputs several things (including multi-line strings).

E.g. it does not split up conditions and it is hard to see which line is part of which condition.

However, on top of that, kubectl describe does not handle \n well. It just prints them out and does not do any additional processing to it (e.g. no additional indent).

Regarding kubectl get -o yaml output:

I tried to reproduce your above output

      - lastTransitionTime: "2025-03-26T05:19:51Z"
        message: "* Machine karthikkn-capi-powervs-control-plane-pbxld:\n  * InfrastructureReady:\n
          \   * IBMPowerVSMachine karthikkn-capi-powervs-control-plane-pbxld: \n  *
          NodeHealthy: Waiting for Cluster control plane to be initialized\n  * Control
          plane components: Waiting for Cluster control plane to be initialized\n
          \ * EtcdMemberHealthy: Waiting for Cluster control plane to be initialized"
        observedGeneration: 2
        reason: NotReady
        status: "False"
        type: ControlPlaneMachinesReady

But was not able to. For me a multi-line message in conditionns (including ControlPlaneMachinesReady) always resulted in rendering the message with:

message: |-
  The
  multiline message

For me it always rendered to:

    - lastTransitionTime: "2025-03-25T19:00:42Z"
      message: |-
        * Machine quick-start-cluster-class-4c5fzs:
          * InfrastructureReady:
            * VirtualMachineProvisioned: Provisioning
          * NodeHealthy: Waiting for Cluster control plane to be initialized
          * Control plane components: Waiting for Cluster control plane to be initialized
          * EtcdMemberHealthy: Waiting for Cluster control plane to be initialized
      observedGeneration: 3
      reason: NotReady
      status: "False"
      type: ControlPlaneMachinesReady

What makes me curious in your output is the additional \ at some lines which do not escape anything. Maybe that leads to rendering it differently in kubectl.

I tried the following kubectl versions, all having the same output:

  • v1.32.2
  • v1.32.0
  • v1.31.6
  • v1.31.0
  • v1.30.10
  • v1.30.0

@Karthik-K-N
Copy link
Contributor Author

Regarding kubectl describe:

kubectl describe poorly formats/outputs several things (including multi-line strings).

E.g. it does not split up conditions and it is hard to see which line is part of which condition.

However, on top of that, kubectl describe does not handle \n well. It just prints them out and does not do any additional processing to it (e.g. no additional indent).

Regarding kubectl get -o yaml output:

I tried to reproduce your above output

  • lastTransitionTime: "2025-03-26T05:19:51Z"
    message: "* Machine karthikkn-capi-powervs-control-plane-pbxld:\n * InfrastructureReady:\n
    \ * IBMPowerVSMachine karthikkn-capi-powervs-control-plane-pbxld: \n *
    NodeHealthy: Waiting for Cluster control plane to be initialized\n * Control
    plane components: Waiting for Cluster control plane to be initialized\n
    \ * EtcdMemberHealthy: Waiting for Cluster control plane to be initialized"
    observedGeneration: 2
    reason: NotReady
    status: "False"
    type: ControlPlaneMachinesReady

But was not able to. For me a multi-line message in conditionns (including ControlPlaneMachinesReady) always resulted in rendering the message with:

message: |-
The
multiline message

For me it always rendered to:

- lastTransitionTime: "2025-03-25T19:00:42Z"
  message: |-
    * Machine quick-start-cluster-class-4c5fzs:
      * InfrastructureReady:
        * VirtualMachineProvisioned: Provisioning
      * NodeHealthy: Waiting for Cluster control plane to be initialized
      * Control plane components: Waiting for Cluster control plane to be initialized
      * EtcdMemberHealthy: Waiting for Cluster control plane to be initialized
  observedGeneration: 3
  reason: NotReady
  status: "False"
  type: ControlPlaneMachinesReady

What makes me curious in your output is the additional \ at some lines which do not escape anything. Maybe that leads to rendering it differently in kubectl.

I tried the following kubectl versions, all having the same output:

* v1.32.2

* v1.32.0

* v1.31.6

* v1.31.0

* v1.30.10

* v1.30.0

Thank you, I was trying with 1.29.3 V, May be I will upgrade the version and update the behavior.

@chrischdi
Copy link
Member

chrischdi commented Mar 26, 2025 •
edited
Loading

(For other readers) there's also discussion on the thread: https://kubernetes.slack.com/archives/C8TSNPY4T/p1742908238199499

It could also be that there is some weird escaping happening.

It would be grate to see the representation from kubectl get -o json to compare.

I doubt earlier versions of kubectl have a different output.

@chrischdi
Copy link
Member

Regarding:

Also observed some changes in IBMPowerVSMachine resource where a Message filed is randomly generated
for v1beta2 which I feel is confusing. Not sure if I missed some configurations while setting.

It could be that #11990 fixed this behaviour.

@sbueringer
Copy link
Member

sbueringer commented Mar 26, 2025 •
edited
Loading

Also I feel the bullet points under Message fields should be properly indented

There's nothing we can do here. This depends on terminal width and could only be done in kubectl

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
kind/bug Categorizes issue or PR as related to a bug. needs-priority Indicates an issue lacks a `priority/foo` label and requires one. needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@chrischdi @sbueringer @k8s-ci-robot @Karthik-K-N