Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for running with Kubernetes as quorum manager instead of ZooKeeper #737

Open
soenkeliebau opened this issue Jan 13, 2025 · 1 comment
Assignees

Comments

@soenkeliebau
Copy link
Member

soenkeliebau commented Jan 13, 2025

NiFi now supports running with Kubernetes as quorum manager instead of ZooKeeper. We should implement this and offer it as an option.

In principle this is just a few settings that could be overriden, but an entry needs to be written in an xml file, which we currently hard-code and effectively kill these overrides.

@nightkr
Copy link
Member

nightkr commented Mar 21, 2025

Looks like this has two components:

  1. Clustering coordination (https://nifi.apache.org/nifi-docs/administration-guide.html#kubernetes-clustering)

This just requires us to set nifi.cluster.leader.election.implementation, and give NiFi permission to manage Lease objects:

---
apiVersion: nifi.stackable.tech/v1alpha1
kind: NifiCluster
metadata:
  name: simple-nifi
spec:
  # [snip]
  nodes:
    configOverrides:
      nifi.properties:
        nifi.cluster.leader.election.implementation: KubernetesLeaderElectionManager
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: nifi-extra
rules:
  # https://nifi.apache.org/nifi-docs/administration-guide.html#kubernetes-clustering
  - apiGroups:
      - coordination.k8s.io
    resources:
      - leases
    verbs:
      - create
      - get
      - update
      # undocumented but required
      - patch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: nifi-extra
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: nifi-extra
subjects:
  - kind: ServiceAccount
    name: simple-nifi-serviceaccount

(Though obviously we should provide proper support, this is just illustrative of the scope of change we'd need to make.)

  1. Cluster state storage (https://nifi.apache.org/nifi-docs/administration-guide.html#kubernetes-configmap-cluster-state-provider)

This is configured by the XML file Sönke refers to in the OP. In theory the change scope should be similarly small, but this is indeed hard-coded and would require an operator change.

@lfrancke lfrancke moved this from Proposed to In Progress in Stackable End-to-End Coordination Mar 26, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: In Progress
Status: Refinement: In Progress
Development

No branches or pull requests

2 participants