Add Radius Flux Controller for GitOps support #8784

willdavsmith · 2025-03-10T16:43:15Z

Description

Add Radius Flux controller and tests
Update test workflows to install and use Flux and Gitea
Add bicep container and changes to allow for Bicep to run server-side in controller pod

Note: I expect the functional tests to fail for this PR because I'm updating the functional test workflow. Here's a link to a successful action run: https://github.com/radius-project/radius/actions/runs/14074009492

Design PR: radius-project/design-notes#79
Docs PR: radius-project/docs#1408

Type of change

This pull request fixes a bug in Radius and has an approved issue (issue link required).
This pull request adds or changes features of Radius and has an approved issue (issue link required).
This pull request is a minor refactor, code cleanup, test improvement, or other maintenance task and doesn't change the functionality of Radius (issue link optional).

Fixes: #6689

Contributor checklist

Please verify that the PR meets the following requirements, where applicable:

An overview of proposed schema changes is included in a linked GitHub issue.
- Yes
- Not applicable
A design document PR is created in the design-notes repository, if new APIs are being introduced.
- Yes
- Not applicable
The design document has been reviewed and approved by Radius maintainers/approvers.
- Yes
- Not applicable
A PR for the samples repository is created, if existing samples are affected by the changes in this PR.
- Yes
- Not applicable
A PR for the documentation repository is created, if the changes in this PR affect the documentation or any user facing updates are made.
- Yes
- Not applicable
A PR for the recipes repository is created, if existing recipes are affected by the changes in this PR.
- Yes
- Not applicable

github-actions · 2025-03-10T16:57:32Z

Unit Tests

3 688 tests +90 3 686 ✅ +92 6m 42s ⏱️ -10s
296 suites ± 0 2 💤 ± 0
1 files ± 0 0 ❌ - 2

Results for commit cc19e18. ± Comparison against base commit 4f8900b.

♻️ This comment has been updated with latest results.

brooke-hamilton

Awesome PR 🚀

.github/actions/install-gitea/install-gitea.sh

brooke-hamilton

I pressed the button on the previous review before finished. Here is part 2. I'm still reviewing so I will add more in another section.

deploy/images/bicep/Dockerfile

deploy/images/controller/Dockerfile

go.mod

pkg/cli/bicep/build.go

deploy/images/controller/Dockerfile

ytimocin

First round of my review.

ytimocin · 2025-03-12T17:10:33Z

.github/actions/create-kind-cluster/action.yaml

+        nodes:
+        - role: control-plane
+          extraPortMappings:
+            - containerPort: 30080
+              hostPort: 30080
+              protocol: TCP


What is the reason behind this addition? Do we need to add some comments here?

this is for Gitea. I added a comment

ytimocin · 2025-03-12T17:11:49Z

.github/actions/install-gitea/gitea-config.yaml

+service:
+  http:
+    type: NodePort
+    nodePort: 30080
+  ssh:
+    type: ClusterIP
+    port: 22


Now it is kind of making sense what you did in the kind cluster creation action. But we may still need to add some comments to the action so that people can understand what it is for.

ytimocin · 2025-03-12T17:13:43Z

.github/actions/install-gitea/install-gitea.sh

+fi
+
+if [ -z "$GITEA_ACCESS_TOKEN_NAME" ]; then
+  echo


This should be removed

I mean the empty echo

ytimocin · 2025-03-12T17:14:21Z

.github/actions/install-gitea/install-gitea.sh

+helm repo add gitea-charts https://dl.gitea.io/charts/
+helm repo update


Are we always going to install the latest version? Can it cause any breaking changes at some point?

true - changed this to install 11.0.0

ytimocin · 2025-03-12T17:17:28Z

.github/actions/install-gitea/install-gitea.sh

+output=$(kubectl exec -it $gitea_pod -n gitea -- gitea admin user generate-access-token --username $GITEA_USERNAME --token-name $GITEA_ACCESS_TOKEN_NAME  --scopes "write:repository,write:user" --raw)
+echo $output
+
+echo "gitea-access-token=$output" >>$GITHUB_OUTPUT


This sh file can actually be tested locally.

ytimocin · 2025-03-12T17:32:05Z

deploy/images/controller/Dockerfile

@@ -1,17 +1,20 @@
-# Use distroless image which already includes ca-certificates
-FROM gcr.io/distroless/static:nonroot
+FROM ubuntu:latest


We should check the size difference since we are changing the base image (I believe ubuntu is probably much larger than what we have now)

We should also lock the version to not have breaking changes

Updated this to use debian-slim and locked the version.

deploy/images/controller/Dockerfile

ytimocin · 2025-03-12T17:36:59Z

pkg/cli/bicep/types.go

@@ -38,12 +41,13 @@ var _ Interface = (*Impl)(nil)

 // Impl is the implementation of Interface.
 type Impl struct {
+	filesystem filesystem.FileSystem


Should this be fileSystem?

yes, updated

ytimocin · 2025-03-12T17:38:50Z

pkg/cli/bicep/types.go

+// Build runs `rad-bicep build` with the given arguments.
+func (i *Impl) Build(args ...string) (map[string]any, error) {
+	buildArgs := make([]string, len(args)+1)
+	buildArgs[0] = "build"
+	copy(buildArgs[1:], args)
+
+	return runBicepJSON(buildArgs...)
+}
+
+// BuildParams runs `rad-bicep build-params` with the given arguments.
+func (i *Impl) BuildParams(args ...string) (map[string]any, error) {
+	buildParamsArgs := make([]string, len(args)+1)
+	buildParamsArgs[0] = "build-params"
+	copy(buildParamsArgs[1:], args)
+
+	return runBicepJSON(buildParamsArgs...)
+}


This could be one generic function with the initial param passed in like: BuildCommand("build-params" or "build", args).

I deleted these functions and just wrote Call() instead

ytimocin · 2025-03-12T17:39:24Z

pkg/cli/bicep/types.go

+
+	version := regexp.MustCompile(SemanticVersionRegex).FindString(string(bytes))
+	if version == "" {
+		return fmt.Sprintf("unknown (failed to parse bicep version from %q)", string(bytes))


Shouldn't this also return an error? Like func (i *Impl) Version() (string, error).

this code is copied over from pkg/cli/bicep/build.go - I don't want to change it in case it breaks something

kachawla

Are all the changes in this PR strictly dependent on one another? Ideally, we should aim to break them down into smaller, more focused PRs, keeping each one as minimal as possible for easier review.

superbeeny · 2025-03-19T16:26:56Z

deploy/images/bicep/Dockerfile

@@ -0,0 +1,14 @@
+FROM alpine:latest


Do we want to pin this to a specific version and make it more deterministic

good call - changing to 3.21.3

superbeeny · 2025-03-19T16:28:12Z

deploy/images/controller/Dockerfile

+# Use Ubuntu image to enable Bicep CLI.
+# Switch to something more lightweight when we can find a 
+# base image that supports running Bicep. 
+FROM ubuntu:latest


Same as the above comment of pinning to a specific version instead of latest

superbeeny · 2025-03-19T16:32:53Z

pkg/cli/bicep/types.go

+
+// Version returns the version of Bicep installed on the local machine,
+// or an error if Bicep cannot be found or is not a valid version.
+func (i *Impl) Version() string {


I think we should return the version or an error and not co-mingle the return

this code is copied over from pkg/cli/bicep/build.go - I don't want to change it in case it breaks something

superbeeny · 2025-03-19T16:33:46Z

pkg/cli/filesystem/memmapfs.go

+	file, exists := m.InternalFileSystem[name]
+	if !exists {
+		return nil, fmt.Errorf("file %s does not exist", name)
+	}


nit: newline

superbeeny · 2025-03-19T16:34:21Z

pkg/cli/filesystem/memmapfs.go

+	_, exists := m.InternalFileSystem[name]
+	if !exists {
+		return fmt.Errorf("file %s does not exist", name)
+	}


nit: newline

Signed-off-by: willdavsmith <[email protected]>

radius-functional-tests · 2025-03-26T00:49:00Z

Radius functional test overview

🔍 Go to test action run

Name	Value
Repository	radius-project/radius
Commit ref	`361e965`
Unique ID	funcfd7cee76c1
Image tag	pr-funcfd7cee76c1

Click here to see the list of tools in the current test run

gotestsum 1.12.0
KinD: v0.20.0
Dapr:
Azure KeyVault CSI driver: 1.4.2
Azure Workload identity webhook: 1.3.0
Bicep recipe location ghcr.io/radius-project/dev/test/testrecipes/test-bicep-recipes/<name>:pr-funcfd7cee76c1
Terraform recipe location http://tf-module-server.radius-test-tf-module-server.svc.cluster.local/<name>.zip (in cluster)
applications-rp test image location: ghcr.io/radius-project/dev/applications-rp:pr-funcfd7cee76c1
dynamic-rp test image location: ghcr.io/radius-project/dev/dynamic-rp:pr-funcfd7cee76c1
controller test image location: ghcr.io/radius-project/dev/controller:pr-funcfd7cee76c1
ucp test image location: ghcr.io/radius-project/dev/ucpd:pr-funcfd7cee76c1
deployment-engine test image location: ghcr.io/radius-project/deployment-engine:latest

Test Status

⌛ Building Radius and pushing container images for functional tests...
✅ Container images build succeeded
⌛ Publishing Bicep Recipes for functional tests...
✅ Recipe publishing succeeded
⌛ Starting corerp-cloud functional tests...
⌛ Starting ucp-cloud functional tests...
❌ Failed to install Radius for corerp-cloud functional test. Please check the logs for more details
❌ Failed to install Radius for ucp-cloud functional test. Please check the logs for more details
❌ corerp-cloud functional test failed. Please check the logs for more details
❌ ucp-cloud functional test failed. Please check the logs for more details

pkg/controller/reconciler/flux_controller.go

Signed-off-by: willdavsmith <[email protected]>

sk593

brooke-hamilton

🚢

radius-functional-tests · 2025-03-27T00:59:25Z

Radius functional test overview

🔍 Go to test action run

Name	Value
Repository	radius-project/radius
Commit ref	`3526ac8`
Unique ID	func9de8fbacaa
Image tag	pr-func9de8fbacaa

Click here to see the list of tools in the current test run

gotestsum 1.12.0
KinD: v0.20.0
Dapr:
Azure KeyVault CSI driver: 1.4.2
Azure Workload identity webhook: 1.3.0
Bicep recipe location ghcr.io/radius-project/dev/test/testrecipes/test-bicep-recipes/<name>:pr-func9de8fbacaa
Terraform recipe location http://tf-module-server.radius-test-tf-module-server.svc.cluster.local/<name>.zip (in cluster)
applications-rp test image location: ghcr.io/radius-project/dev/applications-rp:pr-func9de8fbacaa
dynamic-rp test image location: ghcr.io/radius-project/dev/dynamic-rp:pr-func9de8fbacaa
controller test image location: ghcr.io/radius-project/dev/controller:pr-func9de8fbacaa
ucp test image location: ghcr.io/radius-project/dev/ucpd:pr-func9de8fbacaa
deployment-engine test image location: ghcr.io/radius-project/deployment-engine:latest

Test Status

⌛ Building Radius and pushing container images for functional tests...
✅ Container images build succeeded
⌛ Publishing Bicep Recipes for functional tests...
✅ Recipe publishing succeeded
⌛ Starting ucp-cloud functional tests...
⌛ Starting corerp-cloud functional tests...
❌ Failed to install Radius for ucp-cloud functional test. Please check the logs for more details
❌ ucp-cloud functional test failed. Please check the logs for more details
❌ corerp-cloud functional test cancelled. Please check the logs for more details
⌛ Starting ucp-cloud functional tests...
⌛ Starting corerp-cloud functional tests...
❌ Failed to install Radius for ucp-cloud functional test. Please check the logs for more details
❌ ucp-cloud functional test failed. Please check the logs for more details
❌ corerp-cloud functional test cancelled. Please check the logs for more details

radius-functional-tests · 2025-03-28T20:33:40Z

Radius functional test overview

🔍 Go to test action run

Name	Value
Repository	radius-project/radius
Commit ref	`cc19e18`
Unique ID	func115def619d
Image tag	pr-func115def619d

Click here to see the list of tools in the current test run

gotestsum 1.12.0
KinD: v0.20.0
Dapr:
Azure KeyVault CSI driver: 1.4.2
Azure Workload identity webhook: 1.3.0
Bicep recipe location ghcr.io/radius-project/dev/test/testrecipes/test-bicep-recipes/<name>:pr-func115def619d
Terraform recipe location http://tf-module-server.radius-test-tf-module-server.svc.cluster.local/<name>.zip (in cluster)
applications-rp test image location: ghcr.io/radius-project/dev/applications-rp:pr-func115def619d
dynamic-rp test image location: ghcr.io/radius-project/dev/dynamic-rp:pr-func115def619d
controller test image location: ghcr.io/radius-project/dev/controller:pr-func115def619d
ucp test image location: ghcr.io/radius-project/dev/ucpd:pr-func115def619d
deployment-engine test image location: ghcr.io/radius-project/deployment-engine:latest

Test Status

⌛ Building Radius and pushing container images for functional tests...
✅ Container images build succeeded
⌛ Publishing Bicep Recipes for functional tests...
✅ Recipe publishing succeeded
⌛ Starting corerp-cloud functional tests...
⌛ Starting ucp-cloud functional tests...
❌ Failed to install Radius for corerp-cloud functional test. Please check the logs for more details
❌ Failed to install Radius for ucp-cloud functional test. Please check the logs for more details
❌ corerp-cloud functional test failed. Please check the logs for more details
❌ ucp-cloud functional test failed. Please check the logs for more details

willdavsmith requested review from a team as code owners March 10, 2025 16:43

willdavsmith requested a deployment to functional-tests March 10, 2025 16:43 — with GitHub Actions Waiting

willdavsmith temporarily deployed to publish-bicep March 10, 2025 16:43 — with GitHub Actions Inactive

willdavsmith requested a deployment to functional-tests March 10, 2025 20:41 — with GitHub Actions Waiting

willdavsmith temporarily deployed to publish-bicep March 10, 2025 20:41 — with GitHub Actions Inactive

brooke-hamilton reviewed Mar 12, 2025

View reviewed changes

.github/actions/install-gitea/install-gitea.sh Show resolved Hide resolved

brooke-hamilton requested changes Mar 12, 2025

View reviewed changes

ytimocin reviewed Mar 12, 2025

View reviewed changes

kachawla reviewed Mar 13, 2025

View reviewed changes

willdavsmith requested a deployment to functional-tests March 13, 2025 20:43 — with GitHub Actions Waiting

willdavsmith temporarily deployed to publish-bicep March 13, 2025 20:43 — with GitHub Actions Inactive

willdavsmith requested a deployment to functional-tests March 13, 2025 21:14 — with GitHub Actions Waiting

willdavsmith temporarily deployed to publish-bicep March 13, 2025 21:14 — with GitHub Actions Inactive

willdavsmith requested a deployment to functional-tests March 13, 2025 23:06 — with GitHub Actions Waiting

willdavsmith temporarily deployed to publish-bicep March 13, 2025 23:06 — with GitHub Actions Inactive

brooke-hamilton mentioned this pull request Mar 19, 2025

REQUEST: Membership for brooke-hamilton radius-project/community#80

Open

4 tasks

superbeeny reviewed Mar 19, 2025

View reviewed changes

willdavsmith requested a review from a team as a code owner March 19, 2025 18:58

willdavsmith requested a deployment to functional-tests March 19, 2025 18:59 — with GitHub Actions Waiting

willdavsmith temporarily deployed to publish-bicep March 19, 2025 18:59 — with GitHub Actions Inactive

willdavsmith requested a deployment to functional-tests March 19, 2025 19:19 — with GitHub Actions Waiting

willdavsmith temporarily deployed to publish-bicep March 19, 2025 19:19 — with GitHub Actions Inactive

willdavsmith force-pushed the flux-controller branch from 0b2241a to 90e98b3 Compare March 19, 2025 19:41

willdavsmith requested a deployment to functional-tests March 19, 2025 19:41 — with GitHub Actions Waiting

willdavsmith temporarily deployed to publish-bicep March 19, 2025 19:41 — with GitHub Actions Inactive

willdavsmith requested a deployment to functional-tests March 19, 2025 20:04 — with GitHub Actions Waiting

willdavsmith temporarily deployed to publish-bicep March 19, 2025 20:04 — with GitHub Actions Inactive

willdavsmith requested a deployment to functional-tests March 19, 2025 20:23 — with GitHub Actions Waiting

willdavsmith requested a deployment to functional-tests March 26, 2025 00:08 — with GitHub Actions Waiting

willdavsmith temporarily deployed to publish-bicep March 26, 2025 00:08 — with GitHub Actions Inactive

PR

4e04d45

Signed-off-by: willdavsmith <[email protected]>

willdavsmith requested a deployment to functional-tests March 26, 2025 00:14 — with GitHub Actions Waiting

willdavsmith temporarily deployed to publish-bicep March 26, 2025 00:14 — with GitHub Actions Inactive

PR

6caf0f2

Signed-off-by: willdavsmith <[email protected]>

willdavsmith requested a deployment to functional-tests March 26, 2025 00:33 — with GitHub Actions Waiting

willdavsmith temporarily deployed to publish-bicep March 26, 2025 00:33 — with GitHub Actions Inactive

Merge branch 'main' into flux-controller

361e965

willdavsmith temporarily deployed to functional-tests March 26, 2025 00:47 — with GitHub Actions Inactive

willdavsmith temporarily deployed to publish-bicep March 26, 2025 00:47 — with GitHub Actions Inactive

sk593 reviewed Mar 26, 2025

View reviewed changes

pkg/controller/reconciler/flux_controller.go Outdated Show resolved Hide resolved

Merge branch 'main' into flux-controller

683caa0

willdavsmith requested a deployment to functional-tests March 26, 2025 21:27 — with GitHub Actions Waiting

willdavsmith temporarily deployed to publish-bicep March 26, 2025 21:27 — with GitHub Actions Inactive

removing bicep restore

a031570

Signed-off-by: willdavsmith <[email protected]>

willdavsmith requested a deployment to functional-tests March 26, 2025 21:36 — with GitHub Actions Waiting

willdavsmith temporarily deployed to publish-bicep March 26, 2025 21:37 — with GitHub Actions Inactive

willdavsmith requested a review from sk593 March 26, 2025 22:06

sk593 approved these changes Mar 26, 2025

View reviewed changes

Merge branch 'main' into flux-controller

3526ac8

willdavsmith temporarily deployed to functional-tests March 26, 2025 23:57 — with GitHub Actions Inactive

willdavsmith temporarily deployed to publish-bicep March 26, 2025 23:57 — with GitHub Actions Inactive

brooke-hamilton approved these changes Mar 27, 2025

View reviewed changes

Merge branch 'main' into flux-controller

cc19e18

willdavsmith temporarily deployed to functional-tests March 28, 2025 20:19 — with GitHub Actions Inactive

willdavsmith temporarily deployed to publish-bicep March 28, 2025 20:19 — with GitHub Actions Inactive

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Radius Flux Controller for GitOps support #8784

Add Radius Flux Controller for GitOps support #8784

willdavsmith commented Mar 10, 2025 •

edited

Loading

github-actions bot commented Mar 10, 2025 •

edited

Loading

brooke-hamilton left a comment •

edited

Loading

brooke-hamilton left a comment

ytimocin left a comment

ytimocin Mar 12, 2025

willdavsmith Mar 20, 2025

ytimocin Mar 12, 2025

ytimocin Mar 12, 2025

ytimocin Mar 12, 2025

willdavsmith Mar 20, 2025

ytimocin Mar 12, 2025

willdavsmith Mar 20, 2025

ytimocin Mar 12, 2025

ytimocin Mar 12, 2025 •

edited

Loading

willdavsmith Mar 25, 2025

ytimocin Mar 12, 2025

willdavsmith Mar 25, 2025

ytimocin Mar 12, 2025

willdavsmith Mar 20, 2025 •

edited

Loading

ytimocin Mar 12, 2025

willdavsmith Mar 20, 2025

kachawla left a comment

superbeeny Mar 19, 2025

willdavsmith Mar 20, 2025

superbeeny Mar 19, 2025

superbeeny Mar 19, 2025

willdavsmith Mar 20, 2025

superbeeny Mar 19, 2025

superbeeny Mar 19, 2025

radius-functional-tests bot commented Mar 26, 2025 •

edited

Loading

sk593 left a comment

brooke-hamilton left a comment

radius-functional-tests bot commented Mar 27, 2025 •

edited

Loading

radius-functional-tests bot commented Mar 28, 2025 •

edited

Loading

		helm repo add gitea-charts https://dl.gitea.io/charts/
		helm repo update

Add Radius Flux Controller for GitOps support #8784

Are you sure you want to change the base?

Add Radius Flux Controller for GitOps support #8784

Conversation

willdavsmith commented Mar 10, 2025 • edited Loading

Description

Type of change

Contributor checklist

github-actions bot commented Mar 10, 2025 • edited Loading

Unit Tests

brooke-hamilton left a comment • edited Loading

Choose a reason for hiding this comment

brooke-hamilton left a comment

Choose a reason for hiding this comment

ytimocin left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ytimocin Mar 12, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

willdavsmith Mar 20, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kachawla left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

radius-functional-tests bot commented Mar 26, 2025 • edited Loading

Radius functional test overview

Test Status

sk593 left a comment

Choose a reason for hiding this comment

brooke-hamilton left a comment

Choose a reason for hiding this comment

radius-functional-tests bot commented Mar 27, 2025 • edited Loading

Radius functional test overview

Test Status

radius-functional-tests bot commented Mar 28, 2025 • edited Loading

Radius functional test overview

Test Status

willdavsmith commented Mar 10, 2025 •

edited

Loading

github-actions bot commented Mar 10, 2025 •

edited

Loading

brooke-hamilton left a comment •

edited

Loading

ytimocin Mar 12, 2025 •

edited

Loading

willdavsmith Mar 20, 2025 •

edited

Loading

radius-functional-tests bot commented Mar 26, 2025 •

edited

Loading

radius-functional-tests bot commented Mar 27, 2025 •

edited

Loading

radius-functional-tests bot commented Mar 28, 2025 •

edited

Loading