kgateway-waypoint: initial implementation #10783

stevenctl · 2025-03-07T21:01:02Z

Description

This PR adds support for a new GatewayClass kgateway-waypoint, which can be use in place of Istio's default Waypoint implementation.

There are some sample YAMLs included, but the basic flow is:

Deploy a Gateway with this class
To route traffic through the gateway, label either a Namespace or a Service with istio.io/use-waypoint=<gateway name (and optionally istio.io/use-waypoint-namespace=<ns the gw is in> for out-of-namespace waypoints)
Attach policy like HTTPRoute by parenting either to the Gateway (standard) or the Service (GAMMA-style)

This PR only supports the most basic features of a Waypoint:

Capturing to-service traffic in a single cluster
Attaching HTTPRoute to the Service or Gateway

Potential follow ups include:

TCPRoute
AuthorizationPolicy
Multi-Cluster
ServiceEntry capture and backends

API changes

Add new GatewayClass named kgateway-waypoint
(Helm) Move service.kind out of gateway and onto the gatewayClass and waypointClass values.
(Helm) Allow adding labels/annotation to GatewayClass and the generated podTemplate for each gateway class

Code changes

Helm

Change values to allow per-class customizations
Make templates for GatewayClass and GatewayParameters support passing in values for each class type (standard kgateway and the kgateway-waypoint)

Core

Refactor initialization of Routes and Gateway indexes to happen within CommonCollections rather than in proxy_syncer.
Move some collections into CommonCollections
Make RouteIndex handle the parent group/kind so we can do service-targeted policy.
Remove isOurGw and ExtraGatewayClasses in favor of a statically defined Set of known classes. This can be expanded later to read from an env var, but we shouldn't be passing a closure around for this.
Start actually applying the NetworkFilter plugins on TCP filter chains.
Gateway Queries just hold a reference to CommonCollections so that plugins can reference GatewayQueries (the routes field on it's own won't be ready till after plugins are initialized).

Waypoint Specific

Add a waypoint plugin that provides a new translator
Add some additional queries for waypoints for waypoint attachment and service-attached routes
Add a sandwich plugin that can be added at the Listener level to enable PROXY protocol inbound, and restore the client identity authenticated by zTunnel (Waypoint Sandwiches)

CI changes

E2E added to cluster-two job

Docs changes

Added some samples
README for the samples

Context

#10453

Testing steps

Unit tests against translation in/out
E2E tests for simple traffic flows and basic policy
To test on your own, see the examples/waypoint directory.

Checklist:

I have performed a self-review of my own code
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation
I have added tests that prove my fix is effective or that my feature works

yuval-k · 2025-03-11T13:17:52Z

internal/kgateway/extensions2/plugins/waypoint/waypointquery/service_model.go

+func (s Service) IsHeadless() bool {
+	switch o := s.Object.(type) {
+	case *corev1.Service:
+		return o.Spec.ClusterIP == "None"


Suggested change

return o.Spec.ClusterIP == "None"

return o.Spec.ClusterIP == corev1.ClusterIPNone

internal/kgateway/setup/setup.go

yuval-k · 2025-03-11T14:03:19Z

Remove isOurGw and ExtraGatewayClasses in favor of a statically defined Set of known classes. This can be expanded later to read from an env var, but we shouldn't be passing a closure around for this.

For context, our goal is to eventually use a the controller name on the gateway class to know if we own a gateway class (instead of the gateway class name); so with that in mind a closure seemed like an easier way to get there

stevenctl · 2025-03-11T17:21:55Z

eventually use a the controller name on the gateway class to know if we own a gateway class

Sounds good. Assuming we supported other gateway classes, I'd imagine it would be:

Helm install time adds a new class controlled by our controller
We have a plugin that contributes a translator for that class

Do we really need to have a closure? At the very least, we could probably move where that closure starts getting passed down; it seemed like it was several layers deep but only used in the GatewayIndex. If the list of (or single) controller name is env-driven, it could just be a func in wellknown. If settings driven I guess a closure or the list could be passed around.

stevenctl · 2025-03-11T17:27:02Z

so with that in mind a closure seemed like an easier way to get there

Should I revert this in the short-term?

stevenctl · 2025-03-11T18:29:20Z

@yuval-k you can check 1747e25 for the controller name thing, happy to revert and split it out if it's too much for this one PR.

PetrMc

this is a huge PR. LGTM

PetrMc · 2025-03-12T15:15:04Z

install/helm/kgateway/values.yaml

+waypointClass:
+  enabled: true
+  name: "kgateway-waypoint"
+  description: "KGateway Waypoint Controller"


according to the docs site it seems that kgateway is always lower-case unless the sentence starting with it. Should we unify?

yuval-k

comments re-init code

internal/kgateway/controller/controller.go

yuval-k · 2025-03-12T18:20:23Z

internal/kgateway/controller/controller.go

+			gwl := &apiv1.GatewayList{}
+			if err := cli.List(ctx, gwl); err != nil {
+				return nil
+			}
+
+			var reqs []reconcile.Request
+			for _, gw := range gwl.Items {
+				if string(gw.Spec.GatewayClassName) == gc.Name {


consider: index gw by gateway class name; though not sure it is a big deal in terms of perf

done; if we have a ton of Istio gateways/waypoints then it will matter

#10829

This and the related pieces are moved into this smaller PR.

stevenctl · 2025-03-12T21:02:12Z

@PetrMc

this is a huge PR

You're right!

I've opened #10822 and #10829 to shrink it a bit. I also plan on peeling off the changes to InitCollections/InitPlugins on common collections

stevenctl force-pushed the stevenctl/waypoint-basic branch 5 times, most recently from a349fb4 to 229f833 Compare March 10, 2025 23:22

nfuden requested review from timflannagan and lgadban March 10, 2025 23:23

stevenctl force-pushed the stevenctl/waypoint-basic branch from 257a5cb to 77c095d Compare March 11, 2025 04:16

yuval-k reviewed Mar 11, 2025

View reviewed changes

internal/kgateway/setup/setup.go Outdated Show resolved Hide resolved

stevenctl added 17 commits March 11, 2025 11:27

apply custom NetworkFilters on TCP chains

74083cc

warn when translator isn't enabled

7d8083e

go mod tidy

828570a

sandwich plugin: decap PROXY and load TLV

73a2d4c

base waypoint translator

efa0ca4

helm for Gateway and GatewayClass

cc83ad8

simplify initialization code

192b187

implement waypoint queries for routes and services

eb1edc3

a ton of small fixes

9dad9dd

some example yamls

32e168f

fix service route attachment

ff17c47

gen

2cea4eb

fix translator test setup

796efa9

refactor translator harness to be reusable

f3eb049

test translation

bfb6b1b

remove unsed filter on one test

7205d1a

lint

29f50f7

stevenctl added 7 commits March 11, 2025 11:27

e2e tests

bf66a15

lint

71d4948

update samples

fb5b493

try to fix regex for tests

7f8e0f6

e2e fixes

68d6139

label util for tests

3aebd8d

use controller name to drive is our gw

1747e25

stevenctl force-pushed the stevenctl/waypoint-basic branch from 7dc87fb to 1747e25 Compare March 11, 2025 18:28

stevenctl linked an issue Mar 11, 2025 that may be closed by this pull request

Basic Waypoint Support #10806

Open

3 tasks

stevenctl requested a review from PetrMc March 11, 2025 20:38

PetrMc approved these changes Mar 12, 2025

View reviewed changes

yuval-k reviewed Mar 12, 2025

View reviewed changes

This was referenced Mar 12, 2025

apply plugin network filters to TCP filter chains #10822

Open

use controller name to find relevant Gateways #10829

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

kgateway-waypoint: initial implementation #10783

kgateway-waypoint: initial implementation #10783

stevenctl commented Mar 7, 2025 •

edited

Loading

yuval-k Mar 11, 2025

yuval-k commented Mar 11, 2025

stevenctl commented Mar 11, 2025

stevenctl commented Mar 11, 2025

stevenctl commented Mar 11, 2025 •

edited

Loading

PetrMc left a comment

PetrMc Mar 12, 2025

yuval-k left a comment

yuval-k Mar 12, 2025

stevenctl Mar 12, 2025

stevenctl Mar 12, 2025

stevenctl commented Mar 12, 2025

	return o.Spec.ClusterIP == "None"
	return o.Spec.ClusterIP == corev1.ClusterIPNone

kgateway-waypoint: initial implementation #10783

Are you sure you want to change the base?

kgateway-waypoint: initial implementation #10783

Conversation

stevenctl commented Mar 7, 2025 • edited Loading

Description

API changes

Code changes

CI changes

Docs changes

Context

Testing steps

Checklist:

yuval-k Mar 11, 2025

Choose a reason for hiding this comment

yuval-k commented Mar 11, 2025

stevenctl commented Mar 11, 2025

stevenctl commented Mar 11, 2025

stevenctl commented Mar 11, 2025 • edited Loading

PetrMc left a comment

Choose a reason for hiding this comment

PetrMc Mar 12, 2025

Choose a reason for hiding this comment

yuval-k left a comment

Choose a reason for hiding this comment

yuval-k Mar 12, 2025

Choose a reason for hiding this comment

stevenctl Mar 12, 2025

Choose a reason for hiding this comment

stevenctl Mar 12, 2025

Choose a reason for hiding this comment

stevenctl commented Mar 12, 2025

stevenctl commented Mar 7, 2025 •

edited

Loading

stevenctl commented Mar 11, 2025 •

edited

Loading