Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add EventTransform API types #8447

Merged
merged 4 commits into from
Feb 11, 2025
Merged

Conversation

pierDipi
Copy link
Member

@pierDipi pierDipi commented Feb 10, 2025

Event transformations are essential to connect different systems, with the EventTransform we will support this use cases where a generic and declarative event transformation can be used and we would not require custom code to do that.

The API can be used in different places, between a source/webhook and a broker, between a broker and a sink or even leverage the built-in Trigger transformation feature with response.

source or webhook -> transform -> broker

trigger -> transform -> sink or webhook

broker -> trigger -> (request)  -> transform
broker <- trigger <- (response) <-

Related to:

We would also solve use cases like this https://cloud-native.slack.com/archives/C06AH2C3K8B/p1731786548587349 where a sink is OpenAI compatible LLM endpoint without writing custom code.

First commit is the main commit, rest is codegen and fixes

Proposed Changes

Pre-review Checklist

  • At least 80% unit test coverage
  • E2E tests for any new behavior
  • Docs PR for any user-facing impact
  • Spec PR for any new API feature
  • Conformance test for any change to the spec

Release Note

Add EventTransform API types.

Docs

Signed-off-by: Pierangelo Di Pilato <[email protected]>
Signed-off-by: Pierangelo Di Pilato <[email protected]>
@knative-prow knative-prow bot added the size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. label Feb 10, 2025
@knative-prow knative-prow bot requested review from aslom and creydr February 10, 2025 09:17
@knative-prow knative-prow bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Feb 10, 2025
@pierDipi pierDipi changed the title Add EventTransform API [WIP] Add EventTransform API Feb 10, 2025
@knative-prow knative-prow bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Feb 10, 2025
Copy link

codecov bot commented Feb 10, 2025

Codecov Report

Attention: Patch coverage is 33.33333% with 90 lines in your changes missing coverage. Please review.

Project coverage is 63.92%. Comparing base (4a6e7d2) to head (90e86eb).
Report is 2 commits behind head on main.

Files with missing lines Patch % Lines
...apis/eventing/v1alpha1/eventtransform_lifecycle.go 50.00% 39 Missing and 4 partials ⚠️
...pis/eventing/v1alpha1/eventtransform_validation.go 0.00% 41 Missing ⚠️
pkg/apis/eventing/v1alpha1/eventtransform_types.go 0.00% 6 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #8447      +/-   ##
==========================================
- Coverage   64.10%   63.92%   -0.18%     
==========================================
  Files         388      391       +3     
  Lines       23363    23498     +135     
==========================================
+ Hits        14976    15021      +45     
- Misses       7591     7677      +86     
- Partials      796      800       +4     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Signed-off-by: Pierangelo Di Pilato <[email protected]>
@pierDipi
Copy link
Member Author

/test unit-tests

@pierDipi pierDipi changed the title [WIP] Add EventTransform API Add EventTransform API Feb 10, 2025
@knative-prow knative-prow bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Feb 10, 2025
@pierDipi
Copy link
Member Author

/test reconciler-tests

1 similar comment
@pierDipi
Copy link
Member Author

/test reconciler-tests

@pierDipi pierDipi changed the title Add EventTransform API Add EventTransform API types Feb 10, 2025

func (js *JsonataEventTransformationSpec) Validate(context.Context) *apis.FieldError {
// Jsonata parsers for Go are not maintained, therefore, we will not parse the expression here.
// The downside is that the errors will only be present in the status of the EventTransform resource.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is not the end of the world, if reflected broken expressions via "runtime errors" propagated to the status of the thing

"source": source,
"reason": data.reason,
"message": data.message,
"data": $
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

has the $ a special meaning?

Copy link
Member Author

@pierDipi pierDipi Feb 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, $ is basically the input / root node

{"hello": "Matthias"} with transformation {"data": $ } results in {"data": {"hello": "Matthias"} }

See for example: https://try.jsonata.org/quwrKXzni

For the same input:

{"data": $.hello, "data2": hello, "full": $ }

returns

{
  "data": "Matthias",
  "data2": "Matthias",
  "full": {
    "hello": "Matthias"
  }
}

Copy link
Member

@matzew matzew left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm
/approve

@knative-prow knative-prow bot added the lgtm Indicates that a PR is ready to be merged. label Feb 11, 2025
Copy link

knative-prow bot commented Feb 11, 2025

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: matzew, pierDipi

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@knative-prow knative-prow bot merged commit 2c1c7b8 into knative:main Feb 11, 2025
35 of 36 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. lgtm Indicates that a PR is ready to be merged. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants