Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for partial views #92

Merged
merged 1 commit into from
Jan 23, 2025
Merged

Add support for partial views #92

merged 1 commit into from
Jan 23, 2025

Conversation

ryannedolan
Copy link
Collaborator

Summary

Add support for materializing views with names foo$bar, which output to sink foo. This enables multiple materialized views to write to the same sink.

Details

We introduce the concept of "partial views", which are views materializing part of a larger table. For example, we can have partial views foo$bar, foo$qux, and foo$qiz, all of which are distinct views but which write to the same sink foo. The corresponding pipelines each have distinct names (e.g. foo-bar), enabling them to be individually deleted, updated, etc.

For now, we don't validate that the sink foo's schema matches what the pipeline will try to write. We can introduce a Validator for this purpose later.

Testing

Manually tested creating "partial views" and inspecting the resulting pipeline SQL:

> create materialized view ads."foo$bar" as ...

The resulting pipelines are named correctly:

$ kubectl get pipelines
...
ads-foo-bar         CREATE DATABASE IF NOT EXISTS `ADS` WITH ();...   Deployed.
...

@ryannedolan ryannedolan enabled auto-merge (squash) January 23, 2025 00:42
@ryannedolan ryannedolan merged commit 089fdac into main Jan 23, 2025
1 check passed
@ryannedolan ryannedolan deleted the partial-views branch January 23, 2025 00:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants