Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Destinations cdk: recorddiffer that handles strongly-typed records #44874

Conversation

edgao
Copy link
Contributor

@edgao edgao commented Aug 28, 2024

mostly a copy of the existing RecordDiffer. But this time it operates on OutputRecord + AirbyteValue, which simplifies things by quite a lot. See also #45113 for how this will be used.

closes https://github.com/airbytehq/airbyte-internal-issues/issues/9217

Copy link

vercel bot commented Aug 28, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Skipped Deployment
Name Status Preview Comments Updated (UTC)
airbyte-docs ⬜️ Ignored (Inspect) Visit Preview Sep 19, 2024 9:56pm

@octavia-squidington-iii octavia-squidington-iii added the CDK Connector Development Kit label Aug 28, 2024
@edgao edgao force-pushed the edgao/better_record_differ branch 4 times, most recently from 4ed0e22 to da9d896 Compare September 3, 2024 22:03
Copy link
Contributor Author

edgao commented Sep 3, 2024

@edgao edgao changed the base branch from master to issue-9361/load-cdk-with-e2e-dest-post-refactor September 5, 2024 16:31
@johnny-schmidt johnny-schmidt force-pushed the issue-9361/load-cdk-with-e2e-dest-post-refactor branch 3 times, most recently from d43c364 to 26d247c Compare September 10, 2024 23:01
@octavia-squidington-iii octavia-squidington-iii removed the area/connectors Connector related issues label Sep 11, 2024
@johnny-schmidt johnny-schmidt force-pushed the issue-9361/load-cdk-with-e2e-dest-post-refactor branch from 26d247c to e852485 Compare September 11, 2024 18:40
@johnny-schmidt johnny-schmidt force-pushed the issue-9361/load-cdk-with-e2e-dest-post-refactor branch from e852485 to 7f0ea41 Compare September 13, 2024 17:08
@johnny-schmidt johnny-schmidt force-pushed the issue-9361/load-cdk-with-e2e-dest-post-refactor branch from 7f0ea41 to ee94dc9 Compare September 16, 2024 17:07
@octavia-squidington-iii octavia-squidington-iii added the area/connectors Connector related issues label Sep 16, 2024
"generationId: Expected ${expectedRecord.generationId}, got ${actualRecord.generationId}\n"
)
}
if (expectedRecord.airbyteMeta != actualRecord.airbyteMeta) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there's a bug here that I need to fix - airbyteMeta is a JsonNode, which has issues with comparison (IntNode(42) != LongNode(42))

so I need to either make it a real struct, or write something to compare jsonnodes for equality

@johnny-schmidt johnny-schmidt force-pushed the issue-9361/load-cdk-with-e2e-dest-post-refactor branch 8 times, most recently from 1260f1f to 3c769c5 Compare September 19, 2024 21:46
@edgao edgao force-pushed the issue-9361/load-cdk-with-e2e-dest-post-refactor branch from 3c769c5 to ca834e0 Compare September 19, 2024 21:56
@edgao edgao merged commit f63a834 into issue-9361/load-cdk-with-e2e-dest-post-refactor Sep 20, 2024
27 of 32 checks passed
@edgao edgao deleted the edgao/better_record_differ branch September 20, 2024 20:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants