Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[COST-5773] Handle multiple resource for OCP on GCP managed flow #5427

Merged
merged 24 commits into from
Jan 23, 2025

Conversation

myersCody
Copy link
Contributor

@myersCody myersCody commented Dec 16, 2024

Jira Ticket

COST-5573

Description

This change will:

  1. Add multiple clusters, add a wait, and trigger ocp ingest before cloud ingest to prevent ocp on cloud parquet file error.
  2. Restructure the managed flow to use a dataclass to pass in build the sql parameters in the db accessors.
  3. Restructure the GCP flow to handle row deduplication

Testing

  • Set the unleash flag to True for ocp on cloud managed processing
  1. using this branch Load test customer data for:
  • AWS
  • Azure
  • GCP
  1. The test data populated for OCP on GCP will end with a verification successful return. Whereas if you were to try on master you will get a failed message.

Release Notes

  • proposed release note
* [COST-####](https://issues.redhat.com/browse/COST-####) Fix some things

Copy link

codecov bot commented Jan 10, 2025

Codecov Report

Attention: Patch coverage is 94.59459% with 8 lines in your changes missing coverage. Please review.

Project coverage is 94.1%. Comparing base (d78ebdf) to head (5159732).
Report is 1 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##            main   #5427     +/-   ##
=======================================
- Coverage   94.1%   94.1%   -0.0%     
=======================================
  Files        370     371      +1     
  Lines      31551   31631     +80     
  Branches    3381    3386      +5     
=======================================
+ Hits       29705   29772     +67     
- Misses      1199    1210     +11     
- Partials     647     649      +2     

@myersCody myersCody added the smoke-tests pr_check will build the image and run minimal required smokes label Jan 14, 2025
@myersCody myersCody marked this pull request as ready for review January 14, 2025 18:57
@myersCody myersCody requested review from a team as code owners January 14, 2025 18:57
@myersCody myersCody added aws-smoke-tests pr_check will build the image and run aws + ocp on aws smoke tests and removed aws-smoke-tests pr_check will build the image and run aws + ocp on aws smoke tests labels Jan 15, 2025
Copy link
Contributor

@lcouzens lcouzens left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is looking great Cody!! I left a handful of nitpick/optional considerations. Other than that I'm down to approve this.

Copy link
Contributor

@lcouzens lcouzens left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work Cody!

@myersCody myersCody merged commit 7881165 into main Jan 23, 2025
14 checks passed
@myersCody myersCody deleted the cost-5773-deduplication branch January 23, 2025 16:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
smoke-tests pr_check will build the image and run minimal required smokes smokes-required
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants