Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Kafka connector was throwing null pointer exception when entire record is null, issue fix. #2575

Merged
merged 8 commits into from
Feb 11, 2025

Conversation

VenkatasivareddyTR
Copy link
Contributor

Issue #, if available:

Description of changes:
Kafka connector was throwing null pointer exception when it encountered a completely null record. This change resolves that issue.
PFA testing doc for reference.
Kafka NullPointerException Issue Fix.docx

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Comment on lines 266 to 269
if (record == null || record.value() == null) {
LOGGER.warn("[NullRecord] {} Received a null record or record value, offset: {}", splitParameters, record != null ? record.offset() : "unknown");
return 0;
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we move this code block to before spiller.writeRows? Doesn't seem related to the lambda in spiller.writeRows

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved to before spiller.writeRows.

Comment on lines 353 to 356
if (record == null || record.value() == null) {
LOGGER.warn("[NullRecord] {} Received a null record or record value, offset: {}", splitParameters, record != null ? record.offset() : "unknown");
return 0; // Skip processing this record
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved to before spiller.writeRows.

Comment on lines 439 to 442
if (record == null || record.value() == null) {
LOGGER.warn("[NullRecord] {} Received a null record or record value, offset: {}", splitParameters, record != null ? record.offset() : "unknown");
return 0; // Skip processing this record
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved to before spiller.writeRows.

Copy link

codecov bot commented Feb 10, 2025

Codecov Report

Attention: Patch coverage is 33.33333% with 6 lines in your changes missing coverage. Please review.

Project coverage is 60.71%. Comparing base (f3521ad) to head (9172378).
Report is 43 commits behind head on master.

Files with missing lines Patch % Lines
...ws/athena/connectors/kafka/KafkaRecordHandler.java 33.33% 0 Missing and 6 partials ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##             master    #2575      +/-   ##
============================================
+ Coverage     60.68%   60.71%   +0.03%     
- Complexity     3871     3873       +2     
============================================
  Files           593      593              
  Lines         22130    22130              
  Branches       2732     2735       +3     
============================================
+ Hits          13430    13437       +7     
+ Misses         7398     7385      -13     
- Partials       1302     1308       +6     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Contributor

@ejeffrli ejeffrli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

verified with chngpe about codecov issues. record == null can't be hit with MockConsumer because it gives NullPointerException

@ejeffrli ejeffrli merged commit 4deb3af into awslabs:master Feb 11, 2025
7 of 8 checks passed
github-actions bot pushed a commit that referenced this pull request Feb 17, 2025
  - Upgraded Arrow to 18 (#2576)
  - Fixed Encoding Warning (#2595)
  - Fix codecov format error (#2594)
  - Update Code Cov on Daily Validation Tests (#2593)
  - Allow custom glue endpoint to be used (#2587)
  - Msk connector was throwing null pointer exception when entire record is null, issue fix. (#2588)
  - Kafka connector was throwing null pointer exception when entire record is null, issue fix. (#2575)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants