Enable force overwrite for partial upserts

[himanish-star]

Fixes #14924

This PR removes the logic that stops prev column value to be overwritten by new column value if the new value is NULL.

Before:

(null, newValue) -> newValue
(oldValue, null) -> oldValue
(null, null) -> null

After:

(null, newValue) -> newValue
(oldValue, null) -> null
(null, null) -> null

[tibrewalpratik17]

This change is backward-incompatible. Going through the attached issue, I would recommend to create a new merger strategy and allow null-overwrites there.

[himanish-star]

Okay 👍🏻

[himanish-star]

Done!

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 55.55556% with 4 lines in your changes missing coverage. Please review.

Project coverage is 63.42%. Comparing base (59551e4) to head (d7fa44f).
Report is 1754 commits behind head on master.

Files with missing lines	Patch %	Lines
...cal/upsert/merger/PartialUpsertColumnarMerger.java	25.00%	2 Missing and 1 partial ⚠️
...l/upsert/merger/columnar/ForceOverwriteMerger.java	50.00%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##             master   #15107      +/-   ##
============================================
+ Coverage     61.75%   63.42%   +1.67%     
- Complexity      207     1482    +1275     
============================================
  Files          2436     2749     +313     
  Lines        133233   154451   +21218     
  Branches      20636    23817    +3181     
============================================
+ Hits          82274    97964   +15690     
- Misses        44911    49088    +4177     
- Partials       6048     7399    +1351     

Flag	Coverage Δ
custom-integration1	100.00% <ø> (+99.99%)	⬆️
integration	100.00% <ø> (+99.99%)	⬆️
integration1	100.00% <ø> (+99.99%)	⬆️
integration2	0.00% <ø> (ø)
java-11	63.39% <55.55%> (+1.68%)	⬆️
java-21	63.32% <55.55%> (+1.69%)	⬆️
skip-bytebuffers-false	63.42% <55.55%> (+1.67%)	⬆️
skip-bytebuffers-true	63.30% <55.55%> (+35.57%)	⬆️
temurin	63.42% <55.55%> (+1.67%)	⬆️
unittests	63.42% <55.55%> (+1.67%)	⬆️
unittests1	56.04% <11.11%> (+9.15%)	⬆️
unittests2	33.99% <55.55%> (+6.26%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[raghukn]

Pls update the documentation also with this.

[tibrewalpratik17]

Hey @himanish-star the solution looks good to me! Can you please add unit-test / integration-test for your change too?

[himanish-star]

Hey @himanish-star the solution looks good to me! Can you please add unit-test / integration-test for your change too?

Sure, I'll add today

[himanish-star]

Pls update the documentation also with this.

Yes, I'll add documentation mentioning the new config value to be set

[himanish-star]

added unit tests

[himanish-star]

@tibrewalpratik17 I've tested locally using partial upsert quick start and added unit tests and they're working correctly. Shall we move ahead with this PR?