Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle Oracle NUMBER correctly by letting JdbcArrowTypeConverter hand… #2573

Merged
merged 1 commit into from
Feb 7, 2025

Conversation

ejeffrli
Copy link
Contributor

@ejeffrli ejeffrli commented Feb 5, 2025

Oracle NUMBER with no precision or scale specified allows for Oracle users to have a dynamic number of decimal points. This means NUMBER column can have the value 10.123123 for one entry and 10 for the other entry.

This is not reflected properly in our connector. This fix removes the precision == 0 check (originally, mis-wrote as scale == 0) on line 381 because JdbcArrowTypeConverter.java handles that on line 51. See

boolean needsResolving = jdbcType == Types.NUMERIC && (precision == 0 && scale <= 0);

With above change, customers can now specify default_scale in lambda environment variables to capture the Oracle NUMBER's decimal points.

It also changes line 381 to include 0 to correctly convert 0 scale DECIMAL to arrow BIGINT.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Copy link

codecov bot commented Feb 5, 2025

Codecov Report

Attention: Patch coverage is 75.00000% with 2 lines in your changes missing coverage. Please review.

Project coverage is 60.69%. Comparing base (f3521ad) to head (626bf45).
Report is 22 commits behind head on master.

✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
...thena/connectors/oracle/OracleMetadataHandler.java 75.00% 0 Missing and 2 partials ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##             master    #2573      +/-   ##
============================================
+ Coverage     60.68%   60.69%   +0.01%     
  Complexity     3871     3871              
============================================
  Files           593      593              
  Lines         22130    22134       +4     
  Branches       2732     2732              
============================================
+ Hits          13430    13435       +5     
  Misses         7398     7398              
+ Partials       1302     1301       -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@AbdulR3hman AbdulR3hman merged commit 8480b18 into master Feb 7, 2025
10 checks passed
github-actions bot pushed a commit that referenced this pull request Feb 7, 2025
  - Handle Oracle NUMBER correctly by letting JdbcArrowTypeConverter hand… (#2573)
  - Update aws-cdk cli version to match aws-cdk-lib 2.177.0 (#2570)
  - Fixed Timestamp truncation (#2559)
  - build(deps): bump aws-sdk-v2.version from 2.30.6 to 2.30.11 (#2562)
  - build(deps): bump aws-sdk-v2.version from 2.30.6 to 2.30.11
  - build(deps): bump org.eclipse.rdf4j:rdf4j-repository-sparql from 5.1.0 to 5.1.1 (#2565)
  - build(deps): bump org.eclipse.rdf4j:rdf4j-repository-sparql
  - build(deps): bump software.amazon.awssdk:cloudwatchlogs from 2.30.6 to 2.30.11 (#2567)
  - build(deps): bump software.amazon.awssdk:cloudwatchlogs
  - build(deps): bump org.jetbrains.kotlin:kotlin-stdlib from 2.1.0 to 2.1.10 (#2564)
  - build(deps): bump org.jetbrains.kotlin:kotlin-stdlib
  - build(deps): bump org.jetbrains.kotlin:kotlin-reflect from 2.1.0 to 2.1.10 (#2568)
  - build(deps): bump org.jetbrains.kotlin:kotlin-reflect
  - build(deps): bump com.google.cloud:google-cloud-storage from 2.47.0 to 2.48.0 (#2563)
  - build(deps): bump com.google.cloud:google-cloud-storage
  - build(deps): bump org.jetbrains.kotlin:kotlin-stdlib-jdk8 from 2.1.0 to 2.1.10 (#2566)
  - build(deps): bump org.jetbrains.kotlin:kotlin-stdlib-jdk8
  - build(deps): bump io.lettuce:lettuce-core from 6.5.2.RELEASE to 6.5.3.RELEASE (#2561)
  - build(deps): bump io.lettuce:lettuce-core
  - build(deps-dev): bump nl.jqno.equalsverifier:equalsverifier from 3.18.1 to 3.18.2 (#2560)
  - build(deps-dev): bump nl.jqno.equalsverifier:equalsverifier
  - add dynamodb package-based template back (#2558)
  - build(deps): bump aws-cdk-lib from 2.130.0 to 2.177.0 in /validation_testing/cdk_federation_infra_provisioning/app (#2557)
  - build(deps): bump aws-cdk-lib
  - Upgrade to JDK17. (#2342)
  - build(deps): bump net.snowflake:snowflake-jdbc from 3.20.0 to 3.22.0 in /athena-snowflake (#2556)
  - build(deps): bump net.snowflake:snowflake-jdbc in /athena-snowflake
  - Addressing CVE-2021-42392,CVE-2022-23221,CVE-2022-23305,CVE-2019-17571 (#2555)
  - athena-synapse: Fix Partition Number and Table name for Glue Table API compatibility (#2529)
  - SQLServer glue get table api issue fix (#2524)
  - CLOUDERA-IMPALA Connection String Fix (#2546)
  - CLOUDERA-HIVE Connection String Fix (#2548)
  - build(deps): bump com.sap.cloud.db.jdbc:ngdbc from 2.22.12 to 2.23.8 (#2551)
  - build(deps): bump com.sap.cloud.db.jdbc:ngdbc from 2.22.12 to 2.23.8
  - build(deps): bump com.clickhouse:clickhouse-jdbc from 0.7.2 to 0.8.0 (#2552)
  - build(deps): bump com.clickhouse:clickhouse-jdbc from 0.7.2 to 0.8.0
  - build(deps): bump software.amazon.awssdk:cloudwatchlogs from 2.30.2 to 2.30.6 (#2550)
  - build(deps): bump software.amazon.awssdk:cloudwatchlogs
  - build(deps): bump aws-sdk-v2.version from 2.30.2 to 2.30.6 (#2549)
  - build(deps): bump aws-sdk-v2.version from 2.30.2 to 2.30.6
  - Upgraded RDS Postgresql Version Used In Integ Tests (#2547)
  - Clean up ECR Repo per Connector if exits (#2545)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants