Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat/add support kerberize hivemetastore #1634

Merged

Conversation

Fokko
Copy link
Contributor

@Fokko Fokko commented Feb 10, 2025

Took #766 and addressed the comments, to make sure that it gets into 0.9.0

@Fokko Fokko added this to the PyIceberg 0.9.0 release milestone Feb 10, 2025
@Fokko Fokko force-pushed the feat/add-support-kerberize-hivemetastore branch from e6610b1 to 3ab287f Compare February 10, 2025 10:04
@Fokko Fokko force-pushed the feat/add-support-kerberize-hivemetastore branch from 3ab287f to 38d9c2c Compare February 10, 2025 10:06
@Fokko
Copy link
Contributor Author

Fokko commented Feb 10, 2025

It is not maintained anymore: https://github.com/apple/ccs-pykerberos

@kevinjqliu
Copy link
Contributor

i think to get around this, we need to install some packages on the github runner os

apple/ccs-pykerberos#66

Copy link
Contributor

@kevinjqliu kevinjqliu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I realized that we dont really have a way to verify that this implementation is correct

pyproject.toml Outdated
@@ -80,6 +80,8 @@ sqlalchemy = { version = "^2.0.18", optional = true }
getdaft = { version = ">=0.2.12", optional = true }
cachetools = "^5.5.0"
pyiceberg-core = { version = "^0.4.0", optional = true }
thrift-sasl = { version = ">=0.4.3", optional = true }
kerberos = { version = "1.3.1", optional = true }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is causing the CI issue. Im not sure if we really require this dependency

Copy link
Contributor Author

@Fokko Fokko Feb 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good call, let me check! I was surprised by how little there was as an alternative to this dependency, it is pretty likely that it is not needed at all.

return _HiveClient(
uri,
properties.get("ugi"),
property_as_bool(properties, HIVE_KERBEROS_AUTH, HIVE_KERBEROS_AUTH_DEFAULT),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this change is causing a few tests to fail, we need to add the extra arg to the tests
like so https://github.com/kevinjqliu/iceberg-python/pull/10/files#diff-8a0f847796be6745b3be158f5c39d0e83cbb868c162763a13895175b484cc529

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kevinjqliu all green :)

Copy link
Contributor

@kevinjqliu kevinjqliu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Let's merge this and iterate if necessary

@kevinjqliu kevinjqliu merged commit 8fcdc95 into apache:main Feb 12, 2025
8 checks passed
@Fokko Fokko deleted the feat/add-support-kerberize-hivemetastore branch February 12, 2025 19:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants