Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use cheaper hash function for computing the DataKey #245

Closed
joshua-spacetime opened this issue Aug 25, 2023 · 2 comments
Closed

Use cheaper hash function for computing the DataKey #245

joshua-spacetime opened this issue Aug 25, 2023 · 2 comments
Assignees

Comments

@joshua-spacetime
Copy link
Collaborator

joshua-spacetime commented Aug 25, 2023

We current use Keccak256 for computing a product value's DataKey. This can account for over 90% of the cost of generating the DataKey in some cases, the other portion being the bsatn encoding. Note this hash is invoked once for each row, both on the write path as well as the read path.

Is a cryptographic hash necessary for identifying rows? Can we use something cheaper?

@joshua-spacetime
Copy link
Collaborator Author

The DataKey computation also accounts for nearly 50% of the cost of query and subscription evaluation.

@jdetter
Copy link
Collaborator

jdetter commented Jan 16, 2024

Closed in favor of the memarch refactor

Blocker for fully removing the data key is the SDKs (right now they use data key for identifying rows for deletion). The SDKs should just use the row itself.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants