This repository has been archived by the owner on Jun 14, 2024. It is now read-only.
[FEATURE REQUEST]: Please consider extending indexing support to non - spark implementation of structured storage, specifically, stand - alone Java and Rust implementation of Parquet / Delta Lake #514
Labels
Feature requested
Similar to Access database, an SQL Server engine counterpart, running in process of application, please consider extending indexing support to Spark counterpart, currently provided as libraries, over delta lake. Of those two notable implementations are Java DSR/DSW ( stand - alone reader and writer ):
https://github.com/delta-io/connectors
and Rust delta lake implementation:
https://github.com/delta-io/delta-rs
With proliferation and success of such Data Warehousing implementations as Snowflake, the ETL pipeline of data warehouse increasingly relies on external storage as staging area, mostly, for cost saving features.
Extending support to indexing over external delta lake storage would significantly strengthen appeal of Azure Storage solutions to practically all BI implementations, without exceptions, providing nice financial boost to entire platform as well as competitive edge with regards to query performance.
As a type of user = data engineer, data scientist, developer etc., I want desired feature, in order to goal.
Acceptance criteria
Describe what you consider (e.g., functionality requirement) has to be true to mark this feature as complete
Success criteria
Define any measurements that need to be performed
Additional context
Add any other context or screenshots about the feature request here.
The text was updated successfully, but these errors were encountered: