Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement physical optimizer rule to apply type coercion to physical plans #14324

Open
andygrove opened this issue Jan 27, 2025 · 3 comments
Open
Labels
enhancement New feature or request

Comments

@andygrove
Copy link
Member

Is your feature request related to a problem or challenge?

For DataFusion users who are starting with physical plans rather than logical plans, a physical optimizer rule to apply type coercion to physical plans would be helpful.

See the discussion in #14283 (comment) for one example where this would be helpful.

Describe the solution you'd like

No response

Describe alternatives you've considered

No response

Additional context

No response

@andygrove andygrove added the enhancement New feature or request label Jan 27, 2025
@alamb
Copy link
Contributor

alamb commented Jan 28, 2025

BTW this might be fairly straightforward to add as an extension in comet (maybe it doesn't have to be in the datafusion core if it is only used by systems that use the physical plan)

@edmondop
Copy link
Contributor

Is it possible to do type coercion only at the physical layer? Or does it bring benefits at the logical layer so that we need to do type coercion twice, once at the logical layer for users of the "datafusion frontend" and once at the physical layer for applications like Comet that "skips" the datafusion frontend?

@andygrove
Copy link
Member Author

BTW this might be fairly straightforward to add as an extension in comet (maybe it doesn't have to be in the datafusion core if it is only used by systems that use the physical plan)

We will probably do this in the short term, but don't we want DataFusion to be supported for the execution engine use case, similar to Velox?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants