Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Expose user defined functions in the FFI #14562

Open
4 tasks
timsaucer opened this issue Feb 9, 2025 · 0 comments
Open
4 tasks

Expose user defined functions in the FFI #14562

timsaucer opened this issue Feb 9, 2025 · 0 comments
Labels
enhancement New feature or request

Comments

@timsaucer
Copy link
Contributor

Is your feature request related to a problem or challenge?

By adding the foreign function interface to this project, we exposed a host of features to end users. It allows for easy integration of DataFusion libraries across projects, specifically impacting datafusion-python.

I recommend that we work to expose scalar, aggregate, and window user defined functions via FFI. This is to support the issue apache/datafusion-python#1017

Describe the solution you'd like

Follow the existing pattern within the datafusion repository and expose these types of user defined functions.

  • scalar
  • aggregate
  • window
  • table

Note: The table functions are not currently in use in datafusion-python, but it would be a good addition.

Describe alternatives you've considered

For the datafusion-python project, we can work around via python calls but this would be a far more elegant approach in my opinion.

Additional context

Once we have these exposed, we could expose the function registry, which would get us very close to a FFI SessionContext. That would have impacts on the ballista and datafusion-ray projects.

@timsaucer timsaucer added the enhancement New feature or request label Feb 9, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant