question - what about Wasm support/fallback?

[andreaTP]

Hi 👋 and thanks for the amazing project, it's giving me a solid base to build upon 🙂 .

What I'm trying to do:

I want to have a sqlite-jdbc driver with zero native dependencies, by running SQLite with Chicory.

Where I am:

As of today, I just started implementing the DB astract class and I start to have the first few tests passing in QueryTest using in-memory databases (this is the branch I'm hacking on).

The good:

SQLite is already compiling to wasm-wasi and I'm using Chicory's experimental AOT compiler in a Maven plugin to produce a pure bytecode version of it. Using the SQLite API seems to work completely as expected so far in this early exploration 🎉

The bad:

We need to depend on Chicory's data structures to make it run (Chicory itself is zero dependencies, still...).
Chicory's Java minimum compatibility version is 11, while here I see that we are targeting 8.

The ugly:

Unless I rewrite from scratch a SQLite VFS layer, currently, my plan is to use the wasi integration of Chicory, that is tested only on Jimfs.
This will bring additional (and possibly undesired) transitive dependencies to this project.

Questions:

My initial plan was to try to contribute back this integration as a "fallback mode" to the NativeDB implementation.
Given the considerations above, I start to question my plan, but I want first to open the discussion with the maintainers of this project, since I'm likely going to swap only an alternative implementation of NativeDB and re-use most of the rest of the infrastructure/code.

I'm happy to hear any feedback and consideration!

[gotson]

Hi, that's an interesting topic you have there.

As of now, the project will remain as-is, meaning a single jar distributed, containing all the native libs, and targeting JDK8.

In an ideal world, i think a good sqlite driver for the JVM should be split in multiple artifacts/projects, probably along the lines of:

• 2 interfaces: a simple sqlite wrapper, and a more complex jdbc wrapper (similar to what https://github.com/gwenn/sqlite-jna offers)
• multiple implementations of the runtime:
  • the current JNI version, which would probably need to be split in many Maven artifacts, so you could ship only the OS/arch you want
  • a new implementation using the JDK22 Foreign Function Interface (Project Panama), which has been discussed in Support the Foreign API from project Panama #717
  • any other implementation, for example your WASM project

However, the current state of the project/code would require a tremendous effort to refactor and split everything to achieve this, so it's not on the cards.

Your best bet is probably a fork for now.

[andreaTP]

Thanks for getting back @gotson , appreciated!

I completely understand and respect your position,, I'd start with a fork but I'd be happy to revisit when something changes in the future.

[gotson]

Don't hesitate to report your progress here from times to times !

[andreaTP]

progress report:

I'm working on this branch, the code is not cleaned up etc., but progress is good:

Tests run: 439, Failures: 4, Errors: 35, Skipped: 19

Most of the failures, at the moment, are caused by "missing implementations", but the building blocks seem to be solid enough.
Handlers and UDF functions are working and passing the tests etc.

[andreaTP]

Starting to tight things together, I moved the repo here:
https://github.com/andreaTP/sqlite-jdbc-wasm

We have CI working, initial publishing on Jitpack etc.
There are a few semantics that slightly changed compared to the original version, but it should start to be usable.

Happy to hear if you know any good open source project(possibly testsuite or benchmark) that I can use to stress the implementation more.