-
-
Notifications
You must be signed in to change notification settings - Fork 2.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Use a database as a backend for JabRef library management #12708
Comments
Regarding the de-coupling of the UI with the data, starting points are |
In last year's GSoC we opted for Postgres, because it has many plugins, especially for fast regular expression search so that the DBMS does the regex indexing and resolving - and not the client. |
XTDB? There is so much more to do with literature than a bibtex/biblatex schema covers, and immutable data would enable some great opportunities. Examples include a traceable history of screening or detecting duplicates from the past (recognising previously imported entries even after they have changed). |
Since this is about memory saving, more things need to be considered: Loading of a .bib file
Presenting data to JabRef
Saving of a .bib file
|
Is your suggestion for improvement related to a problem? Please describe.
Currently, JabRef struggles with libraries that have over 1000 entries (#10209).
Short reason and solution: JabRef stores all information in RAM. JabRef needs a mechanism to manage lots of data. This is a perfect use case for databases!
Longer issue description: look at how JabRef manages libraries and entries:
.bib
file..bib
file intoBibDatabase
(withBibDatabaseContext
) andBibEntry
. Those are Java objects that are stored in RAM..bib
file.So, JabRef's original philosophy is to be a file editor. However, when you have a giant library, you just don't have enough JVM heap. It is limited.
Describe the solution you'd like
JabRef should have a mechanism for managing a lot of data and use it for storing and manipulating libraries.
This is the purpose of databases! A DBMS will also cache data: a typical DBMS stores data in pages. Some pages are stored in RAM, some are offloaded to disk. This is a perfect solution for giant libraries, as now you are not limited to RAM space, but to space on your HDD/SDD!
Moreover, DBMS allows you to query data fast and powerful. Here is one place where SQL can be used: #10209 (comment). Search functionality is also a perfect case for databases.
Additional context
This is planned as a GSoC project. Beware, while this project is quite important for JabRef, it might turn out to be very complex.
We aim for a Relational DBMS like SQLite, DuckDB, Postgres. Especially, we want a database to be embedded.
In fact, we want Postgres to be our backend, as Postgres has powerful capabilities for search. It can be used as an embedded database, actually; checkout this library: https://github.com/zonkyio/embedded-postgres.
Here are some materials for this project:
The text was updated successfully, but these errors were encountered: