Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Batch Inserts and Upserts #154

Open
TheSamsa opened this issue Jan 29, 2025 · 0 comments
Open

Batch Inserts and Upserts #154

TheSamsa opened this issue Jan 29, 2025 · 0 comments

Comments

@TheSamsa
Copy link

I would like to have the posibility to perform batch inserts.
Right now every record gets inserted/upserted into the database one by one, this can be a performance bottleneck if too many records are produced.
Its basically the same as a SQL query inside a loop.

A batch inserter mechanism could collect up to N records and also wait up to X seconds before sending the query to the database anyway. Whichever case happens first, so there is not a batch "forever" in the waiting to be sent to the DB, it gets flushed at least at some point.

I'll provide a example implementation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant