Skip to content

Commit

Permalink
perf(accountsdb): bufferpool-integration (#476)
Browse files Browse the repository at this point in the history
* Bufferpool: add CachedRead.borrowSlice

* Accountsdb: add bufferpool field

* use ReadHandle in accountsdb

* get it building

* wip - debugging

* make a mess

* comment all test failures

* add probes

* test and fix ReadHandle's bincode

* misc fixes

* cleanup

* fix line lengths

* make freq atomic

* all tests passing / misc fixes

* add atomic linear fifo

* use threadlocal io_uring

* frames metadata: use atomics for all fields

* make style check happy

* small cleanup

* add flags for benchmark profiling

* rename iterator to byteIterator, add chunked Iterator

* replace iterator with faster one; simplified code

* remove probes

* address comments

* remove .inner on ReadHandle

* misc cleanup

* handle full queue, rollback rcs on error

* blocking io: rc cleanup on error path

* remove allocation from hot path

* line length check

* seek files to 0 before reading all

* use *const ReadHandle

* split out owned and unowned buffers

* remove useless errors from read + related functions

* store and document file_length on accountfiles

* move file_map append

* init_allocator -> allocator

* cloneAndTrack: duplicate cached read handles

* add TODO for io_uring cli flag

* cleanup some io_uring code

* split out FrameManager and properly address thread safety

* remove atomicstack

* make style check happy

* improve threadsafety / add comments on it

* bugfix: use preadall

* batch some lock usages / add comments

* remove .size, factor out Metadata, remove some atomics

* rework locking / fix threadsafety issue / temp regress io_uring

* make fast path / reduce contention

* squashme

* concurrency rework

* bring back io_uring

* simplify io_uring & wait less

* remove dead code

* reduce read calls

* default to blocking io

* align frames

* removed dead code

* style check

* fix bad diff

* remove noinlines used for debugging

* correctly update populated on read

* remove incorrect asserts (triggers rarely, not actually a bug)

* remove dead code

* naming & comments improvements

* small naming/style fixes

* move constants to top

* some renames / style changes

* minor style fixes

* macos: correct page size

* test some errdefers / increase codecov
  • Loading branch information
Sobeston authored Feb 12, 2025
1 parent 805b122 commit 7c39406
Show file tree
Hide file tree
Showing 9 changed files with 1,476 additions and 787 deletions.
11 changes: 9 additions & 2 deletions build.zig
Original file line number Diff line number Diff line change
Expand Up @@ -132,9 +132,10 @@ pub fn build(b: *Build) void {
.root_source_file = b.path("src/tests.zig"),
.target = target,
.optimize = optimize,
.sanitize_thread = enable_tsan,
.filters = filters orelse &.{},
.sanitize_thread = enable_tsan,
});
b.installArtifact(unit_tests_exe);
test_step.dependOn(&unit_tests_exe.step);
install_step.dependOn(&unit_tests_exe.step);

Expand Down Expand Up @@ -210,7 +211,13 @@ pub fn build(b: *Build) void {
benchmark_exe.linkLibC();
benchmark_exe.root_module.addOptions("build-options", build_options);

benchmark_exe.root_module.addImport("xev", xev_mod);
// make sure pyroscope's got enough info to profile
benchmark_exe.build_id = .fast;
benchmark_exe.root_module.omit_frame_pointer = false;
benchmark_exe.root_module.strip = false;

b.installArtifact(benchmark_exe);

benchmark_exe.root_module.addImport("base58", base58_mod);
benchmark_exe.root_module.addImport("zig-network", zig_network_mod);
benchmark_exe.root_module.addImport("zstd", zstd_mod);
Expand Down
Loading

0 comments on commit 7c39406

Please sign in to comment.