Skip to content

Add basic heap profiler #5724

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 2 commits into
base: main
Choose a base branch
from
Draft

Add basic heap profiler #5724

wants to merge 2 commits into from

Conversation

rdettai
Copy link
Collaborator

@rdettai rdettai commented Mar 30, 2025

Description

This PR adds a very lightweight, feature gated, heap profiling endpoint.

Using the jemalloc native profiling is complicated, it's hard to asses its performance impact, and the resulting traces are hard to interpret.

This PR proposes an custom and simple heap profiler:

  • written so that the profiler has almost no performance impact when it's not being used actively. Every allocation/deallocation simply checks for an atomic ENABLED flag with relaxed ordering and does no extra work if that flag is false.
  • the profiler can be activated with a webservice. It logs all allocations larger than a given (configurable) threshold. Instead of printing the entire backtrace, we focus on the first "quickwit related" symbol name (quickwit, chitchat, tantivy)

When started, the profiling allocates ~30MB of symbols.

How was this PR tested?

Describe how you tested this PR.

@rdettai rdettai self-assigned this Mar 30, 2025
@rdettai rdettai marked this pull request as draft April 2, 2025 10:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant