Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

logs_to_metrics: Support optional value_field for counters #10055

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

databus23
Copy link

@databus23 databus23 commented Mar 6, 2025

Support incrementing counters by a specified value

If a value_field is specified for a counter metric, the counter is incremented by the given value instead of just incrementing by 1.

This allows counting more things then just the number of records, such as the total number of bytes sent or received from an access log.


Enter [N/A] in the box, if an item is not applicable to your change.

Testing
Before we can approve your change; please submit the following in a comment:

  • Attached Valgrind output that shows no leaks or memory corruption was found

Documentation

  • Documentation required for this feature

Doc PR: fluent/fluent-bit-docs#1584

Backporting

  • Backport to latest stable release.

Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.

@databus23 databus23 force-pushed the logs_to_metric_counter_value branch 2 times, most recently from 68cd15e to e725188 Compare March 6, 2025 23:09
@databus23 databus23 changed the title logs_to_metric: Support optional value_field for counters logs_to_metrics: Support optional value_field for counters Mar 6, 2025
@databus23 databus23 force-pushed the logs_to_metric_counter_value branch from e725188 to 2c1822f Compare March 6, 2025 23:12
Support incrementing counters by a specified value.

If a `value_field` is specified for a counter, the counter is incremented by the given value instead of just counting the number of records.

This allows tracking different metrics, such as the total number of bytes sent or received in an access log.

Signed-off-by: Fabian Ruff <[email protected]>
@databus23 databus23 force-pushed the logs_to_metric_counter_value branch from 2c1822f to db182ed Compare March 9, 2025 00:10
@databus23
Copy link
Author

databus23 commented Mar 10, 2025

Valgrind output for unit tests:

fluent-bit ➜ /workspaces/fluent-bit/build (logs_to_metric_counter_value) $ valgrind bin/flb-rt-filter_log_
to_metrics 
==35868== Memcheck, a memory error detector
==35868== Copyright (C) 2002-2022, and GNU GPL'd, by Julian Seward et al.
==35868== Using Valgrind-3.19.0 and LibVEX; rerun with -h for copyright info
==35868== Command: bin/flb-rt-filter_log_to_metrics
==35868== 
Test counter_k8s...                             [ OK ]
Test counter...                                 [ OK ]
Test counter_value_field...                     [ OK ]
Test counter_k8s_two_tuples...                  [ OK ]
Test gauge...                                   [ OK ]
Test histogram...                               [ OK ]
Test counter_regex...                           [ OK ]
Test regex_empty_label_keys...                  [ OK ]
Test label...                                   [ OK ]
SUCCESS: All unit tests have passed.
==35868== 
==35868== HEAP SUMMARY:
==35868==     in use at exit: 0 bytes in 0 blocks
==35868==   total heap usage: 22,585 allocs, 22,585 frees, 16,092,238 bytes allocated
==35868== 
==35868== All heap blocks were freed -- no leaks are possible
==35868== 
==35868== For lists of detected and suppressed errors, rerun with: -s
==35868== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

databus23 added a commit to databus23/fluent-bit-docs that referenced this pull request Mar 10, 2025
This documents a feature added with this PR: fluent/fluent-bit#10055

Signed-off-by: Fabian Ruff <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant