Measure rate of growth of metrics #1040
Assignees
Labels
area/data-ingestion-and-visualization
area/flows
area/metrics
control-plane/hubble
control-plane/standard
help wanted
Extra attention is needed
priority/0
P0
type/fix
Fixes something
Comments
anubhabMajumdar
added
area/data-ingestion-and-visualization
area/flows
area/metrics
control-plane/hubble
control-plane/standard
help wanted
7 tasks
7 tasks
github-merge-queue bot
pushed a commit
that referenced
this issue
Jan 23, 2025
# Description Add `metricscardinality` to heartbeat. This will give visibility on the number of time series being exposed by retina. ## Related Issue #1040 ## Checklist - [x] I have read the [contributing documentation](https://retina.sh/docs/contributing). - [x] I signed and signed-off the commits (`git commit -S -s ...`). See [this documentation](https://docs.github.com/en/authentication/managing-commit-signature-verification/about-commit-signature-verification) on signing commits. - [x] I have correctly attributed the author(s) of the code. - [x] I have tested the changes locally. - [x] I have followed the project's style guidelines. - [x] I have updated the documentation, if necessary. - [x] I have added tests, if applicable. ## Screenshots (if applicable) or Testing Completed Metrics exported with heartbeat:  ## Additional Notes Metrics of types `histogram` and `summary` expose multiple time series during a scrape. Code is counting according to number of time series exposed at /metrics endpoint. Ref: https://prometheus.io/docs/concepts/metric_types/#histogram https://prometheus.io/docs/concepts/metric_types/#summary --- Please refer to the [CONTRIBUTING.md](../CONTRIBUTING.md) file for more information on how to contribute to this project. Signed-off-by: Alex Castilio dos Santos <[email protected]>
kamilprz
pushed a commit
to kamilprz/retina
that referenced
this issue
Jan 24, 2025
# Description Add `metricscardinality` to heartbeat. This will give visibility on the number of time series being exposed by retina. ## Related Issue microsoft#1040 ## Checklist - [x] I have read the [contributing documentation](https://retina.sh/docs/contributing). - [x] I signed and signed-off the commits (`git commit -S -s ...`). See [this documentation](https://docs.github.com/en/authentication/managing-commit-signature-verification/about-commit-signature-verification) on signing commits. - [x] I have correctly attributed the author(s) of the code. - [x] I have tested the changes locally. - [x] I have followed the project's style guidelines. - [x] I have updated the documentation, if necessary. - [x] I have added tests, if applicable. ## Screenshots (if applicable) or Testing Completed Metrics exported with heartbeat:  ## Additional Notes Metrics of types `histogram` and `summary` expose multiple time series during a scrape. Code is counting according to number of time series exposed at /metrics endpoint. Ref: https://prometheus.io/docs/concepts/metric_types/#histogram https://prometheus.io/docs/concepts/metric_types/#summary --- Please refer to the [CONTRIBUTING.md](../CONTRIBUTING.md) file for more information on how to contribute to this project. Signed-off-by: Alex Castilio dos Santos <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Metrics generated by Retina is supposed to of low cardinality over time. If there's a label that is unbound, the metrics volume would constantly increase, causing OOMKIll issues for agent, and may overflow storage of these metrics on an external entity.
Add a test that
Measure rate of growth of metrics volume to make sure we don't have unbound growth.
The text was updated successfully, but these errors were encountered: