Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ConsistentReservoirSamplingSpanProcessorTest is sporadically failing #1602

Open
trask opened this issue Dec 13, 2024 · 4 comments
Open

ConsistentReservoirSamplingSpanProcessorTest is sporadically failing #1602

trask opened this issue Dec 13, 2024 · 4 comments

Comments

@trask
Copy link
Member

trask commented Dec 13, 2024

https://github.com/open-telemetry/opentelemetry-java-contrib/actions/runs/12319968072/job/34388221769

(the reason this didn't show up before is because we used to auto-retry tests)

> Task :consistent-sampling:test

ConsistentReservoirSamplingSpanProcessorTest > testConsistentSampling() FAILED
    java.lang.AssertionError: 
    Expecting actual:
      0.006596909041848287
    to be greater than:
      0.01
        at io.opentelemetry.contrib.sampler.consistent.TestUtil.verifyObservedPvaluesUsingGtest(TestUtil.java:72)
        at io.opentelemetry.contrib.sampler.consistent.ConsistentReservoirSamplingSpanProcessorTest.testConsistentSampling(ConsistentReservoirSamplingSpanProcessorTest.java:608)
        at io.opentelemetry.contrib.sampler.consistent.ConsistentReservoirSamplingSpanProcessorTest.testConsistentSampling(ConsistentReservoirSamplingSpanProcessorTest.java:663)
@trask
Copy link
Member Author

trask commented Feb 22, 2025

@oertl @PeterF778 could you look at this? we're still seeing fairly regular sporadic test failures, see #1762

@oertl
Copy link
Contributor

oertl commented Feb 25, 2025

This statistical test may sporadically fail if the result depends on randomness. The seeds for the random generators used by the ConsistentReservoirSamplingSpanProcessor are set in the test. However, it seems that the spans get processed in different orders for some reason, which causes some variations.

@trask
Copy link
Member Author

trask commented Mar 14, 2025

thanks for looking into it, is this fixable, or should we disable the test for now? it's causing us to need to re-run CI a lot

@oertl
Copy link
Contributor

oertl commented Mar 14, 2025

It is fine to temporarily disable the test. I would have to find some more time to investigate, as these variations are not caused by the sampling part.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants