Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consider fuzzy approval for non-deterministic programs #4

Open
maxbechtold opened this issue Jan 21, 2018 · 0 comments
Open

Consider fuzzy approval for non-deterministic programs #4

maxbechtold opened this issue Jan 21, 2018 · 0 comments

Comments

@maxbechtold
Copy link
Owner

In my use case of a concurrent program it proved to be useful to be somewhat lenient when approving (large) outputs against the master files.

The output of two threads was in a race condition, leading to switched lines in the output files. The harness contains a MAX_DEVIATION constant (currently set to 0.0) that can be used to allow minor variations in the output. Non-perfect matches with the master files are then reported as "aborted" tests (equivalent to failed assumptions up to JUnit 4), which are then neither "green" nor "red."

Of course, this can only work for minor differences. If the program output is very loosely related to the input or contains many timestamps (e.g. through logging), this approach is too simple.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant