You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In my use case of a concurrent program it proved to be useful to be somewhat lenient when approving (large) outputs against the master files.
The output of two threads was in a race condition, leading to switched lines in the output files. The harness contains a MAX_DEVIATION constant (currently set to 0.0) that can be used to allow minor variations in the output. Non-perfect matches with the master files are then reported as "aborted" tests (equivalent to failed assumptions up to JUnit 4), which are then neither "green" nor "red."
Of course, this can only work for minor differences. If the program output is very loosely related to the input or contains many timestamps (e.g. through logging), this approach is too simple.
The text was updated successfully, but these errors were encountered:
In my use case of a concurrent program it proved to be useful to be somewhat lenient when approving (large) outputs against the master files.
The output of two threads was in a race condition, leading to switched lines in the output files. The harness contains a
MAX_DEVIATION
constant (currently set to 0.0) that can be used to allow minor variations in the output. Non-perfect matches with the master files are then reported as "aborted" tests (equivalent to failed assumptions up to JUnit 4), which are then neither "green" nor "red."Of course, this can only work for minor differences. If the program output is very loosely related to the input or contains many timestamps (e.g. through logging), this approach is too simple.
The text was updated successfully, but these errors were encountered: