Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

alg/ensemble,internal/results: dedupe expensive tests #102

Open
mmcloughlin opened this issue May 13, 2021 · 0 comments
Open

alg/ensemble,internal/results: dedupe expensive tests #102

mmcloughlin opened this issue May 13, 2021 · 0 comments
Labels
cleanup General cleanup and code health low Low priority testing Tests and supporting infrastructure

Comments

@mmcloughlin
Copy link
Owner

Both alg/ensemble and internal/results run expensive tests to confirm that algorithm results have not changed on targets of interest, but with slightly different goals:

  • alg/ensemble: Ensure chain lengths for every algorithm results are preserved, to verify no unexpected changes.
  • internal/results: Ensure that our list of best-known chains is up-to-date, so our documentation is right. This checks the exact programs generated, not just their length.

These tests run every algorithm against every target in the results list, so they're expensive at the moment. The logic could be merged into one test.

Probably low priority because these should get a lot faster (after #60) and they're only run in long or stress test mode right now.

@mmcloughlin mmcloughlin added testing Tests and supporting infrastructure cleanup General cleanup and code health low Low priority labels May 13, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cleanup General cleanup and code health low Low priority testing Tests and supporting infrastructure
Projects
None yet
Development

No branches or pull requests

1 participant