Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🚀 Feature: New repeat option to run tests an arbitrary number of times #4936

Closed
baileympearson opened this issue Oct 21, 2022 · 1 comment
Labels
duplicate been there, done that, got the t-shirt... type: feature enhancement proposal

Comments

@baileympearson
Copy link

Is your feature request related to a problem or a nice-to-have?? Please describe.
This is definitely a nice-to-have. Oftentimes, I'm debugging flakey tests or tests with race conditions and I want to see if I've fixed the flakiness. I usually approach this by running the test many times as an easy confirmation that the test is fixed. I usually just use a bash loop to run the test command numerous times but this has a few drawbacks

  • Our test runner is slow-ish to start because of custom configuration we need to do (~4-5 seconds), so that dramatically increases the amount of time each test iteration takes (from milliseconds to 4-5 seconds per iteration)
  • Our test runner prints out extra information, and mocha prints out test suite completion information as well, cluttering the test output visually. I usually solve this by either grepping for the test name in the output but it would be nice if I didn't need to do this. This also breaks down if I want to run multiple tests, not just one.
  • Multiple tests are run in the order I don't expect because each iteration of the bash loop runs the full suite. I'd prefer each test to run however many times I specify (in order) before continuing to the next.

Describe the solution you'd like
A repeat option would solve this issue. The option would be numeric and default to one. When mocha executes a test, I envision mocha just looping and running the test however many times are specified.

Describe alternatives you've considered
I've tried the bash loop I mentioned in the description (I outlined some challenges there as well).

The other approach I've used is to wrap each test in a loop, but this breaks down if I'm using a --grep or I'm working with many tests. It's also just slightly cumbersome to wrap each test in a loop, and each loop needs the same hard-coded limit. doable, but annoying.

Additional context
Happy to work with the maintainers and contribute this feature if there is interest!

@baileympearson baileympearson added the type: feature enhancement proposal label Oct 21, 2022
@JoshuaKGoldberg JoshuaKGoldberg changed the title support a new repeat option in Mocha to run tests an arbitrary number of times 🚀 Feature: New repeat option to run tests an arbitrary number of times Dec 27, 2023
@JoshuaKGoldberg
Copy link
Member

Duplicate of #2332. I like the idea! 🙂

@JoshuaKGoldberg JoshuaKGoldberg closed this as not planned Won't fix, can't repro, duplicate, stale Feb 6, 2024
@JoshuaKGoldberg JoshuaKGoldberg added the duplicate been there, done that, got the t-shirt... label Feb 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
duplicate been there, done that, got the t-shirt... type: feature enhancement proposal
Projects
None yet
Development

No branches or pull requests

2 participants