Skip to content

Commit

Permalink
[meta] prepare releases
Browse files Browse the repository at this point in the history
  • Loading branch information
sunshowers committed Feb 12, 2025
1 parent 75f6e57 commit c8b1c79
Show file tree
Hide file tree
Showing 6 changed files with 63 additions and 7 deletions.
9 changes: 7 additions & 2 deletions nextest-runner/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,14 @@
# Changelog

## [0.72.0] - 2024-02-10
## [0.73.0] - 2025-02-12

See the changelog for [cargo-nextest 0.9.90](https://nexte.st/changelog#0.9.90).

## [0.72.0] - 2025-02-10

See the changelog for [cargo-nextest 0.9.89](https://nexte.st/changelog#0.9.89).

## [0.71.0] - 2024-01-15
## [0.71.0] - 2025-01-15

See the changelog for [cargo-nextest 0.9.88](https://nexte.st/changelog#0.9.88).

Expand Down Expand Up @@ -530,6 +534,7 @@ Thanks to [Guiguiprim](https://github.com/Guiguiprim) for their contributions to

- Initial version.

[0.73.0]: https://github.com/nextest-rs/nextest/releases/tag/nextest-runner-0.73.0
[0.72.0]: https://github.com/nextest-rs/nextest/releases/tag/nextest-runner-0.72.0
[0.71.0]: https://github.com/nextest-rs/nextest/releases/tag/nextest-runner-0.71.0
[0.70.0]: https://github.com/nextest-rs/nextest/releases/tag/nextest-runner-0.70.0
Expand Down
11 changes: 11 additions & 0 deletions site/src/changelog.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,16 @@ toc_depth: 1
This page documents new features and bugfixes for cargo-nextest. Please see the [stability
policy](https://nexte.st/docs/stability/) for how versioning works with cargo-nextest.

## [0.9.90] - 2025-02-12

### Added

- Tests are now assigned global and group *slot numbers*. These numbers are non-negative integers starting from 0 that are unique for the lifetime of the test, but are reused after the test ends.

Global and group slot numbers can be accessed via the `NEXTEST_TEST_GLOBAL_SLOT` and `NEXTEST_TEST_GROUP_SLOT` environment variables, respectively. For more, see [*Slot numbers*](http://nexte.st/docs/configuration/test-groups#slot-numbers).

- [Test environments](http://nexte.st/docs/configuration/env-vars/#environment-variables-nextest-sets) now have the `NEXTEST_TEST_GROUP` variable set to the [test group](http://nexte.st/docs/configuration/test-groups) they're in, or `"@global"` if the test is not in any groups.

## [0.9.89] - 2025-02-10

### Added
Expand Down Expand Up @@ -1369,6 +1379,7 @@ Supported in this initial release:
- [Test retries](https://nexte.st/book/retries.md) and flaky test detection
- [JUnit support](https://nexte.st/book/junit.md) for integration with other test tooling

[0.9.90]: https://github.com/nextest-rs/nextest/releases/tag/cargo-nextest-0.9.90
[0.9.89]: https://github.com/nextest-rs/nextest/releases/tag/cargo-nextest-0.9.89
[0.9.88]: https://github.com/nextest-rs/nextest/releases/tag/cargo-nextest-0.9.88
[0.9.87]: https://github.com/nextest-rs/nextest/releases/tag/cargo-nextest-0.9.87
Expand Down
11 changes: 11 additions & 0 deletions site/src/docs/configuration/env-vars.md
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,17 @@ Nextest exposes these environment variables to your tests _at runtime only_. The
`NEXTEST_EXECUTION_MODE`
: Currently, always set to `process-per-test`. More options may be added in the future if nextest gains the ability to run all tests within the same process ([#27]).

`NEXTEST_TEST_GROUP` <!-- md:version 0.9.90 -->
: Set to the [test group](test-groups.md) the test is in, or `"@global"` if the test is not in any groups.

`NEXTEST_TEST_GLOBAL_SLOT` <!-- md:version 0.9.90 -->
: Set to the [global slot number](test-groups.md#slot-numbers). Global slot numbers are non-negative integers starting from 0 that are unique within the run for the lifetime of the test, but are reused after the test finishes.

`NEXTEST_TEST_GROUP_SLOT` <!-- md:version 0.9.90 -->
: If the test is in a group, set to the [group slot number](test-groups.md#slot-numbers). Group slot numbers are non-negative integers that are unique within the test group for the lifetime of the test, but are reused after the test finishes.

If the test is not in any groups, this is set to `"none"`.

`NEXTEST_BIN_EXE_<name>`
: The absolute path to a binary target's executable. This is only set when running an [integration test] or benchmark. The `<name>` is the name of the binary target, exactly as-is. For example, `NEXTEST_BIN_EXE_my-program` for a binary named `my-program`.

Expand Down
35 changes: 32 additions & 3 deletions site/src/docs/configuration/test-groups.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,8 @@ These test groups impact execution in the following ways:

Nextest will continue to schedule as many tests as possible, accounting for global and group concurrency limits.

<!-- md:version 0.9.90 --> The test group a test is running in is exposed via the `NEXTEST_TEST_GROUP` environment variable. If a test is not in any groups, this variable is set to `"@global"`.

## Showing test groups

You can show the test groups currently in effect with `cargo nextest show-config test-groups`.
Expand All @@ -94,6 +96,33 @@ group: <u style="text-decoration-style:single"><b>serial-integration</b></u> (ma

This command accepts [all the same options](../listing.md#options-and-arguments) that `cargo nextest list` does.

## Slot numbers

<!-- md:version 0.9.90 -->

Nextest assigns each test a *global slot number*. Additionally, if a test is in a group, the test is also assigned a *group slot number*.

Slot numbers are integers that start from 0 and go up from there. They are useful to assign resources such as blocks of port numbers to tests.

Slot numbers are:

* **Unique** for the lifetime of the test, either globally or within the group.

For example, if a test in the group `resource-limited` is assigned the global slot 10 and the group slot 5, then while this test is running:

- No other test within the same run will be assigned the global slot 10.
- No other tests in the `resource-limited` group will be assigned the group slot 5.

After this test finishes, the global slot number 10 and the group slot number 5 are freed up, and can be reused for other tests.

* **Stable** across [retries](../features/retries.md) within the same run (though not across runs).

* **Compact**, in the sense that each test is always assigned the smallest possible slot number starting from 0, depending on which numbers are free at the time the test starts.

For example, if a test group is limited to serial execution, the group slot number is always 0 for those tests.

Global and group slot numbers can be accessed via the `NEXTEST_TEST_GLOBAL_SLOT` and `NEXTEST_TEST_GROUP_SLOT` [environment variables](env-vars.md#environment-variables-nextest-sets), respectively. (If a test is not within a group, `NEXTEST_TEST_GROUP_SLOT` is set to `none`.)

## Comparison with `threads-required`

Test groups are similar to [heavy tests and `threads-required`](threads-required.md). The key difference is that test groups are meant to limit concurrency for subsets of tests, while `threads-required` sets global limits across the entire test run.
Expand All @@ -118,7 +147,7 @@ threads-required = 1 # this is the default, shown for clarity
With this configuration:

- Tests whose names start with `group::heavy::`, and tests that start with `group::light::`, are both part of `my-group`.
- The `group::heavy::` tests will take up two slots within _both_ global and group concurrency limits.
- The `group::light::` tests will take up one slot within both limits.
- The `group::heavy::` tests will take up two threads within _both_ global and group concurrency limits.
- The `group::light::` tests will take up one thread within both limits.

> **Note:** Setting `threads-required` to be greater than a test group's `max-threads` will not cause issues; a test that does so will take up all slots available.
> **Note:** Setting `threads-required` to be greater than a test group's `max-threads` will not cause issues; a test that does so will take up all threads available.
2 changes: 1 addition & 1 deletion site/src/docs/configuration/threads-required.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ icon: material/weight-lifter

Nextest achieves [its performance](../benchmarks/index.md) through running [many tests in parallel](../design/how-it-works.md). However, some projects have tests that consume a disproportionate amount of resources like CPU or memory. If too many of these _heavy tests_ are run concurrently, your machine's CPU might be overwhelmed, or it might run out of memory.

With nextest, you can mark heavy tests as taking up multiple threads or "slots" out of the total amount of available parallelism. In other words, you can assign those tests a higher "weight". This is done by using the `threads-required` [per-test override](per-test-overrides.md).
With nextest, you can mark heavy tests as taking up multiple threads out of the total amount of available parallelism. In other words, you can assign those tests a higher "weight". This is done by using the `threads-required` [per-test override](per-test-overrides.md).

For example, on a machine with 16 logical CPUs, nextest will run 16 tests concurrently by default. However, if you mark tests that begin with `tests::heavy::` as requiring 2 threads each:

Expand Down
2 changes: 1 addition & 1 deletion site/src/docs/running.md
Original file line number Diff line number Diff line change
Expand Up @@ -229,7 +229,7 @@ fail-fast = { max-fail = "all" }
* a positive integer (e.g. `8`) to run that many tests simultaneously.
* a negative integer (e.g. `-2`) to run available parallelism minus that many tests simultaneously. For example, on a machine with 8 CPU hyperthreads, `-2` would run 6 tests simultaneously.

Tests can be marked as taking up more than one available slot. For more, see [*Heavy tests and `threads-required`*](configuration/threads-required.md).
Tests can be marked as taking up more than one available thread. For more, see [*Heavy tests and `threads-required`*](configuration/threads-required.md).

`--run-ignored=only` <!-- md:version 0.9.76 -->
: Run only ignored tests. (With prior nextest versions, use `--run-ignored=ignored-only`.)
Expand Down

0 comments on commit c8b1c79

Please sign in to comment.