uploader.md

Uploader CLI Reference

Trunk Flaky Tests detects and tracks flaky tests in your repos by receiving uploads from your test runs in CI. Trunk Flaky Tests takes the JUnit XML format for uploads. These uploads happen in the CI jobs used to run tests in your nightly CI, post-commit jobs, and PR checks.

If you're setting up Trunk Flaky Tests for the first time, you can follow the guides for your CI provider and test framework.

Guides by Test Frameworks			frameworks
Guides by CI Provider			ci-providers

Installing the CLI

The CLI should be downloaded as part of your test workflow in your CI system. The launcher is platform agnostic and will download the latest version of the uploader for your platform.

{% hint style="success" %} You should always use the latest version of the uploader CLI by downloading it fresh in your CI jobs for the best detection results. {% endhint %}

You can download the uploader CLI and mark it executable with the following command:

curl -fsSLO --retry 3 https://trunk.io/releases/trunk && chmod +x trunk

You can invoke an upload like this.

./trunk flakytests upload --junit-paths "test_output.xml" \
   --org-url-slug <TRUNK_ORG_SLUG> \
   --token $TRUNK_API_TOKEN

Uploading from the CLI

{% hint style="info" %} The uploaded tests are processed by Trunk periodically, not in real-time. Wait for at least an hour after the initial upload before they’re displayed in the Uploads tab. Multiple uploads are required before a test can be accurately detected as broken or flaky. {% endhint %}

The trunk command-line tool can upload and analyze test results. The trunk flakytests command accepts the following subcommands:

Command	Description
trunk flakytests upload	Upload data to Trunk Flaky Tests.
trunk flakytests validate	Validates if the provided JUnit XML files and prints any errors.
trunk flakytests test <COMMAND>	Runs tests using the provided command, uploads results, checks whether the failures are quarantined tests, and correct the exit code based on that.

The upload command accepts the following options:

Argument	Description
--junit-paths <JUNIT_PATHS>	A comma separated list of paths containing the test output files. File globs are supported.
--org-url-slug <ORG_URL_SLUG>	Trunk Organization slug, from the Settings page.
--token <TOKEN>	Trunk Organization (not repo) token, from the Settings page. Defaults to the TRUNK_API_TOKEN variable.
-h, --help	Additional detailed description of the upload command.
--repo-root	Path to the repository root. Defaults to the current directory.
--repo-url <REPO_URL>	Value to override URL of repository. Optional.
--repo-head-sha <REPO_HEAD_SHA>	Value to override SHA of repository head. Optional.
--repo-head-branch <REPO_HEAD_BRANCH>	Value to override branch of repository head. Optional.
--repo-head-commit-epoch <REPO_HEAD_COMMIT_EPOCH>	Value to override commit epoch of repository head. Optional.
--tags <TAGS>	Comma separated list of custom tag=value pairs. Optional.
--print-files	Print files which will be uploaded to stdout.
--dry-run	Run metrics CLI without uploading to API. Optional.
--team <TEAM>	Value to tag team owner of upload. Optional.
--codeowners-path <CODEOWNERS_PATH>	Value to override CODEOWNERS file or directory path. Optional.
--use-quarantining	Quarantined tests according to repo settings. Defaults to true.
--allow-empty-test-results	Don't fail commands if test results are empty or missing. Use it when you sometimes skip all tests for certain CI jobs. Defaults to true.

Validating Test Reports

You can validate your test reports locally

Upgrade

If you installed the CLI in your CI jobs following the instructions in the Installing the CLI step, the CI job will automatically install the latest version of the CLI.

If you're using the flakytests CLI subcommand using the Trunk CLI locally, you can upgrade with this command:

trunk flakytests --upgrade

Troubleshooting

As a general rule you should download the release on every CI run. Do not bake the CLI into a container or VM. This ensures your CI runs are always using the latest build.

The trunk binary should be run from the repository root. If you need to run the binary from another location, you must provide the path to the repo root using the --repo-rootargument. The --junit-paths argument accepts the XML file locations as both a list of globs or absolute paths.

Organization not found

If you receive an error that the org slug or API token is not found, double check that the secrets stored in your CI provider are the same as the Organization settings by navigating to Settings -> Manage -> Organization on app.trunk.io.

Make sure you are getting your Organization Slug, not the Organization Name.

Get the Organization Slug

Ensure you get your Organization API Token, not your repo token.

Test results aren't uploading

If the test results aren't uploading from your CI system then one possible cause is malformed XML. Try modifying your job to run ./trunk flakytests validate. Also try using --dry-run and --print-files to show which files will be uploaded.