Skip to content

Latest commit

 

History

History
83 lines (53 loc) · 8.83 KB

metrics.md

File metadata and controls

83 lines (53 loc) · 8.83 KB

Metrics

Your merge experience directly impacts the velocity and productivity of your development team. Merge Queue Metrics provides observability for the health of your Trunk Merge Queue, so you can discover issues early and make informed optimizations.

The Health tab showing metrics in the Trunk Web App.

Access Metrics

You can access the metrics in your Trunk Merge Queue by navigating to the Trunk Web App > Merge Queue > Health.

{% hint style="info" %}

Enabling CI Time and CI Jobs Triggered

Pushing data to CI Analytics is required to enable CI Time and CI Jobs Triggered charts if you're not using GitHub Actions. {% endhint %}

Time Buckets

The date ranges selector at the top left of the dashboard allows you to filter the data displayed by date and time. You can display time buckets by the day or hour in the day/hour dropdown.

The metrics displayed only include data that have completed within the time range, jobs started but not completed during the selected time will not be displayed.

{% hint style="info" %} When working across multiple time zones, enable Time in UTC to ensure everyone sees the same data. {% endhint %}

Conclusion Count

Conclusion count displays the number of pull requests that exited the merge queue during each time bucket. This includes passes, failures, and cancellations. Passes and failures signal a PR that was tested in the queue to completion, while canceled signals that the request to merge terminated before testing finished or before testing began.

Conclusion counts are an important signal to potential bottlenecks or underlying issues with your merging process, as a failure or cancellation in the merge queue can force other PRs to restart their testing. A spike in the number of failures or passes can indicate a potential problem to investigate.

Conclusions are tagged with a reason to give further insights into how merges pass or fail in the queue. You can show or hide conclusions of a particular reason by using the + Add button.

CategoryReasonDescription
✅ PassMerged by TrunkPassed all tests in Merge Queue and merged by Trunk
✅ PassMerged manuallyUser manually merged the PR in Git
❌ FailureTest run timeoutUser-defined timeout for tests exceeded
❌ FailureFailed TestsRequired test failed while testing the PR in the merge queue
❌ FailureMerge conflictA (git) merge conflict encountered
❌ FailureConfig parsing failureMalformed trunk.yaml that couldn't be parsed
❌ FailureConfig bad versionInvalid version field in trunk.yaml
❌ FailureConfig bad required statusesFailed to parse required statuses in trunk.yaml
❌ FailureNo required statusesNo source for required tests was found in trunk.yaml or branch protection settings
❌ FailureGitHub API FailedGitHub returned an error to us that could not be resolved while processing the PR
❌ FailurePR updated at merge timePR updated as Trunk was attempting to merge it
🚫 CancelCanceled by userPR explicitly canceled by user
🚫 CancelPR closedPR closed (not merged)
🚫 CancelPR pushed toNew commits pushed to the PR branch while in the merge queue
🚫 CancelPR draftPR was converted to a draft, which cannot be merged
🚫 CancelPR base branch changedBase branch of PR in the merge queue changed
🚫 CancelAdmin requestedTrunk employee canceled PR during a support session (extreme cases)

Time in Queue

Time in queue shows how long each PR spends in the Merge Queue from the moment the PR enters the queue to the moment when it exits the queue, either from merging, failing, or being canceled.

Understanding the amount of time a pull request spends in the queue is important for ensuring your merge process continues to ship code quickly. A spike in the time to merge indicates a slowdown somewhere that's impacting all developers. For example, it's taking longer to run tests on PRs, PRs are waiting too long to start testing, or constant failures in the queue are causing PRs to take longer to merge

The time in queue can be displayed as different statistical measures. You can show or hide them by using the + Add button.

Measure Explanation
Average Average of all time in queue during the time bucket
Minimum The shortest time in queue in the time bucket.
Maximum The longest time in queue in the time bucket.
Sum The total of all time in queue added together.
P50 The value below 50% of the time in queue falls.
P95 The value below 95% of the time in queue falls.
P99 The value below 99% of the time in queue falls.

CI Time to Test PRs

CI Time measures the time it takes to test each pull request in the merge queue. Specifically, how long CI jobs have to run to test PRs. This measures only CI steps relevant to the merge queue, other CI steps like build or deploy or any workflows outside of the merge queue will not be measured.

{% hint style="info" %} Pushing data to CI Analytics is required to enable CI Time and CI Jobs Triggered charts if you're not using GitHub Actions. {% endhint %}

Monitoring the amount of time it takes for CI workflows to test PRs is important for making sure PRs can move through the queue quickly. The more time it takes to test PRs, the more time a PR will be in the queue. Large spikes signal that work should be devoted to reducing the length of bottlenecking CI jobs, or more CI resources are needed.


CI Time to Test PRs can be shown as different statistical measures by pressing the + Add button

Measure Explanation
Sum The total time it has taken to run all CI jobs for all PRs in the queue
Average Per Test Run The average amount of total CI time it takes to test a PR or batch of PRs in the queue.
Average Per Individual PR

The average amount of total CI time it takes to test an individual PR. This specifically accounts for batching.

For example, if a batch of 5 PRs takes 50 minutes of CI time to test, this would be 50 / 5 = 10 minutes on average per PR.

Average Per Individual Job The average amount of time it takes for an individual job to complete when testing a PR. There are usually multiple jobs launched when testing a PR.

CI Jobs Triggered To Test PRs

The count of Individual CI jobs that are triggered to test PRs in the merge queue. There could be multiple jobs triggered per PR - this metric captures all of them.

Each job will require a CI runner. A spike or large number of requested jobs can result in PRs waiting longer to begin testing.

{% hint style="info" %} Pushing data to CI Analytics is required to enable CI Time and CI Jobs Triggered charts if you're not using GitHub Actions. {% endhint %}