Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Skip commenting memory/perf benches on draft PRs #1694

Merged
merged 6 commits into from
Jul 3, 2024
Merged

Skip commenting memory/perf benches on draft PRs #1694

merged 6 commits into from
Jul 3, 2024

Conversation

sezna
Copy link
Contributor

@sezna sezna commented Jul 2, 2024

This should help reduce some of the comment noise until we have time to customize that action some more.

I tested that this works on this PR -- the benches are not run when the PR is a draft.

I had to pull the two relevant jobs out into a different workflow file, because the triggers are different. For the reporting benches stage (the memory and perf comments that show up on PRs), they'll only run if the PR is in a draft. But, and this is the different part that required them being in a separate file, they also need to trigger when a PR is marked "ready for review".

@sezna sezna marked this pull request as ready for review July 2, 2024 15:32
@sezna sezna marked this pull request as draft July 2, 2024 15:36
@sezna sezna marked this pull request as ready for review July 2, 2024 15:40
@sezna sezna marked this pull request as draft July 2, 2024 15:43
@sezna sezna marked this pull request as ready for review July 2, 2024 15:44
Copy link

github-actions bot commented Jul 2, 2024

Benchmark for ffac5b7

Click to view benchmark
Test Base PR %
Array append evaluation 329.3±3.42µs 330.9±3.04µs +0.49%
Array literal evaluation 189.4±1.27µs 171.2±1.64µs -9.61%
Array update evaluation 409.2±1.54µs 411.6±3.46µs +0.59%
Core + Standard library compilation 20.5±0.49ms 20.8±0.66ms +1.46%
Deutsch-Jozsa evaluation 5.1±0.06ms 5.2±0.06ms +1.96%
Large file parity evaluation 34.4±0.10ms 34.6±0.40ms +0.58%
Large input file compilation 12.6±0.36ms 12.8±0.42ms +1.59%
Large input file compilation (interpreter) 49.7±1.42ms 52.3±1.61ms +5.23%
Large nested iteration 32.0±0.11ms 32.2±0.27ms +0.62%
Perform Runtime Capabilities Analysis (RCA) on Deutsch-Jozsa sample 1575.1±72.51µs 1595.9±126.11µs +1.32%
Perform Runtime Capabilities Analysis (RCA) on large file sample 8.0±0.16ms 8.2±0.20ms +2.50%
Perform Runtime Capabilities Analysis (RCA) on teleport sample 1430.2±67.53µs 1436.6±74.58µs +0.45%
Perform Runtime Capabilities Analysis (RCA) on the core and std libraries 28.0±0.28ms 28.2±0.50ms +0.71%
Teleport evaluation 91.3±3.69µs 93.0±7.77µs +1.86%

@swernli
Copy link
Collaborator

swernli commented Jul 2, 2024

It looks like the tests for the npm package got stuck for the last two hours on macos-14... @billti have you seen anything like this before?

Copy link

github-actions bot commented Jul 2, 2024

Benchmark for d0e85f3

Click to view benchmark
Test Base PR %
Array append evaluation 328.7±4.65µs 330.7±2.29µs +0.61%
Array literal evaluation 188.9±0.78µs 186.6±2.82µs -1.22%
Array update evaluation 409.6±1.59µs 412.8±3.85µs +0.78%
Core + Standard library compilation 21.6±0.88ms 22.0±1.11ms +1.85%
Deutsch-Jozsa evaluation 5.1±0.05ms 5.2±0.06ms +1.96%
Large file parity evaluation 34.3±0.14ms 34.4±0.33ms +0.29%
Large input file compilation 13.0±0.29ms 12.8±0.46ms -1.54%
Large input file compilation (interpreter) 51.5±1.44ms 51.2±1.74ms -0.58%
Large nested iteration 32.1±0.24ms 32.2±0.24ms +0.31%
Perform Runtime Capabilities Analysis (RCA) on Deutsch-Jozsa sample 1561.6±38.76µs 1590.1±88.03µs +1.83%
Perform Runtime Capabilities Analysis (RCA) on large file sample 7.8±0.13ms 8.0±0.16ms +2.56%
Perform Runtime Capabilities Analysis (RCA) on teleport sample 1424.4±45.82µs 1459.1±133.39µs +2.44%
Perform Runtime Capabilities Analysis (RCA) on the core and std libraries 28.0±0.49ms 28.2±0.28ms +0.71%
Teleport evaluation 90.8±4.14µs 91.9±3.67µs +1.21%

@sezna sezna added this pull request to the merge queue Jul 3, 2024
Merged via the queue into main with commit c023200 Jul 3, 2024
19 checks passed
@sezna sezna deleted the alex/ci-noise branch July 3, 2024 16:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants