Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Upgrade to Spock 2.3 #8634

Merged
merged 1 commit into from
Mar 28, 2025
Merged

Upgrade to Spock 2.3 #8634

merged 1 commit into from
Mar 28, 2025

Conversation

smola
Copy link
Member

@smola smola commented Mar 28, 2025

What Does This Do

Release notes

RepeatUntilFailure (PR) is particularly interesting for flaky test troubleshooting.

Motivation

Additional Notes

Contributor Checklist

Jira ticket: [PROJ-IDENT]

@smola smola added tag: no release notes Changes to exclude from release notes comp: tooling Build & Tooling labels Mar 28, 2025
@pr-commenter
Copy link

pr-commenter bot commented Mar 28, 2025

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master smola/spock-2.3
git_commit_date 1743154785 1743159258
git_commit_sha 52b943d 9098f10
release_version 1.48.0-SNAPSHOT~52b943d46e 1.48.0-SNAPSHOT~9098f10dc3
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1743162440 1743162440
ci_job_id 869681772 869681772
ci_pipeline_id 60311637 60311637
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-xjcpkyqs-project-304-concurrent-0-f3w12xgl 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-xjcpkyqs-project-304-concurrent-0-f3w12xgl 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 65 metrics, 6 unstable metrics.

Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.48.0-SNAPSHOT~9098f10dc3, baseline=1.48.0-SNAPSHOT~52b943d46e

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.054 s) : 0, 1053518
Total [baseline] (10.439 s) : 0, 10439049
Agent [candidate] (1.05 s) : 0, 1049881
Total [candidate] (10.442 s) : 0, 10442464
section appsec
Agent [baseline] (1.19 s) : 0, 1190499
Total [baseline] (10.771 s) : 0, 10771145
Agent [candidate] (1.194 s) : 0, 1194329
Total [candidate] (10.753 s) : 0, 10753213
section iast
Agent [baseline] (1.178 s) : 0, 1178339
Total [baseline] (11.029 s) : 0, 11029271
Agent [candidate] (1.177 s) : 0, 1177062
Total [candidate] (10.965 s) : 0, 10965327
section profiling
Agent [baseline] (1.27 s) : 0, 1270089
Total [baseline] (10.846 s) : 0, 10846454
Agent [candidate] (1.278 s) : 0, 1277760
Total [candidate] (10.853 s) : 0, 10853223
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.054 s -
Agent appsec 1.19 s 136.981 ms (13.0%)
Agent iast 1.178 s 124.822 ms (11.8%)
Agent profiling 1.27 s 216.571 ms (20.6%)
Total tracing 10.439 s -
Total appsec 10.771 s 332.096 ms (3.2%)
Total iast 11.029 s 590.222 ms (5.7%)
Total profiling 10.846 s 407.405 ms (3.9%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.05 s -
Agent appsec 1.194 s 144.448 ms (13.8%)
Agent iast 1.177 s 127.18 ms (12.1%)
Agent profiling 1.278 s 227.879 ms (21.7%)
Total tracing 10.442 s -
Total appsec 10.753 s 310.749 ms (3.0%)
Total iast 10.965 s 522.864 ms (5.0%)
Total profiling 10.853 s 410.759 ms (3.9%) 
gantt
    title petclinic - break down per module: candidate=1.48.0-SNAPSHOT~9098f10dc3, baseline=1.48.0-SNAPSHOT~52b943d46e

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (721.014 ms) : 0, 721014
BytebuddyAgent [candidate] (718.963 ms) : 0, 718963
GlobalTracer [baseline] (239.706 ms) : 0, 239706
GlobalTracer [candidate] (239.415 ms) : 0, 239415
AppSec [baseline] (54.409 ms) : 0, 54409
AppSec [candidate] (54.465 ms) : 0, 54465
Debugger [baseline] (5.135 ms) : 0, 5135
Debugger [candidate] (4.446 ms) : 0, 4446
Remote Config [baseline] (686.989 µs) : 0, 687
Remote Config [candidate] (710.604 µs) : 0, 711
Telemetry [baseline] (12.764 ms) : 0, 12764
Telemetry [candidate] (12.16 ms) : 0, 12160
section appsec
BytebuddyAgent [baseline] (736.251 ms) : 0, 736251
BytebuddyAgent [candidate] (738.249 ms) : 0, 738249
GlobalTracer [baseline] (235.045 ms) : 0, 235045
GlobalTracer [candidate] (235.733 ms) : 0, 235733
AppSec [baseline] (174.929 ms) : 0, 174929
AppSec [candidate] (176.018 ms) : 0, 176018
Debugger [baseline] (4.292 ms) : 0, 4292
Debugger [candidate] (4.307 ms) : 0, 4307
Remote Config [baseline] (641.114 µs) : 0, 641
Remote Config [candidate] (636.9 µs) : 0, 637
Telemetry [baseline] (8.873 ms) : 0, 8873
Telemetry [candidate] (8.892 ms) : 0, 8892
IAST [baseline] (21.466 ms) : 0, 21466
IAST [candidate] (21.596 ms) : 0, 21596
section iast
BytebuddyAgent [baseline] (837.215 ms) : 0, 837215
BytebuddyAgent [candidate] (836.847 ms) : 0, 836847
GlobalTracer [baseline] (229.05 ms) : 0, 229050
GlobalTracer [candidate] (228.698 ms) : 0, 228698
AppSec [baseline] (55.928 ms) : 0, 55928
AppSec [candidate] (55.68 ms) : 0, 55680
Debugger [baseline] (4.236 ms) : 0, 4236
Debugger [candidate] (4.172 ms) : 0, 4172
Remote Config [baseline] (602.375 µs) : 0, 602
Remote Config [candidate] (614.477 µs) : 0, 614
Telemetry [baseline] (8.768 ms) : 0, 8768
Telemetry [candidate] (8.787 ms) : 0, 8787
IAST [baseline] (22.873 ms) : 0, 22873
IAST [candidate] (22.654 ms) : 0, 22654
section profiling
BytebuddyAgent [baseline] (708.345 ms) : 0, 708345
BytebuddyAgent [candidate] (713.866 ms) : 0, 713866
GlobalTracer [baseline] (346.907 ms) : 0, 346907
GlobalTracer [candidate] (348.788 ms) : 0, 348788
AppSec [baseline] (54.275 ms) : 0, 54275
AppSec [candidate] (53.272 ms) : 0, 53272
Debugger [baseline] (4.204 ms) : 0, 4204
Debugger [candidate] (4.219 ms) : 0, 4219
Remote Config [baseline] (695.41 µs) : 0, 695
Remote Config [candidate] (687.315 µs) : 0, 687
Telemetry [baseline] (8.789 ms) : 0, 8789
Telemetry [candidate] (8.819 ms) : 0, 8819
ProfilingAgent [baseline] (102.07 ms) : 0, 102070
ProfilingAgent [candidate] (102.556 ms) : 0, 102556
Profiling [baseline] (102.095 ms) : 0, 102095
Profiling [candidate] (102.584 ms) : 0, 102584
Loading
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.48.0-SNAPSHOT~9098f10dc3, baseline=1.48.0-SNAPSHOT~52b943d46e

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.052 s) : 0, 1051754
Total [baseline] (8.689 s) : 0, 8689439
Agent [candidate] (1.049 s) : 0, 1048955
Total [candidate] (8.672 s) : 0, 8671572
section iast
Agent [baseline] (1.178 s) : 0, 1177743
Total [baseline] (9.227 s) : 0, 9226579
Agent [candidate] (1.18 s) : 0, 1179779
Total [candidate] (9.227 s) : 0, 9227051
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.185 s) : 0, 1185180
Total [baseline] (9.253 s) : 0, 9252814
Agent [candidate] (1.187 s) : 0, 1187201
Total [candidate] (9.222 s) : 0, 9221857
section iast_TELEMETRY_OFF
Agent [baseline] (1.173 s) : 0, 1173385
Total [baseline] (9.265 s) : 0, 9264886
Agent [candidate] (1.173 s) : 0, 1173461
Total [candidate] (9.236 s) : 0, 9235533
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.052 s -
Agent iast 1.178 s 125.989 ms (12.0%)
Agent iast_HARDCODED_SECRET_DISABLED 1.185 s 133.426 ms (12.7%)
Agent iast_TELEMETRY_OFF 1.173 s 121.631 ms (11.6%)
Total tracing 8.689 s -
Total iast 9.227 s 537.141 ms (6.2%)
Total iast_HARDCODED_SECRET_DISABLED 9.253 s 563.375 ms (6.5%)
Total iast_TELEMETRY_OFF 9.265 s 575.447 ms (6.6%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.049 s -
Agent iast 1.18 s 130.824 ms (12.5%)
Agent iast_HARDCODED_SECRET_DISABLED 1.187 s 138.246 ms (13.2%)
Agent iast_TELEMETRY_OFF 1.173 s 124.506 ms (11.9%)
Total tracing 8.672 s -
Total iast 9.227 s 555.479 ms (6.4%)
Total iast_HARDCODED_SECRET_DISABLED 9.222 s 550.284 ms (6.3%)
Total iast_TELEMETRY_OFF 9.236 s 563.961 ms (6.5%) 
gantt
    title insecure-bank - break down per module: candidate=1.48.0-SNAPSHOT~9098f10dc3, baseline=1.48.0-SNAPSHOT~52b943d46e

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (719.996 ms) : 0, 719996
BytebuddyAgent [candidate] (717.553 ms) : 0, 717553
GlobalTracer [baseline] (239.031 ms) : 0, 239031
GlobalTracer [candidate] (238.678 ms) : 0, 238678
AppSec [baseline] (54.8 ms) : 0, 54800
AppSec [candidate] (54.33 ms) : 0, 54330
Debugger [baseline] (4.448 ms) : 0, 4448
Debugger [candidate] (5.787 ms) : 0, 5787
Remote Config [baseline] (703.691 µs) : 0, 704
Remote Config [candidate] (694.471 µs) : 0, 694
Telemetry [baseline] (12.998 ms) : 0, 12998
Telemetry [candidate] (12.174 ms) : 0, 12174
section iast
BytebuddyAgent [baseline] (837.051 ms) : 0, 837051
BytebuddyAgent [candidate] (838.153 ms) : 0, 838153
GlobalTracer [baseline] (229.007 ms) : 0, 229007
GlobalTracer [candidate] (229.525 ms) : 0, 229525
AppSec [baseline] (55.851 ms) : 0, 55851
AppSec [candidate] (56.061 ms) : 0, 56061
Debugger [baseline] (4.167 ms) : 0, 4167
Debugger [candidate] (4.214 ms) : 0, 4214
Remote Config [baseline] (615.132 µs) : 0, 615
Remote Config [candidate] (606.062 µs) : 0, 606
Telemetry [baseline] (8.728 ms) : 0, 8728
Telemetry [candidate] (8.783 ms) : 0, 8783
IAST [baseline] (22.685 ms) : 0, 22685
IAST [candidate] (22.765 ms) : 0, 22765
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (842.528 ms) : 0, 842528
BytebuddyAgent [candidate] (844.028 ms) : 0, 844028
GlobalTracer [baseline] (229.797 ms) : 0, 229797
GlobalTracer [candidate] (230.26 ms) : 0, 230260
AppSec [baseline] (56.553 ms) : 0, 56553
AppSec [candidate] (56.285 ms) : 0, 56285
Debugger [baseline] (4.201 ms) : 0, 4201
Debugger [candidate] (4.239 ms) : 0, 4239
Remote Config [baseline] (611.504 µs) : 0, 612
Remote Config [candidate] (626.55 µs) : 0, 627
Telemetry [baseline] (8.833 ms) : 0, 8833
Telemetry [candidate] (8.868 ms) : 0, 8868
IAST [baseline] (23.006 ms) : 0, 23006
IAST [candidate] (23.065 ms) : 0, 23065
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (833.257 ms) : 0, 833257
BytebuddyAgent [candidate] (833.311 ms) : 0, 833311
GlobalTracer [baseline] (228.803 ms) : 0, 228803
GlobalTracer [candidate] (228.523 ms) : 0, 228523
AppSec [baseline] (55.746 ms) : 0, 55746
AppSec [candidate] (56.088 ms) : 0, 56088
Debugger [baseline] (4.151 ms) : 0, 4151
Debugger [candidate] (4.23 ms) : 0, 4230
Remote Config [baseline] (598.145 µs) : 0, 598
Remote Config [candidate] (592.229 µs) : 0, 592
Telemetry [baseline] (8.628 ms) : 0, 8628
Telemetry [candidate] (8.716 ms) : 0, 8716
IAST [baseline] (22.55 ms) : 0, 22550
IAST [candidate] (22.39 ms) : 0, 22390
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2025-03-28T11:17:40 2025-03-28T11:25:27
git_branch master smola/spock-2.3
git_commit_date 1743154785 1743159258
git_commit_sha 52b943d 9098f10
release_version 1.48.0-SNAPSHOT~52b943d46e 1.48.0-SNAPSHOT~9098f10dc3
start_time 2025-03-28T11:17:26 2025-03-28T11:25:13
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1743161527 1743161527
ci_job_id 869681773 869681773
ci_pipeline_id 60311637 60311637
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-vapngg-f-project-304-concurrent-0-euu4nsk2 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-vapngg-f-project-304-concurrent-0-euu4nsk2 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
variant iast iast

Summary

Found 0 performance improvements and 1 performance regressions! Performance is the same for 13 metrics, 16 unstable metrics.

scenario Δ mean http_req_duration Δ mean throughput candidate mean http_req_duration candidate mean throughput baseline mean http_req_duration baseline mean throughput
scenario:load:petclinic:profiling worse
[+37.344µs; +88.742µs] or [+2.454%; +5.831%]		 unstable
[-797.238op/s; +357.677op/s] or [-25.910%; +11.625%]		 1.585ms 2857.143op/s 1.522ms 3076.923op/s
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.48.0-SNAPSHOT~9098f10dc3, baseline=1.48.0-SNAPSHOT~52b943d46e
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.373 ms) : 1354, 1393
.   : milestone, 1373,
appsec (1.745 ms) : 1721, 1769
.   : milestone, 1745,
appsec_no_iast (1.75 ms) : 1727, 1774
.   : milestone, 1750,
code_origins (1.702 ms) : 1676, 1728
.   : milestone, 1702,
iast (1.528 ms) : 1504, 1553
.   : milestone, 1528,
profiling (1.522 ms) : 1499, 1545
.   : milestone, 1522,
tracing (1.503 ms) : 1478, 1528
.   : milestone, 1503,
section candidate
no_agent (1.355 ms) : 1336, 1375
.   : milestone, 1355,
appsec (1.73 ms) : 1706, 1754
.   : milestone, 1730,
appsec_no_iast (1.747 ms) : 1723, 1770
.   : milestone, 1747,
code_origins (1.716 ms) : 1688, 1743
.   : milestone, 1716,
iast (1.54 ms) : 1516, 1564
.   : milestone, 1540,
profiling (1.585 ms) : 1560, 1610
.   : milestone, 1585,
tracing (1.517 ms) : 1493, 1542
.   : milestone, 1517,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.373 ms [1.354 ms, 1.393 ms] -
appsec 1.745 ms [1.721 ms, 1.769 ms] 371.886 µs (27.1%)
appsec_no_iast 1.75 ms [1.727 ms, 1.774 ms] 376.864 µs (27.4%)
code_origins 1.702 ms [1.676 ms, 1.728 ms] 328.821 µs (23.9%)
iast 1.528 ms [1.504 ms, 1.553 ms] 155.043 µs (11.3%)
profiling 1.522 ms [1.499 ms, 1.545 ms] 148.596 µs (10.8%)
tracing 1.503 ms [1.478 ms, 1.528 ms] 129.746 µs (9.4%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.355 ms [1.336 ms, 1.375 ms] -
appsec 1.73 ms [1.706 ms, 1.754 ms] 374.666 µs (27.6%)
appsec_no_iast 1.747 ms [1.723 ms, 1.77 ms] 391.284 µs (28.9%)
code_origins 1.716 ms [1.688 ms, 1.743 ms] 360.061 µs (26.6%)
iast 1.54 ms [1.516 ms, 1.564 ms] 184.656 µs (13.6%)
profiling 1.585 ms [1.56 ms, 1.61 ms] 229.545 µs (16.9%)
tracing 1.517 ms [1.493 ms, 1.542 ms] 161.98 µs (12.0%)
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.48.0-SNAPSHOT~9098f10dc3, baseline=1.48.0-SNAPSHOT~52b943d46e
    dateFormat X
    axisFormat %s
section baseline
no_agent (384.815 µs) : 365, 405
.   : milestone, 385,
iast (518.277 µs) : 496, 540
.   : milestone, 518,
iast_FULL (735.806 µs) : 714, 758
.   : milestone, 736,
iast_GLOBAL (565.971 µs) : 544, 588
.   : milestone, 566,
iast_HARDCODED_SECRET_DISABLED (519.603 µs) : 498, 542
.   : milestone, 520,
iast_INACTIVE (470.454 µs) : 449, 492
.   : milestone, 470,
iast_TELEMETRY_OFF (506.327 µs) : 485, 528
.   : milestone, 506,
tracing (466.368 µs) : 445, 488
.   : milestone, 466,
section candidate
no_agent (391.346 µs) : 371, 412
.   : milestone, 391,
iast (519.055 µs) : 497, 541
.   : milestone, 519,
iast_FULL (729.662 µs) : 708, 752
.   : milestone, 730,
iast_GLOBAL (561.704 µs) : 540, 583
.   : milestone, 562,
iast_HARDCODED_SECRET_DISABLED (512.305 µs) : 491, 534
.   : milestone, 512,
iast_INACTIVE (472.745 µs) : 451, 494
.   : milestone, 473,
iast_TELEMETRY_OFF (498.865 µs) : 477, 520
.   : milestone, 499,
tracing (463.575 µs) : 442, 485
.   : milestone, 464,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 384.815 µs [364.608 µs, 405.022 µs] -
iast 518.277 µs [496.341 µs, 540.214 µs] 133.462 µs (34.7%)
iast_FULL 735.806 µs [713.672 µs, 757.94 µs] 350.991 µs (91.2%)
iast_GLOBAL 565.971 µs [543.637 µs, 588.304 µs] 181.156 µs (47.1%)
iast_HARDCODED_SECRET_DISABLED 519.603 µs [497.589 µs, 541.618 µs] 134.788 µs (35.0%)
iast_INACTIVE 470.454 µs [449.304 µs, 491.604 µs] 85.639 µs (22.3%)
iast_TELEMETRY_OFF 506.327 µs [484.691 µs, 527.963 µs] 121.512 µs (31.6%)
tracing 466.368 µs [444.763 µs, 487.973 µs] 81.553 µs (21.2%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 391.346 µs [371.148 µs, 411.543 µs] -
iast 519.055 µs [497.076 µs, 541.033 µs] 127.709 µs (32.6%)
iast_FULL 729.662 µs [707.693 µs, 751.63 µs] 338.316 µs (86.4%)
iast_GLOBAL 561.704 µs [540.047 µs, 583.361 µs] 170.358 µs (43.5%)
iast_HARDCODED_SECRET_DISABLED 512.305 µs [490.695 µs, 533.915 µs] 120.959 µs (30.9%)
iast_INACTIVE 472.745 µs [451.181 µs, 494.308 µs] 81.399 µs (20.8%)
iast_TELEMETRY_OFF 498.865 µs [477.465 µs, 520.265 µs] 107.519 µs (27.5%)
tracing 463.575 µs [442.275 µs, 484.875 µs] 72.229 µs (18.5%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master smola/spock-2.3
git_commit_date 1743154785 1743159258
git_commit_sha 52b943d 9098f10
release_version 1.48.0-SNAPSHOT~52b943d46e 1.48.0-SNAPSHOT~9098f10dc3
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1743161950 1743161950
ci_job_id 869681774 869681774
ci_pipeline_id 60311637 60311637
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-xjcpkyqs-project-304-concurrent-1-5okvrr31 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-xjcpkyqs-project-304-concurrent-1-5okvrr31 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.48.0-SNAPSHOT~9098f10dc3, baseline=1.48.0-SNAPSHOT~52b943d46e
    dateFormat X
    axisFormat %s
section baseline
no_agent (14.983 s) : 14983000, 14983000
.   : milestone, 14983000,
appsec (15.036 s) : 15036000, 15036000
.   : milestone, 15036000,
iast (18.274 s) : 18274000, 18274000
.   : milestone, 18274000,
iast_GLOBAL (17.998 s) : 17998000, 17998000
.   : milestone, 17998000,
profiling (15.789 s) : 15789000, 15789000
.   : milestone, 15789000,
tracing (14.937 s) : 14937000, 14937000
.   : milestone, 14937000,
section candidate
no_agent (15.318 s) : 15318000, 15318000
.   : milestone, 15318000,
appsec (15.066 s) : 15066000, 15066000
.   : milestone, 15066000,
iast (18.415 s) : 18415000, 18415000
.   : milestone, 18415000,
iast_GLOBAL (17.974 s) : 17974000, 17974000
.   : milestone, 17974000,
profiling (15.592 s) : 15592000, 15592000
.   : milestone, 15592000,
tracing (15.522 s) : 15522000, 15522000
.   : milestone, 15522000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 14.983 s [14.983 s, 14.983 s] -
appsec 15.036 s [15.036 s, 15.036 s] 53.0 ms (0.4%)
iast 18.274 s [18.274 s, 18.274 s] 3.291 s (22.0%)
iast_GLOBAL 17.998 s [17.998 s, 17.998 s] 3.015 s (20.1%)
profiling 15.789 s [15.789 s, 15.789 s] 806.0 ms (5.4%)
tracing 14.937 s [14.937 s, 14.937 s] -46.0 ms (-0.3%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.318 s [15.318 s, 15.318 s] -
appsec 15.066 s [15.066 s, 15.066 s] -252.0 ms (-1.6%)
iast 18.415 s [18.415 s, 18.415 s] 3.097 s (20.2%)
iast_GLOBAL 17.974 s [17.974 s, 17.974 s] 2.656 s (17.3%)
profiling 15.592 s [15.592 s, 15.592 s] 274.0 ms (1.8%)
tracing 15.522 s [15.522 s, 15.522 s] 204.0 ms (1.3%)
Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.48.0-SNAPSHOT~9098f10dc3, baseline=1.48.0-SNAPSHOT~52b943d46e
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.466 ms) : 1455, 1478
.   : milestone, 1466,
appsec (2.335 ms) : 2292, 2379
.   : milestone, 2335,
iast (2.114 ms) : 2058, 2170
.   : milestone, 2114,
iast_GLOBAL (2.159 ms) : 2103, 2215
.   : milestone, 2159,
profiling (1.97 ms) : 1926, 2014
.   : milestone, 1970,
tracing (1.962 ms) : 1919, 2005
.   : milestone, 1962,
section candidate
no_agent (1.465 ms) : 1454, 1476
.   : milestone, 1465,
appsec (2.336 ms) : 2292, 2380
.   : milestone, 2336,
iast (2.114 ms) : 2058, 2169
.   : milestone, 2114,
iast_GLOBAL (2.159 ms) : 2103, 2215
.   : milestone, 2159,
profiling (1.969 ms) : 1925, 2014
.   : milestone, 1969,
tracing (1.95 ms) : 1907, 1992
.   : milestone, 1950,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.466 ms [1.455 ms, 1.478 ms] -
appsec 2.335 ms [2.292 ms, 2.379 ms] 869.258 µs (59.3%)
iast 2.114 ms [2.058 ms, 2.17 ms] 647.76 µs (44.2%)
iast_GLOBAL 2.159 ms [2.103 ms, 2.215 ms] 692.485 µs (47.2%)
profiling 1.97 ms [1.926 ms, 2.014 ms] 503.497 µs (34.3%)
tracing 1.962 ms [1.919 ms, 2.005 ms] 495.709 µs (33.8%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.465 ms [1.454 ms, 1.476 ms] -
appsec 2.336 ms [2.292 ms, 2.38 ms] 870.845 µs (59.4%)
iast 2.114 ms [2.058 ms, 2.169 ms] 648.467 µs (44.3%)
iast_GLOBAL 2.159 ms [2.103 ms, 2.215 ms] 693.482 µs (47.3%)
profiling 1.969 ms [1.925 ms, 2.014 ms] 504.103 µs (34.4%)
tracing 1.95 ms [1.907 ms, 1.992 ms] 484.582 µs (33.1%)

@smola smola marked this pull request as ready for review March 28, 2025 12:25
@smola smola requested a review from a team as a code owner March 28, 2025 12:25
@smola smola merged commit 4f278de into master Mar 28, 2025
250 of 271 checks passed
@smola smola deleted the smola/spock-2.3 branch March 28, 2025 13:18
@github-actions github-actions bot added this to the 1.48.0 milestone Mar 28, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@amarziali amarziali amarziali approved these changes

@manuel-alvarez-alvarez manuel-alvarez-alvarez Awaiting requested review from manuel-alvarez-alvarez manuel-alvarez-alvarez is a code owner automatically assigned from DataDog/apm-java

Assignees
No one assigned
Labels
comp: tooling Build & Tooling tag: no release notes Changes to exclude from release notes
Projects
None yet
Milestone
1.48.0
Development

Successfully merging this pull request may close these issues.
2 participants
@smola @amarziali