Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Decouple gradle JDK from compile/test JDK #7506

Draft
wants to merge 22 commits into
base: master
Choose a base branch
from
Draft

Decouple gradle JDK from compile/test JDK #7506

wants to merge 22 commits into from

Conversation

smola
Copy link
Member

@smola smola commented Aug 26, 2024 •
edited by jira bot
Loading

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Jira ticket: APMJAVA-1342

@smola smola added tag: no release notes Changes to exclude from release notes comp: tooling Build & Tooling labels Aug 26, 2024
@smola smola force-pushed the smola/gradle-java11 branch from dc1fa6e to 702851a Compare August 30, 2024 20:49
@pr-commenter
Copy link

pr-commenter bot commented Aug 30, 2024 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master smola/gradle-java11
git_commit_date 1743177610 1743179438
git_commit_sha 2a848ef d67f8fd
release_version 1.48.0-SNAPSHOT~2a848ef2e2 1.48.0-SNAPSHOT~d67f8fd0fd
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1743182477 1743182477
ci_job_id 870518233 870518233
ci_pipeline_id 60362083 60362083
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-kve8stnm-project-304-concurrent-2-q7b92ly4 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-kve8stnm-project-304-concurrent-2-q7b92ly4 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 67 metrics, 4 unstable metrics.

Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.48.0-SNAPSHOT~d67f8fd0fd, baseline=1.48.0-SNAPSHOT~2a848ef2e2

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.057 s) : 0, 1057132
Total [baseline] (8.703 s) : 0, 8702787
Agent [candidate] (1.062 s) : 0, 1061773
Total [candidate] (8.712 s) : 0, 8711951
section iast
Agent [baseline] (1.18 s) : 0, 1179537
Total [baseline] (9.22 s) : 0, 9220332
Agent [candidate] (1.18 s) : 0, 1179818
Total [candidate] (9.233 s) : 0, 9233216
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.189 s) : 0, 1188698
Total [baseline] (9.245 s) : 0, 9244756
Agent [candidate] (1.189 s) : 0, 1189132
Total [candidate] (9.216 s) : 0, 9216382
section iast_TELEMETRY_OFF
Agent [baseline] (1.186 s) : 0, 1185586
Total [baseline] (9.248 s) : 0, 9248241
Agent [candidate] (1.175 s) : 0, 1175197
Total [candidate] (9.246 s) : 0, 9245552
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.057 s -
Agent iast 1.18 s 122.406 ms (11.6%)
Agent iast_HARDCODED_SECRET_DISABLED 1.189 s 131.567 ms (12.4%)
Agent iast_TELEMETRY_OFF 1.186 s 128.454 ms (12.2%)
Total tracing 8.703 s -
Total iast 9.22 s 517.545 ms (5.9%)
Total iast_HARDCODED_SECRET_DISABLED 9.245 s 541.97 ms (6.2%)
Total iast_TELEMETRY_OFF 9.248 s 545.455 ms (6.3%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.062 s -
Agent iast 1.18 s 118.045 ms (11.1%)
Agent iast_HARDCODED_SECRET_DISABLED 1.189 s 127.36 ms (12.0%)
Agent iast_TELEMETRY_OFF 1.175 s 113.424 ms (10.7%)
Total tracing 8.712 s -
Total iast 9.233 s 521.265 ms (6.0%)
Total iast_HARDCODED_SECRET_DISABLED 9.216 s 504.431 ms (5.8%)
Total iast_TELEMETRY_OFF 9.246 s 533.601 ms (6.1%) 
gantt
    title insecure-bank - break down per module: candidate=1.48.0-SNAPSHOT~d67f8fd0fd, baseline=1.48.0-SNAPSHOT~2a848ef2e2

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (720.874 ms) : 0, 720874
BytebuddyAgent [candidate] (725.347 ms) : 0, 725347
GlobalTracer [baseline] (239.806 ms) : 0, 239806
GlobalTracer [candidate] (241.049 ms) : 0, 241049
AppSec [baseline] (55.44 ms) : 0, 55440
AppSec [candidate] (56.288 ms) : 0, 56288
Debugger [baseline] (4.493 ms) : 0, 4493
Debugger [candidate] (4.526 ms) : 0, 4526
Remote Config [baseline] (682.79 µs) : 0, 683
Remote Config [candidate] (697.906 µs) : 0, 698
Telemetry [baseline] (14.413 ms) : 0, 14413
Telemetry [candidate] (13.072 ms) : 0, 13072
section iast
BytebuddyAgent [baseline] (837.658 ms) : 0, 837658
BytebuddyAgent [candidate] (837.832 ms) : 0, 837832
GlobalTracer [baseline] (228.889 ms) : 0, 228889
GlobalTracer [candidate] (229.392 ms) : 0, 229392
IAST [baseline] (22.842 ms) : 0, 22842
IAST [candidate] (22.901 ms) : 0, 22901
AppSec [baseline] (55.785 ms) : 0, 55785
AppSec [candidate] (55.552 ms) : 0, 55552
Debugger [baseline] (4.23 ms) : 0, 4230
Debugger [candidate] (4.198 ms) : 0, 4198
Remote Config [baseline] (630.614 µs) : 0, 631
Remote Config [candidate] (609.871 µs) : 0, 610
Telemetry [baseline] (8.911 ms) : 0, 8911
Telemetry [candidate] (8.816 ms) : 0, 8816
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (844.65 ms) : 0, 844650
BytebuddyAgent [candidate] (844.964 ms) : 0, 844964
GlobalTracer [baseline] (230.439 ms) : 0, 230439
GlobalTracer [candidate] (230.41 ms) : 0, 230410
IAST [baseline] (23.022 ms) : 0, 23022
IAST [candidate] (23.187 ms) : 0, 23187
AppSec [baseline] (56.092 ms) : 0, 56092
AppSec [candidate] (56.205 ms) : 0, 56205
Debugger [baseline] (4.235 ms) : 0, 4235
Debugger [candidate] (4.249 ms) : 0, 4249
Remote Config [baseline] (621.907 µs) : 0, 622
Remote Config [candidate] (614.126 µs) : 0, 614
Telemetry [baseline] (8.995 ms) : 0, 8995
Telemetry [candidate] (8.867 ms) : 0, 8867
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (841.772 ms) : 0, 841772
BytebuddyAgent [candidate] (833.981 ms) : 0, 833981
GlobalTracer [baseline] (230.569 ms) : 0, 230569
GlobalTracer [candidate] (228.894 ms) : 0, 228894
IAST [baseline] (22.608 ms) : 0, 22608
IAST [candidate] (22.273 ms) : 0, 22273
AppSec [baseline] (56.442 ms) : 0, 56442
AppSec [candidate] (55.914 ms) : 0, 55914
Debugger [baseline] (4.163 ms) : 0, 4163
Debugger [candidate] (4.188 ms) : 0, 4188
Remote Config [baseline] (621.459 µs) : 0, 621
Remote Config [candidate] (630.0 µs) : 0, 630
Telemetry [baseline] (8.802 ms) : 0, 8802
Telemetry [candidate] (8.869 ms) : 0, 8869
Loading
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.48.0-SNAPSHOT~d67f8fd0fd, baseline=1.48.0-SNAPSHOT~2a848ef2e2

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.057 s) : 0, 1057074
Total [baseline] (10.439 s) : 0, 10439368
Agent [candidate] (1.051 s) : 0, 1050980
Total [candidate] (10.424 s) : 0, 10423749
section appsec
Agent [baseline] (1.195 s) : 0, 1194541
Total [baseline] (10.74 s) : 0, 10740155
Agent [candidate] (1.193 s) : 0, 1192851
Total [candidate] (10.761 s) : 0, 10761108
section iast
Agent [baseline] (1.185 s) : 0, 1185419
Total [baseline] (11.055 s) : 0, 11054602
Agent [candidate] (1.196 s) : 0, 1196191
Total [candidate] (11.033 s) : 0, 11033122
section profiling
Agent [baseline] (1.275 s) : 0, 1274755
Total [baseline] (10.847 s) : 0, 10846765
Agent [candidate] (1.276 s) : 0, 1275672
Total [candidate] (10.837 s) : 0, 10836891
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.057 s -
Agent appsec 1.195 s 137.467 ms (13.0%)
Agent iast 1.185 s 128.345 ms (12.1%)
Agent profiling 1.275 s 217.681 ms (20.6%)
Total tracing 10.439 s -
Total appsec 10.74 s 300.787 ms (2.9%)
Total iast 11.055 s 615.235 ms (5.9%)
Total profiling 10.847 s 407.398 ms (3.9%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.051 s -
Agent appsec 1.193 s 141.871 ms (13.5%)
Agent iast 1.196 s 145.211 ms (13.8%)
Agent profiling 1.276 s 224.692 ms (21.4%)
Total tracing 10.424 s -
Total appsec 10.761 s 337.358 ms (3.2%)
Total iast 11.033 s 609.373 ms (5.8%)
Total profiling 10.837 s 413.142 ms (4.0%) 
gantt
    title petclinic - break down per module: candidate=1.48.0-SNAPSHOT~d67f8fd0fd, baseline=1.48.0-SNAPSHOT~2a848ef2e2

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (720.563 ms) : 0, 720563
BytebuddyAgent [candidate] (717.703 ms) : 0, 717703
GlobalTracer [baseline] (240.103 ms) : 0, 240103
GlobalTracer [candidate] (238.655 ms) : 0, 238655
AppSec [baseline] (54.674 ms) : 0, 54674
AppSec [candidate] (54.171 ms) : 0, 54171
Debugger [baseline] (4.454 ms) : 0, 4454
Debugger [candidate] (4.412 ms) : 0, 4412
Remote Config [baseline] (696.173 µs) : 0, 696
Remote Config [candidate] (698.912 µs) : 0, 699
Telemetry [baseline] (15.934 ms) : 0, 15934
Telemetry [candidate] (14.763 ms) : 0, 14763
section appsec
BytebuddyAgent [baseline] (737.74 ms) : 0, 737740
BytebuddyAgent [candidate] (737.217 ms) : 0, 737217
GlobalTracer [baseline] (235.568 ms) : 0, 235568
GlobalTracer [candidate] (235.244 ms) : 0, 235244
IAST [baseline] (21.458 ms) : 0, 21458
IAST [candidate] (21.607 ms) : 0, 21607
AppSec [baseline] (176.431 ms) : 0, 176431
AppSec [candidate] (175.968 ms) : 0, 175968
Debugger [baseline] (4.314 ms) : 0, 4314
Debugger [candidate] (4.267 ms) : 0, 4267
Remote Config [baseline] (650.895 µs) : 0, 651
Remote Config [candidate] (639.206 µs) : 0, 639
Telemetry [baseline] (8.571 ms) : 0, 8571
Telemetry [candidate] (8.197 ms) : 0, 8197
section iast
BytebuddyAgent [baseline] (841.666 ms) : 0, 841666
BytebuddyAgent [candidate] (850.51 ms) : 0, 850510
GlobalTracer [baseline] (230.539 ms) : 0, 230539
GlobalTracer [candidate] (231.45 ms) : 0, 231450
IAST [baseline] (23.013 ms) : 0, 23013
IAST [candidate] (23.227 ms) : 0, 23227
AppSec [baseline] (55.982 ms) : 0, 55982
AppSec [candidate] (56.446 ms) : 0, 56446
Debugger [baseline] (4.22 ms) : 0, 4220
Debugger [candidate] (4.216 ms) : 0, 4216
Remote Config [baseline] (613.279 µs) : 0, 613
Remote Config [candidate] (622.124 µs) : 0, 622
Telemetry [baseline] (8.849 ms) : 0, 8849
Telemetry [candidate] (8.97 ms) : 0, 8970
section profiling
BytebuddyAgent [baseline] (711.388 ms) : 0, 711388
BytebuddyAgent [candidate] (711.269 ms) : 0, 711269
GlobalTracer [baseline] (347.396 ms) : 0, 347396
GlobalTracer [candidate] (349.181 ms) : 0, 349181
AppSec [baseline] (54.009 ms) : 0, 54009
AppSec [candidate] (53.195 ms) : 0, 53195
Debugger [baseline] (4.305 ms) : 0, 4305
Debugger [candidate] (4.295 ms) : 0, 4295
Remote Config [baseline] (715.121 µs) : 0, 715
Remote Config [candidate] (696.462 µs) : 0, 696
Telemetry [baseline] (8.941 ms) : 0, 8941
Telemetry [candidate] (9.0 ms) : 0, 9000
ProfilingAgent [baseline] (102.104 ms) : 0, 102104
ProfilingAgent [candidate] (102.304 ms) : 0, 102304
Profiling [baseline] (102.129 ms) : 0, 102129
Profiling [candidate] (102.329 ms) : 0, 102329
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2025-03-28T16:51:34 2025-03-28T16:59:21
git_branch master smola/gradle-java11
git_commit_date 1743177610 1743179438
git_commit_sha 2a848ef d67f8fd
release_version 1.48.0-SNAPSHOT~2a848ef2e2 1.48.0-SNAPSHOT~d67f8fd0fd
start_time 2025-03-28T16:51:20 2025-03-28T16:59:07
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1743181560 1743181560
ci_job_id 870518234 870518234
ci_pipeline_id 60362083 60362083
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-5gca-jya-project-304-concurrent-0-xpj4ivbg 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-5gca-jya-project-304-concurrent-0-xpj4ivbg 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
variant iast iast

Summary

Found 1 performance improvements and 0 performance regressions! Performance is the same for 13 metrics, 16 unstable metrics.

scenario Δ mean http_req_duration Δ mean throughput candidate mean http_req_duration candidate mean throughput baseline mean http_req_duration baseline mean throughput
scenario:load:petclinic:profiling better
[-91.656µs; -40.047µs] or [-5.792%; -2.531%]		 unstable
[-422.819op/s; +650.739op/s] or [-14.270%; +21.962%]		 1.517ms 3076.923op/s 1.582ms 2962.963op/s
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.48.0-SNAPSHOT~d67f8fd0fd, baseline=1.48.0-SNAPSHOT~2a848ef2e2
    dateFormat X
    axisFormat %s
section baseline
no_agent (384.088 µs) : 363, 405
.   : milestone, 384,
iast (522.044 µs) : 500, 544
.   : milestone, 522,
iast_FULL (730.527 µs) : 709, 753
.   : milestone, 731,
iast_GLOBAL (571.459 µs) : 549, 594
.   : milestone, 571,
iast_HARDCODED_SECRET_DISABLED (516.802 µs) : 495, 539
.   : milestone, 517,
iast_INACTIVE (469.74 µs) : 448, 491
.   : milestone, 470,
iast_TELEMETRY_OFF (503.291 µs) : 481, 525
.   : milestone, 503,
tracing (458.548 µs) : 438, 479
.   : milestone, 459,
section candidate
no_agent (390.604 µs) : 371, 410
.   : milestone, 391,
iast (517.441 µs) : 495, 539
.   : milestone, 517,
iast_FULL (731.734 µs) : 710, 754
.   : milestone, 732,
iast_GLOBAL (568.688 µs) : 546, 591
.   : milestone, 569,
iast_HARDCODED_SECRET_DISABLED (514.958 µs) : 493, 537
.   : milestone, 515,
iast_INACTIVE (469.43 µs) : 448, 491
.   : milestone, 469,
iast_TELEMETRY_OFF (507.359 µs) : 486, 529
.   : milestone, 507,
tracing (464.27 µs) : 443, 486
.   : milestone, 464,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 384.088 µs [363.428 µs, 404.747 µs] -
iast 522.044 µs [500.035 µs, 544.052 µs] 137.956 µs (35.9%)
iast_FULL 730.527 µs [708.521 µs, 752.533 µs] 346.439 µs (90.2%)
iast_GLOBAL 571.459 µs [548.939 µs, 593.979 µs] 187.371 µs (48.8%)
iast_HARDCODED_SECRET_DISABLED 516.802 µs [494.503 µs, 539.102 µs] 132.715 µs (34.6%)
iast_INACTIVE 469.74 µs [448.008 µs, 491.472 µs] 85.652 µs (22.3%)
iast_TELEMETRY_OFF 503.291 µs [481.382 µs, 525.201 µs] 119.203 µs (31.0%)
tracing 458.548 µs [437.633 µs, 479.462 µs] 74.46 µs (19.4%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 390.604 µs [370.917 µs, 410.291 µs] -
iast 517.441 µs [495.45 µs, 539.431 µs] 126.837 µs (32.5%)
iast_FULL 731.734 µs [709.921 µs, 753.548 µs] 341.13 µs (87.3%)
iast_GLOBAL 568.688 µs [546.264 µs, 591.111 µs] 178.084 µs (45.6%)
iast_HARDCODED_SECRET_DISABLED 514.958 µs [493.109 µs, 536.806 µs] 124.354 µs (31.8%)
iast_INACTIVE 469.43 µs [447.636 µs, 491.225 µs] 78.826 µs (20.2%)
iast_TELEMETRY_OFF 507.359 µs [485.705 µs, 529.012 µs] 116.755 µs (29.9%)
tracing 464.27 µs [442.709 µs, 485.832 µs] 73.666 µs (18.9%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.48.0-SNAPSHOT~d67f8fd0fd, baseline=1.48.0-SNAPSHOT~2a848ef2e2
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.363 ms) : 1343, 1383
.   : milestone, 1363,
appsec (1.747 ms) : 1723, 1771
.   : milestone, 1747,
appsec_no_iast (1.722 ms) : 1698, 1746
.   : milestone, 1722,
code_origins (1.696 ms) : 1668, 1724
.   : milestone, 1696,
iast (1.511 ms) : 1486, 1535
.   : milestone, 1511,
profiling (1.582 ms) : 1557, 1607
.   : milestone, 1582,
tracing (1.499 ms) : 1475, 1524
.   : milestone, 1499,
section candidate
no_agent (1.357 ms) : 1338, 1376
.   : milestone, 1357,
appsec (1.73 ms) : 1706, 1753
.   : milestone, 1730,
appsec_no_iast (1.728 ms) : 1704, 1752
.   : milestone, 1728,
code_origins (1.699 ms) : 1672, 1726
.   : milestone, 1699,
iast (1.52 ms) : 1495, 1544
.   : milestone, 1520,
profiling (1.517 ms) : 1494, 1540
.   : milestone, 1517,
tracing (1.49 ms) : 1464, 1516
.   : milestone, 1490,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.363 ms [1.343 ms, 1.383 ms] -
appsec 1.747 ms [1.723 ms, 1.771 ms] 384.141 µs (28.2%)
appsec_no_iast 1.722 ms [1.698 ms, 1.746 ms] 359.152 µs (26.3%)
code_origins 1.696 ms [1.668 ms, 1.724 ms] 333.429 µs (24.5%)
iast 1.511 ms [1.486 ms, 1.535 ms] 147.934 µs (10.9%)
profiling 1.582 ms [1.557 ms, 1.607 ms] 219.417 µs (16.1%)
tracing 1.499 ms [1.475 ms, 1.524 ms] 136.195 µs (10.0%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.357 ms [1.338 ms, 1.376 ms] -
appsec 1.73 ms [1.706 ms, 1.753 ms] 372.71 µs (27.5%)
appsec_no_iast 1.728 ms [1.704 ms, 1.752 ms] 371.17 µs (27.4%)
code_origins 1.699 ms [1.672 ms, 1.726 ms] 342.276 µs (25.2%)
iast 1.52 ms [1.495 ms, 1.544 ms] 162.77 µs (12.0%)
profiling 1.517 ms [1.494 ms, 1.54 ms] 159.767 µs (11.8%)
tracing 1.49 ms [1.464 ms, 1.516 ms] 133.225 µs (9.8%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master smola/gradle-java11
git_commit_date 1743177610 1743179438
git_commit_sha 2a848ef d67f8fd
release_version 1.48.0-SNAPSHOT~2a848ef2e2 1.48.0-SNAPSHOT~d67f8fd0fd
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1743181924 1743181924
ci_job_id 870518235 870518235
ci_pipeline_id 60362083 60362083
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-kve8stnm-project-304-concurrent-3-yjlwaord 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-kve8stnm-project-304-concurrent-3-yjlwaord 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.48.0-SNAPSHOT~d67f8fd0fd, baseline=1.48.0-SNAPSHOT~2a848ef2e2
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.475 s) : 15475000, 15475000
.   : milestone, 15475000,
appsec (14.945 s) : 14945000, 14945000
.   : milestone, 14945000,
iast (18.683 s) : 18683000, 18683000
.   : milestone, 18683000,
iast_GLOBAL (18.093 s) : 18093000, 18093000
.   : milestone, 18093000,
profiling (15.503 s) : 15503000, 15503000
.   : milestone, 15503000,
tracing (14.911 s) : 14911000, 14911000
.   : milestone, 14911000,
section candidate
no_agent (15.626 s) : 15626000, 15626000
.   : milestone, 15626000,
appsec (15.091 s) : 15091000, 15091000
.   : milestone, 15091000,
iast (18.471 s) : 18471000, 18471000
.   : milestone, 18471000,
iast_GLOBAL (18.144 s) : 18144000, 18144000
.   : milestone, 18144000,
profiling (15.594 s) : 15594000, 15594000
.   : milestone, 15594000,
tracing (15.257 s) : 15257000, 15257000
.   : milestone, 15257000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.475 s [15.475 s, 15.475 s] -
appsec 14.945 s [14.945 s, 14.945 s] -530.0 ms (-3.4%)
iast 18.683 s [18.683 s, 18.683 s] 3.208 s (20.7%)
iast_GLOBAL 18.093 s [18.093 s, 18.093 s] 2.618 s (16.9%)
profiling 15.503 s [15.503 s, 15.503 s] 28.0 ms (0.2%)
tracing 14.911 s [14.911 s, 14.911 s] -564.0 ms (-3.6%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.626 s [15.626 s, 15.626 s] -
appsec 15.091 s [15.091 s, 15.091 s] -535.0 ms (-3.4%)
iast 18.471 s [18.471 s, 18.471 s] 2.845 s (18.2%)
iast_GLOBAL 18.144 s [18.144 s, 18.144 s] 2.518 s (16.1%)
profiling 15.594 s [15.594 s, 15.594 s] -32.0 ms (-0.2%)
tracing 15.257 s [15.257 s, 15.257 s] -369.0 ms (-2.4%)
Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.48.0-SNAPSHOT~d67f8fd0fd, baseline=1.48.0-SNAPSHOT~2a848ef2e2
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.479 ms) : 1468, 1491
.   : milestone, 1479,
appsec (2.379 ms) : 2334, 2424
.   : milestone, 2379,
iast (2.142 ms) : 2085, 2199
.   : milestone, 2142,
iast_GLOBAL (2.193 ms) : 2135, 2251
.   : milestone, 2193,
profiling (1.997 ms) : 1952, 2043
.   : milestone, 1997,
tracing (1.966 ms) : 1922, 2009
.   : milestone, 1966,
section candidate
no_agent (1.475 ms) : 1464, 1487
.   : milestone, 1475,
appsec (2.369 ms) : 2324, 2414
.   : milestone, 2369,
iast (2.15 ms) : 2093, 2207
.   : milestone, 2150,
iast_GLOBAL (2.196 ms) : 2138, 2253
.   : milestone, 2196,
profiling (2.009 ms) : 1963, 2056
.   : milestone, 2009,
tracing (1.983 ms) : 1939, 2027
.   : milestone, 1983,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.479 ms [1.468 ms, 1.491 ms] -
appsec 2.379 ms [2.334 ms, 2.424 ms] 899.746 µs (60.8%)
iast 2.142 ms [2.085 ms, 2.199 ms] 662.753 µs (44.8%)
iast_GLOBAL 2.193 ms [2.135 ms, 2.251 ms] 713.827 µs (48.3%)
profiling 1.997 ms [1.952 ms, 2.043 ms] 518.162 µs (35.0%)
tracing 1.966 ms [1.922 ms, 2.009 ms] 486.832 µs (32.9%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.475 ms [1.464 ms, 1.487 ms] -
appsec 2.369 ms [2.324 ms, 2.414 ms] 894.099 µs (60.6%)
iast 2.15 ms [2.093 ms, 2.207 ms] 674.611 µs (45.7%)
iast_GLOBAL 2.196 ms [2.138 ms, 2.253 ms] 720.705 µs (48.9%)
profiling 2.009 ms [1.963 ms, 2.056 ms] 534.186 µs (36.2%)
tracing 1.983 ms [1.939 ms, 2.027 ms] 507.722 µs (34.4%)

@smola smola force-pushed the smola/gradle-java11 branch from 702851a to d1abf3b Compare September 23, 2024 11:18
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jan 1, 2025

This pull request has been marked as stale because it has not had activity over the past quarter. It will be closed in 7 days if no further activity occurs. Feel free to reopen the PR if you are still working on it.

@github-actions github-actions bot added the tag: stale Stale pull requests label Jan 1, 2025
@smola smola force-pushed the smola/gradle-java11 branch 2 times, most recently from 0e79c09 to 0a43968 Compare January 15, 2025 15:19
@pr-commenter
Copy link

pr-commenter bot commented Jan 15, 2025 •
edited
Loading

Debugger benchmarks

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
ci_job_date 1737357243 1737357619
end_time 2025-01-20T07:15:19 2025-01-20T07:21:36
git_branch master smola/gradle-java11
git_commit_sha 0b239ae 352736e
start_time 2025-01-20T07:14:04 2025-01-20T07:20:20
See matching parameters
Baseline Candidate
ci_job_id 772223798 772223798
ci_pipeline_id 53445931 53445931
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
git_commit_date 1737222884 1737222884

Summary

Found 0 performance improvements and 6 performance regressions! Performance is the same for 3 metrics, 6 unstable metrics.

scenario Δ mean agg_http_req_duration_min Δ mean agg_http_req_duration_p50 Δ mean agg_http_req_duration_p75 Δ mean agg_http_req_duration_p99 Δ mean throughput
scenario:basic worse
[+3.576µs; +11.815µs] or [+1.324%; +4.374%]		 same same unstable
[-23.734µs; +39.109µs] or [-4.673%; +7.700%]		 unstable
[-239.733op/s; +111.528op/s] or [-9.350%; +4.350%]
scenario:loop worse
[+442.270µs; +522.596µs] or [+4.356%; +5.147%]		 worse
[+449.109µs; +550.782µs] or [+4.353%; +5.338%]		 worse
[+410.327µs; +544.125µs] or [+3.935%; +5.218%]		 worse
[+195.678µs; +703.388µs] or [+1.812%; +6.512%]		 worse
[-5.491op/s; -3.248op/s] or [-5.738%; -3.394%]
See unchanged results
scenario Δ mean agg_http_req_duration_min Δ mean agg_http_req_duration_p50 Δ mean agg_http_req_duration_p75 Δ mean agg_http_req_duration_p99 Δ mean throughput
scenario:noprobe unstable
[-26.397µs; +21.584µs] or [-9.744%; +7.968%]		 unstable
[-39.342µs; +33.189µs] or [-12.663%; +10.682%]		 unstable
[-52.143µs; +42.935µs] or [-16.038%; +13.206%]		 unstable
[-94.545µs; +97.240µs] or [-16.186%; +16.647%]		 same
Request duration reports for reports 
gantt
    title reports - request duration [CI 0.99] : candidate=None, baseline=None
    dateFormat X
    axisFormat %s
section baseline
noprobe (310.695 µs) : 270, 352
.   : milestone, 311,
basic (301.977 µs) : 293, 310
.   : milestone, 302,
loop (10.317 ms) : 10257, 10378
.   : milestone, 10317,
section candidate
noprobe (307.618 µs) : 281, 334
.   : milestone, 308,
basic (309.887 µs) : 301, 319
.   : milestone, 310,
loop (10.817 ms) : 10788, 10847
.   : milestone, 10817,
Loading
  • baseline results
Scenario Request median duration [CI 0.99]
noprobe 310.695 µs [269.778 µs, 351.611 µs]
basic 301.977 µs [293.457 µs, 310.498 µs]
loop 10.317 ms [10.257 ms, 10.378 ms]
  • candidate results
Scenario Request median duration [CI 0.99]
noprobe 307.618 µs [280.843 µs, 334.393 µs]
basic 309.887 µs [301.101 µs, 318.674 µs]
loop 10.817 ms [10.788 ms, 10.847 ms]

@smola smola force-pushed the smola/gradle-java11 branch 3 times, most recently from 22e3221 to 352736e Compare January 20, 2025 06:59
@pr-commenter
Copy link

pr-commenter bot commented Jan 20, 2025 •
edited
Loading

Kafka / producer-benchmark

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master smola/gradle-java11
git_commit_date 1743177610 1743179438
git_commit_sha 2a848ef d67f8fd
See matching parameters
Baseline Candidate
ci_job_date 1743181110 1743181110
ci_job_id 870518237 870518237
ci_pipeline_id 60362083 60362083
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
jdkVersion 11.0.25 11.0.25
jmhVersion 1.36 1.36
jvm /usr/lib/jvm/java-11-openjdk-amd64/bin/java /usr/lib/jvm/java-11-openjdk-amd64/bin/java
jvmArgs -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/producer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/producer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant
vmName OpenJDK 64-Bit Server VM OpenJDK 64-Bit Server VM
vmVersion 11.0.25+9-post-Ubuntu-1ubuntu122.04 11.0.25+9-post-Ubuntu-1ubuntu122.04

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics.

See unchanged results
scenario Δ mean throughput
scenario:not-instrumented/KafkaProduceBenchmark.benchProduce same
scenario:only-tracing-dsm-disabled-benchmarks/KafkaProduceBenchmark.benchProduce same
scenario:only-tracing-dsm-enabled-benchmarks/KafkaProduceBenchmark.benchProduce same

@pr-commenter
Copy link

pr-commenter bot commented Jan 20, 2025 •
edited
Loading

Kafka / consumer-benchmark

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master smola/gradle-java11
git_commit_date 1743177610 1743179438
git_commit_sha 2a848ef d67f8fd
See matching parameters
Baseline Candidate
ci_job_date 1743181168 1743181168
ci_job_id 870518238 870518238
ci_pipeline_id 60362083 60362083
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
jdkVersion 11.0.25 11.0.25
jmhVersion 1.36 1.36
jvm /usr/lib/jvm/java-11-openjdk-amd64/bin/java /usr/lib/jvm/java-11-openjdk-amd64/bin/java
jvmArgs -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/consumer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/consumer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant
vmName OpenJDK 64-Bit Server VM OpenJDK 64-Bit Server VM
vmVersion 11.0.25+9-post-Ubuntu-1ubuntu122.04 11.0.25+9-post-Ubuntu-1ubuntu122.04

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics.

See unchanged results
scenario Δ mean throughput
scenario:not-instrumented/KafkaConsumerBenchmark.benchConsume same
scenario:only-tracing-dsm-disabled-benchmarks/KafkaConsumerBenchmark.benchConsume same
scenario:only-tracing-dsm-enabled-benchmarks/KafkaConsumerBenchmark.benchConsume same

@smola smola removed the tag: stale Stale pull requests label Jan 20, 2025
@smola smola force-pushed the smola/gradle-java11 branch 2 times, most recently from 920f3ba to 8573d24 Compare January 24, 2025 08:33
@smola smola mentioned this pull request Jan 24, 2025
Merged
@smola smola force-pushed the smola/gradle-java11 branch from 8573d24 to 4812847 Compare January 27, 2025 11:53
@smola smola force-pushed the smola/gradle-java11 branch from 4812847 to 0db0b97 Compare March 28, 2025 15:10
@smola smola force-pushed the smola/gradle-java11 branch from 2a3f2a3 to d67f8fd Compare March 28, 2025 16:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
comp: tooling Build & Tooling tag: no release notes Changes to exclude from release notes
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@smola