Add apm-sdks-benchmarks to CI by sarahchen6 · Pull Request #10775 · DataDog/dd-trace-java

sarahchen6 · 2026-03-09T17:42:02Z

What Does This Do

Add apm-sdks-benchmarks to CI

Motivation

This is a precursor to #10550 that will format these benchmarks and replace the dd-trace-java local ones

Additional Notes

Contributor Checklist

Format the title according to the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any other useful labels
Avoid using close, fix, or any linking keywords when referencing an issue
Use solves instead, and assign the PR milestone to the issue
Update the CODEOWNERS file on source file addition, migration, or deletion
Update public documentation with any new configuration flags or behaviors

Jira ticket: https://datadoghq.atlassian.net/browse/APMLP-714

Note: Once your PR is ready to merge, add it to the merge queue by commenting /merge. /merge -c cancels the queue request. /merge -f --reason "reason" skips all merge queue checks; please use this judiciously, as some checks do not run at the PR-level. For more information, see this doc.

pr-commenter · 2026-03-09T18:24:41Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	sarahchen6/add-benchmarks
git_commit_date	1773331955	1773339704
git_commit_sha	`1d0b64d`	`31857c4`
release_version	1.61.0-SNAPSHOT~1d0b64dbdb	1.61.0-SNAPSHOT~31857c40a6

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1773341522	1773341522
ci_job_id	1501875219	1501875219
ci_pipeline_id	102219374	102219374
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-itj0mx6w 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-itj0mx6w 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 64 metrics, 7 unstable metrics.

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.61.0-SNAPSHOT~31857c40a6, baseline=1.61.0-SNAPSHOT~1d0b64dbdb

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.06 s) : 0, 1059845
Total [baseline] (11.176 s) : 0, 11175803
Agent [candidate] (1.059 s) : 0, 1058581
Total [candidate] (11.064 s) : 0, 11063626
section appsec
Agent [baseline] (1.248 s) : 0, 1248295
Total [baseline] (11.071 s) : 0, 11070996
Agent [candidate] (1.248 s) : 0, 1247938
Total [candidate] (11.174 s) : 0, 11174275
section iast
Agent [baseline] (1.229 s) : 0, 1228568
Total [baseline] (11.419 s) : 0, 11419211
Agent [candidate] (1.233 s) : 0, 1233496
Total [candidate] (11.29 s) : 0, 11290052
section profiling
Agent [baseline] (1.18 s) : 0, 1179912
Total [baseline] (10.879 s) : 0, 10879472
Agent [candidate] (1.187 s) : 0, 1186707
Total [candidate] (11.028 s) : 0, 11027625

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.06 s	-
Agent	appsec	1.248 s	188.45 ms (17.8%)
Agent	iast	1.229 s	168.724 ms (15.9%)
Agent	profiling	1.18 s	120.068 ms (11.3%)
Total	tracing	11.176 s	-
Total	appsec	11.071 s	-104.807 ms (-0.9%)
Total	iast	11.419 s	243.408 ms (2.2%)
Total	profiling	10.879 s	-296.331 ms (-2.7%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.059 s	-
Agent	appsec	1.248 s	189.357 ms (17.9%)
Agent	iast	1.233 s	174.915 ms (16.5%)
Agent	profiling	1.187 s	128.127 ms (12.1%)
Total	tracing	11.064 s	-
Total	appsec	11.174 s	110.649 ms (1.0%)
Total	iast	11.29 s	226.426 ms (2.0%)
Total	profiling	11.028 s	-36.001 ms (-0.3%)

gantt
    title petclinic - break down per module: candidate=1.61.0-SNAPSHOT~31857c40a6, baseline=1.61.0-SNAPSHOT~1d0b64dbdb

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.193 ms) : 0, 1193
crashtracking [candidate] (1.198 ms) : 0, 1198
BytebuddyAgent [baseline] (627.885 ms) : 0, 627885
BytebuddyAgent [candidate] (626.645 ms) : 0, 626645
AgentMeter [baseline] (29.113 ms) : 0, 29113
AgentMeter [candidate] (29.057 ms) : 0, 29057
GlobalTracer [baseline] (256.176 ms) : 0, 256176
GlobalTracer [candidate] (256.499 ms) : 0, 256499
AppSec [baseline] (31.415 ms) : 0, 31415
AppSec [candidate] (31.467 ms) : 0, 31467
Debugger [baseline] (59.181 ms) : 0, 59181
Debugger [candidate] (59.232 ms) : 0, 59232
Remote Config [baseline] (616.339 µs) : 0, 616
Remote Config [candidate] (643.447 µs) : 0, 643
Telemetry [baseline] (8.692 ms) : 0, 8692
Telemetry [candidate] (8.603 ms) : 0, 8603
Flare Poller [baseline] (9.564 ms) : 0, 9564
Flare Poller [candidate] (9.385 ms) : 0, 9385
section appsec
crashtracking [baseline] (1.207 ms) : 0, 1207
crashtracking [candidate] (1.185 ms) : 0, 1185
BytebuddyAgent [baseline] (658.717 ms) : 0, 658717
BytebuddyAgent [candidate] (660.937 ms) : 0, 660937
AgentMeter [baseline] (12.068 ms) : 0, 12068
AgentMeter [candidate] (12.054 ms) : 0, 12054
GlobalTracer [baseline] (258.967 ms) : 0, 258967
GlobalTracer [candidate] (257.514 ms) : 0, 257514
AppSec [baseline] (178.262 ms) : 0, 178262
AppSec [candidate] (177.29 ms) : 0, 177290
Debugger [baseline] (65.648 ms) : 0, 65648
Debugger [candidate] (65.822 ms) : 0, 65822
Remote Config [baseline] (567.593 µs) : 0, 568
Remote Config [candidate] (562.286 µs) : 0, 562
Telemetry [baseline] (8.852 ms) : 0, 8852
Telemetry [candidate] (8.884 ms) : 0, 8884
Flare Poller [baseline] (3.631 ms) : 0, 3631
Flare Poller [candidate] (3.627 ms) : 0, 3627
IAST [baseline] (24.144 ms) : 0, 24144
IAST [candidate] (23.926 ms) : 0, 23926
section iast
crashtracking [baseline] (1.216 ms) : 0, 1216
crashtracking [candidate] (1.202 ms) : 0, 1202
BytebuddyAgent [baseline] (796.847 ms) : 0, 796847
BytebuddyAgent [candidate] (801.042 ms) : 0, 801042
AgentMeter [baseline] (11.392 ms) : 0, 11392
AgentMeter [candidate] (11.545 ms) : 0, 11545
GlobalTracer [baseline] (247.489 ms) : 0, 247489
GlobalTracer [candidate] (248.171 ms) : 0, 248171
AppSec [baseline] (26.522 ms) : 0, 26522
AppSec [candidate] (26.572 ms) : 0, 26572
Debugger [baseline] (64.855 ms) : 0, 64855
Debugger [candidate] (64.852 ms) : 0, 64852
Remote Config [baseline] (518.562 µs) : 0, 519
Remote Config [candidate] (511.711 µs) : 0, 512
Telemetry [baseline] (13.798 ms) : 0, 13798
Telemetry [candidate] (13.753 ms) : 0, 13753
Flare Poller [baseline] (4.623 ms) : 0, 4623
Flare Poller [candidate] (4.59 ms) : 0, 4590
IAST [baseline] (25.212 ms) : 0, 25212
IAST [candidate] (25.28 ms) : 0, 25280
section profiling
ProfilingAgent [baseline] (93.683 ms) : 0, 93683
ProfilingAgent [candidate] (93.65 ms) : 0, 93650
crashtracking [baseline] (1.168 ms) : 0, 1168
crashtracking [candidate] (1.164 ms) : 0, 1164
BytebuddyAgent [baseline] (681.368 ms) : 0, 681368
BytebuddyAgent [candidate] (685.618 ms) : 0, 685618
AgentMeter [baseline] (8.618 ms) : 0, 8618
AgentMeter [candidate] (8.699 ms) : 0, 8699
GlobalTracer [baseline] (215.227 ms) : 0, 215227
GlobalTracer [candidate] (216.438 ms) : 0, 216438
AppSec [baseline] (31.895 ms) : 0, 31895
AppSec [candidate] (32.103 ms) : 0, 32103
Debugger [baseline] (63.565 ms) : 0, 63565
Debugger [candidate] (60.257 ms) : 0, 60257
Remote Config [baseline] (588.458 µs) : 0, 588
Remote Config [candidate] (591.316 µs) : 0, 591
Telemetry [baseline] (9.65 ms) : 0, 9650
Telemetry [candidate] (12.917 ms) : 0, 12917
Flare Poller [baseline] (3.49 ms) : 0, 3490
Flare Poller [candidate] (4.305 ms) : 0, 4305
Profiling [baseline] (94.261 ms) : 0, 94261
Profiling [candidate] (94.215 ms) : 0, 94215

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.61.0-SNAPSHOT~31857c40a6, baseline=1.61.0-SNAPSHOT~1d0b64dbdb

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.058 s) : 0, 1057995
Total [baseline] (8.82 s) : 0, 8820338
Agent [candidate] (1.071 s) : 0, 1071063
Total [candidate] (8.883 s) : 0, 8882850
section iast
Agent [baseline] (1.233 s) : 0, 1233312
Total [baseline] (9.57 s) : 0, 9569751
Agent [candidate] (1.223 s) : 0, 1222950
Total [candidate] (9.557 s) : 0, 9557322

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.058 s	-
Agent	iast	1.233 s	175.317 ms (16.6%)
Total	tracing	8.82 s	-
Total	iast	9.57 s	749.413 ms (8.5%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.071 s	-
Agent	iast	1.223 s	151.887 ms (14.2%)
Total	tracing	8.883 s	-
Total	iast	9.557 s	674.472 ms (7.6%)

gantt
    title insecure-bank - break down per module: candidate=1.61.0-SNAPSHOT~31857c40a6, baseline=1.61.0-SNAPSHOT~1d0b64dbdb

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.203 ms) : 0, 1203
crashtracking [candidate] (1.208 ms) : 0, 1208
BytebuddyAgent [baseline] (628.064 ms) : 0, 628064
BytebuddyAgent [candidate] (634.309 ms) : 0, 634309
AgentMeter [baseline] (29.096 ms) : 0, 29096
AgentMeter [candidate] (29.476 ms) : 0, 29476
GlobalTracer [baseline] (256.946 ms) : 0, 256946
GlobalTracer [candidate] (260.045 ms) : 0, 260045
AppSec [baseline] (31.522 ms) : 0, 31522
AppSec [candidate] (32.054 ms) : 0, 32054
Debugger [baseline] (58.646 ms) : 0, 58646
Debugger [candidate] (59.465 ms) : 0, 59465
Remote Config [baseline] (622.984 µs) : 0, 623
Remote Config [candidate] (614.706 µs) : 0, 615
Telemetry [baseline] (8.666 ms) : 0, 8666
Telemetry [candidate] (8.869 ms) : 0, 8869
Flare Poller [baseline] (7.225 ms) : 0, 7225
Flare Poller [candidate] (8.889 ms) : 0, 8889
section iast
crashtracking [baseline] (1.212 ms) : 0, 1212
crashtracking [candidate] (1.193 ms) : 0, 1193
BytebuddyAgent [baseline] (801.833 ms) : 0, 801833
BytebuddyAgent [candidate] (793.599 ms) : 0, 793599
AgentMeter [baseline] (11.546 ms) : 0, 11546
AgentMeter [candidate] (11.296 ms) : 0, 11296
GlobalTracer [baseline] (248.187 ms) : 0, 248187
GlobalTracer [candidate] (246.737 ms) : 0, 246737
AppSec [baseline] (26.526 ms) : 0, 26526
AppSec [candidate] (26.226 ms) : 0, 26226
Debugger [baseline] (62.404 ms) : 0, 62404
Debugger [candidate] (62.71 ms) : 0, 62710
Remote Config [baseline] (509.076 µs) : 0, 509
Remote Config [candidate] (517.094 µs) : 0, 517
Telemetry [baseline] (14.755 ms) : 0, 14755
Telemetry [candidate] (14.815 ms) : 0, 14815
Flare Poller [baseline] (4.916 ms) : 0, 4916
Flare Poller [candidate] (4.831 ms) : 0, 4831
IAST [baseline] (25.339 ms) : 0, 25339
IAST [candidate] (25.013 ms) : 0, 25013

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	sarahchen6/add-benchmarks
git_commit_date	1773331955	1773339704
git_commit_sha	`1d0b64d`	`31857c4`
release_version	1.61.0-SNAPSHOT~1d0b64dbdb	1.61.0-SNAPSHOT~31857c40a6

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1773342014	1773342014
ci_job_id	1501875221	1501875221
ci_pipeline_id	102219374	102219374
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-hh94417h 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-hh94417h 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 1 performance improvements and 2 performance regressions! Performance is the same for 16 metrics, 17 unstable metrics.

scenario	Δ mean agg_http_req_duration_p50	Δ mean agg_http_req_duration_p95	Δ mean throughput	candidate mean agg_http_req_duration_p50	candidate mean agg_http_req_duration_p95	candidate mean throughput	baseline mean agg_http_req_duration_p50	baseline mean agg_http_req_duration_p95	baseline mean throughput
scenario:load:insecure-bank:iast:high_load	better [-199.505µs; -97.550µs] or [-8.156%; -3.988%]	unsure [-547.136µs; -54.892µs] or [-7.541%; -0.757%]	unstable [-61.202op/s; +224.327op/s] or [-4.248%; +15.571%]	2.298ms	6.954ms	1522.219op/s	2.446ms	7.255ms	1440.656op/s
scenario:load:insecure-bank:iast_GLOBAL:high_load	worse [+180.226µs; +295.180µs] or [+6.625%; +10.851%]	worse [+276.701µs; +844.027µs] or [+3.559%; +10.857%]	unstable [-209.748op/s; +43.810op/s] or [-16.055%; +3.353%]	2.958ms	8.334ms	1223.438op/s	2.720ms	7.774ms	1306.406op/s

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.61.0-SNAPSHOT~31857c40a6, baseline=1.61.0-SNAPSHOT~1d0b64dbdb
    dateFormat X
    axisFormat %s
section baseline
no_agent (17.899 ms) : 17715, 18084
.   : milestone, 17899,
appsec (18.517 ms) : 18328, 18706
.   : milestone, 18517,
code_origins (18.307 ms) : 18124, 18490
.   : milestone, 18307,
iast (18.315 ms) : 18132, 18497
.   : milestone, 18315,
profiling (18.926 ms) : 18736, 19117
.   : milestone, 18926,
tracing (17.789 ms) : 17611, 17967
.   : milestone, 17789,
section candidate
no_agent (18.203 ms) : 18017, 18390
.   : milestone, 18203,
appsec (19.247 ms) : 19050, 19444
.   : milestone, 19247,
code_origins (18.162 ms) : 17981, 18344
.   : milestone, 18162,
iast (17.782 ms) : 17607, 17957
.   : milestone, 17782,
profiling (19.732 ms) : 19530, 19934
.   : milestone, 19732,
tracing (17.541 ms) : 17368, 17714
.   : milestone, 17541,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	17.899 ms [17.715 ms, 18.084 ms]	-
appsec	18.517 ms [18.328 ms, 18.706 ms]	617.6 µs (3.5%)
code_origins	18.307 ms [18.124 ms, 18.49 ms]	407.418 µs (2.3%)
iast	18.315 ms [18.132 ms, 18.497 ms]	415.276 µs (2.3%)
profiling	18.926 ms [18.736 ms, 19.117 ms]	1.027 ms (5.7%)
tracing	17.789 ms [17.611 ms, 17.967 ms]	-110.625 µs (-0.6%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	18.203 ms [18.017 ms, 18.39 ms]	-
appsec	19.247 ms [19.05 ms, 19.444 ms]	1.044 ms (5.7%)
code_origins	18.162 ms [17.981 ms, 18.344 ms]	-40.853 µs (-0.2%)
iast	17.782 ms [17.607 ms, 17.957 ms]	-421.409 µs (-2.3%)
profiling	19.732 ms [19.53 ms, 19.934 ms]	1.529 ms (8.4%)
tracing	17.541 ms [17.368 ms, 17.714 ms]	-662.149 µs (-3.6%)

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.61.0-SNAPSHOT~31857c40a6, baseline=1.61.0-SNAPSHOT~1d0b64dbdb
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.192 ms) : 1180, 1203
.   : milestone, 1192,
iast (3.174 ms) : 3129, 3219
.   : milestone, 3174,
iast_FULL (5.925 ms) : 5864, 5986
.   : milestone, 5925,
iast_GLOBAL (3.508 ms) : 3446, 3571
.   : milestone, 3508,
profiling (2.168 ms) : 2149, 2188
.   : milestone, 2168,
tracing (1.813 ms) : 1797, 1829
.   : milestone, 1813,
section candidate
no_agent (1.193 ms) : 1181, 1204
.   : milestone, 1193,
iast (3.001 ms) : 2966, 3036
.   : milestone, 3001,
iast_FULL (6.066 ms) : 6005, 6128
.   : milestone, 6066,
iast_GLOBAL (3.753 ms) : 3690, 3817
.   : milestone, 3753,
profiling (2.298 ms) : 2277, 2319
.   : milestone, 2298,
tracing (1.754 ms) : 1739, 1768
.   : milestone, 1754,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.192 ms [1.18 ms, 1.203 ms]	-
iast	3.174 ms [3.129 ms, 3.219 ms]	1.982 ms (166.3%)
iast_FULL	5.925 ms [5.864 ms, 5.986 ms]	4.733 ms (397.1%)
iast_GLOBAL	3.508 ms [3.446 ms, 3.571 ms]	2.316 ms (194.4%)
profiling	2.168 ms [2.149 ms, 2.188 ms]	976.532 µs (81.9%)
tracing	1.813 ms [1.797 ms, 1.829 ms]	620.933 µs (52.1%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.193 ms [1.181 ms, 1.204 ms]	-
iast	3.001 ms [2.966 ms, 3.036 ms]	1.809 ms (151.6%)
iast_FULL	6.066 ms [6.005 ms, 6.128 ms]	4.874 ms (408.6%)
iast_GLOBAL	3.753 ms [3.69 ms, 3.817 ms]	2.561 ms (214.7%)
profiling	2.298 ms [2.277 ms, 2.319 ms]	1.105 ms (92.7%)
tracing	1.754 ms [1.739 ms, 1.768 ms]	560.91 µs (47.0%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	sarahchen6/add-benchmarks
git_commit_date	1773331955	1773339704
git_commit_sha	`1d0b64d`	`31857c4`
release_version	1.61.0-SNAPSHOT~1d0b64dbdb	1.61.0-SNAPSHOT~31857c40a6

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1773341852	1773341852
ci_job_id	1501875223	1501875223
ci_pipeline_id	102219374	102219374
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-9nw03sij 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-9nw03sij 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.61.0-SNAPSHOT~31857c40a6, baseline=1.61.0-SNAPSHOT~1d0b64dbdb
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.433 s) : 15433000, 15433000
.   : milestone, 15433000,
appsec (14.94 s) : 14940000, 14940000
.   : milestone, 14940000,
iast (18.081 s) : 18081000, 18081000
.   : milestone, 18081000,
iast_GLOBAL (17.789 s) : 17789000, 17789000
.   : milestone, 17789000,
profiling (14.991 s) : 14991000, 14991000
.   : milestone, 14991000,
tracing (15.138 s) : 15138000, 15138000
.   : milestone, 15138000,
section candidate
no_agent (15.735 s) : 15735000, 15735000
.   : milestone, 15735000,
appsec (14.962 s) : 14962000, 14962000
.   : milestone, 14962000,
iast (18.142 s) : 18142000, 18142000
.   : milestone, 18142000,
iast_GLOBAL (17.864 s) : 17864000, 17864000
.   : milestone, 17864000,
profiling (15.101 s) : 15101000, 15101000
.   : milestone, 15101000,
tracing (15.092 s) : 15092000, 15092000
.   : milestone, 15092000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.433 s [15.433 s, 15.433 s]	-
appsec	14.94 s [14.94 s, 14.94 s]	-493.0 ms (-3.2%)
iast	18.081 s [18.081 s, 18.081 s]	2.648 s (17.2%)
iast_GLOBAL	17.789 s [17.789 s, 17.789 s]	2.356 s (15.3%)
profiling	14.991 s [14.991 s, 14.991 s]	-442.0 ms (-2.9%)
tracing	15.138 s [15.138 s, 15.138 s]	-295.0 ms (-1.9%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.735 s [15.735 s, 15.735 s]	-
appsec	14.962 s [14.962 s, 14.962 s]	-773.0 ms (-4.9%)
iast	18.142 s [18.142 s, 18.142 s]	2.407 s (15.3%)
iast_GLOBAL	17.864 s [17.864 s, 17.864 s]	2.129 s (13.5%)
profiling	15.101 s [15.101 s, 15.101 s]	-634.0 ms (-4.0%)
tracing	15.092 s [15.092 s, 15.092 s]	-643.0 ms (-4.1%)

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.61.0-SNAPSHOT~31857c40a6, baseline=1.61.0-SNAPSHOT~1d0b64dbdb
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.469 ms) : 1458, 1481
.   : milestone, 1469,
appsec (3.767 ms) : 3547, 3987
.   : milestone, 3767,
iast (2.255 ms) : 2186, 2325
.   : milestone, 2255,
iast_GLOBAL (2.283 ms) : 2214, 2352
.   : milestone, 2283,
profiling (2.084 ms) : 2028, 2140
.   : milestone, 2084,
tracing (2.069 ms) : 2015, 2123
.   : milestone, 2069,
section candidate
no_agent (1.471 ms) : 1460, 1483
.   : milestone, 1471,
appsec (3.789 ms) : 3568, 4011
.   : milestone, 3789,
iast (2.248 ms) : 2179, 2318
.   : milestone, 2248,
iast_GLOBAL (2.282 ms) : 2213, 2351
.   : milestone, 2282,
profiling (2.1 ms) : 2043, 2156
.   : milestone, 2100,
tracing (2.058 ms) : 2004, 2111
.   : milestone, 2058,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.469 ms [1.458 ms, 1.481 ms]	-
appsec	3.767 ms [3.547 ms, 3.987 ms]	2.298 ms (156.4%)
iast	2.255 ms [2.186 ms, 2.325 ms]	786.292 µs (53.5%)
iast_GLOBAL	2.283 ms [2.214 ms, 2.352 ms]	814.109 µs (55.4%)
profiling	2.084 ms [2.028 ms, 2.14 ms]	614.651 µs (41.8%)
tracing	2.069 ms [2.015 ms, 2.123 ms]	600.286 µs (40.9%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.471 ms [1.46 ms, 1.483 ms]	-
appsec	3.789 ms [3.568 ms, 4.011 ms]	2.318 ms (157.6%)
iast	2.248 ms [2.179 ms, 2.318 ms]	776.888 µs (52.8%)
iast_GLOBAL	2.282 ms [2.213 ms, 2.351 ms]	810.826 µs (55.1%)
profiling	2.1 ms [2.043 ms, 2.156 ms]	628.317 µs (42.7%)
tracing	2.058 ms [2.004 ms, 2.111 ms]	586.611 µs (39.9%)

sarahchen6 force-pushed the sarahchen6/add-benchmarks branch from 930b901 to 09706c5 Compare March 9, 2026 20:09

sarahchen6 added 2 commits March 12, 2026 13:00

Add apm-sdks-benchmarks to CI

b41dfbb

Ensure artifacts are published before benchmarks run

8c4fb1c

sarahchen6 force-pushed the sarahchen6/add-benchmarks branch from c94c76c to 8c4fb1c Compare March 12, 2026 17:01

sarahchen6 added 2 commits March 12, 2026 13:04

Update benchmarks requirement

46999d6

Re-run benchmarks

31857c4

sarahchen6 mentioned this pull request Mar 12, 2026

Add apm-sdks-benchmarks to CI #10206

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add apm-sdks-benchmarks to CI#10775

Add apm-sdks-benchmarks to CI#10775
sarahchen6 wants to merge 4 commits intomasterfrom
sarahchen6/add-benchmarks

sarahchen6 commented Mar 9, 2026

Uh oh!

pr-commenter bot commented Mar 9, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

sarahchen6 commented Mar 9, 2026

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Uh oh!

pr-commenter bot commented Mar 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

pr-commenter bot commented Mar 9, 2026 •

edited

Loading