Skip to content

POC; migrating DSM integration tests to dsmTest #10199

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
robcarlan-datadog wants to merge 1 commit into from
Closed

POC; migrating DSM integration tests to dsmTest #10199

robcarlan-datadog wants to merge 1 commit into from

Conversation

@robcarlan-datadog
Copy link
Contributor

@robcarlan-datadog robcarlan-datadog commented Dec 12, 2025

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Jira ticket: [PROJ-IDENT]

@robcarlan-datadog robcarlan-datadog changed the title POC; migrating java tests to dsmTest POC; migrating DSM integration tests to dsmTest Dec 12, 2025
@pr-commenter
Copy link

pr-commenter bot commented Dec 12, 2025

Kafka / producer-benchmark

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master rob.carlan/DSMON-1160/refactor-dsm-test-ownership
git_commit_date 1765556506 1765558125
git_commit_sha c2560c4 be2ebd3
See matching parameters
Baseline Candidate
ci_job_date 1765559223 1765559223
ci_job_id 1291494416 1291494416
ci_pipeline_id 86525701 86525701
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
jdkVersion 11.0.25 11.0.25
jmhVersion 1.36 1.36
jvm /usr/lib/jvm/java-11-openjdk-amd64/bin/java /usr/lib/jvm/java-11-openjdk-amd64/bin/java
jvmArgs -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/producer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/producer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant
vmName OpenJDK 64-Bit Server VM OpenJDK 64-Bit Server VM
vmVersion 11.0.25+9-post-Ubuntu-1ubuntu122.04 11.0.25+9-post-Ubuntu-1ubuntu122.04

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics.

See unchanged results
scenario Δ mean throughput
scenario:not-instrumented/KafkaProduceBenchmark.benchProduce same
scenario:only-tracing-dsm-disabled-benchmarks/KafkaProduceBenchmark.benchProduce same
scenario:only-tracing-dsm-enabled-benchmarks/KafkaProduceBenchmark.benchProduce same

@pr-commenter
Copy link

pr-commenter bot commented Dec 12, 2025

Benchmarks

⚠️ Warning: Baseline build not found for merge-base commit. Comparing against the latest commit on master instead.

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master rob.carlan/DSMON-1160/refactor-dsm-test-ownership
git_commit_date 1765557831 1765558125
git_commit_sha f832a5b be2ebd3
release_version 1.57.0-SNAPSHOT~f832a5b4ad 1.57.0-SNAPSHOT~be2ebd3005
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1765559984 1765559984
ci_job_id 1291494411 1291494411
ci_pipeline_id 86525701 86525701
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-0-933ynava 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-0-933ynava 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 58 metrics, 7 unstable metrics.

Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.57.0-SNAPSHOT~be2ebd3005, baseline=1.57.0-SNAPSHOT~f832a5b4ad

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.077 s) : 0, 1077293
Total [baseline] (10.841 s) : 0, 10841382
Agent [candidate] (1.088 s) : 0, 1087790
Total [candidate] (10.808 s) : 0, 10808133
section appsec
Agent [baseline] (1.261 s) : 0, 1261354
Total [baseline] (11.125 s) : 0, 11124512
Agent [candidate] (1.265 s) : 0, 1264593
Total [candidate] (10.968 s) : 0, 10968286
section iast
Agent [baseline] (1.218 s) : 0, 1217835
Total [baseline] (11.199 s) : 0, 11199465
Agent [candidate] (1.224 s) : 0, 1224145
Total [candidate] (11.251 s) : 0, 11250917
section profiling
Agent [baseline] (1.212 s) : 0, 1211700
Total [baseline] (11.016 s) : 0, 11015951
Agent [candidate] (1.211 s) : 0, 1210610
Total [candidate] (10.944 s) : 0, 10944292
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.077 s -
Agent appsec 1.261 s 184.062 ms (17.1%)
Agent iast 1.218 s 140.542 ms (13.0%)
Agent profiling 1.212 s 134.407 ms (12.5%)
Total tracing 10.841 s -
Total appsec 11.125 s 283.13 ms (2.6%)
Total iast 11.199 s 358.082 ms (3.3%)
Total profiling 11.016 s 174.568 ms (1.6%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.088 s -
Agent appsec 1.265 s 176.802 ms (16.3%)
Agent iast 1.224 s 136.355 ms (12.5%)
Agent profiling 1.211 s 122.82 ms (11.3%)
Total tracing 10.808 s -
Total appsec 10.968 s 160.153 ms (1.5%)
Total iast 11.251 s 442.784 ms (4.1%)
Total profiling 10.944 s 136.159 ms (1.3%) 
gantt
    title petclinic - break down per module: candidate=1.57.0-SNAPSHOT~be2ebd3005, baseline=1.57.0-SNAPSHOT~f832a5b4ad

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.18 ms) : 0, 1180
crashtracking [candidate] (1.198 ms) : 0, 1198
BytebuddyAgent [baseline] (646.366 ms) : 0, 646366
BytebuddyAgent [candidate] (652.756 ms) : 0, 652756
GlobalTracer [baseline] (280.744 ms) : 0, 280744
GlobalTracer [candidate] (283.477 ms) : 0, 283477
AppSec [baseline] (32.226 ms) : 0, 32226
AppSec [candidate] (32.655 ms) : 0, 32655
Debugger [baseline] (67.896 ms) : 0, 67896
Debugger [candidate] (68.436 ms) : 0, 68436
Remote Config [baseline] (653.302 µs) : 0, 653
Remote Config [candidate] (636.635 µs) : 0, 637
Telemetry [baseline] (8.959 ms) : 0, 8959
Telemetry [candidate] (9.115 ms) : 0, 9115
Flare Poller [baseline] (3.714 ms) : 0, 3714
Flare Poller [candidate] (3.823 ms) : 0, 3823
section appsec
crashtracking [baseline] (1.18 ms) : 0, 1180
crashtracking [candidate] (1.192 ms) : 0, 1192
BytebuddyAgent [baseline] (687.005 ms) : 0, 687005
BytebuddyAgent [candidate] (688.122 ms) : 0, 688122
GlobalTracer [baseline] (257.812 ms) : 0, 257812
GlobalTracer [candidate] (258.48 ms) : 0, 258480
AppSec [baseline] (174.813 ms) : 0, 174813
AppSec [candidate] (174.796 ms) : 0, 174796
Debugger [baseline] (66.975 ms) : 0, 66975
Debugger [candidate] (68.322 ms) : 0, 68322
Remote Config [baseline] (746.408 µs) : 0, 746
Remote Config [candidate] (668.972 µs) : 0, 669
Telemetry [baseline] (9.083 ms) : 0, 9083
Telemetry [candidate] (9.13 ms) : 0, 9130
Flare Poller [baseline] (3.839 ms) : 0, 3839
Flare Poller [candidate] (3.947 ms) : 0, 3947
IAST [baseline] (24.401 ms) : 0, 24401
IAST [candidate] (24.389 ms) : 0, 24389
section iast
crashtracking [baseline] (1.183 ms) : 0, 1183
crashtracking [candidate] (1.183 ms) : 0, 1183
BytebuddyAgent [baseline] (787.529 ms) : 0, 787529
BytebuddyAgent [candidate] (791.987 ms) : 0, 791987
GlobalTracer [baseline] (255.209 ms) : 0, 255209
GlobalTracer [candidate] (256.076 ms) : 0, 256076
AppSec [baseline] (34.996 ms) : 0, 34996
AppSec [candidate] (34.987 ms) : 0, 34987
Debugger [baseline] (64.361 ms) : 0, 64361
Debugger [candidate] (65.02 ms) : 0, 65020
Remote Config [baseline] (551.268 µs) : 0, 551
Remote Config [candidate] (557.512 µs) : 0, 558
Telemetry [baseline] (8.308 ms) : 0, 8308
Telemetry [candidate] (8.361 ms) : 0, 8361
Flare Poller [baseline] (3.433 ms) : 0, 3433
Flare Poller [candidate] (3.515 ms) : 0, 3515
IAST [baseline] (26.836 ms) : 0, 26836
IAST [candidate] (27.022 ms) : 0, 27022
section profiling
ProfilingAgent [baseline] (96.865 ms) : 0, 96865
ProfilingAgent [candidate] (97.586 ms) : 0, 97586
crashtracking [baseline] (1.237 ms) : 0, 1237
crashtracking [candidate] (1.194 ms) : 0, 1194
BytebuddyAgent [baseline] (706.119 ms) : 0, 706119
BytebuddyAgent [candidate] (705.356 ms) : 0, 705356
GlobalTracer [baseline] (222.015 ms) : 0, 222015
GlobalTracer [candidate] (221.945 ms) : 0, 221945
AppSec [baseline] (32.46 ms) : 0, 32460
AppSec [candidate] (32.382 ms) : 0, 32382
Debugger [baseline] (68.779 ms) : 0, 68779
Debugger [candidate] (68.256 ms) : 0, 68256
Remote Config [baseline] (641.592 µs) : 0, 642
Remote Config [candidate] (635.013 µs) : 0, 635
Telemetry [baseline] (9.076 ms) : 0, 9076
Telemetry [candidate] (9.17 ms) : 0, 9170
Flare Poller [baseline] (3.821 ms) : 0, 3821
Flare Poller [candidate] (3.808 ms) : 0, 3808
Profiling [baseline] (97.435 ms) : 0, 97435
Profiling [candidate] (98.165 ms) : 0, 98165
Loading
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.57.0-SNAPSHOT~be2ebd3005, baseline=1.57.0-SNAPSHOT~f832a5b4ad

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.082 s) : 0, 1082465
Total [baseline] (8.767 s) : 0, 8766985
Agent [candidate] (1.08 s) : 0, 1079973
Total [candidate] (8.755 s) : 0, 8755351
section iast
Agent [baseline] (1.229 s) : 0, 1229189
Total [baseline] (9.382 s) : 0, 9382392
Agent [candidate] (1.218 s) : 0, 1218292
Total [candidate] (9.423 s) : 0, 9423089
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.082 s -
Agent iast 1.229 s 146.724 ms (13.6%)
Total tracing 8.767 s -
Total iast 9.382 s 615.407 ms (7.0%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.08 s -
Agent iast 1.218 s 138.319 ms (12.8%)
Total tracing 8.755 s -
Total iast 9.423 s 667.738 ms (7.6%) 
gantt
    title insecure-bank - break down per module: candidate=1.57.0-SNAPSHOT~be2ebd3005, baseline=1.57.0-SNAPSHOT~f832a5b4ad

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.185 ms) : 0, 1185
crashtracking [candidate] (1.19 ms) : 0, 1190
BytebuddyAgent [baseline] (649.453 ms) : 0, 649453
BytebuddyAgent [candidate] (648.52 ms) : 0, 648520
GlobalTracer [baseline] (282.548 ms) : 0, 282548
GlobalTracer [candidate] (281.461 ms) : 0, 281461
AppSec [baseline] (32.565 ms) : 0, 32565
AppSec [candidate] (32.481 ms) : 0, 32481
Debugger [baseline] (67.714 ms) : 0, 67714
Debugger [candidate] (67.29 ms) : 0, 67290
Remote Config [baseline] (671.975 µs) : 0, 672
Remote Config [candidate] (637.86 µs) : 0, 638
Telemetry [baseline] (9.012 ms) : 0, 9012
Telemetry [candidate] (8.966 ms) : 0, 8966
Flare Poller [baseline] (3.727 ms) : 0, 3727
Flare Poller [candidate] (3.733 ms) : 0, 3733
section iast
crashtracking [baseline] (1.195 ms) : 0, 1195
crashtracking [candidate] (1.176 ms) : 0, 1176
BytebuddyAgent [baseline] (795.743 ms) : 0, 795743
BytebuddyAgent [candidate] (787.671 ms) : 0, 787671
GlobalTracer [baseline] (256.837 ms) : 0, 256837
GlobalTracer [candidate] (255.248 ms) : 0, 255248
IAST [baseline] (27.059 ms) : 0, 27059
IAST [candidate] (26.697 ms) : 0, 26697
AppSec [baseline] (32.744 ms) : 0, 32744
AppSec [candidate] (32.831 ms) : 0, 32831
Debugger [baseline] (67.472 ms) : 0, 67472
Debugger [candidate] (66.859 ms) : 0, 66859
Remote Config [baseline] (559.132 µs) : 0, 559
Remote Config [candidate] (554.687 µs) : 0, 555
Telemetry [baseline] (8.451 ms) : 0, 8451
Telemetry [candidate] (8.373 ms) : 0, 8373
Flare Poller [baseline] (3.471 ms) : 0, 3471
Flare Poller [candidate] (3.512 ms) : 0, 3512
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master rob.carlan/DSMON-1160/refactor-dsm-test-ownership
git_commit_date 1765557831 1765558125
git_commit_sha f832a5b be2ebd3
release_version 1.57.0-SNAPSHOT~f832a5b4ad 1.57.0-SNAPSHOT~be2ebd3005
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1765560567 1765560567
ci_job_id 1291494412 1291494412
ci_pipeline_id 86525701 86525701
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-2-wyg6x8zl 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-2-wyg6x8zl 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 2 performance improvements and 5 performance regressions! Performance is the same for 14 metrics, 15 unstable metrics.

scenario Δ mean agg_http_req_duration_p50 Δ mean agg_http_req_duration_p95 Δ mean throughput candidate mean agg_http_req_duration_p50 candidate mean agg_http_req_duration_p95 candidate mean throughput baseline mean agg_http_req_duration_p50 baseline mean agg_http_req_duration_p95 baseline mean throughput
scenario:load:insecure-bank:iast:high_load worse
[+63.956µs; +152.293µs] or [+2.594%; +6.177%]		 same
[-20.880µs; +377.076µs] or [-0.285%; +5.147%]		 unstable
[-183.988op/s; +124.925op/s] or [-12.968%; +8.805%]		 2.574ms 7.504ms 1389.281op/s 2.466ms 7.326ms 1418.812op/s
scenario:load:insecure-bank:iast_GLOBAL:high_load better
[-308.280µs; -163.810µs] or [-10.588%; -5.626%]		 better
[-664.749µs; -196.661µs] or [-8.142%; -2.409%]		 unstable
[-51.263op/s; +234.450op/s] or [-4.131%; +18.893%]		 2.676ms 7.734ms 1332.562op/s 2.912ms 8.165ms 1240.969op/s
scenario:load:petclinic:no_agent:high_load worse
[+1.296ms; +2.792ms] or [+7.707%; +16.608%]		 worse
[+1.380ms; +4.070ms] or [+4.866%; +14.352%]		 unstable
[-52.850op/s; -1.462op/s] or [-19.654%; -0.544%]		 18.857ms 31.086ms 241.750op/s 16.813ms 28.360ms 268.906op/s
scenario:load:petclinic:profiling:high_load worse
[+458.011µs; +1448.427µs] or [+2.435%; +7.701%]		 worse
[+1.000ms; +2.725ms] or [+3.317%; +9.038%]		 unstable
[-35.583op/s; +10.083op/s] or [-14.527%; +4.117%]		 19.761ms 32.012ms 232.188op/s 18.808ms 30.150ms 244.938op/s
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.57.0-SNAPSHOT~be2ebd3005, baseline=1.57.0-SNAPSHOT~f832a5b4ad
    dateFormat X
    axisFormat %s
section baseline
no_agent (17.348 ms) : 17173, 17522
.   : milestone, 17348,
appsec (18.855 ms) : 18665, 19044
.   : milestone, 18855,
code_origins (17.678 ms) : 17503, 17853
.   : milestone, 17678,
iast (17.655 ms) : 17481, 17830
.   : milestone, 17655,
profiling (19.052 ms) : 18864, 19240
.   : milestone, 19052,
tracing (17.95 ms) : 17774, 18126
.   : milestone, 17950,
section candidate
no_agent (19.306 ms) : 19108, 19504
.   : milestone, 19306,
appsec (19.355 ms) : 19159, 19551
.   : milestone, 19355,
code_origins (17.786 ms) : 17612, 17961
.   : milestone, 17786,
iast (17.75 ms) : 17571, 17929
.   : milestone, 17750,
profiling (20.106 ms) : 19901, 20312
.   : milestone, 20106,
tracing (17.851 ms) : 17677, 18025
.   : milestone, 17851,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 17.348 ms [17.173 ms, 17.522 ms] -
appsec 18.855 ms [18.665 ms, 19.044 ms] 1.507 ms (8.7%)
code_origins 17.678 ms [17.503 ms, 17.853 ms] 330.691 µs (1.9%)
iast 17.655 ms [17.481 ms, 17.83 ms] 307.828 µs (1.8%)
profiling 19.052 ms [18.864 ms, 19.24 ms] 1.704 ms (9.8%)
tracing 17.95 ms [17.774 ms, 18.126 ms] 602.567 µs (3.5%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 19.306 ms [19.108 ms, 19.504 ms] -
appsec 19.355 ms [19.159 ms, 19.551 ms] 49.268 µs (0.3%)
code_origins 17.786 ms [17.612 ms, 17.961 ms] -1.519 ms (-7.9%)
iast 17.75 ms [17.571 ms, 17.929 ms] -1.556 ms (-8.1%)
profiling 20.106 ms [19.901 ms, 20.312 ms] 800.725 µs (4.1%)
tracing 17.851 ms [17.677 ms, 18.025 ms] -1.454 ms (-7.5%)
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.57.0-SNAPSHOT~be2ebd3005, baseline=1.57.0-SNAPSHOT~f832a5b4ad
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.174 ms) : 1162, 1185
.   : milestone, 1174,
iast (3.225 ms) : 3175, 3275
.   : milestone, 3225,
iast_FULL (5.737 ms) : 5680, 5795
.   : milestone, 5737,
iast_GLOBAL (3.697 ms) : 3636, 3758
.   : milestone, 3697,
profiling (1.978 ms) : 1961, 1995
.   : milestone, 1978,
tracing (1.812 ms) : 1795, 1828
.   : milestone, 1812,
section candidate
no_agent (1.168 ms) : 1156, 1179
.   : milestone, 1168,
iast (3.295 ms) : 3255, 3336
.   : milestone, 3295,
iast_FULL (5.738 ms) : 5680, 5796
.   : milestone, 5738,
iast_GLOBAL (3.438 ms) : 3387, 3490
.   : milestone, 3438,
profiling (2.027 ms) : 2009, 2044
.   : milestone, 2027,
tracing (1.757 ms) : 1743, 1772
.   : milestone, 1757,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.174 ms [1.162 ms, 1.185 ms] -
iast 3.225 ms [3.175 ms, 3.275 ms] 2.052 ms (174.8%)
iast_FULL 5.737 ms [5.68 ms, 5.795 ms] 4.564 ms (388.9%)
iast_GLOBAL 3.697 ms [3.636 ms, 3.758 ms] 2.523 ms (215.0%)
profiling 1.978 ms [1.961 ms, 1.995 ms] 804.493 µs (68.6%)
tracing 1.812 ms [1.795 ms, 1.828 ms] 637.933 µs (54.4%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.168 ms [1.156 ms, 1.179 ms] -
iast 3.295 ms [3.255 ms, 3.336 ms] 2.128 ms (182.2%)
iast_FULL 5.738 ms [5.68 ms, 5.796 ms] 4.571 ms (391.5%)
iast_GLOBAL 3.438 ms [3.387 ms, 3.49 ms] 2.271 ms (194.5%)
profiling 2.027 ms [2.009 ms, 2.044 ms] 859.386 µs (73.6%)
tracing 1.757 ms [1.743 ms, 1.772 ms] 589.604 µs (50.5%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master rob.carlan/DSMON-1160/refactor-dsm-test-ownership
git_commit_date 1765557831 1765558125
git_commit_sha f832a5b be2ebd3
release_version 1.57.0-SNAPSHOT~f832a5b4ad 1.57.0-SNAPSHOT~be2ebd3005
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1765560007 1765560007
ci_job_id 1291494413 1291494413
ci_pipeline_id 86525701 86525701
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-3-plg7ox6j 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-3-plg7ox6j 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 1 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 0 unstable metrics.

scenario Δ mean execution_time candidate mean execution_time baseline mean execution_time
scenario:dacapo:tomcat:appsec better
[-1.442ms; -1.099ms] or [-38.552%; -29.382%]		 2.470ms 3.740ms
Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.57.0-SNAPSHOT~be2ebd3005, baseline=1.57.0-SNAPSHOT~f832a5b4ad
    dateFormat X
    axisFormat %s
section baseline
no_agent (14.975 s) : 14975000, 14975000
.   : milestone, 14975000,
appsec (14.53 s) : 14530000, 14530000
.   : milestone, 14530000,
iast (17.869 s) : 17869000, 17869000
.   : milestone, 17869000,
iast_GLOBAL (17.952 s) : 17952000, 17952000
.   : milestone, 17952000,
profiling (14.597 s) : 14597000, 14597000
.   : milestone, 14597000,
tracing (14.661 s) : 14661000, 14661000
.   : milestone, 14661000,
section candidate
no_agent (15.786 s) : 15786000, 15786000
.   : milestone, 15786000,
appsec (14.976 s) : 14976000, 14976000
.   : milestone, 14976000,
iast (18.044 s) : 18044000, 18044000
.   : milestone, 18044000,
iast_GLOBAL (18.153 s) : 18153000, 18153000
.   : milestone, 18153000,
profiling (14.534 s) : 14534000, 14534000
.   : milestone, 14534000,
tracing (14.619 s) : 14619000, 14619000
.   : milestone, 14619000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 14.975 s [14.975 s, 14.975 s] -
appsec 14.53 s [14.53 s, 14.53 s] -445.0 ms (-3.0%)
iast 17.869 s [17.869 s, 17.869 s] 2.894 s (19.3%)
iast_GLOBAL 17.952 s [17.952 s, 17.952 s] 2.977 s (19.9%)
profiling 14.597 s [14.597 s, 14.597 s] -378.0 ms (-2.5%)
tracing 14.661 s [14.661 s, 14.661 s] -314.0 ms (-2.1%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.786 s [15.786 s, 15.786 s] -
appsec 14.976 s [14.976 s, 14.976 s] -810.0 ms (-5.1%)
iast 18.044 s [18.044 s, 18.044 s] 2.258 s (14.3%)
iast_GLOBAL 18.153 s [18.153 s, 18.153 s] 2.367 s (15.0%)
profiling 14.534 s [14.534 s, 14.534 s] -1.252 s (-7.9%)
tracing 14.619 s [14.619 s, 14.619 s] -1.167 s (-7.4%)
Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.57.0-SNAPSHOT~be2ebd3005, baseline=1.57.0-SNAPSHOT~f832a5b4ad
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.485 ms) : 1474, 1497
.   : milestone, 1485,
appsec (3.74 ms) : 3521, 3959
.   : milestone, 3740,
iast (2.227 ms) : 2162, 2291
.   : milestone, 2227,
iast_GLOBAL (2.27 ms) : 2206, 2335
.   : milestone, 2270,
profiling (2.063 ms) : 2011, 2114
.   : milestone, 2063,
tracing (2.057 ms) : 2006, 2108
.   : milestone, 2057,
section candidate
no_agent (1.48 ms) : 1469, 1492
.   : milestone, 1480,
appsec (2.47 ms) : 2418, 2522
.   : milestone, 2470,
iast (2.224 ms) : 2159, 2288
.   : milestone, 2224,
iast_GLOBAL (2.271 ms) : 2206, 2336
.   : milestone, 2271,
profiling (2.082 ms) : 2028, 2136
.   : milestone, 2082,
tracing (2.056 ms) : 2005, 2107
.   : milestone, 2056,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.485 ms [1.474 ms, 1.497 ms] -
appsec 3.74 ms [3.521 ms, 3.959 ms] 2.254 ms (151.8%)
iast 2.227 ms [2.162 ms, 2.291 ms] 741.316 µs (49.9%)
iast_GLOBAL 2.27 ms [2.206 ms, 2.335 ms] 784.956 µs (52.8%)
profiling 2.063 ms [2.011 ms, 2.114 ms] 577.267 µs (38.9%)
tracing 2.057 ms [2.006 ms, 2.108 ms] 571.73 µs (38.5%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.48 ms [1.469 ms, 1.492 ms] -
appsec 2.47 ms [2.418 ms, 2.522 ms] 989.379 µs (66.8%)
iast 2.224 ms [2.159 ms, 2.288 ms] 743.345 µs (50.2%)
iast_GLOBAL 2.271 ms [2.206 ms, 2.336 ms] 791.022 µs (53.4%)
profiling 2.082 ms [2.028 ms, 2.136 ms] 601.659 µs (40.6%)
tracing 2.056 ms [2.005 ms, 2.107 ms] 575.941 µs (38.9%)

@pr-commenter
Copy link

pr-commenter bot commented Dec 12, 2025

Kafka / consumer-benchmark

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master rob.carlan/DSMON-1160/refactor-dsm-test-ownership
git_commit_date 1765556506 1765558125
git_commit_sha c2560c4 be2ebd3
See matching parameters
Baseline Candidate
ci_job_date 1765559369 1765559369
ci_job_id 1291494417 1291494417
ci_pipeline_id 86525701 86525701
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
jdkVersion 11.0.25 11.0.25
jmhVersion 1.36 1.36
jvm /usr/lib/jvm/java-11-openjdk-amd64/bin/java /usr/lib/jvm/java-11-openjdk-amd64/bin/java
jvmArgs -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/consumer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/consumer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant
vmName OpenJDK 64-Bit Server VM OpenJDK 64-Bit Server VM
vmVersion 11.0.25+9-post-Ubuntu-1ubuntu122.04 11.0.25+9-post-Ubuntu-1ubuntu122.04

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics.

See unchanged results
scenario Δ mean throughput
scenario:not-instrumented/KafkaConsumerBenchmark.benchConsume same
scenario:only-tracing-dsm-disabled-benchmarks/KafkaConsumerBenchmark.benchConsume same
scenario:only-tracing-dsm-enabled-benchmarks/KafkaConsumerBenchmark.benchConsume same

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@robcarlan-datadog