Skip to content

fix smoke test flakiness #10347

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
PerfectSlayer merged 1 commit into from
Jan 13, 2026
Merged

fix smoke test flakiness #10347

PerfectSlayer merged 1 commit into from
Jan 13, 2026
+7 −2

Conversation

@jpbempel
Copy link
Member

@jpbempel jpbempel commented Jan 13, 2026 •
edited by atlassian bot
Loading

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Jira ticket: DEBUG-4845

@jpbempel jpbempel requested a review from a team as a code owner January 13, 2026 10:07
@jpbempel jpbempel requested review from evanchooly and removed request for a team January 13, 2026 10:07
@jpbempel jpbempel force-pushed the jpbempel/fix-flaky-smoke-tests branch from ff5c60b to 78f800b Compare January 13, 2026 10:09
@github-actions
Copy link
Contributor

github-actions bot commented Jan 13, 2026

Hi! 👋 Thanks for your pull request! 🎉

To help us review it, please make sure to:

  • Add at least one type, and one component or instrumentation label to the pull request

If you need help, please check our contributing guidelines.

@pr-commenter
Copy link

pr-commenter bot commented Jan 13, 2026

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master jpbempel/fix-flaky-smoke-tests
git_commit_date 1768295376 1768298945
git_commit_sha 7feb988 78f800b
release_version 1.59.0-SNAPSHOT~7feb9885b7 1.59.0-SNAPSHOT~78f800bcdd
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1768300704 1768300704
ci_job_id 1345614656 1345614656
ci_pipeline_id 90364686 90364686
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-0-9t3tpc7c 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-0-9t3tpc7c 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 56 metrics, 9 unstable metrics.

Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.59.0-SNAPSHOT~78f800bcdd, baseline=1.59.0-SNAPSHOT~7feb9885b7

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.086 s) : 0, 1085784
Total [baseline] (8.786 s) : 0, 8785664
Agent [candidate] (1.089 s) : 0, 1089381
Total [candidate] (8.809 s) : 0, 8808828
section iast
Agent [baseline] (1.233 s) : 0, 1233151
Total [baseline] (9.355 s) : 0, 9354678
Agent [candidate] (1.222 s) : 0, 1222038
Total [candidate] (9.339 s) : 0, 9339067
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.086 s -
Agent iast 1.233 s 147.367 ms (13.6%)
Total tracing 8.786 s -
Total iast 9.355 s 569.014 ms (6.5%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.089 s -
Agent iast 1.222 s 132.657 ms (12.2%)
Total tracing 8.809 s -
Total iast 9.339 s 530.239 ms (6.0%) 
gantt
    title insecure-bank - break down per module: candidate=1.59.0-SNAPSHOT~78f800bcdd, baseline=1.59.0-SNAPSHOT~7feb9885b7

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.184 ms) : 0, 1184
crashtracking [candidate] (1.191 ms) : 0, 1191
BytebuddyAgent [baseline] (652.711 ms) : 0, 652711
BytebuddyAgent [candidate] (655.258 ms) : 0, 655258
GlobalTracer [baseline] (283.281 ms) : 0, 283281
GlobalTracer [candidate] (283.961 ms) : 0, 283961
AppSec [baseline] (32.73 ms) : 0, 32730
AppSec [candidate] (32.817 ms) : 0, 32817
Debugger [baseline] (66.87 ms) : 0, 66870
Debugger [candidate] (66.983 ms) : 0, 66983
Remote Config [baseline] (650.97 µs) : 0, 651
Remote Config [candidate] (682.562 µs) : 0, 683
Telemetry [baseline] (8.951 ms) : 0, 8951
Telemetry [candidate] (9.014 ms) : 0, 9014
Flare Poller [baseline] (3.845 ms) : 0, 3845
Flare Poller [candidate] (3.791 ms) : 0, 3791
section iast
crashtracking [baseline] (1.194 ms) : 0, 1194
crashtracking [candidate] (1.174 ms) : 0, 1174
BytebuddyAgent [baseline] (798.588 ms) : 0, 798588
BytebuddyAgent [candidate] (790.001 ms) : 0, 790001
GlobalTracer [baseline] (258.177 ms) : 0, 258177
GlobalTracer [candidate] (256.679 ms) : 0, 256679
IAST [baseline] (27.385 ms) : 0, 27385
IAST [candidate] (26.884 ms) : 0, 26884
AppSec [baseline] (34.537 ms) : 0, 34537
AppSec [candidate] (34.286 ms) : 0, 34286
Debugger [baseline] (65.086 ms) : 0, 65086
Debugger [candidate] (65.145 ms) : 0, 65145
Remote Config [baseline] (558.765 µs) : 0, 559
Remote Config [candidate] (567.801 µs) : 0, 568
Telemetry [baseline] (8.478 ms) : 0, 8478
Telemetry [candidate] (8.422 ms) : 0, 8422
Flare Poller [baseline] (3.601 ms) : 0, 3601
Flare Poller [candidate] (3.528 ms) : 0, 3528
Loading
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.59.0-SNAPSHOT~78f800bcdd, baseline=1.59.0-SNAPSHOT~7feb9885b7

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.103 s) : 0, 1102641
Total [baseline] (10.943 s) : 0, 10942889
Agent [candidate] (1.096 s) : 0, 1095517
Total [candidate] (10.881 s) : 0, 10881289
section appsec
Agent [baseline] (1.266 s) : 0, 1265868
Total [baseline] (11.093 s) : 0, 11093127
Agent [candidate] (1.273 s) : 0, 1273491
Total [candidate] (11.122 s) : 0, 11121698
section iast
Agent [baseline] (1.23 s) : 0, 1229584
Total [baseline] (11.16 s) : 0, 11159516
Agent [candidate] (1.226 s) : 0, 1226064
Total [candidate] (11.105 s) : 0, 11104812
section profiling
Agent [baseline] (1.207 s) : 0, 1207252
Total [baseline] (11.001 s) : 0, 11000502
Agent [candidate] (1.206 s) : 0, 1206124
Total [candidate] (10.935 s) : 0, 10934886
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.103 s -
Agent appsec 1.266 s 163.226 ms (14.8%)
Agent iast 1.23 s 126.943 ms (11.5%)
Agent profiling 1.207 s 104.611 ms (9.5%)
Total tracing 10.943 s -
Total appsec 11.093 s 150.237 ms (1.4%)
Total iast 11.16 s 216.627 ms (2.0%)
Total profiling 11.001 s 57.613 ms (0.5%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.096 s -
Agent appsec 1.273 s 177.975 ms (16.2%)
Agent iast 1.226 s 130.547 ms (11.9%)
Agent profiling 1.206 s 110.607 ms (10.1%)
Total tracing 10.881 s -
Total appsec 11.122 s 240.41 ms (2.2%)
Total iast 11.105 s 223.524 ms (2.1%)
Total profiling 10.935 s 53.597 ms (0.5%) 
gantt
    title petclinic - break down per module: candidate=1.59.0-SNAPSHOT~78f800bcdd, baseline=1.59.0-SNAPSHOT~7feb9885b7

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.207 ms) : 0, 1207
crashtracking [candidate] (1.196 ms) : 0, 1196
BytebuddyAgent [baseline] (661.873 ms) : 0, 661873
BytebuddyAgent [candidate] (657.879 ms) : 0, 657879
GlobalTracer [baseline] (287.597 ms) : 0, 287597
GlobalTracer [candidate] (285.45 ms) : 0, 285450
AppSec [baseline] (33.405 ms) : 0, 33405
AppSec [candidate] (33.097 ms) : 0, 33097
Debugger [baseline] (68.974 ms) : 0, 68974
Debugger [candidate] (68.609 ms) : 0, 68609
Remote Config [baseline] (665.532 µs) : 0, 666
Remote Config [candidate] (649.6 µs) : 0, 650
Telemetry [baseline] (9.097 ms) : 0, 9097
Telemetry [candidate] (8.998 ms) : 0, 8998
Flare Poller [baseline] (3.936 ms) : 0, 3936
Flare Poller [candidate] (3.858 ms) : 0, 3858
section appsec
crashtracking [baseline] (1.18 ms) : 0, 1180
crashtracking [candidate] (1.198 ms) : 0, 1198
BytebuddyAgent [baseline] (691.834 ms) : 0, 691834
BytebuddyAgent [candidate] (696.34 ms) : 0, 696340
GlobalTracer [baseline] (258.717 ms) : 0, 258717
GlobalTracer [candidate] (260.097 ms) : 0, 260097
IAST [baseline] (24.557 ms) : 0, 24557
IAST [candidate] (24.795 ms) : 0, 24795
AppSec [baseline] (173.977 ms) : 0, 173977
AppSec [candidate] (173.95 ms) : 0, 173950
Debugger [baseline] (66.211 ms) : 0, 66211
Debugger [candidate] (67.607 ms) : 0, 67607
Remote Config [baseline] (772.667 µs) : 0, 773
Remote Config [candidate] (783.286 µs) : 0, 783
Telemetry [baseline] (9.311 ms) : 0, 9311
Telemetry [candidate] (9.334 ms) : 0, 9334
Flare Poller [baseline] (3.803 ms) : 0, 3803
Flare Poller [candidate] (3.774 ms) : 0, 3774
section iast
crashtracking [baseline] (1.183 ms) : 0, 1183
crashtracking [candidate] (1.181 ms) : 0, 1181
BytebuddyAgent [baseline] (794.063 ms) : 0, 794063
BytebuddyAgent [candidate] (792.287 ms) : 0, 792287
GlobalTracer [baseline] (258.237 ms) : 0, 258237
GlobalTracer [candidate] (257.153 ms) : 0, 257153
IAST [baseline] (27.264 ms) : 0, 27264
IAST [candidate] (27.034 ms) : 0, 27034
AppSec [baseline] (33.731 ms) : 0, 33731
AppSec [candidate] (34.571 ms) : 0, 34571
Debugger [baseline] (66.914 ms) : 0, 66914
Debugger [candidate] (65.609 ms) : 0, 65609
Remote Config [baseline] (570.568 µs) : 0, 571
Remote Config [candidate] (609.742 µs) : 0, 610
Telemetry [baseline] (8.484 ms) : 0, 8484
Telemetry [candidate] (8.442 ms) : 0, 8442
Flare Poller [baseline] (3.546 ms) : 0, 3546
Flare Poller [candidate] (3.567 ms) : 0, 3567
section profiling
crashtracking [baseline] (1.228 ms) : 0, 1228
crashtracking [candidate] (1.218 ms) : 0, 1218
BytebuddyAgent [baseline] (702.276 ms) : 0, 702276
BytebuddyAgent [candidate] (702.871 ms) : 0, 702871
GlobalTracer [baseline] (222.045 ms) : 0, 222045
GlobalTracer [candidate] (221.305 ms) : 0, 221305
AppSec [baseline] (32.478 ms) : 0, 32478
AppSec [candidate] (32.232 ms) : 0, 32232
Debugger [baseline] (68.381 ms) : 0, 68381
Debugger [candidate] (68.339 ms) : 0, 68339
Remote Config [baseline] (639.214 µs) : 0, 639
Remote Config [candidate] (633.317 µs) : 0, 633
Telemetry [baseline] (9.07 ms) : 0, 9070
Telemetry [candidate] (8.837 ms) : 0, 8837
Flare Poller [baseline] (3.772 ms) : 0, 3772
Flare Poller [candidate] (3.752 ms) : 0, 3752
ProfilingAgent [baseline] (97.638 ms) : 0, 97638
ProfilingAgent [candidate] (96.968 ms) : 0, 96968
Profiling [baseline] (98.214 ms) : 0, 98214
Profiling [candidate] (97.541 ms) : 0, 97541
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master jpbempel/fix-flaky-smoke-tests
git_commit_date 1768295376 1768298945
git_commit_sha 7feb988 78f800b
release_version 1.59.0-SNAPSHOT~7feb9885b7 1.59.0-SNAPSHOT~78f800bcdd
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1768301199 1768301199
ci_job_id 1345614657 1345614657
ci_pipeline_id 90364686 90364686
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-1-vwtfzsmt 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-1-vwtfzsmt 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 2 performance improvements and 1 performance regressions! Performance is the same for 18 metrics, 15 unstable metrics.

scenario Δ mean agg_http_req_duration_p50 Δ mean agg_http_req_duration_p95 Δ mean throughput candidate mean agg_http_req_duration_p50 candidate mean agg_http_req_duration_p95 candidate mean throughput baseline mean agg_http_req_duration_p50 baseline mean agg_http_req_duration_p95 baseline mean throughput
scenario:load:insecure-bank:iast_FULL:high_load worse
[+152.839µs; +367.766µs] or [+3.095%; +7.447%]		 unsure
[+54.715µs; +601.655µs] or [+0.460%; +5.053%]		 unstable
[-109.740op/s; +47.115op/s] or [-13.272%; +5.698%]		 5.199ms 12.235ms 795.562op/s 4.939ms 11.907ms 826.875op/s
scenario:load:petclinic:profiling:high_load better
[-1.984ms; -1.098ms] or [-10.125%; -5.605%]		 better
[-2.033ms; -0.696ms] or [-6.564%; -2.249%]		 unstable
[-8.915op/s; +42.290op/s] or [-3.773%; +17.898%]		 18.050ms 29.602ms 252.969op/s 19.591ms 30.967ms 236.281op/s
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.59.0-SNAPSHOT~78f800bcdd, baseline=1.59.0-SNAPSHOT~7feb9885b7
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.174 ms) : 1163, 1185
.   : milestone, 1174,
iast (3.306 ms) : 3258, 3354
.   : milestone, 3306,
iast_FULL (5.588 ms) : 5533, 5644
.   : milestone, 5588,
iast_GLOBAL (3.579 ms) : 3527, 3631
.   : milestone, 3579,
profiling (2.002 ms) : 1985, 2019
.   : milestone, 2002,
tracing (1.779 ms) : 1765, 1793
.   : milestone, 1779,
section candidate
no_agent (1.188 ms) : 1176, 1200
.   : milestone, 1188,
iast (3.238 ms) : 3198, 3278
.   : milestone, 3238,
iast_FULL (5.813 ms) : 5756, 5871
.   : milestone, 5813,
iast_GLOBAL (3.582 ms) : 3526, 3637
.   : milestone, 3582,
profiling (2.088 ms) : 2069, 2107
.   : milestone, 2088,
tracing (1.872 ms) : 1856, 1888
.   : milestone, 1872,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.174 ms [1.163 ms, 1.185 ms] -
iast 3.306 ms [3.258 ms, 3.354 ms] 2.132 ms (181.7%)
iast_FULL 5.588 ms [5.533 ms, 5.644 ms] 4.415 ms (376.1%)
iast_GLOBAL 3.579 ms [3.527 ms, 3.631 ms] 2.405 ms (204.9%)
profiling 2.002 ms [1.985 ms, 2.019 ms] 828.313 µs (70.6%)
tracing 1.779 ms [1.765 ms, 1.793 ms] 605.154 µs (51.6%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.188 ms [1.176 ms, 1.2 ms] -
iast 3.238 ms [3.198 ms, 3.278 ms] 2.05 ms (172.5%)
iast_FULL 5.813 ms [5.756 ms, 5.871 ms] 4.625 ms (389.3%)
iast_GLOBAL 3.582 ms [3.526 ms, 3.637 ms] 2.394 ms (201.4%)
profiling 2.088 ms [2.069 ms, 2.107 ms] 899.675 µs (75.7%)
tracing 1.872 ms [1.856 ms, 1.888 ms] 683.687 µs (57.5%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.59.0-SNAPSHOT~78f800bcdd, baseline=1.59.0-SNAPSHOT~7feb9885b7
    dateFormat X
    axisFormat %s
section baseline
no_agent (17.636 ms) : 17463, 17809
.   : milestone, 17636,
appsec (18.718 ms) : 18527, 18908
.   : milestone, 18718,
code_origins (17.907 ms) : 17729, 18086
.   : milestone, 17907,
iast (17.636 ms) : 17458, 17814
.   : milestone, 17636,
profiling (19.756 ms) : 19558, 19955
.   : milestone, 19756,
tracing (17.687 ms) : 17510, 17865
.   : milestone, 17687,
section candidate
no_agent (18.264 ms) : 18076, 18452
.   : milestone, 18264,
appsec (18.874 ms) : 18684, 19065
.   : milestone, 18874,
code_origins (17.598 ms) : 17425, 17772
.   : milestone, 17598,
iast (17.612 ms) : 17438, 17785
.   : milestone, 17612,
profiling (18.449 ms) : 18265, 18633
.   : milestone, 18449,
tracing (17.578 ms) : 17405, 17752
.   : milestone, 17578,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 17.636 ms [17.463 ms, 17.809 ms] -
appsec 18.718 ms [18.527 ms, 18.908 ms] 1.081 ms (6.1%)
code_origins 17.907 ms [17.729 ms, 18.086 ms] 271.307 µs (1.5%)
iast 17.636 ms [17.458 ms, 17.814 ms] -0.257 µs (-0.0%)
profiling 19.756 ms [19.558 ms, 19.955 ms] 2.12 ms (12.0%)
tracing 17.687 ms [17.51 ms, 17.865 ms] 51.236 µs (0.3%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 18.264 ms [18.076 ms, 18.452 ms] -
appsec 18.874 ms [18.684 ms, 19.065 ms] 610.676 µs (3.3%)
code_origins 17.598 ms [17.425 ms, 17.772 ms] -665.382 µs (-3.6%)
iast 17.612 ms [17.438 ms, 17.785 ms] -652.172 µs (-3.6%)
profiling 18.449 ms [18.265 ms, 18.633 ms] 185.221 µs (1.0%)
tracing 17.578 ms [17.405 ms, 17.752 ms] -685.435 µs (-3.8%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master jpbempel/fix-flaky-smoke-tests
git_commit_date 1768295376 1768298945
git_commit_sha 7feb988 78f800b
release_version 1.59.0-SNAPSHOT~7feb9885b7 1.59.0-SNAPSHOT~78f800bcdd
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1768300914 1768300914
ci_job_id 1345614658 1345614658
ci_pipeline_id 90364686 90364686
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-0-bzq5duq8 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-0-bzq5duq8 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.59.0-SNAPSHOT~78f800bcdd, baseline=1.59.0-SNAPSHOT~7feb9885b7
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.471 ms) : 1459, 1482
.   : milestone, 1471,
appsec (3.72 ms) : 3501, 3940
.   : milestone, 3720,
iast (2.204 ms) : 2139, 2269
.   : milestone, 2204,
iast_GLOBAL (2.254 ms) : 2188, 2319
.   : milestone, 2254,
profiling (2.091 ms) : 2036, 2146
.   : milestone, 2091,
tracing (2.041 ms) : 1990, 2092
.   : milestone, 2041,
section candidate
no_agent (1.47 ms) : 1459, 1481
.   : milestone, 1470,
appsec (3.726 ms) : 3507, 3944
.   : milestone, 3726,
iast (2.208 ms) : 2143, 2273
.   : milestone, 2208,
iast_GLOBAL (2.246 ms) : 2181, 2312
.   : milestone, 2246,
profiling (2.061 ms) : 2007, 2114
.   : milestone, 2061,
tracing (2.044 ms) : 1993, 2095
.   : milestone, 2044,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.471 ms [1.459 ms, 1.482 ms] -
appsec 3.72 ms [3.501 ms, 3.94 ms] 2.249 ms (153.0%)
iast 2.204 ms [2.139 ms, 2.269 ms] 733.211 µs (49.9%)
iast_GLOBAL 2.254 ms [2.188 ms, 2.319 ms] 782.913 µs (53.2%)
profiling 2.091 ms [2.036 ms, 2.146 ms] 620.178 µs (42.2%)
tracing 2.041 ms [1.99 ms, 2.092 ms] 570.209 µs (38.8%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.47 ms [1.459 ms, 1.481 ms] -
appsec 3.726 ms [3.507 ms, 3.944 ms] 2.256 ms (153.4%)
iast 2.208 ms [2.143 ms, 2.273 ms] 738.059 µs (50.2%)
iast_GLOBAL 2.246 ms [2.181 ms, 2.312 ms] 776.484 µs (52.8%)
profiling 2.061 ms [2.007 ms, 2.114 ms] 590.756 µs (40.2%)
tracing 2.044 ms [1.993 ms, 2.095 ms] 574.043 µs (39.1%)
Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.59.0-SNAPSHOT~78f800bcdd, baseline=1.59.0-SNAPSHOT~7feb9885b7
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.571 s) : 15571000, 15571000
.   : milestone, 15571000,
appsec (14.706 s) : 14706000, 14706000
.   : milestone, 14706000,
iast (18.098 s) : 18098000, 18098000
.   : milestone, 18098000,
iast_GLOBAL (17.55 s) : 17550000, 17550000
.   : milestone, 17550000,
profiling (15.016 s) : 15016000, 15016000
.   : milestone, 15016000,
tracing (14.651 s) : 14651000, 14651000
.   : milestone, 14651000,
section candidate
no_agent (15.128 s) : 15128000, 15128000
.   : milestone, 15128000,
appsec (14.521 s) : 14521000, 14521000
.   : milestone, 14521000,
iast (18.453 s) : 18453000, 18453000
.   : milestone, 18453000,
iast_GLOBAL (17.725 s) : 17725000, 17725000
.   : milestone, 17725000,
profiling (15.046 s) : 15046000, 15046000
.   : milestone, 15046000,
tracing (14.654 s) : 14654000, 14654000
.   : milestone, 14654000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.571 s [15.571 s, 15.571 s] -
appsec 14.706 s [14.706 s, 14.706 s] -865.0 ms (-5.6%)
iast 18.098 s [18.098 s, 18.098 s] 2.527 s (16.2%)
iast_GLOBAL 17.55 s [17.55 s, 17.55 s] 1.979 s (12.7%)
profiling 15.016 s [15.016 s, 15.016 s] -555.0 ms (-3.6%)
tracing 14.651 s [14.651 s, 14.651 s] -920.0 ms (-5.9%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.128 s [15.128 s, 15.128 s] -
appsec 14.521 s [14.521 s, 14.521 s] -607.0 ms (-4.0%)
iast 18.453 s [18.453 s, 18.453 s] 3.325 s (22.0%)
iast_GLOBAL 17.725 s [17.725 s, 17.725 s] 2.597 s (17.2%)
profiling 15.046 s [15.046 s, 15.046 s] -82.0 ms (-0.5%)
tracing 14.654 s [14.654 s, 14.654 s] -474.0 ms (-3.1%)

PerfectSlayer
PerfectSlayer approved these changes Jan 13, 2026
View reviewed changes
Copy link
Contributor

@PerfectSlayer PerfectSlayer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's try it and check how it behaves over time

@PerfectSlayer PerfectSlayer merged commit bff8837 into master Jan 13, 2026
552 of 569 checks passed
@PerfectSlayer PerfectSlayer deleted the jpbempel/fix-flaky-smoke-tests branch January 13, 2026 14:08
@PerfectSlayer PerfectSlayer added tag: no release notes Changes to exclude from release notes tag: flaky test Flaky tests comp: debugger Dynamic Instrumentation labels Jan 13, 2026
@github-actions github-actions bot added this to the 1.59.0 milestone Jan 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@PerfectSlayer PerfectSlayer PerfectSlayer approved these changes

@evanchooly evanchooly Awaiting requested review from evanchooly evanchooly is a code owner automatically assigned from DataDog/debugger-java

Assignees

No one assigned

Labels

comp: debugger Dynamic Instrumentation tag: flaky test Flaky tests tag: no release notes Changes to exclude from release notes

Projects

None yet

Milestone

1.59.0

Development

Successfully merging this pull request may close these issues.

3 participants

@jpbempel @PerfectSlayer