Add telemetry to AI Guard evaluations #9884

manuel-alvarez-alvarez · 2025-10-30T13:09:56Z

What Does This Do

Adds telemetry to the AI Guard evaluations.

Motivation

We want to track customer usage of the SDK through telemetry metrics.

Additional Notes

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any useful labels
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the CODEOWNERS file on source file addition, move, or deletion
Update the public documentation in case of new configuration flag or behavior

Jira ticket: [PROJ-IDENT]

datadog-datadog-prod-us1 · 2025-10-30T13:30:26Z

🎯 Code Coverage
• Patch Coverage: 93.75%
• Total Coverage: 60.18% (+0.48%)

View detailed report

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: 01aed83 | Docs | Datadog PR Page | Was this helpful? Give us feedback!}

pr-commenter · 2025-10-30T13:59:14Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	malvarez/ai-guard-telemetry
git_commit_date	1761927667	1761991527
git_commit_sha	`33bbc70`	`01aed83`
release_version	1.55.0-SNAPSHOT~33bbc70b29	1.55.0-SNAPSHOT~01aed836f3

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1761993528	1761993528
ci_job_id	1210637727	1210637727
ci_pipeline_id	80964930	80964930
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-tn3p526s 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-tn3p526s 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 54 metrics, 11 unstable metrics.

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.55.0-SNAPSHOT~01aed836f3, baseline=1.55.0-SNAPSHOT~33bbc70b29

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.03 s) : 0, 1029748
Total [baseline] (8.702 s) : 0, 8701648
Agent [candidate] (1.032 s) : 0, 1031874
Total [candidate] (8.65 s) : 0, 8649904
section iast
Agent [baseline] (1.164 s) : 0, 1164288
Total [baseline] (9.395 s) : 0, 9394527
Agent [candidate] (1.159 s) : 0, 1158814
Total [candidate] (9.363 s) : 0, 9363197

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.03 s	-
Agent	iast	1.164 s	134.54 ms (13.1%)
Total	tracing	8.702 s	-
Total	iast	9.395 s	692.879 ms (8.0%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.032 s	-
Agent	iast	1.159 s	126.94 ms (12.3%)
Total	tracing	8.65 s	-
Total	iast	9.363 s	713.292 ms (8.2%)

gantt
    title insecure-bank - break down per module: candidate=1.55.0-SNAPSHOT~01aed836f3, baseline=1.55.0-SNAPSHOT~33bbc70b29

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.47 ms) : 0, 1470
crashtracking [candidate] (1.461 ms) : 0, 1461
BytebuddyAgent [baseline] (702.783 ms) : 0, 702783
BytebuddyAgent [candidate] (703.93 ms) : 0, 703930
GlobalTracer [baseline] (244.164 ms) : 0, 244164
GlobalTracer [candidate] (245.165 ms) : 0, 245165
AppSec [baseline] (32.239 ms) : 0, 32239
AppSec [candidate] (32.212 ms) : 0, 32212
Debugger [baseline] (6.37 ms) : 0, 6370
Debugger [candidate] (6.41 ms) : 0, 6410
Remote Config [baseline] (683.785 µs) : 0, 684
Remote Config [candidate] (665.046 µs) : 0, 665
Telemetry [baseline] (13.73 ms) : 0, 13730
Telemetry [candidate] (15.164 ms) : 0, 15164
Flare Poller [baseline] (7.181 ms) : 0, 7181
Flare Poller [candidate] (5.738 ms) : 0, 5738
section iast
crashtracking [baseline] (1.469 ms) : 0, 1469
crashtracking [candidate] (1.46 ms) : 0, 1460
BytebuddyAgent [baseline] (826.304 ms) : 0, 826304
BytebuddyAgent [candidate] (822.722 ms) : 0, 822722
GlobalTracer [baseline] (234.289 ms) : 0, 234289
GlobalTracer [candidate] (233.174 ms) : 0, 233174
AppSec [baseline] (28.925 ms) : 0, 28925
AppSec [candidate] (26.227 ms) : 0, 26227
Debugger [baseline] (6.159 ms) : 0, 6159
Debugger [candidate] (6.154 ms) : 0, 6154
Remote Config [baseline] (599.26 µs) : 0, 599
Remote Config [candidate] (605.655 µs) : 0, 606
Telemetry [baseline] (8.452 ms) : 0, 8452
Telemetry [candidate] (8.317 ms) : 0, 8317
Flare Poller [baseline] (4.11 ms) : 0, 4110
Flare Poller [candidate] (4.093 ms) : 0, 4093
IAST [baseline] (32.687 ms) : 0, 32687
IAST [candidate] (34.928 ms) : 0, 34928

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.55.0-SNAPSHOT~01aed836f3, baseline=1.55.0-SNAPSHOT~33bbc70b29

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.031 s) : 0, 1031334
Total [baseline] (10.854 s) : 0, 10853517
Agent [candidate] (1.03 s) : 0, 1030110
Total [candidate] (10.882 s) : 0, 10881853
section appsec
Agent [baseline] (1.201 s) : 0, 1201293
Total [baseline] (10.859 s) : 0, 10858537
Agent [candidate] (1.204 s) : 0, 1203960
Total [candidate] (10.88 s) : 0, 10879620
section iast
Agent [baseline] (1.16 s) : 0, 1160192
Total [baseline] (11.177 s) : 0, 11176959
Agent [candidate] (1.169 s) : 0, 1169434
Total [candidate] (11.153 s) : 0, 11153348
section profiling
Agent [baseline] (1.173 s) : 0, 1172859
Total [baseline] (10.951 s) : 0, 10951041
Agent [candidate] (1.175 s) : 0, 1174832
Total [candidate] (10.918 s) : 0, 10917898

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.031 s	-
Agent	appsec	1.201 s	169.96 ms (16.5%)
Agent	iast	1.16 s	128.858 ms (12.5%)
Agent	profiling	1.173 s	141.525 ms (13.7%)
Total	tracing	10.854 s	-
Total	appsec	10.859 s	5.019 ms (0.0%)
Total	iast	11.177 s	323.442 ms (3.0%)
Total	profiling	10.951 s	97.524 ms (0.9%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.03 s	-
Agent	appsec	1.204 s	173.85 ms (16.9%)
Agent	iast	1.169 s	139.324 ms (13.5%)
Agent	profiling	1.175 s	144.722 ms (14.0%)
Total	tracing	10.882 s	-
Total	appsec	10.88 s	-2.233 ms (-0.0%)
Total	iast	11.153 s	271.496 ms (2.5%)
Total	profiling	10.918 s	36.045 ms (0.3%)

gantt
    title petclinic - break down per module: candidate=1.55.0-SNAPSHOT~01aed836f3, baseline=1.55.0-SNAPSHOT~33bbc70b29

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.466 ms) : 0, 1466
crashtracking [candidate] (1.458 ms) : 0, 1458
BytebuddyAgent [baseline] (703.191 ms) : 0, 703191
BytebuddyAgent [candidate] (702.602 ms) : 0, 702602
GlobalTracer [baseline] (244.863 ms) : 0, 244863
GlobalTracer [candidate] (244.923 ms) : 0, 244923
AppSec [baseline] (32.426 ms) : 0, 32426
AppSec [candidate] (32.179 ms) : 0, 32179
Debugger [baseline] (6.388 ms) : 0, 6388
Debugger [candidate] (6.407 ms) : 0, 6407
Remote Config [baseline] (676.962 µs) : 0, 677
Remote Config [candidate] (665.165 µs) : 0, 665
Telemetry [baseline] (11.566 ms) : 0, 11566
Telemetry [candidate] (14.254 ms) : 0, 14254
Flare Poller [baseline] (9.533 ms) : 0, 9533
Flare Poller [candidate] (6.385 ms) : 0, 6385
section appsec
crashtracking [baseline] (1.468 ms) : 0, 1468
crashtracking [candidate] (1.466 ms) : 0, 1466
BytebuddyAgent [baseline] (725.755 ms) : 0, 725755
BytebuddyAgent [candidate] (726.963 ms) : 0, 726963
GlobalTracer [baseline] (235.455 ms) : 0, 235455
GlobalTracer [candidate] (236.087 ms) : 0, 236087
AppSec [baseline] (173.785 ms) : 0, 173785
AppSec [candidate] (174.413 ms) : 0, 174413
Debugger [baseline] (5.899 ms) : 0, 5899
Debugger [candidate] (5.909 ms) : 0, 5909
Remote Config [baseline] (625.909 µs) : 0, 626
Remote Config [candidate] (625.299 µs) : 0, 625
Telemetry [baseline] (8.392 ms) : 0, 8392
Telemetry [candidate] (8.365 ms) : 0, 8365
Flare Poller [baseline] (3.88 ms) : 0, 3880
Flare Poller [candidate] (3.891 ms) : 0, 3891
IAST [baseline] (24.914 ms) : 0, 24914
IAST [candidate] (25.043 ms) : 0, 25043
section iast
crashtracking [baseline] (1.465 ms) : 0, 1465
crashtracking [candidate] (1.48 ms) : 0, 1480
BytebuddyAgent [baseline] (823.04 ms) : 0, 823040
BytebuddyAgent [candidate] (830.028 ms) : 0, 830028
GlobalTracer [baseline] (233.513 ms) : 0, 233513
GlobalTracer [candidate] (234.825 ms) : 0, 234825
AppSec [baseline] (28.04 ms) : 0, 28040
AppSec [candidate] (28.275 ms) : 0, 28275
Debugger [baseline] (6.188 ms) : 0, 6188
Debugger [candidate] (6.207 ms) : 0, 6207
Remote Config [baseline] (601.277 µs) : 0, 601
Remote Config [candidate] (607.776 µs) : 0, 608
Telemetry [baseline] (8.453 ms) : 0, 8453
Telemetry [candidate] (8.497 ms) : 0, 8497
Flare Poller [baseline] (4.157 ms) : 0, 4157
Flare Poller [candidate] (4.188 ms) : 0, 4188
IAST [baseline] (33.452 ms) : 0, 33452
IAST [candidate] (33.908 ms) : 0, 33908
section profiling
crashtracking [baseline] (1.453 ms) : 0, 1453
crashtracking [candidate] (1.467 ms) : 0, 1467
BytebuddyAgent [baseline] (726.336 ms) : 0, 726336
BytebuddyAgent [candidate] (727.707 ms) : 0, 727707
GlobalTracer [baseline] (219.819 ms) : 0, 219819
GlobalTracer [candidate] (220.412 ms) : 0, 220412
AppSec [baseline] (32.197 ms) : 0, 32197
AppSec [candidate] (32.283 ms) : 0, 32283
Debugger [baseline] (9.913 ms) : 0, 9913
Debugger [candidate] (13.988 ms) : 0, 13988
Remote Config [baseline] (695.453 µs) : 0, 695
Remote Config [candidate] (1.44 ms) : 0, 1440
Telemetry [baseline] (13.087 ms) : 0, 13087
Telemetry [candidate] (8.301 ms) : 0, 8301
Flare Poller [baseline] (4.136 ms) : 0, 4136
Flare Poller [candidate] (4.162 ms) : 0, 4162
ProfilingAgent [baseline] (109.768 ms) : 0, 109768
ProfilingAgent [candidate] (109.462 ms) : 0, 109462
Profiling [baseline] (110.378 ms) : 0, 110378
Profiling [candidate] (110.094 ms) : 0, 110094

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	malvarez/ai-guard-telemetry
git_commit_date	1761927667	1761991527
git_commit_sha	`33bbc70`	`01aed83`
release_version	1.55.0-SNAPSHOT~33bbc70b29	1.55.0-SNAPSHOT~01aed836f3

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1761994011	1761994011
ci_job_id	1210637728	1210637728
ci_pipeline_id	80964930	80964930
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-1-apawycl5 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-1-apawycl5 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 3 performance improvements and 3 performance regressions! Performance is the same for 6 metrics, 12 unstable metrics.

scenario	Δ mean http_req_duration	Δ mean throughput	candidate mean http_req_duration	candidate mean throughput	baseline mean http_req_duration	baseline mean throughput
scenario:load:insecure-bank:profiling:high_load	better [-183.594µs; -146.386µs] or [-8.835%; -7.045%]	unstable [-79.216op/s; +439.779op/s] or [-3.643%; +20.224%]	1.913ms	2354.812op/s	2.078ms	2174.531op/s
scenario:load:petclinic:no_agent:high_load	worse [+2.218ms; +2.622ms] or [+13.045%; +15.418%]	unstable [-63.580op/s; -4.483op/s] or [-23.178%; -1.634%]	19.425ms	240.281op/s	17.005ms	274.312op/s
scenario:load:petclinic:iast:high_load	worse [+1.588ms; +1.988ms] or [+9.044%; +11.324%]	unstable [-52.185op/s; +3.310op/s] or [-19.634%; +1.245%]	19.343ms	241.344op/s	17.555ms	265.781op/s
scenario:load:petclinic:tracing:high_load	better [-1179.269µs; -785.541µs] or [-6.404%; -4.266%]	unstable [-14.908op/s; +43.346op/s] or [-5.882%; +17.103%]	17.432ms	267.656op/s	18.414ms	253.438op/s
scenario:load:petclinic:appsec:high_load	better [-1.307ms; -0.890ms] or [-6.715%; -4.572%]	unstable [-12.518op/s; +41.080op/s] or [-5.220%; +17.132%]	18.366ms	254.062op/s	19.464ms	239.781op/s
scenario:load:petclinic:profiling:high_load	worse [+532.276µs; +973.351µs] or [+2.678%; +4.896%]	unstable [-33.569op/s; +16.444op/s] or [-14.294%; +7.002%]	20.632ms	226.281op/s	19.879ms	234.844op/s

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.55.0-SNAPSHOT~01aed836f3, baseline=1.55.0-SNAPSHOT~33bbc70b29
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.224 ms) : 1211, 1236
.   : milestone, 1224,
iast (3.241 ms) : 3196, 3285
.   : milestone, 3241,
iast_FULL (5.937 ms) : 5877, 5997
.   : milestone, 5937,
iast_GLOBAL (3.645 ms) : 3591, 3698
.   : milestone, 3645,
profiling (2.078 ms) : 2059, 2097
.   : milestone, 2078,
tracing (1.777 ms) : 1762, 1791
.   : milestone, 1777,
section candidate
no_agent (1.2 ms) : 1189, 1212
.   : milestone, 1200,
iast (3.343 ms) : 3303, 3384
.   : milestone, 3343,
iast_FULL (5.954 ms) : 5893, 6015
.   : milestone, 5954,
iast_GLOBAL (3.649 ms) : 3595, 3702
.   : milestone, 3649,
profiling (1.913 ms) : 1897, 1929
.   : milestone, 1913,
tracing (1.812 ms) : 1797, 1828
.   : milestone, 1812,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.224 ms [1.211 ms, 1.236 ms]	-
iast	3.241 ms [3.196 ms, 3.285 ms]	2.017 ms (164.8%)
iast_FULL	5.937 ms [5.877 ms, 5.997 ms]	4.713 ms (385.2%)
iast_GLOBAL	3.645 ms [3.591 ms, 3.698 ms]	2.421 ms (197.9%)
profiling	2.078 ms [2.059 ms, 2.097 ms]	854.267 µs (69.8%)
tracing	1.777 ms [1.762 ms, 1.791 ms]	553.041 µs (45.2%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.2 ms [1.189 ms, 1.212 ms]	-
iast	3.343 ms [3.303 ms, 3.384 ms]	2.143 ms (178.5%)
iast_FULL	5.954 ms [5.893 ms, 6.015 ms]	4.754 ms (396.0%)
iast_GLOBAL	3.649 ms [3.595 ms, 3.702 ms]	2.448 ms (204.0%)
profiling	1.913 ms [1.897 ms, 1.929 ms]	712.617 µs (59.4%)
tracing	1.812 ms [1.797 ms, 1.828 ms]	611.906 µs (51.0%)

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.55.0-SNAPSHOT~01aed836f3, baseline=1.55.0-SNAPSHOT~33bbc70b29
    dateFormat X
    axisFormat %s
section baseline
no_agent (17.005 ms) : 16835, 17176
.   : milestone, 17005,
appsec (19.464 ms) : 19264, 19664
.   : milestone, 19464,
code_origins (18.688 ms) : 18501, 18875
.   : milestone, 18688,
iast (17.555 ms) : 17377, 17733
.   : milestone, 17555,
profiling (19.879 ms) : 19679, 20079
.   : milestone, 19879,
tracing (18.414 ms) : 18224, 18604
.   : milestone, 18414,
section candidate
no_agent (19.425 ms) : 19222, 19628
.   : milestone, 19425,
appsec (18.366 ms) : 18179, 18553
.   : milestone, 18366,
code_origins (18.379 ms) : 18196, 18563
.   : milestone, 18379,
iast (19.343 ms) : 19149, 19536
.   : milestone, 19343,
profiling (20.632 ms) : 20422, 20841
.   : milestone, 20632,
tracing (17.432 ms) : 17256, 17607
.   : milestone, 17432,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	17.005 ms [16.835 ms, 17.176 ms]	-
appsec	19.464 ms [19.264 ms, 19.664 ms]	2.459 ms (14.5%)
code_origins	18.688 ms [18.501 ms, 18.875 ms]	1.683 ms (9.9%)
iast	17.555 ms [17.377 ms, 17.733 ms]	549.789 µs (3.2%)
profiling	19.879 ms [19.679 ms, 20.079 ms]	2.873 ms (16.9%)
tracing	18.414 ms [18.224 ms, 18.604 ms]	1.409 ms (8.3%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	19.425 ms [19.222 ms, 19.628 ms]	-
appsec	18.366 ms [18.179 ms, 18.553 ms]	-1.059 ms (-5.5%)
code_origins	18.379 ms [18.196 ms, 18.563 ms]	-1.046 ms (-5.4%)
iast	19.343 ms [19.149 ms, 19.536 ms]	-82.487 µs (-0.4%)
profiling	20.632 ms [20.422 ms, 20.841 ms]	1.206 ms (6.2%)
tracing	17.432 ms [17.256 ms, 17.607 ms]	-1.994 ms (-10.3%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	malvarez/ai-guard-telemetry
git_commit_date	1761927667	1761991527
git_commit_sha	`33bbc70`	`01aed83`
release_version	1.55.0-SNAPSHOT~33bbc70b29	1.55.0-SNAPSHOT~01aed836f3

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1761993682	1761993682
ci_job_id	1210637729	1210637729
ci_pipeline_id	80964930	80964930
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-2-1jd5r6b5 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-2-1jd5r6b5 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 10 metrics, 2 unstable metrics.

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.55.0-SNAPSHOT~01aed836f3, baseline=1.55.0-SNAPSHOT~33bbc70b29
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.004 s) : 15004000, 15004000
.   : milestone, 15004000,
appsec (14.709 s) : 14709000, 14709000
.   : milestone, 14709000,
iast (18.308 s) : 18308000, 18308000
.   : milestone, 18308000,
iast_GLOBAL (18.036 s) : 18036000, 18036000
.   : milestone, 18036000,
profiling (15.808 s) : 15808000, 15808000
.   : milestone, 15808000,
tracing (15.334 s) : 15334000, 15334000
.   : milestone, 15334000,
section candidate
no_agent (14.958 s) : 14958000, 14958000
.   : milestone, 14958000,
appsec (14.943 s) : 14943000, 14943000
.   : milestone, 14943000,
iast (18.47 s) : 18470000, 18470000
.   : milestone, 18470000,
iast_GLOBAL (17.867 s) : 17867000, 17867000
.   : milestone, 17867000,
profiling (15.349 s) : 15349000, 15349000
.   : milestone, 15349000,
tracing (15.193 s) : 15193000, 15193000
.   : milestone, 15193000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.004 s [15.004 s, 15.004 s]	-
appsec	14.709 s [14.709 s, 14.709 s]	-295.0 ms (-2.0%)
iast	18.308 s [18.308 s, 18.308 s]	3.304 s (22.0%)
iast_GLOBAL	18.036 s [18.036 s, 18.036 s]	3.032 s (20.2%)
profiling	15.808 s [15.808 s, 15.808 s]	804.0 ms (5.4%)
tracing	15.334 s [15.334 s, 15.334 s]	330.0 ms (2.2%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	14.958 s [14.958 s, 14.958 s]	-
appsec	14.943 s [14.943 s, 14.943 s]	-15.0 ms (-0.1%)
iast	18.47 s [18.47 s, 18.47 s]	3.512 s (23.5%)
iast_GLOBAL	17.867 s [17.867 s, 17.867 s]	2.909 s (19.4%)
profiling	15.349 s [15.349 s, 15.349 s]	391.0 ms (2.6%)
tracing	15.193 s [15.193 s, 15.193 s]	235.0 ms (1.6%)

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.55.0-SNAPSHOT~01aed836f3, baseline=1.55.0-SNAPSHOT~33bbc70b29
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.472 ms) : 1461, 1484
.   : milestone, 1472,
appsec (3.704 ms) : 3485, 3922
.   : milestone, 3704,
iast (2.203 ms) : 2140, 2267
.   : milestone, 2203,
iast_GLOBAL (2.24 ms) : 2176, 2304
.   : milestone, 2240,
profiling (2.466 ms) : 2236, 2696
.   : milestone, 2466,
tracing (2.02 ms) : 1971, 2070
.   : milestone, 2020,
section candidate
no_agent (1.473 ms) : 1461, 1484
.   : milestone, 1473,
appsec (3.628 ms) : 3416, 3840
.   : milestone, 3628,
iast (2.2 ms) : 2137, 2264
.   : milestone, 2200,
iast_GLOBAL (2.25 ms) : 2186, 2313
.   : milestone, 2250,
profiling (2.048 ms) : 1997, 2099
.   : milestone, 2048,
tracing (2.021 ms) : 1972, 2071
.   : milestone, 2021,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.472 ms [1.461 ms, 1.484 ms]	-
appsec	3.704 ms [3.485 ms, 3.922 ms]	2.231 ms (151.6%)
iast	2.203 ms [2.14 ms, 2.267 ms]	731.156 µs (49.7%)
iast_GLOBAL	2.24 ms [2.176 ms, 2.304 ms]	767.638 µs (52.1%)
profiling	2.466 ms [2.236 ms, 2.696 ms]	993.435 µs (67.5%)
tracing	2.02 ms [1.971 ms, 2.07 ms]	547.92 µs (37.2%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.473 ms [1.461 ms, 1.484 ms]	-
appsec	3.628 ms [3.416 ms, 3.84 ms]	2.155 ms (146.3%)
iast	2.2 ms [2.137 ms, 2.264 ms]	727.716 µs (49.4%)
iast_GLOBAL	2.25 ms [2.186 ms, 2.313 ms]	776.901 µs (52.8%)
profiling	2.048 ms [1.997 ms, 2.099 ms]	574.851 µs (39.0%)
tracing	2.021 ms [1.972 ms, 2.071 ms]	548.659 µs (37.3%)

jandro996 · 2025-11-01T10:03:30Z

dd-java-agent/agent-aiguard/src/main/java/com/datadog/aiguard/AIGuardInternal.java

-    map.put(key1, prop1);
-    map.put(key2, prop2);
+  private static Map<String, String> mapOf(final String... props) {
+    final Map<String, String> map = new HashMap<>(props.length << 1);


It’s probably not needed since it looks like there are always pairs, but what do you think about adding a safeguard to check that the args count is even, just to avoid a potential ArrayIndexOutOfBoundsException?

Suggested change

final Map<String, String> map = new HashMap<>(props.length << 1);

if (props.length % 2 != 0) {

throw new IllegalArgumentException("Props must be even");

}

final Map<String, String> map = new HashMap<>(props.length << 1);

jandro996

LGTM just a minor comment

manuel-alvarez-alvarez requested a review from a team as a code owner October 30, 2025 13:09

manuel-alvarez-alvarez requested review from jandro996 and smola October 30, 2025 13:09

manuel-alvarez-alvarez added type: enhancement Enhancements and improvements tag: no release notes Changes to exclude from release notes comp: asm waf Application Security Management (WAF) labels Oct 30, 2025

manuel-alvarez-alvarez force-pushed the malvarez/ai-guard-telemetry branch from 223ce1c to 036f183 Compare October 30, 2025 13:10

Add telemetry to AI Guard evaluations

4707290

manuel-alvarez-alvarez force-pushed the malvarez/ai-guard-telemetry branch from 036f183 to 4707290 Compare October 30, 2025 20:13

Fix coverage and telemetry tests

c82915e

jandro996 reviewed Nov 1, 2025

View reviewed changes

jandro996 approved these changes Nov 1, 2025

View reviewed changes

Merge branch 'master' into malvarez/ai-guard-telemetry

01aed83

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add telemetry to AI Guard evaluations #9884

Add telemetry to AI Guard evaluations #9884

manuel-alvarez-alvarez commented Oct 30, 2025

Uh oh!

datadog-datadog-prod-us1 bot commented Oct 30, 2025 •

edited by datadog-official bot

Loading

Uh oh!

pr-commenter bot commented Oct 30, 2025 •

edited

Loading

Uh oh!

jandro996 Nov 1, 2025

Uh oh!

jandro996 left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Add telemetry to AI Guard evaluations #9884

Are you sure you want to change the base?

Add telemetry to AI Guard evaluations #9884

Conversation

manuel-alvarez-alvarez commented Oct 30, 2025

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Uh oh!

datadog-datadog-prod-us1 bot commented Oct 30, 2025 • edited by datadog-official bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pr-commenter bot commented Oct 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

Uh oh!

jandro996 Nov 1, 2025

Choose a reason for hiding this comment

Uh oh!

jandro996 left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

datadog-datadog-prod-us1 bot commented Oct 30, 2025 •

edited by datadog-official bot

Loading

pr-commenter bot commented Oct 30, 2025 •

edited

Loading