Update metrics: appsec.waf.updates and appsec.waf.init #8280

Mariovido · 2025-01-24T10:53:11Z

What Does This Do

This adds a new value to some metrics which is necessary for the consolidation of ASM Span Tags, Metrics, and Logs across all supported languages. The newly value will be implemented in the following metrics:

appsec.waf.updates:
- success: Whether the update resulted in a usable WAF handle
appsec.waf.init:
- success: Whether the initialization resulted in a usable WAF handle

Motivation

Our goal is to implement all the missing ASM Span Tags, Metrics, and Logs.

Additional Notes

Also, this PR adds tests that were missing and some improvements to the previous ones.

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any usefull labels
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the public documentation in case of new configuration flag or behavior

Jira ticket: APPSEC-56478

pr-commenter · 2025-01-24T11:30:59Z

Benchmarks

Startup

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
end_time	2025-01-31T13:42:12	2025-01-31T13:49:17
git_branch	master	mario.vidal/waf_metrics
git_commit_date	1738318728	1738330321
git_commit_sha	`b24d153`	`036e210`
release_version	1.47.0-SNAPSHOT~b24d15319f	1.47.0-SNAPSHOT~036e210ad8
start_time	2025-01-31T13:41:58	2025-01-31T13:49:03

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1738331715	1738331715
ci_job_id	789186750	789186750
ci_pipeline_id	54569764	54569764
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-yspv7zba-project-304-concurrent-0-opumwiyd 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-yspv7zba-project-304-concurrent-0-opumwiyd 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 10 metrics, 18 unstable metrics.

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.47.0-SNAPSHOT~036e210ad8, baseline=1.47.0-SNAPSHOT~b24d15319f
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.367 ms) : 1348, 1385
.   : milestone, 1367,
appsec (1.778 ms) : 1756, 1801
.   : milestone, 1778,
appsec_no_iast (1.773 ms) : 1749, 1798
.   : milestone, 1773,
iast (1.517 ms) : 1493, 1541
.   : milestone, 1517,
profiling (1.505 ms) : 1481, 1528
.   : milestone, 1505,
tracing (1.483 ms) : 1457, 1508
.   : milestone, 1483,
section candidate
no_agent (1.351 ms) : 1331, 1370
.   : milestone, 1351,
appsec (1.76 ms) : 1736, 1783
.   : milestone, 1760,
appsec_no_iast (1.766 ms) : 1743, 1789
.   : milestone, 1766,
iast (1.51 ms) : 1484, 1535
.   : milestone, 1510,
profiling (1.478 ms) : 1454, 1501
.   : milestone, 1478,
tracing (1.502 ms) : 1477, 1527
.   : milestone, 1502,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.367 ms [1.348 ms, 1.385 ms]	-
appsec	1.778 ms [1.756 ms, 1.801 ms]	411.574 µs (30.1%)
appsec_no_iast	1.773 ms [1.749 ms, 1.798 ms]	406.455 µs (29.7%)
iast	1.517 ms [1.493 ms, 1.541 ms]	150.067 µs (11.0%)
profiling	1.505 ms [1.481 ms, 1.528 ms]	137.662 µs (10.1%)
tracing	1.483 ms [1.457 ms, 1.508 ms]	115.898 µs (8.5%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.351 ms [1.331 ms, 1.37 ms]	-
appsec	1.76 ms [1.736 ms, 1.783 ms]	408.867 µs (30.3%)
appsec_no_iast	1.766 ms [1.743 ms, 1.789 ms]	415.289 µs (30.7%)
iast	1.51 ms [1.484 ms, 1.535 ms]	158.621 µs (11.7%)
profiling	1.478 ms [1.454 ms, 1.501 ms]	126.756 µs (9.4%)
tracing	1.502 ms [1.477 ms, 1.527 ms]	150.894 µs (11.2%)

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.47.0-SNAPSHOT~036e210ad8, baseline=1.47.0-SNAPSHOT~b24d15319f
    dateFormat X
    axisFormat %s
section baseline
no_agent (381.237 µs) : 361, 401
.   : milestone, 381,
iast (513.013 µs) : 490, 536
.   : milestone, 513,
iast_FULL (747.571 µs) : 725, 771
.   : milestone, 748,
iast_GLOBAL (558.971 µs) : 535, 582
.   : milestone, 559,
iast_HARDCODED_SECRET_DISABLED (510.03 µs) : 487, 533
.   : milestone, 510,
iast_INACTIVE (460.161 µs) : 439, 481
.   : milestone, 460,
iast_TELEMETRY_OFF (498.112 µs) : 475, 521
.   : milestone, 498,
tracing (452.713 µs) : 432, 474
.   : milestone, 453,
section candidate
no_agent (381.826 µs) : 362, 401
.   : milestone, 382,
iast (511.858 µs) : 489, 535
.   : milestone, 512,
iast_FULL (743.621 µs) : 722, 766
.   : milestone, 744,
iast_GLOBAL (565.113 µs) : 541, 589
.   : milestone, 565,
iast_HARDCODED_SECRET_DISABLED (510.273 µs) : 488, 532
.   : milestone, 510,
iast_INACTIVE (463.604 µs) : 441, 486
.   : milestone, 464,
iast_TELEMETRY_OFF (498.7 µs) : 475, 522
.   : milestone, 499,
tracing (458.959 µs) : 437, 480
.   : milestone, 459,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	381.237 µs [361.31 µs, 401.165 µs]	-
iast	513.013 µs [490.116 µs, 535.91 µs]	131.776 µs (34.6%)
iast_FULL	747.571 µs [724.555 µs, 770.587 µs]	366.334 µs (96.1%)
iast_GLOBAL	558.971 µs [535.463 µs, 582.479 µs]	177.734 µs (46.6%)
iast_HARDCODED_SECRET_DISABLED	510.03 µs [487.053 µs, 533.007 µs]	128.793 µs (33.8%)
iast_INACTIVE	460.161 µs [439.093 µs, 481.23 µs]	78.924 µs (20.7%)
iast_TELEMETRY_OFF	498.112 µs [474.755 µs, 521.469 µs]	116.875 µs (30.7%)
tracing	452.713 µs [431.674 µs, 473.751 µs]	71.476 µs (18.7%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	381.826 µs [362.238 µs, 401.414 µs]	-
iast	511.858 µs [488.504 µs, 535.212 µs]	130.032 µs (34.1%)
iast_FULL	743.621 µs [721.637 µs, 765.604 µs]	361.795 µs (94.8%)
iast_GLOBAL	565.113 µs [540.977 µs, 589.248 µs]	183.287 µs (48.0%)
iast_HARDCODED_SECRET_DISABLED	510.273 µs [488.157 µs, 532.388 µs]	128.447 µs (33.6%)
iast_INACTIVE	463.604 µs [440.765 µs, 486.444 µs]	81.779 µs (21.4%)
iast_TELEMETRY_OFF	498.7 µs [475.148 µs, 522.253 µs]	116.875 µs (30.6%)
tracing	458.959 µs [437.481 µs, 480.436 µs]	77.133 µs (20.2%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	mario.vidal/waf_metrics
git_commit_date	1738318728	1738330321
git_commit_sha	`b24d153`	`036e210`
release_version	1.47.0-SNAPSHOT~b24d15319f	1.47.0-SNAPSHOT~036e210ad8

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1738332246	1738332246
ci_job_id	789186753	789186753
ci_pipeline_id	54569764	54569764
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-ztgbsgil-project-304-concurrent-1-n39y0xa8 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-ztgbsgil-project-304-concurrent-1-n39y0xa8 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
variant	appsec	appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.47.0-SNAPSHOT~036e210ad8, baseline=1.47.0-SNAPSHOT~b24d15319f
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.208 s) : 15208000, 15208000
.   : milestone, 15208000,
appsec (15.002 s) : 15002000, 15002000
.   : milestone, 15002000,
iast (18.332 s) : 18332000, 18332000
.   : milestone, 18332000,
iast_GLOBAL (18.187 s) : 18187000, 18187000
.   : milestone, 18187000,
profiling (15.701 s) : 15701000, 15701000
.   : milestone, 15701000,
tracing (15.324 s) : 15324000, 15324000
.   : milestone, 15324000,
section candidate
no_agent (15.57 s) : 15570000, 15570000
.   : milestone, 15570000,
appsec (14.761 s) : 14761000, 14761000
.   : milestone, 14761000,
iast (18.714 s) : 18714000, 18714000
.   : milestone, 18714000,
iast_GLOBAL (17.822 s) : 17822000, 17822000
.   : milestone, 17822000,
profiling (15.109 s) : 15109000, 15109000
.   : milestone, 15109000,
tracing (14.837 s) : 14837000, 14837000
.   : milestone, 14837000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.208 s [15.208 s, 15.208 s]	-
appsec	15.002 s [15.002 s, 15.002 s]	-206.0 ms (-1.4%)
iast	18.332 s [18.332 s, 18.332 s]	3.124 s (20.5%)
iast_GLOBAL	18.187 s [18.187 s, 18.187 s]	2.979 s (19.6%)
profiling	15.701 s [15.701 s, 15.701 s]	493.0 ms (3.2%)
tracing	15.324 s [15.324 s, 15.324 s]	116.0 ms (0.8%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.57 s [15.57 s, 15.57 s]	-
appsec	14.761 s [14.761 s, 14.761 s]	-809.0 ms (-5.2%)
iast	18.714 s [18.714 s, 18.714 s]	3.144 s (20.2%)
iast_GLOBAL	17.822 s [17.822 s, 17.822 s]	2.252 s (14.5%)
profiling	15.109 s [15.109 s, 15.109 s]	-461.0 ms (-3.0%)
tracing	14.837 s [14.837 s, 14.837 s]	-733.0 ms (-4.7%)

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.47.0-SNAPSHOT~036e210ad8, baseline=1.47.0-SNAPSHOT~b24d15319f
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.465 ms) : 1454, 1477
.   : milestone, 1465,
appsec (2.347 ms) : 2303, 2390
.   : milestone, 2347,
iast (2.096 ms) : 2042, 2151
.   : milestone, 2096,
iast_GLOBAL (2.143 ms) : 2088, 2198
.   : milestone, 2143,
profiling (1.958 ms) : 1914, 2002
.   : milestone, 1958,
tracing (1.949 ms) : 1906, 1991
.   : milestone, 1949,
section candidate
no_agent (1.465 ms) : 1454, 1477
.   : milestone, 1465,
appsec (2.351 ms) : 2308, 2394
.   : milestone, 2351,
iast (2.105 ms) : 2050, 2160
.   : milestone, 2105,
iast_GLOBAL (2.141 ms) : 2086, 2197
.   : milestone, 2141,
profiling (1.971 ms) : 1926, 2015
.   : milestone, 1971,
tracing (1.946 ms) : 1904, 1989
.   : milestone, 1946,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.465 ms [1.454 ms, 1.477 ms]	-
appsec	2.347 ms [2.303 ms, 2.39 ms]	881.055 µs (60.1%)
iast	2.096 ms [2.042 ms, 2.151 ms]	630.836 µs (43.0%)
iast_GLOBAL	2.143 ms [2.088 ms, 2.198 ms]	677.736 µs (46.2%)
profiling	1.958 ms [1.914 ms, 2.002 ms]	492.547 µs (33.6%)
tracing	1.949 ms [1.906 ms, 1.991 ms]	483.318 µs (33.0%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.465 ms [1.454 ms, 1.477 ms]	-
appsec	2.351 ms [2.308 ms, 2.394 ms]	885.327 µs (60.4%)
iast	2.105 ms [2.05 ms, 2.16 ms]	639.546 µs (43.6%)
iast_GLOBAL	2.141 ms [2.086 ms, 2.197 ms]	675.729 µs (46.1%)
profiling	1.971 ms [1.926 ms, 2.015 ms]	505.125 µs (34.5%)
tracing	1.946 ms [1.904 ms, 1.989 ms]	480.785 µs (32.8%)

github-actions · 2025-01-31T14:19:13Z

Hi! 👋 Thanks for your pull request! 🎉

To help us review it, please make sure to:

Add at least one type, and one component or instrumentation label to the pull request

If you need help, please check our contributing guidelines.

Update metrics to have success value

952ab27

Mariovido added type: enhancement comp: asm iast Application Security Management (IAST) labels Jan 24, 2025

Fix tests

b19e7ef

Mariovido marked this pull request as ready for review January 27, 2025 10:58

Mariovido requested review from a team as code owners January 27, 2025 10:58

Mariovido requested review from smola and ValentinZakharov January 27, 2025 10:58

smola approved these changes Jan 29, 2025

View reviewed changes

smola added comp: asm waf Application Security Management (WAF) and removed comp: asm iast Application Security Management (IAST) labels Jan 31, 2025

smola approved these changes Jan 31, 2025

View reviewed changes

Merge with master

036e210

Mariovido added type: enhancement and removed type: enhancement labels Jan 31, 2025

jandro996 approved these changes Feb 3, 2025

View reviewed changes

Mariovido merged commit 9af5347 into master Feb 3, 2025
197 of 198 checks passed

Mariovido deleted the mario.vidal/waf_metrics branch February 3, 2025 10:25

github-actions bot added this to the 1.47.0 milestone Feb 3, 2025

cbeauchesne mentioned this pull request Feb 3, 2025

Add success to valid WAF telemetry tags DataDog/system-tests#3966

Merged

7 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update metrics: appsec.waf.updates and appsec.waf.init #8280

Update metrics: appsec.waf.updates and appsec.waf.init #8280

Mariovido commented Jan 24, 2025 •

edited by jira bot

Loading

pr-commenter bot commented Jan 24, 2025 •

edited

Loading

github-actions bot commented Jan 31, 2025

Update metrics: appsec.waf.updates and appsec.waf.init #8280

Update metrics: appsec.waf.updates and appsec.waf.init #8280

Conversation

Mariovido commented Jan 24, 2025 • edited by jira bot Loading

What Does This Do

Motivation

Additional Notes

Contributor Checklist

pr-commenter bot commented Jan 24, 2025 • edited Loading

Benchmarks

Startup

Load

Parameters

Summary

Dacapo

Parameters

Summary

github-actions bot commented Jan 31, 2025

Mariovido commented Jan 24, 2025 •

edited by jira bot

Loading

pr-commenter bot commented Jan 24, 2025 •

edited

Loading