feat(stats)!: add whole key cardinality limit by VianneyRuhlmann · Pull Request #2158 · DataDog/libdatadog

VianneyRuhlmann · 2026-06-24T16:43:43Z

What does this PR do?

Add whole-key cardinality limit to the SpanConcentrator. See this RFC for details

The telemetry metrics specified in the RFC are implemented in #2159

The per fields limit from the RFC is out of the scope of this PR

Breaking change

The cardinality limit is applied to all users of the SpanConcentrator however the limit can be configured manually.

Motivation

Avoid unbounded bucket growth when dealing with high cardinality.

Additional Notes

Limit default value

The limit of 7000 entries has been chosen based on the following results

Single Bucket

Component	Typical	Worst
HashMap flat table	3.5 MB (unchanged)	3.5 MB (unchanged)
String heap (7 001 × 76 / 122 B)	0.5 MB	0.9 MB
DDSketch bins (7 001 × 1 600 / 32 768 B)	11.2 MB	229.3 MB
StatsBucket total	~15.2 MB	~233.7 MB

SpanConcentrator

Limit	Typical (3 buckets)	Worst (3 buckets)
7 000	~45.6 MB	~701.2 MB

Trilean change

The overflow sentinel key requires us to use a trilean to represent the is_trace_root in the aggregation key.

VianneyRuhlmann · 2026-06-24T16:43:57Z

This stack of pull requests is managed by Graphite. Learn more about stacking.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a17a2c66d6

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

dd-octo-sts · 2026-06-24T17:23:52Z

Artifact Size Benchmark Report

aarch64-alpine-linux-musl

Artifact	Baseline	Commit	Change
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.so	7.82 MB	7.82 MB	0% (0 B) 👌
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.a	85.14 MB	85.18 MB	+.05% (+44.46 KB) 🔍

aarch64-unknown-linux-gnu

Artifact	Baseline	Commit	Change
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.a	96.27 MB	96.32 MB	+.04% (+46.59 KB) 🔍
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.so	10.51 MB	10.51 MB	+.01% (+1.50 KB) 🔍

libdatadog-x64-windows

Artifact	Baseline	Commit	Change
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.dll	25.14 MB	25.15 MB	+.03% (+8.00 KB) 🔍
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.lib	88.04 KB	88.04 KB	0% (0 B) 👌
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.pdb	183.36 MB	183.41 MB	+.02% (+48.00 KB) 🔍
/libdatadog-x64-windows/debug/static/datadog_profiling_ffi.lib	938.86 MB	939.01 MB	+.01% (+148.33 KB) 🔍
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.dll	8.22 MB	8.23 MB	+.04% (+3.50 KB) 🔍
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.lib	88.04 KB	88.04 KB	0% (0 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.pdb	24.30 MB	24.33 MB	+.09% (+24.00 KB) 🔍
/libdatadog-x64-windows/release/static/datadog_profiling_ffi.lib	48.47 MB	48.49 MB	+.05% (+25.79 KB) 🔍

libdatadog-x86-windows

Artifact	Baseline	Commit	Change
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.dll	21.79 MB	21.80 MB	+.05% (+12.00 KB) 🔍
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.lib	89.42 KB	89.42 KB	0% (0 B) 👌
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.pdb	187.40 MB	187.47 MB	+.03% (+72.00 KB) 🔍
/libdatadog-x86-windows/debug/static/datadog_profiling_ffi.lib	927.44 MB	927.62 MB	+.01% (+181.18 KB) 🔍
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.dll	6.35 MB	6.35 MB	+.06% (+4.00 KB) 🔍
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.lib	89.42 KB	89.42 KB	0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.pdb	26.09 MB	26.11 MB	+.05% (+16.00 KB) 🔍
/libdatadog-x86-windows/release/static/datadog_profiling_ffi.lib	46.11 MB	46.13 MB	+.05% (+25.41 KB) 🔍

x86_64-alpine-linux-musl

Artifact	Baseline	Commit	Change
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.a	75.88 MB	75.92 MB	+.05% (+41.58 KB) 🔍
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.so	8.70 MB	8.70 MB	0% (0 B) 👌

x86_64-unknown-linux-gnu

Artifact	Baseline	Commit	Change
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.a	91.34 MB	91.39 MB	+.04% (+42.22 KB) 🔍
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.so	10.59 MB	10.59 MB	+0% (+896 B) 👌

datadog-official · 2026-06-24T17:43:37Z

Tests

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

🎯 Code Coverage (details)
• Patch Coverage: 99.20%
• Overall Coverage: 74.07% (+0.08%)

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: 8d22e4b | Docs | Datadog PR Page | Give us feedback!}

github-actions · 2026-06-24T17:44:20Z

Clippy Allow Annotation Report

Comparing clippy allow annotations between branches:

Base Branch: origin/main
PR Branch: origin/06-24-feat_stats_add_whole-key_cardinality_limit

Summary by Rule

Rule	Base Branch	PR Branch	Change
unwrap_used	1	1	No change (0%)
Total	1	1	No change (0%)

Annotation Counts by File

File	Base Branch	PR Branch	Change
`libdd-trace-stats/src/stats_exporter.rs`	1	1	No change (0%)

Annotation Stats by Crate

Crate	Base Branch	PR Branch	Change
`clippy-annotation-reporter`	5	5	No change (0%)
`datadog-ffe-ffi`	1	1	No change (0%)
`datadog-ipc`	22	22	No change (0%)
`datadog-live-debugger`	4	4	No change (0%)
`datadog-live-debugger-ffi`	10	10	No change (0%)
`datadog-profiling-replayer`	4	4	No change (0%)
`datadog-sidecar`	45	45	No change (0%)
`libdd-common`	13	13	No change (0%)
`libdd-common-ffi`	12	12	No change (0%)
`libdd-data-pipeline`	6	6	No change (0%)
`libdd-ddsketch`	2	2	No change (0%)
`libdd-dogstatsd-client`	1	1	No change (0%)
`libdd-profiling`	13	13	No change (0%)
`libdd-remote-config`	3	3	No change (0%)
`libdd-telemetry`	20	20	No change (0%)
`libdd-tinybytes`	4	4	No change (0%)
`libdd-trace-normalization`	2	2	No change (0%)
`libdd-trace-obfuscation`	3	3	No change (0%)
`libdd-trace-stats`	1	1	No change (0%)
`libdd-trace-utils`	11	11	No change (0%)
Total	182	182	No change (0%)

About This Report

This report tracks Clippy allow annotations for specific rules, showing how they've changed in this PR. Decreasing the number of these annotations generally improves code quality.

Eldolfin

LGTM

gh-worker-devflow-routing-ef8351 · 2026-06-30T13:05:51Z

View all feedbacks in Devflow UI.

2026-06-30 13:05:51 UTC ℹ️ Start processing command devflow:merge

2026-06-30 13:05:57 UTC ℹ️ MergeQueue: pull request added to the queue

The expected merge time in main is approximately 1h (p90).

2026-06-30 14:12:50 UTC ℹ️ MergeQueue: This merge request was merged

This was referenced Jun 24, 2026

feat(stats)!: send telemetry for cardinality limits #2159

Open

feat(trace_exporter): enable telemetry in stats exporter #2160

Open

VianneyRuhlmann changed the title ~~chore(protobuf): add serde to trilean~~ feat(stats): add whole key cardinality limit Jun 24, 2026

VianneyRuhlmann marked this pull request as ready for review June 24, 2026 17:00

VianneyRuhlmann requested review from a team as code owners June 24, 2026 17:00

chatgpt-codex-connector Bot reviewed Jun 24, 2026

View reviewed changes

Comment thread libdd-trace-stats/src/span_concentrator/aggregation.rs

VianneyRuhlmann force-pushed the 06-24-feat_stats_add_whole-key_cardinality_limit branch from a17a2c6 to 2655975 Compare June 24, 2026 17:41

github-actions Bot added the data-pipeline label Jun 24, 2026

VianneyRuhlmann force-pushed the 06-24-feat_stats_add_whole-key_cardinality_limit branch 3 times, most recently from ce821ea to ebd7e4e Compare June 24, 2026 18:15

ajgajg1134 approved these changes Jun 24, 2026

View reviewed changes

Comment thread libdd-trace-stats/src/span_concentrator/aggregation.rs

Eldolfin approved these changes Jun 25, 2026

View reviewed changes

VianneyRuhlmann changed the title ~~feat(stats): add whole key cardinality limit~~ feat(stats)!: add whole key cardinality limit Jun 26, 2026

VianneyRuhlmann force-pushed the 06-24-feat_stats_add_whole-key_cardinality_limit branch from ebd7e4e to b04c7f6 Compare June 29, 2026 14:20

VianneyRuhlmann mentioned this pull request Jun 29, 2026

feat(sidecar): enable telemetry for stats #2175

Open

VianneyRuhlmann added 3 commits June 29, 2026 17:29

chore(protobuf): add serde to trilean

52c7a87

chore(stats): use trilean for is_trace_root

7f2898a

feat(stats): add whole-key cardinality limit

8d22e4b

VianneyRuhlmann force-pushed the 06-24-feat_stats_add_whole-key_cardinality_limit branch from b04c7f6 to 8d22e4b Compare June 29, 2026 15:35

VianneyRuhlmann added the devflow:merge label Jun 30, 2026 — with Graphite App

gh-worker-dd-devflow-36fce6 Bot added the mergequeue-status: queued label Jun 30, 2026

gh-worker-dd-devflow-36fce6 Bot added mergequeue-status: in_progress and removed mergequeue-status: queued labels Jun 30, 2026

gh-worker-dd-mergequeue-cf854d Bot merged commit a38b630 into main Jun 30, 2026
100 checks passed

gh-worker-dd-devflow-36fce6 Bot added mergequeue-status: done and removed mergequeue-status: in_progress labels Jun 30, 2026

gh-worker-dd-mergequeue-cf854d Bot deleted the 06-24-feat_stats_add_whole-key_cardinality_limit branch June 30, 2026 14:12

gh-worker-devflow-routing-ef8351 Bot removed the devflow:merge label Jun 30, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(stats)!: add whole key cardinality limit#2158

feat(stats)!: add whole key cardinality limit#2158
gh-worker-dd-mergequeue-cf854d[bot] merged 3 commits into
mainfrom
06-24-feat_stats_add_whole-key_cardinality_limit

VianneyRuhlmann commented Jun 24, 2026 •

edited

Loading

Uh oh!

VianneyRuhlmann commented Jun 24, 2026 •

edited

Loading

Uh oh!

chatgpt-codex-connector Bot left a comment

Uh oh!

Uh oh!

dd-octo-sts Bot commented Jun 24, 2026 •

edited

Loading

Uh oh!

datadog-official Bot commented Jun 24, 2026 •

edited by datadog-prod-us1-3 Bot

Loading

Uh oh!

github-actions Bot commented Jun 24, 2026

Uh oh!

Uh oh!

Eldolfin left a comment

Uh oh!

gh-worker-devflow-routing-ef8351 Bot commented Jun 30, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

VianneyRuhlmann commented Jun 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Breaking change

Motivation

Additional Notes

Limit default value

Trilean change

Uh oh!

VianneyRuhlmann commented Jun 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

dd-octo-sts Bot commented Jun 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Artifact Size Benchmark Report

Uh oh!

datadog-official Bot commented Jun 24, 2026 • edited by datadog-prod-us1-3 Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 24, 2026

Clippy Allow Annotation Report

Summary by Rule

Annotation Counts by File

Annotation Stats by Crate

About This Report

Uh oh!

Uh oh!

Eldolfin left a comment

Choose a reason for hiding this comment

Uh oh!

gh-worker-devflow-routing-ef8351 Bot commented Jun 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

VianneyRuhlmann commented Jun 24, 2026 •

edited

Loading

VianneyRuhlmann commented Jun 24, 2026 •

edited

Loading

dd-octo-sts Bot commented Jun 24, 2026 •

edited

Loading

datadog-official Bot commented Jun 24, 2026 •

edited by datadog-prod-us1-3 Bot

Loading

gh-worker-devflow-routing-ef8351 Bot commented Jun 30, 2026 •

edited

Loading