Skip to content

fix(otel-thread-ctx): put the threadlocal attributes at the right place in the context#2167

Merged
gh-worker-dd-mergequeue-cf854d[bot] merged 4 commits into
mainfrom
yannham/fix-context-process-setup
Jun 30, 2026
Merged

fix(otel-thread-ctx): put the threadlocal attributes at the right place in the context#2167
gh-worker-dd-mergequeue-cf854d[bot] merged 4 commits into
mainfrom
yannham/fix-context-process-setup

Conversation

@yannham

@yannham yannham commented Jun 26, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

This PR stores the threadlocal* attributes in the attributes field of the process context, instead of the resource field.
Additionally, fix an unrelated issue (remove undue #[cfg(..)] gate) that was introduced in the otel thread feature-gating, preventing normal tracer metadata from being published in the context when the otel-thread-ctx feature is active.

Motivation

After experimenting more with OTel thread context sharing, I found out the way we set up the process context is incorrect. The threadlocal* attributes should go in the attributes/extra_attributes part of the process context, not the resource part. Tested that it now works with the context reader locally.

Additional Notes

N/A

How to test the change?

I'm using this version in a prototype integration in dd-trace-rs, using the context reader. This is how I uncovered the issue initially. Now I'm seeing context properly read with this fix.

@yannham yannham requested review from ivoanjo and scottgerring June 26, 2026 14:42
@yannham yannham requested a review from a team as a code owner June 26, 2026 14:42
@yannham yannham requested review from mabdinur and removed request for a team June 26, 2026 14:42
@yannham yannham force-pushed the yannham/fix-context-process-setup branch from d266efc to 591a444 Compare June 26, 2026 14:45
@github-actions

Copy link
Copy Markdown
Contributor

Clippy Allow Annotation Report

Comparing clippy allow annotations between branches:

  • Base Branch: origin/main
  • PR Branch: origin/yannham/fix-context-process-setup

Summary by Rule

Rule Base Branch PR Branch Change

Annotation Counts by File

File Base Branch PR Branch Change

Annotation Stats by Crate

Crate Base Branch PR Branch Change
clippy-annotation-reporter 5 5 No change (0%)
datadog-ffe-ffi 1 1 No change (0%)
datadog-ipc 22 22 No change (0%)
datadog-live-debugger 4 4 No change (0%)
datadog-live-debugger-ffi 10 10 No change (0%)
datadog-profiling-replayer 4 4 No change (0%)
datadog-sidecar 45 45 No change (0%)
libdd-common 13 13 No change (0%)
libdd-common-ffi 12 12 No change (0%)
libdd-data-pipeline 6 6 No change (0%)
libdd-ddsketch 2 2 No change (0%)
libdd-dogstatsd-client 1 1 No change (0%)
libdd-profiling 13 13 No change (0%)
libdd-remote-config 3 3 No change (0%)
libdd-telemetry 20 20 No change (0%)
libdd-tinybytes 4 4 No change (0%)
libdd-trace-normalization 2 2 No change (0%)
libdd-trace-obfuscation 3 3 No change (0%)
libdd-trace-stats 1 1 No change (0%)
libdd-trace-utils 11 11 No change (0%)
Total 182 182 No change (0%)

About This Report

This report tracks Clippy allow annotations for specific rules, showing how they've changed in this PR. Decreasing the number of these annotations generally improves code quality.

@yannham yannham changed the title fix: put the threadlocal attributes at the right place in the context fix(otel-thread-ctx): put the threadlocal attributes at the right place in the context Jun 26, 2026
@datadog-official

datadog-official Bot commented Jun 26, 2026

Copy link
Copy Markdown

Tests

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

🎯 Code Coverage (details)
Patch Coverage: 100.00%
Overall Coverage: 74.07% (+0.00%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 957c8f3 | Docs | Datadog PR Page | Give us feedback!

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d266efc1e2

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread libdd-library-config/src/tracer_metadata.rs
Comment thread libdd-library-config/src/tracer_metadata.rs
@dd-octo-sts

dd-octo-sts Bot commented Jun 26, 2026

Copy link
Copy Markdown
Contributor

Artifact Size Benchmark Report

aarch64-alpine-linux-musl
Artifact Baseline Commit Change
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.a 85.18 MB 85.17 MB --.01% (-10.16 KB) 💪
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.so 7.82 MB 7.82 MB 0% (0 B) 👌
aarch64-unknown-linux-gnu
Artifact Baseline Commit Change
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.so 10.51 MB 10.51 MB -0% (-272 B) 👌
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.a 96.32 MB 96.31 MB --.01% (-10.50 KB) 💪
libdatadog-x64-windows
Artifact Baseline Commit Change
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.dll 25.19 MB 25.19 MB 0% (0 B) 👌
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.lib 88.04 KB 88.04 KB 0% (0 B) 👌
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.pdb 183.52 MB 183.53 MB +0% (+8.00 KB) 👌
/libdatadog-x64-windows/debug/static/datadog_profiling_ffi.lib 939.74 MB 939.74 MB +0% (+44 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.dll 8.23 MB 8.23 MB 0% (0 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.lib 88.04 KB 88.04 KB 0% (0 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.pdb 24.33 MB 24.33 MB 0% (0 B) 👌
/libdatadog-x64-windows/release/static/datadog_profiling_ffi.lib 48.52 MB 48.52 MB 0% (0 B) 👌
libdatadog-x86-windows
Artifact Baseline Commit Change
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.dll 21.82 MB 21.82 MB 0% (0 B) 👌
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.lib 89.42 KB 89.42 KB 0% (0 B) 👌
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.pdb 187.54 MB 187.55 MB +0% (+8.00 KB) 👌
/libdatadog-x86-windows/debug/static/datadog_profiling_ffi.lib 926.89 MB 926.89 MB +0% (+4 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.dll 6.35 MB 6.35 MB 0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.lib 89.42 KB 89.42 KB 0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.pdb 26.12 MB 26.12 MB 0% (0 B) 👌
/libdatadog-x86-windows/release/static/datadog_profiling_ffi.lib 46.15 MB 46.15 MB 0% (0 B) 👌
x86_64-alpine-linux-musl
Artifact Baseline Commit Change
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.a 75.92 MB 75.91 MB --.01% (-10.58 KB) 💪
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.so 8.70 MB 8.70 MB 0% (0 B) 👌
x86_64-unknown-linux-gnu
Artifact Baseline Commit Change
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.a 91.39 MB 91.38 MB --.01% (-10.53 KB) 💪
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.so 10.59 MB 10.59 MB -0% (-224 B) 👌

@ivoanjo ivoanjo left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 LGTM. CI is unhappy, but other than that + some very small suggestions, this looks fine! I completely missed that we were putting these attributes in the wrong place too 😅

Comment thread libdd-library-config/src/tracer_metadata.rs Outdated
Comment thread libdd-library-config/src/tracer_metadata.rs
@yannham yannham force-pushed the yannham/fix-context-process-setup branch from 71a5fff to 9dccfc5 Compare June 30, 2026 13:54
@yannham

yannham commented Jun 30, 2026

Copy link
Copy Markdown
Contributor Author

/merge

@gh-worker-devflow-routing-ef8351

gh-worker-devflow-routing-ef8351 Bot commented Jun 30, 2026

Copy link
Copy Markdown

View all feedbacks in Devflow UI.

2026-06-30 13:55:21 UTC ℹ️ Start processing command /merge


2026-06-30 13:55:30 UTC ℹ️ MergeQueue: waiting for PR to be ready

This pull request is not mergeable according to GitHub. Common reasons include pending required checks, missing approvals, or merge conflicts — but it could also be blocked by other repository rules or settings.
It will be added to the queue as soon as checks pass and/or get approvals. View in MergeQueue UI.
Note: if you pushed new commits since the last approval, you may need additional approval.
You can remove it from the waiting list with /remove command.


2026-06-30 14:58:08 UTC ℹ️ MergeQueue: merge request added to the queue

The expected merge time in main is approximately 1h (p90).


2026-06-30 16:58:44 UTCMergeQueue: The build pipeline has timeout

The merge request has been interrupted because the build 682695468372423724 took longer than expected. The current limit for the base branch 'main' is 120 minutes.

@morrisonlevi

Copy link
Copy Markdown
Contributor

The merge queue timed out. I am not skilled in investigating MQ failures, and I was not able to figure out why. I merged in main in hopes that it will work this time.

@bwoebi

bwoebi commented Jun 30, 2026

Copy link
Copy Markdown
Contributor

/merge

@gh-worker-devflow-routing-ef8351

gh-worker-devflow-routing-ef8351 Bot commented Jun 30, 2026

Copy link
Copy Markdown

View all feedbacks in Devflow UI.

2026-06-30 18:14:21 UTC ℹ️ Start processing command /merge


2026-06-30 18:14:25 UTC ℹ️ MergeQueue: pull request added to the queue

The expected merge time in main is approximately 1h (p90).


2026-06-30 19:18:13 UTC ℹ️ MergeQueue: This merge request was merged

@gh-worker-dd-mergequeue-cf854d gh-worker-dd-mergequeue-cf854d Bot merged commit 3630553 into main Jun 30, 2026
99 checks passed
@gh-worker-dd-mergequeue-cf854d gh-worker-dd-mergequeue-cf854d Bot deleted the yannham/fix-context-process-setup branch June 30, 2026 19:18
szegedi added a commit that referenced this pull request Jul 1, 2026
#2167 moved the threadlocal.* attributes from Resource.attributes into
ProcessContext.extra_attributes and renamed the test helper from
find_attr to find_extra_attr. Our tests, added before that fix landed,
still used the old name and lookup surface. Rename to match.
iunanua added a commit that referenced this pull request Jul 7, 2026
…ibdd-data-pipeline, libdd-li... (#2201)

# Release proposal for libdd-capabilities-impl, libdd-common,
libdd-data-pipeline, libdd-library-config, libdd-remote-config,
libdd-sampling, libdd-telemetry, libdd-tinybytes, libdd-trace-utils and
their dependencies

This PR contains version bumps based on public API changes and commits
since last release.

## libdd-capabilities
**Next version:** `2.1.0`
**Semver bump:** `minor`
**Tag:** `libdd-capabilities-v2.1.0`

### Commits

- feat(data-pipeline)!: add stdout log trace exporter (#2074)

## libdd-common
**Next version:** `5.1.0`
**Semver bump:** `minor`
**Tag:** `libdd-common-v5.1.0`

### Commits

- refactor(clippy): prefer core and alloc imports (#2196)
- fix: update rustls-webpki to 0.103.13 (#2187)
- fix: update anyhow for unsoundness (#2186)
- feat(machine id): Add helpers in ddcommon to fetch the machine UUID l…
(#2163)

## libdd-ddsketch
**Next version:** `1.1.0`
**Semver bump:** `minor`
**Tag:** `libdd-ddsketch-v1.1.0`

### Commits

- feat(data-pipeline)!: export client-computed span stats as OTLP trace
metrics (#2067)
- test(ddsketch): add microbenchmarks for add/encode/collapse (#2125)

## libdd-trace-protobuf
**Next version:** `4.0.0`
**Semver bump:** `major`
**Tag:** `libdd-trace-protobuf-v4.0.0`

### Commits

- chore!: update protobufs to be in sync with datadog-agent (#2180)
- feat(stats)!: add whole key cardinality limit (#2158)
- feat(remote-config)!: use the proto file from the agent (#2165)
- feat(data-pipeline): OTLP HTTP/protobuf trace export (#2115)

## libdd-capabilities-impl
**Next version:** `3.0.0`
**Semver bump:** `major`
**Tag:** `libdd-capabilities-impl-v3.0.0`

### ⚠️ major bump forced due to:

- `libdd-common`: ^4.1.0 → ^5.1.0

### Commits

- feat(data-pipeline)!: add stdout log trace exporter (#2074)

## libdd-library-config
**Next version:** `3.0.0`
**Semver bump:** `major`
**Tag:** `libdd-library-config-v3.0.0`

### ⚠️ major bump forced due to:

- `libdd-trace-protobuf`: ^3.0.2 → ^4.0.0

### Commits

- refactor(clippy): prefer core and alloc imports (#2196)
- feat(library-config)!: caller-supplied threadlocal schema and extra
process-context attributes (#2162)
- fix(otel-thread-ctx): put the threadlocal attributes at the right
place in the context (#2167)

## libdd-remote-config
**Next version:** `2.0.0`
**Semver bump:** `major`
**Tag:** `libdd-remote-config-v2.0.0`

### ⚠️ major bump forced due to:

- `libdd-trace-protobuf`: ^3.0.2 → ^4.0.0

### Commits

- refactor(libdd-remote-config)!: hide Target inner properties so they
are not leaked (#2182)
- feat(remote-config)!: use the proto file from the agent (#2165)
- refactor(rc): reexport Endpoint and Tag common types (#2147)

## libdd-trace-normalization
**Next version:** `3.0.0`
**Semver bump:** `major`
**Tag:** `libdd-trace-normalization-v3.0.0`

### ⚠️ major bump forced due to:

- `libdd-trace-protobuf`: ^3.0.1 → ^4.0.0

### Commits

- feat(data-pipeline)!: CSS Trace Filters (#1985)

## libdd-shared-runtime
**Next version:** `2.0.0`
**Semver bump:** `major`
**Tag:** `libdd-shared-runtime-v2.0.0`

### ⚠️ major bump forced due to:

- `libdd-common`: ^4.1.0 → ^5.1.0

### Commits

- feat(shared-runtime)!: SharedRuntime Borrowed & Owned mode (#2061)
- feat(shared-runtime)!: use weak waker in trigger [APMSP-3371] (#2050)

## libdd-trace-utils
**Next version:** `9.0.0`
**Semver bump:** `major`
**Tag:** `libdd-trace-utils-v9.0.0`

### ⚠️ major bump forced due to:

- `libdd-common`: ^4.2.0 → ^5.1.0
- `libdd-trace-protobuf`: ^3.0.2 → ^4.0.0

### Commits

- ci(miri): skip slow miri tests (#2188)
- chore!: update protobufs to be in sync with datadog-agent (#2180)
- feat(data-pipeline): add agentless export (#2081)
- feat(data-pipeline)!: add stdout log trace exporter (#2074)
- feat(data-pipeline): OTLP HTTP/protobuf trace export (#2115)
- feat(otlp)!: Export OTLP spans with attribute-level OTel compatibility
(#2091)
- test(trace-utils): add V05 msgpack decode microbenchmark (#2127)
- feat(data-pipeline)!: export client-computed span stats as OTLP trace
metrics (#2067)
- test(trace-utils): add VecMap microbenchmarks (#2126)
- chore(stats)!: submit p0 telemetry in stats (#2130)
- refactor(change-buffer)!: replace slot index with span_id, fix segment
isolation (#2105)
- feat(data-pipeline)!: CSS Trace Filters (#1985)
- feat(trace-exporter): add v1 span and its encoder (#2039)
- fix(trace-utils): mark decoded span maps as deduped (#2110)
- feat(trace-utils)!: change buffer implementation (#2055)
- feat(native-spans)!: change buffer foundation (#2046)
- refactor(span)!: use VecMap for `meta`, `metrics` and `meta_struct`
for v04 spans (#2043)
- test: fix timeouts on heavily contended scenarios (#2093)

## libdd-telemetry
**Next version:** `6.0.0`
**Semver bump:** `major`
**Tag:** `libdd-telemetry-v6.0.0`

### ⚠️ major bump forced due to:

- `libdd-common`: ^4.2.0 → ^5.1.0
- `libdd-shared-runtime`: ^1.0.0 → ^2.0.0

### Commits

- ci(miri): skip slow miri tests (#2188)
- refactor(libdd-telemetry)!: avoid leaking libdd-common types in the
public API (#2152)
- feat(shared-runtime)!: SharedRuntime Borrowed & Owned mode (#2061)

## libdd-trace-obfuscation
**Next version:** `5.0.0`
**Semver bump:** `major`
**Tag:** `libdd-trace-obfuscation-v5.0.0`

### ⚠️ major bump forced due to:

- `libdd-common`: ^4.2.0 → ^5.1.0
- `libdd-trace-protobuf`: ^3.0.2 → ^4.0.0
- `libdd-trace-utils`: ^8.0.0 → ^9.0.0

### Commits

- refactor(clippy): prefer core and alloc imports (#2196)
- ci(miri): skip slow miri tests (#2188)
- fix: update anyhow for unsoundness (#2186)

## libdd-trace-stats
**Next version:** `6.0.0`
**Semver bump:** `major`
**Tag:** `libdd-trace-stats-v6.0.0`

### ⚠️ major bump forced due to:

- `libdd-common`: ^4.2.0 → ^5.1.0
- `libdd-shared-runtime`: ^1.0.0 → ^2.0.0
- `libdd-trace-protobuf`: ^3.0.2 → ^4.0.0
- `libdd-trace-utils`: ^8.0.0 → ^9.0.0

### Commits

- chore!: update protobufs to be in sync with datadog-agent (#2180)
- feat(stats)!: send telemetry for cardinality limits (#2159)
- feat(stats)!: add whole key cardinality limit (#2158)
- fix(trace-stats)!: add grpc_method to aggregation key (#2151)
- feat(shared-runtime)!: SharedRuntime Borrowed & Owned mode (#2061)
- feat(data-pipeline)!: export client-computed span stats as OTLP trace
metrics (#2067)
- refactor(span)!: use VecMap for `meta`, `metrics` and `meta_struct`
for v04 spans (#2043)

## libdd-data-pipeline
**Next version:** `7.0.0`
**Semver bump:** `major`
**Tag:** `libdd-data-pipeline-v7.0.0`

### ⚠️ major bump forced due to:

- `libdd-common`: ^4.2.0 → ^5.1.0
- `libdd-shared-runtime`: ^1.0.0 → ^2.0.0
- `libdd-telemetry`: ^5.0.1 → ^6.0.0
- `libdd-trace-protobuf`: ^3.0.2 → ^4.0.0
- `libdd-trace-stats`: ^5.0.0 → ^6.0.0
- `libdd-trace-utils`: ^8.0.0 → ^9.0.0

### Commits

- feat(trace_exporter): enable telemetry in stats exporter (#2160)
- refactor(libdd-telemetry)!: avoid leaking libdd-common types in the
public API (#2152)
- feat(stats): emit canonical gRPC status name for OTLP
rpc.response.status_code (#2183)
- feat(data-pipeline): add agentless export (#2081)
- feat(stats)!: send telemetry for cardinality limits (#2159)
- feat(stats)!: add whole key cardinality limit (#2158)
- fix(trace-stats)!: add grpc_method to aggregation key (#2151)
- feat(data-pipeline)!: add stdout log trace exporter (#2074)
- feat(shared-runtime)!: SharedRuntime Borrowed & Owned mode (#2061)
- feat(data-pipeline): OTLP HTTP/protobuf trace export (#2115)
- feat(otlp)!: Export OTLP spans with attribute-level OTel compatibility
(#2091)
- feat(data-pipeline)!: export client-computed span stats as OTLP trace
metrics (#2067)
- chore(stats)!: submit p0 telemetry in stats (#2130)
- feat(data-pipeline)!: CSS Trace Filters (#1985)
- feat(shared-runtime)!: use weak waker in trigger [APMSP-3371] (#2050)
- refactor(span)!: use VecMap for `meta`, `metrics` and `meta_struct`
for v04 spans (#2043)
- feat(stats)!: add endpoint gating to client-side stats [APMSP-3361]
(#2040)

## libdd-dogstatsd-client
**Next version:** `4.0.0`
**Semver bump:** `major`
**Tag:** `libdd-dogstatsd-client-v4.0.0`

### ⚠️ major bump forced due to:

- `libdd-common`: ^4.1.0 → ^5.1.0


## libdd-sampling
**Next version:** `5.0.0`
**Semver bump:** `major`
**Tag:** `libdd-sampling-v5.0.0`

### ⚠️ major bump forced due to:

- `libdd-common`: ^4.2.0 → ^5.1.0
- `libdd-trace-utils`: ^8.0.0 → ^9.0.0


[APMSP-3371]:
https://datadoghq.atlassian.net/browse/APMSP-3371?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: iunanua <18325288+iunanua@users.noreply.github.com>
taegyunkim pushed a commit that referenced this pull request Jul 8, 2026
…ce in the context (#2167)

# What does this PR do?

This PR stores the `threadlocal*` attributes in the `attributes` field of the process context, instead of the `resource` field.
Additionally, fix an unrelated issue (remove undue `#[cfg(..)]` gate) that was introduced in the [otel thread feature-gating](#1843), preventing normal tracer metadata from being published in the context when the `otel-thread-ctx` feature is active.

# Motivation

After experimenting more with OTel thread context sharing, I found out the way we set up the process context is incorrect. The `threadlocal*` attributes should go in the `attributes`/`extra_attributes` part of the process context, not the `resource` part. Tested that it now works with the context reader locally.

# Additional Notes

N/A

# How to test the change?

I'm using this version in a prototype integration in dd-trace-rs, using the [context reader](https://github.com/scottgerring/ctx-sharing-demo/tree/main/context-reader). This is how I uncovered the issue initially. Now I'm seeing context properly read with this fix.

Co-authored-by: morrisonlevi <levi.morrison@datadoghq.com>
Co-authored-by: bob.weinand <bob.weinand@datadoghq.com>
Signed-off-by: Taegyun Kim <taegyun.kim@datadoghq.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants