feat(heap-profiling): add initial heap profiling primitives [PROF-15190] by scottgerring · Pull Request #2166 · DataDog/libdatadog

scottgerring · 2026-06-26T13:19:14Z

What does this PR do?

Adds initial support for eBPF heap profiling to libdatadog for the ebpf heap profiling project. These changes provide the ability for libdatadog users to introduce a USDT into consuming applications allocation and deallocation paths which is fired behind a poisson distributed sampling path every ~0.5 MiB of allocation. This will give us something to hook in user applications from the eBPF side to sample allocations for allocation tracking and sample allocations+frees for live heap.

I expect we will make more changes to this as we go along, but as this is a standalone feature and is largely based on prior work in ddprof it would be nice to get things in now.

A brief description of the change being made with this pull request.

Interesting Details

I have vendored usdt.h; it is not consistently available on all our build environments (read: old alpine) and this seems to be the common pattern. The API is stable and I see no concerns with this and have updated the license stuff appropriately
I have gated the binding generation behind an env var and committed the bindings. This is because the generation imposes a fair bit of extra stuff (llvm and supporting bits) on every build job and developer. I've added a CI job to validate that the bindings generated reflect what's checked in to avoid the footgun.

Motivation

An enthusiasm for heap profiling

Additional Notes

Anything else we should know when reviewing?

PoC-y branch for the full host profiler with heap profiler
The profiling backend + frontend are done; you can see heap profiles pushed through using this infrastructure in staging

How to test the change?

Describe here in detail how the change can be validated.

datadog-prod-us1-4 · 2026-06-26T13:19:20Z

Tests

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

🎯 Code Coverage (details)
• Patch Coverage: 84.99%
• Overall Coverage: 74.44% (+0.09%)

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: aeb870c | Docs | Datadog PR Page | Give us feedback!}

github-actions · 2026-06-26T13:22:21Z

Clippy Allow Annotation Report

Comparing clippy allow annotations between branches:

Base Branch: origin/main
PR Branch: origin/sgg/heap-prof-poc

Summary by Rule

Rule	Base Branch	PR Branch	Change

Annotation Counts by File

File	Base Branch	PR Branch	Change

Annotation Stats by Crate

Crate	Base Branch	PR Branch	Change
`clippy-annotation-reporter`	5	5	No change (0%)
`datadog-ffe-ffi`	1	1	No change (0%)
`datadog-ipc`	22	22	No change (0%)
`datadog-live-debugger`	4	4	No change (0%)
`datadog-live-debugger-ffi`	10	10	No change (0%)
`datadog-profiling-replayer`	4	4	No change (0%)
`datadog-sidecar`	45	45	No change (0%)
`libdd-common`	13	13	No change (0%)
`libdd-common-ffi`	12	12	No change (0%)
`libdd-data-pipeline`	6	6	No change (0%)
`libdd-ddsketch`	2	2	No change (0%)
`libdd-dogstatsd-client`	1	1	No change (0%)
`libdd-profiling`	13	13	No change (0%)
`libdd-remote-config`	3	3	No change (0%)
`libdd-telemetry`	20	20	No change (0%)
`libdd-tinybytes`	4	4	No change (0%)
`libdd-trace-normalization`	2	2	No change (0%)
`libdd-trace-obfuscation`	3	3	No change (0%)
`libdd-trace-stats`	1	1	No change (0%)
`libdd-trace-utils`	11	11	No change (0%)
Total	182	182	No change (0%)

About This Report

This report tracks Clippy allow annotations for specific rules, showing how they've changed in this PR. Decreasing the number of these annotations generally improves code quality.

yannham

Partial review of libdd-heap-gotter

…lback

r1viollet

LGTM
The doc is great, tests are great, overhead looks good, architecture has been discussed in advance. I can't possibly audit all lines, though AI reviews have already done a good job.
I know this is a big one, and I expect we will be iterating on this code in following weeks.
OK to merge this first version

scottgerring · 2026-07-08T12:46:48Z

/merge

gh-worker-devflow-routing-ef8351 · 2026-07-08T12:46:52Z

View all feedbacks in Devflow UI.

2026-07-08 12:46:52 UTC ℹ️ Start processing command /merge

2026-07-08 12:46:57 UTC ℹ️ MergeQueue: pull request added to the queue

The expected merge time in main is approximately 1h (p90).

2026-07-08 13:26:59 UTC ℹ️ MergeQueue: This merge request was merged

danielsn

Was half way through a review, here is what I had so far

…90] (#2166) # What does this PR do? Adds initial support for eBPF heap profiling to libdatadog for the [ebpf heap profiling project](https://docs.google.com/document/d/1cy6OUisjW4_vAIaAu9G5l3RGuPPLT_MRZuhDPJHxXuw/edit?tab=t.mtwnx5ijdb84). These changes provide the ability for libdatadog users to introduce a USDT into consuming applications allocation and deallocation paths which is fired behind a poisson distributed sampling path every ~0.5 MiB of allocation. This will give us something to hook in user applications from the eBPF side to sample allocations for allocation tracking and sample allocations+frees for live heap. I expect we will make more changes to this as we go along, but as this is a standalone feature and is largely based on prior work in ddprof it would be nice to get things in now. A brief description of the change being made with this pull request. ## Interesting Details * I have vendored `usdt.h`; it is not consistently available on all our build environments (read: old alpine) and this seems to be the common pattern. The API is stable and I see no concerns with this and have updated the license stuff appropriately * I have gated the binding generation behind an env var and committed the bindings. This is because the generation imposes a fair bit of extra _stuff_ (llvm and supporting bits) on _every_ build job and developer. I've added a CI job to validate that the bindings generated reflect what's checked in to avoid the footgun. # Motivation An enthusiasm for heap profiling # Additional Notes Anything else we should know when reviewing? * [PoC-y branch for the full host profiler with heap profiler](https://github.com/DataDog/opentelemetry-ebpf-profiler/tree/sgg/heap-prof-poc) * The profiling backend + frontend are done; you can see heap profiles pushed through using this infrastructure in [staging](https://dd.datad0g.com/profiling/explorer?query=service%3A%2Ausdt%2A&agg_m=%40prof_ebpf_cpu_cores&agg_m_source=base&agg_t=sum&extra_search_fields=%7B%22filters_query%22%3A%22%22%2C%22sample_type%22%3A%22cpu-time%22%7D&my_code=enabled&profile_type=ebpf-alloc-size&profiling-flame-graph__expanded_groups=3269946435%2C4099416842&refresh_mode=paused&viz=flame_graph&zoom=2156769548&from_ts=1782885547816&to_ts=1782889147816&live=false) # How to test the change? Describe here in detail how the change can be validated. Co-authored-by: scott.gerring <scott.gerring@datadoghq.com> Signed-off-by: Taegyun Kim <taegyun.kim@datadoghq.com>

danielsn

getting there ...

danielsn · 2026-07-08T19:35:30Z

+
+    void *header = (char *)user - DD_HEADER_BYTES;
+    uint64_t magic;
+    memcpy(&magic, header, sizeof(magic));


why do we need the memcpy vs just looking directly at the bytes?

It's the UB-free way to read a typed value out of raw bytes. Casting header to uint64_t * and dereferencing would assume header is correctly aligned, and I believe also runs afoul of C's strict aliasing rules. memcpy sidesteps both, and for a fixed 8-byte copy I think it's usually lowered to a plain load, so there shouldn't be any meaningful runtime cost.

danielsn · 2026-07-08T19:36:22Z

+/*
+ * arm64: TBI (Top Byte Ignore) pointer tag.
+ *
+ * ARMv8 userspace ignores bits 56..63 of a virtual address, so we stash


what happens if someone else wanted to use the tags too?

We turn ourselves off. The hope is that this doesn't happen much (memory tagging not being useful on amd64-linux because the kernel actively removed support for it hopefully helps folk to not want to use it).

Again here we might need some mechanism for logging so we test this hypothesis. We've talked about having a diagnostic uprobe we can fire - so you'd say otel_heap_diagnostic(0x123) with some magic number that lets us communicate particular states back to the profiler. This frees us of having to log within the user's apps, but still lets us aggregate signal on error cases.

WDYT?

danielsn · 2026-07-08T19:37:11Z

+        return false;
+    }
+
+    uint64_t offset;


have we explored the tradeoff between keeping an offset for every alloc, vs having a hashtable for when magic matches?

We talked about it a bit - there's a writeup over here.

Reasons we like this one:

It's reasonably efficient - we're burning something a bit more than 16 bytes (cuz of wild alignments) per sampled allocation (so on avg. every 512 KiB), which gets us to around 32 KiB per GiB allocated

We have good locality

We don't have to think about how and if to statically size the table and what happens when it is full

We don't have the cost of indirection through the hash table (although this is less of a big deal as this is only on the cold path through free once we know we are sampled)

danielsn · 2026-07-09T21:23:37Z

+When we get a realloc, we treat it as a separate free and a new allocation
+to keep things simple. This means a resized allocation that was sampled
+before the resize is not sampled after it: we report the free and just let
+the new block go untracked, rather than trying to carry the sample


why not check the threshold as if it was a new allocation?

We could, but two issues. First the mechanical one the doc mentions: sampling the new block needs a fresh header stamped with the original user-requested size, which we don't store. The bigger one is byte accounting, the sampler is Poisson-over-bytes and dd_allocation_requested(size) decrements the per-thread counter by size. A realloc's genuinely-new memory is only the growth delta; the carried-forward bytes were already counted at the original allocation. Running the full threshold on new_size would double-count them and bias the size estimator, which is why the unsampled realloc path stays a pure passthrough that never touches the counter. Given that we suspect that realloc-of-sampled-block is rare, we've deferred trying to handle this for now.

danielsn · 2026-07-09T21:27:11Z

+is done through `prctl(PR_SET_TAGGED_ADDR_CTRL)` in
+`dd_sample_flag_thread_init`, which must run once per thread before any
+tagging happens on that thread. If the kernel refuses (older kernels,
+seccomp policies, etc.) we disable sampling entirely on that thread. We might


In what case would it succeed for one thread and not another?

In practice it usually wouldn't. All I can think of for realistic per-thread divergence is a thread that installed its own restrictive seccomp filter blocking prctl or something wildly inappropriate happening with eBPF. So the per-thread handling is really just conservative, we do it per thread because the ABI control is per-thread, not because we expect the result to differ.

danielsn · 2026-07-09T21:28:58Z

+The header holds two 8-byte values:
+
+* a magic constant (`DD_MAGIC`)
+* the offset from the user pointer back to the raw pointer the allocator


why is this needed, isn't the header fixed sized?

The header is fixed-size (16 bytes, always right before the user pointer, which is how we locate it), but the distance from the user pointer back to the raw allocator pointer isn't fixed. For alignment > 16, or when we bump the user pointer forward to avoid landing within 16 bytes of a page boundary (see the Page Alignment section below), raw ends up further back than 16 bytes. The offset records that variable distance so free can recover the real base to hand to the allocator. Without it we'd only be able to handle the alignment <= 16, no-page-bump case.

danielsn · 2026-07-09T21:33:33Z

+(`x86_apply` in `sample_flag.h`), and when we work out how big the original
+allocation was so we can free the right amount (`dd_allocation_freed_slow`
+in `allocation_freed.c`). If any of these disagree we corrupt memory. Touch
+one, check the other two.


can't it be a shared function in the header?

makes sense; i'll see if i can refactor this nicely in the follow-up PR

danielsn · 2026-07-10T16:43:52Z

+    /* Always close the reentry guard, even on allocation failure (raw == NULL),
+     * so the thread isn't permanently locked out of sampling. */
+    dd_tl_state_t *tl = dd_tl_state_get();
+    if (tl) tl->reentry_guard = false;


under what conditions would that happen? Should we assert in debug cases? Should we assert the reentry_guard was true before?

Good idea. To answer the "under what conditions": on a correctly-paired call, never, we only reach the slow path on a sampled request (weight > 0), which means dd_allocation_requested_slow already initialised the thread's TLS and set the guard. So tl non-null and reentry_guard == true both hold; a failure means created was called without a matching requested. Added debug asserts for both.

danielsn · 2026-07-10T16:47:53Z

+
+#if defined(__x86_64__)
+    /* Recover the bumped size the allocator actually holds. This must
+     * exactly mirror allocation_requested.c's bumped_alloc_size():


should this be a shared inline function then?

Good point, same idea as the other thread, I'll pull the size formula into a shared inline in the follow-up PR.

danielsn · 2026-07-10T16:49:34Z

+    };
+
+    if (old_user == NULL) {
+        dd_alloc_req_t req = dd_allocation_requested(


it would be clearer to just create and return the properly initilized out in each branch rather than making it mutable

good call, agreed

danielsn · 2026-07-10T16:50:25Z

+        void *hdr = (char *)old_user - DD_HEADER_BYTES;
+        uint64_t m = DD_MAGIC;
+        uint64_t o = (uint64_t)old_offset;
+        memcpy(hdr, &m, sizeof(m));


Would it be clearer to have a header struct, and copy the thing rather than two seperate fields?

good call, agreed

@danielsn

…2226) # What does this PR do? Clarifies a bunch of stuff that was not obvious, and improves testing, addressing feedback from @danielsn on #2166 # Motivation What inspired you to submit this pull request? # Additional Notes Anything else we should know when reviewing? # How to test the change? Describe here in detail how the change can be validated. Co-authored-by: scott.gerring <scott.gerring@datadoghq.com>

scottgerring force-pushed the sgg/heap-prof-poc branch from b6aab83 to 72d9f77 Compare June 26, 2026 13:19

scottgerring force-pushed the sgg/heap-prof-poc branch 3 times, most recently from b80ec4a to 11cf066 Compare June 26, 2026 14:37

scottgerring changed the title ~~[PROF-15190]: DRAFT - add initial heap profiling crates~~ feat(heap-profiling): add initial heap profiling primitives [PROF-15190] Jun 29, 2026

scottgerring force-pushed the sgg/heap-prof-poc branch from 11cf066 to f40cfdf Compare June 29, 2026 09:53

github-actions Bot added the ci-build label Jun 29, 2026

scottgerring force-pushed the sgg/heap-prof-poc branch 21 times, most recently from 12cd59a to fb19cfa Compare July 1, 2026 07:24

yannham reviewed Jul 3, 2026

View reviewed changes

scottgerring added 4 commits July 7, 2026 07:56

more review comments & a correctness fix

eb8214d

add env flag bypass

244e207

add feature flag to _disable_ live heap

132b6ca

address yann's review comments

743576b

r1viollet reviewed Jul 7, 2026

View reviewed changes

Comment thread libdd-heap-sampler/src/allocation_realloc.c

scottgerring force-pushed the sgg/heap-prof-poc branch from bebaa38 to 743576b Compare July 7, 2026 10:32

scottgerring added 3 commits July 7, 2026 13:24

fix windows lint

797be40

remove the ability to remove GOT hooks

0b8d0e2

fix: hand realloc the raw pointer in the sampled-realloc overflow fal…

37c4b52

…lback

scottgerring force-pushed the sgg/heap-prof-poc branch from 0bc2edd to 37c4b52 Compare July 7, 2026 15:29

scottgerring and others added 2 commits July 8, 2026 08:05

Merge branch 'main' into sgg/heap-prof-poc

48a1c78

address benchmark issue

aeb870c

r1viollet reviewed Jul 8, 2026

View reviewed changes

Comment thread libdd-heap-sampler/docs/tagging.md

r1viollet approved these changes Jul 8, 2026

View reviewed changes

gh-worker-dd-devflow-36fce6 Bot added mergequeue-status: queued mergequeue-status: in_progress and removed mergequeue-status: queued labels Jul 8, 2026

gh-worker-dd-mergequeue-cf854d Bot merged commit 65d8d24 into main Jul 8, 2026
92 checks passed

gh-worker-dd-mergequeue-cf854d Bot deleted the sgg/heap-prof-poc branch July 8, 2026 13:26

gh-worker-dd-devflow-36fce6 Bot added mergequeue-status: done and removed mergequeue-status: in_progress labels Jul 8, 2026

danielsn reviewed Jul 8, 2026

View reviewed changes

danielsn reviewed Jul 10, 2026

View reviewed changes

scottgerring mentioned this pull request Jul 13, 2026

chore(heap-profiling): address heap-profiling feedback [PROF-15190] #2226

Merged

Uh oh!

Conversation

scottgerring commented Jun 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Interesting Details

Motivation

Additional Notes

How to test the change?

Uh oh!

datadog-prod-us1-4 Bot commented Jun 26, 2026 • edited by datadog-datadog-prod-us1-2 Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 26, 2026

Clippy Allow Annotation Report

Summary by Rule

Annotation Counts by File

Annotation Stats by Crate

About This Report

Uh oh!

yannham left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

r1viollet left a comment

Choose a reason for hiding this comment

Uh oh!

scottgerring commented Jul 8, 2026

Uh oh!

gh-worker-devflow-routing-ef8351 Bot commented Jul 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

danielsn left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

danielsn left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

scottgerring commented Jun 26, 2026 •

edited

Loading

datadog-prod-us1-4 Bot commented Jun 26, 2026 •

edited by datadog-datadog-prod-us1-2 Bot

Loading

gh-worker-devflow-routing-ef8351 Bot commented Jul 8, 2026 •

edited

Loading