Skip to content

docs(runbook): eBPF profiling — seccomp gap + node-isolation correction#486

Merged
bdchatham merged 3 commits into
mainfrom
docs/con368-ebpf-runbook-seccomp-dedicated-node
Jul 14, 2026
Merged

docs(runbook): eBPF profiling — seccomp gap + node-isolation correction#486
bdchatham merged 3 commits into
mainfrom
docs/con368-ebpf-runbook-seccomp-dedicated-node

Conversation

@bdchatham

Copy link
Copy Markdown
Collaborator

Two additions to §6 of the gas-vs-execution-time benchmarking runbook, surfaced by the CON-368 follow-on profiling review with security-specialist + platform-engineer. Both live in the gap between the runbook's (correct) advice to minimize the capability grant and what actually makes the minimized grant work.

Step 3 — seccomp (the one that bites). The runbook already says to prefer CAP_BPF+CAP_PERFMON over --profile=sysadmin. But narrowing the caps isn't sufficient: the container's RuntimeDefault seccomp profile blocks bpf() and perf_event_open() behind CAP_SYS_ADMIN, so a pod with only the two caps and default seccomp EPERMs on exactly the syscalls bpftrace/BCC need. --profile=sysadmin works because it also sets seccompProfile: Unconfined; the moment you drop to the two caps you must set that (or a curated allowlist) explicitly. Added so the next person doesn't read the silent failure as a missing-capability bug.

Step 2 — node isolation. The prior text claimed the sei-node/sei-archive pool is 'already tainted/dedicated'. That taint only repels non-Sei pods — other Sei-managed pods and the node's own DaemonSets (eks-pod-identity-agent, cilium-agent) still co-reside, and the pod-identity agent's cached AWS creds are kernel-readable from a CAP_PERFMON pod. Corrected to name the real single-tenant primitive (the sei.io/dedicated-node annotation → hostname anti-affinity) and the two accepted residuals (IMDS→node-IAM-role, co-resident DaemonSets), pointing at PLT-801 for the closed version.

No procedure changed — this sharpens the privileged-attach gate so a scoped-down attach actually runs. Docs-only.

🤖 Generated with Claude Code

…tion claim

Two additions to §6 of the gas-vs-execution-time runbook, from the CON-368
follow-on profiling review:

- Step 3: narrowing from --profile=sysadmin to CAP_BPF+CAP_PERFMON is not
  sufficient alone — RuntimeDefault seccomp blocks bpf()/perf_event_open()
  behind CAP_SYS_ADMIN, so the narrowed pod needs seccompProfile: Unconfined
  (or a curated allowlist) or it silently EPERMs. This bit is easy to
  misread as a missing-capability bug.
- Step 2: the sei-node/sei-archive taint is not single-tenancy (repels
  non-Sei pods only; pod-identity DaemonSet + other Sei pods co-reside and
  are kernel-readable). Name the real primitive (sei.io/dedicated-node
  annotation) and the two accepted residuals (IMDS/node-IAM-role, co-resident
  DaemonSets), pointing at PLT-801 for the closed version.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@cursor

cursor Bot commented Jul 14, 2026

Copy link
Copy Markdown

PR Summary

Low Risk
Documentation-only changes to an internal runbook; no code, config, or deployment behavior is modified.

Overview
§6 (privileged attach gate) in the gas-vs-execution-time runbook is expanded so minimized CAP_BPF/CAP_PERFMON attaches and “dedicated node” pinning match harbor reality.

Step 2 replaces the idea that sei-node/sei-archive taints mean single-tenancy with what isolation actually buys: node-global CAP_PERFMON, co-resident DaemonSets, and the sei.io/dedicated-node annotation as the Sei-pod isolation knob. It adds explicit residual risks (node IAM via pod-identity kernel reads / optional hostNetwork+IMDS) and says those must be raised at sign-off, not self-accepted—escalate to PLT-801 if unacceptable.

Step 3 documents prerequisites when dropping --profile=sysadmin: CAP_SYS_PTRACE for seid uprobes via /proc/.../root, and seccomp (RuntimeDefault blocking bpf()/perf_event_open() unless seccompProfile: Unconfined or a curated profile), plus harbor-specific caveats so silent success isn’t misread as a cap bug. It also notes ≥5.11 memcg BPF memory accounting vs RLIMIT_MEMLOCK.

No runtime or procedure changes—documentation only.

Reviewed by Cursor Bugbot for commit a46b546. Bugbot is set up for automated code reviews on this repo. Configure here.

bdchatham and others added 2 commits July 14, 2026 10:54
…se, IMDS residual

Three independent xreview lenses (systems-engineer, security-specialist,
prose-steward) dissented on the first draft. Fixes:

- CAP_SYS_PTRACE: a uprobe on non-root seid via /proc/<pid>/root is
  ptrace_may_access-gated, so the two-cap grant is NOT sufficient for the
  headline probe — CAP_SYS_PTRACE (or matching uid) is also needed. Kernel
  probes are fine on the two caps.
- seccomp: --profile=sysadmin unconfines seccomp as a side-effect of
  privileged:true (not an explicit field); and on harbor (no PSS, Bottlerocket
  seccomp-default off) the debug container likely runs Unconfined already, so
  the RuntimeDefault EPERM may never appear. Split-caps is no stricter than
  sysadmin on the seccomp axis.
- IMDS residual: hop-limit-1 does NOT close the node-role path in a PSS-less
  eng namespace (hostNetwork bypass; CAP_PERFMON reads creds from kernel
  memory independent of IMDS). Node role is reachable regardless — escalate,
  don't self-mitigate. Residuals reframed as surface-for-sign-off, not
  self-accept.
- Prose: bolded the load-bearing imperatives; 'namespace' not 'node pool'.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…R2 prose polish)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@bdchatham bdchatham merged commit 33280cc into main Jul 14, 2026
5 checks passed
@bdchatham bdchatham deleted the docs/con368-ebpf-runbook-seccomp-dedicated-node branch July 14, 2026 18:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant