chore: merge v5-next into merge-train/fairies-v5#24296
Merged
mverzilli merged 49 commits intoJun 25, 2026
Merged
Conversation
Fixes a race in `e2e_token_bridge_tutorial_test` where L1 setup transactions were submitted but not awaited before dependent bridge calls. - Waits for the `TestERC20.addMinter` transaction receipt before minting through the handler. - Waits for the `TokenPortal.initialize` transaction receipt before simulating `depositToAztecPublic`. - Prevents intermittent `SafeERC20FailedOperation(address token) (0x0)` when the portal deposit simulation runs before `underlying` is initialized. See http://ci.aztec-labs.com/e100f59864b9c18c for sample failed run.
…24169) ## Problem A node that restarts after its bootstrap nodes are gone cannot rejoin the network on its own, even though it has a list of peers persisted on disk. discv5 builds its routing table only from **bootstrap** and **trusted** ENRs (`discV5_service.ts` `start()`), and keeps that table **in memory** — it is never persisted. The peer-manager dial queue is fed exclusively by discv5 discovery events, and libp2p auto-dial is disabled (`minConnections: 0`). So after a restart with no reachable bootstrap node, a node's DHT starts empty and the only way it reconnects is via **inbound** dials from peers that still hold its ENR. That is unreliable and fails entirely on a full-network restart. This is the root cause of the flaky `e2e_p2p_rediscovery` failure (see A-1244): the test only passed because it restarted nodes one at a time, so still-running neighbours dialed the restarted node back. The last nodes to restart often ended up below quorum (`discv5KadPeers: 0`, stuck at 2/3 peers). ## Fix Persist discovered peer ENRs and re-seed discovery from them on startup: - **Persist** valid, non-bootnode peer ENRs to the p2p peer store (`onEnrAdded`), keyed by nodeId, bounded to `MAX_PERSISTED_PEER_ENRS` with simple eviction. - **Re-seed** on `start()`, alongside the existing bootstrap/trusted-ENR seeding: load persisted ENRs and `discv5.addEnr(...)` each (dropping stale/incompatible ones). Re-adding an ENR flows through the existing `onDiscovered` → `DISCOVERED` → `PeerManager` dial path, so no new dial logic is required. ENRs carry both UDP (discv5) and TCP (libp2p) addresses, so this restores both discovery and dialability without a bootstrap node. The persisted store is the same `peerStore` libp2p already uses; the discovery service just opens its own map on it. ## Test `e2e_p2p_rediscovery` now exercises the real capability: - Wait for a full mesh before teardown, so every node has persisted all its peers (replaces a blind `sleep`). - Stop the bootstrap node and **all** validators, then restart them **all** — a full-network bounce — instead of the rolling stop-one/restart-one loop. With no live peers to dial the nodes back, the mesh can only rebuild by re-seeding from persisted ENRs. Existing discv5 unit tests pass unchanged. (The pre-existing `should persist peers without bootnode` unit test remains skipped — it can't be made faithful without spawning fresh node processes; the e2e test is the integration coverage.) Fixes A-1244.
Comment-only annotations across the end-to-end test suite, produced by the e2e consolidation survey. Every test file under `yarn-project/end-to-end/src/` gets a short prose note above its `describe`/`it` describing how it sets up its environment — sequencer type, node topology, prover, cross-chain, time-warp — and the category it maps to in the proposed consolidation, plus inline `// REFACTOR:` markers where a follow-up is expected.
…m gating from prover A node running a validator now always creates the sentinel regardless of SENTINEL_ENABLED, since the sentinel is required to reach consensus on slashing offenses. Non-validators continue to respect SENTINEL_ENABLED (default false). Subsystem gating (sentinel, slashing watchers, slasher) is no longer tied to prover-only mode but to validator status. Non-validators can opt into running the watchers plus a read-only slasher via the new OFFENSE_COLLECTION_ENABLED flag to collect and serve offenses over the admin RPC. The spartan deploy enables it by default, mirroring SENTINEL_ENABLED. Fixes A-1242
…andler for non-validator offense collection Lifts the slotsWithInvalidProposals / slotsWithProposalEquivocation tracking out of ValidatorClient and into ProposalHandler, which now implements InvalidProposalSlotSource (hasInvalidProposals / hasProposalEquivocation). The handler marks these slots from its all-nodes block and checkpoint proposal handlers, so any node that re-executes proposals (the default) populates them, not only validators. The attested-invalid-proposal watcher is now wired to the proposal handler (validator-owned or standalone) instead of the validator client, so a non-validator offense collector (OFFENSE_COLLECTION_ENABLED) can detect attested-to-invalid-checkpoint-proposal offenses. ValidatorClient delegates its has/mark calls to the handler, preserving validator behavior.
Drop the now-redundant invalid-checkpoint slot mark in ValidatorClient: the all-nodes checkpoint handler already marks the slot before invoking the failure callback. The FifoSet add was idempotent so the double-mark was harmless, but the validator-side call is dead. Guard non-validator invalid-block marking with the escape-hatch check, matching the validator path that intentionally disables invalid-block slashing while the escape hatch is open. Mark proposal equivocation from the p2p duplicate-proposal callback in ProposalHandler.register. p2p detects duplicate proposals without routing them through the proposal handlers, so without this a non-validator offense collector could mark a slot invalid without the matching equivocation mark and false-positive slash attesters. Validators overwrite this callback with their own richer handler.
…ting (A-1258) DataTxValidator decodes contract-class publication logs before proof validation. bufferFromFields trusted the declared byte-length field and called Buffer.alloc(byteLength) even when it far exceeded the payload actually present, so a malformed class-publication tx could force attacker-chosen multi-MiB allocations on every node that validates the gossiped tx (req/resp, block proposal, RPC, gossip) before the proof validator rejects it. Add an optional maxByteLength guard to bufferFromFields (preserving the documented blob-reconstruction padding for callers that don't pass it); ContractClassPublishedEvent.fromLog passes the fixed log's payload capacity, so an over-declared length throws before allocation. DataTxValidator already wraps fromLog in try/catch, so the throw becomes a clean pre-proof rejection. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The gossipsub msgId was SHA256(topic || data) with no framing between the topic string and the message bytes, and no allowedTopics was set. A peer could publish on an unsubscribed topic T+data[0] with data[1:], whose (topic, data) concatenation is byte-identical to a real message on topic T, hashing to the same msgId. gossipsub transformed it and inserted the id into seenCache before the subscription check, so the genuine proposal/attestation was later dropped as a duplicate, suppressing a time-sensitive consensus message. - Frame the topic length into the msgId input (uint32be(topicLen) || topic || data) so the (topic, data) boundary is unambiguous and a boundary-shifted pair no longer collides. - Set exact allowedTopics on the gossipsub config so an unsubscribed-topic message is rejected before transform / msgId / seenCache insertion. Defense in depth: either fix alone breaks the attack. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…255)
A discovery peer could advertise a signed, same-version, UDP-contactable
ENR with a malformed TCP field (e.g. 3 bytes instead of a 2-byte port).
validateEnr only checked the aztec key + version, so discv5 emitted
enrAdded; the unguarded async onEnrAdded listener then awaited
getFullMultiaddr('tcp'), which threw RangeError, and as an async listener
the rejection was unhandled and crashed the node.
Audit of every getFullMultiaddr/getLocationMultiaddr site in p2p found
the same pattern on the bootnode's discovered listener (crashes the
bootnode) and config-provided preferredPeers parsing (aborts setup):
- Guard onEnrAdded and the bootstrap discovered listener so a parser
exception is caught, logged and the ENR dropped (no process exit).
- validateEnr rejects an ENR whose TCP multiaddr won't parse, so it's
never emitted as a full peer / re-parsed downstream. A missing TCP addr
stays allowed (those peers are skipped at dial time).
- preferredPeers ENR parsing (peer_manager, libp2p_service) skips and logs
a malformed configured ENR instead of aborting node setup.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…258) Defensive guard from PR review: an empty fields array would destructure length to undefined and throw on length.toNumber(). A valid bufferAsFields output always carries the leading byte-length field, so empty encodes no data — return an empty buffer. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…rs) (#24221) ## Raw merge — do not merge alone This is **PR 1 of a two-PR stack** that catches `merge-train/spartan-v5` up to its base `v5-next` so the long-lived train PR (`merge-train/spartan-v5` → `v5-next`) becomes conflict-free again. This PR is the **raw merge of `v5-next` into `merge-train/spartan-v5` with conflict markers committed** — it is red by design. Review and merge it only together with the stacked resolution PR (PR 2, head `cb/merge-train-spartan-v5-fix`), which targets this branch. ### Parents - train: `merge-train/spartan-v5` @ `bc2c924919fe8880ee9572df779b78f653ef2e15` - base: `v5-next` @ `4df72438bf01f125e791af7e8bb11a613964457e` ### Conflicted files (1) - `yarn-project/end-to-end/src/e2e_nested_utility_calls.test.ts` Resolution lives in the stacked fix PR. --- *Created by [claudebox](https://claudebox.work/v2/sessions/51d528a3438f2849) · group: `slackbot`* --------- Co-authored-by: Mitchell Tracy <mitchellftracy@gmail.com> Co-authored-by: Nicolas Chamo <nicolas@chamo.com.ar> Co-authored-by: Santiago Palladino <santiago@aztec-labs.com> Co-authored-by: just-mitch <68168980+just-mitch@users.noreply.github.com> Co-authored-by: Nicolás Venturo <nicolas.venturo@gmail.com> Co-authored-by: AztecBot <tech@aztecprotocol.com>
…ing (A-1253) (#24212) Fixes A-1253. ## Problem A protocol-valid public log can carry **zero fields** — `PublicLog(nonzeroContractAddress, [])` — reachable by any tx that bypasses the high-level aztec.nr helpers (the AVM `EMITPUBLICLOG` opcode accepts `logSize = 0`). `LogStore` used `fields[0]` as the tag-index key, so `fieldHex(undefined)` threw inside the `addLogs` transaction. Because that runs inside `addProposedBlock`/`addCheckpoints`' larger store transaction, the throw aborted the entire block/checkpoint update — a single such tx in an accepted block stalls L1 sync for every node that ingests it (the same valid checkpoint is retried forever). ## Fix (Fix A — archiver-only, lowest risk) A log with no fields has no tag. Index it under the **empty tag** (`tagHexForLog` returns `''` for a zero-field log): - it stays retrievable via the per-block read (`getPublicLogsForBlock`) and is dropped on reorg via the per-block secondary index, exactly like any other log; - the empty-tag key sorts before every real key for the contract, so a real 64-hex-char tag query never matches it. `PublicLog.fields` is a variable-length `Fr[]` (the actual crash surface); `PrivateLog.fields` is a fixed padded tuple so its `fields[0]` is always present, but the shared helper is applied to both loops for consistency and is behaviour-preserving for private logs. ## Test `log_store.test.ts`: constructs `new PublicLog(CONTRACT, [])`, asserts `addLogs` resolves (previously threw), the log is returned by `getPublicLogsForBlock`, and a tag query omits it. --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…(A-1256) The gossipsub fast-path dedup cache keyed on fastMsgIdFn — a non-cryptographic 64-bit xxhash of the raw data only (no topic), seeded from Math.random (~2^30, non-CSPRNG). A fast-id collision (accidental, or engineered given the weak seed, or simply same-data-on-a-different-topic) makes gossipsub drop a different message as a duplicate with no fallback to the full id, so it's a gossip-suppression vector that the framed full msgId does not close (the fast path short-circuits before it). fastMsgIdFn is optional; removing it leaves dedup resting solely on the cryptographic, topic-framed msgIdFn (SHA-256). Also removes the now-unused xxhash machinery from encoding.ts. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
No longer used after removing fastMsgIdFn. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… EpochSession (A-1212) (#24216) Fixes A-1212. ## Problem `prover_getJobs` exposes each `EpochSession`'s state, but `EpochSession` only ever assigned `initialized`, `awaiting-checkpoints`, and the terminal states. The intermediate phases that exist to describe active work were never set, so a session read `awaiting-checkpoints` continuously while it was actually (a) proving the top tree or (b) publishing to L1. Observed on staging-internal: every `startProof` session sat in `awaiting-checkpoints` even with logs showing `Starting top-tree prove…`. This is a regression vs the old `EpochProvingJob`, which advanced through distinct phases. ## Fix - Add a new **non-terminal** `awaiting-root` state to `EpochProvingJobState` (the zod schema is derived from the same array, so it updates automatically). Not added to `EpochProvingJobTerminalState`. - Set `awaiting-root` when the top-tree (root) prove begins, via the `TopTreeJob` `beforeProve` hook — which fires once the sub-tree (checkpoint block) proofs are ready and the root prove is about to start. `toTopTreeHooks()` now always wires this transition and layers any test-provided hooks on top (previously it returned `undefined` when no test hooks were set). - Set `publishing-proof` at the top of `submitProof()`, before the L1 submit. Both transitions are guarded by `isTerminal()` so a concurrent `cancel()` still wins (the post-submit `isTerminal()` check relies on this). Phase progression is now: `initialized → awaiting-checkpoints → awaiting-root → publishing-proof → terminal`. ## Tests - `epoch-session.test.ts`: new "state reporting" test asserts `getState()` is `awaiting-root` during the prove and `publishing-proof` during the submit, then `completed`. Existing hook-ordering test (`before`/`prove`/`after`) still holds. - `prover-node.test.ts`: the JSON-RPC schema round-trip mock now includes an `awaiting-root` job. Targets the v5 line — the `EpochSession`/`TopTreeJob`/`awaiting-checkpoints` code (post-#23552) exists only on `merge-train/spartan-v5`. --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Resolves conflict in e2e_nested_utility_calls.test.ts (keeps the train's doc comment above the 'denies...private function' test; the msg_sender test from v5-next is already present). Real merge commit so v5-next becomes an ancestor of the train.
…empotent The archiver preloads bundled protocol contract classes and instances at synthetic block 0. When a bundled protocol class id (or instance address) was later published on chain, the contract stores threw "already exists" while re-adding the already-present key, stalling L1 sync. Make the stores idempotent for protocol-preloaded entries: - addContractClass / addContractInstance now skip (no-op) when the key already exists and it belongs to a bundled protocol contract, keeping the existing block-0 entry untouched. Genuine non-protocol redefinitions still throw. - deleteContractClass / deleteContractInstance never delete protocol entries, so they survive reorgs of the publishing block. Adds isProtocolContractClass to protocol-contracts as a sibling of isProtocolContract, plus store unit tests and an integration test that publishes a bundled protocol class id through ArchiverDataStoreUpdater. Fixes A-1257
…ting (A-1258) (#24213) Fixes A-1258. ## Problem `DataTxValidator` decodes contract-class publication logs **before** proof validation (data validation runs ahead of proof validation on the req/resp, block-proposal, RPC, and gossip paths). The decode trusts the declared public-bytecode length field: - `bufferFromFields` (`stdlib/src/abi/buffer.ts`) reads the declared `byteLength` and, when the payload is shorter, does `Buffer.alloc(byteLength)` — trusting the declared length over the bytes actually present. - `ContractClassPublishedEvent.fromLog` decodes the bytecode via `bufferFromFields`, and `DataTxValidator.#hasCorrectContractClassIds` calls `fromLog` before any class-id / proof rejection. A contract-class log is a fixed `CONTRACT_CLASS_LOG_SIZE_IN_FIELDS` array, so the real packed bytecode is capped at ~93 KiB. But the declared length is unbounded: a ~96 KiB log declaring 16 MiB forces a 16 MiB allocation on every node that validates the tx, before the proof check rejects it — a memory-pressure / validation-amplification path. ## Fix - Add an optional `maxByteLength` parameter to `bufferFromFields`; when provided, it throws (before allocating) if the declared length exceeds it. Callers that don't pass it keep the documented blob-reconstruction padding behaviour unchanged. - `fromLog` passes the fixed log's payload capacity (`payloadFields × 31`), so an over-declared length is rejected before allocation. `DataTxValidator` already wraps `fromLog` in try/catch, so the throw becomes a clean pre-proof `TX_ERROR_MALFORMED_CONTRACT_CLASS_LOG` rejection. ## Tests - `data_validator.test.ts`: a log declaring a 16 MiB length is rejected, and `Buffer.alloc` is never called with the declared size (test time drops from ~1.2s to ~40ms — no large allocation). Run against the pre-fix compiled code, the allocation still happened (red); green after the fix. - Existing `contract_class_published_event.test.ts` "fits a max-size public bytecode" and the `buffer.test.ts` padding tests confirm legitimate max-size bytecode and the padding contract are unaffected.
…24214) Fixes A-1256. ## Problem The gossipsub full message id was `SHA256(topic || data)[0..20]` with **no framing** between the topic string and the message bytes, and `LibP2PService` set no `allowedTopics`. Raw concatenation isn't injective: for a real message `(T, D)`, a peer can craft `T' = T + D[0]`, `D' = D[1:]` so `T' || D'` is byte-identical to `T || D` → same msgId. ChainSafe gossipsub transforms the arbitrary-topic message and inserts that id into `seenCache` **before** the subscription check (it isn't delivered to us, since we're not subscribed to `T'`). When the genuine proposal/attestation arrives on `T`, gossipsub drops it as a **duplicate** before application validation, peer scoring, or handling — suppressing a time-sensitive consensus message within the slot. ## Fix (defense in depth — either alone breaks the attack) 1. **Frame the msgId input** as `uint32be(topicLen) || topic || data` (`encoding.ts`). The topic length pins the `(topic, data)` boundary, so a boundary-shifted pair no longer collides. (`getMsgIdFn`'s parameter is narrowed to `Pick<Message, 'topic' | 'data'>` — the only fields it reads — which stays assignable to gossipsub's `msgIdFn` slot.) 2. **Set exact `allowedTopics`** (`libp2p_service.ts`) to the subscribed Aztec topic strings. Verified against the installed gossipsub: the allowlist is enforced in `handleReceivedRpc` *before* `handleReceivedMessage`/`seenCache.put`, so an unsubscribed-topic message is dropped before transform / msgId / seenCache. ## Test `encoding.test.ts`: builds a real `P2PMessage.toMessageData()` buffer (confirming `data[0] === 0x00`), constructs the shifted `(T', D')`, and asserts the two msg ids now differ (they collided before the framing change); plus a determinism check. ## Compatibility `msgId` is computed locally for dedup; changing the function doesn't change any on-wire format. During a rolling upgrade, mixed nodes briefly compute different ids for the same message (minor IHAVE/IWANT inefficiency), with no correctness impact.
## Merge `v5-next` into `merge-train/spartan-v5` — MUST merge as a merge commit (do not squash) ### Why the conflicts came back The previous stack (#24221/#24222) resolved the conflict correctly, but #24221 was **squash-merged** onto the train. A squash creates a brand-new single commit and does **not** record `v5-next` as a parent, so git still sees the train and `v5-next` as diverged (`git merge-base --is-ancestor origin/v5-next origin/merge-train/spartan-v5` → false). The merge-train auto-pull then re-attempts `v5-next → train` and hits the same conflict again. ### What this PR does This is a real **merge commit** of `v5-next` into the train (two parents: train tip `eb7d64d` + `v5-next` `4df7243`), with the one conflict resolved. The resulting tree is **identical to the current train** (0 file changes) because the content already landed via the earlier squash — this PR exists purely to record `v5-next` as an ancestor and advance the merge base. After this lands, `v5-next` is an ancestor of the train and the long-lived `merge-train/spartan-v5` → `v5-next` PR is conflict-free. ### Conflict resolved (1) `yarn-project/end-to-end/src/e2e_nested_utility_calls.test.ts` — kept the train's doc comment above the `denies…private function` test; the `msg_sender` test from `v5-next` is already present. ###⚠️ Merge method **Merge this with "Create a merge commit"** (or `merge_pr merge_method=merge`). A squash or rebase merge will recreate the same divergence and the conflicts will return. --- *Created by [claudebox](https://claudebox.work/v2/sessions/51d528a3438f2849) · group: `slackbot`*
…255) (#24215) Fixes A-1255. ## Problem A discovery peer can advertise a signed ENR with valid `ip`/`udp` contact data and the expected Aztec ENR version key, but a **malformed** `tcp` field (e.g. 3 bytes instead of a 2-byte port). `DiscV5Service.validateEnr` only checked the aztec key + version, so upstream discv5 accepted the session and emitted `enrAdded`. The unguarded async `onEnrAdded` listener then awaited `enr.getFullMultiaddr('tcp')`, which throws `RangeError` on the malformed value; as an async EventEmitter listener the rejection is unhandled and **the node process exits**. One malformed, UDP-contactable, same-version ENR crashes any p2p-enabled node. ## Audit A sweep of every `getFullMultiaddr` / `getLocationMultiaddr` call in `p2p/src` found the same unguarded-parse pattern in more than one place: | Site | ENR source | Crash today | Action | |------|-----------|-------------|--------| | `discV5_service.ts` `onEnrAdded` | remote | yes | guard + drop | | `bootstrap.ts` `discovered` listener | remote | yes (crashes the **bootnode**) | guard + drop | | `peer_manager.ts` `handleDiscoveredPeer` | remote | no (caught by wrapper) | covered by `validateEnr` rejecting malformed-addr ENRs | | `peer_manager.ts` / `libp2p_service.ts` `preferredPeers` | config | aborts setup | skip + log | ## Fix - Guard `onEnrAdded` and the bootnode `discovered` listener so a parser exception is caught, logged, and the ENR dropped — no unhandled rejection. - `validateEnr` rejects an ENR whose TCP multiaddr won't parse, so it's never emitted as a full peer (`PeerEvent.DISCOVERED`) or re-parsed by `handleDiscoveredPeer`. A *missing* TCP addr stays allowed — those peers are simply skipped at dial time; only an *unparseable* one is rejected. - `preferredPeers` ENR parsing skips and logs a malformed configured ENR instead of aborting node / preferred-peer setup. The self-ENR log line (`discV5_service.ts:199-200`) is intentionally left unguarded — it reads our own ENR, which we build and which can't be attacker-malformed. ## Tests `discv5_service.test.ts`: builds a same-version ENR with valid `ip`/`udp` and a 3-byte `tcp` value, and asserts (a) `validateEnr` rejects it while a valid control built from the same identity passes — isolating the TCP check — and (b) `onEnrAdded` resolves (no unhandled rejection, previously a `RangeError`) and never emits `DISCOVERED`. Note: `discV5_service.ts` overlapped with the already-merged PR #24169; this branch is off the post-merge `merge-train/spartan-v5`, so no conflict.
…nt state The archive tree is an append-only accumulator of block-header hashes, so a single bad leaf (e.g. from a mishandled reorg) is never self-corrected: every later root stays noncanonical while the other state trees can re-converge from block effects. sync_block only checked the four non-archive trees, so a self-consistent orphan block — commonly an empty one — could silently fork the archive root from canonical. Verify the archive root against canonical both before appending (the committed root must equal the block's lastArchive) and after (the resulting root must equal the block's archive), failing before commit so the divergence is never persisted. The checks are optional std::optional parameters; the napi and wsdb transports forward the canonical roots from the block being synced.
BEGIN_COMMIT_OVERRIDE fix(p2p): re-seed discovery from persisted peer ENRs after restart (#24169) docs(e2e): annotate e2e tests with setup/category notes (#24191) chore: merge v5-next into merge-train/spartan-v5 (raw, conflict markers) (#24221) fix(archiver): index zero-field logs under empty tag instead of throwing (A-1253) (#24212) fix(prover-node): report awaiting-root and publishing-proof phases in EpochSession (A-1212) (#24216) fix(p2p): bound declared contract-class bytecode length before allocating (A-1258) (#24213) fix(p2p): frame gossipsub msgId and restrict allowedTopics (A-1256) (#24214) chore: merge v5-next into merge-train/spartan-v5 (#24226) fix(p2p): guard ENR address parsing against malformed TCP fields (A-1255) (#24215) END_COMMIT_OVERRIDE
…nt state (#24229) Fixes A-1235 # A-1235 reviewer notes: archive-root divergence and the v4/v5 block-stream race ## Short version The A-1235 symptom was a mainnet fisherman rejecting every proposal with `ReExInitialStateMismatchError`. The proposal/canonical `lastArchive.root` was correct; the local world-state archive root was wrong. The important distinction for review is: - v4.3.0 appears vulnerable to a specific block-stream/cache race that can seed this state. - v5 has already hardened that specific race. - v5 still lacks the world-state archive-root invariant, so if any other path ever writes a bad archive leaf, the corruption can still be committed silently. This change is therefore defense in depth at the only layer that can conclusively reject a bad archive tree before it is persisted: `WorldState::sync_block`. ## What went wrong World state maintains the archive tree as an append-only accumulator of block-header hashes. During `sync_block`, v4.3.0 appended the new block header hash and then checked: - `is_archive_tip`: the just-appended header hash is the tip leaf. - `is_same_state_reference`: nullifier, note hash, public data, and L1-to-L2 message trees match the block state reference. Neither check verifies the archive root against canonical block data: - before append: local archive root must equal the block's `lastArchive.root`; - after append: computed archive root must equal the block's `archive.root`. That means an orphan block can be internally self-consistent and still poison the archive tree. If the orphan and canonical block at the same height have the same effects, commonly because both are empty, the four non-archive trees can reconverge while the archive tree remains permanently forked. The resulting sequence is: 1. World state syncs an orphan block at height `F`. 2. The archiver later canonicalizes a different block at height `F`. 3. The block stream should prune world state to `F - 1`. 4. In the suspected v4 race, it prunes only to `F`, so the orphan archive leaf remains. 5. World state syncs canonical descendants starting at `F + 1` on top of the orphan leaf. 6. The four-tree state-reference check passes, but every archive root from `F` onward is off-chain. ## The v4 block-stream/cache race The original explainer says roughly: `getL2Tips` had processed the reorg, while the archiver had not yet swapped orphan `F` for canonical `F`. More precisely, this is not about committed DB state being partially updated. The block/checkpoint mutation is one LMDB writer transaction. The race is that v4's `L2TipsCache` can publish a tip computed from the uncommitted writer view, while separate block/header reads still use committed read transactions. In v4.3.0: - `ArchiverDataStoreUpdater.addCheckpoints()` wraps prune, checkpoint insertion, logs, contract data, and `l2TipsCache.refresh()` in one `store.transactionAsync(...)`. - `L2TipsCache.refresh()` assigns `#tipsPromise = this.loadFromStore()` before the writer transaction commits. - The LMDB wrapper reuses the active write transaction for reads inside that callback, so `loadFromStore()` can see the post-reorg uncommitted view. - A concurrent consumer calling `archiver.getL2Tips()` can receive that cached future/post-reorg tip. - The same consumer's later `getBlockHeader(F)` call is outside the writer context, so it opens a normal committed read transaction and can still see the pre-reorg orphan at `F`. The v4 `L2BlockStream` performs exactly this mixed read pattern: ```ts const sourceTips = await this.l2BlockSource.getL2Tips(); const localTips = await this.localData.getL2Tips(); let latestBlockNumber = localTips.proposed.number; const sourceCache = new BlockHashCache([sourceTips.proposed]); while (!(await this.areBlockHashesEqualAt(latestBlockNumber, { sourceCache }))) { latestBlockNumber--; } ``` `areBlockHashesEqualAt()` then asks the source for a per-height hash via `getBlockHeader(blockNumber)`. So a single pass can be planned from a future/post-reorg tip but compare old committed per-height headers. If both local world state and the committed archiver still have the orphan at `F`, the walk can falsely conclude `F` is the common ancestor. Pruning to `F` keeps block `F`; the correct target was `F - 1`. That is the suspected seed path for A-1235. The live evidence proves the archive tree forked; the historical interleaving predates retained logs, so treat this as the best source-grounded mechanism, not as directly logged fact. ## What v5 changes in this area v5 has multiple changes that make the specific v4 race much less plausible. ### 1. Tip cache refresh is post-commit In v5, `L2TipsCache` says refresh should happen after the writer transaction has committed, and `ArchiverDataStoreUpdater` does exactly that: ```ts const result = await this.stores.db.transactionAsync(async () => { // mutate blocks/checkpoints/logs/etc. return ...; }); await this.l2TipsCache?.refresh(); return result; ``` So the cache is loaded from committed DB state, and an aborted writer cannot replace the cache with a future view. ### 2. `getL2TipsData()` is one DB snapshot v5 moves chain-tip construction into `BlockStore.getL2TipsData(genesisBlockHash)` and wraps it in a single `db.transactionAsync(...)`. It also validates the resulting tier ordering: - finalized <= proven <= checkpointed <= proposed; - checkpointed block <= proposed block. That is materially stronger than v4's `L2TipsCache.loadFromStore()`, which assembled tip numbers and block data through several independent store calls. ### 3. The block stream fails closed on incoherent source reads v5's `L2BlockStream` no longer uses the old checkpoint-prefetch path in the same way. It drives from a source tips snapshot, compares per-height hashes via `getBlockData({ number })`, and adds guards: - missing local hash compares unequal rather than accidentally stopping the walk; - missing source data at or below the advertised source proposed tip aborts the pass; - source tips are re-read after a prune before downstream reconciliation; - the download pass verifies the delivered proposed block hash matches the snapshot's proposed hash; - tier advancement is skipped if the block download plan did not complete. These are all aimed at preventing a stale or mixed source snapshot from becoming an under-deep prune. Over-deep or skipped reconciliation is recoverable; under-deep pruning is dangerous because it can leave the losing fork's block at the divergence height. ## Why this change still matters on v5 The v5 block stream fixes the known/suspected seed path, but it does not add the missing invariant to world state. Without this patch, `sync_block` can still commit a divergent archive tree if any future path presents it with a self-consistent block whose non-archive state matches: - a different reorg bug; - a cache/snapshot bug elsewhere; - operator/datadir corruption; - a future refactor that bypasses one of the v5 block-stream protections. Once a bad archive leaf is committed, appending more canonical leaves does not repair it. The archive root remains noncanonical forever while the four non-archive trees can look healthy. The fix makes `sync_block` verify the archive tree at the source of truth: ```cpp // Before append: committed local root must be the block's parent archive root. actual_previous_archive_root == expected_previous_archive_root // After append, before commit: uncommitted computed root must be the block's archive root. actual_archive_root == expected_archive_root ``` These checks turn silent permanent corruption into a loud sync failure before commit. The node may need a datadir resync, but it will not write and seal in the divergent archive leaf. ## Reviewer checklist - Confirm the TS/native message path passes both canonical roots: - `expectedPreviousArchiveRoot = l2Block.header.lastArchive.root` - `expectedArchiveRoot = l2Block.archive.root` - Confirm native `sync_block` checks the previous root before `add_value`. - Confirm native `sync_block` checks the resulting root before `commit`. - Confirm errors clearly tell operators that local world state diverged and must be resynced. - Confirm tests cover the archive-only divergence case, not just non-archive state-reference mismatch. ## Expected operator behavior This patch prevents recurrence; it does not repair an already-poisoned archive tree. A node that already has the bad historical leaf must wipe/resync its world-state datadir.
…24202) ## Motivation The sentinel is required for the network to reach consensus on slashing offenses — it's a core duty of participating in consensus, not an optional add-on. Yet `SENTINEL_ENABLED` defaulted to `false`, so a validator could run without one and silently degrade the network's ability to enforce its own rules. While fixing that, two adjacent issues surfaced: - Whether the node runs the sentinel and the slashing-detection watchers was gated on a *prover-only* flag (`enableProverNode && disableValidator`). That coupling is wrong: those subsystems should depend on **validator status**, not on whether the prover is enabled. - A non-validator (e.g. an RPC or full node) had no way to collect and inspect offenses, because the only component that persists them lives inside the slasher client, which was created only for validators. ## Approach - **Force the sentinel for validators.** `createSentinel` now self-gates: if the node runs a validator it always creates the sentinel (ignoring `SENTINEL_ENABLED`); otherwise it respects `SENTINEL_ENABLED` (default `false`). It logs which case applies so operators understand why a `SENTINEL_ENABLED=false` is being overridden. - **Decouple subsystem gating from the prover.** Removed the `proverOnly` guard. The slashing-detection watchers and the slasher are now gated on `collectOffenses = !disableValidator || enableOffenseCollection`. - **Opt-in offense collection for non-validators.** A new `OFFENSE_COLLECTION_ENABLED` flag lets a non-validator run the watchers plus a read-only slasher client (it never writes to L1 — voting/execution flow through the sequencer publisher, which a non-validator lacks) to collect and serve offenses over the existing `getSlashOffenses` admin RPC. The spartan deploy enables it by default, mirroring `SENTINEL_ENABLED`. - **Make the attested-invalid-proposal watcher work without a validator.** Lifted the invalid-proposal / equivocation slot tracking out of `ValidatorClient` into `ProposalHandler`, which now implements `InvalidProposalSlotSource`. The handler populates this from its all-nodes proposal handlers, so any node that re-executes proposals (the default) can feed the watcher. ## API changes - New node config `enableOffenseCollection` (env `OFFENSE_COLLECTION_ENABLED`, default `false`). Named distinctly from the existing L1-deploy `AZTEC_SLASHER_ENABLED` to avoid confusion. - Validator nodes now always run the sentinel regardless of `SENTINEL_ENABLED`. ## Changes - **aztec-node**: `createSentinel` forces the sentinel for validators and logs the reason; `server.ts` removes `proverOnly`, gates watchers + a split-out read-only slasher on `collectOffenses`, and wires the attested-invalid-proposal watcher to the proposal handler. - **aztec-node (config) / foundation**: new `enableOffenseCollection` / `OFFENSE_COLLECTION_ENABLED` flag and env-var entry. - **validator-client**: `ProposalHandler` now owns the invalid-proposal / equivocation slot tracking and implements `InvalidProposalSlotSource`; `ValidatorClient` delegates to it, preserving validator behavior. - **spartan**: `OFFENSE_COLLECTION_ENABLED` wired through the chart value, pod template, network defaults, terraform variable, and deploy script, defaulting on (mirrors `SENTINEL_ENABLED`). Fixes A-1242
…empotent (#24227) ## Motivation The archiver preloads every bundled protocol contract class (and its canonical instance) into its local store at synthetic block 0, before L1 sync. World-state genesis, however, seeds no registration nullifiers for those classes/instances. As a result a *first* on-chain `ContractClassRegistry.publish` of a bundled protocol class id is protocol-valid (fresh class-id nullifier + `ContractClassPublished` log). On replay the archiver recomputes the same class id and unconditionally re-inserts it, but the store throws on the pre-existing block-0 key (`Contract class <id> already exists, cannot add again`). Because that insert runs inside the block/checkpoint store transaction, the throw aborts persistence, and L1 sync retries the same valid checkpoint indefinitely — a sync stall. ## Approach Make the archiver treat protocol-preloaded entries as idempotent and immutable, guarded at the store layer (the single chokepoint for both the add-throw and the block-gated delete): - `addContractClass` / `addContractInstance`: when the key already exists and it is a protocol class id / magic protocol address, treat the (re-)publish as a no-op and keep the existing block-0 entry — crucially **without** bumping its recorded block number. Genuine non-protocol duplicates still throw unchanged. - `deleteContractClass` / `deleteContractInstance`: skip deletion for protocol entries, so a reorg of the publishing block can never roll out the preload. Keeping the preload at block 0 is deliberate: `deleteContractClass` only deletes when the stored `l2BlockNumber >= blockNumber`, so block 0 makes protocol entries survive any reorg; bumping to the publish block would let a deep reorg delete them. The instance-side guards are defensive only: on-chain `publish_for_public_execution` always emits the *derived* address, never a magic address, so the instance store is not reached with a magic address via the on-chain replay path today — the guards exist for symmetry and to protect future code paths. This is a non-breaking, node-local resilience fix. A follow-up PR (PR2) seeds the protocol registration nullifiers into world-state genesis so the on-chain re-publish is rejected at the protocol level — the root-cause fix for the class path. ## Changes - **protocol-contracts**: add `isProtocolContractClass(classId)` (sibling of the existing `isProtocolContract(address)`), backed by a set of the generated `ProtocolContractClassId` values. - **archiver**: idempotent add + protected delete for protocol classes and instances in `ContractClassStore` / `ContractInstanceStore`. - **archiver (tests)**: store unit tests (idempotent re-add stays queryable with block-0 / bytecode-commitment preserved, protected delete, non-protocol duplicate still throws — for both classes and instances) and an A-1257 integration test that preloads protocol contracts, builds an `L2Block` carrying a `ContractClassPublished` log for a bundled class id, and asserts `addProposedBlock` commits and the class stays queryable. Fixes A-1257
…24065) Fixes a race in `e2e_token_bridge_tutorial_test` where L1 setup transactions were submitted but not awaited before dependent bridge calls. - Waits for the `TestERC20.addMinter` transaction receipt before minting through the handler. - Waits for the `TokenPortal.initialize` transaction receipt before simulating `depositToAztecPublic`. - Prevents intermittent `SafeERC20FailedOperation(address token) (0x0)` when the portal deposit simulation runs before `underlying` is initialized. See http://ci.aztec-labs.com/e100f59864b9c18c for sample failed run.
Backports the source-doc corrections from two already-merged `next` docs PRs to the `v5-next` release line, where the same pages were stale: - #24005 — `fix(docs): fable review` (merged Jun 22) - #23830 — `docs: complete and correct the proving historic state page` (merged Jun 22) `v5-next` has no `version-v5.0.0-rc.1` versioned snapshot (its versioned docs are v4.3.0-era), so this touches **source docs only** — the authored pages under `docs/docs-developers/` and `docs/docs-operate/`. ## What's included (12 files) Each fix was re-verified against `v5-next` source, not blindly copied from `next`/v6: **From #23830** - `aztec-nr/.../advanced/how_to_prove_history.md` — rewrite. Every history fn it documents (`assert_note_existed_by`, `assert_note_was_nullified_by`, `assert_note_was_valid_by`, `assert_contract_bytecode_was_published_by`, `assert_contract_was_initialized_by`, `public_storage_historical_read`) exists verbatim in v5-next's `aztec/src/history/`. **From #24005** - `aztec-nr/.../how_to_retrieve_filter_notes.md`, `state_variables.md` — `RetrievedNote`/`HintedNote` → `ConfirmedNote` (type confirmed present on v5-next). - `aztec-nr/.../events_and_logs.md` — nonexistent `self.context.emit_public_log(...)` → `emit_public_log_unsafe` (confirmed at `public_context.nr:110` on v5-next). - `aztec-nr/.../contract_structure.md` — storage struct must be named `Storage` (version-agnostic). - `foundational-topics/contract_creation.md` — `aztec::history::contract_inclusion` → `aztec::history::deployment` (the `deployment` module exists on v5-next). - `foundational-topics/.../outbox.md` — checkpoint-based `IOutbox` (`getRootData(Epoch, uint256)` confirmed in v5-next's `IOutbox.sol`). - `cli/aztec_cli_reference.md` — fills the empty `aztec start` section; all documented module flags (`--network`, `--node`, `--sequencer`, `--prover-node`, `--prover-broker`, `--prover-agent`, `--p2p-bootstrap`, `--txe`, `--bot`, `--admin-port`) exist in v5-next's `aztec_start_options.ts`. - `cli/aztec_wallet_cli_reference.md` — removes generator-machine leakage in default values (`/home/josh/.aztec/wallet` → `~/.aztec/wallet`, `host.docker.internal` → `localhost`). - `operators/.../governance-participation.md`, `useful-commands.md` — stale governance calls fixed: `M()`→`ROUND_SIZE()`, `N()`→`QUORUM_SIZE()`, `yeaCount(...)`→`signalCount(...)`, `proposals(uint256)`→`getProposal(uint256)`. Confirmed: v5-next `Governance.sol` exposes `getProposal`, `EmpireBase.sol` exposes `ROUND_SIZE`/`QUORUM_SIZE`/`signalCount`; old names are gone. - `tutorials/js_tutorials/token_bridge.md` — Hardhat artifact-path / run-command corrections; the `@aztec/l1-contracts` version note uses the `#include_aztec_version` macro, so it resolves to the v5 version automatically. ## Deliberately excluded — needs a v5-specific numeric pass - **`slashing-configuration.md`** — #24005's numbers are `next`/mainnet-framed and **do not match the live v5 testnet**. The v5 upgrade deploy (`DeployRollupForUpgradeV5.s.sol`, Sepolia branch) sets `localEjectionThreshold = 199,000e18` and slash amounts **100k / 250k / 250k** (AZIP-16 full-stake), versus the doc's 190k + 2,000/5,000. Slot duration (72s) and round size (128 slots) do check out for v5 (confirmed via live testnet block spacing of 72s). This page should be rewritten with verified v5-testnet values in a follow-up rather than copied. Source PRs: [#24005](#24005), [#23830](#23830). No tracked issue is closed by this backport. --- *Created by [claudebox](https://claudebox.work/v2/sessions/59cd7d98b38b9d90) · group: `slackbot`*
(cherry picked from commit ef43cb7)
## Problem The nightly `block-capacity-benchmark` job has been failing fast (~5 min) with every transaction rejected: ``` Invalid tx: Insufficient fee payer balance (required=507345496560000, available=0) ``` All 8 benchmark cases fail on the first `send`. The build succeeds and the network is reachable — this is purely a genesis-funding problem. ## Root cause `block_capacity.test.ts` pays **all** fees through `SponsoredFeePaymentMethod` (account deploys, mints, transfers). The fee payer is therefore the sponsored FPC, not the worker wallets. But `spartan/environments/block-capacity.env` never set `SPONSORED_FPC=true`, so the network was deployed **without funding the sponsored FPC at genesis** — its fee-juice balance is `available=0`, so no tx can be paid for. Every other bench env already sets this flag (`tps-scenario`, `bench-10tps`, `prove-n-tps-fake`, `prove-n-tps-real`, `next-scenario`); `block-capacity.env` was the only one missing it. `network-defaults.yml` documents the flag as *"Fund sponsored FPC with fee juice"*. ## Fix Add `SPONSORED_FPC=true` to `block-capacity.env`. Found while investigating the failures in nightly Spartan Benchmarks run #95.
BEGIN_COMMIT_OVERRIDE fix(world-state): verify archive root in sync_block to reject divergent state (#24229) feat(aztec-node): force sentinel for validators + offense collection (#24202) fix(archiver): treat re-publish of preloaded protocol contracts as idempotent (#24227) test(e2e): wait for L1 txs to be mined in token bridge tutorial test (#24065) fix(spartan): fund sponsored FPC in block-capacity bench env (#24248) END_COMMIT_OVERRIDE
Adds a `network-deployed-version` skill that determines which git commit an Aztec network (next-net, devnet, staging) is running now or was running at a past time T, and whether a specific fix or PR is live on it. Useful for incident triage, confirming a fix reached a network, and identifying the exact deployed commit. Notes on the approach: - Networks deploy via scheduled GitHub Actions workflows whose run history lives in **either** the private `aztec-packages-private` repo (next-net, devnet, staging-internal) or the public `aztec-packages` repo (staging-public). The skill includes a table mapping each network to its workflow id, repo, source branch, and cadence. - The actual investigation (the `gh`/`gcloud` queries that produce run lists and file dumps) runs in a **subagent**, so it doesn't pollute the main conversation context — the main agent only relays the condensed answer.
Adds a rule to the `<writing_comments>` section of the root `CLAUDE.md`: code comments must be understandable from the repo alone. - The repo is public but Linear issues are private, so comments must never cite them (e.g. `// see A-1234`). - Comments must not reference an implementation plan that lives outside the repo (e.g. `// this fixes item 4`, `// tackles section C`) — describe the actual constraint or behavior instead. Placed in the root `CLAUDE.md` (not `yarn-project/`) since it applies to comments in any language across all components, alongside the existing rule against referencing PR/issue numbers.
…fetching blobs (A-1252) (#24247) ## Summary Re-scoped replacement for #24183. During L1 sync the archiver fetched and decoded a checkpoint's blob data **before** validating its committee attestations. A checkpoint published with invalid/insufficient attestations **and** malformed-but-hash-matching blob data threw `BlobDeserializationError` during decode before the invalid-attestation rejection path ran — so it was never recorded as rejected, the L1 sync point never advanced past it, and the archiver re-queried the same L1 blocks every poll and re-threw forever, taking any valid checkpoints in the same batch down with it. Fixes A-1252 ## Fix Validate attestations from L1 **calldata** first. The signed consensus payload (header, archive root, fee asset price modifier) is fully available from calldata without any blob, so a checkpoint with invalid attestations (or one descending from an already-rejected ancestor) is rejected — emitting the same events and persisting the same rejected entries as before — **without fetching its blob**. Blobs are then fetched only for the surviving (attestation-valid, non-descendant) checkpoints. - `validation.ts`: add `validateCheckpointAttestationsFromCalldata` and extract the shared core validator (`validateCheckpointAttestations` now delegates to it). The calldata-built `ConsensusPayload` is identical to the blob-decoded one, so no accept/reject verdict changes. - `l1_synchronizer.ts`: reorder `handleCheckpoints` to validate-then-fetch; `lastSeenCheckpoint` is tracked from calldata since rejected checkpoints are no longer built into `PublishedCheckpoint`s. ## What changed vs #24183 This PR is **narrower** than #24183. It drops all of #24183's blob-failure *skipping* machinery (the "rows 4/5" work: the `BlobFetchOutcome` sentinel, the `canPrune`-gated skip, and the `stopAfterBatch` early loop break). A blob fetch/decode failure for a checkpoint that has **valid** attestations stays **fatal** — it throws and propagates, rolling back the L1 sync point so the fetch is retried on the next iteration, exactly as before #24183. We do **not** skip such checkpoints. ## Deferred to [A-1260](https://linear.app/aztec-labs/issue/A-1260/proposer-should-still-call-prune-if-it-cant-propose) Rather than skipping checkpoints if they are prunable, which would lead to the proposer trying to build on top of an incorrect chain tip from the rollup's point of view (since the validators' archive skips checkpoints that the rollup on L1 still considers valid until it's actually pruned), we keep the archiver stuck in them to prevent advancing past an invalid point. To ensure they get pruned, we add a code path in the sequencer so it attempts a prune even when syncing fails, piggybacking on the "vote if propose failed" path. ## Tests - `archiver-sync.test.ts`: regression test — a checkpoint with invalid attestations **and** malformed blob data is rejected without a `BlobDeserializationError` retry loop, and re-polling the same L1 state is stable. Plus a test that a malformed blob with **valid** attestations still throws (stays fatal), and that a matching local proposed checkpoint is promoted even when its on-chain blob is malformed (blob fetch skipped). - `epochs_invalidate_block.parallel.test.ts`: a canonical invalid-attestations checkpoint is rejected from calldata before any blob fetch — a late observer (blob withheld, promotion disabled) syncs past it without ever ingesting it, and once an honest proposer invalidates and replaces it, the on-chain archive at that number differs from the bad checkpoint's and every node progresses past it. Full `archiver-sync.test.ts` (56) and `validation.test.ts` (14) pass; `yarn build` / `format` / `lint` clean.
#24253) ## Motivation When a node fails `checkSync` during its slot as proposer it cannot build a checkpoint, but it already still casts governance/slashing votes so voting keeps passing even if the chain is damaged. It did not, however, call `prune()`. If bad data wedges the pending chain so proposers can't sync and the proof submission window then expires, nobody prunes and the chain stays stuck. This makes the proposer also prune when it can't propose, so the network can recover from data that is blocking syncing. ## Approach L1 `prune()` is permissionless and idempotent (reverts `Rollup__NothingToPrune` when not prunable, emits `PrunedPending`), so the proposer can call it even with a wedged local view. - Add a `prune` action to the sequencer publisher and an `enqueuePruneIfPrunable(slot)` method that checks `canPruneAtTime` — evaluated at the same L1 timestamp the bundle simulator overrides `block.timestamp` with — and enqueues a `prune` request keyed on the `PrunedPending` event. Fails closed on an RPC error. - In the failed-sync fallback (`tryVoteWhenCannotBuild`, renamed to `tryVoteAndPruneWhenCannotBuild`), enqueue the prune alongside the existing votes and submit them in the same multicall at the target slot. The early-return is relaxed so a send still fires when only a prune (and no votes) was enqueued. - The prune is proposer-gated (reuses the existing `checkCanPropose` gate), batched at the target slot, and relies on permissionless idempotency for HA dedup — no new duty type. The `prune` action is ordered before `propose` in the publisher action list. ## Changes - **sequencer-client**: new `SequencerPublisher.enqueuePruneIfPrunable`; `'prune'` action added before `'propose'`; failed-sync fallback renamed to `tryVoteAndPruneWhenCannotBuild` and now enqueues a prune; shared dedup field renamed `lastSlotForFallbackAction`. - **sequencer-client (tests)**: publisher and sequencer unit tests for the prune fallback (prunable/not-prunable/duplicate/RPC-fail, prune-only send, prune alongside votes). - **end-to-end (tests)**: new `e2e_epochs/epochs_prune_when_cannot_build` that pauses sync so the proposer cannot build, expires the proof window, and asserts the pending tip winds back to proven via the fallback path; minor import normalization in two sibling epochs tests. Fixes A-1260
## What
Cherry-picks `ef43cb7fa20` (*"ci: reusable network-teardown action that
runs on GitHub runners"*) from `next` onto the v5 line.
This is a clean port — `.github/actions/network-teardown/action.yml` is
byte-identical to `next`, and the three workflow files
(`nightly-bench-10tps.yml`, `nightly-spartan-bench.yml`,
`test-network-scenarios.yml`) get the same teardown-refactor hunks
applied on top of their v5-specific config.
## Why
The nightly Spartan Benchmarks run on the private repo's default branch
(`next`), whose `cleanup-*` jobs were refactored on 2026-06-17 to call
the local composite action:
```yaml
- name: Checkout
with:
ref: ${{ needs.select-image.outputs.source_ref }} # the nightly TAG
- name: Cleanup network resources
uses: ./.github/actions/network-teardown
```
The checkout pins `source_ref` (the nightly tag). When the deployed tag
is a **v5** nightly tag, that tree did **not** contain
`.github/actions/network-teardown` (the action was only ever on the
`next` line), so cleanup failed with:
```
Can't find 'action.yml' … under '.github/actions/network-teardown'.
Did you forget to run actions/checkout before running your local action?
```
This left network resources orphaned on every failure (observed in
nightly run #95). Porting the action to `v5-next` means future v5
nightly tags include it, so the cleanup checkout resolves the action
correctly.
`weekly-proving-bench.yml` and `ci3.yml` keep the older `ci3.sh
network-teardown` form — matching `next` exactly (the original commit
did not touch them).
## Routing
Targets `v5-next` directly (not via `merge-train/spartan-v5`) since the
commit already exists on `next`; routing through the merge train would
attempt a redundant port back to private `next`. This PR syncs to
private `v5-next`.
BEGIN_COMMIT_OVERRIDE docs: add network-deployed-version skill (#24238) docs: require self-contained code comments (#24239) fix(archiver): validate checkpoint attestations from calldata before fetching blobs (A-1252) (#24247) fix(sequencer): prune in failed-sync fallback so the chain can recover (#24253) END_COMMIT_OVERRIDE
BEGIN_COMMIT_OVERRIDE fix(telemetry): raise span queue size and make telemetry shutdown idempotent (#24121) END_COMMIT_OVERRIDE
…ling (#24105) ## Problem CI DNS failures (`curl: (6) Could not resolve host …`, e.g. the `chonk_inputs.sh` S3 download) are consistent with AWS's **link-local PPS limit**: traffic to the Amazon resolver (the VPC `.2` address / `169.254.169.253`) is capped at **~1024 packets/sec per ENI**, and over that, packets are silently dropped (`linklocal_allowance_exceeded` in `ethtool -S`). We confirmed the build's DNS path makes this likely: the devbox container **and** nested docker-in-docker both get `nameserver 172.31.0.2` and query the VPC resolver directly — no caching. The host's `systemd-resolved` *is* caching (~48% hit on host-only traffic) but listens on loopback only (`127.0.0.53`), so containers can't use it. With the build's parallelism (and the larger spot instances), the aggregate DNS rate blows past 1024 pps. ## Fix Route container DNS through the host's caching `systemd-resolved`: - Expose its stub on the instance's **primary private IP** (derived from `ip route get`, no IMDS dependency) via `DNSStubListenerExtra` — that's the one address reachable from the devbox container *and* nested dind (unlike the docker0 gateway). - Point containers at it with `docker run --dns <priv_ip>`. Non-loopback nameservers propagate through the nested dockerd, so dind inherits it. Repeat lookups become cache hits and never reach the throttled resolver. ## Safety This can only help, never break resolution: if the IP can't be derived, `systemd-resolved` isn't active, or the stub doesn't come up on the IP (5×0.5s health check via `ss`), `priv_ip` is cleared and `--dns` is omitted — leaving DNS exactly as today. No counter instrumentation included — we're treating link-local throttling as the known cause. (PR #379's `linklocal_allowance_exceeded` logging can confirm before/after if desired.) ## Validation - `bash -n` on `ci3/bootstrap_ec2`; rendered+`bash -n` the injected host-script block; verified the `ip route get` parse and the `${priv_ip:+--dns …}` expansion locally. - Full validation is the PR's own CI run: a build instance that resolves through the cache and (ideally) flat `linklocal_allowance_exceeded`.
BEGIN_COMMIT_OVERRIDE fix(ci3): cache DNS on build instances to dodge link-local PPS throttling (#24105) END_COMMIT_OVERRIDE
Keep the fairies train's it.skip on the 'private state is zero' test (TODO(F-741): expectTokenBalance throws 'No public key registered' under the train's in-progress handshake-discovery changes), while retaining the descriptive comment introduced on v5-next.
AztecBot
added
ci-draft
mverzilli
approved these changes
Jun 25, 2026
This was referenced Jun 25, 2026
mverzilli
added a commit
that referenced
this pull request
Jun 25, 2026
Re-integrates `v5-next` into `merge-train/fairies-v5` so the train→v5-next merge ([#24223](#24223)) is conflict-free again. ### Why the conflict came back The previous catch-up PR ([#24296](#24296)) was **squash-merged**. Squashing flattened the merge into a single commit with only `merge-train/fairies-v5` as a parent, dropping the `v5-next` merge parent. Git therefore no longer saw v5-next's version of `e2e_2_pxes.test.ts` as integrated, so the `it.skip` (train) vs `it()` (v5-next) divergence re-conflicted when #24223 tried the reverse merge. v5-next itself had not moved. ### This PR A real merge commit (`chore: merge v5-next into merge-train/fairies-v5`) with both parents — the current train tip and `v5-next` — so v5-next becomes a recorded ancestor of the train. The single conflict (`e2e_2_pxes.test.ts`) is resolved the same way as before: keep the train's `it.skip` + `TODO(F-741)`. Verified locally that with this merge as the train tip, train→v5-next merges cleanly. ### Important — merge with a merge commit, not squash This must land via **"Create a merge commit"**. Squash or rebase would again flatten away the `v5-next` parent and the conflict on #24223 would return. I'll merge it myself with the merge method so the linkage is preserved. Refs #24223. --- *Created by [claudebox](https://claudebox.work/v2/sessions/110d3b4223158cd4) · group: `slackbot`*
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Brings
v5-nextup to date intomerge-train/fairies-v5, which had stopped merging (PR #24223 was inmergeable_state: dirty). Merge this into the train branch to clear that.The work is split into two commits so the conflict resolution is reviewable on its own:
chore: merge v5-next into merge-train/fairies-v5 (raw merge, conflicts as markers)— the baregit merge origin/v5-nextwith the single conflict committed verbatim (conflict markers in the tree), so reviewers can see exactly what git could not auto-merge.fix: resolve v5-next merge conflict in e2e_2_pxes.test.ts— the only human decision; review this commit in isolation to check the resolution.Conflict
One file:
yarn-project/end-to-end/src/e2e_2_pxes.test.ts(theprivate state is "zero" when PXE does not have the account secret keytest).it.skips this test with aTODO(F-741): under the train's in-progress handshake-discovery work,expectTokenBalance(walletB, token, accountAAddress, 0n)throwsNo public key registeredbecause discovery (get_shared_secrets) needs the scope's keys, which this PXE lacks for a foreign account.v5-nextonly reworded the surrounding comment and kept the test enabled (the test body auto-merged identically).Resolution keeps the train's
it.skip+TODO(F-741)(preserving the train's deliberate divergence) while retainingv5-next's accurate description of what the test does.Refs #24223.
Created by claudebox · group:
slackbot