Skip to content

feat(cli): read EQL v3 install SQL from @cipherstash/eql at runtime (CIP-3518)#692

Merged
coderdan merged 1 commit into
mainfrom
feat/eql-v3-source-from-package
Jul 18, 2026
Merged

feat(cli): read EQL v3 install SQL from @cipherstash/eql at runtime (CIP-3518)#692
coderdan merged 1 commit into
mainfrom
feat/eql-v3-source-from-package

Conversation

@coderdan

@coderdan coderdan commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

Why

The v3 install SQL was vendored into packages/cli/src/sql/cipherstash-encrypt-v3.sql (~44k lines) by a build script (gen:eql-v3-sql), sha256-verified against the pinned @cipherstash/eql. That copy defeats the point of depending on @cipherstash/eql:

  • it drifts from the pin on every bump (a manual re-vendor is required), and
  • as ~44k lines of plpgsql in the tree, it made GitHub classify the whole repo as plpgsql (CIP-3518, Supabase-launch milestone).

What

Read it from the package at runtime instead — @cipherstash/eql is the single source of truth (the same readInstallSql() the @cipherstash/stack installer and prisma-next already consume):

  • readV3InstallSql() calls @cipherstash/eql/sql readInstallSql(); the v3 branches of loadBundledEqlSql and EQLInstaller route to it, so resolveBundledFilename is now v2-only.
  • @cipherstash/eql moves devDependency → dependency, externalized in tsup so the real package (and its dist/sql/*.sql) resolves at runtime (the bin already externalizes all deps via skipNodeModulesBundle).
  • Delete cipherstash-encrypt-v3.sql + scripts/build-eql-v3-sql.mjs + the gen:eql-v3-sql script and its CI drift-check.

Net: −44,130 lines. A @cipherstash/eql bump now flows straight through.

No behaviour change

Same one-artifact v3 bundle. There is no Supabase/no-operator-family variant for v3 — the bundle self-adapts to non-superuser environments (its operator-class statements run inside a DO block that catches insufficient_privilege and skips, then disables the ORE domains if absent). The v2 path (vendored .sql, variant selection) is untouched.

Verified

  • readInstallSql() returns the v3 bundle (1.67 MB, correct content) resolved from the installed package.
  • Built bin reads it from an external cwd (verified via the eql migration dry-run on the stacked branch, which shares this installer code).
  • 540 cli unit green (incl. the v3 load path), build + biome clean, pnpm install --frozen-lockfile consistent.

Changeset: stash patch.

Follow-ups (separate): the v2 vendored files (~22k more plpgsql lines) still sit in the tree — fully closing CIP-3518 means dropping v2 or sourcing it likewise; and a repo-wide @cipherstash/eql 3.0.0 → 3.0.1 bump (npm latest) with a prisma-next re-emit.

https://claude.ai/code/session_01MxTTPaPP16m6br7Hoab94w

Summary by CodeRabbit

  • Bug Fixes
    • Improved EQL v3 installation reliability by loading the installation SQL directly from the bundled EQL package.
    • Updated installer messaging to clarify how EQL v3 handles the --latest option.
    • Preserved the existing EQL v2 installation flow.

…CIP-3518)

The v3 install SQL was vendored into packages/cli/src/sql/cipherstash-encrypt-v3.sql
(~44k lines) by a build script (gen:eql-v3-sql), sha256-verified against the pinned
@cipherstash/eql. That copy defeats the point of depending on @cipherstash/eql: it
drifts from the pin on every bump (re-vendor required) and — as ~44k lines of
plpgsql in the tree — made GitHub classify the whole repo as plpgsql (CIP-3518).

Read it from the package at runtime instead:
- `readV3InstallSql()` calls `@cipherstash/eql/sql` `readInstallSql()` (the same
  source the stack + prisma-next consume); the v3 branches of `loadBundledEqlSql`
  and `EQLInstaller` route to it, so `resolveBundledFilename` is now v2-only.
- `@cipherstash/eql` moves devDependency → dependency, externalized in tsup so the
  real package (and its `dist/sql/*.sql`) resolves at runtime.
- Delete `cipherstash-encrypt-v3.sql` + `scripts/build-eql-v3-sql.mjs` + the
  `gen:eql-v3-sql` script and its CI drift-check (tests.yml).

No behaviour change: same one-artifact v3 bundle (self-adapts to non-superuser /
Supabase via the DO-block privilege catch), v2 path untouched. A `@cipherstash/eql`
bump now flows straight through. 557 cli unit green; bundle read verified via the
bin from an external cwd.

Claude-Session: https://claude.ai/code/session_01MxTTPaPP16m6br7Hoab94w
@coderdan
coderdan requested a review from a team as a code owner July 17, 2026 13:53
@changeset-bot

changeset-bot Bot commented Jul 17, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 8d46579

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 11 packages
Name Type
stash Patch
@cipherstash/basic-example Patch
@cipherstash/e2e Patch
@cipherstash/stack Patch
@cipherstash/stack-drizzle Patch
@cipherstash/stack-supabase Patch
@cipherstash/prisma-next Patch
@cipherstash/wizard Patch
@cipherstash/bench Patch
@cipherstash/test-kit Patch
@cipherstash/prisma-next-example Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@coderabbitai

coderabbitai Bot commented Jul 17, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: e5a9b78f-bb93-415f-97a3-c91a192deabb

📥 Commits

Reviewing files that changed from the base of the PR and between 457705c and 8d46579.

⛔ Files ignored due to path filters (1)
  • pnpm-lock.yaml is excluded by !**/pnpm-lock.yaml
📒 Files selected for processing (7)
  • .changeset/eql-v3-source-from-package.md
  • .github/workflows/tests.yml
  • packages/cli/package.json
  • packages/cli/scripts/build-eql-v3-sql.mjs
  • packages/cli/src/installer/index.ts
  • packages/cli/src/sql/cipherstash-encrypt-v3.sql
  • packages/cli/tsup.config.ts
💤 Files with no reviewable changes (2)
  • packages/cli/scripts/build-eql-v3-sql.mjs
  • .github/workflows/tests.yml

📝 Walkthrough

Walkthrough

Changes

EQL v3 runtime source

Layer / File(s) Summary
Runtime package and build wiring
packages/cli/package.json, packages/cli/tsup.config.ts
The CLI uses @cipherstash/eql as a runtime dependency, and the EQL modules are externalized from the build output.
Installer runtime routing
packages/cli/src/installer/index.ts
EQL v3 SQL is read through readInstallSql() with wrapped errors; v2 continues using the bundled-file path, while vendored v3 filename resolution now fails explicitly.
Vendoring workflow removal
.github/workflows/tests.yml, .changeset/eql-v3-source-from-package.md
The vendored SQL drift check is removed and the changeset documents runtime sourcing for v3 and the unchanged v2 path.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Possibly related issues

  • cipherstash/stack issue 690 — Proposes migration generation built on the EQL v3 runtime loader changed here.

Possibly related PRs

Suggested reviewers: calvinbrewer

Sequence Diagram(s)

sequenceDiagram
  participant EQLInstaller
  participant readV3InstallSql
  participant EQLPackage as `@cipherstash/eql`
  EQLInstaller->>readV3InstallSql: request v3 install SQL
  readV3InstallSql->>EQLPackage: readInstallSql()
  EQLPackage-->>readV3InstallSql: return SQL bundle
  readV3InstallSql-->>EQLInstaller: return SQL or wrapped error
Loading
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change: CLI now reads EQL v3 install SQL from @cipherstash/eql at runtime.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/eql-v3-source-from-package

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the stash CLI’s EQL v3 installer path to load the v3 install SQL at runtime from the pinned @cipherstash/eql package, removing the previously vendored ~44k-line SQL artifact and its drift-checking machinery. This keeps @cipherstash/eql as the single source of truth and avoids repo-language misclassification caused by the generated plpgsql file.

Changes:

  • Switch EQL v3 install SQL loading to @cipherstash/eql/sql’s readInstallSql() at runtime, with improved error messaging and v3 routing that bypasses v2-only filename resolution.
  • Promote @cipherstash/eql to a runtime dependency and ensure it stays external in the stash library bundle so its packaged dist/sql/*.sql assets resolve correctly at runtime.
  • Remove the v3 SQL vendoring script and the CI drift-check step; add a patch changeset for stash.

Reviewed changes

Copilot reviewed 6 out of 8 changed files in this pull request and generated no comments.

Show a summary per file
File Description
pnpm-lock.yaml Moves @cipherstash/eql@3.0.0 from devDependency to dependency for stash.
packages/cli/tsup.config.ts Externalizes @cipherstash/eql / @cipherstash/eql/sql in the library bundle so runtime SQL resolution works.
packages/cli/src/installer/index.ts Adds readV3InstallSql() and routes v3 bundled SQL reads through @cipherstash/eql/sql; makes resolveBundledFilename explicitly v2-only.
packages/cli/scripts/build-eql-v3-sql.mjs Removes the v3 vendoring script.
packages/cli/package.json Adds @cipherstash/eql as a dependency and removes the gen:eql-v3-sql script and devDependency entry.
.github/workflows/tests.yml Removes the CI step that regenerated and diff-checked the vendored v3 SQL bundle.
.changeset/eql-v3-source-from-package.md Adds a stash patch changeset documenting the runtime sourcing change and removal of vendored SQL.
Files not reviewed (1)
  • pnpm-lock.yaml: Generated file

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@coderdan

Copy link
Copy Markdown
Contributor Author

Differential Review: cipherstash/stack PR 692

Executive Summary

Severity Count
Critical 0
High 0
Medium 0
Low 0

Overall Risk: Low
Recommendation: Approve
PR: #692
Reviewed range: 457705c7f8787269014527b7d0daec5d953e0c76..8d465791dd873be3fb67ee28ef9c90b554eb7619

No correctness or security defects were found in the reviewed change. The new runtime SQL source is byte-identical to the deleted vendored artifact, the dependency is present in the packed manifest, and the packed ESM and CJS entry points both resolve and read the SQL from an external working directory.

Key metrics:

  • Files analyzed: 8/8 (100%)
  • Production functions changed or added: 3 (readV3InstallSql, EQLInstaller.loadBundledInstallScript, loadBundledEqlSql)
  • Direct production call paths affected: 2 (direct installer and public SQL loader)
  • Test gaps that block merge: 0
  • Security regressions detected: 0
  • CI checks observed: 13/13 successful

What Changed

The PR replaces the CLI's vendored EQL v3 SQL file with a runtime read from the exact-pinned @cipherstash/eql@3.0.0 package. It also removes the generator and CI drift check, promotes @cipherstash/eql from a development dependency to a runtime dependency, and keeps the dependency external in the library bundle.

File Additions Deletions Risk Assessment
.changeset/eql-v3-source-from-package.md 15 0 Low Accurate release note; patch bump present
.github/workflows/tests.yml 0 9 Medium Removed check is obsolete after deleting the vendored copy
packages/cli/package.json 3 4 Medium Exact-pinned package correctly moved to runtime dependencies
packages/cli/scripts/build-eql-v3-sql.mjs 0 57 Low Generator no longer has an output and is safe to remove
packages/cli/src/installer/index.ts 40 4 Medium Both v3 loader paths route to readInstallSql; v2 remains file-backed
packages/cli/src/sql/cipherstash-encrypt-v3.sql 0 44,052 Low Generated artifact removed; replacement is byte-identical
packages/cli/tsup.config.ts 4 1 Medium EQL root and SQL subpath remain external at runtime
pnpm-lock.yaml 3 3 Low Importer classification moves from dev to production only

Total: 65 additions and 44,130 deletions across 8 files, in one commit dated 2026-07-17.

Findings

No actionable defects found.

Non-blocking hardening

Add a packed-artifact regression test that builds and packs stash, installs or lays it out in an isolated temporary consumer, and invokes loadBundledEqlSql({ eqlVersion: 3 }) through both package export conditions. The existing source test at packages/cli/src/__tests__/installer.test.ts:270 proves the v3 SQL reaches the installer, but it does not lock in the packaging contract across package.json, tsup externalization, npm packing, ESM, and CJS. Manual verification during this review proved the current implementation works; automating it would prevent future packaging-only regressions.

Test Coverage Analysis

Existing tests cover the important behavior even though this PR does not add or modify a test file:

Behavior Evidence Result
v2 continues to use vendored SQL packages/cli/src/__tests__/installer.test.ts:118 Pass
v3 installer executes the full bundle packages/cli/src/__tests__/installer.test.ts:270 Pass
v3 bundle contains operator classes packages/cli/src/__tests__/installer.test.ts:289 Pass
v3 --latest remains rejected packages/cli/src/__tests__/installer.test.ts:293 Pass
Full CLI unit suite 46 files / 540 tests Pass
CLI build ESM, CJS, binary, and declarations Pass
CLI Biome check 0 errors; 14 pre-existing warnings Pass
Packed ESM public API 1,666,263-byte SQL; expected markers present Pass
Packed CJS public API 1,666,263-byte SQL; expected markers present Pass

The deleted SQL and @cipherstash/eql@3.0.0/dist/sql/cipherstash-encrypt.sql both hash to:

9311a0c9faeaebe1397fba2abf9e581ce006e94bbd0255596c21e5ada5245afa

This proves the PR changes the source and packaging of the SQL, not its contents.

Blast Radius Analysis

Changed surface Direct callers Blast radius Notes
readV3InstallSql() 2 Medium Feeds direct EQL v3 installation and the public SQL loader
EQLInstaller.loadBundledInstallScript() 1 Low Private method called only by install()
loadBundledEqlSql() 2 repository call sites plus public consumers Medium Repository migration call sites are explicitly v2; default public behavior is v3
Static @cipherstash/eql/sql import All stash module loads Medium Safe because the exact dependency is in the packed production manifest and supports both import/require exports

Failure of the new dependency resolution would prevent v3 installation and could prevent importing the public package entry. That failure mode is handled with an actionable error at packages/cli/src/installer/index.ts:25, and the package-layout checks above confirm the dependency is present and resolvable.

Historical Context

The vendoring path was introduced in July 2026 and subsequently strengthened with a drift gate and hash/version checks. Git history did not show removal of code introduced for a vulnerability, CVE, authentication boundary, or authorization fix. The removed checks protected synchronization of two copies; after this PR there is only one copy, so the synchronization check has no remaining invariant to enforce.

The old generator compared the vendored file against releaseManifest.installSqlSha256. Runtime now trusts the exact-pinned npm package and package-manager integrity checks. Rechecking a manifest shipped inside the same npm artifact would not add an independent trust boundary.

Recommendations

Immediate

  • Approve PR 692.

Before a future packaging refactor

  • Add the isolated packed-artifact ESM/CJS regression test described above.

Analysis Methodology

Strategy: Surgical. The repository contains approximately 777 relevant source files, while this PR changes 8 files and one runtime dependency boundary.

Techniques applied:

  • Retrieved the exact GitHub PR base and head SHAs.
  • Reviewed every changed file and both versions of the installer.
  • Traced both changed loader call paths and public consumers.
  • Examined git history and blame for the deleted generator and drift gate.
  • Compared SHA-256 hashes of the old and new SQL sources.
  • Inspected the published @cipherstash/eql@3.0.0 exports and SQL loader implementation.
  • Built and packed stash, inspected the transformed packed manifest, and exercised ESM and CJS package exports from an isolated temporary consumer.
  • Ran focused and full CLI tests, build, lint, and git diff --check.
  • Checked current GitHub CI conclusions, including CodeQL, OSV, Node 22/24, Bun, E2E, Deno WASM, Drizzle v3, and Supabase v3 jobs.

Limitations:

  • No live PostgreSQL install was run locally. Existing CI reports the Drizzle and Supabase v3 integration jobs successful.
  • External dependency internals were reviewed only as shipped in the installed @cipherstash/eql@3.0.0 package, not as a separate full audit of that repository.

Confidence: High for the PR's changed behavior and package-resolution contract.

Appendix: Verification Results

Focused installer tests: 14/14 passed
Full CLI unit tests:      540/540 passed
CLI build:                passed
CLI Biome check:          passed (14 warnings, 0 errors)
Packed ESM SQL load:      passed
Packed CJS SQL load:      passed
Git diff check:           passed
GitHub CI:                all 13 observed checks successful

@coderdan
coderdan merged commit 175eeb7 into main Jul 18, 2026
14 checks passed
@coderdan
coderdan deleted the feat/eql-v3-source-from-package branch July 18, 2026 10:13
@coderdan coderdan mentioned this pull request Jul 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants