zlib: add ZIP archive support to zlib (ZipFile,ZipBuffer,ZipEntry) by pipobscure · Pull Request #64339 · nodejs/node

pipobscure · 2026-07-07T14:54:34Z

Summary

Add ZIP archive support to the node:zlib module through three classes
and a set of helpers:

ZipEntry: a single archive member, with buffered reads (content()),
bounded-memory streaming reads (contentIterator()), and
create()/createStream() for building members.
ZipFile: random access to an archive backed by a file descriptor,
reading members lazily without retaining their content and writing
new members in place; opened with open()/openSync().
ZipBuffer: a zero-copy, in-memory view over an archive already held
in a Buffer.

createZipArchive() serializes a sequence of entries into an archive
byte stream, and setMaxZipContentSize() bounds the default in-memory
decompression size. Every operation has both an asynchronous and a
synchronous form.

P.S.: my CLA should be on file and I wrote this myself so COO is declared

codecov · 2026-07-07T16:32:30Z

Codecov Report

❌ Patch coverage is 98.48178% with 60 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.36%. Comparing base (1912893) to head (dac8770).
⚠️ Report is 14 commits behind head on main.

Files with missing lines	Patch %	Lines
lib/internal/zip/file.js	97.12%	20 Missing ⚠️
lib/internal/zip/entry.js	98.50%	12 Missing and 2 partials ⚠️
lib/internal/zip/headers.js	97.17%	14 Missing ⚠️
lib/internal/zip/fs-util.js	94.00%	4 Missing and 5 partials ⚠️
lib/internal/zip/compression.js	99.33%	2 Missing ⚠️
lib/internal/zip/archive.js	99.65%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #64339      +/-   ##
==========================================
+ Coverage   90.25%   90.36%   +0.11%     
==========================================
  Files         741      755      +14     
  Lines      241165   245168    +4003     
  Branches    45423    46227     +804     
==========================================
+ Hits       217673   221558    +3885     
- Misses      15069    15168      +99     
- Partials     8423     8442      +19

Files with missing lines	Coverage Δ
lib/internal/errors.js	`97.71% <100.00%> (+<0.01%)`	⬆️
lib/internal/zip.js	`100.00% <100.00%> (ø)`
lib/internal/zip/binary.js	`100.00% <100.00%> (ø)`
lib/internal/zip/buffer.js	`100.00% <100.00%> (ø)`
lib/internal/zip/constants.js	`100.00% <100.00%> (ø)`
lib/internal/zip/content-size.js	`100.00% <100.00%> (ø)`
lib/internal/zip/dos.js	`100.00% <100.00%> (ø)`
lib/internal/zip/extra-fields.js	`100.00% <100.00%> (ø)`
lib/internal/zip/header-builders.js	`100.00% <100.00%> (ø)`
lib/zlib.js	`98.36% <100.00%> (+0.03%)`	⬆️
... and 6 more

... and 45 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Codecov flagged low patch coverage on lib/internal/zip.js and lib/internal/vfs/providers/archive.js in nodejs#64339. Add tests exercising Zip64 extra-field parsing, DOS date/time edge cases, streaming-entry state guards, decodeMemberStream/decodeMemberSync's duplicated error branches, the ZipBuffer/ZipFile iteration protocols, several on-disk ZipFile error paths, and ArchiveFileHandle's direct read/write/stat/ truncate surface plus a handful of provider-level error branches the existing tests didn't reach.

bakkot · 2026-07-07T23:15:44Z

See also #45651

pipobscure · 2026-07-08T05:42:44Z

See also #45651

Thanks @bakkot !!!

I think the time has come for it on the one hand, and on the other I added some „motivation“ links earlier. Here some more detail:

Based on this, we can modify the loader to directly load from an archive. If we do that, we get application bundles. pipobscure#5 & pipobscure#6

Which can then in turn be used to easily create application bundles: https://github.com/pipobscure/experimental-sea

So the world had changed enough that it‘s worth proposing again.

mcollina · 2026-07-08T13:00:36Z

I like this a lot. It's likely better to split this into 2 PRs, one for Zip support and one for VFS-Zip, so the Zip support could theoretically be backportable on its own.

pipobscure · 2026-07-08T16:32:01Z

As per @mcollina I split this into two PRs.

I have the vfs-provider ready to go as follow up one (as it depends on this being merged)

I also made sure that the streaming side of things was actually as clean as I intended, and added a few more tests.

bakkot · 2026-07-08T17:47:34Z

Might be worth borrowing some tests from other projects: e.g. python, go, info-zip. There's a lot of edge cases with zip files. (Python's tests are larger than this whole PR put together.)

Not all directly applicable because this doesn't include the ability to extract to disk, fortunately.

pipobscure · 2026-07-09T05:17:34Z

Might be worth borrowing some tests from other projects

I've added a bunch more tests and compared to what go/python/info-zip do. (info-zip is cli exercise, so it's not entirely clear what gets tested).

I also gave zip.js an once over and concluded that it was too large a file (it originated from separate files that I've had for ages combined into one).

So I split it back out so it would be easier to review. And gave that another look.

mcollina

LGTM

I think it should also emit a usage warning that this API is experimental.

Possibly the Zip should be disposable/asyncdisposable.

nodejs-github-bot · 2026-07-09T10:43:43Z

CI: https://ci.nodejs.org/job/node-test-pull-request/74687/

pipobscure · 2026-07-09T18:34:20Z

Question: is there anything outstanding at this point that I should do on this. I see 4 failing CI items, but I don’t have permissions to look at them. If there is something I can/should do please let me know.

richardlau · 2026-07-09T19:36:51Z

Question: is there anything outstanding at this point that I should do on this. I see 4 failing CI items, but I don’t have permissions to look at them. If there is something I can/should do please let me know.

@pipobscure Yes, unfortunately the failures look related to this PR:

https://ci.nodejs.org/job/node-test-commit-linux-containered/nodes=ubuntu2404_sharedlibs_withoutssl_x64/57337/console

11:56:31 not ok 4612 parallel/test-zlib-zip
11:56:31   ---
11:56:31   duration_ms: 218.98500
11:56:31   severity: fail
11:56:31   exitcode: 1
11:56:31   stack: |-
11:56:31     Test failure: 'an incompressible or empty entry falls back to store'
11:56:31     Location: test/parallel/test-zlib-zip.js:57:1
11:56:31     Error [ERR_NO_CRYPTO]: Node.js is not compiled with OpenSSL crypto support
11:56:31         at assertCrypto (node:internal/util:242:11)
11:56:31         at node:crypto:36:1
11:56:31         at BuiltinModule.compileForInternalLoader (node:internal/bootstrap/realm:409:7)
11:56:31         at BuiltinModule.compileForPublicLoader (node:internal/bootstrap/realm:348:10)
11:56:31         at loadBuiltinModule (node:internal/modules/helpers:133:7)
11:56:31         at loadBuiltinWithHooks (node:internal/modules/cjs/loader:1305:15)
11:56:31         at Module._load (node:internal/modules/cjs/loader:1361:50)
11:56:31         at wrapModuleLoad (node:internal/modules/cjs/loader:261:19)
11:56:31         at Module.require (node:internal/modules/cjs/loader:1682:12)
11:56:31         at require (node:internal/modules/helpers:196:16) {
11:56:31       code: 'ERR_NO_CRYPTO'
11:56:31     }
11:56:31     
11:56:31     Test failure: 'an incompressible entry with method zstd falls back to store'
11:56:31     Location: test/parallel/test-zlib-zip.js:87:1
11:56:31     Error [ERR_NO_CRYPTO]: Node.js is not compiled with OpenSSL crypto support
11:56:31         at assertCrypto (node:internal/util:242:11)
11:56:31         at node:crypto:36:1
11:56:31         at BuiltinModule.compileForInternalLoader (node:internal/bootstrap/realm:409:7)
11:56:31         at BuiltinModule.compileForPublicLoader (node:internal/bootstrap/realm:348:10)
11:56:31         at loadBuiltinModule (node:internal/modules/helpers:133:7)
11:56:31         at loadBuiltinWithHooks (node:internal/modules/cjs/loader:1305:15)
11:56:31         at Module._load (node:internal/modules/cjs/loader:1361:50)
11:56:31         at wrapModuleLoad (node:internal/modules/cjs/loader:261:19)
11:56:31         at Module.require (node:internal/modules/cjs/loader:1682:12)
11:56:31         at require (node:internal/modules/helpers:196:16) {
11:56:31       code: 'ERR_NO_CRYPTO'
11:56:31     }
11:56:31     
11:56:31   ...

https://ci.nodejs.org/job/node-test-binary-windows-js-suites/41560/RUN_SUBSET=1,nodes=win11-COMPILED_BY-vs2022_clang/console

12:42:50 not ok 1287 parallel/test-zlib-zip-files
12:42:50   ---
12:42:50   duration_ms: 2926.03100
12:42:50   severity: fail
12:42:50   exitcode: 1
12:42:50   stack: |-
12:42:50     Test failure: 'zipFiles archives files, directories, contents and Unix mode from disk'
12:42:50     Location: test\parallel\test-zlib-zip-files.js:33:1
12:42:50     AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:
12:42:50     
12:42:50     438 !== 416
12:42:50     
12:42:50         at TestContext.<anonymous> (c:\workspace\node-test-binary-windows-js-suites\node\test\parallel\test-zlib-zip-files.js:44:12)
12:42:50         at process.processTicksAndRejections (node:internal/process/task_queues:104:5)
12:42:50         at async Test.run (node:internal/test_runner/test:1389:7)
12:42:50         at async startSubtestAfterBootstrap (node:internal/test_runner/harness:387:3) {
12:42:50       generatedMessage: true,
12:42:50       code: 'ERR_ASSERTION',
12:42:50       actual: 438,
12:42:50       expected: 416,
12:42:50       operator: 'strictEqual',
12:42:50       diff: 'simple'
12:42:50     }
12:42:50     
12:42:50   ...
12:43:06  failed 10 out of 10
12:43:06 not ok 1288 parallel/test-zlib-zip-interop
12:43:06   ---
12:43:06   duration_ms: 5045.77800
12:43:06   severity: fail
12:43:06   exitcode: 1
12:43:06   stack: |-
12:43:06     Test failure: 'an archive written by Python's zipfile module is readable by ZipFile'
12:43:06     Location: test\parallel\test-zlib-zip-interop.js:98:1
12:43:06     AssertionError [ERR_ASSERTION]: Python was not found; run without arguments to install from the Microsoft Store, or disable this shortcut from Settings > Apps > Advanced app settings > App execution aliases.
12:43:06     
12:43:06     
12:43:06     9009 !== 0
12:43:06     
12:43:06         at TestContext.<anonymous> (c:\workspace\node-test-binary-windows-js-suites\node\test\parallel\test-zlib-zip-interop.js:114:12)
12:43:06         at async Test.run (node:internal/test_runner/test:1389:7)
12:43:06         at async Test.processPendingSubtests (node:internal/test_runner/test:960:7) {
12:43:06       generatedMessage: false,
12:43:06       code: 'ERR_ASSERTION',
12:43:06       actual: 9009,
12:43:06       expected: 0,
12:43:06       operator: 'strictEqual',
12:43:06       diff: 'simple'
12:43:06     }
12:43:06     
12:43:06   ...

pipobscure · 2026-07-09T21:37:32Z

Thank you!!!

This looks very much like we are missing CRC32 which we need. It looks like that‘s imported/made available via OpenSSL (this is just a preliminary, on my iPhone diagnosis and I‘ll lool closer tomorrow).
If my first guess substantiates, how do we want to solve that?

a) gate Zip support on OpenSSL availability
b) do a fallback CRC32 implementation (which I still have from the original, which was written when browsers did not expose CRC32)
c) something else?

That wasn’t it.

Add ZIP archive support to the node:zlib module through three classes and a set of helpers: - ZipEntry: a single archive member, with buffered reads (content()), bounded-memory streaming reads (contentIterator()), and create()/createStream() for building members. - ZipFile: random access to an archive backed by a file descriptor, reading members lazily without retaining their content and writing new members in place; opened with open()/openSync(). - ZipBuffer: a zero-copy, in-memory view over an archive already held in a Buffer. createZipArchive() serializes a sequence of entries into an archive byte stream, and setMaxZipContentSize() bounds the default in-memory decompression size. Every operation has both an asynchronous and a synchronous form. Signed-off-by: Philipp Dunkel <pip@pipobscure.com>

pipobscure · 2026-07-10T10:31:42Z

@richardlau it was the use of random bytes in tests (which failed when crypto was not available to generate them) and to lax of a check for python availability (python3 exists as a stub on Windows) which is used to validate against the python libraries, and the unix-mode check on Windows which Windows simply doesn't have and translates reas-only to 0x666 instead of the expected (unix correct) 0x644. There I did an arch==windows check in the test as the OS simply doesn't support precise unix-modes.

pipobscure · 2026-07-10T10:33:10Z

@mcollina I validated the disposable functionality to make sure everything that has resources (ZipFile has an fd) and owns streams (that may have fds) (ZipEntry) disposes them correctly.

mcollina

lgtm

nodejs-github-bot · 2026-07-10T13:05:47Z

CI: https://ci.nodejs.org/job/node-test-pull-request/74719/

pipobscure · 2026-07-10T14:19:12Z

Well, I'd love to fix the CI issues, but I can't access the putput so I have no clue how to figure out what's the cause. Please ping (@) me if there is anything I can do.

mcollina · 2026-07-10T15:04:44Z

@pipobscure:

---
duration_ms: 1044.392
exitcode: 1
severity: fail
stack: |-
  Test failure: 'an abandoned createZipArchive() destroys the sources of entries it never reached'
  Location: test/parallel/test-zlib-zip-files.js:147:1
  AssertionError [ERR_ASSERTION]: 1 source streams left open

  1 !== 0

      at TestContext.<anonymous> (/home/iojs/build/workspace/node-test-commit-linux-containered/test/parallel/test-zlib-zip-files.js:160:12)
      at async Test.run (node:internal/test_runner/test:1389:7)
      at async Test.processPendingSubtests (node:internal/test_runner/test:960:7) {
    generatedMessage: false,
    code: 'ERR_ASSERTION',
    actual: 1,
    expected: 0,
    operator: 'strictEqual',
    diff: 'simple'
  }
...

Also, there seems to be a conflict:

15:06:16 + git status
15:06:16 HEAD detached at dac877014b7
15:06:16 nothing to commit, working tree clean
15:06:16 + git rev-parse HEAD
15:06:16 dac877014b7e8bb5514153686e23099b64d9a959
15:06:16 + git rev-parse 116302d950c97224a91bb8dace74c2578a04c0f0
15:06:16 116302d950c97224a91bb8dace74c2578a04c0f0
15:06:16 + '[' -z dac877014b7e8bb5514153686e23099b64d9a959 ']'
15:06:16 + echo dac877014b7e8bb5514153686e23099b64d9a959
15:06:16 + grep -qE '^[0-9a-fA-F]+$'
15:06:16 ++ git rev-parse HEAD
15:06:16 ++ git rev-parse dac877014b7e8bb5514153686e23099b64d9a959
15:06:16 + '[' dac877014b7e8bb5514153686e23099b64d9a959 '!=' dac877014b7e8bb5514153686e23099b64d9a959 ']'
15:06:40 + '[' -n 116302d950c97224a91bb8dace74c2578a04c0f0 ']'
15:06:40 + git rebase --committer-date-is-author-date 116302d950c97224a91bb8dace74c2578a04c0f0
15:06:40 Rebasing (1/47026)
15:06:40 Rebasing (2/47026)
15:06:40 Auto-merging Makefile
15:06:40 CONFLICT (add/add): Merge conflict in Makefile

nodejs-github-bot added lib / src Issues and PRs related to general changes in the lib or src directory. needs-ci PRs that need a full CI run. labels Jul 7, 2026

pipobscure changed the title ~~zlib, vfs: add ZIP archive read/write support and an archive vfs provider~~ zlib, vfs: add ZIP archive support and an archive vfs provider Jul 7, 2026

This comment was marked as outdated.

Sign in to view

pipobscure force-pushed the ziparchives branch 3 times, most recently from 0f2b879 to 4258e94 Compare July 7, 2026 19:20

pipobscure force-pushed the ziparchives branch 2 times, most recently from 2a56a15 to 27ab6b4 Compare July 7, 2026 21:54

jasnell reviewed Jul 8, 2026

View reviewed changes

Comment thread doc/api/vfs.md Outdated

JakobJingleheimer self-requested a review July 8, 2026 08:40

pipobscure force-pushed the ziparchives branch 2 times, most recently from f157686 to 233f865 Compare July 8, 2026 11:29

mcollina reviewed Jul 8, 2026

View reviewed changes

Comment thread doc/api/zlib.md

This comment was marked as outdated.

Sign in to view

pipobscure force-pushed the ziparchives branch from 233f865 to 26fb1ab Compare July 8, 2026 16:22

pipobscure changed the title ~~zlib, vfs: add ZIP archive support and an archive vfs provider~~ zlib, vfs: add ZIP archive support to zlib Jul 8, 2026

mcollina reviewed Jul 8, 2026

View reviewed changes

Comment thread test/parallel/test-zlib-zip-vfs.js Outdated

pipobscure force-pushed the ziparchives branch from 26fb1ab to 701dffd Compare July 8, 2026 16:44

pipobscure changed the title ~~zlib, vfs: add ZIP archive support to zlib~~ zlib: add ZIP archive support to zlib (ZipFile,ZipBuffer,ZipEntry) Jul 8, 2026

pipobscure force-pushed the ziparchives branch from 701dffd to 03266e9 Compare July 9, 2026 05:10

trivikr added the zlib Issues and PRs related to the zlib subsystem. label Jul 9, 2026

mcollina approved these changes Jul 9, 2026

View reviewed changes

mcollina added the request-ci Add this label to start a Jenkins CI on a PR. label Jul 9, 2026

github-actions Bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Jul 9, 2026

pipobscure force-pushed the ziparchives branch from 03266e9 to dac8770 Compare July 10, 2026 10:30

mcollina approved these changes Jul 10, 2026

View reviewed changes

mcollina added semver-minor PRs that contain new features and should be released in the next minor version. request-ci Add this label to start a Jenkins CI on a PR. labels Jul 10, 2026

github-actions Bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Jul 10, 2026

bakkot mentioned this pull request Jul 10, 2026

Adds zip support to the zlib module #45651

Open

4 tasks

Uh oh!

Uh oh!

Conversation

pipobscure commented Jul 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Uh oh!

codecov Bot commented Jul 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

This comment was marked as outdated.

bakkot commented Jul 7, 2026

Uh oh!

Uh oh!

pipobscure commented Jul 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mcollina commented Jul 8, 2026

Uh oh!

Uh oh!

This comment was marked as outdated.

pipobscure commented Jul 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

bakkot commented Jul 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pipobscure commented Jul 9, 2026

Uh oh!

mcollina left a comment

Choose a reason for hiding this comment

Uh oh!

nodejs-github-bot commented Jul 9, 2026

Uh oh!

pipobscure commented Jul 9, 2026

Uh oh!

richardlau commented Jul 9, 2026

Uh oh!

pipobscure commented Jul 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pipobscure commented Jul 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pipobscure commented Jul 10, 2026

Uh oh!

mcollina left a comment

Choose a reason for hiding this comment

Uh oh!

nodejs-github-bot commented Jul 10, 2026

Uh oh!

pipobscure commented Jul 10, 2026

Uh oh!

mcollina commented Jul 10, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

pipobscure commented Jul 7, 2026 •

edited

Loading

codecov Bot commented Jul 7, 2026 •

edited

Loading

pipobscure commented Jul 8, 2026 •

edited

Loading

pipobscure commented Jul 8, 2026 •

edited

Loading

bakkot commented Jul 8, 2026 •

edited

Loading

pipobscure commented Jul 9, 2026 •

edited

Loading

pipobscure commented Jul 10, 2026 •

edited

Loading