[cramforce]

A test lane that proves a package behaves the same whether it is compiled natively, run in a Node island, or run under Node, and a list of the cases where it does not.

Why

When scriptc cannot compile a package natively it runs it in an embedded Node. That path is exercised only where a package happens to need it. A comparison with Bun's ahead-of-time compiler forced four packages into islands and one matched Node; a compile-speed scout hit the same family from the other side. scriptc has three ways to run a package and nothing checked that they agree.

What it does

tests/harness/island-conformance.test.ts runs every program three ways:

• under Node, the oracle;
• as its default build (default);
• once per imported package with that package forced into a Node island (island:<package>, the build's --node-island=<package>).

It compares stdout, the exit status and (where the program compares it) stderr for every argv list.

The programs (tests/harness/island-conformance/cases.ts), 234 in all, 482 ways:

• every case of the npm suite (npm/…, 152), with its argv lists;
• the corpus directory programs that carry their own node_modules and need no island today (corpus/…, 68);
• 14 programs over widely used published packages, pinned and trimmed in tests/fixtures/island-conformance (packages/…): commander 15, yaml 2.9, lodash 4.18, chalk 6, semver 7.8, debug 4.4, uuid 14, date-fns 4.4, plus a small process-reader package that reads process state (argv, env, cwd, exitCode, isTTY), one program per aspect. zod and hono are already in the npm suite.

The lane measures the island, so it charges the island only with what the island changes:

• a default build that runs no package in an island and differs from Node is the native tier's difference, which the npm suite and the corpus already pin; that test is skipped with a note;
• a forced island build whose result is byte-equal to the default build's inherits the default build's difference.

The list

tests/harness/island-conformance/known-failures.json lists every island way that does not match Node, keyed by program and way, with a category and a signature (the diagnostic codes of a refused build, or the line that names the difference). The lane passes when exactly the listed failures occur. It fails on a failure the list does not have, on a listed failure that changed, and on a listed failure that no longer occurs. So a fix removes its entries in the same change, and nothing else shrinks or grows the list. SCRIPTC_ISLAND_CONFORMANCE=update rewrites the entries of the programs it ran (the deliberate edit); one racing failure is pinned by category only ("unstable": true).

Scoreboard on main 7dbcdc9dc

node tests/harness/island-conformance/scoreboard.mjs --markdown:

	build	not-islanded	crash	exit	stdout	stderr	total	programs
forced	11	21	0	39	11	1	83	77
automatic	0	0	0	2	0	0	2	2

85 island failures in 79 of 234 programs. forced is a package the lane put in an island; automatic is a default build whose own island fails (yaml, lodash).

What is behind the numbers:

• 21 not-islanded: --node-island=<package> accepted and ignored. Packages scriptc compiles from its own implementation (chalk 5, hot-shots, undici, fs-extra, @hono/node-server), builtin-alias packages, workspace packages compiled from source, types-only packages.
• 17 exit: JSON.stringify of a runtime handle is not supported yet (Follow the value in island method calls and serialize island objects in JSON.stringify #3124's subject).
• About 10 exit: an island object or array meets a program type outside the compile-time crossing rule and the checked conversion throws expected … at $…, got Object: yaml (parse(text) as {…}), lodash (a callback's array parameter), zod and hono cases.
• 8 (exit and stdout): the named run-time refusal this native object cannot cross into a Node island yet.
• 11 build: the island build is refused where the native one is not (SC1090 ×4, SC2004 with SC2013 ×2, SC8006 ×2, SC0001 ×2, SC2013 ×1: versions.sort(semver.compare) with semver in an island).
• Process state: process.argv (commander, packages/process-argv; Node islands: fix error properties, process.argv, thenable awaits and work after 'exit'; pin lifecycle behaviour with 40 corpus programs #3123's subject), process.cwd() and process.exitCode (packages/process-cwd, packages/process-exit-code). env, pid, platform and isTTY agree.
• 11 stdout, exit status matching Node: the silent ones. They are listed one by one in the follow-ups file of the fleet.
• 1 racing failure (npm/node24-promise-timers with its fixture package in an island), pinned by category.

Of the nine popular packages the scouts tried: chalk 6, uuid and debug match Node in an island; commander, yaml, lodash, semver and date-fns do not; zod, which matched in the study's one program, fails in eight of the npm suite's zod cases.

Where it runs

Measured on one machine (Intel Xeon @ 2.50GHz, 8 vCPU, four shards side by side): 214 to 422 s per shard with cold build caches, about 23 minutes of shard time in all; 67 to 177 s per shard warm.

That is too much for every pull request, and most pull requests cannot move the list. So the lane is a CI lane of its own, islands in scripts/ci-lanes.mjs: on when a change touches the island decision (packages/compiler/src/node-islands), the boundary lowering, the island runtime units and host scripts, the libnode bridge, or the lane's own programs and packages; on in the nightly sweep and manual runs; off on pushes to main. island_conformance in ci.yml runs it in four shards and is part of the required check when its lane is on. Linux x64 only: the list pins that platform's results. Without SCRIPTC_ISLAND_CONFORMANCE the file only checks that every entry names a program and a way the lane runs, so the ordinary test lanes are unchanged. Recorded in DECISIONS.md.

For the open island pull requests

The lane job does not run in this pull request: CI takes the lane selector from the base revision, which does not know the islands lane yet. The list here was produced in a Sandbox at 7dbcdc9dc.

Merged into one scratch tree with this lane, #3123, #3124 and #3125 (which carries #3093 and #3005) bring the list from 85 to 73: 12 entries go (nine JSON.stringify ones, packages/process-argv, one checked conversion, one timer), and 16 change to the next failure behind the one they fix. For example commander in an island gets its arguments from #3123 and then fails on the action callback's options record (expected object at $[1], got Object).

Whichever lands second, this lane or one of those, regenerates the list (SCRIPTC_ISLAND_CONFORMANCE=update). After that the lane runs in every pull request that touches island code.

Proof

All on Linux x64 Sandboxes (Intel Xeon @ 2.50GHz, 8 vCPU), Node 24.22.1, a libnode pack built from this branch's bridge, base 7dbcdc9dc.

• The lane, four shards side by side. Update run (cold build caches): 214, 320, 403 and 422 s. Check run against the committed list: green, 67 to 177 s warm. Three more check runs on 827bc093d, where the list was first produced and is identical: all green. 412 ways ran, 70 were skipped (a native difference, or a package the default build already runs in an island).
• Stability. One entry changed its signature between runs and is now pinned by category ("unstable": true); nothing else moved across the runs.
• tests/harness/ci-lanes.test.ts and ci-stack-scope.test.ts: 65 pass, with the new lane in the selector, the workflow outputs and the required check.
• Without SCRIPTC_ISLAND_CONFORMANCE the file runs one test (the list names only programs and ways the lane runs) and skips the rest.
• pnpm lint and pnpm format:check: clean.
• No compiler or runtime source changes: the change is tests, fixtures (about 6 MB of pinned packages, 1,000 files) and CI configuration, plus export on four helpers of npm-suite.ts. The zero-cost gate has nothing to compare.

🤖 Generated with Claude Code

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
scriptc	Ready	Preview, v0	Nov 7, 2026 5:34am UTC

[socket-security]

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff	Package	Supply Chain Security	Vulnerability	Quality	Maintenance	License
	npm/​@​types/​semver@​7.8.0
	npm/​@​types/​lodash@​4.17.27
	npm/​lodash@​4.18.1
	npm/​date-fns@​4.4.0
	npm/​chalk@​6.0.1
	npm/​yaml@​2.9.1
	npm/​uuid@​14.0.3
	npm/​semver@​7.8.6

View full report

[vercel]

Additional findings

1. tests/harness/ci-lanes.test.ts:203 — The push-lane test expects islands=true even though the workflow deliberately emits islands=false, so the CI lane contract test fails on every run.

   

[cramforce]

The macOS failures here are one corpus program, tests/corpus/6125-constants-module.cjs (differential.test.ts and llvm-differential.test.ts; the native stdout is one byte longer than Node's on darwin). #3159 changes no compiler or runtime source and fails on it the same way, so it comes from main, not from this change. These pull requests run the macOS lane only because they carry #3159's CI configuration change.

[cramforce]

The review bot left one finding in a review body here that was not answered before the merge:

tests/harness/ci-lanes.test.ts:203 — The push-lane test expects islands=true even though the workflow deliberately emits islands=false, so the CI lane contract test fails on every run.

Checked on main 78fc2f0800: not real any more. The test expects islands=false for push and merge_group, which is what the workflow prints (a5d2eab8e3 changed the expectation before this PR merged), and tests/harness/ci-lanes.test.ts passes there, 34 tests. Recorded with the other findings of that kind in #3372.