Skip to content

Refresh the Node island conformance list to what main gives - #3293

Merged
ctate merged 2 commits into
mainfrom
test/island-conformance-refresh
Nov 7, 2026
Merged

ctate merged 2 commits into
mainfrom
test/island-conformance-refresh

Conversation

@cramforce

Copy link
Copy Markdown
Contributor

What is wrong

tests/harness/island-conformance/known-failures.json no longer matches main. The Node island conformance lane (tests/harness/island-conformance.test.ts) passes only when exactly the listed failures occur, and on main it fails on eleven entries with nothing applied. Every pull request that touches island code, the npm fixtures or the lane itself runs the lane (scripts/ci-lanes.mjs, islands), so each of them is red for a reason that is main's.

How it went stale

The list was recorded for #3176 at 7dbcdc9dc. Nothing in CI has run the lane since:

The nightly sweep runs every lane, this one included, so it would have shown this at its next run; today's started before #3176 merged.

The eleven entries, by cause

CauseProgram and wayBeforeNow
New text of a failed crossing; the failure is otherwise the samecorpus/7037-js-var-open-writes, island:scalar-marksexpected object at $, got Objectthe same line, followed by from a Node island (an island object stays an island value: declare the island's type or 'unknown' for it, …
npm/hono-zod-default-hook, island:zod[TypeError: expected object at $, got Object]
npm/typed-callbacks, island:typedcbexpected object at $[0], got Object
packages/lodash, default (lodash runs in an island by itself)expected array at $[0], got Object
packages/yaml, default (yaml runs in an island by itself)expected object at $, got Object
Fails at another point of the crossingnpm/zod3-infer, island:zodexpected string at $.name, got undefinedexpected array at $.tags, got Object from a Node island (…
npm/any-chaining, island:mediakitexpected object | undefined at $, got Objectthis native object cannot cross into a Node island yet (scope-B NI-5 exports native values by reference)
Refused at build time (SC8010) instead of failing or diverging when it runsnpm/island-json-stringify, island:arrpackexit 1: JSON.stringify of a runtime handle is not supported yetbuild: SC8010
npm/zod3-adoption, island:zodstdout differs from Node's (a silent difference)build: SC8010
npm/zod4-boundary, island:zodbuild: SC1090build: SC1090,SC8010
Matches Node nownpm/decl-pass-through, island:ref-marksexit 1: expected object at $, got Objectremoved from the list

86 entries become 85 (node tests/harness/island-conformance/scoreboard.mjs: 83 forced in 77 programs, 2 automatic). No entry is new.

What changes

Only the list, regenerated by the lane itself (SCRIPTC_ISLAND_CONFORMANCE=update). No source, test or CI change.

Proof

Linux x64 Sandbox, 32 vCPU, Intel Xeon @ 2.90GHz, Node 24.22.1, a libnode pack built from the commit under test.

  • Regenerated at 4935b7869: 413 tests pass in update mode. The same list, byte for byte, came out of an independent run at dfa7753d3.
  • A second pass in check mode (SCRIPTC_ISLAND_CONFORMANCE=1) against the regenerated list, same commit: 413 tests pass. Before the change the lane fails on exactly the eleven entries above (measured at dfa7753d3).
  • main has merged commits since 4935b7869; none of them changes a path the islands lane is selected by (scripts/ci-lanes.mjs). This pull request changes the list, so its own CI runs the lane against the merge with main's head.
  • The zero-cost gate, the test lanes and the native bootstraps do not apply: no compiler, runtime or test source changes.

Follow-up, not in this change

How to keep the list from going stale again is written up for the maintainers of the lane (the lane on pushes to main that change island code, selected from the push's own diff; or the merge queue #3119 prepares). It would edit the lines of the lanes step that #3119 and #3121 also edit, so it is left until those merge.

🤖 Generated with Claude Code

The list was recorded for #3176 at 7dbcdc9. The island crossing changes
that merged around it changed what several of its programs do, and the
lane has not run on main since: pushes to main do not run it, and #3159
and #3176 were selected by a base revision that predates the lane, so
their own island jobs were skipped. At main's head the lane fails on
eleven entries with nothing applied.

Regenerated with SCRIPTC_ISLAND_CONFORMANCE=update at 4935b78 (Linux
x64, a libnode pack built from that commit), and the same list came out
at dfa7753:

- new text of a failed crossing ("…, got Object from a Node island (an
  island object stays an island value: …"), same failure otherwise:
  corpus/7037-js-var-open-writes, npm/hono-zod-default-hook,
  npm/typed-callbacks, and the default builds of packages/lodash and
  packages/yaml;
- npm/zod3-infer fails at another member, with that text, and
  npm/any-chaining now stops at "this native object cannot cross into a
  Node island yet";
- refused at build time with SC8010 instead of failing or diverging when
  they run: npm/island-json-stringify and npm/zod3-adoption; and
  npm/zod4-boundary reports SC8010 beside its SC1090;
- npm/decl-pass-through with ref-marks in an island matches Node and
  leaves the list.

86 entries become 85. Only the list changes.

Co-Authored-By: Claude Code <noreply@anthropic.com>
@vercel

vercel Bot commented Nov 7, 2026 •

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
scriptc Ready Ready Preview, v0 Nov 7, 2026 11:06am UTC

@cramforce

Copy link
Copy Markdown
Contributor Author

Check of the eleven re-pinned entries: none of them pins a regression

Two of the entries change category to build (npm/zod3-adoption, npm/island-json-stringify), so I checked that this list does not write down a program that used to work. It does not: no way that built and matched Node before has stopped doing so, and the default build of every one of these programs is unchanged.

How it was measured. Each program ran under Node, as its default build and with the package forced into an island, on three built trees (Linux x64 Sandboxes, Node 24.22.1, each tree with a libnode pack from its own bridge): a09254dd3 (main just before #3161), f2b336d18 (#3161's merge commit, which also brought #3005's crossing rule and #3093's live views to main) and main's head 50bcfe1b3. All nine entries whose programs exist on both sides change exactly between the first two. The lane itself, narrowed to these eleven programs (22 tests) at 50bcfe1b3: 22 pass with this PR's list, and exactly the eleven fail with main's list.

Entry Pinned on main At main's head Changed by Verdict
corpus/7037-js-var-open-writes, island:scalar-marks exit: expected object at $, got Object exit: the same line plus from a Node island (an island object stays an island value: … #3161 (message text) Expected: same failure at the same line
npm/typed-callbacks, island:typedcb exit: expected object at $[0], got Object exit: the same line plus the new text #3161 Expected: same failure
packages/lodash, default exit: expected array at $[0], got Object exit: the same line plus the new text #3161 Expected: same failure
packages/yaml, default exit: expected object at $, got Object exit: the same line plus the new text #3161 Expected: same failure
npm/hono-zod-default-hook, island:zod exit: [TypeError: expected object at $, got Object] exit: TypeError: expected object at $, got Object from a Node island (… #3161 for the text; #3231 for the brackets (the caught error keeps its frames, so console.error prints the stack form; placed between acb040e3c and b3b462b01) Expected: same five failed requests
npm/zod3-infer, island:zod exit: expected string at $.name, got undefined exit: expected array at $.tags, got Object from a Node island (… #3161 (by the commit; most likely its fix that makes { ...islandObject } copy the object's keys, which I did not isolate) Expected: gets one member further, then meets an island array where a native array is declared
npm/any-chaining, island:mediakit exit: expected object | undefined at $, got Object exit: this native object cannot cross into a Node island yet (… #3161 (#3093: a program Record over an island object is a live view) Expected: prints one more correct line (resp-42), then passes a native URL to the island (source: mediaSource(true)), which no tree accepts yet
npm/decl-pass-through, island:ref-marks exit: expected object at $, got Object matches Node, entry removed #3161 (#3093) Expected: a fix
npm/zod4-boundary, island:zod build: SC1090 build: SC1090,SC8010 #3161 (#3005's rule 1b) Expected: the SC1090 (items.reduce, line 26) is unchanged; the SC8010 is new, at line 129, const [label, count] = Pair.parse(["n", 4])
npm/zod3-adoption, island:zod stdout, a silent difference: kept … false build: SC8010 #3161 (#3005's rule 1b, #3093) Expected, and better: see below
npm/island-json-stringify, island:arrpack exit: JSON.stringify of a runtime handle is not supported yet build: SC8010 #3161 (#3005's rule 1b) Expected: see below

The two entries that became build

npm/zod3-adoption. The forced build is refused at main.ts:46:1 and 47:1:

error SC8010: an island object typed as the program record '{ id: number; tags: string[]; }' (keep the handle by declaring the island's type or 'unknown'/'any', or copy it explicitly with 'structuredClone(v)') cannot cross into or out of a Node island yet
  46 | pair[0].tags.push("b");

Before #3161 this way built and printed two wrong lines without any error (kept … false and the tuple line: the program wrote into copies, not into the objects zod holds). Now the first of the two is right and the second is refused by name: with lines 46 to 48 removed, the forced build matches Node on every line. The default build matches Node on all three trees.

npm/island-json-stringify. The forced build is refused at main.ts:18:28:

error SC8010: an island object typed as the program record '[number, string]' (keep the handle by …) cannot cross into or out of a Node island yet
  18 | console.log(JSON.stringify(pair()));

This way never matched Node. Before #3161 it built and died on its first output line (JSON.stringify of a runtime handle is not supported yet); with line 18 removed it still builds and still dies there, so the old failure is intact behind the refusal. Line 18 by itself was never a working line either: reduced to console.log(JSON.stringify(pair())) over a package that returns [7, "seven"] declared [number, string], the forced build before #3161 compiled and ended with Uncaught TypeError: expected array at $, got Object; since #3161 it is the SC8010 above. The same holds for const [n, s] = pair() and for passing pair() to a program function; const t = pair(); t[0] matches Node before and after. The default build matches Node on all three trees.

So in both entries a failure at run time (one of them silent) became a named refusal at build time, which is the direction #3005 chose on purpose (rule 1b: the copy out of an island is never implicit).

What is left open by these entries, not by this PR

Not run: the lane's other 461 tests at main's head (this PR's own CI runs them), the sanitized lane, macOS.

@ctate
ctate merged commit 97d3722 into main Nov 7, 2026
76 of 80 checks passed
@ctate
ctate deleted the test/island-conformance-refresh branch November 7, 2026 13:13
cramforce added a commit that referenced this pull request Nov 7, 2026
main refreshed the island conformance list (#3293); nothing conflicts.

Co-Authored-By: Claude Code <noreply@anthropic.com>
cramforce added a commit that referenced this pull request Nov 7, 2026
…with main

Value matrix, 3,680 to 3,668 pinned observations: the 12 that go are `.constructor` reads of a class value that main no longer refuses with SC2020 and that match Node now (`class.*` and `error-class.control`, one of them a control program without an island). main's table still pins them, so the matrix test fails there.

Island conformance, 85 to 82 known failures. Two are this branch: `corpus/7531-buffer-constructor-dependency` (island:legacy-buf) and `corpus/7748-npm-any-bytes-receiver` (island:chunk-buffers) match Node now. The third is main's: `packages/semver` (island:semver, build SC2013) matches Node on main 50bcfe1 without this branch as well, and the refreshed list of #3293 still has it. Everything else in the list is what #3293 put there: the list regenerated on main 50bcfe1 alone and on this tree differ by the two entries above only.

Co-Authored-By: Claude Code <noreply@anthropic.com>
cramforce added a commit that referenced this pull request Nov 7, 2026
#3293 refreshed tests/harness/island-conformance/known-failures.json on
main. The earlier merge of main into this branch had regenerated the same
list inside the merge commit; the two differ in one entry
(npm/hono-zod-default-hook, which #3293 corrected). main's list is taken
as it is: this PR changes the native compiler, and the lane builds with
the Node-hosted one, so it moves no entry of its own.

Co-Authored-By: Claude Code <noreply@anthropic.com>
cramforce added a commit that referenced this pull request Nov 7, 2026
Main's #3293 refreshes the island conformance list. Nothing conflicted.

Co-Authored-By: Claude Code <noreply@anthropic.com>
cramforce added a commit that referenced this pull request Nov 7, 2026
Regenerated on top of main's refreshed list (#3293), with a libnode pack
built from this tree's bridge: 85 entries become 67. Eighteen programs
now match Node, most of them through JSON.stringify of an island value
and a value an island made passing the checked conversions; six fail at
a later point than before. None is new.

Co-Authored-By: Claude Code <noreply@anthropic.com>

This branch was successfully deployed

1 active deployment
Preview — 30d3e3f0 Deployed Nov 7, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants