Skip to content

Train perf 5: the variadic check skips its walk for typed programs, CI's self-compile check fails when a listed compiler class becomes a dynamic value, and the LLVM emitter leaves the dynamic tier of the compiler's own build (4.7% fewer functions) - #3406

Merged
ctate merged 7 commits into
mainfrom
train/perf-5
Nov 8, 2026

Conversation

@cramforce

@cramforce cramforce commented Nov 8, 2026 •

Copy link
Copy Markdown
Contributor

This is a landing train with one member, validated as a whole in Vercel Sandboxes: four commits about the compiler compiling its own source. One removes a whole-program walk that typed programs never needed. One makes CI's own self-compile job fail when a compiler class that must stay typed becomes a dynamic value, and say where. Two use that report: the LLVM emitter and one smaller class leave the dynamic tier of the compiler's own build, which then has 4.7 % fewer functions. The member's signed commits are merged with main, not rebased.

The three numbers

  • Run time. Nothing changes: every program's IR, LLVM and executable is byte-identical (5,549 programs).
  • Compile time. The check that runs when lowering ends takes 21 ms instead of 1,150 ms on the compiler's own source (means of three alternating pairs of the compiler lowering itself, one build at a time on an idle 8-vCPU Sandbox; the difference is 1,129 ms, 95 % CI [920, 1,338]): 0.17 % of that frontend's 697 s. End to end it does not show: the gate's self-compile row is −0.24 % [95 % CI −1.33 %, +1.00 %], inconclusive; benchmarks −0.32 % [−1.03 %, +0.50 %], fixtures −0.74 % [−1.50 %, +0.02 %].
  • Size. For programs, unchanged. The compiler's own source, compiled by itself: 133,132 functions (−4.7 %), 1,008,401,189 LLVM bytes (−1.7 %), 11,682 functions of the dynamic tier (−2,556), from the gate's self-compile budget lines against main 6da2be6a1b (139,652 functions, 1,025,578,714 bytes, 14,238). On the merged tree the last commit takes 145 more functions out (135,862 to 135,717).

What lands

Under Node nothing changes in either: the same functions run, through a record or a local.

Validation

  • Full rounds on the member against main 6da2be6a1b, --gate-corpus=all, no allow list. The first two commits (7e5ca59110): PASS in 44.5 minutes. With the emitter commit (0080bd728f): PASS in 48.3 minutes. In each:
    • Zero-cost gate: 5,549 programs identical, 290 fail to build on both sides (island programs, which the gate builds without a libnode pack), none differ, no runtime-only row. Compile time for programs within tolerance in both (first round above; second: benchmarks +0.22 % [−0.84 %, +1.71 %], self +0.48 % [−2.26 %, +0.91 %]). Self-compile budget in the second: 2 lowering passes, 144 bytes of heap per source byte (budget 146), the three size lines above, and keeps Lowerer, LlEmitter out of the dynamic tier: no instance becomes a dynamic value.
    • Lanes: plain 24,178 passed, 0 failed; sanitized 20,758 passed, 0 failed. Both ran every build in the skip's check mode.
    • Native bootstraps, all five phases in both rounds: plain commands and rebuild, sanitized commands, rebuild-emit and rebuild-frontend. CI does not run these for this pull request (it touches no bootstrap path), so these runs are the evidence.
  • Train perf 2: anchored and non-ASCII regular expressions without a scan, class tables selected by the class, two emitter passes in one walk, variadic functions through fixed slots #3379's three tables of generated callback programs, built and run with main and with the member: 4,944 cells (1,784 match Node, 14 fail loudly, 3,146 are refused), 390 cells (258 match, 132 refused), 117 cells (69 match, 48 refused): the same verdict, output or first diagnostic for every cell on both sides. No silent difference, no match lost.
  • Every program lowered both ways: the corpus (5,772), the diagnostics fixtures (361) and the 5,451 table cells, by main and by the member in check mode: exit status, output and IR are equal for all 11,584, and the check mode never fails.
  • The pushed head (61712ea6f2) is the member merged with main twice, plus the CopyScan commit. First main 5f8c88e410 (one conflict in Lowerer.run: main adds the function views pass's state on the line after the class dispatch pass's, which the second commit moves out of the block; both kept). Then main 2cea97f430 (one conflict in Lowerer.finishModule: main settles the worker module's shareEnv entry as the function's first step, and this train adds two parameters; both kept. That step runs between devirtualization and the variadic check and stores a boxed symbol, no function value; the comments now name it).
    • Zero-cost gate on 6b56f87160 (everything but the second merge of main) against main 5f8c88e410, whole corpus, no allow list: PASS. 5,602 programs identical, 315 fail to build on both sides, none differ, no runtime-only row. Compile time: benchmarks −0.44 % [−1.77 %, +1.11 %], self −0.84 % [−3.82 %, +0.53 %] (inconclusive), fixtures +1.69 % [−0.37 %, +3.76 %] (inconclusive, one timing run). Self-compile budget: 135,717 functions (−4.6 % against main's source), 1,027,681,096 LLVM bytes (−1.6 %), 11,881 dynamic-tier functions (−2,635), and keeps Lowerer, LlEmitter, CopyScan out of the dynamic tier.
    • The compiler's own source compiled by itself on that tree, before and after the CopyScan commit: 135,862 to 135,717 functions, 1,028,098,564 to 1,027,681,096 LLVM bytes, dynamic tier 11,962 to 11,881; the report lists the two calls before and nothing after.
    • The command CI's native compiler job runs, node scripts/self-compile-budget.mjs --native-build with the profile check, on 6b56f87160: exit 0 after 1,024 s, keeps Lowerer, LlEmitter, CopyScan out of the dynamic tier: no instance becomes a dynamic value, 148 bytes of heap per source byte (budget 146, tolerance 5 %). It reports that the committed profile matches 76.0 % of its functions on this tree (a warning, not a failure; the nightly refresh moves the profile).
    • On the first merged tree: the build has no TypeScript error; lint, format, the three generated-file checks and the ts7 baseline check are clean; main's guard tests (130 in 10 files), the policy tests (104), this train's tests (67 in 11 files), the diagnostics suite (372), 550 differential cases (every program with variadic, rest-, arguments, callback, virtual or override in its name) and 275 of them sanitized pass. On the pushed tree: build, lint, format, the generated-file checks, the baseline check, the policy tests, main's guard tests and this train's tests pass.
    • Still running on the pushed tree when it was pushed, results follow as a comment: the diagnostics suite and the differential cases, the gate's identity against main 2cea97f430, and the sanitized bootstrap phases rebuild-frontend and rebuild-emit on 6b56f87160.

Not run

  • bench-runtime and Test262: no program's IR, LLVM or executable changes.
  • macOS.

What is left

  • The general rule for the check: every class of the compiler that is a dynamic value in its own build must be in a listed set, so that a class nobody named fails too. It needs the full list from a build of the then-current main and will fail in-flight changes that box a new compiler class, so it is a change of its own.
  • The general fix in the compiler: a typed program pays for both patterns as well. A five-function program is 130 KB instead of 94 KB when it calls a function-valued member as object.member(…), and 486 KB instead of 107 KB when it hands a class to an interface with a method. Design and measurements are in the fleet's notes; neither is a small change.

🤖 Generated with Claude Code

cramforce and others added 4 commits November 7, 2026 22:54
…unction

The check that runs when lowering ends (variadic-functions.ts, #3379)
walks every expression of the finished program. Each of its refusals
needs one of three things: an expression of a variadic function type, a
store lowering noted, or an adapter whose call is a store. Most typed
programs have none of them.

Devirtualization is the last step before the check on the build path, and
its rewrite passes every expression of the bodies it leaves through one
callback. It now says whether one of them has a variadic function type,
and the check returns without a walk when none has, lowering noted no
store and no storing adapter exists. The longer calls lowering notes are
read only for a store, so they do not keep the walk. The other emit path
(the historical-order relower, coverage) walks as before.

SCRIPTC_CHECK_VARIADIC_SKIP=1 walks every program the check skips and
fails the build when the walk finds something. The test suites run with
it, so a refusal added to the walk without its condition fails there.

No program's output changes.

Co-Authored-By: Claude Code <noreply@anthropic.com>
The compiler's Lowerer became a dynamic value in the compiler's own build
three times in five days (#2878, #3192, #2567): each time one helper took
an interface the Lowerer satisfies, each cost about 6% of the functions of
that build, and each landed green. The output-growth rule sees growth, not
state, so it also misses a second such place that lands while a first one
is in.

The budget file now lists the classes that must stay typed
(neverDynamicClasses: the Lowerer). SCRIPTC_DYNAMIC_CLASSES makes a build
report, per lowering pass, every place of the finished program that makes
an instance of a named class a dynamic value: a conversion of the class or
of a type that holds it, a dynamic function checked into a type that takes
it, a checked cast into it, and the class dispatch pass's own verdict when
no place explains it. scripts/self-compile-budget.mjs sets the variable for
--check and --native-build (the build CI's native compiler job runs),
fails on a reported place and prints it with its source line, and fails
when the build printed no report at all. The repository's gate asks its
traced self-compile for the same report.

Co-Authored-By: Claude Code <noreply@anthropic.com>
…uild

The compiler compiles its own source, and in that build the LlEmitter was
a dynamic value: the keyed tables, checked casts and converters of untyped
code were compiled for it and for every class its fields reach. Three
places made it one, which the build's own report lists
(SCRIPTC_DYNAMIC_CLASSES=LlEmitter):

- emit-function.ts handed the emitter to emitIntegerClone and
  emitIntegerDispatcher, whose host parameter is the IntegerCloneHost
  interface, kept in a field of the clone emitter. They now get
  emitter.integerCloneHost(), a record of the seven members the clone
  emitter reads, as the Lowerer's callers got bindingScope() in #3394.
- wrapping-sums.ts called the function-valued member int32Elements as
  host.int32Elements(...), a member call, which hands the function the
  emitter as its receiver. The function is read into a local first.

LlEmitter joins the Lowerer in the budget's neverDynamicClasses, so the
self-compile check fails on the next such place and names it.

No program's output changes: the same methods run, through a record or a
local instead of through the emitter.

Co-Authored-By: Claude Code <noreply@anthropic.com>
One conflict, in Lowerer.run: main adds the function views pass's state on
the line after the class dispatch pass's, which the second commit of this
train moves out of the block. The dispatch pass stays above the block and
the function views stay in it.

Co-Authored-By: Claude Code <noreply@anthropic.com>
@vercel

vercel Bot commented Nov 8, 2026 •

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
scriptc Ready Ready Preview, v0 Nov 8, 2026 9:15am UTC

cramforce and others added 3 commits November 8, 2026 00:19
The LLVM emitter leaves the dynamic tier of the compiler's own build. No
conflict.

Co-Authored-By: Claude Code <noreply@anthropic.com>
CopyScan.layoutPairs (width-copies.ts) called its function-valued member
as this.unionArms(...). A member call hands the function the scan as its
receiver, which makes the class a dynamic value when the compiler compiles
its own source, as wrapping-sums.ts did for the LLVM emitter. The function
is read into a local first.

CopyScan joins the budget's neverDynamicClasses. The list checks the
classes it names and no others: this class was not caught because nobody
had named it.

Co-Authored-By: Claude Code <noreply@anthropic.com>
One conflict, in Lowerer.finishModule: main settles the worker module's
shareEnv entry as the function's first step, and this train adds two
parameters to it. Both kept. That step runs between devirtualization and
the variadic check and stores a boxed symbol, no function value; the
comments that said nothing runs between the two now name it.

Co-Authored-By: Claude Code <noreply@anthropic.com>
@cramforce cramforce changed the title Train perf 5: the variadic check skips its walk for typed programs, and the self-compile check fails when a listed compiler class becomes a dynamic value Train perf 5: the variadic check skips its walk for typed programs, CI's self-compile check fails when a listed compiler class becomes a dynamic value, and the LLVM emitter leaves the dynamic tier of the compiler's own build (4.7% fewer functions) Nov 8, 2026
@ctate
ctate merged commit f0939e3 into main Nov 8, 2026
30 of 35 checks passed
@ctate
ctate deleted the train/perf-5 branch November 8, 2026 09:24
@cramforce

Copy link
Copy Markdown
Contributor Author

Results that were still running at the second push (head 61712ea6f2):

  • Pushed tree, tests: build (type check), lint, format, the three generated-file checks, the ts7 baseline check, the policy tests (104), main's guard tests (132 in 10 files), this train's tests (77 in 12 files), the diagnostics suite (380), 552 differential cases and 276 of them sanitized: all pass.
  • Sanitized native bootstrap on 6b56f87160 (the tree with the CopyScan commit, before the last merge of main; 32 vCPU Sandbox): rebuild-frontend passes in 3,132 s and rebuild-emit in 2,192 s. The five phases of both bootstraps had passed on the member with the emitter commit (0080bd728f).
  • Gate identity on the pushed head against main 2cea97f430: not run. Two attempts stopped while the base for that commit was being recorded, because GitHub answered HTTP 504 for release downloads (ICU's pinned data file the first time; libnode-linux-x64.tar.gz from both of its sources the second). That is the tooling's download, not this change. What stands: the gate on 6b56f87160 against main 5f8c88e410 (5,602 identical, 315 fail on both sides, none differ), which is this head without the merge of main 2cea97f430, and the tests above on the merged tree.
  • Self-compile time with the emitter and CopyScan commits: three alternating pairs of the compiler compiling its own source to LLVM on an idle 32-vCPU Sandbox give +3.3 % [95 % CI −13.5 %, +23.3 %]: inconclusive. One tree's frontend takes between 353 s and 482 s from run to run there, more than any effect of 6,663 fewer functions; the counts in the description are exact.

CI on this head: the red jobs are the ones that build runtime packs and native artifacts (build (Linux x64 native packages), both bootstrap seeds, the Windows and macOS x64 artifact jobs). I could not read their logs while the run was in progress; the same GitHub release downloads failed in the Sandboxes at that time. On the first head the one red job was test (macOS x64 LLVM artifacts, no clang), step "Node islands (Intel)": tests/harness/node-island-load-reads.test.ts:153, process.hrtime() at load, then the program's: false, an order of two clock reads; main's own runs are green there and nothing in this change is on that path.

This branch was successfully deployed

1 active deployment
Preview — 61712ea6 Deployed Nov 8, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants