Repository navigation
Conversation
The Node-hosted compiler ran comptime callbacks with vm.runInNewContext in its own process. A callback reached the compiler's process through the context's constructor chain and through the result's own methods, which the lowerer called with a compiler function; accessors and promise reactions ran after the budget; an unhandled rejection or import() ended the compiler; and there was no memory limit. The native compiler was not affected: it runs callbacks in a QuickJS process. Both compilers now start the same evaluator through the same driver: - scriptc-comptime (QuickJS-ng and comptime.c, now under native/llvm-codegen/comptime) is built by each LLVM helper package and ships in its bin directory. The native CLI distribution's own copy, its build step and the toolchain manifest's comptime key are removed, and so is the QuickJS archive the runtime packs carried for it. - frontend/comptime.ts replaces native/comptime.ts and frontend/comptime-node.ts. TypeScript's command line erases the types of all of a source file's callbacks in one run. - The time limit is also enforced by a watchdog thread, the memory limit is enforced in the engine's allocator and reported by name, and results the evaluator refuses (cycles, symbols, size bounds) are named. - Pending promise reactions and unhandled rejections are build errors. - Math's transcendental members and hypot compute V8's values (the runtime's fdlibm port), BigInt.asUintN is corrected, and members whose Node result needs locale data, and toString with a radix for values that are not safe integers, throw by name instead of answering something else. The docs state what the evaluator guarantees: a separate process with the built-ins only and two limits, not a hardened sandbox. Co-Authored-By: Claude Code <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
The evaluator the Windows helper package builds with Zig answered the probe correctly, with the carriage return a Windows executable writes before the line feed. The probe compared the bytes exactly. Co-Authored-By: Claude Code <noreply@anthropic.com>
…e result is read A result that is a proxy over an array was written out as a record, so a callback that returned one failed its array type where the Node-hosted compiler used to accept it (Array.isArray looks through proxies). The evaluator now applies the same rule. The checks for pending promise reactions and unhandled rejections ran before the result was written out. Accessors and proxy traps of the result run while it is written, so work they queued passed unnoticed. The checks now follow the encoding. Co-Authored-By: Claude Code <noreply@anthropic.com>
Apple's <malloc/malloc.h> does not compile when _POSIX_C_SOURCE is defined: its typed-allocation declarations need a type the strict POSIX mode hides. The evaluator defines _POSIX_C_SOURCE for clock_gettime and nanosleep and needs one function from that header, so it declares it. Co-Authored-By: Claude Code <noreply@anthropic.com>
One conflict: the node:fs import of packages/compiler/src/native/prepare.ts. main dropped readFileSync from it when the link feature keys moved to one list (#3245); this branch added existsSync for the evaluator's identity. The import is now existsSync alone. Co-Authored-By: Claude Code <noreply@anthropic.com>
|
Merged Conflict. One line, the Re-run on the merged tree (Vercel Sandbox, Xeon 2.90GHz, 8 vCPU; the LLVM helper package rebuilt with
The one failure is Not re-run: the zero-cost gate, the full lanes, the native bootstraps ( |
main moved after the first merge: #3249 and #3251 put readFileSync back into the node:fs import of packages/compiler/src/native/prepare.ts (the build cache reads its recorded inputs there), where this branch has existsSync. The import now names both. Co-Authored-By: Claude Code <noreply@anthropic.com>
|
Conflict. #3249 and #3251 put Re-run on this tree (Vercel Sandbox, Xeon 2.90GHz, 8 vCPU; helper package, Not re-run: the zero-cost gate, the full lanes, the native bootstraps, |
|
Superseded by train security-1, #3430. The train carries this branch at d00f8ec, merged with current main (three conflicts, all in CI files and the test cost table; the resolutions are in the merge commit's message), and adds the two patterns of the evaluator's inputs to the runtime pack's cache key, which a test main gained after this branch was cut requires. Validation of the merged tree is in #3430's description. 🤖 Generated with Claude Code |
Compile-time callbacks run in one evaluator process under both compilers (vercel-labs#3209). Conflicts, resolved by reading both sides: - .github/actions/native-packages/action.yml: the helper cache key keeps the branch's two added inputs (the evaluator's engine sources and the math port), and main's comment on the runtime pack's key stays below it. - .github/workflows/ci.yml, the Windows job: Zig is installed before the helper build, which now builds the evaluator with it, and main's restore of ICU's archives stays before the runtime pack build. Zig is installed once. - scripts/ci-test-costs.json: main's table, with the branch's rename applied: the evaluator's driver test is listed under its new path with main's measured cost, and the deleted test's rows are gone. Co-Authored-By: Claude Code <noreply@anthropic.com>
What this fixes
Under the Node-hosted compiler (
@scriptc/compiler's API,packages/cli/dist/bootstrap.js, this repository's tests and CI) acomptimecallback ran withvm.runInNewContextinside the compiler's process. That is not a boundary. On main827bc093d:comptime(() => globalThis.constructor.constructor("return process")())reaches the compiler'sprocess: the built executable prints the compiler's pid and working directory.(() => 1).constructor("return this")()gets there too.mapreceives one of the compiler's functions, andcallback.constructor("return process")()reaches the host without touching a global.import("node:fs")and an unhandledPromise.reject(...)ended the compiler with an uncaught exception; an array with a hole crashed lowering (TypeError … reading 'kind'); there was no memory limit (4.3 GB peak inside the budget).The native compiler evaluates callbacks in a QuickJS process and none of this applies to it; I built it from the same commit and ran the same programs against it. The docs described the native behaviour for both.
The change: one evaluator
Both compilers now run the same executable through the same driver, so a callback has one behaviour and one set of limits.
scriptc-comptime(QuickJS-ng pluscomptime.c, now innative/llvm-codegen/comptime/) is built by each LLVM helper package'sbuild:nativeand ships in@scriptc/llvm-<host>/binbeside the helper: the helper package is the one per-host package both compilers always resolve. The native CLI distribution's private copy (lib/scriptc-comptime), its build step and the toolchain manifest'scomptimekey are gone.packages/compiler/src/frontend/comptime.ts(wasnative/comptime.ts): TypeScript's command line erases the types, the evaluator runs the JavaScript, and only tagged JSON comes back, so the compiler never calls or reads anything the callback built.frontend/comptime-node.tsand its TypeScript 5 use are deleted.libscriptc-quickjs.a, an archive no program linked; it existed for the native CLI's copy.Options considered (a WebAssembly build of QuickJS in process, an N-API addon, V8 in
node --permission, linking on first use) and why this one: the others leave two builds of the engine or two engines, and results must be identical under both compilers.Limits and what is refused, the same under both compilers
evaluation exceeded the 256 MB compile-time memory limitinstead ofthe callback threw: out of memory.undefined, at most 1,000,000 elements per array, 512 levels, 64 MB; cycles and symbols are refused by name. (the callback threw: [uninitialized]andCommand failed: …/scriptc-comptime …were what the native compiler printed for three of these.)Evaluator disagreements closed
After this change the corpus runs against the evaluator users get, which showed where it differed from Node. Measured on main, QuickJS against V8:
Math:sin,cos,tan,asin,acos,atan,atan2,sinh,cosh,tanh,asinh,acosh,atanh,exp,expm1,log,log1p,log2,log10,cbrtandhypotdiffered in the last digit for 0.03 % to 47 % of 4,000 inputs each (the C library against V8's fdlibm). The native compiler baked4.57642774787696where Node prints4.576427747876961. The evaluator now calls the runtime's port of V8's file and V8'shypot; all 21 agree.BigInt.asUintN: QuickJS-ng 0.15.1 answers-1nforasUintN(64, -1n)and forasUintN(64, 2n ** 64n - 1n). Corrected in the evaluator.toLocaleString,localeCompare,toLocaleLowerCase,toLocaleUpperCase,Date#toString,toTimeString,toLocale*String(no locale data), andNumber#toString(radix)for a value that is not a safe integer (V8's digit generation).No existing corpus program used an affected member: the gate shows every executable byte-identical.
Proof
All in Vercel Sandboxes (Intel Xeon @ 2.90GHz, 32 vCPU, Node 24.22.1, clang 22.1.8); the native compiler was built from the same commit as the Node-hosted one, on main and on this branch.
Escape attempts, each built with both compilers
globalThis.constructor.constructor(...)no processno process(() => 1).constructor("return this")()no processno processmapno processno processimport("node:fs")$[1]$[1]the callback threw: out of memoryevaluation exceeded the 256 MB compile-time memory limitMath.sin + cbrt + exp + hypot + atan2Tests
packages/compiler/src/frontend/comptime.test.ts(14 tests): the driver and executable both compilers run. Constructor chains from every reachable prototype, result hooks,import(),require, blocking, the time limit through loops, patterns, one long builtin, accessors and thrown values, the memory limit on and off the heap, result rules,Mathagainst Node's ownMathon 600 inputs per member,BigInt.asUintN, the refusals, and one TypeScript run per file.packages/compiler/test/comptime-isolation.test.ts(6 tests): the same attempts compiled by the Node-hosted compiler. Run against main, three fail on an assertion (the IR holds the host's answer), two never finish (the compiler is still inside the callback's loop minutes later), one is not an escape test.tests/harness/self-hosting-native-driver.test.ts: the native compiler builds an escape program and theMathcheck, and builds and runs everycomptimecorpus program against Node.8961-comptime-node-values.ts(a 48 by 21 table ofMathmembers in full precision, FNV-1a hashes withBigInt.asUintN; fails natively on main) and8962-comptime-result-data.ts.tests/diagnostics/comptime.tsgains eight cases; the nine existing diagnostics are unchanged.comptimecorpus programs print the same bytes as before under both compilers.Zero cost.
scripts/zero-cost-gate.mjsover benchmarks, the self-hosted compiler and the whole corpus (--corpus=all), no--allow, c64576d against base 4f6bb47: PASS (the commit after it changes one declaration in the macOS build of the evaluator). 5,237 programs are byte-identical, 102 fail to build on both sides, none differ. Compile time, all within tolerance: benchmarks −0.11 % [95 % CI −1.31 %, +0.87 %], self-compile −0.41 % [−2.93 %, +1.88 %], fixtures +0.52 % [−0.59 %, +1.64 %].Compile time (
build --emit=llvm, cache off, median of 11, interleaved):comptimeThe Node host stops loading TypeScript 5 (235 ms once) and pays one
tsc-tsrun per file (45 ms) plus one evaluator process per call (1.6 ms). The native compiler paid atsc-tsrun per call. Run-to-run spread here is about 15 %, so the Node-host columns are equal within it; the native 10 and 100 call rows are real.Lanes and bootstraps (97 sandbox jobs, on 75ad44a; the two commits after it change the evaluator, its build probe and tests, and the
comptimeunit, isolation, diagnostics and corpus tests were rerun on the head, plain and sanitized). Plain lane: 21,990 tests passed, 0 failed. Sanitized lane: 18,760 passed, 0 failed. The native bootstrap passes plain and sanitized (the seed, its command probes, the compiler rebuilding itself, the rebuilt compiler emitting itself). One suite did not run:packages/runtime/test/regex.test.tstimed out in its 60 sbeforeAll, an ASan compile of runtime sources this change does not touch, in a shard of 370 files that held none of this change's tests; alone on the same tree it passes in 37 s.Docs gate.
NEXT_DIST_DIR=.next-check pnpm checkindocs/: 25 of 25 route tests pass.Docs
how-it-works.mdxsaid callbacks run "in an isolated evaluator inside the compiler". It now says what is true: a separate process that sees the ECMAScript built-ins and no object of the compiler or the host, with a time limit and a memory limit. The limitations page gains a "Build-time evaluation" section with the rules, limits and refusals, and says plainly that the evaluator is a process, not a hardened sandbox.For reviewers
packages/llvm-*/scripts/build.mjs, theirpackage.json(bin,prepack) and notices,scripts/verify-llvm-package*.mjs,packages/cli/scripts/install-native.mjs,scripts/package-native-cli.mjs,scripts/verify-native-cli.mjs, the helper cache key in.github/actions/native-packages, and the mode restore inrelease.yml.Date()called as a function; engine error wording.🤖 Generated with Claude Code