[cramforce]

Follow-up to #3369 (the runtime profile refresh), which found three defects around the runtime profile. Test, CI configuration and docs only: no runtime source, compiler source or profile changes.

1. The staleness test counted a floor that no refresh can clear

packages/runtime-pack-common/test/runtime-profile.test.ts fails when more than 10% of the functions the profile describes changed. It read mismatched / (annotated + mismatched) from the pack manifest, and those two numbers add up every unit object's functions. The pack compiles some units more than once (default, workers, young-nursery and other variants) and some names twice (a stub in one unit, the function in another). Where the control flow of two such builds differs, the profile's record matches one of them and reads as mismatched in the other, however fresh the profile is. On main, with a profile collected the same day, that is 382 of 5,226 (7.31%). The limit left room for about 40 really changed functions, and main changed 40 in the three days before #3369.

The measure now counts by profile name: a function is changed when no object's build of it matches the profile. The counters limit (5%) has used that rule since it was added.

Why by name, and not by object and name: the profile is applied when a unit object is compiled (runtime-unit --profile-use), not at link time. LLVM looks each function up by its profile name and takes the record whose control-flow hash is the function's. A record is keyed by name and hash; the format has no object, and a local function's profile name (<file>.c;<name>) is the same in every build of its unit. So a per-object key cannot be expressed, and "this name has a record that some build still uses" is the question the format can answer.

One thing the by-name count does not see, stated here so nobody has to find it: the committed profile holds 1,606 records for 1,485 names, so 121 names have two records (both builds were trained). If an edit changes the control flow of only one of those builds (an edit inside code that differs between the builds), the name stays annotated through the other. The counters limit has the same rule. An exact count needs the LLVM helper to report which hash matched, which is a helper change and not part of this PR.

The manifest keeps the per-object numbers (annotated, mismatched, missing: how much of the pack is optimized with the profile) and gains annotated_names and mismatched_names. The summary moved from build.mjs into scripts/profile-match.mjs with unit tests that run on every host. A pack whose manifest lacks the new fields fails the test with "rebuild the runtime pack".

Numbers (Linux x64 Sandbox, main f88df8d sources)

pack	before: per object	after: by name	counters (unchanged)	test
main, committed profile	382 of 5,226 = 7.31%	2 of 1,464 = 0.14%	0.00%	passes
same sources, the profile from before #3369	460 of 5,002 = 9.20%	40 of 1,409 = 2.84%	2.81%	passes
mutation: 3 profiled functions get one more branch	396 of 5,226 = 7.58%	5 of 1,464 = 0.34%	1.14%	passes
mutation: 303 definitions get one more branch	1,865 of 5,227 = 35.68%	291 of 1,464 = 19.88%	18.40%	fails

The 2 names on main are real staleness: scr_events.c;scr_events_dispatch and scr_run_exit_listeners, changed by #2801 after the profile was collected.

Mutation check

In a scratch copy, a script gave profiled runtime functions one more branch at their start ({ volatile int m = 0; if (m) m = 0; }), which changes their control-flow hash and nothing they compute, and the pack was rebuilt. Three functions (scr_dyn_new_str, scr_dyn_to_string, scr_bytes_write_str): the count goes from 2 to 5, the three are named first in stale, the test passes. 303 definitions: 291 names, and the test fails with 291 of the 1464 runtime functions the profile describes changed since it was collected (most counts: scr_bytes.c;scr_bytes_convert_span, scr_arr_release, …); run node scripts/refresh-runtime-profile.mjs: expected 0.1987704918032787 to be less than or equal to 0.1.

The limit stays at 10%

The number is unchanged; what it measures is now what its comment always said (the share of the described functions that changed). In practice the old measure fired at about 40 changed functions, the new one at 146. At the rate of the three days before #3369 (40 functions, 2.81% of the counters), the counters limit of 5% is reached after about five days and the functions limit after about ten, so the counters limit, which weighs hot functions, decides when to refresh, and the functions limit catches a broad change to cold code.

2. CI did not build a new profile into the pack it tested

The runtime pack cache key in .github/actions/native-packages/action.yml did not name packages/runtime-pack-common/profiles/, so a change to the profile alone restored the pack built with the old one. Checking the rest of that key against what the pack build reads found two more omissions of the same kind:

• packages/runtime-pack-common/unwind-tail.h, forced into every ELF unit since End every packaged ELF object's unwind table with a function no executable links #3199 (only *.mjs of that directory was in the key);
• the LLVM helper's inputs (native/llvm-codegen/**, packages/llvm-*/package.json, packages/llvm-*/scripts/**): the helper compiles every speed unit and applies the profile, so a helper change restored a pack built by the previous helper. scripts/sandbox-test.mjs already lists the helper among the pack's inputs.

All three are in the key now. tests/harness/native-packages-cache.test.ts holds it there: every file of packages/runtime-pack-common outside its tests must match a pattern of the runtime key, and the runtime key must hold every pattern of the helper key. The other cache keys (helper, LLVM distribution, Zig, libnode) do not depend on the profile, and the helper key already hashes all of native/llvm-codegen/, which holds the helper's own profile.

The pack's verify step, which runs in every CI job whether the pack was restored or rebuilt, now prints the match, so a log shows which profile a pack holds: runtime profile: 2 of the 1464 functions it describes changed since it was collected (382 of 5226 per unit object), holding 0.00% of its counts.

What the first CI run does. The action file is itself part of the helper key and the runtime key, so this change misses both caches once for every target: each job that calls the action builds its LLVM helper and its runtime pack, then saves them. The same happens once on main after the merge (a pull request's caches are not visible to main). Later runs restore as before. Because the PR touches .github/, scripts/ci-lanes.mjs turns every lane on for it.

3. RELEASING.md was incomplete

The runtime profile paragraph named only the 5% counters limit. It now names both limits and how functions are counted, says where the three training projects of step 7 come from (the typeorm, t3code-server and vscode entries of tsc-ts's packages/tsc-ts/profiles/projects.json), what the tsc-ts checkout needs, that the island start-up workloads are left out without a libnode pack, and how to measure a refresh (speed against speed; the zero-cost gate builds at release). AGENTS.md gets a paragraph that points there from the failing test.

Proof

• packages/runtime-pack-common/test and tests/harness/native-packages-cache.test.ts: 11 files, 47 tests pass on the Linux x64 Sandbox with the pack built from this branch. pnpm lint: 0 warnings. pnpm format:check: clean.
• The pack's content does not change. The pack built by main's build.mjs from the same sources and the pack built by this branch: every file under artifacts/ is byte-identical, and the two manifests differ only in the two new fields (725,603 and 725,667 bytes).
• Zero-cost gate, because the pack's manifest changes: first commit against main f88df8d, PASS, 4,052 programs identical, 93 fail to build on both sides, none differ; the built compiler is byte-identical to main's (853 files), so the timing shards were skipped. The second commit adds one output line to the pack's verify script.
• Not run: Sandbox lanes, bootstraps, runtime benchmarks at speed. Nothing that is compiled or linked changes, so there is nothing for them to measure.

Not in this PR

• Training the three island start-up workloads (the refresh script passes no libnode pack).
• A nightly refresh job for the runtime profile.
• An exact per-record count (the helper reporting the matched hash).

🤖 Generated with Claude Code

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
scriptc	Ready	Preview, v0	Nov 7, 2026 10:05pm UTC

[cramforce]

CI on f7c64dd (run 42418933560), read once: 98 jobs passed, 20 failed, plus the aggregate test. Both native bootstraps and every Effect shard passed.

The pack was rebuilt with the committed profile. build (Linux x64 native packages): Cache not found for the helper key and for the runtime key (the action file is part of both), Build LLVM helper 23 s, Build runtime pack 5 min 44 s, then the verify step:

verified @scriptc/runtime-linux-x64-gnu@0.4.9: 1517 artifacts
runtime profile: 3 of the 1464 functions it describes changed since it was collected (386 of 5226 per unit object), holding 0.00% of its counts

Both caches were saved. The numbers are for the merge with main afb6825: one more changed function than at the branch point (2, and 382 per object). The three touched test files ran green in the Linux plain lane: runtime-profile.test.ts in 6/20 (it requires the by-name fields, which only this branch's build writes), profile-match.test.ts in 1/20, native-packages-cache.test.ts in 8/20.

The run is red, for reasons that are on main. This PR touches .github/, so every lane ran, including the two that pushes to main do not run (macOS, island conformance). Each failing test, by name:

• lint and format: pnpm lint stops in node scripts/runtime-refusals.mjs --strict (7 refusal sites not in the baseline, in scr_console_native.c, scr_dyn_invoke.c, scr_fs_path_value.c, scr_pki.c, scr_json.c). Main's own runs at afb6825 and 0d142e0 fail the same job.
• Linux plain 2, 3, 5, 7, 10 and san 2, 7, 10 of 20; macOS LLVM artifacts 1/5, 2/5, 5/5 and x64; macOS plain darwin 1/3: node-island-weak-keys.test.ts (3 tests), 8754-node-island-weak-keys, 8850-node-island-weak-cache, runtime-define-sites.test.ts, runtime-pack-matrix.test.ts ("a unit that tests SCR_NODE_ISLANDS has a build that defines it"), fs-write-shared.test.ts. The same tests fail in main's own run at afb6825, and the two weak-key programs fail in the same macOS shard on Train pi 1: lowering fixes on pi's path (generic identity, Proxy slots, shell arguments, stdin as a stream, dictionaries and unions) #3352 and Train node 1: the Node compatibility ledger with its evidence, http client events, http2, Node's error codes, stream, child_process and process fixes #3353.
• Island conformance 1/4 to 4/4: nine entries "fail differently" from the list, each by the diagnostic code that runtime refusals now carry (… is not supported yet [SC1090], … [SC8010]). The lane does not run on pushes to main.
• macOS arm64 LLVM artifacts, contracts: smoke-libnode-opt-in.mjs:161, "no libnode pack was recorded at build time". It fails the same way on Train pi 1: lowering fixes on pi's path (generic identity, Proxy slots, shell arguments, stdin as a stream, dictionaries and unions) #3352 and Train node 1: the Node compatibility ledger with its evidence, http client events, http2, Node's error codes, stream, child_process and process fixes #3353.
• macOS, plain, darwin 3/3: 9474-child-output-order-held.ts, one line differs in the order of two pipe-end events (stdout end, stderr end against stderr end, stdout end). I have no second sighting of this one. The change cannot reach it (no runtime source changes, and the darwin pack is built by its own script from the same sources), and it is an ordering between two pipes on a busy runner. Not rerun.

No failure is in a file this PR touches or in the Linux x64 pack.