fix(scripts): witness entrypoint launches by data flow, not by name - #1606
Merged
Merged
Conversation
`spawnsEntrypoint` answered a weaker question than its name: whether the declared path appeared ANYWHERE in a launcher's argv, with argv identifiers resolved through a flat name to value map. Three facts were asserted that the program does not have. A name is not a binding. The map was keyed by identifier text with no scope, so a same-named `args` in an unrelated function stood in for the launcher's own and witnessed a launch that never happens. An element is not the script slot. Any matching element counted, so a path sitting after `-e` was read as executed when node runs the eval program and never opens the file. A spelling is not an import. The callee list was matched by text, so any function named `spawnProc` counted, including one imported from a local helper. A conditional argv resolved to `undefined` and reached an array-literal test, which threw. From the outside that reads as a refusal while naming a TypeError rather than a reachability fact, so a correct config was rejected for the wrong reason. `launchedPaths` already answers the real question with evidence: bindings resolve by walking outward to the nearest enclosing scope that declares the name, the executed slot is located by node's own flag rules, launcher aliases come from the child_process import, and dead code is excluded. This routes the witness through it. MEASURED at origin/main fe813fe, thirteen fixture shapes, each run through the real command entrypoint. A = main verbatim, B = main with the identifier-resolution line removed and nothing else, C = this change. `!` marks a verdict that disagrees with what the fixture actually does. shape truth A(main) B(-line) C(this) s6-foreign-spawnproc refuse GRADED! REFUSED REFUSED s7-eval-then-entry refuse GRADED! REFUSED REFUSED s8-eval-then-entry-inline refuse GRADED! GRADED! REFUSED s9-param-decoy refuse GRADED! REFUSED REFUSED s11-conditional accept REFUSED! GRADED GRADED wrong on: A 5 of 13, B 1 of 13, C 0 of 13 Main moved to 840e641 while this was in review. The three files this touches are byte-identical at both shas, so the table above was re-verified rather than re-measured: the self-test is 115 passed 0 failed on the rebased base. The four false accepts are the failure that matters: main certifies a suite as exercising an entrypoint it never launches, which is a coverage claim the run does not support. s11 is the TypeError above, wearing a refusal's clothes. s8 is why this is a replacement rather than a deletion. Removing the offending line fixes the three shapes that route through the identifier resolver, and leaves an entrypoint written inline after `-e` still falsely accepted, because the slot rule was never in that line. Nine cells cover the five branches that admit or refuse a launch, each accepting branch paired with a refusing fixture that differs only in the fact under test. Each refusal is anchored on the reason the validator prints, not on a non-zero exit, because a crash also exits non-zero and grading that as a refusal reports success over a tool that fell over. Pre-fix control, the new cells run against the main tool: branch refusing cells FAIL-before/PASS-after accepting cells green B1 slot 2 of 2 0 of 0 B2 scope 1 of 1 0 of 0 B3 conditional 1 of 1 1 of 1 B4 launcher 1 of 1 1 of 1 B5 reachable 1 of 1 1 of 1 branches with no refusing cell that proves the fix: 0 of 5 pre-existing cells regressed: 0 of 115 The two accepting cells that pass on both trees are reported as proving nothing and are kept only as the paired control for their refusing twin. One pinned mutation is re-pointed, not retired. "non-Node executables count as source entrypoint launches" targeted the deleted `executableIsNode` call. The guarantee it protects is unchanged and its cell still names it, so it now targets `isNodeExecutable` at the surviving call site, which is the line that enforces the same rule. Verified that all 50 pinned mutations apply cleanly and unambiguously to this tree, against a control of 0 that fail to apply on main. Refs #1586
The launch witness resolved an argv ARRAY to the declaration in scope at the use site, but the scalars that array holds still came from a flat name to value map. A same-named binding in an unrelated function therefore stood in for the one the launcher passes, so a suite was recorded as launching an entrypoint it never launches. Renaming the unrelated binding alone flipped the verdict, which is what a name lookup does and a binding does not. Identifier resolution now walks outward to the nearest enclosing scope that declares the name, the rule the argv path already used and the rule the source comment already claimed.
The scalar-scope fix deleted the flat name to value map, and one pinned mutation still targeted the `variables.set` line inside it. A find that matches nothing is not a weaker proof, it is no proof, and the run reported ERROR before executing the suite rather than scoring it green. It now targets the equivalent line in the scope walk and reds the same cell it always named. The mutation added alongside the fix was also mis-aimed: replacing the scoped lookup with a source-file-wide one degraded resolution to finding nothing, so it reddened the accepting twin instead of the decoy it named. It now restores the flat whole-file scan, which is the behaviour the fix removed, and reds the decoy cell.
Resolving a use to the nearest enclosing FUNCTION still read a binding the program cannot see. A block is a scope, and the walk skipped it, so a `const` in an `if` branch that never runs supplied the launch slot. Because the scan takes the first match in source order, the same gap read the opposite way round: with the decoy after the real declaration the verdict was correct, and only the other ordering was wrong. Both orderings are now cells, since a first-match walk tested in one order hides half its behaviour. A declaration is no longer evidence for a use that precedes it, which is a temporal dead zone error rather than a launch, and a parameter now shadows an outer binding instead of letting the outer one stand in. Both resolve to a declared-but-unknown value, so the walk stops rather than reporting an outer value the program does not reach. `var` is still collected out of inner blocks, because it hoists. `const A = B` beside `const B = A` is a program, and the resolver answered it with a stack overflow rather than a verdict. Resolution in progress for a name now yields no value. A launch inside a block, using that block's own binding, must still be witnessed. Refusing every block binding would close the false accept by breaking real suites, so that case is a cell too.
…nothing A launcher whose path is its own parameter read an outer same-named binding instead, so a program that launches one file was recorded as launching another. The parameter now shadows, which is a refusal rather than a claim: following a call site's argument into a parameter is analysis this tool has never done, and adding it here would be a new capability rather than a fix. The cycle guard is removed because it guarded a case the use-position rule already covers. Measured rather than reasoned: with the guard disabled and the order rule in place six cyclic shapes all refuse without crashing, with the order rule disabled and the guard in place they also refuse, and with both disabled all six crash. Two independent guards for one case means mutating either proves nothing, because the other still holds, and the proof reported that as a survivor. The block pin now targets the declaration scan's block boundary. Pointed at `scopeOf` it reddened seventy-five cells rather than the one it named, which is a red for the wrong reason and not a proof.
…adow
`node.end > usePos` compared source offsets. A launcher is a function and its
body runs when the function is called, so the two orders differ exactly where it
matters: a call after the declaration is a real launch the offset rule refused,
and a call before it reads the binding in its dead zone while the offset rule
accepted it. The position compared against a declaration is now the earliest
call of each function crossed on the way out to that declaration's scope, and an
invocation order that cannot be located stops resolution rather than being
guessed.
The parameter rule was guarded on `ts.isIdentifier(node.name)`, so
`function run({ENTRY})` did not shadow and an outer target-valued ENTRY resolved
through it, witnessing a launch that is not there. Destructuring parameters and
destructuring lexical declarations now shadow at any nesting depth.
Four cells and four mutants; three pinned definitions re-pointed to the lines
that now carry the same rules.
…ead-* cells The invocation-order rule added by the parent commit refuses a binding read inside a function nobody calls, which is the same case the reach guard `evaluated(node, sf)` refuses. `dead-read`, `dead-launch` and `dead-copy` each reached their mutated file through `join(ROOT, ...)`, and ROOT is declared at top level while the use sits inside `never()`. Resolving it crosses `never`, whose earliest call cannot be located, so the path was already opaque before the reach guard was consulted. Dropping `&& evaluated(node, sf)` then moved nothing and all three definitions survived: two guards for one case, which is the class the parent commit removed once already. The path in each fixture is now `join(process.cwd(), ...)`, which `rootish` evaluates without a scope lookup, so no binding crosses the uncalled function and the reach guard is the only thing left that can refuse.
This was referenced Sep 14, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
spawnsEntrypointdecided whether a suite launches a declared entrypoint by asking whether the declared path appeared anywhere in a launcher's argv, resolving argv identifiers through a flat name to value map. That asserted three facts the program does not have.A name is not a binding. The map was keyed by identifier text with no scope, so a same-named
argsin an unrelated function stood in for the launcher's own and witnessed a launch that never happens.An element is not the script slot. Any matching element counted, so a path sitting after
-ewas read as executed, when node runs the eval program and never opens the file.A spelling is not an import. The callee list was matched by text, so any function named
spawnProccounted, including one imported from a local helper.There was also a crash: a conditional argv resolved to
undefinedand reached an array-literal test, which threw. From the outside that reads as a refusal while naming aTypeErrorinstead of a reachability fact, so a correct config was rejected for the wrong reason.launchedPathsalready answers the real question with evidence. Bindings resolve by walking outward to the nearest enclosing scope that declares the name, the executed slot is located by node's own flag rules, launcher aliases come from thechild_processimport, and dead code is excluded. This routes the witness through it.The measurement
Thirteen fixture shapes, each driven through the real command entrypoint. The measurement baseline was commit
840e64110d28e8f487434de24576d6dcc468ee18. Tree A is that commit verbatim. Tree B is A with only thespawnsEntrypointidentifier-resolution line removed. Tree C is commitd8dfaabefdea7ec3a86b2a5eb09ff14505784c05.!marks a verdict that disagrees with what the fixture actually does.Cryptographic tree definitions:
840e64110d28e8f487434de24576d6dcc468ee18, tree7f6f038ad4a07a09b859c1e02844c0b6c1c00ee3128da73ac949eea99d9db0826972eab3b91e2856, treed78ef804885ab601204279a42dd7125d4004fcb7, changedscripts/mutation-coverage.mjsblob352b6cff3a93017494a97530c0467b2f53ca157ed8dfaabefdea7ec3a86b2a5eb09ff14505784c05, tree907b39c6d0cc7a28024b7e2e6a681617e161f39aB was materialized with a temporary Git index and
git commit-tree; it is not on the PR branch and did not change HEAD or the working tree.C is the parent of this head's parent, not this head. The thirteen-shape comparison was measured there and is not re-run here. What this head adds is in Execution order and destructured bindings and Two guards for one case, found by the proof below, each with its own per-cell and per-mutant evidence.
Wrong on: A 5 of 13, B 1 of 13, C 0 of 13. The other eight shapes agree across all three and are omitted.
Four of the five are false accepts, which is the direction that matters: main certifies a suite as exercising an entrypoint it never launches, and that is a coverage claim the run does not support.
s11is theTypeErrorabove.s8is why this is a replacement and not a deletion. Removing the offending line fixes the three shapes that route through the identifier resolver and leaves an entrypoint written inline after-estill falsely accepted, because the slot rule was never in that line. It is worth being explicit that "the line is dead, delete it" was the starting hypothesis, and theAversusBcolumns are what refuted it.Execution order and destructured bindings
Review found two data-flow facts the scope work still asserted without holding them. Both are fixed here.
Textual order is not execution order.
node.end > usePoscompared a declaration's offset against the identifier's offset. A launcher is a function, and a function body runs when the function is CALLED, so the two orders differ exactly where it matters.function run(){ spawn([ENTRY]) } const ENTRY = target; run()is a real launch that the offset rule refuses, andrun(); const ENTRY = target; function run(){ spawn([ENTRY]) }readsENTRYin its dead zone while the offset rule accepts it, a launch that is not there. The position now compared against a declaration is the earliest CALL of each function crossed on the way out to the declaration's scope. When that call cannot be located, an anonymous function or one never called by name, the order is not established and resolution stops rather than guessing. A refusal is a claim this tool can back; an invocation order it inferred from offsets is not. Both orders are fixtures, not one:call-after-declmust be accepted andcall-before-declmust be refused, and the mutanttextual position is treated as execution orderreddens the first whilean unestablished invocation order resolves anywayreddens the second.A destructuring pattern is a binding. The parameter rule was guarded on
ts.isIdentifier(node.name), sofunction run({ENTRY})did not shadow and an outer target-valuedENTRYresolved through it, the false-witness direction again, one spelling over. At the grandparent head the file contained noisObjectBindingPatternorisBindingElementhandling at all. Parameters and lexical declarations now shadow when a pattern can bind the name, at any nesting depth, and the value a pattern binds is not one this tool claims.destructured-paramanddestructured-declare the fixtures;a destructuring pattern is not a bindinganda destructured declaration is not a bindingare the mutants that redden them.Two guards for one case, found by the proof
The full pinned proof at the previous head reported 54 KILLED, 3 SURVIVED of 57, and the three survivors were pre-existing definitions that had been killed at
d8dfaabefand at main:Each drops
&& evaluated(node, sf)from a call-expression witness, so each is meant to prove that the reach guard is what refuses a call inside a function nobody calls. The change above is what stopped them moving, and the reason is worth stating plainly because it is the same defect class this PR exists to remove.dead-read,dead-launchanddead-copyeach reached their mutated file throughjoin(ROOT, ...). The self-test's fixture writer prependsconst ROOT = process.cwd();at top level, while the use ofROOTsits insidefunction never(). Resolving that use walks out to the source file, crossesnever, and asks for the earliest call ofnever. There is none, by construction, because the whole point of the fixture is that nothing calls it.executionPostherefore returnsundefined,declaredInrecordsOPAQUE, and the path is already unresolvable beforeevaluatedis ever consulted. Dropping the reach guard changed no verdict, all three suites stayed refused, and all three definitions survived.That is two guards for one case, and mutating either one proves nothing while the other holds. It is the same survivor shape the parent commit removed once already by declining to add a visited set beside the declaration-order rule.
The fix is in the fixtures, not the rule. Each path is now
join(process.cwd(), ...), whichrootishevaluates directly with no scope lookup, so no binding crosses the uncalled function and the invocation-order rule is never consulted. The reach guard is then the only thing between the fixture and a witness. Route taken: (a), fixtures. The definitions, theircelland theirexpectRedare unchanged, so what each one grades is still the rule it names.Per-mutant, at this head. Each mutant was applied in place from its own
find/replace, the mutated text was asserted to differ from the source before the result was scored, the self-test was run, and the FIRST cell reddened was recorded:a read is a witness even where control never reaches ita read inside a function nobody calls never happensa launcher is a witness even where control never reaches ita launch inside a function nobody calls never runsa copy is a witness even where control never reaches ita copy inside a function nobody calls never happensIn each case the observed red is the suite being GRADED where the cell requires it to be REFUSED, reported as
1 / 3 cells observed failingforbin/smoke/dead-read.smoke.ts,bin/smoke/dead-launch.smoke.tsandbin/smoke/dead-copy.smoke.tsrespectively. The tool file was restored from the recorded source after each run and the restore was verified.With the reach guard present the three cells are green: 128 passed, 0 failed at this head.
Pinned definitions re-pointed, not retired
A mutation whose
findno longer matches does not fail loudly, it silently stops testing anything, which is worse than a red. Across the PR, five definitions were re-pointed: two byd6c9ae48fa5bcfa1c040a49732004a9a8246db66(bound entry names are ignoredandnon-Node executables count as source entrypoint launches), and three by the parent of this head, measured by a name-set and target-text diff of the definition file against its parent.Re-pointed by
e5489d9f0:a scalar binding resolves by name instead of by scope, itsfindcarried theconst usePos = node.getStart(sf);line, which that change deletes.a declaration after the use is still resolved, itsfindwas the order comparison itself, now spelled against the invocation position.a parameter is not a binding, itsfindcarried thets.isIdentifier(node.name)guard that was the second defect.In each case the guarantee is unchanged, the cell is unchanged, and
expectRedis unchanged; only the target text moved to the line that now enforces the same rule. This head re-points nothing: it changes three fixture strings in the self-test and no definition at all. Every definition targetingscripts/mutation-coverage.mjswas checked for exactly one occurrence, so no mutation is ambiguous and none is a no-op edit:Cells and mutants added by this work
Counts below were measured against
e5489d9f0's own diff against baseaaff9d5e5fbf9ef2f4fa5c0b39f16ca44b9bed07, not carried over from an earlier iteration. This head changes three fixture strings and adds nocheck(site, so every count is unchanged by it; that was verified rather than assumed (git show HEAD -- scripts/mutation-coverage.selftest.mjs | grep -c '^[+-].*check('is0).check(call sitesgit diff aaff9d5e..HEAD | grep -c '^+.*check('git diff aaff9d5e..HEAD | grep '^+.*check(' | grep -vc 'check("'node scripts/mutation-coverage.selftest.mjs(final line)python3 -c 'import json;print(len(json.load(open("scripts/mutations/mutation-coverage.json"))["mutations"]))'e5489d9f0) / 5 (whole PR)The added
check(sites do not equal the added runtime cells: the block-decoy site runs twice, once per declaration order, and the-eslot site and the fabricated-evidence site run once per entry in their loop tables. The cell total is therefore read from the self-test's own printed line rather than counted from the diff.Pre-fix control, per mutant
Each mutant was applied to the tool in place, the self-test run against it, and the FIRST cell it reddens recorded. First, because the self-test aborts on first failure, so a mutant that reddens an earlier cell proves nothing about the one it names. The mutated text was asserted to differ from the source before the result was scored; a
find/replacepair that edits nothing produces a clean run that would otherwise be read as a kill, which is the defect class this tool exists to catch.textual position is treated as execution ordera destructuring pattern is not a bindinga destructured declaration is not a bindinga scalar binding resolves by name instead of by scope(re-pointed)a declaration after the use is still resolved(re-pointed)a parameter is not a binding(re-pointed)Two further candidates were written, measured, and removed rather than registered, because a definition that survives is a definition that tests nothing while reading as coverage:
an unestablished invocation order resolves anyway(if (call === undefined) return pos;), survived, 128 passed. No fixture in the suite reaches the unlocatable-invocation arm, so that arm is currently guarded by code and by nothing else. It is named in the limitations rather than shipped green.the last matching declaration binds instead of the first(last-match scan plus transparent blocks), survived, 128 passed, so it is not the mover the(decoy last)cell needs. It is removed on the same principle.Two cells that still cannot move
These are the blockers this PR does not close, and they are stated as unclosed rather than as done.
block-decoy-afterstill has no mutant that reddens it. The block-transparency mutant is answered correctly by that fixture's declaration order, so the cell is green under it. The candidate written for it, take the last matching declaration instead of the first, with blocks transparent, was applied and survived all 128 cells, so it is not the mover and was removed instead of registered.cyclic-terminationstill has no mutant of its own either, and the reason is structural rather than an oversight. Termination is carried by the same declaration-order rule thatuse-before-decltests, and that cell runs earlier; under abort-on-first-failure any mutant that severs termination reddensuse-before-decland stops there, so the cyclic cell never reports. Adding a second, independent guard purely to have something to mutate would recreate the survivor this PR already removed twice, because mutating either guard leaves the other holding. The honest fix is the separately-filed abort-on-first-failure defect, not a redundant guard here.The unlocatable-invocation arm of
executionPosis the third such gap: it is reached by no fixture, and the mutant written for it survived, so it is guarded by code review and not by this suite.Verification
check(): 128 passed, 0 failed, run at this head on a clean tree.node scripts/mutation-proof.mjs --config scripts/mutations/mutation-coverage.json, exit 0, run at this head on a clean tree; 57 KILLED / 0 SURVIVED counted from the per-mutation verdict lines, not read off the banner. The three reach definitions are named in it:a read is a witness even where control never reaches itred ata read inside a function nobody calls never happens(13 marks, baseline 128),a launcher ...ata launch inside a function nobody calls never runs(14 marks), anda copy ...ata copy inside a function nobody calls never happens(29 marks).)typecheckis a no-op for these files and I would rather say so than report it as a passing gate:tsconfig'sincludedoes not coverscripts/andallowJsis absent, so tsc loads 0 of them, against 91 files underbin/as a control that the check itself works.Scope resolution, and what review found in it
Resolving a use to its nearest enclosing FUNCTION was not enough, and the first attempt at it introduced defects that review caught.
A block is a scope. Without that, a
constinside anifbranch the program never enters supplied the launch slot, which is this issue's own class re-created inside the fix for it. The scan takes the first match in source order, so the gap read both ways round, and both orderings are cells. A launch inside a block using that block's own binding must still be witnessed, so that is a cell too; refusing every block binding would close the false accept by breaking real suites.A declaration is no longer evidence for a use the program reaches before initializing it, and that is now decided by invocation order rather than by offsets. A parameter shadows an outer binding instead of letting the outer one stand in, whether it is written as an identifier or as a pattern. All of these resolve to a declared-but-unknown value, so the walk stops rather than reporting a value the program does not reach.
varis still collected out of inner blocks, because it hoists.const A = Bbesideconst B = Ais a program. The identifier branch it replaced was a flat map lookup that did not recurse, so this change introduces the recursion and terminates it by declaration order: a cycle needs one edge pointing at a later declaration, and a later declaration yields no value. A visited set as well would be a second guard for the same case, and mutating either then proves nothing because the other still holds, which the proof reported as a survivor.Recursion depth is bounded by the length of a binding chain the launch actually references. A chain of about four thousand links overflows the stack and surfaces as a refusal with its reason, not as a wrong verdict. The deepest chain in any suite in this repository is 31 links. The bound is recorded rather than engineered for.
Notes
Three repo-side defects surfaced during this work and are filed separately rather than fixed here, since none is caused by this change.
mutation-proofleaves a live mutation in the tree when it is killed mid-run, and it takes no lock, so a second proof started against the same checkout silently measures a mutated tool. Both were observed here: this clone was found with a stray!inDeadBranch(n)deletion applied from an abandoned run, and a self-test run against it reported a false failure ata same-named scalar in another scope cannot witness an entrypoint launchthat disappeared once the tree was restored and the competing run stopped. A tool whose job is deciding whether tests can fail should not be able to corrupt its own measurement this quietly; a lock file and a restore-on-signal would close both.The self-test's abort-on-first-failure couples every cell's result to its line position. That is what forces the "first cell reddened" discipline in the tables above, and it is what leaves the cyclic-termination cell without a mutant of its own.
Refs #1586
Head note. The pushed head is
b967e2fb0b9ebf82a4a3956682a6bbccb516dd3f. Reproduced on a second machine at that head:node scripts/mutation-coverage.selftest.mjsprinted 128 passed, 0 failed, andnode scripts/mutation-proof.mjs --config scripts/mutations/mutation-coverage.jsonprinted "All 57 mutation(s) killed. The suite discriminates." with exit 0. CI has not yet run on this commit.