Skip to content

Most lib/test/run.sh assertion blocks have no focused module, so the narrowest covering test is the whole monolith shard #1928

Description

@The01Geek

Problem Statement

Most assertion blocks in lib/test/run.sh have no covering focused module, so for a change confined to one of them the narrowest covering test is the whole monolith shard. A one-block edit therefore pays minutes of wall clock instead of seconds.

This is the cost-per-execution half that #1923 explicitly hands off. #1923 reduces how many suite executions a run performs; this issue reduces what one execution costs for the commonest kind of change. The two are independent and can land in either order.

Measured 2026-08-24 across five cloud implement runs' Actions transcripts: 13 suite executions consumed 94.5 minutes, 40.6% of all agent wall clock. Twelve of the 13 were the monolith shard; the focused module runner was invoked zero times, because for those surfaces no module existed to invoke. On the cloud tier monolith ran 5.9–12.6 minutes. Measured locally the same day, six existing modules ran 0.60 s, 4.33 s, 4.71 s, 8.93 s, 18.87 s and 81.19 s — median about 6.8 s. Module cost tracks what the assertions do, not how many there are.

This issue is the refinement #1292 asked for. That ticket was closed as not planned because it was an unrefined tracking record: its own body states that no design or scoping work had been done, and that it exists so a measured finding and the direction it points are not lost. Its closure was for lack of scoping, not a decision against extraction.

#1292's measurement is why extraction is the only remaining lever. Measured 2026-08-04 on an 18-core machine, from two back-to-back coordinator runs on one tree: at DEVFLOW_SUITE_PROCESS_BUDGET 4 the coordinator took 488.7 s and at budget 8 it took 368.4 s, but the monolith shard itself took 362 s at budget 4 and 367 s at budget 8. It ran unimpeded at both, so the extra process budget never reached it. At budget 8 monolith is 99.6% of total coordinator wall clock, 367 s of 368.4 s. monolith is a serial floor, and parallelism against it is already spent: the ceiling in lib/test/run-parallel.sh sits at 8 with no headroom above it, and a separate evaluation of paid higher-CPU runners was declined on this basis. The shard cannot be made faster by adding machine, so moving assertions out of it is the only lever left on cost per execution.

Current Behavior

  • Verified: lib/test/modules/coverage-map.json records assertion blocks with no covering module under the owner value "unmodularized".
  • Verified: lib/test/run-module.sh refuses a raw self-skip, printing "modules may not self-skip (module contract)" and exiting 1.
  • Verified: lib/test/module-harness.sh already carries the shared fixture helpers a module needs, under the header "Shared fixture helpers promoted from lib/test/run.sh (issue Extract the run.sh installer/workflow-wiring region into a focused test module and enforce coverage-map block ownership #695)".
  • Verified: CONTRIBUTING.md names its module-registration checklist "Authoring a new focused module" and its smoke-test step "Focused-runner smoke test".
  • Verified: lib/test/pin-corpus-lint.py enumerates the audited pin population as "AUDITED_PIN_SOURCES = frozenset(".
  • Verified: lib/test/mutation-pin-census.py pins that population's size as "EXPECTED_SOURCE_COUNT = 18".
  • Verified: CLAUDE.md already asks for extraction on a repeat cycle: "cycle on that same surface extracts a durable module instead".
  • Verified: lib/test/run-parallel.sh caps the suite's process budget at the literal "BUDGET_CEILING=8".

Counted 2026-08-24: 469 run_sh_blocks entries, of which 344 are unmodularized; 17 registered modules exist.

Two candidate selection rules were measured over 120 days of lib/test/run.sh history and both fail, so an implementer should not retry them. Extracting the hottest labels does not converge: 419 of 469 blocks were touched in that window, the top 10 labels carry 12% of edit-events and the top 100 only 54%. Carving modules along co-edited clusters does not converge either: at a real cohesion threshold only 10 clusters emerge, covering 8% of edit-events, and 293 labels holding 92% of edit-events have no strong co-move partner. The reason is structural — an assertion label is an issue number, not a subject name, so two issues touching the same file get different labels and co-editing is sparse by construction.

Desired Behavior

Labels are grouped by the source subject their assertions are about, and modules are extracted subject by subject in descending edit volume, several per pull request. A change to skills/review, skills/implement or scripts/workpad.py then has a covering focused module that runs in seconds. CONTRIBUTING.md's checklist names every registration surface a new module must update, so the next batch is mechanical rather than a rediscovery.

Measured 2026-08-24 by segmenting lib/test/run.sh so each line belongs to its nearest preceding assertion label and taking the dominant path that label asserts about: 218 unmodularized labels resolve to a subject across 89 subjects. Excluding the lib/test meta-subject, which is largely un-extractable, the top three subjects are skills/review (14 labels, 11.0% of edit-events), skills/implement (16 labels, 6.4%) and scripts/workpad.py (13 labels, 5.7%). The top eight non-meta subjects hold 35.9% of edit-events in 62 labels.

User Impact

A run editing one of these surfaces pays seconds rather than minutes for its iteration test, on the tier where a run can also die mid-way on an expired credential. The saving compounds: the fix loop runs its test step once per iteration.

Technical Context

Scope note: The files and details below are the known starting points, not the full list. Before implementing, trace the change through the codebase to find every affected call site, consumer, and layer — this issue maps the work, it does not bound it.

  • Relevant Classes/Fileslib/test/run.sh, lib/test/modules/, lib/test/modules/coverage-map.json, lib/test/module-harness.sh, lib/test/run-module.sh, scripts/workflow-flight-recorder-registry.json, .github/workflows/ci.yml, lib/test/pin-corpus-lint.py, lib/test/mutation-pin-census.py, lib/test/test_mutation_pin_census.py, lib/test/test_module_runner.py, CONTRIBUTING.md.
  • Architecture Alignment — this is the established extraction pattern, batched. Commit 5f02f1e62 (issue Continue lib/test/run.sh modularization: a measured first tranche of four low-risk sections #746) extracted four modules in one commit against one set of registration edits; commit dbc18f999 is the single-module shape.
  • Dependencies — none beyond the existing module harness and coverage ratchet.
  • Cross-layer Impact — the shell test suite, the CI shellcheck file list, the module registry, and the contributor documentation.

Two historically expensive costs are already paid down: the shared fixture helpers were promoted into lib/test/module-harness.sh by issue #695, and python3 lib/test/coverage_map_guard.py . --fix repairs a coverage-map ratchet violation mechanically.

Three classes of block cannot move, and the extraction skips them. A block containing a blocking-gate self-skip cannot, because the module runner makes a raw skip fatal and only a host-capability condition is expressible, through module_host_capability_skip; counted 2026-08-24, lib/test/run.sh holds 11 such invocations across the six label families #671, #1219, #719, #1557, #434 and #1621, against 36 host-capability invocations that are portable. A block testing the module system itself cannot move, on circularity. A whole-population meta-guard cannot move; lib/test/modules/harness-python-guards.inventory.md lists 14 of these in its "Deliberate exclusions" table and states the criterion: a guard is extraction-eligible when its subject is a specific code unit and its verification is self-contained.

Three registration surfaces are undocumented and a new module breaks the suite without them: AUDITED_PIN_SOURCES in lib/test/pin-corpus-lint.py, EXPECTED_SOURCE_COUNT in lib/test/mutation-pin-census.py, and the AUDITED tuple in lib/test/test_mutation_pin_census.py. CONTRIBUTING.md's smoke-test step is also stale: it asks for a per-module test "matching the shape the existing module tests use", while lib/test/test_module_runner.py carries exactly one such test, hard-wired to the workflow-flight-recorder module.

Making CLAUDE.md's repeat-cycle extraction rule fire mechanically is follow-up work, not part of this issue. On 2026-08-24 one run executed the monolith shard six times against a single surface and added no module, so the rule is inert today.

Acceptance Criteria

The three subjects this issue extracts are skills/review, skills/implement and scripts/workpad.py, as the modules review-contract, implement-contract and workpad-cli.

  • A tracked script derives, for every lib/test/run.sh assertion label that lib/test/modules/coverage-map.json marks unmodularized, the dominant repository path its assertions name, and prints the labels grouped by that path with each group's label count.
  • Each of review-contract, implement-contract and workpad-cli runs green through lib/test/run-module.sh and reports a passing tally equal to its minimum_assertions floor in scripts/workflow-flight-recorder-registry.json.
  • Every assertion label extracted by this change is asserted by its module, is asserted nowhere in lib/test/run.sh, and names that module as its owner in lib/test/modules/coverage-map.json.
  • Every assertion label of the three subjects that this change leaves resident in lib/test/run.sh is recorded in the relevant module's inventory file together with the reason it stayed.
  • The whole suite's total passing-assertion tally after the change is at least its tally before, so the move drops no assertion.
  • CONTRIBUTING.md's "Authoring a new focused module" checklist names AUDITED_PIN_SOURCES in lib/test/pin-corpus-lint.py, EXPECTED_SOURCE_COUNT in lib/test/mutation-pin-census.py, and the AUDITED tuple in lib/test/test_mutation_pin_census.py as surfaces a new module must update.
  • CONTRIBUTING.md's focused-runner smoke-test step describes the single shared real-runner test that lib/test/test_module_runner.py carries, and requires no per-module test the tree does not have.
  • The three new module files are in the explicit shellcheck file list in .github/workflows/ci.yml.
  • The full test suite passes and shellcheck reports no warning on every changed shell file.

Implementation Notes

  • Approach — write the subject-grouping script first and run it to get the label sets for the three subjects. Then, in one commit per batch following the 5f02f1e62 precedent, move each subject's eligible assertion blocks into a new module file, leaving blocked labels behind with an inventory note. Register all three modules against one set of registration edits: registry entry with a probed floor, the mirrored call-site floor literal, the CI shellcheck list, the coverage map, the two pin-census populations, and the pin-lint audited set. Rename assert_pin_unique to devflow_module_pin_unique and pin_count to devflow_module_pin_count in each moved body, and rebuild any skill bundle the moved blocks used through devflow_module_build_bundle.
  • Relevant files — this touches lib/test/run.sh, three new files under lib/test/modules/ with their .inventory.md siblings, lib/test/modules/coverage-map.json, scripts/workflow-flight-recorder-registry.json, .github/workflows/ci.yml, lib/test/pin-corpus-lint.py, lib/test/mutation-pin-census.py, lib/test/test_mutation_pin_census.py, CONTRIBUTING.md, and a new script for the subject grouping, plausibly beside lib/test/coverage_map_guard.py whose derive_labels already locates assertion labels.
  • Code Patternslib/test/modules/harness-python-guards.inventory.md is the model inventory, recording both what moved and what deliberately stayed. lib/test/coverage_map_guard.py's derive_labels is the label-location pattern the grouping script reuses.
  • Testing Strategy — the extraction's own regression proof is the whole-suite tally comparison in the acceptance criteria, run before and after: a moved assertion that stops executing shows up as a lower total. Each new module gets its floor from the documented over-floor probe, so a silently under-executing module fails its own boundary. For the grouping script, unit-test it against a synthetic lib/test/run.sh fixture holding labels whose assertions name known paths, and assert the emitted grouping, the label counts, and the handling of a label whose assertions name no repository path at all.
  • Documentation NeededCONTRIBUTING.md.
  • Potential Gotchas — the coverage map keeps unmodularized for a partially extracted label, so leaving one assertion of a label behind silently forfeits that label's routing. A merge conflict in lib/test/modules/coverage-map.json is not repaired by the --fix flag. The floor literal lives in two places, the registry and the run.sh boundary call, and they are one coupled contract.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions