You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Most assertion blocks in lib/test/run.sh have no covering focused module, so for a change confined to one of them the narrowest covering test is the whole monolith shard. A one-block edit therefore pays minutes of wall clock instead of seconds.
This is the cost-per-execution half that #1923 explicitly hands off. #1923 reduces how many suite executions a run performs; this issue reduces what one execution costs for the commonest kind of change. The two are independent and can land in either order.
Measured 2026-08-24 across five cloud implement runs' Actions transcripts: 13 suite executions consumed 94.5 minutes, 40.6% of all agent wall clock. Twelve of the 13 were the monolith shard; the focused module runner was invoked zero times, because for those surfaces no module existed to invoke. On the cloud tier monolith ran 5.9–12.6 minutes. Measured locally the same day, six existing modules ran 0.60 s, 4.33 s, 4.71 s, 8.93 s, 18.87 s and 81.19 s — median about 6.8 s. Module cost tracks what the assertions do, not how many there are.
This issue is the refinement #1292 asked for. That ticket was closed as not planned because it was an unrefined tracking record: its own body states that no design or scoping work had been done, and that it exists so a measured finding and the direction it points are not lost. Its closure was for lack of scoping, not a decision against extraction.
#1292's measurement is why extraction is the only remaining lever. Measured 2026-08-04 on an 18-core machine, from two back-to-back coordinator runs on one tree: at DEVFLOW_SUITE_PROCESS_BUDGET 4 the coordinator took 488.7 s and at budget 8 it took 368.4 s, but the monolith shard itself took 362 s at budget 4 and 367 s at budget 8. It ran unimpeded at both, so the extra process budget never reached it. At budget 8 monolith is 99.6% of total coordinator wall clock, 367 s of 368.4 s. monolith is a serial floor, and parallelism against it is already spent: the ceiling in lib/test/run-parallel.sh sits at 8 with no headroom above it, and a separate evaluation of paid higher-CPU runners was declined on this basis. The shard cannot be made faster by adding machine, so moving assertions out of it is the only lever left on cost per execution.
Current Behavior
Verified:lib/test/modules/coverage-map.json records assertion blocks with no covering module under the owner value "unmodularized".
Verified:lib/test/run-module.sh refuses a raw self-skip, printing "modules may not self-skip (module contract)" and exiting 1.
Verified:CONTRIBUTING.md names its module-registration checklist "Authoring a new focused module" and its smoke-test step "Focused-runner smoke test".
Verified:lib/test/pin-corpus-lint.py enumerates the audited pin population as "AUDITED_PIN_SOURCES = frozenset(".
Verified:lib/test/mutation-pin-census.py pins that population's size as "EXPECTED_SOURCE_COUNT = 18".
Verified:CLAUDE.md already asks for extraction on a repeat cycle: "cycle on that same surface extracts a durable module instead".
Verified:lib/test/run-parallel.sh caps the suite's process budget at the literal "BUDGET_CEILING=8".
Counted 2026-08-24: 469 run_sh_blocks entries, of which 344 are unmodularized; 17 registered modules exist.
Two candidate selection rules were measured over 120 days of lib/test/run.sh history and both fail, so an implementer should not retry them. Extracting the hottest labels does not converge: 419 of 469 blocks were touched in that window, the top 10 labels carry 12% of edit-events and the top 100 only 54%. Carving modules along co-edited clusters does not converge either: at a real cohesion threshold only 10 clusters emerge, covering 8% of edit-events, and 293 labels holding 92% of edit-events have no strong co-move partner. The reason is structural — an assertion label is an issue number, not a subject name, so two issues touching the same file get different labels and co-editing is sparse by construction.
Desired Behavior
Labels are grouped by the source subject their assertions are about, and modules are extracted subject by subject in descending edit volume, several per pull request. A change to skills/review, skills/implement or scripts/workpad.py then has a covering focused module that runs in seconds. CONTRIBUTING.md's checklist names every registration surface a new module must update, so the next batch is mechanical rather than a rediscovery.
Measured 2026-08-24 by segmenting lib/test/run.sh so each line belongs to its nearest preceding assertion label and taking the dominant path that label asserts about: 218 unmodularized labels resolve to a subject across 89 subjects. Excluding the lib/test meta-subject, which is largely un-extractable, the top three subjects are skills/review (14 labels, 11.0% of edit-events), skills/implement (16 labels, 6.4%) and scripts/workpad.py (13 labels, 5.7%). The top eight non-meta subjects hold 35.9% of edit-events in 62 labels.
User Impact
A run editing one of these surfaces pays seconds rather than minutes for its iteration test, on the tier where a run can also die mid-way on an expired credential. The saving compounds: the fix loop runs its test step once per iteration.
Technical Context
Scope note: The files and details below are the known starting points, not the full list. Before implementing, trace the change through the codebase to find every affected call site, consumer, and layer — this issue maps the work, it does not bound it.
Dependencies — none beyond the existing module harness and coverage ratchet.
Cross-layer Impact — the shell test suite, the CI shellcheck file list, the module registry, and the contributor documentation.
Two historically expensive costs are already paid down: the shared fixture helpers were promoted into lib/test/module-harness.sh by issue #695, and python3 lib/test/coverage_map_guard.py . --fix repairs a coverage-map ratchet violation mechanically.
Three classes of block cannot move, and the extraction skips them. A block containing a blocking-gate self-skip cannot, because the module runner makes a raw skip fatal and only a host-capability condition is expressible, through module_host_capability_skip; counted 2026-08-24, lib/test/run.sh holds 11 such invocations across the six label families #671, #1219, #719, #1557, #434 and #1621, against 36 host-capability invocations that are portable. A block testing the module system itself cannot move, on circularity. A whole-population meta-guard cannot move; lib/test/modules/harness-python-guards.inventory.md lists 14 of these in its "Deliberate exclusions" table and states the criterion: a guard is extraction-eligible when its subject is a specific code unit and its verification is self-contained.
Three registration surfaces are undocumented and a new module breaks the suite without them: AUDITED_PIN_SOURCES in lib/test/pin-corpus-lint.py, EXPECTED_SOURCE_COUNT in lib/test/mutation-pin-census.py, and the AUDITED tuple in lib/test/test_mutation_pin_census.py. CONTRIBUTING.md's smoke-test step is also stale: it asks for a per-module test "matching the shape the existing module tests use", while lib/test/test_module_runner.py carries exactly one such test, hard-wired to the workflow-flight-recorder module.
Making CLAUDE.md's repeat-cycle extraction rule fire mechanically is follow-up work, not part of this issue. On 2026-08-24 one run executed the monolith shard six times against a single surface and added no module, so the rule is inert today.
Acceptance Criteria
The three subjects this issue extracts are skills/review, skills/implement and scripts/workpad.py, as the modules review-contract, implement-contract and workpad-cli.
A tracked script derives, for every lib/test/run.sh assertion label that lib/test/modules/coverage-map.json marks unmodularized, the dominant repository path its assertions name, and prints the labels grouped by that path with each group's label count.
Each of review-contract, implement-contract and workpad-cli runs green through lib/test/run-module.sh and reports a passing tally equal to its minimum_assertions floor in scripts/workflow-flight-recorder-registry.json.
Every assertion label extracted by this change is asserted by its module, is asserted nowhere in lib/test/run.sh, and names that module as its owner in lib/test/modules/coverage-map.json.
Every assertion label of the three subjects that this change leaves resident in lib/test/run.sh is recorded in the relevant module's inventory file together with the reason it stayed.
The whole suite's total passing-assertion tally after the change is at least its tally before, so the move drops no assertion.
CONTRIBUTING.md's "Authoring a new focused module" checklist names AUDITED_PIN_SOURCES in lib/test/pin-corpus-lint.py, EXPECTED_SOURCE_COUNT in lib/test/mutation-pin-census.py, and the AUDITED tuple in lib/test/test_mutation_pin_census.py as surfaces a new module must update.
CONTRIBUTING.md's focused-runner smoke-test step describes the single shared real-runner test that lib/test/test_module_runner.py carries, and requires no per-module test the tree does not have.
The three new module files are in the explicit shellcheck file list in .github/workflows/ci.yml.
The full test suite passes and shellcheck reports no warning on every changed shell file.
Implementation Notes
Approach — write the subject-grouping script first and run it to get the label sets for the three subjects. Then, in one commit per batch following the 5f02f1e62 precedent, move each subject's eligible assertion blocks into a new module file, leaving blocked labels behind with an inventory note. Register all three modules against one set of registration edits: registry entry with a probed floor, the mirrored call-site floor literal, the CI shellcheck list, the coverage map, the two pin-census populations, and the pin-lint audited set. Rename assert_pin_unique to devflow_module_pin_unique and pin_count to devflow_module_pin_count in each moved body, and rebuild any skill bundle the moved blocks used through devflow_module_build_bundle.
Relevant files — this touches lib/test/run.sh, three new files under lib/test/modules/ with their .inventory.md siblings, lib/test/modules/coverage-map.json, scripts/workflow-flight-recorder-registry.json, .github/workflows/ci.yml, lib/test/pin-corpus-lint.py, lib/test/mutation-pin-census.py, lib/test/test_mutation_pin_census.py, CONTRIBUTING.md, and a new script for the subject grouping, plausibly beside lib/test/coverage_map_guard.py whose derive_labels already locates assertion labels.
Code Patterns — lib/test/modules/harness-python-guards.inventory.md is the model inventory, recording both what moved and what deliberately stayed. lib/test/coverage_map_guard.py's derive_labels is the label-location pattern the grouping script reuses.
Testing Strategy — the extraction's own regression proof is the whole-suite tally comparison in the acceptance criteria, run before and after: a moved assertion that stops executing shows up as a lower total. Each new module gets its floor from the documented over-floor probe, so a silently under-executing module fails its own boundary. For the grouping script, unit-test it against a synthetic lib/test/run.sh fixture holding labels whose assertions name known paths, and assert the emitted grouping, the label counts, and the handling of a label whose assertions name no repository path at all.
Documentation Needed — CONTRIBUTING.md.
Potential Gotchas — the coverage map keeps unmodularized for a partially extracted label, so leaving one assertion of a label behind silently forfeits that label's routing. A merge conflict in lib/test/modules/coverage-map.json is not repaired by the --fix flag. The floor literal lives in two places, the registry and the run.sh boundary call, and they are one coupled contract.
Problem Statement
Most assertion blocks in
lib/test/run.shhave no covering focused module, so for a change confined to one of them the narrowest covering test is the wholemonolithshard. A one-block edit therefore pays minutes of wall clock instead of seconds.This is the cost-per-execution half that #1923 explicitly hands off. #1923 reduces how many suite executions a run performs; this issue reduces what one execution costs for the commonest kind of change. The two are independent and can land in either order.
Measured 2026-08-24 across five cloud implement runs' Actions transcripts: 13 suite executions consumed 94.5 minutes, 40.6% of all agent wall clock. Twelve of the 13 were the
monolithshard; the focused module runner was invoked zero times, because for those surfaces no module existed to invoke. On the cloud tiermonolithran 5.9–12.6 minutes. Measured locally the same day, six existing modules ran 0.60 s, 4.33 s, 4.71 s, 8.93 s, 18.87 s and 81.19 s — median about 6.8 s. Module cost tracks what the assertions do, not how many there are.This issue is the refinement #1292 asked for. That ticket was closed as not planned because it was an unrefined tracking record: its own body states that no design or scoping work had been done, and that it exists so a measured finding and the direction it points are not lost. Its closure was for lack of scoping, not a decision against extraction.
#1292's measurement is why extraction is the only remaining lever. Measured 2026-08-04 on an 18-core machine, from two back-to-back coordinator runs on one tree: at
DEVFLOW_SUITE_PROCESS_BUDGET4 the coordinator took 488.7 s and at budget 8 it took 368.4 s, but themonolithshard itself took 362 s at budget 4 and 367 s at budget 8. It ran unimpeded at both, so the extra process budget never reached it. At budget 8monolithis 99.6% of total coordinator wall clock, 367 s of 368.4 s.monolithis a serial floor, and parallelism against it is already spent: the ceiling inlib/test/run-parallel.shsits at 8 with no headroom above it, and a separate evaluation of paid higher-CPU runners was declined on this basis. The shard cannot be made faster by adding machine, so moving assertions out of it is the only lever left on cost per execution.Current Behavior
lib/test/modules/coverage-map.jsonrecords assertion blocks with no covering module under the owner value "unmodularized".lib/test/run-module.shrefuses a raw self-skip, printing "modules may not self-skip (module contract)" and exiting 1.lib/test/module-harness.shalready carries the shared fixture helpers a module needs, under the header "Shared fixture helpers promoted from lib/test/run.sh (issue Extract the run.sh installer/workflow-wiring region into a focused test module and enforce coverage-map block ownership #695)".CONTRIBUTING.mdnames its module-registration checklist "Authoring a new focused module" and its smoke-test step "Focused-runner smoke test".lib/test/pin-corpus-lint.pyenumerates the audited pin population as "AUDITED_PIN_SOURCES = frozenset(".lib/test/mutation-pin-census.pypins that population's size as "EXPECTED_SOURCE_COUNT = 18".CLAUDE.mdalready asks for extraction on a repeat cycle: "cycle on that same surface extracts a durable module instead".lib/test/run-parallel.shcaps the suite's process budget at the literal "BUDGET_CEILING=8".Counted 2026-08-24: 469
run_sh_blocksentries, of which 344 areunmodularized; 17 registered modules exist.Two candidate selection rules were measured over 120 days of
lib/test/run.shhistory and both fail, so an implementer should not retry them. Extracting the hottest labels does not converge: 419 of 469 blocks were touched in that window, the top 10 labels carry 12% of edit-events and the top 100 only 54%. Carving modules along co-edited clusters does not converge either: at a real cohesion threshold only 10 clusters emerge, covering 8% of edit-events, and 293 labels holding 92% of edit-events have no strong co-move partner. The reason is structural — an assertion label is an issue number, not a subject name, so two issues touching the same file get different labels and co-editing is sparse by construction.Desired Behavior
Labels are grouped by the source subject their assertions are about, and modules are extracted subject by subject in descending edit volume, several per pull request. A change to
skills/review,skills/implementorscripts/workpad.pythen has a covering focused module that runs in seconds.CONTRIBUTING.md's checklist names every registration surface a new module must update, so the next batch is mechanical rather than a rediscovery.Measured 2026-08-24 by segmenting
lib/test/run.shso each line belongs to its nearest preceding assertion label and taking the dominant path that label asserts about: 218 unmodularized labels resolve to a subject across 89 subjects. Excluding thelib/testmeta-subject, which is largely un-extractable, the top three subjects areskills/review(14 labels, 11.0% of edit-events),skills/implement(16 labels, 6.4%) andscripts/workpad.py(13 labels, 5.7%). The top eight non-meta subjects hold 35.9% of edit-events in 62 labels.User Impact
A run editing one of these surfaces pays seconds rather than minutes for its iteration test, on the tier where a run can also die mid-way on an expired credential. The saving compounds: the fix loop runs its test step once per iteration.
Technical Context
lib/test/run.sh,lib/test/modules/,lib/test/modules/coverage-map.json,lib/test/module-harness.sh,lib/test/run-module.sh,scripts/workflow-flight-recorder-registry.json,.github/workflows/ci.yml,lib/test/pin-corpus-lint.py,lib/test/mutation-pin-census.py,lib/test/test_mutation_pin_census.py,lib/test/test_module_runner.py,CONTRIBUTING.md.5f02f1e62(issue Continue lib/test/run.sh modularization: a measured first tranche of four low-risk sections #746) extracted four modules in one commit against one set of registration edits; commitdbc18f999is the single-module shape.Two historically expensive costs are already paid down: the shared fixture helpers were promoted into
lib/test/module-harness.shby issue #695, andpython3 lib/test/coverage_map_guard.py . --fixrepairs a coverage-map ratchet violation mechanically.Three classes of block cannot move, and the extraction skips them. A block containing a
blocking-gateself-skip cannot, because the module runner makes a rawskipfatal and only a host-capability condition is expressible, throughmodule_host_capability_skip; counted 2026-08-24,lib/test/run.shholds 11 such invocations across the six label families #671, #1219, #719, #1557, #434 and #1621, against 36 host-capability invocations that are portable. A block testing the module system itself cannot move, on circularity. A whole-population meta-guard cannot move;lib/test/modules/harness-python-guards.inventory.mdlists 14 of these in its "Deliberate exclusions" table and states the criterion: a guard is extraction-eligible when its subject is a specific code unit and its verification is self-contained.Three registration surfaces are undocumented and a new module breaks the suite without them:
AUDITED_PIN_SOURCESinlib/test/pin-corpus-lint.py,EXPECTED_SOURCE_COUNTinlib/test/mutation-pin-census.py, and theAUDITEDtuple inlib/test/test_mutation_pin_census.py.CONTRIBUTING.md's smoke-test step is also stale: it asks for a per-module test "matching the shape the existing module tests use", whilelib/test/test_module_runner.pycarries exactly one such test, hard-wired to theworkflow-flight-recordermodule.Making
CLAUDE.md's repeat-cycle extraction rule fire mechanically is follow-up work, not part of this issue. On 2026-08-24 one run executed themonolithshard six times against a single surface and added no module, so the rule is inert today.Acceptance Criteria
The three subjects this issue extracts are
skills/review,skills/implementandscripts/workpad.py, as the modulesreview-contract,implement-contractandworkpad-cli.lib/test/run.shassertion label thatlib/test/modules/coverage-map.jsonmarks unmodularized, the dominant repository path its assertions name, and prints the labels grouped by that path with each group's label count.review-contract,implement-contractandworkpad-cliruns green throughlib/test/run-module.shand reports a passing tally equal to itsminimum_assertionsfloor inscripts/workflow-flight-recorder-registry.json.lib/test/run.sh, and names that module as itsownerinlib/test/modules/coverage-map.json.lib/test/run.shis recorded in the relevant module's inventory file together with the reason it stayed.CONTRIBUTING.md's "Authoring a new focused module" checklist namesAUDITED_PIN_SOURCESinlib/test/pin-corpus-lint.py,EXPECTED_SOURCE_COUNTinlib/test/mutation-pin-census.py, and theAUDITEDtuple inlib/test/test_mutation_pin_census.pyas surfaces a new module must update.CONTRIBUTING.md's focused-runner smoke-test step describes the single shared real-runner test thatlib/test/test_module_runner.pycarries, and requires no per-module test the tree does not have..github/workflows/ci.yml.Implementation Notes
5f02f1e62precedent, move each subject's eligible assertion blocks into a new module file, leaving blocked labels behind with an inventory note. Register all three modules against one set of registration edits: registry entry with a probed floor, the mirrored call-site floor literal, the CI shellcheck list, the coverage map, the two pin-census populations, and the pin-lint audited set. Renameassert_pin_uniquetodevflow_module_pin_uniqueandpin_counttodevflow_module_pin_countin each moved body, and rebuild any skill bundle the moved blocks used throughdevflow_module_build_bundle.lib/test/run.sh, three new files underlib/test/modules/with their.inventory.mdsiblings,lib/test/modules/coverage-map.json,scripts/workflow-flight-recorder-registry.json,.github/workflows/ci.yml,lib/test/pin-corpus-lint.py,lib/test/mutation-pin-census.py,lib/test/test_mutation_pin_census.py,CONTRIBUTING.md, and a new script for the subject grouping, plausibly besidelib/test/coverage_map_guard.pywhosederive_labelsalready locates assertion labels.lib/test/modules/harness-python-guards.inventory.mdis the model inventory, recording both what moved and what deliberately stayed.lib/test/coverage_map_guard.py'sderive_labelsis the label-location pattern the grouping script reuses.lib/test/run.shfixture holding labels whose assertions name known paths, and assert the emitted grouping, the label counts, and the handling of a label whose assertions name no repository path at all.CONTRIBUTING.md.unmodularizedfor a partially extracted label, so leaving one assertion of a label behind silently forfeits that label's routing. A merge conflict inlib/test/modules/coverage-map.jsonis not repaired by the--fixflag. The floor literal lives in two places, the registry and therun.shboundary call, and they are one coupled contract.