feat(fabrika): author the graduate skill and derive its CLI contract (#5103) - #5268
Conversation
No preview deploy
|
|
review-skill: FAIL @ ff3456b — changes-requested Reviewed-head: @ ff3456b Verified PR #5268 against #5103's acceptance criteria + the skill-rigor checklist. One AC row fails; everything else passes. The failure is a documentation-completeness gap in This verdict is BINDING, not advisory. Acceptance criteria — 17 PASS, 1 FAIL, 1 N/A
The failing row, in fullField 4 names five v1 surfaces. Three are recorded, and recorded well: the map tool (both scars, including the criterion's named case — the handler that reports a malformed verdict and returns normally, cited at its exact source lines — plus the dangling-ref check that self-disables on an empty read), the dedup tool (prints nothing and exits
Why this is a real gap and not pedantry: it is the artifact's own standard. The contract twice refuses exactly this silence — "Nothing else. … Stated explicitly, because an absent row reads as nobody checked" and "No local scratch state, so the shared-state law has no surface here. … Stated because an absent answer to that question reads as one nobody asked." Remedy — probably three sentences, and no design change is being asked for. The design almost certainly does avoid the scar, cleanly: this group performs no read-modify-write on an existing issue body at all — the emit verb creates a new issue and appends a new comment, so there is nothing to clobber and no lock to take. Say that, cite what the two v1 surfaces get wrong, and the row closes. A "Considered and deliberately not derived" bullet, or a Grounding line under the emit verb, is the natural home. Sibling precedent, named so this is not applied out of nowhere. Skill rigor — all PASS
Specialist fan-out (ADR 0079)
Resolver claims — checked against the landed siblings, all accurateThe whole design rests on importing the two sibling resolvers, so each claim was verified against the landed
#5267's dependency on #5022/#5023 is a sequencing fact about a downstream ticket and is not held against this PR. It is disclosed in the contract, in NOTES, and in the ticket itself, with "nothing here should be read as describing present behaviour." CI at this head: 39 success, 7 skipped, 0 failing. The one failing item must be addressed before merge — contract completeness on two named surfaces. No design change, no re-run of the evals, no re-measurement. The PR stays open and unmerged; #5103 stays open and assigned. Re-request review once it is satisfied. Since this gate is binding, worth saying plainly: everything else clears comfortably, and the rendered-not-authored Decisions mechanism, the spec-digest fix a graded run caught after three review passes had missed it, and NOTES' disclosure of its own coverage gaps are all well above the bar. Verdict-written: 2026-08-10T05:31:45Z |
…5103) Fifth and final skill of the ruled ideation quintet. Synthesis only: runs on a grilling session or a wayfinding map, emits ONE spec issue at status:needs-triage, writes no board state and closes no source (ADR 0246). The name is ruled (#5017 comment 5230781267); the collision with Tracker.graduate and anka-ops flag graduate is recorded and accepted, not an oversight. graduate trail calls the grill read and map read resolvers rather than re-parsing either artifact, so 'is this decision ruled' has one answer in the codebase. The emitted Decisions section is rendered from that resolver output and refused on stdin, which is how #4227's separability criterion is met structurally rather than by convention. No verbs implemented — the lane ends at the spec (#4638). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…run found stranded (#5103) Three reviewers (narrative skill-reviewer, mechanical exit-matrix audit, premise verification) plus a fix-verification pass, then the graded eval runs. The run-found defect: emit bound its marker to the TRAIL digest, so a trail deliberately split across two specs could never graduate its remainder — the one-issue guarantee and the name-the-remainder instruction contradicted. The marker now binds the SPEC digest over the rendered subset, with compose --decisions naming it. Same algorithm, so the two coincide when a spec carries the whole trail. Also: map decisions derive from the map's ## Decisions citations (map read returns no decisions array, so every founder ruling would have rendered as an agent finding); graduate owns its footer and read-back (report's renderFooter carries no source/digest, and readbackMismatch re-asserts the six intake headings); new code 18 for a spec body that no longer matches its trail. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… mis-specified key (#5103) Reading all ten runs side by side, the grader found the skill answering one question two ways: three with-skill runs assumed `ungraduated` and filed on that assumption, while a fourth refused to assume the same thing. The rule read as 'assume when assuming lets you write' — backwards, and eval-4 is the case that proves it, since a trail can read ready and already be graduated. New NEVER-WRITE-ON-AN-ASSUMED-READ anchor names the temptation directly. Also repairs assertion 5.6, which named a ref the pre-run fixture repair had renumbered; both arms were graded generously against the intent. NOTES.md now carries the measured numbers: with-skill 30/30, baseline 22/30, 8 raw / 4 distinct discriminating, +47% tokens — plus the seven terminals no fixture reaches and every annotated leak. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… only half landed (#5103) Pass 2 found 11 blocking. The root cause was mine and worth recording: the script applying the spec-digest change asserted its way to an abort BEFORE writing the file, so that whole batch was silently discarded while I reported success off a later run. The prose and output keys had moved; the mechanics had not. Now actually applied: --decisions is specified (repeatable, not comma-joined, because a map ref contains a space); 18 compares ref-by-ref instead of whole sections, which had reinstated the stranded remainder through a different code; emit parses the body's refs first, computes the spec digest from re-derived entries, then checks 15 against that; the footer carries the spec digest; SKILL steps 1 and 5 no longer contradict step 2. Two independent finds: the footer had dropped 'Filed by an agent', which is ADR 0159's never-auto-close signal and what triage/provenance.ts classifies on — so every filed spec would have read as human-authored. And the marker now carries the refs it covers, so graduate read can answer a coverage question while staying a pure marker read. Trigger optimizer: train 7/13, test 4/7, precision 100%, original returned as best_description. Tenth consecutive flat measurement; description kept. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Pass 3 found the split path stranded one layer higher than the digest fix reached: the terminal-seating table made `state: graduated` a total function onto ALREADY-GRADUATED, so a second run over a deliberately split trail stopped at the terminal even though the digest layer permitted it. The seat now turns on the marker's `covers`, not on `state`. Also: the ref parse that decides which subset a spec covers had no stated grammar and no failure code — both now specified, anchored on the bolded provenance token so a map ref carrying a space and a '#' survives; emit's ordered behaviour never mentioned appending the footer the read-back asserts, and the footer is now appended BEFORE the leak scan so the bytes it interpolates are scanned (#3086); and the digest table claimed compose writes the footer, which it cannot — it emits four sections and no footer. Six non-blocking: 18 reads per-ref on all three surfaces and names the drifted ref, the 15 example no longer speaks pre-fix trail language, two stale trail-digest references, an unreachable compose refusal removed, and a bad-ref refusal reseated off TRAIL-BLOCKED (it would have reported the founder hadn't decided when the caller simply typo'd a ref). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… contract Field 4 of #5103 names five v1 surfaces; the contract recorded three. Adds the missing pair to `graduate emit`'s Grounding block: both carry the same scar — the concurrency guard does not live where the write does — and neither has a surface here, because no verb in the group read-modify-writes an existing issue body. Addresses the one FAIL row (AC 10) on #5268.
ff3456b to
32ebfe6
Compare
|
review-skill: PASS @ 32ebfe6 — merge-ready Reviewed-head: @ 32ebfe6 Re-review at the new head. The head moved twice since This verdict is BINDING, not advisory — a PASS here auto-ships with no human approval. What actually changed between the two headsDiffed the reviewed artifact directly across the two heads ( Every other file is byte-identical to the head the prior gate read. The 17 PASS rows and every rigor row carry forward on unchanged text; the new bullet is verified below from scratch. Acceptance criteria — 18 PASS, 1 N/AThe prior verdict's per-row evidence stands unchanged for rows 1–9 and 11–19 (identical bytes). The one row that moved:
The repair, checked at source rather than taken on the author's wordBoth cited scars are real, accurately stated, and cited at exact lines — I read the v1 sources myself at this head rather than accepting the repair note's claim.
The non-applicability statement is earned, not a wave-past. The bullet's claim — no verb in this group read-modify-writes an existing body, every write is a create — checks out against the contract itself: the verb inventory is Worth naming: the bullet contradicts its own brief, correctly. Field 4 introduces the pair approvingly, as "the guarded read-modify-write and coarse-label claim that keep concurrent writers from clobbering one issue body." The contract went to the source and found the opposite — "the guard does not live where the write does" — for both. That is the criterion working as intended ("read what each computes and what it gets wrong"), and it is the strongest evidence available that these scars were derived rather than reverse-engineered from a desired conclusion. Placement. Under Skill rigor — all PASS
§DEV — the row stays
|
Authors the fifth and final skill of the ruled ideation quintet:
graduate— the emission actthat turns a cleared decision trail into ONE buildable spec issue.
Fixes #5103
What lands
claude-plugins/fabrika/skills/graduate/SKILL.md— the judgment layerclaude-plugins/fabrika/skills/graduate/contract.md— the derived CLI contract for four verbsclaude-plugins/fabrika/skills/graduate/NOTES.md— packaging, pricing, open questionsclaude-plugins/fabrika/skills/graduate/evals/— 5 evals, 31 assertions, with fixturesNo verbs are implemented here. The session's lane ends at the spec (#4638), and the implementation
ticket is filed and named below (ADR 0248).
The name is ruled
graduate, per the founder's ruling on #5017 (comment 5230781267) and ADR 0246. The collision withpipeline-cli tracker graduate,anka-ops flag graduateand fog-graduation is a recorded, accepteddecision — no rename and no disambiguating suffix. The one worth holding in mind while reading:
Tracker.graduatecloses an issue and this skill opens one, which is why the contract says"closes nothing" more than once.
The shape
Synthesis only. It runs on either entry surface — a
grillingsession or awayfindingmap —and works on the short path where no map exists at all. It emits exactly one spec issue at
status:needs-triage, writes no board state, and closes no source (ADR 0246).The load-bearing design decision is that
graduate trailcalls thegrill readandmap readresolvers rather than parsing either artifact itself. "Is this decision ruled" then has exactly one
answer in the codebase instead of two that can disagree — and on that question, a disagreement means
one of them licenses synthesizing over an unproven ruling. The cost is a hard sequencing dependency
on #5022 and #5023, stated in the contract and carried in the implementation ticket.
How the #4227 criterion is met structurally rather than by convention. The emitted issue's
## Decisionssection is rendered by the verb from resolver output, and a stdin body carrying itsown
## Decisionsheading is refused (17). So each entry's provenance —ruled(the founder's,ACL-proven upstream) or
established(an agent's answer to a question of fact) — is not something arun can forget to label or get wrong by being careless. There is deliberately no third word for
inferred: a decision nobody ruled and nobody established is an open question, and it makes the
trail
blocked.Two brief fields that had aged, flagged rather than silently followed
grilling(Authoring brief: thegrillingskill — the shared primitive: frontier rounds of numbered questions, each carrying a recommended answer #5019) andwayfinding(Authoring brief: thewayfindingskill — map + decision tickets, frontier gated by blocking edges (fog only) #5018) landed onmain; their contracts are now this skill's actual inputinterface. The v1 tools remain the right scars to design against and are cited as such in the
Grounding blocks — they are simply no longer where the input shape comes from.
skill-conventions.md§2 now reads"There is no line count" (ruling on fabrika sizing convention is unenforceable by its own ruled gate — line band vs word band #4701, jointly fabrika: the 7-140 SKILL.md line band is dead in practice — 9 of 12 landed skills exceed it; re-price or replace the bound #5219), and the authoring-brief contract's
own field 6 states a brief may carry no sizing acceptance criterion. Authored against the
live rule — the structural split.
SKILL.mdis 268 lines / ~2,740 words, inside the ~1,000–3,000word gate and in line with the landed corpus (front-door 276, plan-epic 304, governance 315).
Reachability
graduateroutes asfabrika:graduateby plugin enablement and skill auto-discovery — no routingwiring is needed or missing, and no file in the repo routes any fabrika skill by path. The known v1
symlink collision (v1 and fabrika skills model-invocable under one bare name) is already filed at
#4829 and is that issue's decision, not this skill's. There is no v1
graduateskill, sograduateitself has no v1/v2 name collision.What was deliberately not widened
graduateemits no verdict marker and gates no merge, sowire/verdict-marker.ts'sNAMESPACEregex, its separate
NAMESPACE_PREFIXESgate,SHIP_NAMESPACESandreview/advisory.ts'sFIRST_LINEare all left alone — widening any would create the second human gate #4631 rules out.report'sREQUIRED_SECTIONSis likewise not widened: a spec body carries four headings, not thesix intake ones, so this group owns its own section list rather than giving a shared constant two
incompatible meanings.
Review and measurement
skill-reviewerran before this PR opened, handedskill-conventions.mdand the landedgrillingsibling as explicit calibration — then ran a second time over the fixed artifacts, alongside a
parallel mechanical exit-matrix audit, a premise-verification pass over the fixtures, and a narrow
fix-verification pass. The four reviewers overlapped on almost nothing.
Evals: 5 cases, 31 assertions, both arms. with-skill 30/30, baseline 22/30. Discriminating
8 raw / 4 distinct (1 vocabulary, 3 substance) — the terminal-token assertion is one property
restated per eval, so counting rows would overstate lift 5×. Cost +47% tokens, +36% wall-clock.
Trigger optimizer: train 7/13, test 4/7, precision 100% throughout; the tool returned the
original description as
best_description, so it is kept. Tenth consecutive flat measurementacross fabrika skills.
Full numbers, every annotated leak, the seven terminals no fixture reaches, and the two defects
found outside the scorecard are in
NOTES.mdand the handoff comment on #5103.Implementation ticket
Filed per ADR 0248: #5267 — the four verbs, the exit table, the
graduate-emittedwireformat, and the #5022/#5023 sequencing dependency.