docs(eval): define the committed scorecard series (#4765)

[usirin]

Fixes #4765

What this lands

The scorecard's defining half was already ruled on the issue (founder, 2026-08-02): one
committed JSON file per eval run at claude-plugins/fabrika/reports/eval/<date>.json, format = the
existing Scorecard type serialized by toJson. The ruling lived only in issue comments,
so a builder of #4679/#4680/#4681 still had to go find it — or invent a second on-disk shape, which
is the exact failure the issue was filed about.

This PR is the doc unit the newest disposition on the issue named as the residue: write the
already-ruled facts down where the builders read, restate the ruled build order, and name what is
still open rather than settling it.

Two files, no code:

• packages/fabrika-cli/src/eval/README.md — a new ### The committed scorecard series section
  under the existing report docs: the path, why it is committed rather than a PR comment, why it is
  colocated with the fabrika plugin, what toJson emits today (described, not prohibited — see
  below), the proposed YYYY-MM-DD UTC date spelling and the argument for it, a three-row table of
  which issue produces and which consume, and an explicit deliberately-not-decided list.
• .glossary/TERMS.md — one committed scorecard row naming the artifact class, with the "Not"
  column separating it from the in-memory Scorecard type / --json output, from a PR comment, and
  from the repo-root reports/.

What is ruled vs what this unit only proposes

The unit's premise is that nothing new is decided here, so the two places where a description could
be misread as a ruling are marked as such in both files:

• The byte contract is a description of today's serializer, not a prohibition. Scorecard at
  head is {decisionRef, framing, baseline, cells} and toJson is a bare JSON.stringify — no
  date, no CLI version, no harness version, no per-row source commit. Founder ruling 4 on Decision (founder-decision-fork): what 'done' means for v2, and the eval bar #4637
  ("committed to the repo, dated, pinned to model + CLI + harness version"), epic fabrika eval layer: the execution harness, the incident corpus, and the ruled bar #4649's Given
  block, and committed scorecards, the trend co-gate, and the model-churn re-run contract #4680's open acceptance criteria all require pins the type does not carry. That tension
  predates this PR and is not resolved here — it is now the first entry in the README's
  not-decided list, and committed scorecards, the trend co-gate, and the model-churn re-run contract #4680 owns the call.
• The YYYY-MM-DD UTC date spelling is this unit's proposal. The 2026-08-02 ruling says
  <date>.json and nothing more; Decision (founder-decision-fork): what 'done' means for v2, and the eval bar #4637 says only "dated". The lexicographic-sort argument is stated
  as an argument, and committed scorecards, the trend co-gate, and the model-churn re-run contract #4680 — which already owns the same-date collision rule — confirms or replaces
  it.

The build order, and where it stays authoritative

The order across the four eval-layer children is #4678 → #4680 → #4681, with #4679 an
independent producer that #4681 also blocks on.

The README states it for a reader only. The authoritative topology stays in epic #4649's
## Dependencies block, which the intake formats contract makes its single home — the earlier
amendment on the issue is right that a per-child restatement would create a second source that can
drift, so this PR does not create one either.

The ordering hazard the report named — #4679 (Phase 3) is told to commit a baseline scorecard while
#4680 (Phase 4) is what defines the committed series — is what the written definition actually
removes: with the path, the type and the open questions all recorded now, #4679 knows what it is
writing and what it must wait on.

Constraint on the sibling lanes — stated, not decided here

Both siblings are separate lanes, and this PR deliberately leaves their calls open. Named in the
README's not-decided list so neither finds its decision pre-made:

• #4678's eval-result record has no defined shape, namespace, or storage location #4769 (the eval-result record has no defined shape or namespace) — not constrained. Whether
  a per-run record is a separate artifact or rides inside a committed scorecard file depends on the
  same open pin question above, which is committed scorecards, the trend co-gate, and the model-churn re-run contract #4680's to settle; where it lives and what it looks like is
  #4678's eval-result record has no defined shape, namespace, or storage location #4769's call.
• fabrika grading chain: dispersion and "two-week decline" are undefined, and one gates a merge-blocking red #4766 (dispersion and the "two-week decline" are undefined) — not constrained either way.
  A trend criterion reads a series of these files; this PR records what a file is, not what a decline
  is. Naming the window, the sample floor and the threshold stays entirely fabrika grading chain: dispersion and "two-week decline" are undefined, and one gates a merge-blocking red #4766's.
• committed scorecards, the trend co-gate, and the model-churn re-run contract #4680 keeps three things this PR could have taken but did not: how Decision (founder-decision-fork): what 'done' means for v2, and the eval bar #4637's required pins land
  in the file, the <date> spelling, and what a second run on the same date does to the
  filename.

Scope

• No source file, no corpus JSON, no harness behavior touched. Docs only.
• Nothing under claude-plugins/ is modified, so none of the fenced fabrika skill trees is touched.
• Not §CP: neither packages/fabrika-cli/ nor .glossary/ matches any CODEOWNERS row (verified by a
  live read of .github/CODEOWNERS on main).

Not done here — a blocked cross-issue write, surfaced rather than dropped

The newest disposition asked for the order to be transcribed onto the four children (#4678,
#4679, #4680, #4681) so each names its dependency. Those are comment writes on four issues this run
did not create and was not pre-authorized to write to, so they were not made. Flagging it rather
than silently skipping: if the per-child transcription is still wanted on top of the epic's
## Dependencies block, it needs an explicit pre-authorization for those four numbers.

Deviations

(repair round 1)

• Narrower than one of the verdict's two offered remedies, by design. The review-code: FAIL
  offered either (a) disclose the Decision (founder-decision-fork): what 'done' means for v2, and the eval bar #4637 tension in the not-decided list and soften the byte contract,
  or (b) scope the byte contract to the scorecard body and leave the pin carrier to committed scorecards, the trend co-gate, and the model-churn re-run contract #4680. This
  round does both halves of (a) plus the body-scoping of (b) and still does not state where
  the pins go — deliberately, because picking a carrier would be the same over-reach one level down.
• The <date> convention is kept, relabelled rather than removed. The verdict allowed either
  marking it a proposal or moving it to the not-decided list. It is now both: marked a proposal at
  its site and listed as committed scorecards, the trend co-gate, and the model-churn re-run contract #4680's open call, so a reader who only reads one of the two still sees it
  is unsettled.
• One PR-body claim was corrected, not just the two files. The body previously asserted that a
  #4678's eval-result record has no defined shape, namespace, or storage location #4769 per-run record is "a separate artifact, not extra keys inside these files" — the same
  undisclosed prohibition applied to a second lane. That sentence is now stated as open.
• Nothing else changed: the path, the committed-not-a-comment rationale, the colocation finding, the
  FOUNDER RULING: move the eval harness into fabrika-cli under the Effect-platform conventions — one ruler, not two #4777 cross-reference, the producer/consumer table, the build order, the single-source-of-topology
  note, and the disclosed cross-issue non-action are untouched.

[github-actions]

No preview deploy

• No preview deploy for this PR — its diff touches no deploy-relevant path, so no preview stack was minted and e2e is not applicable. _(a2a2b02)

• web — Stage pr-5243 torn down.

[usirin]

review-code: FAIL @ d5b3322 — not merge-ready

Verified PR #5243 against the acceptance criteria of #4765, graded against the governing disposition (comment 5234585850, chief-of-staff 2026-08-10): the defining half is discharged by the 2026-08-02 founder ruling, and the residue is one doc unit.

Classification. pipeline-cli class-probe classify --namespaces on the 2-file diff resolves to has-code, i.e. one required namespace: review-code. No review-doc, review-skill or review-design (the glossary and packages trees are code-root paths; nothing under apps/web/src/). Diff shape confirmed first-party: 2 files, +48/-0, zero deletions.

Control-plane re-derived independently, both axes. pipeline-cli cp-classify classify returns not-control-plane [path-clear-no-content-source] (exit 3). Path axis: a live read of .github/CODEOWNERS on main matches no row against .glossary/TERMS.md or packages/fabrika-cli/src/eval/README.md (the pipeline-cli enforcement-core row is a different package). Content axis: the ADR-0164 guard-content-probe clause takes .decisions/ ADR bodies as its input and this diff contains none, so the content axis has nothing to decide — not a fail-open path read. Normal PASS/FAIL namespace, not the control-plane advisory lane.

Closing keywords. Case-insensitive (fix|close|resolve)[a-z]* +#[0-9]+ over the PR body yields exactly 1 hit: line 1, Fixes #4765, plain prose (not inside a code span or fence), intended and correct. None of #4678 / #4679 / #4680 / #4681 / #4766 / #4769 is closed by this PR — each is referenced without a keyword.

---

Criteria

• [PASS] The ruled path is transcribed exactly. README and glossary both state claude-plugins/fabrika/reports/eval/<date>.json, verbatim from the founder ruling ("scorecard" is an undefined artifact forward-referenced by four fabrika epic children #4765 comment 5160520603, 2026-08-02).

• [PASS] "Committed, not a PR comment", and why. The README rationale ("the trend co-gate of committed scorecards, the trend co-gate, and the model-churn re-run contract #4680 reads a series, and only a committed series accumulates anything to read back") is a near-verbatim restatement of the ruling's storage clause. No new claim.

• [PASS] Colocation and the dissolved install-exclusion premise. "Colocated with the fabrika plugin, not the repo-root reports directory", plus the shallow-clone finding that placement cannot exclude a file from an install, matches the ruling's "premise that was checked and dissolved" section. The README does not re-open it.

• [PASS] The FOUNDER RULING: move the eval harness into fabrika-cli under the Effect-platform conventions — one ruler, not two #4777 cross-reference. "FOUNDER RULING: move the eval harness into fabrika-cli under the Effect-platform conventions — one ruler, not two #4777 is what moved the eval harness here" checks out: FOUNDER RULING: move the eval harness into fabrika-cli under the Effect-platform conventions — one ruler, not two #4777 is the founder ruling moving the harness out of the v1 pipeline-cli eval-harness tree into packages/fabrika-cli/, and Scorecard / toJson are in fact at packages/fabrika-cli/src/eval/report.ts at this head.

• [PASS] Producer/consumer split across the v1 cost baseline on the same corpus — the published SOTA proof #4679 / committed scorecards, the trend co-gate, and the model-churn re-run contract #4680 / the ruled bar as the merge gate: 100% floor, 90% graded co-gated on trend, cost at or below baseline #4681. The three-row table (the v1 cost baseline on the same corpus — the published SOTA proof #4679 writes the first file; committed scorecards, the trend co-gate, and the model-churn re-run contract #4680 owns series mechanics; the ruled bar as the merge gate: 100% floor, 90% graded co-gated on trend, cost at or below baseline #4681 reads the series in the merge gate) matches committed scorecards, the trend co-gate, and the model-churn re-run contract #4680's body, the ruled bar as the merge gate: 100% floor, 90% graded co-gated on trend, cost at or below baseline #4681's role, and epic fabrika eval layer: the execution harness, the incident corpus, and the ruled bar #4649's Dependencies block.

• [PASS] Build order. "graded axis: the reused grader-agent judgment path at 5 runs and a median #4678 then committed scorecards, the trend co-gate, and the model-churn re-run contract #4680 then the ruled bar as the merge gate: 100% floor, 90% graded co-gated on trend, cost at or below baseline #4681, with the v1 cost baseline on the same corpus — the published SOTA proof #4679 an independent producer that the ruled bar as the merge gate: 100% floor, 90% graded co-gated on trend, cost at or below baseline #4681 also blocks on" matches epic fabrika eval layer: the execution harness, the incident corpus, and the ruled bar #4649's Dependencies block edge-for-edge (Phase 3 graded axis: the reused grader-agent judgment path at 5 runs and a median #4678/the v1 cost baseline on the same corpus — the published SOTA proof #4679; Phase 4 committed scorecards, the trend co-gate, and the model-churn re-run contract #4680 requires graded axis: the reused grader-agent judgment path at 5 runs and a median #4678; Phase 5 the ruled bar as the merge gate: 100% floor, 90% graded co-gated on trend, cost at or below baseline #4681 requires the fabrika incident corpus: the KEEP incidents and verified v1 failure modes as eval cases #4675, deterministic tier: CLI-layer eval cases run once, with no model in the loop #4677, graded axis: the reused grader-agent judgment path at 5 runs and a median #4678, the v1 cost baseline on the same corpus — the published SOTA proof #4679, committed scorecards, the trend co-gate, and the model-churn re-run contract #4680).

• [PASS] No second source of topology created. The README explicitly keeps the authoritative topology in the epic's Dependencies block and says so in-line. Right call, and it honours the 2026-08-02 amendment's warning that a per-child copy can drift.

• [PASS] "Nothing writes one of these files yet." Verified: claude-plugins/fabrika/reports/ has zero entries on main.

• [PASS] Glossary hygiene. One new committed scorecard row; no collision with the existing eval set, incident corpus, regression floor or live stage key rows; the "Not" column correctly separates it from the in-memory Scorecard type / --json output, from a PR comment, and from the repo-root reports directory; the relative link to the eval README resolves from .glossary/. Placement inside the eval block is consistent with its neighbours.

• [PASS] fabrika grading chain: dispersion and "two-week decline" are undefined, and one gates a merge-blocking red #4766 is genuinely left alone. Nothing in the diff constrains dispersion or the two-week decline, and the README says so explicitly.

• [PASS] The disclosed non-action is a correct refusal, not an incomplete unit. The disposition also asked for the order to be transcribed onto graded axis: the reused grader-agent judgment path at 5 runs and a median #4678-the ruled bar as the merge gate: 100% floor, 90% graded co-gated on trend, cost at or below baseline #4681. Those are four issues this run neither created nor was pre-authorized to write to, and the run surfaced the gap on "scorecard" is an undefined artifact forward-referenced by four fabrika epic children #4765 (comment 5234981657) and in the PR body rather than dropping it. Right call twice over: the constraint was real, and the 2026-08-02 amendment on this very issue argues that a per-child restatement would create exactly the second source that drifts. Not held against this PR.

• [PASS] CI at head. Run-evidence bundle PRESENT for d5b33227; no failing check.

• [FAIL] The core claim — "nothing new was decided" — does not hold. The byte contract silently overrides a standing founder ruling, and the not-decided list does not disclose it.

  The README states, as transcribed fact: "The bytes are toJson output, unchanged — exactly the stable JSON shape documented above, with no wrapper object, no added keys and no second serializer." The glossary repeats it in the canonical vocabulary: "whose bytes are the toJson serialization of the in-memory Scorecard unchanged — no wrapper, no added keys".

  What the type actually is, at this head (packages/fabrika-cli/src/eval/report.ts): Scorecard is {decisionRef, framing, baseline, cells}, and toJson is a bare JSON.stringify(scorecard, null, 2). There is no date field, no CLI version, no harness version, and no per-row source commit. A per-cell model exists; the other three pins do not.

  What a committed scorecard is required to carry, by a standing founder ruling that predates the 2026-08-02 one and was never withdrawn — Decision (founder-decision-fork): what 'done' means for v2, and the eval bar #4637, ruling 4, verbatim: "Scorecards committed to the repo, dated, pinned to model + CLI + harness version."

  That is carried forward into epic fabrika eval layer: the execution harness, the incident corpus, and the ruled bar #4649's Given block ("scorecards committed/dated/pinned to model+CLI+harness version — from Decision (founder-decision-fork): what 'done' means for v2, and the eval bar #4637") and into committed scorecards, the trend co-gate, and the model-churn re-run contract #4680's acceptance criteria, still open and unamended on this point: "A suite run emits a scorecard committed to the repo, dated and pinned to model plus CLI version plus harness version; a scorecard missing any pin is rejected rather than committed." And from its amended set: "Each graded row in the committed series carries the commit its result was bound to [...] A scorecard whose graded rows cite no source eval-result record is rejected on the same footing as one missing a pin."

  So two founder rulings are in tension: Decision (founder-decision-fork): what 'done' means for v2, and the eval bar #4637 requires pins the Scorecard type does not have, while "scorecard" is an undefined artifact forward-referenced by four fabrika epic children #4765's ruling says the format is "the existing Scorecard type, unchanged". A file that is toJson output with no added keys cannot satisfy Decision (founder-decision-fork): what 'done' means for v2, and the eval bar #4637 ruling 4 or committed scorecards, the trend co-gate, and the model-churn re-run contract #4680's first acceptance criterion. That tension is real and pre-existing — this PR did not create it. What this PR does is resolve it, in one direction, silently: it hardens "unchanged" into an explicit prohibition ("no wrapper object, no added keys and no second serializer") and lands that prohibition in the builders' README and in the canonical glossary. The not-decided list — the load-bearing part of a transcription-only unit — names the same-day filename collision, #4678's eval-result record has no defined shape, namespace, or storage location #4769's record shape and fabrika grading chain: dispersion and "two-week decline" are undefined, and one gates a merge-blocking red #4766's definitions, but not this. A stateless coder picking up committed scorecards, the trend co-gate, and the model-churn re-run contract #4680 reads a settled byte contract and then either drops the required pins or re-opens a contract the glossary presents as ruled. That is precisely the "mints doctrine a later reader obeys as ruled" failure mode this unit exists to avoid.

  This also makes the PR's #4678's eval-result record has no defined shape, namespace, or storage location #4769 boundary claim stronger than stated: "a per-run record is a separate artifact, never extra keys inside these files" is not a neutral restatement of a ruling — it is the same undisclosed prohibition, applied to a second open lane.

  What would clear it (either is fine — this is a doc unit, and the gate is not prescribing the design):

  • Add the reconciliation to the not-decided list — e.g. "how Decision (founder-decision-fork): what 'done' means for v2, and the eval bar #4637 ruling 4's required date / CLI-version / harness-version pins and committed scorecards, the trend co-gate, and the model-churn re-run contract #4680's per-row source commit land in the file; the Scorecard type carries none of them today, and reconciling the two rulings is committed scorecards, the trend co-gate, and the model-churn re-run contract #4680's lane" — and soften the byte contract to match in both files; or
  • Scope the byte contract to what is actually ruled: the scorecard body is toJson output unchanged, with whether the required pins ride as a wrapper or as added top-level fields left to committed scorecards, the trend co-gate, and the model-churn re-run contract #4680.

• [FAIL] The UTC YYYY-MM-DD convention is presented as ruled, and is not. The README bullets it under "the founder ruled its storage, location and format": "<date> is the run's UTC date, YYYY-MM-DD — which is what makes a lexicographic sort of the directory a chronological sort of the series." The glossary row repeats it.

  The 2026-08-02 ruling says <date>.json and nothing more; Decision (founder-decision-fork): what 'done' means for v2, and the eval bar #4637 says only "dated". No source rules the timezone or the format. The choice is sane and consistent with repo practice (the reports directory uses YYYY-MM-DD-slug.md), and the lexicographic-sort rationale is a good argument for it — but an argument is a decision, and this PR concedes two lines later that filename mechanics are committed scorecards, the trend co-gate, and the model-churn re-run contract #4680's lane ("what a second run on the same date does to the filename" is listed as not-decided). Deciding the filename's date format while deferring the filename's collision rule to another lane is inconsistent. Lower severity than the finding above, and keeping the convention is fine — but say what it is: mark it as this unit's proposal for committed scorecards, the trend co-gate, and the model-churn re-run contract #4680 to confirm, or move it into the not-decided list. Do not leave it reading as founder-ruled.

Run-evidence bundle: PRESENT for head d5b33227 — producer run 31347574310, artifact 9047912491, manifest.commit == head, schemaVersion 1; checks 2/2 pass; tests 2424/2424 passed, 0 failed, 0 skipped.

Failing criteria above must be addressed before this PR can merge. The PR stays open and unmerged; #4765 stays open and assigned. Re-request review once the failing criteria are satisfied.

Scoping note for the repair: both findings are edits to the two files already in the diff — no code, no new file, no re-opened design question. The transcription is accurate everywhere else, and the disclosed cross-issue non-action is correct and needs no repair.

Verdict-written: 2026-08-10T01:47:56Z

[usirin]

review-code: PASS @ a2a2b02 — merge-ready

Re-gate of PR #5243 after repair round 1. The head moved (round 1 gated d5b33227), so the prior review-code: FAIL is dead by SHA and this verdict is re-derived from scratch against a2a2b02b.

Read the PR head (§HEAD): both files under review were sourced from a2a2b02baf2e80dfcd264218537bdc2f49aedcae through a per-run ref (refs/pr/5243-rev2) whose OID was asserted equal to the pinned head before any read, and read object-wise from that ref — never from the launched session's on-disk copy, which was left untouched throughout.

Required-namespace set (emitted at dispatch, from pipeline-cli class-probe classify --namespaces, exit 0): 1 namespace → review-code. Both changed paths are code roots (.glossary/** and packages/** both match HAS_CODE_RE), so the has-docs carve-out applies and no review-doc marker is owed. No skills/** or agents/** path ⇒ no review-skill. No apps/web/src/** path ⇒ review-design neither required nor dispatched. This independently confirms round 1's derivation.

Diff at head: 2 files, +74/−0, zero deletions — .glossary/TERMS.md (+1), packages/fabrika-cli/src/eval/README.md (+73).

Governing contract. #4765 carries no ### Acceptance criteria heading (it is a type:decision report-shaped issue). Flagged as a process gap upstream, not a defect of this PR — the at-least-one-AC floor should have been established at triage. The contract verified against is the newest disposition (chief-of-staff, 2026-08-10T00:14): the defining half is discharged by the 2026-08-02 founder ruling, and the residue is one doc unit.

The two round-1 FAIL rows — both genuinely discharged

• [PASS] Row 1 — the byte contract no longer resolves the live Decision (founder-decision-fork): what 'done' means for v2, and the eval bar #4637 contradiction. All three claimed repairs verified against the live files at head:

  • (a) The README bullet is reframed as a description, not a prohibition. It states what toJson emits today and then, in the same bullet, "Read that as a description of the current serializer, not a prohibition on what a committed file may carry", scoping the settled part to the scorecard body. That is round 1's remedy (a) plus the body-scoping of (b).
  • (b) A first not-decided entry names the tension in full, and every citation is accurate first-party. Verified against source, not taken from the PR's say-so: Decision (founder-decision-fork): what 'done' means for v2, and the eval bar #4637 ruling 4 reads verbatim "Scorecards committed to the repo, dated, pinned to model + CLI + harness version"; epic fabrika eval layer: the execution harness, the incident corpus, and the ruled bar #4649's ## Given block carries "scorecards committed/dated/pinned to model+CLI+harness version"; committed scorecards, the trend co-gate, and the model-churn re-run contract #4680's acceptance criteria read "a scorecard missing any pin is rejected rather than committed" and its amended AC "Each graded row in the committed series carries the commit its result was bound to". The PR's claim about the type is also true at head: packages/fabrika-cli/src/eval/report.ts L106-112 is Scorecard {decisionRef, framing, baseline, cells} with a per-cell model (L80) — no date, no CLI version, no harness version, no per-row source commit; toJson (L315) is a bare JSON.stringify(scorecard, null, 2).
  • (c) The glossary row names the artifact class, it does not fix the byte contract. "Ruled: the location, and that the format is the existing Scorecard type — the file's body is that type's toJson serialization. Not ruled here: whether the file also carries the date / CLI-version / harness-version pins Decision (founder-decision-fork): what 'done' means for v2, and the eval bar #4637 ruling 4 requires ... and how <date> is spelled — both are committed scorecards, the trend co-gate, and the model-churn re-run contract #4680's open calls, so read this row as naming the artifact class, not as fixing its byte contract."
  • NO CARRIER IS NAMED ANYWHERE IN THE DIFF — the load-bearing finding. The single carrier-adjacent sentence is "whether the pins ride as a wrapper, as added top-level fields, or as a change to Scorecard itself is committed scorecards, the trend co-gate, and the model-churn re-run contract #4680's call" — it enumerates the option space and assigns the choice away; it picks none of the three. The glossary row likewise defers both open questions to committed scorecards, the trend co-gate, and the model-churn re-run contract #4680. Not a wrapper, not added top-level fields, not a Scorecard change. The contradiction is surfaced, not resolved.

• [PASS] Row 2 — the UTC YYYY-MM-DD convention is no longer presented as ruled. All three claims verified:

  • Labelled a proposal at its site: "<date> is proposed as the run's UTC date, YYYY-MM-DD — a proposal from this unit, not a ruling", with the correct grounding that the 2026-08-02 ruling says <date>.json and nothing more while Decision (founder-decision-fork): what 'done' means for v2, and the eval bar #4637 says only "dated". The lexicographic-sort rationale is kept explicitly as an argument, not a settlement.
  • Listed in the not-decided list as committed scorecards, the trend co-gate, and the model-churn re-run contract #4680's call ("The UTC YYYY-MM-DD spelling above is this unit's proposal for committed scorecards, the trend co-gate, and the model-churn re-run contract #4680 to confirm, not a ruling").
  • Absent from the glossary entirely — the row says only that "how <date> is spelled" is open.

Round-1 passes — re-confirmed at the new head, not re-litigated

• [PASS] path, committed-not-a-comment, colocation plus the dissolved install-exclusion premise, the FOUNDER RULING: move the eval harness into fabrika-cli under the Effect-platform conventions — one ruler, not two #4777 cross-reference — all present and unchanged.
• [PASS] producer/consumer table (the v1 cost baseline on the same corpus — the published SOTA proof #4679 writes the first file / committed scorecards, the trend co-gate, and the model-churn re-run contract #4680 owns series mechanics / the ruled bar as the merge gate: 100% floor, 90% graded co-gated on trend, cost at or below baseline #4681 reads the series in the merge gate) — consistent with the epic.
• [PASS] build order checked edge-for-edge against epic fabrika eval layer: the execution harness, the incident corpus, and the ruled bar #4649's ## Dependencies: committed scorecards, the trend co-gate, and the model-churn re-run contract #4680 requires graded axis: the reused grader-agent judgment path at 5 runs and a median #4678 (⇒ graded axis: the reused grader-agent judgment path at 5 runs and a median #4678→committed scorecards, the trend co-gate, and the model-churn re-run contract #4680); the ruled bar as the merge gate: 100% floor, 90% graded co-gated on trend, cost at or below baseline #4681 requires the fabrika incident corpus: the KEEP incidents and verified v1 failure modes as eval cases #4675, deterministic tier: CLI-layer eval cases run once, with no model in the loop #4677, graded axis: the reused grader-agent judgment path at 5 runs and a median #4678, the v1 cost baseline on the same corpus — the published SOTA proof #4679, committed scorecards, the trend co-gate, and the model-churn re-run contract #4680 (⇒ committed scorecards, the trend co-gate, and the model-churn re-run contract #4680→the ruled bar as the merge gate: 100% floor, 90% graded co-gated on trend, cost at or below baseline #4681 and the v1 cost baseline on the same corpus — the published SOTA proof #4679→the ruled bar as the merge gate: 100% floor, 90% graded co-gated on trend, cost at or below baseline #4681); the v1 cost baseline on the same corpus — the published SOTA proof #4679 requires only the fabrika incident corpus: the KEEP incidents and verified v1 failure modes as eval cases #4675 and spawn-and-collect: the unattended eval runner feeding the existing oracle and scorecard #4676 (⇒ independent producer). The stated "graded axis: the reused grader-agent judgment path at 5 runs and a median #4678 → committed scorecards, the trend co-gate, and the model-churn re-run contract #4680 → the ruled bar as the merge gate: 100% floor, 90% graded co-gated on trend, cost at or below baseline #4681, with the v1 cost baseline on the same corpus — the published SOTA proof #4679 an independent producer that the ruled bar as the merge gate: 100% floor, 90% graded co-gated on trend, cost at or below baseline #4681 also blocks on" matches exactly.
• [PASS] no second topology source — the README states the order for a reader and names fabrika eval layer: the execution harness, the incident corpus, and the ruled bar #4649's ## Dependencies as the only authoritative home.
• [PASS] "Nothing writes one of these files yet" — verified at head: claude-plugins/fabrika/reports/ holds zero tracked entries.
• [PASS] glossary hygiene — exactly one committed scorecard row (no collision); the relative link ../packages/fabrika-cli/src/eval/README.md resolves at head, and CI's check docs have no dead internal links is green.
• [PASS] fabrika grading chain: dispersion and "two-week decline" are undefined, and one gates a merge-blocking red #4766 genuinely untouched — named in the not-decided list as unconstrained; no dispersion or decline definition anywhere in the diff.
• [PASS] The disclosed cross-issue non-action (not commenting on graded axis: the reused grader-agent judgment path at 5 runs and a median #4678-the ruled bar as the merge gate: 100% floor, 90% graded co-gated on trend, cost at or below baseline #4681) stands as a correct refusal; not reopened.

The fourth ## Deviations entry, new this round

• [PASS] The corrected #4769 body claim is accurate. Round 0's progress comment recorded the old assertion — "#4678's eval-result record has no defined shape, namespace, or storage location #4769 (the per-run eval-result record is a separate artifact, never extra keys inside these files)" — which was the same undisclosed prohibition applied to a second lane. The body at head now reads "not constrained. Whether a per-run record is a separate artifact or rides inside a committed scorecard file depends on the same open pin question above, which is committed scorecards, the trend co-gate, and the model-churn re-run contract #4680's to settle; where it lives and what it looks like is #4678's eval-result record has no defined shape, namespace, or storage location #4769's call", and the README's not-decided list agrees ("nothing here forecloses it"). The correction is real and the replacement claim defers rather than decides.

Standing sub-gates

• [PASS] Closing keywords. Case-insensitive (fix|close|resolve)[a-z]* +#[0-9]+ over the PR body: exactly one hit — line 1, Fixes #4765, plain prose, not inside a code span and not backticked. A widened whole-word scan for fix/close/resolve found one further occurrence, line 36 "is not resolved here", which is not followed by an issue number and is therefore not a closing form. Plain-prose closing set = {#4765} — graded axis: the reused grader-agent judgment path at 5 runs and a median #4678, the v1 cost baseline on the same corpus — the published SOTA proof #4679, committed scorecards, the trend co-gate, and the model-churn re-run contract #4680, the ruled bar as the merge gate: 100% floor, 90% graded co-gated on trend, cost at or below baseline #4681, fabrika grading chain: dispersion and "two-week decline" are undefined, and one gates a merge-blocking red #4766 and #4678's eval-result record has no defined shape, namespace, or storage location #4769 all appear only as bare #N mentions and are NOT closed. The one TERMS.md scan hit (line 284, Fixes #3543 in the pre-existing read-only fanout row) is not in the diff — the only TERMS.md hunk is at line 43, far from line 284, so it is pre-existing and unrelated exactly as reported. No assumption is encoded about whether a backticked keyword is honored (Seam checks pass a Part of #N PR that also arms a closing keyword for #N (silently closed #5183) #5234) — the sole closing keyword here is unambiguously plain prose and intended.
• [PASS] §CP re-derived on both axes — not-control-plane. Path axis: a live read of .github/CODEOWNERS on main matches no row against either .glossary/TERMS.md or packages/fabrika-cli/src/eval/README.md (the /packages/pipeline-cli/src/* and /packages/ci-required/ rows do not reach packages/fabrika-cli/). Content axis: the diff carries zero .decisions/** files, so guard-content-probe has no input and cannot flag guard-touching content. Classifier output: CONTROL_PLANE_TOUCHED='', GUARD_TOUCHING='', CP_FILES_N=2, ADR_N=0, exit 0. This is a genuine path-and-content clear read, not a path-only shortcut (PR docs(decisions): 0249 — trigger coverage lives in the eval set, not a fourth ship-gate part (#4750) #5238's content-axis §CP tonight is the reason not to shortcut it).
• [PASS] deviation-disclosure (§DEV). ## Deviations is present with four entries, plus a dedicated ## Not done here section disclosing the blocked cross-issue write. The two detectable departures this gate could see — the deliberately-narrower remedy and the relabelled <date> convention — are both disclosed, and each open call is assigned to an owning issue. Nothing undisclosed that this gate could see.
• [PASS] unresolved-threads — the review-thread read ran (§ZS) and returned no unresolved substantive thread.
• comment-discipline: not applicable — the diff adds no code comment lines (two markdown docs).
• glossary-freshness: not applicable — no new feature folder, public package, or public export in this diff (and .glossary/TERMS.md is touched regardless).
• session-caching gate: not applicable — no new session-caching path.
• flag-gating: not applicable — "scorecard" is an undefined artifact forward-referenced by four fabrika epic children #4765 carries no **Containment:** marker.

Evidence

Run-evidence bundle: PRESENT for head a2a2b02b — producer run 31348361292, artifact 9048182213, manifest.commit == head, schemaVersion 1; checks 2/2 pass; tests 2424/2424 passed, 0 failed, 0 skipped.

CI at head: all check-runs green, zero failures.

Note: the 7-140 SKILL.md line band was not cited or enforced anywhere in this gate — it is dead (#4701, landed as #5240).

All criteria pass. This PR is merge-ready. review-code does not merge — ship-it is the authorized merge step; merging will auto-close #4765 via Fixes #4765.

Verdict-written: 2026-08-10T02:06:40Z

[usirin]

ship-it: merge-queue ejection detected — PR #5243 was enqueued but the queue dropped it without merging (still open, no longer queued, not merged). Timeline: added_to_merge_queue 2026-08-10T02:11:55Z → removed_from_merge_queue 2026-08-10T02:17:28Z, with no merged event and no squash commit on main.

Cause: a combined-batch CI failure, not a textual conflict. The batch ref gh-readonly-queue/main/pr-5243-c7d90b13 ran CI (run 31349143625) and the unit + client tests job failed at pnpm --filter './apps/**' test:client; ci-required failed as a consequence. This PR's own head (a2a2b02b) is green on all 45 contexts with an all-pass, commit-bound run-evidence bundle, and the diff is two markdown files (.glossary/TERMS.md, packages/fabrika-cli/src/eval/README.md) that no client test can reach — so the failure is a batch-mate's or a flake, not this PR's.

Routing back to repair/re-queue — this is NOT a shipped state (ADR 0132; GitHub "Managing a merge queue").