feat: step the balanced tier down to effort medium (2.5.13)

[jstrunk]

Closes #607.

What

Steps the balanced tier — the two review-only agents (aidlc-architecture-reviewer-agent, aidlc-product-lead-agent) — down to medium reasoning effort on every harness, instead of inheriting the session's effort. Models are unchanged (Sonnet-class everywhere).

Harness	Before	After
Claude Code	model: sonnet, no effort: line (session effort)	model: sonnet + effort: medium
Codex	model = "openai.gpt-5.4", no effort key	+ model_reasoning_effort = "medium"
opencode	model pin, no variant:	+ variant: medium
Kiro CLI/IDE	chat.modelDefaults sonnet-4.5 → high	sonnet-4.5 → medium (the balanced/templated collapse becomes a no-op — both tiers now pin medium)

Why

Field data from a 12-unit intent (2.2.17, Claude Code, xhigh session): a restarted nfr-design review spent ~36 of 39 minutes in time-to-first-token at ~140k context, and 11/12 nfr-requirements reviews returned READY — session-level effort was buying latency, not verdicts, on work the tier itself defines as "novel input judged against explicit criteria". Since 2.3.5 the review contract is checklist- and evidence-grounded, exactly the shape that does not benefit from maximum-effort deliberation. Full rationale and data in #607.

The judgment tier is untouched — the "never silently downgrade the session" contract still holds where output cascades downstream.

Why keep balanced as a tier at all, when it now projects identically to templated? The tier names classify the WORK, not the dial (aidlc-tiers.ts states this as a design rule): templated is pattern-following output whose methodology already lives in knowledge (delivery plans, CI/CD YAML, runbooks), while balanced is reviewer-shaped work — novel input judged against explicit criteria. Re-tiering the reviewers to templated would make the dials coincide by making the vocabulary lie, and a future contributor adding a review-only agent should be able to write tier: balanced without knowing today's model lineup. Two tiers coinciding in projection is already a designed-for state — balanced and templated have collapsed onto the same model on Kiro since 2.3.1 (the documented collapse rule); after this PR they coincide on every harness. Keeping the classification correct preserves the tuning seam: if a later lineup wants reviewer work on a different dial (a mid-size model at high effort, say), it's a one-line table edit rather than a re-tiering plus roster-doc rewrite.

Escape hatch (documented in docs/reference/05-agent-system.md): a user who wants session effort back on reviews deletes the projected effort: line in the installed .claude/agents/aidlc-*-agent.md (or the harness equivalent).

Changes

• core/tools/aidlc-tiers.ts — TIER_PROJECTIONS.balanced effort/variant pins; KIRO_TIER_EFFORT.balanced high → medium; header rationale + Kiro collapse comments rewritten.
• harness/codex/emit.ts — stale comment fix.
• Test repins: t216 (balanced effort pinned), t220 (EXPECTED.balanced all flavors, kiroModelDefaults, codex shipped-bytes now asserts the effort key present), t148 (cli.json pin).
• Docs: docs/reference/05-agent-system.md, 14-claude-features.md, docs/guide/06-agents.md, 13-customization.md, docs/guide/harnesses/kiro-cli.md, docs/harness-engineering/03-adding-an-agent.md, docs/reference/agents/README.md.
• Version 2.5.1 + CHANGELOG entry + README badge; all dists regenerated.

Testing

• Full smoke + unit tiers: 173 files, 4256 assertions, 0 failed.
• Targeted t216 / t220 / t148 / t68: 70 assertions, 0 failed.
• bun scripts/package.ts --check clean; bun run typecheck and bun run lint exit 0.

[jstrunk]

Rebased onto current `v2` (now at 2.5.11). Re-bumped to 2.5.13 since the original 2.5.1 slot was consumed upstream (per the coordination note, this number was pre-assigned to avoid colliding with sibling PRs rebasing in parallel).

Beyond the mechanical version/changelog/README bookkeeping, this rebase also had to reconcile the tier-projection feature itself against #606 (merged after this branch was authored), which removed all Kiro model pins so every tier now inherits the session model on Kiro. Resolution: the balanced-tier medium-effort pin still applies to Claude Code, Codex, and opencode as designed; on Kiro it's inert (no model surface left to carry it), matching the now-dormant `KIRO_TIER_EFFORT`/`kiroModelDefaults()` state. Updated the affected docs and tests (`05-agent-system.md`, `14-claude-features.md`, `agents/README.md`, `kiro-cli.md`, `13-customization.md`, `03-adding-an-agent.md`, `t220`, `t148`) to match.

Verified green: `bun scripts/package.ts --check` (dist drift), `bun tests/gen-coverage-registry.ts --check`, `bun run check` (package drift + typecheck + lint), and `bash tests/run-tests.sh --smoke --unit` (178 files, 4396 assertions, 0 failures). No reviewer findings to address — this was a clean rebase plus the upstream-conflict reconciliation above.

[leandrodamascena]

Thanks for the thoughtful proposal and the field data behind it. However, we want to preserve the tier contract established in #546: balanced inherits the session effort, while templated is the only tier that explicitly reduces it.

Pinning balanced to medium would override the user’s selected session effort and make balanced and templated operationally equivalent on these harnesses. For that reason, we are keeping the current inherit-by-default behavior.

I'm closing this PR, feel free to reopen if needed.

[jstrunk]

@leandrodamascena Thank you for engaging with this and for the clear rationale. You asked for data demonstrating a tangible improvement — I went back to the field engagement where this pin ran for a month and extracted the timing evidence from the AI-DLC audit ledgers plus the harness's own per-subagent records (toolUseResult.totalDurationMs, observed message.model per transcript line, and the per-turn effort field). 609 subagent dispatches, 210 reviewer runs, Jul 3–28. On that data I'd like to ask that this be reopened — not as-is, but reworked into a dedicated tier for the two review agents (proposal at the end), which I think resolves both contract objections.

The measured improvement

Cleanest cut first: the same model on both sides of the config change, so the only variable is effort. The architecture reviewer ran on sonnet-5 throughout; on Jul 8 the field install added effort: medium to the two balanced agents:

architecture reviewer on sonnet-5	n	p50	p90	max	p50 turns	max turns
inherited session effort	23	42.0 min	102.7 min	195.8 min	55	116
effort medium	102	4.1 min	14.6 min	27.6 min	24	60

~10× at the median; p90 collapsed from 103 to 15 minutes. Engagement-wide: 52 inherited-effort reviews consumed 47.2 reviewer-hours (54 min mean); 158 medium reviews consumed 15.6 hours (5.9 min mean). One concrete worst case that motivated the change: a restarted nfr-design review spent ~36 of its 39 minutes on time-to-first-token alone, at 140k context under xhigh.

A Claude Code bug (anthropics/claude-code#81995) accidentally gave us the model-vs-effort control: some reviewer dispatches resolved to the session model (Fable 5 / Opus) instead of the pinned sonnet while the effort: medium frontmatter still applied. At medium, every model — sonnet-5 (n=110), fable-5 (n=42), opus (n=6) — landed in the same 4–6 min median band. At inherited effort, even fable-5 blew up (p50 19.8 min, p90 95 min). Latency tracks the effort knob, not the model. That's why this PR touched effort and not the model line.

Review value held: NOT-READY rate 22% before (5/23) vs 28% after (28/100), verdict length stable (~2k output tokens both periods), and the medium reviewers kept catching real blockers (e.g. an IndexError/fake-pass bug in a publish path, caught at iteration 1, verified fixed at iteration 2). All 10 post-change bolts hit their definition of done.

On the two contract objections

"Pinning balanced would override the user's selected session effort." For the 9 judgment-tier agents — where reasoning depth cascades into the product — inheritance is untouched; the user's ceiling still governs everything generative. The data shows reviewer-shaped work is different in kind: novel input judged against explicit criteria doesn't convert extra deliberation into different verdicts (the NOT-READY rate and verdict length above), it converts it into time-to-first-token. That's the same work-shape argument the contract already accepts for templated — this PR extends it one tier by measurement rather than by classification. And the user retains control: the documented per-install override (editing the projected frontmatter, docs/guide/13-customization.md) works in both directions.

"balanced and templated become operationally equivalent on these harnesses." Agreed that's a real cost of the PR as written — and it points at the better shape below.

Proposal: a fourth tier for reviewer-shaped work, instead of repinning balanced

aidlc-tiers.ts's own doctrine is that tier names "describe the WORK, not the dial" — and its balanced doc-comment already singles out the work-shape this data is about: "reviewer-shaped work (novel input judged against explicit criteria)." The evidence says that work-shape has a distinct performance profile from the rest of today's balanced: extra deliberation doesn't change verdicts, it only multiplies time-to-first-token. When one tier contains two work-shapes with different measured dial requirements, the tier vocabulary — not the dial — is what's under-specified.

So rather than reopening as-is: reopen and rework into a review tier (name negotiable) between balanced and templated in the TIERS order, holding exactly the two review-only agents (aidlc-architecture-reviewer-agent, aidlc-product-lead-agent), projecting to the balanced model at effort: medium on the harnesses with an effort surface. This preserves everything #546 established:

• balanced keeps inheriting session effort — its contract is untouched for any future agent classified there.
• templated remains distinct: review judges novel input against criteria; templated emits pattern-following output. Different work, different tier, even if today's projections happen to share dials on some harnesses (the module's Kiro collapse rule already accepts that kind of per-harness coincidence).
• The user's session ceiling still governs everything generative — the 9 judgment agents are untouched, and tier_cap clamping composes naturally with a fourth rung.
• No opt-in knob needed: end users only ever copy a dist/<harness>/ tree, so a pack-time setting would reach nobody — the tier table is the maintainers' single tuning point, which is exactly where this decision belongs.

This is a small diff (one TIERS entry, one projection row, two agent frontmatter lines, the t216/t220 projection pins, docs) and I'm happy to rework this PR into that shape.

A consequence for #613: on this data the two PRs only work as a pair, so I'd suggest deciding them together. At medium effort there is a clean gap between normal and runaway reviews (p90 40 turns, max 60), so a 40–60 cap is a real backstop. At inherited effort there is no such gap — legitimate reviews reached 116 turns (p90 112), so a cap of 40 truncates a third of them and a "safe" ~120 cap protects against essentially nothing. The maxTurns half of #613 is only useful if this lands in some form; its §12a missing-verdict guard has standalone value either way (it catches silently incomplete reviews from any cause, not just cap truncation). Happy to share the full methodology writeup, extraction scripts, and per-dispatch CSV.