Skip to content

incomplete-edit costs about one extra iteration beyond what diff size predicts: find what distinguishes it at declare-done - #1944

Merged
The01Geek merged 9 commits into
mainfrom
issue-1827-incomplete-edit-costs-about-one-extra-iteration
Aug 25, 2026
Merged

incomplete-edit costs about one extra iteration beyond what diff size predicts: find what distinguishes it at declare-done#1944
The01Geek merged 9 commits into
mainfrom
issue-1827-incomplete-edit-costs-about-one-extra-iteration

Conversation

@prflow-implementer

@prflow-implementer prflow-implementer Bot commented Aug 25, 2026

Copy link
Copy Markdown

Summary

  • Retrospective Stage A entries now echo the diff-size fields additions, deletions, and changed_files from the context bundle, making diff size a durable, reproducible control for future analysis instead of requiring a live gh re-fetch.
  • Records the incomplete-edit cost investigation as an internal doc page whose answer is that the current durable records are insufficient to decide whether the category is predictable at declare-done.
  • Purely additive: the fields ride the existing schema_version 3 (already 3 on main since A retrospective entry records nothing about what evidence the analyst had, so diff-present and diff-absent entries pool indistinguishably #1829); no schema bump, and existing .prflow/learnings/ entries are byte-unchanged.

Changes

Retrospective schema (skills/retrospective/SKILL.md): Documents additions, deletions, and changed_files in the Stage A output schema, the echo-straight-from-the-bundle prose, and the jq example construction.
Entry cleaner (lib/clean-entry.jq): Preserves the three size fields from the bundle, each defaulting to null via // (so a producer lacking them still cleans without error, and the keys are always present under schema_version 3).
Analysis doc (docs/internal/incomplete-edit-cost-analysis.md, new): Records the cohort method, the rework metric (efficiency_runs[].iterations, never post_bot_commits) and its ~41% coverage, and concludes the durable records cannot decide the question, naming what a future pass would need.
Tests (lib/test/run.sh): Adds a #1827 block asserting the size-field cleaning contract.
Coverage map (lib/test/modules/coverage-map.json): Registers issue 1827 as unmodularized.
Changeset (.changeset/issue-1827-incomplete-edit-size-fields.md, new): patch / Added.

Resolves

Resolves #1827

Test Plan

  • #1827 block in lib/test/run.sh: additions and deletions survive cleaning when present (120, 34).
  • changed_files survives verbatim by contents and order (not merely by length).
  • A bundle lacking the fields cleans without error, defaulting each to null, with the keys still present (has() pins the schema_version 3 contract).
  • Valid-falsy passthrough: a real 0 / [] survives and is not coerced to null, and changed_files stays an array.

Visual Changes

N/A

Breaking Changes

None

Prompt-surface size

Derived at a64b426bc74f60c31561c5ba6946da06d6948c96 against merge-base 9a2147e221859cbbecff25bf23421487c7cd579b (origin/main). Covered: tracked *.md under skills/, agents/, .prflow/prompt-extensions/.

Path Before After Δ bytes Δ %
skills/retrospective/SKILL.md 23,760 24,035 +275 +1.2%
Whole covered surface 2,142,597 2,142,872 +275 +0.0%

Generated via /prflow:implement (v2.34.14, claude-opus-4-8, low)

github-actions Bot and others added 6 commits August 25, 2026 05:49
…field tests (verbatim changed_files, key-present, valid-falsy)
…-edit-costs-about-one-extra-iteration

# Conflicts:
#	lib/test/modules/coverage-map.json
#	lib/test/run.sh
#	skills/retrospective/SKILL.md
…_version claim (already 3 on main) and clean-entry comment default wording

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@prflow-implementer
prflow-implementer Bot marked this pull request as ready for review August 25, 2026 22:57
@The01Geek

Copy link
Copy Markdown
Owner

/prflow:review

@prflow-implementer

Copy link
Copy Markdown
Author

/prflow:review

@prflow-reviewer prflow-reviewer Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: APPROVE (additive schema echo with matching tests; every acceptance criterion met; no findings)

Issue Compliance

Reviewed against issue #1827: incomplete-edit costs about one extra iteration beyond what diff size predicts — criteria from workpad; scope unchanged. Requirement-based checklist items are included in the verification results below.
Criteria came from the /prflow:implement workpad comment — this run's authoritative set, possibly narrowed from the issue.
No divergence between the workpad's criteria and the issue body's.

All four acceptance criteria are satisfied:

  • AC1 — fields echoed + preserved; absent entry still cleans. lib/clean-entry.jq adds additions/deletions/changed_files, each (.field // null), under the unchanged schema_version: 3; skills/retrospective/SKILL.md echoes all three in both the output-schema block and the example construction, extending the "echo straight from the bundle — do not recompute" instruction to them. The producer lib/fetch-pr-context.sh already emits all three fields (verified against head), so "echoed from the bundle" holds. A bundle lacking the fields still cleans, defaulting each to null.
  • AC2 — analysis doc. docs/internal/incomplete-edit-cost-analysis.md exists and records the rework metric (efficiency_runs[].iterations, max non-null per PR), its coverage (45/109 = 41.3% cohort; 175/489 store-wide), and the §2.3 sweep-selection correlation, concluding with counts that the durable records are insufficient to decide. Table arithmetic is internally consistent.
  • AC3 — metric never from post_bot_commits. The doc's Method section states this explicitly and cites #1440 as the reason; no derivation uses that field.
  • AC4 — learnings byte-unchanged. No file under .prflow/learnings/ is touched by the commit range (verified).

Verification Checklist Results

0 failed, 0 inconclusive. The verifiable claims (producer emits the three fields in the shapes the consumer/tests assume; jq // treats 0/[] as truthy so valid-falsy values survive; schema_version 3 always emits the keys) were confirmed by the Phase-3 agents against head a64b426. Diff classification: engine_self_modifying (touches skills/** and lib/**) → full checklist profile.

Code Review Findings

No findings from any of the five dispatched Phase-3 agents (code-reviewer, silent-failure-hunter, comment-analyzer, pr-test-analyzer, requesting-code-review). The one executable change (lib/clean-entry.jq) is a fail-safe additive echo; the #1827 test block pins presence passthrough, absent→null defaulting, the has() key-always-present contract, changed_files verbatim-by-contents (not length), and valid-falsy (0/[]) passthrough — the classic //-coercion trap is explicitly guarded.

over-grade annotation: no finding flagged
truthfulness sweep: no finding promoted
intra-diff contradiction scan: no contradiction found

Prompt-surface edit routing evidence gate

The diff touches skills/retrospective/SKILL.md, so the DevFlow-repo routing gate applies. Discharged: issue #1827's workpad carries a Writing-skills evidence: marker with all four required dispositions explicit — skill-loaded=yes guidance-applied=yes pressure-scenario=no micro-tests=no. A recorded no never draws a finding on its own. The Verification evidence: advisory marker is also present (whole-suite CLEAN pass on a64b426), so the advisory clause is silent.

Shipped-prose cost audit

The skills/retrospective/SKILL.md addition (~9 lines) adds only what documents the three new fields — the schema entries, the echo-list extension, and the example-construction lines — each carrying the necessary "echo these from the bundle, do not recompute" instruction. No excess prose; no finding.

Verdict Criteria

No verification FAIL/INCONCLUSIVE, no self-contradicting-diff finding, no review-agent finding at or above the critical threshold, and no findings at all → APPROVE (rule 7).

Note: this standalone cloud review ran with the checkout on main; the reviewed head a64b426b was verified via git show <head>:<path> per the standalone PR-number routing. The Phase 0.6 stale-prose lint is disabled by config (prflow_review.stale_prose.enabled: false).

@prflow-reviewer prflow-reviewer Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: APPROVE with notes (small, purely-additive change; two low-value Suggestions, neither gating)

Issue Compliance

Reviewed against issue #1827: incomplete-edit costs about one extra iteration — criteria from workpad; scope unchanged. Requirement-based checklist items are included in the verification results below.
Criteria came from the /prflow:implement workpad comment — this run's authoritative set, possibly narrowed from the issue.
No divergence between the workpad's criteria and the issue body's.

All four acceptance criteria are satisfied:

  • AC1 (fields echoed + preserved; absent-field entries still clean): lib/clean-entry.jq adds additions/deletions/changed_files with // null defaults; skills/retrospective/SKILL.md echoes them verbatim from the bundle; the producer lib/fetch-pr-context.sh actually emits all three (additions/deletions as numbers, changed_files = [.files[].path], an array), so the echo carries real data. Satisfied.
  • AC2: docs/internal/incomplete-edit-cost-analysis.md exists and records the rework metric (efficiency_runs[].iterations), its 41% cohort coverage (45 of 109), and the §2.3 sweep-selection correlation, concluding with a counts-backed insufficiency statement — the escape hatch AC2 permits. Satisfied.
  • AC3: the doc explicitly states the metric "is never derived from post_bot_commits" (citing #1440); no table or finding is sourced from that field. Satisfied.
  • AC4: the diff touches no file under .prflow/learnings/ (verified against the merge-base); existing entries are byte-unchanged. Satisfied.

Verification Checklist Results

8 passed, 0 failed, 0 inconclusive — 0 via lite probe, 8 via agent.

✅ Passed items (8 of 8) — click to expand
  • VC-1: additions/deletions/changed_files echoed from the bundle, preserved by clean-entry.jq, absent-field entry still cleans [lib/clean-entry.jq:133-137]
  • VC-2: incomplete-edit-cost-analysis.md records metric, coverage, and counts-backed conclusion [docs/internal/incomplete-edit-cost-analysis.md]
  • VC-3: rework metric never derived from post_bot_commits [docs/internal/incomplete-edit-cost-analysis.md:53]
  • VC-4: existing .prflow/learnings/ entries byte-unchanged (diff touches none)
  • VC-5: bundle producer emits all three fields [lib/fetch-pr-context.sh:888-890]
  • VC-6: changed_files is a JSON array of paths, surviving verbatim [lib/fetch-pr-context.sh:48]
  • VC-7: #1827 fixtures match the real bundle field names/types [lib/test/run.sh:166-188]
  • VC-8: additive under schema_version 3 (no bump); keys always emitted (has() contract) [lib/clean-entry.jq:17,133-137]

Code Review Findings

🟡 Suggestion / Minor

  1. The end-to-end producer→clean-entry.jq join is not pinned anywhere: the #1827 block tests clean-entry.jq in isolation against hand-authored fixtures, and nothing asserts that lib/fetch-pr-context.sh emits changed_files as a path array (vs a scalar count). If the producer later regressed, clean-entry.jq would silently default to null/pass a wrong-typed value with no RED. The producer is pre-existing and unchanged by this PR, so this is a pre-existing-surface gap, not a regression this change introduces. (raised by 1/5 agents)
  2. No test distinguishes an explicit "additions": null in the bundle from an absent field; under .additions // null both yield null, so the case is genuinely redundant — noted for completeness only, not recommended. (raised by 1/5 agents)

Notes

  • Test evidence (CI, reviewed commit a64b426b): lib + python tests: success, all five shards (monolith, python-pool, modules-pin, modules-large, modules-rest) success, and lint (shellcheck + actionlint + ruff): success. Authoritative per the engine ground-truth block.
  • Valid-falsy correctness: the #1827 tests pin that a real 0 and [] survive uncoerced (jq's // treats them as truthy) — the exact #312/#304 off-switch bug class CLAUDE.md flags for //-based extraction, with a regression that fails closed against a naive if .x then rewrite.
  • Prompt-surface edit routing gate (repo policy): discharged — the linked issue #1827 workpad carries Writing-skills evidence: for skills/retrospective/SKILL.md with all four slots recorded (skill-loaded=yes, guidance-applied=yes, pressure-scenario=no, micro-tests=no); all dispositions explicit, so the gate passes (a recorded no never draws a finding).
  • Verification-evidence advisory (repo policy): Verification evidence: marker present on the workpad (cloud-tier run) — advisory clause silent.
  • Shipped-prose cost audit (repo policy): the skills/retrospective/SKILL.md additions are a minimal, factual schema-field/echo edit (3 schema lines + a one-clause echo-list extension + 3 jq-example lines); every added word carries the instruction it names. No wasteful prose.
  • over-grade annotation: no finding flagged.
  • truthfulness sweep: no finding promoted.
  • intra-diff contradiction scan: no contradiction found.
  • stale-prose lint disabled by config.
  • Deferrals: no Scope-Acknowledged Findings block in the PR body.
  • Dirty-tree backstop: disabled this dispatch (the pre-dispatch snapshot redirect was refused by the harness); the advisory review agents are read-only and made no working-tree change.

Verdict Criteria

  • No verification checklist FAIL or INCONCLUSIVE.
  • No self-contradicting-diff finding (every diff-added claim verified true against HEAD a64b426b).
  • No review-agent finding at or above the configured critical threshold — only two Suggestions, both below threshold and one of them pre-existing surface.
  • → APPROVE with notes.

@The01Geek
The01Geek merged commit 91d98a9 into main Aug 25, 2026
16 checks passed
@The01Geek
The01Geek deleted the issue-1827-incomplete-edit-costs-about-one-extra-iteration branch August 25, 2026 23:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

incomplete-edit costs about one extra iteration beyond what diff size predicts: find what distinguishes it at declare-done

1 participant