Skip to content

Make workpad.py own the Verification evidence record's field set #2131

Description

@The01Geek

Problem Statement

As the PRFlow maintainer, I want the Verification evidence: workpad record's required field set to be defined in exactly one place that the implement run actually executes, so that CLAUDE.md and the implement skill stop disagreeing about what a completion-evidence record must carry.

The list of fields this record must carry lives only in CLAUDE.md prose. The shipped implement skill never mentions the record, and scripts/workpad.py writes it as free text that it does not check. So a run can write an incomplete record and nothing notices.

Current Behavior

  • CLAUDE.md, subsection "Recording a whole-suite launch", says each record carries the command, the coordinator's outcome line, the pass/fail/skip tallies, the retained-log root, the elapsed time, and the launch's own start time. For the local tier it says the record carries the full 40-character SHA, each required check's name and conclusion, and the run URL.
  • The record written on issue Grant three drift-detecting helpers to the cloud implement tier and mirror them into the matcher-probe EXTRAS list #2123's workpad by a cloud implement run (comment Grant three drift-detecting helpers to the cloud implement tier and mirror them into the matcher-probe EXTRAS list #2123 (comment)) reads Verification evidence: command=lib/test/run-parallel.sh, outcome=aggregate CLEAN, tallies=22835 passed / 0 failed / 0 skipped, retained-log root=.prflow/tmp/parallel-suite/run-381004-0/logs, elapsed=964s, flight_key=2666de0e…, HEAD 19dd752d9 — no start time, and an abbreviated SHA. Expected: a record carrying every required field. Environment: happens on every tier, because no tool checks the text.
  • Verified: skills/implement/phases/phase-4-documentation.md — "Record the validated flight key on the workpad" is the only evidence-recording instruction the shipped skill carries; no file under skills/ contains the text Verification evidence:, and none under agents/ does.
  • Verified: .prflow/prompt-extensions/implement.md — "governs this run unchanged and is not restated here" is how the repo-local extension defers the record's field list to CLAUDE.md, which no consumer checkout has.
  • Verified: scripts/workpad.py — "lib/fetch-pr-context.sh captures only a bullet's first line" is the only handling --reflection text gets: the kind is checked, the text is never parsed, so a record missing a field is accepted.
  • Verified: scripts/workpad.py — "Record a CI-derived completion-evidence reading" is the help text of --record-completion-evidence-ci, a structured option that validates its fields offline and writes a hidden checkpoint marker; it is the precedent for a structured record; it already carries the local-tier facts (head SHA, each check's name and conclusion, run URL) but none of the launch fields, and writes no Verification evidence: text.

Desired Behavior

scripts/workpad.py update <issue> gains one option, --record-verification-evidence, that owns the record. The caller supplies --command, --outcome, and --run-root (required; --run-root is repeatable for the two-root case, and the literal none is the accepted value for a tier-denied launch and a ceiling-terminated launch, recorded verbatim as run-root=none), plus --tallies, --elapsed, and --started-at, which are accepted when present and not required, because a denied launch and a terminated launch have none. The tool itself stamps the record time in UTC and the full 40-character head from git rev-parse HEAD, writing the literal unestablished for the head when git cannot answer. It refuses, before any PATCH, a call missing a required field, and a call whose --outcome value contains aggregate CLEAN / aggregate FAILED (the coordinator prints these as run-parallel: aggregate CLEAN and run-parallel: aggregate FAILED …) while --run-root is none. Each call appends one note-kind reflection row whose text begins Verification evidence: followed by the fields as key=value pairs; two launches leave two rows.

The shipped implement skill tells the run to call this option at its whole-suite launch site, Phase 4.3 (the only place the shipped skill launches a whole suite; this repository's mid-iteration launches, which CLAUDE.md governs, use the same option). On the local tier, --record-completion-evidence-ci itself appends the Verification evidence: row from the operands it validated (command = the gh pr checks read, outcome = the validated check name and conclusion pairs, run root = the run URL), so the CI reading has one producer. CLAUDE.md's "Recording a whole-suite launch" subsection keeps its tier policy and the four no-run-root paths but names no field: it points at the option's --help for the field set, and names the degraded arm for a local run whose installed plugin predates the option (an unrecognized arguments exit): write the same fields as a --reflection-kind note row that names the plugin version that lacked it.

User Impact

The maintainer reads one field list, in the tool that writes the record, and an incomplete record is refused instead of written. Consumer repos gain the same record on their implement workpads. Nobody else changes how they work.

Technical Context

Scope note: The files and details below are the known starting points, not the full list. Before implementing, trace the change through the codebase to find every affected call site, consumer, and layer — this issue maps the work, it does not bound it.

  • Relevant Classes/Files — scripts/workpad.py (_insert_reflection_bullet, the --record-completion-evidence-ci argparse block and _encode_ci_payload as the structured precedent, _REFLECTION_KINDS); skills/implement/phases/phase-4-documentation.md Phase 4.3; CLAUDE.md subsection "Recording a whole-suite launch"; .prflow/prompt-extensions/implement.md (the sentence naming the note reflection kind for this marker).
  • Architecture Alignment — the same shape as the existing structured evidence options on workpad.py update; the row stays a note-kind reflection because that is the one kind lib/cheap-gate.jq does not count as friction.
  • Dependencies — none new; git rev-parse HEAD is already run by the CI-evidence validator. No new tool grant: scripts/workpad.py is already granted as a vendored-literal head in all three lib/capability-profiles.json profiles, and the option adds only argument-position operands.
  • Data/Schema Considerations — the row is appended, never replaced (unlike the two checkpoint marker families, which replace a prior row). Existing free-text rows on old workpads stay as they are; no reader parses them.
  • Cross-layer Impact — scripts/ (the option), skills/ (one invocation sentence), CLAUDE.md and the prompt extensions that name this marker (pointer wording), and the docs/internal/ pages that name CLAUDE.md as the field-list owner.

Acceptance Criteria

  • scripts/workpad.py update <issue> --record-verification-evidence --command <c> --outcome <o> --run-root <r> appends one ### ℹ️ Notes reflection row whose text starts with Verification evidence: and carries command=, outcome=, run-root=, recorded-at= (UTC, stamped by the tool), and head= (the full 40-character git rev-parse HEAD, with the literal unestablished in its place when git cannot answer); --tallies, --elapsed, and --started-at appear when supplied. git grep -n 'Verification evidence' -- CLAUDE.md skills agents .prflow/prompt-extensions shows no line that enumerates two or more of the record's field names: the option's --help is the field set's single source.
  • A call missing any of --command, --outcome, --run-root, and a call whose --outcome value contains the substring aggregate CLEAN / aggregate FAILED while --run-root is the literal none, exits non-zero with a named breadcrumb and performs no PATCH.
  • Two calls for two launches leave two rows on the workpad; the second does not replace the first. A call with --run-root none and a non-aggregate outcome records run-root=none verbatim.
  • skills/implement/phases/phase-4-documentation.md Phase 4.3, the shipped skill's whole-suite launch site, tells the run to record that launch through this option, and no file under skills/ lists the record's fields, and none under agents/ does. The file stays at or under the 61,750-byte ceiling lib/test/lint-reference-size.py enforces.
  • CLAUDE.md's "Recording a whole-suite launch" subsection names no record field: its per-launch and local-tier bullets point at the option, and the .prflow/prompt-extensions/implement.md sentence that names the note reflection kind for this marker says the option writes it; the CLAUDE.md pointer names the degraded arm for an installed plugin that lacks the option.
  • --record-completion-evidence-ci, on a pass, also appends one Verification evidence: row built from its validated operands, so a local run records its CI reading with one call.
  • The rendered row begins with Verification evidence: so the review extensions' advisory (.prflow/prompt-extensions/review.md, review-and-fix.md), which looks for that text, still finds the record.
  • docs/internal/claude-md-tiered-suite-rationale.md, docs/internal/implement-skill.md, and docs/internal/skills/implement-verification.md name scripts/workpad.py as the owner of the record's fields, and the rationale page's sentence about the launch start time names started-at as an accepted field.
  • A .changeset/*.md file with bump: patch describes the change.

Implementation Notes

  • Approach — add the option beside --record-completion-evidence-ci in scripts/workpad.py, reusing that block's validate-then-write order and the reflection writer for the row, and make the CI option call the same row writer on its pass; then replace CLAUDE.md's field list with a pointer, add one sentence to Phase 4.3, and update the extension sentence and docs pages listed below. The CLAUDE.md edit is made directly by the implementing run under the issue Guard /devflow:implement against nested-Skill tail-call early-stop: add a Skill-completion re-anchor and make the Skill rule exclusionary #366 carve-out.
  • Relevant files — scripts/workpad.py; skills/implement/phases/phase-4-documentation.md; CLAUDE.md; .prflow/prompt-extensions/implement.md; docs/internal/claude-md-tiered-suite-rationale.md; docs/internal/implement-skill.md; docs/internal/skills/implement-verification.md; lib/test/test_python_scripts.py; a new .changeset/ file.
  • Code Patterns — the --record-completion-evidence-ci argparse block and its validate-before-PATCH ordering; _insert_reflection_bullet for the row.
  • Testing Strategy — behavioral tests in lib/test/test_python_scripts.py (the focused test for scripts/workpad.py), written RED first against the stubbed workpad: the refusal cases named in the Acceptance Criteria perform no PATCH; the git-unavailable case records head=unestablished and still succeeds; the two-launch case keeps both rows. The skill edit goes through the writing-skills routing the implement extension prescribes and is checked by lib/test/lint-reference-size.py.
  • Documentation Needed — docs/internal/claude-md-tiered-suite-rationale.md, docs/internal/implement-skill.md, docs/internal/skills/implement-verification.md.
  • Potential Gotchas — the local-tier rule that the CI reading discharges the gate is a PRFlow-repository policy and stays in CLAUDE.md; only the record's shape moves into the tool. The workpad row is read back by lib/fetch-pr-context.sh one line at a time, so the row must stay on one line (the reflection writer already joins lines). skills/implement/phases/phase-1-setup.md is 5 bytes under the ceiling — do not touch it. Record a launch right after it returns: the head is stamped at record time, so a commit between launch and record would stamp a head the suite never ran on. The /prflow:review-and-fix loop is out of scope: it records its verification in iter-<N>.json and keeps doing so.

Generated via /prflow:create-issue (v2.36.15, medium)

Metadata

Metadata

Assignees

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions