Skip to content

fix(ce-code-review): stop the reviewer template from embedding the diff twice - #1512

Merged
tmchow merged 1 commit into
mainfrom
tmchow/issue-1509-eval
Aug 22, 2026
Merged

fix(ce-code-review): stop the reviewer template from embedding the diff twice#1512
tmchow merged 1 commit into
mainfrom
tmchow/issue-1509-eval

Conversation

@tmchow

@tmchow tmchow commented Aug 22, 2026

Copy link
Copy Markdown
Collaborator

Every ce-code-review reviewer prompt carried the changed-file list and the full diff twice, then ended with a note telling the reviewer that inline content "may be file paths" to Read from disk. The closing note of the template's <review-context> block named {file_list} and {diff} in slot syntax, so prompt assembly filled them a second time; the reporter measured the second copy at 33% of a nine-reviewer run's payload, and one reviewer stopped to verify the contradiction. The note now refers to the Changed files: and Diff: labels the reviewer actually sees after substitution and states the condition: inline content is authoritative, a lone staged path is Read. This matches the shape the validator template already used.

A contract test pins each placeholder to exactly one occurrence inside the fenced template, so a future mention of {diff} in the note fails bun run test.

Fixes #1509

Validation

bun run test (3540 pass), bun run release:validate clean. The new test fails on the old template line and passes on the new one.

Fresh-agent eval of the reviewer subagent, read-only, against a throwaway repo with a 2-file diff. Prompts were assembled from the real template (pre = main, post = this branch) by filling every {slot}; the pre prompt reproduces the bug (diff present twice). Three arms on Claude (claude -p) and Codex (codex exec --sandbox read-only):

Arm Claude Codex
pre, inline diff Reviewed inline diff; absorbed the false note silently (no path reads, no contradiction reported) Same; re-derived the diff from git diff rather than trusting the prompt
post, inline diff Reviewed inline diff; no path reads Same; also re-derived from git and ran a node probe
post, staged paths First tool calls: Read diff.txt, Read files.txt, then reviewed First command: sed both staged files, then reviewed

No regression on either host; the staged-path instruction still fires on both. The strong tiers absorbed the old contradiction rather than acting on it (matching 8 of 9 reviewers in the report), so this eval shows the fix holds rather than a before/after flip; the reporter's testing reviewer that halted to verify is the observed failure the rewrite removes.

Security Disclosure

No security-relevant changes.

Agent Disclosure

  • Model: Claude Code · claude-fable-5

https://claude.ai/code/session_012uPY9QcVF4ueBgrpegPcvC

…ff twice

The closing note of the review-context block named {file_list} and {diff}
in slot syntax, so prompt assembly filled them a second time and then told
the reviewer the inline content "may be file paths" to Read. The note now
refers to the Changed files: and Diff: labels the reviewer actually sees
and applies only to a lone staged path. A contract test pins each
placeholder to one occurrence inside the fenced template.

Fixes #1509

Claude-Session: https://claude.ai/code/session_012uPY9QcVF4ueBgrpegPcvC
@cursor

cursor Bot commented Aug 22, 2026

Copy link
Copy Markdown

PR Summary

Low Risk
Prompt-template wording and a contract test only. No auth, data handling, or runtime logic changes; residual risk is that reviewers misread staged path vs inline content, which the new note is meant to prevent.

Overview
Stops ce-code-review reviewer prompts from substituting {file_list} and {diff} a second time. The closing note in subagent-template.md used those names in slot syntax, so prompt assembly pasted the full file list and diff twice (a large share of multi-reviewer payload).

The note now refers to the Changed files: and Diff: labels after substitution: inline content is authoritative; a lone staged path should be Read. A contract test requires each of those placeholders to appear only once inside the fenced template.

Reviewed by Cursor Bugbot for commit c85d44a. Bugbot is set up for automated code reviews on this repo. Configure here.

@tmchow
tmchow merged commit dfeae09 into main Aug 22, 2026
5 checks passed
@github-actions github-actions Bot mentioned this pull request Aug 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ce-code-review: reviewer prompt embeds the diff twice and ends with a false instruction

1 participant