P3 severity + summary pinned to top of PR description by ZhenghuaBao · Pull Request #4 · Continuum-AI-Corp/orca-code-review

ZhenghuaBao · 2026-07-16T08:56:16Z

Orca-Code-Review — push 3

Severity	Count	Δ vs previous push
P0	0	0
P1	1	+1
P2	2	-1
P3	0	-1

Tier: STRONG (final pass) — blocked

❌ 1 finding blocks merge

Summary

Two independent, non-blocking improvements toward Codex-parity review quality. Built cleanly on top of the merged #2 (P1/P2 calibration + PRECISION) — those rules are preserved verbatim; this PR only adds to them.

Summary pinned to top of PR description. The per-push Orca-Code-Review summary moves out of a chronologically-buried issue comment into a marker-delimited region at the top of the PR description body (the only element GitHub anchors at the top). New pure merge primitive scripts/inject-summary.mjs prepends the region on first push and replaces it in place afterward, preserving author-written body text; the driver reads the prior region back for the push counter + Δ column and best-effort deletes any legacy summary issue comment on migrated PRs.
P3 severity level. Splits the overloaded P2 into two: P2 = a real-but-conditional bug (genuine defect that fires only under a precondition the code doesn't normally meet), P3 = a pure style/maintainability nit with no behavioral defect. Both remain non-blocking and are still posted. SEVERITIES flows the new level through the gate, run report, and PR summary automatically. The Calibrate P1/P2 severity and tighten precision rules #2 P1/P2 boundary, security-control carve-out, and PRECISION section are kept intact — only the P2/P3 split + a P2-vs-P3 boundary note are added.

Test plan

node --test scripts/*.test.mjs — 148 pass; the 6 failures are pre-existing Windows-only artifacts (network-mock native crashes + one CRLF regex), green on Linux CI.
Self-review action runs on this PR and posts its summary into the top of this description — the real end-to-end test of the summary-at-top wiring.
Confirm findings carry [P0]/[P1]/[P2]/[P3] tags and the summary table shows all four rows.

🤖 Generated with Claude Code

Move the per-push Orca-Code-Review summary out of a chronologically-buried issue comment and into a marker-delimited region at the top of the PR description body, the only element GitHub anchors at the top. A new pure merge primitive (inject-summary.mjs) prepends the region on first push and replaces it in place afterward, preserving author-written body text. The driver reads the prior region back for the push counter and Δ column, and best-effort deletes any legacy summary issue comment on migrated PRs. Co-Authored-By: Claude Opus 4 <noreply@anthropic.com>

Add a P3 level below P2 to mirror Codex. P2 is redefined as a real-but-conditional bug (a genuine defect that fires only under a precondition the code doesn't normally meet); P3 is a pure style/maintainability nit with no behavioral defect. Both stay non-blocking and are still posted. SEVERITIES flows the new level through the gate, run report, and PR summary automatically. This builds on the P1/P2 calibration + PRECISION rules from #2 (kept intact); it only adds the P2/P3 split and the P2-vs-P3 boundary note. Co-Authored-By: Claude Opus 4 <noreply@anthropic.com>

github-actions

🐳 Orca-Code-Review

Found 3 issues in this PR: 🟠 1 P1 · 🟡 2 P2.

_{Reviewed via OrcaRouter — Route Smarter. Ship Safer. Spend Less.}

github-actions · 2026-07-16T08:58:29Z

            const notice = comments.find((c) => c.body && c.body.includes(stale));
            if (!notice) continue;
            try {
              await github.rest.issues.deleteComment({ ...repo, comment_id: notice.id });


🟡 P2 On the first run after upgrading an existing PR, the previous summary state still lives in the legacy issue comment, but this loop deletes that comment before the later extractRegion(pr.body) path looks for prior state in the PR body. With no body region yet, prevRegion is null, so the push number and Δ column reset to push 1 for migrated PRs. Capture the legacy summary body for --prev before deleting it (or delete it only after the new body update succeeds).

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: fadbe4a358

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

…body Address self-review findings on the P3 change: - summary-comment.mjs stored only p0/p1/p2 in the state line, so on push 2+ the P3 Δ column subtracted an undefined prev.p3 and rendered NaN. Store p3 and treat a missing prev.p3 as 0 (back-compat with older state lines). - action.yml's inline renderer used its own P[012] tag regex + a SEV map without P3, so a [P3] finding fell back to a P1 badge — the inline comment disagreed with the gate/summary. Recognize P3 (⚪) in the regex, badge map, and breakdown. - inject-summary.mjs trimStart() stripped meaningful leading whitespace from the author's PR body on first injection; prepend the body verbatim instead. Co-Authored-By: Claude Opus 4 <noreply@anthropic.com>

github-actions

🐳 Orca-Code-Review

Found 4 issues in this PR: 🟡 3 P2 · ⚪ 1 P3.

_{Reviewed via OrcaRouter — Route Smarter. Ship Safer. Spend Less.}

github-actions · 2026-07-16T09:29:43Z

  lines.push("| Severity | Count | Δ vs previous push |", "|---|---|---|");
  for (const s of SEVERITIES) {
-    lines.push(`| ${s} | ${counts[s]} | ${delta(counts[s] - prev[s.toLowerCase()])} |`);
+    lines.push(`| ${s} | ${counts[s]} | ${delta(counts[s] - (prev[s.toLowerCase()] ?? 0))} |`);


🟡 P2 If the previous summary state is hand-edited or otherwise corrupted so that p3 exists but is not numeric, this subtraction can render NaN in the Δ column because the prev validation still only checks p0–p2. Guard the value as numeric here, or include p3 in the state validation so malformed state omits the delta column as intended.

github-actions · 2026-07-16T09:29:43Z

+// repeated injects don't accumulate blank lines between the region and author text.
+const REGION_RE = new RegExp(
+  `${REGION_START}[\\s\\S]*?${REGION_END}\\n?`,
+);


🟡 P2 If the PR body already contains these marker strings before the first bot run, this treats that author-authored block as an existing Orca region: extractRegion() will feed arbitrary text as prior summary state and injectSummary() will replace the block in place instead of prepending the real summary at the top. That can mangle the author's description and leave the summary in the wrong position under the precondition that the author text contains  / . Consider only recognizing regions that also contain the summary marker emitted by summary-comment.mjs, and otherwise prepend a new region.

github-actions · 2026-07-16T09:29:43Z

+          const comments = await github.paginate(github.rest.issues.listComments, {
+            ...repo, issue_number: num, per_page: 100,
+          });
+          for (const stale of ['<!-- orca-code-review-disabled -->', '<!-- orca-code-review-skip -->', '<!-- orca-code-review-block -->', MARKER]) {


🟡 P2 This retires the legacy summary comment before the new PR-description summary is successfully written. On a migration run from the old comment-based summary path, if the later pulls.update fails (for example due to a transient GitHub API error or insufficient PR write permission), the previous summary comment has already been deleted and no replacement is posted. Remove the legacy marker from this pre-update cleanup and delete it only after pulls.update succeeds.

github-actions · 2026-07-16T09:29:43Z

+  REGION_END,
+} from "./inject-summary.mjs";
+
+const SUMMARY = "<!-- orca-code-review-summary -->\n## Orca-Code-Review — push 1\n\n✅ no blocking findings";


⚪ P3 These contract tests build their expectations from the implementation's exported marker constants, so an accidental marker rename would make the tests pass while the bot can no longer find regions already written to existing PR descriptions. Pin the wire-format markers explicitly in the test suite so compatibility-breaking changes fail loudly.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 243094e1c2

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-07-16T09:33:34Z

+  const current = String(body ?? "");
+  if (REGION_RE.test(current)) {


Avoid replacing author-owned marker regions

If an author includes this documented start/end marker pair in their PR description—for example, while showing the summary format in a Markdown example—this branch treats it as bot state and replaces everything between the markers with the generated summary. That silently deletes the author's description content on the next review; use a bot-owned/verifiable region or avoid destructive replacement when the expected summary marker/state is absent.

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-07-16T09:33:34Z

+          const { data: pr } = await github.rest.pulls.get({ ...repo, pull_number: num });
+          const prevRegion = extractRegion(pr.body);


Carry legacy summary state into the migrated description

For a PR that already has the old marker-bearing issue-comment summary, this migration deletes that comment and then reads prevRegion only from pr.body. Because the legacy body is never supplied to --prev or used for push, the first post-upgrade summary resets to push 1 and drops the prior-count delta instead of continuing the PR's history. Retain the legacy comment body before deletion and use it as the fallback previous state during this migration.

Useful? React with 👍 / 👎.

The dedup key stripped only [P0]-[P2] tags, so a re-tagged P3 finding would not merge across exhaustive passes; the report payload omitted the P3 count entirely. Both are gaps in the new P3 severity level. Co-Authored-By: Claude Opus 4 <noreply@anthropic.com>

github-actions

🐳 Orca-Code-Review

Found 3 issues in this PR: 🟠 1 P1 · 🟡 2 P2.

_{Reviewed via OrcaRouter — Route Smarter. Ship Safer. Spend Less.}

github-actions · 2026-07-16T09:52:39Z

  lines.push("| Severity | Count | Δ vs previous push |", "|---|---|---|");
  for (const s of SEVERITIES) {
-    lines.push(`| ${s} | ${counts[s]} | ${delta(counts[s] - prev[s.toLowerCase()])} |`);
+    lines.push(`| ${s} | ${counts[s]} | ${delta(counts[s] - (prev[s.toLowerCase()] ?? 0))} |`);


🟡 P2 If the saved state line is hand-edited or otherwise garbled so that p3 is present but non-numeric, the existing previous-state validation still accepts it because it only checks p0–p2. This subtraction then renders NaN in the P3 delta instead of treating the state as unparseable (or falling back only when P3 is absent for older summaries). Validate the P3 value before using it, while still allowing it to be missing for backward compatibility.

github-actions · 2026-07-16T09:52:39Z

+// repeated injects don't accumulate blank lines between the region and author text.
+const REGION_RE = new RegExp(
+  `${REGION_START}[\\s\\S]*?${REGION_END}\\n?`,
+);


🟡 P2 If the PR body already contains more than one summary region (for example after a manual edit or a previous bad run), only the first region is replaced and the remaining stale region is preserved. That leaves duplicate/conflicting Orca summaries in the description even though this helper is meant to set a single marker-delimited summary region. Consider removing any extra matched regions while replacing the first one.

github-actions · 2026-07-16T09:52:39Z

+          const newBody = injectSummary(pr.body, summaryMd);
+          if (newBody !== (pr.body || '')) {
+            await github.rest.pulls.update({ ...repo, pull_number: num, body: newBody });
          }


🟠 P1 This updates the entire PR description from the pr.body snapshot read earlier in the step. If the author edits the PR description between that pulls.get and this pulls.update, the update will write the stale snapshot plus the new summary and silently discard the author's concurrent edit. Re-read the body immediately before writing (and merge the summary into that latest body, or otherwise use an optimistic/concurrency guard) so user-authored description text is not clobbered.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a846f34ff9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-07-16T09:56:06Z

+          // Preserve the author's description text; only the marker region changes.
+          const newBody = injectSummary(pr.body, summaryMd);
+          if (newBody !== (pr.body || '')) {
+            await github.rest.pulls.update({ ...repo, pull_number: num, body: newBody });


Protect concurrent PR-description edits

If an author edits the PR description after pulls.get has returned at line 1057 but before this update runs, this sends the stale complete pr.body back to GitHub and overwrites the author's newer text. This is a regression from the issue-comment upsert path, which only changed the bot-owned comment; refetch and merge/retry before updating, or otherwise use a version-aware update strategy.

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-07-16T09:56:06Z

+          for (const stale of ['<!-- orca-code-review-disabled -->', '<!-- orca-code-review-skip -->', '<!-- orca-code-review-block -->', MARKER]) {
            const notice = comments.find((c) => c.body && c.body.includes(stale));
            if (!notice) continue;
            try {
              await github.rest.issues.deleteComment({ ...repo, comment_id: notice.id });


Verify legacy summary ownership before deletion

If any PR participant posts an issue comment containing —for example while quoting the old summary format—this loop treats it as a legacy bot summary and deletes the entire author-owned comment. The migration should require bot ownership and/or a valid generated state marker before calling deleteComment.

Useful? React with 👍 / 👎.

ZhenghuaBao · 2026-07-16T11:06:44Z

Merge note — P3 + summary-at-top

This PR ships the P3 severity level (style/maintainability nits, non-blocking) and pins the per-push summary to the top of the PR description. P3 is now consistent across every consumer: severity.mjs, the inline badge, the summary table + Δ state, the gate, the exhaustive dedup key, and the control-plane report.

Both reviewers' P3-completeness findings are resolved. The remaining open threads are deliberately deferred — none are correctness-blocking for this PR:

Concurrent PR-description edit (action.yml:1102) — flagged P1, but the real pulls.get→pulls.update window is sub-second synchronous exec (the review is a separate earlier step), so the clobber probability is near-zero. Real in kind, over-rated in severity.
Marker collision — author pasting our orca-cr-summary markers verbatim (undocumented internals; effectively never hit).
Legacy comment ownership / migration push-counter reset (action.yml:1044/1048/1058) — one-time upgrade edge cases.
Multiple stale regions, hand-edited non-numeric p3 → NaN Δ — corrupted/hand-edited state only.

These cluster into one follow-up: harden the PR-description write path (re-read before update, verify bot ownership, migration state, multi-region cleanup). Tracked for a subsequent PR alongside the whole-repo impact-tracing work.

ZhenghuaBao and others added 2 commits July 16, 2026 16:50

github-actions Bot reviewed Jul 16, 2026

View reviewed changes

chatgpt-codex-connector Bot reviewed Jul 16, 2026

View reviewed changes

Comment thread scripts/severity.mjs

Comment thread scripts/severity.mjs

Comment thread scripts/inject-summary.mjs Outdated

github-actions Bot reviewed Jul 16, 2026

View reviewed changes

chatgpt-codex-connector Bot reviewed Jul 16, 2026

View reviewed changes

github-actions Bot reviewed Jul 16, 2026

View reviewed changes

chatgpt-codex-connector Bot reviewed Jul 16, 2026

View reviewed changes

ZhenghuaBao merged commit 01edbdf into main Jul 16, 2026
1 check failed

		const current = String(body ?? "");
		if (REGION_RE.test(current)) {

		const { data: pr } = await github.rest.pulls.get({ ...repo, pull_number: num });
		const prevRegion = extractRegion(pr.body);

Uh oh!

Conversation

ZhenghuaBao commented Jul 16, 2026 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Orca-Code-Review — push 3

Summary

Test plan

Uh oh!

github-actions Bot left a comment

Choose a reason for hiding this comment

🐳 Orca-Code-Review

Uh oh!

Uh oh!

Uh oh!

github-actions Bot Jul 16, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions Bot left a comment

Choose a reason for hiding this comment

🐳 Orca-Code-Review

Uh oh!

github-actions Bot Jul 16, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot Jul 16, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot Jul 16, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot Jul 16, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

chatgpt-codex-connector Bot Jul 16, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot Jul 16, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot left a comment

Choose a reason for hiding this comment

🐳 Orca-Code-Review

Uh oh!

github-actions Bot Jul 16, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot Jul 16, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot Jul 16, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot Jul 16, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot Jul 16, 2026

Choose a reason for hiding this comment

Uh oh!

ZhenghuaBao commented Jul 16, 2026

Merge note — P3 + summary-at-top

Uh oh!

Uh oh!

Reviewers

ZhenghuaBao commented Jul 16, 2026 •

edited by github-actions Bot

Loading