From 58d3f1368500e84c598651412c0d15e106baa28c Mon Sep 17 00:00:00 2001 From: kite Date: Sun, 22 Nov 2026 12:14:46 +0800 Subject: [PATCH 1/4] feat(skills): review GitHub PRs from pinned snapshots --- .github/workflows/plugin-contract.yml | 5 + docs/design/native-github-pr-snapshots.md | 35 +++ .../skills/open-code-review/SKILL.md | 6 +- .../open-code-review/scripts/review_pr.py | 204 +++++++++++++++++ scripts/github-actions/test_review_pr.py | 215 ++++++++++++++++++ skills/open-code-review/SKILL.md | 6 +- skills/open-code-review/scripts/review_pr.py | 204 +++++++++++++++++ 7 files changed, 671 insertions(+), 4 deletions(-) create mode 100644 docs/design/native-github-pr-snapshots.md create mode 100644 plugins/open-code-review/skills/open-code-review/scripts/review_pr.py create mode 100644 scripts/github-actions/test_review_pr.py create mode 100644 skills/open-code-review/scripts/review_pr.py diff --git a/.github/workflows/plugin-contract.yml b/.github/workflows/plugin-contract.yml index 5c103a1652..617688efe3 100644 --- a/.github/workflows/plugin-contract.yml +++ b/.github/workflows/plugin-contract.yml @@ -34,6 +34,7 @@ on: - "CONTRIBUTING*.md" - "AGENTS.md" - "scripts/github-actions/check-plugin-contract.*" + - "scripts/github-actions/test_review_pr.py" - ".github/workflows/plugin-contract.yml" pull_request: branches: [main] @@ -48,6 +49,7 @@ on: - "CONTRIBUTING*.md" - "AGENTS.md" - "scripts/github-actions/check-plugin-contract.*" + - "scripts/github-actions/test_review_pr.py" - ".github/workflows/plugin-contract.yml" permissions: @@ -76,6 +78,9 @@ jobs: - name: Test plugin-contract guardrail run: node scripts/github-actions/check-plugin-contract.test.js + - name: Test installed PR review helpers + run: python3 scripts/github-actions/test_review_pr.py + # BLOCKING: no doc or README may link to a path that no longer exists. - name: Check in-repo path links run: node scripts/github-actions/check-plugin-contract.js links diff --git a/docs/design/native-github-pr-snapshots.md b/docs/design/native-github-pr-snapshots.md new file mode 100644 index 0000000000..d903068c6f --- /dev/null +++ b/docs/design/native-github-pr-snapshots.md @@ -0,0 +1,35 @@ +# PR mode for the native review skill + +**Decision:** The maintainer accepted the phase-one pilot. This is the design of the native skill's PR mode for [#2025](https://github.com/alibaba/open-code-review/issues/2025); pilot evidence is tracked separately in [#2023](https://github.com/alibaba/open-code-review/issues/2023). + +## User interface + +A bare positive integer selects PR mode in the [native skill](../../skills/open-code-review/SKILL.md): `42` and `pr 42` both review PR 42 in the current checkout's GitHub repository. `42 --repo owner/repo` selects a different repository; a GitHub PR URL selects the same mode without needing `--repo`. If the repository cannot be resolved unambiguously or the URL conflicts with `--repo`, report the error rather than guessing. This is skill input, not a new `ocr` CLI command. Local workspace, commit, and branch reviews keep their existing behavior and dependencies. + +## The entire PR-specific skill instruction + +> For a bare PR number, `pr `, or a GitHub PR URL, run the installed `scripts/review_pr.py` once, as in `python3 "$SKILL_DIR/scripts/review_pr.py" 42`. Forward an explicit `--repo` or `--background` if supplied. Read the returned run summary and OCR result. Report the reviewed revisions and freshness alongside the normal Review and Analysis outcomes. On failure, report the failed step; do not substitute a branch review. + +`SKILL_DIR` denotes the installed skill directory, never the PR checkout. Ship the same script alongside the standalone and [plugin](../../plugins/open-code-review/skills/open-code-review/SKILL.md) instructions. The skill has no Git command sequence, cleanup checklist, or dependency on the pilot wrapper. + +## What the script guarantees + +| Step | Deterministic behavior | +| --- | --- | +| Resolve | Read the canonical PR identity, actual target branch, base SHA, and head SHA from GitHub. For a fork PR, fetch from the base repository's PR ref, not a similarly named local branch. | +| Freeze | Give each run its own bare Git repository, borrowing local objects from a matching non-shallow checkout when available. Fetch and verify both commit IDs, require a unique merge-base, and create a detached worktree at the verified head. No refs or worktrees are registered in the user's repository. If a fetched revision differs or the comparison cannot be established, stop before OCR. | +| Review once | Invoke installed `ocr review` with `--repo --from --to --audience agent --analysis --format json --output `. Forward background as data. Keep the result outside the worktree. Disable checkout hooks and avoid submodule recursion, external diff drivers, PR-provided scripts, builds, tests, and GitHub writes. | +| Release | On success, failure, or handled interruption, wait for the OCR child to stop, then remove only this run's verified, clean detached worktree without force. Drop the private object store and its refs; there are no shared refs to delete. Preserve the run summary and any OCR output. Record incomplete cleanup rather than pruning other worktrees or silently deleting modified files. | +| Recheck and return | Before returning a review, reread `(target branch, base SHA, head SHA)`. All equal means `current` **at the recorded check time**; any difference means `stale`; an unavailable or incomplete response means `unverified`. Never rerun OCR automatically. | + +The script prints the absolute path to `run.json`. It records the frozen identity and revisions, OCR exit status and result path (if produced), freshness and check time (if checked), cleanup status, and any failed step. `review.json` remains the native OCR output; do not copy its findings or reinterpret coverage in the script. For example, a run can be `current` with partial Analysis, or `stale` with complete Analysis. An interrupted run must not be reported as reviewed merely because its worktree was cleaned up. + +## Acceptance checks + +Use real local Git fixtures, stub GitHub responses and a recording OCR executable. Test the installed script as a process, not just its internal functions. + +- **Routing:** A numeric ID uses the current repository; an override or URL selects the stated repository from an unrelated checkout. Conflicting inputs stop before fetch. Non-PR reviews work without `gh` or Python. +- **Revision identity:** A fork PR targets a non-default branch that has advanced since divergence. Assert exactly one OCR call, from the merge-base rather than the target tip, to the advertised head. A mismatch during fetch or a non-unique merge-base yields zero calls. +- **Isolation and cleanup:** Start two reviews of one PR from a dirty checkout; fail or interrupt one. Confirm independent refs, worktrees, and artifacts, an unchanged source HEAD/index/files, and that one run cannot clean the other. A modified review worktree is left intact and reported. +- **Freshness:** Exercise unchanged, retargeted, advanced-base, changed-head, and unavailable responses. Assert the reported state, retained result, and absence of an automatic rerun. +- **Distribution:** Run the same contract tests from each installed skill layout without the external wrapper. Finish with a read-only smoke review against a real PR. diff --git a/plugins/open-code-review/skills/open-code-review/SKILL.md b/plugins/open-code-review/skills/open-code-review/SKILL.md index c84f64b29e..7af332c9ad 100644 --- a/plugins/open-code-review/skills/open-code-review/SKILL.md +++ b/plugins/open-code-review/skills/open-code-review/SKILL.md @@ -35,7 +35,9 @@ Analyze the review target (commits, branch, or changes) to extract concise busin ### Step 2: Run Code Review -**Run `ocr review` directly.** Do not probe with `command -v ocr` or `ocr --version` first; that adds an unnecessary tool call on the normal path. +**GitHub PR mode:** A bare positive number (e.g. `42`) means PR #42 in the current repository; the `pr` prefix is optional. For a PR number or GitHub PR URL, run `python3 "$SKILL_DIR/scripts/review_pr.py" ` once, resolving `SKILL_DIR` from this installed skill. Forward an explicit `--repo OWNER/REPO` or `--background`. Read the printed `run.json` and its result file; report the reviewed revisions, freshness, and artifact path with Step 3. On failure report the failed step, not a fallback branch review. Only PR mode needs Python 3 and authenticated `gh`. + +**Otherwise run `ocr review` directly.** Do not probe with `command -v ocr` or `ocr --version` first; that adds an unnecessary tool call on the normal path. For a workspace review, omit revision flags and `--analysis`. **Pass business context via `--background`** when available: @@ -60,7 +62,7 @@ ocr review --audience agent --background "business context here" | User says | Command to run | |-----------|---------------| | "review my changes" / "review the working copy" | `ocr review --audience agent -b "context"` | -| "review this PR" / "review feature branch" | `ocr review --audience agent -b "context" --from --to --analysis --format json` | +| "review feature branch" | `ocr review --audience agent -b "context" --from --to --analysis --format json` | | "review the current branch against main" | `ocr review --audience agent -b "context" --from main --analysis --format json` | | "review commit abc123" | `ocr review --audience agent -b "context" --commit abc123 --analysis --format json` | | "what would be reviewed?" (dry-run) | `ocr review --preview` | diff --git a/plugins/open-code-review/skills/open-code-review/scripts/review_pr.py b/plugins/open-code-review/skills/open-code-review/scripts/review_pr.py new file mode 100644 index 0000000000..d14bfdac2f --- /dev/null +++ b/plugins/open-code-review/skills/open-code-review/scripts/review_pr.py @@ -0,0 +1,204 @@ +# SPDX-License-Identifier: Apache-2.0 +# Copyright 2026 alibaba/open-code-review Contributors + +import argparse +import json +import os +import re +import shutil +import signal +import subprocess +import sys +import tempfile +from datetime import datetime, timezone +from pathlib import Path + + +REPOSITORY = r"[A-Za-z0-9-]+/[A-Za-z0-9_.-]+" +PR_URL = re.compile(rf"https://github\.com/({REPOSITORY})/pull/([1-9][0-9]*)/?") +FIELDS = "url,baseRefName,baseRefOid,headRefOid" + + +def target(value, repo): + if repo and not re.fullmatch(REPOSITORY, repo): + raise ValueError("--repo expects OWNER/REPO") + match = PR_URL.fullmatch(value) + if match: + if repo and match[1].lower() != repo.lower(): + raise ValueError("PR URL conflicts with --repo") + return match[2], match[1] + if not re.fullmatch(r"[1-9][0-9]*", value): + raise ValueError("expected a positive PR number or a github.com PR URL") + return value, repo + + +def command(*args, cwd=None, env=None): + result = subprocess.run(args, cwd=cwd, env=env, text=True, stdout=subprocess.PIPE) + if result.returncode: + raise ValueError(f"{args[0]} {args[1]} exited {result.returncode} (see stderr)") + return result.stdout.strip() + + +def lookup(number, repo, cwd, env): + scope = ["--repo", f"github.com/{repo}"] if repo else [] + data = json.loads(command("gh", "pr", "view", number, *scope, "--json", FIELDS, cwd=cwd, env=env)) + if not isinstance(data, dict): + raise ValueError("GitHub did not return a PR object") + match = PR_URL.fullmatch(str(data.get("url", ""))) + if not match or match[2] != number or (repo and match[1].lower() != repo.lower()): + raise ValueError("GitHub returned a different PR identity") + if not isinstance(data.get("baseRefName"), str) or not data["baseRefName"] or not all( + re.fullmatch(r"[0-9a-f]{40}", str(data.get(key, ""))) for key in ("baseRefOid", "headRefOid")): + raise ValueError("GitHub did not return a complete target and commit pair") + return match[1], {"target": data["baseRefName"], "base": data["baseRefOid"], "head": data["headRefOid"]} + + +def local_remote(repo, env): + result = subprocess.run(["git", "config", "--local", "--get-regexp", r"^remote\..*\.url$"], + env=env, capture_output=True, text=True) + pattern = rf"(?:git@github\.com:|(?:https://|ssh://git@)github\.com/){re.escape(repo)}(?:\.git)?/?" + for line in result.stdout.splitlines(): + _, _, url = line.partition(" ") + if re.fullmatch(pattern, url, flags=re.IGNORECASE): + return url + return None + + +def freeze(folder, repo, number, revisions, env, offline): + store, tree = folder / "objects.git", folder / "worktree" + remote = local_remote(repo, env) + if remote and command("git", "rev-parse", "--is-shallow-repository", env=env) == "false": + source = command("git", "rev-parse", "--path-format=absolute", "--git-common-dir", env=env) + command("git", "clone", "--bare", "--shared", "--single-branch", "--no-tags", "--template=", + "--", source, store, cwd=folder, env=dict(offline, GIT_ALLOW_PROTOCOL="file")) + else: + command("git", "init", "--bare", "--template=", store, cwd=folder, env=offline) + if not remote: + protocol = command("gh", "config", "get", "git_protocol", "--host", "github.com", env=env) + remote = f"git@github.com:{repo}.git" if protocol == "ssh" else f"https://github.com/{repo}.git" + command("git", f"--git-dir={store}", "-c", f"core.hooksPath={os.devnull}", "-c", "maintenance.auto=false", + "-c", "credential.https://github.com.helper=", "-c", "credential.https://github.com.helper=!gh auth git-credential", + "fetch", "--atomic", "--no-tags", "--no-write-fetch-head", "--no-recurse-submodules", remote, + f"{revisions['base']}:refs/pr/base", f"refs/pull/{number}/head:refs/pr/head", cwd=store, env=env) + for name in ("base", "head"): + if command("git", "rev-parse", f"refs/pr/{name}^{{commit}}", cwd=store, env=offline) != revisions[name]: + raise ValueError(f"fetched {name} differs from GitHub; no review started") + bases = command("git", "merge-base", "--all", revisions["base"], revisions["head"], cwd=store, env=offline).splitlines() + if len(bases) != 1: + raise ValueError("no unique merge-base") + command("git", "worktree", "add", "--detach", tree, revisions["head"], cwd=store, env=offline) + return bases[0] + + +def release(folder, head, env): + store, tree = folder / "objects.git", folder / "worktree" + if tree.exists(): + if (command("git", "rev-parse", "HEAD", cwd=tree, env=env) != head or + command("git", "rev-parse", "--abbrev-ref", "HEAD", cwd=tree, env=env) != "HEAD" or + command("git", "status", "--porcelain", "--ignored", "--untracked-files=all", cwd=tree, env=env)): + raise ValueError(f"worktree changed; retained {tree}") + command("git", "worktree", "remove", "--", tree, cwd=store, env=env) + if store.exists(): + shutil.rmtree(store) + + +def review(folder, base, head, background, env): + args = ["ocr", "review", "--repo", str(folder / "worktree"), "--from", base, "--to", head, + "--audience", "agent", "--analysis", "--format", "json", "--output", str(folder / "review.json")] + if background: + args.extend(["--background", background]) + with subprocess.Popen(args, cwd=folder, env=env, stdout=sys.stderr, + start_new_session=os.name == "posix") as child: + try: + return child.wait() + except KeyboardInterrupt: + if os.name == "posix": + try: + os.killpg(child.pid, signal.SIGKILL) + except ProcessLookupError: + pass + else: + child.terminate() + child.wait() + raise + + +def save(manifest, summary): + pending = manifest.with_suffix(".pending") + pending.write_text(json.dumps(summary, indent=2) + "\n", encoding="utf-8") + pending.replace(manifest) + + +def run(number, repo, background): + cache = Path.home() / ".cache" / "open-code-review" / "pr" + cache.mkdir(parents=True, exist_ok=True) + folder = Path(tempfile.mkdtemp(prefix=f"pr-{number}-", dir=cache)).resolve() + manifest = folder / "run.json" + result = folder / "review.json" + summary = {"number": int(number), "state": "running", "freshness": "not_checked", "ocr_exit": None, "result": None} + save(manifest, summary) + print(f"PR review artifacts: {folder}", file=sys.stderr, flush=True) + env = {key: value for key, value in os.environ.items() if not key.startswith("GIT_") and key != "GH_REPO"} + env["PATH"] = os.pathsep.join(part for part in os.get_exec_path() if os.path.isabs(part)) + env.update(GH_PROMPT_DISABLED="1", GIT_TERMINAL_PROMPT="0") + offline = dict(env, GIT_CONFIG_GLOBAL=os.devnull, GIT_CONFIG_NOSYSTEM="1", GIT_ALLOW_PROTOCOL="", OCR_NO_UPDATE="1") + step = "lookup" + code = 1 + try: + repo, snapshot = lookup(number, repo, None, env) + summary.update(url=f"https://github.com/{repo}/pull/{number}", snapshot=snapshot) + step = "freeze" + base = freeze(folder, repo, number, snapshot, env, offline) + summary["merge_base"] = base + save(manifest, summary) + step = "review" + summary["ocr_exit"] = review(folder, base, snapshot["head"], background, offline) + if summary["ocr_exit"] != 0 or not result.is_file() or not result.stat().st_size: + raise ValueError(f"OCR exited {summary['ocr_exit']}; no successful review") + code = 0 + except (OSError, ValueError, KeyboardInterrupt) as error: + summary.update(failed_step=step, error=str(error) or "interrupted") + if isinstance(error, KeyboardInterrupt): + code = 130 + finally: + if result.is_file(): + summary["result"] = str(result) + try: + release(folder, summary.get("snapshot", {}).get("head"), offline) + summary["cleanup"] = "complete" + except (OSError, ValueError, KeyboardInterrupt) as error: + summary.update(cleanup="retained", cleanup_error=str(error) or "interrupted") + summary.setdefault("failed_step", "cleanup") + code = 130 if isinstance(error, KeyboardInterrupt) else code or 1 + if summary["result"]: + try: + _, latest = lookup(number, repo, folder, env) + summary.update(latest=latest, freshness="current" if latest == summary["snapshot"] else "stale") + except (OSError, ValueError, KeyboardInterrupt) as error: + summary.update(freshness="unverified", freshness_error=str(error) or "interrupted") + if isinstance(error, KeyboardInterrupt): + summary.setdefault("failed_step", "freshness") + code = 130 + summary["checked_at"] = datetime.now(timezone.utc).isoformat() + summary["state"] = "finished" if code == 0 else "failed" + save(manifest, summary) + print(manifest) + return code + + +def interrupt(signum, frame): + raise KeyboardInterrupt + + +if __name__ == "__main__": + parser = argparse.ArgumentParser(description="Review one pinned GitHub pull request using native OCR.") + parser.add_argument("pr", help="Current repository PR number or full GitHub PR URL") + parser.add_argument("--repo", help="GitHub repository (OWNER/REPO)") + parser.add_argument("--background", help="Business context for the review") + args = parser.parse_args() + try: + number, repo = target(args.pr, args.repo) + except ValueError as error: + parser.error(str(error)) + signal.signal(signal.SIGTERM, interrupt) + sys.exit(run(number, repo, args.background)) diff --git a/scripts/github-actions/test_review_pr.py b/scripts/github-actions/test_review_pr.py new file mode 100644 index 0000000000..eadf9a8af0 --- /dev/null +++ b/scripts/github-actions/test_review_pr.py @@ -0,0 +1,215 @@ +# SPDX-License-Identifier: Apache-2.0 +# Copyright 2026 alibaba/open-code-review Contributors + +import json +import os +import shutil +import signal +import subprocess +import sys +import tempfile +import time +import unittest +from pathlib import Path + + +ROOT = Path(__file__).resolve().parents[2] +SCRIPT = Path("skills/open-code-review/scripts/review_pr.py") +URL = "https://github.com/team/project/pull/42" + +GH = """ +import json, os, sys +from pathlib import Path +if sys.argv[1:3] == ['config', 'get']: + print('https') + sys.exit(0) +assert sys.argv[1:4] == ['pr', 'view', '42'], sys.argv +if '--repo' in sys.argv: + assert sys.argv[sys.argv.index('--repo') + 1] == 'github.com/team/project' +else: + assert Path.cwd() == Path(os.environ['SOURCE']) +data = json.loads(os.environ['METADATA']) +if Path('review.json').exists(): + assert '--repo' in sys.argv + latest = os.environ.get('LATEST', '{}') + if latest == 'unavailable': + sys.exit(1) + data.update(json.loads(latest)) +print(json.dumps(data)) +""" + +OCR = """ +import json, os, sys, time +from pathlib import Path +args = sys.argv[1:] +with open('calls.jsonl', 'a') as log: + log.write(json.dumps(args) + '\\n') +mode = os.environ.get('MODE') +if mode == 'wait': + while not Path('resume').exists(): + time.sleep(0.02) +if mode == 'dirty': + (Path(args[args.index('--repo') + 1]) / 'keep').write_text('retain me') +if mode == 'no-result': + sys.exit(0) +Path(args[args.index('--output') + 1]).write_text(json.dumps({'comments': [], 'analysis_meta': {'declared_completion': 'partial'}})) +sys.exit(5 if mode == 'failed' else 0) +""" + + +class Standalone(unittest.TestCase): + script = ROOT / SCRIPT + + def setUp(self): + build = ROOT / "dist" / "pr-helper-tests" + build.mkdir(parents=True, exist_ok=True) + workspace = tempfile.TemporaryDirectory(dir=build) + self.addCleanup(workspace.cleanup) + self.root = Path(workspace.name) + installed = self.root / "installed" / self.script.relative_to(ROOT) + shutil.copytree(self.script.parent.parent, installed.parent.parent) + self.installed = installed + self.source = self.root / "source" + self.source.mkdir() + tools = self.root / "bin" + tools.mkdir() + for name, program in (("gh", GH), ("ocr", OCR)): + executable = tools / name + executable.write_text(f"#!{sys.executable}\n" + program) + executable.chmod(0o755) + self.env = dict(os.environ, HOME=str(self.root), PATH=str(tools) + os.pathsep + os.environ['PATH'], + SOURCE=str(self.source), XDG_CONFIG_HOME=str(self.root / "config"), GIT_CONFIG_NOSYSTEM="1") + self.git("init", "-q", "-b", "release") + self.git("config", "user.name", "Test") + self.git("config", "user.email", "test@example.com") + self.git("config", "--global", f"url.{self.source.as_uri()}.insteadOf", "https://github.com/team/project.git") + self.git("remote", "add", "origin", "https://github.com/team/project.git") + self.common = self.commit("initial") + self.git("switch", "-qc", "topic") + self.head = self.commit("topic") + self.git("update-ref", "refs/pull/42/head", self.head) + self.git("switch", "-q", "release") + self.base = self.commit("release") + self.metadata = dict(url=URL, baseRefName="release", baseRefOid=self.base, headRefOid=self.head) + self.env["METADATA"] = json.dumps(self.metadata) + (self.source / "staged").write_text("staged") + self.git("add", "staged") + (self.source / "initial").write_text("unstaged") + (self.source / "untracked").write_text("untracked") + + def git(self, *args): + return subprocess.check_output(["git", *args], cwd=self.source, env=self.env, text=True).strip() + + def commit(self, name): + (self.source / name).write_text(name) + self.git("add", name) + self.git("commit", "-qm", name) + return self.git("rev-parse", "HEAD") + + def start(self, *args, cwd=None, **env): + child = subprocess.Popen([sys.executable, str(self.installed), *args], cwd=cwd or self.source, + env=dict(self.env, **env), stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True) + self.addCleanup(self.stop, child) + return child + + @staticmethod + def stop(child): + if child.poll() is None: + child.terminate() + child.communicate(timeout=10) + + def finish(self, child, code=0): + out, err = child.communicate(timeout=30) + self.assertEqual(child.returncode, code, err + out) + path = Path(out.strip()) + self.assertTrue(path.is_absolute()) + return path.parent, json.loads(path.read_text()) + + def test_pinned_range_and_checkout(self): + index = (self.source / ".git/index").read_bytes() + refs = self.git("show-ref") + folder, run = self.finish(self.start("42", "--background", "preserve context")) + self.assertEqual(run["snapshot"], {"target": "release", "base": self.base, "head": self.head}) + self.assertEqual(run["merge_base"], self.common) + self.assertEqual(run["freshness"], "current") + self.assertEqual(run["cleanup"], "complete") + self.assertEqual(json.loads((folder / "calls.jsonl").read_text()), [ + "review", "--repo", str(folder / "worktree"), "--from", self.common, "--to", self.head, + "--audience", "agent", "--analysis", "--format", "json", "--output", run["result"], + "--background", "preserve context"]) + self.assertEqual(json.loads(Path(run["result"]).read_text())["analysis_meta"]["declared_completion"], "partial") + self.assertFalse((folder / "worktree").exists()) + self.assertFalse((folder / "objects.git").exists()) + self.assertEqual((self.source / ".git/index").read_bytes(), index) + self.assertEqual(self.git("show-ref"), refs) + self.assertEqual(self.git("symbolic-ref", "--short", "HEAD"), "release") + self.assertEqual(self.git("status", "--porcelain").splitlines(), ['M initial', 'A staged', '?? untracked']) + + def test_explicit_identity_and_freshness(self): + for delta, expected in (({"baseRefName": "main"}, "stale"), ({"baseRefOid": self.common}, "stale"), + ({"headRefOid": self.base}, "stale"), (None, "unverified")): + with self.subTest(delta=delta): + args = [URL] if delta else ["42", "--repo", "team/project"] + folder, run = self.finish(self.start(*args, cwd=self.root, LATEST=json.dumps(delta) if delta else 'unavailable')) + self.assertEqual(run["freshness"], expected) + self.assertEqual(len((folder / "calls.jsonl").read_text().splitlines()), 1) + self.assertTrue(Path(run["result"]).is_file()) + + def test_conflicting_input_and_moving_head(self): + invalid = self.start(URL, "--repo", "other/project") + out, err = invalid.communicate(timeout=10) + self.assertEqual(invalid.returncode, 2) + self.assertIn('conflicts', err) + self.assertFalse(out) + self.git("update-ref", "refs/pull/42/head", self.base) + folder, run = self.finish(self.start("42"), code=1) + self.assertEqual(run["failed_step"], "freeze") + self.assertEqual(run["cleanup"], "complete") + self.assertFalse((folder / "calls.jsonl").exists()) + + def test_no_unique_merge_base(self): + tree = self.git("rev-parse", "HEAD^{tree}") + first = self.git("commit-tree", tree, "-p", self.head, "-p", self.base, "-m", "first") + second = self.git("commit-tree", tree, "-p", self.base, "-p", self.head, "-m", "second") + self.git("update-ref", "refs/pull/42/head", second) + _, run = self.finish(self.start("42", METADATA=json.dumps(dict(self.metadata, baseRefOid=first, headRefOid=second))), code=1) + self.assertEqual(run["failed_step"], "freeze") + self.assertIn("merge-base", run["error"]) + self.assertIsNone(run["ocr_exit"]) + + def test_cleanup_on_failure(self): + for mode, status, step in (("dirty", "retained", "cleanup"), ("failed", "complete", "review"), + ("no-result", "complete", "review")): + with self.subTest(mode=mode): + folder, run = self.finish(self.start('42', MODE=mode), code=1) + self.assertEqual((run['cleanup'], run['failed_step']), (status, step)) + self.assertEqual((folder / 'worktree/keep').is_file(), mode == 'dirty') + self.assertEqual(run['ocr_exit'], 5 if mode == 'failed' else 0) + + def test_concurrent_interrupt(self): + first, second = self.start('42', MODE='wait'), self.start('42', MODE='wait') + deadline = time.monotonic() + 20 + while time.monotonic() < deadline: + folders = [call.parent for call in self.root.glob('.cache/open-code-review/pr/*/calls.jsonl')] + if len(folders) == 2: + break + time.sleep(0.05) + self.assertEqual(len(folders), 2) + first.send_signal(signal.SIGTERM) + stopped, run = self.finish(first, code=130) + self.assertEqual((run['failed_step'], run['cleanup']), ('review', 'complete')) + remaining, = set(folders) - {stopped} + self.assertTrue((remaining / 'worktree/topic').is_file()) + (remaining / 'resume').touch() + self.finish(second) + + +class Plugin(Standalone): + script = ROOT / 'plugins/open-code-review' / SCRIPT + + def test_helper_matches_standalone(self): + self.assertEqual(self.script.read_bytes(), Standalone.script.read_bytes()) + + +if __name__ == '__main__': + unittest.main() diff --git a/skills/open-code-review/SKILL.md b/skills/open-code-review/SKILL.md index 0a767869e7..1c5b862c65 100644 --- a/skills/open-code-review/SKILL.md +++ b/skills/open-code-review/SKILL.md @@ -30,7 +30,9 @@ Analyze the review target (commits, branch, or changes) to extract concise busin ### Step 2: Run Code Review -**Run `ocr review` directly.** Do not probe with `command -v ocr` or `ocr --version` first; that adds an unnecessary tool call on the normal path. +**GitHub PR mode:** A bare positive number (e.g. `42`) means PR #42 in the current repository; the `pr` prefix is optional. For a PR number or GitHub PR URL, run `python3 "$SKILL_DIR/scripts/review_pr.py" ` once, resolving `SKILL_DIR` from this installed skill. Forward an explicit `--repo OWNER/REPO` or `--background`. Read the printed `run.json` and its result file; report the reviewed revisions, freshness, and artifact path with Step 3. On failure report the failed step, not a fallback branch review. Only PR mode needs Python 3 and authenticated `gh`. + +**Otherwise run `ocr review` directly.** Do not probe with `command -v ocr` or `ocr --version` first; that adds an unnecessary tool call on the normal path. For a workspace review, omit revision flags and `--analysis`. **Pass business context via `--background`** when available: @@ -55,7 +57,7 @@ ocr review --audience agent --background "business context here" | User says | Command to run | |-----------|---------------| | "review my changes" / "review the working copy" | `ocr review --audience agent -b "context"` | -| "review this PR" / "review feature branch" | `ocr review --audience agent -b "context" --from --to --analysis --format json` | +| "review feature branch" | `ocr review --audience agent -b "context" --from --to --analysis --format json` | | "review the current branch against main" | `ocr review --audience agent -b "context" --from main --analysis --format json` | | "review commit abc123" | `ocr review --audience agent -b "context" --commit abc123 --analysis --format json` | | "what would be reviewed?" (dry-run) | `ocr review --preview` | diff --git a/skills/open-code-review/scripts/review_pr.py b/skills/open-code-review/scripts/review_pr.py new file mode 100644 index 0000000000..d14bfdac2f --- /dev/null +++ b/skills/open-code-review/scripts/review_pr.py @@ -0,0 +1,204 @@ +# SPDX-License-Identifier: Apache-2.0 +# Copyright 2026 alibaba/open-code-review Contributors + +import argparse +import json +import os +import re +import shutil +import signal +import subprocess +import sys +import tempfile +from datetime import datetime, timezone +from pathlib import Path + + +REPOSITORY = r"[A-Za-z0-9-]+/[A-Za-z0-9_.-]+" +PR_URL = re.compile(rf"https://github\.com/({REPOSITORY})/pull/([1-9][0-9]*)/?") +FIELDS = "url,baseRefName,baseRefOid,headRefOid" + + +def target(value, repo): + if repo and not re.fullmatch(REPOSITORY, repo): + raise ValueError("--repo expects OWNER/REPO") + match = PR_URL.fullmatch(value) + if match: + if repo and match[1].lower() != repo.lower(): + raise ValueError("PR URL conflicts with --repo") + return match[2], match[1] + if not re.fullmatch(r"[1-9][0-9]*", value): + raise ValueError("expected a positive PR number or a github.com PR URL") + return value, repo + + +def command(*args, cwd=None, env=None): + result = subprocess.run(args, cwd=cwd, env=env, text=True, stdout=subprocess.PIPE) + if result.returncode: + raise ValueError(f"{args[0]} {args[1]} exited {result.returncode} (see stderr)") + return result.stdout.strip() + + +def lookup(number, repo, cwd, env): + scope = ["--repo", f"github.com/{repo}"] if repo else [] + data = json.loads(command("gh", "pr", "view", number, *scope, "--json", FIELDS, cwd=cwd, env=env)) + if not isinstance(data, dict): + raise ValueError("GitHub did not return a PR object") + match = PR_URL.fullmatch(str(data.get("url", ""))) + if not match or match[2] != number or (repo and match[1].lower() != repo.lower()): + raise ValueError("GitHub returned a different PR identity") + if not isinstance(data.get("baseRefName"), str) or not data["baseRefName"] or not all( + re.fullmatch(r"[0-9a-f]{40}", str(data.get(key, ""))) for key in ("baseRefOid", "headRefOid")): + raise ValueError("GitHub did not return a complete target and commit pair") + return match[1], {"target": data["baseRefName"], "base": data["baseRefOid"], "head": data["headRefOid"]} + + +def local_remote(repo, env): + result = subprocess.run(["git", "config", "--local", "--get-regexp", r"^remote\..*\.url$"], + env=env, capture_output=True, text=True) + pattern = rf"(?:git@github\.com:|(?:https://|ssh://git@)github\.com/){re.escape(repo)}(?:\.git)?/?" + for line in result.stdout.splitlines(): + _, _, url = line.partition(" ") + if re.fullmatch(pattern, url, flags=re.IGNORECASE): + return url + return None + + +def freeze(folder, repo, number, revisions, env, offline): + store, tree = folder / "objects.git", folder / "worktree" + remote = local_remote(repo, env) + if remote and command("git", "rev-parse", "--is-shallow-repository", env=env) == "false": + source = command("git", "rev-parse", "--path-format=absolute", "--git-common-dir", env=env) + command("git", "clone", "--bare", "--shared", "--single-branch", "--no-tags", "--template=", + "--", source, store, cwd=folder, env=dict(offline, GIT_ALLOW_PROTOCOL="file")) + else: + command("git", "init", "--bare", "--template=", store, cwd=folder, env=offline) + if not remote: + protocol = command("gh", "config", "get", "git_protocol", "--host", "github.com", env=env) + remote = f"git@github.com:{repo}.git" if protocol == "ssh" else f"https://github.com/{repo}.git" + command("git", f"--git-dir={store}", "-c", f"core.hooksPath={os.devnull}", "-c", "maintenance.auto=false", + "-c", "credential.https://github.com.helper=", "-c", "credential.https://github.com.helper=!gh auth git-credential", + "fetch", "--atomic", "--no-tags", "--no-write-fetch-head", "--no-recurse-submodules", remote, + f"{revisions['base']}:refs/pr/base", f"refs/pull/{number}/head:refs/pr/head", cwd=store, env=env) + for name in ("base", "head"): + if command("git", "rev-parse", f"refs/pr/{name}^{{commit}}", cwd=store, env=offline) != revisions[name]: + raise ValueError(f"fetched {name} differs from GitHub; no review started") + bases = command("git", "merge-base", "--all", revisions["base"], revisions["head"], cwd=store, env=offline).splitlines() + if len(bases) != 1: + raise ValueError("no unique merge-base") + command("git", "worktree", "add", "--detach", tree, revisions["head"], cwd=store, env=offline) + return bases[0] + + +def release(folder, head, env): + store, tree = folder / "objects.git", folder / "worktree" + if tree.exists(): + if (command("git", "rev-parse", "HEAD", cwd=tree, env=env) != head or + command("git", "rev-parse", "--abbrev-ref", "HEAD", cwd=tree, env=env) != "HEAD" or + command("git", "status", "--porcelain", "--ignored", "--untracked-files=all", cwd=tree, env=env)): + raise ValueError(f"worktree changed; retained {tree}") + command("git", "worktree", "remove", "--", tree, cwd=store, env=env) + if store.exists(): + shutil.rmtree(store) + + +def review(folder, base, head, background, env): + args = ["ocr", "review", "--repo", str(folder / "worktree"), "--from", base, "--to", head, + "--audience", "agent", "--analysis", "--format", "json", "--output", str(folder / "review.json")] + if background: + args.extend(["--background", background]) + with subprocess.Popen(args, cwd=folder, env=env, stdout=sys.stderr, + start_new_session=os.name == "posix") as child: + try: + return child.wait() + except KeyboardInterrupt: + if os.name == "posix": + try: + os.killpg(child.pid, signal.SIGKILL) + except ProcessLookupError: + pass + else: + child.terminate() + child.wait() + raise + + +def save(manifest, summary): + pending = manifest.with_suffix(".pending") + pending.write_text(json.dumps(summary, indent=2) + "\n", encoding="utf-8") + pending.replace(manifest) + + +def run(number, repo, background): + cache = Path.home() / ".cache" / "open-code-review" / "pr" + cache.mkdir(parents=True, exist_ok=True) + folder = Path(tempfile.mkdtemp(prefix=f"pr-{number}-", dir=cache)).resolve() + manifest = folder / "run.json" + result = folder / "review.json" + summary = {"number": int(number), "state": "running", "freshness": "not_checked", "ocr_exit": None, "result": None} + save(manifest, summary) + print(f"PR review artifacts: {folder}", file=sys.stderr, flush=True) + env = {key: value for key, value in os.environ.items() if not key.startswith("GIT_") and key != "GH_REPO"} + env["PATH"] = os.pathsep.join(part for part in os.get_exec_path() if os.path.isabs(part)) + env.update(GH_PROMPT_DISABLED="1", GIT_TERMINAL_PROMPT="0") + offline = dict(env, GIT_CONFIG_GLOBAL=os.devnull, GIT_CONFIG_NOSYSTEM="1", GIT_ALLOW_PROTOCOL="", OCR_NO_UPDATE="1") + step = "lookup" + code = 1 + try: + repo, snapshot = lookup(number, repo, None, env) + summary.update(url=f"https://github.com/{repo}/pull/{number}", snapshot=snapshot) + step = "freeze" + base = freeze(folder, repo, number, snapshot, env, offline) + summary["merge_base"] = base + save(manifest, summary) + step = "review" + summary["ocr_exit"] = review(folder, base, snapshot["head"], background, offline) + if summary["ocr_exit"] != 0 or not result.is_file() or not result.stat().st_size: + raise ValueError(f"OCR exited {summary['ocr_exit']}; no successful review") + code = 0 + except (OSError, ValueError, KeyboardInterrupt) as error: + summary.update(failed_step=step, error=str(error) or "interrupted") + if isinstance(error, KeyboardInterrupt): + code = 130 + finally: + if result.is_file(): + summary["result"] = str(result) + try: + release(folder, summary.get("snapshot", {}).get("head"), offline) + summary["cleanup"] = "complete" + except (OSError, ValueError, KeyboardInterrupt) as error: + summary.update(cleanup="retained", cleanup_error=str(error) or "interrupted") + summary.setdefault("failed_step", "cleanup") + code = 130 if isinstance(error, KeyboardInterrupt) else code or 1 + if summary["result"]: + try: + _, latest = lookup(number, repo, folder, env) + summary.update(latest=latest, freshness="current" if latest == summary["snapshot"] else "stale") + except (OSError, ValueError, KeyboardInterrupt) as error: + summary.update(freshness="unverified", freshness_error=str(error) or "interrupted") + if isinstance(error, KeyboardInterrupt): + summary.setdefault("failed_step", "freshness") + code = 130 + summary["checked_at"] = datetime.now(timezone.utc).isoformat() + summary["state"] = "finished" if code == 0 else "failed" + save(manifest, summary) + print(manifest) + return code + + +def interrupt(signum, frame): + raise KeyboardInterrupt + + +if __name__ == "__main__": + parser = argparse.ArgumentParser(description="Review one pinned GitHub pull request using native OCR.") + parser.add_argument("pr", help="Current repository PR number or full GitHub PR URL") + parser.add_argument("--repo", help="GitHub repository (OWNER/REPO)") + parser.add_argument("--background", help="Business context for the review") + args = parser.parse_args() + try: + number, repo = target(args.pr, args.repo) + except ValueError as error: + parser.error(str(error)) + signal.signal(signal.SIGTERM, interrupt) + sys.exit(run(number, repo, args.background)) From 38234d8862c8c48f05f23733882e8751c03b0a49 Mon Sep 17 00:00:00 2001 From: kite Date: Sun, 22 Nov 2026 12:19:15 +0800 Subject: [PATCH 2/4] docs(skills): clarify GitHub repository selection for PR numbers --- docs/design/native-github-pr-snapshots.md | 4 ++-- plugins/open-code-review/skills/open-code-review/SKILL.md | 2 +- .../skills/open-code-review/scripts/review_pr.py | 2 +- scripts/github-actions/test_review_pr.py | 7 +++++++ skills/open-code-review/SKILL.md | 2 +- skills/open-code-review/scripts/review_pr.py | 2 +- 6 files changed, 13 insertions(+), 6 deletions(-) diff --git a/docs/design/native-github-pr-snapshots.md b/docs/design/native-github-pr-snapshots.md index d903068c6f..5d7651cc9b 100644 --- a/docs/design/native-github-pr-snapshots.md +++ b/docs/design/native-github-pr-snapshots.md @@ -4,7 +4,7 @@ ## User interface -A bare positive integer selects PR mode in the [native skill](../../skills/open-code-review/SKILL.md): `42` and `pr 42` both review PR 42 in the current checkout's GitHub repository. `42 --repo owner/repo` selects a different repository; a GitHub PR URL selects the same mode without needing `--repo`. If the repository cannot be resolved unambiguously or the URL conflicts with `--repo`, report the error rather than guessing. This is skill input, not a new `ocr` CLI command. Local workspace, commit, and branch reviews keep their existing behavior and dependencies. +A bare positive integer selects PR mode in the [native skill](../../skills/open-code-review/SKILL.md): `42` and `pr 42` both review PR 42 in the GitHub CLI's default repository for this checkout. In a fork this can be the upstream repository rather than `origin`. Report the resolved PR URL so the target is visible. `42 --repo owner/repo` selects a different repository; a GitHub PR URL selects the same mode without needing `--repo`. If the repository cannot be resolved or the URL conflicts with `--repo`, report the error rather than guessing. This is skill input, not a new `ocr` CLI command. Local workspace, commit, and branch reviews keep their existing behavior and dependencies. ## The entire PR-specific skill instruction @@ -28,7 +28,7 @@ The script prints the absolute path to `run.json`. It records the frozen identit Use real local Git fixtures, stub GitHub responses and a recording OCR executable. Test the installed script as a process, not just its internal functions. -- **Routing:** A numeric ID uses the current repository; an override or URL selects the stated repository from an unrelated checkout. Conflicting inputs stop before fetch. Non-PR reviews work without `gh` or Python. +- **Routing:** A numeric ID follows the GitHub CLI's repository selection, including an upstream default in a fork checkout. An override or URL selects the stated repository from an unrelated checkout. Conflicting inputs stop before fetch. Non-PR reviews work without `gh` or Python. - **Revision identity:** A fork PR targets a non-default branch that has advanced since divergence. Assert exactly one OCR call, from the merge-base rather than the target tip, to the advertised head. A mismatch during fetch or a non-unique merge-base yields zero calls. - **Isolation and cleanup:** Start two reviews of one PR from a dirty checkout; fail or interrupt one. Confirm independent refs, worktrees, and artifacts, an unchanged source HEAD/index/files, and that one run cannot clean the other. A modified review worktree is left intact and reported. - **Freshness:** Exercise unchanged, retargeted, advanced-base, changed-head, and unavailable responses. Assert the reported state, retained result, and absence of an automatic rerun. diff --git a/plugins/open-code-review/skills/open-code-review/SKILL.md b/plugins/open-code-review/skills/open-code-review/SKILL.md index 7af332c9ad..a46d839868 100644 --- a/plugins/open-code-review/skills/open-code-review/SKILL.md +++ b/plugins/open-code-review/skills/open-code-review/SKILL.md @@ -35,7 +35,7 @@ Analyze the review target (commits, branch, or changes) to extract concise busin ### Step 2: Run Code Review -**GitHub PR mode:** A bare positive number (e.g. `42`) means PR #42 in the current repository; the `pr` prefix is optional. For a PR number or GitHub PR URL, run `python3 "$SKILL_DIR/scripts/review_pr.py" ` once, resolving `SKILL_DIR` from this installed skill. Forward an explicit `--repo OWNER/REPO` or `--background`. Read the printed `run.json` and its result file; report the reviewed revisions, freshness, and artifact path with Step 3. On failure report the failed step, not a fallback branch review. Only PR mode needs Python 3 and authenticated `gh`. +**GitHub PR mode:** A bare positive number (e.g. `42`) selects a PR in the GitHub CLI's default repository for this checkout, which can be upstream of a fork. The `pr` prefix is optional. For a PR number or GitHub PR URL, run `python3 "$SKILL_DIR/scripts/review_pr.py" ` once, resolving `SKILL_DIR` from this installed skill. Forward an explicit `--repo OWNER/REPO` or `--background`. Read the printed `run.json` and its result file; report the resolved PR URL, reviewed revisions, freshness, and artifact path with Step 3. On failure report the failed step, not a fallback branch review. Only PR mode needs Python 3 and authenticated `gh`. **Otherwise run `ocr review` directly.** Do not probe with `command -v ocr` or `ocr --version` first; that adds an unnecessary tool call on the normal path. diff --git a/plugins/open-code-review/skills/open-code-review/scripts/review_pr.py b/plugins/open-code-review/skills/open-code-review/scripts/review_pr.py index d14bfdac2f..db672b0ecf 100644 --- a/plugins/open-code-review/skills/open-code-review/scripts/review_pr.py +++ b/plugins/open-code-review/skills/open-code-review/scripts/review_pr.py @@ -192,7 +192,7 @@ def interrupt(signum, frame): if __name__ == "__main__": parser = argparse.ArgumentParser(description="Review one pinned GitHub pull request using native OCR.") - parser.add_argument("pr", help="Current repository PR number or full GitHub PR URL") + parser.add_argument("pr", help="PR number in the checkout's default gh repository, or a full GitHub PR URL") parser.add_argument("--repo", help="GitHub repository (OWNER/REPO)") parser.add_argument("--background", help="Business context for the review") args = parser.parse_args() diff --git a/scripts/github-actions/test_review_pr.py b/scripts/github-actions/test_review_pr.py index eadf9a8af0..254c44afe5 100644 --- a/scripts/github-actions/test_review_pr.py +++ b/scripts/github-actions/test_review_pr.py @@ -145,6 +145,13 @@ def test_pinned_range_and_checkout(self): self.assertEqual(self.git("symbolic-ref", "--short", "HEAD"), "release") self.assertEqual(self.git("status", "--porcelain").splitlines(), ['M initial', 'A staged', '?? untracked']) + def test_bare_number_uses_upstream_default_in_fork(self): + self.git("remote", "rename", "origin", "upstream") + self.git("remote", "add", "origin", "https://github.com/contributor/project.git") + _, run = self.finish(self.start("42")) + self.assertEqual(run["url"], URL) + self.assertEqual(run["freshness"], "current") + def test_explicit_identity_and_freshness(self): for delta, expected in (({"baseRefName": "main"}, "stale"), ({"baseRefOid": self.common}, "stale"), ({"headRefOid": self.base}, "stale"), (None, "unverified")): diff --git a/skills/open-code-review/SKILL.md b/skills/open-code-review/SKILL.md index 1c5b862c65..26e7fd0caf 100644 --- a/skills/open-code-review/SKILL.md +++ b/skills/open-code-review/SKILL.md @@ -30,7 +30,7 @@ Analyze the review target (commits, branch, or changes) to extract concise busin ### Step 2: Run Code Review -**GitHub PR mode:** A bare positive number (e.g. `42`) means PR #42 in the current repository; the `pr` prefix is optional. For a PR number or GitHub PR URL, run `python3 "$SKILL_DIR/scripts/review_pr.py" ` once, resolving `SKILL_DIR` from this installed skill. Forward an explicit `--repo OWNER/REPO` or `--background`. Read the printed `run.json` and its result file; report the reviewed revisions, freshness, and artifact path with Step 3. On failure report the failed step, not a fallback branch review. Only PR mode needs Python 3 and authenticated `gh`. +**GitHub PR mode:** A bare positive number (e.g. `42`) selects a PR in the GitHub CLI's default repository for this checkout, which can be upstream of a fork. The `pr` prefix is optional. For a PR number or GitHub PR URL, run `python3 "$SKILL_DIR/scripts/review_pr.py" ` once, resolving `SKILL_DIR` from this installed skill. Forward an explicit `--repo OWNER/REPO` or `--background`. Read the printed `run.json` and its result file; report the resolved PR URL, reviewed revisions, freshness, and artifact path with Step 3. On failure report the failed step, not a fallback branch review. Only PR mode needs Python 3 and authenticated `gh`. **Otherwise run `ocr review` directly.** Do not probe with `command -v ocr` or `ocr --version` first; that adds an unnecessary tool call on the normal path. diff --git a/skills/open-code-review/scripts/review_pr.py b/skills/open-code-review/scripts/review_pr.py index d14bfdac2f..db672b0ecf 100644 --- a/skills/open-code-review/scripts/review_pr.py +++ b/skills/open-code-review/scripts/review_pr.py @@ -192,7 +192,7 @@ def interrupt(signum, frame): if __name__ == "__main__": parser = argparse.ArgumentParser(description="Review one pinned GitHub pull request using native OCR.") - parser.add_argument("pr", help="Current repository PR number or full GitHub PR URL") + parser.add_argument("pr", help="PR number in the checkout's default gh repository, or a full GitHub PR URL") parser.add_argument("--repo", help="GitHub repository (OWNER/REPO)") parser.add_argument("--background", help="Business context for the review") args = parser.parse_args() From e7ed1c080ca55ceb87dfc5031d30d06e93838845 Mon Sep 17 00:00:00 2001 From: kite Date: Sun, 22 Nov 2026 12:36:28 +0800 Subject: [PATCH 3/4] fix(skills): isolate review policy for GitHub PR snapshots --- .github/workflows/plugin-contract.yml | 13 +++- docs/design/native-github-pr-snapshots.md | 2 + .../skills/open-code-review/SKILL.md | 2 + .../open-code-review/scripts/review_pr.py | 26 +++++++- scripts/github-actions/test_review_pr.py | 65 ++++++++++++++++++- skills/open-code-review/SKILL.md | 2 + skills/open-code-review/scripts/review_pr.py | 26 +++++++- 7 files changed, 128 insertions(+), 8 deletions(-) diff --git a/.github/workflows/plugin-contract.yml b/.github/workflows/plugin-contract.yml index 617688efe3..24d2c765e6 100644 --- a/.github/workflows/plugin-contract.yml +++ b/.github/workflows/plugin-contract.yml @@ -60,6 +60,16 @@ concurrency: cancel-in-progress: true jobs: + installed-pr-helper: + runs-on: ubuntu-latest + timeout-minutes: 10 + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + persist-credentials: false + - name: Test installed PR review helpers + run: python3 scripts/github-actions/test_review_pr.py + plugin-contract: runs-on: self-hosted timeout-minutes: 10 @@ -78,9 +88,6 @@ jobs: - name: Test plugin-contract guardrail run: node scripts/github-actions/check-plugin-contract.test.js - - name: Test installed PR review helpers - run: python3 scripts/github-actions/test_review_pr.py - # BLOCKING: no doc or README may link to a path that no longer exists. - name: Check in-repo path links run: node scripts/github-actions/check-plugin-contract.js links diff --git a/docs/design/native-github-pr-snapshots.md b/docs/design/native-github-pr-snapshots.md index 5d7651cc9b..cee32ef2d8 100644 --- a/docs/design/native-github-pr-snapshots.md +++ b/docs/design/native-github-pr-snapshots.md @@ -18,6 +18,7 @@ A bare positive integer selects PR mode in the [native skill](../../skills/open- | --- | --- | | Resolve | Read the canonical PR identity, actual target branch, base SHA, and head SHA from GitHub. For a fork PR, fetch from the base repository's PR ref, not a similarly named local branch. | | Freeze | Give each run its own bare Git repository, borrowing local objects from a matching non-shallow checkout when available. Fetch and verify both commit IDs, require a unique merge-base, and create a detached worktree at the verified head. No refs or worktrees are registered in the user's repository. If a fetched revision differs or the comparison cannot be established, stop before OCR. | +| Check policy | Refuse to launch OCR if the user's config declares any MCP servers (local or remote); never edit or copy credentials. This is a preflight check, not a process sandbox: keep the trusted user config stable during a run. Omit the checkout's root `.opencodereview` entry using sparse checkout: built-in and trusted user rules still apply, and the committed files remain readable at the pinned revisions. Native Review and Analysis consult versioned Git metadata, so stop for independent review if the PR changes the root `.gitignore`, root `.gitmodules`, or any `.gitattributes`. | | Review once | Invoke installed `ocr review` with `--repo --from --to --audience agent --analysis --format json --output `. Forward background as data. Keep the result outside the worktree. Disable checkout hooks and avoid submodule recursion, external diff drivers, PR-provided scripts, builds, tests, and GitHub writes. | | Release | On success, failure, or handled interruption, wait for the OCR child to stop, then remove only this run's verified, clean detached worktree without force. Drop the private object store and its refs; there are no shared refs to delete. Preserve the run summary and any OCR output. Record incomplete cleanup rather than pruning other worktrees or silently deleting modified files. | | Recheck and return | Before returning a review, reread `(target branch, base SHA, head SHA)`. All equal means `current` **at the recorded check time**; any difference means `stale`; an unavailable or incomplete response means `unverified`. Never rerun OCR automatically. | @@ -33,3 +34,4 @@ Use real local Git fixtures, stub GitHub responses and a recording OCR executabl - **Isolation and cleanup:** Start two reviews of one PR from a dirty checkout; fail or interrupt one. Confirm independent refs, worktrees, and artifacts, an unchanged source HEAD/index/files, and that one run cannot clean the other. A modified review worktree is left intact and reported. - **Freshness:** Exercise unchanged, retargeted, advanced-base, changed-head, and unavailable responses. Assert the reported state, retained result, and absence of an automatic rerun. - **Distribution:** Run the same contract tests from each installed skill layout without the external wrapper. Finish with a read-only smoke review against a real PR. +- **Policy:** Configured MCP or changes to Git filtering metadata stop before OCR starts. A head-only project rule is still available as Git evidence but is never loaded as policy. Set `PR_TEST_NATIVE_OCR` to the absolute path of an installed OCR executable to also exercise its real, provider-free preview on the policy fixture. diff --git a/plugins/open-code-review/skills/open-code-review/SKILL.md b/plugins/open-code-review/skills/open-code-review/SKILL.md index a46d839868..ceead2536a 100644 --- a/plugins/open-code-review/skills/open-code-review/SKILL.md +++ b/plugins/open-code-review/skills/open-code-review/SKILL.md @@ -37,6 +37,8 @@ Analyze the review target (commits, branch, or changes) to extract concise busin **GitHub PR mode:** A bare positive number (e.g. `42`) selects a PR in the GitHub CLI's default repository for this checkout, which can be upstream of a fork. The `pr` prefix is optional. For a PR number or GitHub PR URL, run `python3 "$SKILL_DIR/scripts/review_pr.py" ` once, resolving `SKILL_DIR` from this installed skill. Forward an explicit `--repo OWNER/REPO` or `--background`. Read the printed `run.json` and its result file; report the resolved PR URL, reviewed revisions, freshness, and artifact path with Step 3. On failure report the failed step, not a fallback branch review. Only PR mode needs Python 3 and authenticated `gh`. +PR mode uses built-in and user review rules, not rules from the PR. It stops if MCP servers are configured or the PR changes Git filtering metadata; do not bypass a policy stop. + **Otherwise run `ocr review` directly.** Do not probe with `command -v ocr` or `ocr --version` first; that adds an unnecessary tool call on the normal path. For a workspace review, omit revision flags and `--analysis`. **Pass business context via `--background`** when available: diff --git a/plugins/open-code-review/skills/open-code-review/scripts/review_pr.py b/plugins/open-code-review/skills/open-code-review/scripts/review_pr.py index db672b0ecf..a7816add65 100644 --- a/plugins/open-code-review/skills/open-code-review/scripts/review_pr.py +++ b/plugins/open-code-review/skills/open-code-review/scripts/review_pr.py @@ -102,6 +102,26 @@ def release(folder, head, env): shutil.rmtree(store) +def prepare_review_policy(tree, base, head, env): + config = Path.home() / ".opencodereview" / "config.json" + try: + settings = json.loads(config.read_bytes(), object_pairs_hook=tuple) + except FileNotFoundError: + settings = () + if not isinstance(settings, tuple): + raise ValueError("expected an OCR configuration object") + if any(key.casefold() == "mcp_servers" and value not in (None, ()) for key, value in settings): + raise ValueError("PR mode requires an OCR configuration without MCP servers; no review started") + changed = command("git", "diff-tree", "-r", "--no-commit-id", "--name-only", "--no-renames", base, head, "--", + ":(top,icase).gitignore", ":(top,icase).gitmodules", ":(top,glob,icase)**/.gitattributes", + cwd=tree, env=env) + if changed: + raise ValueError(f"PR changes Git filtering or diff policy; independent review required: {changed}") + command("git", "sparse-checkout", "set", "--no-cone", "--", "/*", "!/.opencodereview", cwd=tree, env=env) + if os.path.lexists(tree / ".opencodereview"): + raise ValueError("project review policy is still present in the worktree; no review started") + + def review(folder, base, head, background, env): args = ["ocr", "review", "--repo", str(folder / "worktree"), "--from", base, "--to", head, "--audience", "agent", "--analysis", "--format", "json", "--output", str(folder / "review.json")] @@ -145,11 +165,15 @@ def run(number, repo, background): step = "lookup" code = 1 try: - repo, snapshot = lookup(number, repo, None, env) + initial_env = dict(env, GH_REPO=os.environ["GH_REPO"]) if not repo and os.environ.get("GH_REPO") else env + repo, snapshot = lookup(number, repo, None, initial_env) summary.update(url=f"https://github.com/{repo}/pull/{number}", snapshot=snapshot) step = "freeze" base = freeze(folder, repo, number, snapshot, env, offline) summary["merge_base"] = base + step = "policy" + prepare_review_policy(folder / "worktree", base, snapshot["head"], offline) + summary["policy"] = {"project_rules": "omitted", "mcp_config": "checked_empty"} save(manifest, summary) step = "review" summary["ocr_exit"] = review(folder, base, snapshot["head"], background, offline) diff --git a/scripts/github-actions/test_review_pr.py b/scripts/github-actions/test_review_pr.py index 254c44afe5..f8334ee203 100644 --- a/scripts/github-actions/test_review_pr.py +++ b/scripts/github-actions/test_review_pr.py @@ -26,8 +26,11 @@ assert sys.argv[1:4] == ['pr', 'view', '42'], sys.argv if '--repo' in sys.argv: assert sys.argv[sys.argv.index('--repo') + 1] == 'github.com/team/project' + assert 'GH_REPO' not in os.environ else: assert Path.cwd() == Path(os.environ['SOURCE']) + if os.environ.get('EXPECT_GH_REPO'): + assert os.environ.get('GH_REPO') == 'team/project' data = json.loads(os.environ['METADATA']) if Path('review.json').exists(): assert '--repo' in sys.argv @@ -39,12 +42,21 @@ """ OCR = """ -import json, os, sys, time +import json, os, subprocess, sys, time from pathlib import Path args = sys.argv[1:] with open('calls.jsonl', 'a') as log: log.write(json.dumps(args) + '\\n') mode = os.environ.get('MODE') +if mode == 'policy': + tree = Path(args[args.index('--repo') + 1]) + head = args[args.index('--to') + 1] + assert not (tree / '.opencodereview').exists() + assert subprocess.check_output(['git', '-C', str(tree), 'show', head + ':.opencodereview/rule.json']).strip() == b'not a rule document' + if os.environ.get('PR_TEST_NATIVE_OCR'): + preview = subprocess.check_output([os.environ['PR_TEST_NATIVE_OCR'], 'review', '--preview', '--format', 'json', + '--repo', str(tree), '--from', args[args.index('--from') + 1], '--to', head]) + assert any(entry['path'] == 'change.go' and entry['will_review'] for entry in json.loads(preview)['files']) if mode == 'wait': while not Path('resume').exists(): time.sleep(0.02) @@ -100,8 +112,8 @@ def setUp(self): def git(self, *args): return subprocess.check_output(["git", *args], cwd=self.source, env=self.env, text=True).strip() - def commit(self, name): - (self.source / name).write_text(name) + def commit(self, name, content=None): + (self.source / name).write_text(name if content is None else content) self.git("add", name) self.git("commit", "-qm", name) return self.git("rev-parse", "HEAD") @@ -152,6 +164,53 @@ def test_bare_number_uses_upstream_default_in_fork(self): self.assertEqual(run["url"], URL) self.assertEqual(run["freshness"], "current") + def test_gh_repo_applies_only_to_initial_number_lookup(self): + _, run = self.finish(self.start("42", GH_REPO="team/project", EXPECT_GH_REPO="1")) + self.assertEqual(run["url"], URL) + self.assertEqual(run["freshness"], "current") + + def test_head_policy_is_evidence_only(self): + self.git("switch", "-q", "topic") + policy = self.source / ".opencodereview" / "rule.json" + policy.parent.mkdir() + policy.write_text("not a rule document") + self.git("add", ".opencodereview/rule.json") + head = self.commit("change.go", "package example\n") + self.git("update-ref", "refs/pull/42/head", head) + self.git("switch", "-q", "release") + _, run = self.finish(self.start("42", MODE="policy", METADATA=json.dumps(dict(self.metadata, headRefOid=head)))) + self.assertEqual(run["cleanup"], "complete") + + def test_configured_mcp_stops_before_ocr(self): + config = self.root / ".opencodereview" / "config.json" + config.parent.mkdir() + for server in ({"command": "not-installed"}, {"type": "remote", "url": "https://example.invalid"}): + with self.subTest(server=server): + data = {"MCP_SERVERS": {"example": server}} + config.write_text(json.dumps(data)) + folder, run = self.finish(self.start("42"), code=1) + self.assertEqual(run["failed_step"], "policy") + self.assertIn("MCP", run["error"]) + self.assertFalse((folder / "calls.jsonl").exists()) + self.assertEqual(json.loads(config.read_text()), data) + + def test_changed_git_policy_requires_independent_review(self): + self.git("switch", "-q", "topic") + for name in (".gitignore", "nested/.gitattributes", ".gitmodules"): + with self.subTest(name=name): + self.git("reset", "--hard", self.head) + (self.source / name).parent.mkdir(parents=True, exist_ok=True) + content = "# Policy fixture\n" + if name == ".gitmodules": + self.git("update-index", "--add", "--cacheinfo", f"160000,{self.common},vendor/lib") + content = '[submodule "lib"]\n\tpath = vendor/lib\n' + head = self.commit(name, content) + self.git("update-ref", "refs/pull/42/head", head) + folder, run = self.finish(self.start("42", METADATA=json.dumps(dict(self.metadata, headRefOid=head))), code=1) + self.assertEqual(run["failed_step"], "policy") + self.assertIn(name, run["error"]) + self.assertFalse((folder / "calls.jsonl").exists()) + def test_explicit_identity_and_freshness(self): for delta, expected in (({"baseRefName": "main"}, "stale"), ({"baseRefOid": self.common}, "stale"), ({"headRefOid": self.base}, "stale"), (None, "unverified")): diff --git a/skills/open-code-review/SKILL.md b/skills/open-code-review/SKILL.md index 26e7fd0caf..88c9c35234 100644 --- a/skills/open-code-review/SKILL.md +++ b/skills/open-code-review/SKILL.md @@ -32,6 +32,8 @@ Analyze the review target (commits, branch, or changes) to extract concise busin **GitHub PR mode:** A bare positive number (e.g. `42`) selects a PR in the GitHub CLI's default repository for this checkout, which can be upstream of a fork. The `pr` prefix is optional. For a PR number or GitHub PR URL, run `python3 "$SKILL_DIR/scripts/review_pr.py" ` once, resolving `SKILL_DIR` from this installed skill. Forward an explicit `--repo OWNER/REPO` or `--background`. Read the printed `run.json` and its result file; report the resolved PR URL, reviewed revisions, freshness, and artifact path with Step 3. On failure report the failed step, not a fallback branch review. Only PR mode needs Python 3 and authenticated `gh`. +PR mode uses built-in and user review rules, not rules from the PR. It stops if MCP servers are configured or the PR changes Git filtering metadata; do not bypass a policy stop. + **Otherwise run `ocr review` directly.** Do not probe with `command -v ocr` or `ocr --version` first; that adds an unnecessary tool call on the normal path. For a workspace review, omit revision flags and `--analysis`. **Pass business context via `--background`** when available: diff --git a/skills/open-code-review/scripts/review_pr.py b/skills/open-code-review/scripts/review_pr.py index db672b0ecf..a7816add65 100644 --- a/skills/open-code-review/scripts/review_pr.py +++ b/skills/open-code-review/scripts/review_pr.py @@ -102,6 +102,26 @@ def release(folder, head, env): shutil.rmtree(store) +def prepare_review_policy(tree, base, head, env): + config = Path.home() / ".opencodereview" / "config.json" + try: + settings = json.loads(config.read_bytes(), object_pairs_hook=tuple) + except FileNotFoundError: + settings = () + if not isinstance(settings, tuple): + raise ValueError("expected an OCR configuration object") + if any(key.casefold() == "mcp_servers" and value not in (None, ()) for key, value in settings): + raise ValueError("PR mode requires an OCR configuration without MCP servers; no review started") + changed = command("git", "diff-tree", "-r", "--no-commit-id", "--name-only", "--no-renames", base, head, "--", + ":(top,icase).gitignore", ":(top,icase).gitmodules", ":(top,glob,icase)**/.gitattributes", + cwd=tree, env=env) + if changed: + raise ValueError(f"PR changes Git filtering or diff policy; independent review required: {changed}") + command("git", "sparse-checkout", "set", "--no-cone", "--", "/*", "!/.opencodereview", cwd=tree, env=env) + if os.path.lexists(tree / ".opencodereview"): + raise ValueError("project review policy is still present in the worktree; no review started") + + def review(folder, base, head, background, env): args = ["ocr", "review", "--repo", str(folder / "worktree"), "--from", base, "--to", head, "--audience", "agent", "--analysis", "--format", "json", "--output", str(folder / "review.json")] @@ -145,11 +165,15 @@ def run(number, repo, background): step = "lookup" code = 1 try: - repo, snapshot = lookup(number, repo, None, env) + initial_env = dict(env, GH_REPO=os.environ["GH_REPO"]) if not repo and os.environ.get("GH_REPO") else env + repo, snapshot = lookup(number, repo, None, initial_env) summary.update(url=f"https://github.com/{repo}/pull/{number}", snapshot=snapshot) step = "freeze" base = freeze(folder, repo, number, snapshot, env, offline) summary["merge_base"] = base + step = "policy" + prepare_review_policy(folder / "worktree", base, snapshot["head"], offline) + summary["policy"] = {"project_rules": "omitted", "mcp_config": "checked_empty"} save(manifest, summary) step = "review" summary["ocr_exit"] = review(folder, base, snapshot["head"], background, offline) From b0022d33d2437b512f8e8c91c2530ab49d503fd8 Mon Sep 17 00:00:00 2001 From: kite Date: Sun, 22 Nov 2026 12:47:08 +0800 Subject: [PATCH 4/4] fix(skills): preserve native review reports --- .../skills/open-code-review/SKILL.md | 47 ++----------------- skills/open-code-review/SKILL.md | 47 ++----------------- 2 files changed, 8 insertions(+), 86 deletions(-) diff --git a/plugins/open-code-review/skills/open-code-review/SKILL.md b/plugins/open-code-review/skills/open-code-review/SKILL.md index ceead2536a..6d5ad56d2b 100644 --- a/plugins/open-code-review/skills/open-code-review/SKILL.md +++ b/plugins/open-code-review/skills/open-code-review/SKILL.md @@ -84,12 +84,12 @@ Omit `--analysis` for workspace reviews, `--preview`, `--resume`, or `--format s ### Step 3: Report -OCR output includes structured `severity` (critical / high / medium / low) and `category` (bug / security / performance / maintainability / test / style / documentation / other) on each comment. Present results grouped by severity, discarding `low` severity items that are likely false positives or nitpicks. +Show OCR's user-facing report in the reply. Summarize, translate, or filter only if requested. For text output, relay the report as emitted. For JSON output, present the report content rather than dumping internal fields such as `thinking`: -For combined runs, report Review and Analysis separately. Use these fields in `ocr.review-with-analysis/v2` JSON; if the caller requested text, use the Review summary and appended Analysis Report instead: +- **Review:** present `comments` in their original order, retaining wording, severity, category, file and line range, and any code suggestions. Include low-severity findings. Report coverage from `review_status` and `manifest.coverage` when available; an empty comment list does not prove everything was reviewed. +- **Analysis (combined runs):** render the `analysis` Markdown directly, not `analysis_raw` or a paraphrase. Preserve its headings, tables (including every row), lists, and caveats. If the original uses prose, keep the prose. -- **Review:** report findings from `comments` and coverage from `review_status` and `manifest.coverage` when available. A zero exit code or empty `comments` does not prove all selected files were reviewed. -- **Analysis:** summarize the before/after behavior, affected workflows, and any decisions supported by `analysis`. Distinguish `analysis_meta.execution_status` from `analysis_meta.declared_completion` (the model's assessment). For example, `completed` with declared `partial` means the investigation ran but left questions open. If analysis was skipped or failed, say why when known and retain the Review findings; do not infer that behavior is unchanged. If the combined schema or analysis metadata is missing, report analysis as unverified. +Keep PR snapshot details and any host verification separate from OCR's report. Distinguish `analysis_meta.execution_status` from `analysis_meta.declared_completion`; report skipped, failed, or unverified Analysis without discarding Review. If the report cannot fit in the reply, identify what was omitted and give the full artifact path. ### Step 4: Fix @@ -105,45 +105,6 @@ When fixing issues and suggestions: - For complex fixes requiring manual intervention, clearly describe what needs to be done - Always verify fixes with the user before committing -## Output Format - -Each comment in OCR's output contains: - -- `path`: File path -- `content`: Review comment text -- `start_line` / `end_line`: Line range (both 0 means positioning failed) -- `category`: Issue category (bug, security, performance, maintainability, test, style, documentation, other) -- `severity`: Issue severity (critical, high, medium, low) -- `suggestion_code`: Optional fix suggestion -- `existing_code`: Optional original code snippet -- `thinking`: Optional LLM reasoning process - -Present results grouped by severity using this template: - -```markdown -## Code Review Results - -**Files reviewed**: N -**Issues found**: X critical, Y high, Z medium - -### Critical - -- **`path/to/file.java:42`** [bug] — Brief description - > Recommendation: How to fix - -### High - -- **`path/to/file.java:26`** [bug] — Brief description - > Recommendation: How to fix - -### Medium - -- **`path/to/file.ts:88`** [performance] — Brief description - > Recommendation: How to fix (if applicable) -``` - -Only use "Review complete — no critical, high, or medium issues found in N files" when Review coverage supports it. Otherwise state the known coverage and gaps. For combined runs, also report the Analysis outcome separately. - **Handling mispositioned comments:** When `start_line` and `end_line` are both `0`, the comment failed to locate the exact position in the file. In such cases: diff --git a/skills/open-code-review/SKILL.md b/skills/open-code-review/SKILL.md index 88c9c35234..a88ca72caa 100644 --- a/skills/open-code-review/SKILL.md +++ b/skills/open-code-review/SKILL.md @@ -79,12 +79,12 @@ Omit `--analysis` for workspace reviews, `--preview`, `--resume`, or `--format s ### Step 3: Report -OCR output includes structured `severity` (critical / high / medium / low) and `category` (bug / security / performance / maintainability / test / style / documentation / other) on each comment. Present results grouped by severity, discarding `low` severity items that are likely false positives or nitpicks. +Show OCR's user-facing report in the reply. Summarize, translate, or filter only if requested. For text output, relay the report as emitted. For JSON output, present the report content rather than dumping internal fields such as `thinking`: -For combined runs, report Review and Analysis separately. Use these fields in `ocr.review-with-analysis/v2` JSON; if the caller requested text, use the Review summary and appended Analysis Report instead: +- **Review:** present `comments` in their original order, retaining wording, severity, category, file and line range, and any code suggestions. Include low-severity findings. Report coverage from `review_status` and `manifest.coverage` when available; an empty comment list does not prove everything was reviewed. +- **Analysis (combined runs):** render the `analysis` Markdown directly, not `analysis_raw` or a paraphrase. Preserve its headings, tables (including every row), lists, and caveats. If the original uses prose, keep the prose. -- **Review:** report findings from `comments` and coverage from `review_status` and `manifest.coverage` when available. A zero exit code or empty `comments` does not prove all selected files were reviewed. -- **Analysis:** summarize the before/after behavior, affected workflows, and any decisions supported by `analysis`. Distinguish `analysis_meta.execution_status` from `analysis_meta.declared_completion` (the model's assessment). For example, `completed` with declared `partial` means the investigation ran but left questions open. If analysis was skipped or failed, say why when known and retain the Review findings; do not infer that behavior is unchanged. If the combined schema or analysis metadata is missing, report analysis as unverified. +Keep PR snapshot details and any host verification separate from OCR's report. Distinguish `analysis_meta.execution_status` from `analysis_meta.declared_completion`; report skipped, failed, or unverified Analysis without discarding Review. If the report cannot fit in the reply, identify what was omitted and give the full artifact path. ### Step 4: Fix @@ -100,45 +100,6 @@ When fixing issues and suggestions: - For complex fixes requiring manual intervention, clearly describe what needs to be done - Always verify fixes with the user before committing -## Output Format - -Each comment in OCR's output contains: - -- `path`: File path -- `content`: Review comment text -- `start_line` / `end_line`: Line range (both 0 means positioning failed) -- `category`: Issue category (bug, security, performance, maintainability, test, style, documentation, other) -- `severity`: Issue severity (critical, high, medium, low) -- `suggestion_code`: Optional fix suggestion -- `existing_code`: Optional original code snippet -- `thinking`: Optional LLM reasoning process - -Present results grouped by severity using this template: - -```markdown -## Code Review Results - -**Files reviewed**: N -**Issues found**: X critical, Y high, Z medium - -### Critical - -- **`path/to/file.java:42`** [bug] — Brief description - > Recommendation: How to fix - -### High - -- **`path/to/file.java:26`** [bug] — Brief description - > Recommendation: How to fix - -### Medium - -- **`path/to/file.ts:88`** [performance] — Brief description - > Recommendation: How to fix (if applicable) -``` - -Only use "Review complete — no critical, high, or medium issues found in N files" when Review coverage supports it. Otherwise state the known coverage and gaps. For combined runs, also report the Analysis outcome separately. - **Handling mispositioned comments:** When `start_line` and `end_line` are both `0`, the comment failed to locate the exact position in the file. In such cases: