Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions .github/workflows/plugin-contract.yml
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ on:
- "CONTRIBUTING*.md"
- "AGENTS.md"
- "scripts/github-actions/check-plugin-contract.*"
- "scripts/github-actions/test_review_pr.py"
- ".github/workflows/plugin-contract.yml"
pull_request:
branches: [main]
Expand All @@ -48,6 +49,7 @@ on:
- "CONTRIBUTING*.md"
- "AGENTS.md"
- "scripts/github-actions/check-plugin-contract.*"
- "scripts/github-actions/test_review_pr.py"
- ".github/workflows/plugin-contract.yml"

permissions:
Expand All @@ -58,6 +60,16 @@ concurrency:
cancel-in-progress: true

jobs:
installed-pr-helper:
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- name: Test installed PR review helpers
run: python3 scripts/github-actions/test_review_pr.py

plugin-contract:
runs-on: self-hosted
timeout-minutes: 10
Expand Down
37 changes: 37 additions & 0 deletions docs/design/native-github-pr-snapshots.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
# PR mode for the native review skill

**Decision:** The maintainer accepted the phase-one pilot. This is the design of the native skill's PR mode for [#2025](https://github.com/alibaba/open-code-review/issues/2025); pilot evidence is tracked separately in [#2023](https://github.com/alibaba/open-code-review/issues/2023).

## User interface

A bare positive integer selects PR mode in the [native skill](../../skills/open-code-review/SKILL.md): `42` and `pr 42` both review PR 42 in the GitHub CLI's default repository for this checkout. In a fork this can be the upstream repository rather than `origin`. Report the resolved PR URL so the target is visible. `42 --repo owner/repo` selects a different repository; a GitHub PR URL selects the same mode without needing `--repo`. If the repository cannot be resolved or the URL conflicts with `--repo`, report the error rather than guessing. This is skill input, not a new `ocr` CLI command. Local workspace, commit, and branch reviews keep their existing behavior and dependencies.

## The entire PR-specific skill instruction

> For a bare PR number, `pr <id>`, or a GitHub PR URL, run the installed `scripts/review_pr.py` once, as in `python3 "$SKILL_DIR/scripts/review_pr.py" 42`. Forward an explicit `--repo` or `--background` if supplied. Read the returned run summary and OCR result. Report the reviewed revisions and freshness alongside the normal Review and Analysis outcomes. On failure, report the failed step; do not substitute a branch review.

`SKILL_DIR` denotes the installed skill directory, never the PR checkout. Ship the same script alongside the standalone and [plugin](../../plugins/open-code-review/skills/open-code-review/SKILL.md) instructions. The skill has no Git command sequence, cleanup checklist, or dependency on the pilot wrapper.

## What the script guarantees

| Step | Deterministic behavior |
| --- | --- |
| Resolve | Read the canonical PR identity, actual target branch, base SHA, and head SHA from GitHub. For a fork PR, fetch from the base repository's PR ref, not a similarly named local branch. |
| Freeze | Give each run its own bare Git repository, borrowing local objects from a matching non-shallow checkout when available. Fetch and verify both commit IDs, require a unique merge-base, and create a detached worktree at the verified head. No refs or worktrees are registered in the user's repository. If a fetched revision differs or the comparison cannot be established, stop before OCR. |
| Check policy | Refuse to launch OCR if the user's config declares any MCP servers (local or remote); never edit or copy credentials. This is a preflight check, not a process sandbox: keep the trusted user config stable during a run. Omit the checkout's root `.opencodereview` entry using sparse checkout: built-in and trusted user rules still apply, and the committed files remain readable at the pinned revisions. Native Review and Analysis consult versioned Git metadata, so stop for independent review if the PR changes the root `.gitignore`, root `.gitmodules`, or any `.gitattributes`. |
| Review once | Invoke installed `ocr review` with `--repo <worktree> --from <merge-base> --to <head> --audience agent --analysis --format json --output <result>`. Forward background as data. Keep the result outside the worktree. Disable checkout hooks and avoid submodule recursion, external diff drivers, PR-provided scripts, builds, tests, and GitHub writes. |
| Release | On success, failure, or handled interruption, wait for the OCR child to stop, then remove only this run's verified, clean detached worktree without force. Drop the private object store and its refs; there are no shared refs to delete. Preserve the run summary and any OCR output. Record incomplete cleanup rather than pruning other worktrees or silently deleting modified files. |
| Recheck and return | Before returning a review, reread `(target branch, base SHA, head SHA)`. All equal means `current` **at the recorded check time**; any difference means `stale`; an unavailable or incomplete response means `unverified`. Never rerun OCR automatically. |

The script prints the absolute path to `run.json`. It records the frozen identity and revisions, OCR exit status and result path (if produced), freshness and check time (if checked), cleanup status, and any failed step. `review.json` remains the native OCR output; do not copy its findings or reinterpret coverage in the script. For example, a run can be `current` with partial Analysis, or `stale` with complete Analysis. An interrupted run must not be reported as reviewed merely because its worktree was cleaned up.

## Acceptance checks

Use real local Git fixtures, stub GitHub responses and a recording OCR executable. Test the installed script as a process, not just its internal functions.

- **Routing:** A numeric ID follows the GitHub CLI's repository selection, including an upstream default in a fork checkout. An override or URL selects the stated repository from an unrelated checkout. Conflicting inputs stop before fetch. Non-PR reviews work without `gh` or Python.
- **Revision identity:** A fork PR targets a non-default branch that has advanced since divergence. Assert exactly one OCR call, from the merge-base rather than the target tip, to the advertised head. A mismatch during fetch or a non-unique merge-base yields zero calls.
- **Isolation and cleanup:** Start two reviews of one PR from a dirty checkout; fail or interrupt one. Confirm independent refs, worktrees, and artifacts, an unchanged source HEAD/index/files, and that one run cannot clean the other. A modified review worktree is left intact and reported.
- **Freshness:** Exercise unchanged, retargeted, advanced-base, changed-head, and unavailable responses. Assert the reported state, retained result, and absence of an automatic rerun.
- **Distribution:** Run the same contract tests from each installed skill layout without the external wrapper. Finish with a read-only smoke review against a real PR.
- **Policy:** Configured MCP or changes to Git filtering metadata stop before OCR starts. A head-only project rule is still available as Git evidence but is never loaded as policy. Set `PR_TEST_NATIVE_OCR` to the absolute path of an installed OCR executable to also exercise its real, provider-free preview on the policy fixture.
55 changes: 10 additions & 45 deletions plugins/open-code-review/skills/open-code-review/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,11 @@ Analyze the review target (commits, branch, or changes) to extract concise busin

### Step 2: Run Code Review

**Run `ocr review` directly.** Do not probe with `command -v ocr` or `ocr --version` first; that adds an unnecessary tool call on the normal path.
**GitHub PR mode:** A bare positive number (e.g. `42`) selects a PR in the GitHub CLI's default repository for this checkout, which can be upstream of a fork. The `pr` prefix is optional. For a PR number or GitHub PR URL, run `python3 "$SKILL_DIR/scripts/review_pr.py" <id-or-url>` once, resolving `SKILL_DIR` from this installed skill. Forward an explicit `--repo OWNER/REPO` or `--background`. Read the printed `run.json` and its result file; report the resolved PR URL, reviewed revisions, freshness, and artifact path with Step 3. On failure report the failed step, not a fallback branch review. Only PR mode needs Python 3 and authenticated `gh`.

PR mode uses built-in and user review rules, not rules from the PR. It stops if MCP servers are configured or the PR changes Git filtering metadata; do not bypass a policy stop.

**Otherwise run `ocr review` directly.** Do not probe with `command -v ocr` or `ocr --version` first; that adds an unnecessary tool call on the normal path.

For a workspace review, omit revision flags and `--analysis`. **Pass business context via `--background`** when available:

Expand All @@ -60,7 +64,7 @@ ocr review --audience agent --background "business context here"
| User says | Command to run |
|-----------|---------------|
| "review my changes" / "review the working copy" | `ocr review --audience agent -b "context"` |
| "review this PR" / "review feature branch" | `ocr review --audience agent -b "context" --from <base> --to <branch> --analysis --format json` |
| "review feature branch" | `ocr review --audience agent -b "context" --from <base> --to <branch> --analysis --format json` |
| "review the current branch against main" | `ocr review --audience agent -b "context" --from main --analysis --format json` |
| "review commit abc123" | `ocr review --audience agent -b "context" --commit abc123 --analysis --format json` |
| "what would be reviewed?" (dry-run) | `ocr review --preview` |
Expand All @@ -80,12 +84,12 @@ Omit `--analysis` for workspace reviews, `--preview`, `--resume`, or `--format s

### Step 3: Report

OCR output includes structured `severity` (critical / high / medium / low) and `category` (bug / security / performance / maintainability / test / style / documentation / other) on each comment. Present results grouped by severity, discarding `low` severity items that are likely false positives or nitpicks.
Show OCR's user-facing report in the reply. Summarize, translate, or filter only if requested. For text output, relay the report as emitted. For JSON output, present the report content rather than dumping internal fields such as `thinking`:

For combined runs, report Review and Analysis separately. Use these fields in `ocr.review-with-analysis/v2` JSON; if the caller requested text, use the Review summary and appended Analysis Report instead:
- **Review:** present `comments` in their original order, retaining wording, severity, category, file and line range, and any code suggestions. Include low-severity findings. Report coverage from `review_status` and `manifest.coverage` when available; an empty comment list does not prove everything was reviewed.
- **Analysis (combined runs):** render the `analysis` Markdown directly, not `analysis_raw` or a paraphrase. Preserve its headings, tables (including every row), lists, and caveats. If the original uses prose, keep the prose.

- **Review:** report findings from `comments` and coverage from `review_status` and `manifest.coverage` when available. A zero exit code or empty `comments` does not prove all selected files were reviewed.
- **Analysis:** summarize the before/after behavior, affected workflows, and any decisions supported by `analysis`. Distinguish `analysis_meta.execution_status` from `analysis_meta.declared_completion` (the model's assessment). For example, `completed` with declared `partial` means the investigation ran but left questions open. If analysis was skipped or failed, say why when known and retain the Review findings; do not infer that behavior is unchanged. If the combined schema or analysis metadata is missing, report analysis as unverified.
Keep PR snapshot details and any host verification separate from OCR's report. Distinguish `analysis_meta.execution_status` from `analysis_meta.declared_completion`; report skipped, failed, or unverified Analysis without discarding Review. If the report cannot fit in the reply, identify what was omitted and give the full artifact path.

### Step 4: Fix

Expand All @@ -101,45 +105,6 @@ When fixing issues and suggestions:
- For complex fixes requiring manual intervention, clearly describe what needs to be done
- Always verify fixes with the user before committing

## Output Format

Each comment in OCR's output contains:

- `path`: File path
- `content`: Review comment text
- `start_line` / `end_line`: Line range (both 0 means positioning failed)
- `category`: Issue category (bug, security, performance, maintainability, test, style, documentation, other)
- `severity`: Issue severity (critical, high, medium, low)
- `suggestion_code`: Optional fix suggestion
- `existing_code`: Optional original code snippet
- `thinking`: Optional LLM reasoning process

Present results grouped by severity using this template:

```markdown
## Code Review Results

**Files reviewed**: N
**Issues found**: X critical, Y high, Z medium

### Critical

- **`path/to/file.java:42`** [bug] — Brief description
> Recommendation: How to fix

### High

- **`path/to/file.java:26`** [bug] — Brief description
> Recommendation: How to fix

### Medium

- **`path/to/file.ts:88`** [performance] — Brief description
> Recommendation: How to fix (if applicable)
```

Only use "Review complete — no critical, high, or medium issues found in N files" when Review coverage supports it. Otherwise state the known coverage and gaps. For combined runs, also report the Analysis outcome separately.

**Handling mispositioned comments:**

When `start_line` and `end_line` are both `0`, the comment failed to locate the exact position in the file. In such cases:
Expand Down
Loading
Loading