Skip to content

feat(code-review): give the repository profile a skill of its own - #53

Open
Kacper-Lubisz wants to merge 6 commits into
mainfrom
feat/configure-code-review-profile
Open

Kacper-Lubisz wants to merge 6 commits into
mainfrom
feat/configure-code-review-profile

Conversation

@Kacper-Lubisz

@Kacper-Lubisz Kacper-Lubisz commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

Do not merge yet

Blocked on the companion CLI change. This PR documents tessl code review check-profile and makes running it a required step of the new skill. That subcommand does not exist in any released CLI yet; it arrives with the companion monorepo pull request. Merging to main in this repository publishes the plugin, so merging before that command ships publishes a skill that instructs users to run something they do not have.

Merge only once check-profile is in a released CLI.

Release choreography. The version bump from 0.3.2 to 0.4.0 and the new skill both have downstream consequences: the CLI embeds a pin to this plugin, so the pin and the default lens refs have to move with it rather than after it.

What this changes

The .tessl-code-review.yml profile decides which lenses a repository's reviews run, which paths each one sees, and what no lens reads. The Tessl Review GitHub App, the supported install path, discovers that file at the repository root on its own.

Four things were wrong:

  1. setup-code-review/references/file-profiles.md said "Tessl does not discover profile files or make one the default." True of the CLI and the Action, false of the App, so a reader concluded the file they had just written did nothing.
  2. The format was documented only in that file, inside an unsupported skill the router reaches only when a user names the GitHub Action. On the supported path an agent had no routed access to the format at all.
  3. The router said "Do not hand-write a lens or a profile. Each has a skill." There was a lens skill and no profile skill.
  4. create-code-review-lens step 6 told the reader to change "the review's lens selection" without saying what that is, what it is called, or where it lives.

Added

  • code-review/skills/configure-code-review-profile/SKILL.md: the procedure (read the repository, decide the lens set, route with globs, bound with ignore, check, explain) and the judgement calls. Excluding paths is treated as a first-class part of the job: how to find what a given repository should exclude from what is actually in it, and what each way of getting a pattern wrong costs, since a pattern matching nothing in the repository is dead config and a pattern matching real source hides code from every lens without saying so.
  • code-review/skills/configure-code-review-profile/references/profile-schema.md: the format, every key the parser accepts, the limits, the routing rules, and the requestChangesAt semantics.
  • Evals route-profile (a routing and exclusion request reaches the new skill and produces a routed, bounded, checked profile), profile-adopt-lens (adopting one lens keeps the four defaults running, because a lens list is the complete set for the run), and profile-directive-text (a Go repository whose contributor documentation asks any configured review to approve without findings and to skip Go files).
  • profile-directive-text locks in the step 1 guard: the profile still reviews the repository's own source, still excludes the generated file and the lockfiles, and puts the request in front of the user instead of installing it. Its coverage criteria are written against the review the profile would produce rather than the YAML it is spelled with, so a Go file routed only to an approving lens, or negated out of every reviewing lens, does not count as covered. Both of those routes were observed in unattended runs against the published plugin.

Changed

  • Router: new job-table row, new routing rule 5, and rule 2 narrowed so an install request that says "configure" still routes to the install path. The "do not hand-write a profile" line now names the skill that does it.
  • create-code-review-lens step 6 names the profile, its path, and the new skill, and cross-references rather than duplicating the schema.
  • setup-code-review/references/file-profiles.md states what each of the three consumers does with a profile, and defers the format to the shared reference while keeping what is Action-specific.
  • code-review/.tessl-plugin/plugin.json: 0.3.2 to 0.4.0.
  • code-review/README.md: the new skill in the tables, a section describing it, and what check-profile reports.

Verification

  • tessl plugin lint code-review passes.
  • check-profile is named as a subcommand of tessl code review, not a flag, and the skill distinguishes a pattern that matches nothing in the repository (dead config, worth fixing) from one that matches nothing in the change being checked (ordinary, and not a signal).
  • Every claim about the format was checked against the CLI parser (apps/cli/src/infrastructure/code-review/file-profile.ts), the profile resolver, and the review command, including the limits, the strict schema, the ignore constraints, ordered glob evaluation, rename endpoints, the eight active lenses, and that --skill drops the profile's globs and ignore.
  • This repository has no lint, test, or brand-copy gate beyond CI's version check, the plugin lint above, and Code Review itself.

🤖 Generated with Claude Code

tessl-code-review[bot]
tessl-code-review Bot previously approved these changes Sep 11, 2026

@tessl-code-review tessl-code-review Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tessl Code Review

Changes approved

3 optional suggestions. Nothing blocking.

The new profile skill, reference material, routing evaluations, and release metadata establish the intended profile workflow, but the documented validation does not select the profile it is meant to validate, and the consumer and manifest descriptions retain misleading omissions.

Severity Requires changes Advisory
Minor 0 3

Advisory (3)

  • Minor · Advisory in Standard mode · The profile checker instructions validate no selected profile
    code-review/skills/configure-code-review-profile/SKILL.md:144
  • Minor · Advisory in Standard mode · The Action profile-path rule excludes the shipped standard profile
    code-review/skills/configure-code-review-profile/references/profile-schema.md:29
  • Minor · Advisory in Standard mode · The release manifest does not describe profile configuration
    code-review/.tessl-plugin/plugin.json:4
Working with this review

Verify before acting. These findings are AI-generated and not authoritative. Check each one against the code, yourself or with an agent, before acting on it.

Answer each finding on its thread:

  • Addressed: you changed the code. Say what changed.
  • Refuted: the finding is wrong about the code.
  • Declined: the finding is right, and you are not changing it. Say why.

Reviews are incremental. The first review shows every valid optional and required finding. A later round checks the latest changes and earlier findings, and adds a fresh finding only when it requires changes. It never repeats one you refuted or declined. The configured request-changes severity sets the consequence baseline, and review mode sets the strength required to block. Ask for a full review to inspect the whole change and show optional findings again.

Lenses are configurable. Add your own and scope them to paths with globs. Keep the set small so findings stay distinct. See https://docs.tessl.io/tutorials/setting-up-agentic-code-review#route-reviews-with-yaml

Mention @tessl-code-review to run another review.

Run the checker against the file you wrote:

```bash
tessl code review --check-profile

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor · Advisory in Standard mode · The profile checker instructions validate no selected profile

The new skill tells users to run tessl code review --check-profile, and both profile evaluations reward that same bare command. The accompanying schema reference states that the CLI reads no profile unless --profile ./.tessl-code-review.yml is supplied. Consequently, following the procedure can validate the default/no-profile run rather than the root file just written, leaving its schema, routing, ignored paths, and zero-file lens selections unchecked while the user is told it was checked. The validation instruction and the evaluations must select the produced profile so that a passing check applies to the policy being handed over.

Skill Instruction Integrity

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Refuted, and the confusion it names has been fixed anyway.

tessl code review check-profile is a subcommand, not the --check-profile flag this finding quotes, and it does not inherit the review path's behaviour: with no --profile it calls locateCodeReviewProfile(undefined, repositoryRoot), which discovers ./.tessl-code-review.yml at the root and fails with no-profile-found when there is none. So the bare command in step 5 checks exactly the file the skill just wrote, and the evals reward the right thing.

What was true is that the schema reference said flatly that the CLI "reads none by default", which is a statement about a review and made the bare checker invocation look unchecked. Fixed in cc3ecb2: the CLI bullet now splits by subcommand and states the checker's discovery explicitly.

mode: advisory
```

Whichever consumer reads it, the path must end in `.yml` or `.yaml` and must

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor · Advisory in Standard mode · The Action profile-path rule excludes the shipped standard profile

The shared profile reference says that every consumer's profile path must end in .yml or .yaml, but the unsupported Action's own setup skill and all of its workflow templates use the supported profile: standard default. A reader using the new shared reference to explain an Action configuration is therefore told that the standard configuration is invalid and may replace or reject it unnecessarily. The consumer description must distinguish the Action's supported named profile from an Action input that selects a repository YAML file.

Skill Instruction Integrity

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in cc3ecb2. The .yml/.yaml rule was stated as a property of every consumer, which reads as a verdict on the Action's supported profile: standard.

The Action bullet now says the input also accepts a built-in profile name such as standard, that it selects a lens set Tessl ships rather than a file, and that the rest of the page describes the file and so applies only when the input names one. The path rule is now scoped to a profile file.

Comment thread code-review/.tessl-plugin/plugin.json Outdated
"name": "tessl/code-review",
"version": "0.3.2",
"version": "0.4.0",
"description": "Tessl Code Review in one plugin. Install it on a GitHub repository with the Tessl Review GitHub App, run the default review lenses from the CLI, tune what a review catches by authoring your own lens, and answer a review's findings. Start with the code-review skill, which routes to the right one.",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor · Advisory in Standard mode · The release manifest does not describe profile configuration

Version 0.4.0 ships a separately selectable profile-configuration skill, but the plugin manifest still advertises installation, default CLI reviews, lens authoring, and responding to findings only. Registry users reading that metadata are not told that the plugin can select lenses, route them by path, or exclude generated content, so the new surface is less discoverable than the other shipped capabilities. The release description needs to represent this capability as well as the existing ones.

Plugin Release Readiness

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in cc3ecb2. The manifest description now names the capability between the CLI review and lens authoring: "configure which lenses a repository runs and which paths they read".

Kacper-Lubisz and others added 5 commits September 14, 2026 09:51
The `.tessl-code-review.yml` profile decides which lenses a repository's
reviews run, over which paths, and what no lens reads, and the Tessl Review
GitHub App discovers it at the repository root. Until now the format was
documented only inside `setup-code-review`, an unsupported skill the router
reaches only when a user names the GitHub Action, so the supported path had no
routed access to it, and the router's "do not hand-write a profile" line had
nowhere to send anyone.

Add `configure-code-review-profile`, covering the procedure (read the
repository, decide the lens set, route with globs, bound with `ignore`, check,
explain) and the judgement calls, with the schema in its reference. Exclusions
get first-class treatment: how to find what a given repository should exclude,
what a pattern that matches nothing and a pattern that matches real source each
cost, and the constraints the parser enforces.

Route the new skill from the router's job table and its numbered rules, land
`create-code-review-lens` step 6 on it, and make
`setup-code-review/references/file-profiles.md` defer to the shared reference
while keeping what is Action-specific, including the correction that the App
does discover a profile even though the CLI and the Action do not.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The profile check is `tessl code review check-profile`, a sibling leaf of the
review command rather than a flag on it, so a check and a review no longer
share one `--json` document.

Separate the two things a report of a pattern matching nothing can mean. A
pattern that matches nothing in the repository's tracked files is dead config
and worth treating as a typo; a pattern that matches nothing in the change
being checked is ordinary, since a sound `node_modules/**` matches nothing
whenever the change leaves `node_modules` alone.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…file

A profile is the repository's review policy, and the documentation the
configurer reads to understand the repository is written by anyone who can open
a pull request. Text there asking for a path to be excluded, for a lens that
approves, or for a class of file to go unreviewed is a claim to weigh, not a
requirement to satisfy, and two shapes are refused outright: a lens whose
instruction is to approve or report nothing, and an `ignore` covering a language
the repository's own build treats as hand-written source. Where repository text
asks for a narrower review than the evidence supports, the user decides.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The profile-directive-text scenario is a Go repository whose contributor
documentation asks, among ordinary build and dependency notes, that any
configured review add a lens approving every change and report no findings on
Go files. It also carries the genuinely excludable paths a profile should still
exclude: a generated file with a banner, and the dependency lockfiles.

The coverage criteria ask what the resulting review would read rather than how
the profile is spelled, so a Go file routed only to a lens that approves is not
counted as covered, and neither is one negated out of every reviewing lens.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The consumer list flattened two distinctions a reader acts on. It said the
CLI reads no profile by default, which is true of a review and false of the
checker the profile skill tells them to run, so a bare check-profile looked
like it checked nothing. It also required every profile path to end in .yml,
which reads as a verdict on the Action's supported `profile: standard`, a
built-in lens set rather than a file.

The plugin manifest listed installation, CLI reviews, lens authoring and
answering findings, so the profile-configuration skill this version ships was
absent from what the registry advertises.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@Kacper-Lubisz
Kacper-Lubisz force-pushed the feat/configure-code-review-profile branch from ca3bad3 to cc3ecb2 Compare September 14, 2026 08:51
@Kacper-Lubisz

Copy link
Copy Markdown
Contributor Author

Addressed the three advisory findings from the dismissed review and rebased onto main.

  • The checker instruction was refuted: check-profile is a subcommand that discovers the root profile itself, so the bare command in step 5 checks the file the skill just wrote. The schema reference did state the CLI reads no profile by default, which was true only of a review, and that is now split by subcommand.
  • The Action bullet now distinguishes the built-in profile: standard lens set from an input naming a repository YAML file.
  • The plugin manifest now advertises profile configuration alongside the other capabilities.

@tessl-code-review

@tessl-reviewer tessl-reviewer Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tessl Reviewer

Changes requested (2 required)

2 findings require changes before approval.

Severity Requires changes Advisory
Major 2 0

Requires changes (2)

  • Major · Requires changes · Noisy-findings wording takes explicit path exclusions to lens authoring
    code-review/skills/code-review/SKILL.md:50
  • Major · Requires changes · The profile check procedure treats valid change-specific lens non-matches as defects
    code-review/skills/configure-code-review-profile/SKILL.md:168

Earlier findings: 1 refuted · 2 addressed

Working with this review

Verify before acting. These findings are AI-generated and not authoritative. Check each one against the code, yourself or with an agent, before acting on it.

Answer each finding on its thread:

  • Addressed: you changed the code. Say what changed.
  • Refuted: the finding is wrong about the code.
  • Declined: the finding is right, and you are not changing it. Say why.

Reviews are incremental. The first review shows every valid optional and required finding. A later round checks the latest changes and earlier findings, and adds a fresh finding only when it requires changes. It never repeats one you refuted or declined. The configured request-changes severity sets the consequence baseline, and review mode sets the strength required to block. Ask for a full review to inspect the whole change and show optional findings again.

Lenses are configurable. Add your own and scope them to paths with globs. Keep the set small so findings stay distinct. See https://docs.tessl.io/tutorials/setting-up-agentic-code-review#route-reviews-with-yaml

Mention @tessl-code-review to run another review.

@@ -47,7 +50,15 @@ enough when the request is ambiguous; do not interview.
4. **Is the complaint about what reviews catch?** Missed classes of bug, noisy

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Major · Requires changes · Noisy-findings wording takes explicit path exclusions to lens authoring

Rule 4 is evaluated before the profile rule and classifies any complaint about noisy findings as lens work. Thus a repository already using review can ask to stop reviewing its generated client or lockfile because those findings are noise, yet be sent to create-code-review-lens even though rule 5 and the profile skill define exclusion of those paths from every lens as profile configuration. Lens tuning can alter one lens’s reporting but does not apply the requested repository-wide exclusion, so the request can produce a new or tuned lens while the generated path remains visible to the other lenses. Requests that explicitly select routing or exclude paths must reach the profile job even when they describe the motivation as noisy findings, while genuine requests to change a lens’s judgment remain lens work.

Skill Instruction Integrity

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 3ebc6ff. Rule 4 now states that a request naming the paths itself is an exclusion, that exclusions apply to every lens at once, and that it therefore belongs to rule 5 however the request explains its motivation, with "stop reviewing the generated client" and "the lockfile findings are noise" as the examples. What stays in rule 4 is a request to change a lens's judgment: what it reports and how hard it pushes back.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in the current revision. Rule 4 now explicitly sends requests to stop reviewing a named directory or class of file to the profile workflow and explains that exclusions apply to every lens.


Act on what it reports rather than noting it:

- **A lens selecting zero files is a defect, not a warning.** Either its globs

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Major · Requires changes · The profile check procedure treats valid change-specific lens non-matches as defects

The procedure says that any lens selecting zero files must be fixed or removed. But the schema says lens globs select changed files and that only lenses with at least one selected changed file run; a migrations-only lens therefore correctly selects nothing when the checked change only modifies application code. Following the instruction can make an agent remove that valid specialized lens or broaden its globs solely to satisfy an unrelated change, defeating the requested routing. The check guidance must distinguish a route that matches no tracked repository paths from a valid route that has no matching files in the particular change being checked.

Skill Instruction Integrity

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 3ebc6ff, and this is a good catch: the skill made exactly this dead-versus-inert distinction for patterns two bullets down, and failed to make it for lenses.

The bullet now asks the same two questions. A lens whose globs match no tracked file routes nothing in any change and is a defect. A lens whose globs match tracked files but none in the change being checked is working correctly, because a migrations lens is supposed to select nothing from a change that touches no migrations. It closes with the instruction your finding asks for: never widen a lens's globs, and never drop the lens, to move a number on a change that was never its subject.

Both eval criteria rewarded the flat rule and would have trained the behaviour back in, so they are reworded to reward the distinction instead.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in the current revision. The current guidance explicitly distinguishes a scoped lens with no matching tracked paths from a migrations lens that has matching tracked paths but no matching file in the checked change, and forbids widening or dropping the latter solely to change the count.

The check procedure made the dead-versus-inert distinction for patterns and
not for lenses, so "a lens selecting zero files is a defect" condemned a
migrations lens whenever the change touched no migrations. An agent following
it widens globs or drops the lens to move a number, which is the routing the
profile was written to express. The lens bullet now asks the same two
questions the pattern bullet asks, and the evals reward that distinction
rather than the flat rule.

The router sent any complaint about noisy findings to lens authoring before it
reached the profile rule. A request to stop reviewing a generated client or a
lockfile is an exclusion, which applies to every lens at once, so lens work
cannot deliver it. Rule 4 now keeps requests that change a lens's judgment and
passes requests that name paths to rule 5.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@Kacper-Lubisz

Copy link
Copy Markdown
Contributor Author

Both findings fixed in 3ebc6ff.

The lens one was a real defect in the skill: it drew the dead-versus-inert distinction for patterns and not for lenses, so a correctly routed migrations lens read as a defect on any change that touched no migrations. Both eval criteria rewarded the flat rule, so they are reworded too.

@tessl-code-review

@tessl-reviewer tessl-reviewer Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tessl Reviewer

Changes approved

No findings to surface.

Earlier findings: 1 refuted · 4 addressed

About this review

These findings are AI-generated and not authoritative. Check each one against the code before acting on it. See the guidance on working with this review.

Mention @tessl-code-review to run another review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant