docs(skills): add pine-visual-test-pr visual regression skill - #788
Conversation
Self-iterating Playwright-MCP visual test harness for Pine PRs, modeled on kajabi-products' kp-visual-test-pr but re-pointed at Pine's Storybook. Builds Stencil, serves Storybook, maps changed pds-* components to their stories via /index.json, and captures screenshots across light/dark themes, viewports, and interactive states. Self-evaluates each shot against embedded rules (dark-mode/theme, focus-ring, console errors, token discipline), retries harness failures, records genuine visual/a11y regressions as findings, and posts a markdown report as a PR comment. Optional before/after baseline diff for modified components. Complements Chromatic (local, fast, pre-CI); does not replace it.
✅ Deploy Preview for pine-design-system ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
Dry run against the pds-tabs PR surfaced three fixes:
- Rule 4: whitelist harness noise (favicon/static 404s, Vite HMR); only
component-origin errors (pine-core/*.js stack) count as findings.
- Screenshots: require an absolute output path into a per-run dir — the
Playwright MCP browser cwd differs from the shell cwd, so relative
filenames land outside the repo.
- Story mapping: match story titles by exact/nested-path equality, not
substring (includes('tab') wrongly matched Table and Sortable).
Running baseline mode against #785 surfaced two more fixes: - Console read must be per-navigation, not all:true — session history floods with stale ERR_CONNECTION_REFUSED / NoStoryMatchError after any Storybook restart, none of which are findings. - Phase 5b: build.stencil dirties tracked generated files (components.d.ts, react createComponent.tsx), aborting the branch switch; reset them before each checkout, restart Storybook on the base (stale index → NoStoryMatchError), and compare only stories shared with base.
pixelflips
left a comment
There was a problem hiding this comment.
Code review — 9 inline findings, see comments on the relevant lines.
Resolves Cursor + pixelflips findings on pine-visual-test-pr: - Chromatic hedged everywhere — the CI job isn't merged (only the local Storybook panel is); this skill is currently Pine's only pixel-level check. - Phase 1 threads an explicit <number> through gh pr diff/view (bare calls resolve against the current branch, mis-scoping an explicitly-passed PR). - Dropped the half-wired RTL axis (planned but never looped/graded). - All Phase 5/6 capture commands use the absolute <REPORT_DIR> path (the L201-213 rule was only in prose, not the example commands). - Console check is per-navigation AND per-theme (a single end-of-story read missed the earlier theme); never all:true. - Phase 5b: createComponent.tsx carries an authored double-registration guard that build.stencil strips — restore via git checkout -- after the final rebuild; added an explicit Storybook stop (lsof kill) before restart; Phase 2 'reuse the server' now carves out the post-branch-switch case. - Phase 6: write the report file before cat-ing it; post via --body-file with --repo (not double-quoted $(cat)); added a once-per-PR dedup guard and the 'Posted by pine-visual-test-pr' marker.
|
Thanks for the thorough review — all findings addressed in e83a33d. pixelflips (9):
Cursor:
(This reply posted via |
Cursor Bugbot re-review flagged two issues introduced by the prior fix: - Cross-branch capture (High): threading the PR number scoped the right files but Phase 2 still builds/screenshots the checked-out tree. Added a Phase 1 branch-match guard that stops on mismatch (no auto-switch). - Stash ordering (Medium): Phase 5b stashed WIP before resetting the generated files, so the stash could snapshot the vendor-stripped createComponent.tsx and reintroduce it on pop. Reordered: reset generated files first, then stash only genuine WIP.
|
Both re-review findings fixed in the latest commit:
|
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit e857714. Configure here.
Re-review flagged that restoring the step-3 WIP stash and then requiring a clean git status contradict each other. The clean check now applies to the skill's own artifacts (the two generated files match committed) and runs before restoring the stash; the stash restore comes last and legitimately re-dirties the tree with the user's own WIP.
|
Fixed in the latest commit. The clean check now applies to skill-induced changes only (the two generated files matching committed), and runs before restoring the step-3 stash. Restoring the user's WIP is done last and legitimately re-dirties the tree — that's expected and no longer treated as a failed clean check. |
pixelflips
left a comment
There was a problem hiding this comment.
Thanks for addressing all the feedback. LGTM! 👍🏼

What
Adds a
pine-visual-test-prClaude Code skill at.claude/skills/pine-visual-test-pr/SKILL.md— a self-iterating, Playwright-MCP visual-test harness for Pine PRs.It's modeled on kajabi-products'
kp-visual-test-pr, re-pointed at Pine's actual visual surface (Storybook) instead of Rails worktree servers.How it works
ghauth, root deps,libs/corescripts, Playwright MCP reachable.libs/core/src/components/pds-*dirs.build.stencil→start.storybook(:6006) → enumerate every story from/index.json(the Pine analog ofworktree-server list; never guesses story IDs).title:, then builds a shot matrix.iframe.html?id=…&globals=theme:…across light/dark themes, viewports, and interactive states (hover/focus/open/disabled/error); self-evaluates each against embedded rules; retries harness failures (max 3), records genuine theme/focus/console/token problems as findings.Optional before/after baseline mode (checkout base → rebuild Stencil → re-shoot the same matrix) labels each delta intended vs regression, gated behind a clean-tree confirmation.
Design notes
@chromatic-com/storybook) remains the authoritative pixel-diff gate in CI.Scope
Docs/tooling only — one new markdown skill file. No component, build, or runtime code touched.
Note
Low Risk
Single new markdown skill file with no changes to components, builds, or production runtime.
Overview
Adds a new Claude Code skill at
.claude/skills/pine-visual-test-pr/SKILL.mdthat documents an end-to-end Playwright MCP visual regression workflow for Pine PRs (adapted from kajabi-products’kp-visual-test-pr, but aimed at Storybook instead of app servers).The skill defines a seven-phase runbook: environment preflight, PR/component scoping from
gh pr diff, Stencil build + Storybook on:6006with story IDs from/index.json, a capture matrix (light/dark, viewports, hover/focus/open states), optional base-branch baseline screenshots (with explicit git/Storybook cleanup aroundcreateComponent.tsxand port 6006), rule-based evaluation with retries vs findings, and a dedupedgh pr commentreport under.claude/visual-test-reports/.It positions the harness as Pine’s current primary local pixel check until Chromatic CI merges, and links it to
pine-design-reviewandpine-run-gauntlet. Docs/tooling only — no component, build, or runtime code changes.Reviewed by Cursor Bugbot for commit ddf7b5d. Bugbot is set up for automated code reviews on this repo. Configure here.