chore(ci): add GitHub Actions with E2E to catch regressions early and reduce AI-contribution risk

[Nuctori]

Summary

npm test (node --experimental-strip-types --test "app/**/*.test.mjs" ..., engines >=22.19) has 587+ tests, but the repo has no CI and 35 pre-existing failures on a clean main. Failures are not product bugs — they are harness/environment issues that remain invisible because nothing runs the suite automatically. This makes "does my change break tests?" unanswerable — a governance gap that gets riskier as AI-assisted contributions increase.

This issue (1) documents the failures with evidence, (2) proposes a minimal CI that runs the suite on every push/PR, and (3) proposes putting E2E into CI so regressions on session-history paths (#509 / #555) are caught quickly in subsequent iterations, reducing the risk of AI-generated regressions.

Evidence

On a clean upstream checkout (2a6e537 Release v0.8.9, no local changes):

npm test
ℹ tests 587
ℹ pass  549
ℹ fail   35

The 35 failures come from two root causes, neither touches application logic:

1. Component/hook tests fail with useI18n must be used inside I18nProvider

Affected: components/MermaidBlock.test.mjs, components/CodeBlock*, components/MessageView.test.mjs, components/AppShell.file-viewer-state.test.mjs, components/ChatInput.test.mjs, components/ExtensionStatusBar.test.mjs, components/ExtensionWidgets.test.mjs, components/TurnWrittenFiles.test.mjs, hooks/useAgentSession.test.mjs, …

Tests render with renderToStaticMarkup(<I18nProvider><Component/></I18nProvider>) and the provider is wrapped (e.g. MermaidBlock.test.mjs:21-25). The error is thrown because useContext(I18nContext) returns null — the provider and the consumer resolve two different I18nContext instances.

Root cause: under node --experimental-strip-types, createJiti({ moduleCache: false }) re-instantiates modules, so react / I18nContext loaded by the test entry and by the imported .tsx are separate module instances. This is a harness incompatibility, not a code defect.

2. lib/project-command-env.test.mjs fails on Windows PATH separator

direct bash updates the platform PATH key expects PATH: '...\bin;/usr/bin' (Windows ;) but gets : (POSIX). Environment-specific assertion.

Both classes fail identically on the unmodified baseline, so they are pre-existing and unrelated to any feature PR.

Update: main@4ec3c5c has landed #588. The same fix (1e5594b on fix/test-harness-react-instance, now merged as bf0da02) takes the suite from 35 → 9 failures; the remaining 9 are platform-conditional / brittle assertions and non-blocking.

What we propose

A. Add CI (GitHub Actions) — minimal, green first

A minimal workflow that runs the existing suite so regressions are caught:

# .github/workflows/ci.yml
name: ci
on:
  push:
    branches: [main]
  pull_request:
jobs:
  lint:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: 22, cache: npm }
      - run: npm ci
      - run: npm run lint
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: 22, cache: npm }
      - run: npm ci
      - run: npm test
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: 22, cache: npm }
      - run: npm ci
      - run: npm run build

node-version: 22 aligns with engines >=22.19. setup-node@v4 with cache: npm as proposed.

B. Fix the component-test harness (so the 35 become green)

Single shared React instance — e.g. one createJiti instance without moduleCache: false, or run component tests through tsx/vitest instead. #588 already proves 35 → 9 with this approach, making npm test trustworthy.

C. Put E2E into CI for session history (#509 / #555)

509 (deep linear chains overflowing the stack) and #555 (full-history transfer on open) are exactly the kind of bug unit tests miss and a real browser catches. We added lib/session-reader.pagination.test.mjs + app/api/sessions/{detail,context}-route.test.mjs covering the data/route boundary (tail window, ?before dedupe, 1000 cap, string-content guard). The remaining UI path — sentinel pagination in ChatWindow — needs a browser.

Prototype e2e/run.mjs already runs locally: writes a 5000-message linear JSONL under ~/.pi/agent/sessions/e2e-ci/, opens ?session= in Chromium, and asserts the tail window (tail=50), no full-forest render, load earlier sentinel, and no pageError/consoleError (E2E_BASE configurable, rmSync cleanup).

CI shape (example, to be refined in a follow-up PR):

  e2e:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: 22, cache: npm }
      - run: npm ci
      - run: npx playwright install chromium
      - run: npm run dev -- -H 127.0.0.1 -p 3100 &
      - run: timeout 120 bash -c 'until curl -sf http://127.0.0.1:3100; do sleep 2; done'
      - run: node e2e/run.mjs

Note: /api/sessions cold scan over ~1700 sessions can take ~110s (dev + Turbopack compile), so the timeout 120 wait is intentional; a follow-up can evaluate build+start for stability.

Why E2E in CI — especially for AI contributions

• Catch regressions fast in later iterations: every push/PR runs lint/test/build + E2E automatically. Regressions on the tail window / pagination are blocked before merge, not discovered by manual runs.
• Reduce AI-contribution risk: AI-generated bulk edits and refactors are most likely to introduce "looks correct, crashes at runtime" regressions — Feature request: conversation management & chat archive (对话管理和对话存档) #509/Performance: page session detail history instead of returning the full active branch #555 are textbook examples. CI+E2E gives every AI PR a deterministic gate: no green, no merge, reducing blind spots in human review.
• Low cost: reuse the existing script-style e2e/run.mjs (no playwright.config.ts needed), add playwright as a single devDependency. ~1–2 min of CI time for continuous coverage of the core session path.

Impact on the current fix PR

Our fix/session-history-tail branch adds 16 passing tests and changes the failure count by exactly 0 (587→603 tests, 549→565 pass, 35 fail unchanged), confirming no regression from the pagination work — but we can only claim that because we ran the suite by hand. CI+E2E would make that automatic for every contributor, including AI-assisted ones.

Ask

1. Add .github/workflows/ci.yml (lint + npm test + build, node 22).
2. Merge the test harness fix so component tests share one React instance (turn the 35 red → green; test: fix component-test harness dual React instance (35 -> 9 failures) #588 already on main).
3. Accept e2e/run.mjs + an e2e job into CI? Happy to contribute .github/workflows/ci.yml + e2e/run.mjs + playwright wiring as a follow-up example PR if you want E2E in-repo — example implementation to follow this issue.

---

Follow-up: example CI+E2E implementation will be sent as a separate PR after this issue.

[agegr]

#722