Land the e2e harness, brand, and capability-aware editors on main (0.5.0 prep)

.dockerignore

@@ -0,0 +1,16 @@
+# Everything the image regenerates from source, plus host cruft that must never
+# cross into a Linux image. node_modules is the critical one: the host's is
+# built for the host platform, and copying it in would shadow the `npm ci` the
+# Dockerfile runs to build the right one.
+node_modules
+**/node_modules
+dist
+web/styled-system
+.git
+.github
+.playwright-mcp
+*.log
+.DS_Store
+bun.lock
+bun.lockb
+profiles.png

.github/workflows/ci.yml

@@ -73,6 +73,16 @@ jobs:
       - name: Test
         run: npm test
 
+      # The end-to-end harness (Tier A): run the REAL bin/swisscode.js against a
+      # recorder and assert on the process it actually launched — the execve
+      # handoff, the argv it forwarded, and the stale env it removed at the OS
+      # level. Hermetic (no network, no credential, no real CLI), so it belongs
+      # on every PR. Runs in every matrix cell, which is how both supported Node
+      # versions exercise it. `npm test` already built dist/, but the script
+      # rebuilds to stay self-contained if this step is ever reordered.
+      - name: End-to-end (hermetic)
+        run: npm run test:e2e
+
       # What users actually download. Shipping the web UI means everyone gets a
       # React bundle they may never open; the budget keeps that a decision that
       # is re-made on every PR rather than one that drifts. One cell is enough —

.github/workflows/e2e-real.yml

@@ -0,0 +1,35 @@
+name: E2E (real CLIs)
+
+# Tier B: swisscode against the actual @anthropic-ai/claude-code, @kilocode/cli
+# and opencode-ai. Manual dispatch ONLY — building on every PR would tie the
+# merge gate to three third-party packages being installable, and the value
+# (catching an upstream flag rename) does not need per-PR frequency. Run it when
+# verifying against a new CLI release, or on a whim.
+#
+# Non-billable: the suite forwards only `--version`, which every CLI answers
+# before auth. No secret is needed and none is referenced.
+
+on:
+  workflow_dispatch:
+
+permissions:
+  contents: read
+
+jobs:
+  real:
+    name: real CLI handoff
+    runs-on: ubuntu-latest
+    steps:
+      # SHA-pinned, matching ci.yml and publish.yml, so Dependabot bumps all
+      # three in one reviewable PR.
+      - uses: actions/checkout@fbc6f3992d24b796d5a048ff273f7fcc4a7b6c09 # v5
+
+      # The whole point is the image: it installs the three real CLIs at pinned
+      # versions and runs the handoff suite as its default command. Building and
+      # running it here is a faithful copy of the documented local command, so a
+      # green run means the documented command works.
+      - name: Build the real-CLI image
+        run: docker build -f test/e2e/Dockerfile -t swisscode-e2e .
+
+      - name: Run the real-CLI handoff suite
+        run: docker run --rm swisscode-e2e

docs/TESTING.md

@@ -13,6 +13,9 @@ npm run typecheck                     # tsc --noEmit over src/ and test/
 node --test "test/core/**/*.test.ts"  # ~0.3s, no build needed
 node --test test/golden.test.ts       # one file
 node --test --test-name-pattern "binding walk" "test/**/*.test.ts"
+
+npm run test:e2e                      # Tier A: the REAL binary vs a recorder
+npm run test:e2e:real                 # Tier B: vs the REAL CLIs (Docker; see below)
 ```
 
 **Most suites need no build.** Node executes `.ts` directly, and `dist/` is
@@ -38,6 +41,8 @@ affecting anything else.
 | `test/registry.test.ts` | Invariant | Descriptor rules every provider must satisfy. |
 | `test/ui.test.ts`, `picker`, `profiles-ui` | End-to-end | The Ink wizard, driven with synthetic keystrokes. |
 | `test/config-commands.test.ts`, `launch-overrides.test.ts` | Integration | Subcommand dispatch and the override pipeline through a composition root. |
+| `test/e2e/*.e2e.ts` | End-to-end (hermetic) | The **real** `bin/swisscode.js`, launched against a recorder. See below. |
+| `test/e2e/*.real.ts` | End-to-end (real CLIs) | The real binary vs the real coding CLIs, in Docker. See below. |
 | `test/support/fixtures.ts` | Helper | Typed constructors for deliberately incomplete domain objects. |
 
 ### The four that are unusual
@@ -154,22 +159,70 @@ a temp directory; never let a test touch the real `~/.config/swisscode`.
   property: `an unknown positional falls through, an unknown flag does not`,
   `no launch inherits a stale ANTHROPIC_API_KEY it did not ask for`.
 
+## The end-to-end harness
+
+Everything above asserts on the launch **plan** — the in-memory object
+`planLaunch` / `buildEnvPlan` produce. Nothing there runs the real binary and
+observes what it launches. The e2e harness closes that gap, in two tiers, both
+under `test/e2e/`.
+
+**Tier A — hermetic (`*.e2e.ts`, `npm run test:e2e`, every PR).** The three
+`SWISSCODE_*_BIN` overrides are pointed at `recorder.mjs` — a tiny fake agent
+that writes down its own argv, env and cwd, then exits. The harness seeds a temp
+config, runs the real `bin/swisscode.js`, and reads that capture. So the whole
+pipeline runs for real — argv parsing, config load, resolution, env lowering,
+the `execve` handoff — with no network, no credential and full determinism.
+
+The assertion a plan test cannot make: for a third-party provider, the stale
+`ANTHROPIC_API_KEY` from the polluted ambient env is **absent from the launched
+child**. Present in `plan.unset` is not the same as gone from the process; only a
+real launch shows the difference. This is `test/golden.test.ts`'s claim, verified
+one layer further out.
+
+The recorder is **copied** into a temp dir, never symlinked or run from inside
+the repo, because swisscode's recursion guard resolves candidates with
+`realpathSync` and rejects anything under its own install directory — a symlink
+resolves back into the repo and is (correctly) refused. A copy under `/tmp` is
+accepted exactly as a real agent would be. The recorder exiting immediately is
+also why this is not the "do not launch for real" hazard `CLAUDE.md` warns
+about: it is the safe launch that warning leaves room for.
+
+**Tier B — real CLIs (`*.real.ts`, `npm run test:e2e:real`, manual only).** A
+Docker image (`test/e2e/Dockerfile`) installs the three actual CLIs at pinned
+versions and runs swisscode against them. It forwards only `--version`, which
+every CLI answers before any auth — non-billable, no secret, no cost. It cannot
+be deterministic (it depends on upstream), so it never gates a PR; it catches an
+upstream flag rename or a rejected env var when someone runs it:
+
+```sh
+docker build -f test/e2e/Dockerfile -t swisscode-e2e .
+docker run --rm swisscode-e2e
+```
+
+The `.real.ts` extension is deliberate: `test:e2e` globs `*.e2e.ts` and
+`npm test` globs `*.test.ts`, so the three sets are disjoint and a PR that has
+not installed the real CLIs stays green.
+
 ## CI
 
-`.github/workflows/ci.yml` runs `npm test` on every push to `main` and every pull
-request, across Node 22 and 24 on ubuntu and macOS.
+`.github/workflows/ci.yml` runs `npm test` and then `npm run test:e2e` on every
+push to `main` and every pull request, across Node 22 and 24 on ubuntu and macOS.
+`.github/workflows/e2e-real.yml` runs Tier B, on manual dispatch only.
 
 Node 22 is not there for symmetry — it is the floor `engines` promises, and the
 version where the toolchain assumptions have to hold: type stripping is what lets
 the suite run straight from `.ts`, and `tsc`'s emit is equivalent to that stripped
 program only because `target: es2023` downlevels nothing there.
 
-One thing it does **not** buy, despite being the version where `process.execve`
-does not exist: extra coverage of the spawn fallback. `spawnFallback` is called
-directly with an injected `SignalHost`, so it is exercised on every version, and
-nothing asserts on the dispatch inside `createNodeProcess().replace()` — taking
-the `execve` branch would replace the test process. The runtime difference is
-real; the test difference is not.
+The `execve` dispatch inside `createNodeProcess().replace()` is now covered — by
+the e2e harness, which runs the real binary in a child process, lets it `execve`
+into the recorder, and reads back what landed. Taking that branch replaces the
+*child*, not the test runner, which is what makes it observable. (Even Node 22.23
+backported `process.execve`, so both supported versions take the `execve` branch;
+the `spawn` fallback fires only where execve is truly absent — Windows and
+Node < 23.11 — and stays unit-tested via `spawnFallback` with an injected
+`SignalHost`. Running the e2e under both Node versions is defence in depth, not
+two different dispatches.)
 
 Windows is deliberately not in the matrix. `execve` does not exist there, the
 POSIX mode bits the config store asserts are meaningless, and nobody has verified

package.json

@@ -22,6 +22,8 @@
     "dev": "node build.js && tsc -p tsconfig.build.json --watch --preserveWatchOutput",
     "typecheck": "tsc --noEmit",
     "test": "tsc --noEmit && npm run typecheck:web && node build.js && node --test \"test/**/*.test.ts\"",
+    "test:e2e": "node build.js && node --test \"test/e2e/**/*.e2e.ts\"",
+    "test:e2e:real": "node --test \"test/e2e/**/*.real.ts\"",
     "prepare": "node build.js",
     "prepublishOnly": "node build.js",
     "size": "node scripts/size-budget.js",

src/adapters/agents/kilo/index.ts

@@ -19,6 +19,7 @@ import {
   ambientUnset,
   anthropicOptions,
   collapsedTierWarning,
+  compatIgnoredWarning,
   sessionUnavailableWarning,
   extendedContextWarning,
   modelRef,
@@ -57,7 +58,7 @@ export const kilo = {
   },
   binary,
   translate(input: TranslateInput): Translation {
-    const { intent, passthrough } = input
+    const { intent, passthrough, profile } = input
     const primary = intent.models.opus
 
     const config: Record<string, unknown> = {
@@ -82,6 +83,8 @@ export const kilo = {
     if (collapse) warnings.push(collapse)
     const ext = extendedContextWarning(intent, primary, 'Kilo')
     if (ext) warnings.push(ext)
+    const compat = compatIgnoredWarning(profile.compat, 'Kilo')
+    if (compat) warnings.push(compat)
 
     return { plan: { set, unset: ambientUnset(intent) }, args: [...passthrough], warnings }
   },

src/adapters/agents/opencode/index.ts

@@ -16,6 +16,7 @@ import {
   ambientUnset,
   anthropicOptions,
   collapsedTierWarning,
+  compatIgnoredWarning,
   sessionUnavailableWarning,
   extendedContextWarning,
   modelRef,
@@ -56,7 +57,7 @@ export const opencode = {
   },
   binary,
   translate(input: TranslateInput): Translation {
-    const { intent, passthrough } = input
+    const { intent, passthrough, profile } = input
     const primary = intent.models.opus
     const small = intent.models.haiku
 
@@ -87,6 +88,8 @@ export const opencode = {
     if (collapse) warnings.push(collapse)
     const ext = extendedContextWarning(intent, primary, 'OpenCode')
     if (ext) warnings.push(ext)
+    const compat = compatIgnoredWarning(profile.compat, 'OpenCode')
+    if (compat) warnings.push(compat)
 
     return { plan: { set, unset: ambientUnset(intent) }, args, warnings }
   },

src/adapters/agents/shared.ts

@@ -103,6 +103,35 @@ export function collapsedTierWarning(
   }
 }
 
+/**
+ * Warn when a profile carries gateway compatibility flags that this agent does
+ * not consume.
+ *
+ * The compat mechanism is Claude-Code-only (capabilities.compatFlags). A flag
+ * toggled on for a Kilo or OpenCode setup changes nothing and, until this
+ * existed, said so NOWHERE — not at edit, not at launch, not in the doctor.
+ * That is the one capability mismatch with no net, and the
+ * capability-gap-never-silently-dropped rule the tier and session warnings
+ * already follow says it should have one. The flags are kept, not stripped, so
+ * the setup still works when it runs Claude Code again.
+ */
+export function compatIgnoredWarning(
+  compat: Record<string, boolean | undefined> | undefined,
+  agentLabel: string,
+): EnvWarning | null {
+  const active = Object.entries(compat ?? {})
+    .filter(([, on]) => on)
+    .map(([id]) => id)
+  if (active.length === 0) return null
+  return {
+    severity: 'medium',
+    code: 'compat-ignored',
+    message:
+      `${agentLabel} does not use gateway compatibility flags; these are set but ignored: ` +
+      `${active.join(', ')}.`,
+  }
+}
+
 /**
  * Warn when a 1M-capable provider is reached without Claude Code's `[1m]`
  * signal, which only Claude Code sends. The model still runs; its window is

test/adapters/agents/kilo.test.ts

@@ -68,3 +68,35 @@ test('capabilities describe a single model slot', () => {
   assert.equal(kilo.binary.name, 'kilo')
   assert.equal(kilo.binary.overrideEnv, 'SWISSCODE_KILO_BIN')
 })
+
+test('a gateway compat flag on a Kilo profile is flagged as ignored — the safety net', () => {
+  // The UI now hides the compat section for Kilo, but a hand-edited or
+  // agent-switched config can still carry a flag. Compat is Claude-Code-only,
+  // so Kilo warns rather than silently doing nothing — the one mismatch that
+  // previously had no net at edit, launch, or doctor.
+  const input: TranslateInput = {
+    intent: intent(),
+    profile: makeProfile({ provider: 'zai', compat: { disableAdaptiveThinking: true } }),
+    provider: null,
+    passthrough: [],
+    ambient: {},
+  }
+  const w = kilo.translate(input).warnings.find((x) => x.code === 'compat-ignored')
+  assert.ok(w, 'Kilo should warn that a compat flag is ignored')
+  assert.match(w.message, /disableAdaptiveThinking/)
+  assert.match(w.message, /Kilo/)
+})
+
+test('a Kilo profile with no compat flags emits no compat-ignored warning', () => {
+  const input: TranslateInput = {
+    intent: intent(),
+    profile: makeProfile({ provider: 'zai' }),
+    provider: null,
+    passthrough: [],
+    ambient: {},
+  }
+  assert.equal(
+    kilo.translate(input).warnings.find((x) => x.code === 'compat-ignored'),
+    undefined,
+  )
+})

test/e2e/Dockerfile

@@ -0,0 +1,54 @@
+# Tier B: swisscode against the three REAL coding CLIs.
+#
+# The hermetic e2e suite (test/e2e/*.e2e.ts) proves swisscode BUILDS the right
+# launch against a recorder. It cannot prove the launch still works after an
+# upstream release — a renamed flag, a rejected env var, a binary that moved.
+# This image installs the actual CLIs and runs `test:e2e:real`, which forwards
+# only `--version` (non-billable: no key, no inference, no cost) and checks that
+# swisscode resolves each one and hands off cleanly.
+#
+# Manual-dispatch only (see .github/workflows/e2e-real.yml). Building on every
+# PR would couple the merge gate to three third-party packages' availability.
+#
+# Build and run from the repo root:
+#   docker build -f test/e2e/Dockerfile -t swisscode-e2e .
+#   docker run --rm swisscode-e2e
+
+FROM node:22-slim
+
+WORKDIR /app
+
+# Install swisscode's own dependencies first, in their own layer, so a source
+# edit does not re-run `npm ci`. `web` is a workspace, so its manifest has to be
+# present before the install resolves.
+#
+# `--ignore-scripts` is REQUIRED here, not an optimisation: the repo's `prepare`
+# lifecycle is `node build.js`, and at this layer the source it needs has not
+# been copied yet. Skipping lifecycle scripts is safe — swisscode has no native
+# dependencies, and esbuild/vite ship their platform binaries via
+# optionalDependencies, which npm resolves without a script. The real build runs
+# explicitly below, once the source is present.
+COPY package.json package-lock.json ./
+COPY web/package.json ./web/package.json
+RUN npm ci --ignore-scripts
+
+# The rest of the source, then the full build — dist/cli.js is what bin/swisscode.js
+# runs, and the e2e harness refuses to start without it.
+COPY . .
+RUN node build.js
+
+# THE POINT OF THIS IMAGE: the three real CLIs, at PINNED versions. Pinning makes
+# an upstream break a deliberate, reviewable bump in this file rather than a
+# nightly mystery — the same discipline the repo already applies to its GitHub
+# Actions (SHA-pinned) and its own lockfile. Bump these when verifying against a
+# newer release on purpose.
+RUN npm install -g \
+      @anthropic-ai/claude-code@2.1.218 \
+      @kilocode/cli@7.4.15 \
+      opencode-ai@1.18.4
+
+# Prove they are all resolvable before the suite runs, so a failed global install
+# is a clear build-time error rather than a confusing skip at test time.
+RUN claude --version && kilo --version && opencode --version
+
+CMD ["npm", "run", "test:e2e:real"]