feat(cli): add migration 0004 for the engine-partitioned .taskless layout

.changeset/engine-partitioned-layout.md

@@ -0,0 +1,9 @@
+---
+"@taskless/cli": minor
+---
+
+Partition `.taskless/` by rule engine. Migration `0004` moves ast-grep rules to `sg/rules/` and `sg/rule-tests/`, the runtime tree to `runtime/rules/` and `runtime/rule-tests/`, and scaffolds an inert `vale/`. Files move byte-for-byte, so runtime rule signatures survive.
+
+The directory a rule sits in now **is** its engine: dispatch reads the path and never parses a rule file to decide who owns it. `check` runs ast-grep against the committed `.taskless/sg/sgconfig.yml` instead of generating an ephemeral config each run.
+
+Existing projects keep working without action. The pre-`0004` `.taskless/rules/` still runs as ast-grep, and a delivered rule that names no engine is still treated as ast-grep — a rule engine this CLI does not recognize is rejected rather than guessed at. A migration that would have to merge a file into an engine directory now refuses up front with `SCAFFOLD_CONFLICT` rather than failing part-way.

openspec/changes/partition-rules-by-engine/.openspec.yaml

@@ -0,0 +1,2 @@
+schema: spec-driven
+created: 2026-07-28

openspec/changes/partition-rules-by-engine/design.md

@@ -0,0 +1,91 @@
+## Context
+
+Taskless rules run today through one engine, ast-grep: `commands/check.ts` writes an ephemeral `sgconfig.yml` (`ruleDirs: [rules]`) and runs `sg scan --config … --json=stream`; rules live as bare ast-grep YAML at `.taskless/rules/*.yml`. A separate **runtime** tier (`runtime-rules/<name>/`: capture YAML + signed `check.ts`) runs only after an authenticated server reconcile. A `.taskless/` migration system already exists (`filesystem/migrate.ts` → `runMigrations`, keyed on `taskless.json`'s `version`, invoked via `ensureTasklessDirectory`).
+
+That shape has room for exactly one engine. Every co-located alternative explored earlier forced either an in-file marker saying which engine owns a rule, or a generated config materialized at check time. Partitioning by directory removes the question: the path _is_ the answer.
+
+Two facts, verified against `sg 0.41.0` and `vale 3.15.1`, make committed per-engine configs viable:
+
+- **Each tool's native config already expresses everything needed.** ast-grep scopes a rule with its own `files`/`ignores` and carries extra data in `metadata`. Vale scopes via `.vale.ini` matchers and silently ignores unknown keys. So Taskless needs **no sidecar or metadata layer** of its own.
+- **Both read rules from disk via `--config` with no size ceiling** — where `--inline-rules` (`ARG_MAX`) and `sg --rule` (one file) cannot scale to hundreds of rules.
+
+## Goals / Non-Goals
+
+**Goals:** partition rules by engine directory; make each engine's committed native config the source of truth; remove config generation from the check path; ride the existing migration ladder; keep every observable behavior identical.
+
+**Non-Goals:** adding the Vale engine (it needs this layout first) and the engine-selection knowledge topic. Also excluded: changing runtime _execution_ — the harness, reconcile, and signing are untouched, only the directory moves; and authoring the committed configs, which belongs to rule generation.
+
+## Directory layout (target)
+
+```
+.taskless/
+  sg/       sgconfig.yml            rules/<rule>.yml            rule-tests/<name>.yml
+  vale/     .vale.ini               rules/<rule>.yml            rule-tests/<name>.yml
+  runtime/  rules/<rule>/{<capture>.yml, check.ts}   rule-tests/<rule>/<fixture>/
+  taskless.json
+```
+
+All three land here. `vale/` is scaffolded but inert — no engine reads it until the Vale change. The `runtime/` realignment is safe now precisely because nothing uses runtime rules yet, and the move is content-preserving so hashes are unaffected. Every scaffolded directory carries a `.gitkeep` so the structure tracks reliably when empty.
+
+## Decisions
+
+### D1 — Engines partition by top-level directory
+
+`.taskless/<engine>/` holds `{config, rules/, rule-tests/}` in that engine's native format. **Directory = engine**: dispatch reads the path, never the file. The config is committed and persisted, not generated.
+
+This is what makes `check` construction-free, and it sidesteps cross-parsing entirely — one engine's config directory never sees another's rule files.
+
+- **Alternative — co-located rules plus a sidecar or envelope:** rejected. Every variant forced either an in-file marker (which Vale rejects outright, `E201`) or a generated config materialized at check time. Per-engine native configs delete that layer rather than manage it.
+
+### D2 — The native config is the source of truth; nothing is generated at check time
+
+`sg/sgconfig.yml` declares `ruleDirs: [rules]` and `testConfigs: [{testDir: rule-tests}]`; rule files carry their own `files`/`ignores` scoping and any Taskless data in native `metadata`. Construction of these configs happens at **rule-authoring time** — by the generator, or by a human editing a committed file — never during a check.
+
+Because `generateSgConfig` no longer runs on the check path, `check` calls `ensureTasklessDirectory` directly so the migration still triggers. That is easy to drop and would silently strand users on an old layout.
+
+### D3 — Migration `0004` is a mechanical, content-preserving move
+
+The existing `.taskless/rules/` and `rule-tests/` are all known-ast-grep, so `0004` moves them under `sg/` without editing contents. `sgconfig.yml`'s `ruleDirs: [rules]` is **relative to the config**, so it survives the move unchanged — no path rewriting. It then scaffolds `vale/` and moves the runtime tier into the same shape.
+
+Content preservation is not incidental: runtime capture bytes determine their reconciliation hashes, so editing during the move would invalidate every signature. Only `discover.ts`'s search path changes.
+
+**Version gating:** `runMigrations` currently returns silently when `version > maxVersion`. It changes to **throw** ("upgrade the CLI") unless `--allow-version-mismatches` is passed, so a newer scaffold fails loudly instead of being half-read by an older CLI.
+
+### D4 — Service-delivered rules default to `sg`, permanently
+
+The ingest path is a separate hazard from the migration: `writeRuleFile`/`writeRuleTestFile` (`rules/files.ts`) hardcode `.taskless/rules`. Left alone, `0004` would relayout existing rules under `sg/` while the next `rule create` wrote straight back into a directory no engine dispatches from — silent, and it reads as a vanished rule.
+
+The API offers nothing to switch on: there is no `engine`, `analysisType`, or `ruleType` field anywhere in `src/generated/api.d.ts`, and `/cli/api/rule/{ruleId}` documents `rules[].content` as "The ast-grep rule definition". The `filename` fields that exist are **client→server** (`references[].filename` on `rule improve`; `files[].file` on reconcile) — the CLI tells the service where things live, not the reverse.
+
+So **no engine identified ⇒ `sg`**, permanently rather than for a migration window, since published CLIs keep receiving engine-less payloads. This is the same judgment `0004` makes about on-disk state, so ingest and migration land a rule in the same place.
+
+**Absence and unrecognized are not the same.** An unrecognized engine means the payload is newer than the CLI; defaulting would file it where the wrong parser reads it, surfacing as a broken rule rather than version skew. Ingest errors and writes nothing.
+
+- **Alternative — sniff the rule body to guess its engine:** rejected; contradicts D1 and guesses where an explicit default is correct and knowable.
+
+### D5 — Both layouts stay readable, so no producer has to cut over
+
+The CLI dispatches the legacy `.taskless/rules/` path as ast-grep alongside `sg/rules/`, de-duplicating when both exist. An unmigrated checkout still runs, and a service that keeps naming the old location keeps working.
+
+This is what decouples this change's release from anyone else's: there is no coordinated cutover, and no window in which a rule silently stops being checked.
+
+### D6 — Reconciliation is path-independent, so the relayout is invisible to the server
+
+Reported reconcile paths are repo-relative POSIX and derived from wherever a rule was discovered (`rules/runtime/run-set.ts`), so they follow the moved trees automatically. The server joins reported files **by content signature, not path** — "content-based, so a moved-but-unchanged rule resolves" — and `0004` is byte-preserving. A migrated rule therefore reconciles to the same server-side rule with no coordinated release.
+
+The API schema's description "Delivered rule filename under `.taskless/rules/` on the client" goes stale when this lands. That is cosmetic and belongs to the platform side; nothing depends on it.
+
+## Risks / Trade-offs
+
+- **Losing the migration trigger when `generateSgConfig` leaves the check path** → `check` calls `ensureTasklessDirectory` directly (D2), and that is worth a test rather than a comment.
+- **Hardcoded `.taskless/rules` literals beyond the ingest writer** — `rules/verify.ts`, `rules/files.ts`, `commands/check.ts`, `commands/rules.ts`, and the `detect/scan.ts` layout probe all name the pre-migration path. Each is the same defect class; a missed one silently reads or writes outside the engine directories.
+- **A partial migration leaves a split-brain `.taskless/`** → `0004` moves whole trees and the legacy path stays readable (D5), so a half-applied state still checks rules rather than dropping them.
+- **Committed configs are maintained artifacts** — scoping now lives in files a human or generator edits, rather than being reconstructed each run. That is the trade: check-time simplicity for author-time maintenance.
+
+## Migration Plan
+
+`0004` runs on first `ensureTasklessDirectory` after upgrade, which `check` triggers. No user action. Rollback is a CLI downgrade plus moving `sg/rules/` back to `rules/` — but the legacy path stays readable, so an old CLI against a new layout degrades to finding no rules rather than erroring, and a new CLI against an old layout still works.
+
+## Open Questions
+
+None outstanding. The engine-selection knowledge topic and the Vale engine are deliberately deferred to the change that adds Vale, since both name engines this change only scaffolds.

openspec/changes/partition-rules-by-engine/proposal.md

@@ -0,0 +1,49 @@
+## Why
+
+Taskless rules run through one engine. `check` writes an ephemeral `sgconfig.yml` and runs `sg scan` over bare ast-grep YAML at `.taskless/rules/*.yml`; the runtime tier sits beside it at `.taskless/runtime-rules/`. Adding any second engine to that shape means either an in-file marker to say which engine owns a rule, or a generated config materialized at check time — both of which were tried and rejected in earlier designs.
+
+This change makes the layout multi-engine **before** any second engine exists: rules partition by top-level directory, each holding that tool's own native, committed config. It ships no new engine and changes no behavior a user can observe — `check` finds and reports exactly what it did before, from a different path. That is the point: the relayout and the migration carry all the risk, so they land on their own where a regression has one obvious cause.
+
+## What Changes
+
+- Partition rules into `.taskless/<engine>/{config, rules/, rule-tests/}` — `sg/`, `vale/` (scaffolded empty), and `runtime/`. The containing directory determines the engine, so dispatch needs no per-file parsing.
+- Make each engine's **committed native config** the source of truth, removing ephemeral `sgconfig.yml` generation from the check path. `check` runs `sg scan --config .taskless/sg/sgconfig.yml`.
+- Add migration `0004`: move `rules/` → `sg/`, `rule-tests/` → `sg/rule-tests/`, `sgconfig.yml` → `sg/`, and `runtime-rules/` → `runtime/rules/` — content-preserving, so runtime capture hashes are unchanged. Scaffold `vale/`, `.gitkeep` every otherwise-empty directory, and bump the scaffold version.
+- Gate on scaffold version: `runMigrations` throws when `taskless.json`'s `version` exceeds what the CLI knows, unless `--allow-version-mismatches` is passed.
+- Write service-delivered rules into `sg/`. The delivery API carries no engine discriminator, so an engine-less payload is ast-grep by definition; an unrecognized engine fails loudly rather than defaulting.
+- Keep dispatching the legacy `.taskless/rules/` path alongside `sg/rules/`, so an unmigrated checkout still runs and no producer has to cut over.
+
+## Capabilities
+
+### New Capabilities
+
+- `cli-rule-format`: The engine-partitioned on-disk layout — `.taskless/<engine>/{config, rules/, rule-tests/}` with each engine's native committed config as the source of truth, directory-based dispatch, the migration that gets there, ingest defaulting, and legacy-layout tolerance. The extension point every future engine plugs into.
+
+### Modified Capabilities
+
+- `cli-check`: Check runs ast-grep against the committed `sg/sgconfig.yml` rather than generating one, and dispatches by engine directory. Because `generateSgConfig` leaves the check path, `check` calls `ensureTasklessDirectory` directly to preserve the migration trigger.
+- `cli-runtime-rule-execution`: Runtime rules are discovered under `.taskless/runtime/rules/<name>/` instead of `runtime-rules/` — a directory move only; execution, reconcile, and signing semantics are untouched.
+
+## Impact
+
+- **CLI (`packages/cli`)**: new `filesystem/migrations/0004-*.ts`; `commands/check.ts` (engine-directory dispatch, migration trigger); `rules/scan.ts` and `rules/verify.ts` (`--config` over `sg/`); `rules/runtime/discover.ts` (new runtime path); `rules/files.ts` (ingest writes into `sg/`); removal of ephemeral `filesystem/sgconfig.ts` generation from the check path.
+- **On-disk**: every existing `.taskless/` is relaid out by `0004` on first run. Content-preserving, so nothing needs re-signing or re-reconciling.
+- **No new engine, no new binary, no user-visible behavior change.** `vale/` is scaffolded empty and nothing executes it yet.
+- **Deliberately excluded**: the Vale engine itself and the engine-selection knowledge topic, which need this layout to exist first.
+
+## Delivery shape
+
+**Stacked, merging down.** Not a preference — a constraint, verified. Task group 1 alone leaves **20 tests failing**: migration `0004` moves rules out from under readers that groups 2–4 update (`rules/scan.ts`, `rules/verify.ts`, `commands/check.ts`, `rules/runtime/discover.ts`). Any intermediate state ships a CLI that has relocated its rules and cannot find them, so no unit can reach production alone.
+
+The stack merges down into the bottom branch and reaches `main` as one commit.
+
+| Unit | Scope                                                                            |
+| ---- | -------------------------------------------------------------------------------- |
+| 1    | Migration `0004`, directory scaffolding, version gating                          |
+| 2    | Directory-based dispatch, service-delivered rule ingest, reconcile compatibility |
+| 3    | Runtime discovery path                                                           |
+| 4    | ast-grep over the committed config, quality gates                                |
+
+Group 1's diff is ~580 lines under `packages/`, 347 of them fixture tests. That exceeds the repo's ~300-line guideline and is the honest cost of a migration that must prove byte-identical moves; splitting the tests from the migration they cover would make review worse, not better.
+
+**Tracking:** OSS-24

openspec/changes/partition-rules-by-engine/specs/cli-check/spec.md

@@ -0,0 +1,36 @@
+## ADDED Requirements
+
+### Requirement: Check subcommand executes ast-grep scan
+
+The CLI SHALL execute `sg scan --config .taskless/sg/sgconfig.yml --json=stream` using `child_process.spawn` with `shell: true` for cross-platform binary resolution, reading the **committed** ast-grep config at `.taskless/sg/sgconfig.yml`. No `sgconfig.yml` is generated at check time. The `sg` binary SHALL be resolved from the `@ast-grep/cli` dependency via PATH. Reconciliation/run-set semantics for runtime rules are unchanged.
+
+#### Scenario: ast-grep scan runs with the committed config
+
+- **WHEN** the CLI executes the ast-grep scanner
+- **THEN** it SHALL invoke `sg scan` with `--config .taskless/sg/sgconfig.yml` and `--json=stream`
+- **AND** it SHALL NOT write or generate a config file
+- **AND** the working directory for the spawned process SHALL be the resolved project directory
+
+#### Scenario: ast-grep binary is not found
+
+- **WHEN** the `sg` binary cannot be resolved from PATH
+- **THEN** the CLI SHALL print an error message indicating ast-grep is not available
+- **AND** the CLI SHALL exit with code 1
+
+### Requirement: Check dispatches static and runtime rules to distinct executors
+
+`taskless check` SHALL dispatch rules to distinct executors by their engine directory: **ast-grep** rules under `.taskless/sg/` via the ast-grep scanner, **Vale** rules under `.taskless/vale/` via the Vale runner (per the `cli-vale-rule-engine` capability), and **runtime** rules under `.taskless/runtime/rules/` via the runtime harness (per the `cli-runtime-rule-execution` capability). Findings from all executors SHALL be aggregated into the same result set and SHALL count toward the exit code identically.
+
+#### Scenario: Mixed corpus runs all executors
+
+- **WHEN** `.taskless/sg/` contains ast-grep rules, `.taskless/vale/` contains Vale rules, and `.taskless/runtime/rules/` contains runtime rules
+- **THEN** the CLI SHALL run ast-grep rules through `sg scan`, Vale rules through the Vale runner, and runtime rules through the runtime harness
+- **AND** SHALL merge their findings into one result set
+
+## REMOVED Requirements
+
+### Requirement: Check subcommand generates ephemeral sgconfig.yml
+
+**Reason**: Engine configs are now committed per-engine directory (`.taskless/sg/sgconfig.yml`), not generated at check time — check reads the committed config directly.
+
+**Migration**: Migration `0004` moves the existing `.taskless/sgconfig.yml` to `.taskless/sg/sgconfig.yml` (its relative `ruleDirs: [rules]` remains valid). No config is written at check time thereafter.