docs(openspec): propose runtime rule execution by thecodedrift · Pull Request #49 · taskless/cli

thecodedrift · 2026-07-03T14:57:33Z

Add the OpenSpec change defining how a runtime rule executes locally in taskless check. This is the deliberate follow-up that server-owned reconciliation was built to enable: reconciliation gates which files may run; this proposal defines the local harness that evaluates a runtime rule.

Stacked on #47 (jakob/server-owned-rule-reconciliation) — base this PR against that branch, not main. Docs/spec only; no code yet. Implementation lands in a later PR off this one.

What it specifies

New capability cli-runtime-rule-execution — a runtime rule is a directory under .taskless/runtime-rules/ (ast-grep capture *.yml + a check.ts). The harness assembles the capture rules into one ast-grep narrow (anchor --inline-rules --json=stream; broad --files-with-matches), gates on matches, then invokes check.ts's default export with (root, matches) via a CLI-bundled pinned tsx. Zero matches ⇒ check.ts never runs. Findings map onto the existing scanner-agnostic CheckResult. Adds a per-check timeout (--timeout override) and a throwing/hanging check is isolated to one error finding.
cli-check (modified) — static ast-grep rules always run (offline linter, unchanged); runtime check.ts runs only on a signature-validated path. Five modes: authed / API-key → reconcile & run blessed rules; logged-out / --anonymous → skip + report; --dangerously-run-scripts → run all trusting local signatures (skips the network). Skipped runtime rules surface as an additive optional skipped array under --json.
cli-rule-reconciliation (refines feat(cli): server-owned rule reconciliation #47) — reconciliation is scoped to each runtime rule's check.ts (the only ACE-bearing artifact). Static rules and inert capture *.yml are not reported or gated. This intentionally evolves spec text feat(cli): server-owned rule reconciliation #47 introduced.

Key decisions (see design.md)

The signature gate is check.ts only — capture *.yml are inert ast-grep patterns and can't execute code, so they're ungated.
Runtime rules live in their own tree (.taskless/runtime-rules/, tests under .taskless/runtime-rule-tests/); location is the primary static-vs-runtime split.
Local harness is eslint-equivalent trust gated by server signature — the hardened --enforce sandbox + commit-bound report remain server-side scope, out of this change.

Refs TSKL-245

Stack (root → tip):

feat(cli): server-owned rule reconciliation #47
- ➡️ docs(openspec): propose runtime rule execution #49 (you are here)
  - feat(cli): implement runtime rule execution #50

Add the OpenSpec change defining how a runtime rule executes, stacked on server-owned rule reconciliation. Reconciliation gates which files may run; this proposal defines the local harness that evaluates a runtime rule. A runtime rule is a directory under .taskless/runtime-rules/ (capture *.yml + a check.ts). The harness assembles the capture rules into one ast-grep narrow, gates on matches, then invokes check.ts's default export via a bundled tsx. check.ts is arbitrary code execution, so it runs only when its signature is validated by reconciliation, or under --dangerously-run-scripts. Static ast-grep rules and inert capture *.yml are never gated. Adds the five auth/flag modes, a --timeout bound, and an additive --json skipped field. Refs TSKL-245 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Verified against workers/generator/src/actions/add-runtime-rule.ts: runtime rules are written to .taskless/runtime-rules/<slug>-<suffix>/ (one <name>.yml per capture rule + a check.ts), with fixtures under .taskless/runtime-rule-tests/. The check file is always check.ts, and its bytes are hashed with the same canonicalHash envelope reconcile uses — so gating on check.ts matches what the server captures. Tighten the recognition spec (drop the metadata.taskless.check locator claim; check.ts is fixed) and record the confirmation in design.md. Refs TSKL-245 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Copilot

Pull request overview

Defines an OpenSpec change proposal for how “runtime rules” execute locally in taskless check, including the on-disk layout, the narrow→gate→check.ts harness, signature-gated execution modes, reconciliation scoping, and related help/test expectations. This fits into the CLI spec suite as the follow-up to server-owned reconciliation by specifying the local execution contract for ACE-bearing runtime rule checks.

Changes:

Adds a new cli-runtime-rule-execution capability spec describing runtime rule discovery, narrowing strategy, match normalization, and check.ts invocation contract.
Modifies cli-check and cli-rule-reconciliation delta specs to scope reconciliation to runtime check.ts, add runtime-rule execution modes (including --dangerously-run-scripts), and define skipped-runtime reporting.
Adds proposal/design/tasks artifacts for the change (including implementation task breakdown and rationale/tradeoffs).

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 8 comments.

Show a summary per file

File	Description
openspec/changes/runtime-rule-execution/tasks.md	Implementation task checklist for runtime rule execution + reconciliation scope changes.
openspec/changes/runtime-rule-execution/specs/cli-runtime-rule-execution/spec.md	New capability spec for runtime rule discovery and the narrow→gate→`check.ts` harness.
openspec/changes/runtime-rule-execution/specs/cli-rule-reconciliation/spec.md	Delta spec refining reconciliation scope to runtime rule `check.ts` only.
openspec/changes/runtime-rule-execution/specs/cli-check/spec.md	Delta spec defining `check` runtime/static dispatch, execution gating modes, and skipped-runtime reporting.
openspec/changes/runtime-rule-execution/proposal.md	High-level proposal summary of behavior and impact across CLI components.
openspec/changes/runtime-rule-execution/design.md	Detailed design decisions, tradeoffs, and mode table for runtime rule execution.
openspec/changes/runtime-rule-execution/.openspec.yaml	Change metadata for the spec-driven workflow.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Group 1 of runtime-rule-execution. Add the harness-side recognition of runtime rules: a directory under .taskless/runtime-rules/ holding kind: runtime capture *.yml plus a check.ts. - src/types/runtime-rule.ts mirrors the structural harness<->check contract (Finding, Match, CheckFunction, CaptureRule, metadata block) from the generator's @taskless/types; a delivered check imports nothing, so the contract is structural. - src/rules/runtime/discover.ts enumerates .taskless/runtime-rules/, parses each capture *.yml, confirms kind: runtime, and returns a typed RuntimeRule (capture rules with id/name/language/match + the check.ts path). .taskless/runtime-rule-tests/ is never enumerated. Refs TSKL-245 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Groups 2-3 of runtime-rule-execution. Implement the local harness that evaluates a runtime rule, plus the bundled tsx loader it runs check.ts under. - narrow.ts: run a rule's capture rules as ONE ast-grep scan (anchor --json=stream, broad --files-with-matches) and normalize matches to the contract shape (0-indexed -> 1-indexed, ruleId -> model name, captures from metaVariables). Uses a temp --config rules dir so multiple captures + full ast-grep config run in a single scan. - invoke.ts: run check.ts's default export (root, matches) via a pinned tsx resolved at runtime (no repo toolchain). An embedded ESM runner writes the returned Finding[] to an out-file; a throw, non-zero exit, or timeout is isolated to an error result. Default 10s bound, overridable. - harness.ts: narrow -> gate-on-matches -> invoke -> map Finding to CheckResult (source: taskless-runtime); process-per-check, sequential. - Add tsx to the CLI dependencies (externalized from the Vite bundle). Verified end-to-end against a temp-dir fixture: discovery, narrow (correct line normalization + rule attribution), tsx invocation, and finding mapping. Refs TSKL-245 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…d rules Group 4 of runtime-rule-execution. Add the runtime reconcile-set: sign each rule's check.ts (only) for reconciliation, select the rules the server blessed, and materialize them for read-hash-execute. - src/rules/runtime/run-set.ts: signRuntimeChecks (check.ts only — capture *.yml are inert), reportRuntimeChecks -> { file, signature }, selectBlessedRuntimeRules (content-join: a rule runs iff its check.ts signature is in the run set; the rest are withheld/advisory), and materializeRuntimeRules (copy blessed rule dirs into .taskless/.run/runtime-rules/ and re-discover so execution uses the blessed bytes; .run/ stays gitignored). - discover.ts: extract discoverRuntimeRulesIn(root) so materialized rules can be re-discovered from .run/. - narrow.ts: copy the original capture *.yml bytes into the temp config instead of re-serializing the parsed object — a YAML round-trip can alter an exotic ast-grep config. Verified end-to-end: report only check.ts, bless on signature match, materialize to .run/, execute the materialized copy; empty run set withholds the rule. Refs TSKL-245 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…ile gating Group 5 of runtime-rule-execution. Rewire `taskless check` around the two rule classes and complete the cutover to signing only untrusted code. - Static ast-grep rules under .taskless/rules/ always run (trusted, no network). - Runtime rules run only on a validated path: planRuntime resolves the mode from auth state + flags — authed reconcile runs blessed rules and withholds the rest (advisory); logged-out / --anonymous / no-remote / reconcile-unavailable skip runtime with a notice; --dangerously-run-scripts runs all runtime rules with no network behind a loud warning. - Add --dangerously-run-scripts and --timeout <seconds>; runtime findings merge into the same results and exit-code logic; --json gains an additive optional `skipped` array (schema updated), warnings/notices stay stderr-only. - Fix a Finding->CheckResult off-by-one: findings are 1-indexed, CheckResult.range is 0-indexed (display/json add 1). Cutover: remove the stacked-under static-reconcile gating — delete src/rules/run-set.ts and the now-obsolete test/reconcile-check.test.ts + test/run-set.test.ts (runtime-dispatch tests land in Group 7). Static rules are no longer signed or gated; only runtime check.ts is. Verified end-to-end via the built CLI: static-only-runs (runtime skipped + notice), --dangerously-run-scripts (both run), and --json (skipped array, warnings suppressed). Full CLI suite green (326 tests). Refs TSKL-245 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Groups 6-7 of runtime-rule-execution. Docs (Group 6): - check.txt (topic v2): static rules always run; runtime check.ts runs only when server-verified; the mode table, --dangerously-run-scripts, --timeout, and the --json skipped array. - ci.txt: unauthenticated CI runs static rules and skips runtime; the TASKLESS_TOKEN backstop is the enforcement point for runtime check.ts. Tests (Group 7): - runtime-harness.test.ts: discovery, gate-on-zero-matches (check never invoked), match normalization + Finding->CheckResult indexing, throwing-check isolation, timeout -> error finding. - runtime-check.test.ts: end-to-end dispatch via the built CLI with a mock reconcile server + git origin — authed-blessed, empty-run withheld, logged-out and --anonymous skip + report, reconcile-unavailable skips, dangerously-run- scripts runs offline; asserts static always runs and only check.ts is reported. Also fix a real timeout bug found by the harness test: tsx re-execs node as a grandchild, so spawn detached and SIGKILL the whole process group — otherwise a runaway check keeps running past the timeout. Full suite green (338 tests). Refs TSKL-245 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…ime-rule-execution

…xecution-impl

…ntation Verification (opsx:verify) caught a spec-vs-impl divergence: the narrow requirement, proposal, and design named `--inline-rules --json=stream`, but the harness assembles the capture rules into a temp `--config` (--inline-rules carries only one rule; a runtime rule has multiple capture rules + full ast-grep config). Reworded to "one scan per mode" via a generated config so the spec that gets synced to canonical on archive matches reality. Refs TSKL-245 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Sync the runtime-rule-execution delta specs into the canonical specs and archive the change (verify → sync → archive). - cli-runtime-rule-execution: new canonical spec (the runtime harness contract). - cli-check: static-vs-runtime dispatch, the validated-path rule, skip+report, and --dangerously-run-scripts added; auth-state/reconcile/degrade requirements updated for the cutover; the now-obsolete 'warns on reconciliation mismatches' and 'exits cleanly when the run set is empty' requirements removed (their static-reconcile behavior was deleted in the cutover — the delta records the removal with reasons). - cli-rule-reconciliation: reporting + run-set requirements rescoped to each runtime rule's check.ts. Change archived to openspec/changes/archive/2026-07-03-runtime-rule-execution/; no unarchived changes remain, so the tip's check-openspec-archived job passes. Refs TSKL-245 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Copilot

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 6 comments.

…ime-rule-execution

Address Copilot review on #49: the 'Blessed runtime rules execute from the materialized run directory' requirement, its scenario, and the design decision referred to the live tree as `.taskless/rules/`, but runtime rules live under `.taskless/runtime-rules/`. Correct all three so the read-hash-execute guarantee names the right tree. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…xecution-impl

Six fixes from Copilot's review of #50, plus the canonical-spec twin of the #49 doc fix: - invoke.ts: on Windows, kill the check via taskkill /T (negative PIDs aren't supported), so a timeout actually terminates the tsx+node tree. - discover.ts: pin the check file to check.ts inside the rule dir; do not resolve metadata.taskless.check as a path (prevents escaping the dir via ../). - run-set.ts: signRuntimeChecks is now per-rule resilient (returns unreadable rules instead of throwing); reported reconcile paths are POSIX-normalized so Windows backslashes don't defeat the server-side path match. - check.ts: a missing/unreadable check.ts is reported as skipped and materialization errors degrade to a runtime-skip — a malformed runtime rule no longer aborts the whole check (static keeps running). - narrow.ts: run one broad scan per broad capture rule so matches are attributed to the right rule (was mislabeling all broad matches as the first rule); treat a signal-killed narrow (null exit code) as a failure instead of success. - specs/cli-runtime-rule-execution: fix the materialize requirement's live-tree path (.taskless/rules/ -> .taskless/runtime-rules/) to match #49. Add an integration test: a runtime rule missing check.ts is skipped (not fatal) and static rules still run. Full suite green (339). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Address the remaining #49 review comments (items 1-4, 6-7): - cli-check: the degrade --json scenario now allows the additive `skipped` array (not 'only { success, results }'); --dangerously-run-scripts is described as 'without server validation' rather than 'trusting local signatures' (which the reconciliation spec forbids as an auth mechanism). - cli-rule-reconciliation: rename the requirement heading to 'Reconcile reports every runtime rule's check.ts' (RENAMED op) so the title matches the scoped body instead of the old 'every held rule file'. - cli-runtime-rule-execution: clarify that match mode is read per capture rule (rules may mix modes); state that a broad path-only match carries line/column = 1 and empty text/captures. - design: note the generator path lives in the internal taskless/taskless repo, not this one. (Item 5 — the 'exactly as before' wording — is left as-is by decision.) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…xecution-impl

Propagate the #49 review-comment fixes into the canonical specs (the archived change copy came via merge): - cli-check: degrade --json scenario allows the additive skipped array; --dangerously-run-scripts described as 'without server validation'. - cli-rule-reconciliation: heading renamed to 'Reconcile reports every runtime rule's check.ts'. - cli-runtime-rule-execution: per-capture match mode; broad matches are path-only (line/column 1, empty text/captures). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

feat(cli): implement runtime rule execution

Address Copilot review on #49: the 'Blessed runtime rules execute from the materialized run directory' requirement, its scenario, and the design decision referred to the live tree as `.taskless/rules/`, but runtime rules live under `.taskless/runtime-rules/`. Correct all three so the read-hash-execute guarantee names the right tree. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Six fixes from Copilot's review of #50, plus the canonical-spec twin of the #49 doc fix: - invoke.ts: on Windows, kill the check via taskkill /T (negative PIDs aren't supported), so a timeout actually terminates the tsx+node tree. - discover.ts: pin the check file to check.ts inside the rule dir; do not resolve metadata.taskless.check as a path (prevents escaping the dir via ../). - run-set.ts: signRuntimeChecks is now per-rule resilient (returns unreadable rules instead of throwing); reported reconcile paths are POSIX-normalized so Windows backslashes don't defeat the server-side path match. - check.ts: a missing/unreadable check.ts is reported as skipped and materialization errors degrade to a runtime-skip — a malformed runtime rule no longer aborts the whole check (static keeps running). - narrow.ts: run one broad scan per broad capture rule so matches are attributed to the right rule (was mislabeling all broad matches as the first rule); treat a signal-killed narrow (null exit code) as a failure instead of success. - specs/cli-runtime-rule-execution: fix the materialize requirement's live-tree path (.taskless/rules/ -> .taskless/runtime-rules/) to match #49. Add an integration test: a runtime rule missing check.ts is skipped (not fatal) and static rules still run. Full suite green (339). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Address the remaining #49 review comments (items 1-4, 6-7): - cli-check: the degrade --json scenario now allows the additive `skipped` array (not 'only { success, results }'); --dangerously-run-scripts is described as 'without server validation' rather than 'trusting local signatures' (which the reconciliation spec forbids as an auth mechanism). - cli-rule-reconciliation: rename the requirement heading to 'Reconcile reports every runtime rule's check.ts' (RENAMED op) so the title matches the scoped body instead of the old 'every held rule file'. - cli-runtime-rule-execution: clarify that match mode is read per capture rule (rules may mix modes); state that a broad path-only match carries line/column = 1 and empty text/captures. - design: note the generator path lives in the internal taskless/taskless repo, not this one. (Item 5 — the 'exactly as before' wording — is left as-is by decision.) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Propagate the #49 review-comment fixes into the canonical specs (the archived change copy came via merge): - cli-check: degrade --json scenario allows the additive skipped array; --dangerously-run-scripts described as 'without server validation'. - cli-rule-reconciliation: heading renamed to 'Reconcile reports every runtime rule's check.ts'. - cli-runtime-rule-execution: per-capture match mode; broad matches are path-only (line/column 1, empty text/captures). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

thecodedrift and others added 2 commits July 3, 2026 07:56

thecodedrift marked this pull request as ready for review July 3, 2026 15:05

Copilot AI review requested due to automatic review settings July 3, 2026 15:05

Copilot started reviewing on behalf of thecodedrift July 3, 2026 15:05 View session

Copilot AI reviewed Jul 3, 2026

View reviewed changes

thecodedrift mentioned this pull request Jul 3, 2026

feat(cli): implement runtime rule execution #50

Merged

7 tasks

thecodedrift and others added 4 commits July 3, 2026 09:06

github-actions Bot mentioned this pull request Jul 3, 2026

feat(cli): server-owned rule reconciliation #47

Merged

thecodedrift and others added 4 commits July 3, 2026 11:59

Merge branch 'jakob/server-owned-rule-reconciliation' into jakob/runt…

9fe0599

…ime-rule-execution

Merge branch 'jakob/runtime-rule-execution' into jakob/runtime-rule-e…

ac76bf9

…xecution-impl

thecodedrift requested a review from Copilot July 3, 2026 22:38

Copilot started reviewing on behalf of thecodedrift July 3, 2026 22:38 View session

Copilot AI reviewed Jul 3, 2026

View reviewed changes

thecodedrift and others added 7 commits July 4, 2026 11:16

Merge branch 'jakob/server-owned-rule-reconciliation' into jakob/runt…

91063f4

…ime-rule-execution

Merge branch 'jakob/runtime-rule-execution' into jakob/runtime-rule-e…

61689a8

…xecution-impl

Merge branch 'jakob/runtime-rule-execution' into jakob/runtime-rule-e…

f1a4656

…xecution-impl

thecodedrift mentioned this pull request Jul 4, 2026

feat(ci): preserve merged PRs in the stack breadcrumb #51

Merged

Merge pull request #50 from taskless/jakob/runtime-rule-execution-impl

a1b4203

feat(cli): implement runtime rule execution

thecodedrift merged commit 44cfaa5 into jakob/server-owned-rule-reconciliation Jul 6, 2026
2 checks passed

thecodedrift deleted the jakob/runtime-rule-execution branch July 6, 2026 20:46

github-actions Bot mentioned this pull request Jul 10, 2026

chore: version packages #59

Merged

Navigation Menu

Uh oh!

docs(openspec): propose runtime rule execution - #49

docs(openspec): propose runtime rule execution#49
thecodedrift merged 19 commits into
jakob/server-owned-rule-reconciliationfrom
jakob/runtime-rule-execution

thecodedrift commented Jul 3, 2026 •

edited by github-actions Bot

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

thecodedrift commented Jul 3, 2026 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What it specifies

Key decisions (see design.md)

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

thecodedrift commented Jul 3, 2026 •

edited by github-actions Bot

Loading