Skip to content

quality: add deterministic runtime-boundary program - #1784

Merged
raphaeltm merged 20 commits into
mainfrom
sam/coordinate-implement-deterministic-runtime-gkq1xt
Aug 10, 2026
Merged

quality: add deterministic runtime-boundary program#1784
raphaeltm merged 20 commits into
mainfrom
sam/coordinate-implement-deterministic-runtime-gkq1xt

Conversation

@simple-agent-manager

@simple-agent-manager simple-agent-manager Bot commented Aug 9, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Adds complete lint and type/template coverage for every current pnpm workspace, including Astro-aware validation for apps/www.
  • Migrates the authoritative host to ESLint 9 flat config with exact 2,372/2,372 legacy finding parity, then adds an unpublished eslint-plugin-sam with three fixture-backed advisory runtime-boundary rules.
  • Adds deterministic net-count ratchets for existing boundary and formatting debt, targeted advisory ts-morph checks, current-tree/PR-range Gitleaks, direct-dependency evidence, and diff-local govulncheck.
  • Measures Oxlint in report-only shadow mode. Its promotion gates failed, so ESLint remains authoritative.
  • Removes the one runtime as any with a bounded navigator.userAgentData interface; application behavior and deployment output are otherwise unchanged.
  • Stabilizes the load-sensitive API contract tests by moving route/client collection and RSA setup outside Vitest's timed test and hook bodies while retaining the unchanged five-second timeout and all behavioral assertions.
  • Pins the staging R2 preflight to the frozen-lockfile apps/api Wrangler binary, eliminating registry-dependent npx bootstrap behavior while preserving its fail-closed status handling.

Migration and rollout

  1. Workspace coverage and ESLint 9 flat-host parity land first; old findings remain passable.
  2. SAM rules provide editor warnings and RuleTester fixtures; the independent repository ratchet alone blocks net-new debt.
  3. Supply-chain checks remain isolated, fail-closed leaf commands with private scanner output.
  4. Oxlint stays report-only. Its alpha SAM plugin host is an 8/8 fixture shadow and is never authoritative in this rollout.

Developer entry point: pnpm check:fast.

Rule ownership and lifecycle metadata live in packages/eslint-plugin-sam/rules.manifest.json. Boundary, format, Astro, and reviewed-secret baselines identify an owner/backlog/review window and reject increases or expired exemptions as applicable. Contributor commands, authoritative/advisory roles, scanner privacy, and rollback switches are documented in CLAUDE.md and scripts/quality/README.md.

Validation

  • Branch rebased onto current origin/main
  • .codex/config.toml matches origin/main exactly
  • pnpm check:fast — formatting ratchet, Oxlint shadow, authoritative ESLint, and type-boundary ratchet pass
  • pnpm typecheck — 19/19 workspace tasks
  • Full pnpm test — 21/21 package tasks; API 6,799/6,799 and web 2,937/2,937
  • pnpm quality:scripts:test — 31 files / 283 tests
  • pnpm --filter @simple-agent-manager/eslint-plugin-sam test — 16/16 RuleTester cases
  • pnpm build --force — 9/9 workspace tasks without cache
  • Focused heartbeat/node-agent contract pair — 10 consecutive 70-test runs during concurrent workspace typecheck
  • Independent completion revalidation — 5 consecutive 70-test runs during forced uncached typecheck; timed test bodies remained below 1.5 seconds
  • VM Agent, CLI, harness, and locked govulncheck-tool Go race suites pass; diff-local govulncheck remains blocking
  • Gitleaks current tree and PR range pass with findings kept private
  • ESLint migration parity — 2,372 legacy / 2,372 flat, zero normalized missing/added findings
  • Oxlint shadow coverage includes all tracked Astro files; promotion gates failed, so it stays shadow
  • All specialist reviews completed with no unresolved finding
  • Fresh final-head GitHub CI — CI, E2E Smoke, and CodSpeed all passed on 384627e36; Specialist Review Evidence is green
  • Consolidated staging regression sweep on the pushed candidate — run 31423110320

The load discrepancy is closed at its root boundary. The heartbeat route, node-agent client, and ephemeral RSA key material are now collected during module setup; tests reset only the fetch boundary. No test was skipped or retried, the five-second timeout is unchanged, and no assertion was relaxed to hide load sensitivity. The staging workflow's R2 bucket preflight now resolves the catalog-pinned Wrangler 4.118.0 from the frozen install instead of depending on an on-demand registry download.

Staging Verification (REQUIRED for all code changes — merge-blocking)

  • Staging deployment green — final content SHA 1ae5808f completed run 31423110320
  • Live app verified via Playwright — authenticated dashboard/projects, project chat, and settings regression sweep passed
  • Existing workflows confirmed working — built-in deployment smoke tests and authenticated browser regression sweep passed
  • New feature/fix verified on staging — pinned Wrangler preflight, full deploy, health check, and application regression sweep passed
  • Infrastructure verification completed — staging ended with zero VMs and zero workspaces; nothing was provisioned
  • Mobile and desktop verification notes added for UI changes — N/A: no UI layout or interaction change

Staging Verification Evidence

The final-content candidate deployed once in run 31423110320: configuration, Cloudflare deployment, migrations, API/tail/web deploys, health check, and built-in smoke tests all passed. Authenticated Playwright used SAM_PLAYWRIGHT_PRIMARY_USER through /api/auth/token-login, rendered the dashboard with 15 projects, opened an existing project and its chat, loaded settings, and observed zero console errors, page errors, or HTTP 5xx responses. Authenticated API checks confirmed zero staging VMs and zero workspaces; no resource was provisioned. Two earlier attempts on superseded SHA 86be87c9 stopped before Cloudflare mutation because the workflow used registry-dependent npx wrangler; the pinned frozen-lockfile correction was independently reviewed and the successful run above is the sole deployed regression sweep.

UI Compliance Checklist (Required for UI changes)

  • Mobile-first layout verified — N/A: no UI layout change
  • Accessibility checks completed — N/A: no rendered UI change
  • Shared UI components used or exception documented — N/A: typing-only browser capability cleanup
  • Playwright visual audit run locally — N/A: no visual change

End-to-End Verification (Required for multi-component changes)

  • Data flow traced from developer command through workspace scripts and CI leaves
  • Capability tests exercise checker fixtures, failure semantics, and CI source contracts
  • Assumptions were re-audited against current main before baselines were captured
  • Local full-suite load discrepancy root-caused and deterministically repaired

Data Flow Trace

package.json:check:fast invokes the format ratchet, Oxlint report-only wrapper, Turbo workspace ESLint scripts, and scripts/quality/check-type-boundaries.ts. .github/workflows/ci.yml calls the same leaves, explicitly runs SAM RuleTester fixtures, and separately invokes privacy-safe supply-chain wrappers. Workspace lint/typecheck scripts cover the audited TS/TSX/MTS/CTS/Astro populations. Runtime application paths do not depend on the new quality tooling.

Untested Gaps

None. Local implementation, full-suite tests, specialist review, label-free CI evidence, and consolidated staging verification are complete.

Post-Mortem (Required for bug fix PRs)

N/A: not a user-visible bug-fix PR; this is progressively introduced repository quality infrastructure.

Specialist Review Evidence

  • All local reviewers completed and findings addressed before merge
  • No reviewer remains pending or failed
Reviewer Status Outcome
Picky code-quality / architecture ADDRESSED Initial concerns repaired; exact-code re-review found no remaining implementation failure
security-auditor PASS Current rebased diff uses ephemeral test keys, preserves real JWT signing, and keeps security gates fail closed
test-engineer ADDRESSED Regression paths and the load-sensitive timed-import boundary are deterministically covered
constitution-validator PASS Exact-code review found no introduced blocking Principle XI violation
doc-sync-validator ADDRESSED Removed stale rollout language and confirmed contributor documentation matches the final implementation
task-completion-validator ADDRESSED Reproduced the original load failure, then verified the corrective SHA under forced concurrent load

Rollback

  • Coverage/ESLint foundation: revert workspace scripts and flat config together; the captured legacy report remains parity evidence.
  • SAM plugin: disable sam/* warnings or remove its workspace reference; the count ratchet is independent.
  • Boundary/format ratchets: remove their individual CI leaf while retaining reports for diagnosis; no runtime code depends on them.
  • Supply-chain layer: disable the affected job/leaf independently; never publish or baseline new secret or vulnerability details during rollback.
  • Oxlint: remove lint:oxlint from check:fast and CI. ESLint 9 remains the complete authoritative layer.

Exceptions

  • Scope: Existing lint, format, Astro, and boundary debt.
  • Rationale: Baseline-and-ratchet rollout preserves main while rejecting net-new debt; broad cleanup is deliberately out of scope.
  • Expiration: Baseline reviews and exemptions carry explicit review/expiry metadata; zero-count classes are promoted and removed from baselines.

Agent Preflight (Required)

  • Preflight completed before code changes

Classification

  • external-api-change
  • cross-component-change
  • business-logic-change
  • public-surface-change
  • docs-sync-change
  • security-sensitive-change
  • ui-change
  • infra-change

External References

Official documentation was consulted for ESLint flat config and migration, typescript-eslint supported versions, Oxlint native and alpha plugin behavior, Astro template checks, Gitleaks scan/report behavior, and govulncheck: https://eslint.org/docs/latest/use/configure/migration-guide, https://typescript-eslint.io/users/dependency-versions/, https://oxc.rs/docs/guide/usage/linter.html, https://docs.astro.build/en/reference/cli-reference/#astro-check, https://github.com/gitleaks/gitleaks, and https://go.dev/doc/security/vuln/.

Codebase Impact Analysis

Root package.json, pnpm-workspace.yaml, turbo.json, and ESLint/Oxlint configuration define the developer contract. apps/www, apps/tail-worker, infra, packages/cloud-init, and tools/og-image gain lint/type/template coverage. packages/eslint-plugin-sam owns syntax diagnostics, scripts/quality owns deterministic ratchets and scanner wrappers, and .github/workflows/ci.yml calls those leaves without changing deployment behavior.

Documentation & Specs

Updated CLAUDE.md, scripts/quality/README.md, .claude/rules/51-runtime-boundary-validation.md, the active task record, rule documentation, lifecycle manifest, and machine-readable adoption evidence. No product feature spec changes are needed because this is contributor/CI quality infrastructure.

Constitution & Risk Check

Checked progressive rollout, no-hardcoded-business-values, security/privacy, deterministic testing, and rollback isolation. Tool versions and audited base SHAs are explicit evidence/protocol pins; existing debt remains baselined; scanner output stays private and temporary; specialist evidence, local validation, CI, and consolidated staging remain fail-closed merge gates.

@simple-agent-manager simple-agent-manager Bot added the needs-human-review Agent could not complete all review gates — human must approve before merge label Aug 9, 2026
@codspeed-hq

codspeed-hq Bot commented Aug 9, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 6 untouched benchmarks


Comparing sam/coordinate-implement-deterministic-runtime-gkq1xt (384627e) with main (8d6296e)

Open in CodSpeed

@raphaeltm raphaeltm removed the needs-human-review Agent could not complete all review gates — human must approve before merge label Aug 10, 2026
@sonarqubecloud

Copy link
Copy Markdown

@raphaeltm
raphaeltm merged commit 3117585 into main Aug 10, 2026
26 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant