diff --git a/capabilities.harness.yaml b/capabilities.harness.yaml index f9e8132..1a5c6d7 100644 --- a/capabilities.harness.yaml +++ b/capabilities.harness.yaml @@ -47,6 +47,7 @@ harness_capability: - list_capabilities.py - verify_async_tasks.py - verify_contract_hash.py + - verify_swarm_contract.py - verify_script_index.py - verify_skills_readme.py diff --git a/docs/BACKLOG.md b/docs/BACKLOG.md index 02600dd..5d96cd6 100644 --- a/docs/BACKLOG.md +++ b/docs/BACKLOG.md @@ -11,3 +11,10 @@ Optional and future work that is **not** required for the core portable harness. **Status:** Not implemented. Security, allowlisting, and host configuration should be documented alongside [MCP_PRIVATE_HOST.md](MCP_PRIVATE_HOST.md) when this is built. **Scope guardrails (from alignment doc):** No new business logic in the server—thin wrapper over existing scripts only. + +## Swarm-readiness milestones (solo-builder-first) + +- **M1 — Swarm decision contract:** establish `swarm_decision_v0_1` fields and policy defaults. +- **M2 — Operating modes + examples:** publish default topology and synthetic low/high risk examples. +- **M3 — Verification:** enforce contract and policy constraints via script checks. +- **M4 — Minimal orchestrator reference:** provide a small runnable reference flow that implements referee override and mandatory high-stakes human gate. diff --git a/docs/CHEATSHEET.md b/docs/CHEATSHEET.md index 5672501..ee8882e 100644 --- a/docs/CHEATSHEET.md +++ b/docs/CHEATSHEET.md @@ -45,6 +45,7 @@ Use this table for **action parity** with humans (same script, agent via `run_te | `list_capabilities.py` | Emit harness manifest (`capabilities.harness.yaml`) as JSON for scripts/checklist discovery | `python scripts/list_capabilities.py` | | `verify_async_tasks.py` | Validate `state/async_tasks.yaml` task ledger schema | `python scripts/verify_async_tasks.py` | | `verify_contract_hash.py` | Verify `docs/contracts/scp_mcp_v1.md` SHA-256 matches `docs/contracts/scp_mcp_v1.sha256` | `python scripts/verify_contract_hash.py` | +| `verify_swarm_contract.py` | Validate `state/swarm_runs/**/*.yaml` against swarm contract v0.1 policy fields | `python scripts/verify_swarm_contract.py` | | `verify_script_index.py` | Parity: `capabilities.harness.yaml` `scripts[]` == on-disk scripts; each basename in this table | `python scripts/verify_script_index.py` | | `verify_skills_readme.py` | `.cursor/skills/README.md` table vs each `SKILL.md` `description:` | `python scripts/verify_skills_readme.py` | | `brain_map_viewer.html` | Local HTML viewer for graph JSON (vis-network; drag-drop or co-located `brain-map-graph.json`) | Open in browser; optional dev aid. **Operator-facing map (canonical UI in OpenGrimoire):** [OpenGrimoire `docs/GUI_ACTION_MAP_BRAIN_MAP.md`](../../OpenGrimoire/docs/GUI_ACTION_MAP_BRAIN_MAP.md) | diff --git a/docs/OPENHARNESS_REVIEW_2026-04-11.md b/docs/OPENHARNESS_REVIEW_2026-04-11.md new file mode 100644 index 0000000..78e77c4 --- /dev/null +++ b/docs/OPENHARNESS_REVIEW_2026-04-11.md @@ -0,0 +1,114 @@ +# OpenHarness review — intent, context, process status (2026-04-11) + +## Goal under review + +Target product goal (from user directive): **OpenHarness as an agnostic template for operating AI agents and AI swarm decision-making processes**. + +## Verified intent and context + +### What OpenHarness already is + +- A **portable harness** focused on context engineering, intent engineering, handoff flow, and state schema (not a monolithic runtime framework). +- Explicitly positioned as **model-agnostic / platform-portable patterns** with Cursor-centric defaults but portable core primitives. +- Uses a **verify-not-trust** philosophy with script/YAML parity checks and canonical bundle hashing. + +### Core architectural pillars in-repo + +1. **Intent-first operations** (intent, scope, constraints, human gate, latency). +2. **Structured memory/state schema** (`state/`) for continuity across sessions and agents. +3. **Document-then-continue handoff protocol** with explicit Done/Next and archive rules. +4. **Operational guardrails** via checklists, scripts, and CI verification. +5. **Async / HITL compatibility** through machine-readable task ledger (`async_tasks.yaml`) and ownership semantics. + +### Agnostic + swarm-readiness signals already present + +- Async multi-session orchestration concepts exist (`docs/ASYNC_HITL_SCOPE.md`, `state/async_tasks.yaml`). +- Intent schema and human-gate semantics are already designed for delegated, multi-step work. +- Delineation docs keep core harness independent from app-specific implementations. + +## Where we are in the development process + +## Phase assessment + +OpenHarness appears to be in a **mature architecture/specification hardening phase** for a public reference harness, with: + +- Strong core docs and schemas in place. +- Foundational verification scripts in place. +- Public-safe placeholders for operational state. +- A small backlog item focused on parity tooling (thin MCP wrapper). + +Not yet a full “batteries-included” swarm runtime; currently a **governance + process substrate** that can host one. + +## Current status indicators + +- Latest handoff (2026-03-26) indicates async/HITL roadmap artifacts were added and verification integrated. +- Backlog has one explicit future item: thin MCP wrapper for allowlisted script invocation. +- Decision and known-issues logs are still intentionally minimal/public-safe placeholders. + +## Gaps relative to your stated north star + +To fully satisfy “agnostic template for operating AI agents and AI swarm decision-making,” the largest gaps are: + +1. **Reference swarm control loop spec** + - Need a canonical pattern for planner/critic/executor/referee roles and arbitration strategy. + +2. **Swarm decision protocol contract** + - Need machine-readable schema for proposals, votes/scores, tie-breakers, and confidence/risk annotations. + +3. **Policy profiles by risk tier** + - Need preset governance profiles (low/medium/high stakes) mapping human gates and escalation triggers. + +4. **Interoperability adapters** + - Need lightweight adapters/examples for non-Cursor environments while preserving core schema. + +5. **Evaluation harness for collective quality** + - Need benchmark tasks and acceptance metrics for single-agent vs multi-agent outcomes. + +## Recommended next steps (ordered) + +1. **Define v0.1 Swarm Decision Contract** + - Add `docs/contracts/swarm_decision_v0_1.md` (proposal schema, scoring fields, arbitration fields, final-decision envelope). + +2. **Publish a canonical “Swarm Operating Modes” doc** + - Add `docs/SWARM_OPERATING_MODES.md` covering at least: + - single-agent + - planner/executor + - planner/critic/executor + - N-agent quorum with referee + +3. **Extend state schema for swarm runs** + - Add optional artifacts like `state/swarm_runs/YYYY-MM-DD/.md` and/or YAML summaries compatible with existing handoff flow. + +4. **Add verification script for swarm artifacts** + - Add `scripts/verify_swarm_contract.py` to enforce required fields and prevent silent format drift. + +5. **Add 2-3 synthetic end-to-end examples** + - Include one low-risk and one high-risk scenario with explicit human gate behavior and async handoff continuity. + +6. **Create roadmap milestones** + - Convert the above into milestones (M1 schema, M2 examples, M3 verification, M4 adapters). + +## Alignment decisions (captured) + +Resolved product decisions: + +1. **Primary audience first:** Optimize first for **solo builders**. +2. **Swarm topology baseline:** Adopt **one default topology**. +3. **Decision semantics:** Use **referee override** as the default final decision rule. +4. **Risk posture:** For high-stakes use, **irreversible actions always require a human gate**. +5. **Execution scope:** Include a **minimal runnable orchestrator reference** in OpenHarness. + +Still-open strategic questions: + +1. **Platform priority:** Which non-Cursor target should come first (CLI-only, GitHub Actions, LangGraph, OpenAI Responses tools, others)? +2. **Success metric:** What defines “v1 ready” first—adoption, reliability, eval gains, or governance completeness? + +## Suggested immediate decision + +Given the resolved decisions above, recommended immediate spike order: + +1. **A:** Swarm Decision Contract first (schemas + validation). +2. **B:** Swarm Operating Modes (single default topology with referee override encoded as default policy). +3. **C:** Minimal runnable orchestrator reference that enforces mandatory human gate for high-stakes irreversible actions. + +After A/B/C, draft the exact file set and acceptance tests for a “solo-builder-first v0.1” release slice. diff --git a/docs/SWARM_OPERATING_MODES.md b/docs/SWARM_OPERATING_MODES.md new file mode 100644 index 0000000..6b03f1d --- /dev/null +++ b/docs/SWARM_OPERATING_MODES.md @@ -0,0 +1,46 @@ +# Swarm Operating Modes (v0.1) + +This document defines portable operating modes for OpenHarness. For v0.1, the default mode is: + +- `planner_critic_executor_referee` +- default decision rule: `referee_override` + +## Mode catalog + +## 1) `single_agent` + +- One agent executes end-to-end. +- Use for low-complexity, low-risk tasks. + +## 2) `planner_executor` + +- Planner proposes steps; executor implements. +- Use for medium complexity when critique overhead is unnecessary. + +## 3) `planner_critic_executor` + +- Critic reviews plan/implementation before finalization. +- Use when quality assurance is needed but formal arbitration is not. + +## 4) `planner_critic_executor_referee` (default) + +- Planner proposes options. +- Critic identifies risk and weakness. +- Executor provides implementation feasibility and constraints. +- Referee performs final arbitration and may override by policy. + +## High-stakes policy + +If an action is both high-stakes and irreversible: + +1. set `human_gate_required: true` +2. block finalization until approval is recorded +3. persist approval reference in run artifacts and handoff + +## Minimal run lifecycle + +1. Initialize run artifact (`state/swarm_runs/...`). +2. Collect proposals and critiques. +3. Referee arbitration and tentative winner. +4. Enforce human gate policy if applicable. +5. Finalize decision, write audit trail, append handoff note. diff --git a/docs/contracts/swarm_decision_v0_1.md b/docs/contracts/swarm_decision_v0_1.md new file mode 100644 index 0000000..e9cc989 --- /dev/null +++ b/docs/contracts/swarm_decision_v0_1.md @@ -0,0 +1,91 @@ +# Swarm Decision Contract v0.1 (synthetic/public) + +Status: draft-v0.1 + +Purpose: provide a portable, machine-readable envelope for multi-agent proposals, scoring, arbitration, and final decision output. + +## Decision envelope + +Required top-level fields: + +- `contract_version`: `swarm_decision_v0_1` +- `run_id`: unique id for one swarm decision run +- `intent`: one-line target objective +- `mode`: operating mode id (see `docs/SWARM_OPERATING_MODES.md`) +- `decision_rule`: `referee_override` (default policy) +- `high_stakes`: boolean +- `irreversible_action`: boolean +- `human_gate_required`: boolean +- `status`: `draft | finalized | blocked_human_gate | aborted` +- `proposals`: list of agent proposals +- `arbitration`: referee evaluation + rationale +- `final_decision`: selected proposal + reason + confidence +- `audit`: timestamps and provenance + +## Proposal object + +Each entry in `proposals[]` must include: + +- `proposal_id` +- `agent_id` +- `summary` +- `plan_steps` (ordered list) +- `risks` (list) +- `confidence` (0.0-1.0) +- `evidence` (paths/links) +- `constraints_checked` (list) + +## Arbitration object + +Required fields: + +- `referee_agent_id` +- `scores` (map: proposal_id -> score) +- `winner_proposal_id` +- `override_applied` (boolean) +- `rationale` +- `dissent_notes` (optional list) + +## Human gate policy + +Normative rule for v0.1: + +- If `high_stakes=true` and `irreversible_action=true`, then `human_gate_required` MUST be `true`. +- When `human_gate_required=true`, `status` MUST be `blocked_human_gate` until approval is recorded. + +## YAML example (minimal) + +```yaml +contract_version: swarm_decision_v0_1 +run_id: swarm-20260411-001 +intent: Select rollout strategy for docs-only migration. +mode: planner_critic_executor_referee +decision_rule: referee_override +high_stakes: false +irreversible_action: false +human_gate_required: false +status: finalized +proposals: + - proposal_id: p1 + agent_id: planner + summary: Stage changes over two weekly releases. + plan_steps: ["prepare", "announce", "rollout"] + risks: ["schedule slip"] + confidence: 0.74 + evidence: ["docs/BACKLOG.md"] + constraints_checked: ["public-safe", "no-secrets"] +arbitration: + referee_agent_id: referee + scores: {p1: 0.82} + winner_proposal_id: p1 + override_applied: false + rationale: Best tradeoff of risk and speed. +final_decision: + selected_proposal_id: p1 + reason: Highest referee score with acceptable risk. + confidence: 0.82 +audit: + created_at: 2026-04-11T00:00:00Z + finalized_at: 2026-04-11T00:15:00Z + sources: ["docs/SWARM_OPERATING_MODES.md"] +``` diff --git a/docs/examples/SWARM_RUN_HIGH_RISK_SYNTHETIC.md b/docs/examples/SWARM_RUN_HIGH_RISK_SYNTHETIC.md new file mode 100644 index 0000000..6bd60f1 --- /dev/null +++ b/docs/examples/SWARM_RUN_HIGH_RISK_SYNTHETIC.md @@ -0,0 +1,15 @@ +# Synthetic swarm run example — high risk + +- Run id: `swarm-20260411-highrisk` +- Intent: evaluate irreversible data purge +- Mode: `planner_critic_executor_referee` +- Decision rule: `referee_override` +- Human gate: required (`high_stakes=true`, `irreversible_action=true`) + +## Outcome + +Referee produced a tentative winner, but the run remains `blocked_human_gate` pending explicit approval. + +## Artifact + +- `state/swarm_runs/2026-04-11/swarm-20260411-highrisk.yaml` diff --git a/docs/examples/SWARM_RUN_LOW_RISK_SYNTHETIC.md b/docs/examples/SWARM_RUN_LOW_RISK_SYNTHETIC.md new file mode 100644 index 0000000..b81b9fa --- /dev/null +++ b/docs/examples/SWARM_RUN_LOW_RISK_SYNTHETIC.md @@ -0,0 +1,15 @@ +# Synthetic swarm run example — low risk + +- Run id: `swarm-20260411-lowrisk` +- Intent: choose release-note style +- Mode: `planner_critic_executor_referee` +- Decision rule: `referee_override` +- Human gate: not required (`high_stakes=false`, `irreversible_action=false`) + +## Outcome + +Referee selected `p1`; status finalized. + +## Artifact + +- `state/swarm_runs/2026-04-11/swarm-20260411-lowrisk.yaml` diff --git a/docs/examples/SWARM_RUN_QUORUM_NOTE_SYNTHETIC.md b/docs/examples/SWARM_RUN_QUORUM_NOTE_SYNTHETIC.md new file mode 100644 index 0000000..733ae8f --- /dev/null +++ b/docs/examples/SWARM_RUN_QUORUM_NOTE_SYNTHETIC.md @@ -0,0 +1,6 @@ +# Synthetic swarm note — quorum variant (non-default) + +This synthetic note documents a non-default pattern where multiple critics submit scores before referee arbitration. + +- Non-default topology: quorum critics + referee final call +- Kept as an extension note; default mode remains `planner_critic_executor_referee`. diff --git a/scripts/verify_swarm_contract.py b/scripts/verify_swarm_contract.py new file mode 100644 index 0000000..8484c38 --- /dev/null +++ b/scripts/verify_swarm_contract.py @@ -0,0 +1,102 @@ +#!/usr/bin/env python3 +"""Validate swarm run YAML artifacts against OpenHarness v0.1 contract requirements. + +Note: this verifier intentionally avoids third-party dependencies so it can run in +minimal environments. +""" + +from __future__ import annotations + +from pathlib import Path +import sys + +REPO_ROOT = Path(__file__).resolve().parents[1] +SWARM_ROOT = REPO_ROOT / "state" / "swarm_runs" + +REQUIRED_TOP_LEVEL = { + "contract_version", + "run_id", + "intent", + "mode", + "decision_rule", + "high_stakes", + "irreversible_action", + "human_gate_required", + "status", + "proposals", + "arbitration", + "final_decision", + "audit", +} + + +def validate_yaml(path: Path) -> list[str]: + errors: list[str] = [] + text = path.read_text(encoding="utf-8") + top_keys = set() + top_values: dict[str, str] = {} + for raw_line in text.splitlines(): + if not raw_line or raw_line.startswith(" ") or raw_line.startswith("\t"): + continue + if ":" not in raw_line: + continue + key, value = raw_line.split(":", 1) + key = key.strip() + if not key: + continue + top_keys.add(key) + top_values[key] = value.strip() + + missing = sorted(REQUIRED_TOP_LEVEL - top_keys) + if missing: + errors.append(f"{path}: missing required fields: {', '.join(missing)}") + + if top_values.get("contract_version") != "swarm_decision_v0_1": + errors.append(f"{path}: contract_version must be 'swarm_decision_v0_1'") + + if top_values.get("decision_rule") != "referee_override": + errors.append(f"{path}: decision_rule must be 'referee_override' for v0.1") + + high_stakes = top_values.get("high_stakes") == "true" + irreversible = top_values.get("irreversible_action") == "true" + gate = top_values.get("human_gate_required") == "true" + status = top_values.get("status") + + if high_stakes and irreversible and not gate: + errors.append( + f"{path}: human_gate_required must be true when high_stakes and irreversible_action are true" + ) + + if gate and status not in {"blocked_human_gate", "finalized"}: + errors.append( + f"{path}: status must be blocked_human_gate or finalized when human_gate_required is true" + ) + + if top_values.get("proposals") is None: + errors.append(f"{path}: proposals must be a non-empty list") + + return errors + + +def main() -> int: + yaml_files = sorted(SWARM_ROOT.rglob("*.yaml")) + if not yaml_files: + print(f"No swarm YAML artifacts found under {SWARM_ROOT}") + return 0 + + errors: list[str] = [] + for file_path in yaml_files: + errors.extend(validate_yaml(file_path)) + + if errors: + print("Swarm contract verification failed:") + for err in errors: + print(f"- {err}") + return 1 + + print(f"Swarm contract verification passed ({len(yaml_files)} files).") + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/state/README.md b/state/README.md index a7034ef..eed5266 100644 --- a/state/README.md +++ b/state/README.md @@ -22,6 +22,7 @@ Shared agent and project memory. Append-only or append-mostly. Use this schema s | intent_surface.md | Optional canonical intent for multi-phase work | | continue_prompt.txt | Canonical continue-from-handoff prompt | | async_tasks.yaml | Machine-readable task ledger (ids, status, owner); validated in CI; see [docs/ASYNC_HITL_SCOPE.md](../docs/ASYNC_HITL_SCOPE.md) | +| swarm_runs/YYYY-MM-DD/*.yaml | Optional machine-readable swarm decision artifacts (synthetic/public examples) | ## Handoff Schema @@ -65,6 +66,13 @@ When `latency_tolerance: async_ok` (or whenever multiple sessions may overlap), See [docs/HANDOFF_FLOW.md](../docs/HANDOFF_FLOW.md) and [docs/INTENT_ENGINEERING.md](../docs/INTENT_ENGINEERING.md). +## swarm_runs (optional) + +- Use for multi-agent/swarm decision records that need machine-readable auditing. +- Contract reference: [docs/contracts/swarm_decision_v0_1.md](../docs/contracts/swarm_decision_v0_1.md). +- Schema helper and examples: [state/swarm_runs/README.md](swarm_runs/README.md). +- Verifier: `python scripts/verify_swarm_contract.py`. + ## decision-log.md - **Section per date:** `## YYYY-MM-DD` diff --git a/state/swarm_runs/2026-04-11/swarm-20260411-highrisk.yaml b/state/swarm_runs/2026-04-11/swarm-20260411-highrisk.yaml new file mode 100644 index 0000000..f0e7b80 --- /dev/null +++ b/state/swarm_runs/2026-04-11/swarm-20260411-highrisk.yaml @@ -0,0 +1,32 @@ +contract_version: swarm_decision_v0_1 +run_id: swarm-20260411-highrisk +intent: Approve irreversible production data purge. +mode: planner_critic_executor_referee +decision_rule: referee_override +high_stakes: true +irreversible_action: true +human_gate_required: true +status: blocked_human_gate +proposals: + - proposal_id: p2 + agent_id: planner + summary: Purge legacy data after 30-day archive snapshot. + plan_steps: ["snapshot", "approval", "purge"] + risks: ["irrecoverable deletion", "compliance"] + confidence: 0.62 + evidence: ["docs/INTENT_ENGINEERING.md"] + constraints_checked: ["human gate required", "audit trail"] +arbitration: + referee_agent_id: referee + scores: {p2: 0.69} + winner_proposal_id: p2 + override_applied: true + rationale: Acceptable only with explicit human approval. +final_decision: + selected_proposal_id: p2 + reason: Blocked pending human authorization. + confidence: 0.69 +audit: + created_at: 2026-04-11T02:00:00Z + finalized_at: null + sources: ["docs/SWARM_OPERATING_MODES.md", "docs/contracts/swarm_decision_v0_1.md"] diff --git a/state/swarm_runs/2026-04-11/swarm-20260411-lowrisk.yaml b/state/swarm_runs/2026-04-11/swarm-20260411-lowrisk.yaml new file mode 100644 index 0000000..78a7fa1 --- /dev/null +++ b/state/swarm_runs/2026-04-11/swarm-20260411-lowrisk.yaml @@ -0,0 +1,32 @@ +contract_version: swarm_decision_v0_1 +run_id: swarm-20260411-lowrisk +intent: Pick a release-note wording style. +mode: planner_critic_executor_referee +decision_rule: referee_override +high_stakes: false +irreversible_action: false +human_gate_required: false +status: finalized +proposals: + - proposal_id: p1 + agent_id: planner + summary: Keep short release notes in bullets. + plan_steps: ["draft", "review", "publish"] + risks: ["less detail"] + confidence: 0.78 + evidence: ["docs/CHEATSHEET.md"] + constraints_checked: ["public-safe"] +arbitration: + referee_agent_id: referee + scores: {p1: 0.85} + winner_proposal_id: p1 + override_applied: false + rationale: Most readable with low risk. +final_decision: + selected_proposal_id: p1 + reason: Highest score and simplest rollout. + confidence: 0.85 +audit: + created_at: 2026-04-11T01:00:00Z + finalized_at: 2026-04-11T01:08:00Z + sources: ["docs/SWARM_OPERATING_MODES.md"] diff --git a/state/swarm_runs/README.md b/state/swarm_runs/README.md new file mode 100644 index 0000000..42f6a4f --- /dev/null +++ b/state/swarm_runs/README.md @@ -0,0 +1,30 @@ +# Swarm run artifacts (synthetic/public schema) + +Store machine-readable summaries for swarm decisions. + +## Layout + +- `state/swarm_runs/YYYY-MM-DD/.yaml` +- Optional narrative companion: `state/swarm_runs/YYYY-MM-DD/.md` + +## YAML required fields (v0.1) + +- `contract_version` +- `run_id` +- `intent` +- `mode` +- `decision_rule` +- `high_stakes` +- `irreversible_action` +- `human_gate_required` +- `status` +- `proposals` +- `arbitration` +- `final_decision` +- `audit` + +## Policy constraint + +When `high_stakes: true` and `irreversible_action: true`, then `human_gate_required` must be `true`. + +Use `python scripts/verify_swarm_contract.py` to validate YAML artifacts.