Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions capabilities.harness.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,7 @@ harness_capability:
- list_capabilities.py
- verify_async_tasks.py
- verify_contract_hash.py
- verify_swarm_contract.py
- verify_script_index.py
- verify_skills_readme.py

Expand Down
7 changes: 7 additions & 0 deletions docs/BACKLOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,3 +11,10 @@ Optional and future work that is **not** required for the core portable harness.
**Status:** Not implemented. Security, allowlisting, and host configuration should be documented alongside [MCP_PRIVATE_HOST.md](MCP_PRIVATE_HOST.md) when this is built.

**Scope guardrails (from alignment doc):** No new business logic in the server—thin wrapper over existing scripts only.

## Swarm-readiness milestones (solo-builder-first)

- **M1 — Swarm decision contract:** establish `swarm_decision_v0_1` fields and policy defaults.
- **M2 — Operating modes + examples:** publish default topology and synthetic low/high risk examples.
- **M3 — Verification:** enforce contract and policy constraints via script checks.
- **M4 — Minimal orchestrator reference:** provide a small runnable reference flow that implements referee override and mandatory high-stakes human gate.
1 change: 1 addition & 0 deletions docs/CHEATSHEET.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,7 @@ Use this table for **action parity** with humans (same script, agent via `run_te
| `list_capabilities.py` | Emit harness manifest (`capabilities.harness.yaml`) as JSON for scripts/checklist discovery | `python scripts/list_capabilities.py` |
| `verify_async_tasks.py` | Validate `state/async_tasks.yaml` task ledger schema | `python scripts/verify_async_tasks.py` |
| `verify_contract_hash.py` | Verify `docs/contracts/scp_mcp_v1.md` SHA-256 matches `docs/contracts/scp_mcp_v1.sha256` | `python scripts/verify_contract_hash.py` |
| `verify_swarm_contract.py` | Validate `state/swarm_runs/**/*.yaml` against swarm contract v0.1 policy fields | `python scripts/verify_swarm_contract.py` |
| `verify_script_index.py` | Parity: `capabilities.harness.yaml` `scripts[]` == on-disk scripts; each basename in this table | `python scripts/verify_script_index.py` |
| `verify_skills_readme.py` | `.cursor/skills/README.md` table vs each `SKILL.md` `description:` | `python scripts/verify_skills_readme.py` |
| `brain_map_viewer.html` | Local HTML viewer for graph JSON (vis-network; drag-drop or co-located `brain-map-graph.json`) | Open in browser; optional dev aid. **Operator-facing map (canonical UI in OpenGrimoire):** [OpenGrimoire `docs/GUI_ACTION_MAP_BRAIN_MAP.md`](../../OpenGrimoire/docs/GUI_ACTION_MAP_BRAIN_MAP.md) |
Expand Down
114 changes: 114 additions & 0 deletions docs/OPENHARNESS_REVIEW_2026-04-11.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,114 @@
# OpenHarness review — intent, context, process status (2026-04-11)

## Goal under review

Target product goal (from user directive): **OpenHarness as an agnostic template for operating AI agents and AI swarm decision-making processes**.

## Verified intent and context

### What OpenHarness already is

- A **portable harness** focused on context engineering, intent engineering, handoff flow, and state schema (not a monolithic runtime framework).
- Explicitly positioned as **model-agnostic / platform-portable patterns** with Cursor-centric defaults but portable core primitives.
- Uses a **verify-not-trust** philosophy with script/YAML parity checks and canonical bundle hashing.

### Core architectural pillars in-repo

1. **Intent-first operations** (intent, scope, constraints, human gate, latency).
2. **Structured memory/state schema** (`state/`) for continuity across sessions and agents.
3. **Document-then-continue handoff protocol** with explicit Done/Next and archive rules.
4. **Operational guardrails** via checklists, scripts, and CI verification.
5. **Async / HITL compatibility** through machine-readable task ledger (`async_tasks.yaml`) and ownership semantics.

### Agnostic + swarm-readiness signals already present

- Async multi-session orchestration concepts exist (`docs/ASYNC_HITL_SCOPE.md`, `state/async_tasks.yaml`).
- Intent schema and human-gate semantics are already designed for delegated, multi-step work.
- Delineation docs keep core harness independent from app-specific implementations.

## Where we are in the development process

## Phase assessment

OpenHarness appears to be in a **mature architecture/specification hardening phase** for a public reference harness, with:

- Strong core docs and schemas in place.
- Foundational verification scripts in place.
- Public-safe placeholders for operational state.
- A small backlog item focused on parity tooling (thin MCP wrapper).

Not yet a full “batteries-included” swarm runtime; currently a **governance + process substrate** that can host one.

## Current status indicators

- Latest handoff (2026-03-26) indicates async/HITL roadmap artifacts were added and verification integrated.
- Backlog has one explicit future item: thin MCP wrapper for allowlisted script invocation.
- Decision and known-issues logs are still intentionally minimal/public-safe placeholders.

## Gaps relative to your stated north star

To fully satisfy “agnostic template for operating AI agents and AI swarm decision-making,” the largest gaps are:

1. **Reference swarm control loop spec**
- Need a canonical pattern for planner/critic/executor/referee roles and arbitration strategy.

2. **Swarm decision protocol contract**
- Need machine-readable schema for proposals, votes/scores, tie-breakers, and confidence/risk annotations.

3. **Policy profiles by risk tier**
- Need preset governance profiles (low/medium/high stakes) mapping human gates and escalation triggers.

4. **Interoperability adapters**
- Need lightweight adapters/examples for non-Cursor environments while preserving core schema.

5. **Evaluation harness for collective quality**
- Need benchmark tasks and acceptance metrics for single-agent vs multi-agent outcomes.

## Recommended next steps (ordered)

1. **Define v0.1 Swarm Decision Contract**
- Add `docs/contracts/swarm_decision_v0_1.md` (proposal schema, scoring fields, arbitration fields, final-decision envelope).

2. **Publish a canonical “Swarm Operating Modes” doc**
- Add `docs/SWARM_OPERATING_MODES.md` covering at least:
- single-agent
- planner/executor
- planner/critic/executor
- N-agent quorum with referee

3. **Extend state schema for swarm runs**
- Add optional artifacts like `state/swarm_runs/YYYY-MM-DD/<run-id>.md` and/or YAML summaries compatible with existing handoff flow.

4. **Add verification script for swarm artifacts**
- Add `scripts/verify_swarm_contract.py` to enforce required fields and prevent silent format drift.

5. **Add 2-3 synthetic end-to-end examples**
- Include one low-risk and one high-risk scenario with explicit human gate behavior and async handoff continuity.

6. **Create roadmap milestones**
- Convert the above into milestones (M1 schema, M2 examples, M3 verification, M4 adapters).

## Alignment decisions (captured)

Resolved product decisions:

1. **Primary audience first:** Optimize first for **solo builders**.
2. **Swarm topology baseline:** Adopt **one default topology**.
3. **Decision semantics:** Use **referee override** as the default final decision rule.
4. **Risk posture:** For high-stakes use, **irreversible actions always require a human gate**.
5. **Execution scope:** Include a **minimal runnable orchestrator reference** in OpenHarness.

Still-open strategic questions:

1. **Platform priority:** Which non-Cursor target should come first (CLI-only, GitHub Actions, LangGraph, OpenAI Responses tools, others)?
2. **Success metric:** What defines “v1 ready” first—adoption, reliability, eval gains, or governance completeness?

## Suggested immediate decision

Given the resolved decisions above, recommended immediate spike order:

1. **A:** Swarm Decision Contract first (schemas + validation).
2. **B:** Swarm Operating Modes (single default topology with referee override encoded as default policy).
3. **C:** Minimal runnable orchestrator reference that enforces mandatory human gate for high-stakes irreversible actions.

After A/B/C, draft the exact file set and acceptance tests for a “solo-builder-first v0.1” release slice.
46 changes: 46 additions & 0 deletions docs/SWARM_OPERATING_MODES.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
# Swarm Operating Modes (v0.1)

This document defines portable operating modes for OpenHarness. For v0.1, the default mode is:

- `planner_critic_executor_referee`
- default decision rule: `referee_override`

## Mode catalog

## 1) `single_agent`

- One agent executes end-to-end.
- Use for low-complexity, low-risk tasks.

## 2) `planner_executor`

- Planner proposes steps; executor implements.
- Use for medium complexity when critique overhead is unnecessary.

## 3) `planner_critic_executor`

- Critic reviews plan/implementation before finalization.
- Use when quality assurance is needed but formal arbitration is not.

## 4) `planner_critic_executor_referee` (default)

- Planner proposes options.
- Critic identifies risk and weakness.
- Executor provides implementation feasibility and constraints.
- Referee performs final arbitration and may override by policy.

## High-stakes policy

If an action is both high-stakes and irreversible:

1. set `human_gate_required: true`
2. block finalization until approval is recorded
3. persist approval reference in run artifacts and handoff

## Minimal run lifecycle

1. Initialize run artifact (`state/swarm_runs/...`).
2. Collect proposals and critiques.
3. Referee arbitration and tentative winner.
4. Enforce human gate policy if applicable.
5. Finalize decision, write audit trail, append handoff note.
91 changes: 91 additions & 0 deletions docs/contracts/swarm_decision_v0_1.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
# Swarm Decision Contract v0.1 (synthetic/public)

Status: draft-v0.1

Purpose: provide a portable, machine-readable envelope for multi-agent proposals, scoring, arbitration, and final decision output.

## Decision envelope

Required top-level fields:

- `contract_version`: `swarm_decision_v0_1`
- `run_id`: unique id for one swarm decision run
- `intent`: one-line target objective
- `mode`: operating mode id (see `docs/SWARM_OPERATING_MODES.md`)
- `decision_rule`: `referee_override` (default policy)
- `high_stakes`: boolean
- `irreversible_action`: boolean
- `human_gate_required`: boolean
- `status`: `draft | finalized | blocked_human_gate | aborted`
- `proposals`: list of agent proposals
- `arbitration`: referee evaluation + rationale
- `final_decision`: selected proposal + reason + confidence
- `audit`: timestamps and provenance

## Proposal object

Each entry in `proposals[]` must include:

- `proposal_id`
- `agent_id`
- `summary`
- `plan_steps` (ordered list)
- `risks` (list)
- `confidence` (0.0-1.0)
- `evidence` (paths/links)
- `constraints_checked` (list)

## Arbitration object

Required fields:

- `referee_agent_id`
- `scores` (map: proposal_id -> score)
- `winner_proposal_id`
- `override_applied` (boolean)
- `rationale`
- `dissent_notes` (optional list)

## Human gate policy

Normative rule for v0.1:

- If `high_stakes=true` and `irreversible_action=true`, then `human_gate_required` MUST be `true`.
- When `human_gate_required=true`, `status` MUST be `blocked_human_gate` until approval is recorded.

## YAML example (minimal)

```yaml
contract_version: swarm_decision_v0_1
run_id: swarm-20260411-001
intent: Select rollout strategy for docs-only migration.
mode: planner_critic_executor_referee
decision_rule: referee_override
high_stakes: false
irreversible_action: false
human_gate_required: false
status: finalized
proposals:
- proposal_id: p1
agent_id: planner
summary: Stage changes over two weekly releases.
plan_steps: ["prepare", "announce", "rollout"]
risks: ["schedule slip"]
confidence: 0.74
evidence: ["docs/BACKLOG.md"]
constraints_checked: ["public-safe", "no-secrets"]
arbitration:
referee_agent_id: referee
scores: {p1: 0.82}
winner_proposal_id: p1
override_applied: false
rationale: Best tradeoff of risk and speed.
final_decision:
selected_proposal_id: p1
reason: Highest referee score with acceptable risk.
confidence: 0.82
audit:
created_at: 2026-04-11T00:00:00Z
finalized_at: 2026-04-11T00:15:00Z
sources: ["docs/SWARM_OPERATING_MODES.md"]
```
15 changes: 15 additions & 0 deletions docs/examples/SWARM_RUN_HIGH_RISK_SYNTHETIC.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# Synthetic swarm run example — high risk

- Run id: `swarm-20260411-highrisk`
- Intent: evaluate irreversible data purge
- Mode: `planner_critic_executor_referee`
- Decision rule: `referee_override`
- Human gate: required (`high_stakes=true`, `irreversible_action=true`)

## Outcome

Referee produced a tentative winner, but the run remains `blocked_human_gate` pending explicit approval.

## Artifact

- `state/swarm_runs/2026-04-11/swarm-20260411-highrisk.yaml`
15 changes: 15 additions & 0 deletions docs/examples/SWARM_RUN_LOW_RISK_SYNTHETIC.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# Synthetic swarm run example — low risk

- Run id: `swarm-20260411-lowrisk`
- Intent: choose release-note style
- Mode: `planner_critic_executor_referee`
- Decision rule: `referee_override`
- Human gate: not required (`high_stakes=false`, `irreversible_action=false`)

## Outcome

Referee selected `p1`; status finalized.

## Artifact

- `state/swarm_runs/2026-04-11/swarm-20260411-lowrisk.yaml`
6 changes: 6 additions & 0 deletions docs/examples/SWARM_RUN_QUORUM_NOTE_SYNTHETIC.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# Synthetic swarm note — quorum variant (non-default)

This synthetic note documents a non-default pattern where multiple critics submit scores before referee arbitration.

- Non-default topology: quorum critics + referee final call
- Kept as an extension note; default mode remains `planner_critic_executor_referee`.
Loading
Loading