Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .github/workflows/publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -49,4 +49,6 @@ jobs:

- run: pnpm install --frozen-lockfile
- run: pnpm release:check
env:
HUMANISH_PUBLIC_DENYLIST_PATTERN: ${{ secrets.HUMANISH_PUBLIC_DENYLIST_PATTERN }}
- run: npm publish --access public
64 changes: 33 additions & 31 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -121,11 +121,11 @@ npx humanish watch --json --no-open
| `humanish lab preflight <lab>` | Check lab routing and optional target reachability before actor/model spend. |
| `humanish lab run <lab>` | Run a lab manifest in human or JSON mode. |
| `humanish verify` | Validate a run bundle and public-safety gates. |
| `humanish cleanup` | Clean resources explicitly recorded as owned by a run and write `cleanup.json`. |
| `humanish cleanup` | Inspect recorded resource evidence and write `cleanup.json`; stored IDs do not authorize provider mutation. |
| `humanish review` | Read review evidence for a run. |
| `humanish runs` | List local runs and latest pointers. |
| `humanish feedback issue` | Print a public-safe GitHub issue draft without API mutation. |
| `humanish lab run oss` | Repo-maintainer dogfood example: Observer-of-Observers for headed authorized-repo app setup attempts. |
| `humanish lab run oss` | Repo-maintainer contract example: dry-run Observer-of-Observers for authorized repo selections. |
| `humanish lab run oss-smoke` | Repo-maintainer dogfood example: disposable clone smoke test against public OSS repos. |

## Exit Codes
Expand Down Expand Up @@ -283,16 +283,22 @@ selected lane ids, and previous lane statuses; the source run's verdict is left
This is intentionally not automatic retry; a passing rerun is evidence of a
nondeterminism candidate, not permission to erase the original red lane.

**Run-owned cleanup.** Live providers can record exact owned resources in `run.json`.
After a run, reclaim only those resources and write a durable receipt:
**Run-owned cleanup.** Live providers can record resource evidence in `run.json`.
Stored bundle IDs are mutable evidence, not provider-mutation authority. The
cleanup command writes a durable inspection receipt until Humanish has a
verified resource-lease contract. Resources already recorded as killed become
`already_clean`; recorded live or unknown resources become `failed`, which
makes cleanup and verification fail closed:

```bash
npx humanish cleanup --run latest
npx humanish verify --run latest
```

Cleanup is exact-id only; Humanish does not enumerate or bulk-delete provider
accounts from this command.
Humanish does not enumerate or bulk-delete provider accounts from this command.
Same-process teardown uses trusted in-memory provider handles. The separate OSS
orphan sweep is maintainer-only, opt-in, and verifies provider metadata before
calling provider cleanup.

Trust note: `serve` commands run inside the disposable sandbox with the declared
subject env provisioned — the same trust class as a repo's package.json scripts.
Expand Down Expand Up @@ -358,43 +364,36 @@ and `waitForSelector`. Supported expectations are `text`, `selectorVisible`,

## Maintainer OSS Meta-Lab Example

This repository includes an experimental authorized-repo dogfood lab:
This repository includes a contract-only authorized-repo dogfood lab:

```bash
pnpm humanish -- watch oss
pnpm humanish -- lab run oss --repos CorentinTh/it-tools,drawdb-io/drawdb,maciekt07/TodoApp,lissy93/dashy
pnpm humanish -- lab run oss --dry-run --repos CorentinTh/it-tools,drawdb-io/drawdb,maciekt07/TodoApp,lissy93/dashy
```

Default lab targets are intentionally app/tool-like repos with visible,
locally runnable user surfaces. Avoid libraries and frameworks for public
dogfood unless the scenario is explicitly testing developer experience.

With `E2B_API_KEY` and `OPENAI_API_KEY` present, Humanish launches headed E2B
desktop lanes, uploads the local package tarball, clones each assigned
repository inside the sandbox, initializes Humanish, runs nested proof commands,
starts the target app when a runnable script is present, opens desktop/mobile
app windows plus the nested Observer in the sandbox browser, and starts a
nonblocking Codex actor attempt.
Install the optional desktop substrate first:

```bash
npm i -D @e2b/desktop
```

The contract-safe path for agents and CI is:
The bundled manifest defaults to dry-run and creates contract evidence without
cloning repos, launching a provider sandbox, or forwarding credentials. Use:

```bash
pnpm humanish -- lab run oss --dry-run --json --no-open
```

The `oss` lab accepts GitHub `owner/repo` slugs. Private repositories are
maintainer-only and should be supplied from ignored local lab manifests with an
authorized `GH_TOKEN` or `GITHUB_TOKEN` loaded via `--env-file`. When a GitHub
token is present, durable run artifacts redact repo labels by default; pass
`--no-redact-repos` only for public-safe repo selections. Live E2B stream URLs
are runtime-only for the attached Observer server and are not persisted to
`run.json` or `observer-data.json`. Local bundles remain ignored under
`.humanish/`; do not publish private screenshots, logs, or upstream details.
Live OSS meta-lab execution is unavailable until repository-derived instructions
have an isolated credential boundary. A live manifest fails closed with
`HUMANISH_OSS_META_LIVE_ISOLATION_REQUIRED` before callbacks, filesystem writes,
network access, or provider launch.

The `oss` lab accepts GitHub `owner/repo` slugs. A CLI `--repos` override redacts
repo labels in durable artifacts by default; pass `--no-redact-repos` only for a
public-safe selection. Dry-run does not access or clone repositories and does
not need or use private-repository credentials. Private-repository execution
remains unavailable while the live lane is disabled. Local bundles remain
ignored under `.humanish/`; do not publish private screenshots, logs, or
upstream details.

## Development

Expand All @@ -416,13 +415,16 @@ pnpm humanish:lab:list

## Docs

Start with the current safety and capability state. Dated design documents may
preserve historical mechanisms and carry explicit amendments near the top.

- [Current safety state and goals](docs/goals/current.md)
- [Ramp for future contributors and agents](docs/ramp/README.md)
- [Current goals](docs/goals/current.md)
- [Project layout](docs/architecture/project-layout.md)
- [Observer architecture](docs/architecture/observer.md)
- [Actor contract (pluggable harnesses)](docs/architecture/actor-contract.md)
- [State-driven executor (drive a local app, no E2B/vision)](docs/architecture/state-driven-executor.md)
- [OSS lab POC](docs/architecture/oss-lab-poc.md)
- [OSS lab design record (historical; see its current safety amendment)](docs/architecture/oss-lab-poc.md)
- [Feedback contract](docs/contracts/feedback.md)
- [Open-source install experience](docs/product/open-source-install-experience.md)
- [Self-driving harness principles](docs/principles/self-driving-harness.md)
Expand Down
8 changes: 8 additions & 0 deletions docs/architecture/oss-lab-poc.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,14 @@ Date: 2026-06-01
Status: implemented as an experimental repo-owned lab manifest plus
compatibility aliases.

Safety amendment (2026-07-14): beginning with `0.15.1`, the bundled `oss`
manifest is a contract-only dry-run. A direct live OSS meta-lab request fails
with `HUMANISH_OSS_META_LIVE_ISOLATION_REQUIRED` before callbacks, filesystem
or network side effects, credential forwarding, or provider launch. The
historical design and evidence description below is preserved as a record; it
is not current execution guidance. The separate `oss-smoke` clone/discard lane
remains available for public repositories.

## Decision

`humanish/labs/oss.yaml` is this repo's authorized-repo meta-simulation
Expand Down
8 changes: 7 additions & 1 deletion docs/contracts/policy.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,12 @@ Date: 2026-06-02
Status: v0 draft contract for credential, network, spend, redaction, and
assisted-run boundaries.

Safety amendment (2026-07-14): beginning with `0.15.1`, stored provider IDs are
evidence and never authorize core provider mutation. The bundled OSS meta-lab
is dry-run only; a live request fails before side effects until
repository-derived instructions have an isolated credential boundary. Any
historical live-OSS examples below do not override that fail-closed behavior.

## Purpose

Policy defines what a run may access, what it may persist, and what it may
Expand Down Expand Up @@ -74,7 +80,7 @@ for credentials.
| `local_only` | Localhost and loopback only. | Observer, local fixtures |
| `public_oss` | Public GitHub clone/fetch of owner/repo slugs only. | disposable OSS smoke |
| `authorized_private` | Token-backed clone/fetch of repos the maintainer is already authorized to access, with repo labels redacted by default. | local maintainer dogfood only |
| `provider_substrate` | Explicit provider substrate such as hosted desktop streams. | live OSS lab with keys |
| `provider_substrate` | Explicit provider substrate such as hosted desktop streams. | live routes with an isolated credential boundary and in-process resource handles |
| `custom_allowlist` | Adapter-declared public hosts. | target-specific adapters |

Synthetic fixture:
Expand Down
21 changes: 20 additions & 1 deletion docs/goals/current.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Current Goals

Status date: 2026-07-08 (rev 13)
Status date: 2026-07-14 (rev 14)

This page is the current public-safe operating goal for `humanish`. Keep it
short enough to reread before a coding session and concrete enough that future
Expand All @@ -16,6 +16,21 @@ Humanish should be the open-source CLI that lets a maintainer ask:
The answer should be observable, verifiable, public-safe, and easy to turn into
actionable feedback.

## Current Safety State (`0.15.1`)

- Managed run, Observer, feedback, lab, actor-output, and source-archive paths
bind to validated physical filesystem identities and fail closed on unsafe
traversal, link, special-file, or retargeting states.
- Provider IDs stored in `run.json` are mutable evidence, not cleanup
authority. `humanish cleanup` writes an inspection receipt; same-process
teardown continues to use the provider handles that created the resources.
- The bundled `oss` manifest defaults to dry-run. Live OSS meta-lab execution
fails with `HUMANISH_OSS_META_LIVE_ISOLATION_REQUIRED` before side effects
until repository-derived instructions have an isolated credential boundary.
- Ordinary Git repositories and verified linked worktrees remain supported.
Git metadata that cannot pass containment validation is recorded as
unavailable rather than followed.

## Definition Of Awesome

A world-class Humanish run should eventually provide:
Expand Down Expand Up @@ -341,6 +356,10 @@ Minimum acceptance:
Make the maintainer `oss` lab report nested lane health back into the
top-level Observer instead of relying on a human watching the desktops.

The `0.15.1` safety state above governs this lane. The completed bullets below
record prior capability and evidence shape; they do not mean the live
entrypoint is currently enabled.

Minimum acceptance:

- each lane records setup status; `done`
Expand Down
10 changes: 9 additions & 1 deletion docs/product/open-source-install-experience.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,14 @@ Date: 2026-06-01

Status: product target for the first world-class `humanish` implementation.

Safety amendment (2026-07-14): the `0.15.1` package binds managed run and
output storage to validated physical paths, treats provider IDs persisted in a
run bundle as evidence rather than cleanup authority, and disables live OSS
meta-lab execution until repository-derived instructions have an isolated
credential boundary. The historical product target below remains useful for
intent and sequencing, but current behavior is defined by the README and
[`docs/goals/current.md`](../goals/current.md).

## Product Promise

Drop Humanish into an app and let a coding agent set up realistic persona
Expand Down Expand Up @@ -171,7 +179,7 @@ Suggested scripts:
| `humanish lab inspect <lab>` | Read a lab manifest | Print the parsed lab config, origin, path, and warnings without executing |
| `humanish lab preflight <lab>` | Check lab readiness before spend | Validate routing and optionally probe declared targets from a hosted desktop without launching actors |
| `humanish lab run <lab>` | Run a lab manifest | Human or JSON execution path for synthetic, OSS meta, and smoke labs |
| `humanish lab run oss` | Maintainer dogfood example | Open the Observer-of-Observers with headed desktop lanes assigned by `--repos`, target app windows, nested Observers, runtime-only stream URLs, and redacted durable evidence for token-backed runs |
| `humanish lab run oss` | Maintainer contract example | Render a dry-run Observer-of-Observers contract for selected repo labels; live execution fails closed pending credential isolation |
| `humanish lab run oss-smoke` | Maintainer smoke example | Shallow clone lightweight GitHub repos, run setup/proof/verify, report, and remove clones |
| `humanish feedback issue` | Produce public-safe issue draft | Print Markdown or prefilled issue URL, no GitHub API mutation |

Expand Down
21 changes: 14 additions & 7 deletions docs/ramp/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,11 @@

Status: public-safe contributor and agent ramp.

Current safety state: `0.15.1` (2026-07-14). Managed run and output paths bind
to validated physical filesystem identities. Stored provider IDs are evidence,
not cleanup authority. The bundled OSS meta-lab is dry-run only until
repository-derived instructions have an isolated credential boundary.

Use this page when you are starting cold on `humanish`. It is meant to be
useful without chat history, private notes, local machine paths, or maintainer
context.
Expand Down Expand Up @@ -71,20 +76,22 @@ Implemented:
drives a lab-owner loopback app in a hosted desktop, and `subject.source:
clone` + `serve` clones, installs, and serves a real app in-sandbox from
config before the actor drives it (`src/cua-actor-lab.ts`);
- experimental maintainer OSS meta-lab and disposable OSS smoke harness;
- OSS dogfood setup-quality filesystem artifacts rendered from the Observer Files
tab with private-run previews suppressed by default.
- containment checks for managed run storage, Observer and feedback reads,
actor artifacts, lab discovery, Git metadata, and source archives;
- an OSS meta-lab dry-run contract and a separate disposable public-repo OSS
smoke harness;
- cleanup inspection receipts that do not treat mutable run-bundle IDs as
provider-mutation authority.

Still not good enough:

- live `--app-url` browser proof now supports app-specific executable
`browser.steps` in `humanish/scenarios/*.yaml`, with headed provider-backed
public proof against two app/tool targets;
- live PTY and Codex UI lanes need stronger completion health;
- OSS lab lanes can report nested Observer health, target app readiness, actor
evidence, setup-quality filesystem checks, and nested browser step summaries
when a target app starts, but need repeated fresh-agent trials across more
disposable public apps;
- live OSS meta-lab execution remains disabled until repository-derived
instructions have an isolated credential boundary; historical headed-lane
evidence does not make the current entrypoint available;
- Observer evidence has real screenshots/traces for browser app proof; richer
multi-step product journeys and broader multi-persona matrices remain the next
gap.
Expand Down
4 changes: 2 additions & 2 deletions humanish/labs/oss.yaml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
schema: humanish.lab.v2
id: oss
title: OSS meta-lab dogfood
description: Experimental Observer-of-Observers for authorized repos; useful for maintainer dogfood and disposable public app trials.
description: Contract-only Observer-of-Observers for authorized repo selections; live execution is gated pending credential isolation.
subject:
source: clone
repos:
Expand All @@ -16,6 +16,6 @@ actors:
execution:
target: e2b-desktop
scenario:
mode: live
mode: dry-run
defaults:
open: true
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "humanish",
"version": "0.15.0",
"version": "0.15.1",
"description": "Open-source-safe CLI for persona simulation, observer review, and public-safe feedback drafts.",
"author": "Daniel G Wilson <daniel@danielgwilson.com>",
"keywords": [
Expand Down
32 changes: 23 additions & 9 deletions src/claude-agent-sdk.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
import { mkdir, writeFile } from "node:fs/promises";
import path from "node:path";

import {
Expand All @@ -12,6 +11,13 @@ import {
type ActorTraceItem
} from "./actor-contract.js";
import { redactText } from "./redaction.js";
import {
prepareContainedOutputDirectory,
prepareContainedOutputFile,
prepareSelectedOutputDirectory,
type PreparedOutputDirectory,
writeContainedOutputFile
} from "./selected-output-paths.js";

// This module holds both halves of the Claude adapter:
// - the PURE mapper (claudeSessionToActorTrace) over a locally-declared
Expand Down Expand Up @@ -349,7 +355,7 @@ const CLAUDE_ARTIFACT_DIR = "claude-agent-sdk";
// Map the session through the pure mapper and write the three evidence artifacts.
// Shared by the normal path and the load-failure path so both always leave a bundle.
async function finishClaudeSession(
runRoot: string,
runRoot: PreparedOutputDirectory,
persona: ActorPersonaRef,
session: ClaudeSessionResult,
envelopeLines: string[]
Expand All @@ -359,10 +365,11 @@ async function finishClaudeSession(
const transcriptPath = path.join(CLAUDE_ARTIFACT_DIR, "transcript.txt");
const trace = claudeSessionToActorTrace(session, persona);
const transcript = renderClaudeTranscript(trace);
await writeFile(path.join(runRoot, eventsPath), envelopeLines.length > 0 ? `${envelopeLines.join("\n")}\n` : "", "utf8");
await writeFile(path.join(runRoot, tracePath), `${JSON.stringify(trace, null, 2)}\n`, "utf8");
await writeFile(
path.join(runRoot, transcriptPath),
await writeContainedOutputFile(runRoot, eventsPath, envelopeLines.length > 0 ? `${envelopeLines.join("\n")}\n` : "", "utf8");
await writeContainedOutputFile(runRoot, tracePath, `${JSON.stringify(trace, null, 2)}\n`, "utf8");
await writeContainedOutputFile(
runRoot,
transcriptPath,
transcript.length > 0 ? transcript : "No Claude Agent SDK transcript output captured.\n",
"utf8"
);
Expand All @@ -386,7 +393,14 @@ async function finishClaudeSession(
* timeout still produces a (failed/timed_out) bundle rather than throwing.
*/
export async function runClaudeAgentSession(options: ClaudeAgentSessionOptions): Promise<ClaudeAgentSessionResult> {
await mkdir(path.join(options.runRoot, CLAUDE_ARTIFACT_DIR), { recursive: true });
const preparedRunRoot = await prepareSelectedOutputDirectory(process.cwd(), options.runRoot);
const runRoot = preparedRunRoot;
await prepareContainedOutputDirectory(runRoot, CLAUDE_ARTIFACT_DIR);
await Promise.all([
prepareContainedOutputFile(runRoot, path.join(CLAUDE_ARTIFACT_DIR, "events.ndjson")),
prepareContainedOutputFile(runRoot, path.join(CLAUDE_ARTIFACT_DIR, "summary.json")),
prepareContainedOutputFile(runRoot, path.join(CLAUDE_ARTIFACT_DIR, "transcript.txt"))
]);
const startedAt = new Date().toISOString();
const startedMs = Date.now();

Expand All @@ -402,7 +416,7 @@ export async function runClaudeAgentSession(options: ClaudeAgentSessionOptions):
completedAt: new Date().toISOString(),
messages: [{ type: "result", subtype: "error_during_execution", is_error: true, duration_ms: Date.now() - startedMs, result: reason }]
};
return finishClaudeSession(options.runRoot, options.persona, session, [JSON.stringify({ at: startedAt, error: reason })]);
return finishClaudeSession(runRoot, options.persona, session, [JSON.stringify({ at: startedAt, error: reason })]);
}

const queryOptions: Record<string, unknown> = {
Expand Down Expand Up @@ -486,5 +500,5 @@ export async function runClaudeAgentSession(options: ClaudeAgentSessionOptions):
});
}

return finishClaudeSession(options.runRoot, options.persona, { messages, startedAt, completedAt }, envelopeLines);
return finishClaudeSession(runRoot, options.persona, { messages, startedAt, completedAt }, envelopeLines);
}
Loading