feat(ephemeral): add inert security groundwork for ephemeral agents

[fanhongy]

Summary

This adds the first security groundwork for the #801 ephemeral-agent support; the design is in #860. The implementation is inert: there is no creation tool and no writer of registry rows or live profile files, so nothing ephemeral can exist yet.

The branch is rebased onto recent main; the reserved-name refusal now covers the local-file and URL install paths added there since the published head.

What changes

• Reserves names matching ^[A-Z][A-Za-z0-9]{1,23}-[a-z][a-z0-9_]{2,31}-[0-9a-f]{4}$.
• Adds the launch-only loader load_launch_profile, which fails closed with EphemeralProfileUnavailable and never falls back to an installed or native profile. Live copies are parsed verbatim, without environment-variable expansion.
• Adds the public predicate agent_profiles.routes_to_ephemeral_store(name) -> bool as the one routing seam that [Feat] Decision points: opt-in, auditable judgments in orchestration, starting with Auto model routing #810 will call.
• Refuses reserved names for installed-profile reads, listing and named writes: every install source (bare name, local file, URL, POST /agents/profiles/install) is refused before anything is written, as are profile-store create, overwrite and replace operations and direct Copilot prompt refresh. Existing copies can still be deleted, and the server warns about them at startup.
• Copilot batch refresh logs and skips reserved files while refreshing the remaining agents. Named memory promotion refuses reserved names; explicit --profile-path behavior is unchanged. Literal reserved frontmatter names are refused before environment resolution; a name built from ${VAR}, or one in frontmatter that only becomes valid YAML after resolution, is refused after resolution.
• Skips plugin-MCP configuration for ephemeral launches on Claude Code and Codex.
• Adds the ephemeral_agents table with read accessors only; it is created additively and has no writer.
• Adds the registry-derived ephemeral terminal field, which metadata PATCH cannot change, and batches list reads.
• Refuses ephemeral callers for assign, handoff, assign_elastic, workflow_run, workflow_resume, and workflow_start. send_message remains allowed. The operator setting ephemeral.child_may_delegate lifts the refusal only when it is literal true, and a failed lookup refuses.
• Refuses local assign and handoff (both handoff paths) for ephemeral targets until claims exist. Remote placement (target_host, assign_elastic) fails closed on the target node, which has no live profile; for assign_elastic that happens after the broker has provisioned a worker. An explicit refusal before any remote call will be added with creation support. Installed names do not cause an extra profile read.
• Moves caller_effective_allowed_tools unchanged to utils/caller_tools.py.
• Terminal metadata reads and terminal creation check registry membership on the session they already hold, avoiding nested connection checkout under a file-backed QueuePool.

Behaviour changes for existing users

Exactly two existing-user behaviours change:

• Installed profiles addressed by names matching the reserved pattern can no longer be installed, created, overwritten, replaced, read, refreshed, promoted by name, launched or listed; existing copies can still be deleted, and the server warns about them at startup.
• Workflow run, resume, and start refuse when CAO_TERMINAL_ID is set but the calling terminal cannot be resolved.

Launch callers

The callers moved to load_launch_profile are create_terminal, resolve_provider, the Claude Code profile load, both Codex reads, and resolve_agent_profile_source. Every other provider and the skill-refresh readers keep load_agent_profile behind the reserved-name refusal. They are safe because create_terminal refuses before any provider initializes.

Deliberately not changed

A missing ephemeral key in terminal JSON reads as false. A new cao-mcp-server can talk to an older cao-server that is still running and returns JSON without the key. Reading it strictly would refuse delegation to every installed caller until that server restarts.

Not in this PR

create_ephemeral_agent, claims and finalize, the live-copy writer, generated names, and GC/TTL sweeps are not included. These are later slices described in #860.

Testing

• Black, isort and Markdown links pass.
• Mypy reports the same 151 existing errors on base and tip; normalized diagnostics are identical, with no new errors.
• Targeted review and regression files: 629 passed, 1 skipped; new, touched and related files have zero environment-file seam hits.
• Full suite: 13,546 passed on tip; the same four shared failures as base (13,338 passed):
  • test_packaged_mcp_server::test_an_unpublished_version_fails_the_build: the local validation interpreter cannot verify PyPI's TLS certificate ("unable to get local issuer certificate"), so the expected "not published" result cannot be produced.
  • test_packages::test_subdir_addressing_works_against_this_repository: the local sensitive-file refusal prevents reading a third-party certificate bundle.
  • test_kimi_code_compat::test_socket_is_skipped: the macOS AF_UNIX path limit under the default pytest temporary path.
  • test_constants::test_cao_home_dir_is_under_aws_cli_agent_orchestrator: the test assumes the default home path while validation deliberately uses an isolated directory.
• The first two failures come from the local validation environment, not this change: one cannot verify PyPI's TLS certificate locally, and the other is stopped by the local refusal to open certificate files. Both pass in CI on main (Python 3.10–3.12).
• Changed runtime lines: 182/182 covered (100%).
• Validation uses -W ignore::ResourceWarning to avoid Python 3.13 SQLite warning records retaining unclosed connections. This is a validation-only deviation from CI, with no file-descriptor limit changes or garbage-collection hooks.
• RED-first regressions cover batch refresh continuation, pre-resolution refusal, memory promotion, install imports and real QueuePool behavior. The single-connection pool tests fail on the previous implementation; 16 concurrent callers complete without nested checkout. Independent review mutations confirmed the earlier security regressions are caught.

Security notes

This is not a same-user privilege boundary: the denial runs in the agent's MCP subprocess, and Codex tool restrictions are advisory, as #860 states.

Merge order

This PR lands before or with ephemeral creation support. #810 will call routes_to_ephemeral_store.

Refs #801.

🤖 Generated with Claude Code

[fanhongy]

CI is green on cd91e0ef (33/33 checks), and the branch is rebased on current main. This PR is ready for review.

Suggested reading order:

1. utils/agent_profiles.py: the reserved-name predicate routes_to_ephemeral_store and the launch-only loader load_launch_profile, which fails closed with EphemeralProfileUnavailable. Also the refusal of reserved names in every other profile consumer, and the best-effort startup warning. [Feat] Decision points: opt-in, auditable judgments in orchestration, starting with Auto model routing #810 S1b calls the predicate.
2. mcp_server/server.py: the ephemeral-caller denial in _tool_denied_reason, which refuses assign, handoff and assign_elastic, and the same check on workflow_run, workflow_resume and workflow_start.
3. clients/database.py and api/main.py: the ephemeral_agents table, which has read accessors and no writer, and the registry-derived ephemeral terminal field.
4. utils/orchestration.py: assign and handoff refusing ephemeral targets until claims exist.
5. providers/claude_code.py and providers/codex.py: the plugin-MCP skip for ephemeral launches.

Existing users see only the two behaviour changes listed in the description. The slice is otherwise inert: nothing in it creates an ephemeral agent.

The next slice, which creates and stores ephemeral agents (still not launchable, and off by default), is being built on top of this branch. It will open as a separate PR once this one is reviewed.

[fanhongy]

Independent automated review commissioned by the PR author and performed by a separate CAO review agent. These findings are the agent's assessment, not the author's manual self-review.

Summary

This PR adds the E1a security groundwork for ephemeral agents (#801, design in #860). It reserves a name pattern, adds a launch-only loader with no fallback, skips plugin MCP servers for ephemeral launches, adds the read-only ephemeral_agents registry and the registry-derived ephemeral terminal field, adds the ephemeral-caller denial on six delegation tools, and refuses ephemeral targets until claims exist.

I verified the security seams against the code:

• All six tools call _tool_denied_reason.
• GET /terminals/{id} serializes ephemeral through response_model=Terminal.
• All five terminal-dict builders carry the key, and PATCH metadata cannot change it.
• Reserved names are refused centrally in _read_agent_profile_source.
• No profile in the repo matches the reserved pattern, and the retargeted test patches hit the right bindings.

One regression must be fixed before merge. The new registry lookup opens a second pooled DB session inside an open one, on the server's hottest read path. I reproduced a pool deadlock that does not occur on base.

Findings

P1: get_terminal_metadata now holds two pooled connections per call, so 15 concurrent readers deadlock cao-server's DB pool

Where: src/cli_agent_orchestrator/clients/database.py:2151. The same pattern is at :2036 in create_terminal, and the helper is at :91.

What happens:

1. get_terminal_metadata runs db.query(TerminalModel)...first() inside with SessionLocal() as db: (:2128). That checks out a connection and keeps it until the block exits.
2. While building the return dict, it calls is_ephemeral_terminal(), which opens its own SessionLocal() and checks out a second connection.
3. The engine (:678) uses SQLAlchemy 2.0's default QueuePool: 5 connections plus 10 overflow, with a 30 s timeout.
4. Once 15 callers each hold an outer connection, every one of them waits for an inner connection that can never be freed.
5. All DB access in the process then stalls for 30 s and fails with TimeoutError: QueuePool limit of size 5 overflow 10 reached.

Base has no nested sessions on this path. create_terminal nests in the same way after db.commit(), because the post-commit attribute refresh re-acquires a connection before is_ephemeral_terminal asks for another.

How it is triggered in production:

• GET /terminals/{id} runs terminal_service.get_terminal, and through it get_terminal_metadata, via asyncio.to_thread (api/main.py:3746).
• That endpoint is "polled heavily by wait_until_terminal_status", in the code's own words.
• Every gated MCP tool call also resolves its caller through that endpoint (_get_terminal_context_from_env).
• Python's default executor has min(32, cpu_count + 4) workers. On any host with 11 or more logical CPUs, the executor alone can put 15 callers inside get_terminal_metadata at once.
• On a 10-CPU host, 14 workers plus the event-loop call in the websocket attach handler (api/main.py:7401) are enough. That variant also blocks the event loop itself for 30 s.

Impact:

• Every DB-backed request in cao-server waits on the exhausted pool.
• GET /terminals/{id} returns 500.
• _tool_denied_reason then refuses assign, handoff and the workflow tools with "cannot authorize".
• Status polls fail.

The stall repeats while the load continues. The PR calls itself inert, but this affects every installed user with enough parallel agents or dashboard polling.

Reproduction (isolated HOME, real clients.database, head venv vs base venv, same scripts):

Probe	Base	Head
N threads looping get_terminal_metadata, N = 8 / 14	n/a	0 errors (max latency 0.04 s / 0.09 s)
Same, N = 15	n/a	28 s with zero completions, then TimeoutError
Same, N = 16 (base also at N = 32)	0 errors, no stalls	28–29 s stall, TimeoutError
Burst of 64 asyncio.to_thread(get_terminal_metadata) with a 16-worker default executor (the size on a 12-CPU host)	0.04 s, 0 errors	30.06 s, 7 × TimeoutError
Unmodified default executor on this 10-CPU host (14 workers), plus one caller on the event loop (the websocket-attach pattern)	0.24 s, 0 errors	120.40 s; the event-loop call raised TimeoutError; 29 thread errors

Why CI is green: the new registry fixture builds its engine with poolclass=StaticPool (test/mcp_server/test_ephemeral_caller_denial.py:26). That is one shared connection, so the nested checkout can never block.

Fix:

• Reuse the session that is already open. For example, factor out _is_ephemeral_terminal(db, terminal_id) and call it with db from both get_terminal_metadata and create_terminal, the same way the list readers already reuse db through _ephemeral_terminal_ids.
• Keep is_ephemeral_terminal(terminal_id) as a thin wrapper for callers that have no session.
• Add a regression test on a file-backed SQLite engine with the default QueuePool that runs 16 or more concurrent get_terminal_metadata calls, or that asserts each call checks out at most one connection.

Validation

• Changed tests, head (all 20 changed or new test files). Command: pytest -p no:cacheprovider --no-cov -q -n 8, with an isolated HOME (DB pre-initialised) and TMPDIR.
  • Run 1: 1021 passed and 1 failed. The failure was test_terminal_service_inbox_registration.py::…::test_create_terminal_kiro_provider_sets_is_kiro_true, which passed 3/3 serially on both head and base, and its file passed 6/6 under -n 8. So it is an xdist shared-HOME flake.
  • Run 2: 1022 passed, 3 skipped, 1 xfailed.
• Formatting: black --check and isort --check-only on the 31 changed .py files are clean.
• Pool probes: head vs base, as in the table above.
• Static verification:
  • The six _tool_denied_reason call sites.
  • response_model=Terminal on the GET, PATCH and POST terminal routes.
  • The ephemeral key in create_terminal, get_terminal_metadata and the three list readers.
  • No repo .md profile matches the reserved pattern.
  • No non-test references to the removed <consumer>.load_agent_profile bindings.
  • No provider module mixes the module-import and function-import styles, so the retargeted patches are effective.
  • No deny_unknown_fields in the TUI.
• Taken from the acquisition run, not re-run: the full CI-equivalent suite (13,392 passed, with one environment-only AF_UNIX failure that also fails on base), mypy parity (151 = 151 errors), and 33/33 CI checks green on this head.
• The checkout was left unmodified: git status --short --ignored is empty.

Notes (not blocking)

• Remote paths. target_host assign and handoff, and assign_elastic, do not reach _refuse_ephemeral_target_without_claim. Design #860 places the "target_host/assign_elastic refuse ephemeral names" rule in E1b, so this is deferred work, not an E1a defect. Consider qualifying the PR body's "Refuses assign and handoff for ephemeral targets" as local targets only.
• Case-insensitive filesystems. The reserved-name refusal is lexical. On case-insensitive APFS, a case variant such as ramones-log_triage-3f9a still loads an installed Ramones-log_triage-3f9a.md. The CHANGELOG's "can no longer be launched" therefore holds for exact-case names only. This is not exploitable in E1a, because routing is keyed by the requested name.

[haofeif]

Review at cd91e0ef (base bcd714c1).

#	Severity	Where	Finding
1	P1	clients/database.py:2151, :2036 → :91	get_terminal_metadata / create_terminal open a second pooled session while holding one, which can deadlock the connection pool

This independently confirms the P1 in review 5398045467. Repro against the real clients.database (file-backed SQLite, SQLAlchemy 2.0.43 default QueuePool: 5 + 10 overflow, 30 s timeout), N threads looping get_terminal_metadata:

• base bcd714c1, N=16: 7,066 calls, 0 errors, 1 connection per call
• head, N=14: no errors, but 2 connections per call
• head, N=16: 30 s stall (14 calls in total), then sqlalchemy.exc.TimeoutError (QueuePool limit reached)

CI can't see this because test/mcp_server/test_ephemeral_caller_denial.py:26 uses StaticPool.

Checked and fine at this head: the MCP Apps submit_command tool is app-only (FastMCP hides visibility: ["app"] tools from tools/list and get_tool), so it doesn't bypass the six-tool ephemeral denial; and init_db() creates ephemeral_agents in the server lifespan, so there is no "no such table" window on upgrade.

[haofeif]

Requesting changes for the P1 in review 5398895325: is_ephemeral_terminal() opens a second pooled session while get_terminal_metadata and create_terminal still hold one (clients/database.py:2151, :2036). With 15 or more concurrent callers this deadlocks the connection pool. The repro, suggested fix and inline comments are in that review. No new findings.

[fanhongy]

Thanks for catching the nested checkout and for the pool repro.
Fixed in e1df2268: get_terminal_metadata and create_terminal now check registry membership on the session they already hold, so each call uses one pooled connection; is_ephemeral_terminal() keeps its behaviour for callers without a session.
test/clients/test_database_ephemeral_pool.py uses a real file-backed QueuePool. With size 1 and no overflow it raises TimeoutError on the previous head cd91e0ef, and passes with a checkout peak of 1 after the fix; 16 concurrent callers also finish without exhausting the default pool.
The branch is now rebased onto recent main.
88b05119 extends the reserved-name refusal to the local-file and URL install paths that main added, before anything is written.
0bcf4a7c refuses literal reserved frontmatter names before environment resolution, lets Copilot batch refresh skip reserved files without aborting, and refuses named memory promotion; explicit --profile-path use is unchanged. 2821ae8b lets profiles whose frontmatter only parses after ${VAR} resolution install as before, and keeps unmanaged Copilot files silent during refresh.
I've updated the description with the refusal surfaces, the local versus remote placement behaviour, the pool fix and the final validation results.

[haofeif]

Re-review at 2821ae8b (merge-base 55dd4a0c): nil — no P1/P2/P3 findings.

Prior P1 (nested SessionLocal() pool exhaustion) — fixed in e1df2268.

• create_terminal (database.py L2041) and get_terminal_metadata (L2156) now call _is_ephemeral_terminal(db, …) on the caller's own session, and the list readers use _ephemeral_terminal_ids(db, …) (L2503/2608/2641). The public is_ephemeral_terminal() wrapper (L103-106) is the only path that opens its own session, and nothing nests it.
• test/clients/test_database_ephemeral_pool.py pins the single-checkout invariant on a real file-backed QueuePool (size 1, peak-checkout assertions, 16-caller barrier).

New commits checked

• 88b05119 / 0bcf4a7c / 2821ae8b: install_agent refuses a reserved source stem, a literal frontmatter name, and a resolved profile.name. All three refusals fire before the ownership guard, before env persistence, and before any store, context, or provider write. Refusals return InstallResult(success=False), and /agents/profiles/install returns 400. Every write_profile/replace_profile caller validates first, so the new store-level refusals are defence in depth.
• The Copilot batch refresh checks managed status first, then skips reserved stems and names with a warning. In src, the new raise in refresh_agent_md_prompt is reachable only behind that filter. The plugin-MCP refresh replays through install_agent, so it inherits the same refusals. memory promote name lookup refuses reserved names.
• #493's ownership guard, picked up by the rebase, runs after the reserved checks. The original four commits are unchanged apart from main context, and the branch merges cleanly with current main (0caec2c9).

All 33 CI checks pass at this head. Remote placement (E1b) and the exact-case lexical caveat are already documented, so I have not raised them again.

[haofeif]

Approving at 2821ae8b: the prior P1 is fixed in e1df2268, the new commits verified clean, and CI is green. Details in review 5405531261.