Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
63 changes: 63 additions & 0 deletions docs/architecture/cua-driver-0-17-contract-migration/plan.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
# CUA Driver 0.17 Contract Migration Plan

## Status

Implementation and host-native validation are complete. Native cross-platform and release-signing
gates remain pending.

## 1. Freeze the reviewed contract

- Record the exact upstream tag, commit, contract metadata, supported targets, tool additions, and
breaking semantics.
- Keep clipboard reads denied until DeepChat has an explicit sensitive-result lifecycle.
- Define closed, bounded model projections and the snapshot-addressing invariant.

## 2. Update the release and host handshake

- Replace the pinned tag, commit, release URL, checksums asset hash, archive names, and archive
hashes in `upstream.json`.
- Update the manifest and package-time embedded adapter contract to driver `0.17.0` and contract
`0.6.0`.
- Keep tools-list schema, capability version, MCP protocol, supported targets, and runtime layout
unchanged.

## 3. Close the new tool surface

- Add the five new tools to both policy copies with the reviewed defaults.
- Regenerate target-local catalogs from the pinned native release binary.
- Retain strict package failure for missing or extra policy entries after platform scoping.

## 4. Adapt element arguments

- Continue removing only an empty optional `element_token` on the seven affected native tools.
- Add a pure guard that rejects a remaining bare `element_index` before dispatch.
- Preserve valid token, index-plus-snapshot, pixel, zero-valued, and unrelated arguments.
- Add unit and ToolManager dispatch tests for all modes.

## 5. Adapt model-visible results

- Project the closed `ActionResult` fields only for the reviewed action-tool set.
- Project bounded `verify_state` control facts without `observed_json`.
- Preserve existing window-handle, browser-chrome, and structured-refusal projections.
- Test valid shapes, enum drift, malformed nested values, bounded evidence, and composition with
existing MCP content.

## 6. Update the Computer Use contract

- Require `element_token` or `element_index + snapshot_id` from the latest same-window snapshot.
- Add all relevant snapshot refusal codes and one-refresh/one-retry recovery.
- Explain `ActionResult` effect, route, delivery, evidence, and escalation semantics.
- Add deterministic `verify_state` to the post-action loop only for supported window/native AX
predicates; retain fresh state tools for desktop, browser DOM, canvas, and visual checks.
- Document the new window/menu tools and the conservative clipboard policy.

## 7. Validate and review

- Run focused adapter, ToolManager, plugin, runtime, catalog, integrity, build-runtime, and package
tests first.
- Build, validate, and verify the host-native CUA plugin artifact.
- Run formatting, i18n, lint, Node/Web type checks, and the appropriate broader suites.
- Review the complete diff for hidden side effects, backward compatibility, edge cases,
performance, security, misleading names, missing tests, and maintenance cost.
- Sort findings by severity, fix every real finding, rerun affected validation, and only then
create one concrete Conventional Commit. Do not push.
201 changes: 201 additions & 0 deletions docs/architecture/cua-driver-0-17-contract-migration/spec.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,201 @@
# CUA Driver 0.17 Contract Migration

## Status

Implementation and host-native validation are complete. This goal upgrades the bundled CUA
runtime from `0.14.1` to `0.17.0` without changing DeepChat's supported target matrix or
external-runtime ownership model. Native cross-platform and release-signing gates remain pending.

## Context

Before this migration, DeepChat pinned `cua-driver-rs-v0.14.1` and validated an exact embedded
handshake before exposing its MCP tools. The native catalog is generated from the release binary,
and packaging requires the catalog and closed tool policy to match exactly. Model-visible tool
content is separate from raw MCP `structuredContent`, so upstream structured contract changes need
an explicit DeepChat projection.

Upstream `0.15.0` and `0.17.0` introduce two breaking contracts:

- successful action tools now return the closed `ActionResult` shape instead of legacy per-tool
structured fields;
- native element actions reject a bare `element_index` and require either `element_token` or the
exact `element_index` plus `snapshot_id` pair from one current window snapshot.

The `0.17.0` release also adds `verify_state`, `set_window_frame`, `invoke_menu`,
`clipboard_read`, and `clipboard_write`. Existing tools are not removed. The embedded
daemon/proxy commands, tools-list schema, capability version, MCP protocol, packaged application
layout, and minimum macOS version remain compatible with the current DeepChat architecture.

## Goals

1. Pin and attest the exact `cua-driver-rs-v0.17.0` release assets.
2. Update the exact embedded handshake from driver/contract `0.14.1/0.2.0` to `0.17.0/0.6.0`.
3. Keep catalog generation and closed policy coverage exact for every supported target.
4. Fail closed before dispatch when a native element action uses a bare `element_index`.
5. Expose bounded, typed `ActionResult` and `verify_state` facts to the model without promoting
arbitrary runtime prose or raw application content.
6. Make the packaged Computer Use loop consume action effects and perform deterministic,
window-scoped postcondition checks when the task has an expressible predicate.
7. Preserve current runtime supervision, integrity verification, signing, and target support.

## Non-goals

- Do not add Linux arm64 support.
- Do not change the embedded daemon/proxy lifecycle or generic MCP result contract.
- Do not persist a DeepChat-owned “latest snapshot” cache or auto-inject a snapshot id.
- Do not expose raw clipboard plaintext to the model or add a new sensitive-data persistence
path in this migration.
- Do not make `verify_state` a desktop or visual-image interpretation engine.
- Do not infer task completion from a delivered action.
- Do not sync this SDD to a GitHub issue unless explicitly requested.

## Pinned Upstream Contract

The runtime pin is:

- tag: `cua-driver-rs-v0.17.0`;
- commit: `10279552e2bbe479e367a082f78b1b98ee85a697`;
- driver version: `0.17.0`;
- contract version: `0.6.0`;
- tools-list schema version: `1`;
- capability version: `1`;
- MCP protocol version: `2025-06-18`.

The five currently supported targets remain `darwin/arm64`, `darwin/x64`, `win32/x64`,
`win32/arm64`, and `linux/x64`. `linux/arm64` remains explicitly unsupported even if upstream
publishes an asset.

## Tool Policy

Every target-local catalog tool must have one explicit policy entry after platform scoping. The
five new cross-platform tools use these reviewed defaults:

| Tool | Policy | Reason |
| --- | --- | --- |
| `verify_state` | `allow` | Bounded, read-only observation of one exact window |
| `set_window_frame` | `ask` | User-visible window mutation |
| `invoke_menu` | `ask` | User-visible native action that can trigger consequential commands |
| `clipboard_write` | `ask` | Mutates privacy-sensitive shared system state |
| `clipboard_read` | `deny` | Can return privacy-sensitive plaintext that DeepChat currently persists as raw MCP structured content |

Explicit denial keeps the tool in the closed catalog and policy while preventing an accidental
sensitive-data path. Enabling reads later requires a separate design for consent, bounded
model-facing projection, transcript persistence, export, and retention.

## Snapshot-safe Element Addressing

The affected native tools are `click`, `double_click`, `right_click`, `type_text`, `press_key`,
`set_value`, and `scroll`.

DeepChat must preserve these invariants:

- prefer the non-empty opaque `element_token` from the latest `get_window_state` for the same
process and window;
- an index fallback is valid only as the pair `element_index + snapshot_id` from that same result;
- a bare `element_index` fails locally before the runtime call;
- an empty or whitespace-only optional token is still removed so it cannot override another
addressing mode;
- the adapter never guesses, caches, or injects a “latest” snapshot id;
- pixel-coordinate actions remain valid when no `element_index` is supplied;
- conflicting token/index/snapshot/window inputs are left for the pinned runtime to reject;
- an addressing refusal causes at most one fresh snapshot and one retry with handles from that
new snapshot. No field from the rejected snapshot may be reused.

`get_window_state` model projection must label the exact pair explicitly. The projected token map
is capped at 256 bounded handles and does not duplicate the accessibility tree. When the map is
truncated, an unlisted element remains addressable through its same-result index-plus-snapshot
pair. Only the pinned 0.17 lexical snapshot/token forms whose snapshot and element-index parts
agree with the structured row enter model-visible content; this boundary check does not decode,
derive, or synthesize either opaque handle for a caller.

## ActionResult Projection

Successful upstream action tools return:

- required `effect`: `confirmed`, `partial`, `unverifiable`, `suspected_noop`, or `refused`;
- required `route`: `accessibility`, `synthetic_events`, `global_input`, `system_api`, `dom`, or
`trusted_input`;
- optional `delivery.mode` and non-negative `delivery.delivered_count`;
- optional evidence kinds `value_readback` and `window_change`;
- optional closed escalation target/reason.

DeepChat preserves the raw structured result for protocol fidelity and app diagnostics, then
appends a typed, bounded projection to model-visible content only for the reviewed action-tool
set. The projection accepts only closed enum values and numeric bounds. It emits no arbitrary
runtime strings and deduplicates the two possible evidence kinds. A non-error action response that
does not satisfy the reviewed shape appends a fixed contract-validation warning, so legacy result
text cannot silently become evidence of success.

Semantics:

- `confirmed` means the driver has action-specific evidence, not that the user's whole task is
complete;
- `partial`, `unverifiable`, and `suspected_noop` require observation or recovery before another
consequential step;
- `refused` is not success;
- delivery only describes dispatch and must not be treated as effect or task completion;
- escalation is advice constrained by current session policy, not permission to broaden scope.

Structured refusal projection remains supported for error results such as stale or malformed
snapshot handles, because those errors occur before a successful `ActionResult` is published.

## Verification Projection and Loop

`verify_state` verifies one exact `(pid, window_id)` against one to eight bounded predicates. Its
aggregate status is `satisfied`, `unsatisfied`, or `unknown`, and `unknown` never means success.

DeepChat projects only:

- aggregate `status`, `stable`, `elapsed_ms`, and `samples`;
- at most eight predicate indices, statuses, and closed `unknown_reason` values.

The upstream `observed_json` field is deliberately not promoted because it can contain application
text and is unnecessary for control-flow decisions. Raw structured content remains available to
the product diagnostics path. A non-error `verify_state` response that violates this shape appends
a fixed warning and cannot be interpreted as verification success.

The Computer Use loop uses `verify_state` after an action only when the requested postcondition is
expressible as a window existence/bounds predicate or trusted native element existence/value/
enabled/selected predicate. It treats success as `status="satisfied"` and `stable=true`. For
desktop scope, browser DOM, canvas, video, screenshots, or other visual effects, it uses a fresh
`get_desktop_state`, `get_browser_state`, or `get_window_state` as appropriate instead.

## Packaging and Compatibility

The existing release-asset staging and integrity model remains unchanged:

- verify `checksums.txt` and each selected archive SHA-256;
- validate the archive layout and executable identity;
- generate the target-local catalog using `dump-docs --type mcp --pretty`;
- scope platform-specific policy entries before exact catalog comparison;
- keep the DeepChat-owned macOS helper name, bundle id, entitlement allowlist, load-path checks,
re-signing order, and notarization gates;
- keep only the unsigned primary Windows driver and the existing Linux binary layout.

The version/contract values intentionally remain duplicated in source manifest and packaging
validation so packaging fails if the source declaration drifts from the reviewed host contract.

## Acceptance Criteria

- The exact `0.17.0/0.6.0` embedded handshake starts; older or mismatched metadata is rejected.
- All five new tools are present with the reviewed policy, and target-local package policy equals
the generated catalog exactly.
- Empty tokens are removed, bare indices fail before dispatch, and index-plus-snapshot or token
inputs are preserved unchanged.
- Model-visible action and verification projections are closed, bounded, injection-resistant, and
covered for valid and malformed inputs; malformed non-error results fail closed visibly.
- The skill requires current snapshot handles, distinguishes delivery/effect/completion, and uses
`verify_state` only for supported window predicates.
- Focused adapter, ToolManager, plugin, embedded-runtime, catalog, integrity, and packaging tests
pass.
- Formatting, i18n validation, lint, Node/Web type checks, CUA plugin validation, and a host-native
CUA bundle/verify run pass where the current machine can execute the release binary.
- Native Windows/Linux behavior, macOS x64, and release-signed/notarized macOS remain explicit
release gates unless run in their matching environments.

## Rollback

Rollback requires reverting the version/contract pin, all release hashes, the five policy entries,
model projections, snapshot argument guard, skill contract, and regenerated catalog as one unit.
Mixing a `0.17.0` binary with the `0.14.1` handshake or skill is intentionally unsupported and
must fail closed rather than degrade silently.
75 changes: 75 additions & 0 deletions docs/architecture/cua-driver-0-17-contract-migration/tasks.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
# CUA Driver 0.17 Contract Migration Tasks

## Contract audit

- [x] Compare the `0.14.1` and `0.17.0` release catalogs and contract manifests.
- [x] Verify the two upstream breaking changes and five added tools against source and release
binaries.
- [x] Trace DeepChat's manifest, packaging, runtime handshake, MCP result, tool-policy, and skill
paths.
- [x] Decide the clipboard-read privacy boundary and model projection limits.

## Release and packaging

- [x] Pin the `0.17.0` tag, commit, release URL, asset names, and SHA-256 values.
- [x] Update both exact embedded adapter contracts to `0.17.0/0.6.0`.
- [x] Keep supported targets and platform-specific catalog scoping unchanged.
- [x] Generate, validate, and verify the host-native CUA plugin artifact.

## Tool policy

- [x] Add `verify_state=allow`.
- [x] Add `set_window_frame=ask`, `invoke_menu=ask`, and `clipboard_write=ask`.
- [x] Add `clipboard_read=deny` and document the sensitive-result rationale.
- [x] Prove both policy copies remain identical and exactly cover target catalogs.

## Model-facing compatibility

- [x] Reject bare `element_index` before runtime dispatch.
- [x] Preserve non-empty tokens, index-plus-snapshot pairs, pixels, zeros, and unrelated values.
- [x] Make the window-state projection explicitly pair indices with the projected snapshot id.
- [x] Add a closed and bounded `ActionResult` projection.
- [x] Add a closed and bounded `verify_state` projection without `observed_json`.
- [x] Preserve raw structured results and existing refusal/browser-chrome projections.

## Skill and documentation

- [x] Require current `element_token` or `element_index + snapshot_id` addressing.
- [x] Add one-refresh/one-retry handling for snapshot-addressing refusals.
- [x] Distinguish action delivery, effect, verification, and task completion.
- [x] Use `verify_state` only for expressible exact-window predicates.
- [x] Document `set_window_frame`, `invoke_menu`, and the clipboard policy.
- [x] Align the maintained CUA architecture and historical feature specifications.

## Automated validation

- [x] Adapter unit tests cover valid and malformed action/verification projections.
- [x] ToolManager tests cover local bare-index rejection and model-visible projection composition.
- [x] Plugin tests cover the new versions, hashes, policies, and skill invariants.
- [x] Embedded adapter, catalog, integrity, build-runtime, and package tests pass.
- [x] CUA plugin validation and host-native bundle/verify pass.
- [x] Formatting, i18n, lint, Node/Web typecheck, and relevant broader tests pass.

## Verification record

Completed on 2026-08-03:

- focused CUA adapter, ToolManager, and plugin tests: 133 passed;
- `pnpm run test:main`: 486 files and 5787 tests passed; 21 files and 285 tests were skipped by
environment gates;
- `pnpm run test:renderer`: 242 files and 1978 tests passed;
- formatting, i18n, lint, Node/Web type checks, and the production build passed;
- macOS arm64 plugin bundle, validation, and verification passed with a development-signed
artifact; its catalog reported driver 0.17.0, 54 tools, and all five added tools;
- the production prebuild rejected a provider-database refresh larger than its 5 MB limit, then
completed normally; the generated ACP registry remained unchanged after its separate refresh.

macOS x64, Windows x64/arm64, Linux x64, native desktop action scenarios, and
release-signed/notarized macOS remain release gates.

## Commit gate

- [x] Review the full diff with findings sorted by severity.
- [x] Fix all real review findings and rerun affected validation.
- [x] Commit with a concrete Conventional Commit message.
- [x] Confirm no push was performed.
16 changes: 14 additions & 2 deletions docs/architecture/plugin-external-runtime-lifecycle/plan.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,8 @@

## Status

Lifecycle and model-facing CUA compatibility implementation are complete with automated validation.
The v0.14.1 native cross-platform release gates remain pending.
Lifecycle ownership and the CUA 0.17 model-facing contract migration are complete and host-native
validated; native cross-platform release gates remain pending.

## 1. Freeze the contracts

Expand Down Expand Up @@ -142,6 +142,18 @@ The v0.14.1 native cross-platform release gates remain pending.
- Keep the optional GNOME Wayland helper outside DeepChat packaging and validate both fresh and
manually installed helper states separately.

## 13. Upgrade the closed CUA contract to 0.17.0

- Follow `docs/architecture/cua-driver-0-17-contract-migration/` as the authoritative goal SDD.
- Pin the `0.17.0/0.6.0` release and handshake without changing lifecycle or supported targets.
- Close the five new tools under explicit policy; deny privacy-sensitive clipboard reads.
- Reject bare indices, require current token or index-plus-snapshot addressing, and never maintain
a DeepChat-owned latest-snapshot cache.
- Project bounded `ActionResult` and `verify_state` facts into model-visible content while keeping
arbitrary runtime/application text out of instructions.
- Regenerate and verify native catalogs and retain all existing signing, integrity, and
cross-platform release gates.

## Compatibility and rollback

- Omitted manifest fields default to eager, all existing MCP surfaces, and legacy environment
Expand Down
Loading