test(repository): document cross-platform gui workflow

.agents/skills/cross-platform-gui-e2e-test/SKILL.md

@@ -0,0 +1,105 @@
+---
+name: cross-platform-gui-e2e-test
+description: Run packaged Ropy desktop GUI end-to-end, smoke, and compatibility tests across macOS, Windows, and Linux. Use when validating a Ropy release or CI artifact, reproducing platform-specific clipboard, global-hotkey, tray, focus, X11, or window behavior, or collecting cross-platform GUI evidence. Run macOS locally; run Windows and Linux in isolated AgentBay virtual desktops.
+---
+
+# Ropy Cross-Platform GUI E2E Test
+
+Validate the packaged desktop application on every supported operating system. Complement GPUI in-process tests with real process, clipboard, global-hotkey, tray, focus, and window interactions.
+
+## Enforce the Platform Contract
+
+| Target | Execution environment | Release artifact |
+| --- | --- | --- |
+| macOS Apple Silicon | Local matching Mac | `ropy-aarch64-apple-darwin.dmg` |
+| macOS Intel | Local matching Mac | `ropy-x86_64-apple-darwin.dmg` |
+| Windows x86-64 | AgentBay Windows virtual desktop | `ropy-x86_64-pc-windows-msvc.zip` |
+| Linux x86-64 | AgentBay Linux X11 virtual desktop | `ropy-x86_64-unknown-linux-gnu.tar.xz` |
+| Linux ARM64 | Matching AgentBay custom Linux X11 image | `ropy-aarch64-unknown-linux-gnu.tar.xz` |
+
+- Inspect the host or guest architecture before selecting an artifact. Never use emulation silently.
+- Run macOS on the local Mac. Do not spend AgentBay resources for macOS unless the user explicitly changes this policy.
+- Run every Windows and Linux GUI test in a fresh virtual desktop. Do not substitute a local container, headless shell, WSL, or compile-only CI job.
+- Require X11 connectivity for Linux. Ropy's Linux GUI integration is X11-specific; a `DISPLAY` variable without a reachable X server is insufficient.
+- Report a cross-platform pass only when every requested platform row passes. Preserve separate results for each OS and architecture.
+
+Read only the reference for the platform being exercised:
+
+- [macOS local run](references/macos.md)
+- [Windows AgentBay run](references/windows.md)
+- [Linux AgentBay run](references/linux.md)
+
+## Apply Shared Guardrails
+
+- Resolve the release tag or commit requested by the user. Otherwise use the latest published release.
+- Download artifacts to a temporary controller directory, inspect their contents, calculate SHA-256, and compare any publisher-provided digest before execution.
+- Keep AgentBay credentials in an environment variable, secret store, or user-level MCP configuration. Never commit, print, or include a credential-bearing URL in output.
+- Confirm paid-session authorization unless the current request already authorizes the virtual test. Do not enable postpaid billing, buy credits, or raise limits without separate authorization.
+- Before creating a virtual session, inspect active sessions and billing controls. Prefer concurrency `1`, a maximum runtime of no more than `30` minutes, and an idle timeout of at least `10` minutes.
+- Create only one virtual session at a time. Record its identifier immediately and release it in a `finally` path. Explicitly terminate it; do not leave it running or hibernating.
+- Use a unique marker such as `ropy-e2e-<platform>-<uuid>` so stale clipboard contents cannot create a false pass.
+- Never record unrelated local clipboard contents, credentials, signed URLs, or user data in screenshots and logs.
+
+## Prefer Stable Virtual Control
+
+For Windows and Linux, prefer these AgentBay paths in order:
+
+1. Use a configured MCP endpoint with command, filesystem, screenshot, keyboard, and window capabilities.
+2. Use the official AgentBay SDK with `AGENTBAY_API_KEY` supplied through the environment.
+3. Use an AgentBay console debug session through the in-app browser only as a manual fallback.
+
+Discover current tool names rather than assuming one SDK version. Prefer filesystem `write_large_file` or an AgentBay Context for artifact transfer. Avoid downloading release assets inside the guest: slow downloads may trigger idle-session reclamation.
+
+If no supported control path is available, stop before creating a paid session and report the missing capability. Never place the secret endpoint in the repository as a workaround.
+
+## Run the Shared GUI Scenario
+
+Use platform-native UI automation, screenshots, and window inspection at every assertion boundary:
+
+1. Launch the verified packaged executable and assert that its process remains alive after three seconds.
+2. Expect no initial visible window: Ropy intentionally starts hidden in the system tray.
+3. Send `Control+Shift+D`, the default activation hotkey on all supported platforms, and assert that the Ropy window becomes visible.
+4. Open a platform-native plain-text editor, enter the unique marker, select it, and copy it.
+5. Wait briefly for clipboard ingestion, reactivate Ropy, and assert that the exact marker appears in history.
+6. Press `/`, search for a unique substring, and assert that the matching record remains visible.
+7. Select the record and press `Enter`. The default confirm mode copies the record back to the clipboard.
+8. Return to the editor, paste, and assert that the pasted text exactly equals the marker.
+
+Prefer semantic window, text, and accessibility queries. Use coordinates only after taking a fresh screenshot. Retry a flaky clipboard ingestion once with a new marker; collect state before retrying.
+
+## Add Focused Checks Deliberately
+
+- Test favorite, delete, clear, plain-text paste, grid navigation, pinning, settings, theme, language, opacity, autostart, or update behavior only when relevant to the requested change.
+- Avoid destructive history operations on local macOS by default because they can touch the user's real Ropy data.
+- Keep the common smoke scenario identical across platforms; isolate platform-specific assertions in the relevant run.
+- Verify Linux tray behavior only when the image has a working desktop panel and session bus. Do not classify a missing tray host as a Ropy regression.
+
+## Record a Platform Matrix
+
+For every requested target, record:
+
+- OS version, architecture, execution environment, and image identifier when virtual;
+- release tag or commit SHA, artifact name, size, and SHA-256;
+- start/end timestamps and elapsed time;
+- checksum, extraction, launch, process, window, and assertion results;
+- screenshots of the visible Ropy window, captured marker, filtered result, and final editor paste;
+- cleanup confirmation and remaining active virtual-session count.
+
+Classify each row independently:
+
+- **pass**: the packaged executable launches and all requested GUI assertions pass;
+- **application failure**: the verified artifact runs in a healthy environment but exits, cannot activate, or violates an assertion;
+- **infrastructure blocked**: architecture, upload, MCP/SDK access, X11, desktop image, session expiry, or provisioning prevents reaching the assertion;
+- **partial**: artifact transfer or launch is proven but the GUI scenario does not finish.
+
+State the last verified boundary. Never describe an infrastructure interruption as a Ropy failure.
+
+## Clean Up Reliably
+
+- Collect platform logs before teardown on failure; use the location in the platform reference.
+- Terminate only the Ropy process started by this run and close helper applications when practical.
+- Remove temporary local mounts and files.
+- Always release AgentBay sessions after Windows or Linux runs, including after exceptions and timeouts.
+- Recheck the AgentBay session list and report any cleanup failure immediately.
+
+Before changing the virtual workflow, consult current official AgentBay Computer Use, desktop UI automation, filesystem, command, application, and window-management documentation.

.agents/skills/cross-platform-gui-e2e-test/agents/openai.yaml

@@ -0,0 +1,4 @@
+interface:
+  display_name: "Ropy Cross-Platform GUI E2E"
+  short_description: "Run Ropy GUI tests locally and in virtual desktops"
+  default_prompt: "Use $cross-platform-gui-e2e-test to validate the latest Ropy build on macOS, Windows, and Linux."

.agents/skills/cross-platform-gui-e2e-test/references/linux.md

@@ -0,0 +1,51 @@
+# Linux AgentBay Run
+
+Use a fresh AgentBay Linux Computer Use virtual desktop. Ropy supports Linux through X11, so require a real desktop session; a successful shell command or headless build is not GUI evidence.
+
+## Match the image
+
+- Inspect `uname -m` and select the matching GNU/Linux artifact. Use `x86_64-unknown-linux-gnu` for an x86-64 `linux_latest` image.
+- Use an ARM64 custom image for `aarch64-unknown-linux-gnu`; never execute it through implicit emulation.
+- Require a non-empty `DISPLAY` and verify the connection with `xdpyinfo` or an equivalent X11 query.
+- Accept XWayland only when the X11 connection is reachable and Ropy's window, hotkey, and clipboard behavior work. Do not claim native Wayland support.
+- Check that a desktop window manager is running. Require a desktop panel and session bus only for tray-specific assertions.
+
+## Transfer and launch
+
+Upload the verified archive to `/tmp/ropy-e2e/ropy.tar.xz` with the AgentBay filesystem API. Run an equivalent shell sequence:
+
+```bash
+set -eu
+root=/tmp/ropy-e2e
+archive="$root/ropy.tar.xz"
+expected='<local-sha256-lowercase>'
+actual="$(sha256sum "$archive" | awk '{print $1}')"
+test "$actual" = "$expected"
+mkdir -p "$root/app"
+tar -xJf "$archive" -C "$root/app"
+exe="$(find "$root/app" -type f -name ropy -perm -u+x -print -quit)"
+test -n "$exe"
+test -n "${DISPLAY:-}"
+xdpyinfo >/dev/null
+ldd "$exe" | tee "$root/ldd.txt"
+if grep -q 'not found' "$root/ldd.txt"; then exit 64; fi
+nohup "$exe" >"$root/ropy.stdout.log" 2>"$root/ropy.stderr.log" &
+pid=$!
+sleep 3
+kill -0 "$pid"
+printf '%s\n' "$pid"
+```
+
+If `ldd` reports missing libraries, classify the stock image as infrastructure-blocked or use an approved custom image containing the runtime equivalents of the repository's GTK3, X11, `libxdo`, and `libxkbcommon-x11` build dependencies. Do not install arbitrary packages silently into a supposedly reproducible compatibility image.
+
+Discover an installed graphical text editor and use it for the shared marker scenario. Send `Control+Shift+D` to reveal Ropy. Use AgentBay screenshot, keyboard, application, and window tools rather than `xdotool` for the assertions unless a lower-level diagnostic is specifically required.
+
+## Troubleshoot and clean up
+
+- If `DISPLAY` is unset or `xdpyinfo` fails, the image is not ready for Ropy GUI testing.
+- If the process lives but activation fails, collect X11 session details and Ropy logs before classifying the result.
+- If the tray is absent while hotkey activation works, check the desktop panel and `DBUS_SESSION_BUS_ADDRESS`; report missing host infrastructure separately.
+- Collect JSONL logs from `~/.config/ropy/logs` plus `$root/ropy.stdout.log` and `$root/ropy.stderr.log` on failure.
+- Stop the test process by its recorded PID, then release the AgentBay session and verify that it is gone.
+
+Current official references: [Computer Use](https://help.aliyun.com/en/agentbay/computeruse), [desktop UI automation](https://help.aliyun.com/en/agentbay/developer-reference/ui-automation), and [application management](https://help.aliyun.com/en/agentbay/developer-reference/application-management).

.agents/skills/cross-platform-gui-e2e-test/references/macos.md

@@ -0,0 +1,34 @@
+# macOS Local Run
+
+Use the local Mac for macOS GUI validation. Inspect `uname -m` first and select the matching DMG; do not run an Intel artifact on Apple Silicon through Rosetta without explicitly labeling that separate compatibility run.
+
+## Protect Local State
+
+- Check for an existing Ropy process. Do not terminate or replace a user-owned running instance without explicit permission.
+- Warn that the smoke test temporarily changes the system clipboard and can add the unique marker to the user's Ropy history.
+- Never run clear-history or bulk-delete checks locally by default. Remove only the unique test marker when that can be done unambiguously.
+- Do not inspect or report pre-existing clipboard history.
+
+## Verify and launch
+
+Download the matching `.dmg` to a temporary directory and verify it against the release digest:
+
+```bash
+shasum -a 256 /path/to/ropy-apple-darwin.dmg
+hdiutil attach -nobrowse -readonly /path/to/ropy-apple-darwin.dmg
+```
+
+Locate `Ropy.app` on the mounted volume and launch that exact bundle with `open`. Record the mount point, bundle path, bundle version, and process identifier. Do not validate an already-installed copy by accident.
+
+Use local Computer Use or another approved macOS UI-control capability for screenshots, keyboard input, focus, and the shared GUI scenario. Ropy starts hidden; send `Control+Shift+D` to reveal it.
+
+If macOS blocks the downloaded application through quarantine or Gatekeeper, capture the dialog and report an application packaging/signing result. Do not remove quarantine attributes or bypass platform security unless the user explicitly asks for that diagnostic.
+
+Use a plain-text editor such as TextEdit in plain-text mode for the marker copy/paste assertion.
+
+## Collect and clean up
+
+- Collect JSONL logs from `~/Library/Application Support/ropy/logs` on failure.
+- Quit only the Ropy instance launched for this test.
+- Eject the mounted DMG with `hdiutil detach` and remove the temporary download.
+- Restore the prior plain-text clipboard value when it was captured without exposing it; otherwise state that clipboard restoration was not lossless.

.agents/skills/cross-platform-gui-e2e-test/references/windows.md

@@ -0,0 +1,38 @@
+# Windows AgentBay Run
+
+Use a fresh AgentBay Windows Computer Use virtual desktop. Record its session and image identifiers before transferring the artifact.
+
+## Transfer and launch
+
+Upload the verified archive to `C:\ropy-e2e\ropy.zip` with the AgentBay filesystem API. Use shell or command APIs for setup; do not type long PowerShell commands through remote-desktop automation because keyboard layouts can corrupt punctuation.
+
+Run an equivalent non-interactive PowerShell sequence:
+
+```powershell
+$ErrorActionPreference = 'Stop'
+$root = 'C:\ropy-e2e'
+$zip = Join-Path $root 'ropy.zip'
+$expected = '<local-sha256-lowercase>'
+$actual = (Get-FileHash -Algorithm SHA256 $zip).Hash.ToLowerInvariant()
+if ($actual -ne $expected) { throw "SHA-256 mismatch: $actual" }
+$app = Join-Path $root 'app'
+Expand-Archive -Path $zip -DestinationPath $app -Force
+$exe = Join-Path $app 'ropy.exe'
+if (-not (Test-Path $exe)) { throw "Missing $exe" }
+$process = Start-Process -FilePath $exe -PassThru
+Start-Sleep -Seconds 3
+if ($process.HasExited) { throw "ropy.exe exited with $($process.ExitCode)" }
+$process.Id
+```
+
+Use Notepad for the shared marker scenario. Treat an alive process with no initial window as expected, then send `Control+Shift+D`.
+
+## Troubleshoot and clean up
+
+- If PowerShell 5 cannot download from GitHub, keep using controller-side upload; do not weaken TLS settings.
+- If typed commands replace `:` or other punctuation, use the command API.
+- If the archive arrives but the session expires before extraction, use direct file upload and an idle timeout of at least `10` minutes.
+- Collect JSONL logs from `%APPDATA%\ropy\logs` on failure.
+- Stop the test process by its recorded PID, then release the AgentBay session and verify that it is gone.
+
+Current official references: [Windows Computer Use](https://help.aliyun.com/en/agentbay/developer-reference/computer-use-windows-server-2022), [desktop UI automation](https://help.aliyun.com/en/agentbay/developer-reference/ui-automation), and [window management](https://help.aliyun.com/en/agentbay/support/window-management).