fix(automations): deliver Hermes prompts via native query#7862
Conversation
Hermes's prompt_toolkit TUI never emits the DECSET 2004 bracketed-paste handshake that the default draft-paste readiness waiter gates on, so the automation prompt was silently dropped and the agent sat idle (terminal opens, nothing entered). Add a 'process-ready' DraftPasteReadySignal that arms the quiet-window on first PTY output and pastes once the TUI settles, and assign it to the hermes agent config. Adds a Hermes unit test covering the no-handshake path. Design for the follow-up PR/issue review loop lives in docs/design/pr-issue-review-loop.md (not committed; gitignored).
📝 WalkthroughWalkthroughThis PR adds a 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
src/renderer/src/lib/agent-paste-draft.test.ts (1)
146-170: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick winAdd
toHaveBeenCalledTimes(1)and keep the quiet-window boundary aligned.
This path only checks the payload; the extra count assertion guards against duplicate paste, and the1499/1split should stay in sync with the quiet-window timeout.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
Run ID: 8e029139-8172-49b9-a759-8dc941775331
📒 Files selected for processing (3)
src/renderer/src/lib/agent-paste-draft.test.tssrc/shared/draft-paste-ready-scanner.tssrc/shared/tui-agent-config.ts
CodeRabbit: the process-ready fallback consulted waitForAgentReady, which compares the foreground process basename via isExpectedAgentProcess. Wrapped interpreter launches surface as 'python3 .../hermes', so the check can never confirm readiness and only drops the paste. process-ready readiness is the PTY-quiet window (handled by waitForAgentDraftInputReady); the process-name fallback is dead for this signal. Skip it and return false on timeout instead.
Guard against a duplicate paste in the no-handshake Hermes path by asserting sendRuntimePtyInputVerified is called exactly once. Addresses the CodeRabbit nitpick on PR stablyai#7862.
…e path pasteDraftToAgentPtyWhenReady (quick-create/work-item route) still ran the python3-vs-hermes process-name fallback that can never match for process-ready agents, burning ~1s and dropping the paste. Mirror the process-ready guard from pasteDraftWhenAgentReady and surface onTimeout. Adds regression tests for both the quiet-window happy path and the fallback-skip on the pty-bound path. Co-authored-by: Orca <help@stably.ai>
Live-testing against Hermes v0.18.2 showed the fixed 50ms post-paste Enter is swallowed: the node ui-tui takes 15s+ to boot, the paste fires ~1.5s in (process-ready quiet window), and the cooked-mode line discipline turns the early \r into \n, which the editor treats as newline-insert. The prompt parked in the input box and the automation never executed. For process-ready agents, defer the Enter until the TUI signals interactivity (DECSET 2004 enable) or echoes the pasted content marker-free (legacy prompt_toolkit), with a 120s best-effort cap. PTY input is FIFO, so the deferred Enter always lands after the buffered paste text. The paste echo of the content itself (raw or caret-notation markers adjacent) is rejected so a cooked-mode echo can't release the Enter early. Verified end-to-end in the dev app: automation prompt pasted, deferred Enter released on tui-ready, Hermes submitted and ran the turn. Co-authored-by: Orca <help@stably.ai>
|
Pushed two commits after an independent review + live end-to-end testing of this PR against the real Hermes CLI (v0.18.2, installed via the official installer): 1. 2. Fix: for Live verification (automation → new_per_run workspace →
Tests: 49 pass across |
|
Thanks @Jinwoo-H this is exactly the layering the unit tests couldn't surface, and the live repro is the part that matters. Confirmed on my end:
One question: keep these as 5 separate commits, or squash before merge? Either works for me; leaving as-is unless a maintainer prefers cleaner history. |
…s-validation-20260711
Co-authored-by: Orca <help@stably.ai>
Jinwoo-H
left a comment
There was a problem hiding this comment.
Thanks @branben — approved after the Windows follow-up. The final implementation uses Hermes’ native startup query, preserves multiline/quoted arguments across POSIX, PowerShell, cmd, SSH, and WSL, removes the transport env before Hermes/tools inherit it, and includes a real Windows PowerShell 5 native-argv regression test. 281 focused tests, typecheck, touched lint/reliability guards, live Hermes validation, and two fresh reviewers are clean.
Co-authored-by: Orca <help@stably.ai>
I appreciate the quick work on this @Jinwoo-H! |
Host-owned agent-launch subsystem, orchestration U6 identity/forget, resolver #7862 Windows-shell + Hermes native-query parity, shared launch contracts, preload/relay/CLI surfaces. One rebase landing split for review; only the branch tip is expected to build/test green. Co-authored-by: Orca <help@stably.ai>
Summary
hermes chat --query=<prompt> --tui.Implementation
hermes-querystartup mode in the shared agent planner.WSLENVin both direct and daemon PTY providers.Test plan
node.exeashermes.exethrough Windows PowerShell 5 and asserts the exact native argv for a multiline quoted query and spaced custom arguments.No issues foundafter the Windows fix.Provider and platform coverage
Performance
Startup planning performs one bounded
O(command + environment)transformation per Hermes launch. It adds no polling, listeners, provider fanout, or retained runtime state. Command plus environment size is capped conservatively at 24,000 characters.Security audit
The literal prompt is absent from the shell command/history. The neutral transport variable remains in the parent terminal environment for compatibility, but the child wrapper removes it before starting Hermes, so Hermes tools do not inherit it. Hermes' native API necessarily places the prompt in the Hermes process argv. No new network endpoint, IPC surface, credential handling, or dependency is introduced.
Screenshots
No Orca layout or copy changes. Reviewer evidence is the real hosted Hermes terminal showing the two-line automation prompt, exact response, and the still-running TUI.
AI review report
CodeRabbit checks passed. Maintainer review replaced the original timing-based paste approach with the native Hermes query path, then completed repeated two-reviewer fix loops and a final Windows-specific review round.