fix(relay): detach primary client on write failure to trigger reconnect#8094
Closed
Jinwoo-H wants to merge 1 commit into
Closed
fix(relay): detach primary client on write failure to trigger reconnect#8094Jinwoo-H wants to merge 1 commit into
Jinwoo-H wants to merge 1 commit into
Conversation
writeFrame caught a write throw and, for the PRIMARY client, only logged and set closed=true WITHOUT notifying detach — unlike non-primary clients which are detached. The framing carries seq/ack but has no retransmit buffer, so the frame (possibly pty.data or pty.exit) was lost, and because detach never fired the owning Orca kept treating the primary as alive until the ~20s keepalive timeout, silently missing output or a pane death in the meantime. Every other primary-death path (stdin end/error, stdout error) already calls invalidateClient() -> notifyClientDetached. Make the socket-write-throw path consistent: fire notifyClientDetached so the reconnect + PTY-reattach machinery runs promptly. Reattach replays the PTY output buffer and surfaces pane death via reattach-not-found, so pty.exit is never lost. This fail-fast approach reuses the tested reconnect path instead of adding a seq-keyed resend buffer to the hot path. Regression test asserts the primary is detached on write throw (fails pre-fix) and recovers via setWrite. Co-authored-by: Orca <help@stably.ai>
Contributor
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Repository UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (2)
📝 WalkthroughWalkthroughUpdated 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Contributor
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The relay dispatcher silently drops frames on a primary-client write failure and never triggers reconnect, so the owning Orca can miss
pty.dataoutput or apty.exitpane-death.writeFrame(src/relay/dispatcher.ts) catches a write throw. For non-primary clients it deletes and detaches them; for the primary client it only logged to stderr and setclosed = truewithout notifying detach. The framing carries seq/ack but there is no retransmit buffer, so the frame is gone. Because detach never fired, none of the primary-death consumers ran and the owning Orca kept treating the primary as alive until the ~20s keepalive timeout — during that window every frame written to the throwing sink was silently lost.Every other primary-death path (
process.stdinend/error,process.stdouterror inrelay.ts) already callsdispatcher.invalidateClient(), which firesnotifyClientDetached. The socket-write-throw path inwriteFramewas the one inconsistency.Fix & design rationale
Fail-fast (reuse existing reconnect + reattach), not a new resend buffer. On a primary write throw,
writeFramenow firesnotifyClientDetached(the primary client object stays in the map because it is reused acrosssetWritereconnects, but listeners must still fire — exactly asinvalidateClient()does for stdin/stdout death). This makes the socket-write-throw path consistent with every other primary-death path and engages the reconnect machinery promptly instead of after the keepalive timeout.Why fail-fast over a seq-keyed resend buffer:
reattachKnownPtys(ssh-relay-session.ts) replays each PTY's buffered output (the relay keeps the last ~100 KB inmanaged.buffered), and a PTY that exited during the failure window is torn out of the relay's map, so reattach throws not-found and the session emitspty:exitto the renderer. Pane death is therefore never lost.Known tradeoff (documented, acceptable): the reattach-not-found path surfaces a pane exit as
code: -1rather than the process's real exit code. The renderer treats any exit as pane death; the exact code is informational. The guarantee the task required —pty.exitis never lost — holds. Preserving the exact code would require the resend buffer, which is out of proportion to the benefit; noted as possible follow-up.Repro / evidence
src/relay/dispatcher.test.ts→detaches the primary client when its write throws:notify('pty.exit', …).onClientDetachedlistener is never called (verified: the test fails on pre-fixdispatcher.ts—expected "vi.fn()" to be called with arguments: [ 1 ]).setWritereconnect makes the client usable again (subsequentpty.datareaches the new sink).Screenshots
No visual change.
Testing
pnpm typecheckoxlinton changed files cleansrc/relay/dispatcher.test.ts(23),pty-handler.test.ts(63),integration.test.ts(27) pass in isolation. New regression test fails on pre-fix, passes on post-fix.AI Review Report
Reviewed the re-entrancy of firing
notifyClientDetachedmid-notify()loop: the primary is not deleted from the client map, so iteration stays valid; non-primary deletion behavior is unchanged. Confirmed the fs-handler detach listener (releaseClientWatches+wakeAllAckWaiters) already runs on primary death viainvalidateClient(), so this path introduces no new listener behavior — only makes the write-throw case consistent. Cross-platform: pure JS control-flow, no path/shell/OS assumptions; the relay runs on Linux/macOS/Windows remote hosts. Git operations are unaffected.Security Audit
No input handling, command execution, path, auth, secrets, or IPC surface added or changed. The change only alters when an existing detach notification fires on a failed write; it cannot be driven by remote input beyond causing the write to fail, which already triggers the same recovery for non-primary clients.
Notes
Fail-fast reuses the reconnect + PTY-reattach machinery in
ssh-relay-session.ts. If exact exit-code fidelity across a primary write blip is ever required, that would need a bounded seq-keyed retransmit buffer (deliberately out of scope here).Made with Orca 🐋