Release the PTY renderer-delivery queue on every teardown path; guard the synthetic-kill window (STA-1397)#7972
Open
brennanb2025 wants to merge 3 commits into
Open
Release the PTY renderer-delivery queue on every teardown path; guard the synthetic-kill window (STA-1397)#7972brennanb2025 wants to merge 3 commits into
brennanb2025 wants to merge 3 commits into
Conversation
…own path (STA-1397) Root cause of the monotonic main-process memory creep under sustained multi-terminal load: pendingData (the main->renderer PTY delivery batcher in ipc/pty.ts) accumulates every output chunk via string append, and its drain is gated on renderer pty:ackData with 512KB/PTY + 8MB global in-flight caps. Panes that never ACK — hidden worktrees, windows that never mounted a terminal pane — wedge the window shut, so main retained the entire output of every hidden terminal as cons-string trees (~312MB after a 3-minute 17-terminal burn; heap snapshots + live-retained allocation profile in the Linear ticket). Worse, the map is closure-local and only sendPtyExitToRenderer deleted entries, so SSH connection shutdown and the synthetic-kill dedup window stranded entries (and their in-flight budget, throttling ALL terminals) for the process lifetime. Fix: - Cap pendingData per PTY (1MB, trim to the newest 640KB). Seq stays truthful for the kept tail; the next payload carries dropped:true and the renderer routes it into the existing hidden-output snapshot-restore machinery instead of painting the spliced stream. - clearPtyRendererDeliveryState hook: clearProviderPtyState now flushes and releases the queue + in-flight budget on every teardown path (SSH connection drop, synthetic kill, daemon spawn failure), not just renderer-observed exits. - Drop renderer delivery for output arriving inside the synthetic-kill dedup window; the daemon's socket backlog can drain for seconds after pty:kill and used to repopulate the just-deleted entry with no cleanup left to run. Validated on the profiling rig (17 hidden terminals, ~1MB/s each, 3min): heapUsed churn floor 90MB vs 250-400MB rising before; post-burn+GC retained strings 55MB vs 348MB; pendingData entries bounded ~1.3MB each vs 14-23MB unbounded; no entries survive worktree removal. Co-authored-by: Orca <help@stably.ai>
Contributor
📝 WalkthroughWalkthroughPTY provider cleanup now flushes remaining renderer data and releases pending and in-flight delivery bookkeeping. PTY output received during the synthetic-kill deduplication window is ignored. Tests cover both teardown under backpressure and late output after a synthetic kill. 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches📝 Generate docstrings
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
#7630 landed the same 2MB cap + droppedBacklog restore flag this branch introduced independently — adopt its mechanism wholesale (constants, flag naming, renderer threading, cap tests). This branch keeps what #7630 does not cover: releasing the queue + in-flight ACK budget on teardown paths that skip sendPtyExitToRenderer (SSH connection drop, synthetic-kill dedup), and dropping renderer delivery for output arriving inside the synthetic-kill window. Co-authored-by: Orca <help@stably.ai>
…t hook during the merge Co-authored-by: Orca <help@stably.ai>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Completes the STA-1397 main-process memory-creep fix on top of
#7630(merged), which independently landed the same 2 MBpendingDatacap +droppedBacklogsnapshot-restore flag this branch originally carried. After rebasing onto that work, this PR keeps the two failure modes#7630does not cover — both proven by heap-snapshot forensics on the profiling rig (details on LinearSTA-1397):Stranded delivery state on teardown.
pendingDataandrendererInFlightCharsByPtyare closure-local toregisterPtyHandlers; onlysendPtyExitToRendererreleased them. Teardown paths that skip it — SSH connection shutdown (clearPtyOwnershipForConnection) and the synthetic-kill dedup early-return — stranded up to a full cap's worth of queue per PTY for the process lifetime, and, worse, left the PTY's in-flight chars counted against the global 8 MB ACK budget forever, silently throttling delivery for all terminals until app restart. In the rig repro, 3 of 17 terminals stranded ~36 MB each through a plain worktree removal (pre-cap; still 2 MB + wedged budget each post-#7630). Fix: aclearPtyRendererDeliveryStatehook thatclearProviderPtyStatenow calls on every teardown path — it flushes the queued tail first (preserving the "last ≤8 ms of output" guarantee regardless of call order relative tosendPtyExitToRenderer), then releases the queue and the in-flight budget.Repopulation inside the synthetic-kill window.
pty:killsends a synthetic exit that flushes and deletes the entry, butonDatahad no kill-window guard: the daemon's socket backlog keeps draining output for seconds after the kill, repopulatingpendingData, and the real exit is then swallowed byconsumeSyntheticKillExit— so nothing ever cleans the entry again. Fix: drop renderer delivery for ids inside the dedup window (runtime/status consumers above still see every byte).Also extends the
#7630platform analysis: the never-ACK wedge is not Windows/Linux-only. A window that has never mounted a terminal pane never attaches the renderer PTY dispatcher at all (ensurePtyDispatcheris transport-created), so nothing ACKs regardless of background-throttling — reproduced on macOS with CLI-created terminals in background worktrees (141 → 568 MB heapUsed in 3 min, zero recession).Diff after reconciliation
src/main/ipc/pty.ts+src/main/ipc/pty.test.tsonly. The cap, flag, preload/dispatcher/renderer threading are#7630's as merged; this branch's duplicate implementation and duplicate cap test were dropped in the merge commit.Screenshots
No visual change.
Testing
pnpm lint(oxlint clean on changed files)pnpm typecheck(node + web)pnpm test— fullsrc/main/ipc/pty.test.tssuite (227 tests incl.#7630's cap tests) + 419 renderer terminal-pane testsAI Review Report
Checked the merge reconciliation specifically: no remnants of the superseded
droppedflag or 1 MB cap constants remain (grep-verified); the teardown hook uses#7630'smakePtyDataPayloadsignature and passesdroppedBacklogthrough the exit flush; hook is assigned before any teardown can run (module-level no-op default otherwise); flush-then-clear ordering analyzed against bothclearProviderPtyState-before-sendPtyExitToRenderer(daemon/SSH exits) and the reverse (kill paths) — no double-send, no tail loss. Cross-platform: pure IPC/queue bookkeeping, no platform branches; original evidence spans Windows (#7375 profiling) and macOS (rig).Security Audit
No new input handling, command execution, path handling, auth, or secrets. Changes are cleanup/bookkeeping on existing
pty:datadelivery structures; the kill-window guard only drops renderer-bound delivery for a PTY already being torn down.Notes
STA-1397: daemon-side unbounded socket write buffering to a slow main (1.9 GB RSS during the rig burn) deserves its own ticket; the bounded main-side headless emulators (~8 MB/terminal) could take an idle-eviction cap.#7630's naming/mechanism wholesale to avoid two flags meaning the same thing; a small follow-up commit restores main's exact formatting on 7 unrelated files that the local pre-commit hook had rewrapped during the merge.