native-devtools sockets: multi-server contention (attach-to-every-sim + latest-bind-wins + unlink-on-stop)

[filip131311]

Summary

When more than one argent tool-server runs on the same host (a second dev
instance, a stale/orphaned instance, or another agent session's server), the
native-devtools injection sockets become a three-way hazard:

1. Attach-to-every-sim: each tool-server's simulator watcher initializes a
   NativeDevtools service — and binds /tmp/argent-nd-<UDID>.sock — for
   every booted simulator, even devices that server will never drive. A
   session doing Android-only work through a dev tool-server knocked out an
   unrelated iOS session's devtools on a different simulator.
2. Latest-bind-wins: the blueprint unlinks any existing socket file before
   binding (fs.unlinkSync(socketPath) in native-devtools.ts, "remove stale
   socket file from a crashed previous run"). A live server's socket is
   indistinguishable from a crashed one, so every server start silently steals
   all future app connections; the losing server keeps its established
   connections but every relaunched app connects to the thief.
3. Unlink-on-stop kills survivors: when the last-binding server stops, its
   cleanup removes the socket file — the surviving servers' listeners are gone
   from the filesystem, so no app can connect to anyone until something
   rebinds.

Observed failure signatures (all reproduced live, 2026-08-11)

• flow launch step: could not connect to native devtools for <bundleId>… —
  the app connected to another server's socket; the flow's server waits out its
  connect window in vain. Reproduced on demand: the failure windows correlate
  1:1 with another server instance's start timestamps (22:07, 22:10:57,
  22:15:51, 22:16:55 across two independent sessions).
• ViewInspector RPC timed out: <method> mid-flow — when the steal happens
  between a flow's app relaunch and its first RPC.
• With three sessions (two scoped experiments + one unrelated Android A/B loop
  restarting its dev server per run), no session could keep a stable devtools
  connection on any simulator.

Also worth noting operationally: there is no in-band way to identify or
negotiate with the process holding the socket — lsof + process-tree forensics
was the only way to find the owner.

Sketch of remedies (not prescriptive)

• Scope: only claim devtools sockets for devices a server is actually asked to
  drive (lazy claim on first tool call for that UDID), or honor an explicit
  device allowlist. (An ARGENT_DEVTOOLS_ONLY_UDID env filter prototype made
  two concurrent sessions coexist cleanly during this investigation.)
• Ownership: before unlinking an existing socket, probe it — a live listener
  answers; only unlink dead sockets. On shutdown, only unlink a socket the
  server still owns (compare inode/identity).
• Diagnosability: embed owner pid + port in a sidecar file next to the socket
  so the "stale or duplicate argent server" error can name the actual owner.

Environment

argent 0.20.0 (installed) + worktree dev builds of the tool-server, macOS,
iOS 26.4/26.5 simulators, Expensify NewDot test app. Full timeline and process
trees available on request.

[filip131311]

Impact data point: a concurrent measurement session lost a full timed round and ~40 minutes across three separate socket-steal incidents on 2026-08-11; two of the three were caused by a tool-server instance belonging to a session doing Android-only work (no shared device with the victim, which was driving an iOS simulator).

[latekvo]

[Diplomat, Opus 5]: A correction to the shutdown remedy sketched above ("On shutdown, only unlink a socket the server still owns (compare inode/identity)") — that is necessary but not sufficient, and on its own it changes nothing.

net.Server.close() unlinks the pipe path itself. libuv records the pathname at uv_pipe_bind and unlinks it by name on close, so a server that bound the path before a rival rebound it still deletes whatever socket is at that name when it shuts down. The explicit fs.unlinkSync(socketPath) in dispose() is not the only deletion — it merely usually wins the race with libuv's.

Measured on this host (node v24.14.1, macOS 24.5.0), two servers on one path, the second self-healing through EADDRINUSE exactly as bindNativeDevtoolsUnixSocket does:

A ino 211437427
B err EADDRINUSE
B ino 211437428
after a.close(), exists: false     <- A's close removed B's inode
after b.close(), exists: false

I gated dispose()'s explicit unlink on socketInode(socketPath) === boundSocketIno while reviewing #712 and the survivor's socket was still gone; that is how this surfaced. So point 3 needs the path renamed out from under the close and restored afterwards, or a listener that notices its own inode has vanished and rebinds — an inode comparison in front of unlinkSync alone is dead code.

Related to #712 only in that its holdsEndpoint() check makes the resulting state diagnosable: the survivor reads its own socket as gone and tells the agent to restart this tool-server (which rebinds the path) instead of blaming the app's dylib. The deletion itself is untouched and long predates that PR.