feat!: default every spawned worker to the socket transport

Windows was held on `transport=stdio` until the share path had a green
run behind it.  It has one now -- all 17 jobs, including
`windows-latest, pypy-3.11`, which was the case that took four rounds to
pin down.  So the platform split goes away: a worker execnet spawns
itself gets a socket, by whichever handoff the platform has.

That also retires the machinery the split needed.  `default=` existed
only to say "can, but does not yet", and `default_spawn_transport()` only
to compute it; with the answer the same on both platforms they collapse
back into `resolve_transport(spec, available=...)`, which is where this
started before Windows needed an exception.

Two tests lose their `posix_only` marks and now cover Windows: that the
protocol really is off fd 0/1 (asserting `--protocol-share` there rather
than `--protocol-fd`), and that a worker's stdout reaches the
coordinator.  The visible consequence on Windows is that one: remote
`print()` now goes to the coordinator's console instead of being folded
onto stderr, matching POSIX.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Author: RonnyPfannschmidt

CHANGELOG.rst

@@ -20,11 +20,12 @@
   ``import execnet, trio`` probe used to decide whether a ``python=``
   interpreter can host a worker directly.
 * The protocol no longer has to be the worker's stdin/stdout. A new
-  ``transport=socket|stdio`` spec key selects; it defaults to ``socket`` on
-  POSIX and ``stdio`` on Windows, where neither ``pass_fds`` nor ``ssh -R``
-  unix-socket forwarding is available. ``socket`` means an inherited
-  socketpair for ``popen`` and an ``ssh -R``-forwarded unix socket that the
-  worker dials back on for ``ssh=``/``vagrant_ssh=``.
+  ``transport=socket|stdio`` spec key selects, and ``socket`` is the default
+  for every worker execnet spawns itself: an inherited socketpair for
+  ``popen`` on POSIX, and a socket duplicated with ``socket.share()`` on
+  Windows. ``ssh=``/``vagrant_ssh=`` gateways use an ``ssh -R``-forwarded
+  unix socket the worker dials back on, which needs ``AF_UNIX`` and
+  ``StreamLocal`` forwarding, so those stay on ``stdio`` on Windows.
 * **A worker's stdio now belongs to the code it runs.** It used to be
   redirected to the null device so it could not corrupt the protocol, which
   meant a remote ``print()`` went nowhere at all. With the socket transport
@@ -51,12 +52,12 @@
   the two errors you got depended on whether the peer had closed yet --
   so the more useful message lost a race.
 * Whether a socket can be handed to a worker is now settled by *doing* it
-  once rather than by looking for ``socket.share``. An implementation with
-  the name but not a working call -- PyPy on Windows -- otherwise passed
-  the check and failed later, at the point where the only thing left to
-  tell the coordinator was a closed socket. Such a host now refuses the
-  request up front, and ``socket=``/``installvia=`` gateways are skipped
-  there rather than failing.
+  once -- sharing to our own pid and rebuilding the result -- rather than
+  by looking for ``socket.share``. An implementation with the name but not
+  a working call would otherwise pass the check and fail later, at the
+  point where the only thing left to tell the coordinator is a closed
+  socket. A host that genuinely cannot hand a socket over refuses the
+  request up front instead.
 * A socket gateway that fails to start no longer takes down the gateway it
   was requested through. It ran as a task on that worker's host, so an
   unsupported sub-gateway used to cost that coordinator as well.
@@ -80,10 +81,12 @@
   ``socket.share()`` (``WSADuplicateSocket``); because that needs the
   child's pid, the flag travels in argv and the blob follows in the config
   on stdin. The blob is bound to that one pid, so it is inert to anything
-  else. This makes ``transport=socket`` work on Windows for ``popen`` and
-  fixes ``socket=``/``installvia=`` gateways served from a Windows host.
-  Windows still *defaults* to ``transport=stdio``; ask for
-  ``transport=socket`` to opt in.
+  else. This makes ``transport=socket`` the Windows default too, and fixes
+  ``socket=``/``installvia=`` gateways served from a Windows host. A socket
+  is handed over *as a socket* rather than reduced to its handle: rebuilding
+  one from a bare handle makes the constructor re-derive family/type/proto
+  by querying it, which PyPy on Windows cannot do to a handle that came
+  from ``WSADuplicateSocket``.
 * ``transport=socket`` on a gateway that cannot provide it is now an error
   at ``makegateway`` time naming the platform, instead of a gateway that
   waits for a worker which was never able to reach back. ssh dial-back
@@ -95,8 +98,8 @@
   POSIX-only. Windows has no async equivalent -- trio's Windows pipe streams
   need OVERLAPPED handles registered with an IOCP, and the stdio a process
   inherits is an ordinary synchronous pipe -- so those reads and writes now
-  run in the thread pool. The socket transport, where it is available, still
-  needs no threads.
+  run in the thread pool. This only affects ``transport=stdio``; the socket
+  transport, which is the default, needs no threads.
 * New ``EXECNET_PROVISION_WHEEL`` environment variable naming a prebuilt
   wheel to provision remote workers from, instead of resolving the
   coordinator's version from an index or building one from its source tree.

src/execnet/_provision.py

@@ -107,35 +107,17 @@ def ssh_dialback_available() -> bool:
     return hasattr(_socket, "AF_UNIX") and not sys.platform.startswith("win")
 
 
-def default_spawn_transport() -> str:
-    """What a worker we spawn gets when the spec does not say.
-
-    Windows can *serve* the socket transport (see :func:`socket_share_required`),
-    but stdio is what has years of exercise there while sharing is new, so it
-    stays the default until the share path has CI behind it.
-    ``transport=socket`` opts in.
-    """
-    if socket_share_required():
-        return "stdio"
-    return "socket" if socket_handoff_available() else "stdio"
-
-
-def resolve_transport(
-    spec: Any, *, available: bool = True, default: str | None = None
-) -> str:
-    """The transport for ``spec``: explicit if given, else the default.
+def resolve_transport(spec: Any, *, available: bool = True) -> str:
+    """The transport for ``spec``: explicit if given, else the best available.
 
     ``available`` is the caller's capability for *its* kind of gateway --
     :func:`socket_handoff_available` for a worker we spawn,
     :func:`ssh_dialback_available` for one that has to reach back to us.
     Asking for a transport that cannot work is an error at makegateway time,
-    rather than a hang once nobody connects.  ``default`` decouples "can
-    work" from "is what you get by default".
+    rather than a hang once nobody connects.
     """
     requested: str | None = getattr(spec, "transport", None)
     if requested is None:
-        if default is not None:
-            return default
         return "socket" if available else "stdio"
     if requested not in TRANSPORTS:
         raise ValueError(f"unknown transport {requested!r} (known: {list(TRANSPORTS)})")

src/execnet/_trio_gateway.py

@@ -1102,9 +1102,7 @@ async def connect_popen_worker(spec: Any) -> tuple[ByteStream, trio.Process]:
     from . import _provision
 
     transport = _provision.resolve_transport(
-        spec,
-        available=_provision.socket_handoff_available(),
-        default=_provision.default_spawn_transport(),
+        spec, available=_provision.socket_handoff_available()
     )
     if transport == "stdio":
         return await connect_command_worker(popen_worker_argv(spec))

testing/test_cli.py

@@ -217,15 +217,18 @@ def test_no_config_source_is_an_error(self) -> None:
 
 
 class TestTransportSelection:
-    def test_defaults_per_platform(self) -> None:
+    def test_a_spawned_worker_defaults_to_the_socket_transport(self) -> None:
+        # every platform now: POSIX hands over the fd, Windows duplicates the
+        # socket with share().  Only a host that can do neither gets stdio.
         spec = execnet.XSpec("popen")
-        expected = "stdio" if sys.platform.startswith("win") else "socket"
+        expected = "socket" if _provision.socket_handoff_available() else "stdio"
         assert (
             _provision.resolve_transport(
-                spec, default=_provision.default_spawn_transport()
+                spec, available=_provision.socket_handoff_available()
             )
             == expected
         )
+        assert expected == "socket"
 
     def test_explicit_wins(self) -> None:
         assert _provision.resolve_transport(
@@ -252,15 +255,12 @@ def test_asking_for_an_impossible_transport_is_an_error(self) -> None:
                 execnet.XSpec("ssh=host//transport=socket"), available=False
             )
 
-    def test_windows_can_serve_the_socket_transport_but_does_not_default_to_it(
-        self,
-    ) -> None:
-        # sharing is new; stdio is what has years of Windows behind it
+    def test_windows_hands_a_socket_over_by_duplicating_it(self) -> None:
+        # `subprocess` refuses pass_fds there, so the capability comes from
+        # socket.share() instead -- including on PyPy, once the socket is
+        # handed over as a socket rather than rebuilt from its handle
         if _provision.socket_share_required():
             assert _provision.socket_handoff_available()
-            assert _provision.default_spawn_transport() == "stdio"
-        else:
-            assert _provision.default_spawn_transport() == "socket"
 
     def test_ssh_cannot_dial_back_on_windows(self) -> None:
         # no AF_UNIX in CPython there, and Win32-OpenSSH cannot -R a unix socket
@@ -279,16 +279,22 @@ def test_socket_transport_roundtrip(self) -> None:
         finally:
             group.terminate(timeout=5.0)
 
-    @posix_only
     def test_socket_transport_keeps_the_protocol_off_stdio(self) -> None:
+        # whichever handoff this platform has, the point is the same: the
+        # protocol is named on the command line, so it is not fd 0/1
+        expected = (
+            "--protocol-share"
+            if _provision.socket_share_required()
+            else "--protocol-fd"
+        )
         group = execnet.Group()
         try:
             gateway = group.makegateway("popen")
             channel = gateway.remote_exec(
                 "import sys; channel.send(sys.argv)",
             )
             argv = channel.receive(TESTTIMEOUT)
-            assert "--protocol-fd" in argv
+            assert expected in argv
         finally:
             group.terminate(timeout=5.0)
 
@@ -305,7 +311,6 @@ def test_stdio_transport_still_works(self) -> None:
 class TestWorkerStdio:
     """Whose stdio is it? The code the worker runs, unless told otherwise."""
 
-    @posix_only
     def test_socket_transport_inherits_stdio(self, capfd) -> None:
         group = execnet.Group()
         try:

testing/test_execmodel_trio.py

@@ -68,7 +68,7 @@ def test_single_thread_on_main(self, trio_gw: Gateway) -> None:
         # async form on Windows, so those reads and writes run in the thread
         # pool.  Only the socket transport is genuinely single-threaded.
         transport = _provision.resolve_transport(
-            trio_gw.spec, default=_provision.default_spawn_transport()
+            trio_gw.spec, available=_provision.socket_handoff_available()
         )
         if transport == "socket":
             assert active == 1