Add isolated context mode: CPython in a child OS process

A mode where Python code cannot take the node down or run forever:
one child process per context over a Unix socket, signals with
SIGKILL as backstop, rlimits and cgroups, restart on crash. Same
public API; the context process is a gen_statem serving one caller
at a time. Validated on macOS and FreeBSD. Version 4.2.0.

Author: benoitc

CHANGELOG.md

@@ -1,5 +1,44 @@
 # Changelog
 
+## 4.2.0 (2026-08-29)
+
+### Added
+
+- **`isolated` context mode** - `py_context:new(#{mode => isolated})` runs
+  CPython in a child OS process per context, with the same `call/eval/exec`,
+  callback, `erlang.send`/`whereis`, worker-loop and pool API as the embedded
+  modes. It is the first mode with a hard bound: `py_context:interrupt/1`
+  stops a blocking C call (a signal in the child) and `SIGKILL` is the
+  backstop after `kill_after` ms; `py_context:kill/1` kills at once. `rlimits`
+  (`as`, `cpu`, `nofile`) and a cgroup v2 directory bound the child; a
+  segfault in a C extension returns `{error, {child_exited, {signal, 11}}}`
+  and the node survives. The child restarts on crash within a budget
+  (`restart`, `max_restarts`, `restart_period`); `py_context:child_info/1`
+  reports its OS pid. Children are reaped by the VM and exit when the BEAM
+  dies (socket EOF watchdog, `PR_SET_PDEATHSIG` on Linux, `PROC_PDEATHSIG_CTL`
+  on FreeBSD). `cgroup` is refused outside Linux; rlimits apply everywhere:
+  `as` is kernel-enforced on Linux and FreeBSD and enforced by an RSS
+  watchdog in the child on macOS (`{child_exited, {memory_limit, Bytes}}`).
+  Validated on macOS (arm64) and FreeBSD 14.3 (OTP 28, Python 3.11).
+- **`py_context:pass_fd/2`** - hands a file descriptor to an isolated child
+  over the control socket (`SCM_RIGHTS`), so `erlang.server.serve` works out
+  of process: Erlang binds once, N killable children accept.
+- **Pure-Python ETF codec** (`priv/_erlang_impl/_etf.py`) with the type
+  mapping of `py_convert.c`; the child needs no C extension. Integers beyond
+  64 bits round-trip exactly in isolated mode.
+- `py:python_executable/0`, `py:kill/1`, `py_nif:os_kill/2`.
+- `py_isolated` is a `gen_statem` (states `idle`, `{busy, Id}`, `looping`,
+  `stopping_loop`, `{restarting, Reason}`): `sys:get_state/1` and
+  `sys:trace/2` work on isolated contexts, requests arriving during a
+  restart are served by the new child, and `py_context:kill/1` returns once
+  the new child is up.
+- Timeouts on an isolated context cancel their own request only (queued
+  requests are dropped, the executing one is interrupted); the kill backstop
+  is bound to that request, so a busy shared context is never killed because
+  another caller gave up. Soak-tested: callback storms, interrupt/kill
+  storms, loop churn, 60 s mixed workload with resource counters checked.
+- Guide: `docs/isolated.md`, with what each of the three modes guarantees.
+
 ## 4.1.0 (2026-08-15)
 
 ### Added

README.md

@@ -603,6 +603,7 @@ When creating Python contexts, you can choose the execution mode:
 |------|----------------|-------------|
 | `worker` | Any | Dedicated pthread per context, main interpreter namespace (default) |
 | `owngil` | 3.14+ | Dedicated pthread + subinterpreter with its own GIL, true parallelism |
+| `isolated` | Any | CPython in a child OS process: killable, rlimit-bounded, crash-contained |
 
 ```erlang
 %% Default: worker mode (recommended)
@@ -612,8 +613,18 @@ When creating Python contexts, you can choose the execution mode:
 %% OWN_GIL mode for true parallelism (Python 3.14+ required)
 %% Each context runs in its own pthread with independent GIL
 {ok, Ctx} = py_context:new(#{mode => owngil}).
+
+%% Isolated mode: a child process per context. A stuck call is killed, a
+%% segfault only takes the child down, rlimits bound memory and CPU.
+{ok, Ctx} = py_context:new(#{mode => isolated, kill_after => 1000,
+                             rlimits => #{as => 512 * 1024 * 1024}}).
 ```
 
+**Isolated mode** is the only mode with a hard bound: `py_context:interrupt/1`
+stops a blocking C call, and `SIGKILL` is the backstop. It costs a process per
+context (about 16 MB and 40 ms to start) and roughly twice the call latency.
+See [Isolated Contexts](docs/isolated.md).
+
 **Worker mode is recommended** because it works with any Python version and automatically benefits from free-threaded Python (3.13t+) when available. Each context owns a dedicated pthread, providing stable thread affinity for libraries with thread-local state (numpy, torch, tensorflow).
 
 **Why OWN_GIL requires Python 3.14+**: Some C extensions (e.g., `_decimal`, `numpy`) have global state bugs in sub-interpreters on Python 3.12/3.13. These are fixed in Python 3.14.
@@ -629,6 +640,7 @@ py:execution_mode().  %% => worker | owngil
 |------|----------------|-------------|
 | `worker` (default) | Any | One pthread per context; true parallelism on free-threaded 3.13t+ |
 | `owngil` | 3.14+ | Per-interpreter GIL, true parallelism across contexts |
+| `isolated` | Any | One OS process per context, parallel and failure-isolated |
 
 ## Error Handling
 
@@ -651,6 +663,7 @@ py:execution_mode().  %% => worker | owngil
 - [Logging and Tracing](docs/logging.md)
 - [Asyncio Event Loop](docs/asyncio.md) - Erlang-native asyncio with TCP/UDP support
 - [Worker Loops](docs/workers.md) - Long-lived loops in owngil contexts, serving on sockets Erlang owns
+- [Isolated Contexts](docs/isolated.md) - Python in a child process: kill, rlimits, crash containment
 - [Reactor](docs/reactor.md) - FD-based protocol handling
 - [Security](docs/security.md) - Sandbox and blocked operations
 - [Changelog](https://github.com/benoitc/erlang-python/releases)

c_src/py_nif.c

@@ -36,6 +36,14 @@
  * - py_callback.c: Callback system and asyncio support
  */
 
+/* pthread_timedjoin_np (used to bound the owngil worker join on Linux)
+ * is declared by <pthread.h> only under _GNU_SOURCE. */
+#ifndef _GNU_SOURCE
+#define _GNU_SOURCE
+#endif
+
+#include <signal.h>
+#include <errno.h>
 #include "py_nif.h"
 #include "py_util.h"
 #include "py_event_loop.h"
@@ -8079,6 +8087,26 @@ static void unload(ErlNifEnv *env, void *priv_data) {
     /* Other cleanup handled by finalize */
 }
 
+/**
+ * @brief Send a signal to an OS process (kill(2)).
+ *
+ * Used by isolated contexts to SIGKILL their child. The caller holds the
+ * child's port open until exit_status arrives, so the pid cannot have been
+ * recycled.
+ */
+static ERL_NIF_TERM nif_os_kill(ErlNifEnv *env, int argc, const ERL_NIF_TERM argv[]) {
+    (void)argc;
+    int pid, sig;
+    if (!enif_get_int(env, argv[0], &pid) || !enif_get_int(env, argv[1], &sig) || pid <= 0) {
+        return enif_make_badarg(env);
+    }
+    if (kill((pid_t)pid, sig) == 0) {
+        return ATOM_OK;
+    }
+    return enif_make_tuple2(env, ATOM_ERROR,
+        enif_make_atom(env, errno == ESRCH ? "esrch" : errno == EPERM ? "eperm" : "einval"));
+}
+
 static ErlNifFunc nif_funcs[] = {
     /* Initialization */
     {"init", 0, nif_py_init, 0},
@@ -8220,6 +8248,7 @@ static ErlNifFunc nif_funcs[] = {
     {"create_test_pipe", 0, nif_create_test_pipe, 0},
     {"close_test_fd", 1, nif_close_test_fd, 0},
     {"dup_fd", 1, nif_dup_fd, 0},
+    {"os_kill", 2, nif_os_kill, 0},
     {"write_test_fd", 2, nif_write_test_fd, 0},
     {"read_test_fd", 2, nif_read_test_fd, 0},
     /* TCP test helpers */

docs/interrupts.md

@@ -65,12 +65,29 @@ context to deal with that:
 ok = py_context:destroy(Ctx).
 ```
 
+## Interrupting a blocking C call: isolated mode
+
+The limits below apply to the embedded modes. An `isolated` context runs
+Python in a child process, where an interrupt is a signal that lands inside
+`time.sleep`, a socket read or any other blocking call, and `SIGKILL` is the
+backstop if the signal is ignored:
+
+```erlang
+{ok, Ctx} = py_context:new(#{mode => isolated, kill_after => 1000}),
+{error, timeout} = py_context:eval(Ctx, <<"__import__('time').sleep(60)">>, #{}, 200),
+%% Usable at once, no 60 s wait
+{ok, 4} = py_context:eval(Ctx, <<"2+2">>, #{}, 5000).
+```
+
+`py_context:kill/1` kills at once. See [Isolated Contexts](isolated.md).
+
 ## Limits
 
 - CPython delivers an async exception at the next bytecode boundary. Code
   blocked inside a C call (`time.sleep`, a numpy kernel, a socket read) is
   not interrupted until that call returns. The call still times out on the
-  Erlang side; the context becomes usable once the C call finishes.
+  Erlang side; the context becomes usable once the C call finishes. Only
+  `isolated` mode interrupts such a call.
 - An interrupt targets the context, not an individual request. Interrupting a
   context that just finished one call and started another stops the new one.
 - `py:call/3,4` and `py:eval/1,2` use `infinity` by default. Pass an explicit