Route browser fs and logs endpoints directly to the VM

[tnsardesai]

Summary

Adds fs and logs/stream to the default KERNEL_BROWSER_ROUTING_SUBRESOURCES prefixes, so every /browsers/{id}/fs/* operation (JSON, binary read/write, multipart upload, watch SSE) and /browsers/{id}/logs/stream goes straight to the browser VM through the existing route cache (base_url + JWT) instead of the control plane.

Browser lifecycle/metadata, extensions, replays, telemetry/events, and anything else under logs/ stay on the control plane. KERNEL_BROWSER_ROUTING_SUBRESOURCES still overrides the list, and an empty value still disables routing entirely.

Two Python-only fixes were needed before fs could route directly:

Indexed multipart array names, scoped to fs.upload. Python previously serialized entries as repeated files[][dest_path] / files[][file] parts. The filesystem parser rejects the empty index, so fs.upload already failed through the control plane as well as against the VM. fs.upload now flattens its own body with indexed names and asks extract_files for matching file-part names, producing files[0][dest_path] / files[0][file]. This fixes the existing upload failure in addition to enabling direct routing. The client's generic multipart array encoding is untouched, so load_extensions, deployments.create, extensions.create, fs.upload_zip and any future multipart array keep their existing wire format.

Body replay safety on a stale-JWT fallback. A stale direct-to-VM 401/403 is retried by rebuilding the request from the original options, so it is only safe when the body can be serialized again byte for byte. The client now:

• evicts the stale route from an httpx response event hook, which runs once the status is known and before any body is read. Eviction used to depend on the retry path or the status-error path, so max_retries=0 left a dead route cached — and because httpx reads a non-streamed body inside send(), a 401/403 whose body read fails surfaces as a connection error that never reaches the status-error path at all. The hook is prepended, so a caller-supplied response hook that reads a failing body or raises cannot pre-empt it, and it is registered once per route cache, so a copied client does not stack hooks;
• retries only when the body is demonstrably replayable — buffered bytes, or a multipart body whose file fields are in-memory or can actually be rewound. seekable() is not treated as proof: a wrapper can report True and still raise from seek(), which would have rendered the fallback body as an empty part, so the rewind httpx would perform is attempted (and the position restored) before classifying the body as replayable;
• records replayability in _prepare_request, before caller request hooks or custom httpx auth can buffer the built request while consuming the original body used for retries;
• marks the request when the response hook observes stale VM auth. If httpx then raises while eagerly reading that error body, both timeout and generic connection-error retry paths consult the recorded replayability instead of blindly rebuilding the request after route eviction.

Streamed bodies (a file object or iterator passed to fs.write_file) are never replayed after stale VM auth. They surface the auth error when its response body is readable and APIConnectionError when reading that error body fails; the caller's next call uses the control plane. Sync and async clients both changed.

Tests

tests/test_browser_routing.py:

• default prefix list and segment-boundary matching include fs and logs/stream, and still exclude telemetry/events, fsx/..., logs, logs/history, logstream, extensions, replays (asserted both at the matcher and through rewrite_direct_vm_options)
• fs JSON (list_files, move), binary read, binary write_file, indexed multipart upload, fs/watch/{id}/events SSE and logs/stream SSE all route to base_url with ?jwt=, no API-key Authorization, and preserved query strings
• async logs/stream cancellation reaches the transport
• stale-JWT fallback: buffered binary body and seekable multipart upload are replayed byte for byte on the control plane; a streamed body (sync iterator, async generator) and a multipart field that lies about seekable() each make exactly one VM call, no control-plane fallback, surface the original auth error, and still evict the route (sync + async)
• max_retries=0 401/403 for buffered and streamed bodies: original error raised, cache empty, next call goes to the control plane with bearer auth and no jwt
• with default retries and caller request hooks that eagerly buffer the built request, an unreplayable fs.write_file body plus a direct 401 (sync) / 403 (async) whose response body raises ReadError or ReadTimeout sends the complete body once, does not retry an empty write on the control plane, surfaces APIConnectionError or APITimeoutError, evicts the route, and leaves the next explicit call on the control plane
• with max_retries=0, direct 401/403 responses whose body stream raises still surface APIConnectionError, evict the route, and leave the next call on the control plane
• with a caller-supplied response hook already registered on a custom httpx client, a direct 401 (sync) / 403 (async) whose body read fails from inside that hook still evicts the route: the eviction hook runs first, the caller hook still executes, and the next call reaches the control plane with bearer auth and no jwt
• a copied client registers exactly one eviction hook, ordered ahead of caller hooks, and shares the route cache
• direct_vm_request_body_is_replayable classification for empty, buffered, streamed, seekable-multipart, lying-seekable(), no-seekable()-with-failing-seek(), and closed-file bodies
• load_extensions still sends extensions[][name] / extensions[][zip_file], and the generic multipart array path still sends array[], pinning that the indexed encoding is scoped to fs.upload
• indexed_multipart_body skips omitted values
• env override excluding fs/logs, and empty env disabling routing

Each new stale-JWT/eviction test was confirmed to fail against the corresponding pre-fix implementation.

Ran locally: full pytest (709 passed, 3257 skipped), targeted pydantic v1 routing tests (60 passed), and targeted Ruff check/format plus mypy for the changed source files.

Live validation

Ran against staging with real headless browsers: fs.write_file, fs.read_file (binary), fs.list_files, fs.upload (bytes and BytesIO entries), fs.upload_zip, fs.download_dir_zip, fs.create_directory, fs.file_info, fs.set_file_permissions, fs.delete_directory, fs.watch.start/stop and logs.stream all hit https://<browser-host>/browser/kernel/...?jwt=... with no API-key header and returned the expected data. Uploaded files read back with the correct per-entry contents, which is what the indexed multipart names fix. telemetry/events and the browser delete stayed on the control plane. Re-verified after narrowing the multipart change and switching to logs/stream.

---

Note

Medium Risk
Changes default request routing for fs/logs and central retry/eviction logic for direct VM auth failures, where incorrect replay could truncate uploads or leave sessions stuck on bad routes.

Overview
Direct-to-VM routing now includes fs and logs/stream in the default KERNEL_BROWSER_ROUTING_SUBRESOURCES list, so browser filesystem calls and live log SSE go to the cached VM URL with JWT query auth instead of the control plane. Extensions, replays, telemetry/events, and other logs/* paths stay on the API origin; env overrides are unchanged.

fs.upload multipart is fixed for the VM (and the prior control-plane failure) by flattening the body to indexed field names (files[0][dest_path] / files[0][file]) via new indexed_multipart_body; other multipart endpoints keep generic array[] encoding.

Stale VM JWT handling is reworked: httpx response hooks evict dead routes as soon as 401/403 is known (before body reads that can fail and wedge the cache). prepare_direct_vm_request records whether the body can be replayed; retries to the control plane (status-based or connection/timeout) only happen when replay is safe—streamed writes, non-rewindable upload files, and known-unreplayable bodies are not blindly retried. Sync/async Kernel clients override _should_retry_on_connection_error for this behavior.

^{Reviewed by Cursor Bugbot for commit 92acf13. Bugbot is set up for automated code reviews on this repo. Configure here.}

[rgarcia]

reviewed alongside the go (#174) and node (#178) siblings, including a control-plane vs kernel-images surface comparison. all 16 fs/* and logs/stream endpoints match on params, body fields, 2xx codes and content types, and metro-api's /browser/kernel/* handler already wakes standby VMs and records session activity for direct requests, so the routing change itself looks safe.

the response-hook placement is sound: it fires inside send() before the body read, so the 401-with-failing-body case evicts, and it runs before _should_retry so the rebuilt request misses the cache. the replayable check matches httpx behavior (FileField.render_data seeks to 0 on each render, so probing seek(0) is the right test), and write_file with a file object correctly lands in the non-replayable branch. the body-read-failure and hook-ordering tests are the ones that justify the design.

Questions

• src/kernel/resources/browsers/fs/fs.py:514 — this is a generated file; is the array_format="indices" / indexed_multipart_body change captured as stainless custom code so it survives the next regen?
• src/kernel/resources/browsers/fs/fs.py:514 — main emits files[][dest_path], which the kernel-images upload parser rejects (empty index fails Atoi) and the control plane forwards untouched. was fs.upload already failing via the control plane? if so this is a user-visible bug fix worth calling out in release notes

Nits

• src/kernel/lib/browser_routing/routing.py:204 — docstring should mention the hook is inserted into a caller-supplied http_client's event_hooks
• tests/test_browser_routing.py — control-plane mocks return 204 for fs/upload / fs/write_file; production returns 201

[Sayan-]

Approving. Verified against a mock control plane and VM over real HTTP: fs/* and logs/stream route to the VM with ?jwt= and no Authorization, telemetry/events and replays stay on the control plane with bearer auth, indexed multipart names arrive as files[0][dest_path] / files[0][file], and the stale-JWT path replays a buffered body byte for byte while a streamed body surfaces the original AuthenticationError with the route evicted.

• p2: with max_retries=0 there is no transparent fallback even for a buffered body. The direct 401 surfaces as AuthenticationError and recovery depends on the eviction, so the caller's next call reaches the control plane. Node and Go fall back inline regardless of retry config. Measured buffered, iterator and file-object bodies at max_retries=0 and 2.

• p2: the old files[][dest_path] encoding was rejected outright rather than mis-paired. The in-VM handler parses the index with strconv.Atoi, which fails on an empty index and returns 400 invalid form field, so fs.upload was failing for every call including single-file ones. Worth reflecting in the release note.

• p2, pre-existing and out of scope: the new load_extensions assertion pins extensions[][name], which the control-plane rewriter regex does not match and the in-VM parser rejects.

[tnsardesai]

@rgarcia @Sayan- addressed the review notes across all three sibling PRs in one pass:

• Python Route browser fs and logs endpoints directly to the VM #164: documented that the eviction hook is installed into a caller-supplied http_client's event_hooks; changed fs.upload / fs.write_file mocks to the real 201 response; and updated the PR summary to call out that the old empty-index multipart encoding already made fs.upload fail through the control plane.
• Route browser fs and logs endpoints directly to the VM kernel-node-sdk#178: made the content-type removal happen before routedInit is built, changed filesystem success mocks to 201, and added a real undici regression test. The test confirms a consumed stream never reaches the control plane after the stale-JWT response and the route is evicted; the existing APIConnectionError behavior is unchanged.
• Route browser fs and logs endpoints directly to the VM kernel-go-sdk#174: documented why bare fs is intentional while only logs/stream is routed, and changed the upload fallback mock to 201.

On the generated Python fs.py change: it will be preserved through the post-merge stlc custom-code seal. The seal-tracking PR in kernel/kernel still needs to land after these SDK changes merge and before the next regeneration.

I left the cross-SDK max_retries=0 asymmetry, the generated Node writeFile example, and the pre-existing Python/Node load_extensions multipart issue unchanged as out of scope.

Validation is green: Python 705 passed, Node 422 passed plus build/lint, Go go test ./... plus lint, and all three PR CI runs passed.

[Sayan-]

• rechecked the post-approval commit, current diff, green CI, and clean merge with current main; no new blocking findings.

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit bb70883. Configure here.}

[Sayan-]

• p1 src/kernel/_base_client.py:1015-1025,1599-1609 reproduced with default retries, an unseekable fs.write_file body, and a direct 401 whose response body raised ReadTimeout: the direct route received payload, the control plane received an empty body, and the call returned normally.
• p1 src/kernel/lib/browser_routing/routing.py:278-292 reproduced with a caller-supplied httpx request hook that called request.read(): replayability changed from false to true, the direct route received payload, fallback received an empty body, and the call returned normally. A custom auth handler requiring the request body produced the same result.

[tnsardesai]

@Sayan- fixed in 5d8ec7f. You were right about both gaps: timeout exceptions used a separate unconditional retry branch, and replayability was being classified after request hooks/custom auth could buffer the built request while consuming the original retry source. Direct requests now record replayability in _prepare_request, before httpx hooks/auth run, and both timeout and generic connection-error retry paths consult that recorded value after stale auth. The sync/async tests now run both ReadError and ReadTimeout with request hooks that eagerly read the request; each case verifies one complete VM write, no automatic control-plane fallback, route eviction, the correct connection/timeout error, and a later explicit control-plane write. Full tests (709), pydantic v1 routing tests (60), lint, build, and CI are green.