Skip to content

Extract nineteen web API domains and harden runtime ownership - #30

Merged
MyButtermilk merged 42 commits into
mainfrom
claude/web-api-hardening-and-route-split
Aug 16, 2026
Merged

Extract nineteen web API domains and harden runtime ownership#30
MyButtermilk merged 42 commits into
mainfrom
claude/web-api-hardening-and-route-split

Conversation

@MyButtermilk

@MyButtermilk MyButtermilk commented Aug 14, 2026

Copy link
Copy Markdown
Owner

Ergebnis

Dieser PR zerlegt die große aiohttp-Komposition in 19 neue, klar verantwortete Route-Domänen, härtet Nebenläufigkeit und persistente Jobs und aktualisiert den Produkt-Runtime auf CPython 3.14.7.

Die öffentlichen HTTP-Routen bleiben unverändert: 164 eindeutige Method/Path-Paare mit identischem kanonischem Route-Hash. src/web_api.py sinkt von 21.083 auf 19.763 Zeilen (−1.320).

Oberste Änderungen

1. Tiefere API-Domänen

Neu extrahiert wurden Runtime, ONNX, YouTube, Transcript, Settings, Local Polishing, Device, Outlook Calendar, Meeting Import, Voice Component, File Transcription, WebSocket, Live Mic, Meeting Capture, Meeting Workspace, Meeting Processing, Meeting Artifacts, Meeting Catalog und Meeting Device Readiness.

Domäne Registrierungen
Runtime 15
ONNX 4
YouTube 4
Transcript 6
Settings 2
Local Polishing 4
Device 4
Outlook Calendar 6
Meeting Import 6
Voice Component 13
File Transcription 1
WebSocket 1
Live Mic 6
Meeting Capture 4
Meeting Workspace 8
Meeting Processing 4
Meeting Artifacts 5
Meeting Catalog 4
Meeting Device Readiness 3

Das sind 100 neue modulare Registrierungen. Einschließlich der schon vor diesem PR vorhandenen Meeting-Delivery-Domäne liegen 103 Registrierungen in 20 Route-Modulen. Vierzehn neue Domänen verwenden schmale Controller-Ports; fünf sind controller-freie Collaborator-Domänen. Die exakten Produktionsadapter werden in den jeweiligen Route-Tests gepinnt.

2. Sichere Async-, Cancellation- und Audio-Ownership

AsyncTaskSupervisor, wiederholungsfeste Cancellation-Barrieren und AudioAdmissionOwner geben Tasks, Thread-Arbeit und der prozessübergreifenden Native-Audio-Lease einen eindeutigen Besitzer. Acquire/Transfer/Loss/Release/Shutdown sind serialisiert; unbekannte Stop- oder Persistenzzustände bleiben fail-closed und retrybar.

Meeting, Live Mic, Device Test und Voice Enrollment teilen die Audio-Admission ohne private Parallelzustände. Ein Meeting wird erst nach dauerhaftem recording-Commit als durable markiert.

3. Restart-sichere File-/YouTube-Jobs

Transcript und Job benutzen dieselbe ID. Enqueue, Adoption, Cancel, Retry, Delete und Resume laufen durch eine serialisierte Lane. Terminale Zustände werden Parent-first persistiert; terminal_projection_pending überlebt Neustarts, verhindert Provider-Replay und wird erst nach Source-Cleanup gelöscht. Commit-then-raise, CAS-Verlust, wiederholte Cancellation, Poison-Pages und Crashfenster sind adversarial getestet.

4. Datenschutz und Meeting-Lifecycle

Voice-Library-Consent wird an HTTP- und SQLite-Schreibgrenzen geprüft; unbekannter Store-Zustand liefert redigierte 503-Antworten. Preview-/Profilantworten sind allowlisted und geben bei deaktiviertem oder unbekanntem Consent weder Embeddings noch PCM aus.

Meeting Start/Pause/Resume/Stop, Workspace-Edits, Processing, Exporte/Playback/E-Mail, Katalog/Discard und Device Readiness besitzen jetzt explizite Transportgrenzen. Finalizer-Reservierung, Audio-Lease, Native Shell, Recorder und durable Zustandswechsel bleiben controller-owned.

5. Datei-Ingest und Upload-Policy

File Transcription besitzt Multipart-Parsing, begrenztes Streaming, ffmpeg-Vorbereitung, Workspace-Cleanup und die eine dauerhafte Job-Übergabe. Eine immutable FileUploadPlan friert Providerroute, Byte-Limits und geprüfte Labels einmal ein. Meeting Import konsumiert dieselbe typisierte Upload-Policy.

6. Frontend-Polish und Browserdiagnostik

Die Zusammenfassungsansicht hat jetzt eine wirklich sticky Inhaltsnavigation und einen sticky, abgerundeten Zusammenfassungs-Header; beim Scrollen gibt es kein Pixel-Driften und keinen scharf abgeschnittenen Kartenrand mehr.

Der reale Desktop-Smoke scheitert jetzt fail-closed bei unerwarteten Console-, Page- und Request-Fehlern. Nur eng begrenzte, phasen- und URL-spezifische React-Query-Abbrüche/Discard-404s sind erlaubt.

7. CPython 3.14.7

Build, CI, Runtime-Lock, Cache-Keys, Tests und Release-Policy verwenden nun exakt CPython 3.14.7. Das übernimmt die rund 499 Maintenance-Korrekturen gegenüber 3.14.6. JIT, Free-Threading und benutzerdefinierte Runtime-Varianten bleiben bewusst deaktiviert, bis installierte A/B-Evidenz ihre Produktreife belegt. Das bestehende CP314-no-BLAS-Wheel bleibt ABI-kompatibel; seine historische Build-Provenienz wird nicht umgeschrieben.

Verifikation (f38d2a4c)

  • Python 3.14.7 Vollsuite: 3.817 passed, 8 skipped.
  • Ruff repository-wide: grün; Ruff format: 449 Dateien.
  • Mypy: 68 Source-Dateien grün.
  • Frontend mit Node 26.5.0: Typecheck und Build grün; 153/153 Tests.
  • Route-Parität: 164/164; kanonischer SHA-256 9fa7c32e2b1e3b8fa2372165a373c4aa3a3ac7fd168cf6b7d50630569da59308.
  • Echter Chrome/Vite-Sticky-Test: ToC-Drift ≤0,375 px, Summary-Header 0 px, 19-px-Radius, keine Browserfehler.
  • Echter Tauri/WebView2→Rust-Sidecar→Python/ONNX→SQLite→React-Smoke: ok=true; Recording→Pause→Resume→Finalize→Ready, Reprocessing, Workspace, Artifacts, Catalog/Discard und Live Mic verifiziert; unerwartete Browserfehler 0.
  • Privacy-minimaler E2E-Report: tmp/meeting-e2e/runs/6aace1f158794349813d62086a7efb8a/result.json, SHA-256 3C2C1A456E3F203C7C4054F747EA9F39A7680226755A5D28CEDB7AC2EAA0879A.

Bewusste Evidenzgrenze

Der Desktop-Smoke benutzt synthetisches Audio. Er beweist den echten Tauri-/Rust-/Python-/SQLite-/React-Pfad, aber nicht, dass Mikrofon und Kamera während einer physischen Microsoft-Teams-Konferenz aktiv bleiben. Scriber öffnet keine Kamera und verwendet WASAPI Shared Mode; die reale Teams-Koexistenz bleibt trotzdem ein separater Operator-Test und wird hier nicht als bestanden behauptet.

Verbleibende Komposition

Der PR beendet nicht jede Extraktion. Insbesondere Speaker-Labeling sowie Shell-/Replay-/SPA-Reste bleiben in create_app. Künftige Slices sollen die abgeschlossenen Ownership-Grenzen bewahren und keine breiten Ports mit Controller-Privatfeldern einführen.

claude and others added 8 commits August 14, 2026 12:47
asyncio holds only a weak reference to a running task, so a task that no
caller stores can be garbage collected mid-await. Six fire-and-forget
tasks were scheduled that way. The overlay stop path was the most
visible: pressing stop on the recording overlay scheduled
stop_listening() with nothing holding it, so the recording could keep
running.

Give each detached task an owner:

- ScriberWebController gains _spawn_detached()/_wait_for_detached_tasks(),
  matching the existing _metrics_persist_tasks pattern, and shutdown now
  drains them alongside the metric and transcript writes.
- MeetingLiveTranscriber holds its backpressure report task.
- _AnalyzerCache holds pending async analyzer cleanups.

Enable Ruff's ASYNC and RUF rule families so this class of defect is
caught at lint time. Rules that do not fit the codebase are switched off
with the reason recorded next to them: ASYNC109 is a Trio convention,
ASYNC230/ASYNC240 need latency measurements on meeting-length audio
before the blocking-IO sites are reworked, and RUF001 flags deliberate
typography in the localized export templates. Tests and scripts poll
external state on purpose, so ASYNC110 and the style-only RUF rules are
relaxed there via per-file-ignores; the three production polls that wait
on third-party state carry an inline noqa with its reason.

The newly enabled rules also surfaced a real typing defect:
unload_model(model_name: str = None) declared an implicit Optional.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Kide1C8g57tv4dTwT6yKXX
First domain lifted out of web_api.create_app, as a template for the rest.
create_app is ~5,900 lines holding 135 nested handler definitions; because
each handler closes over factory locals, none of them can be imported or
tested without building the whole application.

Two new modules:

- src/api/http_security.py holds the transport security helpers
  (loopback checks, session token parsing, attachment headers). These were
  private to web_api, so any extracted route module would have had to
  import web_api and form a cycle. web_api keeps private aliases, leaving
  its existing call sites untouched. The helpers still read
  SCRIBER_SESSION_TOKEN at call time, which is how the tests steer them.

- src/api/runtime_routes.py holds the 15 runtime, diagnostics, and health
  routes as module-level handlers, following the shape already used by
  meeting_delivery_routes: dependencies arrive through a frozen service
  dataclass resolved from a typed web.AppKey. APP_SHUTDOWN_EVENT moves
  here too, since the shutdown handler is its only reader.

The route table is unchanged: method, path, and handler name are identical
for all 164 routes before and after.

src/api is now typechecked, in pyproject and in the CI tranche. That
needed one annotation in meeting_delivery_routes, and the workflow guard
test moves with it.

tests/api/test_runtime_routes.py covers the extracted domain against a
stub controller, without create_app, a pipeline, or an audio device --
which is the point of the change.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Kide1C8g57tv4dTwT6yKXX
Continues the split started with the runtime domain, using the same
shape: module-level handlers, dependencies through a service dataclass
resolved from a typed web.AppKey, and a register_*_routes entry point.

src/api/onnx_routes.py takes the four local-model routes. src.onnx_stt
stays behind function-local imports so listing models never drags
onnxruntime into a process serving only cloud providers.

src/api/youtube_routes.py takes search, lookup, the thumbnail proxy, and
transcribe. safe_thumbnail_url and read_limited_response_body moved with
it: both were private to web_api but used only here. The proxy keeps its
guarantees -- HTTPS only, an allowlist of the two YouTube CDN hosts,
every redirect target revalidated against that allowlist instead of
letting aiohttp follow it, and a hard cap on the buffered body.

src/api/app_keys.py holds APP_HTTP_SESSION, the one key that both
web_api and an extracted domain need. Keys with a single reader stay with
their domain.

Two consequences worth noting in review:

- The ONNX download progress callback created a broadcast task per
  update without keeping a reference, so it could be collected before
  the client saw the progress. The service dataclass now holds those
  tasks until they finish, and a test covers it.
- Three tests in test_web_api_security.py patched get_video_by_id,
  search_youtube_videos, and _safe_youtube_thumbnail_url on web_api.
  Those names resolve in the route module now, so the patch targets move
  with them. Expect the same for the remaining domains.

Route table unchanged: the same 164 method/path pairs before and after.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Kide1C8g57tv4dTwT6yKXX
…ler ports

Two follow-ups on the same PR.

The support-bundle test wrote a fixture named `scriber "support".zip` and
failed on the Windows runner with OSError 22: Windows rejects a quote in a
path outright, so the file could never be created there. The header rules
it was checking are still worth covering, so they move to a parametrised
test on attachment_content_disposition itself -- quote, backslash, slash,
and CRLF injection -- which needs no filesystem. The end-to-end test keeps
a Windows-legal non-ASCII name and now asserts both halves of the header:
the ASCII fallback and the RFC 5987 filename*.

That was the only failure in the run; the other 3,321 tests passed.

src/api/controller_port.py replaces `controller: Any` in the three route
services with Protocols. They are structural, so ScriberWebController
satisfies them by having the methods -- nothing to register, no base class,
no import back into web_api. Each domain declares only the slice it uses:
the ONNX domain needs `broadcast` alone, YouTube needs one coroutine, and
only the runtime domain names the diagnostics surface.

Nothing enforces those Protocols on its own. web_api is outside the mypy
tranche, so the register_*_routes call sites are unchecked and a renamed
controller method would keep compiling and fail on the first request.
tests/api/test_controller_port.py closes that: it compares every declared
port member against the controller for existence, async-ness, parameter
names and kinds, and defaults. Verified by mutation -- changing a default
and renaming a method each make it fail.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Kide1C8g57tv4dTwT6yKXX
Written by the harness when this session subscribed to PR #30 to watch
its CI. Tooling configuration only -- no product code, no test, no gate.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Kide1C8g57tv4dTwT6yKXX
Written by the harness when this session scheduled a CI check-in for
PR #30. Tooling configuration only -- no product code, no test, no gate.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Kide1C8g57tv4dTwT6yKXX
@MyButtermilk MyButtermilk changed the title Fix detached asyncio tasks, widen the Ruff gate, and start splitting web_api Supervise backend async lifecycles and split initial web API domains Aug 14, 2026

Copy link
Copy Markdown
Owner Author

CI red on Rust fmt, clippy, and tests — not caused by this diff. Re-queued the failed job; details below in case the flake is worth chasing separately.

Root cause is one test, not four. synthetic_meeting_capture_relays_raw_system_and_aec_clean_frames panicked first:

audio_sidecar.rs:6591: synthetic Meeting source must carry a signal

It panicked while holding the shared audio env test lock, which poisoned it. The other three then failed at audio_sidecar.rs:5646 with PoisonError without running any of their own assertions:

  • synthetic_pcm_fixture_loader_is_microphone_only_and_redacts_its_path
  • synthetic_pcm_fixture_loader_rejects_relative_paths_without_disclosure
  • wasapi_capture_is_enabled_by_default_unless_disabled

Reported result was 92 passed; 4 failed; the real count is one failure plus three casualties.

Why it is not from this PR: the diff is 36 Python files, one workflow, pyproject.toml (ruff/mypy config only), and one settings JSON — no .rs, no Cargo.toml, no src-tauri. scriber-audio-sidecar is built from unchanged sources. The same job passed at 13:55 on c8be07f, whose Rust tree is byte-identical to the commit that just failed; only Python changed in between. Same code, same job, green then, red now.

Two things worth a separate look, both out of scope here since this PR deliberately contains no Rust:

  1. The failing assertion ("must carry a signal") reads like a timing- or amplitude-sensitive check on synthetic capture — a plausible flake on a loaded runner.
  2. The poisoning turns any single failure in that group into a four-test cascade that buries the original panic. Taking the lock with recovery (unwrap_or_else(|e| e.into_inner())) in the test helper would keep the first failure legible and stop the amplification. Happy to open a separate PR for that if useful.

Generated by Claude Code

claude added 4 commits August 14, 2026 16:19
…r methods

Fourth domain out of create_app, and the first whose handlers reached into
controller internals: the in-memory history index, the deleted-transcript
tombstones, the summary single-flight registry, and the durable
summary-state writes.

Widening the port to name those would have written the boundary violation
down as a contract and frozen the underscore names as de facto API. The
controller instead grew a small public surface shaped by what the routes
need:

- transcript_view() returns one normalized TranscriptView. A transcript
  reaches a route either as a live TranscriptRecord or, once evicted, as
  the durable row; every reader used to repeat that fallback field by
  field, six times over in the export handler alone. The branch now exists
  once and route modules never see TranscriptRecord.
- summarize_transcript() owns the whole lifecycle -- single-flight
  registration, the pending/completed/failed transitions, their durable
  writes, and the history broadcast -- and returns a SummaryOutcome. The
  route maps that domain result onto a status code; deciding what happened
  and deciding how to report it are now separate.
- has_transcript_record() and transcript_was_deleted() replace the two
  remaining direct reads.

Keeping the summary lifecycle on the controller also keeps the existing
tests valid: test_summary_retry_api.py patches _save_transcript_summary_
state_async and _broadcast_history_updated on the controller instance, and
those patch targets are unchanged. Moving that code into the route module
would have silently disabled them, which is the migration cost this PR
already hit once with the YouTube tests.

web_api.py: 21,083 -> 20,297 lines against main. Route table unchanged:
the same 164 method/path pairs.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Kide1C8g57tv4dTwT6yKXX
…ains

Three domains rather than one. Grouping them by their adjacency in
create_app would have lumped together concerns that share nothing but a
line number.

Settings routes are thin because update_settings already owns the settings
lock, validation, and persistence protocol behind its public signature.
The ~390 lines of _update_settings_unlocked stay where they belong.

Local polishing earns its own module through its error taxonomy, not its
size. A nested closure that every handler in the group called is now a
typed failure-code table beside the handlers: each bounded code maps onto
its own status and public message, and an unexpected failure still falls
back to the generic unavailable response rather than reaching the client
verbatim. A test covers that a raised path never appears in a body.

Device routes cover microphone enumeration plus the autostart pair. The
installed Tauri shell owns autostart, so those endpoints stay to answer an
older frontend explicitly instead of 404ing. The contract is now returned
as a copy, so one response cannot mutate the next -- also covered.

None of these handlers touched controller internals, so all three ports
name only public methods. All three joined the drift guard, verified by
mutation.

web_api.py: 21,083 -> 20,143 lines against main. Route table unchanged:
the same 164 method/path pairs.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Kide1C8g57tv4dTwT6yKXX
Eighth domain out of create_app, and the first that needs no controller.
Every handler talks to the calendar collaborator the controller happens to
hold; nothing else about the controller is involved. The service depends
on the calendar directly, so the port describes what the domain actually
uses instead of routing through an object it does not need.

The calendar is resolved per request through a provider callable rather
than captured at registration. Passing it eagerly looked cleaner and broke
77 tests: create_app is built in several lifecycle tests with a controller
stub that never materializes a calendar, and reading the attribute during
composition turned a lazy per-request lookup into a hard requirement.
The provider keeps the original timing.

Two behaviours are load-bearing and now covered directly. The OAuth
callback is opened by the system browser rather than the frontend, so it
answers in HTML and never leaks an exception detail into that page -- a
test asserts a secret in the raised error stays out of the body. And every
sync failure is recorded on the calendar before the response is built, so
a degraded connection stays visible in status() instead of vanishing with
the request; that is asserted for all three failure classes.

The callback's nesting is flattened: authorization failure and first-sync
failure were distinguished by an inner try inside an outer catch-all. They
are now sequential, which is equivalent for every realistic path and makes
the degrade-but-stay-connected rule legible.

The port is checked against OutlookCalendarService rather than the
controller, so the drift guard gained a separate case, plus one asserting
authorization_pending is still a property (a property is not collected as
a method).

web_api.py: 21,083 -> 20,036 lines against main. Route table unchanged:
the same 164 method/path pairs.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Kide1C8g57tv4dTwT6yKXX
Prerequisite for extracting the Meeting import domain, and useful on its
own: Live Mic file uploads and Meeting recording imports validate the same
filenames against the same extensions and render the same size limits, but
those helpers were private to web_api, so no route module could reach them
without importing the module that registers it.

src/api/upload_policy.py now owns filename sanitisation, the accepted media
extensions, and limit formatting. web_api keeps private aliases, so its
existing call sites are untouched.

The rules were previously untested in their own right. They now are, and
one of the new tests had to be rewritten after it failed: the traversal
case is platform-shaped. On Windows a backslash is a separator and
Path.name drops the prefix; on POSIX it is an ordinary character that the
invalid-character substitution replaces instead, so "..\..\escape.wav"
becomes "escape.wav" on one platform and ".._.._escape.wav" on the other.
Both are single flat components, which is the property that matters, so the
test asserts that invariant rather than a literal string.

Also covered: Windows reserved device names get prefixed, long names stay
bounded while keeping their extension, and sanitisation is idempotent.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Kide1C8g57tv4dTwT6yKXX

Copy link
Copy Markdown
Owner Author

CI red on Python full suite for c00a4de — a wall-clock flake, not the diff. Re-run is queued behind the still-running jobs in that workflow; I'll trigger it once the run settles.

The failure

tests/test_web_api_jobs.py::test_youtube_attempt_lease_covers_download_and_long_local_diarization
AssertionError: assert 'failed' == 'completed'

3,451 passed, 1 failed. All 17 tests added in this commit passed.

Why it is timing, not the change

The test compresses a 90-second lease (_TRANSCRIPT_ARTIFACT_LEASE_TTL_SECONDS) into roughly 160 ms of real time — asyncio.sleep(0.07) for the download phase and asyncio.sleep(0.09) for the simulated Sherpa pass — while the fake store enforces expiry against a live time.monotonic():

def renew_attempt_lease(self, _attempt_id, *, owner, expected_version, ttl_seconds):
    if time.monotonic() >= self.expires_at:
        raise ArtifactConflict("Attempt lease has expired")

Any stall longer than the remaining margin — GC pause, CPU contention on a shared Windows runner — makes the renewal miss its window, the attempt is marked failed, and the assertion reports exactly that. Sub-100 ms margins against a real clock is the flake profile.

Why it is not this commit

The diff is three files: a new src/api/upload_policy.py, its tests, and web_api replacing the moved definitions with aliases. What moved is filename sanitisation, the accepted media extensions, and byte-limit formatting. None of those names appear anywhere in tests/test_web_api_jobs.py, and none are on the YouTube attempt-lease or diarization path. The same test passed on be93cdd eleven minutes earlier; the only delta since is the upload policy move.

Worth fixing separately (out of scope here, no Python job changes in this PR): the test would be deterministic with an injected clock instead of time.monotonic(). It is asserting an ordering property — that one guard spans both the pre-provider download and the post-provider speaker phase — which does not need real time to demonstrate. Happy to open a separate PR if useful.


Generated by Claude Code

@MyButtermilk MyButtermilk changed the title Supervise backend async lifecycles and split initial web API domains Supervise backend async lifecycles and extract eight web API domains Aug 14, 2026
claude added 2 commits August 14, 2026 18:26
Prerequisite for both remaining domains. The Meeting import protocol and the
speaker-profile handlers each depend on helpers that were private to
web_api: to_thread_cancellation_barrier (51 call sites),
await_with_delayed_cancellation (30), and remove_tree_if_exists (17).
No route module can reach them without importing the module that registers
it.

src/runtime/cancellation.py now owns them, beside task_supervisor: that
module owns the lifetime of work nobody awaits, this one owns the boundary
of work that must not be abandoned. Call sites use the shared names
directly rather than through aliases, as the upload-policy convention
requires; the one test that reached through web_api moves with them.

Writing their first direct tests surfaced dead code, which is left alone
here. to_thread_cancellation_barrier ends with a handler meant to convert
a failed mutation into the caller's pending CancelledError:

    except BaseException:
        if pending_cancel is not None:
            logger.exception("Durable thread mutation failed while its caller was canceling")
            raise pending_cancel from None

That branch is unreachable. The loop awaits asyncio.shield(worker), which
re-raises the worker's exception, so control leaves the loop before
worker.result() is ever called; the caller sees the mutation's own failure
and that log line never fires. Confirmed by reproducing the loop in
isolation.

The test pins the actual behaviour rather than the apparent intent, and
says why in its docstring. Choosing between them changes a contract, not a
structure, so it does not belong in a move commit -- flagged for review.

web_api.py: 21,083 -> 19,874 lines against main.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Kide1C8g57tv4dTwT6yKXX
Both helpers in src/runtime/cancellation.py ended with a handler meant to
convert a failed mutation into the caller's pending CancelledError. Neither
branch could run: the loop awaits asyncio.shield(worker), which re-raises
the worker's exception, so control left the loop before worker.result() was
ever reached.

Two consequences, both reproduced before the fix:

- A caller that had asked to stop received the mutation's own exception
  instead of CancelledError. Callers of these helpers are already unwinding
  and handle CancelledError there, so an unexpected exception skipped that
  cleanup and escaped into a shutdown path.
- The worker's exception was never retrieved, so asyncio additionally logged
  "exception in shielded future" on its own.

The loop now breaks on a non-cancel exception and lets worker.result()
observe it, which restores the intended contract and retrieves the
exception. The failure is no longer silent either: the barrier already had
a logger.exception for it that had never fired, and the second helper --
whose branch swallowed the failure outright -- gained one.

Verified by reproducing both paths in isolation before and after, and by
the full suite: no new failures across the 81 call sites these helpers
have.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Kide1C8g57tv4dTwT6yKXX

Copy link
Copy Markdown
Owner Author

CI red on Python full suite for 4f04630. I treated this one as a likely regression from my own commit — that commit changes behaviour at 81 call sites — but the evidence says timing. Failed job re-queued.

The failure

tests/test_meeting_live_stt.py::test_stop_cleans_up_when_backpressure_callback_fails
AssertionError: assert supervisor is not None and supervisor.done()
  Task cancelling name='meeting-live-supervisor-system' ... .done() -> False

3,464 passed, 1 failed.

Why it is timing

The test constructs the stream with stop_timeout_s=0.08. stop() then splits that budget:

cleanup_reserve_s = min(1.0, max(0.05, self._stop_timeout_s * 0.5))   # 0.05
graceful_deadline = deadline - cleanup_reserve_s                       # ~30 ms left
...
await asyncio.wait_for(asyncio.shield(task), timeout=remaining)

So the graceful wait gets roughly 30 ms of wall clock, minus whatever the backpressure report consumed first. Blow that budget and wait_for raises TimeoutError, graceful stays False, and the supervisor is still in cancelling when the assertion runs — which is exactly the observed state. The captured stderr confirms the intended path was taken: the callback failure was caught and logged as designed.

Why it is not my commit

4f04630 touches three files: src/runtime/cancellation.py, its tests, and a doc paragraph. src/meeting_live_stt.py does not import that module — not directly and not transitively. I walked the import graph from meeting_live_stt and src.runtime.cancellation does not appear in it. stop() uses its own asyncio.wait_for(asyncio.shield(task)), not either helper.

I also ran the failing test 42 times locally on CPython 3.14 — 30 runs idle, 12 more under eight busy cores — and it passed every time. That does not prove innocence on its own, which is why the import-graph check above matters more.

Worth fixing separately. This is the second wall-clock flake on this PR, after test_youtube_attempt_lease_covers_download_and_long_local_diarization. Both assert an ordering property against a real clock with a sub-100 ms margin, on a shared Windows runner. Both would be deterministic with an injected clock, and neither needs real time to demonstrate what it is checking. Happy to open a separate PR for the pair if that is useful — it does not belong in this one, which contains no changes to those tests.


Generated by Claude Code

claude added 2 commits August 14, 2026 18:47
Claude Code writes auto-approved permissions into
.claude/settings.local.json as tools get used. That file was tracked, so
every session that reached for a new tool produced a diff, and clearing it
by hand only lasted until the next call -- it was reverted twice in this
PR alone.

The name says what the file is for: settings.local.json is per machine.
The durable half moves to .claude/settings.json, which is the shared file
and is now tracked in its place; the local one is gitignored and stays on
disk, free for the harness to write to.

The content is copied verbatim rather than pruned. Everything in it is
project-wide for a Windows-only app -- the venv paths and powershell entry
included -- so this is a move, not a policy change, and effective
permissions are unchanged for anyone who already has the local file.

To undo: delete .claude/settings.json, drop the .gitignore entry, and
git add .claude/settings.local.json.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Kide1C8g57tv4dTwT6yKXX
Tenth domain out of create_app: the six Meeting import routes and the
durable upload protocol behind them move to src/api/meeting_import_routes.py.
web_api drops another 294 lines and no longer holds the streaming, staging,
commit, and cancellation logic for recording imports.

Unlike the other extracted domains this one takes no controller port. The
protocol needs the durable store, the progress broadcast, both task
registries and the shutdown flag -- a surface composition supplies as loose
attributes rather than as a class, and several suites build the app around a
stub that owns exactly those attributes and nothing else. A port would force
each of them to reimplement the protocol just to be allowed to reach it. So
the dependencies arrive as an immutable MeetingImportDeps bundle, assembled
per request: the store is replaced after the app exists, and the shutdown
flag has to answer for the moment cancellation lands rather than for the
moment the upload began.

Both task registries are created on demand and attached to the controller.
Reading a job never needed them, so requiring them up front made listing an
import fail on a controller that owns only the durable store; attaching
rather than handing over keeps the upload and the cancellation racing it in
the same dict.

Verified: all 164 route method/path pairs identical to the previous revision,
the full suite's failure set unchanged against its baseline (2822 passed, the
same 60 environment-only failures), tests/test_meeting_api.py green at 123,
ruff and mypy clean.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Kide1C8g57tv4dTwT6yKXX
@MyButtermilk MyButtermilk changed the title Supervise backend async lifecycles and extract eight web API domains Supervise backend async lifecycles and extract nine web API domains Aug 14, 2026
claude added 2 commits August 14, 2026 19:42
Tenth domain out of create_app: the Voice Library speaker model, the voice
library erase, and the diarization component move to
src/api/voice_component_routes.py. web_api drops another 129 lines.

What makes these six routes a domain rather than six status endpoints is the
biometric opt-in. Voice Library processing needs consent, and that flag is
durable and cross-process, so it can be withdrawn from another Scriber window
while a download is mid-flight. The download therefore checks consent three
times -- before starting, after staging, and again after the atomic replace
that no cancellation can interrupt -- and deletes the model it just promoted
if consent is gone by then. That invariant is now stated in one place instead
of being spread across a create_app closure.

The domain takes two dependency providers rather than one bundle. The Voice
Library routes and the diarization routes share no collaborator, so a single
bundle made reading the model's status fail on a composition that never built
a diarizer. For the same reason the settings-persist hook stays a deferred
call: only the erase route performs it. The wiring guard pins that
granularity, not just the presence of each dependency.

The enrollment routes stay in web_api for now: they carry the audio admission
concern, which needs its own owner first. The two voice mutation locks
therefore stay with composition and are handed to this domain per request;
they move here when enrollment does.

Verified: all 164 route method/path pairs unchanged, the full suite's failure
set identical to its baseline (2838 passed, the same 60 environment-only
failures), tests/test_meeting_api.py green at 123, ruff and mypy clean.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Kide1C8g57tv4dTwT6yKXX
The rules for holding this process's native-audio lease lived as six
module-level functions in web_api, each taking `controller: Any` and
re-deriving eight loose attributes with getattr defaults. They now live in
src/runtime/audio_admission.py as AudioAdmissionOwner, beside the two other
runtime owners: task_supervisor owns the lifetime of work nobody awaits,
cancellation owns the boundary of work that must not be abandoned, this owns
the lifetime of an exclusive resource.

The rules are worth naming, because each exists for a way the lease can be
lost: renew on a heartbeat that never runs twice; adopt this controller's own
pending-to-durable Meeting rebinding rather than mistaking its deliberate CAS
bump for a loss; fail closed against a genuinely different controller; let a
Meeting ride out an unavailable store because its durable row still excludes a
newcomer, while Live Mic gives up before the TTL can lapse; and release a
lease a worker thread created after cancellation already won the race, rather
than leaving a phantom owner for a full TTL.

The claim and the heartbeat task are still stored on the controller, reached
through accessors passed to the owner. That is deliberate: the suite guarding
this concern reads and writes those attributes directly in about forty places,
and relocating the state in the same change that relocates the rules would
remove the check that proves the move was faithful. web_api keeps the six
functions as delegators, so every call site and every monkeypatch seam is
unchanged; _persistent_audio_admission stays the primitive the owner resolves
through on each use, so a substituted store still takes effect.

This is the owner the speaker-profile routes were waiting on: they can now
depend on one collaborator with a documented surface instead of on eight
controller attributes and six web_api functions.

Verified: 23 focused tests, each of the three subtlest rules killed by its own
mutation (never adopting the rebinding, moving the Live Mic failure limit,
skipping the shutdown rollback) and by no other test; all 164 route
method/path pairs unchanged; the full suite's failure set identical to its
baseline (2861 passed, the same 60 environment-only failures); ruff and mypy
clean.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Kide1C8g57tv4dTwT6yKXX
@MyButtermilk MyButtermilk changed the title Supervise backend async lifecycles and extract nine web API domains Give async lifecycles and the audio lease explicit owners, and extract ten web API domains Aug 14, 2026
Serialize native-audio loss and cleanup, enforce Voice Library consent, and make File/YouTube job settlement restart-safe. Finish the Voice enrollment extraction and add adversarial coverage for repeated cancellation, cross-process races, ambiguous commits, and terminal projection recovery.
@MyButtermilk MyButtermilk changed the title Give async lifecycles and the audio lease explicit owners, and extract ten web API domains Harden async, audio, and durable job ownership; extract ten web API domains Aug 15, 2026
Persist the terminal parent in each scheduler/recovery test and use the exact transcript/job identifier contract. This removes the accidental dependency on a developer transcripts.db that clean Windows CI exposed.
@MyButtermilk MyButtermilk changed the title Harden async, audio, and durable job ownership; extract ten web API domains Harden ownership boundaries and extract eleven web API domains Aug 15, 2026
@MyButtermilk MyButtermilk changed the title Harden ownership boundaries and extract eleven web API domains Harden ownership boundaries and extract twelve web API domains Aug 15, 2026
MyButtermilk and others added 5 commits August 15, 2026 06:44
Repointing db._DB_PATH does not move a connection that is already open:
db._get_connection() memoises one per thread. A worker that touched the shared
repository database earlier keeps reading it, which is precisely the ordering
that made this test fail in CI -- the durable Voice Library gate came back
opted in, the constructor disagreed with Config, and the second full persist it
scheduled superseded the JSON-only migration under test.

Closing connections and clearing the thread-local before the patch is what
test_completed_live_mic_session_is_persisted_and_searchable already does in this
file for the same reason.

Verified with a plugin that opens the shared database and sets the flag before
collection, reproducing that worker ordering: with the path patch alone the test
still fails; with the cache reset it passes.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Kide1C8g57tv4dTwT6yKXX
@MyButtermilk MyButtermilk changed the title Harden ownership boundaries and extract twelve web API domains Harden ownership boundaries and extract thirteen web API domains Aug 15, 2026
@MyButtermilk MyButtermilk changed the title Harden ownership boundaries and extract thirteen web API domains Harden ownership boundaries and extract fourteen web API domains Aug 15, 2026
@MyButtermilk MyButtermilk changed the title Harden ownership boundaries and extract fourteen web API domains Harden ownership boundaries and extract fifteen web API domains Aug 15, 2026
@MyButtermilk MyButtermilk changed the title Harden ownership boundaries and extract fifteen web API domains Harden ownership boundaries and extract sixteen web API domains Aug 15, 2026
@MyButtermilk MyButtermilk changed the title Harden ownership boundaries and extract sixteen web API domains Harden ownership boundaries and extract seventeen web API domains Aug 15, 2026
@MyButtermilk MyButtermilk changed the title Harden ownership boundaries and extract seventeen web API domains Harden ownership boundaries and extract eighteen web API domains Aug 15, 2026
@MyButtermilk MyButtermilk changed the title Harden ownership boundaries and extract eighteen web API domains Extract nineteen web API domains and harden runtime ownership Aug 16, 2026
@MyButtermilk
MyButtermilk merged commit 583e05a into main Aug 16, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants