perf(tui): add the reviewed adaptive stream pacer

CHANGELOG.md

@@ -80,6 +80,7 @@ This release upgrades the engine and hardens the terminal agent. The three engin
 - Restored the streaming display coalescer. The raw engine wrapper that precedes every derived text and thinking delta was classified as a synchronous event, so each provider chunk cancelled the pending 16 ms display frame and forced an immediate render request; the coalescer was defeated exactly while a response streamed, which is the one time it exists for. Wrappers carrying text or thinking deltas, which the panel has always ignored, now stop at the renderer instead of reaching it; tool-call formation keeps its synchronous path, and every event the panel acts on arrives in the same order as before.
 - Enabled Node's V8 compile cache for the boot-path module graphs: interactive chat, `run`, `acp`, and native fleet workers. Locally spawned workers inherit the directory through their spawn environment and consume it before any of their children can see it; SSH-placed workers enable the remote install's own cache in-process at the worker entry, with nothing exported to their children. Read-only and dry-run commands never enable it, `paths` and bare `doctor` keep their promise that nothing is created, and the cache enables only on an initialized install so a home Clio never set up stays untouched. `NODE_COMPILE_CACHE` and `NODE_DISABLE_COMPILE_CACHE` always win, and a cache failure never affects a command. The import-only and real-PTY fill/hit observations, with their deliberately narrower endpoint names, are recorded in `docs/performance-methodology.md` rather than generalized into a per-command saving.
 - Corrected boot and TUI performance instrumentation before using it for 0.3.2 decisions. `first TUI paint` is now marked only after the first real frame has issued all of its stdout writes; explicit frame ids group diff, ANSI, hardware-cursor, and IME writes; canonical event and input ids correlate queue, panel, and committed-frame high-water marks; and stdout return, backpressure, and drain are recorded. The trace writer is bounded, asynchronous, awaited at shutdown, and nonfatal on storage failure. A deterministic contract plus a real built-CLI PTY harness cover first frame, input, resize, grouped writes, paused output, and process reaping. The documented metrics are stdout/PTY endpoints, never an in-process claim of literal glass latency.
+- Added the adaptive stream pacer as `terminal.smoothStreaming: off | auto | on`, with the conservative 0.3.2 default left at `off`. Only derived visible text and thinking are paced, through one ordered generation queue; raw text/thinking wrappers remain transparent and public events, transcript persistence, tool formation, cumulative tool state, and ordered boundaries remain synchronous. Pacing is grapheme-safe, arrival-credit and oldest-age bounded, self-stopping, folded-thinking aware, and forced to settle before abort, retry, submit, mode change, final return, or teardown. Fullscreen frozen scrolling and resize survive paced updates, and stdout backpressure gates frame construction rather than creating a second unbounded SSH buffer. `auto` bypasses non-TTY, remote/multiplexed, CI, accessibility-marked, and backpressured sessions; `CLIO_CODER_SMOOTH_STREAM=off` is the immediate per-process escape hatch. Deterministic fake-clock, ordering, grapheme, reset, folded-thinking, scroll/resize, no-drain, and built-CLI PTY contracts cover the rollout.
 - Removed the eager userland Undici graph from `web_fetch` and use the Node 22.19+ built-in Fetch, Headers, Request, Response, stream, and abort implementation. Localhost contracts preserve request headers/body, redirects, UTF-8 streaming and cancellation at the byte ceiling, external abort, timeout, HTTP previews, binary rejection, and transport errors on both supported Node lines; an installed-tarball turn invokes the real tool from a foreign working directory. The built graph contains no Undici source or dependency, while the measured size and deliberately mixed import-time observations are recorded without turning host noise into a boot claim.
 - Made codewiki's tree-sitter graph genuinely lazy and moved runtime indexing off the interactive event loop. Lightweight schema, artifact, and path modules preserve synchronous cached reads without evaluating the builder; actual full, stale, and incremental builds run in a dedicated worker and load only the required grammars. Session startup, parallel `code_nav` demand, mutation batches, explicit index/refresh, bootstrap, wiki grounding, and reset now share one per-workspace generation queue and cross-process lease, so an older build cannot overwrite newer state or resurrect a reset artifact, and shutdown drains admitted work. Built-source and installed-tarball coverage prove nested help is tree-sitter-free and write-free while a real foreign-cwd build loads the runtime and its vendored grammar; measurements and deliberately scoped import observations are in `docs/performance-methodology.md`.
 - Split `context`, `code_nav`, `verify`, `web_fetch`, `dispatch`, `monitor`, and `steer` into immutable lightweight tool surfaces and first-use implementation chunks. Registration order, provider schemas and descriptions, policy metadata, execution modes, argument normalization, admission, permissions, middleware, result shaping, and worker surface attestation remain registry-owned and eager; only an admitted runner imports code. Dispatch keeps its trusted plan and capacity-reservation identities in one synchronous admission controller shared with the lazy runner, deeply freezes every execution-affecting control before middleware or approval can observe it, binds the `apply_winner` repository destination into the approval hash and text, and releases a guard-blocked prepared admission exactly once. Workers import only the core tool bootstrap, so their built entry never evaluates the three orchestrator-only runners. Concurrent first calls share one import, an implementation whose surface drifted fails closed, unrelated tools remain absent, and a missing Clio-owned chunk carries the same named reinstall guidance as a missing command chunk. V8 coverage repeats the absent-before-use/present-on-invocation proof for every tool and the worker exclusion against both the source build and an installed tarball from a foreign working directory.

docs/architecture.md

@@ -272,6 +272,18 @@ decoding and before the application controller mutates editor, overlay, scroll,
 or submit state. The first frame whose input high water includes that id is the
 input-to-stdout-commit endpoint.
 
+Adaptive streaming remains inside that presentation boundary. One semantic
+classifier drops only transparent raw text/thinking mirrors, sends derived
+visible content through one generation/epoch FIFO, and treats every other
+transcript mutation as an ordered drain boundary. Pacer slices mutate the
+panel directly and are never re-emitted on the public bus, so session storage,
+replay/export, tool-call formation, and cumulative tool-result behavior keep
+their canonical synchronous inputs. Abort, retry, interrupt, submit, mode
+change, and teardown drain the queue and can await the containing committed
+frame. The stdout gate stops later frame construction after a false write and
+coalesces to current model state until `drain`; it is not installed for the
+default `off` path and therefore cannot become a second unbounded SSH buffer.
+
 Tracing is opt-in and content-free. Its bounded asynchronous writer never does
 filesystem append I/O on the render stack, and shutdown awaits a bounded flush.
 See [performance-methodology.md](performance-methodology.md) for vocabulary,

docs/configuration-and-targets.md

@@ -207,6 +207,7 @@ terminal:
   outputVerbosity: default
   tuiMode: regular              # regular terminal scrollback or fullscreen sticky layout
   fullscreenScrollbar: auto    # hidden, auto, or always in fullscreen mode
+  smoothStreaming: off         # off, conservative auto, or explicit on
 skills:
   trustProjectCompatRoots: false
 delegation:
@@ -446,7 +447,7 @@ The Settings Center organizes all configuration under four non-selectable group
 | **RUNTIME** | Budget (`budget`) | `budget.sessionCeilingUsd`, `defaults.maxTokens`, and `budget.concurrency` (restart required). |
 | **RUNTIME** | Compaction (`compaction`) | `compaction.auto`, `compaction.threshold`, and `compaction.excludeLastTurns`. |
 | **RUNTIME** | Retry (`retry`) | `retry.enabled`, `retry.maxRetries`, `retry.baseDelayMs`, and `retry.maxDelayMs`. |
-| **EXPERIENCE** | Terminal (`terminal`) | `terminal.showTerminalProgress`, `terminal.outputVerbosity` (`minimal`, `default`, `verbose`), `terminal.tuiMode` (`regular`, `fullscreen`), `terminal.fullscreenScrollbar` (`hidden`, `auto`, `always`), and `theme`. |
+| **EXPERIENCE** | Terminal (`terminal`) | `terminal.showTerminalProgress`, `terminal.outputVerbosity` (`minimal`, `default`, `verbose`), `terminal.tuiMode` (`regular`, `fullscreen`), `terminal.fullscreenScrollbar` (`hidden`, `auto`, `always`), `terminal.smoothStreaming` (`off`, `auto`, `on`), and `theme`. |
 | **EXPERIENCE** | Advanced (`advanced`) | `runtimePlugins`, `compaction.model`, `compaction.systemPrompt`, `delegation.defaults.connectTimeoutMs`, `delegation.defaults.turnTimeoutMs`, `delegation.defaults.permissionTimeoutMs`, `keybindings`, and `delegation.agents`. |
 
 `retry.streamStallMs` has no Settings Center row; edit it in `settings.yaml`.
@@ -495,6 +496,7 @@ Label to config path mapping:
 | Output detail | `terminal.outputVerbosity` (`minimal`, `default`, or `verbose`) |
 | TUI mode | `terminal.tuiMode` (`regular` or `fullscreen`, restart required) |
 | Fullscreen scrollbar | `terminal.fullscreenScrollbar` (`hidden`, `auto`, or `always`, restart required) |
+| Smooth streaming | `terminal.smoothStreaming` (`off`, `auto`, or `on`, live) |
 | Theme | `theme` |
 | Runtime plugins | `runtimePlugins` |
 | Compaction model | `compaction.model` |
@@ -608,6 +610,7 @@ Generic provider and transport errors are classified by transient retry rules, i
 | `terminal.outputVerbosity` | `default` | `minimal`, `default`, `verbose` | immediately |
 | `terminal.tuiMode` | `regular` | `regular`, `fullscreen` | restart |
 | `terminal.fullscreenScrollbar` | `auto` | `hidden`, `auto`, `always` | restart |
+| `terminal.smoothStreaming` | `off` | `off`, `auto`, `on` | immediately |
 | `modelSelector.favorites` | `[]` | list of strings | immediately |
 | `modelSelector.recentLimit` | `12` | integer ≥ 1 | immediately |
 | `keybindings` | `{}` | map of binding id to a key string or list of them | restart |

docs/environment-variables.md

@@ -42,6 +42,9 @@ Durable values live in the `guardrails:` section of settings.yaml (see [configur
 | `CLIO_CODER_SKILL_MARKETPLACE_INDEX` | unset | Skill-marketplace index path override (`src/domains/resources/skills/marketplace.ts`). |
 | `CLIO_CODER_MODEL_CATALOG_DIRS` | unset | Extra model-catalog directories (`src/domains/providers/knowledge-base-path.ts`). |
 | `CLIO_CODER_NO_NETWORK_TOOLS` | off | `1` strips network tools from every registry in the process; the skills-eval harness sets it for hermetic arms; `--allow-network` clears it (`src/tools/network-policy.ts`). |
+| `CLIO_CODER_SMOOTH_STREAM` | settings value | Per-process override for `terminal.smoothStreaming`: `0`/`off`/`false`, `auto`, or `1`/`on`/`true`. A valid value wins over settings; an invalid value fails safely to `off`. |
+| `CLIO_CODER_REDUCE_MOTION` | off | `1` makes smooth-streaming `auto` use the immediate coalescer. Explicit `on` remains an operator request, while stdout backpressure still pauses frame production. |
+| `CLIO_CODER_SCREEN_READER` | off | `1` makes smooth-streaming `auto` use the immediate coalescer so a screen reader receives the existing low-motion update behavior. |
 
 ## Directory and install layout
 

docs/installation-and-lifecycle.md

@@ -232,7 +232,7 @@ clio-coder upgrade
 Key lifecycle and operational updates in v0.3.2:
 - Upgraded the underlying engine SDK libraries to 0.84.0 with signal-aware OAuth cancellation.
 - Hardened migration resilience: damaged `credentials.yaml` files no longer block upgrades when no renames are needed (#121); `--skip-migrations` is available as a recovery override.
-- Fullscreen TUI mode (`terminal.tuiMode`, `terminal.fullscreenScrollbar`) is available via Settings → Terminal (restart required).
+- Fullscreen TUI mode (`terminal.tuiMode`, `terminal.fullscreenScrollbar`) is available via Settings → Terminal (restart required). Adaptive presentation pacing is the live `terminal.smoothStreaming` setting; 0.3.2 defaults it to `off`, with conservative `auto` and explicit `on` available from the same section.
 - Turn settlement is enforced on `/new`, `/resume`, `/tree`, and `/fork` to cleanly commit in-flight streams before session writer replacement (#114).
 - Resumed and forked session entry replays standardize message prefixes through `src/engine/messages.ts`.
 - `AI_AGENT=clio-coder` is set on all child processes for system attribution.

docs/performance-methodology.md

@@ -56,10 +56,12 @@ node dist/cli/index.js
 The deterministic contracts are
 `tests/contracts/render-pipeline-trace.test.ts`. The real built-CLI acceptance
 harness is `tests/smoke/render-trace-pty.test.ts`; it covers first frame, input
-correlation, resize, grouped writes, paused PTY output, and bounded process
-cleanup. Set `CLIO_CODER_PERF_REPORT=1` while running that test to print its
-observation record. Real PTY acceptance is currently unavailable on Windows;
-the fake-stream frame/backpressure contract remains cross-platform.
+correlation, resize, grouped writes, paused PTY output, adaptive pacing against
+a chunked hermetic provider, a controlled `stdout.write() === false`/`drain`
+boundary, final-frame settlement, and bounded process cleanup. Set
+`CLIO_CODER_PERF_REPORT=1` while running that test to print its observation
+records. Real PTY acceptance is currently unavailable on Windows; the
+fake-stream frame/backpressure contracts remain cross-platform.
 
 ## Import-graph method
 
@@ -367,6 +369,52 @@ justified by the deterministic absence/presence and worker-exclusion contracts,
 the smaller evaluated graph, exact registration-order and surface contracts,
 and the dispatch reservation, approval, gate, detach, monitor, and steer suites.
 
+## Adaptive stream-pacer observations
+
+`terminal.smoothStreaming` is presentation-only. `off` is the exact existing
+16 ms coalescer and remains the 0.3.2 default. `auto` uses the pacer only on a
+capable local TTY with no accessibility, remote/multiplexer, CI, or observed
+backpressure signal. `on` requests pacing, but frame construction still stops
+behind stdout backpressure. The pacer never republishes slices on the public
+event bus: canonical events, persistence, replay/export, tool formation, and
+cumulative tool state remain synchronous while one presentation queue owns
+only derived visible text/thinking mutations.
+
+The deterministic fake-clock contracts cover semantic classification, FIFO
+generation/epoch ordering, abort and stale-admission rejection, grapheme
+clusters, fractional arrival credit, event-loop suspension, catch-up, the
+oldest-visible deadline, absolute queue byte/grapheme bounds, idle shutdown,
+folded-thinking fidelity, reset/discard accounting, mode changes, final-frame
+settlement, fullscreen frozen scrolling through resize, and bounded no-drain
+cleanup. The PTY arm uses a built CLI, a four-delta localhost provider, a
+4 KiB reply, an 80x24 `xterm-256color` PTY whose reader is paused, and a
+test-only writable shim that makes exactly one real child `stdout.write()`
+return `false` before emitting a delayed `drain`. This is deterministic
+backpressure acceptance, not a claim about a particular SSH kernel buffer.
+
+Five independent processes per supported Node line were measured on the same
+2026-08-19 WSL2 host as the corrected baseline, with the operating-system page
+cache warm and V8 compile caching disabled. Values are median / largest of the
+five observations; they are diagnostic observations, not timing gates.
+
+| Node | Input-to-stdout commit | First ingress-to-stdout commit | Final ingress-to-stdout commit | Controlled backpressure wait |
+| --- | ---: | ---: | ---: | ---: |
+| 22.22.3 | 5.530 / 8.896 ms | 62.909 / 70.274 ms | 37.798 / 56.412 ms | 403.809 / 426.570 ms |
+| 24.9.0 | 5.468 / 7.225 ms | 67.878 / 74.021 ms | 27.901 / 35.750 ms | 402.582 / 418.783 ms |
+
+The command was:
+
+```bash
+NODE_DISABLE_COMPILE_CACHE=1 CLIO_CODER_PERF_REPORT=1 \
+  node --import tsx --import ./tests/harness/tmp-root.ts --test \
+  --test-name-pattern 'paces provider deltas' \
+  tests/smoke/render-trace-pty.test.ts
+```
+
+These endpoints end at stdout commit. Even the PTY reader assertion stops at
+the pseudo-terminal boundary; none of these values is literal token-to-glass
+latency.
+
 ## Reporting checklist
 
 Every published observation records:

docs/tui-design.md

@@ -152,6 +152,8 @@ The Clio screen maintains a responsive, four-zone structure: the launchpad / ses
 
 In fullscreen mode, `PageUp` and `PageDown` scroll one viewport, `Home` and `End` jump to its bounds, `Ctrl+Shift+Up` and `Ctrl+Shift+Down` jump between semantic prompts, and the mouse wheel scrolls the transcript. Dragging the scrollbar thumb moves the viewport directly. `terminal.fullscreenScrollbar` is `hidden`, `auto` (visible during interaction), or `always`. Manual scrolling suspends follow-end so new output does not steal the operator's position; returning to the bottom resumes it. Both fullscreen settings are restart-scoped because Clio constructs its terminal renderer and component graph once at startup.
 
+`terminal.smoothStreaming` controls presentation-only pacing of derived assistant text and thinking. `off`, the 0.3.2 release default, is the existing immediate 16 ms coalescer. `auto` paces only on a capable local TTY and bypasses pacing for non-TTY, SSH, multiplexers, CI, screen-reader/reduced-motion markers, or observed stdout backpressure. `on` explicitly requests grapheme-safe pacing, while still stopping frame production behind stdout backpressure. Raw provider wrappers never enter the panel, canonical events and persistence remain synchronous, and tool/message/turn/abort/retry/submit/teardown boundaries drain visible state before they continue. `CLIO_CODER_SMOOTH_STREAM` is the one-process escape hatch and takes precedence over settings; invalid values resolve to `off`.
+
 ### 5.1 Welcome Launchpad & Session Header
 
 - **Pre-Submit Launchpad**: Before the first prompt, renders a compact launchpad at line 0 with bold CAPS section tags (`WORKSPACE`, `ROUTE`, `NEXT`), honest readiness state, and a context-sensitive next action (e.g. `ctx missing · /context init`, `ctx checking…`, or `ctx ready · type a task`). Asynchronous repository probes use height-stable dim placeholders (`factsPending`).

src/core/config.ts

@@ -1315,6 +1315,7 @@ export function validateSettings(raw: unknown): SettingsValidationResult {
 				"outputVerbosity",
 				"tuiMode",
 				"fullscreenScrollbar",
+				"smoothStreaming",
 			]);
 			if ("showTerminalProgress" in raw.terminal) {
 				const v = expectBoolean(issues, "terminal.showTerminalProgress", raw.terminal.showTerminalProgress);
@@ -1335,6 +1336,11 @@ export function validateSettings(raw: unknown): SettingsValidationResult {
 				if (v === "hidden" || v === "auto" || v === "always") settings.terminal.fullscreenScrollbar = v;
 				else if (v !== undefined) issues.add("terminal.fullscreenScrollbar", "expected hidden, auto, or always");
 			}
+			if ("smoothStreaming" in raw.terminal) {
+				const v = expectString(issues, "terminal.smoothStreaming", raw.terminal.smoothStreaming);
+				if (v === "off" || v === "auto" || v === "on") settings.terminal.smoothStreaming = v;
+				else if (v !== undefined) issues.add("terminal.smoothStreaming", "expected off, auto, or on");
+			}
 		}
 	}