Skip to content

Explain why a provider model catalog failed to load - #4081

Open
ymichael wants to merge 2 commits into
mainfrom
bb/investigate-when-the-error-appears-thr_4pah43dksr
Open

ymichael wants to merge 2 commits into
mainfrom
bb/investigate-when-the-error-appears-thr_4pah43dksr

Conversation

@ymichael

@ymichael ymichael commented Sep 22, 2026

Copy link
Copy Markdown
Collaborator

Human comments

What was wrong

The model picker had one sentence for every way a catalog can fail. A machine that was asleep and a Codex CLI that was never installed both rendered exactly Could not load models for Codex. — byte-identical panels for problems with opposite fixes. Two causes: the server threw away the host daemon's errorMessage when recording a catalog failure, and a missing CLI never reached the missing_executable code that already exists, because the codex and claude-code bridges replace the spawn ENOENT with prose guidance before rejecting, which defeats the daemon's /ENOENT/ + /spawn/ message classification (apps/host-daemon/src/command-dispatch-support.ts). Everything landed in the catch-all failed. #2646 is this in the wild: reporters had the Codex desktop app and no CLI, and a maintainer had to tell them so by hand.

Where the panel appears also differs by provider — it only renders with an empty model list, so Claude Code's declared fallback catalog usually hides it while Codex, which declares none, always lands there.

What changed

Contract. systemExecutionOptionsModelLoadErrorSchema gains detail: string | null (packages/server-contract/src/api/system.ts). No daemon wire change: errorMessage already exists on the host RPC response, so HOST_DAEMON_PROTOCOL_VERSION is unchanged.

Server. toProviderModelCatalogFailureDetail fills it from ApiError.body.message only — whitespace-collapsed, capped at 300 chars — and returns null for anything else, matching errorToResponse's refusal to leak non-ApiError messages. The catalog store records the detail beside each failure code, includes it in pickerView so a changed message still pushes, and retains the message from a host that dropped mid-refresh (that path records no sticky failure and previously served a bare failed).

Copy. The two lines now have separate jobs. Line one always states what failed; line two states why, in a short phrase: CLI not found, Not signed in, Timed out, Provider plugin failed to load, or the host's own words for failed. The install URL moves onto the reason line. Full host text stays on the tooltip, in bb provider models, and in the 503 details.

Bridge contract (new). BRIDGE_JSON_RPC_ERRORS.MISSING_EXECUTABLE (-32004) and experimental_BridgeMissingExecutableError, which runBridgeRequest maps to that code; the daemon reports it as missing_executable without reading the message. Emitted from the codex bridge's model/list and the claude-code catalog probe — scoped to catalog paths, since a turn failure wants the fuller sentence and a different code there would shift unrelated failure copy. The ACP bridge already rejects with the raw ENOENT and is unchanged. Documented in docs/provider-bridge-protocol.md; audit entry in docs/api_to_audit.md (open question: whether this eventually folds into BridgeRecoveryError as a notInstalled kind — that audit deliberately deferred an installation kind, and this answers classification rather than runtime action).

CLI. bb provider models ignored modelLoadError entirely and printed No models available. It now reports provider, code, and detail on stderr; stdout and --json are unchanged. Guide and skill references updated.

How you verified

pnpm exec turbo run typecheck — 101/101. Tests, serially, all green: @bb/app (5019), @bb/server (3087), @bb/host-daemon (742), @bb/cli (842), @get-bb/plugin-sdk (354), @bb/provider-bridge-protocol (284), bb-plugin-provider-codex (325), bb-plugin-provider-claude-code (365), @bb/templates (47). The workflows plugin is untouched and still passes (248): its local modelLoadError type accepts the added field unchanged.

Tests that fail before this change:

  • apps/host-daemon/src/command-dispatch-support.test.ts — -32004 classifies as missing_executable, and the same message under BRIDGE_ERROR still does not.
  • plugins/provider-codex/src/bridge/bridge.model-list.test.tsmodel/list rejects with MISSING_EXECUTABLE when the app-server binary is absent.
  • plugins/provider-claude-code/src/bridge/missing-cli-error.test.ts — the catalog probe error is typed; the session-start path is deliberately left untyped.
  • apps/app/src/components/pickers/ModelReasoningPicker.test.tsxCLI not found renders as the install link with the prose suppressed; a generic failed shows the host detail; tooltip carries both lines.
  • apps/server/test/providers/provider-model-catalog-store.test.ts — detail survives a sticky failure, a host that drops mid-refresh reports Host is not connected, and a long message is collapsed and truncated to 300 chars.

Fourteen existing server assertions were updated, each now receiving a real diagnostic (Host is suspended, Cursor agent is not authenticated., Local host daemon is not initialized), which is the cross-provider evidence that this is not Codex-specific.

Visual states are a committed Ladle harness, apps/app/src/components/pickers/ModelLoadErrorStates.stories.tsx: pnpm exec turbo run storybook --filter=@bb/app, then pickers → Model Load Error States. End-to-end manual check for the missing CLI: BB_CODEX_BRIDGE_APP_SERVER_COMMAND=/nonexistent pnpm start:worktree, open the model picker on Codex.

Two pre-existing conditions, unrelated and not touched: packages/server-contract/test/payload-size.test.ts asserts exact gzip byte counts and fails locally by 1–4 bytes with identical jsonBytes (confirmed failing on 5eb11ff with this change reverted — a local zlib difference), and apps/app/src/components/plugin/management/BrowsePluginsTab.test.tsx flakes on toast timing under load.

Refs #2646 — this does not make Codex models load; it makes the reason visible and actionable in-product, which is what that thread needed a maintainer to supply by hand. Leaving the issue open for whoever wants the picker to offer installation directly.

🤖 Generated with Claude Code

AGENT GENERATED

@ymichael
ymichael force-pushed the bb/investigate-when-the-error-appears-thr_4pah43dksr branch 2 times, most recently from 1c87313 to c14e02d Compare September 22, 2026 18:54
ymichael and others added 2 commits September 23, 2026 11:42
The model picker could only say "Could not load models for Codex." A host
that was asleep and a Codex CLI that was never installed produced the
identical sentence, because the server dropped the host daemon's
errorMessage and both conditions collapsed into the generic `failed` code.
The missing CLI could not reach `missing_executable` at all: the codex and
claude-code bridges replace the spawn ENOENT with prose guidance, which
defeats the daemon's message-pattern classification.

Carry the underlying failure to the picker as a new nullable `detail` on
SystemExecutionOptionsModelLoadError, filled at the server boundary from the
ApiError message only (unknown errors stay null, matching errorToResponse).
The catalog store records it beside each failure code and keeps the message
from a host that dropped mid-refresh, which previously served a bare
`failed`.

Give the two lines separate jobs so neither repeats the other: the summary
always says what failed, and the reason line says why in a short phrase --
"CLI not found", "Not signed in", "Timed out", "Provider plugin failed to
load", or the host's own words for `failed`. The full host message stays on
the tooltip, in `bb provider models`, and in the 503 details.

Classify a missing provider CLI structurally instead of sniffing text: add
BRIDGE_JSON_RPC_ERRORS.MISSING_EXECUTABLE and
experimental_BridgeMissingExecutableError, emit them from the codex and
claude-code catalog paths, and map the code in the daemon. The ACP bridge
already rejects with the raw ENOENT and is unchanged.

`bb provider models` previously ignored modelLoadError entirely and printed
"No models available"; it now reports the provider, code, and detail on
stderr while leaving stdout and --json unchanged.

Refs #2646

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
experimental_BridgeMissingExecutableError is a new published surface, so a
plugin that throws it needs a version to point engines.bbPluginSdk at.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@ymichael
ymichael force-pushed the bb/investigate-when-the-error-appears-thr_4pah43dksr branch from c14e02d to 4b90c1c Compare September 23, 2026 18:47

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant