From 1ebe2356a53a18a4f778c8ff95fc2498dea467e1 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C3=8Dtalo=20Menezes?= Date: Thu, 9 Jul 2026 21:45:34 -0300 Subject: [PATCH 01/12] feat: replace anthropic-only cli with multi-provider resolution and setup wizard --- cli/README.md | 126 ++++++++++++-- cli/src/acp/session-store.ts | 2 +- cli/src/agent/defaults.ts | 49 ++++++ cli/src/agent/harness.ts | 10 +- cli/src/agent/model.test.ts | 152 +++++++++++++++-- cli/src/agent/model.ts | 120 +++++++++----- cli/src/agent/openai-compat-model.ts | 4 +- cli/src/agent/run.test.ts | 40 ++--- cli/src/agent/run.ts | 10 +- cli/src/agent/types.ts | 42 +++-- cli/src/cli.test.ts | 220 ++++++++++++++++++------ cli/src/cli.ts | 155 +++++++++++------ cli/src/config/config.test.ts | 71 ++++++++ cli/src/config/config.ts | 69 ++++++++ cli/src/config/wizard.test.ts | 161 ++++++++++++++++++ cli/src/config/wizard.ts | 239 +++++++++++++++++++++++++++ cli/src/index.ts | 114 +++++++------ cli/src/iroh/paths.ts | 6 +- cli/src/paths.ts | 19 +++ 19 files changed, 1348 insertions(+), 261 deletions(-) create mode 100644 cli/src/agent/defaults.ts create mode 100644 cli/src/config/config.test.ts create mode 100644 cli/src/config/config.ts create mode 100644 cli/src/config/wizard.test.ts create mode 100644 cli/src/config/wizard.ts create mode 100644 cli/src/paths.ts diff --git a/cli/README.md b/cli/README.md index 2f5f171d6..18303ee3f 100644 --- a/cli/README.md +++ b/cli/README.md @@ -3,8 +3,9 @@ A single-binary terminal coding agent. It operates directly in your working directory with four tools — **bash**, **read**, **write**, **edit** — built on the [Pi harness](https://www.npmjs.com/package/@earendil-works/pi-agent-core) -and talking to Claude. Give it a task as one prompt or drop into an interactive -REPL; there's no daemon, no config file, and nothing to install but the binary. +and talking to models from Anthropic, OpenAI, Google, xAI, and other providers. +Give it a task as one prompt or drop into an interactive REPL; there's no daemon, +and nothing to install but the binary. ## Install @@ -69,6 +70,39 @@ mv "thunderbolt-cli-$TARGET" ~/.local/bin/thunderbolt > Windows also has no prebuilt CLI binary. Build from source above on either > unsupported platform. +## First run + +Run `thunderbolt` in a terminal. When no usable API key exists, guided setup +asks for provider, API key, and model, saves defaults, then continues directly +into requested REPL or one-shot task. API key input is not echoed. + +Run setup again anytime: + +```sh +thunderbolt config +``` + +Config lives at `~/.thunderbolt/config.json`, or +`$THUNDERBOLT_HOME/config.json` when `THUNDERBOLT_HOME` is set. File mode is +`0600` because config may contain a plaintext API key. + +```json +{ + "provider": "openai-compat", + "model": "upstream-model", + "apiKey": "sk-...", + "baseUrl": "https://host.example/v1" +} +``` + +`apiKey` and `baseUrl` are optional. Saved keys and base URLs apply only when +saved provider matches effective provider, preventing cross-provider credential +forwarding. Missing, malformed, or invalid config is treated as absent. + +Resolution order is explicit flag, supported provider environment variable, +config file, then built-in default. Current environment tier contains credential +variables only; provider, model, and base URL have no environment override. + ## Usage Run a single task and exit: @@ -88,6 +122,7 @@ thunderbolt | Command | Purpose | | ------- | ------- | | `thunderbolt agent [options] [prompt]` | Run coding agent; `agent` is optional/default. | +| `thunderbolt config` | Run guided provider setup and overwrite saved defaults. | | `thunderbolt acp serve [options]` | Expose built-in coding agent as stdio ACP server. | | `thunderbolt acp --transport [--port N] -- ` | Bridge stdio ACP agent. | | `thunderbolt mcp --transport [--port N] -- ` | Bridge stdio MCP server. | @@ -121,10 +156,10 @@ elsewhere on the machine are outside its workspace and unavailable. | Flag | Description | | ---- | ----------- | -| `-m`, `--model ` | Provider model id (default: `claude-opus-4-8`). | -| `--provider ` | `anthropic` or `openai-compat` (default: `anthropic`). | -| `--base-url ` | Required OpenAI-compatible endpoint URL. | -| `--api-key ` | OpenAI-compatible bearer key; overrides `THUNDERBOLT_OPENAI_COMPAT_KEY`. | +| `-m`, `--model ` | Provider model id (provider-specific default). | +| `--provider ` | Built-in provider or `openai-compat` (default: `anthropic`). | +| `--base-url ` | Custom endpoint URL (required for `openai-compat`). | +| `--api-key ` | Explicit key for any provider; overrides provider environment. | | `--thinking ` | `off`, `minimal`, `low`, `medium`, `high`, or `xhigh` (default: `medium`). | | `-y`, `--yolo` | Auto-approve tool calls (alias: `--dangerously-skip-permissions`). | | `--no-tui` | Force plain readline REPL. | @@ -135,20 +170,73 @@ ACP/MCP bridge commands accept `--transport wss|iroh` (default `wss`) and `--port <0-65535>` for WSS (defaults: ACP `8839`, MCP `8840`). Arguments after `--` form spawned stdio command. -Anthropic provider requires **`ANTHROPIC_API_KEY`**. OpenAI-compatible provider -requires `--base-url` plus `--api-key` or `THUNDERBOLT_OPENAI_COMPAT_KEY`. +Supported built-in providers: + +`anthropic`, `openai`, `google`, `xai`, `deepseek`, `zai`, `mistral`, `groq`, +`openrouter`, `moonshotai`, `minimax`, `cerebras`, `together`, `fireworks`. + +Each built-in provider uses Pi's generated model catalog and standard API-key +environment variable. `--api-key` overrides that environment key; matching +saved config supplies fallback credentials. Unknown model errors list valid +catalog ids for selected provider. + +`openai-compat` remains custom-endpoint escape hatch: + +```sh +THUNDERBOLT_OPENAI_COMPAT_KEY=sk-... thunderbolt \ + --provider openai-compat \ + --base-url https://host.example/v1 \ + --model upstream-model \ + "review this repository" +``` + +For security, `openai-compat` never reads `OPENAI_API_KEY` or another generic +provider key. Use `--api-key` or `THUNDERBOLT_OPENAI_COMPAT_KEY` explicitly so a +credential cannot be forwarded automatically to an arbitrary custom URL. + +### Provider defaults + +| Provider | Default model | +| -------- | ------------- | +| `anthropic` | `claude-opus-4-8` | +| `openai` | `gpt-5.3-codex` | +| `google` | `gemini-3.1-pro-preview` | +| `xai` | `grok-build-0.1` | +| `deepseek` | `deepseek-v4-pro` | +| `zai` | `glm-5.2` | +| `mistral` | `devstral-medium-latest` | +| `groq` | `openai/gpt-oss-120b` | +| `openrouter` | `anthropic/claude-opus-4.8` | +| `moonshotai` | `kimi-k2.7-code` | +| `minimax` | `MiniMax-M3` | +| `cerebras` | `gpt-oss-120b` | +| `together` | `moonshotai/Kimi-K2.7-Code` | +| `fireworks` | `accounts/fireworks/models/kimi-k2p7-code` | ### Environment -| Variable | Description | -| -------- | ----------- | -| `ANTHROPIC_API_KEY` | Anthropic API key for default provider. | -| `THUNDERBOLT_OPENAI_COMPAT_KEY` | Bearer key for OpenAI-compatible provider. | -| `THUNDERBOLT_HOME` | CLI state root (default `~/.thunderbolt`): iroh identity/allowlist and ACP sessions. | -| `THUNDERBOLT_IROH_RELAY_URL` | Self-hosted iroh-relay WSS URL; unset uses n0 public relays. | -| `THUNDERBOLT_APP_ORIGIN` | Extra comma-separated allowed browser origins for WSS bridges. | -| `THUNDERBOLT_NO_TUI` | Force plain readline REPL when set. | -| `NO_COLOR` | Disable terminal color when set. | +| Variable | Description | +| ----------------------------- | --------------------------------------------------------------------------- | +| `ANTHROPIC_OAUTH_TOKEN`, `ANTHROPIC_API_KEY` | Anthropic credentials, checked in that order. | +| `OPENAI_API_KEY` | OpenAI API key. | +| `GEMINI_API_KEY` | Google Gemini API key. | +| `XAI_API_KEY` | xAI API key. | +| `DEEPSEEK_API_KEY` | DeepSeek API key. | +| `ZAI_API_KEY` | Z.AI API key. | +| `MISTRAL_API_KEY` | Mistral API key. | +| `GROQ_API_KEY` | Groq API key. | +| `OPENROUTER_API_KEY` | OpenRouter API key. | +| `MOONSHOT_API_KEY` | Moonshot AI API key. | +| `MINIMAX_API_KEY` | MiniMax API key. | +| `CEREBRAS_API_KEY` | Cerebras API key. | +| `TOGETHER_API_KEY` | Together API key. | +| `FIREWORKS_API_KEY` | Fireworks API key. | +| `THUNDERBOLT_OPENAI_COMPAT_KEY` | Dedicated fallback key for arbitrary `openai-compat` URLs. | +| `THUNDERBOLT_HOME` | CLI state root containing `config.json`, iroh identity/allowlist, and ACP sessions (default: `~/.thunderbolt`). | +| `THUNDERBOLT_IROH_RELAY_URL` | Self-hosted iroh-relay WSS URL; unset uses n0 public relays. | +| `THUNDERBOLT_APP_ORIGIN` | Extra comma-separated allowed browser origins for WSS bridges. | +| `THUNDERBOLT_NO_TUI` | Force plain readline REPL when set. | +| `NO_COLOR` | Disable terminal color when set. | ## Demo @@ -158,4 +246,8 @@ export ANTHROPIC_API_KEY=sk-ant-... thunderbolt "summarize what this repo does in three bullets" thunderbolt --thinking high "find and fix the off-by-one bug in src/range.ts" thunderbolt --yolo "run the test suite and fix whatever breaks" + +# Or select another built-in provider; omitted --model uses provider default. +OPENAI_API_KEY=sk-... thunderbolt --provider openai "fix the failing tests" +thunderbolt --provider google --api-key AIza... "review this repository" ``` diff --git a/cli/src/acp/session-store.ts b/cli/src/acp/session-store.ts index 3438a7213..1cb726c4b 100644 --- a/cli/src/acp/session-store.ts +++ b/cli/src/acp/session-store.ts @@ -25,7 +25,7 @@ import { join } from 'node:path' import { JsonlSessionRepo } from '@earendil-works/pi-agent-core' import type { Session } from '@earendil-works/pi-agent-core' import { NodeExecutionEnv } from '@earendil-works/pi-agent-core/node' -import { thunderboltHomeDir } from '../iroh/paths.ts' +import { thunderboltHomeDir } from '../paths.ts' /** Creates and resumes disk-backed Pi sessions for ACP session ids. */ export type SessionStore = { diff --git a/cli/src/agent/defaults.ts b/cli/src/agent/defaults.ts new file mode 100644 index 000000000..6acbace55 --- /dev/null +++ b/cli/src/agent/defaults.ts @@ -0,0 +1,49 @@ +/* This Source Code Form is subject to the terms of the Mozilla Public + * License, v. 2.0. If a copy of the MPL was not distributed with this + * file, You can obtain one at http://mozilla.org/MPL/2.0/. */ + +/** Provider defaults and credential environment metadata shared by CLI setup. */ + +import type { BuiltinProvider } from './types.ts' + +/** Default model backend when provider is omitted. */ +export const DEFAULT_PROVIDER: BuiltinProvider = 'anthropic' + +/** Default Anthropic model. */ +export const DEFAULT_MODEL = 'claude-opus-4-8' + +/** Default catalog model for each built-in provider. */ +export const DEFAULT_MODELS: Readonly> = { + anthropic: DEFAULT_MODEL, + openai: 'gpt-5.3-codex', + google: 'gemini-3.1-pro-preview', + xai: 'grok-build-0.1', + deepseek: 'deepseek-v4-pro', + zai: 'glm-5.2', + mistral: 'devstral-medium-latest', + groq: 'openai/gpt-oss-120b', + openrouter: 'anthropic/claude-opus-4.8', + moonshotai: 'kimi-k2.7-code', + minimax: 'MiniMax-M3', + cerebras: 'gpt-oss-120b', + together: 'moonshotai/Kimi-K2.7-Code', + fireworks: 'accounts/fireworks/models/kimi-k2p7-code', +} + +/** Environment variables Pi checks for each exposed built-in provider. */ +export const BUILTIN_PROVIDER_ENV_VARS: Readonly> = { + anthropic: ['ANTHROPIC_OAUTH_TOKEN', 'ANTHROPIC_API_KEY'], + openai: ['OPENAI_API_KEY'], + google: ['GEMINI_API_KEY'], + xai: ['XAI_API_KEY'], + deepseek: ['DEEPSEEK_API_KEY'], + zai: ['ZAI_API_KEY'], + mistral: ['MISTRAL_API_KEY'], + groq: ['GROQ_API_KEY'], + openrouter: ['OPENROUTER_API_KEY'], + moonshotai: ['MOONSHOT_API_KEY'], + minimax: ['MINIMAX_API_KEY'], + cerebras: ['CEREBRAS_API_KEY'], + together: ['TOGETHER_API_KEY'], + fireworks: ['FIREWORKS_API_KEY'], +} diff --git a/cli/src/agent/harness.ts b/cli/src/agent/harness.ts index 8695e531d..d08704d7b 100644 --- a/cli/src/agent/harness.ts +++ b/cli/src/agent/harness.ts @@ -3,11 +3,11 @@ * file, You can obtain one at http://mozilla.org/MPL/2.0/. */ /** - * Assembles the Pi `AgentHarness` — the spine that lets the CLI actually talk - * to Claude. It binds a Node execution environment (real bash + filesystem) to - * the working directory, opens an in-memory session, resolves the model, and - * registers coding tools. Workspace-root harnesses omit bash because arbitrary - * shell commands cannot be confined to that workspace. + * Assembles the Pi `AgentHarness` — the spine that lets the CLI talk to the + * selected model provider. It binds a Node execution environment (real bash + + * filesystem) to the working directory, opens an in-memory session, resolves + * the model, and registers coding tools. Workspace-root harnesses omit bash + * because arbitrary shell commands cannot be confined to that workspace. */ import { AgentHarness, InMemorySessionRepo } from '@earendil-works/pi-agent-core' diff --git a/cli/src/agent/model.test.ts b/cli/src/agent/model.test.ts index e93360394..4d2e44e64 100644 --- a/cli/src/agent/model.test.ts +++ b/cli/src/agent/model.test.ts @@ -3,21 +3,66 @@ * file, You can obtain one at http://mozilla.org/MPL/2.0/. */ /** - * Branch coverage for `resolveModel`: the anthropic-vs-openai-compat routing, - * the required-input guards for openai-compat (base URL + api key), and the - * unknown-Anthropic-id failure. These run with no network — anthropic resolves - * against Pi's wired built-in catalog and openai-compat synthesizes a local - * descriptor. + * Branch coverage for `resolveModel`: built-in provider catalog lookup and + * credentials, explicit key forwarding, plus openai-compat input guards. */ +import { + type Context, + type ProviderStreams, + createAssistantMessageEventStream, + createModels, + createProvider, + envApiKeyAuth, +} from '@earendil-works/pi-ai' import { builtinModels } from '@earendil-works/pi-ai/providers/all' import { describe, expect, test } from 'bun:test' import { resolveModel } from './model.ts' +import { BUILTIN_PROVIDERS } from './types.ts' /** Pull a real catalog id from Pi's wired provider rather than hard-coding one, * so this stays green across catalog churn. */ const KNOWN_ANTHROPIC = builtinModels().getModels('anthropic')[0]!.id +const EMPTY_ENV: Readonly> = {} + +/** Builds a one-model OpenAI catalog whose stream options are observable. */ +const capturingBuiltinModels = () => { + const model = builtinModels().getModels('openai')[0]! + const calls: { readonly fn: 'stream' | 'streamSimple'; readonly options: Record }[] = [] + /** Creates an already-ended stream so Pi's lazy delegate can drain it. */ + const inertStream = () => { + const stream = createAssistantMessageEventStream() + stream.end() + return stream + } + const streams = { + stream: ((_model, _context, options) => { + calls.push({ fn: 'stream', options: { ...options } }) + return inertStream() + }) as ProviderStreams['stream'], + streamSimple: ((_model, _context, options) => { + calls.push({ fn: 'streamSimple', options: { ...options } }) + return inertStream() + }) as ProviderStreams['streamSimple'], + } + const models = createModels({ + authContext: { + env: async (name) => (name === 'OPENAI_API_KEY' ? 'env-key' : undefined), + fileExists: async () => false, + }, + }) + models.setProvider( + createProvider({ + id: 'openai', + auth: { apiKey: envApiKeyAuth('OpenAI API key', ['OPENAI_API_KEY']) }, + models: [model], + api: streams, + }), + ) + return { models, model, calls } +} + describe('resolveModel — openai-compat branch', () => { test('throws when --base-url is missing', () => { expect(() => resolveModel({ model: 'mimo', provider: 'openai-compat', apiKey: 'k' })).toThrow(/--base-url/) @@ -29,6 +74,18 @@ describe('resolveModel — openai-compat branch', () => { ) }) + test('missing custom key error points non-TTY users to guided setup', () => { + expect(() => resolveModel({ model: 'mimo', provider: 'openai-compat', baseUrl: 'https://h/v1' })).toThrow( + /THUNDERBOLT_OPENAI_COMPAT_KEY.*--api-key.*run `thunderbolt` in a terminal for guided setup/, + ) + }) + + test('missing custom key stays actionable when the base URL is also missing', () => { + expect(() => resolveModel({ model: 'mimo', provider: 'openai-compat' })).toThrow( + /THUNDERBOLT_OPENAI_COMPAT_KEY.*--api-key.*run `thunderbolt` in a terminal for guided setup/, + ) + }) + test('rejects an empty-string base URL (falsy guard, not just undefined)', () => { expect(() => resolveModel({ model: 'mimo', provider: 'openai-compat', baseUrl: '', apiKey: 'k' })).toThrow( /--base-url/, @@ -66,22 +123,89 @@ describe('resolveModel — openai-compat branch', () => { }) }) -describe('resolveModel — anthropic branch (default)', () => { +describe('resolveModel — built-in providers', () => { test('defaults to anthropic when no provider is given and resolves a known id', () => { - const { model } = resolveModel({ model: KNOWN_ANTHROPIC }) + const { model } = resolveModel({ model: KNOWN_ANTHROPIC, apiKey: 'explicit-key' }) expect(model.id).toBe(KNOWN_ANTHROPIC) expect(model.provider).toBe('anthropic') }) - test('throws on an unknown Anthropic id', () => { - expect(() => resolveModel({ model: 'claude-does-not-exist', provider: 'anthropic' })).toThrow( - /Unknown Anthropic model/, + test('resolves catalog models for every curated provider', () => { + for (const provider of BUILTIN_PROVIDERS) { + const modelId = builtinModels().getModels(provider)[0]!.id + expect(resolveModel({ model: modelId, provider, apiKey: 'explicit-key' }).model.provider).toBe(provider) + } + }) + + test('unknown-model error includes valid ids read from that provider catalog', () => { + const validIds = builtinModels() + .getModels('google') + .slice(0, 3) + .map((model) => model.id) + expect(() => resolveModel({ model: 'gemini-does-not-exist', provider: 'google', apiKey: 'key' })).toThrow( + new RegExp(validIds.join('|')), ) }) - test('ignores base URL / api key on the anthropic branch (never requires --base-url)', () => { - // anthropic resolution must not trip the openai-compat base-url guard. - const { model } = resolveModel({ model: KNOWN_ANTHROPIC, provider: 'anthropic', baseUrl: 'https://ignored' }) - expect(model.provider).toBe('anthropic') + test('missing-key error names provider env variable and --api-key', () => { + expect(() => + resolveModel( + { model: builtinModels().getModels('google')[0]!.id, provider: 'google' }, + { builtinModels, env: EMPTY_ENV }, + ), + ).toThrow(/GEMINI_API_KEY.*--api-key|--api-key.*GEMINI_API_KEY/) + }) + + test('missing built-in key error points non-TTY users to guided setup', () => { + expect(() => + resolveModel( + { model: builtinModels().getModels('google')[0]!.id, provider: 'google' }, + { builtinModels, env: EMPTY_ENV }, + ), + ).toThrow(/GEMINI_API_KEY.*--api-key.*run `thunderbolt` in a terminal for guided setup/) + }) + + test('missing built-in credentials stay actionable when the model id is also invalid', () => { + expect(() => + resolveModel( + { model: 'not-a-google-model', provider: 'google' }, + { builtinModels, env: EMPTY_ENV }, + ), + ).toThrow(/GEMINI_API_KEY.*--api-key.*run `thunderbolt` in a terminal for guided setup/) + }) + + test('explicit key overrides provider env auth in both Models stream paths', async () => { + const capture = capturingBuiltinModels() + const { models, model } = resolveModel( + { model: capture.model.id, provider: 'openai', apiKey: 'flag-key' }, + { builtinModels: () => capture.models, env: { OPENAI_API_KEY: 'env-key' } }, + ) + + for await (const _event of models.streamSimple(model, {} as Context)) { + // Inert test stream emits no events. + } + for await (const _event of models.stream(model, {} as Context)) { + // Inert test stream emits no events. + } + + expect(capture.calls).toEqual([ + { fn: 'streamSimple', options: expect.objectContaining({ apiKey: 'flag-key' }) }, + { fn: 'stream', options: expect.objectContaining({ apiKey: 'flag-key' }) }, + ]) + }) + + test('explicit key does not become model descriptor data', () => { + const { model } = resolveModel({ model: KNOWN_ANTHROPIC, provider: 'anthropic', apiKey: 'super-secret' }) + expect(JSON.stringify(model)).not.toContain('super-secret') + }) + + test('without an explicit key, Pi resolves the provider environment variable', async () => { + const capture = capturingBuiltinModels() + const { models, model } = resolveModel( + { model: capture.model.id, provider: 'openai' }, + { builtinModels: () => capture.models, env: { OPENAI_API_KEY: 'env-key' } }, + ) + + expect((await models.getAuth(model))?.auth.apiKey).toBe('env-key') }) }) diff --git a/cli/src/agent/model.ts b/cli/src/agent/model.ts index 6189d63fc..c08ee6fd0 100644 --- a/cli/src/agent/model.ts +++ b/cli/src/agent/model.ts @@ -2,69 +2,115 @@ * License, v. 2.0. If a copy of the MPL was not distributed with this * file, You can obtain one at http://mozilla.org/MPL/2.0/. */ -/** - * Resolves the model the harness runs, branching on the requested provider: - * - * - `anthropic` (default): looks the id up in Pi's built-in catalog. Pi's - * `@earendil-works/pi-ai/providers/all` is the only entry point that wires - * the providers (bare `createModels()` returns an empty collection); the - * wired anthropic provider resolves `ANTHROPIC_API_KEY` from the environment. - * - `openai-compat`: synthesizes an OpenAI-compatible model bound to a custom - * base URL + bearer key (see {@link buildOpenAiCompatModel}), so the CLI can - * run a non-Anthropic endpoint like Xiaomi MiMo. - */ +/** Resolves built-in Pi catalog models and custom OpenAI-compatible models. */ -import type { Api, Model, Models } from '@earendil-works/pi-ai' +import type { + Api, + Model, + Models, + MutableModels, + Provider, + ProviderStreams, +} from '@earendil-works/pi-ai' import { builtinModels } from '@earendil-works/pi-ai/providers/all' import { buildOpenAiCompatModel } from './openai-compat-model.ts' -import type { ModelProvider } from './types.ts' +import { BUILTIN_PROVIDER_ENV_VARS, DEFAULT_PROVIDER } from './defaults.ts' +import type { BuiltinProvider, ModelProvider } from './types.ts' -/** Default provider when none is specified. */ -const ANTHROPIC: ModelProvider = 'anthropic' +export { BUILTIN_PROVIDER_ENV_VARS } from './defaults.ts' -/** Inputs for {@link resolveModel}: the model id plus the provider routing. */ +/** Inputs for {@link resolveModel}: model id plus provider routing. */ export type ResolveModelOptions = { - /** Model id to run (Anthropic catalog id, or upstream openai-compat id). */ + /** Catalog id for built-ins, or upstream id for openai-compat. */ readonly model: string /** Backend to resolve against (defaults to `anthropic`). */ readonly provider?: ModelProvider /** OpenAI-compatible base URL — required for `openai-compat`. */ readonly baseUrl?: string - /** Bearer api key — required for `openai-compat`. */ + /** Explicit api key for any provider. */ readonly apiKey?: string } -/** Resolves a single Anthropic model against Pi's wired built-in catalog. */ -const resolveAnthropic = (requestedId: string): { models: Models; model: Model } => { - const models = builtinModels() - const model = models.getModel(ANTHROPIC, requestedId) +/** Injectable runtime inputs for deterministic model-resolution tests. */ +export type ResolveModelDependencies = { + readonly builtinModels: () => MutableModels + readonly env: Readonly> +} + +const DEFAULT_DEPENDENCIES: ResolveModelDependencies = { builtinModels, env: process.env } + +/** Replaces selected provider streams with key-injecting delegates. */ +const applyApiKeyOverride = (models: MutableModels, providerId: BuiltinProvider, apiKey: string): void => { + const provider = models.getProvider(providerId) + if (!provider) throw new Error(`Pi catalog does not contain provider "${providerId}".`) + + const baseStream: ProviderStreams['stream'] = provider.stream + const stream: Provider['stream'] = (model, context, options) => + baseStream(model, context, { ...options, apiKey }) + + models.setProvider({ + ...provider, + stream, + streamSimple: (model, context, options) => provider.streamSimple(model, context, { ...options, apiKey }), + }) +} + +/** Resolves one built-in provider model and validates available credentials. */ +const resolveBuiltin = ( + provider: BuiltinProvider, + requestedId: string, + apiKey: string | undefined, + dependencies: ResolveModelDependencies, +): { models: Models; model: Model } => { + const envVars = BUILTIN_PROVIDER_ENV_VARS[provider] + const hasEnvKey = envVars.some((name) => Boolean(dependencies.env[name])) + if (!apiKey && !hasEnvKey) { + throw new Error( + `No API key configured for provider "${provider}". Set ${envVars.join(' or ')}, pass --api-key, or run ` + + '`thunderbolt` in a terminal for guided setup.', + ) + } + + const models = dependencies.builtinModels() + const model = models.getModel(provider, requestedId) if (!model) { - throw new Error(`Unknown Anthropic model "${requestedId}".`) + const validIds = models + .getModels(provider) + .slice(0, 5) + .map((candidate) => candidate.id) + throw new Error( + `Unknown model "${requestedId}" for provider "${provider}". Valid model ids include: ${validIds.join(', ')}.`, + ) } + + if (apiKey) applyApiKeyOverride(models, provider, apiKey) return { models, model } } -/** Resolves an OpenAI-compatible model, requiring the base URL + key the - * endpoint needs. */ +/** Resolves an OpenAI-compatible model, requiring custom endpoint inputs. */ const resolveOpenAiCompat = (opts: ResolveModelOptions): { models: Models; model: Model } => { - if (!opts.baseUrl) { - throw new Error('the openai-compat provider requires --base-url') - } if (!opts.apiKey) { throw new Error( - 'the openai-compat provider requires an api key (pass --api-key or set THUNDERBOLT_OPENAI_COMPAT_KEY)', + 'The openai-compat provider requires an api key. Set THUNDERBOLT_OPENAI_COMPAT_KEY, pass --api-key, ' + + 'or run `thunderbolt` in a terminal for guided setup.', ) } + if (!opts.baseUrl) { + throw new Error('the openai-compat provider requires --base-url') + } return buildOpenAiCompatModel({ modelId: opts.model, baseUrl: opts.baseUrl, apiKey: opts.apiKey }) } /** - * Builds the wired provider collection and resolves a single model, ready for - * the harness. - * - * @param opts - the model id and provider routing (base URL / api key for openai-compat) - * @returns the provider collection and the resolved model - * @throws if an Anthropic id is unknown, or required openai-compat inputs are missing + * Builds provider collection and resolves requested model for harness use. + * Explicit keys are injected at provider dispatch, after Pi's env resolution, + * so they win without changing ambient credential behavior. */ -export const resolveModel = (opts: ResolveModelOptions): { models: Models; model: Model } => - opts.provider === 'openai-compat' ? resolveOpenAiCompat(opts) : resolveAnthropic(opts.model) +export const resolveModel = ( + opts: ResolveModelOptions, + dependencies: ResolveModelDependencies = DEFAULT_DEPENDENCIES, +): { models: Models; model: Model } => { + const provider = opts.provider ?? DEFAULT_PROVIDER + if (provider === 'openai-compat') return resolveOpenAiCompat(opts) + return resolveBuiltin(provider, opts.model, opts.apiKey, dependencies) +} diff --git a/cli/src/agent/openai-compat-model.ts b/cli/src/agent/openai-compat-model.ts index 686c7635c..b6c71f067 100644 --- a/cli/src/agent/openai-compat-model.ts +++ b/cli/src/agent/openai-compat-model.ts @@ -5,7 +5,7 @@ /** * Builds a Pi `openai-completions` model bound to a custom base URL + bearer * key, so the CLI can run any OpenAI-compatible endpoint (e.g. Xiaomi MiMo at - * `https://token-plan-sgp.xiaomimimo.com/v1`) instead of only Anthropic. + * `https://token-plan-sgp.xiaomimimo.com/v1`) outside Pi's built-in providers. * * This is the CLI sibling of `shared/agent-core/openai-compat-model.ts`, but * deliberately simpler. The app's version SYNCHRONOUSLY swaps `globalThis.fetch` @@ -108,7 +108,7 @@ const synthesizeModel = (opts: BuildOpenAiCompatModelOptions): Model /** * Resolves an OpenAI-compatible model and wires it through a Pi provider bound * to `opts.baseUrl` + `opts.apiKey`. Drop-in sibling of `resolveModel`'s - * Anthropic branch: returns the same `{ models, model }` shape the harness + * built-in branch: returns the same `{ models, model }` shape the harness * consumes. * * @param opts - model id, base URL, and bearer api key diff --git a/cli/src/agent/run.test.ts b/cli/src/agent/run.test.ts index c109dd8c8..0dd645a61 100644 --- a/cli/src/agent/run.test.ts +++ b/cli/src/agent/run.test.ts @@ -3,30 +3,35 @@ * file, You can obtain one at http://mozilla.org/MPL/2.0/. */ /** - * Unit tests for the `runAgent` pre-flight guard: the anthropic provider (the - * default) refuses to start without `ANTHROPIC_API_KEY`, and the guard keys off - * the *resolved* provider (undefined → anthropic) so it fires before any harness - * is built. Only this fail-fast branch is unit-tested; the success path drives a - * live model and belongs to integration coverage. + * Unit tests for provider-aware model preflight plus TUI mode selection. Success + * paths drive live providers and belong to integration coverage. */ import { afterEach, beforeEach, describe, expect, test } from 'bun:test' import { runAgent, shouldUseTui } from './run.ts' import type { RunConfig } from './types.ts' -const KEY = 'ANTHROPIC_API_KEY' -let saved: string | undefined +const KEYS = ['ANTHROPIC_OAUTH_TOKEN', 'ANTHROPIC_API_KEY'] as const +const saved: Partial> = {} beforeEach(() => { - saved = process.env[KEY] - delete process.env[KEY] + for (const key of KEYS) { + const value = process.env[key] + if (value !== undefined) saved[key] = value + delete process.env[key] + } }) afterEach(() => { - if (saved === undefined) delete process.env[KEY] - else process.env[KEY] = saved + for (const key of KEYS) { + const value = saved[key] + if (value === undefined) delete process.env[key] + else process.env[key] = value + delete saved[key] + } }) +/** Builds a one-shot run configuration with targeted overrides. */ const oneshot = (overrides: Partial = {}): RunConfig => ({ model: 'claude-opus-4-8', @@ -38,6 +43,7 @@ const oneshot = (overrides: Partial = {}): RunConfig => ...overrides, }) as RunConfig +/** Builds a REPL run configuration with targeted overrides. */ const repl = (overrides: Partial = {}): RunConfig => ({ model: 'claude-opus-4-8', @@ -49,23 +55,19 @@ const repl = (overrides: Partial = {}): RunConfig => ...overrides, }) as RunConfig -describe('runAgent — ANTHROPIC_API_KEY guard', () => { +describe('runAgent — provider credential preflight', () => { test('throws a friendly error for the explicit anthropic provider with no key', async () => { await expect(runAgent(oneshot({ provider: 'anthropic' }))).rejects.toThrow(/ANTHROPIC_API_KEY/) }) test('the default (unset) provider also requires the key', async () => { - // provider omitted → `?? 'anthropic'` → same guard fires. - await expect(runAgent(oneshot({ provider: undefined }))).rejects.toThrow(/set ANTHROPIC_API_KEY/) + await expect(runAgent(oneshot({ provider: undefined }))).rejects.toThrow(/ANTHROPIC_API_KEY/) }) - test('openai-compat skips the anthropic guard — it fails on its own missing config instead', async () => { - // The inverse branch: with no ANTHROPIC_API_KEY set, an anthropic run throws the - // key error, but openai-compat must get past the guard and fail downstream in - // model resolution (missing --base-url) — proving the provider condition matters. + test('openai-compat reports its dedicated key and guided setup when credentials are missing', async () => { await expect( runAgent(oneshot({ provider: 'openai-compat', baseUrl: undefined, apiKey: undefined })), - ).rejects.toThrow(/base-url/) + ).rejects.toThrow(/THUNDERBOLT_OPENAI_COMPAT_KEY.*guided setup/) }) }) diff --git a/cli/src/agent/run.ts b/cli/src/agent/run.ts index 9963c026a..3ed0ab1c8 100644 --- a/cli/src/agent/run.ts +++ b/cli/src/agent/run.ts @@ -48,18 +48,12 @@ const runRepl = async (harness: AgentHarness, io: TerminalIO): Promise => } /** - * Runs the agent for a single CLI invocation. Requires `ANTHROPIC_API_KEY`; - * exits with a friendly message when it's absent. + * Runs agent for one CLI invocation. Harness model resolution validates + * provider-specific credentials before execution resources are allocated. * * @param config - the resolved configuration from `parseArgs` */ export const runAgent = async (config: RunConfig): Promise => { - // Anthropic resolves its key from the environment; openai-compat carries its - // own (validated in `resolveModel`), so it must not demand ANTHROPIC_API_KEY. - if ((config.provider ?? 'anthropic') === 'anthropic' && !process.env.ANTHROPIC_API_KEY) { - throw new Error('set ANTHROPIC_API_KEY to run the agent (https://console.anthropic.com).') - } - const { harness, dispose } = await buildHarness(config) try { diff --git a/cli/src/agent/types.ts b/cli/src/agent/types.ts index 87afcee19..1869dbd84 100644 --- a/cli/src/agent/types.ts +++ b/cli/src/agent/types.ts @@ -26,9 +26,32 @@ export type HarnessBundle = { /** Reasoning depth passed to the Pi harness (`thinkingLevel`). */ export type ThinkingLevel = 'off' | 'minimal' | 'low' | 'medium' | 'high' | 'xhigh' -/** Which model backend the harness talks to: Anthropic's built-in catalog, or - * any OpenAI-compatible endpoint at a custom base URL (e.g. Xiaomi MiMo). */ -export type ModelProvider = 'anthropic' | 'openai-compat' +/** Built-in Pi providers exposed by thunderbolt. */ +export const BUILTIN_PROVIDERS = [ + 'anthropic', + 'openai', + 'google', + 'xai', + 'deepseek', + 'zai', + 'mistral', + 'groq', + 'openrouter', + 'moonshotai', + 'minimax', + 'cerebras', + 'together', + 'fireworks', +] as const + +/** Built-in Pi provider exposed by thunderbolt. */ +export type BuiltinProvider = (typeof BUILTIN_PROVIDERS)[number] + +/** All model backends accepted by `--provider`. */ +export const MODEL_PROVIDERS = [...BUILTIN_PROVIDERS, 'openai-compat'] as const + +/** Model backend selected for a harness. */ +export type ModelProvider = (typeof MODEL_PROVIDERS)[number] /** Wire protocol whose local stdio process the bridge exposes over the network. * Drives only logging — the stdio↔transport pump is byte-identical for both. */ @@ -80,8 +103,7 @@ export type IrohAdminAction = * consumes exactly this; the run/serve configs extend it with their own fields. */ export type HarnessConfig = { - /** Model id: an Anthropic catalog id (e.g. `claude-opus-4-8`) for the - * `anthropic` provider, or the upstream id (e.g. `mimo-v2.5-pro`) for + /** Pi catalog model id for built-in providers, or upstream model id for * `openai-compat`. */ readonly model: string /** Working directory the agent's bash/fs tools are bound to. */ @@ -96,9 +118,8 @@ export type HarnessConfig = { readonly provider?: ModelProvider /** OpenAI-compatible base URL — required when `provider` is `openai-compat`. */ readonly baseUrl?: string - /** Bearer api key for `openai-compat` (flows in only via flag/env at runtime, - * never persisted). Ignored by the `anthropic` provider, which reads - * `ANTHROPIC_API_KEY` from the environment. */ + /** Explicit provider api key. Built-in providers otherwise resolve their own + * environment variable; openai-compat uses its dedicated CLI env fallback. */ readonly apiKey?: string /** When true, the system prompt names the underlying model so an exposed ACP * agent can self-identify. The standalone CLI leaves this off. */ @@ -126,10 +147,11 @@ export type RunConfig = readonly noTui: boolean }) -/** Result of parsing argv: a run, a bridge, a connect, an ACP server, an iroh - * admin action, or a terminal info action. */ +/** Result of parsing argv: a run, config setup, bridge, connect, ACP server, + * iroh admin action, or terminal info action. */ export type ParsedArgs = | { readonly kind: 'run'; readonly config: RunConfig } + | { readonly kind: 'config' } | { readonly kind: 'bridge'; readonly config: BridgeConfig } | { readonly kind: 'connect'; readonly config: ConnectConfig } | { readonly kind: 'acp-serve'; readonly config: ServeConfig } diff --git a/cli/src/cli.test.ts b/cli/src/cli.test.ts index f7941ba80..178e3d7ee 100644 --- a/cli/src/cli.test.ts +++ b/cli/src/cli.test.ts @@ -9,10 +9,13 @@ * the value-validating flags, and the bridge/serve/connect subcommand routing. */ -import { afterEach, beforeEach, describe, expect, test } from 'bun:test' +import { describe, expect, test } from 'bun:test' import packageJson from '../package.json' with { type: 'json' } import rootPackageJson from '../../package.json' with { type: 'json' } import { parseArgs, VERSION } from './cli.ts' +import type { ParseArgsDependencies } from './cli.ts' +import { BUILTIN_PROVIDERS } from './agent/types.ts' +import type { CliConfig } from './config/config.ts' const ENV_KEY = 'THUNDERBOLT_OPENAI_COMPAT_KEY' @@ -22,24 +25,14 @@ test('VERSION and the CLI package match the released app version', () => { }) /** Narrow a ParsedArgs to a `run` config or fail loudly. */ -const runConfig = (argv: string[]) => { - const parsed = parseArgs(argv) +const runConfig = (argv: string[], dependencies?: ParseArgsDependencies) => { + const parsed = parseArgs(argv, dependencies) if (parsed.kind !== 'run') throw new Error(`expected run, got ${parsed.kind}: ${JSON.stringify(parsed)}`) return parsed.config } describe('parseArgs — resolveApiKey precedence (security)', () => { - const saved = process.env[ENV_KEY] - beforeEach(() => { - delete process.env[ENV_KEY] - }) - afterEach(() => { - if (saved === undefined) delete process.env[ENV_KEY] - else process.env[ENV_KEY] = saved - }) - test('--api-key flag wins over the env var', () => { - process.env[ENV_KEY] = 'env-key' const config = runConfig([ '--provider', 'openai-compat', @@ -48,35 +41,35 @@ describe('parseArgs — resolveApiKey precedence (security)', () => { '--api-key', 'flag-key', 'hi', - ]) + ], { env: { [ENV_KEY]: 'env-key' } }) expect(config.apiKey).toBe('flag-key') }) test('falls back to the env var when no flag is given', () => { - process.env[ENV_KEY] = 'env-key' - const config = runConfig(['--provider', 'openai-compat', '--base-url', 'https://h/v1', 'hi']) + const config = runConfig(['--provider', 'openai-compat', '--base-url', 'https://h/v1', 'hi'], { + env: { [ENV_KEY]: 'env-key' }, + }) expect(config.apiKey).toBe('env-key') }) test('is undefined when neither flag nor env is set', () => { - const config = runConfig(['--provider', 'openai-compat', '--base-url', 'https://h/v1', 'hi']) + const config = runConfig(['--provider', 'openai-compat', '--base-url', 'https://h/v1', 'hi'], { env: {} }) expect(config.apiKey).toBeUndefined() }) test('does not auto-forward a standard OPENAI_API_KEY (dedicated var only)', () => { - const savedOpenai = process.env.OPENAI_API_KEY - process.env.OPENAI_API_KEY = 'sk-real-openai' - try { - const config = runConfig(['--provider', 'openai-compat', '--base-url', 'https://h/v1', 'hi']) - expect(config.apiKey).toBeUndefined() - } finally { - if (savedOpenai === undefined) delete process.env.OPENAI_API_KEY - else process.env.OPENAI_API_KEY = savedOpenai - } + const config = runConfig(['--provider', 'openai-compat', '--base-url', 'https://h/v1', 'hi'], { + env: { OPENAI_API_KEY: 'sk-real-openai' }, + }) + expect(config.apiKey).toBeUndefined() + }) + + test('does not forward the openai-compat env key to a built-in provider', () => { + expect(runConfig(['--provider', 'openai', 'hi'], { env: { [ENV_KEY]: 'custom-host-key' } }).apiKey).toBeUndefined() }) test('the api key never leaks into the prompt positionals', () => { - const config = runConfig(['--api-key', 'super-secret', 'fix', 'the', 'bug']) + const config = runConfig(['--api-key', 'super-secret', 'fix', 'the', 'bug'], { env: {} }) if (config.mode !== 'oneshot') throw new Error('expected oneshot') expect(config.prompt).toBe('fix the bug') expect(config.prompt).not.toContain('super-secret') @@ -84,7 +77,7 @@ describe('parseArgs — resolveApiKey precedence (security)', () => { }) test('an --api-key consumed at the end of argv does not become a positional prompt', () => { - const config = runConfig(['hello', '--api-key', 'k']) + const config = runConfig(['hello', '--api-key', 'k'], { env: {} }) if (config.mode !== 'oneshot') throw new Error('expected oneshot') expect(config.prompt).toBe('hello') }) @@ -119,6 +112,12 @@ describe('parseArgs — flag validation', () => { expect(config.baseUrl).toBe('https://h/v1') }) + test('accepts every curated built-in provider', () => { + for (const provider of BUILTIN_PROVIDERS) { + expect(runConfig(['--provider', provider]).provider).toBe(provider) + } + }) + test('the -m alias sets the model just like --model', () => { expect(runConfig(['-m', 'claude-x', 'go']).model).toBe('claude-x') }) @@ -134,6 +133,132 @@ describe('parseArgs — defaults', () => { expect(config.yolo).toBe(false) expect(config.baseUrl).toBeUndefined() }) + + test('uses provider-specific default models when --model is omitted', () => { + const expected = { + anthropic: 'claude-opus-4-8', + openai: 'gpt-5.3-codex', + google: 'gemini-3.1-pro-preview', + xai: 'grok-build-0.1', + deepseek: 'deepseek-v4-pro', + zai: 'glm-5.2', + mistral: 'devstral-medium-latest', + groq: 'openai/gpt-oss-120b', + openrouter: 'anthropic/claude-opus-4.8', + moonshotai: 'kimi-k2.7-code', + minimax: 'MiniMax-M3', + cerebras: 'gpt-oss-120b', + together: 'moonshotai/Kimi-K2.7-Code', + fireworks: 'accounts/fireworks/models/kimi-k2p7-code', + } as const + + for (const provider of BUILTIN_PROVIDERS) { + expect(runConfig(['--provider', provider]).model).toBe(expected[provider]) + } + }) + + test('an explicit --model wins over the provider default', () => { + expect(runConfig(['--provider', 'google', '--model', 'gemini-custom']).model).toBe('gemini-custom') + }) +}) + +describe('parseArgs — persisted config precedence', () => { + const stored: CliConfig = { + provider: 'openai-compat', + model: 'saved-model', + apiKey: 'saved-key', + baseUrl: 'https://saved.example/v1', + } + + test('uses saved provider, model, key, and base URL when flags and env are silent', () => { + const config = runConfig([], { config: stored, env: {}, cwd: '/repo' }) + + expect(config).toEqual({ + mode: 'repl', + noTui: false, + model: 'saved-model', + cwd: '/repo', + yolo: false, + thinking: 'medium', + provider: 'openai-compat', + apiKey: 'saved-key', + baseUrl: 'https://saved.example/v1', + }) + }) + + test('explicit flags override every saved field', () => { + const config = runConfig( + [ + '--provider', + 'anthropic', + '--model', + 'flag-model', + '--api-key', + 'flag-key', + '--base-url', + 'https://flag.example/v1', + ], + { config: stored, env: {}, cwd: '/repo' }, + ) + + expect(config.provider).toBe('anthropic') + expect(config.model).toBe('flag-model') + expect(config.apiKey).toBe('flag-key') + expect(config.baseUrl).toBe('https://flag.example/v1') + }) + + test('matching built-in provider env suppresses saved-key injection so Pi owns env auth', () => { + const config = runConfig([], { + config: { provider: 'openai', model: 'gpt-5.3-codex', apiKey: 'saved-key' }, + env: { OPENAI_API_KEY: 'env-key' }, + }) + + expect(config.apiKey).toBeUndefined() + }) + + test('dedicated openai-compat env key wins over saved key', () => { + const config = runConfig([], { + config: stored, + env: { THUNDERBOLT_OPENAI_COMPAT_KEY: 'env-key' }, + }) + + expect(config.apiKey).toBe('env-key') + }) + + test('an empty dedicated env key is silent and falls back to saved key', () => { + const config = runConfig([], { + config: stored, + env: { THUNDERBOLT_OPENAI_COMPAT_KEY: '' }, + }) + + expect(config.apiKey).toBe('saved-key') + }) + + test('saved key and base URL do not cross provider boundaries', () => { + const config = runConfig(['--provider', 'anthropic'], { config: stored, env: {} }) + + expect(config.apiKey).toBeUndefined() + expect(config.baseUrl).toBeUndefined() + }) + + test('generic OpenAI env key never forwards to a saved custom endpoint', () => { + const config = runConfig([], { config: stored, env: { OPENAI_API_KEY: 'real-openai-key' } }) + + expect(config.apiKey).toBe('saved-key') + }) +}) + +describe('parseArgs — config subcommand', () => { + test('routes thunderbolt config to interactive setup', () => { + expect(parseArgs(['config'])).toEqual({ kind: 'config' }) + }) + + test('rejects arguments after config', () => { + expect(parseArgs(['config', 'extra'])).toEqual({ + kind: 'error', + message: "thunderbolt config: unexpected argument 'extra'", + }) + }) }) describe('parseArgs — run mode + yolo aliases', () => { @@ -219,33 +344,36 @@ describe('parseArgs — bridge subcommands (acp / mcp)', () => { }) describe('parseArgs — acp serve', () => { - const saved = process.env[ENV_KEY] - afterEach(() => { - if (saved === undefined) delete process.env[ENV_KEY] - else process.env[ENV_KEY] = saved - }) - test('resolves the same flag set as a run, including api-key precedence', () => { - process.env[ENV_KEY] = 'env-key' - const parsed = parseArgs([ - 'acp', - 'serve', - '--provider', - 'openai-compat', - '--base-url', - 'https://h/v1', - '--api-key', - 'flag-key', - ]) + const parsed = parseArgs( + [ + 'acp', + 'serve', + '--provider', + 'openai-compat', + '--base-url', + 'https://h/v1', + '--api-key', + 'flag-key', + ], + { env: { [ENV_KEY]: 'env-key' } }, + ) if (parsed.kind !== 'acp-serve') throw new Error(`expected acp-serve, got ${parsed.kind}`) expect(parsed.config.apiKey).toBe('flag-key') expect(parsed.config.provider).toBe('openai-compat') }) test('rejects a stray positional (serve takes no prompt)', () => { - const parsed = parseArgs(['acp', 'serve', 'unexpected']) + const parsed = parseArgs(['acp', 'serve', 'unexpected'], { env: {} }) expect(parsed).toEqual({ kind: 'error', message: expect.stringContaining("unexpected argument 'unexpected'") }) }) + + test('uses the same provider-specific model default as agent runs', () => { + const parsed = parseArgs(['acp', 'serve', '--provider', 'google'], { env: {} }) + if (parsed.kind !== 'acp-serve') throw new Error(`expected acp-serve, got ${parsed.kind}`) + expect(parsed.config.model).toBe('gemini-3.1-pro-preview') + expect(parsed.config.provider).toBe('google') + }) }) describe('parseArgs — connect + iroh admin', () => { diff --git a/cli/src/cli.ts b/cli/src/cli.ts index b1a045c13..3e118a9fc 100644 --- a/cli/src/cli.ts +++ b/cli/src/cli.ts @@ -9,6 +9,8 @@ */ import packageJson from '../package.json' with { type: 'json' } +import { MODEL_PROVIDERS } from './agent/types.ts' +import { BUILTIN_PROVIDER_ENV_VARS, DEFAULT_MODEL, DEFAULT_MODELS, DEFAULT_PROVIDER } from './agent/defaults.ts' import type { BridgeConfig, BridgeProtocol, @@ -19,19 +21,11 @@ import type { ServeConfig, ThinkingLevel, } from './agent/types.ts' +import type { CliConfig } from './config/config.ts' /** Released version of the CLI, surfaced by `--version` and the banner. */ export const VERSION = packageJson.version -/** Default Anthropic model when `--model` is omitted. */ -const DEFAULT_MODEL = 'claude-opus-4-8' - -/** Default model backend when `--provider` is omitted. */ -const DEFAULT_PROVIDER: ModelProvider = 'anthropic' - -/** All valid `--provider` values. */ -const MODEL_PROVIDERS: readonly ModelProvider[] = ['anthropic', 'openai-compat'] - /** All valid `--thinking` levels, in increasing depth. */ const THINKING_LEVELS: readonly ThinkingLevel[] = ['off', 'minimal', 'low', 'medium', 'high', 'xhigh'] @@ -51,6 +45,7 @@ export const HELP_TEXT = `⚡ thunderbolt v${VERSION} — a single-binary termin USAGE thunderbolt [options] [prompt] thunderbolt agent [options] [prompt] + thunderbolt config thunderbolt acp serve [options] thunderbolt acp --transport [--port N] -- thunderbolt mcp --transport [--port N] -- @@ -58,10 +53,11 @@ USAGE thunderbolt iroh > With a prompt, runs it once and exits. With no prompt, starts an - interactive REPL. Built on the Pi harness; talks to Claude. + interactive REPL. Built on the Pi harness; supports multiple model providers. SUBCOMMANDS agent run the coding agent (default when omitted) + config run guided provider setup and overwrite saved CLI defaults acp serve expose THIS coding agent, rooted at current directory, as a stdio ACP server acp bridge a local stdio ACP agent over the network the app can reach mcp bridge a local stdio MCP server over the network the app can reach @@ -74,11 +70,11 @@ TOOLS edit replace a span within a file OPTIONS - -m, --model model id (default: ${DEFAULT_MODEL}) + -m, --model model id (default: provider-specific; + anthropic uses ${DEFAULT_MODEL}) --provider

model backend: ${MODEL_PROVIDERS.join(' | ')} (default: ${DEFAULT_PROVIDER}) --base-url OpenAI-compatible base URL (required for openai-compat) - --api-key bearer key for openai-compat (or set - THUNDERBOLT_OPENAI_COMPAT_KEY; the flag wins) + --api-key explicit provider api key (flag wins over provider env) --thinking reasoning depth: ${THINKING_LEVELS.join(' | ')} (default: medium) -y, --yolo auto-approve every tool call (alias: --dangerously-skip-permissions) @@ -105,6 +101,9 @@ EXAMPLES thunderbolt "fix the failing test in utils.ts" thunderbolt --thinking high "refactor the auth module" thunderbolt --yolo "run the test suite and fix what breaks" + OPENAI_API_KEY=sk-… thunderbolt --provider openai "fix the failing tests" + thunderbolt --provider google --api-key AIza… "review this repository" + thunderbolt config thunderbolt THUNDERBOLT_OPENAI_COMPAT_KEY=sk-… thunderbolt --provider openai-compat --base-url https://host/v1 --model my-model "hello" thunderbolt acp --transport wss -- npx @zed-industries/claude-code-acp @@ -114,9 +113,11 @@ EXAMPLES thunderbolt iroh id thunderbolt acp connect endpoint1abc… # dial a remote iroh bridge -Requires ANTHROPIC_API_KEY (https://console.anthropic.com) for the default -anthropic provider; openai-compat instead uses --api-key / -THUNDERBOLT_OPENAI_COMPAT_KEY.` +Provider, model, key, and custom URL defaults can be saved in +~/.thunderbolt/config.json (or $THUNDERBOLT_HOME/config.json). Resolution order +is flag, provider environment variable, config, then built-in default. Run +thunderbolt in a terminal for guided first-run setup, or thunderbolt config to +reconfigure. openai-compat never reads generic provider keys.` /** Type guard: is `value` one of the supported {@link ThinkingLevel}s? */ const isThinkingLevel = (value: string): value is ThinkingLevel => @@ -127,22 +128,20 @@ const isProvider = (value: string): value is ModelProvider => (MODEL_PROVIDERS a /** Flag/positional state accumulated while scanning argv. */ type Flags = { - readonly model: string + readonly model?: string readonly yolo: boolean readonly noTui: boolean readonly thinking: ThinkingLevel - readonly provider: ModelProvider + readonly provider?: ModelProvider readonly baseUrl?: string readonly apiKey?: string readonly positionals: readonly string[] } const DEFAULT_FLAGS: Flags = { - model: DEFAULT_MODEL, yolo: false, noTui: false, thinking: 'medium', - provider: DEFAULT_PROVIDER, positionals: [], } @@ -305,11 +304,33 @@ const parseConnectArgs = (protocol: BridgeProtocol, rest: string[]): ParsedArgs return { kind: 'connect', config: { protocol, target, command } } } +/** Injectable inputs for deterministic config and environment resolution. */ +export type ParseArgsDependencies = { + readonly env?: Readonly> + readonly config?: CliConfig | null + readonly cwd?: string +} + +type ResolvedDependencies = { + readonly env: Readonly> + readonly config: CliConfig | null + readonly cwd: string +} + +/** Fills omitted parser dependencies from current process state. */ +const resolveDependencies = (dependencies: ParseArgsDependencies): ResolvedDependencies => ({ + env: dependencies.env ?? process.env, + config: dependencies.config ?? null, + cwd: dependencies.cwd ?? process.cwd(), +}) + +/** Resolves provider flag against saved provider and built-in default. */ +const resolveProvider = (flags: Flags, config: CliConfig | null): ModelProvider => + flags.provider ?? config?.provider ?? DEFAULT_PROVIDER + /** - * Resolves the openai-compat bearer key: the explicit `--api-key` flag wins, - * else the dedicated `THUNDERBOLT_OPENAI_COMPAT_KEY` env var. Returns `undefined` - * when neither is set (the anthropic provider ignores it; `resolveModel` rejects - * openai-compat). + * Resolves an explicit provider key. Every provider accepts `--api-key`, while + * only openai-compat reads a CLI-level fallback environment variable. * * Deliberately scoped to a dedicated var rather than falling back to a standard * key like `OPENAI_API_KEY`: openai-compat sends this key to an arbitrary @@ -317,11 +338,58 @@ const parseConnectArgs = (protocol: BridgeProtocol, rest: string[]): ParsedArgs * leak credentials. `THUNDERBOLT_OPENAI_COMPAT_KEY` is the explicit opt-in for * "this key is meant for whatever host I point at". * - * @param flagApiKey - the value passed via `--api-key`, if any + * @param provider - selected model provider + * @param flagApiKey - value passed via `--api-key`, if any * @returns the resolved api key, or `undefined` */ -const resolveApiKey = (flagApiKey?: string): string | undefined => - flagApiKey ?? process.env.THUNDERBOLT_OPENAI_COMPAT_KEY +const resolveApiKey = ( + provider: ModelProvider, + flagApiKey: string | undefined, + dependencies: ResolvedDependencies, +): string | undefined => { + if (flagApiKey !== undefined) return flagApiKey + if (provider === 'openai-compat') { + return ( + dependencies.env.THUNDERBOLT_OPENAI_COMPAT_KEY || + (dependencies.config?.provider === provider ? dependencies.config.apiKey : undefined) + ) + } + + const hasProviderEnvKey = BUILTIN_PROVIDER_ENV_VARS[provider].some((name) => Boolean(dependencies.env[name])) + if (hasProviderEnvKey) return undefined + return dependencies.config?.provider === provider ? dependencies.config.apiKey : undefined +} + +/** Resolves omitted `--model` against selected provider's catalog default. */ +const resolveModelId = (flags: Flags, provider: ModelProvider, config: CliConfig | null): string => { + if (flags.model !== undefined) return flags.model + if (config !== null) return config.model + return provider === 'openai-compat' ? DEFAULT_MODEL : DEFAULT_MODELS[provider] +} + +/** Resolves custom endpoint flag against provider-scoped saved config. */ +const resolveBaseUrl = ( + flags: Flags, + provider: ModelProvider, + config: CliConfig | null, +): string | undefined => { + if (flags.baseUrl !== undefined) return flags.baseUrl + return config?.provider === provider ? config.baseUrl : undefined +} + +/** Resolves harness fields after argv scanning preserves explicit flags. */ +const resolveAgentFlags = (flags: Flags, dependencies: ResolvedDependencies) => { + const provider = resolveProvider(flags, dependencies.config) + return { + model: resolveModelId(flags, provider, dependencies.config), + cwd: dependencies.cwd, + yolo: flags.yolo, + thinking: flags.thinking, + provider, + baseUrl: resolveBaseUrl(flags, provider, dependencies.config), + apiKey: resolveApiKey(provider, flags.apiKey, dependencies), + } +} /** * Parses an `acp serve` invocation: run the built-in agent as a stdio ACP @@ -330,7 +398,7 @@ const resolveApiKey = (flagApiKey?: string): string | undefined => * that scopes every ACP session. Client-supplied cwd values are ignored. No * positional prompt is accepted. */ -const parseServeArgs = (rest: string[]): ParsedArgs => { +const parseServeArgs = (rest: string[], dependencies: ResolvedDependencies): ParsedArgs => { if (rest.includes('--help') || rest.includes('-h')) return { kind: 'help' } const scan = scanTokens(rest, 0, DEFAULT_FLAGS) @@ -339,15 +407,7 @@ const parseServeArgs = (rest: string[]): ParsedArgs => { return { kind: 'error', message: `thunderbolt acp serve: unexpected argument '${scan.flags.positionals[0]}'` } } - const config: ServeConfig = { - model: scan.flags.model, - cwd: process.cwd(), - yolo: scan.flags.yolo, - thinking: scan.flags.thinking, - provider: scan.flags.provider, - baseUrl: scan.flags.baseUrl, - apiKey: resolveApiKey(scan.flags.apiKey), - } + const config: ServeConfig = resolveAgentFlags(scan.flags, dependencies) return { kind: 'acp-serve', config } } @@ -379,11 +439,18 @@ const parseIrohAdminArgs = (rest: string[]): ParsedArgs => { * @returns a terminal info action (`help`/`version`/`error`), a `run` with a * fully-resolved {@link RunConfig}, or a `bridge` with a {@link BridgeConfig} */ -export const parseArgs = (argv: string[]): ParsedArgs => { +export const parseArgs = (argv: string[], injected: ParseArgsDependencies = {}): ParsedArgs => { + const dependencies = resolveDependencies(injected) const subcommand = argv[0] + if (subcommand === 'config') { + const argument = argv[1] + if (argument === undefined) return { kind: 'config' } + if (argument === '--help' || argument === '-h') return { kind: 'help' } + return { kind: 'error', message: `thunderbolt config: unexpected argument '${argument}'` } + } if (subcommand === 'iroh') return parseIrohAdminArgs(argv.slice(1)) if (subcommand === 'acp' || subcommand === 'mcp') { - if (subcommand === 'acp' && argv[1] === 'serve') return parseServeArgs(argv.slice(2)) + if (subcommand === 'acp' && argv[1] === 'serve') return parseServeArgs(argv.slice(2), dependencies) if (argv[1] === 'connect') return parseConnectArgs(subcommand, argv.slice(2)) return parseBridgeArgs(subcommand, argv.slice(1)) } @@ -396,15 +463,7 @@ export const parseArgs = (argv: string[]): ParsedArgs => { if (!scan.ok) return { kind: 'error', message: scan.message } const prompt = scan.flags.positionals.join(' ') - const base = { - model: scan.flags.model, - cwd: process.cwd(), - yolo: scan.flags.yolo, - thinking: scan.flags.thinking, - provider: scan.flags.provider, - baseUrl: scan.flags.baseUrl, - apiKey: resolveApiKey(scan.flags.apiKey), - } + const base = resolveAgentFlags(scan.flags, dependencies) const config: RunConfig = prompt.length > 0 ? { ...base, mode: 'oneshot', prompt } : { ...base, mode: 'repl', noTui: scan.flags.noTui } return { kind: 'run', config } diff --git a/cli/src/config/config.test.ts b/cli/src/config/config.test.ts new file mode 100644 index 000000000..ebc20480f --- /dev/null +++ b/cli/src/config/config.test.ts @@ -0,0 +1,71 @@ +/* This Source Code Form is subject to the terms of the Mozilla Public + * License, v. 2.0. If a copy of the MPL was not distributed with this + * file, You can obtain one at http://mozilla.org/MPL/2.0/. */ + +import { mkdir, mkdtemp, rm, stat, writeFile } from 'node:fs/promises' +import { tmpdir } from 'node:os' +import { dirname, join } from 'node:path' +import { afterEach, describe, expect, test } from 'bun:test' +import { loadConfig, saveConfig } from './config.ts' +import type { CliConfig } from './config.ts' + +const tempDirs: string[] = [] + +/** Allocates one nested config path and tracks its temp root for cleanup. */ +const temporaryConfigPath = async (): Promise => { + const dir = await mkdtemp(join(tmpdir(), 'thunderbolt-config-')) + tempDirs.push(dir) + return join(dir, 'state', 'config.json') +} + +/** Writes user-edited config text after creating its parent directory. */ +const writeRawConfig = async (path: string, contents: string): Promise => { + await mkdir(dirname(path), { recursive: true }) + await writeFile(path, contents) +} + +afterEach(async () => { + await Promise.all(tempDirs.splice(0).map((dir) => rm(dir, { recursive: true, force: true }))) +}) + +describe('CLI config persistence', () => { + test('roundtrips the typed config shape', async () => { + const path = await temporaryConfigPath() + const config: CliConfig = { + provider: 'openai-compat', + model: 'local-model', + apiKey: 'secret', + baseUrl: 'http://localhost:11434/v1', + } + + await saveConfig(config, path) + + expect(await loadConfig(path)).toEqual(config) + }) + + test('writes config owner-only with mode 0600', async () => { + const path = await temporaryConfigPath() + + await saveConfig({ provider: 'anthropic', model: 'claude-opus-4-8', apiKey: 'secret' }, path) + + expect((await stat(path)).mode & 0o777).toBe(0o600) + }) + + test('treats a missing file as absent', async () => { + expect(await loadConfig(await temporaryConfigPath())).toBeNull() + }) + + test('treats malformed JSON as absent', async () => { + const path = await temporaryConfigPath() + await writeRawConfig(path, '{not-json') + + expect(await loadConfig(path)).toBeNull() + }) + + test('treats an invalid config shape as absent', async () => { + const path = await temporaryConfigPath() + await writeRawConfig(path, JSON.stringify({ provider: 'bogus', model: 42 })) + + expect(await loadConfig(path)).toBeNull() + }) +}) diff --git a/cli/src/config/config.ts b/cli/src/config/config.ts new file mode 100644 index 000000000..fbd332241 --- /dev/null +++ b/cli/src/config/config.ts @@ -0,0 +1,69 @@ +/* This Source Code Form is subject to the terms of the Mozilla Public + * License, v. 2.0. If a copy of the MPL was not distributed with this + * file, You can obtain one at http://mozilla.org/MPL/2.0/. */ + +/** Typed persistence for user-editable CLI defaults. */ + +import { chmod, mkdir, readFile, writeFile } from 'node:fs/promises' +import { dirname } from 'node:path' +import { MODEL_PROVIDERS } from '../agent/types.ts' +import type { ModelProvider } from '../agent/types.ts' +import { configPath } from '../paths.ts' + +const FILE_MODE = 0o600 +const DIR_MODE = 0o700 + +/** Minimal persisted CLI profile. */ +export type CliConfig = { + readonly provider: ModelProvider + readonly model: string + readonly apiKey?: string + readonly baseUrl?: string +} + +/** Narrows unknown JSON values to object records. */ +const isRecord = (value: unknown): value is Record => + typeof value === 'object' && value !== null && !Array.isArray(value) + +/** Narrows user input to supported model providers. */ +const isProvider = (value: unknown): value is ModelProvider => + typeof value === 'string' && (MODEL_PROVIDERS as readonly string[]).includes(value) + +/** Validates unknown JSON and returns a minimal canonical config. */ +const parseConfig = (value: unknown): CliConfig | null => { + if (!isRecord(value) || !isProvider(value.provider) || typeof value.model !== 'string') return null + if (value.apiKey !== undefined && typeof value.apiKey !== 'string') return null + if (value.baseUrl !== undefined && typeof value.baseUrl !== 'string') return null + + return { + provider: value.provider, + model: value.model, + ...(value.apiKey === undefined ? {} : { apiKey: value.apiKey }), + ...(value.baseUrl === undefined ? {} : { baseUrl: value.baseUrl }), + } +} + +/** Loads config, treating missing, malformed, or invalid user input as absent. */ +export const loadConfig = async (path: string = configPath()): Promise => { + try { + const contents = await readFile(path, 'utf8') + try { + return parseConfig(JSON.parse(contents) as unknown) + } catch (error) { + if (error instanceof SyntaxError) return null + throw error + } + } catch (error) { + if (error instanceof Error && (error as NodeJS.ErrnoException).code === 'ENOENT') return null + throw error + } +} + +/** Saves config in an owner-only directory and forces file mode to `0600`. */ +export const saveConfig = async (config: CliConfig, path: string = configPath()): Promise => { + const dir = dirname(path) + await mkdir(dir, { recursive: true, mode: DIR_MODE }) + await chmod(dir, DIR_MODE) + await writeFile(path, `${JSON.stringify(config, null, 2)}\n`, { mode: FILE_MODE }) + await chmod(path, FILE_MODE) +} diff --git a/cli/src/config/wizard.test.ts b/cli/src/config/wizard.test.ts new file mode 100644 index 000000000..946cf0e13 --- /dev/null +++ b/cli/src/config/wizard.test.ts @@ -0,0 +1,161 @@ +/* This Source Code Form is subject to the terms of the Mozilla Public + * License, v. 2.0. If a copy of the MPL was not distributed with this + * file, You can obtain one at http://mozilla.org/MPL/2.0/. */ + +import { describe, expect, test } from 'bun:test' +import type { RunConfig } from '../agent/types.ts' +import { parseArgs } from '../cli.ts' +import type { CliConfig } from './config.ts' +import { runSetupWizard, shouldRunSetupWizard } from './wizard.ts' +import type { SetupWizardIO } from './wizard.ts' + +/** Builds a resolved run configuration for pure setup-decision tests. */ +const runConfig = (overrides: Partial = {}): RunConfig => + ({ + model: 'claude-opus-4-8', + cwd: '/repo', + yolo: false, + thinking: 'medium', + provider: 'anthropic', + mode: 'repl', + noTui: false, + ...overrides, + }) as RunConfig + +/** Builds default TTY runtime facts with targeted overrides. */ +const runtime = ( + overrides: Partial<{ + readonly stdinIsTty: boolean + readonly stdoutIsTty: boolean + readonly env: Readonly> + }> = {}, +) => ({ stdinIsTty: true, stdoutIsTty: true, env: {}, ...overrides }) + +/** Creates setup I/O backed by independent visible and secret answer queues. */ +const scriptedIO = (lines: readonly string[], secrets: readonly string[]) => { + const remainingLines = [...lines] + const remainingSecrets = [...secrets] + const output: string[] = [] + const io: SetupWizardIO = { + readLine: async () => remainingLines.shift() ?? null, + readSecret: async () => remainingSecrets.shift() ?? null, + write: (text) => output.push(text), + } + return { io, output } +} + +describe('shouldRunSetupWizard', () => { + test('uses setup for a credential-less run when stdin and stdout are TTYs', () => { + expect(shouldRunSetupWizard(runConfig(), runtime())).toBe(true) + }) + + test('does not use setup when either terminal stream is not a TTY', () => { + expect(shouldRunSetupWizard(runConfig(), runtime({ stdinIsTty: false }))).toBe(false) + expect(shouldRunSetupWizard(runConfig(), runtime({ stdoutIsTty: false }))).toBe(false) + }) + + test('does not use setup when an explicit or saved key is resolved', () => { + expect(shouldRunSetupWizard(runConfig({ apiKey: 'resolved-key' }), runtime())).toBe(false) + }) + + test('recognizes selected built-in provider environment credentials', () => { + expect( + shouldRunSetupWizard(runConfig({ provider: 'google' }), runtime({ env: { GEMINI_API_KEY: 'env-key' } })), + ).toBe(false) + }) + + test('custom endpoints recognize only the dedicated environment key', () => { + const custom = runConfig({ provider: 'openai-compat', baseUrl: 'https://host/v1' }) + expect( + shouldRunSetupWizard(custom, runtime({ env: { THUNDERBOLT_OPENAI_COMPAT_KEY: 'custom-key' } })), + ).toBe(false) + expect(shouldRunSetupWizard(custom, runtime({ env: { OPENAI_API_KEY: 'real-openai-key' } }))).toBe(true) + }) +}) + +describe('runSetupWizard', () => { + test('saves a built-in vendor, hidden key, and default model from scripted answers', async () => { + const { io, output } = scriptedIO(['2', ''], ['sk-openai']) + const saved: CliConfig[] = [] + + const result = await runSetupWizard(io, { + path: '/tmp/thunderbolt/config.json', + save: async (config) => { + saved.push(config) + }, + catalogIds: () => ['gpt-catalog-a', 'gpt-catalog-b'], + }) + + const expected: CliConfig = { provider: 'openai', model: 'gpt-5.3-codex', apiKey: 'sk-openai' } + expect(result).toEqual(expected) + expect(saved).toEqual([expected]) + expect(output.join('')).toContain('2. OpenAI') + expect(output.join('')).toContain('gpt-catalog-a') + expect(output.join('')).toContain('Saved config to /tmp/thunderbolt/config.json') + }) + + test('uses Ollama preset URL and a placeholder key when secret input is blank', async () => { + const { io } = scriptedIO(['15', 'qwen3-coder'], ['']) + const saved: CliConfig[] = [] + + await runSetupWizard(io, { + path: '/tmp/config.json', + save: async (config) => { + saved.push(config) + }, + catalogIds: () => [], + }) + + expect(saved).toEqual([ + { + provider: 'openai-compat', + model: 'qwen3-coder', + apiKey: 'local', + baseUrl: 'http://localhost:11434/v1', + }, + ]) + }) + + test('prompts custom endpoints for base URL, key, and free-form model', async () => { + const { io } = scriptedIO(['17', 'https://custom.example/v1', 'custom-model'], ['custom-key']) + const saved: CliConfig[] = [] + + await runSetupWizard(io, { + path: '/tmp/config.json', + save: async (config) => { + saved.push(config) + }, + catalogIds: () => [], + }) + + expect(saved).toEqual([ + { + provider: 'openai-compat', + model: 'custom-model', + apiKey: 'custom-key', + baseUrl: 'https://custom.example/v1', + }, + ]) + }) + + test('re-prompts when a choice conflicts with an explicit provider flag', async () => { + const { io, output } = scriptedIO(['2', '3', ''], ['google-key']) + const saved: CliConfig[] = [] + + const freshConfig = await runSetupWizard(io, { + path: '/tmp/config.json', + requiredProvider: 'google', + save: async (config) => { + saved.push(config) + }, + catalogIds: () => [], + }) + + expect(saved).toEqual([{ provider: 'google', model: 'gemini-3.1-pro-preview', apiKey: 'google-key' }]) + expect(output.join('')).toContain('--provider google overrides that choice') + const reparsed = parseArgs(['--provider', 'google'], { config: freshConfig, env: {} }) + if (reparsed.kind !== 'run') throw new Error(`expected run, got ${reparsed.kind}`) + expect(reparsed.config.apiKey).toBe('google-key') + expect(reparsed.config.model).toBe('gemini-3.1-pro-preview') + }) +}) diff --git a/cli/src/config/wizard.ts b/cli/src/config/wizard.ts new file mode 100644 index 000000000..6d3918b56 --- /dev/null +++ b/cli/src/config/wizard.ts @@ -0,0 +1,239 @@ +/* This Source Code Form is subject to the terms of the Mozilla Public + * License, v. 2.0. If a copy of the MPL was not distributed with this + * file, You can obtain one at http://mozilla.org/MPL/2.0/. */ + +/** First-run provider setup and its injectable terminal I/O seam. */ + +import { createInterface } from 'node:readline/promises' +import { Writable } from 'node:stream' +import { builtinModels } from '@earendil-works/pi-ai/providers/all' +import { BUILTIN_PROVIDER_ENV_VARS, DEFAULT_MODELS, DEFAULT_PROVIDER } from '../agent/defaults.ts' +import type { BuiltinProvider, ModelProvider, RunConfig } from '../agent/types.ts' +import { configPath } from '../paths.ts' +import { saveConfig } from './config.ts' +import type { CliConfig } from './config.ts' + +type BuiltinChoice = { + readonly kind: 'builtin' + readonly label: string + readonly provider: BuiltinProvider +} + +type CompatChoice = { + readonly kind: 'compat' + readonly label: string + readonly baseUrl?: string + readonly local: boolean +} + +type ProviderChoice = BuiltinChoice | CompatChoice + +const PROVIDER_CHOICES: readonly ProviderChoice[] = [ + { kind: 'builtin', label: 'Anthropic', provider: 'anthropic' }, + { kind: 'builtin', label: 'OpenAI', provider: 'openai' }, + { kind: 'builtin', label: 'Google (Gemini)', provider: 'google' }, + { kind: 'builtin', label: 'xAI (Grok)', provider: 'xai' }, + { kind: 'builtin', label: 'DeepSeek', provider: 'deepseek' }, + { kind: 'builtin', label: 'Z.AI', provider: 'zai' }, + { kind: 'builtin', label: 'Moonshot (Kimi)', provider: 'moonshotai' }, + { kind: 'builtin', label: 'Mistral', provider: 'mistral' }, + { kind: 'builtin', label: 'Groq', provider: 'groq' }, + { kind: 'builtin', label: 'Cerebras', provider: 'cerebras' }, + { kind: 'builtin', label: 'OpenRouter', provider: 'openrouter' }, + { kind: 'builtin', label: 'Together', provider: 'together' }, + { kind: 'builtin', label: 'Fireworks', provider: 'fireworks' }, + { kind: 'builtin', label: 'MiniMax', provider: 'minimax' }, + { kind: 'compat', label: 'Ollama (local)', baseUrl: 'http://localhost:11434/v1', local: true }, + { kind: 'compat', label: 'LM Studio (local)', baseUrl: 'http://localhost:1234/v1', local: true }, + { kind: 'compat', label: 'Custom OpenAI-compatible endpoint', local: false }, +] + +/** Interactive operations consumed by setup; tests provide scripted answers. */ +export type SetupWizardIO = { + readonly readLine: (prompt: string) => Promise + readonly readSecret: (prompt: string) => Promise + readonly write: (text: string) => void +} + +/** Production setup I/O includes lifecycle cleanup for its readline handle. */ +export type SetupWizardTerminalIO = SetupWizardIO & { readonly close: () => void } + +type SetupWizardDependencies = { + readonly path?: string + readonly save?: (config: CliConfig, path: string) => Promise + readonly catalogIds?: (provider: BuiltinProvider) => readonly string[] + readonly requiredProvider?: ModelProvider +} + +/** Runtime terminal and environment facts used by setup mode selection. */ +export type SetupWizardRuntime = { + readonly stdinIsTty: boolean + readonly stdoutIsTty: boolean + readonly env: Readonly> +} + +/** Reports whether selected provider has an explicit/config key or supported env credential. */ +export const hasUsableCredentials = ( + config: Pick, + env: Readonly>, +): boolean => { + if (config.apiKey) return true + const provider = config.provider ?? DEFAULT_PROVIDER + if (provider === 'openai-compat') return Boolean(env.THUNDERBOLT_OPENAI_COMPAT_KEY) + return BUILTIN_PROVIDER_ENV_VARS[provider].some((name) => Boolean(env[name])) +} + +/** Selects guided setup only for credential-less runs owning stdin and stdout TTYs. */ +export const shouldRunSetupWizard = (config: RunConfig, runtime: SetupWizardRuntime): boolean => + runtime.stdinIsTty && runtime.stdoutIsTty && !hasUsableCredentials(config, runtime.env) + +/** Reads one required value, retrying blank input and surfacing EOF as cancellation. */ +const readRequiredLine = async (io: SetupWizardIO, prompt: string): Promise => { + while (true) { + const answer = await io.readLine(prompt) + if (answer === null) throw new Error('Setup cancelled.') + if (answer.trim() !== '') return answer.trim() + io.write('Value required.\n') + } +} + +/** Resolves one menu choice to its effective provider id. */ +const choiceProvider = (choice: ProviderChoice): ModelProvider => + choice.kind === 'builtin' ? choice.provider : 'openai-compat' + +/** Prints ordered provider menu and reads a valid provider-compatible choice. */ +const chooseProvider = async (io: SetupWizardIO, requiredProvider?: ModelProvider): Promise => { + io.write('Choose a model provider:\n') + PROVIDER_CHOICES.forEach((choice, index) => io.write(` ${index + 1}. ${choice.label}\n`)) + + while (true) { + const answer = await io.readLine(`Provider [1-${PROVIDER_CHOICES.length}]: `) + if (answer === null) throw new Error('Setup cancelled.') + const index = Number(answer.trim()) - 1 + const choice = PROVIDER_CHOICES[index] + if (!Number.isInteger(index) || choice === undefined) { + io.write(`Enter a number from 1 to ${PROVIDER_CHOICES.length}.\n`) + continue + } + if (requiredProvider === undefined || choiceProvider(choice) === requiredProvider) return choice + io.write(`--provider ${requiredProvider} overrides that choice. Select ${requiredProvider}.\n`) + } +} + +/** Reads a hidden key, allowing local servers to use a non-empty placeholder. */ +const readApiKey = async (io: SetupWizardIO, local: boolean): Promise => { + while (true) { + const answer = await io.readSecret(local ? 'API key [local]: ' : 'API key: ') + if (answer === null) throw new Error('Setup cancelled.') + if (answer.trim() !== '') return answer.trim() + if (local) return 'local' + io.write('API key required.\n') + } +} + +/** Builds unique default-first model suggestions from Pi catalog ids. */ +const modelChoices = (provider: BuiltinProvider, catalogIds: readonly string[]): readonly string[] => + [...new Set([DEFAULT_MODELS[provider], ...catalogIds])].slice(0, 4) + +/** Reads a built-in model, accepting Enter for default, number, or free-form id. */ +const chooseBuiltinModel = async ( + io: SetupWizardIO, + provider: BuiltinProvider, + catalogIds: readonly string[], +): Promise => { + const choices = modelChoices(provider, catalogIds) + io.write('Available models:\n') + choices.forEach((model, index) => io.write(` ${index + 1}. ${model}${index === 0 ? ' (default)' : ''}\n`)) + const answer = await io.readLine(`Model [${DEFAULT_MODELS[provider]}]: `) + if (answer === null) throw new Error('Setup cancelled.') + if (answer.trim() === '') return DEFAULT_MODELS[provider] + const numbered = choices[Number(answer.trim()) - 1] + return numbered ?? answer.trim() +} + +/** Resolves selected choice into provider and optional custom base URL. */ +const resolveChoice = async ( + choice: ProviderChoice, + io: SetupWizardIO, +): Promise<{ readonly provider: ModelProvider; readonly baseUrl?: string }> => { + if (choice.kind === 'builtin') return { provider: choice.provider } + if (choice.baseUrl !== undefined) return { provider: 'openai-compat', baseUrl: choice.baseUrl } + return { provider: 'openai-compat', baseUrl: await readRequiredLine(io, 'Base URL: ') } +} + +/** Runs setup, securely persists selected profile, and returns it for immediate reuse. */ +export const runSetupWizard = async ( + io: SetupWizardIO, + dependencies: SetupWizardDependencies = {}, +): Promise => { + const choice = await chooseProvider(io, dependencies.requiredProvider) + const resolved = await resolveChoice(choice, io) + const apiKey = await readApiKey(io, choice.kind === 'compat' && choice.local) + const catalogIds = + dependencies.catalogIds ?? + ((provider) => + builtinModels() + .getModels(provider) + .slice(0, 3) + .map(({ id }) => id)) + const model = + choice.kind === 'builtin' + ? await chooseBuiltinModel(io, choice.provider, catalogIds(choice.provider)) + : await readRequiredLine(io, 'Model id: ') + const config: CliConfig = { + provider: resolved.provider, + model, + apiKey, + ...(resolved.baseUrl === undefined ? {} : { baseUrl: resolved.baseUrl }), + } + const path = dependencies.path ?? configPath() + await (dependencies.save ?? saveConfig)(config, path) + io.write(`Saved config to ${path}.\n`) + return config +} + +/** Creates readline setup I/O while suppressing terminal echo for secret input. */ +export const createSetupWizardIO = ( + input: NodeJS.ReadableStream = process.stdin, + output: NodeJS.WritableStream = process.stdout, +): SetupWizardTerminalIO => { + const state = { muted: false } + const readlineOutput = new Writable({ + write: (chunk, _encoding, callback) => { + if (!state.muted) output.write(chunk) + callback() + }, + }) + const rl = createInterface({ input, output: readlineOutput, terminal: true }) + const eof = new AbortController() + rl.on('close', () => eof.abort()) + + /** Reads one readline answer and maps only expected stream closure to EOF. */ + const question = async (prompt: string, secret: boolean): Promise => { + if (secret) { + output.write(prompt) + state.muted = true + } + try { + const answer = await rl.question(secret ? '' : prompt, { signal: eof.signal }) + return answer.trim() + } catch (error) { + if (error instanceof Error && error.name === 'AbortError') return null + throw error + } finally { + if (secret) { + state.muted = false + output.write('\n') + } + } + } + + return { + readLine: (prompt) => question(prompt, false), + readSecret: (prompt) => question(prompt, true), + write: (text) => { + output.write(text) + }, + close: () => rl.close(), + } +} diff --git a/cli/src/index.ts b/cli/src/index.ts index 23c04432e..722226607 100644 --- a/cli/src/index.ts +++ b/cli/src/index.ts @@ -4,9 +4,8 @@ * file, You can obtain one at http://mozilla.org/MPL/2.0/. */ /** - * Binary entrypoint. Parses argv, dispatches the terminal info actions - * (`help`/`version`/`error`) inline, and runs the agent inside the single - * top-level try/catch that turns any uncaught failure into a clean exit. + * Binary entrypoint. Loads persisted defaults, parses argv, dispatches commands, + * and turns uncaught failures into clean terminal errors at one boundary. */ import { HELP_TEXT, VERSION, parseArgs } from './cli.ts' @@ -16,59 +15,76 @@ import { runBridge } from './commands/bridge.ts' import { runIrohBridge } from './iroh/bridge.ts' import { runIrohConnect } from './iroh/connect.ts' import { runIrohAdmin } from './iroh/admin.ts' +import { loadConfig } from './config/config.ts' +import type { CliConfig } from './config/config.ts' +import { createSetupWizardIO, runSetupWizard, shouldRunSetupWizard } from './config/wizard.ts' +import type { ModelProvider } from './agent/types.ts' -const parsed = parseArgs(Bun.argv.slice(2)) +/** Runs interactive setup with production terminal I/O and guaranteed cleanup. */ +const configure = async (requiredProvider?: ModelProvider): Promise => { + if (!process.stdin.isTTY || !process.stdout.isTTY) { + throw new Error('config requires stdin and stdout to be terminals') + } + const io = createSetupWizardIO() + try { + return await runSetupWizard(io, { requiredProvider }) + } finally { + io.close() + } +} -switch (parsed.kind) { - case 'help': - console.log(HELP_TEXT) - break - case 'version': - console.log(VERSION) - break - case 'error': - process.stderr.write(parsed.message + '\n') - process.exitCode = 1 - break - case 'run': - try { - await runAgent(parsed.config) - } catch (err) { - process.stderr.write(`thunderbolt: ${err instanceof Error ? err.message : String(err)}\n`) +try { + const argv = Bun.argv.slice(2) + const storedConfig = await loadConfig() + const parsed = parseArgs(argv, { config: storedConfig }) + + switch (parsed.kind) { + case 'help': + console.log(HELP_TEXT) + break + case 'version': + console.log(VERSION) + break + case 'error': + process.stderr.write(parsed.message + '\n') process.exitCode = 1 + break + case 'config': + await configure() + break + case 'run': { + if ( + shouldRunSetupWizard(parsed.config, { + stdinIsTty: Boolean(process.stdin.isTTY), + stdoutIsTty: Boolean(process.stdout.isTTY), + env: process.env, + }) + ) { + const requiredProvider = argv.includes('--provider') ? parsed.config.provider : undefined + const freshConfig = await configure(requiredProvider) + const reparsed = parseArgs(argv, { config: freshConfig }) + if (reparsed.kind !== 'run') throw new Error('failed to resume agent after setup') + await runAgent(reparsed.config) + break + } + await runAgent(parsed.config) + break } - break - case 'bridge': - try { + case 'bridge': if (parsed.config.transport === 'iroh') await runIrohBridge(parsed.config) else await runBridge(parsed.config) - } catch (err) { - process.stderr.write(`thunderbolt: ${err instanceof Error ? err.message : String(err)}\n`) - process.exitCode = 1 - } - break - case 'connect': - try { + break + case 'connect': await runIrohConnect(parsed.config) - } catch (err) { - process.stderr.write(`thunderbolt: ${err instanceof Error ? err.message : String(err)}\n`) - process.exitCode = 1 - } - break - case 'acp-serve': - try { + break + case 'acp-serve': await runAcpServe(parsed.config) - } catch (err) { - process.stderr.write(`thunderbolt: ${err instanceof Error ? err.message : String(err)}\n`) - process.exitCode = 1 - } - break - case 'iroh-admin': - try { + break + case 'iroh-admin': await runIrohAdmin(parsed.action) - } catch (err) { - process.stderr.write(`thunderbolt: ${err instanceof Error ? err.message : String(err)}\n`) - process.exitCode = 1 - } - break + break + } +} catch (error) { + process.stderr.write(`thunderbolt: ${error instanceof Error ? error.message : String(error)}\n`) + process.exitCode = 1 } diff --git a/cli/src/iroh/paths.ts b/cli/src/iroh/paths.ts index c2b54dcaf..de0a4b8a5 100644 --- a/cli/src/iroh/paths.ts +++ b/cli/src/iroh/paths.ts @@ -10,13 +10,9 @@ * the allowlist gate rejects an unknown peer). */ -import { homedir } from 'node:os' import { join } from 'node:path' import type { BridgeProtocol } from '../agent/types.ts' - -/** Root for all thunderbolt CLI state. `THUNDERBOLT_HOME` overrides the default - * `~/.thunderbolt`, enabling isolated identities for testing/multi-account. */ -export const thunderboltHomeDir = (): string => process.env.THUNDERBOLT_HOME ?? join(homedir(), '.thunderbolt') +import { thunderboltHomeDir } from '../paths.ts' /** Directory holding the iroh identity and allowlist. */ export const irohDir = (): string => join(thunderboltHomeDir(), 'iroh') diff --git a/cli/src/paths.ts b/cli/src/paths.ts new file mode 100644 index 000000000..6eb1e53de --- /dev/null +++ b/cli/src/paths.ts @@ -0,0 +1,19 @@ +/* This Source Code Form is subject to the terms of the Mozilla Public + * License, v. 2.0. If a copy of the MPL was not distributed with this + * file, You can obtain one at http://mozilla.org/MPL/2.0/. */ + +/** Shared filesystem locations for persistent thunderbolt CLI state. */ + +import { homedir } from 'node:os' +import { join } from 'node:path' + +/** Root for all thunderbolt CLI state. `THUNDERBOLT_HOME` overrides the default + * `~/.thunderbolt`, enabling isolated identities for testing/multi-account. */ +export const thunderboltHomeDir = ( + env: Readonly> = process.env, + home: string = homedir(), +): string => env.THUNDERBOLT_HOME ?? join(home, '.thunderbolt') + +/** Resolves persisted CLI config path. */ +export const configPath = (env: Readonly> = process.env): string => + join(thunderboltHomeDir(env), 'config.json') From 8dfc3d8072fcb6219f6aa4de54acee876b3a423f Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C3=8Dtalo=20Menezes?= Date: Thu, 9 Jul 2026 21:45:39 -0300 Subject: [PATCH 02/12] feat: add self-hosted iroh relay deployment config --- deploy/iroh-relay/config.example.toml | 42 +++++++++++++++++++++++++++ deploy/iroh-relay/docker-compose.yml | 22 ++++++++++++++ 2 files changed, 64 insertions(+) create mode 100644 deploy/iroh-relay/config.example.toml create mode 100644 deploy/iroh-relay/docker-compose.yml diff --git a/deploy/iroh-relay/config.example.toml b/deploy/iroh-relay/config.example.toml new file mode 100644 index 000000000..4ccdb0a96 --- /dev/null +++ b/deploy/iroh-relay/config.example.toml @@ -0,0 +1,42 @@ +# Production config for a self-hosted iroh relay (iroh-relay v1.0.x). +# Mount into the container and run WITHOUT `--dev`: +# +# docker run -d -p 80:80 -p 443:443 -p 4433:4433/udp \ +# -v ./config.toml:/config.toml -v ./certs:/certs \ +# n0computer/iroh-relay:v1.0.2 --config-path /config.toml +# +# Field reference: https://github.com/n0-computer/iroh/tree/main/iroh-relay +# Walkthrough + rationale: docs/architecture/iroh-relay-self-hosting.md + +# Relay traffic proxying (the actual job of this server). +enable_relay = true + +# Port 80 serves the captive-portal page and the ACME HTTP-01 challenge. +http_bind_addr = "[::]:80" + +# QUIC address discovery lets clients learn their public IP/port (the role STUN +# played pre-1.0). Requires TLS. UDP 4433 must be reachable — skip on hosts +# that can't expose UDP (relaying still works; discovery falls back). +enable_quic_addr_discovery = true + +[tls] +https_bind_addr = "[::]:443" +quic_bind_addr = "[::]:4433" +hostname = "relay.thunderbolt.example" # <- real domain pointing at this host +cert_mode = "LetsEncrypt" +contact = "ops@thunderbolt.example" # <- ACME contact email +cert_dir = "/certs" + +# Without this section the relay is open like the n0 public relays. A shared +# token gates it to our own clients: the CLI/app would append `?token=…` to the +# relay URL. Start open for the pilot; lock down before announcing the URL. +# [access] +# shared_token = [""] # or env IROH_RELAY_ACCESS_TOKEN + +# Per-client receive-rate limiting — sane flood protection for a public host. +[limits.client.rx] +bytes_per_second = 1048576 # 1 MiB/s steady +max_burst_bytes = 4194304 # 4 MiB burst + +# Prometheus metrics on :9090 (keep firewalled / internal-only). +enable_metrics = true diff --git a/deploy/iroh-relay/docker-compose.yml b/deploy/iroh-relay/docker-compose.yml new file mode 100644 index 000000000..4ca7a9083 --- /dev/null +++ b/deploy/iroh-relay/docker-compose.yml @@ -0,0 +1,22 @@ +# Self-hosted iroh relay — local development / PoC. +# +# Runs the official n0 relay in `--dev` mode: plain HTTP (no TLS) on :3340, +# which is exactly what the CLI/app accept for a localhost relay URL. +# +# docker compose up -d +# +# Point the two sides at it: +# CLI : THUNDERBOLT_IROH_RELAY_URL=http://localhost:3340 +# App : VITE_IROH_RELAY_URL=http://localhost:3340 (build-time) +# +# Production runs the same image WITHOUT `--dev`, with a config file mounted +# (see config.example.toml) and real TLS on 443. Full walkthrough: +# docs/architecture/iroh-relay-self-hosting.md +services: + iroh-relay: + image: n0computer/iroh-relay:v1.0.2 + container_name: thunderbolt-iroh-relay + command: ['--dev'] + ports: + - '3340:3340' + restart: unless-stopped From 05e6c6c83314071af78c61961adf58f31406ad7d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C3=8Dtalo=20Menezes?= Date: Thu, 9 Jul 2026 21:45:44 -0300 Subject: [PATCH 03/12] docs: explain self-hosting the iroh relay --- docs/architecture/iroh-relay-self-hosting.md | 101 +++++++++++++++++++ 1 file changed, 101 insertions(+) create mode 100644 docs/architecture/iroh-relay-self-hosting.md diff --git a/docs/architecture/iroh-relay-self-hosting.md b/docs/architecture/iroh-relay-self-hosting.md new file mode 100644 index 000000000..cc485ddb6 --- /dev/null +++ b/docs/architecture/iroh-relay-self-hosting.md @@ -0,0 +1,101 @@ +# Self-hosting the iroh relay + +The CLI↔app bridge (`thunderbolt acp|mcp --transport iroh`) runs on [iroh](https://iroh.computer), +which by default rides on relay servers operated by n0 (iroh's authors). This doc explains what a +relay actually does, what we self-host versus what still comes from n0, the local PoC, and the +production plan. + +## What a relay does (and doesn't) + +An iroh connection is QUIC between two endpoints identified by ed25519 keys (NodeIds). Two peers +behind NATs usually can't open a direct UDP path immediately, so every endpoint keeps one +long-lived connection to a **home relay** — an HTTPS server that: + +1. **Forwards encrypted packets** between peers when no direct path exists (launch traffic, or + permanently when hole-punching fails — e.g. the browser, which can't do UDP at all and is + relay-only by design, see `crates/thunderbolt-acp-client`). +2. **Assists hole-punching**: peers exchange candidate addresses through the relay, then attempt a + direct QUIC path and migrate to it when it works. +3. **Names the meeting point**: a connection ticket embeds the NodeId + home-relay URL, so a dialer + knows where to find the peer without any discovery. + +What a relay can NOT do: read traffic. Everything it forwards is QUIC end-to-end encrypted to the +peer's NodeId — a hostile relay can drop or delay packets and observe metadata (who talks to whom, +when, how much), but never content. Self-hosting is therefore about **availability and metadata**, +not about payload confidentiality (which we already have). + +## Why self-host + +- **Availability**: the n0 public relays are a free best-effort service with per-client rate limits; + they can throttle, change, or disappear. Product traffic needs infrastructure with our SLOs. +- **Metadata privacy**: connection graph + traffic timing of our users stays with us. +- **Control**: access control (token-gate the relay to our clients), rate limits, metrics. + +What still comes from n0 after the swap: **DNS discovery** (`presetN0` keeps it). Dialing by *bare +NodeId* resolves the peer's current relay via n0's DNS service; dialing by *ticket* doesn't need it +(the relay URL is in the ticket). Our flows exchange tickets, so n0 DNS is a convenience, not a +dependency — replacing it (iroh supports custom DNS/pkarr discovery) is a possible follow-up, +documented here so nobody thinks the relay swap removed every third party. + +## The plumbing already in the code + +Both sides accept a relay override; unset keeps today's n0 default: + +| Side | Variable | Where it lands | +| --- | --- | --- | +| CLI (Bun) | `THUNDERBOLT_IROH_RELAY_URL` (runtime) | `cli/src/iroh/endpoint.ts` `configureTransport` — swaps ONLY the relay in the n0 preset | +| Web app (wasm) | `VITE_IROH_RELAY_URL` (build-time) | `src/acp/iroh/iroh-transport.ts` → `crates/thunderbolt-acp-client` relay-only endpoint | + +## Local PoC (validated) + +```sh +cd deploy/iroh-relay && docker compose up -d # n0computer/iroh-relay:v1.0.2 --dev → plain HTTP :3340 +``` + +Validation run (2026-07-09, iroh 1.x both sides): + +1. `THUNDERBOLT_IROH_RELAY_URL=http://localhost:3340 thunderbolt iroh id` → the printed ticket + decodes to `http://localhost:3340/` as the home relay (no n0 URL present), and `endpoint.online()` + only resolves once the home relay accepted us — proof the override is fully in effect. +2. Full round-trip through the bridge: identity A ran + `thunderbolt acp --transport iroh -- cat`, identity B was allowlisted and dialed A's ticket with + `thunderbolt acp connect `; a JSON-RPC line piped into B came back byte-identical through + A's `cat`. Same-host peers may migrate to a direct path after the relay-mediated handshake — + which is exactly the intended behavior in production too. + +`--dev` mode = plain HTTP, no TLS, no QUIC address discovery. Fine on localhost, never on the +internet. + +## Production plan (decision pending) + +The relay is a single small Rust binary (the public n0 relays run on modest VMs; CPU cost is +forwarding ciphertext). What it needs from a host: + +- A **domain** (e.g. `relay.thunderbolt.io`) — tickets and configs carry this URL. +- **TCP 80 + 443** terminated by the relay itself (`cert_mode = "LetsEncrypt"` auto-provisions); + port 80 also serves the ACME challenge. +- **UDP 4433** (optional) for QUIC address discovery — the 1.x replacement for STUN. Without it, + relaying still works; peers just lose one address-discovery mechanism. +- Prometheus metrics on :9090 (internal only). + +Two candidate shapes, in order of fit: + +1. **Small VPS (Hetzner/Fly/EC2)** — full fit: raw 80/443 + UDP, LetsEncrypt built in, ~$5–10/mo. + `config.example.toml` in `deploy/iroh-relay/` is this shape. +2. **Render (current infra)** — partial fit: web services proxy HTTP/TLS (the relay's WebSocket + upgrade works), but no UDP and no self-terminated TLS, so no QUIC address discovery and + `LetsEncrypt` mode is out (Render terminates TLS; the relay would run `dangerous_http_only` + behind it — needs a validation spike before committing). + +Rollout: stand up the relay → bake the URL as the default in the CLI (still overridable by env) and +`VITE_IROH_RELAY_URL` in app builds → mixed fleets keep working because tickets carry the relay URL +of whoever minted them. Lock down with `access.shared_token` once our clients send it. + +## Operational notes + +- **Version coupling**: the relay speaks the iroh relay protocol; keep the server minor-version + aligned with the `iroh`/`@number0/iroh` 1.x clients when bumping either. +- The container logs nothing per-connection at default log level; use `RUST_LOG=info` (or metrics) + when debugging. +- Access control modes: `everyone` (default), `allowlist`/`denylist` (NodeIds), `shared_token`, + or an `http` callback — see `config.example.toml`. From 0c2a82150ba83642c924e52307cc9dcda8087a96 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C3=8Dtalo=20Menezes?= Date: Tue, 14 Jul 2026 09:58:52 -0300 Subject: [PATCH 04/12] fix: scope the saved model to its provider like the sibling resolvers --- cli/src/cli.test.ts | 1 + cli/src/cli.ts | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/cli/src/cli.test.ts b/cli/src/cli.test.ts index 178e3d7ee..76c3234d3 100644 --- a/cli/src/cli.test.ts +++ b/cli/src/cli.test.ts @@ -237,6 +237,7 @@ describe('parseArgs — persisted config precedence', () => { test('saved key and base URL do not cross provider boundaries', () => { const config = runConfig(['--provider', 'anthropic'], { config: stored, env: {} }) + expect(config.model).toBe('claude-opus-4-8') expect(config.apiKey).toBeUndefined() expect(config.baseUrl).toBeUndefined() }) diff --git a/cli/src/cli.ts b/cli/src/cli.ts index 3e118a9fc..d952b145c 100644 --- a/cli/src/cli.ts +++ b/cli/src/cli.ts @@ -363,7 +363,7 @@ const resolveApiKey = ( /** Resolves omitted `--model` against selected provider's catalog default. */ const resolveModelId = (flags: Flags, provider: ModelProvider, config: CliConfig | null): string => { if (flags.model !== undefined) return flags.model - if (config !== null) return config.model + if (config?.provider === provider) return config.model return provider === 'openai-compat' ? DEFAULT_MODEL : DEFAULT_MODELS[provider] } From cc308aa5c52702b1c72c6324b0d7a5c9f218d29d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C3=8Dtalo=20Menezes?= Date: Wed, 15 Jul 2026 13:22:08 -0300 Subject: [PATCH 05/12] feat: fetch wizard model suggestions live from each provider --- cli/src/config/model-listing.test.ts | 207 +++++++++++++++++++++++++++ cli/src/config/model-listing.ts | 185 ++++++++++++++++++++++++ cli/src/config/wizard.test.ts | 117 ++++++++++++++- cli/src/config/wizard.ts | 95 +++++++++--- 4 files changed, 581 insertions(+), 23 deletions(-) create mode 100644 cli/src/config/model-listing.test.ts create mode 100644 cli/src/config/model-listing.ts diff --git a/cli/src/config/model-listing.test.ts b/cli/src/config/model-listing.test.ts new file mode 100644 index 000000000..c5b513867 --- /dev/null +++ b/cli/src/config/model-listing.test.ts @@ -0,0 +1,207 @@ +/* This Source Code Form is subject to the terms of the Mozilla Public + * License, v. 2.0. If a copy of the MPL was not distributed with this + * file, You can obtain one at http://mozilla.org/MPL/2.0/. */ + +import { describe, expect, test } from 'bun:test' +import { listModels } from './model-listing.ts' +import type { ModelListingFetch } from './model-listing.ts' + +describe('listModels', () => { + test('reads an OpenAI-compatible model list with bearer authentication', async () => { + const requests: { readonly input: string | URL | Request; readonly init?: RequestInit }[] = [] + const fetchFn: ModelListingFetch = async (input, init) => { + requests.push({ input, init }) + return Response.json({ + object: 'list', + data: [ + { id: 'gpt-live-a', object: 'model' }, + { id: 'gpt-live-b', object: 'model' }, + ], + }) + } + + const result = await listModels({ provider: 'openai', apiKey: 'secret-key', fetchFn }) + + expect(result).toEqual({ source: 'live', ids: ['gpt-live-a', 'gpt-live-b'] }) + expect(String(requests[0]?.input)).toBe('https://api.openai.com/v1/models') + expect(new Headers(requests[0]?.init?.headers).get('Authorization')).toBe('Bearer secret-key') + }) + + test('reads Anthropic models with Anthropic authentication headers', async () => { + const requests: { readonly input: string | URL | Request; readonly init?: RequestInit }[] = [] + const fetchFn: ModelListingFetch = async (input, init) => { + requests.push({ input, init }) + return Response.json({ + data: [ + { id: 'claude-live-b', type: 'model', created_at: '2026-06-01T00:00:00Z' }, + { id: 'claude-live-a', type: 'model', created_at: '2026-07-01T00:00:00Z' }, + ], + }) + } + + const result = await listModels({ provider: 'anthropic', apiKey: 'anthropic-key', fetchFn }) + + expect(result).toEqual({ source: 'live', ids: ['claude-live-a', 'claude-live-b'] }) + expect(String(requests[0]?.input)).toBe('https://api.anthropic.com/v1/models') + const headers = new Headers(requests[0]?.init?.headers) + expect(headers.get('x-api-key')).toBe('anthropic-key') + expect(headers.get('anthropic-version')).toBe('2023-06-01') + expect(headers.has('Authorization')).toBe(false) + }) + + test('reads only Gemini models supporting generateContent and strips their prefix', async () => { + const requests: { readonly input: string | URL | Request; readonly init?: RequestInit }[] = [] + const fetchFn: ModelListingFetch = async (input, init) => { + requests.push({ input, init }) + return Response.json({ + models: [ + { name: 'models/gemini-live-chat', supportedGenerationMethods: ['generateContent', 'countTokens'] }, + { name: 'models/text-embedding-004', supportedGenerationMethods: ['embedContent'] }, + { name: 'models/gemini-no-chat', supportedGenerationMethods: ['countTokens'] }, + ], + }) + } + + const result = await listModels({ provider: 'google', apiKey: 'gemini-key', fetchFn }) + + expect(result).toEqual({ source: 'live', ids: ['gemini-live-chat'] }) + expect(String(requests[0]?.input)).toBe( + 'https://generativelanguage.googleapis.com/v1beta/models?key=gemini-key', + ) + expect(new Headers(requests[0]?.init?.headers).has('Authorization')).toBe(false) + }) + + test('filters non-chat model id patterns from compatible responses', async () => { + const fetchFn: ModelListingFetch = async () => + Response.json({ + data: [ + { id: 'chat-model' }, + { id: 'text-embedding-3-large' }, + { id: 'whisper-large-v3' }, + { id: 'gpt-4o-mini-tts' }, + { id: 'dall-e-3' }, + { id: 'gpt-image-1' }, + { id: 'sora-2' }, + { id: 'omni-moderation-latest' }, + { id: 'cohere-rerank-v3' }, + ], + }) + + expect(await listModels({ provider: 'openrouter', apiKey: 'key', fetchFn })).toEqual({ + source: 'live', + ids: ['chat-model'], + }) + }) + + test('sorts by created descending and limits live suggestions to eight', async () => { + const fetchFn: ModelListingFetch = async () => + Response.json({ + data: Array.from({ length: 10 }, (_, index) => ({ id: `model-${index}`, created: index })), + }) + + expect(await listModels({ provider: 'groq', apiKey: 'key', fetchFn })).toEqual({ + source: 'live', + ids: ['model-9', 'model-8', 'model-7', 'model-6', 'model-5', 'model-4', 'model-3', 'model-2'], + }) + }) + + test('derives compatible listing routes from Pi descriptors', async () => { + const urls: string[] = [] + const fetchFn: ModelListingFetch = async (input) => { + urls.push(String(input)) + return Response.json({ data: [{ id: 'chat-model' }] }) + } + + for (const provider of [ + 'xai', + 'deepseek', + 'zai', + 'mistral', + 'groq', + 'openrouter', + 'moonshotai', + 'minimax', + 'cerebras', + 'together', + 'fireworks', + ] as const) { + await listModels({ provider, apiKey: 'key', fetchFn }) + } + + expect(urls).toEqual([ + 'https://api.x.ai/v1/models', + 'https://api.deepseek.com/models', + 'https://api.z.ai/api/coding/paas/v4/models', + 'https://api.mistral.ai/v1/models', + 'https://api.groq.com/openai/v1/models', + 'https://openrouter.ai/api/v1/models', + 'https://api.moonshot.ai/v1/models', + 'https://api.minimax.io/v1/models', + 'https://api.cerebras.ai/v1/models', + 'https://api.together.ai/v1/models', + 'https://api.fireworks.ai/inference/v1/models', + ]) + }) + + test('returns catalog models on timeout even when injected fetch ignores abort', async () => { + const fetchFn: ModelListingFetch = async () => new Promise(() => {}) + + expect(await listModels({ provider: 'openai', apiKey: 'key', fetchFn, timeoutMs: 1 })).toEqual({ + source: 'catalog', + ids: ['gpt-4', 'gpt-4-turbo', 'gpt-4.1'], + }) + }) + + test('returns catalog models for non-success and malformed responses', async () => { + const unavailable = await listModels({ + provider: 'openai', + apiKey: 'key', + fetchFn: async () => new Response('unavailable', { status: 503 }), + }) + const malformed = await listModels({ + provider: 'openai', + apiKey: 'key', + fetchFn: async () => Response.json({ unexpected: [] }), + }) + + expect(unavailable).toEqual({ source: 'catalog', ids: ['gpt-4', 'gpt-4-turbo', 'gpt-4.1'] }) + expect(malformed).toEqual({ source: 'catalog', ids: ['gpt-4', 'gpt-4-turbo', 'gpt-4.1'] }) + }) + + test('marks 401 and 403 catalog fallbacks as authentication rejections', async () => { + const unauthorized = await listModels({ + provider: 'openai', + apiKey: 'bad-key', + fetchFn: async () => new Response(null, { status: 401 }), + }) + const forbidden = await listModels({ + provider: 'openai', + apiKey: 'bad-key', + fetchFn: async () => new Response(null, { status: 403 }), + }) + + expect(unauthorized).toEqual({ + source: 'catalog', + ids: ['gpt-4', 'gpt-4-turbo', 'gpt-4.1'], + authRejected: true, + status: 401, + }) + expect(forbidden).toEqual({ + source: 'catalog', + ids: ['gpt-4', 'gpt-4-turbo', 'gpt-4.1'], + authRejected: true, + status: 403, + }) + }) + + test('treats an empty chat-capable result as catalog fallback', async () => { + const result = await listModels({ + provider: 'openai-compat', + baseUrl: 'http://localhost:11434/v1', + apiKey: 'local', + fetchFn: async () => Response.json({ data: [{ id: 'nomic-embed-text' }] }), + }) + + expect(result).toEqual({ source: 'catalog', ids: [] }) + }) +}) diff --git a/cli/src/config/model-listing.ts b/cli/src/config/model-listing.ts new file mode 100644 index 000000000..e6474bb9a --- /dev/null +++ b/cli/src/config/model-listing.ts @@ -0,0 +1,185 @@ +/* This Source Code Form is subject to the terms of the Mozilla Public + * License, v. 2.0. If a copy of the MPL was not distributed with this + * file, You can obtain one at http://mozilla.org/MPL/2.0/. */ + +/** Live provider model discovery with Pi catalog fallback. */ + +import { builtinModels, builtinProviders } from '@earendil-works/pi-ai/providers/all' +import type { ModelProvider } from '../agent/types.ts' + +const DEFAULT_TIMEOUT_MS = 3_000 +const MAX_LIVE_MODELS = 8 +const NON_CHAT_MODEL_PATTERN = + /embed(?:ding)?|whisper|tts|speech|transcri|dall-?e|gpt-image|imagen|(?:^|[-_/])sora(?:[-_/]|$)|moderation|rerank/i + +export type ModelListingResult = { + readonly source: 'live' | 'catalog' + readonly ids: readonly string[] + readonly authRejected?: true + readonly status?: 401 | 403 +} + +export type ModelListingFetch = (input: string | URL | Request, init?: RequestInit) => Promise + +export type ListModelsOptions = { + readonly provider: ModelProvider + readonly apiKey: string + readonly baseUrl?: string + readonly fetchFn?: ModelListingFetch + readonly timeoutMs?: number +} + +type ListedModel = { + readonly id: string + readonly created?: number +} + +/** Reads numeric OpenAI or ISO Anthropic creation metadata. */ +const createdTimestamp = (candidate: Readonly>): number | undefined => { + if (typeof candidate.created === 'number') return candidate.created + if (typeof candidate.created_at !== 'string') return undefined + const timestamp = Date.parse(candidate.created_at) + return Number.isNaN(timestamp) ? undefined : timestamp +} + +/** Narrows parsed JSON objects without weakening unknown input types. */ +const isRecord = (value: unknown): value is Record => + typeof value === 'object' && value !== null && !Array.isArray(value) + +/** Reads OpenAI-compatible `{ data: [{ id, created? }] }` responses. */ +const parseOpenAiModels = (value: unknown): readonly ListedModel[] => { + if (!isRecord(value) || !Array.isArray(value.data)) throw new Error('Invalid model list response.') + return value.data.map((candidate) => { + if (!isRecord(candidate) || typeof candidate.id !== 'string') throw new Error('Invalid model entry.') + const created = createdTimestamp(candidate) + return { + id: candidate.id, + ...(created === undefined ? {} : { created }), + } + }) +} + +/** Reads Gemini model entries capable of `generateContent`. */ +const parseGeminiModels = (value: unknown): readonly ListedModel[] => { + if (!isRecord(value) || !Array.isArray(value.models)) throw new Error('Invalid Gemini model list response.') + return value.models.flatMap((candidate) => { + if (!isRecord(candidate) || typeof candidate.name !== 'string') throw new Error('Invalid Gemini model entry.') + if (!Array.isArray(candidate.supportedGenerationMethods)) throw new Error('Invalid Gemini methods.') + if (!candidate.supportedGenerationMethods.includes('generateContent')) return [] + return [{ id: candidate.name.replace(/^models\//, '') }] + }) +} + +/** Keeps model ids intended for chat rather than specialized media or ranking APIs. */ +const chatModels = (models: readonly ListedModel[]): readonly ListedModel[] => + models.filter(({ id }) => !NON_CHAT_MODEL_PATTERN.test(id)) + +/** Sorts newest-first when creation metadata exists, preserving order otherwise. */ +const newestModelsFirst = (models: readonly ListedModel[]): readonly ListedModel[] => { + if (!models.some(({ created }) => created !== undefined)) return models + return [...models].sort( + (left, right) => + (right.created ?? Number.NEGATIVE_INFINITY) - (left.created ?? Number.NEGATIVE_INFINITY), + ) +} + +/** Returns current Pi catalog ids using setup wizard's existing three-item limit. */ +const catalogIds = (provider: ModelProvider): readonly string[] => { + if (provider === 'openai-compat') return [] + try { + return builtinModels() + .getModels(provider) + .slice(0, 3) + .map(({ id }) => id) + } catch { + return [] + } +} + +/** Resolves provider base URL from Pi descriptors, except caller-owned custom targets. */ +const providerBaseUrl = (provider: ModelProvider, customBaseUrl?: string): string => { + if (provider === 'openai-compat') { + if (!customBaseUrl) throw new Error('Missing OpenAI-compatible base URL.') + return customBaseUrl + } + const descriptor = builtinProviders().find(({ id }) => id === provider) + if (!descriptor?.baseUrl) throw new Error(`Missing Pi base URL for ${provider}.`) + return descriptor.baseUrl +} + +/** Joins one endpoint path without duplicating a trailing slash. */ +const endpoint = (baseUrl: string, path: string): string => `${baseUrl.replace(/\/$/, '')}/${path}` + +/** Adapts Pi inference bases to providers' OpenAI-compatible model-listing bases. */ +const compatibleBaseUrl = (provider: ModelProvider, baseUrl?: string): string => { + const resolvedBaseUrl = providerBaseUrl(provider, baseUrl) + if (provider === 'mistral') return endpoint(resolvedBaseUrl, 'v1') + if (provider === 'minimax') return new URL('/v1', resolvedBaseUrl).toString() + if (provider !== 'fireworks') return resolvedBaseUrl + + const openAiModel = builtinModels() + .getModels(provider) + .find(({ api }) => api === 'openai-completions') + return openAiModel?.baseUrl ?? resolvedBaseUrl +} + +/** Builds provider-specific listing URL and authentication headers. */ +const listingRequest = ( + provider: ModelProvider, + apiKey: string, + baseUrl?: string, +): { readonly url: string; readonly headers: Readonly> } => { + if (provider === 'anthropic') { + return { + url: endpoint(providerBaseUrl(provider), 'v1/models'), + headers: { 'x-api-key': apiKey, 'anthropic-version': '2023-06-01' }, + } + } + if (provider === 'google') { + const url = new URL(endpoint(providerBaseUrl(provider), 'models')) + url.searchParams.set('key', apiKey) + return { url: url.toString(), headers: {} } + } + return { + url: endpoint(compatibleBaseUrl(provider, baseUrl), 'models'), + headers: { Authorization: `Bearer ${apiKey}` }, + } +} + +/** Lists live provider models, returning Pi catalog ids for every failure mode. */ +export const listModels = async (options: ListModelsOptions): Promise => { + const fallback = (): ModelListingResult => ({ source: 'catalog', ids: catalogIds(options.provider) }) + const controller = new AbortController() + const timeout = Promise.withResolvers() + const timeoutId = setTimeout(() => { + controller.abort() + timeout.reject(new DOMException('Model listing timed out.', 'TimeoutError')) + }, options.timeoutMs ?? DEFAULT_TIMEOUT_MS) + + try { + const request = listingRequest(options.provider, options.apiKey, options.baseUrl) + const response = await Promise.race([ + (options.fetchFn ?? globalThis.fetch)(request.url, { + headers: request.headers, + signal: controller.signal, + }), + timeout.promise, + ]) + if (!response.ok) { + if (response.status === 401 || response.status === 403) { + return { ...fallback(), authRejected: true, status: response.status } + } + return fallback() + } + const parsed = (await response.json()) as unknown + const models = newestModelsFirst( + chatModels(options.provider === 'google' ? parseGeminiModels(parsed) : parseOpenAiModels(parsed)), + ) + if (models.length === 0) return fallback() + return { source: 'live', ids: models.slice(0, MAX_LIVE_MODELS).map(({ id }) => id) } + } catch { + return fallback() + } finally { + clearTimeout(timeoutId) + } +} diff --git a/cli/src/config/wizard.test.ts b/cli/src/config/wizard.test.ts index 946cf0e13..8780f995a 100644 --- a/cli/src/config/wizard.test.ts +++ b/cli/src/config/wizard.test.ts @@ -6,6 +6,7 @@ import { describe, expect, test } from 'bun:test' import type { RunConfig } from '../agent/types.ts' import { parseArgs } from '../cli.ts' import type { CliConfig } from './config.ts' +import type { ModelListingFetch } from './model-listing.ts' import { runSetupWizard, shouldRunSetupWizard } from './wizard.ts' import type { SetupWizardIO } from './wizard.ts' @@ -44,6 +45,13 @@ const scriptedIO = (lines: readonly string[], secrets: readonly string[]) => { return { io, output } } +/** Returns one injected OpenAI-compatible model-list response. */ +const liveModels = (...ids: readonly string[]): ModelListingFetch => async () => + Response.json({ data: ids.map((id) => ({ id })) }) + +/** Simulates provider unavailability without touching ambient network state. */ +const unavailableModels: ModelListingFetch = async () => new Response(null, { status: 503 }) + describe('shouldRunSetupWizard', () => { test('uses setup for a credential-less run when stdin and stdout are TTYs', () => { expect(shouldRunSetupWizard(runConfig(), runtime())).toBe(true) @@ -83,14 +91,14 @@ describe('runSetupWizard', () => { save: async (config) => { saved.push(config) }, - catalogIds: () => ['gpt-catalog-a', 'gpt-catalog-b'], + fetchFn: liveModels('gpt-live-a', 'gpt-live-b'), }) const expected: CliConfig = { provider: 'openai', model: 'gpt-5.3-codex', apiKey: 'sk-openai' } expect(result).toEqual(expected) expect(saved).toEqual([expected]) expect(output.join('')).toContain('2. OpenAI') - expect(output.join('')).toContain('gpt-catalog-a') + expect(output.join('')).toContain('gpt-live-a') expect(output.join('')).toContain('Saved config to /tmp/thunderbolt/config.json') }) @@ -103,7 +111,7 @@ describe('runSetupWizard', () => { save: async (config) => { saved.push(config) }, - catalogIds: () => [], + fetchFn: unavailableModels, }) expect(saved).toEqual([ @@ -125,7 +133,7 @@ describe('runSetupWizard', () => { save: async (config) => { saved.push(config) }, - catalogIds: () => [], + fetchFn: unavailableModels, }) expect(saved).toEqual([ @@ -138,6 +146,30 @@ describe('runSetupWizard', () => { ]) }) + test('lists and selects numbered models from the entered compatible base URL', async () => { + const { io, output } = scriptedIO(['17', 'https://custom.example/v1/', '2'], ['custom-secret']) + const urls: string[] = [] + const fetchFn: ModelListingFetch = async (input) => { + urls.push(String(input)) + return Response.json({ data: [{ id: 'custom-live-a' }, { id: 'custom-live-b' }] }) + } + + const result = await runSetupWizard(io, { + path: '/tmp/config.json', + save: async () => {}, + fetchFn, + }) + + expect(urls).toEqual(['https://custom.example/v1/models']) + expect(result).toEqual({ + provider: 'openai-compat', + model: 'custom-live-b', + apiKey: 'custom-secret', + baseUrl: 'https://custom.example/v1/', + }) + expect(output.join('')).not.toContain('custom-secret') + }) + test('re-prompts when a choice conflicts with an explicit provider flag', async () => { const { io, output } = scriptedIO(['2', '3', ''], ['google-key']) const saved: CliConfig[] = [] @@ -148,7 +180,10 @@ describe('runSetupWizard', () => { save: async (config) => { saved.push(config) }, - catalogIds: () => [], + fetchFn: async () => + Response.json({ + models: [{ name: 'models/gemini-3.1-pro-preview', supportedGenerationMethods: ['generateContent'] }], + }), }) expect(saved).toEqual([{ provider: 'google', model: 'gemini-3.1-pro-preview', apiKey: 'google-key' }]) @@ -158,4 +193,76 @@ describe('runSetupWizard', () => { expect(reparsed.config.apiKey).toBe('google-key') expect(reparsed.config.model).toBe('gemini-3.1-pro-preview') }) + + test('selects a numbered live model through the full scripted wizard flow', async () => { + const { io } = scriptedIO(['2', '3'], ['sk-live']) + + const result = await runSetupWizard(io, { + path: '/tmp/config.json', + save: async () => {}, + fetchFn: liveModels('gpt-live-a', 'gpt-live-b'), + }) + + expect(result).toEqual({ provider: 'openai', model: 'gpt-live-b', apiKey: 'sk-live' }) + }) + + test('shows the curated default once before deduplicated live ids', async () => { + const { io, output } = scriptedIO(['2', ''], ['sk-live']) + + await runSetupWizard(io, { + path: '/tmp/config.json', + save: async () => {}, + fetchFn: liveModels('gpt-5.3-codex', 'gpt-live-a', 'gpt-live-a'), + }) + + expect(output.join('').match(/gpt-5\.3-codex/g)).toHaveLength(1) + expect(output.join('').match(/gpt-live-a/g)).toHaveLength(1) + }) + + test('marks catalog suggestions as an offline list', async () => { + const { io, output } = scriptedIO(['2', ''], ['sk-offline']) + + await runSetupWizard(io, { + path: '/tmp/config.json', + save: async () => {}, + fetchFn: unavailableModels, + }) + + expect(output.join('')).toContain('Available models (offline list):') + }) + + test('warns and re-prompts the key once after a 401 before using live models', async () => { + const { io, output } = scriptedIO(['2', '2'], ['bad-key', 'good-key']) + const requestedKeys: string[] = [] + const fetchFn: ModelListingFetch = async (_input, init) => { + const key = new Headers(init?.headers).get('Authorization') ?? '' + requestedKeys.push(key) + if (key === 'Bearer bad-key') return new Response(null, { status: 401 }) + return Response.json({ data: [{ id: 'gpt-live' }] }) + } + + const result = await runSetupWizard(io, { + path: '/tmp/config.json', + save: async () => {}, + fetchFn, + }) + + expect(requestedKeys).toEqual(['Bearer bad-key', 'Bearer good-key']) + expect(result).toEqual({ provider: 'openai', model: 'gpt-live', apiKey: 'good-key' }) + expect(output.join('')).toContain('provider rejected this API key (401) — check it.') + }) + + test('continues with catalog models after a second authentication rejection', async () => { + const { io, output } = scriptedIO(['2', ''], ['bad-key', 'still-bad']) + const fetchFn: ModelListingFetch = async () => new Response(null, { status: 403 }) + + const result = await runSetupWizard(io, { + path: '/tmp/config.json', + save: async () => {}, + fetchFn, + }) + + expect(result).toEqual({ provider: 'openai', model: 'gpt-5.3-codex', apiKey: 'still-bad' }) + expect(output.join('')).toContain('Available models (offline list):') + }) }) diff --git a/cli/src/config/wizard.ts b/cli/src/config/wizard.ts index 6d3918b56..752de2982 100644 --- a/cli/src/config/wizard.ts +++ b/cli/src/config/wizard.ts @@ -6,12 +6,13 @@ import { createInterface } from 'node:readline/promises' import { Writable } from 'node:stream' -import { builtinModels } from '@earendil-works/pi-ai/providers/all' import { BUILTIN_PROVIDER_ENV_VARS, DEFAULT_MODELS, DEFAULT_PROVIDER } from '../agent/defaults.ts' import type { BuiltinProvider, ModelProvider, RunConfig } from '../agent/types.ts' import { configPath } from '../paths.ts' import { saveConfig } from './config.ts' import type { CliConfig } from './config.ts' +import { listModels } from './model-listing.ts' +import type { ModelListingFetch, ModelListingResult } from './model-listing.ts' type BuiltinChoice = { readonly kind: 'builtin' @@ -61,7 +62,8 @@ export type SetupWizardTerminalIO = SetupWizardIO & { readonly close: () => void type SetupWizardDependencies = { readonly path?: string readonly save?: (config: CliConfig, path: string) => Promise - readonly catalogIds?: (provider: BuiltinProvider) => readonly string[] + readonly fetchFn?: ModelListingFetch + readonly modelListingTimeoutMs?: number readonly requiredProvider?: ModelProvider } @@ -131,18 +133,18 @@ const readApiKey = async (io: SetupWizardIO, local: boolean): Promise => } } -/** Builds unique default-first model suggestions from Pi catalog ids. */ -const modelChoices = (provider: BuiltinProvider, catalogIds: readonly string[]): readonly string[] => - [...new Set([DEFAULT_MODELS[provider], ...catalogIds])].slice(0, 4) +/** Builds unique default-first model suggestions from live or Pi catalog ids. */ +const modelChoices = (provider: BuiltinProvider, ids: readonly string[]): readonly string[] => + [...new Set([DEFAULT_MODELS[provider], ...ids])] /** Reads a built-in model, accepting Enter for default, number, or free-form id. */ const chooseBuiltinModel = async ( io: SetupWizardIO, provider: BuiltinProvider, - catalogIds: readonly string[], + listing: ModelListingResult, ): Promise => { - const choices = modelChoices(provider, catalogIds) - io.write('Available models:\n') + const choices = modelChoices(provider, listing.ids) + io.write(listing.source === 'catalog' ? 'Available models (offline list):\n' : 'Available models:\n') choices.forEach((model, index) => io.write(` ${index + 1}. ${model}${index === 0 ? ' (default)' : ''}\n`)) const answer = await io.readLine(`Model [${DEFAULT_MODELS[provider]}]: `) if (answer === null) throw new Error('Setup cancelled.') @@ -151,6 +153,61 @@ const chooseBuiltinModel = async ( return numbered ?? answer.trim() } +/** Reads a numbered or free-form model id for a custom compatible endpoint. */ +const chooseCompatModel = async (io: SetupWizardIO, listing: ModelListingResult): Promise => { + if (listing.source === 'catalog') { + io.write('Available models (offline list):\n') + return readRequiredLine(io, 'Model id: ') + } + + const choices = [...new Set(listing.ids)] + io.write('Available models:\n') + choices.forEach((model, index) => io.write(` ${index + 1}. ${model}\n`)) + while (true) { + const answer = await io.readLine('Model id or number: ') + if (answer === null) throw new Error('Setup cancelled.') + if (answer.trim() === '') { + io.write('Value required.\n') + continue + } + return choices[Number(answer.trim()) - 1] ?? answer.trim() + } +} + +type ApiKeyAndListing = { + readonly apiKey: string + readonly listing: ModelListingResult +} + +/** Lists models and allows one key correction after provider authentication rejection. */ +const listModelsWithKeyRetry = async ( + io: SetupWizardIO, + provider: ModelProvider, + apiKey: string, + baseUrl: string | undefined, + local: boolean, + dependencies: SetupWizardDependencies, +): Promise => { + const listForKey = (key: string) => + listModels({ + provider, + apiKey: key, + ...(baseUrl === undefined ? {} : { baseUrl }), + ...(dependencies.fetchFn === undefined ? {} : { fetchFn: dependencies.fetchFn }), + ...(dependencies.modelListingTimeoutMs === undefined ? {} : { timeoutMs: dependencies.modelListingTimeoutMs }), + }) + const firstListing = await listForKey(apiKey) + if (!firstListing.authRejected) return { apiKey, listing: firstListing } + + io.write(`provider rejected this API key (${firstListing.status}) — check it.\n`) + const retriedApiKey = await readApiKey(io, local) + const secondListing = await listForKey(retriedApiKey) + if (secondListing.authRejected) { + io.write(`provider rejected this API key (${secondListing.status}) — check it.\n`) + } + return { apiKey: retriedApiKey, listing: secondListing } +} + /** Resolves selected choice into provider and optional custom base URL. */ const resolveChoice = async ( choice: ProviderChoice, @@ -168,18 +225,20 @@ export const runSetupWizard = async ( ): Promise => { const choice = await chooseProvider(io, dependencies.requiredProvider) const resolved = await resolveChoice(choice, io) - const apiKey = await readApiKey(io, choice.kind === 'compat' && choice.local) - const catalogIds = - dependencies.catalogIds ?? - ((provider) => - builtinModels() - .getModels(provider) - .slice(0, 3) - .map(({ id }) => id)) + const local = choice.kind === 'compat' && choice.local + const initialApiKey = await readApiKey(io, local) + const { apiKey, listing } = await listModelsWithKeyRetry( + io, + resolved.provider, + initialApiKey, + resolved.baseUrl, + local, + dependencies, + ) const model = choice.kind === 'builtin' - ? await chooseBuiltinModel(io, choice.provider, catalogIds(choice.provider)) - : await readRequiredLine(io, 'Model id: ') + ? await chooseBuiltinModel(io, choice.provider, listing) + : await chooseCompatModel(io, listing) const config: CliConfig = { provider: resolved.provider, model, From dab95db59aa554e28a7d25c8d365b0a3c43b7eef Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C3=8Dtalo=20Menezes?= Date: Wed, 15 Jul 2026 14:53:34 -0300 Subject: [PATCH 06/12] fix: verify model listing endpoints and refresh provider defaults --- cli/README.md | 120 +++++++++++----------- cli/src/agent/defaults.ts | 2 +- cli/src/agent/model.test.ts | 45 ++++---- cli/src/agent/openai-compat-model.test.ts | 18 ++-- cli/src/agent/openai-compat-model.ts | 8 +- cli/src/cli.test.ts | 28 ++--- cli/src/cli.ts | 8 +- cli/src/config/model-listing.test.ts | 80 ++++++++++++--- cli/src/config/model-listing.ts | 109 ++++++++++++++------ cli/src/config/wizard.test.ts | 18 ++-- 10 files changed, 260 insertions(+), 176 deletions(-) diff --git a/cli/README.md b/cli/README.md index 18303ee3f..5b48ade41 100644 --- a/cli/README.md +++ b/cli/README.md @@ -60,7 +60,7 @@ mv "thunderbolt-cli-$TARGET" ~/.local/bin/thunderbolt > **What the checksum covers.** `SHA256SUMS` and the binary come from the same > release over the same TLS connection, so the checksum catches a corrupted or -> truncated download but *not* a compromised release host — whoever could swap the +> truncated download but _not_ a compromised release host — whoever could swap the > binary could swap its digest too. The binaries are unsigned and the quarantine > strip bypasses macOS Gatekeeper. Signature verification (minisign) over the > manifest against a pinned key is the planned follow-up hardening. @@ -119,15 +119,15 @@ thunderbolt ### Subcommands -| Command | Purpose | -| ------- | ------- | -| `thunderbolt agent [options] [prompt]` | Run coding agent; `agent` is optional/default. | -| `thunderbolt config` | Run guided provider setup and overwrite saved defaults. | -| `thunderbolt acp serve [options]` | Expose built-in coding agent as stdio ACP server. | -| `thunderbolt acp --transport [--port N] -- ` | Bridge stdio ACP agent. | -| `thunderbolt mcp --transport [--port N] -- ` | Bridge stdio MCP server. | -| `thunderbolt connect [-- ]` | Dial iroh bridge. | -| `thunderbolt iroh id` / `pair` / `allow ` | Inspect ACP identity, print pairing ticket, or authorize peer. | +| Command | Purpose | +| ---------------------------------------------------------------------------- | -------------------------------------------------------------- | +| `thunderbolt agent [options] [prompt]` | Run coding agent; `agent` is optional/default. | +| `thunderbolt config` | Run guided provider setup and overwrite saved defaults. | +| `thunderbolt acp serve [options]` | Expose built-in coding agent as stdio ACP server. | +| `thunderbolt acp --transport [--port N] -- ` | Bridge stdio ACP agent. | +| `thunderbolt mcp --transport [--port N] -- ` | Bridge stdio MCP server. | +| `thunderbolt connect [-- ]` | Dial iroh bridge. | +| `thunderbolt iroh id` / `pair` / `allow ` | Inspect ACP identity, print pairing ticket, or authorize peer. | ### Served agent workspace @@ -154,17 +154,17 @@ elsewhere on the machine are outside its workspace and unavailable. ### Agent and `acp serve` flags -| Flag | Description | -| ---- | ----------- | -| `-m`, `--model ` | Provider model id (provider-specific default). | -| `--provider ` | Built-in provider or `openai-compat` (default: `anthropic`). | -| `--base-url ` | Custom endpoint URL (required for `openai-compat`). | -| `--api-key ` | Explicit key for any provider; overrides provider environment. | +| Flag | Description | +| -------------------- | -------------------------------------------------------------------------- | +| `-m`, `--model ` | Provider model id (provider-specific default). | +| `--provider ` | Built-in provider or `openai-compat` (default: `anthropic`). | +| `--base-url ` | Custom endpoint URL (required for `openai-compat`). | +| `--api-key ` | Explicit key for any provider; overrides provider environment. | | `--thinking ` | `off`, `minimal`, `low`, `medium`, `high`, or `xhigh` (default: `medium`). | -| `-y`, `--yolo` | Auto-approve tool calls (alias: `--dangerously-skip-permissions`). | -| `--no-tui` | Force plain readline REPL. | -| `-h`, `--help` | Show help and exit. | -| `-v`, `--version` | Print version and exit. | +| `-y`, `--yolo` | Auto-approve tool calls (alias: `--dangerously-skip-permissions`). | +| `--no-tui` | Force plain readline REPL. | +| `-h`, `--help` | Show help and exit. | +| `-v`, `--version` | Print version and exit. | ACP/MCP bridge commands accept `--transport wss|iroh` (default `wss`) and `--port <0-65535>` for WSS (defaults: ACP `8839`, MCP `8840`). Arguments after @@ -185,8 +185,8 @@ catalog ids for selected provider. ```sh THUNDERBOLT_OPENAI_COMPAT_KEY=sk-... thunderbolt \ --provider openai-compat \ - --base-url https://host.example/v1 \ - --model upstream-model \ + --base-url http://localhost:11434/v1 \ + --model llama3.3 \ "review this repository" ``` @@ -196,47 +196,47 @@ credential cannot be forwarded automatically to an arbitrary custom URL. ### Provider defaults -| Provider | Default model | -| -------- | ------------- | -| `anthropic` | `claude-opus-4-8` | -| `openai` | `gpt-5.3-codex` | -| `google` | `gemini-3.1-pro-preview` | -| `xai` | `grok-build-0.1` | -| `deepseek` | `deepseek-v4-pro` | -| `zai` | `glm-5.2` | -| `mistral` | `devstral-medium-latest` | -| `groq` | `openai/gpt-oss-120b` | -| `openrouter` | `anthropic/claude-opus-4.8` | -| `moonshotai` | `kimi-k2.7-code` | -| `minimax` | `MiniMax-M3` | -| `cerebras` | `gpt-oss-120b` | -| `together` | `moonshotai/Kimi-K2.7-Code` | -| `fireworks` | `accounts/fireworks/models/kimi-k2p7-code` | +| Provider | Default model | +| ------------ | ------------------------------------------ | +| `anthropic` | `claude-opus-4-8` | +| `openai` | `gpt-5.6-sol` | +| `google` | `gemini-3.1-pro-preview` | +| `xai` | `grok-build-0.1` | +| `deepseek` | `deepseek-v4-pro` | +| `zai` | `glm-5.2` | +| `mistral` | `devstral-medium-latest` | +| `groq` | `openai/gpt-oss-120b` | +| `openrouter` | `anthropic/claude-opus-4.8` | +| `moonshotai` | `kimi-k2.7-code` | +| `minimax` | `MiniMax-M3` | +| `cerebras` | `gpt-oss-120b` | +| `together` | `moonshotai/Kimi-K2.7-Code` | +| `fireworks` | `accounts/fireworks/models/kimi-k2p7-code` | ### Environment -| Variable | Description | -| ----------------------------- | --------------------------------------------------------------------------- | -| `ANTHROPIC_OAUTH_TOKEN`, `ANTHROPIC_API_KEY` | Anthropic credentials, checked in that order. | -| `OPENAI_API_KEY` | OpenAI API key. | -| `GEMINI_API_KEY` | Google Gemini API key. | -| `XAI_API_KEY` | xAI API key. | -| `DEEPSEEK_API_KEY` | DeepSeek API key. | -| `ZAI_API_KEY` | Z.AI API key. | -| `MISTRAL_API_KEY` | Mistral API key. | -| `GROQ_API_KEY` | Groq API key. | -| `OPENROUTER_API_KEY` | OpenRouter API key. | -| `MOONSHOT_API_KEY` | Moonshot AI API key. | -| `MINIMAX_API_KEY` | MiniMax API key. | -| `CEREBRAS_API_KEY` | Cerebras API key. | -| `TOGETHER_API_KEY` | Together API key. | -| `FIREWORKS_API_KEY` | Fireworks API key. | -| `THUNDERBOLT_OPENAI_COMPAT_KEY` | Dedicated fallback key for arbitrary `openai-compat` URLs. | -| `THUNDERBOLT_HOME` | CLI state root containing `config.json`, iroh identity/allowlist, and ACP sessions (default: `~/.thunderbolt`). | -| `THUNDERBOLT_IROH_RELAY_URL` | Self-hosted iroh-relay WSS URL; unset uses n0 public relays. | -| `THUNDERBOLT_APP_ORIGIN` | Extra comma-separated allowed browser origins for WSS bridges. | -| `THUNDERBOLT_NO_TUI` | Force plain readline REPL when set. | -| `NO_COLOR` | Disable terminal color when set. | +| Variable | Description | +| -------------------------------------------- | --------------------------------------------------------------------------------------------------------------- | +| `ANTHROPIC_OAUTH_TOKEN`, `ANTHROPIC_API_KEY` | Anthropic credentials, checked in that order. | +| `OPENAI_API_KEY` | OpenAI API key. | +| `GEMINI_API_KEY` | Google Gemini API key. | +| `XAI_API_KEY` | xAI API key. | +| `DEEPSEEK_API_KEY` | DeepSeek API key. | +| `ZAI_API_KEY` | Z.AI API key. | +| `MISTRAL_API_KEY` | Mistral API key. | +| `GROQ_API_KEY` | Groq API key. | +| `OPENROUTER_API_KEY` | OpenRouter API key. | +| `MOONSHOT_API_KEY` | Moonshot AI API key. | +| `MINIMAX_API_KEY` | MiniMax API key. | +| `CEREBRAS_API_KEY` | Cerebras API key. | +| `TOGETHER_API_KEY` | Together API key. | +| `FIREWORKS_API_KEY` | Fireworks API key. | +| `THUNDERBOLT_OPENAI_COMPAT_KEY` | Dedicated fallback key for arbitrary `openai-compat` URLs. | +| `THUNDERBOLT_HOME` | CLI state root containing `config.json`, iroh identity/allowlist, and ACP sessions (default: `~/.thunderbolt`). | +| `THUNDERBOLT_IROH_RELAY_URL` | Self-hosted iroh-relay WSS URL; unset uses n0 public relays. | +| `THUNDERBOLT_APP_ORIGIN` | Extra comma-separated allowed browser origins for WSS bridges. | +| `THUNDERBOLT_NO_TUI` | Force plain readline REPL when set. | +| `NO_COLOR` | Disable terminal color when set. | ## Demo diff --git a/cli/src/agent/defaults.ts b/cli/src/agent/defaults.ts index 6acbace55..a6bd73176 100644 --- a/cli/src/agent/defaults.ts +++ b/cli/src/agent/defaults.ts @@ -15,7 +15,7 @@ export const DEFAULT_MODEL = 'claude-opus-4-8' /** Default catalog model for each built-in provider. */ export const DEFAULT_MODELS: Readonly> = { anthropic: DEFAULT_MODEL, - openai: 'gpt-5.3-codex', + openai: 'gpt-5.6-sol', google: 'gemini-3.1-pro-preview', xai: 'grok-build-0.1', deepseek: 'deepseek-v4-pro', diff --git a/cli/src/agent/model.test.ts b/cli/src/agent/model.test.ts index 4d2e44e64..ac45032e5 100644 --- a/cli/src/agent/model.test.ts +++ b/cli/src/agent/model.test.ts @@ -65,58 +65,58 @@ const capturingBuiltinModels = () => { describe('resolveModel — openai-compat branch', () => { test('throws when --base-url is missing', () => { - expect(() => resolveModel({ model: 'mimo', provider: 'openai-compat', apiKey: 'k' })).toThrow(/--base-url/) + expect(() => resolveModel({ model: 'llama3.3', provider: 'openai-compat', apiKey: 'local' })).toThrow(/--base-url/) }) test('throws when the api key is missing even with a base URL', () => { - expect(() => resolveModel({ model: 'mimo', provider: 'openai-compat', baseUrl: 'https://h/v1' })).toThrow( - /requires an api key/, - ) + expect(() => + resolveModel({ model: 'llama3.3', provider: 'openai-compat', baseUrl: 'http://localhost:11434/v1' }), + ).toThrow(/requires an api key/) }) test('missing custom key error points non-TTY users to guided setup', () => { - expect(() => resolveModel({ model: 'mimo', provider: 'openai-compat', baseUrl: 'https://h/v1' })).toThrow( - /THUNDERBOLT_OPENAI_COMPAT_KEY.*--api-key.*run `thunderbolt` in a terminal for guided setup/, - ) + expect(() => + resolveModel({ model: 'llama3.3', provider: 'openai-compat', baseUrl: 'http://localhost:11434/v1' }), + ).toThrow(/THUNDERBOLT_OPENAI_COMPAT_KEY.*--api-key.*run `thunderbolt` in a terminal for guided setup/) }) test('missing custom key stays actionable when the base URL is also missing', () => { - expect(() => resolveModel({ model: 'mimo', provider: 'openai-compat' })).toThrow( + expect(() => resolveModel({ model: 'llama3.3', provider: 'openai-compat' })).toThrow( /THUNDERBOLT_OPENAI_COMPAT_KEY.*--api-key.*run `thunderbolt` in a terminal for guided setup/, ) }) test('rejects an empty-string base URL (falsy guard, not just undefined)', () => { - expect(() => resolveModel({ model: 'mimo', provider: 'openai-compat', baseUrl: '', apiKey: 'k' })).toThrow( + expect(() => resolveModel({ model: 'llama3.3', provider: 'openai-compat', baseUrl: '', apiKey: 'local' })).toThrow( /--base-url/, ) }) test('rejects an empty-string api key', () => { - expect(() => resolveModel({ model: 'mimo', provider: 'openai-compat', baseUrl: 'https://h/v1', apiKey: '' })).toThrow( - /requires an api key/, - ) + expect(() => + resolveModel({ model: 'llama3.3', provider: 'openai-compat', baseUrl: 'http://localhost:11434/v1', apiKey: '' }), + ).toThrow(/requires an api key/) }) test('resolves a synthetic model carrying the upstream id and base URL', () => { const { models, model } = resolveModel({ - model: 'mimo-v2.5-pro', + model: 'llama3.3', provider: 'openai-compat', - baseUrl: 'https://h/v1', - apiKey: 'secret', + baseUrl: 'http://localhost:11434/v1', + apiKey: 'local', }) - expect(model.id).toBe('mimo-v2.5-pro') + expect(model.id).toBe('llama3.3') expect(model.provider).toBe('openai-compat') - expect(model.baseUrl).toBe('https://h/v1') + expect(model.baseUrl).toBe('http://localhost:11434/v1') // The model is registered in the returned collection under its provider. - expect(models.getModel('openai-compat', 'mimo-v2.5-pro')?.id).toBe('mimo-v2.5-pro') + expect(models.getModel('openai-compat', 'llama3.3')?.id).toBe('llama3.3') }) test('the resolved model descriptor does not embed the secret key', () => { const { model } = resolveModel({ - model: 'mimo', + model: 'llama3.3', provider: 'openai-compat', - baseUrl: 'https://h/v1', + baseUrl: 'http://localhost:11434/v1', apiKey: 'super-secret-key', }) expect(JSON.stringify(model)).not.toContain('super-secret-key') @@ -167,10 +167,7 @@ describe('resolveModel — built-in providers', () => { test('missing built-in credentials stay actionable when the model id is also invalid', () => { expect(() => - resolveModel( - { model: 'not-a-google-model', provider: 'google' }, - { builtinModels, env: EMPTY_ENV }, - ), + resolveModel({ model: 'not-a-google-model', provider: 'google' }, { builtinModels, env: EMPTY_ENV }), ).toThrow(/GEMINI_API_KEY.*--api-key.*run `thunderbolt` in a terminal for guided setup/) }) diff --git a/cli/src/agent/openai-compat-model.test.ts b/cli/src/agent/openai-compat-model.test.ts index 47c046f2e..ba2914a0a 100644 --- a/cli/src/agent/openai-compat-model.test.ts +++ b/cli/src/agent/openai-compat-model.test.ts @@ -14,7 +14,7 @@ import type { Api, AssistantMessageEventStream, Context, Model } from '@earendil import { describe, expect, test } from 'bun:test' import { type OpenAiStreamFns, buildOpenAiCompatModel } from './openai-compat-model.ts' -const opts = { modelId: 'mimo-v2.5-pro', baseUrl: 'https://h.example/v1', apiKey: 'k' } +const opts = { modelId: 'llama3.3', baseUrl: 'http://localhost:11434/v1', apiKey: 'local' } /** A fake pair of raw stream fns that record the options they were handed and * return an inert stream, so the bearer-key injection is observable offline. */ @@ -37,10 +37,10 @@ const capturingStreamFns = () => { describe('buildOpenAiCompatModel — synthetic descriptor', () => { test('carries the upstream id, base URL, and provider id', () => { const { model } = buildOpenAiCompatModel(opts) - expect(model.id).toBe('mimo-v2.5-pro') - expect(model.name).toBe('mimo-v2.5-pro') + expect(model.id).toBe('llama3.3') + expect(model.name).toBe('llama3.3') expect(model.provider).toBe('openai-compat') - expect(model.baseUrl).toBe('https://h.example/v1') + expect(model.baseUrl).toBe('http://localhost:11434/v1') expect(model.api).toBe('openai-completions') }) @@ -53,8 +53,8 @@ describe('buildOpenAiCompatModel — synthetic descriptor', () => { test('registers the model in the returned collection under its provider', () => { const { models } = buildOpenAiCompatModel(opts) - expect(models.getModel('openai-compat', 'mimo-v2.5-pro')?.id).toBe('mimo-v2.5-pro') - expect(models.getProvider('openai-compat')?.baseUrl).toBe('https://h.example/v1') + expect(models.getModel('openai-compat', 'llama3.3')?.id).toBe('llama3.3') + expect(models.getProvider('openai-compat')?.baseUrl).toBe('http://localhost:11434/v1') }) }) @@ -86,7 +86,7 @@ describe('buildOpenAiCompatModel — bearer key injection (security)', () => { const provider = models.getProvider('openai-compat')! provider.stream(model, {} as Context, { maxTokens: 256 }) expect(calls).toHaveLength(1) - expect(calls[0]!.options.apiKey).toBe('k') + expect(calls[0]!.options.apiKey).toBe('local') // Caller-supplied options must be preserved alongside the injected key. expect(calls[0]!.options.maxTokens).toBe(256) }) @@ -97,7 +97,7 @@ describe('buildOpenAiCompatModel — bearer key injection (security)', () => { const provider = models.getProvider('openai-compat')! // A caller trying to slip a different key in must not win over the resolved one. provider.stream(model, {} as Context, { apiKey: 'attacker-supplied' } as never) - expect(calls[0]!.options.apiKey).toBe('k') + expect(calls[0]!.options.apiKey).toBe('local') }) test('streamSimple also injects the configured key', () => { @@ -105,6 +105,6 @@ describe('buildOpenAiCompatModel — bearer key injection (security)', () => { const { models, model } = buildOpenAiCompatModel(opts, streamFns) const provider = models.getProvider('openai-compat')! provider.streamSimple(model, {} as Context, {}) - expect(calls[0]).toMatchObject({ fn: 'streamSimple', options: { apiKey: 'k' } }) + expect(calls[0]).toMatchObject({ fn: 'streamSimple', options: { apiKey: 'local' } }) }) }) diff --git a/cli/src/agent/openai-compat-model.ts b/cli/src/agent/openai-compat-model.ts index b6c71f067..fce3604a5 100644 --- a/cli/src/agent/openai-compat-model.ts +++ b/cli/src/agent/openai-compat-model.ts @@ -4,8 +4,8 @@ /** * Builds a Pi `openai-completions` model bound to a custom base URL + bearer - * key, so the CLI can run any OpenAI-compatible endpoint (e.g. Xiaomi MiMo at - * `https://token-plan-sgp.xiaomimimo.com/v1`) outside Pi's built-in providers. + * key, so the CLI can run any OpenAI-compatible endpoint (e.g. Ollama at + * `http://localhost:11434/v1`) outside Pi's built-in providers. * * This is the CLI sibling of `shared/agent-core/openai-compat-model.ts`, but * deliberately simpler. The app's version SYNCHRONOUSLY swaps `globalThis.fetch` @@ -53,9 +53,9 @@ const DEFAULT_MAX_TOKENS = 8_192 /** Inputs for {@link buildOpenAiCompatModel}. */ export type BuildOpenAiCompatModelOptions = { - /** Upstream model id sent on the wire, e.g. `mimo-v2.5-pro`. */ + /** Upstream model id sent on the wire, e.g. `llama3.3`. */ readonly modelId: string - /** OpenAI-compatible base URL, e.g. `https://token-plan-sgp.xiaomimimo.com/v1`. */ + /** OpenAI-compatible base URL, e.g. `http://localhost:11434/v1`. */ readonly baseUrl: string /** Bearer key handed to the OpenAI SDK (sent as `Authorization: Bearer `). */ readonly apiKey: string diff --git a/cli/src/cli.test.ts b/cli/src/cli.test.ts index 76c3234d3..113bf22f5 100644 --- a/cli/src/cli.test.ts +++ b/cli/src/cli.test.ts @@ -33,15 +33,10 @@ const runConfig = (argv: string[], dependencies?: ParseArgsDependencies) => { describe('parseArgs — resolveApiKey precedence (security)', () => { test('--api-key flag wins over the env var', () => { - const config = runConfig([ - '--provider', - 'openai-compat', - '--base-url', - 'https://h/v1', - '--api-key', - 'flag-key', - 'hi', - ], { env: { [ENV_KEY]: 'env-key' } }) + const config = runConfig( + ['--provider', 'openai-compat', '--base-url', 'https://h/v1', '--api-key', 'flag-key', 'hi'], + { env: { [ENV_KEY]: 'env-key' } }, + ) expect(config.apiKey).toBe('flag-key') }) @@ -137,7 +132,7 @@ describe('parseArgs — defaults', () => { test('uses provider-specific default models when --model is omitted', () => { const expected = { anthropic: 'claude-opus-4-8', - openai: 'gpt-5.3-codex', + openai: 'gpt-5.6-sol', google: 'gemini-3.1-pro-preview', xai: 'grok-build-0.1', deepseek: 'deepseek-v4-pro', @@ -209,7 +204,7 @@ describe('parseArgs — persisted config precedence', () => { test('matching built-in provider env suppresses saved-key injection so Pi owns env auth', () => { const config = runConfig([], { - config: { provider: 'openai', model: 'gpt-5.3-codex', apiKey: 'saved-key' }, + config: { provider: 'openai', model: 'gpt-5.6-sol', apiKey: 'saved-key' }, env: { OPENAI_API_KEY: 'env-key' }, }) @@ -347,16 +342,7 @@ describe('parseArgs — bridge subcommands (acp / mcp)', () => { describe('parseArgs — acp serve', () => { test('resolves the same flag set as a run, including api-key precedence', () => { const parsed = parseArgs( - [ - 'acp', - 'serve', - '--provider', - 'openai-compat', - '--base-url', - 'https://h/v1', - '--api-key', - 'flag-key', - ], + ['acp', 'serve', '--provider', 'openai-compat', '--base-url', 'https://h/v1', '--api-key', 'flag-key'], { env: { [ENV_KEY]: 'env-key' } }, ) if (parsed.kind !== 'acp-serve') throw new Error(`expected acp-serve, got ${parsed.kind}`) diff --git a/cli/src/cli.ts b/cli/src/cli.ts index d952b145c..efd8e4633 100644 --- a/cli/src/cli.ts +++ b/cli/src/cli.ts @@ -105,7 +105,7 @@ EXAMPLES thunderbolt --provider google --api-key AIza… "review this repository" thunderbolt config thunderbolt - THUNDERBOLT_OPENAI_COMPAT_KEY=sk-… thunderbolt --provider openai-compat --base-url https://host/v1 --model my-model "hello" + THUNDERBOLT_OPENAI_COMPAT_KEY=local thunderbolt --provider openai-compat --base-url http://localhost:11434/v1 --model llama3.3 "hello" thunderbolt acp --transport wss -- npx @zed-industries/claude-code-acp thunderbolt mcp --transport wss --port 9001 -- uvx mcp-server-fetch thunderbolt acp --transport iroh -- thunderbolt acp serve # share THIS agent @@ -368,11 +368,7 @@ const resolveModelId = (flags: Flags, provider: ModelProvider, config: CliConfig } /** Resolves custom endpoint flag against provider-scoped saved config. */ -const resolveBaseUrl = ( - flags: Flags, - provider: ModelProvider, - config: CliConfig | null, -): string | undefined => { +const resolveBaseUrl = (flags: Flags, provider: ModelProvider, config: CliConfig | null): string | undefined => { if (flags.baseUrl !== undefined) return flags.baseUrl return config?.provider === provider ? config.baseUrl : undefined } diff --git a/cli/src/config/model-listing.test.ts b/cli/src/config/model-listing.test.ts index c5b513867..5101fb490 100644 --- a/cli/src/config/model-listing.test.ts +++ b/cli/src/config/model-listing.test.ts @@ -65,12 +65,48 @@ describe('listModels', () => { const result = await listModels({ provider: 'google', apiKey: 'gemini-key', fetchFn }) expect(result).toEqual({ source: 'live', ids: ['gemini-live-chat'] }) - expect(String(requests[0]?.input)).toBe( - 'https://generativelanguage.googleapis.com/v1beta/models?key=gemini-key', - ) + expect(String(requests[0]?.input)).toBe('https://generativelanguage.googleapis.com/v1beta/models?key=gemini-key') expect(new Headers(requests[0]?.init?.headers).has('Authorization')).toBe(false) }) + test('uses xAI language-model listing instead of mixed-modality models', async () => { + const requests: { readonly input: string | URL | Request; readonly init?: RequestInit }[] = [] + const fetchFn: ModelListingFetch = async (input, init) => { + requests.push({ input, init }) + return Response.json({ + models: [ + { id: 'grok-live-a', created: 2 }, + { id: 'grok-live-b', created: 1 }, + ], + }) + } + + const result = await listModels({ provider: 'xai', apiKey: 'xai-key', fetchFn }) + + expect(result).toEqual({ source: 'live', ids: ['grok-live-a', 'grok-live-b'] }) + expect(String(requests[0]?.input)).toBe('https://api.x.ai/v1/language-models') + expect(new Headers(requests[0]?.init?.headers).get('Authorization')).toBe('Bearer xai-key') + }) + + test('reads Together bare-array model responses', async () => { + const requests: { readonly input: string | URL | Request; readonly init?: RequestInit }[] = [] + const result = await listModels({ + provider: 'together', + apiKey: 'together-key', + fetchFn: async (input, init) => { + requests.push({ input, init }) + return Response.json([ + { id: 'chat-model', type: 'chat', created: 2 }, + { id: 'embedding-model', type: 'embedding', created: 1 }, + ]) + }, + }) + + expect(result).toEqual({ source: 'live', ids: ['chat-model'] }) + expect(String(requests[0]?.input)).toBe('https://api.together.ai/v1/models') + expect(new Headers(requests[0]?.init?.headers).get('Authorization')).toBe('Bearer together-key') + }) + test('filters non-chat model id patterns from compatible responses', async () => { const fetchFn: ModelListingFetch = async () => Response.json({ @@ -107,40 +143,50 @@ describe('listModels', () => { test('derives compatible listing routes from Pi descriptors', async () => { const urls: string[] = [] - const fetchFn: ModelListingFetch = async (input) => { + const authorizations: (string | null)[] = [] + const fetchFn: ModelListingFetch = async (input, init) => { urls.push(String(input)) + authorizations.push(new Headers(init?.headers).get('Authorization')) return Response.json({ data: [{ id: 'chat-model' }] }) } for (const provider of [ - 'xai', 'deepseek', - 'zai', 'mistral', 'groq', 'openrouter', 'moonshotai', 'minimax', 'cerebras', - 'together', - 'fireworks', ] as const) { await listModels({ provider, apiKey: 'key', fetchFn }) } expect(urls).toEqual([ - 'https://api.x.ai/v1/models', 'https://api.deepseek.com/models', - 'https://api.z.ai/api/coding/paas/v4/models', 'https://api.mistral.ai/v1/models', 'https://api.groq.com/openai/v1/models', 'https://openrouter.ai/api/v1/models', 'https://api.moonshot.ai/v1/models', 'https://api.minimax.io/v1/models', 'https://api.cerebras.ai/v1/models', - 'https://api.together.ai/v1/models', - 'https://api.fireworks.ai/inference/v1/models', ]) + expect(authorizations).toEqual(Array.from({ length: 7 }, () => 'Bearer key')) + }) + + test('uses catalog fallback without network calls when official docs expose no usable list route', async () => { + const requestedProviders: string[] = [] + const fetchFn: ModelListingFetch = async (input) => { + requestedProviders.push(String(input)) + return Response.json({ data: [{ id: 'unexpected-live-model' }] }) + } + + const zai = await listModels({ provider: 'zai', apiKey: 'key', fetchFn }) + const fireworks = await listModels({ provider: 'fireworks', apiKey: 'key', fetchFn }) + + expect(zai.source).toBe('catalog') + expect(fireworks.source).toBe('catalog') + expect(requestedProviders).toEqual([]) }) test('returns catalog models on timeout even when injected fetch ignores abort', async () => { @@ -195,13 +241,21 @@ describe('listModels', () => { }) test('treats an empty chat-capable result as catalog fallback', async () => { + const urls: string[] = [] + const authorizations: (string | null)[] = [] const result = await listModels({ provider: 'openai-compat', baseUrl: 'http://localhost:11434/v1', apiKey: 'local', - fetchFn: async () => Response.json({ data: [{ id: 'nomic-embed-text' }] }), + fetchFn: async (input, init) => { + urls.push(String(input)) + authorizations.push(new Headers(init?.headers).get('Authorization')) + return Response.json({ data: [{ id: 'nomic-embed-text' }] }) + }, }) expect(result).toEqual({ source: 'catalog', ids: [] }) + expect(urls).toEqual(['http://localhost:11434/v1/models']) + expect(authorizations).toEqual(['Bearer local']) }) }) diff --git a/cli/src/config/model-listing.ts b/cli/src/config/model-listing.ts index e6474bb9a..26bb7aaaf 100644 --- a/cli/src/config/model-listing.ts +++ b/cli/src/config/model-listing.ts @@ -34,6 +34,19 @@ type ListedModel = { readonly created?: number } +type ListingRequest = { + readonly url: string + readonly headers: Readonly> +} + +/** Z.AI API reference has no model-list operation: https://docs.z.ai/llms.txt */ +const ZAI_FALLBACK_ONLY: ModelProvider = 'zai' + +/** Fireworks listing needs an account ID: https://docs.fireworks.ai/api-reference/list-models */ +const FIREWORKS_FALLBACK_ONLY: ModelProvider = 'fireworks' + +const FALLBACK_ONLY_PROVIDERS: ReadonlySet = new Set([ZAI_FALLBACK_ONLY, FIREWORKS_FALLBACK_ONLY]) + /** Reads numeric OpenAI or ISO Anthropic creation metadata. */ const createdTimestamp = (candidate: Readonly>): number | undefined => { if (typeof candidate.created === 'number') return candidate.created @@ -59,7 +72,7 @@ const parseOpenAiModels = (value: unknown): readonly ListedModel[] => { }) } -/** Reads Gemini model entries capable of `generateContent`. */ +/** Gemini listing schema: https://ai.google.dev/api/models#method:-models.list */ const parseGeminiModels = (value: unknown): readonly ListedModel[] => { if (!isRecord(value) || !Array.isArray(value.models)) throw new Error('Invalid Gemini model list response.') return value.models.flatMap((candidate) => { @@ -70,6 +83,37 @@ const parseGeminiModels = (value: unknown): readonly ListedModel[] => { }) } +/** xAI language-model schema: https://docs.x.ai/developers/rest-api-reference/inference/models#list-language-models */ +const parseXaiModels = (value: unknown): readonly ListedModel[] => { + if (!isRecord(value) || !Array.isArray(value.models)) throw new Error('Invalid xAI model list response.') + return value.models.map((candidate) => { + if (!isRecord(candidate) || typeof candidate.id !== 'string') throw new Error('Invalid xAI model entry.') + const created = createdTimestamp(candidate) + return { id: candidate.id, ...(created === undefined ? {} : { created }) } + }) +} + +/** Together listing schema: https://docs.together.ai/reference/models */ +const parseTogetherModels = (value: unknown): readonly ListedModel[] => { + if (!Array.isArray(value)) throw new Error('Invalid Together model list response.') + return value.flatMap((candidate) => { + if (!isRecord(candidate) || typeof candidate.id !== 'string' || typeof candidate.type !== 'string') { + throw new Error('Invalid Together model entry.') + } + if (!['chat', 'language', 'code'].includes(candidate.type)) return [] + const created = createdTimestamp(candidate) + return [{ id: candidate.id, ...(created === undefined ? {} : { created }) }] + }) +} + +/** Selects the documented response schema for the requested provider. */ +const parseListedModels = (provider: ModelProvider, value: unknown): readonly ListedModel[] => { + if (provider === 'google') return parseGeminiModels(value) + if (provider === 'xai') return parseXaiModels(value) + if (provider === 'together') return parseTogetherModels(value) + return parseOpenAiModels(value) +} + /** Keeps model ids intended for chat rather than specialized media or ranking APIs. */ const chatModels = (models: readonly ListedModel[]): readonly ListedModel[] => models.filter(({ id }) => !NON_CHAT_MODEL_PATTERN.test(id)) @@ -78,8 +122,7 @@ const chatModels = (models: readonly ListedModel[]): readonly ListedModel[] => const newestModelsFirst = (models: readonly ListedModel[]): readonly ListedModel[] => { if (!models.some(({ created }) => created !== undefined)) return models return [...models].sort( - (left, right) => - (right.created ?? Number.NEGATIVE_INFINITY) - (left.created ?? Number.NEGATIVE_INFINITY), + (left, right) => (right.created ?? Number.NEGATIVE_INFINITY) - (left.created ?? Number.NEGATIVE_INFINITY), ) } @@ -110,36 +153,44 @@ const providerBaseUrl = (provider: ModelProvider, customBaseUrl?: string): strin /** Joins one endpoint path without duplicating a trailing slash. */ const endpoint = (baseUrl: string, path: string): string => `${baseUrl.replace(/\/$/, '')}/${path}` +/** Mistral listing route: https://docs.mistral.ai/api/endpoint/models */ +const mistralListingBaseUrl = (baseUrl: string): string => endpoint(baseUrl, 'v1') + +/** MiniMax listing route: https://platform.minimax.io/docs/api-reference/models/openai/list-models */ +const minimaxListingBaseUrl = (baseUrl: string): string => new URL('/v1', baseUrl).toString() + /** Adapts Pi inference bases to providers' OpenAI-compatible model-listing bases. */ const compatibleBaseUrl = (provider: ModelProvider, baseUrl?: string): string => { const resolvedBaseUrl = providerBaseUrl(provider, baseUrl) - if (provider === 'mistral') return endpoint(resolvedBaseUrl, 'v1') - if (provider === 'minimax') return new URL('/v1', resolvedBaseUrl).toString() - if (provider !== 'fireworks') return resolvedBaseUrl + if (provider === 'mistral') return mistralListingBaseUrl(resolvedBaseUrl) + if (provider === 'minimax') return minimaxListingBaseUrl(resolvedBaseUrl) + return resolvedBaseUrl +} - const openAiModel = builtinModels() - .getModels(provider) - .find(({ api }) => api === 'openai-completions') - return openAiModel?.baseUrl ?? resolvedBaseUrl +/** Anthropic listing request: https://platform.claude.com/docs/en/api/models/list */ +const anthropicListingRequest = (apiKey: string): ListingRequest => ({ + url: endpoint(providerBaseUrl('anthropic'), 'v1/models'), + headers: { 'x-api-key': apiKey, 'anthropic-version': '2023-06-01' }, +}) + +/** Gemini listing request: https://ai.google.dev/api/models#method:-models.list */ +const googleListingRequest = (apiKey: string): ListingRequest => { + const url = new URL(endpoint(providerBaseUrl('google'), 'models')) + url.searchParams.set('key', apiKey) + return { url: url.toString(), headers: {} } } +/** xAI chat-model route: https://docs.x.ai/developers/rest-api-reference/inference/models#list-language-models */ +const xaiListingRequest = (apiKey: string): ListingRequest => ({ + url: endpoint(providerBaseUrl('xai'), 'language-models'), + headers: { Authorization: `Bearer ${apiKey}` }, +}) + /** Builds provider-specific listing URL and authentication headers. */ -const listingRequest = ( - provider: ModelProvider, - apiKey: string, - baseUrl?: string, -): { readonly url: string; readonly headers: Readonly> } => { - if (provider === 'anthropic') { - return { - url: endpoint(providerBaseUrl(provider), 'v1/models'), - headers: { 'x-api-key': apiKey, 'anthropic-version': '2023-06-01' }, - } - } - if (provider === 'google') { - const url = new URL(endpoint(providerBaseUrl(provider), 'models')) - url.searchParams.set('key', apiKey) - return { url: url.toString(), headers: {} } - } +const listingRequest = (provider: ModelProvider, apiKey: string, baseUrl?: string): ListingRequest => { + if (provider === 'anthropic') return anthropicListingRequest(apiKey) + if (provider === 'google') return googleListingRequest(apiKey) + if (provider === 'xai') return xaiListingRequest(apiKey) return { url: endpoint(compatibleBaseUrl(provider, baseUrl), 'models'), headers: { Authorization: `Bearer ${apiKey}` }, @@ -149,6 +200,7 @@ const listingRequest = ( /** Lists live provider models, returning Pi catalog ids for every failure mode. */ export const listModels = async (options: ListModelsOptions): Promise => { const fallback = (): ModelListingResult => ({ source: 'catalog', ids: catalogIds(options.provider) }) + if (FALLBACK_ONLY_PROVIDERS.has(options.provider)) return fallback() const controller = new AbortController() const timeout = Promise.withResolvers() const timeoutId = setTimeout(() => { @@ -172,9 +224,8 @@ export const listModels = async (options: ListModelsOptions): Promise id) } } catch { diff --git a/cli/src/config/wizard.test.ts b/cli/src/config/wizard.test.ts index 8780f995a..14e28fe3c 100644 --- a/cli/src/config/wizard.test.ts +++ b/cli/src/config/wizard.test.ts @@ -46,8 +46,10 @@ const scriptedIO = (lines: readonly string[], secrets: readonly string[]) => { } /** Returns one injected OpenAI-compatible model-list response. */ -const liveModels = (...ids: readonly string[]): ModelListingFetch => async () => - Response.json({ data: ids.map((id) => ({ id })) }) +const liveModels = + (...ids: readonly string[]): ModelListingFetch => + async () => + Response.json({ data: ids.map((id) => ({ id })) }) /** Simulates provider unavailability without touching ambient network state. */ const unavailableModels: ModelListingFetch = async () => new Response(null, { status: 503 }) @@ -74,9 +76,7 @@ describe('shouldRunSetupWizard', () => { test('custom endpoints recognize only the dedicated environment key', () => { const custom = runConfig({ provider: 'openai-compat', baseUrl: 'https://host/v1' }) - expect( - shouldRunSetupWizard(custom, runtime({ env: { THUNDERBOLT_OPENAI_COMPAT_KEY: 'custom-key' } })), - ).toBe(false) + expect(shouldRunSetupWizard(custom, runtime({ env: { THUNDERBOLT_OPENAI_COMPAT_KEY: 'custom-key' } }))).toBe(false) expect(shouldRunSetupWizard(custom, runtime({ env: { OPENAI_API_KEY: 'real-openai-key' } }))).toBe(true) }) }) @@ -94,7 +94,7 @@ describe('runSetupWizard', () => { fetchFn: liveModels('gpt-live-a', 'gpt-live-b'), }) - const expected: CliConfig = { provider: 'openai', model: 'gpt-5.3-codex', apiKey: 'sk-openai' } + const expected: CliConfig = { provider: 'openai', model: 'gpt-5.6-sol', apiKey: 'sk-openai' } expect(result).toEqual(expected) expect(saved).toEqual([expected]) expect(output.join('')).toContain('2. OpenAI') @@ -212,10 +212,10 @@ describe('runSetupWizard', () => { await runSetupWizard(io, { path: '/tmp/config.json', save: async () => {}, - fetchFn: liveModels('gpt-5.3-codex', 'gpt-live-a', 'gpt-live-a'), + fetchFn: liveModels('gpt-5.6-sol', 'gpt-live-a', 'gpt-live-a'), }) - expect(output.join('').match(/gpt-5\.3-codex/g)).toHaveLength(1) + expect(output.join('').match(/gpt-5\.6-sol/g)).toHaveLength(1) expect(output.join('').match(/gpt-live-a/g)).toHaveLength(1) }) @@ -262,7 +262,7 @@ describe('runSetupWizard', () => { fetchFn, }) - expect(result).toEqual({ provider: 'openai', model: 'gpt-5.3-codex', apiKey: 'still-bad' }) + expect(result).toEqual({ provider: 'openai', model: 'gpt-5.6-sol', apiKey: 'still-bad' }) expect(output.join('')).toContain('Available models (offline list):') }) }) From a5dc942d2c54a95552521f90b180f5c1cb8cf89a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C3=8Dtalo=20Menezes?= Date: Wed, 15 Jul 2026 14:53:39 -0300 Subject: [PATCH 07/12] docs: rewrite the relay self-hosting guide for a public audience --- deploy/iroh-relay/docker-compose.yml | 7 +- docs/architecture/iroh-relay-self-hosting.md | 168 +++++++++++-------- 2 files changed, 101 insertions(+), 74 deletions(-) diff --git a/deploy/iroh-relay/docker-compose.yml b/deploy/iroh-relay/docker-compose.yml index 4ca7a9083..dae8d199c 100644 --- a/deploy/iroh-relay/docker-compose.yml +++ b/deploy/iroh-relay/docker-compose.yml @@ -1,4 +1,4 @@ -# Self-hosted iroh relay — local development / PoC. +# Self-hosted iroh relay for local development. # # Runs the official n0 relay in `--dev` mode: plain HTTP (no TLS) on :3340, # which is exactly what the CLI/app accept for a localhost relay URL. @@ -9,8 +9,9 @@ # CLI : THUNDERBOLT_IROH_RELAY_URL=http://localhost:3340 # App : VITE_IROH_RELAY_URL=http://localhost:3340 (build-time) # -# Production runs the same image WITHOUT `--dev`, with a config file mounted -# (see config.example.toml) and real TLS on 443. Full walkthrough: +# Production runs on Render as a derived GHCR image with the config baked in +# (deploy/docker/iroh-relay.Dockerfile + deploy/config/iroh-relay.toml); the +# VPS-shaped alternative with real TLS is config.example.toml. Full walkthrough: # docs/architecture/iroh-relay-self-hosting.md services: iroh-relay: diff --git a/docs/architecture/iroh-relay-self-hosting.md b/docs/architecture/iroh-relay-self-hosting.md index cc485ddb6..88e7f33df 100644 --- a/docs/architecture/iroh-relay-self-hosting.md +++ b/docs/architecture/iroh-relay-self-hosting.md @@ -1,101 +1,127 @@ # Self-hosting the iroh relay -The CLI↔app bridge (`thunderbolt acp|mcp --transport iroh`) runs on [iroh](https://iroh.computer), -which by default rides on relay servers operated by n0 (iroh's authors). This doc explains what a -relay actually does, what we self-host versus what still comes from n0, the local PoC, and the -production plan. +The CLI↔app bridge (`thunderbolt acp|mcp --transport iroh`) uses +[iroh](https://iroh.computer). Without an override, iroh uses relay servers operated by n0, its +authors. This guide explains the relay's role and how contributors or self-hosters can run one. ## What a relay does (and doesn't) -An iroh connection is QUIC between two endpoints identified by ed25519 keys (NodeIds). Two peers -behind NATs usually can't open a direct UDP path immediately, so every endpoint keeps one -long-lived connection to a **home relay** — an HTTPS server that: +An iroh connection is QUIC between two endpoints identified by ed25519 keys (NodeIds). Peers behind +NATs often cannot open a direct UDP path immediately, so every endpoint maintains a long-lived +connection to a **home relay**, an HTTPS server that: -1. **Forwards encrypted packets** between peers when no direct path exists (launch traffic, or - permanently when hole-punching fails — e.g. the browser, which can't do UDP at all and is - relay-only by design, see `crates/thunderbolt-acp-client`). -2. **Assists hole-punching**: peers exchange candidate addresses through the relay, then attempt a - direct QUIC path and migrate to it when it works. -3. **Names the meeting point**: a connection ticket embeds the NodeId + home-relay URL, so a dialer - knows where to find the peer without any discovery. +1. **Forwards encrypted packets** when no direct path exists. This covers connection startup, + permanent NAT traversal failures, and the browser client in `crates/thunderbolt-acp-client`, + which cannot use UDP and therefore remains relay-only. +2. **Assists hole-punching** by carrying candidate addresses between peers. Native peers then try a + direct QUIC path and migrate to it when possible. +3. **Names the meeting point** because a connection ticket embeds the NodeId and home-relay URL. + Ticket recipients can dial the peer without separate discovery. -What a relay can NOT do: read traffic. Everything it forwards is QUIC end-to-end encrypted to the -peer's NodeId — a hostile relay can drop or delay packets and observe metadata (who talks to whom, -when, how much), but never content. Self-hosting is therefore about **availability and metadata**, -not about payload confidentiality (which we already have). +A relay cannot read forwarded traffic. QUIC encrypts traffic end to end for the peer's NodeId. A +hostile relay can drop or delay packets and observe connection metadata, including participants, +timing, and volume, but cannot read payloads. Self-hosting controls availability and metadata +exposure; payload confidentiality does not depend on relay trust. ## Why self-host -- **Availability**: the n0 public relays are a free best-effort service with per-client rate limits; - they can throttle, change, or disappear. Product traffic needs infrastructure with our SLOs. -- **Metadata privacy**: connection graph + traffic timing of our users stays with us. -- **Control**: access control (token-gate the relay to our clients), rate limits, metrics. +- **Availability**: operate capacity and rate limits appropriate for your deployment instead of + relying on n0's free, best-effort public relays. +- **Metadata privacy**: keep user connection graphs and traffic timing within infrastructure you + control. +- **Control**: configure access tokens, rate limits, logging, and metrics. -What still comes from n0 after the swap: **DNS discovery** (`presetN0` keeps it). Dialing by *bare -NodeId* resolves the peer's current relay via n0's DNS service; dialing by *ticket* doesn't need it -(the relay URL is in the ticket). Our flows exchange tickets, so n0 DNS is a convenience, not a -dependency — replacing it (iroh supports custom DNS/pkarr discovery) is a possible follow-up, -documented here so nobody thinks the relay swap removed every third party. +Relay traffic moves fully to the configured self-hosted relay. DNS discovery for bare-NodeId dials +still queries n0's DNS service because the transport retains `presetN0`. Ticket-based dials, the +normal Thunderbolt flow, never use DNS discovery because each ticket contains its relay URL. A +custom discovery service can remove the remaining n0 DNS dependency; that integration is separate +follow-up work. -## The plumbing already in the code +## Client configuration -Both sides accept a relay override; unset keeps today's n0 default: +Both clients accept a relay override. Leaving it unset keeps the n0 default. -| Side | Variable | Where it lands | -| --- | --- | --- | -| CLI (Bun) | `THUNDERBOLT_IROH_RELAY_URL` (runtime) | `cli/src/iroh/endpoint.ts` `configureTransport` — swaps ONLY the relay in the n0 preset | -| Web app (wasm) | `VITE_IROH_RELAY_URL` (build-time) | `src/acp/iroh/iroh-transport.ts` → `crates/thunderbolt-acp-client` relay-only endpoint | +| Client | Variable | Configuration path | +| -------------- | -------------------------------------- | ------------------------------------------------------------------------------------------ | +| CLI (Bun) | `THUNDERBOLT_IROH_RELAY_URL` (runtime) | `cli/src/iroh/endpoint.ts` `configureTransport`, replacing only the relay in the n0 preset | +| Web app (wasm) | `VITE_IROH_RELAY_URL` (build time) | `src/acp/iroh/iroh-transport.ts` → `crates/thunderbolt-acp-client` relay-only endpoint | -## Local PoC (validated) +## Local development + +Start the official relay in development mode: + +```sh +docker compose -f deploy/iroh-relay/docker-compose.yml up -d +``` + +The compose service runs `n0computer/iroh-relay:v1.0.2 --dev` over plain HTTP on port `3340`. +Configure both clients with `http://localhost:3340`: + +```sh +THUNDERBOLT_IROH_RELAY_URL=http://localhost:3340 thunderbolt iroh id +VITE_IROH_RELAY_URL=http://localhost:3340 bun run dev +``` + +Development mode has no TLS or QUIC address discovery. Use it only on localhost. + +### Verifying your relay + +First, decode a generated endpoint ticket and confirm its embedded relay URL: ```sh -cd deploy/iroh-relay && docker compose up -d # n0computer/iroh-relay:v1.0.2 --dev → plain HTTP :3340 +cd cli +TICKET='' bun -e \ + 'import { EndpointTicket } from "@number0/iroh"; console.log(EndpointTicket.fromString(process.env.TICKET!).endpointAddr().relayUrl())' ``` -Validation run (2026-07-09, iroh 1.x both sides): +The command must print `http://localhost:3340/` and no n0 relay URL. + +Then run a round-trip with two state directories so each process has a distinct identity: -1. `THUNDERBOLT_IROH_RELAY_URL=http://localhost:3340 thunderbolt iroh id` → the printed ticket - decodes to `http://localhost:3340/` as the home relay (no n0 URL present), and `endpoint.online()` - only resolves once the home relay accepted us — proof the override is fully in effect. -2. Full round-trip through the bridge: identity A ran - `thunderbolt acp --transport iroh -- cat`, identity B was allowlisted and dialed A's ticket with - `thunderbolt acp connect `; a JSON-RPC line piped into B came back byte-identical through - A's `cat`. Same-host peers may migrate to a direct path after the relay-mediated handshake — - which is exactly the intended behavior in production too. +1. Set `THUNDERBOLT_HOME=/tmp/thunderbolt-a` for identity A and + `THUNDERBOLT_HOME=/tmp/thunderbolt-b` for identity B. Set + `THUNDERBOLT_IROH_RELAY_URL=http://localhost:3340` for both. +2. Run `thunderbolt iroh id` as identity B and copy its NodeId. +3. Run `thunderbolt iroh allow ` as identity A. +4. Start `thunderbolt acp --transport iroh -- cat` as identity A and copy its ticket. +5. Pipe one JSON-RPC line through + `thunderbolt acp connect ` as identity B. Confirm output is byte-identical. -`--dev` mode = plain HTTP, no TLS, no QUIC address discovery. Fine on localhost, never on the -internet. +Same-host native peers can migrate to a direct path after the relay-mediated handshake. Embedded +relay URL and successful authenticated round-trip verify relay configuration even when migration +occurs. -## Production plan (decision pending) +## Production deployment -The relay is a single small Rust binary (the public n0 relays run on modest VMs; CPU cost is -forwarding ciphertext). What it needs from a host: +This repository ships a Render/GHCR deployment pattern. `.github/workflows/images-publish.yml` builds +`deploy/docker/iroh-relay.Dockerfile`, which embeds `deploy/config/iroh-relay.toml`, and publishes +`ghcr.io/thunderbird/thunderbolt/thunderbolt-iroh-relay:`. A Render image service runs that +stateless image behind Render's TLS-terminating proxy. -- A **domain** (e.g. `relay.thunderbolt.io`) — tickets and configs carry this URL. -- **TCP 80 + 443** terminated by the relay itself (`cert_mode = "LetsEncrypt"` auto-provisions); - port 80 also serves the ACME challenge. -- **UDP 4433** (optional) for QUIC address discovery — the 1.x replacement for STUN. Without it, - relaying still works; peers just lose one address-discovery mechanism. -- Prometheus metrics on :9090 (internal only). +The relay listens for plain HTTP/WebSocket traffic on one container port (`10000`). Render exposes +that port through an HTTPS service URL, so the relay config has no `[tls]` section. `/` serves as the +health-check path. Metrics use a separate listener and remain disabled because Render exposes only +one service port. -Two candidate shapes, in order of fit: +Render does not expose UDP, so this deployment cannot provide relay-side QUIC address discovery. +Encrypted relaying and hole-punch coordination still work, and the browser path remains relay-only. +This shape suits deployments that prioritize managed HTTPS and a single forwarded port. -1. **Small VPS (Hetzner/Fly/EC2)** — full fit: raw 80/443 + UDP, LetsEncrypt built in, ~$5–10/mo. - `config.example.toml` in `deploy/iroh-relay/` is this shape. -2. **Render (current infra)** — partial fit: web services proxy HTTP/TLS (the relay's WebSocket - upgrade works), but no UDP and no self-terminated TLS, so no QUIC address discovery and - `LetsEncrypt` mode is out (Render terminates TLS; the relay would run `dangerous_http_only` - behind it — needs a validation spike before committing). +Self-hosters who need UDP/QUIC discovery can use the VPS-shaped +`deploy/iroh-relay/config.example.toml`: expose TCP 80/443 and UDP 4433 directly, let the relay +terminate TLS with Let's Encrypt, and optionally enable its metrics listener. A small VPS or VM is +sufficient because the relay is stateless and forwards ciphertext without terminating peer QUIC. -Rollout: stand up the relay → bake the URL as the default in the CLI (still overridable by env) and -`VITE_IROH_RELAY_URL` in app builds → mixed fleets keep working because tickets carry the relay URL -of whoever minted them. Lock down with `access.shared_token` once our clients send it. +Set the deployed HTTPS URL as `THUNDERBOLT_IROH_RELAY_URL` at CLI runtime and +`VITE_IROH_RELAY_URL` when building the app. Fleets using different relays remain interoperable +because tickets carry the relay URL chosen by their issuer. To restrict access, configure +`access.shared_token` and append the token query parameter to client relay URLs. ## Operational notes -- **Version coupling**: the relay speaks the iroh relay protocol; keep the server minor-version - aligned with the `iroh`/`@number0/iroh` 1.x clients when bumping either. -- The container logs nothing per-connection at default log level; use `RUST_LOG=info` (or metrics) - when debugging. -- Access control modes: `everyone` (default), `allowlist`/`denylist` (NodeIds), `shared_token`, - or an `http` callback — see `config.example.toml`. +- **Version coupling**: keep the server minor version aligned with the `iroh` and `@number0/iroh` + 1.x clients when upgrading either side. +- Default logging omits per-connection details. Set `RUST_LOG=info` or enable metrics when + debugging. +- Access modes include `everyone`, `allowlist`, `denylist`, `shared_token`, and an HTTP callback. + See `deploy/iroh-relay/config.example.toml` for field examples. From f8aa49b9d2cd2509d0cde470ad15e4d44df64170 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C3=8Dtalo=20Menezes?= Date: Wed, 15 Jul 2026 15:05:23 -0300 Subject: [PATCH 08/12] docs: drop follow-up phrasing from the relay dns note --- docs/architecture/iroh-relay-self-hosting.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/docs/architecture/iroh-relay-self-hosting.md b/docs/architecture/iroh-relay-self-hosting.md index 88e7f33df..cc739270f 100644 --- a/docs/architecture/iroh-relay-self-hosting.md +++ b/docs/architecture/iroh-relay-self-hosting.md @@ -34,8 +34,7 @@ exposure; payload confidentiality does not depend on relay trust. Relay traffic moves fully to the configured self-hosted relay. DNS discovery for bare-NodeId dials still queries n0's DNS service because the transport retains `presetN0`. Ticket-based dials, the normal Thunderbolt flow, never use DNS discovery because each ticket contains its relay URL. A -custom discovery service can remove the remaining n0 DNS dependency; that integration is separate -follow-up work. +custom discovery service can remove the remaining n0 DNS dependency. ## Client configuration From 445e484079058f5afa88ce64ad28e019c9b456e8 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C3=8Dtalo=20Menezes?= Date: Wed, 15 Jul 2026 15:33:02 -0300 Subject: [PATCH 09/12] fix: narrow the model listing fallback to expected io failures --- cli/src/config/model-listing.test.ts | 56 ++++++++++++++++ cli/src/config/model-listing.ts | 99 ++++++++++++++++------------ 2 files changed, 114 insertions(+), 41 deletions(-) diff --git a/cli/src/config/model-listing.test.ts b/cli/src/config/model-listing.test.ts index 5101fb490..4876e1890 100644 --- a/cli/src/config/model-listing.test.ts +++ b/cli/src/config/model-listing.test.ts @@ -198,6 +198,27 @@ describe('listModels', () => { }) }) + test('returns catalog models when fetch rejects with a network TypeError', async () => { + const fetchFn: ModelListingFetch = async () => { + throw new TypeError('Network request failed.') + } + + expect(await listModels({ provider: 'openai', apiKey: 'key', fetchFn })).toEqual({ + source: 'catalog', + ids: ['gpt-4', 'gpt-4-turbo', 'gpt-4.1'], + }) + }) + + test('returns catalog models when response JSON is invalid', async () => { + const fetchFn: ModelListingFetch = async () => + new Response('{"data":', { headers: { 'Content-Type': 'application/json' } }) + + expect(await listModels({ provider: 'openai', apiKey: 'key', fetchFn })).toEqual({ + source: 'catalog', + ids: ['gpt-4', 'gpt-4-turbo', 'gpt-4.1'], + }) + }) + test('returns catalog models for non-success and malformed responses', async () => { const unavailable = await listModels({ provider: 'openai', @@ -214,6 +235,41 @@ describe('listModels', () => { expect(malformed).toEqual({ source: 'catalog', ids: ['gpt-4', 'gpt-4-turbo', 'gpt-4.1'] }) }) + test('treats unrecognized provider responses as empty model lists', async () => { + const providerResponses = [ + ['openai', { data: [{ broken: true }] }], + ['google', { models: [{ broken: true }] }], + ['xai', { models: [{ broken: true }] }], + ['together', [{ broken: true }]], + ] as const + + for (const [provider, response] of providerResponses) { + const result = await listModels({ + provider, + apiKey: 'key', + fetchFn: async () => Response.json(response), + }) + + expect(result.source).toBe('catalog') + } + }) + + test('propagates unexpected errors from model post-processing', async () => { + const unexpectedError = new Error('Unexpected post-processing failure.') + const parsed = new Proxy>({}, { + get: () => { + throw unexpectedError + }, + }) + class PostProcessingResponse extends Response { + override readonly json = async (): Promise => parsed + } + + await expect( + listModels({ provider: 'openai', apiKey: 'key', fetchFn: async () => new PostProcessingResponse() }), + ).rejects.toBe(unexpectedError) + }) + test('marks 401 and 403 catalog fallbacks as authentication rejections', async () => { const unauthorized = await listModels({ provider: 'openai', diff --git a/cli/src/config/model-listing.ts b/cli/src/config/model-listing.ts index 26bb7aaaf..f1346f77a 100644 --- a/cli/src/config/model-listing.ts +++ b/cli/src/config/model-listing.ts @@ -61,23 +61,25 @@ const isRecord = (value: unknown): value is Record => /** Reads OpenAI-compatible `{ data: [{ id, created? }] }` responses. */ const parseOpenAiModels = (value: unknown): readonly ListedModel[] => { - if (!isRecord(value) || !Array.isArray(value.data)) throw new Error('Invalid model list response.') - return value.data.map((candidate) => { - if (!isRecord(candidate) || typeof candidate.id !== 'string') throw new Error('Invalid model entry.') + if (!isRecord(value) || !Array.isArray(value.data)) return [] + return value.data.flatMap((candidate) => { + if (!isRecord(candidate) || typeof candidate.id !== 'string') return [] const created = createdTimestamp(candidate) - return { - id: candidate.id, - ...(created === undefined ? {} : { created }), - } + return [ + { + id: candidate.id, + ...(created === undefined ? {} : { created }), + }, + ] }) } /** Gemini listing schema: https://ai.google.dev/api/models#method:-models.list */ const parseGeminiModels = (value: unknown): readonly ListedModel[] => { - if (!isRecord(value) || !Array.isArray(value.models)) throw new Error('Invalid Gemini model list response.') + if (!isRecord(value) || !Array.isArray(value.models)) return [] return value.models.flatMap((candidate) => { - if (!isRecord(candidate) || typeof candidate.name !== 'string') throw new Error('Invalid Gemini model entry.') - if (!Array.isArray(candidate.supportedGenerationMethods)) throw new Error('Invalid Gemini methods.') + if (!isRecord(candidate) || typeof candidate.name !== 'string') return [] + if (!Array.isArray(candidate.supportedGenerationMethods)) return [] if (!candidate.supportedGenerationMethods.includes('generateContent')) return [] return [{ id: candidate.name.replace(/^models\//, '') }] }) @@ -85,21 +87,19 @@ const parseGeminiModels = (value: unknown): readonly ListedModel[] => { /** xAI language-model schema: https://docs.x.ai/developers/rest-api-reference/inference/models#list-language-models */ const parseXaiModels = (value: unknown): readonly ListedModel[] => { - if (!isRecord(value) || !Array.isArray(value.models)) throw new Error('Invalid xAI model list response.') - return value.models.map((candidate) => { - if (!isRecord(candidate) || typeof candidate.id !== 'string') throw new Error('Invalid xAI model entry.') + if (!isRecord(value) || !Array.isArray(value.models)) return [] + return value.models.flatMap((candidate) => { + if (!isRecord(candidate) || typeof candidate.id !== 'string') return [] const created = createdTimestamp(candidate) - return { id: candidate.id, ...(created === undefined ? {} : { created }) } + return [{ id: candidate.id, ...(created === undefined ? {} : { created }) }] }) } /** Together listing schema: https://docs.together.ai/reference/models */ const parseTogetherModels = (value: unknown): readonly ListedModel[] => { - if (!Array.isArray(value)) throw new Error('Invalid Together model list response.') + if (!Array.isArray(value)) return [] return value.flatMap((candidate) => { - if (!isRecord(candidate) || typeof candidate.id !== 'string' || typeof candidate.type !== 'string') { - throw new Error('Invalid Together model entry.') - } + if (!isRecord(candidate) || typeof candidate.id !== 'string' || typeof candidate.type !== 'string') return [] if (!['chat', 'language', 'code'].includes(candidate.type)) return [] const created = createdTimestamp(candidate) return [{ id: candidate.id, ...(created === undefined ? {} : { created }) }] @@ -114,6 +114,9 @@ const parseListedModels = (provider: ModelProvider, value: unknown): readonly Li return parseOpenAiModels(value) } +/** Identifies fetch failures that should use catalog models. */ +const isExpectedFetchError = (error: unknown): boolean => error instanceof DOMException || error instanceof TypeError + /** Keeps model ids intended for chat rather than specialized media or ranking APIs. */ const chatModels = (models: readonly ListedModel[]): readonly ListedModel[] => models.filter(({ id }) => !NON_CHAT_MODEL_PATTERN.test(id)) @@ -197,10 +200,11 @@ const listingRequest = (provider: ModelProvider, apiKey: string, baseUrl?: strin } } -/** Lists live provider models, returning Pi catalog ids for every failure mode. */ +/** Lists live provider models, returning Pi catalog ids for expected provider failures. */ export const listModels = async (options: ListModelsOptions): Promise => { const fallback = (): ModelListingResult => ({ source: 'catalog', ids: catalogIds(options.provider) }) if (FALLBACK_ONLY_PROVIDERS.has(options.provider)) return fallback() + const request = listingRequest(options.provider, options.apiKey, options.baseUrl) const controller = new AbortController() const timeout = Promise.withResolvers() const timeoutId = setTimeout(() => { @@ -208,29 +212,42 @@ export const listModels = async (options: ListModelsOptions): Promise => { + try { + return await Promise.race([ + (options.fetchFn ?? globalThis.fetch)(request.url, { + headers: request.headers, + signal: controller.signal, + }), + timeout.promise, + ]) + } catch (error) { + if (isExpectedFetchError(error)) return undefined + throw error + } finally { + clearTimeout(timeoutId) + } + })() + if (!response) return fallback() + if (!response.ok) { + if (response.status === 401 || response.status === 403) { + return { ...fallback(), authRejected: true, status: response.status } } - const parsed = (await response.json()) as unknown - const listedModels = parseListedModels(options.provider, parsed) - const models = newestModelsFirst(chatModels(listedModels)) - if (models.length === 0) return fallback() - return { source: 'live', ids: models.slice(0, MAX_LIVE_MODELS).map(({ id }) => id) } - } catch { return fallback() - } finally { - clearTimeout(timeoutId) } + + const parsed = await (async (): Promise => { + try { + return (await response.json()) as unknown + } catch (error) { + if (error instanceof SyntaxError) return undefined + throw error + } + })() + if (parsed === undefined) return fallback() + + const listedModels = parseListedModels(options.provider, parsed) + const models = newestModelsFirst(chatModels(listedModels)) + if (models.length === 0) return fallback() + return { source: 'live', ids: models.slice(0, MAX_LIVE_MODELS).map(({ id }) => id) } } From 54755c66ae17fc57a3dae01d7462c37203fd8da9 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C3=8Dtalo=20Menezes?= Date: Wed, 15 Jul 2026 16:01:40 -0300 Subject: [PATCH 10/12] fix: scope saved openai-compat key to its endpoint, not just provider --- cli/README.md | 7 ++++--- cli/src/cli.test.ts | 32 ++++++++++++++++++++++++++++++++ cli/src/cli.ts | 11 ++++++++--- 3 files changed, 44 insertions(+), 6 deletions(-) diff --git a/cli/README.md b/cli/README.md index 5b48ade41..5043bb039 100644 --- a/cli/README.md +++ b/cli/README.md @@ -95,9 +95,10 @@ Config lives at `~/.thunderbolt/config.json`, or } ``` -`apiKey` and `baseUrl` are optional. Saved keys and base URLs apply only when -saved provider matches effective provider, preventing cross-provider credential -forwarding. Missing, malformed, or invalid config is treated as absent. +`apiKey` and `baseUrl` are optional. Saved keys apply only when saved provider +and base URL match effective provider and base URL, preventing cross-provider or +cross-endpoint credential forwarding. Missing, malformed, or invalid config is +treated as absent. Resolution order is explicit flag, supported provider environment variable, config file, then built-in default. Current environment tier contains credential diff --git a/cli/src/cli.test.ts b/cli/src/cli.test.ts index 113bf22f5..bf277227e 100644 --- a/cli/src/cli.test.ts +++ b/cli/src/cli.test.ts @@ -229,6 +229,38 @@ describe('parseArgs — persisted config precedence', () => { expect(config.apiKey).toBe('saved-key') }) + test('saved openai-compat key is not forwarded when --base-url targets a different endpoint', () => { + const config = runConfig(['--base-url', 'https://other.example/v1'], { config: stored, env: {} }) + + expect(config.baseUrl).toBe('https://other.example/v1') + expect(config.apiKey).toBeUndefined() + }) + + test('saved openai-compat key is used when effective base URL matches saved endpoint', () => { + const config = runConfig(['--base-url', 'https://saved.example/v1'], { config: stored, env: {} }) + + expect(config.baseUrl).toBe('https://saved.example/v1') + expect(config.apiKey).toBe('saved-key') + }) + + test('--api-key remains explicit opt-in when --base-url targets a different endpoint', () => { + const config = runConfig(['--base-url', 'https://other.example/v1', '--api-key', 'flag-key'], { + config: stored, + env: {}, + }) + + expect(config.apiKey).toBe('flag-key') + }) + + test('dedicated env key remains explicit opt-in when --base-url targets a different endpoint', () => { + const config = runConfig(['--base-url', 'https://other.example/v1'], { + config: stored, + env: { THUNDERBOLT_OPENAI_COMPAT_KEY: 'env-key' }, + }) + + expect(config.apiKey).toBe('env-key') + }) + test('saved key and base URL do not cross provider boundaries', () => { const config = runConfig(['--provider', 'anthropic'], { config: stored, env: {} }) diff --git a/cli/src/cli.ts b/cli/src/cli.ts index efd8e4633..e424e72f8 100644 --- a/cli/src/cli.ts +++ b/cli/src/cli.ts @@ -340,18 +340,22 @@ const resolveProvider = (flags: Flags, config: CliConfig | null): ModelProvider * * @param provider - selected model provider * @param flagApiKey - value passed via `--api-key`, if any + * @param effectiveBaseUrl - endpoint selected after flag and config precedence * @returns the resolved api key, or `undefined` */ const resolveApiKey = ( provider: ModelProvider, flagApiKey: string | undefined, + effectiveBaseUrl: string | undefined, dependencies: ResolvedDependencies, ): string | undefined => { if (flagApiKey !== undefined) return flagApiKey if (provider === 'openai-compat') { return ( dependencies.env.THUNDERBOLT_OPENAI_COMPAT_KEY || - (dependencies.config?.provider === provider ? dependencies.config.apiKey : undefined) + (dependencies.config?.provider === provider && effectiveBaseUrl === dependencies.config.baseUrl + ? dependencies.config.apiKey + : undefined) ) } @@ -376,14 +380,15 @@ const resolveBaseUrl = (flags: Flags, provider: ModelProvider, config: CliConfig /** Resolves harness fields after argv scanning preserves explicit flags. */ const resolveAgentFlags = (flags: Flags, dependencies: ResolvedDependencies) => { const provider = resolveProvider(flags, dependencies.config) + const baseUrl = resolveBaseUrl(flags, provider, dependencies.config) return { model: resolveModelId(flags, provider, dependencies.config), cwd: dependencies.cwd, yolo: flags.yolo, thinking: flags.thinking, provider, - baseUrl: resolveBaseUrl(flags, provider, dependencies.config), - apiKey: resolveApiKey(provider, flags.apiKey, dependencies), + baseUrl, + apiKey: resolveApiKey(provider, flags.apiKey, baseUrl, dependencies), } } From d993285ff271e3362330546c4067d39e70d94225 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C3=8Dtalo=20Menezes?= Date: Wed, 15 Jul 2026 16:01:47 -0300 Subject: [PATCH 11/12] test: derive expected catalog fallback ids from the pi catalog --- cli/src/config/model-listing.test.ts | 56 +++++++++++++++------------- 1 file changed, 30 insertions(+), 26 deletions(-) diff --git a/cli/src/config/model-listing.test.ts b/cli/src/config/model-listing.test.ts index 4876e1890..b0a43246e 100644 --- a/cli/src/config/model-listing.test.ts +++ b/cli/src/config/model-listing.test.ts @@ -2,10 +2,17 @@ * License, v. 2.0. If a copy of the MPL was not distributed with this * file, You can obtain one at http://mozilla.org/MPL/2.0/. */ +import { builtinModels } from '@earendil-works/pi-ai/providers/all' import { describe, expect, test } from 'bun:test' import { listModels } from './model-listing.ts' import type { ModelListingFetch } from './model-listing.ts' +/** Derives fallback expectations from Pi's wired catalog so catalog churn does not break behavior tests. */ +const openAiCatalogIds = builtinModels() + .getModels('openai') + .slice(0, 3) + .map(({ id }) => id) + describe('listModels', () => { test('reads an OpenAI-compatible model list with bearer authentication', async () => { const requests: { readonly input: string | URL | Request; readonly init?: RequestInit }[] = [] @@ -191,11 +198,10 @@ describe('listModels', () => { test('returns catalog models on timeout even when injected fetch ignores abort', async () => { const fetchFn: ModelListingFetch = async () => new Promise(() => {}) + const result = await listModels({ provider: 'openai', apiKey: 'key', fetchFn, timeoutMs: 1 }) - expect(await listModels({ provider: 'openai', apiKey: 'key', fetchFn, timeoutMs: 1 })).toEqual({ - source: 'catalog', - ids: ['gpt-4', 'gpt-4-turbo', 'gpt-4.1'], - }) + expect(result.source).toBe('catalog') + expect(result.ids).toEqual(openAiCatalogIds) }) test('returns catalog models when fetch rejects with a network TypeError', async () => { @@ -203,20 +209,20 @@ describe('listModels', () => { throw new TypeError('Network request failed.') } - expect(await listModels({ provider: 'openai', apiKey: 'key', fetchFn })).toEqual({ - source: 'catalog', - ids: ['gpt-4', 'gpt-4-turbo', 'gpt-4.1'], - }) + const result = await listModels({ provider: 'openai', apiKey: 'key', fetchFn }) + + expect(result.source).toBe('catalog') + expect(result.ids).toEqual(openAiCatalogIds) }) test('returns catalog models when response JSON is invalid', async () => { const fetchFn: ModelListingFetch = async () => new Response('{"data":', { headers: { 'Content-Type': 'application/json' } }) - expect(await listModels({ provider: 'openai', apiKey: 'key', fetchFn })).toEqual({ - source: 'catalog', - ids: ['gpt-4', 'gpt-4-turbo', 'gpt-4.1'], - }) + const result = await listModels({ provider: 'openai', apiKey: 'key', fetchFn }) + + expect(result.source).toBe('catalog') + expect(result.ids).toEqual(openAiCatalogIds) }) test('returns catalog models for non-success and malformed responses', async () => { @@ -231,8 +237,10 @@ describe('listModels', () => { fetchFn: async () => Response.json({ unexpected: [] }), }) - expect(unavailable).toEqual({ source: 'catalog', ids: ['gpt-4', 'gpt-4-turbo', 'gpt-4.1'] }) - expect(malformed).toEqual({ source: 'catalog', ids: ['gpt-4', 'gpt-4-turbo', 'gpt-4.1'] }) + expect(unavailable.source).toBe('catalog') + expect(unavailable.ids).toEqual(openAiCatalogIds) + expect(malformed.source).toBe('catalog') + expect(malformed.ids).toEqual(openAiCatalogIds) }) test('treats unrecognized provider responses as empty model lists', async () => { @@ -282,18 +290,14 @@ describe('listModels', () => { fetchFn: async () => new Response(null, { status: 403 }), }) - expect(unauthorized).toEqual({ - source: 'catalog', - ids: ['gpt-4', 'gpt-4-turbo', 'gpt-4.1'], - authRejected: true, - status: 401, - }) - expect(forbidden).toEqual({ - source: 'catalog', - ids: ['gpt-4', 'gpt-4-turbo', 'gpt-4.1'], - authRejected: true, - status: 403, - }) + expect(unauthorized.source).toBe('catalog') + expect(unauthorized.ids).toEqual(openAiCatalogIds) + expect(unauthorized.authRejected).toBe(true) + expect(unauthorized.status).toBe(401) + expect(forbidden.source).toBe('catalog') + expect(forbidden.ids).toEqual(openAiCatalogIds) + expect(forbidden.authRejected).toBe(true) + expect(forbidden.status).toBe(403) }) test('treats an empty chat-capable result as catalog fallback', async () => { From 80ac90b58afa461def1c634ddcc7379e32c7bd63 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C3=8Dtalo=20Menezes?= Date: Wed, 15 Jul 2026 16:30:26 -0300 Subject: [PATCH 12/12] docs: drop the production deployment section until its artifacts land --- deploy/iroh-relay/docker-compose.yml | 6 ++--- docs/architecture/iroh-relay-self-hosting.md | 26 -------------------- 2 files changed, 2 insertions(+), 30 deletions(-) diff --git a/deploy/iroh-relay/docker-compose.yml b/deploy/iroh-relay/docker-compose.yml index dae8d199c..e163ac570 100644 --- a/deploy/iroh-relay/docker-compose.yml +++ b/deploy/iroh-relay/docker-compose.yml @@ -9,10 +9,8 @@ # CLI : THUNDERBOLT_IROH_RELAY_URL=http://localhost:3340 # App : VITE_IROH_RELAY_URL=http://localhost:3340 (build-time) # -# Production runs on Render as a derived GHCR image with the config baked in -# (deploy/docker/iroh-relay.Dockerfile + deploy/config/iroh-relay.toml); the -# VPS-shaped alternative with real TLS is config.example.toml. Full walkthrough: -# docs/architecture/iroh-relay-self-hosting.md +# See docs/architecture/iroh-relay-self-hosting.md for the relay's role and +# self-hosting guidance. services: iroh-relay: image: n0computer/iroh-relay:v1.0.2 diff --git a/docs/architecture/iroh-relay-self-hosting.md b/docs/architecture/iroh-relay-self-hosting.md index cc739270f..8a7b5157d 100644 --- a/docs/architecture/iroh-relay-self-hosting.md +++ b/docs/architecture/iroh-relay-self-hosting.md @@ -90,32 +90,6 @@ Same-host native peers can migrate to a direct path after the relay-mediated han relay URL and successful authenticated round-trip verify relay configuration even when migration occurs. -## Production deployment - -This repository ships a Render/GHCR deployment pattern. `.github/workflows/images-publish.yml` builds -`deploy/docker/iroh-relay.Dockerfile`, which embeds `deploy/config/iroh-relay.toml`, and publishes -`ghcr.io/thunderbird/thunderbolt/thunderbolt-iroh-relay:`. A Render image service runs that -stateless image behind Render's TLS-terminating proxy. - -The relay listens for plain HTTP/WebSocket traffic on one container port (`10000`). Render exposes -that port through an HTTPS service URL, so the relay config has no `[tls]` section. `/` serves as the -health-check path. Metrics use a separate listener and remain disabled because Render exposes only -one service port. - -Render does not expose UDP, so this deployment cannot provide relay-side QUIC address discovery. -Encrypted relaying and hole-punch coordination still work, and the browser path remains relay-only. -This shape suits deployments that prioritize managed HTTPS and a single forwarded port. - -Self-hosters who need UDP/QUIC discovery can use the VPS-shaped -`deploy/iroh-relay/config.example.toml`: expose TCP 80/443 and UDP 4433 directly, let the relay -terminate TLS with Let's Encrypt, and optionally enable its metrics listener. A small VPS or VM is -sufficient because the relay is stateless and forwards ciphertext without terminating peer QUIC. - -Set the deployed HTTPS URL as `THUNDERBOLT_IROH_RELAY_URL` at CLI runtime and -`VITE_IROH_RELAY_URL` when building the app. Fleets using different relays remain interoperable -because tickets carry the relay URL chosen by their issuer. To restrict access, configure -`access.shared_token` and append the token query parameter to client relay URLs. - ## Operational notes - **Version coupling**: keep the server minor version aligned with the `iroh` and `@number0/iroh`