Skip to content

feat: add experimental Codex CLI web search - #322

Merged
joshavant merged 72 commits into
mainfrom
feat/codex-standalone-search
Oct 23, 2026
Merged

joshavant merged 72 commits into
mainfrom
feat/codex-standalone-search

Conversation

@joshavant

@joshavant joshavant commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

The setup helper currently disables Codex CLI web search, and ClawRouter has no qualified standalone search route. Add experimental text search/navigation with caller-isolated sessions, one qualified public OpenAI API account and an explicit search-specific router price. New eligible CLI profiles default to cached search; saved profile modes are preserved and Desktop search stays disabled.

Changes

  • Forward bounded search/open/click/find requests through the standalone OpenAI endpoint. Preserve upstream responses, opaque references and tool errors; bind session ownership to the caller, policy and upstream account without search account failover.
  • Require an explicit requestCostsMicros["search.web"] entry in both catalog discovery and dispatch. Existing general request prices cannot activate standalone search. Zero remains a valid explicit router price; removing the entry disables new and resumed search sessions. Other capability/inference pricing retains its existing behavior.
  • Gate eligibility on declared models and a single public API account. Setup also requires a native Responses Lite descriptor and Codex CLI 0.163.0 or newer; empirical native qualification is pinned to 0.163.0. Eligible new CLI profiles still default to cached search.
  • Add capability-specific request prices and admission-time tariff freezing. Admin inputs use dollars with exact conversion to existing integer-microdollar API/storage values.
  • Define the separately authorized production pilot: $0.001 per standalone request, token-priced inference, a separate $25 shared monthly router allowance, Sol/Astra only, retention off and no account fallback. Activation and live qualification passed as recorded below; existing policies and shared provider limits remain unchanged.
  • Retain bounded qualification fixtures and archived experiment/cleanup records. Guarded FakeCo teardown checks both empty queues, disconnects/verifies the exact owned consumer before Worker deletion, and preserves Access on earlier failures.

Verification

Refreshed against current main, including the deployment asset-gate fix and newer usage reporting.

  • pnpm check: 3,761 passed (2,575 Worker, 236 admin, 950 script tests); 18 optional native-binary tests skipped. Worker and admin TypeScript checks passed.
  • Local workerd search regression, included in the script suite: WebSocket inference, model switching, compaction transcript/reference reuse and SQL accounting passed.
  • New regression coverage rejects absent/legacy-only tariffs on native and manifest routes, including uncapped policies with zero/one/1,000-microdollar general prices. Catalog/dispatch agree on explicit zero/nonzero search prices; removing the tariff blocks resumed and new sessions without additional spend. Inference pricing and in-flight settlement remain covered.
  • pnpm --dir admin build: passed, with nonfatal vendor module-directive warnings.
  • Incremental commit d0843e595d5acbb509493a56cc837c2829bc618f: one-pass shared autoreview completed scoped-clean through P2, no findings, final source verification passed. The reviewer assessed supplied evidence and did not rerun tests.
  • Earlier full-branch review completed seven passes through P2 with no findings on the recorded pre-redaction candidate; historical full build, 22-provider smoke-plan, pricing browser and local Worker lifecycle checks remain documented in the handoff. They are distinct from this incremental review and current check run.

Remote CI and CodeQL passed for the published head. Worker/package required one retry: its first attempt exceeded an unchanged Realtime ASR authority timing assertion (327 ms versus 250 ms); the focused case passed locally at 86 ms and the same commit passed the CI retry. No runtime code or timing limit was changed for that retry.

The pre-merge local check did not perform new paid-provider or native-binary qualification. The 18 skipped cases require an explicitly selected native CLI binary. The separately authorized production rollout and native pilot below add live inference, search, transport and accounting evidence for the recorded CLI/models/mode; they do not qualify every native client combination.

Deployment status

Merged as 7e1ddb4f73ca719a018264dfee8e9b99961d5cd4; its tree matches the reviewed head. All seven exact-main CI jobs and CodeQL passed.

The production zero-smoke rehearsal passed and restored blue. The separately authorized full gradual rollout then passed 0%, 10%, 50% and 100% gates, with 12 live OpenAI inference/usage checks. Version 2e16e6f4-bbee-4fb4-8297-401f11791bc4 now serves 100%; rollback version 1e828d8d-b8ab-43fd-a059-7780d915d1db remains at 0%. The final comparative analytics sample had 260 blue requests and 322 green requests, with zero runtime errors in either sample.

Independent readback confirmed the owned deployment history, production health and version, all eight candidate assets, preserved-secret lineage, no remaining temporary tail sessions and an empty dead-letter queue. The live usage queue continued processing normally; its post-pilot sample held one message. Queue version samples remain observations, not a platform guarantee.

The dedicated production pilot was separately approved and activated with the proposed $0.001 per standalone request, token-priced inference, $25 shared monthly cap, retention off and no account failover/fallback. All 14 existing policies matched their pre-pilot values on canonical readback; no user/group bindings or provider-wide limits were changed. One operator credential expires October 24 at 21:00 UTC and is stored in the authorized password vault. A prematurely submitted model-unrestricted draft was corrected to Sol/Astra before any pilot credential, binding or paid pilot request existed.

The one-shot production pilot passed on native Codex CLI 0.163.0 in cached mode: Sol to Astra model switch, strict WebSocket inference without HTTP inference fallback, search/open/click/find, source citation, one search namespace and original-reference reuse through two automatic compactions. Sixteen successful receipts (five standalone search requests and eleven inference/control operations) reconciled exactly with the dedicated policy authority and usage report; every receipt recorded content retention off. Router charge: $0.978612 ($0.005 search plus $0.973612 inference), leaving $24.021388. The shared provider ledger was not independently inspected, and this is not a provider invoice measurement. No paid retry or new hosting/retrieval experiment ran.

The private operational harness adapted its global-retention expectation to production's unchanged default; the policy was independently verified with retention disabled and per-response/receipt checks remained enforced. Its one-pass static autoreview found a credential-validation diagnostic disclosure; the boolean guard was fixed and malformed-input redaction verified before reading the key. Review assessment completed, but report publication failed because its output directory was absent; the streamed finding and fix evidence were retained locally. Production application source was not changed for the harness.

Historical FakeCo resources and dedicated credentials remain removed. Broader policy enablement requires a separate pricing/cap decision. The unrelated deployment issue was fixed in #323, with its own successful rollout.

Remaining risk

Fresh-origin retrieval, strict cache-only behavior and upstream Alpha billing remain unqualified. The router price is per standalone HTTP request, including HTTP-200 tool errors, and is not an upstream invoice estimate. The $25 pilot cap is a separate combined search/inference router allowance, not an upstream invoice cap or a shared balance with an existing policy. An HTTP 200 or native completion does not establish retrieval success.

Desktop search, media/feed commands, non-Lite models and account-pool failover remain outside qualification. Historical native evidence covers eight recorded models and deployed Sol/Astra runs, not every later CLI/model/mode/transport combination. Complete account qualification must use the actual authority-backed guard; grant inventory/readiness alone is insufficient. Broader enablement requires deliberate policy pricing and budget decisions.

Evidence: production pilot proposal, setup guide, final historical handoff, qualification assessment, verified cleanup. Private live-production audit receipts remain local and are not included in this PR.

Preflight both managed queues and direct consumer identity before writes. Verify consumer absence before Worker deletion and recheck empty queues before removal. Stop on API/refusal failures with Access last; reject malformed acknowledgements.

Validation: 55 focused teardown, deployment and bootstrap tests pass. Shared autoreview: scoped-clean through P2 (gpt-6.1-sol, high). Local-only change; no live Cloudflare mutations or teardown replay.
Seven-pass GPT-6.1 Sol high review of 1a359ae against pinned main c130b15: scoped-clean through P2, zero findings, final source verification passed. Record generated-snapshot review limits and separate local validation. Runtime unchanged.
@clawsweeper

clawsweeper Bot commented Oct 23, 2026 •

Copy link
Copy Markdown

🦞👀
ClawSweeper picked this up.

Pull request received. I will update this pull request when review starts.

ClawSweeper review blocked

Automated review did not run, so no review verdict was produced.

Reason: The input-safety check rejected material in this revision. No detected value, path, or scanner output is reproduced here.

Automatic review is on hold, including after source changes. Request a fresh re-review after resolving the failure; maintainers can also release the hold through an intentional scanner-policy update.

Next step: If this is a genuine credential, remove and rotate it. If it is an intentional test fixture, a maintainer must review and qualify it.

View the workflow run.

Comment thread scripts/search-semantics-funnel.mjs Fixed
Comment thread test/responses-search-probe.test.mjs Fixed
@clawsweeper clawsweeper Bot added the clawsweeper:intake:feature Preliminary submission classification; full source review pending. label Oct 23, 2026
Replace the historical fixture host with a reserved invalid example and mark the archived request examples as redacted. Keep live experiments gated on a separately approved hostname. Focused Funnel and endpoint-comparison tests: 41 pass.
@joshavant
joshavant force-pushed the feat/codex-standalone-search branch from e81be67 to d52e3c3 Compare October 23, 2026 16:41
@joshavant
joshavant merged commit 7e1ddb4 into main Oct 23, 2026
25 of 27 checks passed
@joshavant
joshavant deleted the feat/codex-standalone-search branch October 23, 2026 20:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

clawsweeper:intake:feature Preliminary submission classification; full source review pending.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants