feat(bin): add codex-only ultra reasoning effort to spawn profiles - #3109
feat(bin): add codex-only ultra reasoning effort to spawn profiles#3109gksbrandon wants to merge 7 commits into
Conversation
Confidence Score: 5/5The PR appears safe to merge. No blocking failure remains. Reviews (2): Last reviewed commit: "chore(ci): apply CI fixes" | Re-trigger Greptile |
|
Speaking as Kun's firstmate: Reviewed HEAD Class: opt-in. Extends the accepted effort vocabulary with Codex-only VISION (per rule, inspected evidence):
First-time fork CI: after that full-diff review I approved workflow runs Not merging. Waiting on the author for a proper no-mistakes pipeline signature (and a gate-green attestation body). Not with the captain. Merge-eligible: NO. |
|
Speaking as Kun's firstmate: Reviewed HEAD Class: opt-in. VISION (per rule, inspected evidence):
Attestation: Waiting on the author, not the captain: re-push via Merge-eligible: NO. |
|
Closing this; we're carrying this change on our own fork instead. |
2 participants
Intent
Teach Firstmate's spawn layer the Codex-only ultra reasoning effort so gpt-5.6-sol lanes can use OpenAI's ultra tier. Codex CLI 0.149.1's embedded schema accepts ultra, while the current PATH codex is nix-pinned 0.133.0 and does not support it until the separately owned dotfiles rebuild; do not change PATH, Nix, or perform that cutover. Pi has no ultra concept and its ladder ends at max. Extend fm-spawn's accepted effort vocabulary and emit -c model_reasoning_effort="ultra" only for harness=codex, while every non-Codex harness keeps its existing accepted effort set and follows the existing unsupported-effort behavior by recording and omitting the value or refusing it according to the current path. Ensure fm-control relaunch passes ultra through, including relevant local and remote secondmate spawn routes. Bootstrap crew-dispatch harness and effort validation must accept codex+ultra and reject ultra elsewhere. Do not hard-enforce the gpt-5.6-sol pairing in code because that sol-only restriction is a moving vendor fact and Codex already rejects an invalid model-effort pairing at launch; state it in documentation. The canonical operator docs must warn that ultra requires codex-cli 0.149.1 or newer on PATH and fails until the currently pinned CLI is rebuilt. The harness-adapters Codex row must retain prior codex-cli 0.142.1 low through xhigh evidence alongside 0.149.1 ultra schema evidence and the current 0.133.0 limitation. Explicitly distinguish Firstmate's Codex-only ultra profile value from Muse's unrelated native ultra level: direct --effort ultra on Muse is recorded and omitted, while shared max maps to Muse-native ultra. Add executable behavioral regressions for Codex acceptance and emission, non-Codex omission or refusal, bootstrap validation, relaunch, and the remote route. Keep general remote profile-axis coverage in a normally-run dedicated suite rather than the trace-context suite, and make it hermetic with a remote-path Pi stub instead of any host executable. Keep the Codex emission comment explicit about the 0.149.1 evidence and current pinned 0.133.0 caveat, and disambiguate the Muse-native level at its emission site. Keep all touched scripts shellcheck-clean. Codex ultra uses token-intensive internal sub-agent decomposition and requires current explicit captain approval before selection. Deliver through the captain-created gksbrandon/firstmate fork while opening the cross-repository pull request against kunchenguid/firstmate, leaving origin on upstream.
What Changed
Effort vocabulary now accepts
ultraacrossbin/fm-spawn.sh,bin/fm-control.shrelaunch (including theconfig/secondmate-harnesspin),bin/fm-remote-secondmate-control.sh, andcrew_dispatch_validateinbin/fm-bootstrap.sh. Onlyharness=codexemits-c model_reasoning_effort="ultra"; every other harness keeps its existing accepted set, so a non-Codex profile recordseffort=ultrain task meta and omits the launch flag, and bootstrap rejects pairings likeclaude:ultra.Docs record the CLI floor and disambiguate Muse.
docs/configuration.mdstates thegpt-5.6-solpairing and warns thatultraneeds codex-cli 0.149.1 or newer, failing on the currently nix-pinned 0.133.0. The harness-adapters Codex row keeps its 0.142.1 low-through-xhigh evidence alongside the 0.149.1ultraschema, adds a captain-approval requirement before selectingultra, and separates Muse's nativeultralevel (reached only by mapping an explicitmax) from this Codex-only profile value.New behavioral regressions. Adds
tests/fm-remote-secondmate-profile-axes.test.sh, registered inbin/fm-test-run.shand made hermetic with a remote-pathpistub, covering the full parent-to-remote-pane chain for codexultra, pimax, pi record-and-omit, and refusal of an unverified token. Existing suites gain cases for codex emission, claude and muse omission, bootstrap validation, and both direct and configured-pin relaunch pass-through.Risk Assessment
✅ Low: The change is an additive, well-bounded effort-vocabulary extension with exactly one new emission site gated on harness=codex; every non-Codex harness keeps its prior accepted set and record-and-omit behavior, all touched scripts are shellcheck-clean, the docs carry the required 0.149.1/0.133.0 and gpt-5.6-sol warnings, and executable regressions cover Codex acceptance and emission, non-Codex omission and refusal, bootstrap validation, both relaunch routes, and the hermetic remote route.
Testing
Ran the six targeted suites that own this change (spawn dispatch profile, the new remote profile-axes suite, secondmate harness, control relaunch, muse harness, bootstrap) and all passed with no failures or gate skips. Because pass/fail alone does not show the behavior, I also produced three operator transcripts that drive the real scripts against fake tmux and herdr backends and read back the literal launch command each pane receives: codex emits
-c 'model_reasoning_effort="ultra"'locally, throughfm-control relaunch, and across the SSH boundary on the remote secondmate route; claude, muse and pi recordeffort=ultrain durable task metadata and omit the flag; muse's native max-class level stays reachable only via--effort max; bootstrap accepts codex+ultra and namesclaude:ultra,muse:ultra,pi:ultraandgrok:ultraas invalid; and an unverified remote effort is refused before any remote launch. I confirmed the new remote suite is registered in the normally-run portable-serial lane and that the runner's coverage guard is complete. The change is CLI and documentation only with no rendered surface, so there is no screenshot or GIF; the pane launch literals are the end-user-visible artifact. Documentation edits (the codex-cli 0.149.1 floor, the 0.133.0 pinned-CLI caveat, the retained 0.142.1 evidence, and the muse-vs-codex ultra disambiguation) are prose and were reviewed rather than executed;codex --versionon this host returns 0.133.0, matching the documented caveat. Per the phase rules I did not run linters, so the shellcheck-clean requirement is unverified here and belongs to the lint phase. The worktree is clean, with all evidence written outside it.Evidence: Local spawn and bootstrap transcript: codex ultra emission, non-codex record-and-omit, muse disambiguation, crew-dispatch validation
Source: Local spawn and bootstrap transcript: codex ultra emission, non-codex record-and-omit, muse disambiguation, crew-dispatch validation
Evidence: fm-control relaunch transcript: explicit ultra and a durable codex+ultra secondmate pin reaching the replacement launch
Source: fm-control relaunch transcript: explicit ultra and a durable codex+ultra secondmate pin reaching the replacement launch
Evidence: Remote secondmate route transcript: the launch literal the remote pane received for codex ultra, pi max, pi ultra, and a refused unverified effort
Source: Remote secondmate route transcript: the launch literal the remote pane received for codex ultra, pi max, pi ultra, and a refused unverified effort
Evidence: Reproduction driver: local spawn and bootstrap transcript
Source: Reproduction driver: local spawn and bootstrap transcript
Evidence: Pinned PATH codex version on this host, matching the documented caveat
Pipeline
✅ **intent** - passed
✅ No issues found.
✅ **Rebase** - passed
✅ No issues found.
bin/fm-spawn.sh:1402- bin/fm-spawn.sh:1402 emits -c model_reasoning_effort="ultra" with no CLI capability guard, so on the currently nix-pinned codex 0.133.0 the pane's codex exits at launch. Concrete path: config/secondmate-harness pinned to 'codex gpt-5.6-sol ultra' -> fm-spawn.sh:1292 accepts the token -> effort_flag_for_harness emits ultra -> codex 0.133.0 rejects the unknown value and the pane dies -> secondmate_liveness_one (bin/fm-bootstrap.sh:753) sees dead/missing and respawns through fm-spawn.sh --secondmate, which re-resolves the same pin, so the secondmate never comes up and each session start reports another relaunch. This is explicitly authorized containment: the intent says ultra 'fails until the currently pinned CLI is rebuilt' and requires only that the docs warn, which docs/configuration.md:341 does. Recorded as a tradeoff, not a defect. If a spawn-time refusal is ever preferred over a launch-time one, the codebase already has the pattern in pi_supports_tui_mode (bin/fm-spawn.sh:1109), which probes the resolved executable's --help before composing a version-dependent flag.tests/fm-remote-secondmate-profile-axes.test.sh:50- tests/fm-remote-secondmate-profile-axes.test.sh:50-163 duplicates roughly 110 lines of remote fixture setup verbatim from tests/fm-remote-secondmate-trace-context.test.sh:37-155: the tar of the code root into REMOTE_ROOT, the whole bin/tmux stub (has-session/list-windows/new-window/display-message/capture-pane/send-keys/kill-window/list-panes), the fake-ssh wrapper including its base64 doctor-readiness shortcut, and the remote_env variable wrapper. tests/remote-herdr-fixture.sh is already the shared fixture module both suites source and is the natural owner for this. As written, a change to the tmux or ssh stub (for example a new tmux subcommand the production code starts calling) has to be made in two files, and fixing only one leaves the other suite silently exercising a stale boundary. Extracting the common setup into tests/remote-herdr-fixture.sh, parameterized by state/log paths, keeps both suites separate as the intent requires while giving the boundary one owner.✅ **Test** - passed
✅ No issues found.
bin/fm-test-run.sh tests/fm-spawn-dispatch-profile.test.shbin/fm-test-run.sh tests/fm-remote-secondmate-profile-axes.test.shbin/fm-test-run.sh tests/fm-secondmate-harness.test.shbin/fm-test-run.sh tests/fm-control-relaunch.test.shbin/fm-test-run.sh tests/fm-muse-harness.test.shbin/fm-test-run.sh tests/fm-bootstrap.test.shbin/fm-test-run.sh --check-coverageand--list --lane portable-serialto confirm the new remote suite is registered in a normally-run laneManual operator transcript driving the realbin/fm-spawn.shandbin/fm-bootstrap.shagainst a fake tmux backend that records the literal pane launch command (codex ultra, codex ultra with a non-sol model, claude ultra, muse max vs muse ultra, and six crew-dispatch validation cases)Manual operator transcript driving the realbin/fm-control.shrelaunch for an explicit--harness codex --model gpt-5.6-sol --effort ultraand for a durableconfig/secondmate-harnesscodex+ultra pinManual operator transcript driving the real remote chain (parentfm-spawn->fm-on->fm-remote-entrypoint.sh->fm-remote-secondmate-control.sh-> the remote host's ownfm-spawn) against a fake herdr CLI, reading back the launch literal the remote pane receivedcodex --versionon this host to confirm the documented nix-pinned 0.133.0 caveatdocs/configuration.md:343- The moving vendor fact 'PATH codex is nix-pinned at 0.133.0' now lives in two documents: this operator warning and the harness-adapters Codex row. I kept both because the user intent requires each one explicitly (operator warning about the codex-cli 0.149.1 floor, adapter row retaining the 0.133.0 limitation alongside its version evidence), and they serve different audiences. Follow-up when the dotfiles rebuild lands: both places must be updated together, since neither is a pointer to the other.AGENTS.md:214- AGENTS.md's condensed effort-fallback line names only the 'never max without explicit captain preference' guard, not the new ultra approval gate. I left it unchanged: the sentence already names harness-adapters as the owner, its ladder stops at xhigh so an agent following it alone cannot reach ultra, and adding the gate would duplicate owner prose into always-loaded guidance. Flagging it in case the captain wants the token-spend gate visible in the always-loaded contract too.✅ **Lint** - passed
✅ No issues found.
✅ **Push** - passed
✅ No issues found.