fix(auth): rotate API-key model pools for all configured providers - #208
fix(auth): rotate API-key model pools for all configured providers#208warelik wants to merge 2 commits into
Conversation
Previously, only openai-compatibility builds resolved a multi-model alias pool and applied nextModelPoolOffset rotation. Claude, Gemini, Codex, xAI, and Vertex API-key configs fell through to applyAPIKeyModelAlias and used only the first resolved model, losing sibling-model failover and cache-warm opportunities. Resolve the API-key alias pool for every provider whose config carries Models [] by reusing resolveModelAliasPoolFromConfigModels and the existing nextModelPoolOffset / rotateStrings machinery. Keep the OpenAI-compat pool key behavior; add an apiKeyModelPoolKey for other providers so rotation is per-auth and per-alias. filterExecutionModels continues to drop currently-blocked pool members. Add tests covering multi-model alias rotation, suffix preservation, blocked member filtering, and single-model config behavior. Refs: MISSION-CLIPROXY.md Gap 5
predictedHomeConcurrencyModel only consulted resolveOpenAICompatUpstreamModelPool, so legacy Home responses with a multi-model Claude/Gemini/Codex/xAI/Vertex API-key alias would predict a single stable upstream from the first pool member and allow the Home selection to be retained. Subsequent messages could then rotate to a different upstream while reusing the original Home concurrency scope. Use resolveAPIKeyUpstreamModelPool for the prediction and reject when it contains multiple members, matching the existing OpenAI-compat behavior. Add tests for multi-model pool rejection and single-model acceptance. Refs: router-for-me/CLIProxyAPI#5143 (comment)
eaa67f4 to
f8aabc1
Compare
warelik
left a comment
warelik
left a comment
There was a problem hiding this comment.
Requesting changes — one finding.
The pool generalization in conductor_models.go itself is correct: resolveAPIKeyUpstreamModelPool resolves the same config entry for openai-compat auths as the old helper (via the default branch of configuredModelAliasEntries), the modelPoolKey dispatch preserves existing rotation-state keys for openai-compat, and single-model / no-alias-match requests produce exactly the old results. The new tests cover rotation, suffix preservation, blocked-member filtering, and the single-model regression.
One gap: predictedHomeConcurrencyModel (sdk/cliproxy/auth/conductor_home.go:375) still resolves pools via m.resolveOpenAICompatUpstreamModelPool. With this change, claude/gemini/codex/xai/vertex multi-model pools now rotate at execution time, but the Home prediction path falls through to applyAPIKeyModelAlias, which always resolves the first pool member. For openai-compat multi-model pools that function deliberately declines to predict (return "", false) precisely because the executed member is unpredictable. selection.accountedModel (used when Home supplies no lease envelope, conductor_home.go:461) will therefore mis-attribute every rotated request that lands on a non-first member. Swapping that one call site to m.resolveAPIKeyUpstreamModelPool keeps openai-compat behavior identical and fixes the mismatch.
warelik
left a comment
warelik
left a comment
There was a problem hiding this comment.
Approving — the follow-up commit resolves the previous finding.
predictedHomeConcurrencyModel (sdk/cliproxy/auth/conductor_home.go:375) now uses resolveAPIKeyUpstreamModelPool, so multi-model API-key pools on claude/gemini/codex/xai/vertex decline prediction (return "", false) exactly as openai-compat pools already did, and single-model pools still resolve to the configured upstream name. Behavior for openai-compat auths is unchanged: configuredModelAliasEntries resolves the same compat entry as the old helper. The two new tests (multi-model decline, single-model accept) match validCanonicalHomeConcurrencyModelKey semantics.
Rest of the change stands as previously reviewed: pool resolution and rotation generalization is behavior-preserving for single-model and openai-compat paths, with good coverage of rotation, suffix preservation, and blocked-member filtering. LGTM.
1 participant
Fixes Gap 5 from failover-auth-audit.md.
Previously only openai-compatibility resolved multi-model alias pools; Claude, Gemini, Codex, xAI, and Vertex API-key providers used only the first resolved model. This change resolves the API-key alias pool for every provider whose config carries Models [] and reuses the existing
nextModelPoolOffsetrotation, while keepingfilterExecutionModelsdropping blocked members.resolveAPIKeyUpstreamModelPoolandmodelPoolKey.executionModelCandidatesandexecutionModelCandidatesWithAlias.conductor_models_test.gowith multi-model rotation, suffix preservation, blocked-member filtering, and single-model regression tests.Cross-link: router-for-me/CLIProxyAPI#5143