Pre-emptively de-prefer a Claude credential on allowed_warning

[chansearrington]

Pre-emptively de-prefer a Claude credential on allowed_warning

Today a Claude credential is only de-preferred after Anthropic rejects a request: something has
to fail before the router reacts, so a caller sees a real error on every window transition.

Anthropic already tells us in advance. On successful responses it sends
Anthropic-Ratelimit-Unified-{5h,7d,7d_oi}-Status: allowed_warning before it starts rejecting. This
PR reads that signal and starts preferring another credential while the warned one is still serving.

Behaviour

• Per Claude credential, per window (5h, 7d, 7d_oi): a warning mark is set when a successful
  response's Anthropic-Ratelimit-Unified-<window>-Status is exactly allowed_warning, carrying
  that window's -Reset epoch when present.
• The mark clears when that window's own status is exactly allowed, when the unsuffixed aggregate
  status reports allowed, or when the Reset epoch passes.
• The selector prefers enabled, non-exhausted, non-warned credentials.
• 🔴 It never starves. If every candidate is warned, the warned credential is still served from.
  A warning is a soft preference, never a block — no caller ever sees an error that the router
  could have avoided by just using the warned credential.
• A disabled credential is never a candidate. The existing per-model 429 quota marks keep their
  precedence; this sits above them as an earlier, softer signal.
• One log line per transition, naming only an 8-character credential fingerprint prefix — never a
  label, email, token, or full id.

Why per-window, and not just the aggregate status

The unsuffixed Anthropic-Ratelimit-Unified-Status is worst-of-windows. Measured against a
real production ledger: 5h warned on 25 responses and 7d_oi on 236, and the aggregate carried a
warning on exactly 261 — the sum, with no overlap. So the aggregate tells you something is warned
but never which, and it cannot clear one window while another is still warning.

That matters in practice, not just in theory. In one observed event the 7d_oi window read
allowed_warning continuously for 3h43m while 5h stayed allowed the entire time. An
implementation keyed on 5h alone, or on the aggregate alone, would have been inert for that whole
window.

A defect this PR fixes in its own first commit

The first commit collected only allowed_warning and treated every other value — including an
explicit allowed — the same as an absent header. Because the only other clear path was the
worst-of-windows aggregate, a window that had recovered could not clear its mark while a different
window was warning. Simulated over a real ledger of 7,788 responses, that produced 12 cases where
a mark was held past an explicit allowed, the longest for 4.8 minutes.

The second commit adds Result.RateLimitClearedWindows and clears each window on its own explicit
allowed. Two properties are deliberately preserved, and both are tested: an absent header still
never clears (absence is not an all-clear), and rejected never clears (it is worse than warned,
not better — the existing 429 path owns that case). They share a single default: no-op arm so they
cannot drift apart.

That commit also updates rateLimitWarningsWouldChange, which is load-bearing rather than cosmetic:
it drives resultNeedsGlobalTransition, which selects the atomic StateMutator.MutateAuthState
path. Without it, a clear-only response reports "nothing changed" and the clear is inert on that
path.

Scoping that precisely, because the obvious reading over-claims it: this is about
cluster consistency, not about the write vanishing on every topology. On a single-node deployment
a clear-only response takes the local path and still applies and persists correctly via
stageResultPersist/scheduleResultPersist. The failure this guards against is the multi-node case,
where the write is routed through the cluster mutator and a false "no change" lets it be dropped or
clobbered.

Concurrency note

authRateLimitWarned iterates the fixed three-window array with a keyed map lookup instead of
range-ing auth.RateLimitWarnings. Selection runs unlocked while MarkResult mutates that map
under a lock, and range is the shape that triggers Go's fatal
concurrent map iteration and map write. The keyed lookup matches the pre-existing unsynchronized
keyed read of auth.ModelStates in isAuthBlockedForModel, so this introduces no new crash mode and
changes no locking. To be clear about what this is not: it de-amplifies an existing unsynchronized
read; it does not fix the underlying lack of synchronization, which is pre-existing and unchanged.

Tests

17 tests in internal/cliproxy/auth/rate_limit_warning_test.go, covering: warning sets the mark;
selector prefers non-warned; falls back to the warned credential when every candidate is warned;
sole-warned-credential never starves; clears on allowed; clears on Reset expiry; per-window clear
while another window warns; rejected does not clear; absent header does not clear; disabled never
selected; the anti-inert guard; 7d_oi header canonicalization; and clone non-aliasing.

7d_oi deserves a note: Go's textproto.CanonicalMIMEHeaderKey does not treat _ as a word
separator, so that key does not canonicalize the way the hyphenated ones do. Header lookups here
are therefore explicitly case-insensitive rather than relying on canonical form.

To be straight about that helper rather than let a reviewer discover it: getHeaderCaseInsensitive's
linear-scan fallback is belt-and-suspenders, not load-bearing. Canonicalization is deterministic
for these keys, so headers.Get alone would suffice for any realistic net/http header set. The
fallback is kept to match this codebase's existing defensive posture for header reads, and it is
exercised directly by TestGetHeaderCaseInsensitiveFallsBackToLinearScan rather than left untested.
Happy to drop it if you would rather not carry the extra path.

gofmt -l clean; go build ./... clean; go vet ./... reports only the pre-existing
internal/cluster/refresh.go:172: unreachable code; go test -count=1 ./... green across all 31
packages. Each new test was observed passing individually rather than inferred from a package-level
ok.

🔴 Warning-avoidance outranks the operator-set PRIORITY field

Stating this explicitly because it is a real behaviour change and it is not obvious from the diff.
collectAvailableByPriority splits candidates into a non-warned bucket and a warned bucket, each
keyed by priority, and getAvailableAuths tries the non-warned bucket first regardless of
priority level. So a PRIORITY 1 credential with no warning will be preferred over a
PRIORITY 10 credential that is warned. Warning-avoidance is a stronger signal than the operator's
own ordering, and only falls back to priority ordering within each bucket.

That is intended — the whole point is to move off a credential before it errors — but an operator
who has deliberately ranked their credentials should know that a warning overrides that ranking
until the warning clears.

🔴 This PR contains #115

It is branched on top of fix/claude-quota-reset-scope, so it carries that PR's 4 commits as well as
its own 2. #115 is where the unified rate-limit header allowlist and the per-credential reset scoping
live, and this work is not separable from them. If #115 merges first, this reduces to the last two
commits. Reviewing them together is fine; reviewing this one instead of #115 is not.

🤖 Generated with Claude Code