Prevent Redis outages from amplifying latency, source load, and logs

[lan17]

Priority

P1 — High — bound avoidable cache-layer amplification before broad production adoption.

Current problem

A failed Redis read already pays the caller-owned client's timeout/queue/retry cost and logs once. DialCache then runs the source fallback. For untracked keys it currently attempts a Redis write afterward, potentially paying and logging a second failed Redis operation on the same request. Tracked keys already suppress that post-read-error write to preserve invalidation safety.

The earlier lazy-factory retry concern was removed by #67. Redis clients are now created, connected, configured, and lifecycle-owned by the application.

Current evidence:

• Remote read failure path
• Fallback and possible second Redis write
• Caller-owned client contract
• Shutdown and lifecycle documentation

Ownership boundary

The underlying Redis/Valkey client should own command timeouts, reconnect policy, retry strategy, and offline queueing. The injected logger should own deployment-specific sampling/rate limiting. DialCache should not duplicate those systems with a second general-purpose client breaker unless production evidence shows the client boundary is insufficient.

Recommended minimal scope — approval required

• After any Redis read exception, run fallback but skip the post-fallback Redis write for both tracked and untracked keys. A later request retries Redis normally, so recovery requires no DialCache breaker state.
• Preserve local population only where it does not weaken tracked invalidation correctness.
• Document required finite client command/connect/retry/queue budgets.
• Add sustained concurrent and high-cardinality read-failure tests proving one Redis attempt per invocation after the initial read fails.
• Preserve surfaced invalidateRemote() failures.
• Keep metrics changes narrowly tied to an operational need; do not reopen the broad write/end-to-end telemetry bundle closed in Add Redis write-path and end-to-end cache outcome telemetry #44.

Deferred unless measured evidence requires it

• DialCache-level circuit-breaker states, half-open probes, or cooldown timers.
• Internal retry/backoff or reconnect policy.
• Library-owned global log throttling.
• Cross-process coordination from Design optional cross-process cache stampede protection #7.

Acceptance criteria

• No automatic second Redis attempt after a failed read.
• Source fallbacks remain fail open.
• Tracked keys cannot publish process-local data after a failed tracked read.
• Recovery occurs naturally when a later read succeeds or returns a normal miss.
• Client-owned timeout/retry/queue requirements are documented with node-redis and GLIDE examples.
• Focused tests cover concurrency, sustained failure, high key cardinality, and recovery without hidden breaker state.