Skip to content

[cache] Propagate prompt cache hints and service tier across translators #5156

Description

@warelik

Problem

The translator paths that convert between OpenAI/Codex, Claude, and Gemini currently drop or mishandle cache-control markers:

  • Codex/OpenAI Chat→Codex drops prompt_cache_key, prompt_cache_options, prompt_cache_breakpoint, and does not map service_tier to Codex-accepted values.
  • OpenAI Responses→Codex drops client prompt_cache_options/prompt_cache_breakpoint.
  • Claude→OpenAI/Codex and Claude→Gemini drop cache_control on content parts, messages, and tools.
  • Gemini→Claude/Codex has no cache_control handling.

Proposed change

Apply the same implementation as kaitranntt/CLIProxyAPIPlus#215:

  • Add shared helpers: ModelSupportsExplicitPromptCache, NormalizeCodexServiceTier, CopyPromptCacheBreakpoint, AttachPromptCacheBreakpoint, AttachMessagePromptCacheBreakpoint.
  • Map Claude cache_control to OpenAI/Codex prompt_cache_breakpoint when rebuilding content arrays.
  • Preserve message-level cache_control on the last content block.
  • Echo prompt_cache_key in Codex/OpenAI Responses responses.
  • Propagate prompt_cache_key/prompt_cache_retention and conditional prompt_cache_options/prompt_cache_breakpoint in Chat/Responses→Codex.
  • Normalize service_tier to Codex values.
  • Document cache_control drop in Claude→Gemini (no direct cachedContent mapping).

Plus implementation

kaitranntt/CLIProxyAPIPlus#215

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions