Skip to content

fix: restore Anthropic prompt caching - #1

Open
luisherranz wants to merge 1 commit into
heymaaz:mainfrom
luisherranz:fix-prompt-cache
Open

fix: restore Anthropic prompt caching#1
luisherranz wants to merge 1 commit into
heymaaz:mainfrom
luisherranz:fix-prompt-cache

Conversation

@luisherranz

Copy link
Copy Markdown

Important

AI disclosure: All changes in this PR, including the implementation, tests, commit message, and this PR description, were generated entirely by GPT-5.6 Sol. I, Luis Herranz, have no idea what was changed or why. The technical explanation below is also AI-generated, so please treat the contribution as untrusted code requiring careful maintainer review.

Summary

This PR restores Anthropic prompt caching for OAuth requests routed through the plugin's custom AI SDK provider.

It adds a bounded default cache policy when the caller did not supply one, preserves explicit cache policies and TTLs, and ensures transient session-warming requests refresh history that subsequent real prompts can reuse.

Problem

The plugin replaces OpenCode's native Anthropic route with:

aisdk:file://.../dist/provider.js

That sends requests through OpenCode's generic AI SDK adapter rather than its native Anthropic protocol.

In this path:

  • OpenCode's native Anthropic cache policy does not reach the final request body.
  • transformBody received requests without cache_control markers and did not add any.
  • Relocating non-core system content into the first user message discarded any marker attached to that content.
  • Repeated prompts and automatic session-warming requests therefore reprocessed the complete context instead of reading it from Anthropic's prompt cache.
  • Cache telemetry in responses was already mapped correctly, so the problem was request construction rather than response handling or OAuth support.

This was particularly expensive with session warming enabled because each warm request resent the complete context every four minutes without refreshing a reusable cache entry.

Solution

When the input has no existing prompt-cache policy, transformBody now adds at most four ephemeral cache boundaries after completing the OAuth rewrites:

  • The last custom tool.
  • The final relocated system block.
  • The last reusable history block before the current request suffix.
  • Anthropic's top-level automatic cache control for the current tail.

The reusable history boundary is important for warming. A warm request has a transient user suffix that will not appear in the next real prompt. Caching only that tail would refresh an entry the next prompt could not reuse. The additional history boundary lets warming refresh the shared conversation prefix independently of its temporary suffix.

The implementation also:

  • Detects caller-provided cache policies before rewriting the request.
  • Leaves explicit policies unchanged instead of adding another automatic policy.
  • Preserves existing TTLs, including ttl: "1h".
  • Preserves markers when system content is relocated into a user message.
  • Removes markers from the OAuth billing header and exact Claude Code identity block.
  • Does not invent a replacement policy when a caller placed a forbidden marker on the identity block.
  • Treats null cache policies as absent.
  • Supports text, thinking, and provider-tool history boundaries.
  • Stays within Anthropic's four-breakpoint limit.

Testing

make all passes:

  • Lint: 0 warnings and 0 errors.
  • Formatting: all files correctly formatted.
  • TypeScript build: passed.
  • Tests: 299 passed, 0 failed.

Regression coverage includes:

  • Bounded default cache boundaries.
  • Reusable history before a warming suffix.
  • Provider-tool history blocks.
  • Existing explicit policies.
  • Existing top-level automatic policies.
  • One-hour TTL preservation.
  • Relocated system markers.
  • String-form system prompts.
  • Null policies.
  • Billing and identity markers.
  • OAuth system-prompt rewrites.

Manual OAuth verification against Anthropic also succeeded:

  • A cold Sonnet 4.5 request wrote 14,239 cache tokens.
  • An identical second request read all 14,239 tokens.
  • A real scheduled Fable 5 warm request read the complete 32,834-token conversation prefix, wrote only the new 1,706-token suffix, and had only 2 uncached input tokens.
  • A separate suffix-change test confirmed that a later real prompt could still reuse the shared history cached by transient warm requests.

Checklist

  • PR title follows Conventional Commits
  • make all passes locally
  • Tests added or updated where applicable
  • README or docs updated where applicable (no user-facing configuration change)

@jeroenev

Copy link
Copy Markdown

this is highly needed
A single prompt to opus consumed 50% of my AI usage with opencode2 😓

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants