Skip to content

feat(ai): modernize the Anthropic path and fix conversation replay - #663

Open
yupanzi wants to merge 1 commit into
kite-org:mainfrom
yupanzi:feature/ai-claude-updates
Open

feat(ai): modernize the Anthropic path and fix conversation replay#663
yupanzi wants to merge 1 commit into
kite-org:mainfrom
yupanzi:feature/ai-claude-updates

Conversation

@yupanzi

@yupanzi yupanzi commented Jul 30, 2026

Copy link
Copy Markdown

Summary

Modernize the Anthropic path and fix conversation replay:

Anthropic request surface

  • Move to the Beta Messages API with adaptive thinking, context management, and output_config.effort. Feature use is gated per model (anthropicModelSupportsModernFeatures) so models that reject the modern surface keep the classic request shape.
  • New Reasoning Effort setting (low/medium/high/xhigh/max, default xhigh), exposed in general settings next to Max Tokens, with hints explaining both knobs.
  • AIMaxTokens is sent as configured (never clamped) and the default is provider-aware: 64000 for Anthropic, 8192 for OpenAI. Existing installs with the old 4096 default are migrated (covered by general_setting_upgrade_test.go). Default Anthropic model bumped to claude-opus-5.

Conversation replay

  • The frontend now sends tool round-trips structurally (role: tool with id/name/args/result) and the backend rebuilds real tool_use/tool_result blocks. Tool messages without a result (denied/cancelled) are skipped so no dangling tool_use is produced. Covered by conversation_history_test.go.

Robustness

  • Secret data in tool results is masked before reaching the model.
  • SSE streams emit a keepalive comment every 20s; the chart's gateway request timeout is raised to 900s and the matching ingress-nginx annotations are documented in values.yaml — these are two halves of the same fix.
  • Resource tool schemas now state their caps (list items, log tail lines/bytes) so the model narrows queries instead of truncating blindly.
  • AI chat markdown tables no longer overflow on long unbreakable tokens.

Why

The Anthropic integration predates the current Claude API surface: budget_tokens now returns 400 on current models, the 4096-token default truncates answers (thinking and answer share max_tokens), and tool turns were replayed to the model as flattened [Tool: ...] text, which poisons later turns into emitting tool calls as plain text/XML instead of native tool use. Long agent turns also died behind ingress-nginx, which closes a connection after 60s of backend silence while a tool is legitimately still running.

Related issue

Closes #668

Validation

  • go test ./pkg/ai/... ./pkg/model/... ./pkg/settings/... (new: conversation history, handler streaming, settings migration tests)
  • pnpm run type-check && pnpm run lint && pnpm run test
  • helm lint + helm template for the chart change

Checklist

  • I reviewed this PR myself before requesting review.
  • I understand the changes, including AI-generated parts (if any).
  • For new features, a feature request issue is linked.
  • I cleaned up AI noise (unnecessary comments, dead code, and unrelated changes).
  • This PR is reasonably scoped (or split into smaller PRs).

🤖 Generated with Claude Code

Update the Claude integration to the current API surface and fix several
agent-loop bugs:

- Move the Anthropic path to the Beta Messages API with adaptive
  thinking, context management, and output_config.effort. Effort is the
  reasoning-depth knob on current Claude models (budget_tokens returns
  400 there); it is exposed as a new AIEffort setting
  (low/medium/high/xhigh/max, default xhigh) in the general settings.
  Feature use is gated per model so older models keep the classic
  request shape.
- Send AIMaxTokens as configured and make the default provider-aware
  (64000 for Anthropic, 8192 for OpenAI): on current Claude models
  thinking and answer share max_tokens, so the old 4096 default
  truncated answers mid-sentence. Default Anthropic model is now
  claude-opus-5.
- Replay tool turns structurally instead of flattening them to
  "[Tool: ...]" text. Textual replay poisoned the model into emitting
  tool calls as plain text/XML on later turns; the frontend now sends
  the tool round-trip (id, name, args, result) and the backend rebuilds
  real tool_use/tool_result blocks.
- Mask Secret data in tool results before they reach the model.
- Emit an SSE keepalive comment every 20s: an agent turn is
  legitimately silent while a tool runs, and ingress-nginx closes the
  connection after 60s of backend silence. Raise the chart's gateway
  request timeout to 900s and document the matching ingress-nginx
  annotations next to it.
- Tighten resource tool schemas/descriptions with explicit caps, and
  fix long-token overflow in AI chat markdown tables.
- Cover conversation replay, handler streaming, settings migration, and
  the agent loop with tests.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

AI: Anthropic requests fail or truncate on current Claude models

1 participant