feat: Implement shared provider utilities and Proxy Api Server #11495
Conversation
- Added provider API host formatting utilities to handle differences between Cherry Studio and AI SDK. - Introduced functions for formatting provider API hosts, including support for Azure OpenAI and Vertex AI. - Created a simple API key rotator for managing API key rotation. - Developed shared provider initialization and mapping utilities for resolving provider IDs. - Implemented AI SDK configuration utilities for converting Cherry Studio providers to AI SDK configurations. - Added support for various providers including OpenRouter, Google Vertex AI, and Amazon Bedrock. - Enhanced error handling and logging in the unified messages service for better debugging. - Introduced functions for streaming and generating unified messages using AI SDK.
There was a problem hiding this comment.
Note
This review was translated by Claude.
PR Review Summary
Thank you for submitting this PR. After careful review, I found some issues that need to be fixed, including several critical bugs that would cause core functionality to not work.
❓ Need Additional Explanation
Please provide a detailed description of the problem this PR solves:
- User Scenario: Who is the target audience for this Proxy API Server? What problems are they encountering?
- Feature Value: Why do we need to expose any AI provider as an Anthropic-compatible API? What are the specific use cases?
- Relationship with Existing Features: How does this feature differ from Cherry Studio's existing API Server? Is it a supplement or a replacement?
- Breaking Changes: Will this PR affect existing users' experience?
The following parts of the PR template need to be filled:
- "Before this PR" / "After this PR"
- "Why we need it and why it was done in this way"
- "Fixes #" (if there are related issues)
🔴 Critical Issues (Must Fix)
1. Tool result content is cleared
Location: `src/main/apiServer/services/unified-messages.ts:187-190`
```typescript
// After the values array is correctly populated...
return {
type: 'content',
value: [] // ❌ Should be value: values
}
```
The `values` array is built but never used, causing all tool_result content to be lost. This would completely break the tool calling chain.
2. Pure tool_result messages generate empty user messages
Location: `src/main/apiServer/services/unified-messages.ts:308-327`
When messages only contain `tool_result`, the code first pushes a `role: 'tool'` message, then unconditionally continues to push a `role: 'user'` message, where both `textParts` and `imageParts` are empty arrays:
```typescript
messages.push({
role: 'user',
content: [...textParts, ...imageParts] // Empty array []
})
```
AI SDK/models usually reject empty messages, causing the tool chain to break.
3. Provider filtering contradicts feature declaration
Location: `src/main/apiServer/utils/index.ts:32-34`
```typescript
const supportedProviders = providers.filter(
(p: Provider) => p.enabled && (p.type === 'openai' || p.type === 'anthropic')
)
```
The PR description claims to support "any AI provider supported by AI SDK", but the code only allows `openai` and `anthropic` types. Providers like gemini/vertexai/bedrock cannot be used, which contradicts the claimed functionality.
⚠️ Important Issues
4. Missing test coverage
About 1500 lines of new code in `packages/shared/` have no unit tests:
- `AiSdkToAnthropicSSE.ts` (649 lines)
- `sdk-config.ts` (240 lines)
- `api/index.ts` (177 lines)
It's recommended to add at least tests for core conversion logic.
5. Duplicate code
The `/count_tokens` endpoint has two nearly identical implementations in router and providerRouter (`messages.ts:568-715`). Consider extracting shared logic.
6. Token estimation is too simple
```typescript
// ~4 characters per token for English text
const estimatedTokens = Math.ceil(totalChars / 4) + messages.length * 3
```
Estimation is inaccurate for non-English text (like Chinese), and doesn't handle tool token counting.
📝 Minor Suggestions
- Type Safety: `AiSdkToAnthropicSSE.ts:180` uses `as any`, consider improving type definitions
- Hardcoded Values: There are hardcoded `APP-Code` and API hosts in `aihubmix.ts`
Summary
Please:
- Complete the PR description, explaining the feature's purpose and use cases
- Fix Critical Issues, especially tool calling related bugs
- Add tests, at least covering core conversion logic
- Run `yarn build:check` to ensure everything passes
🤖 Review by Claude Code
- Fix tool result content bug: return `values` array instead of empty array - Fix empty message bug: skip pushing user/assistant messages when content is empty - Expand provider support: remove type restrictions to support all AI SDK providers - Add missing alias for @cherrystudio/ai-sdk-provider in main process config 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
Extract duplicated token estimation code from both count_tokens endpoints into a shared `estimateTokenCount` function to improve maintainability. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
This comment was marked as resolved.
This comment was marked as resolved.
…ndling in AI SDK integration
…nified-messages integration
… in unified messages
… cache service - Deleted the old ReasoningCache class and its instance. - Introduced CacheService for managing reasoning caches. - Updated unified-messages service to utilize new googleReasoningCache and openRouterReasoningCache. - Added AiSdkToAnthropicSSE adapter to handle streaming events and integrate with new cache service. - Reorganized shared adapters to include the new AiSdkToAnthropicSSE adapter. - Created openrouter adapter with detailed reasoning schemas for better type safety and validation.
…ate the tool call model filter.
There was a problem hiding this comment.
Pull request overview
This PR implements a comprehensive Proxy API Server that exposes Cherry Studio's configured AI providers through an Anthropic-compatible API endpoint. This enables external tools like Claude Code and Cursor to use any AI provider configured in Cherry Studio via a standardized interface.
Key Changes:
- Extracted shared provider utilities to
packages/sharedfor code reuse between renderer and main processes - Implemented
AiSdkToAnthropicSSEadapter to convert AI SDK responses to Anthropic SSE format - Created unified message processing service supporting both streaming and non-streaming modes
- Added support for PPIO provider's Anthropic-compatible models
- Enhanced Claude Code integration to work with any provider via the unified adapter
Reviewed changes
Copilot reviewed 64 out of 65 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
packages/shared/provider/* |
Shared provider utilities for detection, mapping, formatting, and SDK configuration |
packages/shared/middleware/* |
Shared AI SDK middlewares (Gemini thought signature, OpenRouter reasoning) |
packages/shared/api/* |
Shared API URL formatting and validation utilities |
packages/shared/anthropic/* |
Enhanced Anthropic SDK utilities with sanitization |
src/main/apiServer/adapters/* |
AI SDK to Anthropic SSE conversion adapter |
src/main/apiServer/services/unified-messages.ts |
Core unified message processing service |
src/main/apiServer/routes/messages.ts |
Enhanced message routing with unified processing |
src/main/services/agents/services/claudecode/* |
Claude Code integration improvements |
src/renderer/src/aiCore/provider/* |
Refactored to use shared utilities |
No critical issues found. The implementation is well-structured with proper error handling and type safety. The code follows established patterns and includes comprehensive documentation.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| new TransformStream<LanguageModelV2StreamPart, LanguageModelV2StreamPart>({ | ||
| transform( | ||
| chunk: LanguageModelV2StreamPart, | ||
| controller: TransformStreamDefaultController<LanguageModelV2StreamPart> | ||
| ) { | ||
| if (chunk.type === 'reasoning-delta' && chunk.delta.includes(REDACTED_BLOCK)) { | ||
| controller.enqueue({ | ||
| ...chunk, | ||
| delta: chunk.delta.replace(REDACTED_BLOCK, '') | ||
| }) | ||
| } else { | ||
| controller.enqueue(chunk) | ||
| } | ||
| } | ||
| }) |
There was a problem hiding this comment.
Superfluous argument passed to function TransformStream.
CheckList
Summary
This PR implements a Proxy API Server that exposes Cherry Studio's configured AI providers as an Anthropic-compatible API endpoint. This allows external tools and applications to use any AI provider configured in Cherry Studio through a standardized Anthropic Messages API.
Key Features
What this PR does
Before this PR:
After this PR:
http://localhost:<port>/v1/messagesand use any provider<provider-id>:<model-id>(e.g.,my-openai:gpt-4o,gemini:gemini-2.0-flash)Why we need it and why it was done in this way
User Scenario:
Developers and power users who want to use their existing Cherry Studio provider configurations with external AI tools (Claude Code, Cursor, Continue, custom scripts, etc.) without re-configuring API keys in multiple places.
Feature Value:
Implementation Approach:
Tradeoffs:
Alternatives Considered:
Breaking changes
None. This is a new feature that doesn't affect existing functionality.
Special notes for your reviewer
Checklist
Release note