Skip to content

ai: unify claude JSON instructions and error handling#174

Open
derekbarbosa wants to merge 5 commits into
sashiko-dev:mainfrom
derekbarbosa:fixes/vertex-response-format
Open

ai: unify claude JSON instructions and error handling#174
derekbarbosa wants to merge 5 commits into
sashiko-dev:mainfrom
derekbarbosa:fixes/vertex-response-format

Conversation

@derekbarbosa
Copy link
Copy Markdown
Collaborator

@derekbarbosa derekbarbosa commented May 11, 2026

Centralizes JSON instruction logic in src/ai/mod.rs by adding AiResponseFormat::format_json_schema_instruction().

Improves claude-cli in src/ai/claude_cli.rs:

  • Introduces ClaudeCliError implementing ClassifyAiError for better retry handling.
  • Switches build_prompt to use the new centralized JSON instructions.

Unifies Claude API in src/ai/claude.rs:

  • Injects the centralized JSON instructions into the system prompt.
  • Removes redundant/duplicate formatting logic.

Updates error classification in src/ai/mod.rs to support the new ClaudeCliError.

fixes #171

@HouMinXi
Copy link
Copy Markdown

Tested PR #174 on Vertex AI with claude-sonnet-4-6. Still fails at Stage 1:

Error: Stage 1 failed to produce valid 'concerns' array after 3 attempts

The response_format injection into system prompt looks correct (Phase 0 should work now), but Stage 1 uses tool use, and the tool schemas in src/worker/prompts.rs still have uppercase type names:

// prompts.rs:486-490
"type": "OBJECT",
"type": "ARRAY",
"type": "STRING"

Claude API requires lowercase ("object", "array", "string"). Gemini accepts both cases but Claude rejects uppercase with:

tools.0.custom.input_schema.type: Input should be 'object'

This is the same issue as #107. PR #108 (closed) fixed both problems (response_format + uppercase schemas), but only the response_format half made it into this PR.

Quick fix -- in src/worker/prompts.rs, change all inline JSON schema types to lowercase:

-"type": "OBJECT"
+"type": "object"
-"type": "ARRAY"
+"type": "array"
-"type": "STRING"
+"type": "string"

Also check src/worker/tools.rs if it has the same pattern (I didn't find uppercase there, but worth double-checking).

derekbarbosa and others added 2 commits May 12, 2026 23:06
Co-authored-by: Gemini CLI <noreply@google.com>
Signed-off-by: derekbarbosa <derekasobrab@gmail.com>
Co-authored-by: Gemini CLI <noreply@google.com>
Signed-off-by: derekbarbosa <derekasobrab@gmail.com>
@derekbarbosa derekbarbosa force-pushed the fixes/vertex-response-format branch from b27f7e1 to 9bf0d40 Compare May 13, 2026 03:32
@derekbarbosa
Copy link
Copy Markdown
Collaborator Author

Reworded some commits, added some more error handling for claude_cli, and unified JSON prompt building/formatting for claude responses.

@derekbarbosa derekbarbosa changed the title WIP: DO NOT MERGE: forward response_format to Anthropic-based providers ai: unify claude JSON instrctuons and error handling May 13, 2026
@derekbarbosa derekbarbosa changed the title ai: unify claude JSON instrctuons and error handling ai: unify claude JSON instructions and error handling May 13, 2026
derekbarbosa and others added 3 commits May 14, 2026 17:28
Co-authored-by: Gemini CLI <noreply@google.com>
Signed-off-by: derekbarbosa <derekasobrab@gmail.com>
Co-authored-by: Gemini CLI <noreply@google.com>
Signed-off-by: derekbarbosa <derekasobrab@gmail.com>
Co-authored-by: Gemini CLI <noreply@google.com>
Signed-off-by: derekbarbosa <derekasobrab@gmail.com>
@derekbarbosa derekbarbosa force-pushed the fixes/vertex-response-format branch from 9bf0d40 to 9b49275 Compare May 14, 2026 23:47
@derekbarbosa
Copy link
Copy Markdown
Collaborator Author

derekbarbosa commented May 14, 2026

Addressed some review-pr concerns, mainly:

src/ai/claude.rs:L322-326: 🟡 risk: system_blocks.push appends a new text block. Some Anthropic-compatible proxies (e.g. AWS Bedrock, Google Vertex AI) might reject multiple consecutive
text blocks in the system prompt array. Consider appending the instruction to the last text block if one exists.

src/ai/mod.rs:L95: 🔵 nit: if serde_json::to_string fails (unwrap_or_default returns ""), the schema instruction becomes invalid JSON matching this schema: .. Since schema is dynamic,
might be safer to handle serialization errors gracefully (e.g., fallback to schema-less instruction).

Tested using us-east5 endpoint, E2E patch submission successful with sonnet & opus models via vertex.

ready for review! @rgushchin

@derekbarbosa
Copy link
Copy Markdown
Collaborator Author

Hi @rgushchin, sorry for the noise. Seems like there are others encountering the JSON encoding issue. Kindly asking for a review :-)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Vertex AI Claude provider drops response_format, Stage 1 always fails

2 participants