fix(bedrock): unify streaming error handling for in-band and out-of-band errors#1479
Open
sevakva wants to merge 4 commits into
Open
fix(bedrock): unify streaming error handling for in-band and out-of-band errors#1479sevakva wants to merge 4 commits into
sevakva wants to merge 4 commits into
Conversation
Two streaming error paths previously failed to surface as typed APIStatusError subclasses: - HTTP 200 streams where the first SSE chunk is an in-band error frame (e.g. cross-region inference profile rate limits). The decoder hardcoded event="completion", so the streaming layer parsed the payload as RawMessageStartEvent with message=None and crashed with AttributeError (anthropics#1472). - Out-of-band stream errors where botocore parses an EventStreamMessage with status_code != 200 and a :exception-type header (e.g. internalServerException, throttlingException). The decoder raised a bare ValueError that bypassed retry/error handling (anthropics#1477). Funnel both shapes through the existing event="error" SSE path: - Decoder normalizes Bedrock :exception-type into Anthropic error.type (throttlingException → rate_limit_error, serviceUnavailableException → overloaded_error, internalServerException/modelStreamError/modelTimeout → api_error, validationException → invalid_request_error). Unknown types pass through verbatim and fall back to a generic APIStatusError. - _make_status_error gains a body-aware pre-dispatch on body.error.type so in-band errors on HTTP 200 select the right exception class (RateLimitError, OverloadedError, etc.) instead of being misclassified as success. - HTTP 529 → OverloadedError, matching anthropic._client behavior.
Author
|
@localden Hello! Not sure who to ask, but is this repo alive? Do I need do anything extra? These issues with bedrock is really critical for our company and blocking us from using it |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes #1472. Closes #1477. Supersedes #1475, #1485, #1486, #1488
Two stream-error paths in
lib/bedrockfailed to surface as typedAPIStatusErrorsubclasses. This PR routes both through the existingevent="error"SSE path and adds a body-aware exception dispatch.Bugs
#1472 — HTTP 200 stream where the first SSE chunk is an in-band error envelope (
{"type":"error","error":{...}}), reliably reproducible with cross-region inference profiles. The decoder hardcodedevent="completion", so the streaming layer parsed it asRawMessageStartEvent(message=None)and crashed withAttributeError: 'NoneType' object has no attribute 'model'.#1477 — Out-of-band exception frame (
:message-type: exceptionwith a:exception-typeheader). botocore parses these withstatus_code != 200. The decoder raised a bareValueError, uncatchable asAPIStatusError.Bundled: HTTP 529 →
OverloadedError, matchinganthropic._client.Changes
Decoder normalizes both shapes into
event="error"and translates Bedrock:exception-typeto the Anthropicerror.type::exception-typeerror.typethrottlingExceptionrate_limit_errorRateLimitErrorserviceUnavailableExceptionoverloaded_errorOverloadedErrorinternalServerExceptionapi_errorInternalServerErrormodelStreamErrorExceptionapi_errorInternalServerErrormodelTimeoutExceptionapi_errorInternalServerErrorvalidationExceptioninvalid_request_errorBadRequestErrorUnknown types pass through verbatim → generic
APIStatusError. The eventstatus_codeis not used for class selection (botocore hardcodes it to 400 for any exception frame);:exception-typeis the source of truth.Client —
_make_status_errorgains a body-aware pre-dispatch keyed onbody.error.type. This is what lets in-band HTTP 200 errors reach the right subclass. The lookup table usesErrorTypeliterals fromanthropic.types.shared.The body dispatch is additive — non-stream 4xx/5xx already produce the correct subclass via HTTP status; the new path only kicks in when the body carries an
error.typethe status fallback would miss.Tests
tests/lib/test_bedrock_stream_decoder.py— decoder unit tests, parametrized over the full mapping table.tests/lib/streaming/test_bedrock_stream_errors.py— E2E viahttpx.MockTransportwith hand-encoded eventstream frames, sync + async, plus 529 parity.