Skip to content

fix(parser): pass non-JSON octet-stream payloads to the model in EventBridgeV2Envelope - #8669

Open
alphasafal wants to merge 1 commit into
aws-powertools:developfrom
alphasafal:fix/eventbridge-v2-octet-stream-non-json
Open

alphasafal wants to merge 1 commit into
aws-powertools:developfrom
alphasafal:fix/eventbridge-v2-octet-stream-non-json

Conversation

@alphasafal

Copy link
Copy Markdown

Issue number: closes #8666

Summary

Changes

EventBridgeV2Envelope.parse ran payload.decode() on every application/octet-stream payload and handed the str to _parse, which treats any str as JSON. Only octet-stream payloads that were UTF-8 JSON worked.

This change adds _decode_octet_stream, which tries json.loads on the bytes and falls back to the raw bytes when the payload isn't JSON (json.loads raises ValueError for both invalid UTF-8 and invalid JSON):

  • JSON octet-stream payloads parse into the model as before.
  • A bytes model receives the raw payload, matching EventBridgeV2Event(...).records[...].data.
  • Any other model raises ValidationError instead of a raw UnicodeDecodeError.

Octet-stream payloads are no longer passed through _unwrap_put_events, which keeps today's behaviour for JSON bytes (they were never unwrapped, since _unwrap_put_events only saw a str).

Tests added to tests/unit/parser/_pydantic/test_eventbridge_v2.py:

  • binary and plain-text octet-stream payloads parsed with model=bytes return the raw bytes
  • a binary octet-stream payload parsed with an object model raises ValidationError

All three fail on develop and pass with this change. tests/unit/parser, tests/unit/data_classes and tests/functional/parser pass (428 tests), and ruff format / ruff check are clean.

User experience

Before:

parse(event=octet_stream_event(b"\xff\xd8\xff\xe0JFIF"), model=bytes, envelope=envelopes.EventBridgeV2Envelope)
# UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 0: invalid start byte

parse(event=octet_stream_event(b"hello world"), model=bytes, envelope=envelopes.EventBridgeV2Envelope)
# ValidationError: 1 validation error for bytes

After:

parse(event=octet_stream_event(b"\xff\xd8\xff\xe0JFIF"), model=bytes, envelope=envelopes.EventBridgeV2Envelope)
# [b'\xff\xd8\xff\xe0JFIF']

parse(event=octet_stream_event(b"hello world"), model=bytes, envelope=envelopes.EventBridgeV2Envelope)
# [b'hello world']

parse(event=octet_stream_event(b"\xff\xd8\xff\xe0JFIF"), model=Order, envelope=envelopes.EventBridgeV2Envelope)
# ValidationError: 1 validation error for Order

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Disclaimer: We value your time and bandwidth. As such, any pull requests created on non-triaged issues might not be successful.

…tBridgeV2Envelope

EventBridgeV2Envelope decoded every application/octet-stream payload with
bytes.decode() and parsed the result as JSON, so binary payloads raised
UnicodeDecodeError and plain-text payloads failed even with model=bytes.

Try JSON first and fall back to the raw bytes, matching the
EventBridgeV2Event data class: bytes models get the payload, and any other
model raises ValidationError.

Closes aws-powertools#8666
@alphasafal
alphasafal requested a review from a team as a code owner October 25, 2026 19:23
@alphasafal
alphasafal requested review from svozza and a balanced review from Copilot October 25, 2026 19:23
@powertools-for-aws-oss-automation powertools-for-aws-oss-automation Bot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Oct 25, 2026
@boring-cyborg boring-cyborg Bot added the tests label Oct 25, 2026
@boring-cyborg

boring-cyborg Bot commented Oct 25, 2026

Copy link
Copy Markdown

Thanks a lot for your first contribution! Please check out our contributing guidelines and don't hesitate to ask whatever you need.
In the meantime, check out the #python channel on our Powertools for AWS Lambda Discord: Invite link

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.

@sonarqubecloud

Copy link
Copy Markdown

@leandrodamascena leandrodamascena added this to the 3.38.0 milestone Oct 25, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/M Denotes a PR that changes 30-99 lines, ignoring generated files. tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bug: EventBridgeV2Envelope raises UnicodeDecodeError for application/octet-stream payloads that are not UTF-8 JSON

3 participants