[takke1986]

Description of Changes

Attaching a Shift_JIS encoded text file fails with:

ValidationException: Unsupported MIME type: application/octet-stream.
Retry your request with a supported file type: xlsx, txt, pdf, csv, md, doc, html, xls, docx

I measured where this comes from by calling bedrock-runtime:Converse directly in
ap-northeast-1 (jp.anthropic.claude-haiku-4-5-20251001-v1:0), sending the same CSV in
different encodings and changing nothing else:

Bytes sent	format	Result
UTF-8	csv	accepted, model reads the content correctly
Shift_JIS	csv	ValidationException: Unsupported MIME type: application/octet-stream
Shift_JIS	txt	same
Shift_JIS re-encoded to UTF-8	csv	accepted, model reads the content correctly

So the rejection comes from Bedrock, and the declared format is ignored — Bedrock
sniffs the bytes. This confirms what @kazuhitogo found in #1399 and what @toshikwa noted
in #1181.

The last row is the part I would like to raise: the same content goes through once it is
re-encoded, so this can be fixed rather than only reported.

In #1399 the plan was to surface a clearer "UTF-8 only" message. That is a real improvement
over the current error, but #1181 asks for CSV files to be accepted regardless of
encoding, and Shift_JIS is still what several Japanese SaaS products emit when you export
a file. Converting on the server removes the manual step entirely.

If you would rather keep the error-message approach, I am happy to close this and send
that instead — I wrote this partly to show that transcoding is viable and cheap.

What this PR does

Adds convertTextDocumentToUtf8 and applies it where the Converse document content block
is built:

• only txt, csv, md and html are considered; binary formats (pdf, doc,
  docx, xls, xlsx) are never re-encoded
• bytes that are already valid UTF-8 are returned untouched (this also covers ASCII)
• otherwise shift_jis then euc-jp are tried with a strict decoder, and the decoded text
  is re-encoded as UTF-8
• bytes that no candidate decodes cleanly are left alone, so Bedrock still reports the
  problem rather than us silently sending mojibake

No new dependency. Node's built-in TextDecoder supports shift_jis and euc-jp.

Compatibility: UTF-8 and ASCII content is byte-for-byte unchanged, and binary formats
are untouched. The only behavior change is that files which previously failed the API call
now succeed.

Out of scope: the Bedrock Agents path in bedrockAgentApi.ts uses byteContent with a
different validation path, and UTF-16 is not attempted because a strict UTF-16 decode
accepts almost any byte sequence and would misfire.

Verified end to end

Passing the real output of convertTextDocumentToUtf8 to Converse:

rejected  Shift_JIS as-is                     → ValidationException: Unsupported MIME type
accepted  convertTextDocumentToUtf8(bytes)    → "山田太郎"

Checklist

• Modified relevant documentation — n/a (internal utility; the function carries a
  comment explaining the Bedrock behavior)
• Verified operation in local environment
• Executed npm run cdk:test — 49 tests / 15 snapshots passed, no snapshot differences

Also run:

• npm -w packages/cdk test -- textEncoding — 10 passed
• npm run web:test — 278 passed
• npm -w packages/cdk run lint (--max-warnings 0) and prettier --check clean

Related Issues

• Shift-JIS encoded files are rejected with "Unsupported MIME type: application/octet-stream" error #1399
• Error occurs when uploading CSV files encoded by Shift-JIS #1181