feat(s3): S3プロバイダにSigV4署名を追加 - #93
Open
tishin-endou wants to merge 9 commits into
Open
Conversation
Co-Authored-By: An Qiuyu <qiuyu.an@hotmail.com>
This was referenced Jun 5, 2026
tishin-endou
changed the title
Feature/s3 addon migration
- test_region_host: mock get_s3_bucket_object_location to use pre-registered URL and avoid timestamp divergence between two aiobotocore sessions - test_validate_v1_path_file: register root_listing_url (my-subfolder/) instead of bucket root - test_validate_v1_path_file_with_subfolder: fix listing_url to my-subfolder/ and add proper XML body - test_validate_v1_path_folder: fix listing_url to my-subfolder/Photos/ - test_chunked_upload_upload_part: pass params= to register_uri/has_call so ImmutableFurl hash matches when make_request passes params separately - test_metadata_folder: assert 'photos' (xmltodict strips leading whitespace) - test_errors_out/test_creates: GET 200 empty XML + HEAD 404 to let exists() return False; DownloadError from GET 404 is not caught by exists() - test_errors_out_metadata: expect DownloadError (not MetadataError) on GET 403
tishin-endou
marked this pull request as ready for review
July 17, 2026 00:53
yacchin1205
requested changes
Aug 10, 2026
There was a problem hiding this comment.
過去に意図的に導入した仕様や既知問題への対策が複数失われているように見えるため、変更をご検討ください。また、コメントアウトして残しているコードがいくつか散見されますが、不要なものであれば削除、採用すべきものであれば、そちらを採用すべきと考えます。どちらか、ご判断いただければと思います。
以下、見つけた回帰と思われる箇所です(これが全てではないかもしれません):
- PR #77 で追加された、Version/DeleteMarkerの全削除および1000件超のVersion対応が失われているように見えます -
# Check and delete all versions of the file in batchesのコードを削除しており、移植されているように見えない - PR #53 で5GB超のCopyObject問題を理由に無効化したintra-copy/moveが、サイズ制限なしで再有効化されている -
can_intra_copy等のコードが仕様変更されている
また、テストの多くが既存の振る舞いを保証する形ではなく、新しい内部ヘルパーをモックして呼び出しだけを確認する形へ置き換えられているため、CIが成功していても上記の回帰を検出できません。
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
概要
S3プロバイダをレガシーSignature V2からSignature V4認証に移行し、SigV4が必要なAWSリージョンとの互換性を実現します。
変更内容
プロバイダ (
waterbutler/providers/s3/provider.py)Content-MD5整合性チェックを追加VersionId='null'レスポンスを処理ListObjectsV2フォールバックを_delete_folderに追加ListObjectVersionsレスポンスのキーをURLデコードメタデータ (
waterbutler/providers/s3/metadata.py)コア (
waterbutler/core/metadata.py)依存関係 (
requirements.txt)テスト (
tests/providers/s3/)VersionId='null'フィルタリングロジックにテストアサーションを整合関連PR
Based on anqiuy/RDM-waterbutler@feature/s3-addon-migration