test: unit tests for fsProvider, s3Provider, tileDeletionStrategy(MAPCO-9806)#25
Merged
Merged
Conversation
almog8k
changed the title
|
🎫 Related Jira Issue: MAPCO-9806 |
CL-SHLOMIKONCHA
requested changes
May 13, 2026
…to be more readable
Co-authored-by: Shlomi k <65117898+CL-SHLOMIKONCHA@users.noreply.github.com>
…es/cleaner into test/deletion-strategy
CL-SHLOMIKONCHA
approved these changes
May 13, 2026
almog8k
added a commit
that referenced
this pull request
May 13, 2026
* feat: add AWS SDK S3 client and new storage provider constant - Added "@aws-sdk/client-s3" dependency to package.json for S3 interactions. - Introduced STORAGE_PROVIDERS constant in constants.ts for future storage provider implementations. * feat: integrate S3 storage provider with configuration updates * feat: update tilesDeletion strategy configuration for S3 integration * feat: tiles deletion strategy with S3 integration and validation schemas Co-authored-by: Copilot <copilot@github.com> * feat: register task context and update tiles deletion strategy for progress tracking * feat: enhance file deletion process by adding cleanup of empty directories * feat: improve logging level during S3 deletion and clean up unused type imports Co-authored-by: Copilot <copilot@github.com> * feat: integrate @map-colonies/raster-shared and update tiles deletion strategy Co-authored-by: Copilot <copilot@github.com> * feat: remove unused tiles deletion validation exports * feat: update storage provider keys to use SourceType constants Co-authored-by: Copilot <copilot@github.com> * fix: correct file extension formatting in generateRangePaths method * fix: replace console.debug with logger.debug in validateSchema function * chore: update dependencies for mc-priority-queue and raster-shared - upgraded @map-colonies/mc-priority-queue from ^9.0.2 to ^9.1.0 - upgraded @map-colonies/raster-shared from ^8.1.0-alpha.2 to ^8.1.0-alpha.3 * fix: pr issues * feat: update js-logger to version 5.0.0 and switch schemas to commonWorkerBoilerplateV2 * feat: add opentelemetry options to logger configuration in JSON files * feat: implement targetExists method for filesystem and S3 storage providers Co-authored-by: Copilot <copilot@github.com> * helm: deployment adjustments * feat: update simulate-deletion script and Dockerfile for tile deletion testing * feat: update @map-colonies/schemas dependency to version 1.20.0 * feat: update ConfigMap and Deployment templates for CA secret handling and logging configuration * feat: count not found paths as failed paths * feat: update metrics URL definition and refactor CA configuration in values.yaml * feat: handle undefined values in failed paths during file deletion * feat: integrate CA configuration into deployment and template rendering * feat: add K8S_POD_UID environment variable to deployment * feat: update log level key in configmap and values.yaml for consistency * test: unit tests for fsProvider, s3Provider, tileDeletionStrategy(MAPCO-9806) (#25) * test: unit tests for fsProvider, s3Provider, tileDeletionStrategy * fix: correct mocked return value for stat in FsStorageProvider tests to be more readable * Update tests/storageProviders/fsStorageProvider.spec.ts Co-authored-by: Shlomi k <65117898+CL-SHLOMIKONCHA@users.noreply.github.com> * test: improve clarity of targetExists test description and expectation --------- Co-authored-by: Shlomi k <65117898+CL-SHLOMIKONCHA@users.noreply.github.com> --------- Co-authored-by: Copilot <copilot@github.com> Co-authored-by: Shlomi k <65117898+CL-SHLOMIKONCHA@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
New and Improved Unit Tests:
FsStorageProvider, covering file deletion, directory cleanup, and existence checks, including error scenarios and edge cases.S3StorageProvider, covering object deletion (including batching, error handling, and S3-specific behaviors), prefix existence checks, and client construction.Test Helpers and Mocks:
tests/helpers/mocks.tswith new mock creators for storage providers, S3 config, and strategy config, and added a mock implementation forupdateProgresson the queue client. [1] [2] [3]Test Infrastructure Improvements:
strategyFactory.spec.tsto use the new mock config, queue client, and storage providers, improving test isolation and reliability. [1] [2]Minor Implementation Improvements:
FsStorageProvider.deleteto ensure failed paths are correctly reported for all error types.S3StorageProvider.