Skip to content

Fix metadata file placement using outdir param#320

Open
drernie wants to merge 2 commits into
mainfrom
fix/outdir-aware-packaging
Open

Fix metadata file placement using outdir param#320
drernie wants to merge 2 commits into
mainfrom
fix/outdir-aware-packaging

Conversation

@drernie

@drernie drernie commented Apr 6, 2026

Copy link
Copy Markdown
Member

Summary

  • When outdir points deeper than the Quilt package root (e.g. s3://bucket/ns/pkg/run-name/), README_NF_QUILT.md, quilt_summarize.json, and nf-quilt/*.json now land inside the run subdirectory instead of the package root
  • The observer extracts outdir from session.params and passes it to QuiltProduct, which strips bucket/namespace/name to compute the sub-path prefix
  • Bumps plugin version to 0.9.2

Context

Reported by Austin Hovland: when using automatic S3 packaging (plain s3:// outdir rather than quilt+s3://), the uriFromS3File() algorithm correctly maps files into the package subdirectory, but README/summarize/metadata files were always written to the package root. This made the package structure inconsistent — pipeline outputs under run-name/ but metadata at the top level.

Test plan

  • New unit tests for computeOutdirPrefix with various outdir formats (null, empty, exact match, with sub-paths, trailing slashes)
  • New tests verifying prefixedPath prepends correctly when outdir prefix is present vs absent
  • New tests verifying writeReadme and writeSummarize place files under the prefix
  • All 233 existing tests pass (3 pre-existing QuiltPkgTest failures due to version bump — dest-0.9.2 package doesn't exist on S3 yet)

🤖 Generated with Claude Code

When the Nextflow outdir param points deeper than the Quilt package root
(e.g. s3://bucket/ns/pkg/run-name/), README_NF_QUILT.md, quilt_summarize.json,
and nf-quilt/*.json metadata files now land inside the run subdirectory rather
than at the package root. This fixes the reported issue where these files
appeared one level above the pipeline output.

The observer extracts `outdir` from session params and passes it through to
QuiltProduct, which computes the prefix by stripping the bucket/namespace/name
components from the outdir path.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@github-actions

github-actions Bot commented Apr 6, 2026

Copy link
Copy Markdown

🦙 MegaLinter status: ✅ SUCCESS

Descriptor Linter Files Fixed Errors Warnings Elapsed time
✅ MARKDOWN markdown-link-check 1 0 0 1.16s
✅ MARKDOWN markdown-table-formatter 1 0 0 0 0.18s
✅ REPOSITORY checkov yes no no 10.34s
✅ REPOSITORY git_diff yes no no 0.01s
✅ REPOSITORY grype yes no no 27.89s
✅ REPOSITORY secretlint yes no no 0.48s
✅ REPOSITORY syft yes no no 0.94s
✅ REPOSITORY trivy yes no no 5.24s
✅ REPOSITORY trivy-sbom yes no no 0.09s
✅ REPOSITORY trufflehog yes no no 2.22s

See detailed report in MegaLinter reports
Set VALIDATE_ALL_CODEBASE: true in mega-linter.yml to validate all sources, not only the diff

MegaLinter is graciously provided by OX Security

Tests the outdir-aware packaging fix by using a sub-path
(s3-test/deep-run) that extends past the package root,
verifying README/summarize land inside the run subdirectory.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant