Fix metadata file placement using outdir param#320
Open
drernie wants to merge 2 commits into
Open
Conversation
When the Nextflow outdir param points deeper than the Quilt package root (e.g. s3://bucket/ns/pkg/run-name/), README_NF_QUILT.md, quilt_summarize.json, and nf-quilt/*.json metadata files now land inside the run subdirectory rather than at the package root. This fixes the reported issue where these files appeared one level above the pipeline output. The observer extracts `outdir` from session params and passes it through to QuiltProduct, which computes the prefix by stripping the bucket/namespace/name components from the outdir path. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
🦙 MegaLinter status: ✅ SUCCESS
See detailed report in MegaLinter reports |
Tests the outdir-aware packaging fix by using a sub-path (s3-test/deep-run) that extends past the package root, verifying README/summarize land inside the run subdirectory. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Summary
outdirpoints deeper than the Quilt package root (e.g.s3://bucket/ns/pkg/run-name/),README_NF_QUILT.md,quilt_summarize.json, andnf-quilt/*.jsonnow land inside the run subdirectory instead of the package rootoutdirfromsession.paramsand passes it toQuiltProduct, which strips bucket/namespace/name to compute the sub-path prefixContext
Reported by Austin Hovland: when using automatic S3 packaging (plain
s3://outdir rather thanquilt+s3://), theuriFromS3File()algorithm correctly maps files into the package subdirectory, but README/summarize/metadata files were always written to the package root. This made the package structure inconsistent — pipeline outputs underrun-name/but metadata at the top level.Test plan
computeOutdirPrefixwith various outdir formats (null, empty, exact match, with sub-paths, trailing slashes)prefixedPathprepends correctly when outdir prefix is present vs absentwriteReadmeandwriteSummarizeplace files under the prefixQuiltPkgTestfailures due to version bump —dest-0.9.2package doesn't exist on S3 yet)🤖 Generated with Claude Code