Repository navigation
Conversation
input_multiallelic.vcf.gz is the GIAB HG002 chr22:20572272-21735973 window of input.vcf.gz before bcftools norm: 986 records, 14 multiallelic. Splitting it with bcftools norm -m -both gives the records of input.vcf.gz, so the vcf_annotate_vepyr subworkflow can test its normalization step. Generated by Claude Code
mwiewior
added a commit
to mwiewior/modules
that referenced
this pull request
Oct 4, 2026
- Prefix val_hf_repo with datasets/, so HUGGINGFACE_DOWNLOAD needs no required --repo-type in ext.args; only --revision/--include remain. - Fail when both ch_cache and val_hf_repo are set, or neither, and test both failures. - Test normalisation on input_multiallelic.vcf.gz (986 records, 14 multiallelic; nf-core/test-datasets#2298): the output has 1,000 biallelic records and the same variantsMD5 as annotating the pre-split input.vcf.gz without normalisation. - Drop the basename test (covered by the bcftools/norm module tests) and use channel instead of Channel. - meta.yml: reword the bcftools norm configuration, drop the ext.prefix paragraph, keep one "default to true", document that ch_cache and ch_fasta must be value channels and that BCF or plain VCF output makes VEPYR_ANNOTATE run a single pipeline. Generated by Claude Code
This was referenced Oct 4, 2026
|
I think you dont need more testdata, you can use setup to manipulate right? setup {
run("UNTAR") { ... } // unchanged
run("BCFTOOLS_NORM", alias: "BCFTOOLS_JOIN_MULTIALLELICS") {
script "modules/nf-core/bcftools/norm/main.nf"
process {
"""
input[0] = channel.of([
[ id:'test_split' ],
file(params.modules_testdata_base_path + 'genomics/homo_sapiens/vepyr/input.vcf.gz', checkIfExists: true),
file(params.modules_testdata_base_path + 'genomics/homo_sapiens/vepyr/input.vcf.gz.tbi', checkIfExists: true)
])
input[1] = channel.value([
[ id:'GRCh38' ],
file(params.modules_testdata_base_path + 'genomics/homo_sapiens/vepyr/reference.fa.gz', checkIfExists: true)
])
"""
}
}
}
// when: input[0] = BCFTOOLS_JOIN_MULTIALLELICS.out.vcf.map { _meta, vcf -> [ [ id:'test' ], vcf, [] ] }It also needs this in tests/nextflow.config: withName: 'BCFTOOLS_JOIN_MULTIALLELICS' {
ext.args = '--multiallelics +both --output-type z'
}Rest looks good to me |
mwiewior
added a commit
to mwiewior/modules
that referenced
this pull request
Oct 5, 2026
Instead of a new test-datasets file (nf-core/test-datasets#2298), the normalise tests join input.vcf.gz back into multiallelic records with an aliased BCFTOOLS_NORM in setup: --multiallelics +both turns 1,000 records into 986, 14 of them multiallelic, and the subworkflow splits them again. --do-not-normalize is needed because the module always passes --fasta-ref, and left-aligning fails on chr22 vs 22. Snapshots are unchanged. Generated by Claude Code
mwiewior
added a commit
to biodatageeks/vepyr
that referenced
this pull request
Oct 5, 2026
Sync the vcf_annotate_vepyr tests with nf-core/modules#13070 643d2b157: the normalise tests join input.vcf.gz back into multiallelic records with an aliased BCFTOOLS_NORM in setup, so the staged input_multiallelic.vcf.gz (nf-core/test-datasets#2298) is no longer needed. Revert its staging in stage_testdata.py and the harness hash, run the aliased process under amd64 emulation like BCFTOOLS_NORM, and drop the #2298 references from the README. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Author
|
hey @jonasscheid thanks for the hint! I fixed that in the main sub-workflow PR. Closing this one. |
mwiewior
added a commit
to biodatageeks/vepyr
that referenced
this pull request
Oct 5, 2026
…d badge (#167) * nf-core: sync vcf_annotate_vepyr with nf-core/modules#13070 Mirror the subworkflow as it stands after the #13070 review round: optional Hugging Face cache download (val_hf_repo, datasets/ prefix added by the subworkflow), errors when both or neither of ch_cache and val_hf_repo are set, normalization tests on a multiallelic input, and the reworded meta.yml. The dev harness follows: bcftools/norm pinned past #13076 (default ${meta.id}_norm prefix and collision guard), huggingface/download fetched at the same commit and run under amd64 emulation, the parity test passes the new seventh argument, and stage_testdata.py builds input_multiallelic.vcf.gz from raw_chr22.vcf.gz, checked against the md5 published in nf-core/test-datasets#2298. Docs: nextflow.md drops the required ext.prefix, documents the Hugging Face cache and value-channel inputs, and copies both helper modules; the subworkflow diagram gains the download branch. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * nf-core: build the multiallelic test input in setup, as upstream Sync the vcf_annotate_vepyr tests with nf-core/modules#13070 643d2b157: the normalise tests join input.vcf.gz back into multiallelic records with an aliased BCFTOOLS_NORM in setup, so the staged input_multiallelic.vcf.gz (nf-core/test-datasets#2298) is no longer needed. Revert its staging in stage_testdata.py and the harness hash, run the aliased process under amd64 emulation like BCFTOOLS_NORM, and drop the #2298 references from the README. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs: advertise the merged nf-core subworkflow vcf_annotate_vepyr merged as nf-core/modules#13070 (7d5d5ab); the mirror here is byte-identical to it. Install it with `nf-core subworkflows install vcf_annotate_vepyr`, which also installs vepyr/annotate, bcftools/norm and huggingface/download, instead of copying it from this repository. Add an nf-core subworkflow badge next to the module badge and mark the submission steps done. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
famosab
pushed a commit
to famosab/modules
that referenced
this pull request
Oct 5, 2026
* Add vcf_annotate_vepyr subworkflow Compose optional bcftools normalisation with vepyr annotation and add published-fixture tests for indexed, unindexed, bypass and stub paths. Generated by Codex * vcf_annotate_vepyr: support Hugging Face cache repositories Add an optional repository input that downloads one shared cache before annotation, retaining the existing local-cache path. Document revision and contig filters. Exercise local-cache bypass, remote stubs and a real two-sample annotation using the published chr22 cache. Generated by Codex * vcf_annotate_vepyr: apply test config per test Load ./nextflow.config only in the tests that rely on its ext.args (BCFTOOLS_NORM, HUGGINGFACE_DOWNLOAD), following the nf-core testing spec and the review of nf-core#13071. The no-normalise test runs without it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * vcf_annotate_vepyr: run UNTAR setup only where the cache tarball is used Move the UNTAR setup into the three local-cache tests so the Hugging Face and stub tests no longer fetch and extract cache.tar.gz. Mention the optional Hugging Face download in the meta.yml summary. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * bcftools/norm: refuse to write output over the staged input With no ext.prefix and meta.id equal to the input basename, the default output name matches the staged input symlink, so bcftools norm rewrites the source file through it. Add the input/output collision guard used by bcftools/annotate, csq, filter, convert and pluginfilltags, in script and stub, with module and vcf_annotate_vepyr regression tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * vcf_annotate_vepyr: move the bcftools/norm guard to nf-core#13076 The input/output collision guard is a module fix, so it now lives in its own PR. Keep the subworkflow regression test, which passes once nf-core#13076 is merged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * vcf_annotate_vepyr: drop the collision test until nf-core#13076 lands The test asserts the bcftools/norm guard from nf-core#13076, which is not on master yet, so it fails in CI. Restore the meta.yml wording that matches the current module; both come back once nf-core#13076 is merged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * vcf_annotate_vepyr: rely on the bcftools/norm default prefix from nf-core#13076 bcftools/norm now defaults to ${meta.id}_norm and refuses colliding prefixes, so drop the ext.prefix workaround from the documented config. Add a test where the sample ID matches the input basename. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * vcf_annotate_vepyr: address review of nf-core#13070 - Prefix val_hf_repo with datasets/, so HUGGINGFACE_DOWNLOAD needs no required --repo-type in ext.args; only --revision/--include remain. - Fail when both ch_cache and val_hf_repo are set, or neither, and test both failures. - Test normalisation on input_multiallelic.vcf.gz (986 records, 14 multiallelic; nf-core/test-datasets#2298): the output has 1,000 biallelic records and the same variantsMD5 as annotating the pre-split input.vcf.gz without normalisation. - Drop the basename test (covered by the bcftools/norm module tests) and use channel instead of Channel. - meta.yml: reword the bcftools norm configuration, drop the ext.prefix paragraph, keep one "default to true", document that ch_cache and ch_fasta must be value channels and that BCF or plain VCF output makes VEPYR_ANNOTATE run a single pipeline. Generated by Claude Code * vcf_annotate_vepyr: build the multiallelic test input in setup Instead of a new test-datasets file (nf-core/test-datasets#2298), the normalise tests join input.vcf.gz back into multiallelic records with an aliased BCFTOOLS_NORM in setup: --multiallelics +both turns 1,000 records into 986, 14 of them multiallelic, and the subworkflow splits them again. --do-not-normalize is needed because the module always passes --fasta-ref, and left-aligning fails on chr22 vs 22. Snapshots are unchanged. Generated by Claude Code --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds an unnormalized input VCF for the
vcf_annotate_vepyrsubworkflow (nf-core/modules#13070). The reviewer there noted that the existinginput.vcf.gzhas no multiallelic records, so the subworkflow'sbcftools normstep changes nothing and is never really tested.Files,
data/genomics/homo_sapiens/vepyr/input_multiallelic.vcf.gz+.tbiinput.vcf.gz(chr22:20572272-21735973), before normalization: 986 records, 14 of them multiallelicIt reuses the existing
cache.tar.gzandreference.fa.gz, so nothing else is added.Checked
bcftools norm --fasta-ref reference.fa.gz -m -both --do-not-normalize(the subworkflow's documented configuration) splits 14 records into 28. The record body of the output is byte-identical toinput.vcf.gz(md5ca50096a40b2796376f02df03587516e). Normalizing this file and then annotating it should therefore give the same result as annotatinginput.vcf.gz.##bcftools_*Commandlines removed.Generated by Claude Code
PR checklist
README.md