feat: Add vcf_annotate_vepyr subworkflow

[mwiewior]

Adds vcf_annotate_vepyr, an end-to-end VCF annotation subworkflow built on vepyr, a Rust reimplementation of Ensembl VEP.

It composes three existing modules:

• huggingface/download, which gained whole-repository downloads in feat: huggingface/download: support whole repository downloads #13071
• bcftools/norm
• vepyr/annotate, added in Add vepyr/annotate module #13001

Steps

1. Download the cache (optional). When val_hf_repo is set (e.g. biodatageeks/vepyr_116_GRCh38_ensembl), the vepyr Parquet cache is downloaded from that Hugging Face dataset once and shared across all samples. Otherwise the local cache in ch_cache is used. Setting both, or neither, is an error. An optional revision and chromosome filters are passed through HUGGINGFACE_DOWNLOAD's ext.args.
2. Normalise (optional, val_normalize). Runs bcftools norm. meta.yml documents the configuration used in vepyr's Ensembl VEP parity tests: split multiallelic records, no indel left-alignment. If no index is written, vepyr builds one.
3. Annotate. vepyr annotate --everything emits [ meta, vcf.gz, tbi ].

Tests

All tests use the HG002 / Ensembl 116 chr22 fixtures from nf-core/test-datasets#2270:

• normalise, with and without index generation, on an input with 14 multiallelic records. A setup step builds that input by joining input.vcf.gz with bcftools norm --multiallelics +both. The tests assert 1,000 biallelic output records and the same variantsMD5 as annotating input.vcf.gz without normalisation.
• no normalise
• Hugging Face cache shared across two samples (revision pinned, chr22 only), asserting a single download
• failures when both or neither of ch_cache and val_hf_repo are set
• stubs for the local cache and the Hugging Face path

Validation:

• nf-test 0.9.5, Nextflow 26.04.6, --ci: 8/8 pass with --profile=+docker,arm64 and with --profile=+conda (osx-arm64).
• Annotated outputs from both the local and the Hugging Face cache have the same record-body MD5, 1d4a92b815eb6193e6f4546f4ab35978, across 1,000 records.
• nf-core subworkflows lint passes (35 passed, 0 warnings) and the prek hooks pass.
• x64 Docker, Singularity and Conda run in upstream CI.

Generated by Claude Code

PR checklist

• This comment contains a description of changes (with reason).
• If you've fixed a bug or added code that should be tested, add tests!
• Follow the component conventions in the contribution docs.
• If necessary, include test data in your PR.
• Remove all TODO statements.
• Broadcast software version numbers to topic: versions (provided by the modules).
• Follow the naming conventions.
• Follow the parameters requirements.
• Follow the input/output options guidelines.
• Add a resource label (provided by the modules).
• Use BioConda and BioContainers if possible to fulfil software requirements.
• Ensure that the test works with either Docker / Singularity. Conda CI tests can be quite flaky:
  • nf-core subworkflows test vcf_annotate_vepyr --profile docker
  • nf-core subworkflows test vcf_annotate_vepyr --profile singularity (upstream CI)
  • nf-core subworkflows test vcf_annotate_vepyr --profile conda (upstream CI)

[jonasscheid]

Nice, I suggest using the nf-core/agents AGENTS.md for future PRs

--> The AI disclosure doesn't match nf-core/agents. The commits say "Generated by Codex" plus a Claude co-author, and the PR body ends with a Claude Code footer. nf-core/agents asks for "Generated by {name}" placed above the checklist.

[jonasscheid]

I think more user friendly would be to throw an error if ch_cach and val_hf_repo are set

[mwiewior]

Done in cb18242. The subworkflow now fails with set either ch_cache or val_hf_repo, not both when both are set. For symmetry it also fails with no cache given, set ch_cache or val_hf_repo when neither is set; before, that case surfaced later as an unclear VEPYR_ANNOTATE error. Two new stub tests, cache and Hugging Face repo - fails and no cache - fails, assert workflow.failed and the message.

[jonasscheid]

AI-hint:

The Hugging Face path only works if the pipeline also sets ext.args = '--repo-type dataset' on HUGGINGFACE_DOWNLOAD. That flag is required, not optional, and nf-core guidance says required arguments should be inputs, not ext.args.

[mwiewior]

Done in cb18242, without a new input. vepyr caches are always HF datasets, so the subworkflow passes datasets/<owner>/<repo> to HUGGINGFACE_DOWNLOAD (a datasets/ prefix already in val_hf_repo is kept). hf download 1.18.0, the version the module pins, resolves that ID without --repo-type; the bare ID is looked up as a model. Nothing required is left in ext.args, only the optional --revision/--include. I've removed --repo-type dataset from the test config and meta.yml. The real two-sample HF test still passes with the same variantsMD5.

[jonasscheid]

optional:

The normalisation branch is never actually tested. (tests L18, tests L75) I checked the test data: input.vcf.gz has 1,000 records and none are multiallelic. Combined with --do-not-normalize, bcftools norm changes nothing.
That is why the normalise and no-normalise snapshots share the same variantsMD5 (fdddbf25…).
That MD5 is computed over full record lines, INFO included, so the annotation itself is covered. Only the split step is untested.
A test VCF with one or two multiallelic sites would fix this.

[mwiewior]

Fixed in cb18242. nf-core/test-datasets#2298 adds input_multiallelic.vcf.gz: the same HG002 window before normalisation, 986 records, 14 of them multiallelic. Both normalise tests now read it and assert 1,000 output records, all biallelic. Splitting it reproduces input.vcf.gz exactly, so the snapshots keep variantsMD5 fdddbf25…. That is the same as the no-normalise test, but now it shows split + annotate equals annotating the pre-split input; before, it was a no-op. The two tests need #2298 merged before they pass in CI; locally they pass against its branch.

[mwiewior]

Follow-up in 643d2b1: per the suggestion on nf-core/test-datasets#2298, the normalise tests no longer need new test data. A setup step runs BCFTOOLS_NORM (aliased BCFTOOLS_JOIN_MULTIALLELICS) with --multiallelics +both --do-not-normalize on input.vcf.gz. That joins the 1,000 records into 986, 14 of them multiallelic, and the subworkflow splits them again. --do-not-normalize is needed because the module always passes --fasta-ref, and left-aligning fails on the VCF's chr22 vs the FASTA's 22. The task logs confirm 14 records joined in setup and 14 split in the subworkflow. Snapshots are unchanged, and the tests pass against the published test data.

[jonasscheid]

The "sample id matches input basename" test checks bcftools/norm behaviour, not subworkflow logic. (tests L231) That module already has its own test for it (from #13076). Dropping it here saves a full vepyr run in CI.

[mwiewior]

Removed in cb18242, together with its snapshot.

[jonasscheid]

de-capitalize Channel. Wont work anymore with higher nextflow versions. Should be channel

[mwiewior]

Done in cb18242. Every Channel. in the tests is now channel..

[jonasscheid]

Docs and prose (nf-core/agents prose rules)

One sentence doesn't parse. (meta.yml L26-29) "The configuration vepyr's Ensembl VEP parity tests are validated on splits multiallelic records…" needs rewording. It also says the index "vepyr needs", but VEPYR_ANNOTATE builds the index itself when it is missing. Writing it in bcftools norm only saves a tabix step.
The ext.prefix paragraph describes module internals. (meta.yml L37-39) Under the rule to scope prose to the immediate task, it can go.
"Pipelines should default it to true" appears three times. Keep one. The three places are:
meta.yml L25, in the description
meta.yml L78, in the val_normalize input
main.nf L16, as a comment
The constraints on ch_fasta and ch_cache aren't documented. (meta.yml L57, meta.yml L62)
Both must be value channels. If a pipeline passes a queue channel, only the first sample gets annotated.
BCFTOOLS_NORM output must be bgzipped VCF. -Ob and -Ou produce BCF, and -Ov produces plain VCF, which makes vepyr fall back to --fork 1.

[mwiewior]

All addressed in cb18242:

• Reworded the configuration sentence. --write-index=tbi is now described as optional: it only saves the tabix step, because VEPYR_ANNOTATE builds a missing index.
• Removed the ext.prefix paragraph.
• "Default it to true" now appears once, in the val_normalize description.
• ch_cache and ch_fasta are documented as value channels (channel.value(...) or .collect()), noting that a queue channel annotates only the first sample.
• Documented keeping BCFTOOLS_NORM output as bgzip VCF (--output-type z): VEPYR_ANNOTATE runs a single pipeline for BCF (-Ob/-Ou) and plain VCF (-Ov).

[mwiewior]

@jonasscheid I've fixed the disclosure: the PR body now ends with Generated by Claude Code above the checklist, and the review commit (cb18242) ends its message with the same line, without the co-author trailer. I haven't rewritten the earlier commits because the PR is squash-merged. All six threads are answered. The two normalise tests need nf-core/test-datasets#2298 merged before CI passes.

[jonasscheid]

LGTM