feat: add per-sample assemblies to the metagenome binning dataset by dialvarezs · Pull Request #2255 · nf-core/test-datasets

dialvarezs · 2026-09-02T04:11:52Z

SemiBin2 multi_easy_bin and Vamb, and co-assembly binners in general, need one assembly per sample combined into a single FASTA with the sample as a header prefix. The binning dataset added in #2237 shares one assembly across its three samples, so it cannot drive that mode, and nf-core/modules still tests semibin/multieasybin against data under delete_me/.

This adds binning/multisample/, covering the same community in that form. It is derived entirely from the files already in binning/, so there is no second simulation to keep in step:

the reads come back out of the published BAMs with samtools fastq, split by genome on the reference name
the contigs are the published 5 kb pieces stitched into blocks of 8, with each sample taking a seeded 60% of them, which keeps the contig count near what a real assembly would give
each sample takes one genome's reads from one published sample, giving five abundance ratios out of the three that exist

Contents are contigs.fasta.gz, s1 to s5 .sorted.bam with indexes, s1 to s5 .aemb.tsv and abundances.tsv.

It carries five samples rather than three because SemiBin2 only accepts --abundance in place of BAMs from five samples up. Both tools' abundance forms are included, since they differ: SemiBin reads one file per sample over the split contigs SemiBin2 split_contigs produces, Vamb one wide table over the contigs themselves.

The separator is C, which is Vamb's default and what SemiBin takes as -s C. Using SemiBin's own default of : works for both tools but leaves a colon in the names of the bins Vamb writes.

Verified on all four paths. SemiBin2 2.4.1 returns 2, 2, 3, 2 and 2 bins from the BAMs and 2 per sample from the abundance files, about 70 seconds a run. Vamb 5.0.4 returns 53 clusters in 163 split bins from either input, at the low epoch count nf-core/modules uses for testing. The README section documents the full recipe and the seeds.

Draft until nf-core/modules#12863 is pointed at it.

🤖 Generated with Claude Code

`multi_easy_bin` needs one assembly per sample, which the binning dataset did not provide, so these tests still pointed at `delete_me/semibin2/`. The dataset now carries that form under `binning/multisample/`, and all three tests use it. Note the new data has five samples, not the three the rest of the binning dataset uses. SemiBin2 only accepts `--abundance` in place of BAMs from five samples up, so that is what the abundance test needs, and it also matches what the old data had. Depends on nf-core/test-datasets#2255. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

`SemiBin2 multi_easy_bin` and `Vamb`, and co-assembly binners in general, need one assembly per sample combined into a single FASTA with the sample as a header prefix. The existing files share one assembly across the three samples, so they cannot drive that mode, and the modules repo still tests it against data under `delete_me/`. `multisample/` covers the same community in that form, derived entirely from the files already here so there is no second simulation to keep in step. The reads come back out of the published BAMs with `samtools fastq`, split by genome on the reference name. The contigs are the published 5 kb pieces stitched into blocks of 8, with each sample taking a seeded 60% of them, which keeps the contig count near what a real assembly would give. Each sample takes one genome's reads from one published sample, giving five abundance ratios out of the three that exist. It carries five samples rather than three because SemiBin2 only accepts abundance tables in place of BAMs from five samples up. Both tools' abundance forms are included, since they differ: SemiBin reads one file per sample over the split contigs, Vamb one wide table over the contigs themselves. The separator is `C`, Vamb's default, which SemiBin takes as `-s C`. Using SemiBin's own default of `:` works too but leaves a colon in the names of the bins Vamb writes. Verified on all four paths. SemiBin2 2.4.1 returns 2, 2, 3, 2 and 2 bins from the BAMs and 2 per sample from the abundance files. Vamb 5.0.4 returns 53 clusters in 163 split bins from either input. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

`multi_easy_bin` needs one assembly per sample, which the binning dataset did not provide, so these tests still pointed at `delete_me/semibin2/`. The dataset now carries that form under `binning/multisample/`, and all three tests use it. Note the new data has five samples, not the three the rest of the binning dataset uses. SemiBin2 only accepts `--abundance` in place of BAMs from five samples up, so that is what the abundance test needs, and it also matches what the old data had. Depends on nf-core/test-datasets#2255. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

erikrikarddaniel

👍

`multi_easy_bin` needs one assembly per sample, which the binning dataset did not provide, so these tests still pointed at `delete_me/semibin2/`. The dataset now carries that form under `binning/multisample/`, added in nf-core/test-datasets#2255, and all three tests use it. Note the new data has five samples, not the three the rest of the binning dataset uses. SemiBin2 only accepts `--abundance` in place of BAMs from five samples up, so that is what the abundance test needs, and it also matches what the old data had. The contigs use `C` as the sample separator, which is Vamb's default, so the tests pass `-s C` rather than relying on SemiBin's own default of `:`. That keeps the data usable by both tools without a colon ending up in the names of the bins Vamb writes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

…f-core#12863) * feat(semibin): add GPU support to single_easy_bin and multi_easy_bin Select the container and conda environment from `task.accelerator`, and pass the resolved device to SemiBin2 with `--engine`. The default is `auto`, which silently falls back to CPU, so making it explicit keeps the declared accelerator and the device actually used in agreement. Both modules also emit the CUDA runtime version to the versions topic, and carry the `process_gpu` label so the accelerator comes from the pipeline's gpu profile. The singleeasybin test moves to the shared metagenome binning dataset, with all three BAMs so the multi-BAM coverage path is exercised. That means dropping `--environment global`: the pretrained model only supports single-sample binning, and it skips training altogether, which is the part the GPU path is about. Bin names and contents are not stable across machines, so the snapshot follows what multieasybin already does and keeps the stable outputs plus a bin count assertion. multieasybin keeps its own test data. `multi_easy_bin` needs a concatenated FASTA of several assemblies, which the binning dataset does not provide. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(semibin/multieasybin): move off the delete_me test data `multi_easy_bin` needs one assembly per sample, which the binning dataset did not provide, so these tests still pointed at `delete_me/semibin2/`. The dataset now carries that form under `binning/multisample/`, added in nf-core/test-datasets#2255, and all three tests use it. Note the new data has five samples, not the three the rest of the binning dataset uses. SemiBin2 only accepts `--abundance` in place of BAMs from five samples up, so that is what the abundance test needs, and it also matches what the old data had. The contigs use `C` as the sample separator, which is Vamb's default, so the tests pass `-s C` rather than relying on SemiBin's own default of `:`. That keeps the data usable by both tools without a colon ending up in the names of the bins Vamb writes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(semibin): say why the bins are left out of the snapshots The comment said the names and number of bins are not stable, without saying how that was established. Trying the opposite established it: snapshotting everything reproduces byte for byte across repeated local runs, including every bin checksum, and then fails on CI for both modules, on every profile. Pinning the visible cores and the OpenMP and BLAS thread counts does not change the local result either, so the drift is not down to threading and cannot be pinned from a test config. So the sanitized form stays, and the comments now say the drift is between machines rather than between runs. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(semibin): check that every contig lands in a bin With the bins out of the snapshot, the tests only checked that some bins came out and that the csv and tsv files had the expected names. One thing holds regardless of how the clustering lands, so it is asserted directly: the contigs SemiBin featurised in `data.csv` are exactly the contigs written across the bins, each once. For multi_easy_bin that is checked per sample, which also covers the sample split itself. The bins are read with the nft-fasta plugin, which is loaded for the repo here, so a bin that is not FASTA fails the same assertion. The snapshot stays for the versions and file names. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>

dialvarezs force-pushed the binning-multisample branch from 07354da to 6a2f5c0 Compare September 2, 2026 04:23

dialvarezs mentioned this pull request Sep 2, 2026

feat(semibin): add GPU support to single_easy_bin and multi_easy_bin nf-core/modules#12863

Merged

12 tasks done

dialvarezs force-pushed the binning-multisample branch from 6a2f5c0 to 2f48388 Compare September 2, 2026 13:23

dialvarezs force-pushed the binning-multisample branch from 2f48388 to 7e7af39 Compare September 2, 2026 13:42

dialvarezs marked this pull request as ready for review September 2, 2026 14:04

erikrikarddaniel approved these changes Sep 2, 2026

View reviewed changes

dialvarezs merged commit 561cacd into nf-core:modules Sep 2, 2026
2 checks passed

Navigation Menu

Repository navigation

feat: add per-sample assemblies to the metagenome binning dataset - #2255

feat: add per-sample assemblies to the metagenome binning dataset#2255
dialvarezs merged 1 commit into
nf-core:modulesfrom
dialvarezs:binning-multisample

dialvarezs commented Sep 2, 2026 •

edited

Loading

Uh oh!

erikrikarddaniel left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

dialvarezs commented Sep 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

erikrikarddaniel left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

dialvarezs commented Sep 2, 2026 •

edited

Loading