Add in-browser zip download for small dandisets

[bendichter]

Motivation

Requiring the DANDI CLI to download a dandiset is overkill for small datasets. The current Download menu only hands the user a dandi download ... command — there's no way to grab a small dandiset straight from the browser.

This adds a Download .zip button to the dandiset Download menu that bundles the dataset client-side and saves it through the normal browser download mechanism.

How it works

• Fetches each asset's blob directly from S3 (the bucket already serves Access-Control-Allow-Origin: *, so browser fetch works) and zips them client-side with client-zip (~5 KB).
• No backend changes and no server bandwidth cost — bytes go S3 → browser, preserving the existing S3-direct model. The button is purely additive; the CLI flow is untouched.
• Files nest under a single root folder matching the zip filename (e.g. 000123-draft/sub-01/...).
• Byte-level progress bar and a Cancel button (backed by an AbortController that interrupts both the asset listing and in-flight fetches).
• Zarr assets are skipped (they're not single blobs) with a user-facing note pointing to the CLI.

Gating

Shown only for "small" versions: ≤ 2 GiB and ≤ 1000 files (constants at the top of the component). The archive is buffered in memory (client-zip → Blob) before saving, which is why the cap exists.

Video of sandbox testing below

Status / open questions

• Memory cap: the in-memory buffer is the reason for the 2 GiB limit. A follow-up could stream to disk via the File System Access API (createWritable()) to lift the cap.
• Thresholds (2 GiB / 1000 files) are guesses — feedback welcome.

🤖 Generated with Claude Code

[bendichter]

download_button_demo.mov

[bendichter]

@yarikoptic , what do you think? Is this useful?

[yarikoptic]

this reminded me about

• https://github.com/con/jsdownloader

and see there

• Please contribute your ideas and feedback con/jsdownloader#1
• OpenNeuro implementation con/jsdownloader#2

[bendichter]

In-browser zip download eligibility

The branch gates the feature on: has data (size > 0), size ≤ 2 GiB, and asset_count ≤ 1000.

┌──────────────────────────────────────┬───────┐
│                Metric                │ Count │
├──────────────────────────────────────┼───────┤
│ Total dandisets                      │ 869   │
├──────────────────────────────────────┼───────┤
│ With any data (size > 0)             │ 605   │
├──────────────────────────────────────┼───────┤
│ Eligible (≤ 2 GiB and ≤ 1000 assets) │ 209   │
└──────────────────────────────────────┴───────┘

• ~24% of all dandisets are eligible.
• ~35% of dandisets that have data are eligible (the other 264 dandisets are empty placeholders with 0 bytes).

@CodyCBakerPhD

[bendichter]

@yarikoptic @CodyCBakerPhD, summarizing our discussion from Monday's call, plus my response to the jsdownloader idea.

Where we landed on the call:

• Is the feature worth having? Yarik: yes, this solves a real problem. The main open questions were scope of applicability and modularity.
• How many dandisets does this apply to? (Cody's question) Answered above: 209 of 869 dandisets are eligible under the current thresholds (~24% overall, ~35% of dandisets that actually contain data). So this isn't an edge-case feature.
• Thresholds: the 2 GiB / 1000 file caps are deliberately conservative because the zip is buffered in memory before saving. They can be raised later by streaming to disk (File System Access API), which is noted as a follow-up in the PR description.
• Yarik's concerns: (1) the generic download-list-of-URLs logic should live in a reusable library (per con/jsdownloader) so DataLad catalogs, OpenNeuro, etc. could share it; (2) a hard threshold invites "one byte over" complaints; (3) this should generalize beyond whole dandisets to folders, subjects, or any collection of assets ("a dandiset is just a folder").

On the library question:

I'd argue the generic library already exists: it's client-zip (~5 KB, maintained, MIT), which is what this PR uses. It takes named byte streams and produces a zip client-side, which is exactly the reusable core. What this PR adds on top is almost entirely DANDI-specific: asset pagination via our REST client, Zarr filtering, eligibility gating on version metadata, and Vuetify UI. The truly generic glue (sequential fetch with byte-level progress tracking and an AbortController) is about 60 lines, which I don't think clears the bar for a separately versioned and released package. My general rule is to generalize once something is needed in three places, and I'm happy to help extract a library at that point.

Middle path I'd propose: I'll refactor the generic part into a composable inside this repo, roughly useZipDownload(entries: {name, url, size}[]), with DownloadDialog.vue feeding it dandiset assets. That gives us the modular seam Yarik is asking for: a folder- or subject-level download button in the file browser later just feeds it a filtered asset list, and if a third consumer ever shows up, publishing it as a standalone package becomes mechanical rather than speculative.

On the threshold complaint-magnet concern: when a dandiset is over the limit, nothing is lost. The CLI instructions still show, exactly as today. The button is purely additive, and we can tune messaging (e.g., "too large for browser download, use the CLI") rather than architecture.

If that plan sounds reasonable, I'll do the composable refactor in this PR and file a follow-up issue for the folder-level button and the streaming-to-disk cap raise.

[bendichter]

One known gap: the zip path uses a raw fetch(), which doesn't attach the API auth token that our axios client adds, so asset downloads from an embargoed dandiset would fail with a 401. I'll either hide the button on embargoed versions or route the requests through the authenticated client as part of the refactor.

[bendichter]

I've pushed the refactor described above. The generic logic now lives in a composable, web/src/composables/useZipDownload.ts, which takes a zip filename and a function returning a list of {name, url, size} entries, and handles the sequential fetching, byte-level progress, cancellation, and client-side zipping. DownloadDialog.vue keeps only the DANDI-specific parts: asset pagination, Zarr filtering, eligibility gating, and the UI. This gives us the modular seam discussed on the call: a folder- or subject-level download button in the file browser would just feed the same composable a filtered asset list, and extracting a standalone package later becomes mechanical if a third consumer shows up.

Two other changes in the same commit:

1. The embargo gap I noted earlier is closed by hiding the button on embargoed dandisets. The eligibility check now requires embargo_status === 'OPEN'. The composable uses a plain unauthenticated fetch, so embargoed asset downloads would have failed with a 401. Routing those requests through the authenticated client is possible but depends on CORS behavior for embargoed presigned URLs that I haven't verified, so I went with the conservative option. Embargoed dandisets still get the CLI instructions as before.

2. I removed the length option previously passed to downloadZip. client-zip interprets that option as a predicted byte length for the archive, not a file count, so the value was wrong. It was harmless in practice because we buffer the response to a Blob and never use the Content-Length header, but there was no reason to keep it.

Still to do: file the follow-up issue for the folder-level download button and for streaming to disk to raise the 2 GiB cap.