Add secure, transactional CUDA and WebGPU EP bootstrapping

[Baiju Meswani (baijumeswani)]

Summary

This change adds first-class CUDA execution-provider bootstrapping for Windows x64, Windows ARM64, and Linux x64. It also moves WebGPU onto the same reusable bundle installer so both providers follow one secure, transactional installation flow.

CUDA is treated as one compatible bundle made from independently reusable artifacts. The large CUDA and cuDNN packages change less frequently than the execution-provider package, so Foundry Local validates each installed artifact against the new manifest before downloading anything. If the CUDA or cuDNN runtime-file hashes are unchanged, those files are reused and only the changed archive, such as the CUDA EP package, is downloaded. The reused and newly downloaded files are then assembled, verified, and atomically activated as one generation, avoiding unnecessary large downloads without exposing a partially updated or incompatible installation.

Why

The CUDA and WebGPU bootstrappers previously owned their package-management logic independently. Downloading, extracting, validating, caching, and activating provider binaries inside each bootstrapper made the behavior harder to reuse and maintain.

The new CUDA package layout also requires Foundry Local to manage several archives as one compatible installation:

• CUDA toolkit binaries.
• cuDNN binaries.
• ONNX Runtime CUDA provider and GenAI CUDA add-on binaries.

The shared installer provides a single implementation for:

• Archive and runtime-file SHA-256 validation.
• Selective reuse and repair.
• Immutable bundle generations.
• Atomic activation.
• Cross-process synchronization.
• Cancellation and cleanup.
• Secure archive extraction.

Code flow before this change

CUDA and WebGPU each managed most of the package lifecycle themselves.

EpDetector::DownloadAndRegister()
|
+-- CudaEpBootstrapper::DownloadAndRegister()
|   |
|   +-- Detect whether CUDA should be offered
|   +-- Resolve the CUDA package for the current platform
|   +-- Download and extract CUDA provider files
|   +-- Perform CUDA-specific cache and hash checks
|   +-- Configure dependency discovery
|   `-- Register CUDAExecutionProvider with ORT
|
`-- WebGpuEpBootstrapper::DownloadAndRegister()
    |
    +-- Resolve the WebGPU package for the current platform
    +-- Download and extract WebGPU provider files
    +-- Perform separate WebGPU cache and hash checks
    +-- Configure dependency discovery
    `-- Register Foundry.WebGPU with ORT

Problems with this flow:

* Package installation behavior was duplicated across bootstrappers.
* CUDA could not naturally represent three archives as one compatible installation.
* Cache existence did not provide one reusable transaction boundary for the complete bundle.
* Adding the same safety and cleanup behavior to another provider required bootstrapper-specific work.

Code flow after this change

The bootstrappers now describe their packages and handle provider-specific loading. The shared installer owns the package lifecycle.

EpDetector::DownloadAndRegister()
|
+-- CudaEpBootstrapper::DownloadAndRegister()
|   |
|   +-- NvmlGpuDetector::HasNvidiaGpu()
|   +-- BuildCudaEpManifest(HostCudaEpPlatform())
|   |   |
|   |   +-- Windows x64
|   |   |   +-- CUDA toolkit archive
|   |   |   +-- cuDNN archive
|   |   |   `-- CUDA EP archive
|   |   |
|   |   +-- Windows ARM64
|   |   |   +-- CUDA toolkit archive
|   |   |   +-- cuDNN archive
|   |   |   `-- CUDA EP archive
|   |   |
|   |   `-- Linux x64
|   |       `-- CUDA EP and GenAI CUDA archive
|   |
|   `-- EpBundleInstaller::EnsureInstalled(cuda_manifest)
|
`-- WebGpuEpBootstrapper::DownloadAndRegister()
    |
    +-- BuildWebGpuManifest()
    `-- EpBundleInstaller::EnsureInstalled(webgpu_manifest)


EpBundleInstaller::EnsureInstalled(manifest)
|
+-- ValidateManifest()
|   +-- Validate bundle and artifact identifiers
|   +-- Require HTTPS package URLs
|   +-- Validate archive and runtime-file SHA-256 values
|   `-- Validate all relative paths
|
+-- FileLock(provider_lock)
|
+-- ReadActiveMarker()
|
+-- CleanupStaleGenerations()
|   `-- Reject symlinked or otherwise unsafe managed directories
|
+-- ReuseVerified?
|   |
|   `-- VerifyBundleDir(active_generation)
|       +-- Rehash every declared runtime file
|       `-- Reject missing, additional, or unsupported filesystem entries
|
+-- If the complete active bundle is valid
|   `-- Return an install transaction for the active generation
|
`-- Otherwise create a private staging generation
    |
    +-- For each EpBundleArtifact
    |   |
    |   +-- If the artifact's installed runtime files still verify
    |   |   `-- CopyArtifactFiles(active_generation, staging_generation)
    |   |
    |   `-- Otherwise
    |       |
    |       +-- HttpDownloadFile()
    |       |   +-- Enforce cancellation
    |       |   +-- Enforce Content-Length and streaming byte limits
    |       |   `-- Reject malformed Content-Length values
    |       |
    |       +-- Verify the downloaded archive SHA-256
    |       |
    |       +-- ExtractZip()
    |       |   +-- Validate the complete ZIP before writing
    |       |   +-- Reject absolute and parent-traversal paths
    |       |   +-- Reject symlinks and special files
    |       |   +-- Reject duplicate and conflicting entries
    |       |   +-- Enforce entry-count and uncompressed-size limits
    |       |   `-- Extract only into an empty, validated destination
    |       |
    |       +-- Require the exact manifest-declared archive contents
    |       +-- Ignore authenticated metadata such as version.json
    |       +-- Rehash every extracted runtime file
    |       `-- Copy only declared runtime files into staging
    |
    +-- VerifyBundleDir(staging_generation)
    |
    `-- Atomically move staging into an immutable bundle generation


EpInstallTransaction::CommitActive()
|
+-- Reverify the completed generation while holding the provider lock
+-- Atomically replace the active marker
`-- Remove stale generations


Provider-specific registration
|
+-- CUDA on Windows
|   +-- EpBundleDependencyOwner::Load()
|   +-- Retain CUDA and cuDNN DLL handles
|   `-- Register CUDAExecutionProvider
|
+-- CUDA on Linux
|   +-- Load libonnxruntime-genai-cuda.so
|   `-- Register CUDAExecutionProvider
|
`-- WebGPU
    +-- EpBundleDependencyOwner::Load() on Windows
    +-- Preserve delayed sibling dependency discovery
    `-- Register Foundry.WebGPU

Provider teardown after this change

The teardown sequence now keeps GenAI, ORT, provider libraries, and their dependencies alive in the required order.

Manager::~Manager()
|
+-- Shutdown active services
+-- Destroy sessions and loaded models
+-- Destroy catalog, download, and telemetry consumers
+-- OgaShutdown()
+-- OrtApi::UnregisterExecutionProviderLibrary()
+-- Destroy EpDetector and bootstrapper dependency owners
|   +-- Release retained Windows dependency DLL handles
|   `-- Release the Linux GenAI CUDA add-on handle
`-- OrtApi::ReleaseEnv()

CUDA platform support

Platform	Package layout	Status
Windows x64	CUDA toolkit, cuDNN, and CUDA EP archives	Supported
Windows ARM64	CUDA toolkit, cuDNN, and CUDA EP archives	Supported
Linux x64	CUDA EP and GenAI CUDA archive	Supported
Linux ARM64	No published bundle	Unsupported

WebGPU behavior

WebGPU continues to support its existing published platforms, but now uses the shared bundle installer. It gains the same:

• Archive and runtime-file verification.
• Immutable generations and atomic activation.
• Cancellation and cleanup.
• Secure extraction.
• Ignored authenticated metadata handling.
• Windows dependency ownership.

Installation and repair behavior

Valid active bundle

Every declared runtime file is rehashed. If all hashes and the exact installed file set match, the existing generation is reused without downloading.

Partially damaged bundle

Each artifact is checked independently. Valid artifact files are copied into a new staging generation, while only invalid artifacts are downloaded again. The combined generation is then fully verified before activation.

Forced installation

A forced request downloads every artifact instead of selectively reusing files.

Failure or cancellation

Failed downloads, hash mismatches, malformed archives, cancellation, loading failures, and marker-publication failures return failure. Temporary staging data is removed, and a partially assembled generation is never made active.

Security and reliability improvements

• HTTPS-only package manifests.
• Hard-coded SHA-256 values for archives and installed runtime files.
• Exact archive and installed file-set validation.
• Cross-process installation lock.
• Unique private staging directories.
• Immutable generation directories.
• Atomic active-marker replacement.
• ZIP path-traversal protection.
• Symlink and special-file rejection.
• Duplicate-entry and file/directory collision rejection.
• Entry-count and uncompressed-size limits.
• HTTP body-size enforcement with or without Content-Length.
• Strict Content-Length parsing.
• Cancellation checks before provider registration.
• RAII ownership and rollback for Windows dependency DLLs.
• WindowsStore-safe NVML behavior.

NVIDIA GPU detection

CUDA eligibility is detected through NVML instead of starting a subprocess. Foundry Local requires at least one NVIDIA device with the supported compute capability. NVML is loaded dynamically and released after detection.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
foundry-local	Ready	Preview	Aug 6, 2026 7:44am

[Copilot]

Pull request overview

Adds a shared, transactional installer for CUDA and WebGPU execution-provider bundles.

Changes:

• Adds verified, reusable bundle installation with atomic activation.
• Adds CUDA platform manifests, NVML detection, and dependency ownership.
• Replaces external ZIP extraction and expands security-focused tests.

Reviewed changes

Copilot reviewed 32 out of 32 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
sdk_v2/cpp/vcpkg.json	Adds archive dependencies.
sdk_v2/cpp/CMakeLists.txt	Builds and links new components.
sdk_v2/cpp/src/ep_detection/cuda_ep_bootstrapper.cc	Installs and registers CUDA bundles.
sdk_v2/cpp/src/ep_detection/cuda_ep_bootstrapper.h	Updates CUDA bootstrapper ownership.
sdk_v2/cpp/src/ep_detection/cuda_ep_manifest.cc	Defines platform CUDA bundles.
sdk_v2/cpp/src/ep_detection/cuda_ep_manifest.h	Declares CUDA manifest APIs.
sdk_v2/cpp/src/ep_detection/ep_bundle_installer.cc	Implements transactional installation.
sdk_v2/cpp/src/ep_detection/ep_bundle_installer.h	Declares installer transactions.
sdk_v2/cpp/src/ep_detection/ep_bundle_manifest.h	Defines bundle metadata.
sdk_v2/cpp/src/ep_detection/ep_utils.cc	Manages Windows dependencies.
sdk_v2/cpp/src/ep_detection/ep_utils.h	Exposes dependency helpers.
sdk_v2/cpp/src/ep_detection/nvml_gpu_detector.cc	Implements NVML GPU detection.
sdk_v2/cpp/src/ep_detection/nvml_gpu_detector.h	Declares GPU detection APIs.
sdk_v2/cpp/src/ep_detection/webgpu_ep_bootstrapper.cc	Migrates WebGPU to bundles.
sdk_v2/cpp/src/ep_detection/webgpu_ep_bootstrapper.h	Updates WebGPU bootstrapper ownership.
sdk_v2/cpp/src/http/http_download.cc	Adds strict size handling.
sdk_v2/cpp/src/http/http_download.h	Extends downloader contract.
sdk_v2/cpp/src/manager.cc	Updates EP lifecycle and teardown.
sdk_v2/cpp/src/manager.h	Documents revised ownership order.
sdk_v2/cpp/src/util/zip_extract.cc	Adds in-process bounded extraction.
sdk_v2/cpp/src/util/zip_extract.h	Defines extraction limits.
sdk_v2/cpp/test/CMakeLists.txt	Registers new unit tests.
sdk_v2/cpp/test/internal_api/c_api_test.cc	Tests manager recreation.
sdk_v2/cpp/test/internal_api/cuda_ep_bootstrapper_test.cc	Tests CUDA manifests and behavior.
sdk_v2/cpp/test/internal_api/ep_bundle_installer_test.cc	Tests installation transactions.
sdk_v2/cpp/test/internal_api/ep_utils_test.cc	Tests dependency selection.
sdk_v2/cpp/test/internal_api/http_download_test.cc	Tests Content-Length parsing.
sdk_v2/cpp/test/internal_api/nvml_gpu_detector_test.cc	Tests capability filtering.
sdk_v2/cpp/test/internal_api/webgpu_ep_bootstrapper_test.cc	Tests WebGPU registration.
sdk_v2/cpp/test/internal_api/zip_extract_test.cc	Tests secure extraction.
sdk_v2/cpp/test/utils/scoped_environment_variable.h	Adds environment test helper.
sdk_v2/cpp/test/utils/zip_builder.h	Adds ZIP fixture builder.

[Scott McKay (skottmckay)]

nit: do you need another try/catch here if safe_log already has that? same for the "Exception while shutting down Manager subsystems during destruction" usage of safe_log.

[Scott McKay (skottmckay)]

I assume we'll manage the manifests using AI, so if we have the SHA for each file could we also have the exact bytes for individual files, and infer the max values from that?

[Baiju Meswani (baijumeswani)]

According to AI:

A ZIP can be larger than the sum of the declared file sizes because:

 - Incompressible files may slightly expand when compressed.
 - ZIP headers and the central directory add overhead.
 - The archive contains ignored entries such as  version.json .

we could derive the max from total file sizes plus some overhead.

[Scott McKay (skottmckay)]

Is this a known issue? are we downloading from an endpoint we don't control and we need extra protection?

[Baiju Meswani (baijumeswani)]

The ep archives are hosted on endpoints controlled by us. The sha256 validation only happens post download. But if the url was wrong, or the endpoint compromised or misconfigured, this may protect users from downloading excessively large archives.

Will make clarifying code comment.

[Scott McKay (skottmckay)]

nit: using friend means you give up control of all your internals. as this is an internal only class that seems like a high price just to protect the ctor.

[Scott McKay (skottmckay)]

what's the point of including/testing this on other platforms?

[Scott McKay (skottmckay)]

Not clear how FOUNDRY_LOCAL_USE_WINHTTP_TRANSPORT relates to CUDA usage.

[Baiju Meswani (baijumeswani)]

from AI:

FOUNDRY_LOCAL_USE_WINHTTP_TRANSPORT means “this build uses WinHTTP for networking.” The NVML detector uses it as a proxy for “this is desktop Windows,” because CMake currently defines it only for desktop Windows and not Windows Store/UWP builds.

Windows Store/UWP applications cannot use the same native-loading approach as an unpackaged desktop application.

The detector currently:

 - Loads  nvml.dll  dynamically from  System32  or NVIDIA’s  Program Files  directory.
 - Uses desktop APIs such as  SHGetKnownFolderPath .
 - Relies on access to the machine-wide NVIDIA driver installation.

A UWP process is sandboxed and generally must load packaged native libraries through  LoadPackagedLibrary ; it cannot freely discover and load DLLs from arbitrary machine paths. The CUDA bootstrapper also downloads native DLLs at runtime, which does not fit the normal UWP package model.

So CUDA/NVML is intentionally disabled for WindowsStore builds today. Supporting it would require a separate packaged-library design, not simply changing the preprocessor condition

Will introduce FOUNDRY_LOCAL_DESKTOP_WINDOWS.

[Scott McKay (skottmckay)]

should there be something logged if we fail this way?

[Scott McKay (skottmckay)]

nit: do we need these vs. putting the values inline like we do in cuda_ep_manifest.cc? I found the latter a bit easier to read, but not sure if one way or the other is going to be easier to update.

[Scott McKay (skottmckay)]

Why do we need to do this? Is this potentially loading files that may not be needed for a specific model?

[Baiju Meswani (baijumeswani)]

We should not be preloading all binaries.

We need to preload libonnxrutime-genai-cuda.so so onnxruntime-genai can find it when it tries to load it.

On windows, we can do AddDllDirectory so cuda and cudnn and onnxruntime-genai-cuda.dll binaries be found.