Skip to content

Add HSTU attention CuTe DSL kernels for Blackwell - #487

Draft
jiayus-nvidia wants to merge 14 commits into
NVIDIA:developfrom
jiayus-nvidia:agent/hstu-blackwell-sm100-bf16
Draft

Add HSTU attention CuTe DSL kernels for Blackwell#487
jiayus-nvidia wants to merge 14 commits into
NVIDIA:developfrom
jiayus-nvidia:agent/hstu-blackwell-sm100-bf16

Conversation

@jiayus-nvidia

@jiayus-nvidia jiayus-nvidia commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Add packed variable-length HSTU attention forward and backward CuTe DSL kernels for Blackwell SM100/SM103 GPUs.
  • Support FP16/BF16 head dimensions 64, 128, and 256; full, causal, local, and arbitrary masks; paged-KV forward; and strided or preallocated gradient outputs.
  • Add the experimental FE API, lazy top-level exports, documentation, and L0/L1 pytest coverage.
  • Apply the repository's existing licensing pattern: NVIDIA-owned implementation, API, and test files use Apache-2.0, while seven retained low-level derived helpers use MIT and preserve their original author notices.
  • Make explicit-stream execution allocation- and lifetime-safe, reject cross-device PyTorch streams, and validate output overlap with all read-only metadata.
  • Preserve the original HSTU development history while adapting the port to the current develop package layout and conventions.

Validation

  • pre-commit run --files <changed files>
  • python -m compileall -q python/cudnn/hstu_attention
  • cd test/python && PYTHONDONTWRITEBYTECODE=1 PYTHONPATH=../../python python -m pytest -p no:cacheprovider -q -m "L0 or L1" fe_api/hstu_attention/
    • 185 passed on NVIDIA B200 (SM100), CUDA 13.2, cuDNN 9.20, PyTorch 2.11, CUTLASS DSL 4.6.1
  • License audit: 19 Apache-2.0 files and 7 MIT files in the HSTU package and tests, with no file-level BSD source in the HSTU tree.

Closes #369

jiayus-nvidia and others added 9 commits August 5, 2026 06:53
Migrate the Blackwell two-kernel dQ and dK/dV implementation, integrate it with the FE OSS API lifecycle, and add correctness, layout, cache, mask, and CUDA Graph coverage.
Build private device-side Q2K/K2Q metadata for FP16/BF16 forward and backward paths. Specialize MASK/FULL/EMPTY traversal, use R2P or endpoint reuse where profitable, and add API, correctness, tail, and CUDA Graph coverage.
- add padded CLC scheduling and optimized R2P, SiLU, and TMA forward paths
- select the SM107 two-CTA one-Q-stage forward pipeline while retaining the SM100/SM103 one-CTA path
- add the two-CTA backward pipeline, padded-static work mapping, and optimized gradient conversion
- compile public CUTLASS DSL kernels as family-portable sm_100f binaries
- preserve existing local, arbitrary-mask, paged, BF16, and FP16 paths
@coderabbitai

coderabbitai Bot commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 90c14234-9bca-4002-b34b-e8e971e25278

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Comment @coderabbitai help to get the list of available commands.

@jiayus-nvidia
jiayus-nvidia force-pushed the agent/hstu-blackwell-sm100-bf16 branch from 494914e to aab82f0 Compare August 5, 2026 07:50
@jiayus-nvidia jiayus-nvidia changed the title Add HSTU attention CuTe DSL kernels for Blackwell and Rubin Add HSTU attention CuTe DSL kernels for Blackwell Aug 5, 2026
@jiayus-nvidia
jiayus-nvidia force-pushed the agent/hstu-blackwell-sm100-bf16 branch 2 times, most recently from c8dc3d4 to a20539e Compare August 5, 2026 08:34
@jiayus-nvidia
jiayus-nvidia force-pushed the agent/hstu-blackwell-sm100-bf16 branch from a20539e to 3245fb1 Compare August 5, 2026 08:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[feature request] Add blackwell HSTU mha kernel.

2 participants