Skip to content

frost(sdpa): drop the redundant zero-fill of the never-read THD sinks dummy (SM100) - #573

Open
vedaanta wants to merge 1 commit into
NVIDIA:developfrom
vedaanta:vagarwalla/frost-sm100-thd-sink-dummy-nofill
Open

frost(sdpa): drop the redundant zero-fill of the never-read THD sinks dummy (SM100)#573
vedaanta wants to merge 1 commit into
NVIDIA:developfrom
vedaanta:vagarwalla/frost-sm100-thd-sink-dummy-nofill

Conversation

@vedaanta

@vedaanta vedaanta commented Aug 13, 2026

Copy link
Copy Markdown
Collaborator

Problem

A per-execute fill kernel on the SM100 THD execute hot path zeroes the dummy sinks buffer — scratch whose contents provably do not matter: the sinks slot is always part of the kernel ABI, but CFG.HAS_SINK is a compile-time fold, so when the graph declares no sink the kernel never reads the buffer (and execute() enforces has_sink <=> sinks is not None, so the dummy only exists in the never-read case).

The fill dates to the original FROST landing (#476) as belt-and-braces; AGENTS.md Rule 1 bans adapter-side fills on the execute hot path.

Fix

Bind the carved/allocated dummy without the zero_() / torch.zeros fill (one hunk in _execute_thd). Split out of #543 so it can land independently; #543 keeps the matching O-descriptor no-fill and touches the same else-branch for stream-binding — whichever lands second has a trivial one-hunk rebase.

Verification

What Where Result
THD fp16/graph suites (incl. the sink-less THD cases that bind the deliberately unfilled dummy, and the #522 cu_seq_len forms) SM100 (B200-class) 180 passed

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Performance
    • Reduced unnecessary initialization overhead in SM100 THD execution when sink support is disabled.
  • Bug Fixes
    • Ensured the unused no-sink buffer is safely provided without requiring explicit zero-filling.

… dummy (SM100)

A per-execute fill kernel on the THD execute hot path zeroed the dummy
sinks buffer, whose contents provably do not matter: the sinks slot is
always part of the kernel ABI, but CFG.HAS_SINK is a compile-time fold —
when the graph declares no sink the kernel never reads the buffer (and
execute() enforces has_sink <=> sinks is not None, so the dummy only
exists in the never-read case).

The fill dates to the original FROST landing (NVIDIA#476) as belt-and-braces.
Rule 1: no adapter-side fills on the execute hot path.

Split out of NVIDIA#543 (its O-descriptor twin stays there); textual overlap
with NVIDIA#543's stream-binding of the same else-branch — whichever lands
second has a trivial rebase.

Verified on SM100 (B200-class): 180 passed across the THD fp16/graph
suites, including the sink-less THD cases that bind the unfilled dummy.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@vedaanta vedaanta added orig-nv-eng Reported or requested by NVIDIA engineering. mod-cutedsl CuTeDSL kernels, generated kernels, examples, or related integration work. cat-cleanup mod-frost labels Aug 13, 2026
@coderabbitai

coderabbitai Bot commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: ecdc0bc9-b121-4638-87ef-bd3e4081a4cc

📥 Commits

Reviewing files that changed from the base of the PR and between a7b4ca2 and f050b57.

📒 Files selected for processing (1)
  • python/cudnn/sdpa/fwd/api_dsl.py

📝 Walkthrough

Walkthrough

SM100 THD execution now obtains an uninitialized ABI-only dummy buffer when no sinks are provided. The buffer comes from workspace or torch.empty instead of zero-filled storage.

Changes

SM100 THD dummy buffer

Layer / File(s) Summary
No-sink dummy buffer allocation
python/cudnn/sdpa/fwd/api_dsl.py
The no-sink path uses uninitialized workspace or torch.empty storage for the ABI-only dummy buffer.

Estimated code review effort: 1 (Trivial) | ~5 minutes

Mergeability Score: ⚪ Minimal · up to f050b

This change removes an unnecessary zero-fill from the SM100 THD execute path while retaining the required dummy buffer binding; no actionable merge-blocking risk remains after normal checks and review.

Suggested reviewers: aneureka

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Description check ⚠️ Warning The description explains the problem, fix, and verification, but omits the required checklist, affected area, related issues, and API or compatibility sections. Add the template sections and complete the checklist; state the affected area, related issues, API or compatibility impact, and exact test commands and results.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly identifies the SM100 THD change and the removal of the redundant zero-fill.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@vedaanta
vedaanta requested a review from Aneureka August 13, 2026 07:11
@vedaanta

Copy link
Copy Markdown
Collaborator Author

@cudnn-ci-bot run frost

@cudnn-ci-bot

Copy link
Copy Markdown

🚀 Running mirror pipeline

Branch: cudnn-gh/pr-573-f050b57
Pipeline: 62478929
Targets: frost

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cat-cleanup mod-cutedsl CuTeDSL kernels, generated kernels, examples, or related integration work. mod-frost orig-nv-eng Reported or requested by NVIDIA engineering.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants