Skip to content

frost(sdpa): filter arch-specific tests correctly to fix sm120 pipeline failures - #495

Closed
Aneureka wants to merge 1 commit into
NVIDIA:developfrom
Aneureka:haobing/frost/filter_non_sm120_tests
Closed

frost(sdpa): filter arch-specific tests correctly to fix sm120 pipeline failures#495
Aneureka wants to merge 1 commit into
NVIDIA:developfrom
Aneureka:haobing/frost/filter_non_sm120_tests

Conversation

@Aneureka

@Aneureka Aneureka commented Aug 6, 2026

Copy link
Copy Markdown
Member

Before submitting

  • I agree to license this contribution under the terms of LICENSE.txt.
  • I ran pre-commit run and committed any formatting changes.
  • I added GitHub labels: one cat-*, one or more mod-*, and one orig-* (see label list).

Affected area

  • CI or test infrastructure

Summary

Filter architecture-specific tests correctly to fix SM120 pipeline failures.

Why

Currently some tests are intended to run in an SM100 environment (either real or mocked), but are currently executed in the SM120 pipeline and fail as a result. This PR filters out those cases accordingly.

Related issues

Related to #381

API and compatibility impact

N/A.

Testing

pytest \
    gemm/frost \
    sdpa/frost \
    linear_attention/frost \
    test_mhas_v2.py \
    -n 4 \
    --tb=short

========= 1507 passed, 8341 skipped, 413 warnings in 222.41s (0:03:42) =========

Summary by CodeRabbit

  • Tests
    • Improved GPU architecture coverage by standardizing simulated SM100 and SM103 test environments.
    • Updated matrix multiplication and grouped matrix multiplication tests to use shared architecture fixtures.
    • Restricted specialized compilation and backward-engine tests to environments with the required GPU and runtime support.
    • Consolidated GPU availability checks for dual-GEMM tests.

@Aneureka Aneureka added cat-bug Reports of incorrect behavior, crashes, regressions, or unexpected results. orig-nv-eng Reported or requested by NVIDIA engineering. mod-frost labels Aug 6, 2026
@coderabbitai

coderabbitai Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

FROST tests now share SM100 and SM103 architecture fixtures. Selected GPU tests also use SM100 and runtime availability markers.

Changes

FROST test environment gating

Layer / File(s) Summary
Shared architecture fixtures
test/python/gemm/frost/conftest.py, test/python/gemm/frost/test_block_scale_matmul.py, test/python/gemm/frost/test_moe_grouped_block_scale_matmul_fwd.py
Added shared SM100 and SM103 fixtures. The SM103 fixture sets the shared-memory budget to 227 KiB. Selected tests now use these fixtures.
GPU and runtime capability gating
test/python/gemm/frost/test_multi_gemm.py, test/python/linear_attention/frost/test_gdn_bprop_kernel.py
Applied SM100 GPU and runtime availability markers to the relevant tests and removed a duplicate GPU marker assignment.

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related PRs

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly describes the primary change: filtering architecture-specific Frost tests to prevent SM120 pipeline failures.
Description check ✅ Passed The description includes all required sections, explains the issue and solution, states compatibility impact, and provides testing commands with results.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@Aneureka
Aneureka force-pushed the haobing/frost/filter_non_sm120_tests branch from 9feae2f to 5834856 Compare August 6, 2026 07:47

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@python/cudnn/sdpa/fwd/kernels/prefill_f16_sm120.py`:
- Around line 291-292: Update the initializer containing self.head_tile_qk and
self.head_tile_v to validate both values against SUPPORTED_HEAD_TILES before
deriving any kernel or fragment attributes; reject unsupported widths such as
17, while preserving valid specializations and existing tensor-shape validation.

In `@test/python/sdpa/frost/test_sdpa_graph_analyzer.py`:
- Line 539: Add the `@pytest.mark.L0` decorator to
test_sm120_probe_accepts_mixed_head_dims, preserving the test’s existing
implementation and ensuring the marker is applied directly to this new Python
test.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 3c55ddaa-6e97-4404-9abf-a96a9c062b52

📥 Commits

Reviewing files that changed from the base of the PR and between bce80fb and 9feae2f.

📒 Files selected for processing (9)
  • python/cudnn/sdpa/fwd/api_dsl.py
  • python/cudnn/sdpa/fwd/kernels/prefill_f16_sm120.py
  • test/python/gemm/frost/conftest.py
  • test/python/gemm/frost/test_block_scale_matmul.py
  • test/python/gemm/frost/test_moe_grouped_block_scale_matmul_fwd.py
  • test/python/gemm/frost/test_multi_gemm.py
  • test/python/linear_attention/frost/test_gdn_bprop_kernel.py
  • test/python/sdpa/frost/test_sdpa_fwd_dsl_sm120.py
  • test/python/sdpa/frost/test_sdpa_graph_analyzer.py

Comment thread python/cudnn/sdpa/fwd/kernels/prefill_f16_sm120.py Outdated
Comment thread test/python/sdpa/frost/test_sdpa_graph_analyzer.py Outdated
@coderabbitai

coderabbitai Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Note

GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer.

@Aneureka Aneureka self-assigned this Aug 6, 2026
@Aneureka

Aneureka commented Aug 6, 2026

Copy link
Copy Markdown
Member Author

@cudnn-ci-bot run frost

Signed-off-by: Haobin Guo <haobing@nvidia.com>
@Aneureka
Aneureka force-pushed the haobing/frost/filter_non_sm120_tests branch from 5834856 to 3048fd3 Compare August 6, 2026 08:12
@coderabbitai

coderabbitai Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Note

GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer.

@Aneureka

Aneureka commented Aug 6, 2026

Copy link
Copy Markdown
Member Author

@cudnn-ci-bot run frost

@cudnn-ci-bot

Copy link
Copy Markdown

🚀 Running mirror pipeline

Branch: cudnn-gh/pr-495-3048fd3
Pipeline: 61361135
Targets: frost

@vedaanta

vedaanta commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator

lets just change the CI internally to only run sm120 for spda tests.
closing

@vedaanta vedaanta closed this Aug 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cat-bug Reports of incorrect behavior, crashes, regressions, or unexpected results. mod-frost orig-nv-eng Reported or requested by NVIDIA engineering.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants