Skip to content

hip-tests: skip peer-device graph test when hipErrorNotSupported - #8671

Merged
sunhmy-amd merged 1 commit into
developfrom
mingsun/fix-multigpu-graph-skip
Jul 17, 2026
Merged

hip-tests: skip peer-device graph test when hipErrorNotSupported#8671
sunhmy-amd merged 1 commit into
developfrom
mingsun/fix-multigpu-graph-skip

Conversation

@sunhmy-amd

@sunhmy-amd sunhmy-amd commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Motivation

On systems where hipDeviceCanAccessPeer returns true (e.g., gfx1250 with AIFM
fabric), Unit_hipGraphAddMemcpyNode1D_Functional's peer device section proceeds
to call hipGraphInstantiate, which fails with hipErrorNotSupported (error 801)
because multi-device graphs are not yet supported on either the classic or segmented
graph scheduling paths in CLR. The test hard-fails rather than skipping, producing
a misleading failure on otherwise healthy systems.

Technical Details

In validateMemcpyNode1DArray, replace the unconditional
HIP_CHECK(hipGraphInstantiate(...)) in the peer device path with an explicit
return-code check. When hipGraphInstantiate returns hipErrorNotSupported, clean
up allocated resources and call SKIP() (Catch2 v3.3+) instead of failing. All
other error codes continue through HIP_CHECK as before.

The root cause is a known CLR limitation documented in both
GraphExecClassic::Init() and GraphExecSegmented::FindStreamsReqPerDevForSegments():
[hipGraph] Multi-device graph is not supported on the segment scheduling path

The fix is test-side only. validateMemcpyNode1DArray is static and called from
exactly three SECTION blocks within a single test case; the new code path triggers
only on hipErrorNotSupported, leaving the single-device sections unaffected.

JIRA ID

ROCM-28108

Test Plan

Rebuilt GraphsTest1 from the patched source on a gfx1250 system with AIFM fabric
active (4 GPUs, all ACCEL_STATE: READY), with PRs #8254 and #8534 applied to
CLR/ROCr. Ran with AMD_COMGR_HOTSWAP_ENTRY_TRAMPOLINES=1 HSA_ENABLE_SDMA=1,
all 4 GPUs visible.

Test Result

Test Before After
Unit_hipGraphAddMemcpyNode1D_Functional (default device) Passed Passed
Unit_hipGraphAddMemcpyNode1D_Functional (peer device) Failed (801) Skipped

Submission Checklist

Copilot AI review requested due to automatic review settings July 16, 2026 03:19
@sunhmy-amd
sunhmy-amd requested a review from a team as a code owner July 16, 2026 03:19
@therock-pr-bot

Copy link
Copy Markdown

❌ PR Check — Action Required

Check Status Details
🌿 Branch Name ❌ Fail Branch name does not match allowed patterns.
Branch: mingsun/fix-multigpu-graph-skip
Allowed patterns:
- ^users\/[A-Za-z0-9][A-Za-z0-9\-]*\/.+
- ^shared\/.+
- ^[A-Za-z0-9][A-Za-z0-9\-_]*$
- ^dependabot\/.+
- ^revert-[0-9]+-.+
📝 PR Title/Description ❌ Fail Error: Title does not follow Conventional Commits style.
Expected: start with a valid type (feat, fix, docs, …).
Desired format: type(optional-scope): short description
Forbidden Files ✅ Pass
🧪 Unit Test ❌ Fail Error: Source/code files changed without an accompanying unit test.
Expected: add at least one test file named like test_<name>.py / test_<name>.cpp (or <name>_test.*).
Current: code file(s) changed: projects/hip-tests/catch/unit/graph/hipGraphAddMemcpyNode1D_old.cc; no test file found
🔎 pre-commit ⏳ Pending ⏳ Still running…
🚫 Draft PR 🔜 To Be Enabled
🚩 Feature Flag 🔜 To Be Enabled
📊 Code Coverage 🔜 To Be Enabled

⚠️ 3 policy check(s) failed. Please address the issues above before this PR can be Reviewed.

🚫 Please fix the failed policies

  • ❌ Branch Name
  • ❌ PR Title/Description
  • ❌ Unit Test

The Not ready to Review label was added to this PR. Once all policies pass, the label is removed automatically.

📖 Need help? See the Policy FAQ for details on every check and how to fix failures.

@therock-pr-bot

Copy link
Copy Markdown

🚫 Please fix the failed policies before requesting reviews.

The following policy checks failed:

  • ❌ Branch Name
  • ❌ PR Title/Description
  • ❌ Unit Test

The Not ready to Review label has been added to this PR.
Once all policies pass, the label will be removed automatically.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates the HIP graph memcpy 1D functional test to avoid reporting a hard failure on platforms where multi-device graph instantiation is not supported, by converting a specific instantiation failure into a test skip.

Changes:

  • Replaces unconditional HIP_CHECK(hipGraphInstantiate(...)) with an explicit return-code check.
  • On hipErrorNotSupported, performs cleanup and skips the test instead of failing.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread projects/hip-tests/catch/unit/graph/hipGraphAddMemcpyNode1D_old.cc
hipGraphInstantiate returns hipErrorNotSupported (801) for multi-device
graphs on both classic and segmented scheduling paths. When peer access
is available but multi-device graph instantiation is not yet supported,
skip the section rather than failing.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@sunhmy-amd
sunhmy-amd merged commit 54bfe63 into develop Jul 17, 2026
35 of 44 checks passed
@sunhmy-amd
sunhmy-amd deleted the mingsun/fix-multigpu-graph-skip branch July 17, 2026 18:26
systems-assistant Bot pushed a commit to ROCm/hip-tests that referenced this pull request Jul 17, 2026
 hipErrorNotSupported (#8671)

## Motivation
On systems where `hipDeviceCanAccessPeer` returns true (e.g., gfx1250
with AIFM
fabric), `Unit_hipGraphAddMemcpyNode1D_Functional`'s peer device section
proceeds
to call `hipGraphInstantiate`, which fails with `hipErrorNotSupported`
(error 801)
because multi-device graphs are not yet supported on either the classic
or segmented
graph scheduling paths in CLR. The test hard-fails rather than skipping,
producing
a misleading failure on otherwise healthy systems.
## Technical Details
In `validateMemcpyNode1DArray`, replace the unconditional
`HIP_CHECK(hipGraphInstantiate(...))` in the peer device path with an
explicit
return-code check. When `hipGraphInstantiate` returns
`hipErrorNotSupported`, clean
up allocated resources and call `SKIP()` (Catch2 v3.3+) instead of
failing. All
other error codes continue through `HIP_CHECK` as before.
The root cause is a known CLR limitation documented in both
`GraphExecClassic::Init()` and
`GraphExecSegmented::FindStreamsReqPerDevForSegments()`:
[hipGraph] Multi-device graph is not supported on the segment scheduling
path
The fix is test-side only. `validateMemcpyNode1DArray` is `static` and
called from
exactly three SECTION blocks within a single test case; the new code
path triggers
only on `hipErrorNotSupported`, leaving the single-device sections
unaffected.
## JIRA ID
ROCM-28108
## Test Plan
Rebuilt `GraphsTest1` from the patched source on a gfx1250 system with
AIFM fabric
active (4 GPUs, all ACCEL_STATE: READY), with PRs #8254 and #8534
applied to
CLR/ROCr. Ran with AMD_COMGR_HOTSWAP_ENTRY_TRAMPOLINES=1
HSA_ENABLE_SDMA=1,
all 4 GPUs visible.
## Test Result
Test | Before | After
[rocm-systems] ROCm/rocm-systems#8671 (commit 54bfe63)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants