Skip to content

fix(rocprof-systems): Fix transpose to work under MPI#8389

Open
sputhala-amd wants to merge 2 commits into
developfrom
users/sputhala/transposeMPIFix
Open

fix(rocprof-systems): Fix transpose to work under MPI#8389
sputhala-amd wants to merge 2 commits into
developfrom
users/sputhala/transposeMPIFix

Conversation

@sputhala-amd

@sputhala-amd sputhala-amd commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Motivation

Under MPI, transpose only ran GPU work on ranks mapped 1:1 to a distinct device, and worker threads never selected their stream's device — so ranks bound to a non-zero GPU crashed with "invalid resource handle" and ranks beyond the device count did no GPU work at all.

Technical Details

  • Per-thread device selection: added hipSetDevice(devid) at the start of run(). HIP tracks the active device per host thread, and freshly spawned std::thread workers default to device 0; without this they launched kernels on a stream owned by a different device and failed with "invalid resource handle" whenever a rank was mapped to a non-zero device (e.g. rank 1 → device 1).
  • Round-robin GPU sharing: replaced the restrictive guard if(rank == devid && rank < ndevice) with if(ndevice > 0), so ranks map round-robin via devid = rank % ndevice and ranks beyond the device count share a GPU (e.g. 2 ranks on 1 GPU both use device 0) instead of skipping GPU work.
  • Removed the stale "temporary workaround in rocprof-sys when HIP + MPI is enabled" comment; the underlying limitation no longer applies — verified that multiple ranks sharing a single GPU each collect full HW counter data.
  • Thread wiring: the device id is now threaded through to every worker (run(rank, i, devid, stream, ...)) and the main-thread invocation, and the device-assignment + GPU-work blocks are merged into a single if(ndevice > 0) block

JIRA ID

JIRA ID : AIPROFSYST-622

Test Plan

Manual and CI test.

Test Result

Pass.

Submission Checklist

@therock-pr-bot

therock-pr-bot Bot commented Jul 10, 2026

Copy link
Copy Markdown

✅ All Policy Checks Passed

Check Status Details
🌿 Branch Name ✅ Pass
📝 PR Title/Description ✅ Pass
Forbidden Files ✅ Pass
🧪 Unit Test ✅ Pass
🚫 Draft PR 🔜 To Be Enabled
🚩 Feature Flag 🔜 To Be Enabled
📊 Code Coverage 🔜 To Be Enabled

🎉 All policy checks passed!

📖 Need help? See the Policy FAQ for details on every check and how to fix failures.

@therock-pr-bot

therock-pr-bot Bot commented Jul 10, 2026

Copy link
Copy Markdown

🚫 Please fix the failed policies before requesting reviews.

The following policy checks failed:

  • ❌ Unit Test

The Not ready to Review label has been added to this PR.
Once all policies pass, the label will be removed automatically.

@github-actions

github-actions Bot commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Code Coverage Report

Code Coverage Report

Unit Tests

Code Coverage: unit-tests

Lines: 🔴 14.0% — 4,521/32,243 across 290 files
Functions: 🔴 13.7% — 462/3,361

🔴 0-20% (206 files)
Lines Functions File
0.0% 0/105 0/9 source/bin/common/argument_registration.hpp
0.0% 0/259 0/29 source/bin/common/tool_runner.cpp
0.0% 0/931 0/47 source/bin/rocprof-sys-avail/avail.cpp
0.0% 0/61 0/6 source/bin/rocprof-sys-avail/avail.hpp
0.0% 0/231 0/26 source/bin/rocprof-sys-avail/common.cpp
0.0% 0/17 0/3 source/bin/rocprof-sys-avail/component_categories.hpp
0.0% 0/257 0/12 source/bin/rocprof-sys-avail/generate_config.cpp
0.0% 0/39 0/2 source/bin/rocprof-sys-avail/get_availability.hpp
0.0% 0/10 0/1 source/bin/rocprof-sys-avail/get_categories.hpp
0.0% 0/102 0/6 source/bin/rocprof-sys-avail/hw_counter_query.cpp
0.0% 0/4 0/1 source/bin/rocprof-sys-avail/info_type.cpp
0.0% 0/9 0/1 source/bin/rocprof-sys-avail/info_type.hpp
0.0% 0/440 0/47 source/bin/rocprof-sys-causal/impl.cpp
0.0% 0/80 0/2 source/bin/rocprof-sys-causal/rocprof-sys-causal.cpp
0.0% 0/588 0/42 source/bin/rocprof-sys-instrument/details.cpp
0.0% 0/93 0/7 source/bin/rocprof-sys-instrument/function_signature.cpp
0.0% 0/20 0/5 source/bin/rocprof-sys-instrument/function_signature.hpp
0.0% 0/156 0/7 source/bin/rocprof-sys-instrument/info.hpp
0.0% 0/261 0/22 source/bin/rocprof-sys-instrument/internal_libs.cpp
0.0% 0/63 0/6 source/bin/rocprof-sys-instrument/log.cpp
0.0% 0/5 0/1 source/bin/rocprof-sys-instrument/log.hpp
0.0% 0/529 0/40 source/bin/rocprof-sys-instrument/module_function.cpp
0.0% 0/93 0/7 source/bin/rocprof-sys-instrument/module_function.hpp
0.0% 0/1550 0/87 source/bin/rocprof-sys-instrument/rocprof-sys-instrument.cpp
0.0% 0/156 0/12 source/bin/rocprof-sys-instrument/rocprof-sys-instrument.hpp
0.0% 0/2 0/1 source/bin/rocprof-sys-run/rocprof-sys-run.cpp
0.0% 0/2 0/1 source/bin/rocprof-sys-sample/rocprof-sys-sample.cpp
0.0% 0/22 0/1 source/lib/backends/amd_smi/wrapper.hpp
0.0% 0/43 0/2 source/lib/backends/rocprofiler_sdk/backend.hpp
0.0% 0/16 0/4 source/lib/binary/address_multirange.cpp
0.0% 0/9 0/1 source/lib/binary/address_multirange.hpp
0.0% 0/117 0/5 source/lib/binary/analysis.cpp
0.0% 0/7 0/1 source/lib/binary/binary_info.hpp
0.0% 0/102 0/9 source/lib/binary/dwarf_entry.cpp
0.0% 0/1 0/1 source/lib/binary/dwarf_entry.hpp
0.0% 0/69 0/6 source/lib/binary/link_map.cpp
0.0% 0/3 0/1 source/lib/binary/link_map.hpp
0.0% 0/10 0/1 source/lib/binary/scope_filter.cpp
0.0% 0/5 0/2 source/lib/binary/scope_filter.hpp
0.0% 0/159 0/18 source/lib/binary/symbol.cpp
0.0% 0/8 0/8 source/lib/binary/symbol.hpp
0.0% 0/9 0/2 source/lib/common/env_vars.hpp
0.0% 0/36 0/3 source/lib/common/invoke.hpp
0.0% 0/164 0/14 source/lib/common/md5sum.hpp
0.0% 0/20 0/2 source/lib/common/setup.hpp
0.0% 0/22 0/3 source/lib/common/static_object.hpp
0.0% 0/10 0/3 source/lib/common/synchronized.hpp
0.0% 0/3 0/1 source/lib/common/units.hpp
0.0% 0/166 0/2 source/lib/core/agent_info.hpp
0.0% 0/33 0/3 source/lib/core/amd_smi.cpp
0.0% 0/80 0/18 source/lib/core/binary/address_range.cpp
0.0% 0/9 0/3 source/lib/core/binary/address_range.hpp
0.0% 0/49 0/10 source/lib/core/categories.cpp
0.0% 0/11 0/11 source/lib/core/components/fwd.hpp
0.0% 0/26 0/3 source/lib/core/config.hpp
0.0% 0/14 0/2 source/lib/core/containers/aligned_static_vector.hpp
0.0% 0/11 0/3 source/lib/core/containers/c_array.hpp
0.0% 0/1 0/1 source/lib/core/containers/operators.hpp
0.0% 0/46 0/12 source/lib/core/containers/stable_vector.hpp
0.0% 0/23 0/5 source/lib/core/containers/static_vector.hpp
0.0% 0/89 0/6 source/lib/core/cpu.cpp
0.0% 0/1 0/1 source/lib/core/cpu.hpp
0.0% 0/53 0/6 source/lib/core/dynamic_library.cpp
0.0% 0/28 0/7 source/lib/core/exception.cpp
0.0% 0/163 0/22 source/lib/core/gpu.cpp
0.0% 0/160 0/12 source/lib/core/gpu_metrics.cpp
0.0% 0/32 0/6 source/lib/core/mpi.hpp
0.0% 0/56 0/4 source/lib/core/mproc.cpp
0.0% 0/24 0/2 source/lib/core/node_info.cpp
0.0% 0/2 0/2 source/lib/core/node_info.hpp
0.0% 0/79 0/5 source/lib/core/perf.cpp
0.0% 0/56 0/6 source/lib/core/perfetto.hpp
0.0% 0/51 0/5 source/lib/core/perfetto/cached_perfetto_session.cpp
0.0% 0/33 0/4 source/lib/core/perfetto/counter_track.hpp
0.0% 0/29 0/7 source/lib/core/perfetto/emitter.cpp
0.0% 0/48 0/9 source/lib/core/perfetto/emitter.hpp
0.0% 0/7 0/1 source/lib/core/perfetto/sinks/append_mode.hpp
0.0% 0/13 0/3 source/lib/core/perfetto/sinks/tee_sink.cpp
0.0% 0/1 0/1 source/lib/core/perfetto/sinks/tee_sink.hpp
0.0% 0/1 0/1 source/lib/core/progress/bar.hpp
0.0% 0/290 0/33 source/lib/core/rocpd/data_processor.cpp
0.0% 0/7 0/3 source/lib/core/rocpd/data_processor.hpp
0.0% 0/73 0/11 source/lib/core/rocpd/data_storage/database.cpp
0.0% 0/49 0/6 source/lib/core/rocpd/data_storage/database.hpp
0.0% 0/28 0/4 source/lib/core/rocpd/data_storage/insert_query_builders.hpp
0.0% 0/7 0/2 source/lib/core/rocpd/data_storage/table_insert_query.hpp
0.0% 0/342 0/22 source/lib/core/rocprofiler-sdk.cpp
0.0% 0/54 0/3 source/lib/core/trace_cache/cache_manager.cpp
0.0% 0/7 0/3 source/lib/core/trace_cache/cache_manager.hpp
0.0% 0/516 0/71 source/lib/core/trace_cache/metadata_registry.cpp
0.0% 0/758 0/103 source/lib/core/trace_cache/perfetto_processor.cpp
0.0% 0/119 0/12 source/lib/core/trace_cache/post_processor.cpp
0.0% 0/589 0/34 source/lib/core/trace_cache/rocpd_processor.cpp
0.0% 0/77 0/1 source/lib/core/trace_cache/sample_processor.hpp
0.0% 0/44 0/4 source/lib/core/utility.cpp
0.0% 0/542 0/90 source/lib/rocprof-sys-dl/dl.cpp
0.0% 0/16 0/1 source/lib/rocprof-sys-dl/main.c
0.0% 0/280 0/31 source/lib/rocprof-sys-rt/src/RTcommon.c
0.0% 0/29 0/4 source/lib/rocprof-sys-rt/src/RTheap-linux.c
0.0% 0/65 0/4 source/lib/rocprof-sys-rt/src/RTheap.c
0.0% 0/212 0/22 source/lib/rocprof-sys-rt/src/RTlinux.c
0.0% 0/97 0/8 source/lib/rocprof-sys-rt/src/RTposix.c
0.0% 0/6 0/2 source/lib/rocprof-sys-rt/src/RTsignal.c
0.0% 0/11 0/2 source/lib/rocprof-sys-rt/src/RTthread-x86-64.c
0.0% 0/3 0/1 source/lib/rocprof-sys-rt/src/RTthread.c
0.0% 0/79 0/13 source/lib/rocprof-sys-user/user.cpp
0.0% 0/70 0/18 source/lib/rocprof-sys/api.cpp
0.0% 0/619 0/36 source/lib/rocprof-sys/library.cpp
0.0% 0/77 0/6 source/lib/rocprof-sys/library/causal/components/backtrace.cpp
0.0% 0/7 0/2 source/lib/rocprof-sys/library/causal/components/backtrace.hpp
0.0% 0/93 0/11 source/lib/rocprof-sys/library/causal/components/blocking_gotcha.cpp
0.0% 0/33 0/7 source/lib/rocprof-sys/library/causal/components/causal_gotcha.cpp
0.0% 0/85 0/22 source/lib/rocprof-sys/library/causal/components/progress_point.cpp
0.0% 0/23 0/3 source/lib/rocprof-sys/library/causal/components/progress_point.hpp
0.0% 0/44 0/8 source/lib/rocprof-sys/library/causal/components/unblocking_gotcha.cpp
0.0% 0/551 0/33 source/lib/rocprof-sys/library/causal/data.cpp
0.0% 0/87 0/16 source/lib/rocprof-sys/library/causal/delay.cpp
0.0% 0/373 0/29 source/lib/rocprof-sys/library/causal/experiment.cpp
0.0% 0/3 0/2 source/lib/rocprof-sys/library/causal/experiment.hpp
0.0% 0/24 0/5 source/lib/rocprof-sys/library/causal/sample_data.cpp
0.0% 0/269 0/25 source/lib/rocprof-sys/library/causal/sampling.cpp
0.0% 0/4 0/1 source/lib/rocprof-sys/library/causal/selected_entry.cpp
0.0% 0/5 0/3 source/lib/rocprof-sys/library/causal/selected_entry.hpp
0.0% 0/71 0/11 source/lib/rocprof-sys/library/components/backtrace.cpp
0.0% 0/1 0/1 source/lib/rocprof-sys/library/components/backtrace.hpp
0.0% 0/218 0/30 source/lib/rocprof-sys/library/components/backtrace_metrics.cpp
0.0% 0/11 0/2 source/lib/rocprof-sys/library/components/backtrace_metrics.hpp
0.0% 0/9 0/3 source/lib/rocprof-sys/library/components/backtrace_timestamp.cpp
0.0% 0/4 0/2 source/lib/rocprof-sys/library/components/backtrace_timestamp.hpp
0.0% 0/71 0/10 source/lib/rocprof-sys/library/components/callchain.cpp
0.0% 0/1 0/1 source/lib/rocprof-sys/library/components/callchain.hpp
0.0% 0/367 0/30 source/lib/rocprof-sys/library/components/comm_data.cpp
0.0% 0/19 0/5 source/lib/rocprof-sys/library/components/comm_data.hpp
0.0% 0/8 0/2 source/lib/rocprof-sys/library/components/ensure_storage.hpp
0.0% 0/28 0/6 source/lib/rocprof-sys/library/components/exit_gotcha.cpp
0.0% 0/1 0/1 source/lib/rocprof-sys/library/components/exit_gotcha.hpp
0.0% 0/67 0/8 source/lib/rocprof-sys/library/components/fork_gotcha.cpp
0.0% 0/1 0/1 source/lib/rocprof-sys/library/components/fork_gotcha.hpp
0.0% 0/13 0/3 source/lib/rocprof-sys/library/components/kill_gotcha.cpp
0.0% 0/1 0/1 source/lib/rocprof-sys/library/components/kill_gotcha.hpp
0.0% 0/158 0/22 source/lib/rocprof-sys/library/components/mpi_gotcha.cpp
0.0% 0/2 0/1 source/lib/rocprof-sys/library/components/mpi_gotcha.hpp
0.0% 0/533 0/12 source/lib/rocprof-sys/library/components/mpip.hpp
0.0% 0/78 0/19 source/lib/rocprof-sys/library/components/numa_gotcha.cpp
0.0% 0/2 0/1 source/lib/rocprof-sys/library/components/numa_gotcha.hpp
0.0% 0/323 0/20 source/lib/rocprof-sys/library/components/pthread_create_gotcha.cpp
0.0% 0/3 0/1 source/lib/rocprof-sys/library/components/pthread_create_gotcha.hpp
0.0% 0/34 0/9 source/lib/rocprof-sys/library/components/pthread_gotcha.cpp
0.0% 0/23 0/2 source/lib/rocprof-sys/library/components/pthread_mutex_gotcha_policy.hpp
0.0% 0/164 0/31 source/lib/rocprof-sys/library/components/vaapi_gotcha.cpp
0.0% 0/2 0/1 source/lib/rocprof-sys/library/components/vaapi_gotcha.hpp
0.0% 0/145 0/10 source/lib/rocprof-sys/library/coverage.cpp
0.0% 0/29 0/5 source/lib/rocprof-sys/library/coverage.hpp
0.0% 0/60 0/16 source/lib/rocprof-sys/library/coverage/impl.hpp
0.0% 0/254 0/37 source/lib/rocprof-sys/library/kokkosp.cpp
0.0% 0/245 0/31 source/lib/rocprof-sys/library/perf.cpp
0.0% 0/11 0/3 source/lib/rocprof-sys/library/perf.hpp
0.0% 0/182 0/11 source/lib/rocprof-sys/library/pmc/collectors/common/settings.hpp
0.0% 0/66 0/5 source/lib/rocprof-sys/library/pmc/collectors/cpu/cache_policy.hpp
0.0% 0/47 0/6 source/lib/rocprof-sys/library/pmc/collectors/cpu/cpu_traits.hpp
0.0% 0/52 0/9 source/lib/rocprof-sys/library/pmc/collectors/cpu/perfetto_policy.hpp
0.0% 0/100 0/6 source/lib/rocprof-sys/library/pmc/collectors/gpu/cache_policy.hpp
0.0% 0/40 0/3 source/lib/rocprof-sys/library/pmc/collectors/gpu/gpu_traits.hpp
0.0% 0/222 0/36 source/lib/rocprof-sys/library/pmc/collectors/gpu/perfetto_policy.hpp
0.0% 0/25 0/3 source/lib/rocprof-sys/library/pmc/collectors/gpu/sample.hpp
0.0% 0/32 0/3 source/lib/rocprof-sys/library/pmc/collectors/gpu_perf_counter/cache_policy.hpp
0.0% 0/39 0/7 source/lib/rocprof-sys/library/pmc/collectors/gpu_perf_counter/perfetto_policy.hpp
0.0% 0/16 0/2 source/lib/rocprof-sys/library/pmc/collectors/nic/sample.hpp
0.0% 0/14 0/2 source/lib/rocprof-sys/library/pmc/device_providers/procfs/provider.hpp
0.0% 0/148 0/18 source/lib/rocprof-sys/library/pmc/sampler.cpp
0.0% 0/112 0/11 source/lib/rocprof-sys/library/process_sampler.cpp
0.0% 0/13 0/1 source/lib/rocprof-sys/library/process_sampler.hpp
0.0% 0/2 0/1 source/lib/rocprof-sys/library/rocm.cpp
0.0% 0/1307 0/88 source/lib/rocprof-sys/library/rocprofiler-sdk.cpp
0.0% 0/110 0/13 source/lib/rocprof-sys/library/rocprofiler-sdk/counters.cpp
0.0% 0/17 0/6 source/lib/rocprof-sys/library/rocprofiler-sdk/counters.hpp
0.0% 0/129 0/9 source/lib/rocprof-sys/library/rocprofiler-sdk/fwd.cpp
0.0% 0/34 0/7 source/lib/rocprof-sys/library/rocprofiler-sdk/fwd.hpp
0.0% 0/265 0/25 source/lib/rocprof-sys/library/rocprofiler-sdk/kfd_events.cpp
0.0% 0/36 0/9 source/lib/rocprof-sys/library/rocprofiler-sdk/marker_writer.cpp
0.0% 0/136 0/13 source/lib/rocprof-sys/library/rocprofiler-sdk/rccl.cpp
0.0% 0/112 0/10 source/lib/rocprof-sys/library/rocprofiler-sdk/roctx_client.cpp
0.0% 0/118 0/20 source/lib/rocprof-sys/library/runtime.cpp
0.0% 0/2 0/1 source/lib/rocprof-sys/library/runtime.hpp
0.0% 0/995 0/65 source/lib/rocprof-sys/library/sampling.cpp
0.0% 0/145 0/31 source/lib/rocprof-sys/library/thread_data.hpp
0.0% 0/10 0/3 source/lib/rocprof-sys/library/thread_data_growth.hpp
0.0% 0/10 0/1 source/lib/rocprof-sys/library/thread_deleter.cpp
0.0% 0/5 0/1 source/lib/rocprof-sys/library/thread_deleter.hpp
0.0% 0/148 0/25 source/lib/rocprof-sys/library/thread_info.cpp
0.0% 0/2 0/1 source/lib/rocprof-sys/library/thread_info.hpp
0.0% 0/112 0/11 source/lib/rocprof-sys/library/tracing.cpp
0.0% 0/109 0/30 source/lib/rocprof-sys/library/tracing.hpp
0.0% 0/3 0/1 source/lib/rocprof-sys/library/tracing/annotation.cpp
0.0% 0/45 0/4 source/lib/rocprof-sys/library/tracing/annotation.hpp
0.0% 0/8 0/2 source/lib/rocprof-sys/progress.cpp
0.0% 0/36 0/10 source/lib/rocprof-sys/regions.cpp
0.0% 0/82 0/4 source/lib/rocprof-sys/timeout.cpp
0.0% 0/457 0/46 source/python/libpyrocprofsys.cpp
0.6% 1/166 1/13 source/lib/core/perfetto.cpp
0.8% 7/859 1/74 source/lib/core/argparse.cpp
2.5% 42/1705 7/288 source/lib/core/config.cpp
8.7% 15/172 3/25 source/lib/core/constraint.cpp
15.4% 4/26 0/6 source/lib/core/trace_cache/metadata_registry.hpp
16.7% 6/36 2/5 source/bin/common/common_utils.hpp
18.8% 9/48 2/5 source/lib/core/output_file_registry.cpp
🟠 20-50% (8 files)
Lines Functions File
20.0% 1/5 0/2 source/lib/core/argparse.hpp
21.3% 13/61 3/9 source/lib/core/utility.hpp
22.2% 2/9 0/6 source/lib/core/constraint.hpp
23.8% 15/63 3/12 source/lib/core/agent_manager.cpp
31.1% 23/74 6/14 source/lib/core/state.cpp
38.2% 13/34 1/4 source/lib/core/perfetto/sinks/io_helpers.hpp
42.9% 12/28 1/4 source/lib/common/join.hpp
47.4% 176/371 18/30 source/bin/common/common_utils.cpp
🟡 50-80% (22 files)
Lines Functions File
50.0% 127/254 14/33 source/lib/rocprof-sys/library/components/category_region.hpp
50.0% 1/2 0/1 source/lib/rocprof-sys/library/pmc/collectors/nic/types.hpp
50.0% 1/2 0/1 source/lib/rocprof-sys/library/pmc/common/types.hpp
52.0% 13/25 2/3 source/lib/rocprof-sys/library/pmc/collectors/gpu_perf_counter/gpu_perf_counter_traits.hpp
53.3% 8/15 3/3 source/lib/core/perfetto/sinks/per_pid_file_sink.cpp
55.7% 34/61 5/8 source/lib/rocprof-sys/library/pmc/collectors/cpu/sample.hpp
60.0% 3/5 1/1 source/lib/core/agent.hpp
62.1% 18/29 1/1 source/lib/core/perfetto/locked_file_append.cpp
62.1% 41/66 7/8 source/lib/rocprof-sys/library/pmc/collectors/base/collector.hpp
62.3% 48/77 4/5 source/lib/core/trace_cache/discovery.cpp
63.1% 70/111 7/10 source/lib/core/progress/bar.cpp
64.1% 336/524 26/40 source/bin/common/json_config.cpp
66.7% 2/3 0/2 source/lib/rocprof-sys/library/pmc/collectors/cpu/types.hpp
67.2% 43/64 11/14 source/lib/core/trace_cache/data_types.cpp
68.8% 22/32 3/8 source/lib/rocprof-sys/library/pmc/collectors/gpu_perf_counter/types.hpp
72.3% 86/119 14/17 source/lib/core/perfetto/engine.cpp
72.5% 216/298 14/16 source/bin/common/preset_registry.cpp
75.0% 3/4 0/1 source/lib/backends/amd_smi/nic_types.hpp
75.0% 3/4 1/2 source/lib/core/perfetto/sinks/single_file_sink.hpp
76.5% 78/102 4/4 source/lib/core/perfetto/sinks/single_file_sink.cpp
77.8% 77/99 10/11 source/lib/core/perfetto/engine_impl.hpp
78.1% 107/137 13/15 source/lib/common/path.hpp
🟢 80-100% (54 files)
Lines Functions File
80.3% 61/76 8/9 source/lib/rocprof-sys/library/pmc/device_providers/rocprofiler_sdk/provider.hpp
83.9% 135/161 12/16 source/lib/backends/procfs/backend.hpp
84.6% 11/13 2/4 source/lib/rocprof-sys/library/rocprofiler-sdk/roctx_client.hpp
84.8% 84/99 11/15 source/lib/logger/logger.hpp
85.1% 40/47 2/2 source/lib/core/trace_cache/storage_parser.hpp
87.7% 265/302 27/32 source/lib/common/environment.hpp
89.1% 49/55 4/4 source/lib/rocprof-sys/library/pmc/collectors/gpu_perf_counter/device.hpp
90.0% 54/60 4/4 source/lib/core/trace_cache/buffer_storage.cpp
90.1% 100/111 8/8 source/lib/rocprof-sys/library/pmc/collectors/cpu/device.hpp
90.7% 303/334 28/31 source/lib/core/trace_cache/sample_type.hpp
91.0% 243/267 19/20 source/lib/core/trace_cache/unified_memory_processor.cpp
91.3% 178/195 14/18 source/lib/rocprof-sys/library/components/shmem_gotcha.hpp
91.7% 11/12 2/2 source/lib/core/perfetto/session_backend.hpp
92.3% 156/169 8/8 source/lib/rocprof-sys/library/pmc/collectors/gpu/device.hpp
92.7% 38/41 6/11 source/lib/core/demangler.hpp
92.9% 13/14 0/1 source/lib/rocprof-sys/library/pmc/collectors/common/collector_slice.hpp
93.0% 66/71 7/8 source/lib/core/trace_cache/buffer_storage.hpp
94.7% 54/57 6/8 source/lib/rocprof-sys/library/components/pthread_mutex_gotcha.hpp
94.8% 55/58 1/5 source/lib/core/trace_cache/cacheable.hpp
94.8% 92/97 9/10 source/lib/rocprof-sys/library/rocprofiler-sdk/trace_control.cpp
95.8% 23/24 1/2 source/lib/common/delimit.hpp
95.9% 162/169 19/22 source/lib/rocprof-sys/library/components/ucx_gotcha.hpp
96.8% 30/31 3/4 source/lib/rocprof-sys/library/pmc/collectors/gpu_perf_counter/sample.hpp
97.3% 73/75 6/6 source/lib/core/common_types.hpp
97.3% 73/75 3/3 source/lib/core/perfetto/packet_framing.hpp
97.8% 45/46 7/7 source/lib/rocprof-sys/library/rocprofiler-sdk/rccl_internal.hpp
97.9% 47/48 3/3 source/lib/rocprof-sys/library/pmc/collectors/nic/device.hpp
98.1% 51/52 7/7 source/lib/rocprof-sys/library/pmc/device_providers/amd_smi/provider.hpp
98.6% 73/74 14/15 source/lib/backends/amd_smi/backend.hpp
100.0% 1/1 0/1 source/bin/common/json_config.hpp
100.0% 105/105 13/14 source/lib/backends/amd_smi/device.hpp
100.0% 7/7 0/1 source/lib/backends/amd_smi/gpu_types.hpp
100.0% 1/1 0/1 source/lib/backends/rocprofiler_sdk/types.hpp
100.0% 3/3 0/1 source/lib/common/span.hpp
100.0% 2/2 0/2 source/lib/core/agent_manager.hpp
100.0% 1/1 0/1 source/lib/core/output_file_registry.hpp
100.0% 1/1 0/1 source/lib/core/perfetto/category_registry.hpp
100.0% 1/1 0/1 source/lib/core/perfetto/engine.hpp
100.0% 7/7 0/1 source/lib/core/perfetto/locked_file_append.hpp
100.0% 9/9 2/3 source/lib/core/perfetto/sinks/polymorphic_sink_view.hpp
100.0% 6/6 2/2 source/lib/core/perfetto/sinks/recording_sink.cpp
100.0% 5/5 0/2 source/lib/core/perfetto/sinks/recording_sink.hpp
100.0% 6/6 2/2 source/lib/core/progress/tracker.cpp
100.0% 1/1 0/1 source/lib/core/progress/tracker.hpp
100.0% 2/2 0/1 source/lib/core/state.hpp
100.0% 4/4 0/3 source/lib/core/trace_cache/data_types.hpp
100.0% 9/9 3/3 source/lib/core/trace_cache/type_registry.hpp
100.0% 24/24 4/5 source/lib/core/trace_cache/unified_memory_processor.hpp
100.0% 5/5 2/2 source/lib/core/track_registry.cpp
100.0% 4/4 0/2 source/lib/core/track_registry.hpp
100.0% 10/10 1/1 source/lib/logger/logger.cpp
100.0% 9/9 0/2 source/lib/rocprof-sys/library/pmc/collectors/gpu/types.hpp
100.0% 34/34 3/4 source/lib/rocprof-sys/library/rocprofiler-sdk/marker_writer.hpp
100.0% 3/3 1/2 source/lib/rocprof-sys/library/rocprofiler-sdk/trace_control.hpp

Distribution

Range Files
0-20% 206
20-50% 8
50-80% 22
80-100% 54

E2E Tests

Code Coverage: e2e-tests

Lines: 🟡 62.5% — 20,148/32,243 across 290 files
Functions: 🟡 62.5% — 2,099/3,361

🔴 0-20% (49 files)
Lines Functions File
0.0% 0/102 0/6 source/bin/rocprof-sys-avail/hw_counter_query.cpp
0.0% 0/22 0/1 source/lib/backends/amd_smi/wrapper.hpp
0.0% 0/43 0/2 source/lib/backends/rocprofiler_sdk/backend.hpp
0.0% 0/166 0/2 source/lib/core/agent_info.hpp
0.0% 0/89 0/6 source/lib/core/cpu.cpp
0.0% 0/1 0/1 source/lib/core/cpu.hpp
0.0% 0/53 0/6 source/lib/core/dynamic_library.cpp
0.0% 0/28 0/7 source/lib/core/exception.cpp
0.0% 0/160 0/12 source/lib/core/gpu_metrics.cpp
0.0% 0/79 0/5 source/lib/core/perf.cpp
0.0% 0/13 0/3 source/lib/core/perfetto/sinks/tee_sink.cpp
0.0% 0/1 0/1 source/lib/core/perfetto/sinks/tee_sink.hpp
0.0% 0/7 0/3 source/lib/core/rocpd/data_processor.hpp
0.0% 0/6 0/2 source/lib/rocprof-sys-rt/src/RTsignal.c
0.0% 0/11 0/2 source/lib/rocprof-sys-rt/src/RTthread-x86-64.c
0.0% 0/3 0/1 source/lib/rocprof-sys-rt/src/RTthread.c
0.0% 0/1 0/1 source/lib/rocprof-sys/library/components/exit_gotcha.hpp
0.0% 0/1 0/1 source/lib/rocprof-sys/library/components/kill_gotcha.hpp
0.0% 0/164 0/31 source/lib/rocprof-sys/library/components/vaapi_gotcha.cpp
0.0% 0/52 0/9 source/lib/rocprof-sys/library/pmc/collectors/cpu/perfetto_policy.hpp
0.0% 0/100 0/6 source/lib/rocprof-sys/library/pmc/collectors/gpu/cache_policy.hpp
0.0% 0/40 0/3 source/lib/rocprof-sys/library/pmc/collectors/gpu/gpu_traits.hpp
0.0% 0/222 0/36 source/lib/rocprof-sys/library/pmc/collectors/gpu/perfetto_policy.hpp
0.0% 0/25 0/3 source/lib/rocprof-sys/library/pmc/collectors/gpu/sample.hpp
0.0% 0/32 0/3 source/lib/rocprof-sys/library/pmc/collectors/gpu_perf_counter/cache_policy.hpp
0.0% 0/39 0/7 source/lib/rocprof-sys/library/pmc/collectors/gpu_perf_counter/perfetto_policy.hpp
0.0% 0/16 0/2 source/lib/rocprof-sys/library/pmc/collectors/nic/sample.hpp
0.0% 0/2 0/1 source/lib/rocprof-sys/library/rocm.cpp
0.0% 0/110 0/13 source/lib/rocprof-sys/library/rocprofiler-sdk/counters.cpp
0.0% 0/17 0/6 source/lib/rocprof-sys/library/rocprofiler-sdk/counters.hpp
0.0% 0/129 0/9 source/lib/rocprof-sys/library/rocprofiler-sdk/fwd.cpp
0.0% 0/34 0/7 source/lib/rocprof-sys/library/rocprofiler-sdk/fwd.hpp
0.0% 0/265 0/25 source/lib/rocprof-sys/library/rocprofiler-sdk/kfd_events.cpp
0.0% 0/136 0/13 source/lib/rocprof-sys/library/rocprofiler-sdk/rccl.cpp
0.0% 0/112 0/10 source/lib/rocprof-sys/library/rocprofiler-sdk/roctx_client.cpp
0.0% 0/3 0/1 source/lib/rocprof-sys/library/tracing/annotation.cpp
0.0% 0/45 0/4 source/lib/rocprof-sys/library/tracing/annotation.hpp
0.0% 0/457 0/46 source/python/libpyrocprofsys.cpp
0.6% 1/166 1/13 source/lib/core/perfetto.cpp
2.5% 33/1307 3/88 source/lib/rocprof-sys/library/rocprofiler-sdk.cpp
6.1% 2/33 1/3 source/lib/core/amd_smi.cpp
7.1% 4/56 1/6 source/lib/core/perfetto.hpp
8.3% 3/36 1/9 source/lib/rocprof-sys/library/rocprofiler-sdk/marker_writer.cpp
11.3% 8/71 4/10 source/lib/rocprof-sys/library/components/callchain.cpp
13.6% 38/280 7/31 source/lib/rocprof-sys-rt/src/RTcommon.c
13.8% 4/29 1/4 source/lib/rocprof-sys-rt/src/RTheap-linux.c
14.2% 30/212 4/22 source/lib/rocprof-sys-rt/src/RTlinux.c
17.3% 102/589 7/34 source/lib/core/trace_cache/rocpd_processor.cpp
19.6% 32/163 6/22 source/lib/core/gpu.cpp
🟠 20-50% (22 files)
Lines Functions File
22.2% 76/342 8/22 source/lib/core/rocprofiler-sdk.cpp
22.5% 49/218 5/30 source/lib/rocprof-sys/library/components/backtrace_metrics.cpp
23.6% 122/516 24/71 source/lib/core/trace_cache/metadata_registry.cpp
26.3% 199/758 24/103 source/lib/core/trace_cache/perfetto_processor.cpp
27.8% 27/97 4/8 source/lib/rocprof-sys-rt/src/RTposix.c
32.4% 59/182 6/11 source/lib/rocprof-sys/library/pmc/collectors/common/settings.hpp
35.7% 10/28 3/6 source/lib/rocprof-sys/library/components/exit_gotcha.cpp
38.5% 5/13 2/3 source/lib/rocprof-sys/library/components/kill_gotcha.cpp
38.8% 31/80 1/2 source/bin/rocprof-sys-causal/rocprof-sys-causal.cpp
39.8% 37/93 5/11 source/lib/rocprof-sys/library/causal/components/blocking_gotcha.cpp
41.3% 26/63 7/12 source/lib/core/agent_manager.cpp
43.8% 21/48 6/9 source/lib/core/perfetto/emitter.hpp
43.9% 36/82 2/4 source/lib/rocprof-sys/timeout.cpp
44.4% 4/9 1/3 source/lib/rocprof-sys/library/components/backtrace_timestamp.cpp
44.9% 22/49 4/6 source/lib/core/rocpd/data_storage/database.hpp
45.0% 27/60 6/16 source/lib/rocprof-sys/library/coverage/impl.hpp
46.2% 72/156 3/7 source/bin/rocprof-sys-instrument/info.hpp
46.2% 36/78 6/19 source/lib/rocprof-sys/library/components/numa_gotcha.cpp
47.7% 31/65 3/4 source/lib/rocprof-sys-rt/src/RTheap.c
48.3% 14/29 5/7 source/lib/core/perfetto/emitter.cpp
48.5% 178/367 16/30 source/lib/rocprof-sys/library/components/comm_data.cpp
49.4% 39/79 6/13 source/lib/rocprof-sys-user/user.cpp
🟡 50-80% (57 files)
Lines Functions File
50.0% 8/16 2/4 source/lib/binary/address_multirange.cpp
50.0% 1/2 0/1 source/lib/rocprof-sys/library/components/numa_gotcha.hpp
50.0% 1/2 0/1 source/lib/rocprof-sys/library/components/vaapi_gotcha.hpp
50.0% 1/2 0/1 source/lib/rocprof-sys/library/pmc/collectors/nic/types.hpp
50.0% 1/2 0/1 source/lib/rocprof-sys/library/thread_info.hpp
50.0% 4/8 1/2 source/lib/rocprof-sys/progress.cpp
50.4% 128/254 25/37 source/lib/rocprof-sys/library/kokkosp.cpp
50.5% 55/109 12/30 source/lib/rocprof-sys/library/tracing.hpp
52.0% 13/25 2/3 source/lib/rocprof-sys/library/pmc/collectors/gpu_perf_counter/gpu_perf_counter_traits.hpp
53.2% 41/77 0/1 source/lib/core/trace_cache/sample_processor.hpp
53.4% 79/148 12/18 source/lib/rocprof-sys/library/pmc/sampler.cpp
55.7% 39/70 13/18 source/lib/rocprof-sys/api.cpp
58.0% 142/245 20/31 source/lib/rocprof-sys/library/perf.cpp
60.0% 63/105 5/9 source/bin/common/argument_registration.hpp
60.0% 3/5 1/1 source/lib/core/agent.hpp
61.1% 22/36 1/3 source/lib/common/invoke.hpp
61.9% 616/995 51/65 source/lib/rocprof-sys/library/sampling.cpp
62.1% 18/29 1/1 source/lib/core/perfetto/locked_file_append.cpp
62.3% 48/77 4/5 source/lib/core/trace_cache/discovery.cpp
62.5% 50/80 13/18 source/lib/core/binary/address_range.cpp
62.7% 42/67 5/8 source/lib/rocprof-sys/library/components/fork_gotcha.cpp
63.1% 70/111 7/10 source/lib/core/progress/bar.cpp
64.7% 22/34 3/4 source/lib/core/perfetto/sinks/io_helpers.hpp
65.7% 113/172 17/25 source/lib/core/constraint.cpp
66.1% 37/56 3/4 source/lib/core/mproc.cpp
67.1% 106/158 14/22 source/lib/rocprof-sys/library/components/mpi_gotcha.cpp
67.9% 197/290 21/33 source/lib/core/rocpd/data_processor.cpp
68.6% 81/118 15/20 source/lib/rocprof-sys/library/runtime.cpp
68.8% 22/32 3/8 source/lib/rocprof-sys/library/pmc/collectors/gpu_perf_counter/types.hpp
69.2% 18/26 2/6 source/lib/core/trace_cache/metadata_registry.hpp
69.2% 9/13 0/1 source/lib/rocprof-sys/library/process_sampler.hpp
69.7% 46/66 7/8 source/lib/rocprof-sys/library/pmc/collectors/base/collector.hpp
70.6% 24/34 5/9 source/lib/rocprof-sys/library/components/pthread_gotcha.cpp
71.4% 10/14 1/2 source/lib/core/containers/aligned_static_vector.hpp
71.6% 388/542 46/90 source/lib/rocprof-sys-dl/dl.cpp
72.2% 26/36 8/10 source/lib/rocprof-sys/regions.cpp
72.7% 24/33 5/7 source/lib/rocprof-sys/library/causal/components/causal_gotcha.cpp
72.7% 32/44 5/8 source/lib/rocprof-sys/library/causal/components/unblocking_gotcha.cpp
73.0% 54/74 12/14 source/lib/core/state.cpp
73.9% 407/551 30/33 source/lib/rocprof-sys/library/causal/data.cpp
74.5% 35/47 4/6 source/lib/rocprof-sys/library/pmc/collectors/cpu/cpu_traits.hpp
74.8% 440/588 31/42 source/bin/rocprof-sys-instrument/details.cpp
74.8% 223/298 15/16 source/bin/common/preset_registry.cpp
75.0% 3/4 0/1 source/lib/backends/amd_smi/nic_types.hpp
75.0% 18/24 4/5 source/lib/rocprof-sys/library/causal/sample_data.cpp
75.4% 1168/1550 52/87 source/bin/rocprof-sys-instrument/rocprof-sys-instrument.cpp
75.4% 395/524 30/40 source/bin/common/json_config.cpp
75.6% 90/119 11/12 source/lib/core/trace_cache/post_processor.cpp
76.2% 48/63 5/6 source/bin/rocprof-sys-instrument/log.cpp
76.5% 284/371 28/30 source/bin/common/common_utils.cpp
76.8% 53/69 5/6 source/lib/binary/link_map.cpp
77.3% 17/22 3/3 source/lib/common/static_object.hpp
77.8% 7/9 4/6 source/lib/core/constraint.hpp
77.9% 482/619 27/36 source/lib/rocprof-sys/library.cpp
78.1% 210/269 23/25 source/lib/rocprof-sys/library/causal/sampling.cpp
78.8% 204/259 25/29 source/bin/common/tool_runner.cpp
79.2% 19/24 2/2 source/lib/core/node_info.cpp
🟢 80-100% (162 files)
Lines Functions File
80.0% 4/5 0/1 source/bin/rocprof-sys-instrument/log.hpp
80.1% 209/261 20/22 source/bin/rocprof-sys-instrument/internal_libs.cpp
80.3% 61/76 8/9 source/lib/rocprof-sys/library/pmc/device_providers/rocprofiler_sdk/provider.hpp
80.4% 1371/1705 263/288 source/lib/core/config.cpp
81.2% 69/85 17/22 source/lib/rocprof-sys/library/causal/components/progress_point.cpp
81.2% 13/16 1/1 source/lib/rocprof-sys-dl/main.c
81.7% 432/529 40/40 source/bin/rocprof-sys-instrument/module_function.cpp
81.8% 121/148 22/25 source/lib/rocprof-sys/library/thread_info.cpp
81.8% 81/99 11/11 source/lib/core/perfetto/engine_impl.hpp
82.3% 190/231 23/26 source/bin/rocprof-sys-avail/common.cpp
82.4% 42/51 5/5 source/lib/core/perfetto/cached_perfetto_session.cpp
82.7% 129/156 12/12 source/bin/rocprof-sys-instrument/rocprof-sys-instrument.hpp
83.3% 85/102 4/4 source/lib/core/perfetto/sinks/single_file_sink.cpp
84.2% 314/373 18/29 source/lib/rocprof-sys/library/causal/experiment.cpp
84.2% 16/19 3/5 source/lib/rocprof-sys/library/components/comm_data.hpp
84.3% 86/102 8/9 source/lib/binary/dwarf_entry.cpp
84.6% 11/13 2/4 source/lib/rocprof-sys/library/rocprofiler-sdk/roctx_client.hpp
84.9% 135/159 17/18 source/lib/binary/symbol.cpp
85.1% 74/87 14/16 source/lib/rocprof-sys/library/causal/delay.cpp
85.2% 52/61 8/9 source/lib/core/utility.hpp
85.7% 96/112 9/11 source/lib/rocprof-sys/library/process_sampler.cpp
85.9% 85/99 11/15 source/lib/logger/logger.hpp
86.0% 80/93 5/7 source/bin/rocprof-sys-instrument/module_function.hpp
86.3% 63/73 11/11 source/lib/core/rocpd/data_storage/database.cpp
87.0% 20/23 4/5 source/lib/core/containers/static_vector.hpp
87.0% 47/54 3/3 source/lib/core/trace_cache/cache_manager.cpp
87.5% 42/48 4/5 source/lib/core/output_file_registry.cpp
87.5% 98/112 10/11 source/lib/rocprof-sys/library/tracing.cpp
87.6% 141/161 13/16 source/lib/backends/procfs/backend.hpp
87.6% 283/323 16/20 source/lib/rocprof-sys/library/components/pthread_create_gotcha.cpp
88.2% 388/440 35/47 source/bin/rocprof-sys-causal/impl.cpp
88.6% 39/44 4/4 source/lib/core/utility.cpp
88.7% 268/302 28/32 source/lib/common/environment.hpp
89.0% 129/145 10/10 source/lib/rocprof-sys/library/coverage.cpp
89.1% 49/55 4/4 source/lib/rocprof-sys/library/pmc/collectors/gpu_perf_counter/device.hpp
89.3% 25/28 4/4 source/lib/common/join.hpp
89.3% 767/859 59/74 source/lib/core/argparse.cpp
89.6% 147/164 9/14 source/lib/common/md5sum.hpp
90.0% 18/20 5/5 source/bin/rocprof-sys-instrument/function_signature.hpp
90.0% 9/10 1/1 source/lib/binary/scope_filter.cpp
90.0% 9/10 3/3 source/lib/common/synchronized.hpp
90.0% 54/60 4/4 source/lib/core/trace_cache/buffer_storage.cpp
90.0% 9/10 1/1 source/lib/rocprof-sys/library/thread_deleter.cpp
90.1% 64/71 9/11 source/lib/rocprof-sys/library/components/backtrace.cpp
90.5% 124/137 14/15 source/lib/common/path.hpp
90.6% 106/117 5/5 source/lib/binary/analysis.cpp
90.7% 303/334 28/31 source/lib/core/trace_cache/sample_type.hpp
91.0% 243/267 19/20 source/lib/core/trace_cache/unified_memory_processor.cpp
91.3% 178/195 14/18 source/lib/rocprof-sys/library/components/shmem_gotcha.hpp
91.4% 85/93 6/7 source/bin/rocprof-sys-instrument/function_signature.cpp
91.4% 235/257 11/12 source/bin/rocprof-sys-avail/generate_config.cpp
91.7% 11/12 2/2 source/lib/core/perfetto/session_backend.hpp
91.7% 133/145 29/31 source/lib/rocprof-sys/library/thread_data.hpp
91.9% 102/111 8/8 source/lib/rocprof-sys/library/pmc/collectors/cpu/device.hpp
92.3% 156/169 8/8 source/lib/rocprof-sys/library/pmc/collectors/gpu/device.hpp
92.7% 863/931 40/47 source/bin/rocprof-sys-avail/avail.cpp
92.9% 26/28 3/4 source/lib/core/rocpd/data_storage/insert_query_builders.hpp
92.9% 13/14 0/1 source/lib/rocprof-sys/library/pmc/collectors/common/collector_slice.hpp
92.9% 13/14 2/2 source/lib/rocprof-sys/library/pmc/device_providers/procfs/provider.hpp
93.0% 66/71 7/8 source/lib/core/trace_cache/buffer_storage.hpp
93.1% 27/29 3/5 source/lib/rocprof-sys/library/coverage.hpp
93.3% 111/119 16/17 source/lib/core/perfetto/engine.cpp
93.3% 14/15 3/3 source/lib/core/perfetto/sinks/per_pid_file_sink.cpp
93.8% 30/32 6/6 source/lib/core/mpi.hpp
94.1% 239/254 29/33 source/lib/rocprof-sys/library/components/category_region.hpp
94.1% 16/17 3/3 source/bin/rocprof-sys-avail/component_categories.hpp
94.8% 55/58 1/5 source/lib/core/trace_cache/cacheable.hpp
95.0% 19/20 2/2 source/lib/common/setup.hpp
95.1% 58/61 7/8 source/lib/rocprof-sys/library/pmc/collectors/cpu/sample.hpp
95.7% 45/47 2/2 source/lib/core/trace_cache/storage_parser.hpp
95.8% 23/24 1/2 source/lib/common/delimit.hpp
95.9% 162/169 19/22 source/lib/rocprof-sys/library/components/ucx_gotcha.hpp
96.1% 74/77 6/6 source/lib/rocprof-sys/library/causal/components/backtrace.cpp
96.8% 30/31 3/4 source/lib/rocprof-sys/library/pmc/collectors/gpu_perf_counter/sample.hpp
96.9% 62/64 14/14 source/lib/core/trace_cache/data_types.cpp
96.9% 94/97 10/10 source/lib/rocprof-sys/library/rocprofiler-sdk/trace_control.cpp
97.0% 32/33 4/4 source/lib/core/perfetto/counter_track.hpp
97.3% 73/75 6/6 source/lib/core/common_types.hpp
97.3% 73/75 3/3 source/lib/core/perfetto/packet_framing.hpp
97.8% 45/46 10/12 source/lib/core/containers/stable_vector.hpp
97.8% 45/46 7/7 source/lib/rocprof-sys/library/rocprofiler-sdk/rccl_internal.hpp
97.9% 47/48 3/3 source/lib/rocprof-sys/library/pmc/collectors/nic/device.hpp
98.1% 51/52 7/7 source/lib/rocprof-sys/library/pmc/device_providers/amd_smi/provider.hpp
98.2% 56/57 6/8 source/lib/rocprof-sys/library/components/pthread_mutex_gotcha.hpp
98.6% 73/74 14/15 source/lib/backends/amd_smi/backend.hpp
100.0% 36/36 4/5 source/bin/common/common_utils.hpp
100.0% 1/1 0/1 source/bin/common/json_config.hpp
100.0% 61/61 6/6 source/bin/rocprof-sys-avail/avail.hpp
100.0% 39/39 2/2 source/bin/rocprof-sys-avail/get_availability.hpp
100.0% 10/10 1/1 source/bin/rocprof-sys-avail/get_categories.hpp
100.0% 4/4 1/1 source/bin/rocprof-sys-avail/info_type.cpp
100.0% 9/9 0/1 source/bin/rocprof-sys-avail/info_type.hpp
100.0% 2/2 1/1 source/bin/rocprof-sys-run/rocprof-sys-run.cpp
100.0% 2/2 1/1 source/bin/rocprof-sys-sample/rocprof-sys-sample.cpp
100.0% 105/105 13/14 source/lib/backends/amd_smi/device.hpp
100.0% 7/7 0/1 source/lib/backends/amd_smi/gpu_types.hpp
100.0% 1/1 0/1 source/lib/backends/rocprofiler_sdk/types.hpp
100.0% 9/9 0/1 source/lib/binary/address_multirange.hpp
100.0% 7/7 1/1 source/lib/binary/binary_info.hpp
100.0% 1/1 0/1 source/lib/binary/dwarf_entry.hpp
100.0% 3/3 0/1 source/lib/binary/link_map.hpp
100.0% 5/5 1/2 source/lib/binary/scope_filter.hpp
100.0% 8/8 7/8 source/lib/binary/symbol.hpp
100.0% 9/9 2/2 source/lib/common/env_vars.hpp
100.0% 3/3 0/1 source/lib/common/span.hpp
100.0% 3/3 1/1 source/lib/common/units.hpp
100.0% 2/2 1/2 source/lib/core/agent_manager.hpp
100.0% 5/5 0/2 source/lib/core/argparse.hpp
100.0% 9/9 1/3 source/lib/core/binary/address_range.hpp
100.0% 49/49 10/10 source/lib/core/categories.cpp
100.0% 11/11 11/11 source/lib/core/components/fwd.hpp
100.0% 26/26 3/3 source/lib/core/config.hpp
100.0% 11/11 1/3 source/lib/core/containers/c_array.hpp
100.0% 1/1 0/1 source/lib/core/containers/operators.hpp
100.0% 41/41 7/11 source/lib/core/demangler.hpp
100.0% 2/2 2/2 source/lib/core/node_info.hpp
100.0% 1/1 0/1 source/lib/core/output_file_registry.hpp
100.0% 1/1 0/1 source/lib/core/perfetto/category_registry.hpp
100.0% 1/1 0/1 source/lib/core/perfetto/engine.hpp
100.0% 7/7 0/1 source/lib/core/perfetto/locked_file_append.hpp
100.0% 7/7 1/1 source/lib/core/perfetto/sinks/append_mode.hpp
100.0% 9/9 2/3 source/lib/core/perfetto/sinks/polymorphic_sink_view.hpp
100.0% 6/6 2/2 source/lib/core/perfetto/sinks/recording_sink.cpp
100.0% 5/5 0/2 source/lib/core/perfetto/sinks/recording_sink.hpp
100.0% 4/4 2/2 source/lib/core/perfetto/sinks/single_file_sink.hpp
100.0% 1/1 0/1 source/lib/core/progress/bar.hpp
100.0% 6/6 2/2 source/lib/core/progress/tracker.cpp
100.0% 1/1 0/1 source/lib/core/progress/tracker.hpp
100.0% 7/7 1/2 source/lib/core/rocpd/data_storage/table_insert_query.hpp
100.0% 2/2 0/1 source/lib/core/state.hpp
100.0% 7/7 1/3 source/lib/core/trace_cache/cache_manager.hpp
100.0% 4/4 0/3 source/lib/core/trace_cache/data_types.hpp
100.0% 9/9 3/3 source/lib/core/trace_cache/type_registry.hpp
100.0% 24/24 4/5 source/lib/core/trace_cache/unified_memory_processor.hpp
100.0% 5/5 2/2 source/lib/core/track_registry.cpp
100.0% 4/4 0/2 source/lib/core/track_registry.hpp
100.0% 10/10 1/1 source/lib/logger/logger.cpp
100.0% 7/7 1/2 source/lib/rocprof-sys/library/causal/components/backtrace.hpp
100.0% 23/23 2/3 source/lib/rocprof-sys/library/causal/components/progress_point.hpp
100.0% 3/3 1/2 source/lib/rocprof-sys/library/causal/experiment.hpp
100.0% 4/4 1/1 source/lib/rocprof-sys/library/causal/selected_entry.cpp
100.0% 5/5 2/3 source/lib/rocprof-sys/library/causal/selected_entry.hpp
100.0% 1/1 1/1 source/lib/rocprof-sys/library/components/backtrace.hpp
100.0% 11/11 1/2 source/lib/rocprof-sys/library/components/backtrace_metrics.hpp
100.0% 4/4 0/2 source/lib/rocprof-sys/library/components/backtrace_timestamp.hpp
100.0% 1/1 1/1 source/lib/rocprof-sys/library/components/callchain.hpp
100.0% 8/8 2/2 source/lib/rocprof-sys/library/components/ensure_storage.hpp
100.0% 1/1 0/1 source/lib/rocprof-sys/library/components/fork_gotcha.hpp
100.0% 2/2 0/1 source/lib/rocprof-sys/library/components/mpi_gotcha.hpp
100.0% 533/533 12/12 source/lib/rocprof-sys/library/components/mpip.hpp
100.0% 3/3 0/1 source/lib/rocprof-sys/library/components/pthread_create_gotcha.hpp
100.0% 23/23 1/2 source/lib/rocprof-sys/library/components/pthread_mutex_gotcha_policy.hpp
100.0% 11/11 0/3 source/lib/rocprof-sys/library/perf.hpp
100.0% 66/66 5/5 source/lib/rocprof-sys/library/pmc/collectors/cpu/cache_policy.hpp
100.0% 3/3 0/2 source/lib/rocprof-sys/library/pmc/collectors/cpu/types.hpp
100.0% 9/9 0/2 source/lib/rocprof-sys/library/pmc/collectors/gpu/types.hpp
100.0% 2/2 0/1 source/lib/rocprof-sys/library/pmc/common/types.hpp
100.0% 34/34 3/4 source/lib/rocprof-sys/library/rocprofiler-sdk/marker_writer.hpp
100.0% 3/3 1/2 source/lib/rocprof-sys/library/rocprofiler-sdk/trace_control.hpp
100.0% 2/2 0/1 source/lib/rocprof-sys/library/runtime.hpp
100.0% 10/10 3/3 source/lib/rocprof-sys/library/thread_data_growth.hpp
100.0% 5/5 1/1 source/lib/rocprof-sys/library/thread_deleter.hpp

Distribution

Range Files
0-20% 49
20-50% 22
50-80% 57
80-100% 162

@kcossett-amd

Copy link
Copy Markdown
Contributor

I had to rerun the gfx950 workflow, some odd GPU hangs occurred:

2026-07-10T02:55:18.4245615Z HW Exception by GPU node-6 (Agent handle: 0x5c8feb1f4aa0) reason :GPU Hang

I don't believe these are due to your changes.

@kcossett-amd

kcossett-amd commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

On another note, this time to myself, the sampling variants of transpose are still using more than 1 GPU. I suspect its because base_environment sets "ROCPROFSYS_SAMPLING_GPUS": "all",.

Inside our code, all maps to "enumerate everything" which is done though AMD-SMI, which most likely bypasses HIP_VISIBLE_DEVICES/ROCR_VISIBLE_DEVICES... which is not good because that is how the OSCII MIXXX runners enforce isolation to 1 GPU.

I'll have to investigate this quickly.

@sputhala-amd sputhala-amd changed the title fix(rocprof-systems): Enable per-rank GPU work and correct multi-device usage under MPI fix(rocprof-systems): Fix transpose to work under MPI Jul 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants