[DOC] System Tuning Tutorial + tune_system.py description gaps for IGX Thor DevKit

[RamyaGuru]

Body

Three documentation gaps surfaced while running the raw GPUDirect sweep on an IGX Thor devkit (aarch64, driver 580.00/CUDA 13.0, integrated Thor iGPU sm_110 + discrete RTX PRO 6000 sm_120).

---

1. tune_system.py points to the wrong package for ibdev2netdev

tune_system.py warns users to run apt install infiniband-diags when ibdev2netdev is not found. On Ubuntu 24.04, infiniband-diags ships without ibdev2netdev — that script is part of mlnx-tools (or DOCA-Host), not infiniband-diags. Installing the suggested package leaves the tool in the same broken state, and the MRRS/MPS/MTU checks are silently skipped.

Fixes needed:

• Correct the hint to point at mlnx-tools / DOCA-Host (or ibv_devinfo from ibverbs-utils as a fallback).
• Ideally add a fallback that reads MRRS directly from PCIe config via /sys/class/net/<dev>/device → lspci -vvs <bdf> | grep MaxReadReq (or setpci on the DevCtl register) when ibdev2netdev is absent.

---

2. tune_system.py GPU checks always target index 0 — wrong on Tegra iGPU+dGPU systems

All GPU checks in tune_system.py (BAR1 size, GPUDirect-RDMA support, peermem, PIX-PXB topology) default to GPU index 0. On IGX Thor, index 0 is the integrated Thor iGPU (sm_110). The tool therefore reports "integrated GPU, no resizable BAR1, use kind: host_pinned, peermem N/A" — all of which is l target: the discrete RTX PRO 6000 (index 1), which has a real 256 MiB BAR1 and workingdma-buf GPUDirect.
Additional complexity — CUDA vs nvidia-smi index inversion on Tegra: the CUDA runtime enumerates GPUs in the opposite order from nvidia-smi/NVML on Tegra. With CUDA_VISIBLE_DEVICES unset, libcuda sonly the iGPU (count=1). Setting CUDA_VISIBLE_DEVICES=1 selects the iGPU (CUDA index 11 = dGPU — opposite). The only reliable selector is the UUID:CUDA_VISIBLE_DEVICES=GPU-<uuid>.
Fix needed: - Add a --gpu N flag (or auto-select the discrete GPU, or check every GPU) so the BAR1/GPUet the intended device.

• Document the Tegra CUDA-vs-nvidia-smi index inversion and recommend using the GPU UUID as the selector.

---

3. Docs gap: --privileged overrides --gpus '"device=N"' on Tegra; no guidance on GPU pinning

The documented container launch uses --privileged (required for DPDK). On a system with both an integrated and a discrete NVIDIA GPU, --privileged floods /dev with all Tegra iGPU nodes, which causes the CUDA runtime to enumerate only the iGPU regardless of --gpus '"device=N"'. affinity and wrong device.

The reliable workaround is -e CUDA_VISIBLE_DEVICES=GPU-<uuid> (by UUID, not by index — seeersion).

Docs to update:

• docs/benchmarks/raw_benchmarking.md
• docs/tutorials/system_configuration.md

Add a "Tegra / IGX with a discrete GPU" note: when running with --privileged on a system t dGPU, pin the target GPU with CUDA_VISIBLE_DEVICES=GPU-<uuid> rather than by index.

---

Discovered on: IGX Thor aarch64, kernel 6.8.0-1019-nvidia-tegra-rt, host driver 580.00oceP4p3s0f0/f1), daqiri 1134167`.