[Question] Multi-Node Distributed RL Validation Status on DGX Spark

[iadityaharsh]

Hi everyone,

I run a robotics company developing autonomous UGV/rover-class robots that operate in GPS-denied environments. Each robot carries multiple camera feeds, LiDAR, depth sensors, thermal imagers, and environment detection models running on Jetson Orin Nano Super at the edge.

We’re designing our training infrastructure and would love feedback from anyone who has deployed Isaac Sim and Isaac Lab at scale.

Proposed Setup

• 4× DGX Spark (expanding to 8× later) connected via Mikrotik CRS804-DDQ (4× 400GbE, supports up to 8× Sparks from a single switch)
• 1× RTX 5090 workstation dedicated to Isaac Sim GUI authoring (environment design, sensor rig setup)
• Jetson Orin Nano Super on each robot for edge inference
• Sim-to-real pipeline: author in Isaac Sim → train headless on Spark cluster → validate HIL → deploy to Orin

Following the review on the NVIDIA Developer Forums for this reinforcement learning cluster design, the moderator recommended opening an inquiry here to assess the current validation status and software compatibility regarding Isaac Lab’s multi-node distributed training execution on the NVIDIA DGX Spark platform.

https://forums.developer.nvidia.com/t/dgx-spark-cluster-4-8-nodes-running-isaac-sim-isaac-lab-for-autonomous-robot-training/370549

Questions for the Isaac Lab Team:

1. Multi-Node Execution Status: Has Isaac Lab’s multi-node distributed training module (source/standalone/workflows/rsl_rl/train.py distributed) been actively tested and validated on DGX Spark environment?
2. Unified Memory Scaling & Performance Overhead: Under heavy multi-node scaling, the system's coherent unified memory handles both the local simulation loops (PhysX and RTX ray-tracing for sensor generation) and PyTorch distributed task orchestration simultaneously. Are there any known memory system overhead bottlenecks or performance degradation flags during NCCL multi-node gradient reduction passes under this specific architecture?
3. CUDA 13+ / PyTorch Dependency Baseline: Given that the DGX Spark platform maps to newer CUDA 13+ software baselines, do the repository's installation scripts cleanly target and compile extensions (e.g., rsl_rl or Isaac Lab bindings) against cu13 PyTorch wheels on Spark, or should we anticipate manual packaging workarounds?
4. Native arm64 Container Multi-Stage Stability: Are the repository's native multi-stage Docker build recipes stable when building and executing directly on a native arm64 container deployment engine?

We are looking to catch potential toolchain or multi-node execution hurdles early in our physical AI deployment pipeline. Any insights or scaling guidance would be highly valuable!
Thanks in advance.

[rdsa-nv]

Hi @iadityaharsh, Isaac Lab’s distributed training path uses standard PyTorch torch.distributed/NCCL, so multi-node execution on DGX-class systems should work if the CUDA, PyTorch, and driver stack are aligned. The main thing I would watch on DGX Spark is memory bandwidth. The 128GB vram on the spark is excellent for inference of large models and high Lab env count, but the memory bandwidth will likely be the limiting factor for training. And yes, we build containers for both x86 and arm64 https://catalog.ngc.nvidia.com/orgs/nvidia/containers/isaac-lab/tags?version=3.0.0-beta1

[iadityaharsh]

Hi @rdsa-nv, thanks for the clarification on multi-node and the NGC container link — that's very helpful.

One follow-up on the memory bandwidth point: could you help clarify whether bandwidth constraints affect training wall-clock time (steps/sec throughput) or whether there are scenarios where it could also affect convergence quality (e.g. forcing smaller batch sizes, fewer parallel environments, or truncated rollouts that change the effective sample distribution)? Trying to understand whether it's purely a speed concern or whether it can have downstream effects on policy quality at scale.

Also, we're evaluating several hardware configurations and would appreciate any guidance on relative training performance for our Isaac Lab RL workload (headless, sensor-heavy: multi-camera RGB+depth+thermal, RTX LiDAR, GPS denied confined geometry):

Configuration	VRAM	Notes
RTX 5090 (single)	32GB	Consumer Blackwell, PCIe
RTX Pro 5000 Blackwell (single)	48GB	Pro Blackwell workstation
RTX Pro 6000 Blackwell (single)	96GB	Pro Blackwell workstation
DGX Spark × 1	128GB unified	GB10
DGX Spark × 2 cluster	256GB unified	200GbE ConnectX-7
DGX Spark × 4 cluster	512GB unified	200GbE ConnectX-7

Specific questions:

1. For the GPU-based workstation options (RTX 5090, Pro 5000, Pro 6000), what system RAM would you recommend alongside the GPU? Is there a minimum to avoid the CPU & GPU transfer becoming a bottleneck when reading back sensor observations?
2. How does the DGX Spark's unified memory architecture compare to a discrete GPU + system RAM setup for Isaac Lab specifically does the unified model meaningfully reduce transfer overhead for sensor heavy environments?
3. At what point in scaling (single Spark to 2-node to 4-node) does the inter-node communication overhead start to noticeably offset the gains from additional parallel environments?

This would really help us calibrate whether the cluster investment is justified vs a high-end single node workstation for our workload. Thanks in advance!