General Visualizer Fixes

docs/index.rst

@@ -130,7 +130,6 @@ Table of Contents
    source/features/hydra
    source/features/multi_gpu
    source/features/population_based_training
-   Tiled Rendering</source/overview/core-concepts/sensors/camera>
    source/features/ray
    source/features/reproducibility
 

docs/source/how-to/optimize_stage_creation.rst

@@ -49,7 +49,7 @@ Stage in memory can be toggled by setting the :attr:`isaaclab.sim.SimulationCfg.
     env = ManagerBasedRLEnv(cfg=cfg)
 
 When using stage in memory without an existing RL environment class, wrap the stage creation steps
-in a :py:keyword:`with` statement to set the stage context. The stage is automatically attached
+in a ``with`` statement to set the stage context. The stage is automatically attached
 to the USD context when ``SimulationContext`` is created with ``create_stage_in_memory=True``.
 
 **Using Stage in Memory with a manual scene setup**

docs/source/overview/core-concepts/visualization.rst

@@ -222,11 +222,16 @@ Note, Kit tiled camera views require launching with ``--enable_cameras``.
      - ``tiled_cam_view=False``, ``eye=(4, -4, 3)``, ``lookat=(0, 0, 0)``
      - Interactive visualizer camera starts at ``eye`` and looks at the fixed ``lookat`` coordinate.
    * - Generated tiled camera
-     - ``tiled_cam_view=True``, ``tiled_cam_prim_path=None``, ``tiled_cam_target_prim_path="/World/envs/*/Robot/base"``
+     - ``tiled_cam_view=True``, ``tiled_cam_prim_path=None``, ``tiled_cam_target_prim_path="/World/envs/*/Robot"``
      - The visualizer creates per-env cameras. Each camera looks at the matched target prim, with ``tiled_cam_eye`` as an offset from that target.
+       Note that the ``tiled_cam_target_prim_path`` has a default value, but different environments may require different paths.
    * - Existing tiled camera sensors
      - ``tiled_cam_view=True``, ``tiled_cam_prim_path="/World/envs/*/Camera"``
-     - The visualizer displays existing Isaac Lab ``Camera`` sensor output. Generated-camera fields such as ``tiled_cam_eye`` and ``tiled_cam_target_prim_path`` are ignored.
+     - The visualizer displays existing Isaac Lab ``Camera`` sensor output. Generated-camera fields such as ``tiled_cam_eye`` and
+       ``tiled_cam_target_prim_path`` are ignored. Note that the ``tiled_cam_prim_path`` has a default value, but different
+       environments may require different paths. This mode requires an environment that registers Isaac Lab ``Camera`` sensors
+       in ``scene.sensors``. For Cartpole, use a camera task such as ``Isaac-Cartpole-Camera``. The plain ``Isaac-Cartpole``
+       task has no ``/World/envs/*/Camera`` sensor, so leave ``tiled_cam_prim_path=None`` to use generated visualizer cameras.
 
 **How to Access the Tiled Camera View in the UI**
 
@@ -419,8 +424,8 @@ Newton Visualizer
         tiled_cam_prim_path=None,                 # Existing Camera sensor prim path, e.g. "/World/envs/*/Camera"
         tiled_cam_eye=(4.0, -4.0, 3.0),           # Eye offset for generated tiled cameras
         tiled_cam_target_prim_path=(              # Prim that generated cameras follow/look at
-            "/World/envs/*/Robot/base"
-        ),
+            "/World/envs/*/Robot"                 # This is the default value, but different environments
+        ),                                        # may require a different paths.
 
         # Performance tuning
         update_frequency=1,                       # Update every N frames (1=every frame)
@@ -606,6 +611,13 @@ The FPS control in the Rerun visualizer UI may not affect the visualization fram
 Currently, live plots are only available in the Kit Visualizer.
 
 
+**Newton Contact Visualization**
+
+Newton's native ``Show Contacts`` view can show all contacts from the Newton physics contact buffer. When running
+with PhysX, the Newton visualizer can only show contacts reported by configured Isaac Lab contact sensors, so
+currently the set of displayed contacts may differ across backends.
+
+
 **Viser Visualizer Renderer Requirement**
 
 The Viser visualizer requires a Newton model, which is provided automatically by

docs/source/refs/migration.rst

@@ -157,7 +157,7 @@ Two key patterns support this:
    initialized.
 
 For full details, examples, and the ``{DIR}`` placeholder convention, see the
-:ref:`contributing` guide — in particular the
+:doc:`contributing` guide — in particular the
 `Lazy Loading & Module Exports <contributing.html#lazy-loading-module-exports>`__,
 `Resolvable Strings <contributing.html#resolvable-strings>`__, and
 `Config + Implementation File Split <contributing.html#config-implementation-file-split>`__
@@ -197,7 +197,7 @@ With this in place, ``import my_package`` will not eagerly import any submodules
 are loaded on first access, giving ``SimulationApp`` time to initialize and auto-detect the
 correct backend.
 
-For more details, refer to the :ref:`contributing` guide.
+For more details, refer to the :doc:`contributing` guide.
 
 
 .. _simple script: https://gist.github.com/kellyguo11/3e8f73f739b1c013b1069ad372277a85

source/isaaclab/changelog.d/clarify-single-video-stream.rst

@@ -0,0 +1,6 @@
+Changed
+^^^^^^^
+
+* Clarified ``--video`` behavior when multiple video-capable visualizers are active:
+  Gymnasium video recording captures one ``env.render()`` stream, with Kit taking
+  priority over Newton.

source/isaaclab/changelog.d/clarify-visualizer-camera-sensor-mode.rst

@@ -0,0 +1,6 @@
+Fixed
+^^^^^
+
+* Improved visualizer tiled-camera errors when ``tiled_cam_prim_path`` is set but
+  the scene has no Isaac Lab ``Camera`` sensors, and clarified the camera-mode
+  documentation for Cartpole camera tasks.

source/isaaclab/changelog.d/fix-scene-data-articulation-transforms.rst

@@ -0,0 +1,6 @@
+Added
+^^^^^
+
+* Added a scene-data backend hook for active ``InteractiveScene`` access so
+  backends can source scene-owned entity transforms without relying on global
+  rigid-body views, and visualizers can discover scene-owned contact sensors.

source/isaaclab/isaaclab/cli/commands/install.py

@@ -652,7 +652,7 @@ def _install_extra_feature(feature_name: str, selector: str = "") -> None:
     elif feature_name == "newton":
         if selector:
             print_warning(f"'newton' does not support selectors (got '{selector}'). Installing all newton extras.")
-        print_info("Installing newton extras (newton[sim], PyOpenGL-accelerate, imgui-bundle)...")
+        print_info("Installing newton extras (newton[sim], PyOpenGL-accelerate, imgui-bundle, typing-extensions)...")
         run_command(pip_cmd + ["install", "--editable", f"{source_dir}/isaaclab_newton[all]"])
         run_command(pip_cmd + ["install", "--editable", f"{source_dir}/isaaclab_physx[newton]"])
         run_command(pip_cmd + ["install", "--editable", f"{source_dir}/isaaclab_visualizers[newton]"])

source/isaaclab/isaaclab/envs/utils/camera_view.py

@@ -92,7 +92,20 @@ def find_camera_by_prim_path(camera_sensors: dict[str, Camera], cam_prim_path: s
             f"cam_prim_path={cam_prim_path!r} matched USD camera prims, but no Isaac Lab Camera sensor owns them. "
             "Add the camera to scene.sensors or leave tiled_cam_prim_path unset to use generated tiled cameras."
         )
-    raise RuntimeError(f"No Isaac Lab Camera sensor matched cam_prim_path={cam_prim_path!r}.")
+    if not camera_sensors:
+        raise RuntimeError(
+            f"No Isaac Lab Camera sensors are registered in the scene, so tiled_cam_prim_path={cam_prim_path!r} "
+            "cannot be used. Use an environment that defines Camera sensors, or leave tiled_cam_prim_path unset "
+            "to use generated tiled cameras."
+        )
+    available_paths = {
+        getattr(camera.cfg, "prim_path", None) for camera in camera_sensors.values() if getattr(camera, "cfg", None)
+    }
+    raise RuntimeError(
+        f"No Isaac Lab Camera sensor matched cam_prim_path={cam_prim_path!r}. "
+        f"Available Camera sensor prim paths: {sorted(path for path in available_paths if path)}. "
+        "Leave tiled_cam_prim_path unset to use generated tiled cameras."
+    )
 
 
 def ensure_camera_initialized(camera: Camera) -> None:

source/isaaclab/isaaclab/envs/utils/video_recorder.py

@@ -71,6 +71,14 @@ def _resolve_video_backend(
     # Prefer the visualizer backend when --visualizer is active alongside --video.
     visualizer_types: list[str] = scene.sim.resolve_visualizer_types() if backend_source == "visualizer" else []
     if visualizer_types:
+        supported_visualizers = [viz for viz in ("kit", "newton") if viz in visualizer_types]
+        if len(supported_visualizers) > 1:
+            logger.warning(
+                "[VideoRecorder] Multiple video-capable visualizers are active (%s), but --video records one "
+                "env.render() stream. Using Kit because it has priority. Run with only --viz newton to record "
+                "a Newton GL video.",
+                supported_visualizers,
+            )
         # kit takes priority when multiple visualizers are active
         for preferred in ("kit", "newton"):
             if preferred in visualizer_types:

source/isaaclab/isaaclab/markers/visualization_markers.py

@@ -111,6 +111,7 @@ def __init__(self, cfg: VisualizationMarkersCfg):
         self.prim_path = cfg.prim_path
         self._count = len(cfg.markers)
         self._is_visible = True
+        self._has_visualized = False
         self._backends: list[object] = []
         self._ensure_backends_initialized()
 
@@ -226,7 +227,11 @@ def visualize(
             if value is not None:
                 num_markers = value.shape[0]
 
-        if norm_marker_indices is None and num_markers != 0 and num_markers != self._count:
+        if (
+            norm_marker_indices is None
+            and num_markers != 0
+            and (not self._has_visualized or num_markers != self._count)
+        ):
             norm_marker_indices = torch.zeros(num_markers, dtype=torch.int32, device=target_device)
         elif norm_marker_indices is None and num_markers == 0:
             if all(value is None for value in (norm_translations, norm_orientations, norm_scales)):
@@ -238,6 +243,7 @@ def visualize(
 
         if num_markers != 0:
             self._count = num_markers
+            self._has_visualized = True
 
     def __del__(self):
         for backend in getattr(self, "_backends", []):

source/isaaclab/isaaclab/scene_data/scene_data_provider.py

@@ -56,6 +56,18 @@ def get_camera_sensors(self) -> dict[str, Any]:
             if isinstance(sensor, Camera)
         }
 
+    def get_contact_sensors(self) -> dict[str, Any]:
+        """Return Isaac Lab contact sensors keyed by scene sensor name."""
+        if self._interactive_scene is None:
+            return {}
+        from isaaclab.sensors.contact_sensor import BaseContactSensor
+
+        return {
+            name: sensor
+            for name, sensor in getattr(self._interactive_scene, "sensors", {}).items()
+            if isinstance(sensor, BaseContactSensor)
+        }
+
     @property
     def transform_count(self) -> int:
         """Number of transforms available from the sim backend."""

source/isaaclab/test/envs/test_video_recorder.py

@@ -172,6 +172,17 @@ def test_resolve_backend_kit_wins_over_newton_visualizer():
     assert matched == "kit"
 
 
+def test_resolve_backend_warns_when_multiple_video_capable_visualizers(caplog: pytest.LogCaptureFixture):
+    """The Gymnasium video wrapper records one stream even if both Kit and Newton are active."""
+    scene = _make_scene(["newton", "kit"])
+    with caplog.at_level("WARNING"):
+        backend, matched = _resolve_video_backend(scene)
+
+    assert backend == "kit"
+    assert matched == "kit"
+    assert any("Multiple video-capable visualizers are active" in record.getMessage() for record in caplog.records)
+
+
 def test_resolve_backend_unsupported_visualizer_falls_through():
     """viser/rerun visualizers fall through to physics stack detection."""
     scene = _make_scene(["viser"], physics_name="PhysxPhysicsManager")

source/isaaclab/test/markers/test_visualization_markers.py

@@ -144,6 +144,26 @@ def test_rendering_context_authors_visible_usd_point_instancer(sim):
     assert list(instancer.GetProtoIndicesAttr().Get()) == [0, 1]
 
 
+def test_first_visualize_defaults_to_first_prototype_when_count_matches_prototypes(sim):
+    """Omitted marker indices should not preserve initialization prototype placeholders."""
+    from pxr import UsdGeom
+
+    sim._has_offscreen_render = True
+    config = VisualizationMarkersCfg(
+        prim_path="/World/Visuals/default_marker_indices",
+        markers={
+            "frame": sim_utils.SphereCfg(radius=0.1),
+            "line": sim_utils.CuboidCfg(size=(0.1, 0.1, 0.1)),
+        },
+    )
+    test_marker = VisualizationMarkers(config)
+
+    test_marker.visualize(translations=torch.tensor([[0.0, 0.0, 0.0], [0.2, 0.0, 0.0]], device=sim.device))
+
+    instancer = UsdGeom.PointInstancer(sim_utils.get_current_stage().GetPrimAtPath(test_marker.prim_path))
+    assert list(instancer.GetProtoIndicesAttr().Get()) == [0, 0]
+
+
 def test_usd_marker(sim):
     """Test with marker from a USD."""
     # create a marker
@@ -253,6 +273,7 @@ class _FakeViewer:
 
         def __init__(self):
             self.calls = []
+            self.show_contacts = False
 
         def is_paused(self):
             return False
@@ -263,6 +284,9 @@ def begin_frame(self, sim_time):
         def log_state(self, state):
             self.calls.append(("log_state", state))
 
+        def log_arrows(self, name, starts, ends, colors):
+            pass
+
         def end_frame(self):
             self.calls.append(("end_frame",))
 
@@ -276,6 +300,10 @@ def get_state(scene_data_provider=None):
         def get_num_envs() -> int:
             return 4
 
+        @staticmethod
+        def get_contacts():
+            return None
+
     def _fake_render_markers(viewer, visible_env_ids, num_envs):
         marker_calls.append((viewer, visible_env_ids, num_envs))
 

source/isaaclab/test/visualization/check_scene_xr_visualization.py → source/isaaclab/test/xr_visualization/check_scene_xr_visualization.py

@@ -9,7 +9,7 @@
 .. code-block:: bash
 
     # Usage
-    ./isaaclab.sh -p source/isaaclab/test/visualization/check_scene_visualization.py
+    ./isaaclab.sh -p source/isaaclab/test/xr_visualization/check_scene_xr_visualization.py
 
 """
 

source/isaaclab_newton/changelog.d/expose-newton-contact-buffer.rst

@@ -0,0 +1,5 @@
+Added
+^^^^^
+
+* Added a ``NewtonManager.get_contacts()`` accessor so visualizers can render
+  Newton contact buffers without reaching into manager internals.

source/isaaclab_newton/isaaclab_newton/physics/newton_manager.py

@@ -1584,6 +1584,11 @@ def get_state(cls, scene_data_provider: SceneDataProvider | None = None) -> Stat
         cls.update_visualization_state(scene_data_provider)
         return cls.get_state_0()
 
+    @classmethod
+    def get_contacts(cls) -> Contacts | None:
+        """Get the current Newton contact buffer, if the active solver exposes one."""
+        return cls._contacts
+
     @classmethod
     def get_num_envs(cls) -> int:
         return cls._num_envs

source/isaaclab_physx/changelog.d/reduce-ant-joint-wrench-warnings.rst

@@ -0,0 +1,6 @@
+Fixed
+^^^^^
+
+* Fixed excessive PhysX tensor warnings from Ant tasks with ``JointWrenchSensor``
+  by sourcing scene-data transforms for articulation links from Isaac Lab
+  articulation views instead of a global PhysX rigid-body view.

source/isaaclab_physx/isaaclab_physx/physics/physx_manager.py

@@ -173,9 +173,10 @@ def simulation_view(self, simulation_view: omni.physics.tensors.SimulationView |
     def get_rigid_body_view(self) -> omni.physics.tensors.RigidBodyView | None:
         """Lazily create a rigid body view covering all rigid bodies in the scene.
 
-        Discovers rigid body prims by traversing the USD stage and converts
-        per-environment paths (``/World/envs/env_N/...``) into wildcard
-        patterns so a single PhysX view covers every environment instance.
+        Discovers exact rigid body prims by traversing USD, then compacts cloned
+        environment paths into wildcard patterns. If a rigid body name is also
+        used by a non-rigid prim, the exact path is kept to avoid PhysX resolving
+        the wildcard to the non-rigid prim.
         """
         if self._rigid_body_view is not None:
             return self._rigid_body_view
@@ -187,15 +188,29 @@ def get_rigid_body_view(self) -> omni.physics.tensors.RigidBodyView | None:
         if stage is None:
             return None
 
-        patterns: set[str] = set()
+        rigid_body_paths: list[str] = []
+        non_rigid_body_names: set[str] = set()
         for prim in stage.Traverse():
+            prim_path = prim.GetPath().pathString
             if prim.HasAPI(UsdPhysics.RigidBodyAPI):
-                patterns.add(re.sub(r"/World/envs/env_\d+", "/World/envs/env_*", prim.GetPath().pathString))
+                rigid_body_paths.append(prim_path)
+            elif re.search(r"/World/envs/env_\d+/", prim_path):
+                non_rigid_body_names.add(prim_path.rsplit("/", 1)[-1])
 
-        if not patterns:
+        patterns: set[str] = set()
+        exact_paths: list[str] = []
+        for prim_path in rigid_body_paths:
+            body_name = prim_path.rsplit("/", 1)[-1]
+            if body_name in non_rigid_body_names:
+                exact_paths.append(prim_path)
+            else:
+                patterns.add(re.sub(r"/World/envs/env_\d+", "/World/envs/env_*", prim_path))
+
+        body_paths = [*sorted(patterns), *exact_paths]
+        if not body_paths:
             return None
 
-        self._rigid_body_view = self._simulation_view.create_rigid_body_view(list(patterns))
+        self._rigid_body_view = self._simulation_view.create_rigid_body_view(body_paths)
         return self._rigid_body_view
 
     @property

source/isaaclab_visualizers/changelog.d/fix-newton-contact-visualization.rst

@@ -0,0 +1,6 @@
+Fixed
+^^^^^
+
+* Fixed Newton visualizer contact rendering by logging Newton contact buffers
+  when available and falling back to scene contact sensors for PhysX-backed
+  scenes.

source/isaaclab_visualizers/changelog.d/fix-newton-hud-imgui-dependency.rst

@@ -0,0 +1,15 @@
+Fixed
+^^^^^
+
+* Fixed Newton visualizer HUD dependency checks by requiring
+  ``typing-extensions>=4.15.0`` for the Newton visualizer extra and failing
+  integration tests when Newton reports that ``imgui_bundle`` could not be
+  imported. Removed the legacy ``setup.py`` for ``isaaclab_visualizers`` now that
+  ``pyproject.toml`` carries the package metadata.
+
+* Fixed Rerun and Viser visualizers rendering Newton infinite ground planes too
+  small by expanding non-positive plane extents to the same large finite size
+  used by Newton GL.
+
+* Fixed Viser visualizer ground-grid flickering by reusing unchanged plane grid
+  line segments instead of removing and re-adding them every frame.