diff --git a/docs/index.rst b/docs/index.rst index 448d897852e0..2a304b8f8d7b 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -130,7 +130,6 @@ Table of Contents source/features/hydra source/features/multi_gpu source/features/population_based_training - Tiled Rendering source/features/ray source/features/reproducibility diff --git a/docs/source/how-to/optimize_stage_creation.rst b/docs/source/how-to/optimize_stage_creation.rst index 293ee84ec7d3..3b6a6b28924a 100644 --- a/docs/source/how-to/optimize_stage_creation.rst +++ b/docs/source/how-to/optimize_stage_creation.rst @@ -49,7 +49,7 @@ Stage in memory can be toggled by setting the :attr:`isaaclab.sim.SimulationCfg. env = ManagerBasedRLEnv(cfg=cfg) When using stage in memory without an existing RL environment class, wrap the stage creation steps -in a :py:keyword:`with` statement to set the stage context. The stage is automatically attached +in a ``with`` statement to set the stage context. The stage is automatically attached to the USD context when ``SimulationContext`` is created with ``create_stage_in_memory=True``. **Using Stage in Memory with a manual scene setup** diff --git a/docs/source/overview/core-concepts/visualization.rst b/docs/source/overview/core-concepts/visualization.rst index 8fa20c86a652..f25042e7437b 100644 --- a/docs/source/overview/core-concepts/visualization.rst +++ b/docs/source/overview/core-concepts/visualization.rst @@ -222,11 +222,16 @@ Note, Kit tiled camera views require launching with ``--enable_cameras``. - ``tiled_cam_view=False``, ``eye=(4, -4, 3)``, ``lookat=(0, 0, 0)`` - Interactive visualizer camera starts at ``eye`` and looks at the fixed ``lookat`` coordinate. * - Generated tiled camera - - ``tiled_cam_view=True``, ``tiled_cam_prim_path=None``, ``tiled_cam_target_prim_path="/World/envs/*/Robot/base"`` + - ``tiled_cam_view=True``, ``tiled_cam_prim_path=None``, ``tiled_cam_target_prim_path="/World/envs/*/Robot"`` - The visualizer creates per-env cameras. Each camera looks at the matched target prim, with ``tiled_cam_eye`` as an offset from that target. + Note that the ``tiled_cam_target_prim_path`` has a default value, but different environments may require different paths. * - Existing tiled camera sensors - ``tiled_cam_view=True``, ``tiled_cam_prim_path="/World/envs/*/Camera"`` - - The visualizer displays existing Isaac Lab ``Camera`` sensor output. Generated-camera fields such as ``tiled_cam_eye`` and ``tiled_cam_target_prim_path`` are ignored. + - The visualizer displays existing Isaac Lab ``Camera`` sensor output. Generated-camera fields such as ``tiled_cam_eye`` and + ``tiled_cam_target_prim_path`` are ignored. Note that the ``tiled_cam_prim_path`` has a default value, but different + environments may require different paths. This mode requires an environment that registers Isaac Lab ``Camera`` sensors + in ``scene.sensors``. For Cartpole, use a camera task such as ``Isaac-Cartpole-Camera``. The plain ``Isaac-Cartpole`` + task has no ``/World/envs/*/Camera`` sensor, so leave ``tiled_cam_prim_path=None`` to use generated visualizer cameras. **How to Access the Tiled Camera View in the UI** @@ -419,8 +424,8 @@ Newton Visualizer tiled_cam_prim_path=None, # Existing Camera sensor prim path, e.g. "/World/envs/*/Camera" tiled_cam_eye=(4.0, -4.0, 3.0), # Eye offset for generated tiled cameras tiled_cam_target_prim_path=( # Prim that generated cameras follow/look at - "/World/envs/*/Robot/base" - ), + "/World/envs/*/Robot" # This is the default value, but different environments + ), # may require a different paths. # Performance tuning update_frequency=1, # Update every N frames (1=every frame) @@ -606,6 +611,13 @@ The FPS control in the Rerun visualizer UI may not affect the visualization fram Currently, live plots are only available in the Kit Visualizer. +**Newton Contact Visualization** + +Newton's native ``Show Contacts`` view can show all contacts from the Newton physics contact buffer. When running +with PhysX, the Newton visualizer can only show contacts reported by configured Isaac Lab contact sensors, so +currently the set of displayed contacts may differ across backends. + + **Viser Visualizer Renderer Requirement** The Viser visualizer requires a Newton model, which is provided automatically by diff --git a/docs/source/refs/migration.rst b/docs/source/refs/migration.rst index 3ae599e91509..11db227bc142 100644 --- a/docs/source/refs/migration.rst +++ b/docs/source/refs/migration.rst @@ -157,7 +157,7 @@ Two key patterns support this: initialized. For full details, examples, and the ``{DIR}`` placeholder convention, see the -:ref:`contributing` guide — in particular the +:doc:`contributing` guide — in particular the `Lazy Loading & Module Exports `__, `Resolvable Strings `__, and `Config + Implementation File Split `__ @@ -197,7 +197,7 @@ With this in place, ``import my_package`` will not eagerly import any submodules are loaded on first access, giving ``SimulationApp`` time to initialize and auto-detect the correct backend. -For more details, refer to the :ref:`contributing` guide. +For more details, refer to the :doc:`contributing` guide. .. _simple script: https://gist.github.com/kellyguo11/3e8f73f739b1c013b1069ad372277a85 diff --git a/source/isaaclab/changelog.d/clarify-single-video-stream.rst b/source/isaaclab/changelog.d/clarify-single-video-stream.rst new file mode 100644 index 000000000000..39e5bacdc191 --- /dev/null +++ b/source/isaaclab/changelog.d/clarify-single-video-stream.rst @@ -0,0 +1,6 @@ +Changed +^^^^^^^ + +* Clarified ``--video`` behavior when multiple video-capable visualizers are active: + Gymnasium video recording captures one ``env.render()`` stream, with Kit taking + priority over Newton. diff --git a/source/isaaclab/changelog.d/clarify-visualizer-camera-sensor-mode.rst b/source/isaaclab/changelog.d/clarify-visualizer-camera-sensor-mode.rst new file mode 100644 index 000000000000..cb6f9c0081f7 --- /dev/null +++ b/source/isaaclab/changelog.d/clarify-visualizer-camera-sensor-mode.rst @@ -0,0 +1,6 @@ +Fixed +^^^^^ + +* Improved visualizer tiled-camera errors when ``tiled_cam_prim_path`` is set but + the scene has no Isaac Lab ``Camera`` sensors, and clarified the camera-mode + documentation for Cartpole camera tasks. diff --git a/source/isaaclab/changelog.d/fix-scene-data-articulation-transforms.rst b/source/isaaclab/changelog.d/fix-scene-data-articulation-transforms.rst new file mode 100644 index 000000000000..14be85dbc997 --- /dev/null +++ b/source/isaaclab/changelog.d/fix-scene-data-articulation-transforms.rst @@ -0,0 +1,6 @@ +Added +^^^^^ + +* Added a scene-data backend hook for active ``InteractiveScene`` access so + backends can source scene-owned entity transforms without relying on global + rigid-body views, and visualizers can discover scene-owned contact sensors. diff --git a/source/isaaclab/isaaclab/cli/commands/install.py b/source/isaaclab/isaaclab/cli/commands/install.py index 2fe6368b65d0..f4d141dc4a08 100644 --- a/source/isaaclab/isaaclab/cli/commands/install.py +++ b/source/isaaclab/isaaclab/cli/commands/install.py @@ -652,7 +652,7 @@ def _install_extra_feature(feature_name: str, selector: str = "") -> None: elif feature_name == "newton": if selector: print_warning(f"'newton' does not support selectors (got '{selector}'). Installing all newton extras.") - print_info("Installing newton extras (newton[sim], PyOpenGL-accelerate, imgui-bundle)...") + print_info("Installing newton extras (newton[sim], PyOpenGL-accelerate, imgui-bundle, typing-extensions)...") run_command(pip_cmd + ["install", "--editable", f"{source_dir}/isaaclab_newton[all]"]) run_command(pip_cmd + ["install", "--editable", f"{source_dir}/isaaclab_physx[newton]"]) run_command(pip_cmd + ["install", "--editable", f"{source_dir}/isaaclab_visualizers[newton]"]) diff --git a/source/isaaclab/isaaclab/envs/utils/camera_view.py b/source/isaaclab/isaaclab/envs/utils/camera_view.py index 6f5c193d4ecd..a686e0e65b16 100644 --- a/source/isaaclab/isaaclab/envs/utils/camera_view.py +++ b/source/isaaclab/isaaclab/envs/utils/camera_view.py @@ -92,7 +92,20 @@ def find_camera_by_prim_path(camera_sensors: dict[str, Camera], cam_prim_path: s f"cam_prim_path={cam_prim_path!r} matched USD camera prims, but no Isaac Lab Camera sensor owns them. " "Add the camera to scene.sensors or leave tiled_cam_prim_path unset to use generated tiled cameras." ) - raise RuntimeError(f"No Isaac Lab Camera sensor matched cam_prim_path={cam_prim_path!r}.") + if not camera_sensors: + raise RuntimeError( + f"No Isaac Lab Camera sensors are registered in the scene, so tiled_cam_prim_path={cam_prim_path!r} " + "cannot be used. Use an environment that defines Camera sensors, or leave tiled_cam_prim_path unset " + "to use generated tiled cameras." + ) + available_paths = { + getattr(camera.cfg, "prim_path", None) for camera in camera_sensors.values() if getattr(camera, "cfg", None) + } + raise RuntimeError( + f"No Isaac Lab Camera sensor matched cam_prim_path={cam_prim_path!r}. " + f"Available Camera sensor prim paths: {sorted(path for path in available_paths if path)}. " + "Leave tiled_cam_prim_path unset to use generated tiled cameras." + ) def ensure_camera_initialized(camera: Camera) -> None: diff --git a/source/isaaclab/isaaclab/envs/utils/video_recorder.py b/source/isaaclab/isaaclab/envs/utils/video_recorder.py index 3ff6a7e1a0a4..0925b4a0ab1c 100644 --- a/source/isaaclab/isaaclab/envs/utils/video_recorder.py +++ b/source/isaaclab/isaaclab/envs/utils/video_recorder.py @@ -71,6 +71,14 @@ def _resolve_video_backend( # Prefer the visualizer backend when --visualizer is active alongside --video. visualizer_types: list[str] = scene.sim.resolve_visualizer_types() if backend_source == "visualizer" else [] if visualizer_types: + supported_visualizers = [viz for viz in ("kit", "newton") if viz in visualizer_types] + if len(supported_visualizers) > 1: + logger.warning( + "[VideoRecorder] Multiple video-capable visualizers are active (%s), but --video records one " + "env.render() stream. Using Kit because it has priority. Run with only --viz newton to record " + "a Newton GL video.", + supported_visualizers, + ) # kit takes priority when multiple visualizers are active for preferred in ("kit", "newton"): if preferred in visualizer_types: diff --git a/source/isaaclab/isaaclab/markers/visualization_markers.py b/source/isaaclab/isaaclab/markers/visualization_markers.py index 667f9b4ddf7d..ef9eb65e9927 100644 --- a/source/isaaclab/isaaclab/markers/visualization_markers.py +++ b/source/isaaclab/isaaclab/markers/visualization_markers.py @@ -111,6 +111,7 @@ def __init__(self, cfg: VisualizationMarkersCfg): self.prim_path = cfg.prim_path self._count = len(cfg.markers) self._is_visible = True + self._has_visualized = False self._backends: list[object] = [] self._ensure_backends_initialized() @@ -226,7 +227,11 @@ def visualize( if value is not None: num_markers = value.shape[0] - if norm_marker_indices is None and num_markers != 0 and num_markers != self._count: + if ( + norm_marker_indices is None + and num_markers != 0 + and (not self._has_visualized or num_markers != self._count) + ): norm_marker_indices = torch.zeros(num_markers, dtype=torch.int32, device=target_device) elif norm_marker_indices is None and num_markers == 0: if all(value is None for value in (norm_translations, norm_orientations, norm_scales)): @@ -238,6 +243,7 @@ def visualize( if num_markers != 0: self._count = num_markers + self._has_visualized = True def __del__(self): for backend in getattr(self, "_backends", []): diff --git a/source/isaaclab/isaaclab/scene_data/scene_data_provider.py b/source/isaaclab/isaaclab/scene_data/scene_data_provider.py index 9d888347e38e..ba4c051920e1 100644 --- a/source/isaaclab/isaaclab/scene_data/scene_data_provider.py +++ b/source/isaaclab/isaaclab/scene_data/scene_data_provider.py @@ -56,6 +56,18 @@ def get_camera_sensors(self) -> dict[str, Any]: if isinstance(sensor, Camera) } + def get_contact_sensors(self) -> dict[str, Any]: + """Return Isaac Lab contact sensors keyed by scene sensor name.""" + if self._interactive_scene is None: + return {} + from isaaclab.sensors.contact_sensor import BaseContactSensor + + return { + name: sensor + for name, sensor in getattr(self._interactive_scene, "sensors", {}).items() + if isinstance(sensor, BaseContactSensor) + } + @property def transform_count(self) -> int: """Number of transforms available from the sim backend.""" diff --git a/source/isaaclab/test/envs/test_video_recorder.py b/source/isaaclab/test/envs/test_video_recorder.py index 91af56632d6b..e22cbe656147 100644 --- a/source/isaaclab/test/envs/test_video_recorder.py +++ b/source/isaaclab/test/envs/test_video_recorder.py @@ -172,6 +172,17 @@ def test_resolve_backend_kit_wins_over_newton_visualizer(): assert matched == "kit" +def test_resolve_backend_warns_when_multiple_video_capable_visualizers(caplog: pytest.LogCaptureFixture): + """The Gymnasium video wrapper records one stream even if both Kit and Newton are active.""" + scene = _make_scene(["newton", "kit"]) + with caplog.at_level("WARNING"): + backend, matched = _resolve_video_backend(scene) + + assert backend == "kit" + assert matched == "kit" + assert any("Multiple video-capable visualizers are active" in record.getMessage() for record in caplog.records) + + def test_resolve_backend_unsupported_visualizer_falls_through(): """viser/rerun visualizers fall through to physics stack detection.""" scene = _make_scene(["viser"], physics_name="PhysxPhysicsManager") diff --git a/source/isaaclab/test/markers/test_visualization_markers.py b/source/isaaclab/test/markers/test_visualization_markers.py index b9ae8387cf0f..79a66443eca1 100644 --- a/source/isaaclab/test/markers/test_visualization_markers.py +++ b/source/isaaclab/test/markers/test_visualization_markers.py @@ -144,6 +144,26 @@ def test_rendering_context_authors_visible_usd_point_instancer(sim): assert list(instancer.GetProtoIndicesAttr().Get()) == [0, 1] +def test_first_visualize_defaults_to_first_prototype_when_count_matches_prototypes(sim): + """Omitted marker indices should not preserve initialization prototype placeholders.""" + from pxr import UsdGeom + + sim._has_offscreen_render = True + config = VisualizationMarkersCfg( + prim_path="/World/Visuals/default_marker_indices", + markers={ + "frame": sim_utils.SphereCfg(radius=0.1), + "line": sim_utils.CuboidCfg(size=(0.1, 0.1, 0.1)), + }, + ) + test_marker = VisualizationMarkers(config) + + test_marker.visualize(translations=torch.tensor([[0.0, 0.0, 0.0], [0.2, 0.0, 0.0]], device=sim.device)) + + instancer = UsdGeom.PointInstancer(sim_utils.get_current_stage().GetPrimAtPath(test_marker.prim_path)) + assert list(instancer.GetProtoIndicesAttr().Get()) == [0, 0] + + def test_usd_marker(sim): """Test with marker from a USD.""" # create a marker @@ -253,6 +273,7 @@ class _FakeViewer: def __init__(self): self.calls = [] + self.show_contacts = False def is_paused(self): return False @@ -263,6 +284,9 @@ def begin_frame(self, sim_time): def log_state(self, state): self.calls.append(("log_state", state)) + def log_arrows(self, name, starts, ends, colors): + pass + def end_frame(self): self.calls.append(("end_frame",)) @@ -276,6 +300,10 @@ def get_state(scene_data_provider=None): def get_num_envs() -> int: return 4 + @staticmethod + def get_contacts(): + return None + def _fake_render_markers(viewer, visible_env_ids, num_envs): marker_calls.append((viewer, visible_env_ids, num_envs)) diff --git a/source/isaaclab/test/visualization/check_scene_xr_visualization.py b/source/isaaclab/test/xr_visualization/check_scene_xr_visualization.py similarity index 98% rename from source/isaaclab/test/visualization/check_scene_xr_visualization.py rename to source/isaaclab/test/xr_visualization/check_scene_xr_visualization.py index 0ecded8048a8..c8d25c51f140 100644 --- a/source/isaaclab/test/visualization/check_scene_xr_visualization.py +++ b/source/isaaclab/test/xr_visualization/check_scene_xr_visualization.py @@ -9,7 +9,7 @@ .. code-block:: bash # Usage - ./isaaclab.sh -p source/isaaclab/test/visualization/check_scene_visualization.py + ./isaaclab.sh -p source/isaaclab/test/xr_visualization/check_scene_xr_visualization.py """ diff --git a/source/isaaclab_newton/changelog.d/expose-newton-contact-buffer.rst b/source/isaaclab_newton/changelog.d/expose-newton-contact-buffer.rst new file mode 100644 index 000000000000..178e170378b7 --- /dev/null +++ b/source/isaaclab_newton/changelog.d/expose-newton-contact-buffer.rst @@ -0,0 +1,5 @@ +Added +^^^^^ + +* Added a ``NewtonManager.get_contacts()`` accessor so visualizers can render + Newton contact buffers without reaching into manager internals. diff --git a/source/isaaclab_newton/isaaclab_newton/physics/newton_manager.py b/source/isaaclab_newton/isaaclab_newton/physics/newton_manager.py index d5991ed395ab..41aea04368f4 100644 --- a/source/isaaclab_newton/isaaclab_newton/physics/newton_manager.py +++ b/source/isaaclab_newton/isaaclab_newton/physics/newton_manager.py @@ -1584,6 +1584,11 @@ def get_state(cls, scene_data_provider: SceneDataProvider | None = None) -> Stat cls.update_visualization_state(scene_data_provider) return cls.get_state_0() + @classmethod + def get_contacts(cls) -> Contacts | None: + """Get the current Newton contact buffer, if the active solver exposes one.""" + return cls._contacts + @classmethod def get_num_envs(cls) -> int: return cls._num_envs diff --git a/source/isaaclab_physx/changelog.d/reduce-ant-joint-wrench-warnings.rst b/source/isaaclab_physx/changelog.d/reduce-ant-joint-wrench-warnings.rst new file mode 100644 index 000000000000..ceab9cc95ca3 --- /dev/null +++ b/source/isaaclab_physx/changelog.d/reduce-ant-joint-wrench-warnings.rst @@ -0,0 +1,6 @@ +Fixed +^^^^^ + +* Fixed excessive PhysX tensor warnings from Ant tasks with ``JointWrenchSensor`` + by sourcing scene-data transforms for articulation links from Isaac Lab + articulation views instead of a global PhysX rigid-body view. diff --git a/source/isaaclab_physx/isaaclab_physx/physics/physx_manager.py b/source/isaaclab_physx/isaaclab_physx/physics/physx_manager.py index df30c9e268a7..36701a248f5f 100644 --- a/source/isaaclab_physx/isaaclab_physx/physics/physx_manager.py +++ b/source/isaaclab_physx/isaaclab_physx/physics/physx_manager.py @@ -173,9 +173,10 @@ def simulation_view(self, simulation_view: omni.physics.tensors.SimulationView | def get_rigid_body_view(self) -> omni.physics.tensors.RigidBodyView | None: """Lazily create a rigid body view covering all rigid bodies in the scene. - Discovers rigid body prims by traversing the USD stage and converts - per-environment paths (``/World/envs/env_N/...``) into wildcard - patterns so a single PhysX view covers every environment instance. + Discovers exact rigid body prims by traversing USD, then compacts cloned + environment paths into wildcard patterns. If a rigid body name is also + used by a non-rigid prim, the exact path is kept to avoid PhysX resolving + the wildcard to the non-rigid prim. """ if self._rigid_body_view is not None: return self._rigid_body_view @@ -187,15 +188,29 @@ def get_rigid_body_view(self) -> omni.physics.tensors.RigidBodyView | None: if stage is None: return None - patterns: set[str] = set() + rigid_body_paths: list[str] = [] + non_rigid_body_names: set[str] = set() for prim in stage.Traverse(): + prim_path = prim.GetPath().pathString if prim.HasAPI(UsdPhysics.RigidBodyAPI): - patterns.add(re.sub(r"/World/envs/env_\d+", "/World/envs/env_*", prim.GetPath().pathString)) + rigid_body_paths.append(prim_path) + elif re.search(r"/World/envs/env_\d+/", prim_path): + non_rigid_body_names.add(prim_path.rsplit("/", 1)[-1]) - if not patterns: + patterns: set[str] = set() + exact_paths: list[str] = [] + for prim_path in rigid_body_paths: + body_name = prim_path.rsplit("/", 1)[-1] + if body_name in non_rigid_body_names: + exact_paths.append(prim_path) + else: + patterns.add(re.sub(r"/World/envs/env_\d+", "/World/envs/env_*", prim_path)) + + body_paths = [*sorted(patterns), *exact_paths] + if not body_paths: return None - self._rigid_body_view = self._simulation_view.create_rigid_body_view(list(patterns)) + self._rigid_body_view = self._simulation_view.create_rigid_body_view(body_paths) return self._rigid_body_view @property diff --git a/source/isaaclab_visualizers/changelog.d/fix-newton-contact-visualization.rst b/source/isaaclab_visualizers/changelog.d/fix-newton-contact-visualization.rst new file mode 100644 index 000000000000..255a3a270ceb --- /dev/null +++ b/source/isaaclab_visualizers/changelog.d/fix-newton-contact-visualization.rst @@ -0,0 +1,6 @@ +Fixed +^^^^^ + +* Fixed Newton visualizer contact rendering by logging Newton contact buffers + when available and falling back to scene contact sensors for PhysX-backed + scenes. diff --git a/source/isaaclab_visualizers/changelog.d/fix-newton-hud-imgui-dependency.rst b/source/isaaclab_visualizers/changelog.d/fix-newton-hud-imgui-dependency.rst new file mode 100644 index 000000000000..bcbbbafcebcc --- /dev/null +++ b/source/isaaclab_visualizers/changelog.d/fix-newton-hud-imgui-dependency.rst @@ -0,0 +1,15 @@ +Fixed +^^^^^ + +* Fixed Newton visualizer HUD dependency checks by requiring + ``typing-extensions>=4.15.0`` for the Newton visualizer extra and failing + integration tests when Newton reports that ``imgui_bundle`` could not be + imported. Removed the legacy ``setup.py`` for ``isaaclab_visualizers`` now that + ``pyproject.toml`` carries the package metadata. + +* Fixed Rerun and Viser visualizers rendering Newton infinite ground planes too + small by expanding non-positive plane extents to the same large finite size + used by Newton GL. + +* Fixed Viser visualizer ground-grid flickering by reusing unchanged plane grid + line segments instead of removing and re-adding them every frame. diff --git a/source/isaaclab_visualizers/isaaclab_visualizers/newton/newton_visualizer.py b/source/isaaclab_visualizers/isaaclab_visualizers/newton/newton_visualizer.py index 7c4f82234d0f..3dbecdec794f 100644 --- a/source/isaaclab_visualizers/isaaclab_visualizers/newton/newton_visualizer.py +++ b/source/isaaclab_visualizers/isaaclab_visualizers/newton/newton_visualizer.py @@ -13,6 +13,7 @@ import sys from typing import TYPE_CHECKING +import torch import warp as wp from newton.viewer import ViewerGL from pyglet.math import Vec3 as PygletVec3 @@ -37,6 +38,15 @@ logger = logging.getLogger(__name__) +CONTACT_ARROW_PATH = "/contacts" +"""Viewer path used for native and synthesized contact arrows.""" + +CONTACT_ARROW_COLOR = (0.0, 1.0, 0.0) +"""Color used by Newton's native contact visualization.""" + +CONTACT_ARROW_LENGTH = 0.1 +"""Length of synthesized contact arrows in meters.""" + if TYPE_CHECKING: from isaaclab.scene_data import SceneDataProvider @@ -488,6 +498,11 @@ def step(self, dt: float) -> None: if hasattr(body_q, "shape") and body_q.shape[0] == 0: return self._viewer.log_state(self._state) + contacts = NewtonManager.get_contacts() + if contacts is not None: + self._viewer.log_contacts(contacts, self._state) + else: + self._log_scene_contact_sensor_arrows(num_envs) if self.cfg.enable_markers: render_newton_visualization_markers( self._viewer, self._resolved_visible_env_ids, num_envs=num_envs @@ -511,6 +526,115 @@ def close(self) -> None: self._camera_sensor = None self._is_closed = True + def _log_scene_contact_sensor_arrows(self, num_envs: int) -> None: + """Render contact sensor data as Newton-style arrows when native contacts are unavailable.""" + if self._viewer is None: + return + if not self._viewer.show_contacts: + self._viewer.log_arrows(CONTACT_ARROW_PATH, None, None, None) + return + contact_sensors = ( + self._scene_data_provider.get_contact_sensors() if self._scene_data_provider is not None else {} + ) + if not contact_sensors: + self._viewer.log_arrows(CONTACT_ARROW_PATH, None, None, None) + return + + starts: list[torch.Tensor] = [] + ends: list[torch.Tensor] = [] + for sensor in contact_sensors.values(): + sensor_starts, sensor_ends = self._contact_sensor_arrow_tensors(sensor, num_envs) + if sensor_starts is not None and sensor_ends is not None: + starts.append(sensor_starts) + ends.append(sensor_ends) + + if not starts: + self._viewer.log_arrows(CONTACT_ARROW_PATH, None, None, None) + return + + starts_t = torch.cat(starts, dim=0).detach().to(dtype=torch.float32, device="cpu").contiguous() + ends_t = torch.cat(ends, dim=0).detach().to(dtype=torch.float32, device="cpu").contiguous() + self._viewer.log_arrows( + CONTACT_ARROW_PATH, + wp.array(starts_t.numpy(), dtype=wp.vec3, device=self._viewer.device), + wp.array(ends_t.numpy(), dtype=wp.vec3, device=self._viewer.device), + CONTACT_ARROW_COLOR, + ) + + def _contact_sensor_arrow_tensors(self, sensor, num_envs: int) -> tuple[torch.Tensor | None, torch.Tensor | None]: + """Build Newton-style arrow starts/ends from an Isaac Lab contact sensor.""" + try: + data = sensor.data + net_forces_proxy = data.net_forces_w + net_forces = net_forces_proxy.torch if net_forces_proxy is not None else None + except (AttributeError, NotImplementedError, RuntimeError): + return None, None + + if net_forces is None or net_forces.numel() == 0: + return None, None + net_forces = self._filter_visible_env_tensor(net_forces, num_envs) + + force_threshold = getattr(getattr(sensor, "cfg", None), "force_threshold", None) + if force_threshold is None: + force_threshold = 0.0 + + try: + contact_pos = getattr(data, "contact_pos_w", None) + force_matrix = getattr(data, "force_matrix_w", None) + except NotImplementedError: + contact_pos = None + force_matrix = None + if contact_pos is not None and force_matrix is not None: + contact_pos_t = self._filter_visible_env_tensor(contact_pos.torch, num_envs) + force_matrix_t = self._filter_visible_env_tensor(force_matrix.torch, num_envs) + if contact_pos_t.numel() != 0 and force_matrix_t.numel() != 0: + force_norm = torch.linalg.norm(force_matrix_t, dim=-1) + finite_pos = torch.isfinite(contact_pos_t).all(dim=-1) + active = (force_norm > force_threshold) & finite_pos + if torch.any(active): + starts = contact_pos_t[active] + directions = torch.nn.functional.normalize(force_matrix_t[active], dim=-1) + return starts, starts + directions * CONTACT_ARROW_LENGTH + + origins = self._contact_sensor_origin_positions(sensor, data, net_forces) + if origins is None: + return None, None + origins = self._filter_visible_env_tensor(origins, num_envs) + + force_norm = torch.linalg.norm(net_forces, dim=-1) + active = force_norm > force_threshold + if not torch.any(active): + return None, None + + starts = origins[active] + directions = torch.nn.functional.normalize(net_forces[active], dim=-1) + return starts, starts + directions * CONTACT_ARROW_LENGTH + + def _contact_sensor_origin_positions(self, sensor, data, net_forces: torch.Tensor) -> torch.Tensor | None: + """Return per-sensor origins for contact arrow starts.""" + try: + pos_w = getattr(data, "pos_w", None) + except NotImplementedError: + pos_w = None + if pos_w is not None: + return pos_w.torch + + body_physx_view = getattr(sensor, "body_physx_view", None) + if body_physx_view is None: + return None + try: + pose = body_physx_view.get_transforms() + except RuntimeError: + return None + return wp.to_torch(pose).view(*net_forces.shape[:-1], 7)[..., :3] + + def _filter_visible_env_tensor(self, tensor: torch.Tensor, num_envs: int) -> torch.Tensor: + """Apply Newton visualizer visible-world filtering to a sensor tensor.""" + if self._resolved_visible_env_ids is None or tensor.ndim == 0 or tensor.shape[0] != num_envs: + return tensor + ids = torch.as_tensor(self._resolved_visible_env_ids, dtype=torch.long, device=tensor.device) + return tensor.index_select(0, ids) + def is_running(self) -> bool: """Return whether the visualizer should continue stepping. diff --git a/source/isaaclab_visualizers/isaaclab_visualizers/newton_adapter.py b/source/isaaclab_visualizers/isaaclab_visualizers/newton_adapter.py index 6bc3d5a2b4f1..8a1554a253fd 100644 --- a/source/isaaclab_visualizers/isaaclab_visualizers/newton_adapter.py +++ b/source/isaaclab_visualizers/isaaclab_visualizers/newton_adapter.py @@ -7,6 +7,52 @@ from __future__ import annotations +from collections.abc import Callable +from typing import Any + +VISUALIZER_INFINITE_PLANE_SIZE = 1000.0 +"""Finite render size used for Newton planes encoded as infinite.""" + + +def expand_infinite_plane_scale( + geo_scale: tuple[float, ...], plane_size: float = VISUALIZER_INFINITE_PLANE_SIZE +) -> tuple[float, ...]: + """Return a finite visual scale for Newton planes encoded with non-positive extents. + + Newton uses non-positive X/Y plane scale values to represent an effectively + infinite plane. Newton GL renders those with a large finite mesh; web viewers + also need a finite size, otherwise their world-extents heuristic can shrink + the floor to just the actor bounds. + """ + scale = tuple(float(value) for value in geo_scale) + width = scale[0] if len(scale) > 0 else 0.0 + length = scale[1] if len(scale) > 1 else 0.0 + if width > 0.0 and length > 0.0: + return scale + tail = scale[2:] if len(scale) > 2 else () + return ( + width if width > 0.0 else float(plane_size), + length if length > 0.0 else float(plane_size), + *tail, + ) + + +def log_geo_with_expanded_plane_scale( + super_log_geo: Callable[..., Any], + plane_geo_type: int, + name: str, + geo_type: int, + geo_scale: tuple[float, ...], + geo_thickness: float, + geo_is_solid: bool, + geo_src=None, + hidden: bool = False, +): + """Log geometry after expanding Newton infinite-plane extents for web viewers.""" + if geo_type == plane_geo_type: + geo_scale = expand_infinite_plane_scale(geo_scale) + return super_log_geo(name, geo_type, geo_scale, geo_thickness, geo_is_solid, geo_src, hidden) + def resolve_visible_env_indices( env_ids: list[int] | None, diff --git a/source/isaaclab_visualizers/isaaclab_visualizers/rerun/rerun_visualizer.py b/source/isaaclab_visualizers/isaaclab_visualizers/rerun/rerun_visualizer.py index 8b9eea880813..631ae43c7168 100644 --- a/source/isaaclab_visualizers/isaaclab_visualizers/rerun/rerun_visualizer.py +++ b/source/isaaclab_visualizers/isaaclab_visualizers/rerun/rerun_visualizer.py @@ -16,6 +16,7 @@ from typing import TYPE_CHECKING from urllib.parse import quote +import newton import rerun as rr import rerun.blueprint as rrb from newton.viewer import ViewerRerun @@ -23,7 +24,11 @@ from isaaclab.visualizers.base_visualizer import BaseVisualizer from isaaclab_visualizers.newton.newton_visualization_markers import render_newton_visualization_markers -from isaaclab_visualizers.newton_adapter import apply_viewer_visible_worlds, resolve_visible_env_indices +from isaaclab_visualizers.newton_adapter import ( + apply_viewer_visible_worlds, + log_geo_with_expanded_plane_scale, + resolve_visible_env_indices, +) from .rerun_visualizer_cfg import RerunVisualizerCfg @@ -134,6 +139,29 @@ def _render_ui(self): if imgui.button("Pause Rendering" if not self._paused_rendering else "Resume Rendering"): self._paused_rendering = not self._paused_rendering + def log_geo( + self, + name: str, + geo_type: int, + geo_scale: tuple[float, ...], + geo_thickness: float, + geo_is_solid: bool, + geo_src=None, + hidden: bool = False, + ): + """Log geometry, preserving large render extents for infinite ground planes.""" + return log_geo_with_expanded_plane_scale( + super().log_geo, + newton.GeoType.PLANE, + name, + geo_type, + geo_scale, + geo_thickness, + geo_is_solid, + geo_src, + hidden, + ) + class RerunVisualizer(BaseVisualizer): """Rerun visualizer for Isaac Lab.""" diff --git a/source/isaaclab_visualizers/isaaclab_visualizers/viser/viser_visualizer.py b/source/isaaclab_visualizers/isaaclab_visualizers/viser/viser_visualizer.py index 5bd9a831adce..17611db34b6e 100644 --- a/source/isaaclab_visualizers/isaaclab_visualizers/viser/viser_visualizer.py +++ b/source/isaaclab_visualizers/isaaclab_visualizers/viser/viser_visualizer.py @@ -16,12 +16,18 @@ from pathlib import Path from typing import TYPE_CHECKING, Any +import newton +import numpy as np from newton.viewer import ViewerViser from isaaclab.visualizers.base_visualizer import BaseVisualizer from isaaclab_visualizers.newton.newton_visualization_markers import render_newton_visualization_markers -from isaaclab_visualizers.newton_adapter import apply_viewer_visible_worlds, resolve_visible_env_indices +from isaaclab_visualizers.newton_adapter import ( + apply_viewer_visible_worlds, + log_geo_with_expanded_plane_scale, + resolve_visible_env_indices, +) from .viser_visualizer_cfg import ViserVisualizerCfg @@ -108,12 +114,85 @@ def _viser_server_with_bind_address(*args, **kwargs): record_to_viser=record_to_viser, ) self._metadata = metadata or {} + self._isaaclab_plane_grid_cache: dict[str, tuple] = {} @property def share_url(self) -> str | None: """Return the public share URL created by Viser, if any.""" return self._share_url + def clear_model(self) -> None: + """Clear cached static plane-grid signatures with the viewer model.""" + cache = getattr(self, "_isaaclab_plane_grid_cache", None) + if cache is not None: + cache.clear() + return super().clear_model() + + @staticmethod + def _array_signature(array) -> tuple[tuple[int, ...], bytes] | None: + """Return a stable signature for small transform/scale arrays.""" + if array is None: + return None + array_np = np.ascontiguousarray(np.asarray(array, dtype=np.float32)) + return tuple(int(dim) for dim in array_np.shape), array_np.tobytes() + + def _log_plane_instances( + self, + name: str, + plane_info: dict[str, float | bool], + xforms, + scales, + hidden: bool = False, + ) -> None: + """Avoid removing/re-adding unchanged Viser plane grids every frame.""" + cache = getattr(self, "_isaaclab_plane_grid_cache", None) + if hidden or xforms is None: + if cache is not None: + cache.pop(name, None) + return super()._log_plane_instances(name, plane_info, xforms, scales, hidden=hidden) + + xforms_np = self._to_numpy(xforms) + if xforms_np is None or len(xforms_np) == 0: + if cache is not None: + cache.pop(name, None) + return super()._log_plane_instances(name, plane_info, xforms, scales, hidden=hidden) + + scales_np = self._to_numpy(scales) if scales is not None else None + signature = ( + float(plane_info["width"]), + float(plane_info["length"]), + self._array_signature(xforms_np), + self._array_signature(scales_np), + ) + if cache is not None and cache.get(name) == signature and name in self._plane_handles: + return None + if cache is not None: + cache[name] = signature + return super()._log_plane_instances(name, plane_info, xforms, scales, hidden=hidden) + + def log_geo( + self, + name: str, + geo_type: int, + geo_scale: tuple[float, ...], + geo_thickness: float, + geo_is_solid: bool, + geo_src=None, + hidden: bool = False, + ): + """Log geometry, preserving large render extents for infinite ground planes.""" + return log_geo_with_expanded_plane_scale( + super().log_geo, + newton.GeoType.PLANE, + name, + geo_type, + geo_scale, + geo_thickness, + geo_is_solid, + geo_src, + hidden, + ) + class ViserVisualizer(BaseVisualizer): """Viser web-based visualizer backed by Newton's ViewerViser.""" diff --git a/source/isaaclab_visualizers/pyproject.toml b/source/isaaclab_visualizers/pyproject.toml index 77396b839fad..e39c6f7f943c 100644 --- a/source/isaaclab_visualizers/pyproject.toml +++ b/source/isaaclab_visualizers/pyproject.toml @@ -32,10 +32,12 @@ newton = [ "newton[sim] @ git+https://github.com/newton-physics/newton.git@v1.2.1rc2", "PyOpenGL-accelerate", "imgui-bundle>=1.92.5", + "typing-extensions>=4.15.0", ] rerun = [ "newton[sim] @ git+https://github.com/newton-physics/newton.git@v1.2.1rc2", "rerun-sdk>=0.29.0", + "pyarrow==22.0.0", ] viser = [ "newton[sim] @ git+https://github.com/newton-physics/newton.git@v1.2.1rc2", @@ -45,7 +47,10 @@ all = [ "imgui-bundle>=1.92.5", "newton[sim] @ git+https://github.com/newton-physics/newton.git@v1.2.1rc2", "PyOpenGL-accelerate", + # Match rerun-sdk's supported Arrow stack and avoid resolver drift across environments. + "pyarrow==22.0.0", "rerun-sdk>=0.29.0", + "typing-extensions>=4.15.0", "viser>=1.0.16", "warp-lang", ] diff --git a/source/isaaclab_visualizers/test/test_newton_adapter.py b/source/isaaclab_visualizers/test/test_newton_adapter.py index 3c020a8d10ee..50d76902d3f0 100644 --- a/source/isaaclab_visualizers/test/test_newton_adapter.py +++ b/source/isaaclab_visualizers/test/test_newton_adapter.py @@ -3,11 +3,63 @@ # # SPDX-License-Identifier: BSD-3-Clause -"""Unit tests for viewer env resolution helpers.""" +"""Unit tests for Newton viewer adapter helpers.""" from __future__ import annotations -from isaaclab_visualizers.newton_adapter import apply_viewer_visible_worlds, resolve_visible_env_indices +from types import SimpleNamespace + +import torch +from isaaclab_visualizers.newton import NewtonVisualizer, NewtonVisualizerCfg +from isaaclab_visualizers.newton_adapter import ( + VISUALIZER_INFINITE_PLANE_SIZE, + apply_viewer_visible_worlds, + expand_infinite_plane_scale, + log_geo_with_expanded_plane_scale, + resolve_visible_env_indices, +) + + +def test_expand_infinite_plane_scale_expands_non_positive_extents(): + assert expand_infinite_plane_scale((0.0, 0.0, 1.0, 0.0)) == ( + VISUALIZER_INFINITE_PLANE_SIZE, + VISUALIZER_INFINITE_PLANE_SIZE, + 1.0, + 0.0, + ) + assert expand_infinite_plane_scale((-1.0, 25.0)) == ( + VISUALIZER_INFINITE_PLANE_SIZE, + 25.0, + ) + assert expand_infinite_plane_scale((25.0, 0.0)) == ( + 25.0, + VISUALIZER_INFINITE_PLANE_SIZE, + ) + + +def test_expand_infinite_plane_scale_preserves_finite_extents(): + assert expand_infinite_plane_scale((100.0, 50.0, 1.0)) == (100.0, 50.0, 1.0) + + +def test_log_geo_with_expanded_plane_scale_delegates_with_adjusted_plane_scale(): + calls = [] + + def _log_geo(*args): + calls.append(args) + return "logged" + + assert log_geo_with_expanded_plane_scale(_log_geo, 1, "ground", 1, (0.0, 25.0), 0.0, True) == "logged" + assert calls == [("ground", 1, (VISUALIZER_INFINITE_PLANE_SIZE, 25.0), 0.0, True, None, False)] + + +def test_log_geo_with_expanded_plane_scale_preserves_non_plane_scale(): + calls = [] + + def _log_geo(*args): + calls.append(args) + + log_geo_with_expanded_plane_scale(_log_geo, 1, "box", 2, (0.0, 25.0), 0.0, True, hidden=True) + assert calls == [("box", 2, (0.0, 25.0), 0.0, True, None, True)] def test_resolve_visible_env_indices_truncates_explicit_list(): @@ -47,3 +99,127 @@ def set_visible_worlds(self, worlds): apply_viewer_visible_worlds(_V(), env_ids=None, max_visible_envs=None, num_envs=3) assert calls[-1] is None + + +class _BodyQ: + shape = (1,) + + +class _Viewer: + _update_frequency = 1 + + def __init__(self): + self.device = "cpu" + self.show_contacts = False + self.logged_state = None + self.logged_contacts = None + self.logged_arrows = None + + def is_paused(self): + return False + + def begin_frame(self, _time): + pass + + def log_state(self, state): + self.logged_state = state + + def log_contacts(self, contacts, state): + self.logged_contacts = (contacts, state) + + def log_arrows(self, name, starts, ends, colors): + self.logged_arrows = (name, starts, ends, colors) + + def end_frame(self): + pass + + +class _Proxy: + def __init__(self, tensor): + self.torch = tensor + + +class _ContactSensorData: + def __init__(self, net_forces_w, pos_w): + self.net_forces_w = _Proxy(net_forces_w) + self.pos_w = _Proxy(pos_w) + self.contact_pos_w = None + self.force_matrix_w = None + + +class _ContactSensor: + def __init__(self, net_forces_w, pos_w, force_threshold=1.0): + self.cfg = SimpleNamespace(force_threshold=force_threshold) + self.data = _ContactSensorData(net_forces_w, pos_w) + + +class _SceneDataProvider: + def __init__(self, contact_sensors=None): + self._contact_sensors = contact_sensors or {} + + def get_contact_sensors(self): + return self._contact_sensors + + +def _make_newton_visualizer(viewer, scene_data_provider=None): + visualizer = NewtonVisualizer.__new__(NewtonVisualizer) + visualizer.cfg = NewtonVisualizerCfg(enable_markers=False) + visualizer._is_initialized = True + visualizer._is_closed = False + visualizer._sim_time = 0.0 + visualizer._step_counter = 0 + visualizer._viewer = viewer + visualizer._state = None + visualizer._scene_data_provider = scene_data_provider + visualizer._resolved_visible_env_ids = None + visualizer._log_camera_sensor_image = lambda: None + return visualizer + + +def test_newton_visualizer_logs_native_contacts_when_available(monkeypatch): + from isaaclab_newton.physics import NewtonManager + + state = SimpleNamespace(body_q=_BodyQ()) + contacts = object() + viewer = _Viewer() + + monkeypatch.setattr(NewtonManager, "get_state", lambda _scene_data_provider=None: state) + monkeypatch.setattr(NewtonManager, "get_contacts", lambda: contacts) + monkeypatch.setattr(NewtonManager, "get_num_envs", lambda: 1) + + _make_newton_visualizer(viewer).step(0.1) + + assert viewer.logged_state is state + assert viewer.logged_contacts == (contacts, state) + + +def test_newton_visualizer_contact_sensor_fallback_obeys_show_contacts(monkeypatch): + from isaaclab_newton.physics import NewtonManager + + state = SimpleNamespace(body_q=_BodyQ()) + viewer = _Viewer() + sensor = _ContactSensor( + net_forces_w=torch.tensor([[[0.0, 0.0, 2.0], [0.0, 0.0, 0.5]]], dtype=torch.float32), + pos_w=torch.tensor([[[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]]], dtype=torch.float32), + force_threshold=1.0, + ) + scene_data_provider = _SceneDataProvider({"contact_forces": sensor}) + + monkeypatch.setattr(NewtonManager, "get_state", lambda _scene_data_provider=None: state) + monkeypatch.setattr(NewtonManager, "get_contacts", lambda: None) + monkeypatch.setattr(NewtonManager, "get_num_envs", lambda: 1) + + visualizer = _make_newton_visualizer(viewer, scene_data_provider) + visualizer.step(0.1) + assert viewer.logged_arrows == ("/contacts", None, None, None) + + viewer.show_contacts = True + visualizer.step(0.1) + + name, starts, ends, colors = viewer.logged_arrows + assert name == "/contacts" + assert len(starts) == 1 + assert len(ends) == 1 + assert colors == (0.0, 1.0, 0.0) + assert torch.allclose(torch.tensor(starts.numpy()[0]), torch.tensor([1.0, 2.0, 3.0])) + assert torch.allclose(torch.tensor(ends.numpy()[0]), torch.tensor([1.0, 2.0, 3.1])) diff --git a/source/isaaclab_visualizers/test/test_visualizer_integration_newton.py b/source/isaaclab_visualizers/test/test_visualizer_integration_newton.py index a5567735e5cf..ea8e35fab1ae 100644 --- a/source/isaaclab_visualizers/test/test_visualizer_integration_newton.py +++ b/source/isaaclab_visualizers/test/test_visualizer_integration_newton.py @@ -28,9 +28,12 @@ pytestmark = [pytest.mark.isaacsim_ci, pytest.mark.flaky(max_runs=2, min_passes=1)] -def test_cartpole_env_visualizers_motion_with_play_pause_newton(caplog: pytest.LogCaptureFixture) -> None: +def test_cartpole_env_visualizers_motion_with_play_pause_newton( + caplog: pytest.LogCaptureFixture, capsys: pytest.CaptureFixture[str] +) -> None: """Cartpole env + all non-tiled visualizers on Newton MJWarp.""" run_cartpole_env_visualizers_motion_with_play_pause("newton", caplog) + _viz_utils.assert_no_newton_imgui_bundle_warning(capsys, caplog) if __name__ == "__main__": diff --git a/source/isaaclab_visualizers/test/test_visualizer_integration_physx.py b/source/isaaclab_visualizers/test/test_visualizer_integration_physx.py index dac89b283b00..c8b909ab532e 100644 --- a/source/isaaclab_visualizers/test/test_visualizer_integration_physx.py +++ b/source/isaaclab_visualizers/test/test_visualizer_integration_physx.py @@ -28,9 +28,12 @@ pytestmark = [pytest.mark.isaacsim_ci, pytest.mark.flaky(max_runs=2, min_passes=1)] -def test_cartpole_env_visualizers_motion_with_play_pause_physx(caplog: pytest.LogCaptureFixture) -> None: +def test_cartpole_env_visualizers_motion_with_play_pause_physx( + caplog: pytest.LogCaptureFixture, capsys: pytest.CaptureFixture[str] +) -> None: """Cartpole env + all non-tiled visualizers on PhysX.""" run_cartpole_env_visualizers_motion_with_play_pause("physx", caplog) + _viz_utils.assert_no_newton_imgui_bundle_warning(capsys, caplog) if __name__ == "__main__": diff --git a/source/isaaclab_visualizers/test/test_visualizer_tiled_integration_newton.py b/source/isaaclab_visualizers/test/test_visualizer_tiled_integration_newton.py index e77ee9be05b5..0baaf7d8d5f7 100644 --- a/source/isaaclab_visualizers/test/test_visualizer_tiled_integration_newton.py +++ b/source/isaaclab_visualizers/test/test_visualizer_tiled_integration_newton.py @@ -28,9 +28,12 @@ pytestmark = [pytest.mark.isaacsim_ci, pytest.mark.flaky(max_runs=2, min_passes=1)] -def test_visualizer_tiled_integration_newton(caplog: pytest.LogCaptureFixture) -> None: +def test_visualizer_tiled_integration_newton( + caplog: pytest.LogCaptureFixture, capsys: pytest.CaptureFixture[str] +) -> None: """Cartpole env + tiled Kit/Newton visualizers on Newton MJWarp.""" run_cartpole_env_visualizers_tiled_camera_motion("newton", caplog) + _viz_utils.assert_no_newton_imgui_bundle_warning(capsys, caplog) if __name__ == "__main__": diff --git a/source/isaaclab_visualizers/test/test_visualizer_tiled_integration_physx.py b/source/isaaclab_visualizers/test/test_visualizer_tiled_integration_physx.py index 5693c1131ee0..bafffe831e02 100644 --- a/source/isaaclab_visualizers/test/test_visualizer_tiled_integration_physx.py +++ b/source/isaaclab_visualizers/test/test_visualizer_tiled_integration_physx.py @@ -28,9 +28,12 @@ pytestmark = [pytest.mark.isaacsim_ci, pytest.mark.flaky(max_runs=2, min_passes=1)] -def test_visualizer_tiled_integration_physx(caplog: pytest.LogCaptureFixture) -> None: +def test_visualizer_tiled_integration_physx( + caplog: pytest.LogCaptureFixture, capsys: pytest.CaptureFixture[str] +) -> None: """Cartpole env + tiled Kit/Newton visualizers on PhysX.""" run_cartpole_env_visualizers_tiled_camera_motion("physx", caplog) + _viz_utils.assert_no_newton_imgui_bundle_warning(capsys, caplog) if __name__ == "__main__": diff --git a/source/isaaclab_visualizers/test/visualizer_integration_utils.py b/source/isaaclab_visualizers/test/visualizer_integration_utils.py index 8cc79b5ba08b..3b4bc8f3449d 100644 --- a/source/isaaclab_visualizers/test/visualizer_integration_utils.py +++ b/source/isaaclab_visualizers/test/visualizer_integration_utils.py @@ -61,6 +61,7 @@ # When True, tests also fail on WARNING-level records from visualizer-related loggers. ASSERT_VISUALIZER_WARNINGS = False +_NEWTON_IMGUI_BUNDLE_PRINT_WARNING = "Warning: imgui_bundle not found" _MAX_FRAME_CHECK_STEPS = 5 """Steps for Rerun / Viser smoke tests.""" @@ -221,6 +222,19 @@ def _assert_no_visualizer_log_issues(caplog: pytest.LogCaptureFixture, *, fail_o ) +def assert_no_newton_imgui_bundle_warning(capsys: pytest.CaptureFixture[str], caplog: pytest.LogCaptureFixture) -> None: + """Fail when Newton reports that its imgui HUD dependency is missing.""" + captured = capsys.readouterr() + captured_output = captured.out + captured.err + printed_warning = _NEWTON_IMGUI_BUNDLE_PRINT_WARNING in captured_output + logged_warnings = [record for record in caplog.records if _NEWTON_IMGUI_BUNDLE_PRINT_WARNING in record.getMessage()] + assert not printed_warning and not logged_warnings, ( + "Newton viewer reported that imgui_bundle could not be imported, which disables HUD controls. " + f"Captured output: {captured_output!r}. " + "Captured logs: " + "; ".join(f"{record.name}: {record.getMessage()}" for record in logged_warnings) + ) + + def _configure_sim_for_visualizer_test(env: CartpoleCameraEnv) -> None: """Settings used by the previous smoke tests; keep RTX sensors enabled for camera paths.""" AppLauncher.apply_rtx_determinism_settings() diff --git a/tools/wheel_builder/res/python_packages.toml b/tools/wheel_builder/res/python_packages.toml index 24867efd8561..998dc8a05de3 100644 --- a/tools/wheel_builder/res/python_packages.toml +++ b/tools/wheel_builder/res/python_packages.toml @@ -106,7 +106,7 @@ pyproject.optional-dependencies.all = [ { "rsl-rl" = ["rsl-rl-lib==5.0.1", "onnxscript>=0.5"] }, { "rsl_rl" = ["rsl-rl-lib==5.0.1", "onnxscript>=0.5"] }, # ================================================================================ - # https://github.com/isaac-sim/IsaacLab/blob/main/source/isaaclab_visualizers/setup.py + # https://github.com/isaac-sim/IsaacLab/blob/main/source/isaaclab_visualizers/pyproject.toml # ================================================================================ # Viser visualizer (opt-in: viser pulls websockets>=13.1 which collides with # isaacsim-kernel==6.0.0.0's websockets==12.0; do not include in [all]).