Skip to content

Warp-first experimental envs implements lazy export + cfg implementation split#5916

Merged
ooctipus merged 10 commits into
isaac-sim:developfrom
ooctipus:fix/warp_envs
Jun 3, 2026
Merged

Warp-first experimental envs implements lazy export + cfg implementation split#5916
ooctipus merged 10 commits into
isaac-sim:developfrom
ooctipus:fix/warp_envs

Conversation

@ooctipus
Copy link
Copy Markdown
Collaborator

@ooctipus ooctipus commented Jun 2, 2026

Summary

  • Adds Warp-first experimental environments and MDP terms under isaaclab_experimental and isaaclab_tasks_experimental (direct + manager-based cartpole/humanoid/ant/reach/locomotion), with Warp-kernel implementations of actions, observations, rewards, terminations, and events.
  • Adds lazy __init__.py + .pyi stubs (with explicit __all__) across the new/updated packages.

Test plan

  • ./isaaclab.sh -p -m pytest source/isaaclab_tasks_experimental/test
  • Train the experimental Warp cartpole/humanoid/ant configs on the Newton backend and confirm parity with the stable envs.
  • ./isaaclab.sh -p -m pytest source/isaaclab_physx/test/assets/test_newton_actuators_physx.py source/isaaclab_newton/test/assets/test_newton_actuators_newton.py
  • Verify randomize_rigid_body_scale raises on Newton and warns (deprecated) on PhysX.

@github-actions github-actions Bot added bug Something isn't working isaac-lab Related to Isaac Lab team labels Jun 2, 2026
Copy link
Copy Markdown

@isaaclab-review-bot isaaclab-review-bot Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review: Warp-first experimental envs + physics-event backend decoupling

Reviewed at: 4b5b475b

Summary

This PR refactors physics-event randomization terms into per-backend implementations (isaaclab_physx, isaaclab_newton, isaaclab_ovphysx), separates env configs from env implementations to enable lazy imports via lazy_export() + .pyi stubs, and moves heavyweight imports (pxr, carb, Articulation, etc.) behind TYPE_CHECKING guards to keep the env-cfg parse path simulator-free.

The architecture is clean and well-motivated. Below are specific findings.


Findings

1. 🔴 [Critical] Missing isaaclab.envs.mdp.physics_events module — broken imports in backend files and tests

Files:

  • source/isaaclab_newton/isaaclab_newton/envs/mdp/physics_events.py (line 19)
  • source/isaaclab_physx/isaaclab_physx/envs/mdp/physics_events.py (line 20)
  • source/isaaclab_newton/test/assets/test_newton_actuators_newton.py (line 531)
  • source/isaaclab_physx/test/assets/test_newton_actuators_physx.py (line 477)

Both backend physics_events.py files import:

from isaaclab.envs.mdp.physics_events import randomize_prop_by_op, validate_scale_range

And both test files import:

from isaaclab.envs.mdp.physics_events import randomize_actuator_gains

However, source/isaaclab/isaaclab/envs/mdp/physics_events.py does not exist in the repository tree at this commit (nor on develop). The base events.py has _randomize_prop_by_op and _validate_scale_range (private names), but no physics_events submodule.

This appears to depend on a prerequisite PR that extracts these utilities into physics_events.py. Without that dependency merged first, all new backend files will fail to import at runtime.

Action needed: Either include the physics_events.py extraction in this PR, or document/enforce the merge dependency.


2. 🔴 [High] PhysX RandomizePhysicsSceneGravity ignores the distribution parameter at runtime

File: source/isaaclab_physx/isaaclab_physx/envs/mdp/physics_events.py (line ~235)

gravity = randomize_prop_by_op(
    gravity,
    (self._dist_param_0.cpu(), self._dist_param_1.cpu()),
    None,
    slice(None),
    operation=operation,
    distribution="uniform",  # ← hard-coded, ignores `distribution` argument
)

The Newton implementation correctly resolves the distribution function at init time via self._dist_fn. The PhysX implementation hard-codes "uniform", so users configuring distribution="gaussian" or "log_uniform" will silently get uniform sampling.

Suggested fix:

    distribution=distribution,

3. 🟡 [Medium] Newton RandomizeRigidBodyMaterial ignores runtime call parameters

File: source/isaaclab_newton/isaaclab_newton/envs/mdp/physics_events.py (lines 75–90 in __call__)

The __call__ signature accepts static_friction_range, dynamic_friction_range, and restitution_range as arguments, but the implementation samples from self._static_friction_range and self._restitution_range (cached at __init__ time):

friction_range = torch.tensor(self._static_friction_range, device=device)
restitution_range_t = torch.tensor(self._restitution_range, device=device)

Unlike PhysX (which pre-samples into fixed buckets due to the 64K material limit), Newton has no such constraint. If the event manager ever passes updated ranges at call time (e.g., curriculum-driven), Newton will silently ignore them.

Suggested fix: Use the call-time static_friction_range / restitution_range parameters for sampling. Reserve self._*_range only as a fallback when call-time values are not provided.


4. 🟡 [Medium] Top-level pxr / sim_utils imports in stable modules may break pre-SimulationApp workflows

Files:

  • source/isaaclab/isaaclab/envs/utils/camera_view.py (line 17: from pxr import UsdGeom)
  • source/isaaclab/isaaclab/scene_data/scene_data_provider.py (line 16: from pxr import UsdGeom)
  • source/isaaclab/isaaclab/sim/schemas/schemas_actuators.py (line 27: from pxr import Sdf, Usd)
  • source/isaaclab/isaaclab/terrains/utils.py (line 14: from pxr import UsdGeom)

These files previously deferred pxr and sim_utils imports to prevent Isaac Sim from launching at module import time. The original code had explicit comments:

# need to import these here to prevent isaacsim launching when importing this module

The stated goal of this PR is to keep the env-cfg parse path clean, but these utility modules are not part of the env-cfg parse path. If any downstream code imports isaaclab.terrains.utils or isaaclab.sim.schemas.schemas_actuators before SimulationApp is launched, it will now crash.

Suggested fix: Verify that these modules are never imported prior to SimulationApp initialization. If there's any risk, keep the deferred import pattern — it was explicitly added to prevent this class of bug.


5. 🟡 [Medium] {DIR} string-based class_type convention undocumented

File: source/isaaclab_experimental/isaaclab_experimental/envs/mdp/actions/actions_cfg.py (lines 47, 65)

class_type: type[JointPositionAction] | str = "{DIR}.joint_actions:JointPositionAction"

This is a novel pattern for lazy class resolution. Contributors unfamiliar with the mechanism will not understand:

  • What {DIR} resolves to at runtime
  • When and how the string is converted to an actual class reference
  • Whether this is an officially supported pattern or experimental

Suggested fix: Add a brief docstring or comment explaining the {DIR} convention (e.g., "Resolved at runtime by the experimental manager; {DIR} is replaced with the directory of this config file").


6. 🟢 [Low] CI "Check changelog fragments" failing — missing fragments for backend packages

The PR adds new envs/mdp/physics_events.py modules to isaaclab_newton, isaaclab_physx, and isaaclab_ovphysx, but only provides changelog fragments for isaaclab_experimental and a .skip file for isaaclab_tasks. The failing CI check likely expects fragments for the backend packages as well.


7. 🟢 [Low] Newton RandomizeRigidBodyCom stability warning could be a runtime log

File: source/isaaclab_newton/isaaclab_newton/envs/mdp/physics_events.py (class docstring, ~line 165)

The docstring notes that CoM randomization may cause simulation instability on Newton because notify_model_changed(BODY_INERTIAL_PROPERTIES) doesn't fully recompute the mass matrix. This is valuable information, but users who don't read source docstrings will miss it.

Suggestion: Emit a logger.warning() on first call to surface this caveat at runtime.


Overall Assessment

The core architecture — backend-dispatched physics events, lazy exports, and import deferral for env configs — is sound and addresses a real pain point (importing env configs before SimulationApp).

Key blockers:

  1. The missing isaaclab.envs.mdp.physics_events module makes the new backend implementations non-functional. This is either a missing file in this PR or an undocumented dependency on another PR.
  2. The PhysX gravity distribution bug introduces a silent behavioral difference from configuration intent.

Non-blocking concerns:
3. Newton material randomizer ignoring runtime params (medium risk for curriculum workflows).
4. Elevated pxr imports in utility modules (risk depends on import graphs).

CI changelog failure should be resolved before merge.


Update (12ede03): Reviewed incremental changes from 4b5b47512ede035.

New commits complete the lazy-import refactoring for the experimental packages:

  • ✅ Env configs extracted to dedicated *_cfg.py modules (ant, cartpole, humanoid) — clean separation of config from runtime env class
  • ✅ All experimental __init__.py files converted to lazy_export() pattern
  • ✅ MDP term leaves (rewards, observations, terminations, events, curriculums) correctly guard Articulation, ContactSensor, TerrainImporter under TYPE_CHECKING
  • env_cfg_entry_point strings updated to reference the new cfg modules
  • ✅ Changelog fragment added

No new issues found in this increment. Previous findings (1–7) remain as-is — they apply to code not touched in this push.

@ooctipus ooctipus changed the title Warp-first experimental envs + physics-event backend decoupling Warp-first experimental envs implements lazy export + cfg implementation skip Jun 2, 2026
@ooctipus ooctipus changed the title Warp-first experimental envs implements lazy export + cfg implementation skip Warp-first experimental envs implements lazy export + cfg implementation split Jun 2, 2026
@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented Jun 2, 2026

Greptile Summary

This PR decouples physics-engine-specific event terms from the shared isaaclab.envs.mdp layer by extracting them into a new physics_events.py module that dispatches at runtime to per-backend implementations (isaaclab_physx, isaaclab_newton, isaaclab_ovphysx). It also adds Warp-first experimental environments (Cartpole, Humanoid, Ant, Reach, Locomotion) in isaaclab_tasks_experimental, and splits env cfg classes into dedicated *_cfg.py files to keep the cfg parse path free of heavy pxr/Isaac Sim imports.

  • Backend dispatch layer (physics_events.py): all physics-property randomizers resolve their active backend via SimulationContext.instance().physics_manager.__name__ and delegate to isaaclab_{backend}.envs.mdp; randomize_rigid_body_scale is deprecated (PhysX-only) and raises NotImplementedError on Newton.
  • Experimental envs: Direct and manager-based Warp environments added with Warp-kernel MDP terms; cfg classes split into separate *_cfg.py files and imported under TYPE_CHECKING to prevent Isaac Sim from booting at cfg-parse time.
  • Import cleanup: deferred pxr/sim_utils function-level imports in camera_view.py, scene_data_provider.py, terrains/utils.py, and schemas_actuators.py are promoted to module level (safe because none sit on the cfg parse path and all parent packages use lazy_export()).

Confidence Score: 4/5

The backend dispatch mechanism and .pyi stubs are consistent across all three backends; the primary runtime risk is the unguarded SimulationContext.instance() dereference in _resolve_backend_event, which would produce a cryptic AttributeError outside a running simulation.

The architecture of the backend dispatch layer is sound — constructor signatures in Newton, PhysX, and OVPhysX all match what _resolve_backend_event forwards, and the lazy-export mechanism ensures pxr/carb imports stay deferred until runtime. The notable issues are: _resolve_backend_event does not guard against SimulationContext.instance() returning None; randomize_rigid_body_scale unconditionally emits DeprecationWarning even on Newton where the function is completely unsupported; and Newton's RandomizeRigidBodyColliderOffsets samples random values across all environments before discarding non-target rows.

source/isaaclab/isaaclab/envs/mdp/physics_events.py (the _resolve_backend_event None-dereference and the Newton deprecation-warning ordering) and source/isaaclab_newton/isaaclab_newton/envs/mdp/physics_events.py (over-broad env sampling in RandomizeRigidBodyColliderOffsets).

Important Files Changed

Filename Overview
source/isaaclab/isaaclab/envs/mdp/physics_events.py New module centralizing physics-property event terms; delegates to backend-specific implementations via _resolve_backend_event. Minor issues: SimulationContext.instance() can return None causing a cryptic AttributeError, and randomize_rigid_body_scale fires DeprecationWarning on Newton before raising NotImplementedError.
source/isaaclab/isaaclab/envs/mdp/events.py ~1665 lines of physics-property randomizers removed and relocated to physics_events.py; non-physics event terms (resets, visual randomization, force/velocity push) retained. Clean extraction with no functional changes to remaining terms.
source/isaaclab_physx/isaaclab_physx/envs/mdp/physics_events.py New PhysX backend implementing all 8 randomizer classes. Module-level from pxr import … and import carb are fine since the module is loaded lazily via lazy_export(). All constructor signatures match the base-module dispatch calls.
source/isaaclab_newton/isaaclab_newton/envs/mdp/physics_events.py New Newton backend implementing all 8 randomizer classes. RandomizeRigidBodyColliderOffsets.__call__ passes dim_0_ids=None to randomize_prop_by_op, over-computing random samples for all environments rather than just env_ids. Functionally correct but wasteful on partial resets.
source/isaaclab_ovphysx/isaaclab_ovphysx/envs/mdp/physics_events.py OVPhysX backend provides only a no-op RandomizeRigidBodyMaterial (with a clear warning); all other terms fall back to PhysX via ovphysx_uses_physx=True. Minimal and correct.
source/isaaclab/isaaclab/envs/mdp/init.pyi Updated stub correctly splits exports between .events (resets/visual) and .physics_events (physics randomizers). All exported symbols exist in their respective source modules.
source/isaaclab_tasks_experimental/isaaclab_tasks_experimental/direct/cartpole/cartpole_warp_env_cfg.py New file splitting the Cartpole Warp config out of the env module; env module now uses TYPE_CHECKING-guarded import. Keeps pxr/Newton imports out of the cfg parse path.
source/isaaclab/isaaclab/envs/utils/camera_view.py Moves from pxr import UsdGeom and from isaaclab.sim.views import FrameView from inside prim_world_positions() to module level, removing the previous guard comment. Safe because camera_view.py is not on the cfg parse path.
source/isaaclab_tasks/test/core/test_env_cfg_no_forbidden_imports.py Extends the forbidden-imports test to cover isaaclab_tasks_experimental task registry, ensuring experimental configs are also checked for pxr/sim boot imports.

Sequence Diagram

sequenceDiagram
    participant EC as EventManager
    participant PE as isaaclab.envs.mdp.physics_events
    participant RBE as _resolve_backend_event()
    participant SC as SimulationContext
    participant BACK as isaaclab_{backend}.envs.mdp

    EC->>PE: randomize_rigid_body_material.__init__(cfg, env)
    PE->>RBE: _resolve_backend_event("RandomizeRigidBodyMaterial", cfg, env, asset, asset_cfg)
    RBE->>SC: SimulationContext.instance().physics_manager.__name__
    SC-->>RBE: "PhysxManager" / "NewtonManager" / "OVPhysxManager"
    RBE->>BACK: "importlib.import_module("isaaclab_{backend}.envs.mdp")"
    BACK-->>RBE: lazy module proxy
    RBE->>BACK: "getattr(module, "RandomizeRigidBodyMaterial")(*args)"
    BACK-->>RBE: impl instance
    RBE-->>PE: self._impl
    EC->>PE: randomize_rigid_body_material.__call__(env, env_ids, ...)
    PE->>BACK: self._impl(env, env_ids, ...)
    BACK-->>EC: (done)
Loading

Comments Outside Diff (3)

  1. source/isaaclab/isaaclab/envs/mdp/physics_events.py, line 55-57 (link)

    P2 None-dereference when SimulationContext has no instance

    SimulationContext.instance() returns None when no simulation context has been created. In that case None.physics_manager raises AttributeError with a cryptic message. This path is hit in any unit test or CLI tool that calls a physics-event term outside a running environment. A defensive check (or at least a guard that raises a RuntimeError with a clear message) would make failures easier to diagnose.

  2. source/isaaclab/isaaclab/envs/mdp/physics_events.py, line 115-123 (link)

    P2 Deprecation warning fires on Newton before NotImplementedError

    randomize_rigid_body_scale emits DeprecationWarning unconditionally before dispatching to the backend. On Newton, the user then immediately gets a NotImplementedError from RandomizeRigidBodyScale.__call__. The warning message ("deprecated and only supported on the PhysX backend") does accurately cover the Newton case, but a user who catches or filters DeprecationWarning will silently proceed past the warning and hit the NotImplementedError without the helpful migration advice. Considering the test plan explicitly says "raises on Newton and warns (deprecated) on PhysX", it may be more consistent to guard the warning behind a backend check.

    Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

  3. source/isaaclab_newton/isaaclab_newton/envs/mdp/physics_events.py, line 154-166 (link)

    P2 Over-broad randomization in RandomizeRigidBodyColliderOffsets.__call__

    randomize_prop_by_op is called with dim_0_ids=None, which samples new random values across all environments. Then only env_ids rows are written back to default_margin and margin_view. The discarded samples for the non-env_ids environments are computed but immediately thrown away, wasting GPU work proportional to (num_envs − len(env_ids)). Every other Newton implementation in this file (e.g., RandomizeRigidBodyMaterial.__call__) passes env_ids directly as the row selector. The fix is to pass env_ids as dim_0_ids and drop the per-env_ids slice when writing back, which is the pattern used by the comparable PhysX implementation.

Reviews (1): Last reviewed commit: "fix warp environemtns." | Re-trigger Greptile

Copy link
Copy Markdown

@isaaclab-review-bot isaaclab-review-bot Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review: Lazy export + cfg implementation split for Warp-first experimental envs

Summary

This PR implements lazy imports via lazy_export() + .pyi stubs and separates env configs from env implementations to allow importing task configurations before SimulationApp is launched. It also moves heavyweight runtime type imports (Articulation, InteractiveScene, ContactSensor, etc.) behind TYPE_CHECKING guards.


Findings

1. 🟠 [Medium] Top-level pxr and sim_utils imports may break pre-SimulationApp workflows

Files:

  • source/isaaclab/isaaclab/envs/utils/camera_view.py (lines 17, 21)
  • source/isaaclab/isaaclab/scene_data/scene_data_provider.py (lines 16, 18)
  • source/isaaclab/isaaclab/sim/schemas/schemas_actuators.py (lines 27-29)
  • source/isaaclab/isaaclab/terrains/utils.py (lines 14, 17)

These files previously used deferred imports inside functions to prevent pxr and sim_utils from being loaded at module import time. The original code had explicit comments:

# need to import these here to prevent isaacsim launching when importing this module

The refactor hoists these imports to module level. While this simplifies the code, it may break downstream workflows that import these modules before SimulationApp is initialized.

Risk assessment: If any of these modules are reachable from env-cfg parse paths, the PR's stated goal of keeping env-cfg imports simulator-free would be violated.

Suggested action: Verify that isaaclab.terrains.utils, isaaclab.envs.utils.camera_view, isaaclab.scene_data.scene_data_provider, and isaaclab.sim.schemas.schemas_actuators are not transitively imported during env-cfg loading. If they are, the deferred import pattern should be preserved.


2. 🟡 [Low] Undocumented {DIR} string-based class_type convention

File: source/isaaclab_experimental/isaaclab_experimental/envs/mdp/actions/actions_cfg.py (lines 47, 65)

class_type: type[JointPositionAction] | str = "{DIR}.joint_actions:JointPositionAction"

This introduces a lazy class resolution pattern where {DIR} is presumably replaced at runtime with the directory of the config file. However:

  • There's no documentation explaining what {DIR} resolves to
  • Contributors unfamiliar with this mechanism may not understand how to use it
  • The | type union syntax (type[X] | str) is valid in Python 3.10+ but the runtime resolution mechanism is not standard

Suggested fix: Add a brief docstring or comment explaining the convention, e.g.:

# {DIR} is resolved at runtime to the directory containing this config module.
# This enables lazy class loading without eagerly importing the implementation.

3. 🟡 [Low] Missing .pyi stub files referenced by lazy_export()

Files:

  • source/isaaclab_experimental/isaaclab_experimental/envs/__init__.py
  • source/isaaclab_experimental/isaaclab_experimental/envs/mdp/__init__.py
  • source/isaaclab_experimental/isaaclab_experimental/envs/mdp/actions/__init__.py
  • source/isaaclab_experimental/isaaclab_experimental/managers/__init__.py

The module docstrings reference "lazy __init__.py + .pyi stubs (with explicit __all__)", but the diff doesn't include the corresponding .pyi stub files. The lazy_export() function relies on these stubs to determine which symbols to export.

Impact: Without the stub files, the lazy export mechanism may not expose the expected symbols, causing ImportError or AttributeError at runtime when accessing module members.

Question: Are the .pyi files included in the PR but not shown in this diff, or do they need to be added?


4. 🟢 [Nit] Inconsistent import aliasing in scene_data_provider.py

File: source/isaaclab/isaaclab/scene_data/scene_data_provider.py

The diff shows:

+import isaaclab.sim as sim_utils

But the original code used:

import isaaclab.sim as isaaclab_sim

And the usage was changed from isaaclab_sim.resolve_prim_pose(prim) to sim_utils.resolve_prim_pose(prim).

While sim_utils is the more common alias across the codebase, this creates a minor inconsistency within the file's history. Not a functional issue.


5. 🟢 [Good] Proper use of TYPE_CHECKING guards

The PR correctly moves runtime type imports behind TYPE_CHECKING blocks in:

  • events.py
  • observations.py
  • rewards.py
  • terminations.py
  • scene_entity_cfg.py
  • All task-specific MDP modules

This is the correct pattern for avoiding circular imports and keeping config modules lightweight.


6. 🟢 [Good] Config/implementation split is clean

The separation of *Cfg classes into dedicated *_cfg.py files (e.g., ant_env_warp_cfg.py, cartpole_warp_env_cfg.py, humanoid_warp_env_cfg.py) is well-executed:

  • Entry points correctly updated to point to *_cfg:*Cfg
  • Environment classes import their configs locally with TYPE_CHECKING guards where needed
  • No code duplication

Overall Assessment

The PR's core approach—lazy exports, TYPE_CHECKING guards, and config/implementation separation—is architecturally sound and addresses a real pain point (importing env configs before SimulationApp).

Items to verify before merge:

  1. Ensure the hoisted pxr imports in stable isaaclab modules don't break any pre-SimulationApp import paths
  2. Confirm the .pyi stub files exist and are properly configured

Minor improvements:

  • Document the {DIR} string-based class resolution convention

Update (c117f97): New commit adds resample_interval_on_reset feature to EventTermCfg (both stable and experimental managers). Implementation is clean—proper docstring, correct logic, good test coverage with deterministic intervals. No issues found with the new code.

Previous concerns (1–3) from the original review remain unaddressed—they concern different files not touched in this commit. No action needed on those unless the lazy-export paths are affected.


Update (5251d1c): New commit adds a changelog entry (fix-warp-env-cfg-forbidden-imports.skip) documenting the intentional import hoisting from concern #1. The entry confirms this was deliberate and states no user-facing change, implying the affected modules are not on pre-SimulationApp import paths. ✅ Concern #1 acknowledged by author. No new issues found.---

Update (b103091): ✅ Concern #3 resolved — all .pyi stub files are now included (envs/__init__.pyi, envs/mdp/__init__.pyi, envs/mdp/actions/__init__.pyi, managers/__init__.pyi, plus task-specific stubs). The stubs have proper __all__ lists and explicit imports matching the lazy export pattern.

Additional change: scene_entity_cfg.py replaces isinstance(entity, BaseArticulation) with hasattr(entity, "num_joints") duck-typing to avoid importing BaseArticulation at module level — reasonable trade-off for the lazy-import architecture.

No new issues found.


Update (c67e2b0): Major cartpole task consolidation commit — merges direct_cartpole and manager_cartpole into unified isaaclab_tasks.core.cartpole package, drops -v0 suffixes from task IDs, removes all deprecated per-variant aliases (35+ retired gym.register entries), and updates docs/tests/scripts throughout. The lazy_export() + .pyi pattern is also extended to more experimental task MDP packages (locomotion, humanoid, reach) with proper stubs. The experimental env env_cfg_entry_point registrations are corrected to point at the new *_cfg.py files. Clean, consistent refactoring — no new issues found.

Previous concerns status:

  • #1 (hoisted imports): ✅ Resolved/acknowledged
  • #2 ({DIR} convention): Still a minor nit, no change
  • #3 (.pyi stubs): ✅ Resolved

Update (ea7a37d): Release cut commit — version bumps across all packages (isaaclab 6.2.0, isaaclab_experimental 0.1.0, isaaclab_tasks 2.0.0, etc.) and changelog fragment compilation. Two substantive code additions:

  1. get_scales() now returns ProxyArray (breaking change, properly documented) across all frame view backends — consistent with existing get_world_poses/get_local_poses contract. Implementation is correct; test coverage added in frame_view_contract_utils.py.
  2. reshape_data_to_view_3d gains torch.Tensor input support on PhysX/OvPhysX/Newton rigid object collections — clean implementation with proper type dispatch and tests.
  3. Docstring correction in math.py for combine_frame_transforms/subtract_frame_transforms ("from A to B" → "from B to A").

No new issues found. Previous concerns status unchanged (#1 ✅, #2 minor nit, #3 ✅).


Update (78ef61d): Empty commit ("ci: re-trigger checks") — no code changes. No new issues.


Update (a178287): Small consistency fix — removes the eager from . import mdp from envs/__init__.py and moves it into the .pyi stub (with mdp added to __all__). This makes the mdp submodule fully lazy-exported like the other symbols, consistent with the PR's architecture. ✅ No issues found.

Update (f10c5c6): Two small follow-up commits:

  1. scene_entity_cfg.py: Replaces the duck-typing hasattr(entity, "num_joints") check (introduced in b103091) with isinstance(entity.cfg, ArticulationCfg) — a more robust type check against the config object rather than runtime attributes. This adds an explicit from isaaclab.assets import ArticulationCfg import, which is acceptable since ArticulationCfg is a lightweight config class (no simulator dependency). ✅ Good improvement over duck-typing.

  2. managers/__init__.pyi: Adds from isaaclab.managers import * re-export so the experimental managers stub also exposes all base manager symbols. This ensures IDE autocompletion and type checking correctly resolve symbols inherited from the stable isaaclab.managers package. ✅ Proper stub fix.

No new issues found. All previous concerns remain in their current state (#1 ✅, #2 minor nit, #3 ✅).


Update (860bf3b): Major infrastructure commit with two key changes:

  1. RTX Scene Partitioning — Adds renderer.scenePartitioning.enabled = true to all Kit app files and implements per-env omni:scenePartition token authoring in IsaacRtxRenderer.prepare_stage() and KitVisualizer. This enables RTX to cull geometry per env tile. Includes new regression tests in test_isaac_rtx_renderer_scene_partitioning.py. Implementation is clean — uses Sdf.ChangeBlock for efficient batch authoring.

  2. Breaking: launch_simulation relocated — The sim launcher module moved from isaaclab_tasks.utils.sim_launcher to isaaclab.app.sim_launcher. A new scan() function consolidates the single-walk config tree analysis (replacing the old multi-scan approach). All training scripts updated to import from isaaclab.app. Changelog entry documents the breaking change.

Additional changes:

  • Extended lazy-export pattern to experimental task MDP packages (cartpole, humanoid, ant, locomotion, reach) with proper .pyi stubs
  • Environment config classes split into dedicated *_cfg.py files to support pre-SimulationApp import
  • Visualizer camera now tagged with scene partition token for correct Kit viewport rendering

No new issues found. Previous concerns status unchanged (#1 ✅, #2 minor nit — {DIR} convention still undocumented, #3 ✅).


Update (f4fba26 — reviewed 2026-06-03):

New commits since the previous review (f10c5c6) introduce two additional areas of change:

1. launch_simulation / sim_launcher moved from isaaclab_tasks.utilsisaaclab.app

  • The module isaaclab_tasks/utils/sim_launcher.py is deleted and re-implemented as isaaclab/app/sim_launcher.py.
  • A new Scan dataclass and scan() function replace the previous multi-pass _scan_config() approach with a single-walk config inspection.
  • add_launcher_args, launch_simulation, make_physics_cfg, scan, and Scan are now exported from isaaclab.app.
  • All scripts (benchmarks, environments, reinforcement_learning/*/train|play*.py, demos/quadrupeds.py) and tests are updated to import from the new location.
  • This is a breaking change for downstream code importing from isaaclab_tasks.utils.sim_launcher.

2. RTX scene partitioning enabled across all Kit app files + renderer/visualizer

  • Six .kit files gain renderer.scenePartitioning.enabled = true.
  • IsaacRtxRenderer.prepare_stage() now authors per-env primvars:omni:scenePartition and camera omni:scenePartition tokens via an efficient Sdf.ChangeBlock.
  • KitVisualizer tags the viewport camera with the first visible env partition token.
  • camera_view.py creates the partition attribute on generated visualizer cameras.
  • A new dedicated test file (test_isaac_rtx_renderer_scene_partitioning.py) validates rigid-object and articulation per-env isolation.
  • Golden test images are regenerated accordingly.

3. Minor: quadrupeds.py demo refactored to use launch_simulation context manager + PhysicsCfg.

All previously raised concerns (top-level pxr/sim_utils imports, {DIR} string-based class_type resolution, lazy_export() usage) remain applicable to the existing code and are not re-addressed here. The new code looks well-structured. No new blocking issues found in this incremental diff.


Update (f4fba26): Major infrastructure commit with two key changes:

  1. RTX Scene Partitioning — Adds renderer.scenePartitioning.enabled = true to all Kit app files and implements per-env omni:scenePartition token authoring in IsaacRtxRenderer.prepare_stage() and KitVisualizer. This enables RTX to cull geometry per env tile, improving rendering isolation between environments. Includes new regression tests in test_isaac_rtx_renderer_scene_partitioning.py. Implementation is clean — uses Sdf.ChangeBlock for efficient batch authoring and handles both env-root primvars and direct camera attributes.

  2. Breaking: launch_simulation relocated — The sim launcher module moved from isaaclab_tasks.utils.sim_launcher to isaaclab.app.sim_launcher. A new scan() function consolidates the config tree analysis into a single walk (returning a Scan dataclass with all signals). All training/benchmark scripts updated to import add_launcher_args and launch_simulation from isaaclab.app instead of isaaclab_tasks.utils. Proper changelog entry documents the breaking change.

Additional changes:

  • camera_view.py: Generated visualizer cameras now receive per-env omni:scenePartition tokens
  • KitVisualizer: Viewport camera is tagged with the first visible env partition token for correct rendering with scene partitioning enabled
  • Extended lazy-export pattern to more experimental task MDP packages (cartpole, humanoid, locomotion, reach) with proper .pyi stubs
  • Test files updated to import from the new module location (isaaclab.app.sim_launcher)
  • Golden images regenerated to match per-env scene partition authoring

No new issues found. Previous concerns status unchanged (#1 ✅, #2 minor nit — {DIR} convention still undocumented, #3 ✅).


Update (c91c18b): Single-line change — adds UsdGeom to the existing from pxr import Sdf line in camera_view.py. This is consistent with the file already importing from pxr at module level (concern #1, previously acknowledged). No new issues found.

@ooctipus ooctipus force-pushed the fix/warp_envs branch 2 times, most recently from b103091 to c67e2b0 Compare June 2, 2026 10:17
Copy link
Copy Markdown
Collaborator

@hujc7 hujc7 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!
Does it mean all of #5826 can be reverted?


# -- Warp joint mask / ids for articulations
if isinstance(entity, BaseArticulation):
if hasattr(entity, "num_joints"):
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this also because of importing baseArticulation pulls extra dependencies? The only concern is that it's not very obvious when things can be done and cannot. Wondering if there's a way to generalize or referred in agents.md

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you could check against cfg, for example
if isinstance(entity.Cfg, BaseArticulation):

keeping stable task configs and the stable `isaaclab.managers` package intact.
"""

from isaaclab.managers import * # noqa: F401,F403
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This fallback is needed I think like that in mdp in .pyi?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it seems we covered all managers? but just in case ill add it back

Comment thread source/isaaclab_experimental/isaaclab_experimental/envs/__init__.py Outdated
Comment thread source/isaaclab/isaaclab/envs/utils/camera_view.py
@ooctipus
Copy link
Copy Markdown
Collaborator Author

ooctipus commented Jun 3, 2026

Thanks! Does it mean all of #5826 can be reverted?

yes that PR can be reverted

ooctipus added 8 commits June 2, 2026 19:35
The experimental Warp packages use lazy_export(), which requires a sibling
.pyi stub to resolve names. Those stubs are gitignored (**/__*) and were
never force-added, so a clean checkout (CI) had no managers/envs/mdp stubs.
Loading any Warp env config then crashed (e.g. cannot import
ObservationTermCfg / UniformVelocityCommandCfg), failing
test_config_load_does_not_import_backend_modules for all 23 Warp tasks.

Add and track the missing stubs for isaaclab_experimental (managers, envs,
envs/mdp, envs/mdp/actions) and the four task mdp packages (velocity,
cartpole, humanoid, reach).

Also drop the top-level BaseArticulation import in the experimental
SceneEntityCfg, which transitively pulled scene_data_provider -> pxr during
config parsing. The resolve() articulation branch now duck-types on
num_joints, mirroring the body branch.
YizeWang pushed a commit to YizeWang/IsaacLab that referenced this pull request Jun 3, 2026
@ooctipus ooctipus merged commit 429aff2 into isaac-sim:develop Jun 3, 2026
37 checks passed
@ooctipus ooctipus deleted the fix/warp_envs branch June 3, 2026 09:36
@hujc7
Copy link
Copy Markdown
Collaborator

hujc7 commented Jun 3, 2026

@ooctipus Is there a cherry-picked to release branch? I think it's needed.

@kellyguo11
Copy link
Copy Markdown
Contributor

@ooctipus Is there a cherry-picked to release branch? I think it's needed.

looks like it was cherry picked in #5916

@kellyguo11
Copy link
Copy Markdown
Contributor

ah nvm was looking at the wrong branch. thanks @hujc7 let's get the cherry pick on merged

hujc7 pushed a commit to hujc7/IsaacLab that referenced this pull request Jun 3, 2026
…ion split (isaac-sim#5916)

- Adds Warp-first experimental environments and MDP terms under
`isaaclab_experimental` and `isaaclab_tasks_experimental` (direct +
manager-based cartpole/humanoid/ant/reach/locomotion), with Warp-kernel
implementations of actions, observations, rewards, terminations, and
events.
- Adds lazy `__init__.py` + `.pyi` stubs (with explicit `__all__`)
across the new/updated packages.

- [ ] `./isaaclab.sh -p -m pytest
source/isaaclab_tasks_experimental/test`
- [ ] Train the experimental Warp cartpole/humanoid/ant configs on the
Newton backend and confirm parity with the stable envs.
- [ ] `./isaaclab.sh -p -m pytest
source/isaaclab_physx/test/assets/test_newton_actuators_physx.py
source/isaaclab_newton/test/assets/test_newton_actuators_newton.py`
- [ ] Verify `randomize_rigid_body_scale` raises on Newton and warns
(deprecated) on PhysX.
hujc7 pushed a commit to hujc7/IsaacLab that referenced this pull request Jun 3, 2026
…ion split (isaac-sim#5916)

- Adds Warp-first experimental environments and MDP terms under
`isaaclab_experimental` and `isaaclab_tasks_experimental` (direct +
manager-based cartpole/humanoid/ant/reach/locomotion), with Warp-kernel
implementations of actions, observations, rewards, terminations, and
events.
- Adds lazy `__init__.py` + `.pyi` stubs (with explicit `__all__`)
across the new/updated packages.

- [ ] `./isaaclab.sh -p -m pytest
source/isaaclab_tasks_experimental/test`
- [ ] Train the experimental Warp cartpole/humanoid/ant configs on the
Newton backend and confirm parity with the stable envs.
- [ ] `./isaaclab.sh -p -m pytest
source/isaaclab_physx/test/assets/test_newton_actuators_physx.py
source/isaaclab_newton/test/assets/test_newton_actuators_newton.py`
- [ ] Verify `randomize_rigid_body_scale` raises on Newton and warns
(deprecated) on PhysX.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working isaac-lab Related to Isaac Lab team

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants