Fix LEAPP projected gravity export#6007
Conversation
Expose root orientation as the LEAPP graph input for projected gravity observations and compute the projected gravity tensor inside the exported graph. This keeps the policy observation contract unchanged while matching deployment inputs.
There was a problem hiding this comment.
Code Review Summary
This PR correctly fixes the LEAPP export of projected_gravity by exposing the root quaternion orientation (root_quat_w) at the graph boundary instead of the pre-computed projected gravity vector. This is the right architectural decision since deployment backends (robot hardware/simulation) typically provide body orientation from IMUs rather than pre-computed projected gravity.
✅ Strengths
-
Sound Architecture: Exposing
root_quat_was aBODY_ROTATIONinput instead ofprojected_gravityasVECTOR3Daligns with what deployment backends actually provide. The projection computation happens inside the exported graph where it belongs. -
Correct Quaternion Math: The inline quaternion inverse rotation implementation correctly matches the existing
math_utils.quat_apply_inverseformula and properly handles Isaac Sim's (x, y, z, w) scalar-last quaternion convention. -
Good Test Coverage: The new tests verify both the direct proxy path (preserving
VECTOR3Dsemantics) and the observation export path (usingroot_quat_wwithBODY_ROTATIONsemantics), with mathematical correctness validated against the reference implementation. -
Proper Changelog Entry: Follows the project's changelog conventions.
📝 Follow-up Review (d30f2b7)
The new commits since the initial review (30d66c0→d30f2b7) contain only housekeeping changes:
- Documentation updates to use the new task naming convention (
Isaac-Reach-UR10instead ofIsaac-Reach-UR10-v0) - Test file rename and task name updates to match the codebase-wide rename
- Changelog
.skipmarker added
No changes to the core LEAPP projected gravity fix logic. The suggestions from the initial review remain optional enhancements and do not block merge.
📋 Overall Assessment
This is a well-implemented fix that addresses a real deployment concern. The approach is architecturally sound, the math is correct, and the test coverage validates the key behaviors. Ready to merge. ✅
Update (e290070): New commits are a merge of develop into the feature branch (cloner refactor #5770, OvPhysX runtime fix #5995, CI changelog bump). No changes to the LEAPP projected gravity fix. Previous assessment stands — ready to merge.
Update (2bd4a4b): New commit fixes hidden_state → hidden_states attribute name in two places in export.py (lines 202 and 332). Straightforward correctness fix aligning with the actual API. No new issues introduced. Previous assessment unchanged — ready to merge. ✅
Update (ae5481a): Merge of develop into the feature branch — brings in docs command updates (#6023) and PPISP optional-dependency refactor (#5986). No changes to LEAPP export logic or tests. No new issues. ✅
Update (6f18d1c): This commit reverts the previous hidden_states back to hidden_state (singular) in export.py (lines 202 and 332). The commit message explains the rationale well: rsl_rl 5.0.1 (which Isaac Lab pins) uses hidden_state (singular), while the plural form (hidden_states) only exists in older/vendored forks like AGILE's 2.3.3.
This is a good catch — the earlier commit (2bd4a4b) was incorrect for the pinned rsl_rl version. The revert restores the correct attribute name. Net effect: the export.py file is unchanged from its state before the 2bd4a4b commit, which is correct for rsl_rl 5.0.1.
No new issues. Core LEAPP projected gravity fix remains intact. ✅
|
can you move the tests to: IsaacLab/source/isaaclab_rl/test/export. that's where all the leapp export tests are. |
Greptile SummaryThis PR fixes the LEAPP graph export for the
Confidence Score: 4/5Safe to merge; the quaternion convention and rotation formula are correct, and regression tests validate both the annotated and unannotated paths. The core math (xyzw convention, Rodrigues inverse-rotation formula) matches the existing quat_apply_inverse helper exactly, the asset-name fallback to robot handles the default-config case correctly, and the tests assert numerical correctness against the reference implementation. The only things worth noting are a minor duplication of the rotation formula and a stale file path in the PR description test-plan commands that could confuse contributors running the tests manually. The new test file at source/isaaclab_rl/test/export/test_leapp_proxy.py — verify the CI configuration picks it up from that location rather than the path quoted in the PR description. Important Files Changed
Sequence DiagramsequenceDiagram
participant ObsManager as Observation Manager
participant Patcher as ExportPatcher
participant Wrapper as _wrap_projected_gravity
participant ProxyEnv as _EnvProxy / _DataProxy
participant LEAPP as LEAPP annotate
ObsManager->>Patcher: _patch_observation_manager(obs_manager, proxy_env)
Patcher->>Patcher: "detect func_name == projected_gravity"
Patcher->>Wrapper: _wrap_projected_gravity(original_func, proxy_env)
Patcher-->>ObsManager: "replace term_cfg.func, set noise=None"
Note over ObsManager: At policy rollout / export trace
ObsManager->>Wrapper: term_cfg.func(env, [asset_cfg])
Wrapper->>ProxyEnv: proxy_env.scene[asset_name].data.root_quat_w.torch
ProxyEnv->>LEAPP: annotate.input_tensors(task_name, BODY_ROTATION semantics)
LEAPP-->>ProxyEnv: annotated root_quat_w tensor
ProxyEnv-->>Wrapper: root_quat_w (Nx4, x,y,z,w)
Wrapper->>Wrapper: "gravity_w = [0,0,-1] broadcast to (N,3)"
Wrapper->>Wrapper: "projected_gravity_b = quat_apply_inverse(root_quat_w, gravity_w)"
Wrapper-->>ObsManager: projected_gravity_b (Nx3)
Reviews (1): Last reviewed commit: "Merge branch 'develop' into lgulich/leap..." | Re-trigger Greptile |
The export wrote the traced recurrent state to `memory.hidden_state`, but rsl_rl's Memory module reads/writes `hidden_states` (plural). The registered state tensors were therefore never fed into the LSTM: LEAPP flagged the `actor_state_*` inputs as unused and dropped them, producing a stateless ONNX (the LSTM reset to zeros every call). Write to `hidden_states` so the hidden state is threaded as graph I/O and the recurrent policy exports as stateful. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
My earlier commit changed the actor memory write to `hidden_states` (plural), but rsl_rl 5.0.1 (which Isaac Lab pins) stores the RNN state in `hidden_state` (singular, rsl_rl/modules/rnn.py). The plural form only exists in older/vendored rsl_rl (e.g. AGILE's 2.3.3 fork) and as the algorithm-level (actor, critic) tuple. The original `hidden_state` is correct for 5.0.1, so reverting; the AGILE recurrent-export issue is a vendored-rsl_rl version mismatch to be fixed by upgrading AGILE to rsl_rl 5.0.1. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
## Summary Backports #6007 to `release/3.0.0-beta2`. This fixes LEAPP export for `mdp.projected_gravity` so the exported graph takes `root_quat_w` as the input and computes projected gravity inside the graph, instead of exporting projected gravity directly as an input. ## Changes - Export `mdp.projected_gravity` through `root_quat_w` for LEAPP graphs. - Keep the policy observation as projected gravity by computing it inside the exported graph. - Add regression coverage and changelog fragments. ## Testing - Red check: temporarily removed the projected-gravity dispatch and ran `uv run --with pytest --with leapp python -m pytest source/isaaclab_rl/test/export/test_leapp_proxy.py -q` -> failed as expected in `test_projected_gravity_observation_exports_root_quat_w_input`. - Green check: restored the fix and reran `uv run --with pytest --with leapp python -m pytest source/isaaclab_rl/test/export/test_leapp_proxy.py -q` -> `2 passed, 37 warnings`. - `uv run --with ruff --with leapp ruff check source/isaaclab/isaaclab/utils/leapp/export_annotator.py source/isaaclab/isaaclab/utils/leapp/proxy.py source/isaaclab_rl/test/export/test_leapp_proxy.py` -> passed. - `./isaaclab.sh -f` -> passed.
Summary
The current LEAPP export exports projected_gravity as an input instead of root_quat_w.
The reason is that with the change to use warp for observation terms LEAPP can no longer automatically trace projected gravity back to root_quat_w. For new we fix this manually, once we have warp support in LEAPP this will no longer be needed.
Changes
mdp.projected_gravitythroughroot_quat_wfor LEAPP graphs.Test Plan
uv run --with pytest --with leapp python -m pytest source/isaaclab_rl/test/export/test_leapp_proxy.py -quv run --with ruff --with leapp ruff check source/isaaclab/isaaclab/utils/leapp/export_annotator.py source/isaaclab/isaaclab/utils/leapp/proxy.py source/isaaclab_rl/test/export/test_leapp_proxy.py./isaaclab.sh -fIsaac-Velocity-Flat-G1-v0checkpointmodel_1499.ptand verified YAML usesstate/body/rotationforrobot_root_quat_w.