locomotion-rough-ground-rl#142
Merged
Merged
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR updates the Wato humanoid locomotion RL setup for rough-ground training, aligning key command ranges and stabilizing training while sharing reward shaping across flat/rough configs.
Changes:
- Adds shared anti leg-crossing reward term and introduces a rough-specific fall termination (
base_tilt) in the rough locomotion config. - Adjusts rough-terrain training parameters (feet air-time reward strength, termination penalty, velocity command ranges) and mirrors training command ranges in PLAY config.
- Updates PPO runner configuration for rough locomotion (empirical normalization, std parameterization, entropy tweak) and refreshes session progress documentation.
Reviewed changes
Copilot reviewed 5 out of 6 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| autonomy/simulation/Humanoid_Wato/HumanoidRL/HumanoidRLPackage/rsl_rl_scripts/train.py | Minor formatting-only change near helper definition spacing. |
| autonomy/simulation/Humanoid_Wato/HumanoidRL/HumanoidRLPackage/HumanoidRLSetup/tasks/locomotion/SESSION_PROGRESS_2026-07-13.md | Adds detailed session notes documenting tuning decisions and outcomes. |
| autonomy/simulation/Humanoid_Wato/HumanoidRL/HumanoidRLPackage/HumanoidRLSetup/tasks/locomotion/config/wato_humanoid_v1/rough_env_cfg.py | Adds shared reward term(s), introduces rough-specific termination, and updates rough env tuning parameters and PLAY ranges. |
| autonomy/simulation/Humanoid_Wato/HumanoidRL/HumanoidRLPackage/HumanoidRLSetup/tasks/locomotion/config/wato_humanoid_v1/flat_env_cfg.py | Removes flat-only duplicate reward term implementation and disables rough-only reward that depends on removed sensors. |
| autonomy/simulation/Humanoid_Wato/HumanoidRL/HumanoidRLPackage/HumanoidRLSetup/tasks/locomotion/config/wato_humanoid_v1/agents/rsl_rl_ppo_cfg.py | Updates PPO runner settings for rough training stability and exploration behavior. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| torch.backends.cudnn.benchmark = False | ||
|
|
||
|
|
||
|
|
Comment on lines
+88
to
+92
| base_height_l2 = RewTerm( | ||
| func=mdp.base_height_l2, | ||
| weight=-5.0, | ||
| params={"target_height": 0.75, "sensor_cfg": SceneEntityCfg("height_scanner")}, | ||
| ) |
Comment on lines
22
to
39
| policy = RslRlPpoActorCriticCfg( | ||
| init_noise_std=1.0, | ||
| actor_hidden_dims=[512, 256, 128], | ||
| critic_hidden_dims=[512, 256, 128], | ||
| activation="elu", | ||
| # Root cause of the recurring "RuntimeError: normal expects all elements of | ||
| # std >= 0.0" crash (confirmed via torch.autograd.set_detect_anomaly + cross- | ||
| # referenced against rsl_rl's own upstream GitHub issue #33): with the default | ||
| # noise_std_type="scalar", the policy's std is a raw nn.Parameter with no | ||
| # lower bound -- the optimizer can push it negative or to NaN directly, or an | ||
| # out-of-distribution observation (e.g. mid-fall on rough terrain) can drive | ||
| # the PPO ratio's exp() to overflow, both of which crash torch.normal(). | ||
| # rsl_rl's own maintainers added "log" mode as the structural fix: std is | ||
| # stored as log(std) and exponentiated, so it's mathematically guaranteed | ||
| # positive no matter what the optimizer does to the underlying parameter. | ||
| # We were already on rsl-rl-lib 3.1.2 (which has this), just never set it. | ||
| noise_std_type="log", | ||
| ) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.