Pith.

Robotics · arXiv:2611.16337 v1 · submitted 12 Nov 2026 · 50 references

Memory-Efficient Parallel Policy Learning for Collapse-Reactive Retreat via Hybrid Terrain Modeling

Yuki Kadokawa, Ritsuki Matsunaga, Yuqi Yang, Edgar Anarossi, Takamitsu Matsubara

The paper says

“At disaster sites and on unstable slopes, robot contact can collapse uninspected terrain, requiring retreat upon support loss. Massively parallel reinforcement learning of collapse-reactive retreat requires independent state–action experiences.”

more of the abstract

However, on shared rigid-body collapsible terrain, one robot's collapse alters others' contacts without direct interaction. We call this indirect interference. Duplicating terrain objects prevents this but is memory-intensive, limiting parallel sampling. Our key insight is that policy learning does not necessarily require rigid-body collapse modeling, which is feasible but computationally expensive; collapse-induced contact responses can suffice. We therefore propose a memory-efficient parallel policy-learning framework for collapse-reactive retreat via hybrid terrain modeling. Hybrid terrain modeling combines rigid-body simulation of non-collapsible terrain with virtual contact forces for collapsible regions. A State-Multiplexed Contact Model shares terrain geometry and a contact-response law while maintaining environment-specific collapse states, enabling independent experiences without terrain objects for collapsible regions. The framework trains a proprioceptive policy for both normal locomotion and collapse-reactive retreat. It supports over 100 times as many parallel environments as fine-resolution rigid-block terrain within the same GPU memory limit. This scalability and improved sample efficiency enable over 50 times faster policy convergence with comparable retreat performance. Sim-to-sim and real-robot experiments demonstrate retreat under multiple collapse conditions. A supplementary video is available at: https://youtu.be/_RrhwjzEPE0

from the abstract

Pith says

Machine review · deepseek-v4-flash · 13 Nov 2026 · what was read was not recorded · not checked by a person

It can be wrong. Challenge this review or reply to it.

Holds up, with conditions. Clever hybrid terrain model for parallel RL on collapsible ground; real-robot validation is thin but the simulation results are solid. This line is wrong

4 major objections · 6 minor

How often objections from reviews like this one are right: not yet measured

Are you an author? What you can do.

Not checked: figures, code, what the 50 references say.

The paper says, at Section V-K, Fig. 15

The review did not quote this passage.

Pith says: Objection 1 of 4 · major

The real-robot validation uses only 25 trials per method per scenario (5 policies × 5 trials each). This sample size is small, and no statistical test is reported to compare Ours against baselines. While the results are suggestive, the claim that the policy 'transfers without additional training to a real robot in four physical collapse scenarios' (Section V-K) would be strengthened by a larger number of trials and appropriate statistical analysis (e.g., confidence intervals, effect sizes).

more of objection 1

The authors acknowledge in Section VI-B that 'this transfer does not establish the physical fidelity of the virtual contact model,' but the real-robot results are a central part of the paper's contribution. A more rigorous evaluation would increase confidence in the transferability claim.

The paper says, at Section IV-B, Appendix D

The review did not quote this passage.

A classifier read this sentence of the paper as acknowledging the concern (probability 0.95):

“However, this transfer does not establish the physical fidelity of the virtual contact model, whose correspon- dence to real ground responses remains unquantified despite the randomized contact and collapse parameters (Table V).”

Pith says: Objection 2 of 4 · major

The three collapse primitives (Touch, Sink, Slip) are described as 'simple realizations' and 'not physical models of ground failure' (Section IV-B). The virtual contact model's parameters (stiffness, damping, friction, trigger forces, etc.) are randomized over wide ranges (Table V). While this randomization may help the policy generalize, the paper does not provide any validation that the simulated contact responses (forces, support-loss timing, robot motion) are representative of real ground collapse.

more of objection 2

The sim-to-real transfer is demonstrated only on four ad-hoc physical scenarios (sponge sheet, rubber sheet, wooden blocks, plastic balls). Without a quantitative comparison of simulated vs. real collapse dynamics, it is unclear whether the policy would work on unseen real collapse terrain. The authors acknowledge this limitation in Section VI-B, but it remains a significant gap in the evidence for the paper's central claim.

The paper says, at Section V-H, Fig. 11

The review did not quote this passage.

Pith says: Objection 3 of 4 · major

The sim-to-sim transfer experiment tests policies trained with Ours and Grid on Particle representation. However, Particle is used only as a testing representation, not for training. The paper states that 'Particle is used only to measure computational cost and as a testing representation in the sim-to-sim experiment' (Section V-B1).

more of objection 3

This is a reasonable design choice given Particle's high cost, but it means that the generalization to Particle is tested only on policies trained with simpler models. The claim that 'a policy trained only with hybrid terrain modeling achieved over 80% success in Particle' (Section VI-B) is based on this experiment. However, the Particle representation itself is a simplified simulation (particles on a floor that is removed cell by cell), not a physically accurate granular simulation. The paper does not compare against a more realistic granular simulator (e.g., GranularGym [22] or MPM [25]) to assess whether the virtual contact model captures essential collapse dynamics. This limits the strength of the generalization claim.

The paper says, at Section V-I, Fig. 12

The review did not quote this passage.

Pith says: Objection 4 of 4 · major

The comparison with estimator-triggered retreat (Naive and RTDD) shows that Ours achieves >90% fall avoidance while Naive and RTDD achieve <20%. However, the baselines use a locomotion policy trained on non-collapsible terrain (Legged Gym), which is not designed to handle support loss.

more of objection 4

This is a valid comparison to show the benefit of training a single policy for both locomotion and collapse response, but it is not a fair comparison of the retreat method itself. The Naive and RTDD estimators could potentially be combined with a policy trained on collapsible terrain (e.g., Ours's policy) to isolate the effect of the retreat command timing. The paper's conclusion that 'detecting collapse and issuing a retreat command may be insufficient if the underlying locomotion policy cannot handle support loss' (Section VI-F) is supported, but the experiment conflates the estimator quality with the locomotion policy's robustness. A cleaner ablation would use Ours's policy with and without the estimator-triggered retreat command.

6 minor objections

The paper says, at General

Pith says: minor objection 1 of 6

The paper is generally well-written and the experiments are clearly presented. However, there are a few minor issues: (1) The abstract states 'over 100 times as many parallel environments' and 'over 50 times faster policy convergence' – these numbers are from specific comparisons (Ours vs.

more of objection 1

Grid (10) at N=16384 vs. N=128). It would be helpful to explicitly state the comparison point in the abstract. (2) The supplementary video link is provided but not referenced in the main text beyond the abstract. (3) The paper uses the term 'collapse-reactive retreat' but the policy also performs normal locomotion; the title could be slightly misleading.

The paper says, at Section III-C1, Eq. (1)

Pith says: minor objection 2 of 6

The notation for the contact-response law H and state transition Φ is clear, but the dependence on the simulation step j and the environment index e could be made more explicit. The superscript (j) is used for the step, but the environment index e is sometimes omitted. Consider adding a brief note on the indexing convention.

show 4 more

The paper says, at Table I and Table II

Pith says: minor objection 3 of 6

The comparison tables are useful but some entries are ambiguous. For example, in Table I, property (b) 'No visual or probing inspection before contact' is marked ✓ for Ours, but the paper acknowledges that exteroceptive sensing could complement proprioception (Section VI-D).

more of objection 3

The ✓ is correct for the current implementation, but a note about the limitation would be helpful. Similarly, in Table II, property (e) 'Learning with ≥10^3 parallel environments' is marked ✓ for Ours, but the paper shows that Ours runs up to 16,384 environments. The ✓ is appropriate.

The paper says, at Appendix C, Table VI

Pith says: minor objection 4 of 6

The reward function is complex with many terms. The paper provides a detailed table, but the gating variables (D, B, S, u, ℓ̄) are defined in the text. It would be helpful to include a brief summary of the gating logic in the table caption or a separate note.

The paper says, at Section V-L, Fig. 16

Pith says: minor objection 5 of 6

The outdoor demonstration is qualitative and not included in the quantitative evaluation. The paper states that 'the robot retreated to the supported ground after a wheel lost support' but does not provide success rates or failure cases. This is acceptable as a demonstration, but the claim that it 'further demonstrate[s] successful retreat' (Section V-L) should be tempered given the lack of quantitative data.

The paper says, at References

Pith says: minor objection 6 of 6

Reference [49] is listed as 'arXiv preprint arXiv:2606.09183, 2026' which appears to be a future date (2026). This may be a typo or a preprint that has not yet been published. Please verify the reference.

What would settle it

Conduct a systematic real-robot evaluation with at least 100 trials per scenario across a wider range of collapse materials (e.g., sand, gravel, loose soil, snow) and measure contact forces and support-loss timing. Compare the distribution of robot trajectories and success rates against simulation predictions. If the success rate drops significantly (e.g., below 70%) on any new material, or if the contact forces differ systematically from the virtual model, then the claim of sim-to-real transfer is not robust.

from the skeptic's reading

How the review went

4 readings by deepseek-v4-flash · 13 Nov 2026

  1. Reader holds up, with conditions confidence high
  2. Skeptic agrees 6 s · 29,964 tokens read, 574 written
  3. Editor wrote the verdict above 9 s · 30,882 tokens read, 631 written
  4. Referee major revision: 4 major objections, 6 minor 21 s · 30,339 tokens read, 2,051 written

Not run on this paper: the simulated authors' defence.

Pith did not check this review's objections against the paper.

The review in full

The review in brief, machine-written

The paper's core idea is genuinely clever: instead of giving each parallel environment its own rigid-body collapsible terrain (which eats memory), they replace the collapsible region with a shared contact-response law plus a tiny per-environment collapse state. This State-Multiplexed Contact Model (SMCM) eliminates indirect interference without duplicating terrain objects. The 100x improvement in parallel environments and 50x faster convergence are real and well-measured against sensible baselines (Grid at various resolutions, Particle). The paper is thorough in defining the indirect interference problem, measuring it, and running ablations on collapse primitives, terrain types, and collapse-exposure direction. The writing is clear and the limitations are stated explicitly—they don't oversell the physical fidelity of the virtual contact model.

The soft spots are real but not fatal. The main one is that the virtual contact model's correspondence to real ground failure is unquantified. The authors say this themselves (Section VI-B). The real-robot validation uses 25 trials per method per scenario on four ad-hoc scenarios (sponge sheet, rubber sheet, wooden blocks, plastic balls). That's enough to show the policy does something reasonable, but not enough to claim it generalizes to real collapse dynamics. No contact forces or support-loss timing are measured on the real robot to validate the simulation. The collapse primitives are acknowledged as 'simple realizations' and 'not physical models of ground failure' (Section IV-B). So the central claim that the policy transfers to real collapsible terrain is supported by a demonstration, not by rigorous evidence. That said, the paper is transparent about this gap, and the sim-to-sim results (including >80% success on Particle, which the policy never trained on) suggest the approach has real merit.

Minor concerns: the false retreat rate on Stepped Downhill terrain (up to ~10%) is acknowledged but not deeply analyzed. The outdoor demonstration is qualitative. The paper does not release code or data, which would help reproducibility.

Who this is for: researchers working on parallel RL for locomotion on state-changing terrain, especially those dealing with memory bottlenecks. The SMCM formulation is clean and could be extended to other morphologies or terrain types.

Recommendation: This paper deserves a serious referee. The method is novel, the experiments are well-designed (if limited in real-robot scope), and the authors are honest about limitations. I would accept peer review and ask for stronger real-robot validation or a clearer statement that the real-robot results are demonstrative rather than definitive. The paper is not ready for publication as-is, but it's a solid contribution that should go through review.

The strongest objection

The paper's central claim that the policy transfers to real collapse scenarios (Section V-K) rests on 25 trials per method per scenario on four constructed physical scenarios (Touch, Sink, Slip, Particle Slip). The authors explicitly state that 'this transfer does not establish the physical fidelity of the virtual contact model, whose correspondence to real ground responses remains unquantified' (Section VI-B). The three collapse primitives are 'simple realizations' and 'not physical models of ground failure' (Section IV-B). The real-robot scenarios are ad-hoc constructions (sponge sheet, rubber sheet, wooden blocks, plastic balls) that may not represent the diversity of real ground failure. The sample size is small (25 trials per scenario), and no statistical test is reported to compare Ours against baselines. The paper does not measure contact forces, support-loss timing, or robot motion on the real robot to validate that the simulated collapse dynamics match reality. Without such validation, the claim that the policy 'transfers without additional training to a real robot in four physical collapse scenarios' is supported only by a limited demonstration, not by rigorous evidence that the policy would work on unseen real collapse terrain.

The reader's weakest_assumption correctly identifies that the virtual contact model's correspondence to real ground responses is unquantified and that the real-robot validation is limited. My stress-test confirms this as the most load-bearing concern. The paper is transparent about this limitation (Section VI-B), but the central claim of sim-to-real transfer is not fully established. The concrete test I propose would directly address this gap by requiring more extensive real-robot evaluation with quantitative comparison. The verdict remains CONDITIONAL because the paper's claims are plausible and well-supported in simulation, but the real-robot evidence is insufficient to fully validate the central claim. I agree with the reader's assessment.

The referee's report

If the results hold, the paper makes a significant contribution to massively parallel RL for locomotion on state-changing terrain. The hybrid terrain modeling approach is a clever engineering solution to the memory bottleneck that arises when each parallel environment requires its own copy of deformable/collapsible terrain. The SMCM formulation is general and could be extended to other robot morphologies and collapse types. The paper is thorough in its experimental design, comparing against multiple baselines (Grid at various resolutions, Particle, Naive, RTDD) and testing on a real robot. The authors are also transparent about limitations (unquantified physical fidelity of virtual contact model, limited real-robot validation, reliance on simple collapse primitives). The work addresses a practical problem in disaster response robotics.

Pith's reading of the paper (a machine's summary, not the authors')

The paper is trying to establish that a robot can learn, entirely in simulation, a single control policy that performs normal locomotion and, when the ground gives way under a wheel, automatically retreats to firm ground—without any prior visual inspection, separate collapse detector, or controller switching. The authors argue that the key obstacle to training such a policy with massively parallel reinforcement learning is memory: giving each simulated robot its own copy of a collapsible terrain model exhausts GPU memory, whereas letting them share one model causes "indirect interference" (one robot's collapse alters the terrain another robot experiences). Their central claim is that this obstacle can be bypassed by replacing the collapsible region with a compact virtual contact model that computes contact forces from a shared law and per-environment collapse states, eliminating the need for terrain objects. If true, this would make it practical to train collapse-reactive behaviors at scale, and the paper reports that the resulting policy transfers to a real wheeled quadruped in four physical collapse scenarios without additional training.

What carries the argument. The central object is the State-Multiplexed Contact Model (SMCM), which separates a collapsible-terrain model into: (1) shared reference terrain geometry, (2) a shared contact-response law that computes virtual normal and tangential forces from wheel positions and velocities, and (3) a shared state transition function, all shared across parallel environments, plus (4) a per-environment collapse state that evolves independently and irreversibly within each episode. The SMCM computes virtual contact forces for each wheel based on its environment-specific collapse state (support ratio, surface height, tangential traction), applies them as external forces to the robot, and updates the collapse state—all without creating any terrain objects for the collapsible region. This design eliminates indirect interference while keeping memory and computation per added environment minimal.

Core claim. The paper introduces a hybrid terrain modeling framework for massively parallel reinforcement learning of collapse-reactive retreat. The key discovery is that policy learning does not require detailed rigid-body simulation of collapsing terrain; instead, collapse-induced contact responses can be computed by a State-Multiplexed Contact Model (SMCM) that shares terrain geometry and a contact-response law across parallel environments while maintaining only a small, independent collapse state per environment. This eliminates indirect interference (where one robot's collapse alters another's terrain) without duplicating terrain objects, enabling over 100 times as many parallel environments as fine-resolution rigid-block terrain within the same GPU memory. The framework trains a single proprioceptive policy for both normal locomotion and collapse-reactive retreat, and the paper demonstrates that this policy transfers without additional training to a real wheeled quadruped robot in four physical collapse scenarios, achieving over 50 times faster policy convergence with comparable retreat performance.

The premise it rests on. The virtual contact model's correspondence to real ground responses is unquantified; the three collapse primitives (Touch, Sink, Slip) are simple realizations, not physical models of ground failure, and the paper assumes these produce contact responses sufficiently similar to real ground failure for the policy to generalize to physical collapse scenarios.

If the paper is right (Pith's inferences, not the authors')

The SMCM formulation is not inherently limited to wheeled quadrupeds; it can be extended to other robot morphologies by computing contact positions, penetration depths, and relative velocities for any link.

The framework supports adding new collapse phenomena (e.g., mud, snow, gravel) by defining their state variables and update rules, without building new terrain geometry or duplicating terrain objects.

Because the policy relies only on proprioception, it may mistake a traversable downward step for a collapse; combining it with exteroceptive terrain assessment or tactile probing could reduce false retreats while retaining collapse-reactive retreat.

The results suggest that task-relevant contact responses can provide the state–action experiences needed to learn collapse-reactive retreat without detailed ground-failure simulation, potentially reducing simulation cost for other contact-based tasks.

The framework's claim that training with three simple collapse primitives (Touch, Sink, Slip) generalizes to physical collapse scenarios rests on an unquantified correspondence between the virtual contact model and real ground responses; direct measurement of contact forces, support-loss timing, and robot motion in real collapse scenarios would be needed to establish which aspects of the virtual m

The paper's finding that a policy trained only with forward collapse exposure succeeds in over 85% of forward entries but under 20% of backward entries suggests that the retreat behavior is direction-specific; a robot that might need to retreat from terrain entered backward would require additional training.

The 50x faster convergence claim depends on the specific comparison to fine-resolution rigid-block terrain (Grid (10)); the improvement over coarser grids (Grid (50)) is about 4x in time to convergence, which is still substantial but less dramatic than the headline ratio suggests.

Check this review

Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.

How often reviews like this one are right

Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.

How often objections from reviews like this one are right: not yet measured

Every value above, in the review's public record (JSON)

Reviews by people

This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.

Request a human review

How Pith checks papers · The full audit of this paper

The paper

Abstract

At disaster sites and on unstable slopes, robot contact can collapse uninspected terrain, requiring retreat upon support loss. Massively parallel reinforcement learning of collapse-reactive retreat requires independent state–action experiences. However, on shared rigid-body collapsible terrain, one robot's collapse alters others' contacts without direct interaction. We call this indirect interference. Duplicating terrain objects prevents this but is memory-intensive, limiting parallel sampling. Our key insight is that policy learning does not necessarily require rigid-body collapse modeling, which is feasible but computationally expensive; collapse-induced contact responses can suffice. We therefore propose a memory-efficient parallel policy-learning framework for collapse-reactive retreat via hybrid terrain modeling. Hybrid terrain modeling combines rigid-body simulation of non-collapsible terrain with virtual contact forces for collapsible regions. A State-Multiplexed Contact Model shares terrain geometry and a contact-response law while maintaining environment-specific collapse states, enabling independent experiences without terrain objects for collapsible regions. The framework trains a proprioceptive policy for both normal locomotion and collapse-reactive retreat. It supports over 100 times as many parallel environments as fine-resolution rigid-block terrain within the same GPU memory limit. This scalability and improved sample efficiency enable over 50 times faster policy convergence with comparable retreat performance. Sim-to-sim and real-robot experiments demonstrate retreat under multiple collapse conditions. A supplementary video is available at: https://youtu.be/_RrhwjzEPE0

The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.

abstract compiled · 2026-11-13

The authors' main claim, as the classifier picked it out 0.62

“This scalability and improved sample efficiency enable over 50 times faster policy convergence with comparable retreat performance.”

How the main claim is stated: asserted 1.00

What the abstract itself states, as probabilities: claims first 0.05 · states limitation 0.45 · human participants 0.02 · controlled comparison 0.48 · randomized assignment 0.02 · states quantitative result 0.98 · compares with prior results 0.65 · mentions code or data availability 0.12

The abstract, sentence by sentence

This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.

SentenceRoleThe authors' own workStates the main conclusionStrength
At disaster sites and on unstable slopes, robot contact can collapse uninspected terrain, requiring retreat upon support loss. background 1.00 0.08 0.02 asserted 0.99
Massively parallel reinforcement learning of collapse-reactive retreat requires independent state--action experiences. background 0.92 0.13 0.04 asserted 1.00
However, on shared rigid-body collapsible terrain, one robot's collapse alters others' contacts without direct interaction. background 0.71 0.15 0.03 asserted 1.00
We call this indirect interference. background 0.61 0.46 0.03 asserted 0.90
Duplicating terrain objects prevents this but is memory-intensive, limiting parallel sampling. background 0.73 0.28 0.04 asserted 1.00
Our key insight is that policy learning does not necessarily require rigid-body collapse modeling, which is feasible but computationally expensive; collapse-induced contact responses can suffice. contribution 0.74 0.49 0.44 hedged possibility 0.62
We therefore propose a memory-efficient parallel policy-learning framework for collapse-reactive retreat via hybrid terrain modeling. contribution 0.98 0.83 0.30 asserted 0.98
Hybrid terrain modeling combines rigid-body simulation of non-collapsible terrain with virtual contact forces for collapsible regions. method 0.98 0.69 0.08 asserted 1.00
A State-Multiplexed Contact Model shares terrain geometry and a contact-response law while maintaining environment-specific collapse states, enabling independent experiences without terrain objects for collapsible regions. method 0.96 0.83 0.16 asserted 1.00
The framework trains a proprioceptive policy for both normal locomotion and collapse-reactive retreat. method 0.56 0.85 0.07 asserted 1.00
It supports over 100 times as many parallel environments as fine-resolution rigid-block terrain within the same GPU memory limit. result 0.99 0.93 0.12 asserted 1.00
This scalability and improved sample efficiency enable over 50 times faster policy convergence with comparable retreat performance. result 0.99 0.93 0.28 asserted 1.00
Sim-to-sim and real-robot experiments demonstrate retreat under multiple collapse conditions. result 0.90 0.94 0.11 asserted 1.00
A supplementary video is available at: https://youtu.be/_RrhwjzEPE0 contribution 0.57 0.46 0.02 asserted 0.91

What kind of paper the classifier reports this is: method or tool 1.00 · evidence from observation or experiment 0.61

What was examined, and what was not

Sources read

  • arXiv listing abstract v1

Not examined

  • full text
  • figures and tables
  • references and citing sentences
  • relations to other papers

Levels run

  • P abstract

Not run

  • S sections
  • T body sentences
  • C citations
  • R cross-paper relations

How the labels were made

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-11-13: 1 request, 9,916 input tokens.

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON

Figures (15)
Figure 1 of the paper
Figure 1. Concept of our policy-learning framework. (Top) A wheeled
Figure 2 of the paper
Figure 2. Overview of the proposed policy-learning framework.
Figure 3 of the paper
Figure 3. Collapse primitives implemented as representative examples. (a) Collapse-capable opening and a randomly shaped collapse region; the inset shows the support ratio of a wheel across the region boundary after support loss. (b)–(d) Touch, Sink, and Slip, each in three stages from left to right. references: its scaled entries give position targets for the leg joints, as offsets from the default posture, and velocity targets for the wheel joints. These targets are held over the control step and tracked by joint PD controllers (Appendix B). This action space lets a single policy combine…
Figure 4 of the paper
Figure 4. Training terrain types (columns) at the lowest (Lv. Min,
Figure 5 of the paper
Figure 5. Terrain representations of Ours, Grid, and Particle, illustrated with the
Figure 6 of the paper
Figure 6. Sim-to-sim success rate (line, left axis) and indirect-interference rate
Figure 9 of the paper
Figure 9. Retreat success rates in the collapse-primitive ablation: Panels show
Figure 11 of the paper
Figure 11. Sim-to-sim retreat success rates across terrain representations: Rows indicate the representation used for training, and columns indicate the
Figure 12 of the paper
Figure 12. Moving speed versus success rate under sudden ground collapse.
Figure 14 of the paper
Figure 14. Snapshots of Ours and Naive in the sim-to-real experiment: Panels show the four physical collapse scenarios, (a) Touch, (b) Sink, (c) Slip, and (d)
Figure 15 of the paper
Figure 15. Number of successes of Ours, Naive, and RTDD in the sim-to-real

Showing 11 of the paper's 15 figures. Figures are the authors', reproduced from the paper.

References (50)
  1. [22] D. R. Millard, D. Pastor, J. Bowkett, P. Backes, and G. S. Sukhatme, “GranularGym: High performance simulation for robotic tasks with granular materials,” inRobotics: Science and Systems, 2023 on Pith
  2. [25] D. Sulsky, Z. Chen, and H. L. Schreyer, “A particle method for history- dependent materials,”Computer Methods in Applied Mechanics and Engineering, vol. 118, no. 1–2, pp. 179–196, 1994 on Pith
  3. [1] S. Godon, A. Ristolainen, and M. Kruusmaa, “An insight on mud behavior upon stepping,”IEEE Robotics and Automation Letters, vol. 7, no. 4, pp. 11 039–11 046, 2022 on Pith
  4. [2] S. Godon, C. P. Sesmero, A. Chemori, A. Ristolainen, and M. Kruusmaa, “Walking in mud: Modeling, control, and experiments of quadruped locomotion,”IEEE/ASME Transactions on Mechatronics, 2025 on Pith
  5. [3] M. H. Raibert, “Trotting, pacing, and bounding by a quadruped robot,” Journal of Biomechanics, vol. 23, no. Supplement 1, pp. 79–98, 1990 on Pith
  6. [4] M. Hutter, C. Gehring, D. Jud, A. Lauber, C. D. Bellicoso, V . Tsounis, J. Hwangbo, K. Bodie, P. Fankhauser, M. Bloesch, R. Diethelm, S. Bach- mann, A. Melzer, and M. Hoepflinger, “ANYmal—a highly mobile and dynamic quadrupedal robot,” inIEEE/RSJ International Conference on In... on Pith
  7. [5] M. Bjelonic, P. K. Sankar, C. D. Bellicoso, H. Vallery, and M. Hutter, “Rolling in the deep—hybrid locomotion for wheeled-legged robots using online trajectory optimization,”IEEE Robotics and Automation Letters, vol. 5, no. 2, pp. 3626–3633, 2020. 17 on Pith
  8. [6] N. He, Z. Yang, X. Fan, W. Que, S. Liu, H. Xu, C. Bu, and B. Zhang, “Residual policy optimization with trust region constraints: A learning framework for stable and agile wheel-legged locomotion,”IEEE Trans- actions on Automation Science and Engineering, vol. 22, pp. 23 352– 2... on Pith
  9. [7] G. Haddeler, M. Y . M. Chuah, Y . You, J. Chan, A. H. Adiwahono, W. Y . Yau, and C.-M. Chew, “Traversability analysis with vision and terrain probing for safe legged robot navigation,”Frontiers in Robotics and AI, vol. 9, p. 887910, 2022 on Pith
  10. [8] G. Haddeler, H. P. Palanivelu, Y . C. Ng, F. Colonnier, A. H. Adiwahono, Z. Li, C.-M. Chew, and M. Y . M. Chuah, “Real-time digital double framework to predict collapsible terrains for legged robots,” inInterna- tional Conference on Intelligent Robots and Systems, 2022 on Pith
  11. [9] T. Miki, J. Lee, J. Hwangbo, L. Wellhausen, V . Koltun, and M. Hutter, “Learning robust perceptive locomotion for quadrupedal robots in the wild,”Science Robotics, vol. 7, no. 62, p. eabk2822, 2022 on Pith
  12. [10] E. Tennakoon, T. Peynot, J. Roberts, and N. Kottege, “Probe-before-step walking strategy for multi-legged robots on terrain with risk of collapse,” inIEEE International Conference on Robotics and Automation, 2020, pp. 5530–5536 on Pith
  13. [11] J. Lee, J. Hwangbo, L. Wellhausen, V . Koltun, and M. Hutter, “Learning quadrupedal locomotion over challenging terrain,”Science Robotics, vol. 5, no. 47, p. eabc5986, 2020 on Pith
  14. [12] T. Kim and S.-H. Lee, “Quadruped locomotion on non-rigid terrain using reinforcement learning,”arXiv preprint arXiv:2107.02955, 2021 on PithPith's reviewarXiv
  15. [13] S. Choi, G. Ji, J. Park, H. Kim, J. Mun, J. H. Lee, and J. Hwangbo, “Learning quadrupedal locomotion on deformable terrain,”Science Robotics, vol. 8, no. 74, p. eade2256, 2023 on Pith
  16. [14] S. Dey, D. Fan, R. Schmid, A. Dixit, K. Otsu, T. Touma, A. F. Schilling, and A.-a. Agha-mohammadi, “PrePARE: Predictive proprioception for agile failure event detection in robotic exploration of extreme terrains,” inIEEE/RSJ International Conference on Intelligent Robots and S... on Pith
  17. [15] T. He, C. Zhang, W. Xiao, G. He, C. Liu, and G. Shi, “Agile but safe: Learning collision-free high-speed legged locomotion,” inRobotics: Science and Systems, 2024 on Pith
  18. [16] J. Hwangbo, J. Lee, A. Dosovitskiy, C. D. Bellicoso, V . Tsounis, V . Koltun, and M. Hutter, “Learning agile and dynamic motor skills for legged robots,”Science Robotics, vol. 4, no. 26, p. eaau5872, 2019 on Pith
  19. [17] J. Tan, T. Zhang, E. Coumans, A. Iscen, Y . Bai, D. Hafner, S. Bohez, and V . Vanhoucke, “Sim-to-real: Learning agile locomotion for quadruped robots,” inRobotics: Science and Systems, 2018 on Pith
  20. [18] X. B. Peng, E. Coumans, T. Zhang, T.-W. E. Lee, and S. Levine, “Learning agile robotic locomotion skills by imitating animals,” in Robotics: Science and Systems, 2020 on Pith
show 30 more
  1. [19] N. Rudin, D. Hoeller, P. Reist, and M. Hutter, “Learning to walk in minutes using massively parallel deep reinforcement learning,” in Conference on Robot Learning, vol. 164, 2022, pp. 91–100 on Pith
  2. [20] V . Makoviychuk, L. Wawrzyniak, Y . Guo, M. Lu, K. Storey, M. Macklin, D. Hoeller, N. Rudin, A. Allshire, A. Handa, and G. State, “Isaac Gym: High performance GPU based physics simulation for robot learning,” inNeural Information Processing Systems Track on Datasets and Benchm... on Pith
  3. [21] M. Mittal, C. Yu, Q. Yu, J. Liu, N. Rudin, D. Hoeller, J. L. Yuan, R. Singh, Y . Guo, H. Mazhar, A. Mandlekar, B. Babich, G. State, M. Hutter, and A. Garg, “Orbit: A unified simulation framework for interactive robot learning environments,”IEEE Robotics and Automation Letters,... on Pith
  4. [23] Y . Kadokawa, H. Tahara, and T. Matsubara, “Progressive-resolution policy distillation: Leveraging coarse-resolution simulations for time- efficient fine-resolution policy learning,”IEEE Transactions on Automa- tion Science and Engineering, vol. 22, pp. 18 682–18 693, 2025 on Pith
  5. [24] P. A. Cundall and O. D. L. Strack, “A discrete numerical model for granular assemblies,”G ´eotechnique, vol. 29, no. 1, pp. 47–65, 1979 on Pith
  6. [26] C. Li, T. Zhang, and D. I. Goldman, “A terradynamics of legged locomotion on granular media,”Science, vol. 339, no. 6126, pp. 1408– 1412, 2013 on Pith
  7. [27] M. Kalakrishnan, J. Buchli, P. Pastor, M. Mistry, and S. Schaal, “Learn- ing, planning, and control for quadruped locomotion over challenging terrain,”The International Journal of Robotics Research, vol. 30, no. 2, pp. 236–258, 2011 on Pith
  8. [28] D. D. Fan, K. Otsu, Y . Kubo, A. Dixit, J. Burdick, and A.-a. Agha- mohammadi, “STEP: Stochastic traversability evaluation and planning for risk-aware off-road navigation,” inRobotics: Science and Systems, 2021 on Pith
  9. [29] C. Yan, Y . Jiang, Q. Liu, and J. Qin, “Learning policy-dependent traversability for terrain-aware quadruped navigation,”IEEE Transac- tions on Automation Science and Engineering, vol. 23, pp. 10 325– 10 336, 2026 on Pith
  10. [30] L. Wellhausen, A. Dosovitskiy, R. Ranftl, K. Walas, C. Cadena, and M. Hutter, “Where should I walk? predicting terrain properties from images via self-supervised learning,”IEEE Robotics and Automation Letters, vol. 4, no. 2, pp. 1509–1516, 2019 on Pith
  11. [31] X. Fang, Z. Li, Y . Chen, W. Tan, M. Zhang, R. Song, and W. Zhang, “MoTL: Modality-balanced terrain-aware locomotion learn- ing for quadruped robots,”IEEE Transactions on Automation Science and Engineering, vol. 23, pp. 11 868–11 880, 2026 on Pith
  12. [32] A. Luo, D. Li, S. Wang, S. Kong, K. Zhu, and J. Yu, “Perceptive locomotion and navigation for quadruped robots via depth-based repre- sentation,”IEEE Transactions on Automation Science and Engineering, vol. 23, pp. 8672–8684, 2026 on Pith
  13. [33] H. Kolvenbach, C. B ¨artschi, L. Wellhausen, R. Grandia, and M. Hutter, “Haptic inspection of planetary soils with legged robots,”IEEE Robotics and Automation Letters, vol. 4, no. 2, pp. 1626–1632, 2019 on Pith
  14. [34] A. Kumar, Z. Fu, D. Pathak, and J. Malik, “RMA: Rapid motor adaptation for legged robots,” inRobotics: Science and Systems, 2021 on Pith
  15. [35] T.-Y . Yang, T. Zhang, L. Luu, S. Ha, J. Tan, and W. Yu, “Safe rein- forcement learning for legged locomotion,” inIEEE/RSJ International Conference on Intelligent Robots and Systems, 2022, pp. 2454–2461 on Pith
  16. [36] C. Zhang, J. Bai, Z. Chen, J. Gao, and H. Qiao, “Hierarchical control for robust standing stability and fall recovery of task-performing humanoid robots,”IEEE Transactions on Automation Science and Engineering, vol. 22, pp. 20 920–20 933, 2025 on Pith
  17. [37] J. Y . Wong,Theory of Ground Vehicles, 3rd ed., 2001 on Pith
  18. [38] S. Gangapurwala, M. Geisert, R. Orsolino, M. Fallon, and I. Havoutis, “RLOC: Terrain-aware legged locomotion using reinforcement learning and optimal control,”IEEE Transactions on Robotics, vol. 38, no. 5, pp. 2908–2927, 2022 on Pith
  19. [39] L. Pinto, M. Andrychowicz, P. Welinder, W. Zaremba, and P. Abbeel, “Asymmetric actor critic for image-based robot learning,” inRobotics: Science and Systems, 2018 on Pith
  20. [40] G. Gilardi and I. Sharf, “Literature survey of contact dynamics mod- elling,”Mechanism and Machine Theory, vol. 37, no. 10, pp. 1213–1239, 2002 on Pith
  21. [41] D. W. Marhefka and D. E. Orin, “A compliant contact model with non- linear damping for simulation of robotic systems,”IEEE Transactions on Systems, Man, and Cybernetics – Part A: Systems and Humans, vol. 29, no. 6, pp. 566–572, 1999 on Pith
  22. [42] A. M. Castro, A. Qu, N. Kuppuswamy, A. Alspach, and M. Sherman, “A transition-aware method for the simulation of compliant contact with regularized friction,”IEEE Robotics and Automation Letters, vol. 5, no. 2, pp. 1859–1866, 2020 on Pith
  23. [43] K. Terzaghi,Theoretical Soil Mechanics, 1943 on Pith
  24. [44] Y . Bengio, J. Louradour, R. Collobert, and J. Weston, “Curriculum learning,” inInternational Conference on Machine Learning, 2009, pp. 41–48 on Pith
  25. [45] Y . Zhou, J. Qiu, S. Jia, F. Ni, and W. Zhang, “Curriculum-enhanced rein- forcement learning for robust humanoid locomotion,”IEEE Transactions on Automation Science and Engineering, vol. 23, pp. 5779–5789, 2026 on Pith
  26. [46] J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Prox- imal policy optimization algorithms,”arXiv preprint arXiv:1707.06347, 2017 on PithPith's reviewarXiv
  27. [47] J. Tobin, R. Fong, A. Ray, J. Schneider, W. Zaremba, and P. Abbeel, “Domain randomization for transferring deep neural networks from simulation to the real world,” inIEEE/RSJ International Conference on Intelligent Robots and Systems, 2017, pp. 23–30 on Pith
  28. [48] X. B. Peng, M. Andrychowicz, W. Zaremba, and P. Abbeel, “Sim-to- real transfer of robotic control with dynamics randomization,” inIEEE International Conference on Robotics and Automation, 2018, pp. 3803– 3810 on Pith
  29. [49] Y . Kadokawa, S. M. Alcantara Tacora, T. Abe, D. Endo, G. Yamauchi, T. Hashimoto, and T. Matsubara, “Autonomous obstacle removal for excavators through policy learning with particle simulation,”arXiv preprint arXiv:2606.09183, 2026 on PithPith's reviewarXiv
  30. [50] P. Fankhauser, M. Bloesch, D. Rodriguez, R. Kaestner, and M. Hutter, “Probabilistic terrain mapping for mobile robots with uncertain localiza- tion,”IEEE Robotics and Automation Letters, vol. 3, no. 4, pp. 3019– 3026, 2018 on Pith

Challenges and discussion

Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.

No comments yet.

Sign in to comment