Pith. sign in

REVIEW 4 major objections 6 minor 44 references

Learned Perceptive Forward Dynamics Model for Safe and Platform-aware Robotic Navigation

T0 review · 4 major / 6 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that a learned perceptive forward dynamics model can replace hand-tuned traversability costs for legged-robot navigation, predicting five seconds of future poses and failure risk from a height scan and proprioceptive…

desk verdict Solid incremental extension of Kim et al.'s learned FDM, but the headline 27% success-rate gain rests on a 2D-trained baseline and no variance estimates. read the letter →

arxiv 2504.19322 v2 pith:VZZUWQBU submitted 2025-04-27 cs.RO

Pascal Roth , Jonas Frey , Cesar Cadena , Marco Hutter This is my paper · ORCID
classification cs.RO
keywords forwarddynamicsmodelquadrupedalnavigationMPPIplanningterraintraversabilitysim-to-realtransferperceptivepredictionfailureriskrough
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Rough-terrain navigation usually separates traversability assessment from motion planning and requires hand-tuned cost functions. This paper tries to collapse both into one learned forward dynamics model: given a height scan, a history of poses and proprioceptive readings, and a sequence of velocity commands, the model predicts where the robot will be and how likely it is to fail over the next five seconds. The authors claim that this perceptive prediction makes a sampling-based MPPI planner work with only a goal-distance reward and a risk penalty, with no environment-specific cost tuning, and that training on simulated plus real data transfers to a real quadruped. Their reported numbers on ANYmal are a 41% average reduction in final position error over competitive baselines and a 27% higher navigation success rate in rough simulated environments.

What carries the argument

The load-bearing object is the perceptive forward dynamics model itself, a compact recurrent network of about 1.16 million parameters. A GRU encoder compresses the past-state and proprioceptive history while a small CNN compresses the height scan; their embeddings initialize the hidden state of a second GRU that rolls the action sequence forward. Two heads decode, at each step, a velocity correction $\Delta\tilde{a}$ and a failure probability $\tilde{r}$; the velocity corrections are integrated into poses, and a "stop loss" keeps predicted poses constant once a failure is predicted. On the planning side, the central mechanism is the replacement of hand-designed traversability costs by the model's implicit risk output, so MPPI's reward is $R = \lambda_{\text{pose}} R_{\text{pose}} + \lambda_{\text{risk}} R_{\text{risk}}$ with a terminal goal-distance term and a cumulative neighboring-path risk penalty.

What would settle it

Run the trained FDM through a course where a low obstacle is hidden behind a ridge for the first two prediction steps and only becomes visible from the robot's onboard camera later; if the predicted poses keep passing through the obstacle with low risk, the single-scan assumption fails. A quantitative version would measure rollout error against a variant that re-encodes the height scan at every step.

Watch

Extended reading notes

Core claim

The paper's central claim is that a learned perceptive forward dynamics model can stand in for both a dynamics simulator and a traversability classifier in local planning. At time $t$, the model receives the current height scan $h_t$, a history of $n$ past states and proprioceptive measurements, and a planned action sequence; a recurrent network predicts the future poses $\tilde{p}_{t+1},\dots,\tilde{p}_{t+n}$ and failure risks $\tilde{r}_{t+1},\dots,\tilde{r}_{t+n}$. The pose prediction is done in residual form: the network outputs velocity corrections that are integrated with a constant-velocity model into $\mathrm{SE}(2)$ poses, and the risk head is trained with binary cross-entropy. The authors integrate this model into an MPPI planner whose reward is just a terminal goal-distance term and a risk penalty. They report final-step position errors of 0.28 m in 2D, 0.30 m in 2D-3D and 0.28 m in 3D environments versus 0.37--0.45 m for the perceptive baseline, a 41.28% average error reduction, and planning success rates of 88.33% (2D) and 73.75% (3D) versus 78.33% and 48.75% for the baseline FDM planner.

Load-bearing premise

The height scan is encoded once at the start of the rollout, so the model must assume that a single static, partly occluded view of the terrain contains all geometry relevant to the next five seconds of motion.

Editorial extensions

If this is right

  • If the claim is right, sampling-based planners for legged robots can be deployed in new rough environments by retraining or fine-tuning one FDM instead of tuning traversability cost maps.
  • A failure-risk estimate learned from data, rather than from a geometric heuristic, should transfer across obstacle shapes such as walls, stairs, and ramps without per-obstacle thresholds.
  • The same architecture and training recipe can be reused for different embodiments: the paper demonstrates platform-aware predictions for a quadruped, a wheeled-legged platform, and another quadruped under the same action sequence.
  • Real-world fine-tuning becomes a small-data correction step: adding safe human-guided trajectories from snow, forest, or pavement cuts position error by about 30% in those domains.
  • Because the planner still samples and scores many trajectories, planning behavior can be adjusted at deployment by changing reward weights, noise, and command ranges without retraining the model.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the height scan enters only once, the model treats the world as static for five seconds; an obvious extension is to re-feed perception at every prediction step, which would also let the planner handle obstacles revealed as the robot moves.
  • The failure-risk head could double as an uncertainty or curiosity signal for data collection: a planner could seek out trajectories where predicted risk is high but not catastrophic, actively closing the model's blind spots.
  • The same residual velocity-integration design could be embedded as a differentiable world model for model-based policy optimization or for refining physics simulators, consequences the paper mentions only as future work.
  • A direct test of the framework's limits would be navigation among moving agents, since the current observations and failure labels assume static geometry; the paper states it has not tested this regime.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes a learned perceptive Forward Dynamics Model (FDM) for quadrupedal robots, predicting future SE2 poses and failure risk over a 5-second horizon from proprioceptive histories, velocity actions, and a current height scan. The model is pre-trained on large-scale simulated data, optionally fine-tuned with real-world data, and integrated into an MPPI planner using a two-term reward composed of pose error and predicted failure risk. Experiments on ANYmal, Barry, and ANYmal-on-Wheels report improved position prediction over the Kim et al. baseline and constant-velocity assumption, higher planning success in 2D and 3D simulation, and qualitative real-world transfers. The headline quantitative claims are a 41.28% average reduction in final-step position error and a 27% higher navigation success rate in rough simulation environments.

Significance. If the claims hold, this is a valuable contribution to legged navigation: it demonstrates that a learned perceptive dynamics model can replace hand-designed traversability costs in sampling-based planning, and the authors provide public code and models, modality ablations in Appendix D, platform-aware experiments across three robots, and a real-world fine-tuning study. The 41% position-error improvement in Table II is well supported because the Kim et al. baseline is retrained on the same data for that experiment. However, the downstream 27% success-rate claim is currently supported by an unequal planning comparison and lacks statistical confidence, so the headline result is not yet fully established.

major comments (4)
  1. [§VI-E and Table III] The headline downstream claim—27% higher navigation success in rough environments—rests on an unfair comparison. The text explicitly states that the Kim et al. baseline 'was trained solely in a 2D environment,' while the proposed FDM uses full 3D data, so the 3D success-rate gap (73.75% vs. 48.75%) could be caused by training-distribution mismatch rather than by the proposed model. To support the claim, the authors should retrain the baseline on the same 3D data, as was done for the dynamics comparison in §VI-B, or clearly reframe the experiment as a domain-shift test and add a like-for-like baseline.
  2. [Table III] The planning results report success rates without any number of trials, starts, seeds, or confidence intervals. Given that the 2D success rates are 88.33%, 78.33%, and 82.50%, the 10-percentage-point difference between the proposed method and Kim et al. may not be significant; the same concern applies to the 3D numbers. The authors should report the number of runs per condition and provide repeated-run statistics, such as bootstrap confidence intervals or per-seed standard deviations.
  3. [§V-B and Fig. 2] The height scan ht is encoded once by the CNN and only initializes the hidden state of the forward prediction GRU; no observation is re-encoded during the 10-step, 5-second rollout. The model therefore assumes that the static initial height scan contains all geometric information needed for the entire prediction horizon, which is questionable in cluttered or occluded rough terrain where obstacles become visible only as the robot moves. This assumption should be explicitly acknowledged as a limitation and, ideally, tested with an ablation that re-injects observations at each step or with scenarios where later-visible obstacles are present.
  4. [§V-D and Appendix H] The abstract describes the approach as a 'heuristic-free solution' and the paper claims that the simplified cost formulation 'eliminates the need for extensive cost-tuning to ensure safety,' but Appendix H states that MPPI parameters such as noise magnitude, time correlation, reward scaling, and sampling space 'require carefully tuned parameters' in complex scenarios. This tension should be resolved by either softening the heuristic-free claim or quantifying the remaining tuning effort, for example with a sensitivity analysis of the MPPI hyperparameters.
minor comments (6)
  1. [Abstract and Table III] The phrase '27% higher navigation success rate' is ambiguous: in Table III the 3D gap is 25 percentage points (73.75% vs. 48.75%), which is not 27% higher in relative terms; the authors should specify whether they report relative improvement, percentage points, or an average over environments, and give the exact arithmetic.
  2. [Table III] The caption should define what counts as a successful run and how many runs per environment were executed, since success rate without a trial count is not interpretable.
  3. [Fig. 5 and Fig. 13] The y-axis label 'Position Delta (m)' with the text 'linear logarithmic' is confusing; the authors should state explicitly which scale is used and how the 95% quantile is displayed.
  4. [§V-A] Typo: 'Synethic Data Generation' should be 'Synthetic Data Generation.'
  5. [Fig. 2] Typo: 'Data-Colletion' should be 'Data Collection.'
  6. [Eq. (15)] The term 'neighboring paths' is used without a precise definition; the authors should define q and the neighborhood relation explicitly, since Appendix F refers to a 'neighboring filter parameter' that is not formally introduced.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the FDM is supervised from executed trajectories and evaluated on held-out data and external baselines.

full rationale

The paper's central claims are empirical rather than derivational. The FDM is trained with a supervised pose loss and a binary cross-entropy risk loss (Eqs. 9-10) using replay-buffer labels from executed simulated and real-world trajectories, and the same labels are never used as the evaluation output. Position accuracy is measured on held-out samples against Kim et al. [5] and a constant-velocity model, with the baseline retrained on the same data for the dynamics comparison (Sec. VI-B). Planning success (Table III) is measured by executing the MPPI planner in simulation, not by re-inserting the planner's own reward values into the reported metric. The failure-risk term contributes to planning, but its effect is tested by ablating it (Table VI), so it is not assumed by construction. Self-citations such as [5] for time-correlated command sampling, [30] as a heuristic baseline, [37] for simulator identification, and [40] for noise augmentation support implementation details rather than load-bearing derivations. The limitation sections (Sec. VII) honestly acknowledge domain constraints and residual tuning. The only substantive concern raised by the skeptic—that the planning baseline of Kim et al. was trained only in 2D in Sec. VI-E—is a fairness-of-comparison issue, not a circularity issue, since neither the baseline nor the FDM's own results are constructed from its own outputs. No equation-level reduction from outputs back to inputs was found.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the observation model (static height scan, proprioceptive history), the simulator-to-real transfer assumption, and a small set of hand-tuned planner and loss hyperparameters that are not fully specified. No new physical entities are introduced.

free parameters (4)
  • Prediction horizon n and timesteps delta_tp, delta_th = n=10, delta_tp=0.5s, delta_th=0.05s
    The model predicts 10 states over 5s; these hand-chosen design decisions determine the problem difficulty and affect all reported errors.
  • MPPI reward weights lambda_pose, lambda_risk, thresholds delta_pose, delta_risk, lambda_pull = not reported in text
    The planner reward in Sec. V-D depends on these weights and thresholds; the paper says MPPI parameters were tuned for each baseline but does not give the values used for the proposed method, making the results hard to reproduce.
  • FDM loss weights epsilon_pose, epsilon_risk, epsilon_stop = not reported
    The combined loss in Eq. 12 uses these weights, presumably tuned by hand; no values are given in the paper.
  • MPPI population size C and iterations k = C=2048, planner at 7 Hz
    These control planning compute and stochasticity and directly affect success rate and runtime.
assumptions (4)
  • domain assumption The static height scan h_t is sufficient to predict the full n-step future trajectory; the model does not re-encode observations during the rollout.
    Entered in the model architecture (Sec. V-B and Fig. 2): the height scan only initializes the prediction GRU hidden state, so the network cannot adapt to newly visible obstacles.
  • domain assumption Simulation (IsaacLab) provides a sufficiently accurate model of robot-terrain interaction for pre-training, and real-world fine-tuning covers the residual gap.
    The claim of effective sim-to-real transfer (Sec. VI-D) rests on this; the paper itself notes in Sec. V-A that simulation misses effects like snow and vegetation entanglement, which motivated real data.
  • domain assumption The observation space (proprioceptive history, height scan, commands) is sufficient to determine the robot's future state and failure risk.
    The POMDP approximation in Sec. III-A and IV-A assumes these observations capture the relevant state; the ablations in Appendix D show all modalities matter, but there is no guarantee of sufficiency in unseen environments.
  • domain assumption Ground-truth pose labels are accurate: simulator state in simulation, fusion of dual RTK-GNSS and total station in real data.
    The supervised losses (Eq. 9) use these labels; any bias or noise in the labels transfers into the model. The paper describes the real-world setup in Sec. V-A but does not report label uncertainty.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Learned Perceptive Forward Dynamics Model for Safe and Platform-aware Robotic Navigation." pith.science (2026). https://pith.science/paper/VZZUWQBU

@misc{pith2026250419322,
  author       = {Pith},
  title        = {Pith review of: Learned Perceptive Forward Dynamics Model for Safe and Platform-aware Robotic Navigation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VZZUWQBU}},
  note         = {Machine review of arXiv:2504.19322}
}
read the original abstract

Ensuring safe navigation in complex environments requires accurate real-time traversability assessment and understanding of environmental interactions relative to the robot`s capabilities. Traditional methods, which assume simplified dynamics, often require designing and tuning cost functions to safely guide paths or actions toward the goal. This process is tedious, environment-dependent, and not generalizable. To overcome these issues, we propose a novel learned perceptive Forward Dynamics Model (FDM) that predicts the robot`s future state conditioned on the surrounding geometry and history of proprioceptive measurements, proposing a more scalable, safer, and heuristic-free solution. The FDM is trained on multiple years of simulated navigation experience, including high-risk maneuvers, and real-world interactions to incorporate the full system dynamics beyond rigid body simulation. We integrate our perceptive FDM into a zero-shot Model Predictive Path Integral (MPPI) planning framework, leveraging the learned mapping between actions, future states, and failure probability. This allows for optimizing a simplified cost function, eliminating the need for extensive cost-tuning to ensure safety. On the legged robot ANYmal, the proposed perceptive FDM improves the position estimation by on average 41% over competitive baselines, which translates into a 27% higher navigation success rate in rough simulation environments. Moreover, we demonstrate effective sim-to-real transfer and showcase the benefit of training on synthetic and real data. Code and models are made publicly available under https://github.com/leggedrobotics/fdm.

Figures

Figures reproduced from arXiv: 2504.19322 by the authors.

Figure 1
Figure 1. Demonstration of the proposed perceptive Forward Dynamics Model for robust navigation in complex environments. The model, trained with real-world and simulation data, predicts the robot’s future states given a sequence of velocity actions. It takes as input the surrounding geometry in the form of a height scan, along with past states and proprioceptive measurements. A sampling-based planner evaluates the integrated … view at source ↗
Figure 2
Figure 2. Overview of the FDM training. Data is collected in a parallelized simulation setting and from real-world experiments. The proprioceptive and exteroceptive measurements, along with velocity actions, are saved in a replay buffer from which training data is sampled. The information about the current and past state of the robotic system is encoded and given to a recurrent unit, which generates a latent of the robot’s fu… view at source ↗
Figure 3
Figure 3. Overview of the MPPI-based planning approach. A population of action trajectories is generated by perturbating an initial solution with Gaussian noise. The presented FDM is then used to predict the future states and the risk of the individual action sequences, which are evaluated using a reward formulation. After k iterations, using the previous highest reward action sequence as a starting point, the sequence with t… view at source ↗
Figures from the paper (11 more)
Figure 4
Figure 4. Figure 4: Demonstration of environment- and platform-aware state predictions using the presented FDM. Collision-free predictions of our method are displayed in , in collision ones in , whereas the actual path is presented in . (a) Simulation: The same four action sequences are r…
Figure 5
Figure 5. Figure 5: Comparison of the position error at the final prediction step in different environments for the presented FDM , the perceptive FDM by Kim et al. [5] and the constant velocity model . For each environment, 50k samples are evaluated, and the error is displayed up to the …
Figure 6
Figure 6. Figure 6: Comparison of the position error over the prediction steps between the presented method, the perceptive FDM by Kim et al. [5], and the constant velocity model. Our FDM demonstrates the highest accuracy with the lowest errors and smallest standard deviation. B. Baseline…
Figure 7
Figure 7. Figure 7: Comparison of state predictions of the presented method on the quadrupedal platforms ANYmal [41], Barry [42], and ANYmal￾On-Wheels (AoW) [43]. For Barry, a robust locomotion policy capable of traversing rough environments and a ”quiet” locomotion policy optimized for m…
Figure 8
Figure 8. Figure 8: Comparison of the position error at two prediction steps in real-world environments. Shown is the presented method , trained only with simulated data and fine-tuned with real-world data and the constant velocity model . The presented method can already bridge successfu…
Figure 9
Figure 9. Figure 9: Demonstration of the pose and failure rewards across various simulation scenarios. The proposed FDM accurately predicts failures due to collisions and early path terminations caused by untraversable stairs and ramps. As a result, the simple combination of a pose reward…
Figure 10
Figure 10. Figure 10: The simulation training environment consists of four distinct segments. The first segment features a randomized mix of stairs, ramps, walls, and rough surfaces. The second contains obstacles arranged in structured patterns. The third includes pillars of varying dimens…
Figure 11
Figure 11. Figure 11: Combined visualization of the height scan and traversability estimates generated by the heuristics-based method of [26] for four environments. It is visible that stairs often have very low traversability scores, even if they should be traversable. Nevertheless, the sc…
Figure 12
Figure 12. Figure 12: Comparison of the position error over the prediction steps between the presented method , the perceptive FDM by Kim et al. [5] , and the constant velocity model for the different environments. Our FDM demonstrates the highest accuracy and smallest standard deviation a…
Figure 13
Figure 13. Figure 13: Comparison of position error at the final prediction step across different environments for the presented FDM , the perceptive FDM by Kim et al. [5] , and the constant velocity model . For each environment, 50k samples are evaluated, and the error is shown up to the 9…
Figure 14
Figure 14. Figure 14: Additional real-world demonstration of environment- and platform-aware state predictions using the presented FDM in comparison to constant velocity estimation for the same action sequences across multiple scenarios. Collision-free predictions of our method are display…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

44 extracted references · 34 canonical work pages

  1. [1]

    Theodorou

    Jason Gibson, Bogdan Vlahov, David Fan, Patrick Spieler, Daniel Pastor, Ali-akbar Agha-mohammadi, and Evange- los A. Theodorou. A multi-step dynamics modeling frame- work for autonomous driving in multiple environments. In 2023 IEEE International Conference on Robotics and Automation (ICRA) , pages 7959–7965, 2023. doi: 10.1109/ICRA48891.2023.10161330

    arXiv 2023

  2. [2]

    Learning inverse kinodynamics for accurate high-speed off-road navigation on unstructured terrain

    Xuesu Xiao, Joydeep Biswas, and Peter Stone. Learning inverse kinodynamics for accurate high-speed off-road navigation on unstructured terrain. IEEE Robotics and Automation Letters, 6(3):6054–6060, 2021

    2021

  3. [3]

    Cahsor: Competence-aware high-speed off- road ground navigation in se (3)

    Anuj Pokhrel, Aniket Datar, Mohammad Nazeri, and Xuesu Xiao. Cahsor: Competence-aware high-speed off- road ground navigation in se (3). IEEE Robotics and Automation Letters, 2024

    work page 2024

  4. [4]

    Dynamics modeling using visual terrain features for high-speed autonomous off-road driving

    Jason Gibson, Anoushka Alavilli, Erica Tevere, Evange- los A Theodorou, and Patrick Spieler. Dynamics modeling using visual terrain features for high-speed autonomous off-road driving. arXiv preprint arXiv:2412.00581 , 2024

    arXiv 2024

  5. [5]

    Learning forward dynamics model and informed trajectory sampler for safe quadruped navigation

    Yunho Kim, Chanyoung Kim, and Jemin Hwangbo. Learning forward dynamics model and informed trajectory sampler for safe quadruped navigation. In Robotics: Science and Systems (RSS 2022) , 2022

    work page 2022

  6. [6]

    Duong, Abdullah Altawaitan, Jason Stanley, and Nikolay Atanasov

    Thai P. Duong, Abdullah Altawaitan, Jason Stanley, and Nikolay Atanasov. Port-hamiltonian neural ode networks on lie groups for robot dynamics learning and control. IEEE Transactions on Robotics , 40:3695–3715, 2024

    work page 2024

  7. [7]

    Badgr: An autonomous self-supervised learning-based navigation system

    Gregory Kahn, Pieter Abbeel, and Sergey Levine. Badgr: An autonomous self-supervised learning-based navigation system. IEEE Robotics and Automation Letters , 6(2): 1312–1319, 2021

    work page 2021

  8. [8]

    Combining physics and deep learning to learn continuous-time dynamics models

    Michael Lutter and Jan Peters. Combining physics and deep learning to learn continuous-time dynamics models. The International Journal of Robotics Research , 42(3): 83–107, 2023

    work page 2023

Show all 44 references
  1. [9]

    Lagrangian neural networks

    Miles Cranmer, Sam Greydanus, Stephan Hoyer, Peter Battaglia, David Spergel, and Shirley Ho. Lagrangian neural networks. Proc. ICLR Workshop Integration Deep Neural Models Differ . Equ., 2020

    2020

  2. [10]

    Modeling system dynamics with physics-informed neural networks based on lagrangian mechanics

    Manuel A Roehrl, Thomas A Runkler, Veronika Brandt- stetter, Michel Tokic, and Stefan Obermayer. Modeling system dynamics with physics-informed neural networks based on lagrangian mechanics. IF AC-PapersOnLine, 53 (2):9195–9200, 2020

    2020

  3. [11]

    Bridging active exploration and uncertainty-aware deployment using probabilistic ensemble neural network dynamics

    Taekyung Kim, Jungwi Mun, Junwon Seo, Beomsu Kim, and Seongil Hong. Bridging active exploration and uncertainty-aware deployment using probabilistic ensemble neural network dynamics. In Robotics: Science and Systems (RSS 2023) , 2023

    2023

  4. [12]

    Learning terrain-aware kinodynamic model for autonomous off-road rally driving with model predictive path integral control

    Hojin Lee, Taekyung Kim, Jungwi Mun, and Wonsuk Lee. Learning terrain-aware kinodynamic model for autonomous off-road rally driving with model predictive path integral control. IEEE Robotics and Automation Letters, 2023

    2023

  5. [13]

    Pilco: A model- based and data-efficient approach to policy search

    Marc Deisenroth and Carl E Rasmussen. Pilco: A model- based and data-efficient approach to policy search. In Proceedings of the 28th International Conference on machine learning (ICML-11) , pages 465–472, 2011

    2011

  6. [14]

    Learning neural net- work policies with guided policy search under unknown dynamics

    Sergey Levine and Pieter Abbeel. Learning neural net- work policies with guided policy search under unknown dynamics. Advances in neural information processing systems, 27, 2014

    2014

  7. [15]

    Deep reinforcement learning in a handful of trials using probabilistic dynamics models

    Kurtland Chua, Roberto Calandra, Rowan McAllister, and Sergey Levine. Deep reinforcement learning in a handful of trials using probabilistic dynamics models. Advances in neural information processing systems , 31, 2018

    2018

  8. [16]

    Pose prediction for mobile ground robots in uneven terrain based on difference of heightmaps

    Stefan Fabian, Stefan Kohlbrecher, and Oskar V on Stryk. Pose prediction for mobile ground robots in uneven terrain based on difference of heightmaps. In 2020 IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR) , pages 49–56. IEEE, 2020

    2020

  9. [17]

    Navigation world models

    Amir Bar, Gaoyue Zhou, Danny Tran, Trevor Darrell, and Yann LeCun. Navigation world models. arXiv preprint arXiv:2412.03572, 2024

    2024 arXiv

  10. [18]

    Identifying terrain physical parameters from vision-towards physical-parameter-aware locomotion and navigation

    Jiaqi Chen, Jonas Frey, Ruyi Zhou, Takahiro Miki, Georg Martius, and Marco Hutter. Identifying terrain physical parameters from vision-towards physical-parameter-aware locomotion and navigation. IEEE Robotics and Automa- tion Letters , 2024

    2024

  11. [19]

    Dream to control: Learning behaviors by latent imagination

    Danijar Hafner, Timothy Lillicrap, Jimmy Ba, and Mo- hammad Norouzi. Dream to control: Learning behaviors by latent imagination. In International Conference on Learning Representations, 2020

    2020

  12. [20]

    Td-mpc2: Scalable, robust world models for continuous control

    Nicklas Hansen, Hao Su, and Xiaolong Wang. Td-mpc2: Scalable, robust world models for continuous control. In International Conference on Learning Representations (ICLR), 2024

    2024

  13. [21]

    Nomad: Goal masked diffusion policies for navigation and exploration

    Ajay Sridhar, Dhruv Shah, Catherine Glossop, and Sergey Levine. Nomad: Goal masked diffusion policies for navigation and exploration. In 2024 IEEE International Conference on Robotics and Automation (ICRA) , pages 63–70. IEEE, 2024

    2024

  14. [22]

    Risk-predictive planning for off-road autonomy

    Lukas Lao Beyer, Gilhyun Ryou, Patrick Spieler, and Sertac Karaman. Risk-predictive planning for off-road autonomy. In 2024 IEEE International Conference on Robotics and Automation (ICRA) , pages 16452–16458. IEEE, 2024

    2024

  15. [23]

    Terrainnet: Visual modeling of complex terrain for high-speed, off- road navigation

    Xiangyun Meng, Nathan Hatch, Alexander Lambert, Anqi Li, Nolan Wagener, Matthew Schmittle, JoonHo Lee, Wentao Yuan, Zoey Chen, Samuel Deng, et al. Terrainnet: Visual modeling of complex terrain for high-speed, off- road navigation. In Robotics: Science and Systems (RSS 2023), 2023

    2023

  16. [24]

    Fast traversability estimation for wild visual navigation

    Jonas Frey, Matias Eduardo Mattamala Aravena, Nived Chebrolu, Cesar Cadena, Maurice Fallon, and Marco Hutter. Fast traversability estimation for wild visual navigation. Proceedings of Robotics: Science and System XIX, page p054, 2023

    2023

  17. [25]

    How does it feel? self-supervised costmap learning for off-road vehicle traversability

    Mateo Guaman Castro, Samuel Triest, Wenshan Wang, Jason M Gregory, Felix Sanchez, John G Rogers, and Sebastian Scherer. How does it feel? self-supervised costmap learning for off-road vehicle traversability. In 2023 IEEE International Conference on Robotics and Automation (ICR...

    2023

  18. [26]

    Artplanner: Robust legged robot navigation in the field

    Lorenz Wellhausen and Marco Hutter. Artplanner: Robust legged robot navigation in the field. Field Robotics, 3(1): 413–434, 2023

    2023

  19. [27]

    An efficient locally reactive controller for safe navigation in visual teach and repeat missions

    Matias Mattamala, Nived Chebrolu, and Maurice Fallon. An efficient locally reactive controller for safe navigation in visual teach and repeat missions. IEEE Robotics and Automation Letters, 7(2):2353–2360, 2022

    2022

  20. [28]

    Learning ground traversability from simulations

    R Omar Chavez-Garcia, J ´erˆome Guzzi, Luca M Gam- bardella, and Alessandro Giusti. Learning ground traversability from simulations. IEEE Robotics and Automation letters, 3(3):1695–1702, 2018

    2018

  21. [29]

    Roadrunner m&m - learning multi-range multi-resolution traversability maps for autonomous off-road navigation

    Manthan Patel, Jonas Frey, Deegan Atha, Patrick Spieler, Marco Hutter, and Shehryar Khattak. Roadrunner m&m - learning multi-range multi-resolution traversability maps for autonomous off-road navigation. IEEE Robotics and Automation Letters, 9(12):11425–11432, 2024. doi: 10. 1...

    2024

  22. [30]

    Rough terrain navigation for legged robots using reachability planning and template learning

    Lorenz Wellhausen and Marco Hutter. Rough terrain navigation for legged robots using reachability planning and template learning. In 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , pages 6914–6921. IEEE, 2021

    2021

  23. [31]

    Information theoretic mpc for model-based reinforcement learning

    Grady Williams, Nolan Wagener, Brian Goldfain, Paul Drews, James M Rehg, Byron Boots, and Evangelos A Theodorou. Information theoretic mpc for model-based reinforcement learning. In 2017 IEEE international conference on robotics and automation (ICRA) , pages 1714–1721. IEEE, 2017

    2017

  24. [32]

    Sample-efficient cross-entropy method for real-time planning

    Cristina Pinneri, Shambhuraj Sawant, Sebastian Blaes, Jan Achterhold, Joerg Stueckler, Michal Rolinek, and Georg Martius. Sample-efficient cross-entropy method for real-time planning. In Conference on Robot Learning 2020, 2020

    2020

  25. [33]

    Viplanner: Visual semantic imperative learning for local navigation

    Pascal Roth, Julian Nubert, Fan Yang, Mayank Mittal, and Marco Hutter. Viplanner: Visual semantic imperative learning for local navigation. In 2024 IEEE International Conference on Robotics and Automation (ICRA) , pages 5243–5249. IEEE, 2024

    2024

  26. [34]

    iplanner: Imperative path planning

    Fan Yang. iplanner: Imperative path planning. In Robotics: Science and Systems (RSS 2023) , 2023

    2023

  27. [35]

    Learn- ing robust autonomous navigation and locomotion for wheeled-legged robots

    Joonho Lee, Marko Bjelonic, Alexander Reske, Lorenz Wellhausen, Takahiro Miki, and Marco Hutter. Learn- ing robust autonomous navigation and locomotion for wheeled-legged robots. Science Robotics , 9(89):eadi9641, 2024

    2024

  28. [36]

    Learning a state representation and navigation in cluttered and dynamic environments

    David Hoeller, Lorenz Wellhausen, Farbod Farshidian, and Marco Hutter. Learning a state representation and navigation in cluttered and dynamic environments. IEEE Robotics and Automation Letters , 6(3):5081–5088, 2021

    2021

  29. [37]

    Learning agile and dynamic motor skills for legged robots

    Jemin Hwangbo, Joonho Lee, Alexey Dosovitskiy, Dario Bellicoso, Vassilios Tsounis, Vladlen Koltun, and Marco Hutter. Learning agile and dynamic motor skills for legged robots. Science Robotics , 4(26):eaau5872, 2019

    2019

  30. [38]

    Boxi: Design Decisions in the Context of Algorithmic Performance for Robotics

    Jonas Frey, Turcan Tuna, Lanke Frank Tarimo Fu, Cedric Weibel, Katharine Patterson, Benjamin Krummenacher, Matthias M ¨uller, Julian Nubert, Maurice Fallon, Cesar Cadena, and Marco Hutter. Boxi: Design Decisions in the Context of Algorithmic Performance for Robotics. In Procee...

    2025

  31. [39]

    Holistic fusion: Task-and setup-agnostic robot localization and state estimation with factor graphs

    Julian Nubert, Turcan Tuna, Jonas Frey, Cesar Cadena, Katherine J Kuchenbecker, Shehryar Khattak, and Marco Hutter. Holistic fusion: Task-and setup-agnostic robot localization and state estimation with factor graphs. arXiv preprint arXiv:2504.06479, 2025

    2025 arXiv

  32. [40]

    Learning to walk in minutes using massively parallel deep reinforcement learning

    Nikita Rudin, David Hoeller, Philipp Reist, and Marco Hutter. Learning to walk in minutes using massively parallel deep reinforcement learning. In Conference on Robot Learning , pages 91–100. PMLR, 2022

    2022

  33. [41]

    Anymal-a highly mobile and dynamic quadrupedal robot

    Marco Hutter, Christian Gehring, Dominic Jud, Andreas Lauber, C Dario Bellicoso, Vassilios Tsounis, Jemin Hwangbo, Karen Bodie, Peter Fankhauser, Michael Bloesch, et al. Anymal-a highly mobile and dynamic quadrupedal robot. In 2016 IEEE/RSJ international conference on intellig...

    2016

  34. [42]

    Barry: A high-payload and agile quadruped robot

    Giorgio Valsecchi, Nikita Rudin, Lennart Nachtigall, Konrad Mayer, Fabian Tischhauser, and Marco Hutter. Barry: A high-payload and agile quadruped robot. IEEE Robotics and Automation Letters , 8(11):6939–6946, 2023. doi: 10.1109/LRA.2023.3313923

    2023

  35. [43]

    Keep rollin’—whole-body motion control and planning for wheeled quadrupedal robots

    Marko Bjelonic, C Dario Bellicoso, Yvain de Viragh, Dhionis Sako, F Dante Tresoldi, Fabian Jenelten, and Marco Hutter. Keep rollin’—whole-body motion control and planning for wheeled quadrupedal robots. IEEE Robotics and Automation Letters , 4(2):2116–2123, 2019

    2019

  36. [44]

    Orbit: A unified simulation framework for in- teractive robot learning environments

    Mayank Mittal, Calvin Yu, Qinxi Yu, Jingzhou Liu, Nikita Rudin, David Hoeller, Jia Lin Yuan, Ritvik Singh, Yunrong Guo, Hammad Mazhar, Ajay Mandlekar, Buck Babich, Gavriel State, Marco Hutter, and Animesh Garg. Orbit: A unified simulation framework for in- teractive robot lear...

    2023

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.