Pith. sign in

REVIEW 4 major objections 5 minor 38 references

Re4MPC: Reactive Nonlinear MPC for Multi-model Motion Planning via Deep Reinforcement Learning

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Reactive, learned switching among robot models for NMPC raises a mobile-manipulation task's success rate from 61% to 86% while cutting per-solve compute time.

desk verdict A useful engineering contribution with a real confound: the efficiency and success gains are partly written into the reward, and end-to-end compute is never measured. read the letter →

arxiv 2506.08344 v1 pith:QJO6FQXU submitted 2025-06-10 cs.RO cs.AIcs.LGcs.SYeess.SY

Neşet Ünver Akmandor , Sarvesh Prajapati , Mark Zolotas , Taşk{i}n Pad{i}r This is my paper
classification cs.ROcs.AIcs.LGcs.SYeess.SY
keywords reactivemotionplanningnonlinearmodelpredictivecontrolmulti-modeldeepreinforcementlearningmobilemanipulationsequentiallinearquadraticcomputationalefficiencywhole-body
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a motion planner's compute cost and success rate both improve when a learned policy is allowed to choose, at each decision step, which robot model the NMPC trajectory optimizer should use, and what sub-target to aim for. The target robot is a wheeled mobile manipulator, and the choice is among base-only, arm-only, and whole-body kinematic models. In a physics-based simulation with a stationary conveyor and a box, the best trained variant succeeds at 86% of 108 test configurations versus 61% for a whole-body-only NMPC baseline, while per-solve compute times drop when simpler models are selected. If this result holds beyond the tested task, it points to a template for scaling NMPC to high-DOF robots by spending solver effort only where it is needed.

What carries the argument

The central mechanism is the parameterized action space $A_c$ (and its discrete counterpart $A_d$), which encodes three sub-actions: model selection $a_{model} \in [0,1]$ mapped to one of three kinematic models via thresholds, constraint selection $a_{constraint} \in [0,1]^{N_c}$ gated at 0.5, and target selection $a_{target} \in \mathbb{R}^7$ that picks either a sub-goal or the goal pose. These decoded settings define the NMPC problem in Eq. 1, which is solved by Sequential Linear Quadratic (SLQ), a DDP-based optimizer with relaxed barrier functions for constraints. The reward function includes the DOF-proportional model penalty, a goal-distance reward, and a target reward that encourages sub-goals for non-arm modes; together these terms drive the policy to prefer cheaper models when they suffice.

What would settle it

Retrain Re4MPC with the model penalty set to a constant regardless of the selected model's degrees of freedom, keeping everything else identical. If whole-body model usage stays high and per-solve compute time does not drop, then the reported computational gains come from the reward, not from the multi-model selection mechanism itself. Alternatively, run the trained policy in an environment with obstacles and goal positions outside the training distribution; if its success rate falls to the baseline level, the advantage is overfitting to the training layout.

Watch

Extended reading notes

Core claim

Re4MPC formulates motion planning as a partially observable Markov decision process in which the agent's action is decoded into the NMPC problem definition: the robot model index (base, arm, or whole body), the constraint set, and the end-effector target (either a goal or a sub-goal). The NMPC solver runs for an action horizon with those settings, and the resulting state transitions and rewards shape the policy. The paper reports that after 100k training steps, the SAC-trained variant reaches 86% success versus 61% for the whole-body baseline, with fewer rollovers and collisions, and that the learned policy uses the whole-body model only about 30% of the time in its successful episodes. The computational gain is attributed to a reward term that penalizes the active model in proportion to its degrees of freedom, together with a target reward that steers the agent toward feasible sub-goals.

Load-bearing premise

The load-bearing premise is that the hand-designed reward function — specifically the DOF-proportional model penalty and the target-reward thresholds — is what teaches the policy to choose simpler models, so if those weights do not transfer to new tasks or environments, the reported efficiency and success gains will not transfer either.

Editorial extensions

If this is right

  • Re4MPC reduces whole-body model usage in successful episodes to roughly 30% for the SAC variant, implying the learned policy discovers when full-body optimization is unnecessary.
  • Per-solve computation time for base-model planning is roughly half that of whole-body planning, so a policy that switches models can cut online compute while maintaining or improving task success.
  • All Re4MPC variants surpass the baseline success rate by 100k training timesteps, with SAC variants converging fastest.
  • The discrete action space filters infeasible target poses, leading to different model and target selection frequencies than the continuous action space, a design choice the paper links to learning behavior.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the model-penalty reward is indeed the driver, the same pipeline could be retargeted to other high-DOF systems such as humanoids or multi-arm platforms by swapping the model set while keeping the POMDP action structure; this extension is left implicit in the paper.
  • The dependency on hand-tuned reward weights and thresholds (for example $\tau_{target}$, $\delta_{com}$, and $\alpha_{sub}$) suggests a testable bound: performance should degrade gracefully as these parameters are perturbed, but no sensitivity analysis is provided.
  • A natural next experiment is to replace the learned policy with a hand-coded heuristic (e.g., use base-only when far from the goal and whole-body when close) and compare; if the heuristic matches Re4MPC, the contribution lies in the multi-model formulation rather than in the learning.
  • Because the training environment keeps the box fixed and the conveyor static, the reported gains may be specific to that layout; testing in randomized or dynamic scenes would clarify whether the policy learns true reactivity or a fixed switching schedule.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes Re4MPC, a framework that couples deep reinforcement learning (DRL) with nonlinear model predictive control (NMPC) for mobile manipulator motion planning. A DRL policy selects, at each action interval, one of three robot models (base, arm, or whole-body), the NMPC cost function and constraints, and a target (sub-goal or final goal). The framework is evaluated in a PyBullet-based simulation with a Jackal mobile base and a 6-DOF Kinova Jaco arm. The authors report that Re4MPC trained with PPO, SAC, or DQN achieves higher success rates than a whole-body NMPC baseline (ocs2wb), and that it computes trajectories more efficiently because it often selects lower-dimensional models.

Significance. If substantiated, the framework would be a useful contribution to integrated DRL/NMPC for mobile manipulation, and the open-source implementation (mobiman) is a clear strength. The mathematical formulation of integrating NMPC into a DRL loop is clearly presented. However, the current evidence does not establish the computational efficiency claim, and the attribution of gains to reactive model selection is confounded by the reward function's explicit DOF penalty. The success-rate advantage is plausible but lacks statistical support and has at least one internal contradiction.

major comments (4)
  1. [Section IV.B, Table I] The headline claim that Re4MPC is 'more computationally efficient' than the NMPC baseline is not supported by the reported metrics. Table I lists mean per-call SLQ solve times for each model and the number of NMPC calls per model, but it does not report total computation per episode, end-to-end wall-clock time, or the overhead of the learned policy (neural-network forward pass, action decoding, model switching, sub-goal computation). The paper itself notes that ocs2wb 'exhibits fewer total computations' because Re4MPC episodes tend to last longer. Without total computation or end-to-end time, lower per-call times for simpler models cannot establish the efficiency claim.
  2. [Section III.B.3, Table I] The reduced whole-body usage is at least partly enforced by design. The reward term R_model penalizes the selected model with a constant whose value is proportional to the model's DOF, directly incentivizing low-DOF choices. Presenting the resulting reduction in whole-body usage as evidence of Re4MPC's efficiency is therefore circular to some degree. An ablation that removes the DOF penalty from R_model, or a comparison against a policy that selects models without this reward shaping, is needed to attribute the efficiency gain to reactive model selection rather than to the reward design.
  3. [Section IV.B, Fig. 3, Table I] Success rates are reported as point estimates without error bars, confidence intervals, or statistical significance tests, even though each configuration is averaged over five runs. This makes it difficult to assess whether the observed differences over the baseline are meaningful, especially for variants with small differences. The paper should report per-run results, standard deviations, or a statistical test to support the claim of higher success rates.
  4. [Section IV.A] The baseline ocs2wb is not described in sufficient detail to ensure a fair comparison. It is stated to be a whole-body SLQ formulation with the goal as the MPC target, but the manuscript does not specify whether it uses the same cost function weights and constraints as Re4MPC when Re4MPC selects the whole-body model and goal target. Differences in cost tuning could confound the success-rate comparison.
minor comments (5)
  1. [Table I] Table I is hard to read: entries like '8684' and '01 2' appear to be two numbers concatenated without a separator. Please reformat the table for clarity.
  2. [Section IV.B] The statement that 'all variants achieve higher success rates' than the baseline is contradicted by Table I, where re4mpc-DQN-w/oRT shows a 60% success rate versus 61% for ocs2wb.
  3. [Section III.B.3] The numerical values of the reward constants (tau_success, tau_boundary, tau_collision, tau_roll, tau_max_step, tau_target) and thresholds (delta_g, delta_cd, delta_psi, delta_vartheta, delta_com) used in Eqs. (2)-(5) are not reported. Please provide these values in the paper or a clear pointer to the code.
  4. [Table I] The timing measurements in Table I lack a description of the compute platform. Please specify the hardware and software environment used for the timing experiments.
  5. [References] Some references appear incomplete; for example, [32] does not include publication venue or page numbers. Please ensure all references are complete.

Circularity Check

1 steps flagged · score 4.0 of 10

Efficiency conclusion partially restates the R_model reward objective; total-computation claim is not independently measured.

  1. self definitional [Section III.B.3 (Reward Function) and Section V (Conclusion)]
    "To discourage the robot from wandering about and to encourage the selection of efficient system models, we design R_model as a constant value penalty function where the penalty values are proportional to the DOF of the chosen system model. ... Therefore, Re4MPC converged to a computationally more efficient motion planning policy for the given task."

    R_model is defined as a penalty proportional to the DOF of the selected model, so maximizing the reward necessarily drives the policy away from whole-body (high-DOF) models. The conclusion then infers computational efficiency from this reduced whole-body usage, making that part of the efficiency evidence an artifact of the reward definition rather than an independently derived result. Table I does supply measured per-call solve times, which partially validate the DOF proxy; however, the paper never reports total per-episode computation, and it concedes that the ocs2wb baseline 'exhibits fewer total computations,' so the headline efficiency claim still leans on the reward-forced model-selection pattern.

full rationale

The success-rate comparison against the OCS2 whole-body baseline is an external benchmark and is not circular: the training reward shapes behavior, but success is measured by reaching the goal in simulation, independent of the reward's model-selection term. The main circularity concern is confined to the efficiency narrative. The reward R_model (Section III.B.3) explicitly penalizes high-DOF models, and the conclusion (Section V) cites reduced whole-body usage as evidence of computational efficiency. Since that reduction is the optimized consequence of R_model, treating it as validation of efficiency is partially self-referential. The per-call SLQ solve times in Table I provide independent support for the proxy, but they are reported per model only; no end-to-end time or total NMPC-call count per episode is given, and the paper admits the baseline has fewer total computations during the tests. Thus the abstract claim that Re4MPC is more computationally efficient than the NMPC baseline is partly forced by the reward design and partly unsupported by missing aggregate measurements. No load-bearing self-citation or imported uniqueness theorem is present; the only prior self-citation, [32], is a non-central example of an alternative observation representation.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The paper introduces no new physical entities. The central claim rests on a set of hand-tuned reward parameters, assumed kinematic models, and simulation fidelity. The most notable free parameters are the DOF-proportional model penalties, which directly shape the reported efficiency gains.

free parameters (6)
  • R_model penalty values per model = not reported
    Penalties proportional to DOF of selected model; directly incentivizes low-DOF selections and shapes the observed model-switching behavior (Section III.B.3).
  • Terminal reward constants (tau_success, tau_boundary, tau_collision, tau_roll, tau_max_step) = not reported
    Magnitudes of terminal rewards in Eq. 2; hand-tuned and not disclosed, affecting training convergence and policy behavior.
  • Thresholds delta_g, delta_cd, delta_psi, delta_vartheta, delta_com = not reported
    Goal tolerance, collision distance, rollover limits, CoM stability threshold; all hand-set, and their values determine success and failure criteria.
  • alpha_sub (sub-goal interpolation factor) = not reported
    Scales the interpolation between robot and goal for sub-goal pose; a heuristic that directly shapes target rewards (Eq. 4).
  • gamma (exponential scaling for target rewards) = not reported
    Scales R_base_target rewards in Eqs. 4-5; hand-tuned.
  • Empirical CoM position p*_com = not reported
    Reference center-of-mass trajectory used in R_arm_target (Eq. 3); derived empirically from the simulator rather than from a model.
assumptions (5)
  • domain assumption The observation space o is sufficient for the agent to choose good NMPC settings.
    Section III.B.1; the agent has no global position or full state, but the paper assumes the structured sensor features suffice for the task.
  • domain assumption The three kinematic models (base, arm, whole-body) adequately capture robot motion for planning.
    Section III.A.1; decoupled kinematic concatenation ignores dynamic coupling and wheel slip beyond the cited skid-steer model.
  • ad hoc to paper The reward function encodes the true objectives of successful motion planning.
    Section III.B.3; many constants and heuristics, including R_model and sub-goal interpolation, are chosen by hand and treated as ground truth.
  • domain assumption PyBullet/iGibson simulation faithfully represents the robot and environment.
    Section IV; evaluation never leaves simulation, so all success and computation claims rest on simulator fidelity.
  • standard math SLQ converges to a solution of Eq. 1 in the allowed iterations.
    Section III.A.2; the paper relies on the solver's local convergence guarantees without reporting cases of non-convergence.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Re4MPC: Reactive Nonlinear MPC for Multi-model Motion Planning via Deep Reinforcement Learning." pith.science (2026). https://pith.science/paper/QJO6FQXU

@misc{pith2026250608344,
  author       = {Pith},
  title        = {Pith review of: Re4MPC: Reactive Nonlinear MPC for Multi-model Motion Planning via Deep Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QJO6FQXU}},
  note         = {Machine review of arXiv:2506.08344}
}
read the original abstract

Traditional motion planning methods for robots with many degrees-of-freedom, such as mobile manipulators, are often computationally prohibitive for real-world settings. In this paper, we propose a novel multi-model motion planning pipeline, termed Re4MPC, which computes trajectories using Nonlinear Model Predictive Control (NMPC). Re4MPC generates trajectories in a computationally efficient manner by reactively selecting the model, cost, and constraints of the NMPC problem depending on the complexity of the task and robot state. The policy for this reactive decision-making is learned via a Deep Reinforcement Learning (DRL) framework. We introduce a mathematical formulation to integrate NMPC into this DRL framework. To validate our methodology and design choices, we evaluate DRL training and test outcomes in a physics-based simulation involving a mobile manipulator. Experimental results demonstrate that Re4MPC is more computationally efficient and achieves higher success rates in reaching end-effector goals than the NMPC baseline, which computes whole-body trajectories without our learning mechanism.

Figures

Figures reproduced from arXiv: 2506.08344 by the authors.

Figure 1
Figure 1. The Re4MPC pipeline consists of two main compo [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Training curves for Re4MPC when trained using [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Success rate per training timestep for the NMPC [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Map of robot trajectories shown as connected dots, [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

38 extracted references · 25 canonical work pages

  1. [1]

    A survey of robotic motion planning in dynamic environments,

    M. Mohanan and A. Salgoankar, “A survey of robotic motion planning in dynamic environments,”Robotics and Autonomous Systems, vol. 100, pp. 171–185, 2018

    work page 2018

  2. [2]

    A survey of wheeled mobile manipulation: A decision-making perspective,

    S. Thakar, S. Srinivasan, S. Al-Hussaini, P. M. Bhatt, P. Rajendran, Y . Jung Yoon, N. Dhanaraj, R. K. Malhan, M. Schmid, V . N. Krovi, et al., “A survey of wheeled mobile manipulation: A decision-making perspective,”Journal of Mechanisms and Robotics, vol. 15, no. 2, p. 020801, 2023

    work page 2023

  3. [3]

    Plan-time multi-model switching for motion planning,

    B. Styler and R. Simmons, “Plan-time multi-model switching for motion planning,” inProceedings of the International Conference on Automated Planning and Scheduling, vol. 27, 2017, pp. 558–566

    work page 2017

  4. [4]

    Adaptive complexity model predictive control,

    J. Norby, A. Tajbakhsh, Y . Yang, and A. M. Johnson, “Adaptive complexity model predictive control,”IEEE Transactions on Robotics, 2024

    2024

  5. [5]

    Real-time mixed-integer quadratic programming for vehicle decision-making and motion plan- ning,

    R. Quirynen, S. Safaoui, and S. Di Cairano, “Real-time mixed-integer quadratic programming for vehicle decision-making and motion plan- ning,”IEEE Transactions on Control Systems Technology, 2024

    work page 2024

  6. [6]

    Multi-modal model predictive control through batch non-holonomic trajectory optimization: Application to highway driving,

    V . K. Adajania, A. Sharma, A. Gupta, H. Masnavi, K. M. Krishna, and A. K. Singh, “Multi-modal model predictive control through batch non-holonomic trajectory optimization: Application to highway driving,”IEEE Robotics and Automation Letters, vol. 7, no. 2, pp. 4220–4227, 2022

    work page 2022

  7. [7]

    Manipulation plan- ning among movable obstacles using physics-based adaptive motion primitives,

    D. M. Saxena, M. S. Saleem, and M. Likhachev, “Manipulation plan- ning among movable obstacles using physics-based adaptive motion primitives,” in2021 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2021, pp. 6570–6576

    work page 2021

  8. [8]

    Constrained stochastic optimal control with learned importance sampling: A path integral approach,

    J. Carius, R. Ranftl, F. Farshidian, and M. Hutter, “Constrained stochastic optimal control with learned importance sampling: A path integral approach,”The International Journal of Robotics Research, vol. 41, no. 2, pp. 189–209, 2022

    work page 2022

Show all 38 references
  1. [9]

    A holistic approach to reactive mobile manipulation,

    J. Haviland, N. S ¨underhauf, and P. Corke, “A holistic approach to reactive mobile manipulation,”IEEE Robotics and Automation Letters, vol. 7, no. 2, pp. 3122–3129, 2022

    2022

  2. [10]

    An architecture for reactive mobile manipulation on-the-move,

    B. Burgess-Limerick, C. Lehnert, J. Leitner, and P. Corke, “An architecture for reactive mobile manipulation on-the-move,” in2023 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2023, pp. 1623–1629

    2023

  3. [11]

    Coupled mobile manip- ulation via trajectory optimization with free space decomposition,

    M. Spahn, B. Brito, and J. Alonso-Mora, “Coupled mobile manip- ulation via trajectory optimization with free space decomposition,” in2021 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2021, pp. 12 759–12 765

    2021

  4. [12]

    A review of mobile robot motion planning methods: from classical motion planning workflows to reinforcement learning-based architectures,

    L. Dong, Z. He, C. Song, and C. Sun, “A review of mobile robot motion planning methods: from classical motion planning workflows to reinforcement learning-based architectures,”Journal of Systems Engineering and Electronics, vol. 34, no. 2, pp. 439–459, 2023

    2023

  5. [13]

    Perceptive model predictive control for continuous mobile manipulation,

    J. Pankert and M. Hutter, “Perceptive model predictive control for continuous mobile manipulation,”IEEE Robotics and Automation Letters, vol. 5, no. 4, pp. 6177–6184, 2020

    2020

  6. [14]

    A unified mpc framework for whole-body dynamic locomotion and manipula- tion,

    J.-P. Sleiman, F. Farshidian, M. V . Minniti, and M. Hutter, “A unified mpc framework for whole-body dynamic locomotion and manipula- tion,”IEEE Robotics and Automation Letters, vol. 6, no. 3, pp. 4688– 4695, 2021

    2021

  7. [15]

    An efficient sequential linear quadratic algorithm for solving nonlinear optimal control problems,

    A. Sideris and J. E. Bobrow, “An efficient sequential linear quadratic algorithm for solving nonlinear optimal control problems,” inProceed- ings of the 2005, American Control Conference, 2005.IEEE, 2005, pp. 2275–2280

    2005

  8. [16]

    Constraint handling in continuous-time ddp-based model predictive control,

    J.-P. Sleiman, F. Farshidian, and M. Hutter, “Constraint handling in continuous-time ddp-based model predictive control,” in2021 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2021, pp. 8209–8215

    2021

  9. [17]

    Learning mobile manipulation through deep reinforcement learning,

    C. Wang, Q. Zhang, Q. Tian, S. Li, X. Wang, D. Lane, Y . Petillot, and S. Wang, “Learning mobile manipulation through deep reinforcement learning,”Sensors, vol. 20, no. 3, p. 939, 2020

    2020

  10. [18]

    Learning positioning policies for mobile manipulation operations with deep reinforcement learning,

    A. Iriondo, E. Lazkano, A. Ansuategi, A. Rivera, I. Lluvia, and C. Tub ´ıo, “Learning positioning policies for mobile manipulation operations with deep reinforcement learning,”International Journal of Machine Learning and Cybernetics, vol. 14, no. 9, pp. 3003–3023, 2023

    2023

  11. [19]

    Hrl4in: Hierar- chical reinforcement learning for interactive navigation with mobile manipulators,

    C. Li, F. Xia, R. Martin-Martin, and S. Savarese, “Hrl4in: Hierar- chical reinforcement learning for interactive navigation with mobile manipulators,” inConference on Robot Learning. PMLR, 2020, pp. 603–616

    2020

  12. [20]

    An online training method for augmenting mpc with deep reinforcement learning,

    G. Bellegarda and K. Byl, “An online training method for augmenting mpc with deep reinforcement learning,” in2020 IEEE/RSJ Interna- tional Conference on Intelligent Robots and Systems (IROS). IEEE, 2020, pp. 5453–5459

    2020

  13. [21]

    Motion planner augmented reinforcement learning for robot manipulation in obstructed environments,

    J. Yamada, Y . Lee, G. Salhotra, K. Pertsch, M. Pflueger, G. Sukhatme, J. Lim, and P. Englert, “Motion planner augmented reinforcement learning for robot manipulation in obstructed environments,” inCon- ference on Robot Learning. PMLR, 2021, pp. 589–603

    2021

  14. [22]

    Relmogen: Integrating motion generation in reinforcement learning for mobile manipulation,

    F. Xia, C. Li, R. Mart ´ın-Mart´ın, O. Litany, A. Toshev, and S. Savarese, “Relmogen: Integrating motion generation in reinforcement learning for mobile manipulation,” in2021 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2021, pp. 4583–4590

    2021

  15. [23]

    Cacto: Continuous actor-critic with trajectory opti- mization - towards global optimality,

    G. Grandesso, E. Alboni, G. P. Papini, P. M. Wensing, and A. Del Prete, “Cacto: Continuous actor-critic with trajectory opti- mization - towards global optimality,”IEEE Robotics and Automation Letters, vol. 8, no. 6, pp. 3318–3325, 2023

    2023

  16. [24]

    Data efficient reinforcement learning for legged robots,

    Y . Yang, K. Caluwaerts, A. Iscen, T. Zhang, J. Tan, and V . Sindhwani, “Data efficient reinforcement learning for legged robots,” inConfer- ence on Robot Learning. PMLR, 2020, pp. 1–10

    2020

  17. [25]

    Hierarchical evasive path planning using reinforcement learning and model predictive control,

    ´A. Feh ´er, S. Aradi, and T. B ´ecsi, “Hierarchical evasive path planning using reinforcement learning and model predictive control,”IEEE Access, vol. 8, pp. 187 470–187 482, 2020

    2020

  18. [26]

    OCS2: An open source library for optimal control of switched systems,

    F. Farshidianet al., “OCS2: An open source library for optimal control of switched systems,” [Online]. Available: https://github.com/ leggedrobotics/ocs2

  19. [27]

    Stable-baselines3: Reliable reinforcement learning implementations,

    A. Raffin, A. Hill, A. Gleave, A. Kanervisto, M. Ernestus, and N. Dormann, “Stable-baselines3: Reliable reinforcement learning implementations,”Journal of Machine Learning Research, vol. 22, no. 268, pp. 1–8, 2021. [Online]. Available: http://jmlr.org/papers/ v22/20-1364.html

    2021

  20. [28]

    Pybullet, a python module for physics sim- ulation for games, robotics and machine learning,

    E. Coumans and Y . Bai, “Pybullet, a python module for physics sim- ulation for games, robotics and machine learning,” http://pybullet.org, 2016–2021

    2016

  21. [29]

    A friction-based kinematic model for skid- steer wheeled mobile robots,

    S. Rabiee and J. Biswas, “A friction-based kinematic model for skid- steer wheeled mobile robots,” in2019 International Conference on Robotics and Automation (ICRA). IEEE, 2019, pp. 8563–8569

    2019

  22. [30]

    Relaxed logarithmic barrier function based model predictive control of linear systems,

    C. Feller and C. Ebenbauer, “Relaxed logarithmic barrier function based model predictive control of linear systems,”IEEE Transactions on Automatic Control, vol. 62, no. 3, pp. 1223–1238, 2016

    2016

  23. [31]

    Planning and acting in partially observable stochastic domains,

    L. P. Kaelbling, M. L. Littman, and A. R. Cassandra, “Planning and acting in partially observable stochastic domains,”Artificial intelli- gence, vol. 101, no. 1-2, pp. 99–134, 1998

    1998

  24. [32]

    Deep rein- forcement learning based robot navigation in dynamic environments using occupancy values of motion primitives,

    N. ¨U. Akmandor, H. Li, G. Lvov, E. Dusel, and T. Padir, “Deep rein- forcement learning based robot navigation in dynamic environments using occupancy values of motion primitives,” in2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2022, p...

    2022

  25. [33]

    Siciliano,Kinematics

    B. Siciliano,Kinematics. London: Springer London, 2009, pp. 39–

    2009

  26. [34]

    igibson 2.0: Object-centric simulation for robot learning of everyday household tasks,

    C. Li, F. Xia, R. Mart ´ın-Mart´ın, M. Lingelbach, S. Srivastava, B. Shen, K. E. Vainio, C. Gokmen, G. Dharan, T. Jain, A. Kurenkov, K. Liu, H. Gweon, J. Wu, L. Fei-Fei, and S. Savarese, “igibson 2.0: Object-centric simulation for robot learning of everyday household tasks,” i...

    2022

  27. [35]

    Proximal policy optimization algorithms,

    J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization algorithms,”arXiv preprint arXiv:1707.06347, 2017

    2017 arXiv

  28. [36]

    Soft actor-critic: Off- policy maximum entropy deep reinforcement learning with a stochastic actor,

    T. Haarnoja, A. Zhou, P. Abbeel, and S. Levine, “Soft actor-critic: Off- policy maximum entropy deep reinforcement learning with a stochastic actor,” inInternational conference on machine learning. PMLR, 2018, pp. 1861–1870

    2018

  29. [37]

    Human-level control through deep reinforcement learning,

    V . Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski, et al., “Human-level control through deep reinforcement learning,” Nature, vol. 518, no. 7540, pp. 529–533, 2015

    2015

  30. [103]

    Available: https://doi.org/10.1007/978-1-84628-642-1 2

    [Online]. Available: https://doi.org/10.1007/978-1-84628-642-1 2

    doi

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.