REVIEW 3 major objections 6 minor 34 references
A manifold-constrained Hamilton-Jacobi reachability framework lets robots plan collision-free, task-compliant motions in real time without knowing other agents' policies.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-03 23:53 UTC pith:YCSOJ3T6
load-bearing objection Solid manifold-constrained HJR extension with a clean 2D validation, but the planner's safety claim only checks the horizon endpoint and the paper doesn't justify the gap. the 3 major comments →
Manifold-constrained Hamilton-Jacobi Reachability Learning for Decentralized Multi-Agent Motion Planning
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The central claim is that a value function satisfying the manifold-constrained HJB equation, learned by a neural network trained against a terminal condition and a PDE residual, encodes when a state is safe under a given task constraint. The paper derives this PDE, gives a closed-form Hamiltonian for velocity-controlled systems via projection onto the manifold's tangent space, and shows the learned backward reachable set on a 2D circle-constrained particle matches ground truth while the unconstrained version over-approximates. The value function then serves as a pairwise safety certificate in a receding-horizon optimizer, enforcing the equality constraint at the planning endpoint and requiri
What carries the argument
The load-bearing object is the manifold-constrained Hamilton-Jacobi-Bellman PDE, together with the projection matrix P(x)=I−J_C^T(J_C J_C^T)^{-1}J_C that maps control inputs into the tangent space of the constraint manifold. For velocity-controlled systems, the constrained Hamiltonian reduces to −ū||P(x)∇V||₂, yielding a closed-form worst-case control. A neural network is trained to satisfy this PDE (via a terminal loss and a PDE residual loss) on samples from the manifold, and the resulting value function is evaluated pairwise in a trajectory optimizer that enforces C(x)=0 at the planning endpoint and requires V_θ > ε for predicted future states.
Load-bearing premise
The safety check only has meaning while the robot state lies exactly on the task's constraint manifold (the set of configurations satisfying the equality constraint); the planner enforces this only at the end of each planning step, so drift off the manifold between replans invalidates the safety guarantee.
What would settle it
Execute a trajectory planned by the proposed method while artificially perturbing the robot's configuration off the constraint manifold between replans (e.g., tilting a cup-carrying end effector by a few degrees), and check whether a collision occurs even though the planned value function stayed above the threshold; or, on the 2D circle test, compare the learned backward reachable set to the analytic geodesic set and measure any mismatch.
If this is right
- Because the value function is learned on the manifold and used pairwise, robots can replan in real time (about 0.1–0.2 seconds per step) while maintaining collision avoidance, as the experiments demonstrate.
- The closed-form Hamiltonian for velocity-controlled systems makes the approach applicable to any smooth equality constraint of the form C(x)=0, so tasks with different orientation or alignment constraints can be handled by exchanging C.
- The adversarial (zero-sum) treatment of other agents removes the need to know their control policies, which enables decentralization and robustness to non-cooperative agents.
- When the optimizer cannot find a feasible solution, a fail-safe control derived from the constrained Hamiltonian keeps the robot moving tangentially to the manifold, providing a conservative avoidance action.
- Scaling from two to five agents degrades success rate (from 85% to 57%) but keeps collision rate low and planning time modest, indicating the safety analysis remains usable in denser teams.
Where Pith is reading between the lines
- A natural extension is to wrap the planner in a high-frequency manifold-tracking controller that projects executed states back onto the constraint manifold between replans, which would restore the validity of the learned value function as a safety certificate.
- The same projection argument used for velocity-controlled systems could be applied to linear control-affine systems, broadening the framework to actuated robots with second-order dynamics.
- The pairwise, worst-case formulation likely becomes overly conservative as the number of agents grows; a probabilistic model of other agents' behavior, or conformal calibration of the value threshold as the paper suggests, could reduce conservatism while preserving safety with high probability.
- The 2D circle experiment offers a quick unit test for implementations: if the learned backward reachable set on that toy problem deviates measurably from the analytic geodesic set, the neural solver is not actually solving the constrained HJB equation.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents HaMMAR, a learning-based Hamilton-Jacobi reachability (HJR) framework for decentralized multi-agent motion planning under equality manifold constraints. It extends DeepReach to constrained systems by deriving a manifold-constrained HJB PDE and a closed-form projected Hamiltonian for velocity-controlled dynamics, trains a neural value function against PDE residuals and a 2D ground-truth BRS, and embeds the learned value function into a receding-horizon trajectory optimization with pairwise collision-avoidance constraints. Experiments on a 2D particle system and on UR5 manipulators in object-carrying, cup-holding, and doorway-crossing tasks report higher success rates and lower collision rates than the considered baselines. The paper claims that the method produces safe and task-feasible motion plans without assumptions about other agents' policies.
Significance. If the claims are fully supported, the core idea is valuable: formulating HJR on constraint manifolds and exploiting a closed-form projected Hamiltonian for velocity-controlled systems could extend learning-based reachability to a practically important class of constrained multi-robot problems. The 2D circle-constrained comparison against an analytic ground truth is a useful sanity check, and the hardware-style cup-holding demonstration supports practical relevance. However, the safety claim rests on the HJR value function being evaluated only at the planning-horizon endpoint, and the paper itself concedes the absence of formal safety guarantees. As written, the planner is not actually certified safe by the HJR theory, so the contribution is an interesting empirical method rather than a demonstrated safety guarantee. The nonzero collision rates in the multi-agent experiments are consistent with this gap. With a reworked safety-integration argument and explicit treatment of off-manifold drift, the approach could become a solid contribution.
major comments (3)
- [§IV-C, Eq. (23)] The safety constraint is enforced only at the planning-horizon endpoint: Vθ(T−t_safe, x1(t_plan), xi(t_plan)) > ε. HJR guarantees that if V(t,x)>0, there exists an optimal control keeping the state safe; it does not imply that the trajectory returned by the IPOPT optimization, which minimizes a different cost, stays inside the superlevel set along the entire horizon. A two-agent 'swap' trajectory can pass through an unsafe region mid-horizon while satisfying the endpoint constraint. This is a load-bearing issue for the abstract's claim that HaMMAR 'generates motion plans that are both safe and task-feasible.' The nonzero collision rates in Tables III and IV are consistent with this gap, which is not listed among the limitations in Section VI. The optimization should either impose Vθ>ε at multiple intermediate time nodes or use the HJR-derived control law as the nominal policy, with the c
- [§IV-C, Eq. (23) and §VI] The planner is supposed to keep x(t) on M 'making sure the value function Vθ is well-defined,' but the only explicit manifold constraint is C(x(t_plan))=0. The paper acknowledges in Section VI that 'discretized trajectories may drift off the constraint manifold ... and trajectory segments between planning updates are not explicitly constrained to the constraint manifold.' Since the value function is trained and theoretically defined only on M, evaluating Vθ off M is outside its valid domain and cannot provide a safety certificate. This is exactly the kind of limitation the review process should surface; as written, the method's central safety and task-feasibility guarantees are conditional on an unverified assumption. A projection/retraction step, a manifold-aware integration scheme, or a reformulation that constrains the entire trajectory is needed.
- [§IV-B, Eqs. (12)–(14) and §V] The training procedure is described only by analogy to DeepReach. There is no statement of the network architecture, the sampling distribution over the constraint manifold, the form of the terminal cost l(x) for the multi-agent tasks, the choice of the safety threshold ε, or the details of the adversarial pairwise value function (e.g., state dimension, whether the relative state is used, how the disturbance set D is realized). Because the safety argument depends on Vθ closely approximating the constrained HJB solution, the absence of these details makes the experimental evaluation difficult to interpret and reproduce. This is not a mere presentation issue: without knowing how the value function was trained and validated, the reader cannot assess whether the reported collision rates reflect a failure of the learned approximation or of the safety-integration logic.
minor comments (6)
- [§IV-A, Eqs. (15)–(22)] The derivation uses the Euclidean gradient ∇V while noting that V is only defined on M. The projection argument is valid for any smooth extension of V, but this should be stated explicitly to avoid ambiguity about which gradient is being computed and why the projection P(x) makes the expression well-defined.
- [§IV-A, Eq. (8)] The notation H_M is introduced but the dependence of the Hamiltonian on t and x is inconsistent across equations (e.g., H_M(t,x,p) in Eq. (8) vs. H_M(t,x,∇V(t,x)) in Eq. (15)). This should be harmonized.
- [§IV-C, Eq. (23)] The time indices in the optimization are confusing: τ=T−t_plan is defined but then the safety constraint uses T−t_safe. It is unclear whether the value function argument should be a time-to-go or an absolute time, and how t_safe and t_plan relate. This should be clarified.
- [Tables II–IV] The method name appears as 'MaNeHMO (ours)' in the tables while the text calls it 'HaMMAR.' This typo should be fixed for consistency.
- [§V-C] The scaling experiment measures planning time only for the surrounded manipulator ('Planning time is therefore only measured on the surrounded manipulator'). This is a reasonable choice, but it should be stated explicitly in the metric definition in §V so the reader does not infer average per-agent planning time.
- [§VI] The limitation paragraph correctly notes that neural-network-based HJR 'loses formal safety guarantees' and suggests conformal prediction as future work. This is an honest statement, but it is in tension with the unqualified 'safe' language in the abstract and introduction. If the final paper retains the current claims, the abstract should be softened to 'empirically collision-free' or the safety guarantees must be established formally.
Circularity Check
No significant circularity: the manifold-constrained HJB derivation and the learned-value-function validation are self-contained; the NeHMO self-citation is attribution, not load-bearing.
full rationale
The paper's central derivation is the manifold-constrained HJB PDE in Eq. (7), obtained by applying dynamic programming with the explicit feasibility constraint x(τ) ∈ M. The closed-form constrained Hamiltonian in Eqs. (15)-(22) follows from projecting the velocity control onto the tangent bundle via P(x), not from assuming the conclusion. The value function is trained with the DeepReach residual loss Eqs. (12)-(14), whose only targets are the terminal condition and PDE residual; the 2D validation in Section V-A compares against an analytically derived ground-truth BRS, so no reported success or collision quantity is baked into the loss. The planner in Eq. (23) uses the learned value function as a safety constraint, but this is an application of the trained function rather than a fitted parameter being renamed as a prediction; the safety threshold, horizons, and costs are user-chosen hyperparameters. The only self-citation is [21] (NeHMO) as the base planner being extended in Section IV-C, and the paper states the full optimization problem there, so the argument does not reduce to an unverified self-citation. Section VI explicitly acknowledges that neural-network value functions lose formal safety guarantees and that trajectories may drift off the manifold; these are correctness/robustness limitations, not circularity. Overall, the derivation chain is self-contained and independently checked against an analytic 2D benchmark and external baselines.
Axiom & Free-Parameter Ledger
free parameters (4)
- safety_threshold_epsilon
- safety_horizon (T - t_safe)
- planning_horizon t_plan
- DeepReach loss weight lambda
axioms (6)
- domain assumption The system is velocity-controlled with bounded control ||u|| <= u_bar, so the constrained Hamiltonian has the closed form in Eq. (20).
- domain assumption Constraint Jacobian J_C(x) has full row rank on M so that (J_C J_C^T)^{-1} exists in Eq. (22).
- standard math Standard viscosity-solution conditions for the HJB framework hold, and gradients of constraints are bounded.
- domain assumption Each agent perfectly perceives the states of other agents and knows their dynamics.
- ad hoc to paper The neural value function Vtheta closely approximates the true constrained HJB solution over the relevant state distribution.
- ad hoc to paper Planned and executed trajectories remain on M whenever Vtheta is evaluated.
Cite this review
Pith. "Pith review of Manifold-constrained Hamilton-Jacobi Reachability Learning for Decentralized Multi-Agent Motion Planning." pith.science (2026). https://pith.science/paper/YCSOJ3T6
@misc{pith2026251103591,
author = {Pith},
title = {Pith review of: Manifold-constrained Hamilton-Jacobi Reachability Learning for Decentralized Multi-Agent Motion Planning},
year = {2026},
howpublished = {\url{https://pith.science/paper/YCSOJ3T6}},
note = {Machine review of arXiv:2511.03591}
}
read the original abstract
Safe multi-agent motion planning (MAMP) under task-induced constraints is a critical challenge in robotics. Many real-world scenarios require robots to navigate dynamic environments while adhering to manifold constraints imposed by tasks. For example, service robots must carry cups upright while avoiding collisions with humans or other robots. Despite recent advances in decentralized MAMP for high-dimensional systems, incorporating manifold constraints remains difficult. To address this, we propose a manifold-constrained Hamilton-Jacobi reachability (HJR) learning framework for decentralized MAMP. Our method solves HJR problems under manifold constraints to capture task-aware safety conditions, which are then integrated into a decentralized trajectory optimization planner. This enables robots to generate motion plans that are both safe and task-feasible without requiring assumptions about other agents' policies. Our approach generalizes across diverse manifold-constrained tasks and scales effectively to high-dimensional multi-agent manipulation problems. Experiments show that our method outperforms existing constrained motion planners and operates at speeds suitable for real-world applications. Video demonstrations are available at https://youtu.be/RYcEHMnPTH8 .
Figures
Reference graph
Works this paper leans on
-
[1]
Learning a decentralized multi-arm motion planner,
H. Ha, J. Xu, and S. Song, “Learning a decentralized multi-arm motion planner,” 2020. [Online]. Available: https://arxiv.org/abs/2011.02608
Pith/arXiv arXiv 2020
-
[2]
Roboballet: Planning for multirobot reaching with graph neural networks and reinforcement learning,
M. Lai, K. Go, Z. Li, T. Kr ¨oger, S. Schaal, K. Allen, and J. Scholz, “Roboballet: Planning for multirobot reaching with graph neural networks and reinforcement learning,”Science Robotics, vol. 10, no. 106, Sept. 2025. [Online]. Available: http://dx.doi.org/10.1126/scirobotics.ads1204
-
[3]
Dynamic collision and deadlock avoidance for multiple robotic manipulators,
N. Gafur, G. Kanagalingam, A. Wagner, and M. Ruskowski, “Dynamic collision and deadlock avoidance for multiple robotic manipulators,”IEEE Access, vol. 10, p. 55766–55781, 2022. [Online]. Available: http://dx.doi.org/10.1109/ACCESS.2022.3176626 (a) (b) (c) (d) Fig. 6: An example of HaMMAR successfully solving adoorway-crossingtask. The robots’ current conf...
arXiv 2022
-
[4]
Synchronous minimum-time cooperative manipulation using distributed model predictive control,
A. Tika and N. Bajcinca, “Synchronous minimum-time cooperative manipulation using distributed model predictive control,” in2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2020, pp. 7675–7681
2020
-
[5]
Long-horizon multi-robot rearrangement planning for construction assembly,
V . N. Hartmann, A. Orthey, D. Driess, O. S. Oguz, and M. Toussaint, “Long-horizon multi-robot rearrangement planning for construction assembly,”IEEE Transactions on Robotics, vol. 39, no. 1, p. 239–252, Feb. 2023. [Online]. Available: http://dx.doi.org/10.1109/TRO.2022. 3198020
doi:10.1109/tro.2022 2023
-
[6]
Multi-robot local motion planning using dynamic optimization fabrics,
S. Bakker, L. Knoedler, M. Spahn, W. B ¨ohmer, and J. Alonso-Mora, “Multi-robot local motion planning using dynamic optimization fabrics,” 2023. [Online]. Available: https://arxiv.org/abs/2310.12816
Pith/arXiv arXiv 2023
-
[7]
Guaranteed real-time coop- erative collision avoidance for n-dof manipulators,
E. J. Rodr ´ıguez-Seda and M. D. Kutzer, “Guaranteed real-time coop- erative collision avoidance for n-dof manipulators,”Robotica, vol. 42, no. 9, pp. 3149–3173, 2024
2024
-
[8]
Manipulation planning on constraint manifolds,
D. Berenson, S. Srinivasa, D. Ferguson, and J. Kuffner, “Manipulation planning on constraint manifolds,” inProceedings of (ICRA) Interna- tional Conference on Robotics and Automation, May 2009, pp. 625 – 632
2009
-
[9]
Sampling-based methods for motion planning with constraints,
Z. Kingston, M. Moll, and L. E. Kavraki, “Sampling-based methods for motion planning with constraints,”Annual review of control, robotics, and autonomous systems, vol. 1, no. 1, pp. 159–185, 2018
2018
-
[10]
Asymptotically-optimal path planning on manifolds
L. Jaillet and J. M. Porta, “Asymptotically-optimal path planning on manifolds.” inRobotics: Science and Systems, 2012, pp. 145–152
2012
-
[11]
Task space regions: A framework for pose-constrained manipulation planning,
D. Berenson, S. Srinivasa, and J. Kuffner, “Task space regions: A framework for pose-constrained manipulation planning,”The Interna- tional Journal of Robotics Research, vol. 30, no. 12, pp. 1435–1460, 2011
2011
-
[12]
Tangent bundle rrt: A randomized algorithm for constrained motion planning,
B. Kim, T. T. Um, C. Suh, and F. C. Park, “Tangent bundle rrt: A randomized algorithm for constrained motion planning,”Robotica, vol. 34, no. 1, pp. 202–225, 2016
2016
-
[13]
A sampling-based path planner for dual-arm manipulation,
M. Gharbi, J. Cortes, and T. Simeon, “A sampling-based path planner for dual-arm manipulation,” in2008 IEEE/ASME International Con- ference on Advanced Intelligent Mechatronics, 2008, pp. 383–388
2008
-
[14]
W. Guo, Z. Kingston, K. Hang, and L. E. Kavraki, “Efficient multi-robot motion planning for manifold-constrained manipulators by randomized scheduling and informed path generation,” 2024. [Online]. Available: https://arxiv.org/abs/2412.00366
arXiv 2024
-
[15]
DeepReach: A deep learning approach to high-dimensional reachability,
S. Bansal and C. Tomlin, “DeepReach: A deep learning approach to high-dimensional reachability,” inIEEE International Conference on Robotics and Automation (ICRA), 2021
2021
-
[16]
Bridging hamilton-jacobi safety analysis and reinforcement learning,
J. F. Fisac, N. F. Lugovoy, V . Rubies-Royo, S. Ghosh, and C. J. Tomlin, “Bridging hamilton-jacobi safety analysis and reinforcement learning,” in2019 International Conference on Robotics and Automation (ICRA), 2019, pp. 8550–8556
2019
-
[17]
Isaacs: Iterative soft adversarial actor-critic for safety,
K.-C. Hsu, D. P. Nguyen, and J. F. Fisac, “Isaacs: Iterative soft adversarial actor-critic for safety,” 2024. [Online]. Available: https://arxiv.org/abs/2212.03228
Pith/arXiv arXiv 2024
-
[18]
Safety and liveness guarantees through reach-avoid reinforcement learning,
K.-C. Hsu*, V . Rubies-Royo*, C. Tomlin, and J. Fisac, “Safety and liveness guarantees through reach-avoid reinforcement learning,” inRobotics: Science and Systems XVII, ser. RSS2021. Robotics: Science and Systems Foundation, July 2021. [Online]. Available: http://dx.doi.org/10.15607/RSS.2021.XVII.077
-
[19]
Parameterized fast and safe tracking (FaSTrack) using DeepReach,
H. J. Jeong, Z. Gong, S. Bansal, and S. Herbert, “Parameterized fast and safe tracking (FaSTrack) using DeepReach,” inProceedings of the 6th Annual Learning for Dynamics & Control Conference, ser. Proceedings of Machine Learning Research, A. Abate, M. Cannon, K. Margellos, and A. Papachristodoulou, Eds., vol. 242. PMLR, 15–17 Jul 2024, pp. 1006–1017. [Onl...
2024
-
[21]
Q. Chen and A. H. Qureshi, “Nehmo: Neural hamilton-jacobi reachability learning for decentralized safe multi-agent motion planning,” 2025. [Online]. Available: https://arxiv.org/abs/2507.13940
Pith/arXiv arXiv 2025
-
[22]
Constrained nonlinear kaczmarz projection on intersections of manifolds for coor- dinated multi-robot mobile manipulation,
A. Agrawal, P. Mayer, Z. Kingston, and G. A. Hollinger, “Constrained nonlinear kaczmarz projection on intersections of manifolds for coor- dinated multi-robot mobile manipulation,” in2025 IEEE International Conference on Robotics and Automation (ICRA), 2025, pp. 7726– 7732
2025
-
[23]
On infusing reachability-based safety assurance within planning frameworks for human-robot vehicle interactions,
K. Leung, E. Schmerling, M. Zhang, M. Chen, J. Talbot, J. C. Gerdes, and M. Pavone, “On infusing reachability-based safety assurance within planning frameworks for human-robot vehicle interactions,”
-
[24]
Multi-vehicle collision avoidance via hamilton-jacobi reachability and mixed integer program- ming,
M. Chen, J. C. Shih, and C. J. Tomlin, “Multi-vehicle collision avoidance via hamilton-jacobi reachability and mixed integer program- ming,” in2016 IEEE 55th Conference on Decision and Control (CDC). IEEE, 2016, pp. 1695–1700
2016
-
[25]
Provably safe and scalable multivehicle trajectory planning,
S. Bansal, M. Chen, K. Tanabe, and C. J. Tomlin, “Provably safe and scalable multivehicle trajectory planning,”IEEE Transactions on Control Systems Technology, vol. 29, no. 6, pp. 2473–2489, 2021
2021
-
[26]
Bridging model predictive control and deep learning for scalable reachability analysis,
Z. Feng, L. Qiu, and S. Bansal, “Bridging model predictive control and deep learning for scalable reachability analysis,”arXiv preprint arXiv:2505.03830, 2025
Pith/arXiv arXiv 2025
-
[27]
I. M. Mitchell,Application of level set methods to control and reachability problems in continuous and hybrid systems. stanford university, 2002
2002
-
[28]
Reach-avoid problems with time-varying dynamics, targets and constraints,
J. F. Fisac, M. Chen, C. J. Tomlin, and S. S. Sastry, “Reach-avoid problems with time-varying dynamics, targets and constraints,” 2014. [Online]. Available: https://arxiv.org/abs/1410.6445
Pith/arXiv arXiv 2014
-
[29]
Hamilton-jacobi reachability: A brief overview and recent advances,
S. Bansal, M. Chen, S. Herbert, and C. J. Tomlin, “Hamilton-jacobi reachability: A brief overview and recent advances,” 2017. [Online]. Available: https://arxiv.org/abs/1709.07523
Pith/arXiv arXiv 2017
-
[30]
S. M. LaValle,Planning algorithms. Cambridge university press, 2006
2006
-
[31]
On the implementation of an interior- point filter line-search algorithm for large-scale nonlinear program- ming,
A. W ¨achter and L. T. Biegler, “On the implementation of an interior- point filter line-search algorithm for large-scale nonlinear program- ming,”Mathematical programming, vol. 106, pp. 25–57, 2006
2006
-
[32]
Safe Planning for Articulated Robots Using Reachability-based Obstacle Avoidance With Spheres,
J. Michaux, A. Li, Q. Chen, C. Chen, and R. Vasudevan, “Safe Planning for Articulated Robots Using Reachability-based Obstacle Avoidance With Spheres,” inProceedings of Robotics: Science and Systems, Delft, Netherlands, July 2024
2024
-
[33]
trimesh
Dawson-Haggerty et al., “trimesh.” [Online]. Available: https: //trimesh.org/
-
[34]
Verification of neural reachable tubes via scenario optimization and conformal prediction,
A. Lin and S. Bansal, “Verification of neural reachable tubes via scenario optimization and conformal prediction,” inProceedings of the 6th Annual Learning for Dynamics & Control Conference, ser. Proceedings of Machine Learning Research, A. Abate, M. Cannon, K. Margellos, and A. Papachristodoulou, Eds., vol. 242. PMLR, 15–17 Jul 2024, pp. 719–731. [Online...
2024
-
[2020]
Available: https://arxiv.org/abs/2012.03390
[Online]. Available: https://arxiv.org/abs/2012.03390
Pith/arXiv arXiv 2012
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.