Pith. sign in

REVIEW 3 major objections 5 minor 99 references

Solving Multi-Agent Safe Optimal Control with Distributed Epigraph Form MARL

T0 review · 3 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper proves that the centralized epigraph-form safe optimal control problem can be solved distributedly, with each agent finding its own minimal cost bound and the team taking the maximum.

desk verdict A solid, honestly evaluated extension of the authors' epigraph RL to multi-agent safe control, with a genuine distributed-decomposition theorem; the unverified injectivity assumption and the gap between 'zero violation' and the 97-100% empirical safety rates are the main things I'd want fixed before trusting the theory. read the letter →

arxiv 2504.15425 v1 pith:VYGMS24D submitted 2025-04-21 cs.RO cs.AIcs.LGcs.MAmath.OC

Songyuan Zhang , Oswin So , Mitchell Black , Zachary Serlin , Chuchu Fan This is my paper · ORCID
classification cs.ROcs.AIcs.LGcs.MAmath.OC
keywords multi-agentsafeoptimalcontrolzeroconstraintviolationepigraphformconstrainedMarkovdecisionprocesscentralizedtrainingdistributedexecutionproximalpolicyoptimizationgraphneuralnetworkCrazyflie
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Def-MARL tackles the multi-agent safe optimal control problem, where safety means zero constraint violation at every step. The paper rewrites the constrained problem in epigraph form, turning it into an outer one-dimensional search over a cost bound $z$ and an unconstrained inner policy optimization. It then proves (Theorem 1) that the centralized outer problem is equivalent to $z=\max_i z_i$, where each agent solves a local problem using only its own constraint-value function. A sympathetic reading of the evidence: across eight simulated tasks and Crazyflie hardware experiments, Def-MARL keeps near-100% safety with costs matching the aggressive baselines, while training is stable with constant hyperparameters. This matters because Lagrangian methods become unstable when the violation threshold is zero, and Def-MARL removes that instability without sacrificing the safety constraint.

What carries the argument

The epigraph form is the central object: a constrained optimization $\min_\pi J(\pi)$ subject to $h(\pi)\le 0$ is replaced by $\min_z z$ subject to $\min_\pi \max\{h(\pi), J(\pi)-z\}\le 0$. The paper extends this to multi-agent systems by defining the total value function $V(x,z;\pi)=\max_i \max\{V_i^h(o_i;\pi), V^l(x;\pi)-z\}$ and showing it satisfies dynamic programming in the augmented state with $z_{k+1}=z_k-l(x_k,u_k)$. This dynamic programming property makes the inner policy problem solvable by PPO with GNN value functions, and Theorem 1 then decomposes the outer problem across agents, replacing a centralized optimization with local root-finding plus a max.

What would settle it

Run Def-MARL on a task engineered to make the learned policy yield equal cumulative cost for two different $z$ values (for instance, symmetric goals with identical distances), record $z_i$ and $z=\max_i z_i$ during execution, and compare with the centralized outer solution found by exhaustive search over $z$; if they differ, Theorem 1's decomposition fails in that regime. A simpler probe: evaluate the learned $V_i^h(o_i;\pi(\cdot,z))$ over a fine $z$-grid and check monotonicity.

Watch

Extended reading notes

Core claim

The paper's central claim is that the multi-agent safe optimal control problem with hard safety constraints can be solved by a centralized-training, distributed-execution algorithm built on the epigraph form. Concretely, for the joint policy $\pi$ and per-agent constraint values $V_i^h$, the safety constraint $V^h(x;\pi)\le 0$ is equivalent to $\max_i V_i^h(o_i;\pi)\le 0$, and the outer epigraph objective becomes $z=\max_i z_i$ with $z_i=\min\{z' : V_i^h(o_i;\pi(\cdot,z'))\le 0\}$ (Theorem 1, under an injectivity assumption). During execution each agent solves its own one-dimensional root-finding problem for $z_i$ using its learned local constraint-value function, then takes the maximum, so no centralized cost-value function is needed online. Simulations on modified MPE and Safe Multi-agent MuJoCo, plus Crazyflie hardware experiments, show near-100% safety and costs competitive with the best unsafe baselines.

Load-bearing premise

The load-bearing premise is that no two distinct values of the cost bound $z$ produce the same cumulative cost under the learned policy, so the constraint-value function is monotone in $z$; the paper assumes this in Theorem 1 but does not test it.

Editorial extensions

If this is right

  • With Def-MARL, zero-constraint-violation safety is achieved without the usual penalty-performance trade-off: safety matches the most conservative baselines while cost matches the least safe ones.
  • Distributed execution needs only a local constraint-value function and a one-dimensional root-find per agent; no centralized value network is required at run time.
  • Because the epigraph objective adds $z$ instead of multiplying by a Lagrange multiplier, policy gradients do not blow up with constraint severity, giving smoother training curves than Lagrangian baselines.
  • Increasing the safety buffer $\xi$ in the outer problem (toward $\nu$) trades a small cost increase for higher safety rates, giving a practical dial for robustness to estimation error.
  • A policy trained with 8 agents generalizes to 512 agents at constant density with safety rates near 99.5%, and hardware trials on Crazyflie quadcopters show 100% success in corridor and inspect tasks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The injectivity assumption behind Theorem 1 is never verified; a natural testbed is to construct a symmetric goal configuration where two different cost bounds yield equal cumulative cost and compare the distributed $z$ with the centralized epigraph optimum.
  • The paper's observation that omitting $z$-communication barely hurts performance suggests a stronger statement may hold, but it also means the practical algorithm operates outside the theorem's conditions; quantifying the gap would turn an empirical finding into a characterization.
  • The same epigraph machinery could be combined with a certified constraint-value function, such as one trained with formal reachability margins, to turn the near-100% safety into a provable guarantee, which the paper itself lists as an open issue.
  • The homogeneous-MAS assumption and GNN backbone are conveniences, not requirements; Theorem 1 only needs per-agent constraint-value functions, so heterogeneous-agent extensions are a plausible next step.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This paper introduces Def-MARL, a centralized-training/distributed-execution MARL algorithm for the multi-agent safe optimal control problem (MASOCP) with zero constraint violation. The method reformulates the constrained problem in epigraph form with an auxiliary cost-upper-bound variable z, trains a z-conditioned policy and value functions with PPO and GNNs, and during execution solves a distributed outer problem in which each agent computes the smallest z_i making its local constraint-value function Vh_i non-positive, with the joint z taken as max_i z_i. The central theoretical claim (Theorem 1) is that this distributed computation is equivalent to the centralized epigraph outer problem under an injectivity assumption on the map from z to the optimal cost. Experiments on six MPE tasks and two Safe Multi-agent MuJoCo tasks compare Def-MARL against penalty and Lagrangian baselines, reporting near-100% safety with lower cost and more stable training using fixed hyperparameters, plus Crazyflie hardware demonstrations against MPC baselines. The paper also reports generalization to MAS sizes up to 512 agents and ablations on z-communication and the safety margin parameter xi.

Significance. If the theoretical decomposition is correct, this is a valuable and non-obvious contribution: it extends single-agent epigraph-form safe RL to the multi-agent CTDE setting, provides a distributed execution rule based on local constraint-value functions, and avoids the well-documented training instability of Lagrangian methods at zero violation threshold. The empirical package is strong: eight simulated tasks across two simulators with consistent hyperparameters, comparison against multiple penalty and Lagrangian baselines, ablations, hardware experiments on Crazyflie drones, and a public code project. The generalization experiments to hundreds of agents and the hardware success rates are particularly interesting. The main qualification is that the zero-violation guarantee is not achieved in simulation (safety rates are often 93-100%), and the proof of Theorem 1 rests on an injectivity assumption that is unverified and appears to be violated in the paper's own settings. The paper would be a solid contribution after the theoretical claim is repaired or appropriately weakened.

major comments (3)
  1. [Section IV-C, Appendix B (Lemma 3), Eq. (15)] Theorem 1 is stated under the assumption that no two distinct values of z yield the same unique optimal cost, i.e., the map z -> Vl(x; pi(.,z)) is injective. This assumption is load-bearing: the proof of Lemma 3 uses it in Case 2 to conclude from Vl(pi_{z*}) = Vl(pi_{z*+epsilon}) that pi_{z*} = pi_{z*+epsilon}, which yields the monotonicity Vh(pi_{z*+epsilon}) <= Vh(pi_{z*}) needed for the second half of the double-inequality proof of z* = max_i z_i. Without that conclusion, the proof only shows consistency inequalities and does not establish that Vh_i(pi(.,z_distr)) <= 0 for all i, so the distributed outer problem can return a z at which some agent is unsafe even though each individual z_i is the first safe z for that agent. The assumption is also in direct tension with Remark 1: for all sufficiently large z, the inner objective max{Vh, Vl - z} is dominated by Vh, so the optimal policy minimizes Vh alone and is independent of z over an entire interval whenever a least-violating policy has finite cost. All environments in the paper have bounded finite-horizon costs, so such a flat tail exists and this generic failure of injectivity applies. The theorem as stated therefore does not cover the paper's own experimental settings, and the claimed equivalence between the centralized epigraph outer problem and z = max_i z_i is not established by the proof as written. Please either prove monotonicity of Vh_i(pi(.,z)) under weaker assumptions, state and empirically verify an explicit nonincreasing property of Vh_i in z, or weaken the claim to a conservative bound with a margin.
  2. [Section V-B, Tables I-III, Fig. 4, Section VIII] The paper defines MASOCP (2) as requiring zero constraint violation, and the abstract states that Def-MARL "satisfies safety constraints", but the simulation results do not achieve zero violation: Table II reports safety rates of 96.9-99.0%, Table III reports 93.4-100%, Table I reports 99.5-99.8%, and Fig. 4 shows similar near-but-not-perfect values. The Limitations section correctly concedes that guarantees do not hold under inexact minimization of the losses or with learned value functions. The empirical claims should be reworded to "near-100% safety" in the abstract and results, and the gap between the idealized zero-violation guarantee and the learned-function-approximation setting should be presented as a central caveat rather than relegated to the final limitations paragraph. This does not diminish the empirical comparison, but the current wording overstates the extent to which problem (2) is solved in practice.
  3. [Section V-C and Section IV-C] The main experiments disable z-communication and use the approximation z <- z_i, while Theorem 1 and Algorithm 2 describe the communicated variant z = max_i z_i. The paper states this discrepancy and provides an ablation (Table II), but the reported headline results are all for the approximate variant. Since the approximation is not covered by the theorem, the paper should make this separation prominent in Section V-A and in the abstract, and should not imply that the experimental safety/cost numbers validate the exact distributed equivalence of Theorem 1. The ablation shows the two variants are close in these environments, which is useful evidence, but it is not a substitute for verifying the theorem's assumptions or for a theoretical statement about the approximate variant.
minor comments (5)
  1. [Appendix B, Lemma 3] In the proof of Lemma 3, after considering the case Vh(x; pi_{z*}) >= Vl(x; pi_{z*}) - (z* + epsilon), the text says "We thus suppose that Vh(x; pi_{z*}) >= Vl(x; pi_{z*}) - (z* + epsilon)", which should be the complementary strict inequality "<" for the rest of the proof.
  2. [Section IV-C] The phrase "Assume no two unique values of z achieves the same unique cost" is grammatically unclear and should be restated as a precise injectivity condition, e.g., "Assume the map z -> Vl(x; pi(.,z)) is injective for the relevant range of z."
  3. [Appendix E-C, Table IV] The claim that Def-MARL uses constant hyperparameters across all environments is stronger than the implementation: zmax is computed per environment (Eqs. 63-64), the number of Vh GNN layers differs for ConnectSpread (Table V), and training steps vary by environment. Please clarify which hyperparameters are fixed and which are environment-specific.
  4. [Table IV] There is a typo in the hyperparameter table: "massage passing dimension" should be "message passing dimension".
  5. [Abstract and Section V-B] The abstract's claim that Def-MARL "satisfies safety constraints" should be qualified as "near-100% safety" or "satisfies constraints up to value-function estimation error", consistent with the numbers in Fig. 4 and Tables I-III.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the distributed epigraph decomposition is proven, not assumed, and the only load-bearing citations are standard or jointly backed by external results.

full rationale

The paper's derivation chain is (2) -> epigraph form (5) -> Bellman recursion (Prop. 1) -> inner/outer split -> Theorem 1 distributed outer problem. Theorem 1 is proven in Appendix B via Lemmas 1-3; the key equality z*=max_i z_i is reached by a double inequality, and the proof does not define z* to be max_i z_i. The injectivity assumption ('no two unique values of z achieves the same unique cost') is an explicit regularity condition used in Lemma 3 to obtain monotonicity of Vh_i in z; it is not a restatement of the theorem's conclusion, so the argument is conditional rather than circular. The value functions are trained by regression on rollouts, but the reported safety metric is defined directly by h, not by the fitted Vh, so no fitted parameter is renamed as a prediction. Reliance on the authors' prior epigraph work [66] is for a standard constrained-optimization equivalence (also in [10]) and for the single-agent inner-problem training scheme; the multi-agent decomposition is new, and the convergence claim in Appendix G is backed by an external theorem [81] in addition to the authors' [67]. The Limitations section explicitly states that the theoretical guarantee requires optimal value functions/policies and that disabling z-communication invalidates the optimality guarantee, which further indicates the paper is not claiming a by-construction equivalence. The unverified injectivity assumption and the near-100% (not exactly 100%) empirical safety rates are legitimate correctness/rigor concerns, but they do not constitute circularity under the criteria requiring a specific reduction of a prediction to its inputs.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The central claim rests on standard epigraph theory, the dynamic programming recursion (Prop. 1), the convergence of PPO to local optima (cited from prior work, partly the authors' own), and the injectivity assumption in Theorem 1. The safety guarantee also depends on the learned value functions being accurate, which is not certified.

free parameters (6)
  • z_max = Environment-specific: conservative estimate for MPE; doubled unconstrained MAPPO cost for MuJoCo
    Sampling interval for z in training; Appendix E-C shows performance degrades if too small or too large.
  • xi = 0.4
    Safety buffer in outer problem (15b); Table III shows it trades safety against cost.
  • nu = 0.5
    Safety margin inserted into h (Appendix E-B) to create a discontinuity at the safety boundary.
  • z_min = -0.5
    Lower bound for z sampling; represents an estimate of minimum possible cost.
  • Vh GNN layers = 2 for ConnectSpread, 1 for others
    Per-environment architecture choice (Table V).
  • training steps = 1e5-2e5 for MPE, 7e3 for MuJoCo
    Set per environment without a stated rule; affects convergence.
assumptions (4)
  • standard math Epigraph form equivalence: problem (3) and (4)/(5) have the same optimum (Boyd and Vandenberghe; [66, Theorem 3]).
    Invoked in Section III-B and Appendix H.
  • domain assumption Infinite-horizon dynamic programming recursion for the total value function V (Proposition 1).
    Proved in Appendix A; assumes interchange of max and sum over time and the z-dynamics z_{k+1}=z_k-l(x_k,u_k).
  • domain assumption The inner RL problem (13b) converges to a locally optimal policy under PPO (via [81, Theorem 5.5] or [67, Theorem 4]).
    Appendix G; both cited works are not machine-checked and one is from the same authors.
  • ad hoc to paper Theorem 1's injectivity assumption: no two distinct z give the same unique optimal cost Vl(x; pi_z).
    Section IV-C; needed for Lemma 3's monotonicity of Vh in z; not verified in experiments or theory beyond an assertion.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Solving Multi-Agent Safe Optimal Control with Distributed Epigraph Form MARL." pith.science (2026). https://pith.science/paper/VYGMS24D

@misc{pith2026250415425,
  author       = {Pith},
  title        = {Pith review of: Solving Multi-Agent Safe Optimal Control with Distributed Epigraph Form MARL},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VYGMS24D}},
  note         = {Machine review of arXiv:2504.15425}
}
read the original abstract

Tasks for multi-robot systems often require the robots to collaborate and complete a team goal while maintaining safety. This problem is usually formalized as a constrained Markov decision process (CMDP), which targets minimizing a global cost and bringing the mean of constraint violation below a user-defined threshold. Inspired by real-world robotic applications, we define safety as zero constraint violation. While many safe multi-agent reinforcement learning (MARL) algorithms have been proposed to solve CMDPs, these algorithms suffer from unstable training in this setting. To tackle this, we use the epigraph form for constrained optimization to improve training stability and prove that the centralized epigraph form problem can be solved in a distributed fashion by each agent. This results in a novel centralized training distributed execution MARL algorithm named Def-MARL. Simulation experiments on 8 different tasks across 2 different simulators show that Def-MARL achieves the best overall performance, satisfies safety constraints, and maintains stable training. Real-world hardware experiments on Crazyflie quadcopters demonstrate the ability of Def-MARL to safely coordinate agents to complete complex collaborative tasks compared to other methods.

Figures

Figures reproduced from arXiv: 2504.15425 by the authors.

Figure 1
Figure 1. Two agents using Def-MARL to safely and collaboratively inspect a moving target. We propose a novel safe MARL algorithm, Def-MARL, that solves the multi-agent safe optimal control problem. Def-MARL translates the original problem to its epigraph form to avoid unstable training and extends the epigraph form to the CTDE paradigm for distributed execution. (a): Long exposure photo of the trajectories of the drones. The… view at source ↗
Figure 2
Figure 2. Def-MARL algorithm. Randomly sampled initial states and z 0 are used to collect trajectories in x and z using the current policy π. In the centralized training (orange blocks), distributed constraint-value functions V h i and policies πi and a centralized cost-value function V l are jointly trained. During distributed execution (green blocks), the distributed V h i are used to solve the outer problem (15b) to comput… view at source ↗
Figure 3
Figure 3. Simulation Environments. Visualization of the (top) modified MPE [40] and (bottom) Safe Multi-agent MuJoCo [32] environments we consider. constraint-value function V h of other agents. As a result, it is sufficient for only the connected agents to communicate their zi . Furthermore, we observe experimentally that the agents can achieve low cost while maintaining safety even if zi is not communicated (see Section V-C… view at source ↗
Figures from the paper (13 more)
Figure 4
Figure 4. Figure 4: Comparison on modified MPE (N = 3) and Safe Multi-agent MuJoCo. Def-MARL is consistently closest to the top-left corner in all environments, achieving low cost with near 100% safety rate. The dots show the mean values and the error bars show one standard deviation. use…
Figure 5
Figure 5. Figure 5: Converged states in CORRIDOR. Def-MARL achieves the global minimum, while other baselines converge to a different optimum (partly) due to training using a different cost function. 0.0 0.5 1.0 Step ×105 0.0 0.5 1.0 1.5 Cost Target 0.0 0.5 1.0 Step ×10 5 0.0 0.5 1.0 Safe…
Figure 6
Figure 6. Figure 6: Training Curves in TARGET and SPREAD. Def-MARL has a smoother, more stable training curve compared to Lagr(lr). We plot the mean and shade the ±1 standard deviation. TABLE I: Policy Generalization. Testing Def-MARL on TARGET with more agents after training with N = 8 a…
Figure 7
Figure 7. Figure 7: Comparison on larger-scale modified MPE. Def-MARL remains in the top-left corner even when the number of agents increases. The dots show the mean and the error bars show one standard deviation. TABLE II: Effect of zi communication (Section IV-C) in different environmen…
Figure 8
Figure 8. Figure 8: Hardware tasks. We perform hardware experiments using a swarm of CF drones on the CORRIDOR and INSPECT tasks. In CORRIDOR, the team must cross a narrow corridor and cover a set of goals collectively without prior assignment. In INSPECT, the team must maintain visual co…
Figure 9
Figure 9. Figure 9: Hardware Results on CORRIDOR (N = 3). Left to right: key frames of the trajectories generated by different algorithms. Arrows with different colors indicate the positions of different drones. Def-MARL (top) finishes the task with 100% success rate because the drones le…
Figure 10
Figure 10. Figure 10: Hardware Results of Def-MARL on CORRIDOR (N = 7). Even in this crowded environment, Def-MARL maintains a success rate of 100%. Def-MARL CMPC DMPC [PITH_FULL_IMAGE:figures/full_fig_p011_10.png]
Figure 11
Figure 11. Figure 11: Hardware Results on INSPECT. The CF drone overlayed with the yellow/green sphere is the target. The sphere turns green when the target is observed and yellow otherwise. The CF drones overlayed with blue spheres are agents, which turn red if the agents become unsafe. T…
Figure 12
Figure 12. Figure 12: h value with respect to distance. where sign is the sign function, and ν = 0.5 in all our experiments. This represents a linear function w.r.t. the inter-agent distance with a discontinuity at the safe-unsafe boundary ( [PITH_FULL_IMAGE:figures/full_fig_p022_12.png]
Figure 13
Figure 13. Figure 13: Cost and safety rate of Def-MARL and the baselines during training in MPE. wcontrol denote the corresponding weights of the different cost terms in the cost function l in (58) and (59). For the multi-agent MuJoCo environments, we first train the agents with (unconstra…
Figure 14
Figure 14. Figure 14: Cost and safety rate of Def-MARL and Lagr(lr) during training in MPE. 0 5 Step ×103 0 200 400 Cost Safe HalfCheetah 2x3 0 5 Step ×10 3 0.2 0.4 0.6 0.8 1.0 Safety rate Safe HalfCheetah 2x3 0 5 Step ×10 3 0 200 400 600 800 Cost Safe Coupled HalfCheetah 4x3 0 5 Step ×10 …
Figure 15
Figure 15. Figure 15: Cost and safety rate of Def-MARL and all baselines during training in Safe Multi-agent MuJoCo environments. Using the MoT form, the constraint-value function of the Lagrangian method becomes the same as the one used in Def-MARL (Equation (9)). We create 3 more baselin…
Figure 16
Figure 16. Figure 16: Cost and safety rate of Def-MARL and Lagr-MoT with different learning rates of λ during training in the TARGET environment, and the λ values during training. instability of the Lagrangian method. F. Sensitivity analysis on the choice of zmax In Appendix E-C, we have i…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

99 extracted references · 60 canonical work pages

  1. [1]

    Constrained policy optimization

    Joshua Achiam, David Held, Aviv Tamar, and Pieter Abbeel. Constrained policy optimization. In Interna- tional Conference on Machine Learning , pages 22–31. PMLR, 2017

    2017

  2. [2]

    Learning transferable cooperative be- havior in multi-agent team

    Akshat Agarwal, Sumit Kumar, Katia Sycara, and Michael Lewis. Learning transferable cooperative be- havior in multi-agent team. In International Conference on Autonomous Agents and Multiagent Systems (AA- MAS’2020). IFMAS, 2020

    2020

  3. [3]

    Constrained Markov decision processes

    Eitan Altman. Constrained Markov decision processes . Routledge, 2004

    2004

  4. [4]

    Casadi: a software framework for nonlinear optimization and optimal con- trol

    Joel AE Andersson, Joris Gillis, Greg Horn, James B Rawlings, and Moritz Diehl. Casadi: a software framework for nonlinear optimization and optimal con- trol. Mathematical Programming Computation, 11:1–36, 2019

    2019

  5. [5]

    Hamilton-jacobi reachability: A brief overview and recent advances

    Somil Bansal, Mo Chen, Sylvia Herbert, and Claire J Tomlin. Hamilton-jacobi reachability: A brief overview and recent advances. In 2017 IEEE 56th Annual Confer- ence on Decision and Control (CDC) , pages 2242–2253. IEEE, 2017

    2017

  6. [6]

    Dynamic programming and optimal control: Volume I, volume 4

    Dimitri Bertsekas. Dynamic programming and optimal control: Volume I, volume 4. Athena scientific, 2012

    2012

  7. [7]

    Synthesis of minimum-cost shields for multi-agent systems

    Suda Bharadwaj, Roderik Bloem, Rayna Dimitrova, Bettina Konighofer, and Ufuk Topcu. Synthesis of minimum-cost shields for multi-agent systems. In ACC. IEEE, 2019

    2019

  8. [8]

    An actor-critic algorithm for constrained markov decision processes

    Vivek S Borkar. An actor-critic algorithm for constrained markov decision processes. Systems & Control Letters , 54(3):207–213, 2005

    2005

Show all 99 references
  1. [9]

    Stochastic Approximation: A Dynamical Systems Viewpoint, volume 48

    Vivek S Borkar. Stochastic Approximation: A Dynamical Systems Viewpoint, volume 48. Springer, 2009

    2009

  2. [10]

    Convex optimization

    Stephen P Boyd and Lieven Vandenberghe. Convex optimization. Cambridge university press, 2004

    2004

  3. [11]

    Safe multi-agent reinforcement learning through decentralized multiple control barrier functions

    Zhiyuan Cai, Huanhui Cao, Wenjie Lu, Lin Zhang, and Hao Xiong. Safe multi-agent reinforcement learning through decentralized multiple control barrier functions. arXiv preprint arXiv:2103.12553 , 2021

    2021 arXiv

  4. [12]

    A new hybrid quadratic/bisection algorithm for finding the zero of a nonlinear function without using derivatives

    Tirupathi R Chandrupatla. A new hybrid quadratic/bisection algorithm for finding the zero of a nonlinear function without using derivatives. Advances in Engineering Software , 28(3):145–149, 1997

    1997

  5. [13]

    Socially aware motion planning with deep re- inforcement learning

    Yu Fan Chen, Michael Everett, Miao Liu, and Jonathan P How. Socially aware motion planning with deep re- inforcement learning. In 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , pages 1343–1350. IEEE, 2017

    2017

  6. [14]

    Decentralized non-communicating multiagent col- lision avoidance with deep reinforcement learning

    Yu Fan Chen, Miao Liu, Michael Everett, and Jonathan P How. Decentralized non-communicating multiagent col- lision avoidance with deep reinforcement learning. In 2017 IEEE International Conference on Robotics and Automation (ICRA), pages 285–292. IEEE, 2017

    2017

  7. [15]

    On the duality gap of constrained cooperative multi-agent reinforcement learning

    Ziyi Chen, Yi Zhou, and Heng Huang. On the duality gap of constrained cooperative multi-agent reinforcement learning. In The Twelfth International Conference on Learning Representations, 2024

    2024

  8. [16]

    Computational aspects of distributed optimization in model predictive control

    Christian Conte, Tyler Summers, Melanie N Zeilinger, Manfred Morari, and Colin N Jones. Computational aspects of distributed optimization in model predictive control. In 2012 IEEE 51st IEEE conference on decision and control (CDC) , pages 6819–6824. IEEE, 2012

    2012

  9. [17]

    De- tecting, localizing, and tracking an unknown number of moving targets using a team of mobile robots

    Philip Dames, Pratap Tokekar, and Vijay Kumar. De- tecting, localizing, and tracking an unknown number of moving targets using a team of mobile robots. The International Journal of Robotics Research , 36(13-14): 1540–1553, 2017

    2017

  10. [18]

    Provably efficient gener- alized lagrangian policy optimization for safe multi-agent reinforcement learning

    Dongsheng Ding, Xiaohan Wei, Zhuoran Yang, Zhaoran Wang, and Mihailo Jovanovic. Provably efficient gener- alized lagrangian policy optimization for safe multi-agent reinforcement learning. In Learning for Dynamics and Control Conference, pages 315–332. PMLR, 2023

    2023

  11. [19]

    Safe multi-agent reinforcement learning via shielding

    Ingy ElSayed-Aly, Suda Bharadwaj, Christopher Am- ato, R ¨udiger Ehlers, Ufuk Topcu, and Lu Feng. Safe multi-agent reinforcement learning via shielding. arXiv preprint arXiv:2101.11196, 2021

    2021 arXiv

  12. [20]

    Safe multi- agent reinforcement learning via shielding

    Ingy ElSayed-Aly, Suda Bharadwaj, Christopher Amato, R¨udiger Ehlers, Ufuk Topcu, and Lu Feng. Safe multi- agent reinforcement learning via shielding. AAMAS ’21, 2021

    2021

  13. [21]

    Mo- tion planning among dynamic, decision-making agents with deep reinforcement learning

    Michael Everett, Yu Fan Chen, and Jonathan P How. Mo- tion planning among dynamic, decision-making agents with deep reinforcement learning. In 2018 IEEE/RSJ In- ternational Conference on Intelligent Robots and Systems (IROS), pages 3052–3059. IEEE, 2018

    2018

  14. [22]

    A distributed model predictive control strategy for constrained multi- agent systems: The uncertain target capturing scenario

    Giuseppe Fedele and Giuseppe Franz `e. A distributed model predictive control strategy for constrained multi- agent systems: The uncertain target capturing scenario. IEEE Transactions on Automation Science and Engineer- ing, 2023

    2023

  15. [23]

    Counterfactual multi-agent policy gradients

    Jakob Foerster, Gregory Farquhar, Triantafyllos Afouras, Nantas Nardelli, and Shimon Whiteson. Counterfactual multi-agent policy gradients. In Proceedings of the AAAI conference on artificial intelligence , volume 32, 2018

    2018

  16. [24]

    Iterative reachability estimation for safe reinforcement learning

    Milan Ganai, Zheng Gong, Chenning Yu, Sylvia Herbert, and Sicun Gao. Iterative reachability estimation for safe reinforcement learning. Advances in Neural Information Processing Systems, 36, 2024

    2024

  17. [25]

    Learning safe control for multi- robot systems: Methods, verification, and open chal- lenges

    Kunal Garg, Songyuan Zhang, Oswin So, Charles Daw- son, and Chuchu Fan. Learning safe control for multi- robot systems: Methods, verification, and open chal- lenges. Annual Reviews in Control , 57:100948, 2024

    2024

  18. [26]

    A reinforce- ment learning framework for vehicular network routing under peak and average constraints

    Nan Geng, Qinbo Bai, Chenyi Liu, Tian Lan, Vaneet Aggarwal, Yuan Yang, and Mingwei Xu. A reinforce- ment learning framework for vehicular network routing under peak and average constraints. IEEE Transactions on Vehicular Technology, 2023

    2023

  19. [27]

    Crazyflie 2.0 quadrotor as a platform for research and education in robotics and control engineering

    Wojciech Giernacki, Mateusz Skwierczy ´nski, Wojciech Witwicki, Paweł Wro´nski, and Piotr Kozierski. Crazyflie 2.0 quadrotor as a platform for research and education in robotics and control engineering. In 2017 22nd International Conference on Methods and Models in Automation ...

    2017

  20. [28]

    Snopt: An sqp algorithm for large-scale constrained optimization

    Philip E Gill, Walter Murray, and Michael A Saunders. Snopt: An sqp algorithm for large-scale constrained optimization. SIAM review, 47(1):99–131, 2005

    2005

  21. [29]

    Nonlinear model predictive control: theory and algorithms

    Lars Grne and Jrgen Pannek. Nonlinear model predictive control: theory and algorithms . Springer Publishing Company, Incorporated, 2013

    2013

  22. [30]

    Multi-agent constrained policy optimisation

    Shangding Gu, Jakub Grudzien Kuba, Munning Wen, Ruiqing Chen, Ziyan Wang, Zheng Tian, Jun Wang, Alois Knoll, and Yaodong Yang. Multi-agent constrained policy optimisation. arXiv preprint arXiv:2110.02793 , 2021

    2021 arXiv

  23. [31]

    A review of safe reinforcement learning: Methods, theory and applications

    Shangding Gu, Long Yang, Yali Du, Guang Chen, Florian Walter, Jun Wang, Yaodong Yang, and Alois Knoll. A review of safe reinforcement learning: Methods, theory and applications. arXiv preprint arXiv:2205.10330, 2022

    2022 arXiv

  24. [32]

    Safe multi-agent reinforcement learning for multi-robot control

    Shangding Gu, Jakub Grudzien Kuba, Yuanpei Chen, Yali Du, Long Yang, Alois Knoll, and Yaodong Yang. Safe multi-agent reinforcement learning for multi-robot control. Artificial Intelligence, 319:103905, 2023

    2023

  25. [33]

    Coordinated reinforcement learning

    Carlos Guestrin, Michail Lagoudakis, and Ronald Parr. Coordinated reinforcement learning. In ICML, volume 2, pages 227–234. Citeseer, 2002

    2002

  26. [34]

    Deep recurrent q-learning for partially observable mdps

    Matthew Hausknecht and Peter Stone. Deep recurrent q-learning for partially observable mdps. In 2015 aaai fall symposium series , 2015

    2015

  27. [35]

    Autocost: Evolving intrinsic cost for zero-violation reinforcement learning

    Tairan He, Weiye Zhao, and Changliu Liu. Autocost: Evolving intrinsic cost for zero-violation reinforcement learning. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 37, pages 14847–14855, 2023

    2023

  28. [36]

    Safedreamer: Safe reinforcement learning with world models

    Weidong Huang, Jiaming Ji, Chunhe Xia, Borong Zhang, and Yaodong Yang. Safedreamer: Safe reinforcement learning with world models. In The Twelfth International Conference on Learning Representations , 2024

    2024

  29. [37]

    Distributed optimization in multi-agent robotics for industry 4.0 warehouses

    Ajay Kattepur, Hemant Kumar Rath, Anantha Simha, and Arijit Mukherjee. Distributed optimization in multi-agent robotics for industry 4.0 warehouses. In Proceedings of the 33rd Annual ACM Symposium on Applied Computing, pages 808–815, 2018

    2018

  30. [38]

    Cmix: Deep multi- agent reinforcement learning with peak and average con- straints

    Chenyi Liu, Nan Geng, Vaneet Aggarwal, Tian Lan, Yuan Yang, and Mingwei Xu. Cmix: Deep multi- agent reinforcement learning with peak and average con- straints. In Machine Learning and Knowledge Discovery in Databases. Research Track: European Conference, ECML PKDD 2021, Bilbao...

    2021

  31. [39]

    Towards optimally decentralized multi-robot collision avoidance via deep reinforcement learning

    Pinxin Long, Tingxiang Fan, Xinyi Liao, Wenxi Liu, Hao Zhang, and Jia Pan. Towards optimally decentralized multi-robot collision avoidance via deep reinforcement learning. In 2018 IEEE International Conference on Robotics and Automation (ICRA) , pages 6252–6259. IEEE, 2018

    2018

  32. [40]

    Multi-agent actor- critic for mixed cooperative-competitive environments

    Ryan Lowe, Yi I Wu, Aviv Tamar, Jean Harb, OpenAI Pieter Abbeel, and Igor Mordatch. Multi-agent actor- critic for mixed cooperative-competitive environments. Advances in neural information processing systems , 30, 2017

    2017

  33. [41]

    Decentralized policy gradient descent ascent for safe multi-agent reinforcement learning

    Songtao Lu, Kaiqing Zhang, Tianyi Chen, Tamer Bas ¸ar, and Lior Horesh. Decentralized policy gradient descent ascent for safe multi-agent reinforcement learning. In Proceedings of the AAAI Conference on Artificial Intel- ligence, volume 35, pages 8767–8775, 2021

    2021

  34. [42]

    Trajectory generation for multiagent point-to-point transitions via distributed model predictive control

    Carlos E Luis and Angela P Schoellig. Trajectory generation for multiagent point-to-point transitions via distributed model predictive control. IEEE Robotics and Automation Letters, 4(2):375–382, 2019

    2019

  35. [43]

    Online trajectory generation with distributed model predictive control for multi-robot motion planning

    Carlos E Luis, Marijan Vukosavljev, and Angela P Schoellig. Online trajectory generation with distributed model predictive control for multi-robot motion planning. IEEE Robotics and Automation Letters , 5(2):604–611, 2020

    2020

  36. [44]

    On reachability and minimum cost optimal control

    John Lygeros. On reachability and minimum cost optimal control. Automatica, 40(6):917–927, 2004

    2004

  37. [45]

    Lifelong multi-agent path finding for online pickup and delivery tasks

    Hang Ma, Jiaoyang Li, TK Kumar, and Sven Koenig. Lifelong multi-agent path finding for online pickup and delivery tasks. arXiv preprint arXiv:1705.10868 , 2017

    2017 arXiv

  38. [46]

    Hamilton–jacobi formulation for reach–avoid differential games

    Kostas Margellos and John Lygeros. Hamilton–jacobi formulation for reach–avoid differential games. IEEE Transactions on automatic control , 56(8):1849–1861, 2011

    2011

  39. [47]

    Safe value functions

    Pierre-Franc ¸ois Massiani, Steve Heim, Friedrich Solowjow, and Sebastian Trimpe. Safe value functions. IEEE Transactions on Automatic Control , 68(5): 2743–2757, 2023

    2023

  40. [48]

    Shield decentralization for safe multi-agent reinforce- ment learning

    Daniel Melcer, Christopher Amato, and Stavros Tripakis. Shield decentralization for safe multi-agent reinforce- ment learning. In Advances in Neural Information Processing Systems, 2022

    2022

  41. [49]

    A time-dependent hamilton-jacobi formulation of reachable sets for continuous dynamic games

    Ian M Mitchell, Alexandre M Bayen, and Claire J Tomlin. A time-dependent hamilton-jacobi formulation of reachable sets for continuous dynamic games. IEEE Transactions on automatic control, 50(7):947–957, 2005

    2005

  42. [50]

    Distributed model predictive safety certification for learning-based control

    Simon Muntwiler, Kim P Wabersich, Andrea Carron, and Melanie N Zeilinger. Distributed model predictive safety certification for learning-based control. IFAC- PapersOnLine, 53(2):5258–5265, 2020

    2020

  43. [51]

    Scalable multi-agent reinforcement learning through intelligent information aggregation

    Siddharth Nayak, Kenneth Choi, Wenqi Ding, Sydney Dolan, Karthik Gopalakrishnan, and Hamsa Balakrish- nan. Scalable multi-agent reinforcement learning through intelligent information aggregation. In International Conference on Machine Learning , pages 25817–25833. PMLR, 2023

    2023

  44. [52]

    Distributed optimization for control

    Angelia Nedi ´c and Ji Liu. Distributed optimization for control. Annual Review of Control, Robotics, and Autonomous Systems, 1:77–103, 2018

    2018

  45. [53]

    Numerical opti- mization

    Jorge Nocedal and Stephen J Wright. Numerical opti- mization. Springer, 1999

    1999

  46. [54]

    Facmac: Factored multi- agent centralised policy gradients

    Bei Peng, Tabish Rashid, Christian Schroeder de Witt, Pierre-Alexandre Kamienny, Philip Torr, Wendelin B¨ohmer, and Shimon Whiteson. Facmac: Factored multi- agent centralised policy gradients. Advances in Neural Information Processing Systems, 34:12208–12221, 2021

    2021

  47. [55]

    Decentralized safe multi- agent stochastic optimal control using deep fbsdes and admm

    Marcus A Pereira, Augustinos D Saravanos, Oswin So, and Evangelos A Theodorou. Decentralized safe multi- agent stochastic optimal control using deep fbsdes and admm. arXiv preprint arXiv:2202.10658 , 2022

    2022 arXiv

  48. [56]

    Learning safe multi-agent control with decentralized neural barrier certificates

    Zengyi Qin, Kaiqing Zhang, Yuxiao Chen, Jingkai Chen, and Chuchu Fan. Learning safe multi-agent control with decentralized neural barrier certificates. In International Conference on Learning Representations , 2021

    2021

  49. [57]

    Weighted qmix: Expanding monotonic value function factorisation for deep multi-agent reinforcement learning

    Tabish Rashid, Gregory Farquhar, Bei Peng, and Shimon Whiteson. Weighted qmix: Expanding monotonic value function factorisation for deep multi-agent reinforcement learning. Advances in neural information processing systems, 33:10199–10210, 2020

    2020

  50. [58]

    Monotonic value function factorisation for deep multi-agent reinforcement learning

    Tabish Rashid, Mikayel Samvelyan, Christian Schroeder De Witt, Gregory Farquhar, Jakob Foerster, and Shimon Whiteson. Monotonic value function factorisation for deep multi-agent reinforcement learning. Journal of Machine Learning Research , 21(178):1–51, 2020

    2020

  51. [59]

    A stochastic approx- imation method

    Herbert Robbins and Sutton Monro. A stochastic approx- imation method. The Annals of Mathematical Statistics , pages 400–407, 1951

    1951

  52. [60]

    Con- strained markov decision processes via backward value functions

    Harsh Satija, Philip Amortila, and Joelle Pineau. Con- strained markov decision processes via backward value functions. In International Conference on Machine Learning, pages 8502–8511. PMLR, 2020

    2020

  53. [61]

    Trust region policy optimiza- tion

    John Schulman, Sergey Levine, Pieter Abbeel, Michael Jordan, and Philipp Moritz. Trust region policy optimiza- tion. In International Conference on Machine Learning , pages 1889–1897. PMLR, 2015

    2015

  54. [62]

    High-dimensional continuous control using generalized advantage estimation

    John Schulman, Philipp Moritz, Sergey Levine, Michael Jordan, and Pieter Abbeel. High-dimensional continuous control using generalized advantage estimation. arXiv preprint arXiv:1506.02438, 2015

    2015 arXiv

  55. [63]

    Proximal policy optimization algorithms

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347 , 2017

    2017 arXiv

  56. [64]

    Multi-agent motion planning for dense and dynamic environments via deep reinforcement learning

    Samaneh Hosseini Semnani, Hugh Liu, Michael Everett, Anton De Ruiter, and Jonathan P How. Multi-agent motion planning for dense and dynamic environments via deep reinforcement learning. IEEE Robotics and Automation Letters, 5(2):3221–3226, 2020

    2020

  57. [65]

    Masked label predic- tion: Unified message passing model for semi-supervised classification

    Yunsheng Shi, Zhengjie Huang, Shikun Feng, Hui Zhong, Wenjin Wang, and Yu Sun. Masked label predic- tion: Unified message passing model for semi-supervised classification. arXiv preprint arXiv:2009.03509 , 2020

    2009 arXiv

  58. [66]

    Solving stabilize-avoid optimal control via epigraph form and deep reinforce- ment learning

    Oswin So and Chuchu Fan. Solving stabilize-avoid optimal control via epigraph form and deep reinforce- ment learning. In Proceedings of Robotics: Science and Systems, 2023

    2023

  59. [67]

    Solving minimum-cost reach avoid using reinforcement learning

    Oswin So, Cheng Ge, and Chuchu Fan. Solving minimum-cost reach avoid using reinforcement learning. In The Thirty-eighth Annual Conference on Neural In- formation Processing Systems , 2024

    2024

  60. [68]

    Predictive control of aerial swarms in cluttered environ- ments

    Enrica Soria, Fabrizio Schiano, and Dario Floreano. Predictive control of aerial swarms in cluttered environ- ments. Nature Machine Intelligence, 3(6):545–554, 2021

    2021

  61. [69]

    Value-decomposition networks for cooperative multi-agent learning

    Peter Sunehag, Guy Lever, Audrunas Gruslys, Woj- ciech Marian Czarnecki, Vinicius Zambaldi, Max Jader- berg, Marc Lanctot, Nicolas Sonnerat, Joel Z Leibo, Karl Tuyls, et al. Value-decomposition networks for cooperative multi-agent learning. arXiv preprint arXiv:1706.05296, 2017

    2017 arXiv

  62. [70]

    Mankowitz, and Shie Mannor

    Chen Tessler, Daniel J. Mankowitz, and Shie Mannor. Reward constrained policy optimization. In International Conference on Learning Representations , 2019

    2019

  63. [71]

    A game theoretic approach to controller design for hybrid systems

    Claire J Tomlin, John Lygeros, and S Shankar Sastry. A game theoretic approach to controller design for hybrid systems. Proceedings of the IEEE, 88(7):949–970, 2000

    2000

  64. [72]

    Decentralized multi-agent planning using model predictive control and time-aware safe corridors

    Charbel Toumieh and Alain Lambert. Decentralized multi-agent planning using model predictive control and time-aware safe corridors. IEEE Robotics and Automa- tion Letters, 7(4):11110–11117, 2022

    2022

  65. [73]

    Initial guess generation for aircraft landing trajec- tory optimization

    Panagiotis Tsiotras, Efstathios Bakolas, and Yiming Zhao. Initial guess generation for aircraft landing trajec- tory optimization. In AIAA Guidance, Navigation, and Control Conference, page 6689, 2011

    2011

  66. [74]

    Qplex: Duplex dueling multi-agent q- learning

    Jianhao Wang, Zhizhou Ren, Terry Liu, Yang Yu, and Chongjie Zhang. Qplex: Duplex dueling multi-agent q- learning. arXiv preprint arXiv:2008.01062 , 2020

    2008 arXiv

  67. [75]

    A synthesis approach of distributed model predictive control for homogeneous multi-agent system with collision avoidance

    Peng Wang and Baocang Ding. A synthesis approach of distributed model predictive control for homogeneous multi-agent system with collision avoidance. Interna- tional Journal of Control , 87(1):52–63, 2014

    2014

  68. [76]

    Multi-agent deep reinforcement learning for urban traffic light control in vehicular networks

    Tong Wu, Pan Zhou, Kai Liu, Yali Yuan, Xiumin Wang, Huawei Huang, and Dapeng Oliver Wu. Multi-agent deep reinforcement learning for urban traffic light control in vehicular networks. IEEE Transactions on Vehicular Technology, 69(8):8243–8256, 2020

    2020

  69. [77]

    Model-based dynamic shielding for safe and efficient multi-agent re- inforcement learning

    Wenli Xiao, Yiwei Lyu, and John Dolan. Model-based dynamic shielding for safe and efficient multi-agent re- inforcement learning. arXiv preprint arXiv:2304.06281 , 2023

    2023 arXiv

  70. [78]

    Crpo: A new approach for safe reinforcement learning with convergence guarantee

    Tengyu Xu, Yingbin Liang, and Guanghui Lan. Crpo: A new approach for safe reinforcement learning with convergence guarantee. In International Conference on Machine Learning, pages 11480–11491. PMLR, 2021

    2021

  71. [79]

    Qat- ten: A general framework for cooperative multiagent re- inforcement learning

    Yaodong Yang, Jianye Hao, Ben Liao, Kun Shao, Guangyong Chen, Wulong Liu, and Hongyao Tang. Qat- ten: A general framework for cooperative multiagent re- inforcement learning. arXiv preprint arXiv:2002.03939 , 2020

    2002 arXiv

  72. [80]

    The surprising effectiveness of ppo in cooperative multi-agent games

    Chao Yu, Akash Velu, Eugene Vinitsky, Jiaxuan Gao, Yu Wang, Alexandre Bayen, and Yi Wu. The surprising effectiveness of ppo in cooperative multi-agent games. Advances in Neural Information Processing Systems , 35: 24611–24624, 2022

    2022

  73. [81]

    Reachability constrained reinforcement learning

    Dongjie Yu, Haitong Ma, Shengbo Li, and Jianyu Chen. Reachability constrained reinforcement learning. In International conference on machine learning , pages 25636–25655. PMLR, 2022

    2022

  74. [82]

    Safe reinforcement learning using robust mpc

    Mario Zanon and S ´ebastien Gros. Safe reinforcement learning using robust mpc. IEEE Transactions on Auto- matic Control, 66(8):3638–3652, 2020

    2020

  75. [83]

    Fully decentralized multi-agent re- inforcement learning with networked agents

    Kaiqing Zhang, Zhuoran Yang, Han Liu, Tong Zhang, and Tamer Basar. Fully decentralized multi-agent re- inforcement learning with networked agents. In Inter- national conference on machine learning , pages 5872–

  76. [84]

    Multi- agent reinforcement learning: A selective overview of theories and algorithms

    Kaiqing Zhang, Zhuoran Yang, and Tamer Bas ¸ar. Multi- agent reinforcement learning: A selective overview of theories and algorithms. Handbook of reinforcement learning and control , pages 321–384, 2021

    2021

  77. [85]

    Neu- ral graph control barrier functions guided distributed collision-avoidance multi-agent control

    Songyuan Zhang, Kunal Garg, and Chuchu Fan. Neu- ral graph control barrier functions guided distributed collision-avoidance multi-agent control. In Conference on Robot Learning , pages 2373–2392. PMLR, 2023

    2023

  78. [86]

    Discrete GCBF proximal policy optimization for multi-agent safe optimal control

    Songyuan Zhang, Oswin So, Mitchell Black, and Chuchu Fan. Discrete GCBF proximal policy optimization for multi-agent safe optimal control. In The Thirteenth International Conference on Learning Representations , 2025

    2025

  79. [87]

    GCBF+: A neural graph control barrier function framework for distributed safe multiagent control

    Songyuan Zhang, Oswin So, Kunal Garg, and Chuchu Fan. GCBF+: A neural graph control barrier function framework for distributed safe multiagent control. IEEE Transactions on Robotics , 41:1533–1552, 2025

    2025

  80. [88]

    Mamps: Safe multi-agent reinforcement learning via model pre- dictive shielding

    Wenbo Zhang, Osbert Bastani, and Vijay Kumar. Mamps: Safe multi-agent reinforcement learning via model pre- dictive shielding. arXiv preprint arXiv:1910.12639 , 2019

    1910 arXiv

  81. [89]

    Model-free safe control for zero-violation reinforcement learning

    Weiye Zhao, Tairan He, and Changliu Liu. Model-free safe control for zero-violation reinforcement learning. In 5th Annual Conference on Robot Learning , 2021

    2021

  82. [90]

    Multi-agent first order con- strained optimization in policy space

    Youpeng Zhao, Yaodong Yang, Zhenbo Lu, Wengang Zhou, and Houqiang Li. Multi-agent first order con- strained optimization in policy space. Advances in Neural Information Processing Systems , 36, 2024

    2024

  83. [91]

    Fast, on-line collision avoidance for dynamic vehicles using buffered voronoi cells

    Dingjiang Zhou, Zijian Wang, Saptarshi Bandyopadhyay, and Mac Schwager. Fast, on-line collision avoidance for dynamic vehicles using buffered voronoi cells. IEEE Robotics and Automation Letters, 2(2):1047–1054, 2017

    2017

  84. [92]

    Trajectory optimization for nonlinear multi-agent systems using decentralized learning model predictive control

    Edward L Zhu, Yvonne R St ¨urz, Ugo Rosolia, and Francesco Borrelli. Trajectory optimization for nonlinear multi-agent systems using decentralized learning model predictive control. In 2020 59th IEEE Conference on Decision and Control (CDC) , pages 6198–6203. IEEE, 2020. APPEN...

    2020

  85. [93]

    In other words, for a given z0, the value at the kth timestep is only a function of zk and xk instead of the z0 and the entire trajectory up to the kth timestep

    Satisfying dynamic programming implies that the value function is Markovian. In other words, for a given z0, the value at the kth timestep is only a function of zk and xk instead of the z0 and the entire trajectory up to the kth timestep

  86. [94]

    Consequently, this implies that the optimal policy will also be Markovian and is only a function of zk and xk

  87. [95]

    Rephrased differently, since the value function is Markovian, this implies that, for a given z0 and x0, the value at the kth timestep is equal to the value (at the initial timestep) of a new problem where we start with ˜z0 =zk and ˜x0 =xk

  88. [96]

    Since we relate the value function of consecutive timesteps, given a value function estimator, we can now control the bias-variance tradeoff of the value function estimate by using k-step estimates instead of the Monte Carlo estimates

  89. [97]

    APPENDIX D ALGORITHM PSEUDOCODE We describe the centralized training process of Def-MARL in Algorithm 1 and the distributed execution process in Algorithm 2

    Instead of only using the k-step estimates for a single choice of k, we can compute a weighted average of the k-step estimates as in GAE to further control the bias-variance tradeoff. APPENDIX D ALGORITHM PSEUDOCODE We describe the centralized training process of Def-MARL in A...

  90. [98]

    E⊆{ (i,j )|i∈V a,j ∈V} is the set of edges, denoting the information flow from a sender node j to a receiver agent i

    Multi-partical environments (MPE): We use directed graphs G = (V,E) to represent MPE, where V is the set of nodes containing the objects in the multi-agent environment (e.g., agents Va, goals Vg, landmarks Vl, and obstacles Vo). E⊆{ (i,j )|i∈V a,j ∈V} is the set of edges, deno...

  91. [99]

    Safe multi-agent MuJoCo environments: We also test on the S AFE HALF CHEETAH (2X3) and S AFE COUPLED HALF CHEETAH (4X3) tasks from the Safe Multi-Agent Mujoco benchmark suite [32]. Each agent controls a subset of joints and must cooperate to minimize the cost (which we take to...

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.