Pith. sign in

REVIEW 4 major objections 4 minor 1 cited by

Scalable Safe Multi-Agent Reinforcement Learning for Multi-Agent System

T0 review · 4 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read SS-MARL claims safe multi-agent reinforcement learning that scales zero-shot from 3 to 96 agents by combining graph message passing with constrained joint policy optimization.

desk verdict The paper's central safety claim is falsified by its own Table 2, but the multi-constraint recovery step and GNN architecture are worth a second look. read the letter →

arxiv 2501.13727 v2 pith:PVGIMSWA submitted 2025-01-23 cs.MA cs.AI

Haikuo Du , Fandi Gou , Yunze Cai This is my paper · ORCID
classification cs.MAcs.AI
keywords safemulti-agentreinforcementlearningconstrainedMarkovgamegraphneuralnetworkzero-shottransferscalabilitytrust-regionpolicyoptimizationcooperativenavigationsystems
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to solve two problems at once: multi-agent reinforcement learning that respects hard safety constraints instead of soft reward penalties, and policies that transfer to far more agents than they were trained with. It claims that both can be achieved in one framework, called SS-MARL, by combining a graph neural network that aggregates variable-size local observations and communications with a constrained joint policy optimization that keeps expected collision costs under a user-set bound. On cooperative navigation tasks, the paper reports that SS-MARL reaches goals while keeping collision costs lower than reward-shaping baselines, and that a model trained with three agents keeps a high success rate when tested with up to 96 agents. A sympathetic reader would take the central claim to be that safety constraints and scalability are not in conflict, and that both can be delivered by one architecture plus one constrained updating rule.

What carries the argument

The load-bearing mechanism is a graph-attention message-passing backbone with two aggregation modes. The actor uses agent aggregation, selecting the learned embedding of the agent's own vertex after several message-passing rounds, so its input dimension never depends on agent count; the critic and cost critic use graph aggregation that attention-pools all agent vertices into fixed-length reward and cost value estimates. On the optimization side, the paper adapts trust-region policy optimization to the multi-agent setting and adds a weighted recovery step for the case where a policy update violates several cost constraints at once.

What would settle it

Run the final SS-MARL(PS) policy from Table 2 for 100 test episodes at n=3, 6, and 9 and record the distribution of per-agent per-episode costs; if mean costs stay above the declared c=1 bound (2.06, 4.11, and 7.00 respectively), the central safety claim is refuted as stated.

Watch

Extended reading notes

Core claim

SS-MARL's central proposal is a combination: encode the multi-agent system as a graph whose vertices are agents, goals, and obstacles; pass messages along perception and communication edges with attention; aggregate per-agent to form the actor's local representation and graph-wide to form the critic and cost critic; then update policies with a trust-region constrained optimization that maximizes a reward surrogate subject to each agent's cost constraints. The paper argues that this yields monotonic reward improvement and cost constraint satisfaction in the idealized setting where value functions are known, and that the network's fixed-length representations let the same policy act in environments with different numbers of agents. Empirically it claims SS-MARL outperforms fixed-input safe MARL and reward-shaping baselines in both reward and collision cost, and demonstrates zero-shot transfer from three agents to 96 agents.

Load-bearing premise

The safety guarantee assumes the value functions used in the constrained update are exact; when they are approximated by learned critics operating on local observations and communication, the guarantee may break.

Editorial extensions

If this is right

  • A single trained policy can be deployed in environments with many more agents than seen in training, since the actor's input and output sizes do not depend on the number of agents.
  • Safety becomes an adjustable upper bound on expected per-episode cost, so the same algorithm can be tuned between conservative and reward-seeking behavior by changing one parameter.
  • The recovery step extends constrained multi-agent updates to settings with more than one cost constraint per agent, which earlier trust-region safe MARL methods do not handle.
  • Because policy sharing among homogeneous agents speeds convergence while retaining safety, the method is practical for large homogeneous fleets.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial extension: The safety guarantee is only as strong as the learned cost critics that replace the exact value functions used in the theory; the paper's own Table 2 reports average costs of 2.06, 4.11, and 7.00 against a declared bound of c=1, so in the reported test setting the constraint is not actually met.
  • Editorial extension: The attention weights over communication edges could be inspected to see whether agents learn to ignore far-away or irrelevant entities; if they do, the method might scale to even larger swarms with bounded communication load.
  • Editorial extension: A natural next test is whether zero-shot transfer holds under shifted dynamics, such as denser obstacles or heterogeneous agents, where the graph structure changes more dramatically than simply adding more identical agents.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes SS-MARL, a framework for safe and scalable multi-agent reinforcement learning. It combines a graph neural network backbone with message passing over agents, obstacles, and goals to obtain policies that accept variable-size observations, and a constrained joint policy optimization step derived from MACPO/CPO that includes a weighted recovery rule intended to handle multiple cost constraints. The authors evaluate the method on a modified MPE environment (Safe MPE) in cooperative navigation, compare it with RMAPPO, RMACPO, and InforMARL, and demonstrate zero-shot transfer from n=3 to up to n=96 agents, including hardware experiments.

Significance. If the safety guarantees were valid, the paper would be a useful contribution: it offers an end-to-end architecture for variable-size multi-agent systems, a trust-region recovery step for multiple constraints, and evidence of zero-shot scalability, all in a safety-constrained setting. The paper also ships an environment modification (Safe MPE) and hardware validation, which are practical strengths. However, the central claim that SS-MARL 'satisfies the cost constraint' is contradicted by the paper's own experimental table, and the theoretical feasibility proof is largely imported from prior work and is not shown to survive the approximations used in implementation. As a result, the main advertised advantage over reward-shaping baselines is not established by the evidence.

major comments (4)
  1. [§5.2, Table 2] The central safety claim is contradicted by the paper's own evaluation. With c = 1 defined in §5.2 as the expected per-episode cost per agent, and Appendix §2.3 confirming that c = 1 corresponds to an undiscounted episode bound, Table 2 reports SS-MARL(PS) test costs of 2.06, 4.11, and 7.00 for n = 3, 6, and 9. These exceed the bound by factors of 2–7. The sentence in §5.2 stating that at n = 9 SS-MARL 'not only obtains positive rewards but also satisfies the cost constraint' is therefore directly inconsistent with the table. If the table is instead reporting a different metric (e.g., discounted episode cost), the units must be stated and the comparison to c must be re-derived; as written, the paper does not support its headline safety claim.
  2. [Appendix §1.1, Theorem 1; §2.1] The theoretical feasibility guarantee is not established for the algorithm actually implemented. Theorem 1 relies on Lemma 1.2, which is stated as a bound from [Gu et al., 2023] under the assumption that exact state value functions are available, as the paper itself notes in §2.1. In the implemented method, the advantages in Eq. (5) are computed with learned critics, GAE estimates, and local observations over a communication graph. No error bound, pessimism correction, or robustness argument is provided to show that the constrained update and the recovery step preserve feasibility under these approximations. The empirical violation in Table 2 is consistent with this gap, so the theory cannot be used to repair the safety claim.
  3. [§4.2, Eq. (9); Appendix §1.2] The proposed multi-constraint recovery step is a heuristic rather than a proven mechanism. The update in Eq. (9) moves along a weighted combination of the constraint gradients, with weights defined in Eq. (8) for currently violated constraints. The paper does not prove that a backtracking line search along this direction decreases every violated constraint; with multiple constraints sharing a Hessian, reducing a weighted sum does not guarantee reduction of each individual cost. Moreover, all experiments use a single cost per agent (collisions), so the multi-constraint recovery is never empirically tested. Either a proof or an experiment with m_i >= 2 is needed to support the claim that SS-MARL handles multiple constraints.
  4. [§5.3, Figure 6] The scalability results are presented without reference to the safety constraint used during training. The text states that SS-MARL trained with n = 3 can be scaled to 96 agents while 'maintaining collision times per agent below 5', but it does not state the cost bound c for these transfer experiments. If the same c = 1 applies, the reported average episode costs per agent below 5 also violate the constraint; if a larger bound is allowed, the safety claim needs to be qualified accordingly. Please report the constraint value and the fraction of episodes satisfying it for the transferred policies.
minor comments (4)
  1. [§2.3] 'tranfer' should be 'transfer'.
  2. [§4.2] 'cosntraints' should be 'constraints'.
  3. [Table 2] The caption does not specify whether 'Cost' is per episode, per step, discounted, or undiscounted; please define the metric explicitly and align it with Eq. (1).
  4. [Figures 4 and 6] The text refers to 'average costs per step' in Figure 4 and 'average episode costs per agent' in Figure 6; use consistent terminology and units across all cost plots and tables.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; the central algorithm is not fitted to its outputs, and the safety guarantee rests on external prior work, with the Table 2 violation being a correctness issue rather than a circular one.

full rationale

The paper's derivation is self-contained in the relevant sense: the constrained joint policy optimization in Eq. (5) uses cost-advantage estimates from a learned cost critic and a recovery step whose weights beta_j are computed from estimated constraint violations; no parameter is fitted to reproduce the reported rewards, costs, or success rates, so the empirical results are not determined by construction. The theoretical guarantee of monotonic improvement and cost feasibility is imported from Kuba et al. (2022) and Gu et al. (2023), which are external works with no author overlap with the present paper, and the cited bounds apply to exact value functions, not to values fitted in this paper; per the review rules, such external citations are independent support and do not constitute circularity. The appendix's Theorem 1 proves the reward-improvement part directly and delegates the 'specific form' of the KL/cost limit to Gu et al. (2023), but this is a normal external-theorem dependency, not a self-citation chain. The apparent inconsistency between the claimed cost satisfaction in Section 5.2 and the reported costs in Table 2 (2.06, 4.11, 7.00 against c=1) is an empirical/correctness concern, not a circular-derivation concern; it does not show that any equation reduces to its input. No circular step was found.

Assumptions & free parameters 1 free parameters · 6 assumptions · 0 invented entities

No new physical entities or mediators are introduced. The main free input is the cost bound c chosen by the user; the theoretical proof is inherited from prior work (Kuba et al., Gu et al.) and assumes exact value functions.

free parameters (1)
  • Cost constraint upper bound c = 1 (comparative experiments), 1 and 6 (Section 5.1)
    Chosen by the experimenter. The safety claim depends on the algorithm respecting it, but Table 2 test costs exceed it.
assumptions (6)
  • standard math Multi-agent advantage decomposition (Kuba et al., 2021)
    Used in Lemma 1.1 to expand the joint surrogate return in Appendix Equation 3.
  • standard math Cost bounds lemma from [Gu et al., 2023] (Lemma 1.2)
    The paper states it as given and uses it for the cost-satisfaction part of Theorem 1; it is not proven in this paper.
  • standard math TRPO/CPO trust-region bounds and KL inequality
    Used to justify the surrogate objective and the recovery step (Appendix 1.2).
  • domain assumption Known state value functions for theoretical guarantees
    The monotonic improvement and cost-satisfaction theorem assumes exact value functions (as noted for MACPO in Section 2.1), while the implementation uses learned critics and local observations.
  • domain assumption Fully cooperative setting with factorized joint policy and shared reward
    Used throughout the problem formulation, Section 3.
  • domain assumption Graph structure with agents, goals, obstacles and communication range
    The GNN backbone relies on this graph representation; Section 4, Table 1.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Scalable Safe Multi-Agent Reinforcement Learning for Multi-Agent System." pith.science (2026). https://pith.science/paper/PVGIMSWA

@misc{pith2026250113727,
  author       = {Pith},
  title        = {Pith review of: Scalable Safe Multi-Agent Reinforcement Learning for Multi-Agent System},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PVGIMSWA}},
  note         = {Machine review of arXiv:2501.13727}
}
read the original abstract

Safety and scalability are two critical challenges faced by practical Multi-Agent Systems (MAS). However, existing Multi-Agent Reinforcement Learning (MARL) algorithms that rely solely on reward shaping are ineffective in ensuring safety, and their scalability is rather limited due to the fixed-size network output. To address these issues, we propose a novel framework, Scalable Safe MARL (SS-MARL), to enhance the safety and scalability of MARL methods. Leveraging the inherent graph structure of MAS, we design a multi-layer message passing network to aggregate local observations and communications of varying sizes. Furthermore, we develop a constrained joint policy optimization method in the setting of local observation to improve safety. Simulation experiments demonstrate that SS-MARL achieves a better trade-off between optimality and safety compared to baselines, and its scalability significantly outperforms the latest methods in scenarios with a large number of agents.

Figures

Figures reproduced from arXiv: 2501.13727 by the authors.

Figure 1
Figure 1. Overview of SS-MARL. AA: Agent Aggregation, GA: Graph Aggregation, GAE: Generalized Advantage Estimation. [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Visual Representation of message passing and aggrega [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 4
Figure 4. Scenarios with (a) c = 1 and (b) c = 6 and (c) average re￾wards per step per agent and average costs per step per agent during the training phase. From Figure 4c, it can be observed that the average costs per step per agent during the training phase gradually con￾verges around their expected upper bounds, with an episode length of 100 steps. Figures 4a and 4b illustrate the trajecto￾ries of final policies trained wi… view at source ↗
Figures from the paper (10 more)
Figure 3
Figure 3. Figure 3: Comparison of the training performance of SS-MARL with baselines. (a)(b)(c) are average rewards per step per agent during the [PITH_FULL_IMAGE:figures/full_fig_p006_3.png]
Figure 6
Figure 6. Figure 6: Comparison of the zero-shot transfer performance of SS [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 5
Figure 5. Figure 5: Zero-shot transfer to n = 24 using a model trained on scenarios with n = 3, when the test episode (left) begins and (right) ends During the testing phase, the graph that represents the agent’s communication and observation undergoes dynamic changes. Concurrently, the a…
Figure 1
Figure 1. Figure 1: Cooperative Navigation scenario in Safe MPE, when the test episode (left) begins and (right) ends [PITH_FULL_IMAGE:figures/full_fig_p014_1.png]
Figure 2
Figure 2. Figure 2: Formation task(a) and Line task(b), the blue circle is agent, the grey circle is landmark and the black lines between agents are [PITH_FULL_IMAGE:figures/full_fig_p015_2.png]
Figure 3
Figure 3. Figure 3: The training performance of SS-MARL on Formation and Line tasks. (a)(c) are average step rewards per agent during the training [PITH_FULL_IMAGE:figures/full_fig_p015_3.png]
Figure 4
Figure 4. Figure 4: Zero-shot transfer experiment on Formation task(a) and Line task(b), the blue circle is agent, the grey circle is landmark and the [PITH_FULL_IMAGE:figures/full_fig_p016_4.png]
Figure 5
Figure 5. Figure 5: RoverC, a programmable and omnidirectional mobile miniature vehicle. [PITH_FULL_IMAGE:figures/full_fig_p016_5.png]
Figure 6
Figure 6. Figure 6: Cooperative navigation task with 3 agents. The hardware implementation (a) and the trajectory (b). [PITH_FULL_IMAGE:figures/full_fig_p017_6.png]
Figure 7
Figure 7. Figure 7: Cooperative navigation task with 6 agents. The hardware implementation (a) and the trajectory (b). [PITH_FULL_IMAGE:figures/full_fig_p017_7.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. High-Precision Formation Control for Heterogeneous Multi-Robot Systems via Hierarchical Hybrid Physics-Informed Deep Reinforcement Learning

    cs.RO 2026-07 conditional novelty 4.0 of 10

    Hybrid physics-plus-PD-plus-residual-SAC control yields 100% simulated success for Ackermann-leader navigation and omnidirectional formation tracking, outperforming pure residual RL ablations.

Reference graph

Works this paper leans on

31 extracted references · 24 canonical work pages · cited by 1 Pith paper

  1. [1]

    Constrained policy optimization

    Joshua Achiam, David Held, Aviv Tamar, and Pieter Abbeel. Constrained policy optimization. In International Conference on Machine Learning (ICML) , pages 22--31. PMLR, 2017

    work page 2017

  2. [2]

    Learning transferable cooperative behavior in multi-agent teams

    Akshat Agarwal, Sumit Kumar, Katia Sycara, and Michael Lewis. Learning transferable cooperative behavior in multi-agent teams. In Proceedings of the 19th International Conference on Autonomous Agents and Multi-Agent Systems (AAMAS) , pages 1741--1743, 2020

    work page 2020

  3. [3]

    Safe learning in robotics: From learning-based control to safe reinforcement learning

    Lukas Brunke, Melissa Greeff, Adam W Hall, Zhaocong Yuan, Siqi Zhou, Jacopo Panerati, and Angela P Schoellig. Safe learning in robotics: From learning-based control to safe reinforcement learning. Annual Review of Control, Robotics, and Autonomous Systems , 5(1):411--444, 2022

    work page 2022

  4. [4]

    Drqn-based 3d obstacle avoidance with a limited field of view

    Yu’an Chen, Guangda Chen, Lifan Pan, Jun Ma, Yu Zhang, Yanyong Zhang, and Jianmin Ji. Drqn-based 3d obstacle avoidance with a limited field of view. In 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , pages 8137--8143. IEEE, 2021

    work page 2021

  5. [5]

    Safe rlhf: Safe reinforcement learning from human feedback

    Josef Dai, Xuehai Pan, Ruiyang Sun, Jiaming Ji, Xinbo Xu, Mickel Liu, Yizhou Wang, and Yaodong Yang. Safe rlhf: Safe reinforcement learning from human feedback. In 2024-12th International Conference on Learning Representations (ICLR) , 2024

    work page 2024

  6. [6]

    Safe multi-agent reinforcement learning for multi-robot control

    Shangding Gu, Jakub Grudzien Kuba, Yuanpei Chen, Yali Du, Long Yang, Alois Knoll, and Yaodong Yang. Safe multi-agent reinforcement learning for multi-robot control. Artificial Intelligence , page 103905, 2023

    work page 2023

  7. [7]

    Scalable communication for multi-agent reinforcement learning via transformer-based email mechanism

    Xudong Guo, Daming Shi, and Wenhui Fan. Scalable communication for multi-agent reinforcement learning via transformer-based email mechanism. In Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence (IJCAI) , pages 126--134, 2023

    work page 2023

  8. [8]

    Long short-term memory

    S Hochreiter. Long short-term memory. Neural Computation MIT-Press , 1997

    1997

Show all 31 references
  1. [9]

    Collision avoidance and navigation for a quadrotor swarm using end-to-end deep reinforcement learning

    Zhehui Huang, Zhaojing Yang, Rahul Krupani, Bask n S enba s lar, Sumeet Batra, and Gaurav S Sukhatme. Collision avoidance and navigation for a quadrotor swarm using end-to-end deep reinforcement learning. In 2024 IEEE International Conference on Robotics and Automation (ICRA) ...

    2024

  2. [10]

    Graph convolutional reinforcement learning

    Jiechuan Jiang, Chen Dun, Tiejun Huang, and Zongqing Lu. Graph convolutional reinforcement learning. In 2020-8th International Conference on Learning Representations (ICLR) , 2020

    2020

  3. [11]

    Semi-supervised classification with graph convolutional networks

    Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In 2017-5th International Conference on Learning Representations (ICLR) , 2017

    2017

  4. [12]

    Trust region policy optimisation in multi-agent reinforcement learning

    JG Kuba, R Chen, M Wen, Y Wen, F Sun, J Wang, and Y Yang. Trust region policy optimisation in multi-agent reinforcement learning. In 2022-10th International Conference on Learning Representations (ICLR) , page 1046, 2022

    2022

  5. [13]

    Cmix: Deep multi-agent reinforcement learning with peak and average constraints

    Chenyi Liu, Nan Geng, Vaneet Aggarwal, Tian Lan, Yuan Yang, and Mingwei Xu. Cmix: Deep multi-agent reinforcement learning with peak and average constraints. In Machine Learning and Knowledge Discovery in Databases. Research Track: European Conference, ECML PKDD 2021, Bilbao, S...

    2021

  6. [14]

    Multi-agent actor-critic for mixed cooperative-competitive environments

    Ryan Lowe, Yi Wu, Aviv Tamar, Jean Harb, Pieter Abbeel, and Igor Mordatch. Multi-agent actor-critic for mixed cooperative-competitive environments. Advances in Neural Information Processing Systems (NeurIPS) , 30, 2017

    2017

  7. [15]

    Deep learning for safe autonomous driving: Current challenges and future directions

    Khan Muhammad, Amin Ullah, Jaime Lloret, Javier Del Ser, and Victor Hugo C de Albuquerque. Deep learning for safe autonomous driving: Current challenges and future directions. IEEE Transactions on Intelligent Transportation Systems , 22(7):4316--4336, 2020

    2020

  8. [16]

    Scalable multi-agent reinforcement learning through intelligent information aggregation

    Siddharth Nayak, Kenneth Choi, Wenqi Ding, Sydney Dolan, Karthik Gopalakrishnan, and Hamsa Balakrishnan. Scalable multi-agent reinforcement learning through intelligent information aggregation. In International Conference on Machine Learning (ICML) , pages 25817--25833. PMLR, 2023

    2023

  9. [17]

    A concise introduction to decentralized POMDPs , volume 1

    Frans A Oliehoek, Christopher Amato, et al. A concise introduction to decentralized POMDPs , volume 1. Springer, 2016

    2016

  10. [18]

    Dealing with non-stationarity in multi-agent deep reinforcement learning

    Georgios Papoudakis, Filippos Christianos, Arrasy Rahman, and Stefano V Albrecht. Dealing with non-stationarity in multi-agent deep reinforcement learning. arXiv preprint arXiv:1906.04737 , 2019

    1906 arXiv

  11. [19]

    Monotonic value function factorisation for deep multi-agent reinforcement learning

    Tabish Rashid, Mikayel Samvelyan, Christian Schroeder De Witt, Gregory Farquhar, Jakob Foerster, and Shimon Whiteson. Monotonic value function factorisation for deep multi-agent reinforcement learning. Journal of Machine Learning Research , 21(178):1--51, 2020

    2020

  12. [20]

    Benchmarking safe exploration in deep reinforcement learning

    Alex Ray, Joshua Achiam, and Dario Amodei. Benchmarking safe exploration in deep reinforcement learning. arXiv preprint arXiv:1910.01708 , 7(1):2, 2019

    1910 arXiv

  13. [21]

    High-dimensional continuous control using generalized advantage estimation

    John Schulman, Philipp Moritz, Sergey Levine, Michael Jordan, and Pieter Abbeel. High-dimensional continuous control using generalized advantage estimation. arXiv preprint arXiv:1506.02438 , 2015

    2015 arXiv

  14. [22]

    Trust region policy optimization

    John Schulman. Trust region policy optimization. arXiv preprint arXiv:1502.05477 , 2015

    2015 arXiv

  15. [23]

    Masked label prediction: Unified message passing model for semi-supervised classification

    Yunsheng Shi, Zhengjie Huang, Shikun Feng, Hui Zhong, Wenjin Wang, and Yu Sun. Masked label prediction: Unified message passing model for semi-supervised classification. In Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence (IJCAI) , pages 1...

    2021

  16. [24]

    Learning multiagent communication with backpropagation

    Sainbayar Sukhbaatar, Rob Fergus, et al. Learning multiagent communication with backpropagation. Advances in Neural Information Processing Systems (NeurIPS) , 29, 2016

    2016

  17. [25]

    Relative distributed formation and obstacle avoidance with multi-agent reinforcement learning

    Yuzi Yan, Xiaoxiang Li, Xinyou Qiu, Jiantao Qiu, Jian Wang, Yu Wang, and Yuan Shen. Relative distributed formation and obstacle avoidance with multi-agent reinforcement learning. In 2022 IEEE International Conference on Robotics and Automation (ICRA) , pages 1661--1667. IEEE, 2022

    2022

  18. [26]

    The surprising effectiveness of ppo in cooperative multi-agent games

    Chao Yu, Akash Velu, Eugene Vinitsky, Jiaxuan Gao, Yu Wang, Alexandre Bayen, and Yi Wu. The surprising effectiveness of ppo in cooperative multi-agent games. Advances in Neural Information Processing Systems (NeurIPS) , 35:24611--24624, 2022

    2022

  19. [27]

    Attention-based reinforcement learning for real-time uav semantic communication

    Won Joon Yun, Byungju Lim, Soyi Jung, Young-Chai Ko, Jihong Park, Joongheon Kim, and Mehdi Bennis. Attention-based reinforcement learning for real-time uav semantic communication. In 2021 17th International Symposium on Wireless Communication Systems (ISWCS) , pages 1--6. IEEE, 2021

    2021

  20. [28]

    A survey of multi-agent deep reinforcement learning with communication

    Changxi Zhu, Mehdi Dastani, and Shihan Wang. A survey of multi-agent deep reinforcement learning with communication. In Proceedings of the 23rd International Conference on Autonomous Agents and Multiagent Systems (AAMAS) , pages 2845--2847, 2024

    2024

  21. [29]

    Reducing overestimation bias in multi-agent domains using double centralized critics

    Johannes Ackermann, Volker Gabler, Takayuki Osa, and Masashi Sugiyama. Reducing overestimation bias in multi-agent domains using double centralized critics. arXiv preprint arXiv:1910.01465 , 2019

    1910 arXiv

  22. [30]

    Settling the variance of multi-agent policy gradients

    Jakub Grudzien Kuba, Muning Wen, Linghui Meng, Haifeng Zhang, David Mguni, Jun Wang, Yaodong Yang, et al. Settling the variance of multi-agent policy gradients. Advances in Neural Information Processing Systems (NeurIPS) , 34:13458--13470, 2021

    2021

  23. [31]

    write newline

    " write newline "" before.all 'output.state := FUNCTION fin.entry add.period write newline FUNCTION new.block output.state before.all = 'skip after.block 'output.state := if FUNCTION new.sentence output.state after.block = 'skip output.state before.all = 'skip after.sentence '...

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.