REVIEW 4 major objections 6 minor 53 references
Achieving Collective Welfare in Multi-Agent Reinforcement Learning via Suggestion Sharing
T0 review · 4 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read This paper proposes Suggestion Sharing, a multi-agent reinforcement learning method where agents exchange only action suggestions, and proves a lower bound showing that maximizing the suggestion-based surrogate improves the collective…
desk verdict New suggestion-sharing mechanism with credible experiments, but the theoretical guarantee in Theorem 1 is weaker than claimed because of the dropped f^{pi_old} constant. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the suggesting joint policy, $\tilde{\pi}_i(a|s)=\prod_j \pi_{ij}(a_j|s)$, where $\pi_{ii}$ is agent $i$'s own policy and $\pi_{ij}$ is the action suggestion agent $i$ holds for agent $j$. The theoretical engine is Theorem 1, which lower-bounds $\eta(\pi^{\mathrm{new}})$ by $\eta(\pi^{\mathrm{old}}) + \zeta^{\pi^{\mathrm{old}}}(\tilde{\Pi}^{\mathrm{new}}) - C\sum_i D_{\mathrm{KL}}^{\max}(\pi_{ii}^{\mathrm{old}}\|\pi_{ii}^{\mathrm{new}}) - f^{\pi^{\mathrm{old}}} - \sum_i \frac{1}{2}\max_{s,a}|A_i^{\pi^{\mathrm{old}}}| \sum_{s,a}(\tilde{\pi}_i^{\mathrm{new}}(a|s)-\pi^{\mathrm{new}}(a|s))^2$; the first two terms form the surrogate objective, and the penalties become constraints in the per-agent optimization. The practical algorithm implements the bound through a dual clipped objective (Eq. 16) that combines a PPO-style clipped ratio for the agent's own policy with penalty terms activated exactly when a gradient step would widen the discrepancy between suggestions and actual policies.
What would settle it
On a finite-state social dilemma (or a discretized version of Cooperative Predation), compute the right-hand side of Theorem 1 from logged policies and rewards and compare it with the realized collective return; the theorem is falsified if the realized return is ever below the bound.
Extended reading notes
Core claim
On the paper's own terms, the central discovery is that the gap between individual and collective objectives in MARL can be controlled by the discrepancy between the policies agents actually follow and the suggestions they exchange. Specifically, Theorem 1 states that the collective return of a new joint policy is at least the return of the old policy plus the expected sum of individual advantages evaluated under each agent's suggesting joint policy $\tilde{\pi}_i$, minus a KL-divergence penalty on each agent's own policy change and minus a squared-discrepancy penalty between the suggesting joint policies and the true joint policy. Because the objective decomposes per agent, each agent can maximize its own advantage subject to constraints that keep its suggestions close to others' actual policies and its own policy close to others' suggestions. The paper then turns this bound into the Suggestion Sharing algorithm, in which agents share only action distributions and suggestions, and shows empirically that this suffices to reach cooperation in social dilemmas.
Load-bearing premise
The load-bearing premise is that the stated lower bound actually applies, which requires finite state and action spaces and suggestions that stay close to others' true policies; if either condition fails, the theoretical justification for SS collapses.
Editorial extensions
If this is right
- If Theorem 1 holds, maximizing the suggestion-based surrogate under the stated constraints improves a lower bound on collective return, so agents can cooperate without a shared reward or value signal.
- Because only action distributions and suggestions are exchanged, SS is designed to reveal less private information than methods that share rewards, value functions, or policy parameters, making it relevant where such sharing is infeasible.
- The constraint penalties in Eq. 12 are what keep the suggestions informative; the ablation with $\rho=0$ shows that without them performance drops, confirming the mechanism.
- In sequential social dilemmas like Cooperative Predation, SS converges to joint cooperation with a C-C rate near 1 despite individual incentives to defect.
- Because the objective decomposes into per-agent terms, SS can run with only local communication with neighbours, and the scalability study shows that reducing communication frequency or the number of neighbours still yields competitive performance.
Reading between the lines
- Beyond the paper's own claims, a testable extension is to verify whether the stated bound holds in continuous-state dilemmas: Theorem 1 sums over states and actions, while the Cooperative Predation environment uses a continuous state space, so the empirical result there is not covered by the theorem.
- The suggestion-sharing mechanism can be read as a soft contracting protocol: because each agent optimizes suggestions that would benefit itself, the exchanged suggestions carry implicit information about the sender's interests; quantifying that leakage (for instance, via mutual information between suggestions and rewards) would sharpen the paper's qualitative privacy claim.
- The assumption that agents truthfully share suggestions is an untested vulnerability; injecting one deceptive agent that sends suggestions steering others toward its own benefit is a direct stress test for whether collective welfare survives without trust mechanisms.
- The KL-decomposition property for product policies is generic, so the lower-bound proof may transfer to other trust-region MARL settings that keep the same per-agent decomposition, such as team-reward or mixed-motive games.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes Suggestion Sharing (SS), a multi-agent reinforcement learning method for collective welfare under individual rewards. Each agent learns a suggesting joint policy composed of its own policy and action suggestions for other agents, and agents exchange only these action distributions rather than rewards, values, or full policies. The authors derive a TRPO-style lower bound on the collective return (Theorem 1) in terms of the sum of individual advantages evaluated under the suggesting joint policies, minus KL divergence penalties, a joint-action discrepancy penalty, and a constant f^{π_old}. From this bound they formulate a constrained surrogate objective (Eqs. 11--12) and a practical PPO-style objective with clipping and indicator penalties (Eq. 16). Experiments on Cleanup, Harvest, Cooperative Navigation, and Cooperative Predation show that SS performs competitively with baselines that share values, policies, or use intrinsic rewards.
Significance. If fully established, the theoretical claim would be significant: SS offers a privacy-motivated communication protocol that exchanges only action suggestions, and the empirical results suggest it can resolve sequential social dilemmas without value, reward, or policy sharing. The paper includes detailed proofs in the appendix and a clean ablation (ρ = 0) showing that the suggestion-based penalty is essential for performance. However, the theoretical support for the algorithm is currently incomplete: the lower bound is shifted by a negative constant, the practical objective is heuristic rather than derived from the bound, and the finite-state theorem does not cover the continuous-state C. Predation experiments. The empirical contribution is useful, but the claim that the algorithm is supported by the lower-bound analysis needs substantial reframing.
major comments (4)
- [4.2, Eqs. (10)--(11)] The lower bound in Theorem 1 contains the negative constant −f^{π_old}. Evaluating at the incumbent policy with exact suggestions, π_new = π_old and tildeπ_i_new = π_old for all i, reduces Eq. (10) to η(π_old) ≥ η(π_old) − f^{π_old}, so the bound is slack by the positive amount f^{π_old}. Since dropping a constant does not change the argmax, maximizing Eq. (11) is not certified to produce a policy with η(π_new) ≥ η(π_old); the theorem only relates η(π_new) to a shifted quantity. The boundedness argument in Section 4.2 is not sufficient: a bounded negative constant still weakens the guarantee. The authors should either add a condition such as ζ^{π_old}(tildeΠ_new) − C·Σ_i D_KL^max(π_ii_old||π_ii_new) − discrepancy ≥ f^{π_old}, or explicitly state that the surrogate is a heuristic motivated by the lower bound rather than a certified monotone improvement step. This is load-bearing because the abstract and introduction claim that SS is supported by a lower-bound analysis.
- [4.3, Eq. (16)] The practical objective in Eq. (16) is not shown to follow from the constrained problem in Eq. (12) or from Theorem 1. It replaces the hard constraints of Eq. (12) with indicator penalties whose activation sets X_ij and X_ii are defined by comparing signed quantities in Eq. (14); the accompanying text explains only the case A_i > 0, and for negative advantages the inequality condition reverses. More importantly, no argument establishes that optimizing Eq. (16) decreases the joint-action discrepancy term in Theorem 1 or satisfies constraints (b) and (c) of Eq. (12). The penalty coefficient ρ is a free parameter that is tuned separately per environment (Table 3). Consequently, the empirical performance of SS is not formally explained by Theorem 1; the paper should either prove a connection between Eq. (16) and the theoretical surrogate, or present Eq. (16) as a heuristic algorithm motivated by, but not guaranteed by, the theory.
- [5.1 and Theorem 1] Theorem 1, Lemma 1, and Lemma 2 are proved for finite state and action spaces: the proofs sum over s ∈ S, use |A| (the size of the joint action space), and use ∥d^{π'}∥_2^2 = Σ_s (d^{π'}(s))^2. In C. Predation (Section 5.1), the state space is continuous, X = [0,30], so f^{π_old} and the discrepancy sums over states are not defined. Since C. Predation is the main sequential social dilemma environment used to demonstrate the method (Figs. 1, 3--5), the stated theorem does not cover a central part of the empirical evaluation. The paper should either state the finite-state assumption explicitly and mark C. Predation as outside the theoretical scope, or provide a continuous-state extension.
- [4.2, Eqs. (11)--(12)] The constraints in Eq. (12) bound pairwise marginal discrepancies between π_ij and π_jj (constraint b) and between π_ii and π_ji (constraint c), while the penalty term in Theorem 1 is Σ_{s,a} (tildeπ_i_new(a|s) − π_new(a|s))^2 over joint actions. The paper does not provide an inequality relating these quantities. For product policies, the joint-action L2 distance is not bounded by the sum of marginal L2 distances in the form used here without additional constants depending on N and |A|. Thus, satisfying Eq. (12) does not by itself imply that the last penalty term in Eq. (10) is small, and the claimed decomposition of the surrogate objective is incomplete.
minor comments (6)
- [Appendix A.1.3, Eq. (37)] In the proof of Theorem 1, Eq. (37) writes max_s d^{π_old}(s)^2 inside the sum over states and actions; this should be d^{π_old}(s)^2 for the subsequent replacement by f^{π_old} via Eq. (31) to be valid. Also, the sentence 'Using Theorem 2' in the same proof should read 'Using Lemma 2'.
- [Appendix A.1.1, proof of Lemma 1] The relationship between total variation and KL divergence is written as (D_TV(p||q))^2 ≤ D_KL(q||q); this should be D_KL(p||q).
- [Eqs. (14) and (16)] The set X_ii in Eq. (14) depends on π_ji, which is different for each neighbor j, but in Eq. (16) it is written without a neighbor index inside the sum over j ∈ N_i; the notation should be clarified, for example by writing X_ii^{(j)} or by defining the set per neighbor.
- [Section 5.3, Figs. 1--2] The paper reports that 5 runs with different seeds were conducted, but the training curves and final results do not show error bars or confidence intervals; reporting variability would strengthen the claim that SS performs competitively with the baselines.
- [References] Several references are incomplete, including 'Siedler and Alpha' (missing title and venue) and 'Macy and Flache' (missing year and complete venue); the reference for 'Huang and Zhou' also lacks a year. Please complete the bibliography.
- [Table 3] The penalty coefficient ρ varies by orders of magnitude across environments (1e3 in Cleanup, 0.1 in Harvest and C. Predation, 1 in C. Navigation). Since the ablation in Fig. 4 indicates that this term is important, the selection procedure for ρ should be described, or a sensitivity analysis should be provided.
Circularity Check
No circularity: the lower-bound derivation is an independent TRPO-style argument; the noted f^{pi_old} slack and continuous-state gaps are correctness concerns, not definitional circularity.
full rationale
The paper's central claim is Theorem 1, which lower-bounds eta(pi_new) in terms of eta(pi_old), zeta^{pi_old}(Pi_tilde_new), KL penalties, the f^{pi_old} constant, and suggestion-policy discrepancies. This bound is derived from external TRPO results (Schulman et al., 2015), an independently stated KL property (Su and Lu, 2022), and the paper's own Lemma 2, whose proof is explicit in Appendix A.1.2 and does not presuppose the algorithm's objective. The surrogate objective in Eq. 11 is the policy-dependent part of that bound with constants discarded; maximizing a bound's variable part is standard constructive design, not circular reasoning, because the theorem is not defined in terms of the algorithm's objective and the bound is not fitted to experimental data. No parameter is fitted to a subset of outcomes and then relabeled as a prediction. The self-citations that appear (Jin et al. 2021, Qiu et al. 2023) are application-domain references and are not load-bearing for the theoretical claim. The manuscript's own statement that f^{pi_old} can be ignored because it is bounded (Section 4.2) is a mathematical-rigor concern: at pi_new = pi_old with exact suggestions, Eq. 10 reduces to eta(pi_old) >= eta(pi_old) - f^{pi_old}, so the lower bound is slack and maximizing Eq. 11 is not certified to improve collective return. This is a weakness of the stated guarantee, not a circular step, since the bound's content does not reduce to its inputs by definition. Similarly, applying finite-state sums to the continuous-state C. Predation environment is an applicability gap, not circularity. The empirical results are used only as evaluation, not as premises of the theorem. I therefore find no significant circularity.
Assumptions & free parameters
free parameters (1)
- penalty coefficient rho =
Cleanup 1e3, Harvest 0.1, C.Predation 0.1, C.Navigation 1
assumptions (5)
- standard math TRPO lower bound for policy improvement (Schulman et al. 2015)
- domain assumption Finite state and action spaces with well-defined sums over s,a and finite |A|
- domain assumption Agents truthfully share suggestions and incorporate received suggestions into policy updates
- ad hoc to paper The penalty sets X_ij in Eq. 14 enforce the discrepancy constraints (b) and (c) of Eq. 12
- ad hoc to paper Dropping f^{pi_old} from the bound in Eq. 10 does not change the argmax or harm the lower-bound guarantee after each iteration
invented entities (2)
-
suggesting joint policy pi_tilde_i = prod_j pi_ij
-
suggestion policy pi_ij
Cite this review
Pith. "Pith review of Achieving Collective Welfare in Multi-Agent Reinforcement Learning via Suggestion Sharing." pith.science (2026). https://pith.science/paper/HC5GQPNK
@misc{pith2026241212326,
author = {Pith},
title = {Pith review of: Achieving Collective Welfare in Multi-Agent Reinforcement Learning via Suggestion Sharing},
year = {2026},
howpublished = {\url{https://pith.science/paper/HC5GQPNK}},
note = {Machine review of arXiv:2412.12326}
}
read the original abstract
In human society, the conflict between self-interest and collective well-being often obstructs efforts to achieve shared welfare. Related concepts like the Tragedy of the Commons and Social Dilemmas frequently manifest in our daily lives. As artificial agents increasingly serve as autonomous proxies for humans, we propose a novel multi-agent reinforcement learning (MARL) method to address this issue - learning policies to maximise collective returns even when individual agents' interests conflict with the collective one. Unlike traditional cooperative MARL solutions that involve sharing rewards, values, and policies or designing intrinsic rewards to encourage agents to learn collectively optimal policies, we propose a novel MARL approach where agents exchange action suggestions. Our method reveals less private information compared to sharing rewards, values, or policies, while enabling effective cooperation without the need to design intrinsic rewards. Our algorithm is supported by our theoretical analysis that establishes a bound on the discrepancy between collective and individual objectives, demonstrating how sharing suggestions can align agents' behaviours with the collective objective. Experimental results demonstrate that our algorithm performs competitively with baselines that rely on value or policy sharing or intrinsic rewards.
Figures
Figures from the paper (5 more)
Reference graph
Works this paper leans on
-
[1]
Stefano V. Albrecht and Peter Stone. Autonomous agents modelling other agents: A comprehensive survey and open problems . Artificial Intelligence, 258 0 (September): 0 66--95, 2018. ISSN 00043702. doi:10.1016/j.artint.2018.01.002
-
[2]
Tianyi Chen, Kaiqing Zhang, Georgios B. Giannakis, and Tamer Basar. Communication-Efficient Policy Gradient Methods for Distributed Reinforcement Learning . IEEE Transactions on Control of Network Systems, 9 0 (2): 0 917--929, 2022. ISSN 23255870. doi:10.1109/TCNS.2021.3078100
arXiv 2022
-
[3]
Phillip J. K. Christoffersen, Andreas A. Haupt, and Dylan Hadfield-Menell. Get It in Writing: Formal Contracts Mitigate Social Dilemmas in Multi-Agent RL . Proceedings of the 2023 International Conference on Autonomous Agents and Multiagent Systems, pages 448--456, 2023. URL http://arxiv.org/abs/2208.10469
arXiv 2023
-
[4]
Multi-agent Reinforcement Learning for Networked System Control
Tianshu Chu, Sandeep Chinchali, and Sachin Katti. Multi-agent Reinforcement Learning for Networked System Control . International Conference on Learning Representations, 0 (1), 2020 a . URL http://arxiv.org/abs/2004.01339
arXiv 2020
-
[5]
Multi-Agent Deep Reinforcement Learning for Large-Scale Traffic Signal Control
Tianshu Chu, Jie Wang, Lara Codec \` a , and Zhaojian Li. Multi-Agent Deep Reinforcement Learning for Large-Scale Traffic Signal Control . IEEE Transactions on Intelligent Transportation Systems, 21 0 (3): 0 1086--1095, 2020 b . ISSN 15582914
work page 2020
-
[6]
Scalable Model-based Policy Optimization for Decentralized Networked Systems
Yali Du, Chengdong Ma, Yuchen Liu, Runji Lin, Hao Dong, Jun Wang, and Yaodong Yang. Scalable Model-based Policy Optimization for Decentralized Networked Systems . International Conference on Intelligent Robots and Systems (IROS), pages 9019--9026, 2022. URL http://arxiv.org/abs/2207.06559
work page Pith review arXiv 2022
-
[7]
Torr, Pushmeet Kohli, and Shimon Whiteson
Jakob Foerster, Nantas Nardell, Gregory Farquhar, Trtantafyllos Afouras, Philip H.S. Torr, Pushmeet Kohli, and Shimon Whiteson. Stabilising experience replay for deep multi-agent reinforcement learning . 34th International Conference on Machine Learning, ICML 2017, 3: 0 1879--1888, 2017
work page 2017
-
[8]
Mordechai Gersani, Joel S. Brown, Erin E. O'Brien, Godfrey M. Maina, and Zvika Abramsky. Tragedy of the commons as a result of root competition . Journal of Ecology, 89 0 (4): 0 660--669, 2001. ISSN 00220477. doi:10.1046/j.0022-0477.2001.00609.x
Show all 53 references
-
[9]
Hauser, Christian Hilbe, Krishnendu Chatterjee, and Martin A
Oliver P. Hauser, Christian Hilbe, Krishnendu Chatterjee, and Martin A. Nowak. Social dilemmas among unequals . Nature, 572 0 (7770): 0 524--527, 8 2019. ISSN 14764687. doi:10.1038/s41586-019-1488-5
2019 doi
-
[10]
Opponent modeling in deep reinforcement learning
He He, Jordan Boyd-Graber, Kevin Kwok, and Hal Daume. Opponent modeling in deep reinforcement learning . 33rd International Conference on Machine Learning, ICML 2016, 4: 0 2675--2684, 2016
2016
-
[11]
Importance-Aware Message Exchange and Prediction for Multi-Agent Reinforcement Learning
Xiufeng Huang and Sheng Zhou. Importance-Aware Message Exchange and Prediction for Multi-Agent Reinforcement Learning . 2022 IEEE Global Communications Conference, GLOBECOM 2022 - Proceedings, pages 6493--6498, 2022. doi:10.1109/GLOBECOM48099.2022.10001408
2022
-
[12]
Leibo, Matthew Phillips, and Karl Tuyls
Edward Hughes, Joel Z. Leibo, Matthew Phillips, and Karl Tuyls. Inequity aversion improves cooperation in intertemporal social dilemmas . Advances in Neural Information Processing Systems, pages 3326--3336, 2018. ISSN 10495258
2018
-
[13]
Actor-attention-critic for multi-agent reinforcement learning
Shariq Iqbal and Fei Sha. Actor-attention-critic for multi-agent reinforcement learning . 36th International Conference on Machine Learning, ICML 2019, 2019-June: 0 5261--5270, 2019
2019
-
[14]
Ortega, D
Natasha Jaques, Angeliki Lazaridou, Edward Hughes, Caglar Gulcehre, Pedro A. Ortega, D. J. Strouse, Joel Z. Leibo, and Nando de Freitas. Social influence as intrinsic motivation for multi-agent deep reinforcement learning . 36th International Conference on Machine Learning, IC...
2019
-
[15]
I2Q : A Fully Decentralized Q-Learning Algorithm
Jiechuan Jiang and Zongqing Lu. I2Q : A Fully Decentralized Q-Learning Algorithm . Advances in Neural Information Processing Systems, 35: 0 20469--20481, 2022
2022
-
[16]
Hierarchical and Stable Multiagent Reinforcement Learning for Cooperative Navigation Control
Yue Jin, Shuangqing Wei, Jian Yuan, and Xudong Zhang. Hierarchical and Stable Multiagent Reinforcement Learning for Cooperative Navigation Control . IEEE Transactions on Neural Networks and Learning Systems, 34 0 (1): 0 90--103, 2021. ISSN 21622388. doi:10.1109/TNNLS.2021.3089834
2021
-
[17]
Communication in Multi-Agent Reinforcement Learning: Intention Sharing
Woojun Kim, Jongeui Park, and Youngchul Sung. Communication in Multi-Agent Reinforcement Learning: Intention Sharing . ICLR, pages 1--15, 2021
2021
-
[18]
SOCIAL DILEMMAS: The Anatomy of Cooperation
Peter Kollock. SOCIAL DILEMMAS: The Anatomy of Cooperation . Technical report, 1998. URL www.sscnet.ucla.edu/soc/faculty/kollock/dilemmas
1998
-
[19]
Communication-Efficient and Federated Multi-Agent Reinforcement Learning
Mounssif Krouka, Anis Elgabli, Chaouki Ben Issaid, and Mehdi Bennis. Communication-Efficient and Federated Multi-Agent Reinforcement Learning . IEEE Transactions on Cognitive Communications and Networking, 8 0 (1): 0 311--320, 2022. ISSN 23327731. doi:10.1109/TCCN.2021.3130993
2022
-
[20]
Trust Region Policy Optimisation in Multi-Agent Reinforcement Learning
Jakub Grudzien Kuba, Ruiqing Chen, Muning Wen, Ying Wen, Fanglei Sun, Jun Wang, and Yaodong Yang. Trust Region Policy Optimisation in Multi-Agent Reinforcement Learning . International Conference on Learning Representations, page 1046, 2022
2022
-
[21]
Adaptive Stochastic ADMM for Decentralized Reinforcement Learning in Edge IoT
Wanlu Lei, Yu Ye, Ming Xiao, Mikael Skoglund, and Zhu Han. Adaptive Stochastic ADMM for Decentralized Reinforcement Learning in Edge IoT . IEEE Internet of Things Journal, 9 0 (22): 0 22958--22971, 2022. ISSN 23274662. doi:10.1109/JIOT.2022.3187067
2022
-
[22]
Leibo, Vinicius Zambaldi, Marc Lanctot, Janusz Marecki, and Thore Graepel
Joel Z. Leibo, Vinicius Zambaldi, Marc Lanctot, Janusz Marecki, and Thore Graepel. Multi-agent Reinforcement Learning in Sequential Social Dilemmas . Proceedings of the 16th International Conference on Autonomous Agents and Multiagent Systems, pages 464--473, 2017. URL http://...
2017 arXiv
-
[23]
Multi-agent actor-critic for mixed cooperative-competitive environments
Ryan Lowe, Yi Wu, Aviv Tamar, Jean Harb, Pieter Abbeel, and Igor Mordatch. Multi-agent actor-critic for mixed cooperative-competitive environments . Advances in Neural Information Processing Systems, 2017-Decem: 0 6380--6391, 2017. ISSN 10495258
2017
-
[24]
Learning dynamics in social dilemmas
Michael W Macy and Andreas Flache. Learning dynamics in social dilemmas . Technical report. URL www.pnas.orgcgidoi10.1073pnas.092080099
-
[25]
Milinski, D
M. Milinski, D. Semmann, and HJ. Krambeck. Reputation helps solve the `tragedy of the commons' . Nature, 415 0 (6870): 0 424--426, 1 2002. ISSN 00368075. doi:10.1126/science.1064748
2002 doi
-
[26]
Deep Decentralized Multi-task Multi-Agent Reinforcement Learning under Partial Observability
Shayegan Omidshafiei, Jason Pazis, Christopher Amato, Jonathan P How, and John Vian. Deep Decentralized Multi-task Multi-Agent Reinforcement Learning under Partial Observability . 2017. doi:10.5555/3305890.3305958
2017
-
[27]
E. Ostrom. Governing the commons: the evolution of institutions for collective action , volume 32. 1990. ISBN 0521371015. doi:10.2307/3146384
1990 doi
-
[28]
Schroeder de Witt, Pierre Alexandre Kamienny, Philip H.S
Bei Peng, Tabish Rashid, Christian A. Schroeder de Witt, Pierre Alexandre Kamienny, Philip H.S. Torr, Wendelin B \" o hmer, and Shimon Whiteson. FACMAC: Factored Multi-Agent Centralised Policy Gradients . Advances in Neural Information Processing Systems, 15 0 (NeurIPS): 0 122...
2021
-
[29]
Improving Sample Efficiency of Multi-Agent Reinforcement Learning with Non-expert Policy for Flocking Control
Yunbo Qiu, Yue Jin, Lebin Yu, Jian Wang, Yu Wang, and Xudong Zhang. Improving Sample Efficiency of Multi-Agent Reinforcement Learning with Non-expert Policy for Flocking Control . IEEE Internet of Things Journal, 10 0 (14): 0 14014--14027, 2023. doi:10.1109/JIOT.2023.3240671
2023
-
[30]
Trust region policy optimization
John Schulman, Sergey Levine, Philipp Moritz, Michael Jordan, and Pieter Abbeel. Trust region policy optimization . 32nd International Conference on Machine Learning, ICML 2015, 3: 0 1889--1897, 2015
2015
-
[31]
Jordan, and Pieter Abbeel
John Schulman, Philipp Moritz, Sergey Levine, Michael I. Jordan, and Pieter Abbeel. High-dimensional continuous control using generalized advantage estimation . 4th International Conference on Learning Representations, ICLR 2016 - Conference Track Proceedings, pages 1--14, 2016
2016
-
[32]
Proximal Policy Optimization Algorithms
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal Policy Optimization Algorithms . arXiv preprint arXiv:1707.06347, 2017
2017 arXiv
-
[33]
Policy evaluation for reinforcement learning over asynchronous multi-agent networks
Xingyu Sha, Jiaqi Zhang, and Keyou You. Policy evaluation for reinforcement learning over asynchronous multi-agent networks . Chinese Control Conference, CCC, 2021-July: 0 5373--5378, 2021. ISSN 21612927. doi:10.23919/CCC52363.2021.9550466
2021
-
[34]
Dynamic Collaborative Multi-Agent Reinforcement Learning Communication for Autonomous Drone Reforestation
Philipp D Siedler and Aleph Alpha. Dynamic Collaborative Multi-Agent Reinforcement Learning Communication for Autonomous Drone Reforestation . 0 (NeurIPS 2022)
2022
-
[35]
Stankovic, Marko Beko, and Srdjan S
Milos S. Stankovic, Marko Beko, and Srdjan S. Stankovic. Distributed Actor-Critic Learning Using Emphatic Weightings . 2022 8th International Conference on Control, Decision and Information Technologies, CoDIT 2022, pages 1167--1172, 2022 a . doi:10.1109/CoDIT55151.2022.9804022
2022
-
[36]
Stankovic, Marko Beko, and Srdjan S
Miloš S. Stankovic, Marko Beko, and Srdjan S. Stankovic. Convergent Distributed Actor-Critic Algorithm Based on Gradient Temporal Difference . European Signal Processing Conference, 2022-Augus: 0 2066--2070, 2022 b . ISSN 22195491. doi:10.23919/eusipco55093.2022.9909762
2022
-
[37]
Decentralized Policy Optimization
Kefan Su and Zongqing Lu. Decentralized Policy Optimization . arXiv preprint arXiv:2211.03032, 2022
2022 arXiv
-
[38]
Chuangchuang Sun, Macheng Shen, and Jonathan P. How. Scaling up multiagent reinforcement learning for robotic systems: Learn an adaptive sparse communication graph . IEEE International Conference on Intelligent Robots and Systems, pages 11755--11762, 2020. ISSN 21530866. doi:1...
2020
-
[39]
Trust Region Bounds for Decentralized PPO Under Non-stationarity
Mingfei Sun, Sam Devlin, Jacob Beck, Katja Hofmann, and Shimon Whiteson. Trust Region Bounds for Decentralized PPO Under Non-stationarity . Proceedings of the 2023 International Conference on Autonomous Agents and Multiagent Systems, pages 5--13, 2022. URL http://arxiv.org/abs...
2023 arXiv
-
[40]
A multi-agent off-policy actor-critic algorithm for distributed reinforcement learning
Wesley Suttle, Zhuoran Yang, Kaiqing Zhang, Zhaoran Wang, Tamer Basar, and Ji Liu. A multi-agent off-policy actor-critic algorithm for distributed reinforcement learning . IFAC-PapersOnLine, 53: 0 1549--1554, 2020. ISSN 24058963. doi:10.1016/j.ifacol.2020.12.2021
2020 doi
-
[41]
Modeling Moral Choices in Social Dilemmas with Multi-Agent Reinforcement Learning
Elizaveta Tennant, Stephen Hailes, and Mirco Musolesi. Modeling Moral Choices in Social Dilemmas with Multi-Agent Reinforcement Learning . arXiv preprint arXiv:2301.08491, 2023. URL https://arxiv.org/abs/2301.08491v1
2023 arXiv
-
[42]
Van Lange, Jeff Joireman, Craig D
Paul A.M. Van Lange, Jeff Joireman, Craig D. Parks, and Eric Van Dijk. The psychology of social dilemmas: A review . Organizational Behavior and Human Decision Processes, 120 0 (2): 0 125--141, 3 2013. ISSN 07495978. doi:10.1016/j.obhdp.2012.11.003
2013 doi
-
[43]
Distributed Reinforcement Learning for Robot Teams: A Review
Yutong Wang, Mehul Damani, Pamela Wang, Yuhong Cao, and Guillaume Sartoretti. Distributed Reinforcement Learning for Robot Teams: A Review . 2022. URL http://arxiv.org/abs/2204.03516
2022 arXiv
-
[44]
Probabilistic recursive reasoning for multi-agent reinforcement learning
Ying Wen, Yaodong Yang, Rui Luo, Jun Wang, and Wei Pan. Probabilistic recursive reasoning for multi-agent reinforcement learning . 7th International Conference on Learning Representations, ICLR 2019, pages 1--20, 2019
2019
-
[45]
Coordinated Proximal Policy Optimization
Zifan Wu, Chao Yu, Deheng Ye, Junge Zhang, Haiyin Piao, and Hankz Hankui Zhuo. Coordinated Proximal Policy Optimization . Advances in Neural Information Processing Systems, 32: 0 26437--26448, 2021. ISSN 10495258
2021
-
[46]
Multi-Agent Reinforcement Learning Aided Intelligent UAV Swarm for Target Tracking
Zhaoyue Xia, Jun Du, Jingjing Wang, Chunxiao Jiang, Yong Ren, Gang Li, and Zhu Han. Multi-Agent Reinforcement Learning Aided Intelligent UAV Swarm for Target Tracking . IEEE Transactions on Vehicular Technology, 71 0 (1): 0 931--945, 2022. ISSN 19399359. doi:10.1109/TVT.2021.3129504
2022
-
[47]
Learning to Share in Multi-Agent Reinforcement Learning
Yuxuan Yi, Ge Li, Yaowei Wang, and Zongqing Lu. Learning to Share in Multi-Agent Reinforcement Learning . ICLR 2022 Workshop on Gamification and Multiagent Solutions, 2022. URL http://arxiv.org/abs/2112.08702
2022 arXiv
-
[48]
Networked Multi-Agent Reinforcement Learning in Continuous Spaces
Kaiqing Zhang, Zhuoran Yang, and Tamer Basar. Networked Multi-Agent Reinforcement Learning in Continuous Spaces . Proceedings of the IEEE Conference on Decision and Control, 2018-Decem 0 (Cdc): 0 2771--2776, 2018 a . ISSN 25762370. doi:10.1109/CDC.2018.8619581
2018
-
[49]
Fully decentralized multi-agent reinforcement learning with networked agents
Kaiqing Zhang, Zhuoran Yang, Han Liu, Tong Zhang, and Tamer Ba s ar. Fully decentralized multi-agent reinforcement learning with networked agents . 35th International Conference on Machine Learning, ICML 2018, 13: 0 9340--9371, 2018 b
2018
-
[50]
Finite-sample analysis for decentralized cooperative multi-agent reinforcement learning from batch data
Kaiqing Zhang, Zhuoran Yang, Han Liu, Tong Zhang, and Tamer Basar. Finite-sample analysis for decentralized cooperative multi-agent reinforcement learning from batch data . IFAC-PapersOnLine, 53 0 (2): 0 1049--1056, 2020. ISSN 24058963. doi:10.1016/j.ifacol.2020.12.1290
2020 doi
-
[51]
Zavlanos
Yan Zhang and Michael M. Zavlanos. Distributed off-Policy Actor-Critic Reinforcement Learning with Policy Consensus . Proceedings of the IEEE Conference on Decision and Control, 2019-Decem 0 (Cdc): 0 4674--4679, 2019. ISSN 25762370. doi:10.1109/CDC40024.2019.9029969
2019
-
[52]
Distributed policy evaluation via inexact ADMM in multi-agent reinforcement learning
Xiaoxiao Zhao, Peng Yi, and Li Li. Distributed policy evaluation via inexact ADMM in multi-agent reinforcement learning . Control Theory and Technology, 18 0 (4): 0 362--378, 2020. ISSN 21980942. doi:10.1007/s11768-020-00007-x
2020 doi
-
[53]
A deep Bayesian policy reuse approach against non-stationary agents
Yan Zheng, Zhaopeng Meng, Jianye Hao, Zongzhang Zhang, Tianpei Yang, and Changjie Fan. A deep Bayesian policy reuse approach against non-stationary agents . Advances in Neural Information Processing Systems, 2018-Decem 0 (NeurIPS): 0 954--964, 2018. ISSN 10495258
2018
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.