Pith. sign in

REVIEW 4 major objections 5 minor 42 references

Networked Agents in the Dark: Team Value Learning under Partial Observability

T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that decentralized agents under partial observability can approximate a central critic by running consensus on both local value targets and model parameters, and that this outperforms previous networked-agent methods.

desk verdict Useful empirical extension of networked MARL to partial observability, but the team-value consensus mechanism is asserted without theory and the ablation data already shows it can hurt. read the letter →

arxiv 2501.08778 v1 pith:MHQFQCXV submitted 2025-01-15 cs.LG cs.AIcs.MA

Guilherme S. Varela , Alberto Sardinha , Francisco S. Melo This is my paper · ORCID
classification cs.LGcs.AIcs.MA
keywords multi-agentreinforcementlearningpartialobservabilityconsensusdecentralizedtrainingactor-criticdeepQ-networkteamvaluefunctionnetworkedagents
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that cooperative reinforcement learning does not require a central trainer or shared observations: agents that only see their own observations and collect their own rewards can still learn a shared team objective. To that end it introduces the networked dynamic partially observable Markov game and DNA-MARL, a class of algorithms that runs consensus steps on local value targets in addition to consensus on actor and critic parameters. The central claim is that averaging these local targets approximates the team value function, so each agent's policy gradient points toward the team advantage rather than only its own reward. If the claim holds, distributed systems in privacy-sensitive or lossy communication settings can approach the performance of centralized training without ever pooling observations. The paper's experiments on level-based foraging and multi-agent particle tasks show DNA-MARL matching or closely approaching the centralized baselines and outperforming earlier networked-agent methods.

What carries the argument

The load-bearing object is the team-value consensus step: agent $i$ turns its local TD target $y_t^i$ into a consensus variable and applies $K$ rounds of distributed averaging with Metropolis weights, which are local weights that guarantee asymptotic average consensus over a switching graph. This is the 'double' in DNA-MARL, because the same consensus operator is also applied to critic parameters and actor parameters, following earlier networked-agent work. The paper's idealized identity (15) asserts that the averaged target equals the team value, which is what turns local actor updates into updates in the direction of the team advantage. The empirical section then treats $K$ as a cooperation dial between independent learning ($K=0$) and full cooperative mode.

What would settle it

On a small task where the true team value can be computed exactly, measure the mean squared error between the consensus target $\bar{y}_t^i$ from (15) and the true central-critic target as $K$ increases; if DNA-MARL already matches the central critic while this error stays large, then the consensus approximation is not the source of the improvement, and if the error fails to shrink with $K$ while performance degrades toward independent learning, the central claim is refuted.

Watch

Extended reading notes

Core claim

The paper claims that DNA-MARL lets decentralized agents under partial observability approximate a central critic that sees the full system. The mechanism is a second consensus layer: after each episode agents compute their own TD target $y_t^i = r_{t+1}^i + \gamma V(o_{t+1}^i; \omega_-^i)$, average these scalars over a randomly switching communication graph for $K$ rounds, and use the averaged target $\bar{y}_t^i$ in both critic and actor updates. The authors assert, via equation (15), that the consensus average of local targets can stand in for the true team value $V(o;\omega)$, and they supplement it with periodic consensus on the critic and actor parameters from earlier work. Across six on-policy and six off-policy benchmark tasks, DNA-A2C and DNA-Q-learning generally track the centralized CTDE baselines more closely than prior decentralized methods do, and the ablation study attributes the gain to the team-value consensus. The intended conclusion is that a distributed, private, partially observable learning system can emulate centralized training.

Load-bearing premise

The method works only if the consensus average of local reward-plus-next-value estimates really approximates the team value function; the paper states this as an ideal approximation without an error bound, and the experiments use only 1 or 5 consensus rounds, so a biased average would feed systematic bias into every actor and critic update.

Editorial extensions

If this is right

  • In privacy-sensitive domains, agents can cooperate without sharing observations or rewards: only scalar value targets and averaged parameters leave the node.
  • The $K$ consensus rounds give a direct dial between independent learning ($K=0$) and fully cooperative behavior, so a system can tune how much communication budget buys how much cooperation.
  • Because the updates are agnostic to the base RL algorithm, the same double-averaging recipe transfers to on-policy methods such as TRPO or PPO and to off-policy Q-learning, as demonstrated with A2C and DQN.
  • If DNA-MARL closes most of the gap to centralized critics in the tested benchmarks, decentralized training with random packet dropout no longer requires a fully observable state or agent-chosen communication, covering applications such as power dispatch, packet routing, and sensor networks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable consequence the paper does not examine: the bias in (15) should grow with reward variance and network diameter, so the number of consensus rounds needed to match a central critic should scale with both; current results only probe small networks with $K \in \{1,5,10\}$.
  • The team-target consensus can also be viewed as a variance-reduction mechanism that might help even when agents have nothing to coordinate; comparing DNA-MARL against independent learners that simply average their rewards would separate the cooperative contribution from the noise-reduction contribution.
  • The failure in the off-policy Tag setting, where DNAQL only matches independent learning, suggests the max operator inside Q-learning targets may amplify consensus bias; swapping team-$V$ consensus for team-$Q$ consensus on identical tasks would localize whether the fragile part is the target type or the consensus step.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes DNA-MARL, a decentralized training and decentralized execution (DTDE) multi-agent reinforcement learning method for cooperative agents under partial observability. Agents communicate over a time-varying network to run consensus not only on actor and critic parameters, as in prior networked-agent work, but also on local value targets (team-V for A2C, team-Q for DQN). The paper introduces the ND-POMG framework, derives a factorized team policy gradient, and evaluates DNAA2C and DNAQL on Level-Based Foraging and Multi-Agent Particle Environment benchmarks against CTDE, DTDE, and independent-learner baselines. The central claims are that DNA-MARL outperforms previous networked-agent methods and that, despite randomized communication with limited rounds, it can emulate a central critic that uses system-wide observations.

Significance. If the central mechanism were sound, the paper would make a useful contribution to privacy-preserving and communication-limited cooperative MARL: it relaxes full observability assumptions in networked-agent methods and provides an open-source implementation with a systematic benchmark comparison using bootstrap confidence intervals and ablations. The formalization of ND-POMG is clear, and the experiments cover both on-policy and off-policy instantiations. However, the load-bearing step is the team-value consensus approximation in Eq. (15), which is asserted heuristically rather than derived or bounded. The empirical evidence is mixed: team-V consensus helps in some on-policy tasks, but the off-policy ablation shows team-Q consensus hurting performance in LBF. The contribution is therefore better characterized as an empirical heuristic with promising results than as a theoretically grounded networked MARL method.

major comments (4)
  1. [§4.1.3, Eq. (15)] The central approximation is asserted without proof or error bound. Even in the limit K→∞ with perfect consensus, the consensus target is (1/N)∑_j (r_j + γV(o_j'; ω_j^-)). For this to equal the central TD target r + γV(o'; ω^-), one must assume an additive decomposition V(o; ω) ≈ (1/N)∑_j V(o_j; ω_j) and a common parameter vector ω. Neither condition is stated, checked, or derived, and the finite parameter consensus used in the experiments (K=5 or K=1, C=1, Table 5) does not provide identical critics. The paper's own wording, 'Ideally, the following approximation will hold,' confirms that this is a heuristic, and no experiment verifies the approximation error directly. Because every actor and critic update in Eqs. (iii) and (iv) uses this quantity, the validity of the method rests on an unexamined assumption.
  2. [§4.1.2, Eq. (11)] The team policy gradient is derived for the centralized case in Eq. (10) and then localized by replacing the global state s with the local observation o_i and the team advantage A_θ with A_i. This replacement is described as 'standard practice,' but the paper provides no analysis of the resulting bias. The system dynamics still depend on the joint policy and the joint state, so the local gradient is not the true team policy gradient. If the approximation in Eq. (11) is poor, the actor updates are biased even when the critic is perfectly accurate. The paper should either provide a formal bias bound, a reference that establishes such a bound in this setting, or an ablation that measures the gap between Eq. (10) and Eq. (11) in a tractable environment.
  3. [§6.1 and Appendix D, Fig. 7] The off-policy ablation directly contradicts the general claim that team-value consensus emulates a central critic. In all three LBF instances, the DNA group (team-Q consensus plus parameter consensus) performs worse than the DQ group (parameter consensus only), and the paper's own hypothesis is that 'the best Q policy has the agents acting independently.' This is evidence that the team-Q consensus mechanism can inject a systematic bias rather than improve cooperation. The paper should either restrict the emulation claim to the on-policy setting, explain why team-Q consensus fails while team-V consensus succeeds, or provide additional evidence that the failure is an artifact of the specific Q-learning instantiation.
  4. [Appendix C, Table 5] The hyperparameters K, I, and C were selected by grid search on three seeds and then evaluated on the same tasks with ten seeds. Because the selection and evaluation use the same environments, the reported maximum average episodic returns may reflect selection bias toward configurations that happened to perform well in the small validation sample. The paper should discuss this limitation and, ideally, validate the selected hyperparameters on held-out tasks or report the variance across the grid search. This concern does not invalidate the empirical results, but it weakens the strength of the 'superior performance' claim.
minor comments (5)
  1. [§4.1.3, Eq. (15)] The summation index in Eq. (15) is inconsistent: the left-hand side is written as a sum over i of (1/N) terms, but the right-hand side should sum over the N agents; using the same symbol i on both sides is confusing. Please rewrite with distinct indices.
  2. [Algorithm 1, line 13] The pseudocode computes y_i^t = r_i^{t+1} + γV(o_{t+1}; ω_i), but the text in Eq. (13) uses the target network parameters ω_i^-; the pseudocode should be updated to match the target-network notation.
  3. [Appendix D, Fig. 7 caption] The caption says 'For six tasks the component that improves performance is consensus on the parameters and not team-Q consensus,' but only five panels are shown in the figure; please check the panel count and caption wording.
  4. [§5.2, DVA2C baseline] DVA2C is described as 'a model-based algorithm, whereby it has a neural network that estimates the discounted team return.' This phrasing is unclear, since the implementation follows Zhang et al. [38], which is a model-free networked actor-critic method; please clarify what 'model-based' means here.
  5. [Table 1, off-policy MPE Tag] The abstract claims 'superior performance of DNA-MARL over previous methods,' but in Table 1 DNAQL in MPE Tag (15.77) is far below VDN (23.27) and statistically indistinguishable from INDQL (15.54). The text should qualify the superiority claim in this setting.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central team-value consensus step is an explicitly labeled approximation, and reported gains are measured against external benchmarks.

full rationale

The derivation chain is not circular. The load-bearing step, Eq. (15), is introduced with the caveat "Ideally, the following approximation will hold" and is an asserted, unproven approximation rather than a quantity reconstructed from fitted parameters or from the paper's own output. The consensus average of local TD targets is not, by construction, the Bellman target of a central critic; for the claim to be valid one would need additivity of the value decomposition and enough consensus rounds to equalize the critics, and the paper provides no error bound. That is a soundness and justification gap, not a circular reduction: Eq. (15) is not derived from the algorithm's outputs, and no fitted parameter is renamed as a prediction. The empirical claims are evaluated against external baselines (MAA2C, VDN, PIC, DVA2C, INDQL, INDA2C) using independent benchmark environments; the hyperparameters K, I, and C are tuned, which introduces potential selection bias in the reported superiority, but that is a statistical concern rather than circularity. The paper contains no load-bearing self-citations: the citations of Zhang et al. [38] and Chen et al. [4] are to prior work by other authors, and they are used as building blocks (parameter consensus), not to prove the team-value approximation. The off-policy ablation in Fig. 7 showing that team-Q consensus degrades performance in LBF is consistent with Eq. (15) failing, but it is an empirical inconsistency, not evidence that the derivation assumes its conclusion. Overall, the central claim has independent empirical content and the derivation is self-contained apart from the unproved approximation in Eq. (15).

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The paper's central claim depends on four premises: the standard consensus convergence theorem (safe), the heuristic replacement of states with local observations (unproven but common), the asserted team value approximation in Eq. (15) (ad hoc and load-bearing), and the sufficiency of finite consensus steps (ad hoc and unanalyzed). The free parameters K, I, C are fitted per benchmark, not derived. No new physical or mathematical entities are introduced beyond the ND-POMG formalization, which is a modeling framework rather than an invented entity.

free parameters (3)
  • K (number of consensus rounds) = 5 for DNAA2C, 1 for DNAQL (Table 5)
    Grid-searched over {1, 5, 10} and selected per task by maximum average episodic return across three seeds; affects the accuracy of the team value approximation.
  • I (interval between parameter consensus rounds) = 10 for DNAA2C and DNAQL-LBF, 1 for DNAQL-MPE (Table 5)
    Grid-searched over {1, 5, 10} and selected per task; controls how often actor and critic parameters are averaged.
  • C (number of edges per communication round) = 1 for all tasks (Table 5)
    Grid-searched over {1, 2} and selected per task; with C=1, the communication graph has a single edge at each round, so consensus mixing is slow relative to the asymptotic guarantee.
assumptions (4)
  • standard math Average consensus converges to the global average for jointly connected switching graphs (Xiao et al., Theorem 1).
    Invoked in Appendix B.1 to justify that the uniform edge-set distribution achieves average consensus over sufficiently long communication rounds; this is a standard distributed averaging result from the cited literature.
  • domain assumption Replacing state s with local observation o_i in the policy gradient and advantage is a valid approximation.
    Used in Eq. (11) in Section 4.1.2; the paper cites this as standard practice in MARL but provides no bound or formal justification, and the joint state remains unobserved.
  • ad hoc to paper The consensus average of local TD targets approximates the true team value function (Eq. 15).
    Section 4.1.3, Eq. (15): asserted with 'Ideally, the following approximation will hold' and no derivation. This is the load-bearing premise of the team-V mechanism.
  • ad hoc to paper Finite K consensus steps with C=1 edge per round yield a good enough team value approximation.
    Section 4.1.3 and Table 5: convergence theory is asymptotic, but the method uses K in {1,5} and C=1; the adequacy of this finite-step mixing is unanalyzed.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Networked Agents in the Dark: Team Value Learning under Partial Observability." pith.science (2026). https://pith.science/paper/MHQFQCXV

@misc{pith2026250108778,
  author       = {Pith},
  title        = {Pith review of: Networked Agents in the Dark: Team Value Learning under Partial Observability},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MHQFQCXV}},
  note         = {Machine review of arXiv:2501.08778}
}
read the original abstract

We propose a novel cooperative multi-agent reinforcement learning (MARL) approach for networked agents. In contrast to previous methods that rely on complete state information or joint observations, our agents must learn how to reach shared objectives under partial observability. During training, they collect individual rewards and approximate a team value function through local communication, resulting in cooperative behavior. To describe our problem, we introduce the networked dynamic partially observable Markov game framework, where agents communicate over a switching topology communication network. Our distributed method, DNA-MARL, uses a consensus mechanism for local communication and gradient descent for local computation. DNA-MARL increases the range of the possible applications of networked agents, being well-suited for real world domains that impose privacy and where the messages may not reach their recipients. We evaluate DNA-MARL across benchmark MARL scenarios. Our results highlight the superior performance of DNA-MARL over previous methods.

Figures

Figures reproduced from arXiv: 2501.08778 by the authors.

Figure 1
Figure 1. Diagram illustrating the information flow from the [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. From left to right, episodic returns for on-policy setting and episodic returns for the off-policy setting, for two [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Ablation for DNAA2C: From left to right, DV [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Train rollouts for the on-policy algorithms. The bullets represent the average of evaluation checkpoints for ten random seeds. The shaded area represents a 95% bootstrap confidence interval around the average. In blue DNAA2C (ours), in orange INDA2C, the independent ag…
Figure 5
Figure 5. Figure 5: Train rollouts for the off-policy algorithms. The bullets represent the average of evaluation checkpoints for ten random seeds. The shaded area represents a 95% bootstrap confidence interval around the average. In blue DNAQL (ours), in orange INDQL, the independent age…
Figure 6
Figure 6. Figure 6: DNAA2C ablations: DV (distributed-V) group learns from individual rewards, performs consensus rounds on the critic. [PITH_FULL_IMAGE:figures/full_fig_p016_6.png]
Figure 7
Figure 7. Figure 7: DNAQL ablations: DQ (distributed-𝑄) performs consensus on parameters only. DNA group has both team-𝑄 consensus and parameter consensus. For six tasks the component that improves performance is consensus on the parameters and not team-𝑄 consensus. E EXTENDED RELATED WOR…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

42 extracted references · 31 canonical work pages

  1. [1]

    Boyan and Michael L

    Justin A. Boyan and Michael L. Littman. 1993. Packet routing in dynamically changing networks: a reinforcement learning approach. In Proceedings of the 6th International Conference on Neural Information Processing Systems (Denver, Colorado) (NIPS’93). Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 671–678

    work page 1993

  2. [2]

    Lucian Busoniu, Robert Babuska, and Bart De Schutter. 2010. Multi-agent Rein- forcement Learning: An Overview. Vol. 310. 183–221

    work page 2010

  3. [3]

    Alghunaim, and Ali H

    Lucas Cassano, Sulaiman A. Alghunaim, and Ali H. Sayed. 2019. Team Policy Learning for Multi-agent Reinforcement Learning. In ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . 3062–3066

    work page 2019

  4. [4]

    Dingyang Chen, Yile Li, and Qi Zhang. 2022. Communication-Efficient Actor- Critic Methods for Homogeneous Markov Games. In International Conference on Learning Representations

    work page 2022

  5. [5]

    Cédric Colas, Olivier Sigaud, and Pierre-Yves Oudeyer. 2019. A Hitchhiker’s Guide to Statistical Comparisons of Reinforcement Learning Algorithms. (04 2019)

    work page 2019

  6. [6]

    Enrique Munoz de Cote, Alessandro Lazaric, and Marcello Restelli. 2006. Learn- ing to Cooperate in Multi-Agent Social Dilemmas. In Proceedings of the Fifth International Joint Conference on Autonomous Agents and Multiagent Systems (Hakodate, Japan) (AAMAS ’06). Association for Computing Machinery, New York, NY, USA, 783–785

    work page 2006

  7. [7]

    Prafulla Dhariwal, Christopher Hesse, Oleg Klimov, Alex Nichol, Matthias Plap- pert, Alec Radford, John Schulman, Szymon Sidor, Yuhuai Wu, and Peter Zhokhov

    work page

  8. [8]

    Foerster, Gregory Farquhar, Triantafyllos Afouras, Nantas Nardelli, and Shimon Whiteson

    Jakob N. Foerster, Gregory Farquhar, Triantafyllos Afouras, Nantas Nardelli, and Shimon Whiteson. 2018. Counterfactual Multi-Agent Policy Gradients. InProceed- ings of the Thirty-Second AAAI Conference on Artificial Intelligence and Thirtieth In- novative Applications of Artificial Intelligence Conference and Eighth AAAI Sympo- sium on Educational Advance...

    work page 2018

Show all 42 references
  1. [9]

    Sven Gronauer and Klaus Diepold. 2022. Multi-agent deep reinforcement learning: a survey. Artificial Intelligence Review 55, 2 (2022), 895–943

    2022

  2. [10]

    Gupta, Maxim Egorov, and Mykel Kochenderfer

    Jayesh K. Gupta, Maxim Egorov, and Mykel Kochenderfer. 2017. Cooperative Multi-agent Control Using Deep Reinforcement Learning. In Autonomous Agents and Multiagent Systems. Springer International Publishing, 66–83

    2017

  3. [11]

    Pablo Hernandez-Leal, Bilal Kartal, and Matthew E. Taylor. 2019. A Survey and Critique of Multiagent Deep Reinforcement Learning. Autonomous Agents and Multi-Agent Systems 33 (2019), 750–797

    2019

  4. [12]

    Soummya Kar and José M. F. Moura. 2010. Distributed Consensus Algorithms in Sensor Networks: Quantized Data and Random Link Failures. IEEE Transactions on Signal Processing 58, 3 (2010), 1383–1400. https://doi.org/10.1109/TSP.2009. 2036046

    2010 doi

  5. [13]

    Mert Kayaalp, Fatima Ghadieh, and Ali H. Sayed. 2023. Policy Evaluation in Decentralized POMDPs With Belief Sharing.IEEE Open Journal of Control Systems 2 (2023), 125–145

    2023

  6. [14]

    Kipf and Max Welling

    Thomas N. Kipf and Max Welling. 2017. Semi-Supervised Classification with Graph Convolutional Networks. In International Conference on Learning Repre- sentations. https://openreview.net/forum?id=SJU4ayYgl

    2017

  7. [15]

    Tor Lattimore and Csaba Szepesvári. 2020. Bandit Algorithms. Cambridge Uni- versity Press

    2020

  8. [16]

    Michael L. Littman. 1994. Markov Games as a Framework for Multi-Agent Reinforcement Learning. In Machine Learning Proceedings 1995 . Elsevier, 157– 163

    1994

  9. [17]

    Yeh, and Alexander G

    Iou-Jen Liu, Raymond A. Yeh, and Alexander G. Schwing. 2020. PIC: Permutation Invariant Critic for Multi-Agent Deep Reinforcement Learning. In Proceedings of the Conference on Robot Learning (Proceedings of Machine Learning Research, Vol. 100). PMLR, 590–602

    2020

  10. [18]

    Ryan Lowe, Yi Wu, Aviv Tamar, Jean Harb, Pieter Abbeel, and Igor Mordatch. 2017. Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments. Neural Information Processing Systems (NIPS) (2017)

    2017

  11. [19]

    Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin Riedmiller. 2015. Playing Atari with Deep Reinforcement Learning. Nature 518, 7540 (2015), 529–533

    2015

  12. [20]

    Oliehoek and Christopher Amato

    Frans A. Oliehoek and Christopher Amato. 2016. A Concise Introduction to Decentralized POMDPs. Springer International Publishing

    2016

  13. [21]

    How, and John Vian

    Shayegan Omidshafiei, Jason Pazis, Christopher Amato, Jonathan P. How, and John Vian. 2017. Deep Decentralized Multi-task Multi-Agent Reinforcement Learning under Partial Observability. In Proceedings of the 34th International Con- ference on Machine Learning (Proceedings of M...

    2017

  14. [22]

    Afshin Oroojlooy and Davood Hajinezhad. 2022. A Review of Cooperative Multi-Agent Deep Reinforcement Learning. Applied Intelligence 53, 11 (2022), 13677–13722

    2022

  15. [23]

    Gregory Palmer, Karl Tuyls, Daan Bloembergen, and Rahul Savani. 2018. Le- nient Multi-Agent Deep Reinforcement Learning. In Proceedings of the 17th In- ternational Conference on Autonomous Agents and MultiAgent Systems (AAMAS ’18). International Foundation for Autonomous Agent...

    2018

  16. [24]

    Georgios Papoudakis, Filippos Christianos, Lukas Schäfer, and Stefano Albrecht

  17. [25]

    Marijana Peti, Frano Petric, and Stjepan Bogdan. 2023. Decentralized Coordi- nation of Multi-Agent Systems Based on POMDPs and Consensus for Active Perception. IEEE Access 11 (2023), 52480–52491

    2023

  18. [26]

    Potter and Kenneth A

    Mitchell A. Potter and Kenneth A. De Jong. 1994. A cooperative coevolutionary approach to function optimization. In Parallel Problem Solving from Nature — PPSN III. Springer Berlin Heidelberg, 249–257

    1994

  19. [27]

    Tabish Rashid, Mikayel Samvelyan, Christian Schroeder, Gregory Farquhar, Jakob Foerster, and Shimon Whiteson. 2018. QMIX: Monotonic Value Function Factori- sation for Deep Multi-Agent Reinforcement Learning. In Proceedings of the 35th International Conference on Machine Learni...

    2018

  20. [28]

    John Schulman, Sergey Levine, Philipp Moritz, Michael Jordan, and Pieter Abbeel

  21. [29]

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov

  22. [30]

    Stimpson and Michael A

    Jeffrey L. Stimpson and Michael A. Goodrich. 2003. Learning to Cooperate in a Social Dilemma: A Satisficing Approach to Bargaining. In Proceedings of the Twentieth International Conference on International Conference on Machine Learning (ICML’03). AAAI Press, 728–735

    2003

  23. [31]

    Leibo, Karl Tuyls, and Thore Graepel

    Peter Sunehag, Guy Lever, Audrunas Gruslys, Wojciech Marian Czarnecki, Vini- cius Zambaldi, Max Jaderberg, Marc Lanctot, Nicolas Sonnerat, Joel Z. Leibo, Karl Tuyls, and Thore Graepel. 2018. Value-Decomposition Networks For Cooper- ative Multi-Agent Learning Based On Team Rewa...

    2018

  24. [32]

    Sutton and A.G

    R.S. Sutton and A.G. Barto. 2018. Reinforcement Learning: An Introduction. MIT Press

    2018

  25. [33]

    Sutton, David A

    R. Sutton, David A. McAllester, Satinder Singh, and Y. Mansour. 1999. Policy Gradient Methods for Reinforcement Learning with Function Approximation. In NIPS

    1999

  26. [34]

    Hao Tu, Yuhua Du, Hui Yu, Xiaonan Lu, and Srdjan Lukic. 2024. Privacy- Preserving Robust Consensus for Distributed Microgrid Control Applications. IEEE Transactions on Industrial Electronics 71, 4 (2024), 3684–3697. https: //doi.org/10.1109/TIE.2023.3274846

    2024

  27. [35]

    Wang, Michael Everett, and Jonathan P

    Rose E. Wang, Michael Everett, and Jonathan P. How. 2020. R-MADDPG for Partially Observable Environments and Limited Communication. CoRR abs/2002.06684 (2020)

    2020 arXiv

  28. [36]

    Lin Xiao, Stephen Boyd, and Seung-Jean Kim. 2007. Distributed average consensus with least-mean-square deviation. J. Parallel and Distrib. Comput. 67, 1 (2007), 33–46

    2007

  29. [37]

    2021.Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms

    Kaiqing Zhang, Zhuoran Yang, and Tamer Başar. 2021.Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms . Springer International Publishing, Cham, 321–384

    2021

  30. [38]

    Kaiqing Zhang, Zhuoran Yang, Han Liu, Tong Zhang, and Tamer Basar. 2018. Fully Decentralized Multi-Agent Reinforcement Learning with Networked Agents. In Proceedings of the 35th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 80) . ...

    2018

  31. [39]

    Zavlanos

    Yan Zhang and Michael M. Zavlanos. 2019. Distributed off-Policy Actor-Critic Reinforcement Learning with Policy Consensus. In 2019 IEEE 58th Conference on Decision and Control (CDC) . 4674–4679. A EXTENDED BACKGROUND Consider a graphG(N,E), an undirected graph with nodesN ={1,...

    2019

  32. [2015]

    In Proceedings of the 32nd International Conference on International Conference on Machine Learning - Volume 37 (ICML’15)

    Trust Region Policy Optimization. In Proceedings of the 32nd International Conference on International Conference on Machine Learning - Volume 37 (ICML’15). JMLR.org, 1889–1897

  33. [2017]

    CoRR abs/1707.06347 (2017)

    Proximal Policy Optimization Algorithms. CoRR abs/1707.06347 (2017)

    2017 arXiv

  34. [2021]

    In Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks , J

    Benchmarking Multi-Agent Deep Reinforcement Learning Algorithms in Cooperative Tasks. In Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks , J. Vanschoren and S. Yeung (Eds.), Vol. 1

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.