Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

Contextual Knowledge Sharing in Multi-Agent Reinforcement Learning with Decentralized Communication and Coordination

T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read In decentralized multi-agent reinforcement learning, sharing knowledge filtered by goal relevance and freshness significantly improves performance in dynamic environments.

desk verdict A thoughtful framework, but the paper's own tables contradict its claim of significant improvement—A5 beats independent A1 only in small environments. read the letter →

arxiv 2501.15695 v1 pith:X6N55W6O submitted 2025-01-26 cs.MA cs.AI

Hung Du , Srikanth Thudumu , Hy Nguyen , Rajesh Vasa , Kon Mouzakis This is my paper · ORCID
classification cs.MAcs.AI
keywords multi-agentreinforcementlearningdecentralizedtrainingandexecutiongoalawarenesstimeintrinsicrewardcommunicationcoordinationknowledgesharingdynamicobstacles
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes a framework for decentralized multi-agent reinforcement learning where each agent pursues its own goal with limited visibility. The central claim is that agents learn better when they filter shared knowledge by two criteria: whether the information is relevant to the receiver's goal and how recently the information was obtained. The authors build this into a Share-Reason-Aggregate cycle, where peers and advisors exchange observations, an agent reasons about stale versus fresh knowledge, and selected peers average their learning parameters. They test the framework on grid-world tasks with dynamically appearing obstacles and report that the full system outperforms independent agents and most ablations. If the claim holds, it suggests that context-aware communication can replace centralized coordination in decentralized teams.

What carries the argument

The machinery is the Share-Reason-Aggregate cycle with a time-aware mental state. An agent's mental state $M_i$ stores masked labels and visit durations; the novelty score $u_t^i(o) = e^{\frac{1}{2}d_{t'}}$ from Equation 3 makes old observations decay and rediscovered ones fresh again. This score enters the mental-state embedding (Equation 4) and an intrinsic reward (Equation 5) that balances extrinsic goal reward against exploration. In a session the agent broadcasts its goal to identify peers and advisors, merges shared observations by freshness (Equation 6), selects peers by Jaccard overlap (Equation 7), and averages learning parameters (Equations 8 and 9) with a low blending factor $\beta$.

What would settle it

In the Easy environments (static obstacles), remove the time-aware intrinsic reward entirely; if the full A5 framework still shows the same margin over the A4 ablation, then the paper's central attribution of gains to time awareness is not supported.

Watch

Extended reading notes

Core claim

The authors claim that a fully decentralized MARL framework combining goal-aware communication with time-aware coordination improves both exploration and task success compared with independent agents and partial ablations. The strongest form of the claim is carried by the full A5 agent, which broadcasts its goal to separate same-goal peers from advisors, merges only fresh observations into its mental state, and averages parameters only with peers whose mental-state overlap is low. In the paper's experiments, A5 is the best or near-best performer in most tested environments and completes tasks in fewer steps than the independent baseline, with the largest gains in hard, dynamic environments. The work is presented as a demonstration that context—what the receiver wants and how old the information is—should determine what is shared, not just whether to communicate.

Load-bearing premise

The central claim depends on two assumptions: a simple time-decay formula can correctly measure how stale an agent's knowledge is, and averaging learning parameters with peers chosen by overlap helps more than it hurts.

Editorial extensions

If this is right

  • A fully decentralized team can match or beat independent learners without a centralized critic, shared reward, or hand-designed communication protocol.
  • Goal-aware filtering matters in mixed-goal scenarios: agents that always share regardless of goal (A4) sometimes perform worse than independent agents, while goal-aware A5 is more consistent.
  • Time awareness is most valuable where knowledge goes stale: in large or hard environments A5 beats A4, while in small static environments mental state alone can suffice.
  • Advisors contribute heuristic shortest-path plans rather than learned parameters, so advice can help without transferring goal-specific policies.
  • The parameter-averaging step should be used with low $\beta$, since the paper observes that poor-performing peers can negatively affect others.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • One extension not tested by the paper: replacing the hand-set time-decay formula with a learned or count-based staleness measure, to see whether the exponential form is essential to the reported gains.
  • The Jaccard-overlap rule for peer selection could be replaced by an estimate of each peer's actual contribution; the paper's own admission that poor agents hurt others suggests adaptive weighting would stabilize aggregation.
  • Because agents share an ontology and a bounded map, goal-aware filtering might transfer to semantic labels or learned representations, but only if the mental-state encoding is learned rather than predefined.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes a Decentralized Multi-Agent Reinforcement Learning (Dec-MARL) framework that integrates peer-to-peer communication and coordination with goal awareness and time awareness. The method introduces a mental-state representation, a time-aware intrinsic reward, and a three-phase Share-Reason-Aggregate mechanism for knowledge sharing. The authors evaluate five agent variants (A1 through A5) in 10x10 and 20x20 grid-world environments with static or dynamically appearing obstacles, under two goal scenarios. The central claim is that the full framework, A5, significantly improves overall performance over independent agents.

Significance. If the claimed improvements were real, the framework would be a useful contribution to decentralized MARL with heterogeneous goals and dynamic obstacles. The paper's modular ablation design from A1 to A5 and its explicit treatment of information decay are appropriate and address an important problem. However, the reported evidence does not establish the central claim: the full model A5 is worse than the independent baseline A1 in all four Large-environment cells, no statistical testing is provided, and the main hyperparameters are tuned on the same tasks used for evaluation. The contribution is therefore not currently substantiated.

major comments (4)
  1. [Section V-C, Tables I and II] The central claim that A5 significantly enhances performance is contradicted by the reported means. In Table I, A5 is worse than A1 in Large-Easy (0.229 vs 0.233) and Large-Hard (0.235 vs 0.243); in Table II, A5 is worse than A1 in Large-Easy (0.232 vs 0.249) and Large-Hard (0.233 vs 0.251). Across the eight settings, A5 is best only in Base-Easy (0.139) and Scenario 2 Base-Hard (0.225), and in Scenario 1 Base-Hard a simpler variant A2 (0.207) outperforms A5 (0.191). The tables report only Roverall, yet Section V-C.1 claims '5% fewer steps' and '15% fewer steps' without any step-based results, and no number of seeds or statistical tests are given. These results do not support the abstract's assertion of significant enhancement.
  2. [Section IV-B and Section V-B] Several core hyperparameters are chosen by 'empirical analysis' on the same environments used for evaluation: alpha=0.1 in Eq. (5), the time increment 0.01 for dt in Eq. (3), the Jaccard threshold J<=0.5 in Eq. (7), and beta=0.1 in Eqs. (8)-(9). Because the reported A5 gains depend on these values and no held-out validation or sensitivity analysis is provided, the improvements are fitted to the test tasks rather than independent predictions. This circularity is load-bearing for the claim of general enhancement.
  3. [Section IV-B, Eq. (3)] Equation (3) is ambiguous and unverified. The expression for u_t_i(o) is not typeset unambiguously (it is unclear whether the exponent is 1/(2 dt') or dt'/2), and the stated condition that the value of an observation decays over time is not demonstrated against the definition of dt'. Since the time-aware intrinsic reward is the key mechanism intended to improve exploration in dynamic environments, its behavior must be specified precisely and validated with a clear derivation or experiment.
  4. [Section IV-C.3, Eqs. (8)-(9)] The parameter-averaging update is acknowledged in the text to 'sometimes lead to situations where poor-performing agents negatively impact the performance of others,' and beta is merely set low. No mechanism or experiment isolates when averaging helps versus hurts, so the ablation from A3 to A4 to A5 cannot separate the negative effect of aggregation from the benefit of communication. This weakens the interpretation of the A4 and A5 results.
minor comments (5)
  1. [Abstract/Conclusion] There is a typo 'Muti-Agent' in the Conclusion; 'relay buffer' in Section V-B should be 'replay buffer'; and Section V-B contains an incomplete sentence, 'we conducted our experiments a complex 2D environment'.
  2. [Section III and Section IV-A] The symbol S is overloaded: it denotes the set of states in Section III and the aggregation function in Eq. (1), which is confusing for the reader.
  3. [Figure 3] The axes and the relationship between dt' and time steps are not labeled in Figure 3, making the illustrative '100 steps' difficult to interpret.
  4. [Section V-C.1] The text reports percentage reductions in steps, but the tables report only Roverall; a separate table of step counts or episode-completion statistics is needed to support those claims.
  5. [Overall] No code or data availability statement is included, which hampers reproducibility of the experimental results.

Circularity Check

1 steps flagged · score 6.0 of 10

Reported 'overall performance' metric includes the method's own intrinsic-reward term, so the central enhancement claim is partly built into the evaluation.

  1. self definitional [Section V-B (Implementation Details, Eqs. 5, 10, 12) and Section V-C (Results)]
    "Moreover, to incentivize movement towards the goal, an agent receives a reward of ragg that is the same as Equation 5. Specifically, rext = 1 − ∆ (s^t_i, g_i) ... Since this type of agent does not have time awareness, the intrinsic reward in Equation 5 is always set to 0. ... The overall performance of the system is then estimated as: R_overall = 1/M Σ_{i=1}^M 1/N Σ_{j=1}^N AvgR_j"

    Equation 5 adds a positive intrinsic exploration reward u_t^i to the shaped reward of every non-goal move, and Equation 10 routes that shaped reward into each agent's reward stream. Equation 12 defines 'overall performance' as the mean of exactly these shaped rewards. For A1, the intrinsic term is explicitly forced to zero, while A2–A5 accumulate positive u_t^i rewards for moving through states. The headline comparison is therefore not on equivalent task-success scales: the evaluation metric itself contains the method's added exploration bonus. The claimed 'significant enhancement' is partially guaranteed by the construction of the metric rather than being an independent prediction of coordination or task-completion benefit.

full rationale

The central empirical claim is that goal-aware and time-aware knowledge sharing 'significantly enhances overall performance.' But the paper defines 'overall performance' as the mean of per-agent rewards that are themselves shaped by Equation 10, which for all non-A1 agents includes the positive intrinsic exploration term from Equation 5. Because A1 is defined with that term set to zero, the comparison between A1 and A2–A5 is not a neutral measure of task success: the metric already contains the reward component that the method injects. This is a self-definitional reduction of the measured outcome to an input of the proposed framework. The hyperparameter choices (α=0.1, β=0.1, J≤0.5, dt increment 0.01) are selected via 'empirical analysis' on the same environments; I do not count this as circular per se, though it weakens out-of-sample validity. I also note as a correctness matter that the tabulated results actually conflict with the abstract's conclusion, since A5 is worse than A1 in all four Large-environment cells, but that is an evidence problem, not circularity. No load-bearing self-citation or imported uniqueness theorem was found; reference [1] is a background citation.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The central claim rests on several hand-tuned parameters (alpha, beta, J, dt) and ad hoc assumptions about the benefit of time-decayed novelty and parameter averaging. No independent evidence is provided for these choices beyond self-comparisons on the same tasks.

free parameters (6)
  • alpha (dampening factor for intrinsic reward) = 0.1
    Chosen from empirical analysis over [0.1, 0.5] on the same test environments (Section V-B).
  • beta (parameter aggregation factor) = 0.1
    Set to a low value based on empirical analysis to avoid negative impact from poor-performing agents (Section IV-C.3).
  • Jaccard similarity threshold (J) = <= 0.5
    Hand-chosen threshold for selecting peers with substantial novel knowledge; no justification beyond 'designed to encourage learning from novel knowledge' (Section V-B).
  • Time increment for dt in Equation 3 = 0.01
    Set based on 'empirical analysis' for gradually decaying information; not cross-validated (Section V-B).
  • lambda_stay (stay penalty) = 0.5
    Set to encourage movement; no sensitivity analysis (Section V-B, Equation 10).
  • Exponential decay coefficient in Equation 3 = implied 1/2 (unclear)
    The form of the time-decay function is ad hoc; the coefficient is not derived and the rendering is ambiguous.
assumptions (4)
  • standard math The Dec-MARL problem is modeled as a decentralized POMDP with individual goals.
    Standard framework for decentralized decision making, cited in Section III.
  • domain assumption All agents share the same ontology and bounded environment, making observations and knowledge transferable.
    Stated in Section III and IV-C.1: 'agents share the same ontology and bounded environment'. This is necessary for the communication and parameter sharing to be meaningful.
  • ad hoc to paper The time-aware intrinsic reward in Equation 3 correctly models the value of information decay and encourages beneficial exploration.
    No derivation or theoretical justification is given for the exponential form; the behavior described in the text is not consistent with the equation as rendered.
  • ad hoc to paper Averaging learning parameters with selected peers (Equations 8 and 9) improves learning when Jaccard overlap is low.
    The paper itself admits that poor-performing agents can hurt others (Section IV-C.3), yet uses this heuristic without a safeguard.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Contextual Knowledge Sharing in Multi-Agent Reinforcement Learning with Decentralized Communication and Coordination." pith.science (2026). https://pith.science/paper/X6N55W6O

@misc{pith2026250115695,
  author       = {Pith},
  title        = {Pith review of: Contextual Knowledge Sharing in Multi-Agent Reinforcement Learning with Decentralized Communication and Coordination},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/X6N55W6O}},
  note         = {Machine review of arXiv:2501.15695}
}
read the original abstract

Decentralized Multi-Agent Reinforcement Learning (Dec-MARL) has emerged as a pivotal approach for addressing complex tasks in dynamic environments. Existing Multi-Agent Reinforcement Learning (MARL) methodologies typically assume a shared objective among agents and rely on centralized control. However, many real-world scenarios feature agents with individual goals and limited observability of other agents, complicating coordination and hindering adaptability. Existing Dec-MARL strategies prioritize either communication or coordination, lacking an integrated approach that leverages both. This paper presents a novel Dec-MARL framework that integrates peer-to-peer communication and coordination, incorporating goal-awareness and time-awareness into the agents' knowledge-sharing processes. Our framework equips agents with the ability to (i) share contextually relevant knowledge to assist other agents, and (ii) reason based on information acquired from multiple agents, while considering their own goals and the temporal context of prior knowledge. We evaluate our approach through several complex multi-agent tasks in environments with dynamically appearing obstacles. Our work demonstrates that incorporating goal-aware and time-aware knowledge sharing significantly enhances overall performance.

Figures

Figures reproduced from arXiv: 2501.15695 by the authors.

Figure 1
Figure 1. An illustration of a fully decentralized environment with multiple agents ( [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. A demonstration of the intrinsic reward guiding Agent 1 [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 4
Figure 4. An illustration of the Base environment [PITH_FULL_IMAGE:figures/full_fig_p009_4.png] view at source ↗
Figures from the paper (1 more)
Figure 5
Figure 5. Figure 5: An illustration of the Large environment [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Intelligent System of Emergent Knowledge: A Coordination Fabric for Billions of Minds

    cs.MA 2025-06 reject novelty 2.0 of 10

    ISEK is a conceptual blockchain-and-token architecture for coordinating human and AI agents, with no implementation, experiments, or formal results reported.

Reference graph

Works this paper leans on

46 extracted references · 39 canonical work pages · cited by 1 Pith paper

  1. [1]

    A survey on context- aware multi-agent systems: Techniques, challenges and future direc- tions,

    H. Du, S. Thudumu, R. Vasa, and K. Mouzakis, “A survey on context- aware multi-agent systems: Techniques, challenges and future direc- tions,” arXiv preprint arXiv:2402.01968 , 2024

    arXiv 2024

  2. [2]

    Survey of multi-agent systems for microgrid control,

    A. Kantamneni, L. E. Brown, G. Parker, and W. W. Weaver, “Survey of multi-agent systems for microgrid control,” Engineering applications of artificial intelligence, vol. 45, pp. 192–203, 2015

    work page 2015

  3. [3]

    Survey of agent-based cloud computing applications,

    F. De la Prieta, S. Rodr ´ıguez-Gonz´alez, P. Chamoso, J. M. Corchado, and J. Bajo, “Survey of agent-based cloud computing applications,” Future generation computer systems , vol. 100, pp. 223–236, 2019

    work page 2019

  4. [4]

    Consensus in multi-agent systems: a review,

    A. Amirkhani and A. H. Barshooi, “Consensus in multi-agent systems: a review,” Artificial Intelligence Review , vol. 55, no. 5, pp. 3897–3935, 2022

    2022

  5. [5]

    Multi-agent deep reinforcement learning: a survey,

    S. Gronauer and K. Diepold, “Multi-agent deep reinforcement learning: a survey,” Artificial Intelligence Review , vol. 55, no. 2, pp. 895–943, 2022

    work page 2022

  6. [6]

    Multi-agent reinforcement learning as a rehearsal for decentralized planning,

    L. Kraemer and B. Banerjee, “Multi-agent reinforcement learning as a rehearsal for decentralized planning,” Neurocomputing, vol. 190, pp. 82– 94, 2016

    work page 2016

  7. [7]

    Cooperative multi-agent control using deep reinforcement learning,

    J. K. Gupta, M. Egorov, and M. Kochenderfer, “Cooperative multi-agent control using deep reinforcement learning,” in Autonomous Agents and Multiagent Systems: AAMAS 2017 Workshops, Best Papers, S ˜ao Paulo, Brazil, May 8-12, 2017, Revised Selected Papers 16, pp. 66–83, Springer, 2017

    work page 2017

  8. [8]

    Multi-agent actor-critic for mixed cooperative-competitive environ- ments,

    R. Lowe, Y . I. Wu, A. Tamar, J. Harb, O. Pieter Abbeel, and I. Mordatch, “Multi-agent actor-critic for mixed cooperative-competitive environ- ments,” Advances in neural information processing systems , vol. 30, 2017

    2017

Show all 46 references
  1. [9]

    Counterfactual multi-agent policy gradients,

    J. Foerster, G. Farquhar, T. Afouras, N. Nardelli, and S. Whiteson, “Counterfactual multi-agent policy gradients,” in Proceedings of the AAAI conference on artificial intelligence , vol. 32, 2018

    2018

  2. [10]

    Shapley q-value: A local reward approach to solve global reward games,

    J. Wang, Y . Zhang, T.-K. Kim, and Y . Gu, “Shapley q-value: A local reward approach to solve global reward games,” in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 34, pp. 7285–7292, 2020

    2020

  3. [11]

    Monotonic value function factorisation for deep multi- agent reinforcement learning,

    T. Rashid, M. Samvelyan, C. S. De Witt, G. Farquhar, J. Foerster, and S. Whiteson, “Monotonic value function factorisation for deep multi- agent reinforcement learning,” Journal of Machine Learning Research , vol. 21, no. 178, pp. 1–51, 2020

    2020

  4. [12]

    Roma: Multi-agent reinforcement learning with emergent roles,

    T. Wang, H. Dong, V . Lesser, and C. Zhang, “Roma: Multi-agent reinforcement learning with emergent roles,” inInternational Conference on Machine Learning , pp. 9876–9886, PMLR, 2020

    2020

  5. [13]

    Gcs: Graph-based coordination strategy for multi-agent rein- forcement learning,

    J. Ruan, Y . Du, X. Xiong, D. Xing, X. Li, L. Meng, H. Zhang, J. Wang, and B. Xu, “Gcs: Graph-based coordination strategy for multi-agent rein- forcement learning,” in Proceedings of the 21st International Conference on Autonomous Agents and Multiagent Systems, AAMAS ’22, (Ric...

    2022

  6. [14]

    Scalable multi-agent reinforcement learning through intelligent information aggregation,

    S. Nayak, K. Choi, W. Ding, S. Dolan, K. Gopalakrishnan, and H. Balakrishnan, “Scalable multi-agent reinforcement learning through intelligent information aggregation,” in International Conference on Machine Learning, pp. 25817–25833, PMLR, 2023

    2023

  7. [15]

    Multi-agent reinforcement learning: Independent vs. cooper- ative agents,

    M. Tan, “Multi-agent reinforcement learning: Independent vs. cooper- ative agents,” in Proceedings of the tenth international conference on machine learning, pp. 330–337, 1993

    1993

  8. [16]

    I2q: A fully decentralized q-learning algorithm,

    J. Jiang and Z. Lu, “I2q: A fully decentralized q-learning algorithm,” Advances in Neural Information Processing Systems, vol. 35, pp. 20469– 20481, 2022

    2022

  9. [17]

    Learning when to communicate at scale in multiagent cooperative and competitive tasks,

    A. Singh, T. Jain, and S. Sukhbaatar, “Learning when to communicate at scale in multiagent cooperative and competitive tasks,” in International Conference on Learning Representations , 2018

    2018

  10. [18]

    Learning attentional communication for multi- agent cooperation,

    J. Jiang and Z. Lu, “Learning attentional communication for multi- agent cooperation,” Advances in neural information processing systems , vol. 31, 2018

    2018

  11. [19]

    Tarmac: Targeted multi-agent communication,

    A. Das, T. Gervet, J. Romoff, D. Batra, D. Parikh, M. Rabbat, and J. Pineau, “Tarmac: Targeted multi-agent communication,” in Interna- tional Conference on machine learning , pp. 1538–1546, PMLR, 2019

    2019

  12. [20]

    Efficient communication in multi- agent reinforcement learning via variance based control,

    S. Q. Zhang, Q. Zhang, and J. Lin, “Efficient communication in multi- agent reinforcement learning via variance based control,” Advances in neural information processing systems , vol. 32, 2019

    2019

  13. [21]

    Who2com: Collaborative perception via learnable handshake commu- nication,

    Y .-C. Liu, J. Tian, C.-Y . Ma, N. Glaser, C.-W. Kuo, and Z. Kira, “Who2com: Collaborative perception via learnable handshake commu- nication,” in 2020 IEEE International Conference on Robotics and Automation (ICRA), pp. 6876–6883, IEEE, 2020

    2020

  14. [22]

    When2com: Multi-agent perception via communication graph grouping,

    Y .-C. Liu, J. Tian, N. Glaser, and Z. Kira, “When2com: Multi-agent perception via communication graph grouping,” in Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition , pp. 4106–4115, 2020

    2020

  15. [23]

    Where2comm: Communication-efficient collaborative perception via spatial confidence maps,

    Y . Hu, S. Fang, Z. Lei, Y . Zhong, and S. Chen, “Where2comm: Communication-efficient collaborative perception via spatial confidence maps,” Advances in neural information processing systems , vol. 35, pp. 4874–4886, 2022

    2022

  16. [24]

    Multi-agent incentive communication via decentralized teammate mod- eling,

    L. Yuan, J. Wang, F. Zhang, C. Wang, Z. Zhang, Y . Yu, and C. Zhang, “Multi-agent incentive communication via decentralized teammate mod- eling,” in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 36, pp. 9466–9474, 2022

    2022

  17. [25]

    Learning efficient and robust multi-agent communication via graph information bottleneck,

    S. Ding, W. Du, L. Ding, L. Guo, and J. Zhang, “Learning efficient and robust multi-agent communication via graph information bottleneck,” in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 38, pp. 17346–17353, 2024

    2024

  18. [26]

    Expressive multi-agent communication via identity-aware learning,

    W. Du, S. Ding, L. Guo, J. Zhang, and L. Ding, “Expressive multi-agent communication via identity-aware learning,” in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 38, pp. 17354–17361, 2024

    2024

  19. [27]

    Pmac: Personalized multi-agent communication,

    X. Meng and Y . Tan, “Pmac: Personalized multi-agent communication,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 38, pp. 17505–17513, 2024

    2024

  20. [28]

    Simultaneously learning and advising in multiagent reinforcement learning,

    F. L. Da Silva, R. Glatt, and A. H. R. Costa, “Simultaneously learning and advising in multiagent reinforcement learning,” in Proceedings of the 16th conference on autonomous agents and multiagent systems , pp. 1100–1108, 2017

    2017

  21. [29]

    Cautiously-optimistic knowledge sharing for cooperative multi-agent reinforcement learning,

    Y . Ba, X. Liu, X. Chen, H. Wang, Y . Xu, K. Li, and S. Zhang, “Cautiously-optimistic knowledge sharing for cooperative multi-agent reinforcement learning,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 38, pp. 17299–17307, 2024

    2024

  22. [30]

    Learning individually inferred commu- nication for multi-agent cooperation,

    Z. Ding, T. Huang, and Z. Lu, “Learning individually inferred commu- nication for multi-agent cooperation,” Advances in neural information processing systems, vol. 33, pp. 22069–22079, 2020

    2020

  23. [31]

    Learning multi-agent communication with double attentional deep reinforcement learning,

    H. Mao, Z. Zhang, Z. Xiao, Z. Gong, and Y . Ni, “Learning multi-agent communication with double attentional deep reinforcement learning,” Autonomous Agents and Multi-Agent Systems , vol. 34, pp. 1–34, 2020

    2020

  24. [32]

    Value-decomposition networks for cooperative multi-agent learning based on team reward,

    P. Sunehag, G. Lever, A. Gruslys, W. M. Czarnecki, V . Zambaldi, M. Jaderberg, M. Lanctot, N. Sonnerat, J. Z. Leibo, K. Tuyls, et al. , “Value-decomposition networks for cooperative multi-agent learning based on team reward,” in Proceedings of the 17th International Con- feren...

    2018

  25. [33]

    Deep coordination graphs,

    W. B ¨ohmer, V . Kurin, and S. Whiteson, “Deep coordination graphs,” in International Conference on Machine Learning , pp. 980–991, PMLR, 2020

    2020

  26. [34]

    Learning multi-agent coordination through connectivity-driven communication,

    E. Pesce and G. Montana, “Learning multi-agent coordination through connectivity-driven communication,” Machine Learning, vol. 112, no. 2, pp. 483–514, 2023

    2023

  27. [35]

    Settling decentralized multi-agent coordinated exploration by novelty sharing,

    H. Jiang, Z. Ding, and Z. Lu, “Settling decentralized multi-agent coordinated exploration by novelty sharing,” in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 38, pp. 17444–17452, 2024

    2024

  28. [36]

    Is independent learning all you need in the starcraft multi-agent challenge?,

    C. S. De Witt, T. Gupta, D. Makoviichuk, V . Makoviychuk, P. H. Torr, M. Sun, and S. Whiteson, “Is independent learning all you need in the starcraft multi-agent challenge?,” arXiv preprint arXiv:2011.09533 , 2020

    2011 arXiv

  29. [37]

    V-learning–a simple, efficient, decentralized algorithm for multiagent rl,

    C. Jin, Q. Liu, Y . Wang, and T. Yu, “V-learning–a simple, efficient, decentralized algorithm for multiagent rl,” in ICLR 2022 Workshop on Gamification and Multiagent Solutions , 2022

    2022

  30. [38]

    The complexity of markov equilibrium in stochastic games,

    C. Daskalakis, N. Golowich, and K. Zhang, “The complexity of markov equilibrium in stochastic games,” in The Thirty Sixth Annual Conference on Learning Theory , pp. 4180–4234, PMLR, 2023

    2023

  31. [39]

    Learn to follow: Decentralized lifelong multi-agent pathfinding via planning and learning,

    A. Skrynnik, A. Andreychuk, M. Nesterova, K. Yakovlev, and A. Panov, “Learn to follow: Decentralized lifelong multi-agent pathfinding via planning and learning,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 38, pp. 17541–17549, 2024

    2024

  32. [40]

    Shared experience actor- critic for multi-agent reinforcement learning,

    F. Christianos, L. Sch ¨afer, and S. Albrecht, “Shared experience actor- critic for multi-agent reinforcement learning,” Advances in neural infor- mation processing systems , vol. 33, pp. 10707–10717, 2020

    2020

  33. [41]

    Exploration-guided reward shaping for reinforcement learning under sparse rewards,

    R. Devidze, P. Kamalaruban, and A. Singla, “Exploration-guided reward shaping for reinforcement learning under sparse rewards,” Advances in Neural Information Processing Systems , vol. 35, pp. 5829–5842, 2022

    2022

  34. [42]

    F. A. Oliehoek, C. Amato, et al., A concise introduction to decentralized POMDPs, vol. 1. Springer, 2016

    2016

  35. [43]

    Universal value func- tion approximators,

    T. Schaul, D. Horgan, K. Gregor, and D. Silver, “Universal value func- tion approximators,” in International conference on machine learning , pp. 1312–1320, PMLR, 2015

    2015

  36. [44]

    Natural actor-critic,

    J. Peters and S. Schaal, “Natural actor-critic,” Neurocomputing, vol. 71, no. 7-9, pp. 1180–1190, 2008

    2008

  37. [45]

    Continuous control with deep reinforcement learning,

    T. P. Lillicrap, J. J. Hunt, A. Pritzel, N. M. O. Heess, T. Erez, Y . Tassa, D. Silver, and D. Wierstra, “Continuous control with deep reinforcement learning,” CoRR, 2015

    2015

  38. [46]

    Adam: A method for stochastic optimization,

    D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980 , 2014. APPENDIX 1 - D ESCRIPTION OF DATASETS A. The Base Environment Fig. 4: An illustration of the Base environment Figure 4 shows the Base environment mentioned in the paper...

    2014 arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.