M2I2: Learning Efficient Multi-Agent Communication via Masked State Modeling and Intention Inference

Chuxiong Sun; Jiangmeng Li; Peng He; Qirui Ji; Rui Wang; Wei Wang; Zehua Zang

REVIEW 4 major objections 6 minor 35 references

M2I2: Learning Efficient Multi-Agent Communication via Masked State Modeling and Intention Inference

T0 review · 4 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read M2I2 claims that improving how agents integrate received messages, through masked state reconstruction and joint-action prediction, makes cooperative agents communicate less while coordinating better.

desk verdict A promising but under-specified MARL communication method: masked state modeling plus intention inference and a meta-learned dimensional mask; the meta-update and gradient path need clarification, and the evaluation lacks error bars. read the letter →

arxiv 2501.00312 v1 pith:G3WATKJH submitted 2024-12-31 cs.MA cs.AI

Chuxiong Sun , Peng He , Qirui Ji , Zehua Zang , Jiangmeng Li , Rui Wang , Wei Wang This is my paper · ORCID

classification cs.MAcs.AI

keywords multi-agentreinforcementlearningcommunicationefficiencymaskedstatemodelingintentioninferencedimensionalrationalnetworkmeta-learningself-supervisedauxiliarytasks

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that the bottleneck in multi-agent communication is not just what agents send but how they digest what they receive. To close that gap, M2I2 gives each agent two self-supervised jobs while training: reconstruct the environment's global state from the messages it received, and predict the joint action that took the team from one state representation to the next. A separate network, the Dimensional Rational Network, is meta-trained to score each dimension of an observation and keep only the top-k most useful dimensions when sharing. The paper reports that this combination outperforms existing communication methods on Hallway, Predator-Prey, and SMAC-style benchmarks, while transmitting at only 60% frequency and achieving higher communication efficiency on every tested scenario.

What carries the argument

The load-bearing pieces are: (1) a masked state auto-encoder, a message encoder plus state decoder that predicts the global state from received messages; (2) an inverse model that predicts the joint action from two consecutive message-integrated representations, providing intention inference; and (3) the Dimensional Rational Network (DRN), a meta-learned importance scorer whose top-k mask selects which observation dimensions each agent transmits. The DRN's importance weights are supposed to be updated through a meta-learning step that evaluates their effect on the combined RL-reconstruction-inverse loss, which is what ties the communication masks to both task success and the auxiliary objectives.

What would settle it

Re-run M2I2 on the SMAC-Communication map 1o_2r_vs_4r with the DRN replaced by fixed random top-k masks chosen at the same 0.6 communication rate; if random masks reproduce the reported win rate, the claim that the meta-learned importance masks drive the gains is falsified. A second check is to instrument the training loop: if the gradient of the M2I2 loss with respect to $\theta_{\mathrm{DRN}}$ is zero at every step, the DRN is not being trained as described.

Watch

Extended reading notes

Core claim

M2I2's central claim is that information integration at the receiving end is the neglected factor in multi-agent communication, and that it can be taught directly by two auxiliary objectives. The state decoder and message encoder form a masked auto-encoder that reconstructs the full global state from a partial, masked set of received messages; the inverse model takes two consecutive integrated representations and predicts the joint action, giving agents an implicit model of teammates' intentions. The Dimensional Rational Network meta-learns per-dimension importance weights from their gradient contribution to both the auxiliary losses and the RL loss, and a top-k scheduler masks out everything below the top fraction, so agents share only the dimensions that matter. On Hallway, Predator-Prey, SMAC, and SMAC-Communication, the paper reports consistent improvements over TarMAC, MAIC, SMS, MASIA, and communication-free QMIX, and a communication-efficiency lead at 60% communication frequency.

Load-bearing premise

The central premise is that the meta-learning update can train the Dimensional Rational Network to pick useful dimensions even though the top-k selection is discrete and the paper gives no explicit gradient path into it; if that premise fails, the claimed efficiency gains from selective masking do not follow.

Editorial extensions

If this is right

At the communication frequency the paper uses (60%), M2I2 reports higher communication efficiency, measured as performance gain per unit of communication, than all tested baselines in every evaluated environment.
The ablations indicate both the inverse model and the DRN contribute: removing them degrades performance on the SMAC-Communication maps.
M2I2's message encoder and auxiliary objectives are compatible with multiple MARL backbones, including QMIX, VDN, QPLEX, MAPPO, and MADDPG, so the integration recipe transfers across value-based and policy-gradient training.
The communication-rate ablation suggests that 0.6 beats 0.8, implying that in these tasks extra shared information can be noise rather than signal.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A testable extension is to use the state-reconstruction loss as a live signal to adapt the top-k ratio per agent and per phase, instead of the fixed 0.6 frequency reported here.
Because the top-k operation in Equation 2 is discrete, the described meta-learning update will likely need a differentiable relaxation, such as a straight-through estimator or Gumbel-softmax, to actually train the DRN; the paper does not specify one.
The intention-inference objective could be used as an auxiliary loss in a fully decentralized MARL system with no explicit messages at all, which would test whether representation quality alone, rather than selective sharing, drives the reported gains.
If reconstructions from received messages are rich enough to predict global states, the learned representations may transfer to zero-shot coordination with new teammates, a scenario beyond the paper's benchmarks.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Let me give you the short version: this paper is worth your attention if you work on multi-agent communication, but it is not ready as written. The core idea—use self-supervised reconstruction of the global state and inverse prediction of joint actions to help agents integrate incoming messages, plus a learned dimensional mask to decide what to say—is plausible and the empirical results look consistently positive across Hallway, Predator-Prey, and SMAC variants. The novelty is real but incremental: MASIA already applied self-supervised aggregation; M2I2 extends this with masked state modeling and a meta-learned importance network. That's a reasonable contribution, though not a breakthrough.

The writing is clear about the problem: existing methods focus on what to communicate, when, and with whom, but pay less attention to how receivers synthesize the messages. The two auxiliary losses are well-motivated, and the visualization of what the DRN focuses on (e.g., ally vs. enemy features during exploration vs. combat) is a nice touch.

The soft spots are significant. First, the DRN training is not actually specified. Equation 9 is an arg min over θ_DRN, but no gradient update rule is given; the top-k selection in Eq. 2 is hard, non-differentiable, and no straight-through estimator or relaxation is mentioned. As written, I cannot implement the central mechanism. That's not a minor omission; it's the thing that makes the efficiency claim work. Second, there are no error bars. The authors say results are averaged over five seeds, but no variance is shown, so I can't judge whether the improvements are meaningful. Third, the efficiency metric (performance improvement divided by communication frequency) is not carefully controlled; the ablation that removes DRN also changes from dimension-level top-k masking to random observation-level masking, so the two are confounded. Finally, Figure 2 is corrupted in the manuscript, which hurts readability.

Who should read this? Researchers working on self-supervised methods for MARL communication, and anyone interested in how meta-learned masking could reduce communication costs. It deserves a serious referee: the core idea is sound, the issues are addressable with a careful revision, and the empirical claims, if they hold up, would be useful to the community. I'd recommend sending it to review, but with a strong request to clarify the DRN update (and ideally provide code), add error bars, and redo the ablation with a cleaner control.

Referee Report

4 major / 6 minor

Summary. The paper proposes M2I2, a framework for cooperative multi-agent reinforcement learning with communication. It combines masked state modeling (a state-level masked auto-encoder reconstructing global states from received messages), an inverse model for joint-action prediction (intention inference), and a Dimensional Rational Network (DRN) that learns dimension-level importance weights for observations. A top-k mechanism masks low-importance dimensions before sharing, reducing communication cost, and the DRN is trained with a meta-learning paradigm. The method is evaluated on Hallway, Predator-Prey, SMAC, and SMAC-Communication against QMIX, TarMAC, MAIC, SMS, and MASIA, reporting performance, communication efficiency, ablations, and generalization results.

Significance. If the claims are correct, M2I2 would be a meaningful contribution to communication-efficient MARL: it explicitly targets the receiving-end integration problem and uses self-supervised auxiliary tasks to shape representations, an interesting and timely direction. The evaluation is external (standard benchmarks and baselines), so there is no fitting-is-prediction circularity. The paper also ships auxiliary-loss learning curves and visualizations of the learned masks, and reports parameter counts and runtime, which helps reproducibility. However, the central DRN training mechanism is underspecified, the ablation does not isolate the DRN, and the empirical evidence lacks uncertainty quantification. These gaps prevent the paper from being accepted in its current form.

major comments (4)

[Section 4.4, Eq. (2), Eq. (9), Eq. (10), Algorithm 1] The training procedure for the DRN is not implementable from the text. In Eq. (2), topK is a hard selection: dimensions outside the top-k are set to zero, so the mask has discontinuous dependence on theta_DRN. The paper does not state a gradient estimator (straight-through, Gumbel-Softmax, REINFORCE, or another surrogate) for this discrete selection. Furthermore, Eq. (9) is written as an arg min, not an update, and Algorithm 1 says 'Update theta_DRN by Equation 9' without giving an explicit gradient step. The 'second-derivative technique' is mentioned but not specified. Because the DRN and its top-k masking are load-bearing for both the communication-efficiency claim and the ablation, the authors must provide the exact update rule and the differentiable treatment of the top-k selection.
[Section 5.4 (ablation)] The ablation 'M2I2 w/o DRN' removes the DRN and the top-k filter, and replaces them with random observation-level masking. This simultaneously changes two factors: (a) learned importance weights versus random selection, and (b) dimension-level masking versus whole-observation binary gating. Consequently, the performance difference between M2I2 and this variant cannot be attributed solely to the DRN's learned importance; it could stem from the change in mask granularity or from the randomness of the gating. An additional ablation is needed, for example using random dimension-level top-k masking, or retaining the DRN with observation-level masking, to separate these factors.
[Section 5.2 and Appendix C] The paper states that results are averaged over 5 random seeds but does not report variances, standard deviations, confidence intervals, or significance tests anywhere. The claims in Section 5.2 of 'consistently outperforming all baselines by a significant margin' are not supported without uncertainty quantification. MARL results typically exhibit non-negligible seed variance, so the authors should add error bars or shaded regions to the learning curves, report standard deviations in tables, and, if they wish to use the word 'significant,' accompany it with an appropriate statistical test.
[Section 5.3, Table 1] The communication-efficiency metric divides performance improvement by communication frequency, but the improvement is measured relative to different baselines for different methods: DOP for SMS and QMIX for all other algorithms. This makes the improvement values non-comparable across methods. In addition, 'communication frequency' appears to be defined differently for different methods (fraction of retained dimensions for M2I2 versus a gating probability for SMS, versus 100% for always-communicating baselines). The authors should use a consistent baseline and a consistent definition of communication frequency, or justify why the current heterogeneous comparison is fair.

minor comments (6)

[Figures 2 and 3] Figure 2 is corrupted in the submitted manuscript (it renders as a long string of font paths), and Figure 3 is also garbled. Please re-upload the actual figures.
[Section 5.5] The heading 'Generation' should be 'Generalization.'
[Eq. (7)] The inverse-model loss uses L2 error between a softmax output and a one-hot action vector; this is unconventional, and cross-entropy would be more natural. Please justify the choice or change the loss.
[Algorithm 1] The phrase 'trail weight' should be 'trial weight.'
[Section 1 and Related Work] The claim of being the 'first instance' of incorporating self-supervised objectives into multi-agent communication is too strong given that MASIA already uses self-supervised learning (an autoencoder and a forward model); the novelty should be positioned as the masked modeling and inverse-model combination rather than self-supervision per se.
[Section 3] The notation c_i^t for received messages is introduced but not used in the rest of the paper; please clarify how c_i^t relates to m_j^t and the integrated representation z_i^t.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity identified; the paper's empirical claims are benchmark-driven, and its self-citations are to published components used in standard ways.

full rationale

No load-bearing step in the manuscript reduces a claimed result to its own inputs. M2I2's performance claims (Section 5.2, Figure 2) are comparisons against external baselines on standard benchmarks (Hallway, PP, SMAC, SMAC-Communication), so there is no fitted-input-renamed-as-prediction structure. The communication efficiency metric (Section 5.3, Table 1) is a normalized comparison: (performance improvement over a communication-free baseline)/(communication frequency); even though M2I2's '60% communication frequency' is the chosen top-k mask ratio hyperparameter rather than a measured quantity, the numerator is still independent benchmark performance, so the table entry is a measurement, not an identity. The self-cited works (IMMAC for importance-based gating; MetAug for second-derivative meta-learning) are peer-reviewed prior components and are not invoked as uniqueness theorems or as definitions of the target claim; the DRN itself is evaluated in-paper, but that evaluation is an ablation, not a prediction derived from the same training objective. A genuine gap exists in Section 4.4: Eq. 9 is written as an arg min and no gradient rule is given for the discrete topK selection of Eq. 2, and the DRN ablation (Section 5.4) confounds mask generation with mask granularity. These are reproducibility and experimental-design concerns, not circularity. Accordingly no circular step can be quoted, and the score is 0.

Assumptions & free parameters 5 free parameters · 5 assumptions · 1 invented entities

The central empirical claims rest on hand-tuned hyperparameters (mask ratio, beta), on the assumption that auxiliary reconstruction and inverse losses transfer to the RL objective, and on an underspecified meta-learning update for DRN. No code or error bars are provided, and the main figure is unreadable, so the ledger cannot be audited beyond the text.

free parameters (5)

mask ratio (communication frequency) = 0.4 masked, 60% communicated
Hand-chosen hyperparameter in Table 5 (mask ratio 0.4); determines the top-k dimension selection in Equation 2 and directly underlies the communication efficiency claims; ablation shows 0.6 communication rate is best.
beta (auxiliary loss weight) = 1
Coefficient balancing L_RL and auxiliary losses in Equation 8; set to 1 by hand, no sweep reported.
top-k selection threshold = top 60% of dimensions
Derived from mask ratio; the DRN weights are thresholded by this k, and the choice affects the masking behavior.
message hidden dimension multiplier = 8*n_agent
Architecture size in Table 4 for message encoder output and decoder input; chosen without reported tuning, affects capacity for state reconstruction.
network hidden sizes for DRN = 32, 32
Architecture in Table 4; shapes what importance signals can be represented.

assumptions (5)

domain assumption Dec-POMDP with CTDE, global states available during training
Section 3; the reconstruction loss Equation 6 requires access to s_t in training, and the RL loss uses centralized Q_tot, which is standard but nontrivial.
domain assumption Reconstructing global states and predicting joint actions from message representations improves cooperative decision-making
Section 4.3 states these self-supervised objectives make representations sufficient and informative, but no theoretical or ablation-only evidence isolates the mechanism from other changes.
domain assumption The top-k masking preserves decision-critical information
Section 4.2 assumes that zeroing non-top-k dimensions does not remove information needed for effective decisions; ablation at 0.6 com. rate supports it empirically but only for that setting.
ad hoc to paper Meta-learning with trial weights trains DRN without trivial solutions
Section 4.4 asserts second-derivative meta-learning avoids trivial solutions and local optima, but the exact update for theta_DRN is not specified; this is an unproved design claim.
ad hoc to paper Communication frequency measured by the fraction of unmasked dimensions reflects real communication cost
Section 5.3 computes efficiency as performance gain divided by frequency; this equates zeroed dimensions with saved bandwidth, which may not hold if zeros are still transmitted.

invented entities (1)

Dimensional Rational Network (DRN)
purpose: Learns per-dimension importance weights for each agent's observation and drives top-k masking for information sharing (Section 4.4)
No external benchmark or independent source validates DRN; its usefulness is based only on the paper's own ablations and visualizations, and its training update is underspecified.

how reviews work

0 comments

Cite this review

Pith. "Pith review of M2I2: Learning Efficient Multi-Agent Communication via Masked State Modeling and Intention Inference." pith.science (2026). https://pith.science/paper/G3WATKJH

@misc{pith2026250100312,
  author       = {Pith},
  title        = {Pith review of: M2I2: Learning Efficient Multi-Agent Communication via Masked State Modeling and Intention Inference},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/G3WATKJH}},
  note         = {Machine review of arXiv:2501.00312}
}

read the original abstract

Communication is essential in coordinating the behaviors of multiple agents. However, existing methods primarily emphasize content, timing, and partners for information sharing, often neglecting the critical aspect of integrating shared information. This gap can significantly impact agents' ability to understand and respond to complex, uncertain interactions, thus affecting overall communication efficiency. To address this issue, we introduce M2I2, a novel framework designed to enhance the agents' capabilities to assimilate and utilize received information effectively. M2I2 equips agents with advanced capabilities for masked state modeling and joint-action prediction, enriching their perception of environmental uncertainties and facilitating the anticipation of teammates' intentions. This approach ensures that agents are furnished with both comprehensive and relevant information, bolstering more informed and synergistic behaviors. Moreover, we propose a Dimensional Rational Network, innovatively trained via a meta-learning paradigm, to identify the importance of dimensional pieces of information, evaluating their contributions to decision-making and auxiliary tasks. Then, we implement an importance-based heuristic for selective information masking and sharing. This strategy optimizes the efficiency of masked state modeling and the rationale behind information sharing. We evaluate M2I2 across diverse multi-agent tasks, the results demonstrate its superior performance, efficiency, and generalization capabilities, over existing state-of-the-art methods in various complex scenarios.

Figures

Figures reproduced from arXiv: 2501.00312 by the authors.

**Figure 2.** Performance on multiple benchmarks. multi-agent communication strategies. Detailed descriptions of each environment can be found in Appendix C. Baselines. For comparative analysis, we select a diverse set of baselines. This includes QMIX (Rashid et al. 2018), a well-established MARL algorithm that operates without a communication mechanism. To assess our method’s performance in the context of communication-enhanced… view at source ↗

**Figure 4.** Generation. tently show the best performance at a communication rate of 0.6. This result suggests that an excessively high communication rate can introduce redundant and misleading information, whereas too low a rate may lead to critical information being overlooked. Intriguingly, reducing information by 0.6 using the meta mask still outperforms a random mask reduction of 0.4, underscoring the meta mask’s profi… view at source ↗

Figures from the paper (6 more)

**Figure 5.** Figure 5: Multiple environments considered in our experiments. [PITH_FULL_IMAGE:figures/full_fig_p011_5.png]

**Figure 6.** Figure 6: Visualization of weight learn by DRN [PITH_FULL_IMAGE:figures/full_fig_p013_6.png]

**Figure 7.** Figure 7: (a) Presents the probability of masking for the three types of observations at different stages. (b) Illustrates the [PITH_FULL_IMAGE:figures/full_fig_p014_7.png]

**Figure 8.** Figure 8: Visualization of learned representations [PITH_FULL_IMAGE:figures/full_fig_p015_8.png]

**Figure 9.** Figure 9: Reconstruct Loss E Details of Computational Resources The computational experiments described in this paper were executed on a dedicated high-performance computing cluster to ensure the reproducibility and efficiency of the results. Below, we provide the detail of the …

**Figure 10.** Figure 10: Inverse Loss – Quantity: 3 NVIDIA TITAN Xps – Memory: 12 GB GDDR6X per GPU • CPU Specifications: – Model: Intel(R) Xeon(R) Silver 4114 CPU – Architecture: x86 64 – Base Clock Speed: 2.20GHz • Software and Frameworks: – Operating System: 16.04.1-Ubuntu SMP – Machine Le…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

35 extracted references · 21 canonical work pages

[1]

, " * write output.state after.block = add.period write newline

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...
[2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...
[3]

M.; Baker, B.; Chociej, M.; Jozefowicz, R.; McGrew, B.; Pachocki, J.; Petron, A.; Plappert, M.; Powell, G.; Ray, A.; et al

Andrychowicz, O. M.; Baker, B.; Chociej, M.; Jozefowicz, R.; McGrew, B.; Pachocki, J.; Petron, A.; Plappert, M.; Powell, G.; Ray, A.; et al. 2020. Learning dexterous in-hand manipulation. The International Journal of Robotics Research, 39(1): 3--20

2020
[4]

Das, A.; Gervet, T.; Romoff, J.; Batra, D.; Parikh, D.; Rabbat, M.; and Pineau, J. 2019. Tarmac: Targeted multi-agent communication. In International Conference on Machine Learning, 1538--1546

work page 2019
[5]

Devlin, J.; Chang, M.; Lee, K.; and Toutanova, K. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Burstein, J.; Doran, C.; and Solorio, T., eds., Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapo...

2019
[6]

Ding, Z.; Huang, T.; and Lu, Z. 2020. Learning individually inferred communication for multi-agent cooperation. Advances in Neural Information Processing Systems, 33: 22069--22079

work page 2020
[7]

Etel, E.; and Slaughter, V. 2019. Theory of mind and peer cooperation in two play contexts. Journal of Applied Developmental Psychology, 60: 87--95

work page 2019
[8]

Guan, C.; Chen, F.; Yuan, L.; Wang, C.; Yin, H.; Zhang, Z.; and Yu, Y. 2022. Efficient Multi-agent Communication via Self-supervised Information Aggregation. Advances in Neural Information Processing Systems, 35: 1020--1033

work page 2022

Show all 35 references

[9]

He, K.; Chen, X.; Xie, S.; Li, Y.; Doll \'a r, P.; and Girshick, R. 2022. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 16000--16009

2022
[10]

J.; Lee, T.; Son, K.; and Yi, Y

Kim, D.; Moon, S.; Hostallero, D.; Kang, W. J.; Lee, T.; Son, K.; and Yi, Y. 2019. Learning to schedule communication in multi-agent reinforcement learning. arXiv preprint arXiv:1902.01554

2019 arXiv
[11]

Leurent, E. 2018. A survey of state-action representations for autonomous driving

2018
[12]

Li, J.; Qiang, W.; Zheng, C.; Su, B.; and Xiong, H. 2022. MetAug: Contrastive Learning via Meta Feature Augmentation. In Chaudhuri, K.; Jegelka, S.; Song, L.; Szepesv \' a ri, C.; Niu, G.; and Sabato, S., eds., International Conference on Machine Learning, ICML 2022, 17-23 Jul...

2022
[13]

Liu, F.; Liu, H.; Grover, A.; and Abbeel, P. 2022. Masked autoencoding for scalable and generalizable decision making. Advances in Neural Information Processing Systems, 35: 12608--12618

2022
[14]

Liu, S.; Davison, A.; and Johns, E. 2019. Self-supervised generalisation with meta auxiliary learning. Advances in Neural Information Processing Systems, 32

2019
[15]

I.; Tamar, A.; Harb, J.; Abbeel, O

Lowe, R.; Wu, Y. I.; Tamar, A.; Harb, J.; Abbeel, O. P.; and Mordatch, I. 2017. Multi-agent actor-critic for mixed cooperative-competitive environments. In Advances in neural information processing systems, 6379--6390

2017
[16]

R.; and Gombolay, M

Niu, Y.; Paleja, R. R.; and Gombolay, M. C. 2021. Multi-Agent Graph-Attention Communication and Teaming. In AAMAS, 964--973

2021
[17]

A.; Amato, C.; et al

Oliehoek, F. A.; Amato, C.; et al. 2016. A concise introduction to decentralized POMDPs, volume 1. Springer

2016
[18]

Osband, I.; Blundell, C.; Pritzel, A.; and Van Roy, B. 2016. Deep exploration via bootstrapped DQN. In Advances in neural information processing systems, 4026--4034

2016
[19]

S.; Farquhar, G.; Foerster, J.; and Whiteson, S

Rashid, T.; Samvelyan, M.; De Witt, C. S.; Farquhar, G.; Foerster, J.; and Whiteson, S. 2018. QMIX: Monotonic value function factorisation for deep multi-agent reinforcement learning. arXiv preprint arXiv:1803.11485

2018 arXiv
[20]

S.; Farquhar, G.; Nardelli, N.; Rudner, T

Samvelyan, M.; Rashid, T.; de Witt, C. S.; Farquhar, G.; Nardelli, N.; Rudner, T. G.; Hung, C.-M.; Torr, P. H.; Foerster, J.; and Whiteson, S. 2019. The starcraft multi-agent challenge. arXiv preprint arXiv:1902.04043

2019 arXiv
[21]

Silver, D.; Hubert, T.; Schrittwieser, J.; Antonoglou, I.; Lai, M.; Guez, A.; Lanctot, M.; Sifre, L.; Kumaran, D.; Graepel, T.; et al. 2018. A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play. Science, 362(6419): 1140--1144

2018
[22]

Silver, D.; Schrittwieser, J.; Simonyan, K.; Antonoglou, I.; Huang, A.; Guez, A.; Hubert, T.; Baker, L.; Lai, M.; Bolton, A.; et al. 2017. Mastering the game of go without human knowledge. nature, 550(7676): 354--359

2017
[23]

Singh, A.; Jain, T.; and Sukhbaatar, S. 2018. Learning when to communicate at scale in multiagent cooperative and competitive tasks. arXiv preprint arXiv:1812.09755

2018 arXiv
[24]

Sukhbaatar, S.; Szlam, A.; and Fergus, R. 2016. Learning Multiagent Communication with Backpropagation. In Proceedings of the 30th International Conference on Neural Information Processing Systems, NIPS'16, 2252–2260. Red Hook, NY, USA: Curran Associates Inc. ISBN 9781510838819

2016
[25]

Sun, C.; Wu, B.; Wang, R.; Hu, X.; Yang, X.; and Cong, C. 2021. Intrinsic Motivated Multi-Agent Communication. In Proceedings of the 20th International Conference on Autonomous Agents and MultiAgent Systems, AAMAS '21, 1668–1670. Richland, SC: International Foundation for Auto...

2021
[26]

Van der Maaten, L.; and Hinton, G. 2008. Visualizing data using t-SNE. Journal of machine learning research, 9(11)

2008
[27]

N.; Kaiser, L.; and Polosukhin, I

Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A. N.; Kaiser, L.; and Polosukhin, I. 2017. Attention is All You Need

2017
[28]

M.; Mathieu, M.; Dudzik, A.; Chung, J.; Choi, D

Vinyals, O.; Babuschkin, I.; Czarnecki, W. M.; Mathieu, M.; Dudzik, A.; Chung, J.; Choi, D. H.; Powell, R.; Ewalds, T.; Georgiev, P.; et al. 2019. Grandmaster level in StarCraft II using multi-agent reinforcement learning. Nature, 575(7782): 350--354

2019
[29]

Wang , T.; Wang , J.; Zheng , C.; and Zhang , C. 2020. Learning Nearly Decomposable Value Functions Via Communication Minimization. In ICLR 2020 : Eighth International Conference on Learning Representations

2020
[30]

Wang, Y.; Han, B.; Wang, T.; Dong, H.; and Zhang, C. 2020. Dop: Off-policy multi-agent decomposed policy gradients. In International Conference on Learning Representations

2020
[31]

Xue, D.; Yuan, L.; Zhang, Z.; and Yu, Y. 2022. Efficient Multi-Agent Communication via Shapley Message Value. In Raedt, L. D., ed., Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence, IJCAI-22 , 578--584. International Joint Conferences o...

2022
[32]

Yu, C.; Velu, A.; Vinitsky, E.; Gao, J.; Wang, Y.; Bayen, A.; and Wu, Y. 2022. The surprising effectiveness of ppo in cooperative multi-agent games. Advances in Neural Information Processing Systems, 35: 24611--24624

2022
[33]

Yuan, L.; Wang, J.; Zhang, F.; Wang, C.; Zhang, Z.; Yu, Y.; and Zhang, C. 2022. Multi-agent incentive communication via decentralized teammate modeling. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, 9466--9474

2022
[34]

Q.; Zhang, Q.; and Lin, J

Zhang, S. Q.; Zhang, Q.; and Lin, J. 2019. Efficient communication in multi-agent reinforcement learning via variance based control. In Advances in Neural Information Processing Systems, 3235--3244

2019
[35]

Q.; Zhang, Q.; and Lin, J

Zhang, S. Q.; Zhang, Q.; and Lin, J. 2020. Succinct and robust multi-agent communication with temporal message control. Advances in Neural Information Processing Systems, 33: 17271--17282

2020

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.

[1] [1]

, " * write output.state after.block = add.period write newline

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

[2] [2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

[3] [3]

M.; Baker, B.; Chociej, M.; Jozefowicz, R.; McGrew, B.; Pachocki, J.; Petron, A.; Plappert, M.; Powell, G.; Ray, A.; et al

Andrychowicz, O. M.; Baker, B.; Chociej, M.; Jozefowicz, R.; McGrew, B.; Pachocki, J.; Petron, A.; Plappert, M.; Powell, G.; Ray, A.; et al. 2020. Learning dexterous in-hand manipulation. The International Journal of Robotics Research, 39(1): 3--20

2020

[4] [4]

Das, A.; Gervet, T.; Romoff, J.; Batra, D.; Parikh, D.; Rabbat, M.; and Pineau, J. 2019. Tarmac: Targeted multi-agent communication. In International Conference on Machine Learning, 1538--1546

work page 2019

[5] [5]

Devlin, J.; Chang, M.; Lee, K.; and Toutanova, K. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Burstein, J.; Doran, C.; and Solorio, T., eds., Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapo...

2019

[6] [6]

Ding, Z.; Huang, T.; and Lu, Z. 2020. Learning individually inferred communication for multi-agent cooperation. Advances in Neural Information Processing Systems, 33: 22069--22079

work page 2020

[7] [7]

Etel, E.; and Slaughter, V. 2019. Theory of mind and peer cooperation in two play contexts. Journal of Applied Developmental Psychology, 60: 87--95

work page 2019

[8] [8]

Guan, C.; Chen, F.; Yuan, L.; Wang, C.; Yin, H.; Zhang, Z.; and Yu, Y. 2022. Efficient Multi-agent Communication via Self-supervised Information Aggregation. Advances in Neural Information Processing Systems, 35: 1020--1033

work page 2022