{"id":"61386bb8-30e9-4448-89c0-c239e63749e3","arxiv_id":"2501.00312","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"M2I2 improves multi-agent communication by adding masked state reconstruction and joint-action prediction auxiliary tasks, plus a meta-learned module that selects which observation dimensions to share.","lead":"Deep RL agents that talk to each other usually struggle to use what they receive. This paper adds self-supervised tasks, reconstructing the hidden global state and predicting teammates' next actions, plus a learned masking network, and reports large gains on multi-agent benchmarks.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The claimed ability of the DRN to learn useful top-k masks is load-bearing for M2I2's efficiency and ablation, but the paper gives no implementable gradient rule for the discrete selection in Eq. 2, leaving the central mechanism underdetermined.","rationale":"The reader's conditional verdict and its identified weakest assumption both target the DRN, and I agree that the DRN is load-bearing. My primary concern is slightly more empirical: even granting that a gradient through the selected weights exists, the paper never specifies how the discrete top-k selection is differentiated, and the only ablation isolating DRN (Figure 3(a)) changes two variables simultaneously—learned versus random masking and dimension-level versus observation-level granularity. This confound means the reported DRN contribution and the efficiency advantage in Table 1 could be explained by the auxiliary losses or by the finer-grained dimension mask rather than by the learned importance model. The paper provides no code, no error bars, and a corrupted key figure, so these claims cannot currently be checked. The proposed control experiment would settle whether the DRN's learned importance is actually responsible for the reported gains. Because the underlying idea is plausible and the flaws are about missing specification and experimental control rather than a demonstrated negative result, I would keep the reader's conditional verdict rather than escalate to rejection.","tokens_in":18359,"tokens_out":11124,"duration_ms":125867,"concrete_test":"Run the ablation of Figure 3(a) with an additional control that replaces DRN with random dimension-level masks at the same 60% retention rate, keeping all other training and communication details identical. If M2I2's DRN variant does not outperform this random-dimension control across the three SMAC-Communication scenarios, the claimed meta-learned mask advantage is not established.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central efficiency and ablation claims depend on the DRN learning to mask observation dimensions. The only DRN update is Eq. 9, written as an arg min, after the trial-weight step Eq. 10; Algorithm 1 ends there. From Eq. 2, topK is a hard selection: selected dimensions retain their weight, all others are zero. The paper does not state whether gradients flow through the selected weights, through the selection itself, or through a relaxation; no straight-through estimator, Gumbel-Softmax, or REINFORCE rule is given. Without such a rule, the discrete choice of which dimensions enter the mask has no defined gradient, so the claimed meta-learning for mask selection is not implementable from the text. This matters because Figure 3(a) and Table 1 attribute the efficiency gain to DRN, but the ablation also confounds DRN removal with a change from dimension-level top-k masking to random observation-level masking. Thus the reported advantage could arise from the auxiliary losses or from the mask granularity rather than from a learned importance model. The paper's central mechanism is consequently unverified as written.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes M2I2, a framework for cooperative multi-agent reinforcement learning with communication. It combines masked state modeling (a state-level masked auto-encoder reconstructing global states from received messages), an inverse model for joint-action prediction (intention inference), and a Dimensional Rational Network (DRN) that learns dimension-level importance weights for observations. A top-k mechanism masks low-importance dimensions before sharing, reducing communication cost, and the DRN is trained with a meta-learning paradigm. The method is evaluated on Hallway, Predator-Prey, SMAC, and SMAC-Communication against QMIX, TarMAC, MAIC, SMS, and MASIA, reporting performance, communication efficiency, ablations, and generalization results.","tokens_in":18613,"tokens_out":7134,"duration_ms":68234,"significance":"If the claims are correct, M2I2 would be a meaningful contribution to communication-efficient MARL: it explicitly targets the receiving-end integration problem and uses self-supervised auxiliary tasks to shape representations, an interesting and timely direction. The evaluation is external (standard benchmarks and baselines), so there is no fitting-is-prediction circularity. The paper also ships auxiliary-loss learning curves and visualizations of the learned masks, and reports parameter counts and runtime, which helps reproducibility. However, the central DRN training mechanism is underspecified, the ablation does not isolate the DRN, and the empirical evidence lacks uncertainty quantification. These gaps prevent the paper from being accepted in its current form.","major_comments":[{"comment":"The training procedure for the DRN is not implementable from the text. In Eq. (2), topK is a hard selection: dimensions outside the top-k are set to zero, so the mask has discontinuous dependence on theta_DRN. The paper does not state a gradient estimator (straight-through, Gumbel-Softmax, REINFORCE, or another surrogate) for this discrete selection. Furthermore, Eq. (9) is written as an arg min, not an update, and Algorithm 1 says 'Update theta_DRN by Equation 9' without giving an explicit gradient step. The 'second-derivative technique' is mentioned but not specified. Because the DRN and its top-k masking are load-bearing for both the communication-efficiency claim and the ablation, the authors must provide the exact update rule and the differentiable treatment of the top-k selection.","section":"Section 4.4, Eq. (2), Eq. (9), Eq. (10), Algorithm 1"},{"comment":"The ablation 'M2I2 w/o DRN' removes the DRN and the top-k filter, and replaces them with random observation-level masking. This simultaneously changes two factors: (a) learned importance weights versus random selection, and (b) dimension-level masking versus whole-observation binary gating. Consequently, the performance difference between M2I2 and this variant cannot be attributed solely to the DRN's learned importance; it could stem from the change in mask granularity or from the randomness of the gating. An additional ablation is needed, for example using random dimension-level top-k masking, or retaining the DRN with observation-level masking, to separate these factors.","section":"Section 5.4 (ablation)"},{"comment":"The paper states that results are averaged over 5 random seeds but does not report variances, standard deviations, confidence intervals, or significance tests anywhere. The claims in Section 5.2 of 'consistently outperforming all baselines by a significant margin' are not supported without uncertainty quantification. MARL results typically exhibit non-negligible seed variance, so the authors should add error bars or shaded regions to the learning curves, report standard deviations in tables, and, if they wish to use the word 'significant,' accompany it with an appropriate statistical test.","section":"Section 5.2 and Appendix C"},{"comment":"The communication-efficiency metric divides performance improvement by communication frequency, but the improvement is measured relative to different baselines for different methods: DOP for SMS and QMIX for all other algorithms. This makes the improvement values non-comparable across methods. In addition, 'communication frequency' appears to be defined differently for different methods (fraction of retained dimensions for M2I2 versus a gating probability for SMS, versus 100% for always-communicating baselines). The authors should use a consistent baseline and a consistent definition of communication frequency, or justify why the current heterogeneous comparison is fair.","section":"Section 5.3, Table 1"}],"minor_comments":[{"comment":"Figure 2 is corrupted in the submitted manuscript (it renders as a long string of font paths), and Figure 3 is also garbled. Please re-upload the actual figures.","section":"Figures 2 and 3"},{"comment":"The heading 'Generation' should be 'Generalization.'","section":"Section 5.5"},{"comment":"The inverse-model loss uses L2 error between a softmax output and a one-hot action vector; this is unconventional, and cross-entropy would be more natural. Please justify the choice or change the loss.","section":"Eq. (7)"},{"comment":"The phrase 'trail weight' should be 'trial weight.'","section":"Algorithm 1"},{"comment":"The claim of being the 'first instance' of incorporating self-supervised objectives into multi-agent communication is too strong given that MASIA already uses self-supervised learning (an autoencoder and a forward model); the novelty should be positioned as the masked modeling and inverse-model combination rather than self-supervision per se.","section":"Section 1 and Related Work"},{"comment":"The notation c_i^t for received messages is introduced but not used in the rest of the paper; please clarify how c_i^t relates to m_j^t and the integrated representation z_i^t.","section":"Section 3"}],"recommendation":"major_revision","confidential_remarks":"The manuscript leans heavily on the authors' own prior work (IMMAC, MetAug) for the meta-learning recipe and gating heuristic, which is not by itself disqualifying, but the DRN update is specified so loosely that the central mechanism cannot be reproduced. The missing error bars and the confounded ablation are additional load-bearing issues. If the authors can provide a precise, implementable DRN update, an ablation that isolates the DRN, and proper uncertainty quantification, the paper could become a solid contribution."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Let me give you the short version: this paper is worth your attention if you work on multi-agent communication, but it is not ready as written. The core idea—use self-supervised reconstruction of the global state and inverse prediction of joint actions to help agents integrate incoming messages, plus a learned dimensional mask to decide what to say—is plausible and the empirical results look consistently positive across Hallway, Predator-Prey, and SMAC variants. The novelty is real but incremental: MASIA already applied self-supervised aggregation; M2I2 extends this with masked state modeling and a meta-learned importance network. That's a reasonable contribution, though not a breakthrough.\n\nThe writing is clear about the problem: existing methods focus on what to communicate, when, and with whom, but pay less attention to how receivers synthesize the messages. The two auxiliary losses are well-motivated, and the visualization of what the DRN focuses on (e.g., ally vs. enemy features during exploration vs. combat) is a nice touch.\n\nThe soft spots are significant. First, the DRN training is not actually specified. Equation 9 is an arg min over θ_DRN, but no gradient update rule is given; the top-k selection in Eq. 2 is hard, non-differentiable, and no straight-through estimator or relaxation is mentioned. As written, I cannot implement the central mechanism. That's not a minor omission; it's the thing that makes the efficiency claim work. Second, there are no error bars. The authors say results are averaged over five seeds, but no variance is shown, so I can't judge whether the improvements are meaningful. Third, the efficiency metric (performance improvement divided by communication frequency) is not carefully controlled; the ablation that removes DRN also changes from dimension-level top-k masking to random observation-level masking, so the two are confounded. Finally, Figure 2 is corrupted in the manuscript, which hurts readability.\n\nWho should read this? Researchers working on self-supervised methods for MARL communication, and anyone interested in how meta-learned masking could reduce communication costs. It deserves a serious referee: the core idea is sound, the issues are addressable with a careful revision, and the empirical claims, if they hold up, would be useful to the community. I'd recommend sending it to review, but with a strong request to clarify the DRN update (and ideally provide code), add error bars, and redo the ablation with a cleaner control.","headline":"A promising but under-specified MARL communication method: masked state modeling plus intention inference and a meta-learned dimensional mask; the meta-update and gradient path need clarification, and the evaluation lacks error bars.","tokens_in":19141,"tokens_out":5966,"would_cite":true,"duration_ms":53631,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"M2I2 claims that improving how agents integrate received messages, through masked state reconstruction and joint-action prediction, makes cooperative agents communicate less while coordinating better.","keywords":["multi-agent reinforcement learning","multi-agent communication","communication efficiency","masked state modeling","intention inference","dimensional rational network","meta-learning","self-supervised auxiliary tasks"],"falsifier":"Re-run M2I2 on the SMAC-Communication map 1o_2r_vs_4r with the DRN replaced by fixed random top-k masks chosen at the same 0.6 communication rate; if random masks reproduce the reported win rate, the claim that the meta-learned importance masks drive the gains is falsified. A second check is to instrument the training loop: if the gradient of the M2I2 loss with respect to $\\theta_{\\mathrm{DRN}}$ is zero at every step, the DRN is not being trained as described.","tokens_in":18141,"feed_emoji":"📡","tokens_out":8781,"duration_ms":78547,"temperature":0.7,"pith_summary":"This paper argues that the bottleneck in multi-agent communication is not just what agents send but how they digest what they receive. To close that gap, M2I2 gives each agent two self-supervised jobs while training: reconstruct the environment's global state from the messages it received, and predict the joint action that took the team from one state representation to the next. A separate network, the Dimensional Rational Network, is meta-trained to score each dimension of an observation and keep only the top-k most useful dimensions when sharing. The paper reports that this combination outperforms existing communication methods on Hallway, Predator-Prey, and SMAC-style benchmarks, while transmitting at only 60% frequency and achieving higher communication efficiency on every tested scenario.","feed_headline":"Masked state modeling helps agents share less and win more","feed_subtitle":"M2I2 rebuilds global states from received messages, predicts joint actions, and reports higher efficiency at 60 percent bandwidth","key_machinery":"The load-bearing pieces are: (1) a masked state auto-encoder, a message encoder plus state decoder that predicts the global state from received messages; (2) an inverse model that predicts the joint action from two consecutive message-integrated representations, providing intention inference; and (3) the Dimensional Rational Network (DRN), a meta-learned importance scorer whose top-k mask selects which observation dimensions each agent transmits. The DRN's importance weights are supposed to be updated through a meta-learning step that evaluates their effect on the combined RL-reconstruction-inverse loss, which is what ties the communication masks to both task success and the auxiliary objectives.","core_discovery":"M2I2's central claim is that information integration at the receiving end is the neglected factor in multi-agent communication, and that it can be taught directly by two auxiliary objectives. The state decoder and message encoder form a masked auto-encoder that reconstructs the full global state from a partial, masked set of received messages; the inverse model takes two consecutive integrated representations and predicts the joint action, giving agents an implicit model of teammates' intentions. The Dimensional Rational Network meta-learns per-dimension importance weights from their gradient contribution to both the auxiliary losses and the RL loss, and a top-k scheduler masks out everything below the top fraction, so agents share only the dimensions that matter. On Hallway, Predator-Prey, SMAC, and SMAC-Communication, the paper reports consistent improvements over TarMAC, MAIC, SMS, MASIA, and communication-free QMIX, and a communication-efficiency lead at 60% communication frequency.","pith_inferences":["A testable extension is to use the state-reconstruction loss as a live signal to adapt the top-k ratio per agent and per phase, instead of the fixed 0.6 frequency reported here.","Because the top-k operation in Equation 2 is discrete, the described meta-learning update will likely need a differentiable relaxation, such as a straight-through estimator or Gumbel-softmax, to actually train the DRN; the paper does not specify one.","The intention-inference objective could be used as an auxiliary loss in a fully decentralized MARL system with no explicit messages at all, which would test whether representation quality alone, rather than selective sharing, drives the reported gains.","If reconstructions from received messages are rich enough to predict global states, the learned representations may transfer to zero-shot coordination with new teammates, a scenario beyond the paper's benchmarks."],"forward_implications":["At the communication frequency the paper uses (60%), M2I2 reports higher communication efficiency, measured as performance gain per unit of communication, than all tested baselines in every evaluated environment.","The ablations indicate both the inverse model and the DRN contribute: removing them degrades performance on the SMAC-Communication maps.","M2I2's message encoder and auxiliary objectives are compatible with multiple MARL backbones, including QMIX, VDN, QPLEX, MAPPO, and MADDPG, so the integration recipe transfers across value-based and policy-gradient training.","The communication-rate ablation suggests that 0.6 beats 0.8, implying that in these tasks extra shared information can be noise rather than signal."],"supporting_citations":[{"why":"Prior self-supervised information aggregation in multi-agent communication; M2I2 extends it by relaxing the assumption that agents see all peers' observations.","marker":"(Guan et al. 2022)"},{"why":"Source of the masked autoencoding technique that M2I2 adapts to multi-agent state reconstruction.","marker":"(He et al. 2022)"},{"why":"TarMAC is the attention-based message-integration baseline that M2I2 compares against and aims to surpass at the receiving end.","marker":"(Das et al. 2019)"},{"why":"Provides the meta-learning and second-derivative update used to train the Dimensional Rational Network.","marker":"(Liu, Davison, and Johns 2019)"},{"why":"Source of the communication-efficiency metric M2I2 uses to compare performance gain per communication frequency.","marker":"(Niu, Paleja, and Gombolay 2021)"},{"why":"Provides the scaled dot-product self-attention module used to integrate received messages.","marker":"(Vaswani et al. 2017)"},{"why":"QMIX is the communication-free value-based baseline and the backbone for several M2I2 variants.","marker":"(Rashid et al. 2018)"},{"why":"Defines the SMAC benchmark and its reward structure used in the main experiments.","marker":"(Samvelyan et al. 2019)"},{"why":"Supplies the Hallway benchmark and the SMAC-Communication configuration used in the evaluations.","marker":"(Wang et al. 2020)"}],"fun_headline_variants":["M2I2: Mask what's irrelevant, infer teammates' moves","Agents learn to share less, predict more with M2I2","Meta-masking makes multi-agent communication efficient","Selective masking helps agents infer intents and act","Infer teammates' intentions, share less: M2I2"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The central premise is that the meta-learning update can train the Dimensional Rational Network to pick useful dimensions even though the top-k selection is discrete and the paper gives no explicit gradient path into it; if that premise fails, the claimed efficiency gains from selective masking do not follow.","fun_headline_variants_meta":{"raw":{"variants":["M2I2: Mask what's irrelevant, infer teammates' moves","Agents learn to share less, predict more with M2I2","Meta-masking makes multi-agent communication efficient","Selective masking helps agents infer intents and act","Infer teammates' intentions, share less: M2I2"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000713,"raw_usage":{"total_tokens":3219,"prompt_tokens":970,"completion_tokens":2249,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":586,"completion_tokens_details":{"reasoning_tokens":2164}},"tokens_in":586,"tokens_out":2249,"duration_ms":34369,"temperature":1.0,"reasoning_tokens":2164,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T22:53:42.862951+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run M2I2 on the SMAC-Communication map 1o_2r_vs_4r with the DRN replaced by fixed random top-k masks chosen at the same 0.6 communication rate; if random masks reproduce the reported win rate, the claim that the meta-learned importance masks drive the gains is falsified. A second check is to instrument the training loop: if the gradient of the M2I2 loss with respect to $\\theta_{\\mathrm{DRN}}$ is zero at every step, the DRN is not being trained as described.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Prior self-supervised information aggregation in multi-agent communication; M2I2 extends it by relaxing the assumption that agents see all peers' observations."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"TarMAC is the attention-based message-integration baseline that M2I2 compares against and aims to surpass at the receiving end."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the meta-learning and second-derivative update used to train the Dimensional Rational Network."},{"cited_title":"R.; and Gombolay, M","cited_arxiv_id":null,"evidence_quote":"Source of the communication-efficiency metric M2I2 uses to compare performance gain per communication frequency."},{"cited_title":"N.; Kaiser, L.; and Polosukhin, I","cited_arxiv_id":null,"evidence_quote":"Provides the scaled dot-product self-attention module used to integrate received messages."}],"review_version":1}