{"id":"796b4e17-a7c9-4cdc-92fe-f7ea342b7af7","arxiv_id":"2504.21278","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"DMAC is an adversarial training method that masks critical communication channels to force multi-agent communication policies into a more balanced, decentralized structure, improving robustness and task performance.","lead":"This paper introduces DMAC, a training method that makes multi-agent communication policies more robust by having an adversary learn which communication channels matter and then masking them during retraining. The retrained policies become less dependent on any single channel, which the authors find improves their win rate under attacks and in normal play.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Algorithm 1 never retrains the communication policy, so the central robustness mechanism is asserted but not specified or verified.","rationale":"The paper has real independent scaffolding: a concrete adversary formulation (Equations 1–3, Qtot/IGM in Equations 4–5), two target policies, four environments, and code release, so I read the central claim in good faith as an empirical training-time defense. The reader's weakest assumption concerned transfer from binary masking to the message-perturbation attacks used in evaluation; that is a legitimate concern about surrogate robustness. I think a more load-bearing gap sits earlier: Algorithm 1, the only training procedure provided, never updates the communication policy. The abstract and Section 3.2 promise 'adversarial learning of the communication policy,' but the algorithm's output is only πc and C, with π fixed throughout. Without a specified CP update, the mechanism that would produce Table 1's robustness improvements is absent from the paper. This is not an internal contradiction in the math; it is an omitted procedural step that is essential to the central claim. The missing appendix, referenced for 'more evaluation results' and training details, cannot be assumed to fill the gap because the main text should at minimum state the CP loss and update loop. My proposed test is therefore to verify the released code: if the CP update exists there, the concern reduces to a presentation problem and the conditional accept stands; if it does not exist, the reported gains are not explained by the described method. I do not accuse the authors of anything; the code may well implement the missing step, but the manuscript as written is not self-contained on the point that matters most. Because the issue is checkable and potentially presentation-only, CONDITIONAL (not REJECT) is the right final verdict, with the condition being explicit verification and description of the CP retraining loop.","tokens_in":13321,"tokens_out":4707,"duration_ms":49249,"concrete_test":"Inspect the released code at https://anonymous.4open.science/r/IJCAI2025-FB61 and locate the training loop for DMAC: identify any weight update applied to the communication policy CP (T2MAC or I2C) when DMAC_Adv's masks are active. If no such update exists, the central claim fails. If an update exists, run a controlled experiment on one environment (e.g., SC with T2MAC+DMAC) with that CP update disabled; if the attacked win rate drops back to the unoptimized T2MAC level, the retraining step is indeed load-bearing and the paper should describe it. This single check settles whether Algorithm 1 omits the defense mechanism or merely omits its description.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim is that DMAC is a training-time defense that retrains a communication policy CP against adversarial masking, forcing it into a decentralized pattern. The only training procedure given is Algorithm 1: its input is the already-trained target policy π, lines 4–15 compute masking actions, apply masks (line 9), execute the fixed policy (line 11), and update only the masking policy πc and critic C (line 14). No update rule for CP/π appears anywhere in Section 3.2; there is no objective for CP, no alternation schedule, and no hyperparameters for the adversarial retraining step. The sentence in Section 3.2 saying 'the communication policy is retrained in the process of adversarial training' is an assertion without a corresponding equation or pseudocode, and the referenced appendix is not included in the manuscript. If Algorithm 1 is the complete method, the reported robustness gains in Table 1 have no mechanism in the paper: a fixed policy evaluated under masks cannot become more robust from this procedure. This is a missing-support gap in the central claim, not a stylistic issue. A secondary concern is that the training adversary masks entire channels (Eq. 1) while the evaluation attacks add message perturbations (Section 4.1); even if CP retraining were specified, transfer from masking to perturbation would need a random-masking control to be credible.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes DMAC, an adversarial training method intended to improve the robustness of learned multi-agent communication policies. DMAC trains an adversary (DMAC_Adv) to identify and mask critical communication channels, then claims to retrain the target communication policy on the resulting adversarial samples so that the policy becomes less dependent on any single channel and shifts toward a decentralized communication structure. The authors evaluate DMAC on two communication policies, T2MAC and I2C, across four tasks (SMAC, Cooperative Navigation, Traffic Junction, and Predator-Prey), reporting higher win rates than AME and R-MACRL under learned adaptive and heuristic attacks, slightly improved performance in normal conditions, and reduced variance in per-channel communication frequencies.","tokens_in":13588,"tokens_out":4747,"duration_ms":48092,"significance":"If the mechanism were fully specified and the empirical results were reported with uncertainty quantification, DMAC would be a useful contribution: it is a training-time defense that can be combined with any learnable communication policy and adds no inference-time overhead. The adversary formulation as a MARL problem with an IGM-constrained critic is coherent, and evaluating under both learned and heuristic attacks is a sensible protocol. The authors state that source code is released, which supports reproducibility. However, the central retraining step for the communication policy is not described anywhere in the manuscript, and the empirical evidence consists of single-point win rates without seeds, error bars, or statistical tests. As a result, the strength of the claimed robustness gains cannot currently be assessed.","major_comments":[{"comment":"The central claim that DMAC retrains the communication policy is not supported by the presented algorithm. Algorithm 1 takes the already-trained target policy π as input and updates only the masking policy πc and the critic C (line 14), while line 11 executes the fixed policy π on masked observations. There is no update rule, loss function, alternation schedule, or set of hyperparameters for retraining CP/π anywhere in Section 3.2. The sentence stating that 'the communication policy is retrained in the process of adversarial training' is an assertion without a corresponding equation or pseudocode, and the referenced appendix is not included in the manuscript. Because a fixed policy evaluated under masks cannot change its behavior, the robustness gains in Tables 1 and 2 have no mechanism in the paper as written. Please provide the complete retraining loop for CP, including the objective, the gradient update, how adversarial samples are incorporated, and how the adversary and CP interact over training iterations.","section":"§3.2, Algorithm 1"},{"comment":"The adversary's training objective uses binary channel masking, as defined in Eq. (1), while the evaluation attacks in Section 4.1 are message perturbations: the learned adaptive attack and the heuristic attack perturb or replace message content rather than masking whole channels. If masking-based adversarial training is supposed to confer robustness against perturbation attacks, the paper needs an explicit transfer argument or a control experiment, such as training with randomly masked channels or evaluating under masking attacks as well. Without such evidence, the reported robustness gains under perturbation attacks may not be attributable to the masking-based training procedure described in Section 3.2.","section":"§3.2 Eq. (1) and §4.1"},{"comment":"Tables 1 and 2 report a single win-rate value per condition with no number of seeds, no standard deviation or confidence interval, and no statistical test. The headline improvements (47.9%–81.9% for T2MAC and 54.4%–99.0% for I2C under the learned adaptive attack) are therefore not statistically established. Please report mean ± standard deviation over at least five independent runs and include pairwise significance tests or effect sizes for the comparisons against AME and R-MACRL.","section":"§4.2, Tables 1 and 2"},{"comment":"Table 3 reports communication-frequency statistics only for SC and PP, although the decentralization evaluation in Section 4.2 is presented as a general result for the adjusted communication policy. The claim that DMAC achieves a decentralized communication structure across all four tasks requires supporting numbers for CN and TJ as well, or an explicit statement that only two environments were analyzed in this part of the evaluation.","section":"§4.2, Table 3"}],"minor_comments":[{"comment":"The manuscript repeatedly refers to an appendix ('More results can be found in the appendix', including ablation studies and additional experimental details), but the arXiv submission contains no appendix. Include the appendix or remove the references to it.","section":"General"},{"comment":"Several references are duplicated: Das et al. 2019 appears as [Das et al., 2019a] and [Das et al., 2019b], Sukhbaatar et al. 2016 appears as [Sukhbaatar et al., 2016a] and [Sukhbaatar et al., 2016b], and the Mitchell et al. 2020 citation is repeated in Section 2. Please consolidate these entries.","section":"References"},{"comment":"Equation (3) uses the reciprocal of the edge weight w(u,v) after w is defined as the distance between two agents. Please clarify whether this is inverse-distance weighting, how the normalization is performed, and why distance alone is sufficient to capture interaction strength for identifying critical communication channels; an ablation or justification would be helpful.","section":"§3.2, Eq. (3)"},{"comment":"Equation (2) is ambiguous: the objective says it maximizes r̂, but the expression is written as the reciprocal of a weighted sum, with ξ preventing division by zero. Please state the exact transformation and the roles of w1, w2, and ξ, and provide their values in the experimental setup.","section":"§3.2, Eq. (2)"},{"comment":"The number of masking agents N = n(n−1)/2 is defined in Section 3.2 but never used explicitly in Algorithm 1. Please clarify whether each masking agent is responsible for exactly one channel and how the pairing (i,j) is assigned in the loop over training batches.","section":"Algorithm 1"}],"recommendation":"major_revision","confidential_remarks":"The main barrier is the missing specification of the communication-policy retraining step; this is fixable in a revision if the authors can provide the actual update. The empirical claims also need proper seed reporting and statistical testing. I do not see grounds for rejection if these points are addressed, but as written the central mechanism is absent from the algorithmic description."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper's central idea is attractive: train a masking adversary to identify critical communication channels, then adversarially retrain the communication policy to become less reliant on them, yielding a decentralized and more robust structure. That recipe is new in this combination, and the design of the adversary, with its dual reward and graph-based features, is coherent. The reported improvements over AME and R-MACRL are large and consistent across four environments, so something substantive is likely happening. But the manuscript as submitted has a load-bearing gap: the retraining of the communication policy is never actually specified. Algorithm 1, which is the only training loop given, updates only the masking policy and critic. It takes the target policy π as fixed input, executes it under masks, and returns the adversary's policy. There is no update rule for CP, no adversarial training objective, no alternation schedule, no hyperparameters for the retraining step. The sentence in Section 3.2 saying 'the communication policy is retrained in the process of adversarial training' is an assertion without any supporting pseudocode or equation, and the referenced appendix is not present. A fixed policy evaluated under masks cannot become more robust from this procedure, so Table 1's gains currently have no explained mechanism. That is the main problem, and it is not a matter of style. The secondary issues are also real but minor by comparison: single-point results with no error bars or seed counts, no statistical tests, and no random-masking or dropout control to show that the adversary's targeted masking, rather than any perturbation, drives the robustness. The evaluation attacks perturb messages while the training adversary masks channels, so the transfer needs explicit evidence. The decentralization measurements are a nice addition, but they show correlation, not the claimed causal path. Who should read this? Researchers working on robust communication in MARL would find the adversary formulation useful, and the empirical data is suggestive enough to warrant a closer look. But the paper is not ready as is. I would send it to peer review, because the idea is timely and the empirical claims are substantial, but I would make the authors specify the retraining algorithm, add seeds and confidence intervals, and include a random-masking baseline. If the code is available, a referee can verify, but the version I saw cannot be evaluated as a complete method.","headline":"The adversary is well-specified, but the paper's core claim that DMAC retrains the communication policy has no corresponding algorithm or equation, so the reported robustness gains lack a mechanism as submitted.","tokens_in":14131,"tokens_out":2417,"would_cite":false,"duration_ms":27395,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Retraining a communication policy against an adversary that masks its critical channels makes multi-agent teams markedly more robust against attacks, and slightly more performant when no attack occurs.","keywords":["multi-agent reinforcement learning","communication policy","adversarial training","robustness","decentralization","critical channel identification","channel masking","cooperative multi-agent systems"],"falsifier":"Run DMAC exactly as specified, then attack the final policy with a continuous adversarial perturbation that alters message values while leaving all channels open; if the post-attack win rate falls back to the unprotected baseline level, the claimed robustness mechanism would be disproved.","tokens_in":13121,"feed_emoji":"🛡️","tokens_out":9176,"duration_ms":86906,"temperature":0.7,"pith_summary":"This paper proposes DMAC, a training-time method for making learned communication policies in multi-agent reinforcement learning harder to break. The central claim is that a communication policy retrained against an adversary which dynamically identifies and masks its most-used channels will spread its communication across more channels, become more decentralized, and therefore survive attacks that previously crippled it. Across two communication policies (T2MAC and I2C) and four tasks, DMAC raises post-attack win rates by 47.9% to 81.9% for T2MAC and by 54.4% to 99.0% for I2C relative to the AME and R-MACRL baselines under a learned adaptive attack, with comparable gains under a heuristic attack. Under normal conditions the retrained policies are not worse and are usually slightly better, and the standard deviation of channel communication frequencies drops, which the paper treats as the signature of decentralization. If the claim holds, DMAC is a portable training-time defense that adds no extra computation at inference time.","feed_headline":"Channel-masking adversary retraining boosts multi-agent resilience","feed_subtitle":"Training-time defense spreads messages across channels, so knocking out one link no longer collapses the team.","key_machinery":"The load-bearing object is DMAC_Adv, an adversary that assigns one masking agent to every unordered pair of target agents. At each time step it chooses $a^c_{i,j} \\in \\{0,1\\}$, and the masked observation is $\\hat{o}_{i,j} = o_{i,j} \\odot a^c_{i,j}$, where a masked channel delivers null to both endpoints. The adversary's reward is $\\hat{r} = 1/(\\sum_{t=0}^T \\gamma^t (w_1 R(s^t,a^t,s^{t+1}) + w_2 R_m(a^{ct})) + \\xi)$, so maximizing it means both hurting the team's reward and masking as few channels as possible; this pressures the adversary to find the smallest set of channels whose loss is most damaging. The adversary's policy network consumes each agent's observation concatenated with a graph embedding $e_v$ built by one round of distance-weighted neighbor aggregation, $e^{k+1}_v \\leftarrow \\mathrm{Sum}(e^k_v, \\mathrm{Mean}(\\frac{1}{w(u,v)} e^k_u), u \\in N(v))$. The masking policy is trained with a value-decomposed critic that enforces the Individual-Global-Max property, so the joint optimum equals per-channel optimality, and the masks it produces are used as adversarial training samples to retrain the communication policy while the target agents' decision policy stays fixed.","core_discovery":"The paper's claim is that robustness of a learned communication policy can be improved not by filtering or repairing attacked messages at inference time, but by deliberately attacking the policy during training. An adversary trained with a dual-objective reward, maximizing the drop in team reward while minimizing the number of masked channels, generates adversarial samples that close the currently critical channels. Retraining the communication policy on these samples forces it to fall back on alternative channels, shifting its usage pattern from a concentrated, hub-like structure to a decentralized one. The paper reports that this shift is what makes the policy resilient: under learned adaptive and heuristic message attacks, the win rate of policies trained with DMAC roughly doubles compared with the AME and R-MACRL defenses, while the standard deviation of per-channel communication frequency drops by about a third to a half. DMAC is framed as a portable wrapper: the target agents' decision policy stays fixed during adversarial training, and only the communication policy is retrained, so the method can be attached to any learnable communication policy without changing the deployed decision model.","pith_inferences":["An implication the paper leaves implicit is that the same recipe could apply to communication policies with continuous-valued or directional channels, not just binary links, by training the adversary to select message dimensions to corrupt.","A testable extension is to measure decentralization directly with graph-theoretic indices such as effective resistance or spectral gap rather than only the standard deviation of channel frequencies; if decentralization is the true causal mechanism, those indices should track the robustness gain.","The paper's protocol trains the learned adaptive attack on the victim policy, but an adaptive adversary that re-optimizes after each DMAC update is not fully explored, so whether the defense survives repeated attacker re-optimization remains open.","Because the adversary is rewarded for masking as few channels as possible, DMAC implicitly assumes the target policy has a small set of critical channels to find; in regimes where all channels matter equally, the training signal may be much weaker."],"forward_implications":["DMAC improves post-attack win rates substantially for both tested communication policies; for example, T2MAC rises from 27.8% to 60.4% on SMAC under the learned adaptive attack, so teams hardened this way no longer collapse when one critical channel is damaged.","Because DMAC is a training wrapper over the communication policy and leaves the target agents' decision policy fixed, the defense adds no inference-time computation and can be paired with any learnable communication policy.","Under normal conditions the retrained policy remains at least as good as the original policy, with T2MAC on SMAC going from 81.2% to 83.7% win rate, so robustness is not bought by sacrificing ordinary performance.","The reduced standard deviation of per-channel communication frequency indicates the retrained policy spreads messages more evenly, so attacking the previously dominant channel no longer removes most of the team's information flow."],"supporting_citations":[{"why":"This is the T2MAC policy, one of the two communication policies whose robustness DMAC retrains and evaluates.","marker":"[Sun et al., 2024]"},{"why":"This is the I2C policy, the second communication policy whose robustness DMAC retrains and evaluates.","marker":"[Ding et al., 2020]"},{"why":"This is the AME defense, the baseline whose robustness numbers DMAC must beat in the comparison tables.","marker":"[Sun et al., 2023]"},{"why":"This is the R-MACRL defense, the second baseline compared in the robustness and performance tables.","marker":"[Xue et al., 2022]"},{"why":"This supplies the decentralization-theory premise that centralized networks collapse when key nodes fail, motivating the whole approach.","marker":"[Boss and Gralla, 2023]"},{"why":"This provides the SMAC environment, one of the four tasks used for the robustness and performance evaluations.","marker":"[Samvelyan et al., 2019]"},{"why":"This provides the Multi-Agent Particle Environments used for the Cooperative Navigation and Predator Prey tasks.","marker":"[Lowe et al., 2017]"},{"why":"This provides the Traffic Junction environment used as the fourth task.","marker":"[Singh et al., 2019]"}],"fun_headline_variants":["Adversarial channel masking forces robust decentralized multi-agent communication","Train-time attacks make multi-agent communication resilient to link loss","Decentralizing communication via adversarial training boosts multi-agent robustness","Mask critical channels in training to harden multi-agent communication","Adversarial training spreads messages across channels for robust MARL"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The argument rests on the assumption that robustness against binary channel masking, the only attack form used during adversarial training, transfers to the message-perturbation attacks used in evaluation; if that transfer fails, the reported gains would not be explained by the training method.","fun_headline_variants_meta":{"raw":{"variants":["Adversarial channel masking forces robust decentralized multi-agent communication","Train-time attacks make multi-agent communication resilient to link loss","Decentralizing communication via adversarial training boosts multi-agent robustness","Mask critical channels in training to harden multi-agent communication","Adversarial training spreads messages across channels for robust MARL"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000568,"raw_usage":{"total_tokens":2706,"prompt_tokens":982,"completion_tokens":1724,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":598,"completion_tokens_details":{"reasoning_tokens":1642}},"tokens_in":598,"tokens_out":1724,"duration_ms":12756,"temperature":1.0,"reasoning_tokens":1642,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T05:08:13.023468+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run DMAC exactly as specified, then attack the final policy with a continuous adversarial perturbation that alters message values while leaving all channels open; if the post-attack win rate falls back to the unprotected baseline level, the claimed robustness mechanism would be disproved.","supporting_citations":[{"cited_title":"T2MAC: targeted and trusted multi-agent communication through selective engagement and evidence-driven inte- gration","cited_arxiv_id":null,"evidence_quote":"This is the T2MAC policy, one of the two communication policies whose robustness DMAC retrains and evaluates."},{"cited_title":"Learning individually inferred communication for multi-agent cooperation","cited_arxiv_id":null,"evidence_quote":"This is the I2C policy, the second communication policy whose robustness DMAC retrains and evaluates."},{"cited_title":"Certifiably robust policy learning against adversarial multi-agent communication","cited_arxiv_id":null,"evidence_quote":"This is the AME defense, the baseline whose robustness numbers DMAC must beat in the comparison tables."},{"cited_title":"Mis- spoke or mis-lead: Achieving robustness in multi-agent communicative reinforcement learning","cited_arxiv_id":null,"evidence_quote":"This is the R-MACRL defense, the second baseline compared in the robustness and performance tables."},{"cited_title":"Robustness of decentralized decision-making architectures in command and control systems","cited_arxiv_id":null,"evidence_quote":"This supplies the decentralization-theory premise that centralized networks collapse when key nodes fail, motivating the whole approach."},{"cited_title":"Multi-agent actor- critic for mixed cooperative-competitive environments","cited_arxiv_id":null,"evidence_quote":"This provides the Multi-Agent Particle Environments used for the Cooperative Navigation and Predator Prey tasks."},{"cited_title":"Learning when to communicate at scale in multiagent cooperative and competitive tasks","cited_arxiv_id":null,"evidence_quote":"This provides the Traffic Junction environment used as the fourth task."}],"review_version":1}