{"id":"5825e5f0-b454-42ca-bc1e-1f2508ebf50d","arxiv_id":"2412.00534","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A fault-tolerant MARL method using attention in actor and critic networks plus per-module prioritized experience replay improves team performance when agents suddenly fail.","lead":"This paper proposes AACFT, a MARL method that adds attention modules to actor and critic networks and uses per-module prioritized experience replay so teams keep performing when agents suddenly fail. The authors report improved task completion in several simulated multi-agent scenarios and open-source a platform for fault-tolerant MARL research.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The claim that attention 'automatically identifies unexpected faults' is unsupported: fault detection is performed by an external oracle flag F_j that injects a hand-crafted z·1, and no value or sensitivity analysis for z is provided.","rationale":"I read the paper in good faith. The empirical scaffolding is coherent: AACFT combines known components (attention and PER) in a sensible way, ablations isolate their contributions, and the visualizations suggest the attention does reweight information in the intended direction in the shown episode. However, the central claim that the method 'automatically identify[ies] unexpected faults' is not supported by the design. Section IV-B and IV-C explicitly condition the special flag z·1 on the ground-truth fault indicator F_j, which is an oracle signal unavailable in genuinely unexpected fault scenarios. The attention mechanism merely learns to reweight inputs after being told which agent is faulty through this flag. The paper never reports z, despite acknowledging that too large a z disrupts training, and it provides no sensitivity analysis or seed-averaged attention statistics. The reader's weakest assumption correctly targets the flag encoding; I am sharpening it into a claim-level concern about the 'automatic' property. A no-flag ablation and a z sweep would directly test whether the mechanism works without the oracle and whether the reported gains are robust. If those tests fail, the method's fault tolerance advantage is conditional on an unreported and possibly environment-specific hyperparameter, so the paper should be revised to frame the contribution as fault-tolerance given a provided fault indicator, not automatic fault identification. The reader's CONDITIONAL verdict remains appropriate; I would not change it.","tokens_in":23894,"tokens_out":6371,"duration_ms":89152,"concrete_test":"Retrain AACFT and the strongest baseline (MADDPG+MC) on the Recovery scenario with (i) the flag z removed entirely (faulty observations left at their raw post-fault values or zeroed), and (ii) z swept over {1, 10, 50, 100, 500, 1000} with the reported setting included as control, using at least 10 independent seeds per condition and reporting mean plus standard error of task completion rate. If AACFT's advantage over baselines collapses without the flag, or fluctuates by more than, say, 10 percentage points across the z sweep, the claimed automatic fault identification is not achieved and the headline result depends on the unreported oracle flag value.","verdict_should_be":"UNCHANGED","load_bearing_attack":"In Section IV-B, the critic input for a faulty agent j is set to o_j = z·1 and a_j = 0 'when F_j = 0,' where F_j is a Boolean fault status supplied by the environment. Section IV-C applies the same z·1 to the observations of the faulty agent in other agents' actor inputs. Thus the network does not automatically detect a fault; it is handed a perfect, noise-free fault indicator and a specially encoded input. If F_j is unavailable or imperfect in a real 'unexpected fault' setting, the distinguishing input feature disappears and the attention modules have nothing to key on. Moreover, z is a method-critical hyperparameter: the paper states its absolute value 'should be much larger than the normal values' yet never reports it, and it acknowledges that an excessive z disrupts training. No sensitivity analysis is given, so we cannot tell whether the reported improvements in Figs. 6 and 7 are robust to z or specific to an unreported value. Finally, the attention visualizations in Fig. 9 come from a single episode and do not establish that the learned weights reliably downweight faulty agents across seeds. Since the entire mechanism of 'automatically identifying' faults and reweighting them reduces to the hand-crafted flag, this is the most load-bearing assumption in the paper.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes AACFT, a MADDPG extension that adds attention modules to both the critic and actor networks and applies per-module prioritized experience replay (PER) to improve fault tolerance in cooperative multi-agent reinforcement learning. In the critic, the observation and action of a faulty agent are manually replaced by a special constant vector z·1 and zero, respectively; in the actor, the observation of a faulty agent is replaced either by its fault-time state or by z·1. The attention modules are intended to learn to downweight these abnormal inputs. Experiments on four MPE-based particle scenarios compare AACFT against MADDPG, M3DDPG, and MADDPG+MC using four training sessions per method, with additional ablation studies and an open-source platform for fault-tolerant MARL.","tokens_in":24164,"tokens_out":4397,"duration_ms":42628,"significance":"If the empirical results hold, the paper addresses a relatively underexplored problem—agent faults in MARL—with a practical architecture and a reusable open-source platform. The comparison against a robust MARL baseline (M3DDPG) and a manual multiple-critic baseline (MADDPG+MC) is useful, and the ablation studies for both attention modules and PER give some evidence for the design choices. However, the headline claim that AACFT 'automatically identifies unexpected faults' is not supported by the method as described, because fault detection is entirely delegated to an external Boolean flag F_j and a hand-crafted input encoding z·1. The missing sensitivity analysis for the critical parameter z and the absence of error bars in Table I further limit the strength of the empirical claims.","major_comments":[{"comment":"The paper repeatedly claims that AACFT 'automatically identifies unexpected faults' (abstract), that the attention mechanism 'automatically detect[s] faults' (Section IV), and that the algorithm 'automatically identify faults' (conclusion). In fact, fault detection is provided by an external Boolean flag F_j, and when F_j=0 the observation and action are manually overwritten with z·1 and 0 in the critic (Section IV-B) and similarly in the actor (Section IV-C). The attention modules re-weight inputs only after this hand-crafted encoding has been injected; there is no learned fault-detection mechanism. If F_j is unavailable or noisy, the distinctive input feature disappears and the method has nothing to key on. Please revise the claims to state that AACFT adapts to a provided fault flag, or add experiments where the flag is not perfectly known.","section":"Abstract; Section IV-B; Section IV-C"},{"comment":"The special flag z is a method-critical hyperparameter, but no value is reported and no sensitivity analysis is provided. The text only states that its absolute value 'should be much larger than the normal values' and acknowledges that an excessive value 'significantly disrupts critic network training.' Without knowing the chosen z and without sensitivity curves over a range of z (and ideally over the PER exponents α and β and the replay period K), readers cannot determine whether the improvements shown in Figs. 6 and 7 are robust or specific to an unreported setting. Please report the chosen z and evaluate at least several values around it.","section":"Section IV-B; Figs. 6-7; Table I"},{"comment":"Table I reports task completion rates at different fault times but gives no error bars, confidence intervals, or number of evaluation episodes per cell, despite the rest of the paper using four training sessions. The differences among 0.770, 0.789, 0.798, and 0.808 are small, and without variance information it is unclear whether these differences are meaningful or whether the model is stable across seeds. This is load-bearing for the claim that AACFT is adaptable to fault timing; please add variance measures or a statistical comparison.","section":"Table I"}],"minor_comments":[{"comment":"The attention-distribution visualizations in Fig. 9 come from a single episode and are described qualitatively. Please state explicitly that these are illustrative examples, and if possible aggregate attention weights over multiple episodes or seeds to support the claim that the learned weights reliably downweight faulty agents.","section":"Section VI-B4; Fig. 9"},{"comment":"No hyperparameter table is provided. Please report batch size k, replay period K, learning rates, network sizes, PER exponents α and β, the specific value of z, and the fault probabilities used in each scenario either in the paper or in the open-source repository.","section":"Section VI-A"},{"comment":"The captions state '4 training sessions' but do not specify how the reward curves and boxplots are computed (e.g., per-episode reward, evaluation interval, number of evaluation episodes). Please clarify the evaluation protocol.","section":"Section VI-B2; Figs. 6-7"},{"comment":"There are several typos and duplicated references: 'Notablely' (Section III-B), 'martix' (Eq. 3), 'impertive' (Section IV), 'priorty' (Section VI-B5 heading), and reference [10] duplicates reference [9]. A careful proofread is needed.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The paper fits the journal's scope and the empirical core is plausible, but the advertised contribution of 'automatic fault detection' is contradicted by the method's reliance on an external F_j flag and a hand-crafted z·1 encoding. The missing z value and sensitivity analysis is the main technical gap; I would be willing to reconsider after a revision that tempers the claims and adds the requested experiments. The duplicate reference and mild self-citation are not concerns for me."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Bottom line: the empirical package is mostly solid and the ablations earn their keep, but the central \"automatically identifies faults\" claim is not supported—the method is handed a perfect fault indicator and an unreported flag magnitude z. That is a fixable overstatement, not a fatal flaw.\n\nWhat is new: attention in both actor and critic, with the actor token, combined with per-module prioritized replay, is a genuine combination; I have not seen that specific set in the fault-tolerant MARL literature. The paper does the right experimental legwork: four scenarios, comparisons against M3DDPG, MADDPG, and MADDPG+MC, separate ablations for the two attention modules and for PER, and fault-time adaptability. The PER analysis in the patrol scenario is the strongest part—vanilla AACFT cannot learn, PER makes it work, and the additional-sampling-rate plots show it shifts toward post-fault and late pre-fault transitions. The attention visualizations line up with the episode narratives.\n\nSoft spots, in rough order. First, the \"automatic detection\" claim. Section IV-B literally does: if F_j = 0, set o_j = z·1 and a_j = 0. F_j is a Boolean supplied by the environment. The network never detects a fault; it receives a perfect oracle and a hand-crafted input. Attention then learns to reweight, which may be useful, but calling that \"unexpected fault\" identification is an overstatement. The paper would be honest if it said \"given access to fault status.\" Second, z is method-critical and unreported. The paper says its absolute value should be much larger than normal values and that excessive z disrupts training. That is a red flag: a sensitive hyperparameter with no value and no sensitivity analysis leaves the reader unable to judge robustness. Third, Table I has no error bars and the attention figure is from a single episode, so the variance story is incomplete. Fourth, the GitHub link in the text has a space in it and looks broken. Minor.\n\nThe citation pattern is fine; ref 22 is background, not evidence for the main claim.\n\nWho this is for: researchers working on fault-tolerant MARL or on input encoding tricks for handling anomalous agents. It deserves a serious referee. The right revision makes the oracle assumption explicit, reports z and a sensitivity sweep, adds error bars, and softens the language from \"automatically identifies\" to \"uses fault-status indicators.\" I would accept it for review; the core combination is sound and the experiments are reproducible in principle, but the current framing needs correction.","headline":"Solid empirical combination with a fixable overclaim: the method relies on an oracle fault flag and an unreported flag magnitude, so the 'automatically identifies faults' claim is not supported as stated.","tokens_in":24700,"tokens_out":3444,"would_cite":false,"duration_ms":105322,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Attention and prioritized replay make multi-agent RL teams resilient to sudden agent failures.","keywords":["Multi-agent Reinforcement Learning","Fault Tolerance","Attention Mechanism","Prioritized Experience Replay","MADDPG","Actor-Critic","Sample Imbalance","Agent Faults"],"falsifier":"Train AACFT in one of the reported scenarios, such as the recovery scenario, across a range of fault-flag magnitudes $z$ from small to very large and record task completion; if completion collapses outside a narrow interval of $z$, or if attention weights do not concentrate away from the failed agent in the critic and onto or away from it in the actor as the task phase demands, the central claim of learned fault reweighting is not supported.","tokens_in":23662,"feed_emoji":"🛡️","tokens_out":5875,"duration_ms":55370,"temperature":0.7,"pith_summary":"The paper argues that a multi-agent reinforcement learning system can be made fault-tolerant by combining two mechanisms: attention modules in both the actor and critic networks, which learn to handle the corrupted inputs left by a failed agent, and per-module prioritized experience replay, which samples the transitions that matter most after a fault. If this is right, teams of learning agents can keep executing cooperative tasks at high success rates even when one or more members fail at unpredictable times, without hand-designing separate policies or replay buffers for the failure case. The method, called AACFT, is built on the MADDPG actor-critic framework and is validated in four simulated multi-agent scenarios with different fault effects. The core claim is that the attention reweighting and the prioritization together, not either alone, close the performance gap opened by faults.","feed_headline":"Fault-tolerant MARL achieved with attention and prioritized replay","feed_subtitle":"Attention redirects focus from failed agents while replay prioritizes post-fault transitions, lifting task completion.","key_machinery":"The load-bearing object is AACFT, an extension of the MADDPG actor-critic architecture. In the critic, an attention layer over agent embeddings computes weights that can suppress the embedding of a failed agent; in the actor, an attention layer over the pieces of the agent's own observation computes weights that can emphasize or suppress the failed teammate's state, using an extra token as the query. On top of this, the method extends prioritized experience replay by keeping one priority queue for the shared critics and one per actor, ranking transitions by loss and sampling with probability proportional to priority. The attention modules are what let a single network absorb a fault-induced distribution shift without retraining, and the priority queues are what keep post-fault transitions from being starved by the plentiful, similar pre-fault transitions.","core_discovery":"On the paper's own terms, the central discovery is that a single actor-critic model can tolerate unexpected agent faults if its inputs are explicitly flagged and its networks can reweight those flags. When an agent fails, AACFT sets the failed agent's observation to a large constant vector and its action to zero, making the anomaly visible; the critic's attention module learns to shift weight away from that embedding, while the actor's attention module learns to give the failed agent's frozen state high or low weight depending on the task phase. At the same time, separate priority queues for the critic and each actor sample transitions according to their losses, so that scarce post-fault experience is reused once pre-fault behavior is already learned. Experiments across abandonment, recovery, navigation, and patrol scenarios report task-completion rates between 0.77 and 0.85 for faults at different time steps, against 0.38 for a MADDPG baseline trained without faults.","pith_inferences":["A testable follow-up is to replace the hand-set fault flag $z\\cdot\\mathbf{1}$ with a learned fault indicator; if AACFT's gains disappear, the method is buying fault tolerance with a hand-crafted input convention rather than a learned mechanism.","The attention visualizations suggest the actor learns a two-phase policy in the recovery scenario—attend to the failed agent until recovery, then reallocate attention to the prey—so the method's reliability depends on attention discovering this task-dependent schedule instead of always suppressing faults.","Because priorities are kept per module, the scheme costs $N+1$ priority queues; for large agent counts the memory and update overhead may dominate, and an aggregated or shared priority design would be a natural extension.","The open-sourced fault controller could serve as a common testbed for fault tolerance, allowing future algorithms to be compared on standardized fault types, timings, and probabilities."],"forward_implications":["AACFT maintains task completion in the 0.77–0.85 range across fault times 5 through 25 and no-fault 0.841, while a MADDPG model trained without faults drops to 0.382 when agent 2 fails at time step 5.","Ablations show that removing attention from either the critic (AAFT) or the actor (ACFT) hurts performance, so both attention modules are load-bearing for the reported gains.","PER changes the sampled distribution: post-fault transitions are sampled nearly 30% more than uniform sampling late in training, and later pre-fault transitions are preferred over early ones.","AACFT beats the robust baseline M3DDPG and the manual multi-critic baseline MADDPG+MC in most scenarios, with the largest gains in task-redistribution cases such as the recovery scenario when the faulty agent is not the one that can recover."],"supporting_citations":[{"why":"Supplies the MADDPG actor-critic algorithm and the four MPE-based scenarios that AACFT modifies; it is the base method and the main baseline.","marker":"[15]"},{"why":"Prioritized Experience Replay; the method is extended to per-module priority queues for critic and actors.","marker":"[17]"},{"why":"Actor-attention-critic; provides the attention-mechanism design for the critic that attends over agent embeddings, which AACFT adapts for fault handling.","marker":"[42]"},{"why":"Attention for image classification; supplies the query-token design used to give the actor a single output token over observation embeddings.","marker":"[41]"},{"why":"M3DDPG; the robust-MARL baseline whose minimax robustness is compared against AACFT's explicit fault tolerance.","marker":"[24]"}],"fun_headline_variants":["Attention and replay make MARL resilient to agent failures","MARL copes with unexpected agent faults via attention and replay","Fault-tolerant MARL: attention redirects, replay rebalances","Handling random agent failures in MARL with attention and replay"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that marking a failed agent's observation with a single large hand-picked value (the same $z\\cdot\\mathbf{1}$ vector every time) and its action as zero will give the attention modules a signal they can learn to reweight correctly, even though the paper does not specify $z$ or test how sensitive the results are to it.","fun_headline_variants_meta":{"raw":{"variants":["Attention and replay make MARL resilient to agent failures","MARL copes with unexpected agent faults via attention and replay","Fault-tolerant MARL: attention redirects, replay rebalances","Handling random agent failures in MARL with attention and replay"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001042,"raw_usage":{"total_tokens":4359,"prompt_tokens":900,"completion_tokens":3459,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":516,"completion_tokens_details":{"reasoning_tokens":3389}},"tokens_in":516,"tokens_out":3459,"duration_ms":27718,"temperature":1.0,"reasoning_tokens":3389,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T05:16:07.223569+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train AACFT in one of the reported scenarios, such as the recovery scenario, across a range of fault-flag magnitudes $z$ from small to very large and record task completion; if completion collapses outside a narrow interval of $z$, or if attention weights do not concentrate away from the failed agent in the critic and onto or away from it in the actor as the task phase demands, the central claim of learned fault reweighting is not supported.","supporting_citations":[{"cited_title":"Multi-agent actor-critic for mixed cooperative-competitive environments,","cited_arxiv_id":null,"evidence_quote":"Supplies the MADDPG actor-critic algorithm and the four MPE-based scenarios that AACFT modifies; it is the base method and the main baseline."},{"cited_title":"Prioritized experience replay,","cited_arxiv_id":null,"evidence_quote":"Prioritized Experience Replay; the method is extended to per-module priority queues for critic and actors."},{"cited_title":"Actor-attention-critic for multi-agent reinforcement learning,","cited_arxiv_id":null,"evidence_quote":"Actor-attention-critic; provides the attention-mechanism design for the critic that attends over agent embeddings, which AACFT adapts for fault handling."},{"cited_title":"Robust multi- agent reinforcement learning via minimax deep deterministic policy gradient,","cited_arxiv_id":null,"evidence_quote":"M3DDPG; the robust-MARL baseline whose minimax robustness is compared against AACFT's explicit fault tolerance."}],"review_version":1}