Pith. sign in

REVIEW 3 major objections 4 minor 1 cited by

Towards Fault Tolerance in Multi-Agent Reinforcement Learning

T0 review · 3 major / 4 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Attention and prioritized replay make multi-agent RL teams resilient to sudden agent failures.

desk verdict Solid empirical combination with a fixable overclaim: the method relies on an oracle fault flag and an unreported flag magnitude, so the 'automatically identifies faults' claim is not supported as stated. read the letter →

arxiv 2412.00534 v1 pith:T5M6SY4S submitted 2024-11-30 cs.LG cs.AIcs.MA

Yuchen Shi , Huaxin Pei , Liang Feng , Yi Zhang , Danya Yao This is my paper · ORCID
classification cs.LGcs.AIcs.MA
keywords Multi-agentReinforcementLearningFaultToleranceAttentionMechanismPrioritizedExperienceReplayMADDPGActor-CriticSampleImbalanceAgentFaults
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that a multi-agent reinforcement learning system can be made fault-tolerant by combining two mechanisms: attention modules in both the actor and critic networks, which learn to handle the corrupted inputs left by a failed agent, and per-module prioritized experience replay, which samples the transitions that matter most after a fault. If this is right, teams of learning agents can keep executing cooperative tasks at high success rates even when one or more members fail at unpredictable times, without hand-designing separate policies or replay buffers for the failure case. The method, called AACFT, is built on the MADDPG actor-critic framework and is validated in four simulated multi-agent scenarios with different fault effects. The core claim is that the attention reweighting and the prioritization together, not either alone, close the performance gap opened by faults.

What carries the argument

The load-bearing object is AACFT, an extension of the MADDPG actor-critic architecture. In the critic, an attention layer over agent embeddings computes weights that can suppress the embedding of a failed agent; in the actor, an attention layer over the pieces of the agent's own observation computes weights that can emphasize or suppress the failed teammate's state, using an extra token as the query. On top of this, the method extends prioritized experience replay by keeping one priority queue for the shared critics and one per actor, ranking transitions by loss and sampling with probability proportional to priority. The attention modules are what let a single network absorb a fault-induced distribution shift without retraining, and the priority queues are what keep post-fault transitions from being starved by the plentiful, similar pre-fault transitions.

What would settle it

Train AACFT in one of the reported scenarios, such as the recovery scenario, across a range of fault-flag magnitudes $z$ from small to very large and record task completion; if completion collapses outside a narrow interval of $z$, or if attention weights do not concentrate away from the failed agent in the critic and onto or away from it in the actor as the task phase demands, the central claim of learned fault reweighting is not supported.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that a single actor-critic model can tolerate unexpected agent faults if its inputs are explicitly flagged and its networks can reweight those flags. When an agent fails, AACFT sets the failed agent's observation to a large constant vector and its action to zero, making the anomaly visible; the critic's attention module learns to shift weight away from that embedding, while the actor's attention module learns to give the failed agent's frozen state high or low weight depending on the task phase. At the same time, separate priority queues for the critic and each actor sample transitions according to their losses, so that scarce post-fault experience is reused once pre-fault behavior is already learned. Experiments across abandonment, recovery, navigation, and patrol scenarios report task-completion rates between 0.77 and 0.85 for faults at different time steps, against 0.38 for a MADDPG baseline trained without faults.

Load-bearing premise

The load-bearing premise is that marking a failed agent's observation with a single large hand-picked value (the same $z\cdot\mathbf{1}$ vector every time) and its action as zero will give the attention modules a signal they can learn to reweight correctly, even though the paper does not specify $z$ or test how sensitive the results are to it.

Editorial extensions

If this is right

  • AACFT maintains task completion in the 0.77–0.85 range across fault times 5 through 25 and no-fault 0.841, while a MADDPG model trained without faults drops to 0.382 when agent 2 fails at time step 5.
  • Ablations show that removing attention from either the critic (AAFT) or the actor (ACFT) hurts performance, so both attention modules are load-bearing for the reported gains.
  • PER changes the sampled distribution: post-fault transitions are sampled nearly 30% more than uniform sampling late in training, and later pre-fault transitions are preferred over early ones.
  • AACFT beats the robust baseline M3DDPG and the manual multi-critic baseline MADDPG+MC in most scenarios, with the largest gains in task-redistribution cases such as the recovery scenario when the faulty agent is not the one that can recover.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable follow-up is to replace the hand-set fault flag $z\cdot\mathbf{1}$ with a learned fault indicator; if AACFT's gains disappear, the method is buying fault tolerance with a hand-crafted input convention rather than a learned mechanism.
  • The attention visualizations suggest the actor learns a two-phase policy in the recovery scenario—attend to the failed agent until recovery, then reallocate attention to the prey—so the method's reliability depends on attention discovering this task-dependent schedule instead of always suppressing faults.
  • Because priorities are kept per module, the scheme costs $N+1$ priority queues; for large agent counts the memory and update overhead may dominate, and an aggregated or shared priority design would be a natural extension.
  • The open-sourced fault controller could serve as a common testbed for fault tolerance, allowing future algorithms to be compared on standardized fault types, timings, and probabilities.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes AACFT, a MADDPG extension that adds attention modules to both the critic and actor networks and applies per-module prioritized experience replay (PER) to improve fault tolerance in cooperative multi-agent reinforcement learning. In the critic, the observation and action of a faulty agent are manually replaced by a special constant vector z·1 and zero, respectively; in the actor, the observation of a faulty agent is replaced either by its fault-time state or by z·1. The attention modules are intended to learn to downweight these abnormal inputs. Experiments on four MPE-based particle scenarios compare AACFT against MADDPG, M3DDPG, and MADDPG+MC using four training sessions per method, with additional ablation studies and an open-source platform for fault-tolerant MARL.

Significance. If the empirical results hold, the paper addresses a relatively underexplored problem—agent faults in MARL—with a practical architecture and a reusable open-source platform. The comparison against a robust MARL baseline (M3DDPG) and a manual multiple-critic baseline (MADDPG+MC) is useful, and the ablation studies for both attention modules and PER give some evidence for the design choices. However, the headline claim that AACFT 'automatically identifies unexpected faults' is not supported by the method as described, because fault detection is entirely delegated to an external Boolean flag F_j and a hand-crafted input encoding z·1. The missing sensitivity analysis for the critical parameter z and the absence of error bars in Table I further limit the strength of the empirical claims.

major comments (3)
  1. [Abstract; Section IV-B; Section IV-C] The paper repeatedly claims that AACFT 'automatically identifies unexpected faults' (abstract), that the attention mechanism 'automatically detect[s] faults' (Section IV), and that the algorithm 'automatically identify faults' (conclusion). In fact, fault detection is provided by an external Boolean flag F_j, and when F_j=0 the observation and action are manually overwritten with z·1 and 0 in the critic (Section IV-B) and similarly in the actor (Section IV-C). The attention modules re-weight inputs only after this hand-crafted encoding has been injected; there is no learned fault-detection mechanism. If F_j is unavailable or noisy, the distinctive input feature disappears and the method has nothing to key on. Please revise the claims to state that AACFT adapts to a provided fault flag, or add experiments where the flag is not perfectly known.
  2. [Section IV-B; Figs. 6-7; Table I] The special flag z is a method-critical hyperparameter, but no value is reported and no sensitivity analysis is provided. The text only states that its absolute value 'should be much larger than the normal values' and acknowledges that an excessive value 'significantly disrupts critic network training.' Without knowing the chosen z and without sensitivity curves over a range of z (and ideally over the PER exponents α and β and the replay period K), readers cannot determine whether the improvements shown in Figs. 6 and 7 are robust or specific to an unreported setting. Please report the chosen z and evaluate at least several values around it.
  3. [Table I] Table I reports task completion rates at different fault times but gives no error bars, confidence intervals, or number of evaluation episodes per cell, despite the rest of the paper using four training sessions. The differences among 0.770, 0.789, 0.798, and 0.808 are small, and without variance information it is unclear whether these differences are meaningful or whether the model is stable across seeds. This is load-bearing for the claim that AACFT is adaptable to fault timing; please add variance measures or a statistical comparison.
minor comments (4)
  1. [Section VI-B4; Fig. 9] The attention-distribution visualizations in Fig. 9 come from a single episode and are described qualitatively. Please state explicitly that these are illustrative examples, and if possible aggregate attention weights over multiple episodes or seeds to support the claim that the learned weights reliably downweight faulty agents.
  2. [Section VI-A] No hyperparameter table is provided. Please report batch size k, replay period K, learning rates, network sizes, PER exponents α and β, the specific value of z, and the fault probabilities used in each scenario either in the paper or in the open-source repository.
  3. [Section VI-B2; Figs. 6-7] The captions state '4 training sessions' but do not specify how the reward curves and boxplots are computed (e.g., per-episode reward, evaluation interval, number of evaluation episodes). Please clarify the evaluation protocol.
  4. [Throughout] There are several typos and duplicated references: 'Notablely' (Section III-B), 'martix' (Eq. 3), 'impertive' (Section IV), 'priorty' (Section VI-B5 heading), and reference [10] duplicates reference [9]. A careful proofread is needed.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; the empirical comparisons are self-contained and the only self-citation is background.

full rationale

The paper presents an empirical MARL method (AACFT) that combines attention modules with per-module prioritized experience replay. It offers no analytic derivation, so the circularity test is whether any claimed result is forced by construction or by self-citation. It is not. The central comparisons (Figs. 6, 7, 10-12 and Table I) are against external baselines (MADDPG, M3DDPG, MADDPG+MC, and ablations) on MPE-derived scenarios, and the reported improvements are measured rather than fitted. The only reference to the authors' own prior work is [22] (Pei et al.), used as background on rule-based fault-tolerant driving; it does not supply any premise of AACFT and is not load-bearing. One wording overclaim should be noted as a correctness risk, not circularity: the abstract says attention 'automatically detect[s] faults', but Section IV-B injects a hand-crafted fault flag (oj = z·1, aj = 0 when Fj = 0) and Section IV-C likewise encodes oij = z·1; the network is handed a perfect fault indicator rather than discovering faults from raw observations. That undermines the generalizability claim if F_j is unavailable or imperfect, but it is not a case of the paper deriving a result from its own output, renaming a fit as a prediction, or relying on a self-citation chain. No load-bearing circular step is present.

Assumptions & free parameters 4 free parameters · 5 assumptions · 1 invented entities

The central claim is empirical, so the main burden falls on the unverified flag encoding z and the learned attention that compensates for it. Several PER hyperparameters and all network architecture details are unreported, and the fault model is a single total-failure mode tested only in toy scenarios.

free parameters (4)
  • special flag z = not reported
    In Section IV-B, faulty observations are set to o_j = z·1, and the text requires |z| to be much larger than normal values; the chosen magnitude and its sensitivity are never given, yet the whole fault-detection input design depends on it.
  • PER exponent alpha = not reported
    Eq. 10 uses alpha to control prioritization, with alpha = 0 as uniform sampling; the value used in the experiments is not stated.
  • PER importance-sampling exponent beta = not reported, annealed to 1
    Eq. 11 uses beta for unbiasedness correction; the initial value and schedule are not stated.
  • batch size k and replay update period K = not reported
    Algorithm 1 depends on these values, but the experimental setup in Section VI-A does not list them.
assumptions (5)
  • domain assumption The Dec-POMDP fault model p = F(s,t) adequately represents real agent faults.
    Section III-A defines the problem through this fault function, but only total loss of observation and action is evaluated in Section VI; partial or gradual faults are not tested.
  • ad hoc to paper Setting faulty observations to z·1 and actions to 0 improves identifiability without breaking training.
    This is the core input encoding in Section IV-B and IV-C, asserted rather than derived, with no sensitivity study.
  • domain assumption Attention modules in critic and actor learn to reweight faulty information appropriately.
    The method relies on learned weights in Eq. 6 and Eq. 8 compensating for the flag encoding; no convergence or robustness guarantee is provided.
  • domain assumption The scenario reward r = r_com + r_out + r_col + r_dis + r_goal is a suitable training signal.
    Section VI-A2 defines rewards without testing sensitivity to reward shaping.
  • standard math Softmax attention with query, key, and value projections (Eq. 3, 4) provides a valid differentiable mechanism for weighting inputs.
    Standard attention formulation used as background; not new.
invented entities (1)
  • actor token e_i0
    purpose: Extra learnable embedding in the actor that aggregates attention-weighted observation pieces and is decoded into the action.
    Introduced in Section IV-C and Fig. 2; the ACFT ablation indirectly tests its contribution, but the token itself is not independently validated outside the proposed architecture.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Towards Fault Tolerance in Multi-Agent Reinforcement Learning." pith.science (2026). https://pith.science/paper/T5M6SY4S

@misc{pith2026241200534,
  author       = {Pith},
  title        = {Pith review of: Towards Fault Tolerance in Multi-Agent Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/T5M6SY4S}},
  note         = {Machine review of arXiv:2412.00534}
}
read the original abstract

Agent faults pose a significant threat to the performance of multi-agent reinforcement learning (MARL) algorithms, introducing two key challenges. First, agents often struggle to extract critical information from the chaotic state space created by unexpected faults. Second, transitions recorded before and after faults in the replay buffer affect training unevenly, leading to a sample imbalance problem. To overcome these challenges, this paper enhances the fault tolerance of MARL by combining optimized model architecture with a tailored training data sampling strategy. Specifically, an attention mechanism is incorporated into the actor and critic networks to automatically detect faults and dynamically regulate the attention given to faulty agents. Additionally, a prioritization mechanism is introduced to selectively sample transitions critical to current training needs. To further support research in this area, we design and open-source a highly decoupled code platform for fault-tolerant MARL, aimed at improving the efficiency of studying related problems. Experimental results demonstrate the effectiveness of our method in handling various types of faults, faults occurring in any agent, and faults arising at random times.

Figures

Figures reproduced from arXiv: 2412.00534 by the authors.

Figure 1
Figure 1. (a) An illustration of a predator-prey system before and after the agent fault, where different shades of blue circles [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. An illustration of the main components of our method. Actor of agent outputs action [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. The overall framework of FTMAL. note that if the transition involves faulty agent i, the priority queue Qa,i does not add a new priority for this transition, to avoid affecting the priority ranking and prevent the transitions from being sampled when updating agent i’s actor. For Q ∈ {Qc, Qa,1, ..., Qa,N }, denote pi as the priority of transition i. pi = 1/rank(i) is rank-based priority of transition i. The rank of a… view at source ↗
Figures from the paper (10 more)
Figure 4
Figure 4. Figure 4: Schematic diagram of the scenarios before and after faults. (a) Abandonment scenario and recovery scenario if agent 2 [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: Testing of the basic MADDPG without considering [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: Reward curves from 4 training sessions for each method in 4 scenarios. (a) No-fault predator-prey scenario; (b) [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]
Figure 7
Figure 7. Figure 7: Boxplots of evaluations results from 4 training sessions for each method in 3 scenarios. (a) Abandonment scenario; (b) [PITH_FULL_IMAGE:figures/full_fig_p010_7.png]
Figure 8
Figure 8. Figure 8: Illustration of an episode in the recovery scenario, where agent 2 and agent 3 fail respectively. In the figure, a series of [PITH_FULL_IMAGE:figures/full_fig_p011_8.png]
Figure 9
Figure 9. Figure 9: Attention allocation on critic and actor before and after fault in the recovery scenario. The [PITH_FULL_IMAGE:figures/full_fig_p011_9.png]
Figure 10
Figure 10. Figure 10: Reward curves from 4 training sessions for each method in 2 scenarios. (a) Abandonment scenario; (b) Recovery [PITH_FULL_IMAGE:figures/full_fig_p012_10.png]
Figure 11
Figure 11. Figure 11: Boxplots of evaluations results from 4 training sessions for each method in 2 scenarios. (a) Abandonment scenario; (b) [PITH_FULL_IMAGE:figures/full_fig_p012_11.png]
Figure 12
Figure 12. Figure 12: Reward curves from 4 training sessions for AACFT [PITH_FULL_IMAGE:figures/full_fig_p012_12.png]
Figure 13
Figure 13. Figure 13: Additional sampling rate in batches for training the critic for AACFT with PER in patrol scenario. (a) Additional [PITH_FULL_IMAGE:figures/full_fig_p013_13.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Exploring Critical Testing Scenarios for Decision-Making Policies: An LLM Approach

    cs.LG 2024-12 conditional novelty 6.0 of 10

    An LLM-driven generate-test-feedback loop with a multi-scale mutation strategy finds more and more diverse failure scenarios for black-box decision-making policies than MDPFuzz and random testing.

Reference graph

Works this paper leans on

54 extracted references · 42 canonical work pages · cited by 1 Pith paper

  1. [1]

    Deep reinforcement learning for autonomous driving: A survey,

    B. R. Kiran, I. Sobh, V . Talpaert, P. Mannion, A. A. A. Sallab, S. Yogamani, and P. P´erez, “Deep reinforcement learning for autonomous driving: A survey,” IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 6, pp. 4909–4926, 2022

    work page 2022

  2. [2]

    Distributed multi-vehicle task assignment and motion planning in dense environments,

    G. Xu, X. Kang, H. Yang, Y . Wu, W. Liu, J. Cao, and Y . Liu, “Distributed multi-vehicle task assignment and motion planning in dense environments,” IEEE Transactions on Automation Science and Engineering, vol. 21, no. 4, pp. 7027–7039, 2024

    work page 2024

  3. [3]

    A survey on multi- agent reinforcement learning applications in the internet of vehicles,

    E. M. Mianji, M. Fardad, G.-M. Muntean, and I. Tal, “A survey on multi- agent reinforcement learning applications in the internet of vehicles,” in 2024 IEEE 99th Vehicular Technology Conference (VTC2024-Spring) , 2024, pp. 1–7

    work page 2024

  4. [4]

    Theory and experiment on formation-containment control of multiple multirotor unmanned aerial vehicle systems,

    X. Dong, Y . Hua, Y . Zhou, Z. Ren, and Y . Zhong, “Theory and experiment on formation-containment control of multiple multirotor unmanned aerial vehicle systems,” IEEE Transactions on Automation Science and Engineering, vol. 16, no. 1, pp. 229–240, 2019

    work page 2019

  5. [5]

    Cooperative internet of uavs: Distributed trajectory design by multi-agent deep reinforcement learning,

    J. Hu, H. Zhang, L. Song, R. Schober, and H. V . Poor, “Cooperative internet of uavs: Distributed trajectory design by multi-agent deep reinforcement learning,” IEEE Transactions on Communications , vol. 68, no. 11, pp. 6807–6821, 2020

    work page 2020

  6. [6]

    Heterogeneous multi-robot reinforcement learning,

    M. Bettini, A. Shankar, and A. Prorok, “Heterogeneous multi-robot reinforcement learning,” in Proceedings of the 2023 International Conference on Autonomous Agents and Multiagent Systems , 2023, pp. 1485–1494

    2023

  7. [7]

    On the effects of communication failures in a multi-agent consensus network,

    M. E. Valcher and G. Parlangeli, “On the effects of communication failures in a multi-agent consensus network,” in 2019 23rd International Conference on System Theory, Control and Computing (ICSTCC) , 2019, pp. 709–720

    work page 2019

  8. [8]

    The impact of agent definitions and interactions on multiagent learning for coordination in traffic management domains,

    J. J. Chung, D. Mikli ´c, L. Sabattini, K. Tumer, and R. Siegwart, “The impact of agent definitions and interactions on multiagent learning for coordination in traffic management domains,” Autonomous Agents and Multi-Agent Systems, vol. 34, pp. 1–27, 2020

    work page 2020

Show all 54 references
  1. [9]

    Fault-tolerant cooperative control of multiagent systems: A survey of trends and methodologies,

    H. Yang, Q.-L. Han, X. Ge, L. Ding, Y . Xu, B. Jiang, and D. Zhou, “Fault-tolerant cooperative control of multiagent systems: A survey of trends and methodologies,” IEEE Transactions on Industrial Informatics , vol. 16, no. 1, pp. 4–17, 2020

    2020

  2. [10]

    Fault-tolerant cooperative control of multiagent systems: A survey of trends and methodologies,

    ——, “Fault-tolerant cooperative control of multiagent systems: A survey of trends and methodologies,” IEEE Transactions on Industrial Informatics, vol. 16, no. 1, pp. 4–17, 2020

    2020

  3. [11]

    Fault-tolerant consensus of leader–following multi-agent systems with jointly connected topologies,

    Y . Li, F. Wang, M. Sader, Z. Liu, and Z. Chen, “Fault-tolerant consensus of leader–following multi-agent systems with jointly connected topologies,” Transactions of the Institute of Measurement and Control , vol. 45, no. 9, pp. 1747–1754, 2023

    2023

  4. [12]

    Fault-tolerant formation for multi-uav via improved artificial potential field method,

    W. Liu, W. Liu, X. Lv, Z. Mao, and B. Jiang, “Fault-tolerant formation for multi-uav via improved artificial potential field method,” in 2021 CAA Symposium on Fault Detection, Supervision, and Safety for Technical Processes (SAFEPROCESS), 2021, pp. 1–6

    2021

  5. [13]

    Policy gradient methods for reinforcement learning with function approximation,

    R. S. Sutton, D. McAllester, S. Singh, and Y . Mansour, “Policy gradient methods for reinforcement learning with function approximation,” Advances in neural information processing systems , vol. 12, 1999

    1999

  6. [14]

    Multi-agent reinforcement learning with decentral- ized distribution correction,

    K. Li and Q.-S. Jia, “Multi-agent reinforcement learning with decentral- ized distribution correction,” IEEE Transactions on Automation Science and Engineering, pp. 1–13, 2024. 14

    2024

  7. [15]

    Multi-agent actor-critic for mixed cooperative-competitive environments,

    R. Lowe, Y . I. Wu, A. Tamar, J. Harb, O. Pieter Abbeel, and I. Mordatch, “Multi-agent actor-critic for mixed cooperative-competitive environments,” Advances in neural information processing systems , vol. 30, 2017

    2017

  8. [16]

    The surprising effectiveness of ppo in cooperative multi-agent games,

    C. Yu, A. Velu, E. Vinitsky, J. Gao, Y . Wang, A. Bayen, and Y . WU, “The surprising effectiveness of ppo in cooperative multi-agent games,” in Advances in Neural Information Processing Systems , vol. 35, 2022, pp. 24 611–24 624

    2022

  9. [17]

    Prioritized experience replay,

    T. Schaul, “Prioritized experience replay,” 4rd International Conference on Learning Representations, ICLR 2016 , 2016

    2016

  10. [18]

    Towards a fault-tolerant multi-agent system architecture,

    S. Kumar and P. R. Cohen, “Towards a fault-tolerant multi-agent system architecture,” in Proceedings of the fourth international conference on Autonomous agents, 2000, pp. 459–466

    2000

  11. [19]

    A survey on fault tolerant multi agent system,

    Y . Arfat and F. E. Eassa, “A survey on fault tolerant multi agent system,” IJ Inf. Technol. Comput. Sci , vol. 9, pp. 39–48, 2016

    2016

  12. [20]

    Adaptive fault-tolerant boundary control of an autonomous aerial refueling hose system with prescribed constraints,

    Z. Liu, Z. Han, and W. He, “Adaptive fault-tolerant boundary control of an autonomous aerial refueling hose system with prescribed constraints,” IEEE Transactions on Automation Science and Engineering , vol. 19, no. 4, pp. 2678–2688, 2022

    2022

  13. [21]

    A goa-based fault-tolerant trajectory tracking control for an underwater vehicle of multi-thruster system without actuator saturation,

    D. Zhu, L. Wang, H. Zhang, and S. X. Yang, “A goa-based fault-tolerant trajectory tracking control for an underwater vehicle of multi-thruster system without actuator saturation,” IEEE Transactions on Automation Science and Engineering , vol. 21, no. 1, pp. 771–782, 2024

    2024

  14. [22]

    Fault-tolerant cooperative driving at signal-free intersections,

    H. Pei, J. Zhang, Y . Zhang, X. Pei, S. Feng, and L. Li, “Fault-tolerant cooperative driving at signal-free intersections,” IEEE Transactions on Intelligent Vehicles, vol. 8, no. 1, pp. 121–134, 2022

    2022

  15. [23]

    Fault-tolerant cooperative control design of multiple wheeled mobile robots,

    M. A. Kamel, X. Yu, and Y . Zhang, “Fault-tolerant cooperative control design of multiple wheeled mobile robots,” IEEE Transactions on Control Systems Technology, vol. 26, no. 2, pp. 756–764, 2018

    2018

  16. [24]

    Robust multi- agent reinforcement learning via minimax deep deterministic policy gradient,

    S. Li, Y . Wu, X. Cui, H. Dong, F. Fang, and S. Russell, “Robust multi- agent reinforcement learning via minimax deep deterministic policy gradient,” in Proceedings of the AAAI conference on artificial intelligence, vol. 33, no. 01, 2019, pp. 4213–4220

    2019

  17. [25]

    Byzantine robust cooperative multi-agent reinforcement learning as a bayesian game,

    S. Li, J. Guo, J. Xiu, R. Xu, X. Yu, J. Wang, A. Liu, Y . Yang, and X. Liu, “Byzantine robust cooperative multi-agent reinforcement learning as a bayesian game,” arXiv preprint arXiv:2305.12872 , 2023

    2023 arXiv

  18. [26]

    QMIX: Monotonic value function factorisation for deep multi-agent reinforcement learning,

    T. Rashid, M. Samvelyan, C. Schroeder, G. Farquhar, J. Foerster, and S. Whiteson, “QMIX: Monotonic value function factorisation for deep multi-agent reinforcement learning,” in Proceedings of the 35th International Conference on Machine Learning , ser. Proceedings of Machine L...

    2018

  19. [27]

    Counterfactual multi-agent policy gradients,

    J. Foerster, G. Farquhar, T. Afouras, N. Nardelli, and S. Whiteson, “Counterfactual multi-agent policy gradients,” in Proceedings of the AAAI conference on artificial intelligence , vol. 32, no. 1, 2018

    2018

  20. [28]

    Evolutionary population curriculum for scaling multi-agent reinforcement learning,

    Q. Long, Z. Zhou, A. Gupta, F. Fang, Y . Wu, and X. Wang, “Evolutionary population curriculum for scaling multi-agent reinforcement learning,” in International Conference on Learning Representations , 2019

    2019

  21. [29]

    Scalable autonomous separation assurance with heterogeneous multi-agent reinforcement learning,

    M. Brittain and P. Wei, “Scalable autonomous separation assurance with heterogeneous multi-agent reinforcement learning,” IEEE Transactions on Automation Science and Engineering , vol. 19, no. 4, pp. 2837–2848, 2022

    2022

  22. [30]

    Asynchronous multi-agent reinforcement learning for efficient real-time multi-robot cooperative exploration,

    C. Yu, X. Yang, J. Gao, J. Chen, Y . Li, J. Liu, Y . Xiang, R. Huang, H. Yang, Y . Wuet al., “Asynchronous multi-agent reinforcement learning for efficient real-time multi-robot cooperative exploration,” inProceedings of the 2023 International Conference on Autonomous Agents a...

    2023

  23. [31]

    Race: improve multi-agent reinforcement learning with representation asymmetry and collaborative evolution,

    P. Li, J. Hao, H. Tang, Y . Zheng, and X. Fu, “Race: improve multi-agent reinforcement learning with representation asymmetry and collaborative evolution,” in International Conference on Machine Learning , 2023, pp. 19 490–19 503

    2023

  24. [32]

    Effective multi-agent deep reinforcement learning control with relative entropy regularization,

    C. Miao, Y . Cui, H. Li, and X. Wu, “Effective multi-agent deep reinforcement learning control with relative entropy regularization,” IEEE Transactions on Automation Science and Engineering , pp. 1–15, 2024

    2024

  25. [33]

    Multi-task multi- agent reinforcement learning with task-entity transformers and value decomposition training,

    Y . Zhu, S. Huang, B. Zuo, D. Zhao, and C. Sun, “Multi-task multi- agent reinforcement learning with task-entity transformers and value decomposition training,” IEEE Transactions on Automation Science and Engineering, pp. 1–14, 2024

    2024

  26. [34]

    R-maddpg for partially observable environments and limited communication,

    R. E. Wang, M. Everett, and J. P. How, “R-maddpg for partially observable environments and limited communication,” arXiv preprint arXiv:2002.06684, 2020

    2002 arXiv

  27. [35]

    Multi-agent deep reinforcement learning with extremely noisy observations,

    O. Kilinc and G. Montana, “Multi-agent deep reinforcement learning with extremely noisy observations,” arXiv preprint arXiv:1812.00922 , 2018

    2018 arXiv

  28. [36]

    A general survey on attention mechanisms in deep learning,

    G. Brauwers and F. Frasincar, “A general survey on attention mechanisms in deep learning,” IEEE Transactions on Knowledge & Data Engineering , no. 01, pp. 1–1, 2021

    2021

  29. [37]

    Neural machine translation by jointly learning to align and translate,

    D. Bahdanau, K. H. Cho, and Y . Bengio, “Neural machine translation by jointly learning to align and translate,” in 3rd International Conference on Learning Representations, ICLR 2015 , 2015

    2015

  30. [38]

    Attention is all you need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in neural information processing systems , vol. 30, 2017

    2017

  31. [39]

    Large language models: A survey,

    S. Minaee, T. Mikolov, N. Nikzad, M. Chenaghlu, R. Socher, X. Ama- triain, and J. Gao, “Large language models: A survey,” arXiv preprint arXiv:2402.06196, 2024

    2024 arXiv

  32. [40]

    Recurrent models of visual attention,

    V . Mnih, N. Heess, A. Graveset al., “Recurrent models of visual attention,” Advances in neural information processing systems , vol. 27, 2014

    2014

  33. [41]

    An image is worth 16x16 words: Transformers for image recognition at scale,

    A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly et al., “An image is worth 16x16 words: Transformers for image recognition at scale,” in International Conference on Learning Representations , 2020

    2020

  34. [42]

    Actor-attention-critic for multi-agent reinforcement learning,

    S. Iqbal and F. Sha, “Actor-attention-critic for multi-agent reinforcement learning,” in International conference on machine learning , 2019, pp. 2961–2970

    2019

  35. [43]

    Attention-based recurrence for multi-agent rein- forcement learning under stochastic partial observability,

    T. Phan, F. Ritz, P. Altmann, M. Zorn, J. N ¨ußlein, M. K ¨olle, T. Gabor, and C. Linnhoff-Popien, “Attention-based recurrence for multi-agent rein- forcement learning under stochastic partial observability,” in International Conference on Machine Learning , 2023, pp. 27 840–27 853

    2023

  36. [44]

    Attention- guided contrastive role representations for multi-agent reinforcement learning,

    Z. Hu, Z. Zhang, H. Li, C. Chen, H. Ding, and Z. Wang, “Attention- guided contrastive role representations for multi-agent reinforcement learning,” arXiv preprint arXiv:2312.04819 , 2023

    2023 arXiv

  37. [45]

    Novel distributed grus based on hybrid self-attention mechanism for dynamic soft sensing,

    Y . He, X. Li, Y . Xu, Q. Zhu, and S. Lu, “Novel distributed grus based on hybrid self-attention mechanism for dynamic soft sensing,” IEEE Transactions on Automation Science and Engineering , vol. 21, no. 4, pp. 5161–5172, 2024

    2024

  38. [46]

    Learning from noisy labels with distillation,

    Y . Li, J. Yang, Y . Song, L. Cao, J. Luo, and L.-J. Li, “Learning from noisy labels with distillation,” in Proceedings of the IEEE international conference on computer vision , 2017, pp. 1910–1918

    2017

  39. [47]

    Curriculum reinforcement learning from avoiding collisions to navigating among movable obstacles in diverse environments,

    H.-C. Wang, S.-C. Huang, P.-J. Huang, K.-L. Wang, Y .-C. Teng, Y .-T. Ko, D. Jeon, and I.-C. Wu, “Curriculum reinforcement learning from avoiding collisions to navigating among movable obstacles in diverse environments,” IEEE Robotics and Automation Letters , vol. 8, no. 5, pp...

    2023

  40. [48]

    Training region-based object detectors with online hard example mining,

    A. Shrivastava, A. Gupta, and R. Girshick, “Training region-based object detectors with online hard example mining,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2016, pp. 761– 769

    2016

  41. [49]

    Markov games as a framework for multi-agent rein- forcement learning,

    M. L. Littman, “Markov games as a framework for multi-agent rein- forcement learning,” in Machine learning proceedings 1994 , 1994, pp. 157–163

    1994

  42. [50]

    Planning and acting in partially observable stochastic domains,

    L. P. Kaelbling, M. L. Littman, and A. R. Cassandra, “Planning and acting in partially observable stochastic domains,” Artificial intelligence, vol. 101, no. 1-2, pp. 99–134, 1998

    1998

  43. [51]

    Continuous control with deep reinforcement learning,

    T. Lilicrap, J. Hunt, A. Pritzel, N. Hess, T. Erez, D. Silver, Y . Tassa, and D. Wiestra, “Continuous control with deep reinforcement learning,” in International Conference on Learning Representations , 2016

    2016

  44. [52]

    Ray rllib: A composable and scalable reinforcement learning library,

    E. Liang, R. Liaw, R. Nishihara, P. Moritz, R. Fox, J. Gonzalez, K. Goldberg, and I. Stoica, “Ray rllib: A composable and scalable reinforcement learning library,” arXiv preprint arXiv:1712.09381, vol. 85, p. 245, 2017

    2017 arXiv

  45. [53]

    rlpyt: A research code base for deep reinforcement learning in pytorch,

    A. Stooke and P. Abbeel, “rlpyt: A research code base for deep reinforcement learning in pytorch,” arXiv preprint arXiv:1909.01500 , 2019

    1909 arXiv

  46. [54]

    Experiment tracking with weights and biases,

    L. Biewald, “Experiment tracking with weights and biases,” 2020, software available from wandb.com. [Online]. Available: https: //www.wandb.com/

    2020

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.