Pith. sign in

REVIEW 3 major objections 5 minor 33 references

Adaptive Episode Length Adjustment for Multi-agent Reinforcement Learning

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read AELA proposes to start MARL training with short episodes and lengthen them only as Q-value entropy falls; the paper claims this speeds convergence and lifts final win rates, rescuing QMIX on maps where it otherwise wins almost never.

desk verdict AELA is a simple, plausible MARL training heuristic with real-looking gains on hard SMAC maps, but the entropy trigger is never isolated from plain truncation plus tuned growth, so the central claim overreaches the evidence. read the letter →

arxiv 2505.19637 v1 pith:OF4HVW2C submitted 2025-05-26 cs.MA

Byunghyun Yoo , Younghwan Shin , Hyunwoo Kim , Euisok Chung , Jeongmin Yang This is my paper
classification cs.MA
keywords multi-agentreinforcementlearningepisodelengthadjustmentdead-endstatesentropy-basedconvergenceassessmentvaluedecompositionQMIXVDNSMAC
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes Adaptive Episode Length Adjustment (AELA), a way to train multi-agent reinforcement learning agents by starting each episode short and only lengthening it once the policy shows signs of converging, measured by falling entropy of action-value distributions. The authors argue that short episodes keep agents away from dead-end states early in training, where no policy can recover, while gradually increasing length lets them eventually learn long-horizon coordination. They test AELA on top of QMIX and VDN in SMAC and a modified predator-prey environment. Their central claim is that AELA improves both convergence speed and final performance, and in maps like 6h_vs_8z and Corridor it turns win rates near zero for QMIX into substantially higher win rates. If true, it offers a general, plug-in training schedule that helps MARL without changing rewards, network architecture, or value factorization.

What carries the argument

The central object is the entropy of the agents' action distributions, computed from Q-values via softmax and summed over batch, time, and agents. AELA collects this total entropy over a window of size w, fits a line, and increments episode length by one when the slope is negative; the initial length EL0 and window w are the only added hyper-parameters. This entropy-triggered length schedule carries the argument because it is what turns the theoretical claim—shorter episodes do not reduce secure-state visit counts—into an algorithm that gradually extends the horizon as learning stabilizes. The supporting theoretical machinery is Lemma 1's monotonicity of secure-state probability over interaction steps, which yields Theorem 1 (expected secure visits do not decrease when episode length is reduced) and Theorem 2 (lower dead-end probability reduces regret under a goal-dominance assumption).

What would settle it

Compare AELA-QMIX on 6h_vs_8z to a version that keeps the episode length fixed at the initial value for the entire run. If the fixed-short version matches AELA's win rate, the entropy-triggered lengthening is not responsible for the reported recovery from near-zero wins.

Watch

Extended reading notes

Core claim

Under AELA, the episode length starts at a fraction of the task maximum (typically one quarter, or 15% in a dead-end-heavy scenario) and increases by one step whenever a linear fit to the entropy of Q-values over a window of updates has negative slope. The paper argues, under a Dec-POMDP model, that the probability of being in a secure (non-dead-end) state is non-increasing in the interaction step, so truncating episodes keeps expected secure-state visits constant or higher, and under the assumption that goal reward dominates intermediate rewards, lower dead-end probability lowers regret. The empirical discovery is that applying this schedule to QMIX and VDN improves convergence speed in easy SMAC maps and final win rate in hard maps, with the largest effect in 6h_vs_8z and Corridor, where original QMIX nearly always loses but AELA-QMIX wins at a significantly higher rate. The authors support the dead-end explanation with snapshots showing that QMIX's final policy in 6h_vs_8z wanders into a dead-end after losing units, while AELA's policy ends episodes earlier and avoids that state.

Load-bearing premise

The load-bearing premise is that when the agents' choices become more predictable (lower entropy), the right next step is to let episodes run longer; if that trigger does not matter, the improvement might simply come from starting with short episodes and tuning two extra hyper-parameters.

Editorial extensions

If this is right

  • On easy SMAC maps (3m, 2s_vs_1sc), AELA reaches the same near-100% win rate as QMIX/VDN but faster.
  • On hard maps (MMM2, 3s5z_vs_3s6z, 6h_vs_8z, Corridor), AELA improves final win rate; for 6h_vs_8z and Corridor, AELA-QMIX wins where QMIX's win rate is near zero.
  • AELA concentrates collected samples in early interaction steps, which the paper argues reduces visits to dead-end states and lets agents learn strategies like luring.
  • Because AELA changes no rewards and no network architecture, it can be layered onto value-decomposition MARL methods, demonstrated with QMIX and VDN.
  • The theory implies the benefit is tied to dead-end-rich environments; in dead-end-free tasks, truncating episodes should not reduce secure-state visits, though the paper does not claim it would improve them.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not compare AELA against a fixed short-episode schedule or a random extension schedule, so the adaptive entropy trigger's standalone contribution is an open question, not a demonstrated result.
  • If the gains come mainly from the initial truncation, a simpler schedule that keeps episodes at the initial length over the full run could match AELA with fewer hyper-parameters; the SMAC maps used here would make that comparison straightforward.
  • Because policy entropy is already computed in policy-gradient MARL methods, the same trigger could be lifted directly into those settings, which the paper does not test.
  • In tasks with dense intermediate rewards or where long horizons are essential for cooperation, truncation can cut off credit assignment, so a boundary condition on when AELA helps is a natural follow-up.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes Adaptive Episode Length Adjustment (AELA) for multi-agent reinforcement learning, in which episodes are initialized at a fraction of the maximum episode length and extended by one step whenever a linear fit to the entropy of a window of softmax Q-values has negative slope. The authors motivate the approach with Lemma 1 and Theorem 1, which state that the probability of being in a secure state is non-increasing in the interaction step and that reducing the episode length increases or preserves the expected number of secure-state visits, and with Theorem 2, which claims that lower dead-end probability reduces regret. They evaluate AELA combined with VDN and QMIX on a modified predator-prey task and six SMAC maps, reporting faster convergence and higher final win rates, particularly for QMIX on 6h_vs_8z and Corridor, where standard QMIX is reported as being close to zero.

Significance. The problem addressed is real and under-explored: episode-length management is a potentially cheap and general intervention for MARL, and the reported observation that QMIX fails on 6h_vs_8z and Corridor while AELA-QMIX succeeds is empirically striking. The manuscript contains a reasonable amount of experimental work, including six SMAC maps, a modified predator-prey environment, and a detailed hyperparameter discussion in Appendix C. However, the theoretical results are largely a restatement of the monotonicity of the secure-state probability, and the experimental design does not isolate the entropy-adaptive mechanism from the initial truncation and the tuned growth schedule. Therefore the headline claim that entropy-based assessment drives the improvements is currently unsupported, and the significance of the contribution depends on whether the missing ablations confirm that attribution.

major comments (3)
  1. [Section 4.1 / Appendix A / Appendix B] The theoretical support in Section 4.1 does not establish a benefit of the adaptive schedule. Theorem 1 (Appendix A, Eq. (14)-(27)) is a re-arrangement of Lemma 1: with N_total fixed, N_s(EL) = (N_total/EL) * sum_{l=1}^{EL} P_s(l), and because P_s(l) is non-increasing in l, N_s(EL) is non-increasing in EL. Corollary 1 then defines P_d = 1 - N_s/N_total, so the stated decrease in P_d is a restatement of the same monotonicity and carries no information about learning over time. Theorem 2's proof in Appendix B assumes P_d and P_g are constant, omits discounting, mixes realized and expected rewards, and drops the term -sum_{t=1}^T r_t between Eq. (30) and Eq. (31). Consequently, the proof does not rigorously show that reducing dead-end probability reduces regret under the proposed schedule. This matters because Section 4.2 explicitly says that AELA is built on these theorems.
  2. [Section 5 / Appendix C / Algorithm 1] The central empirical claim is not isolated. AELA differs from the fixed-maximum-length baselines in two ways: it starts from a reduced EL0 and it grows EL through the entropy trigger in Algorithm 1, lines 22-25. The experiments in Section 5 never compare AELA against (i) training with a fixed short episode length, (ii) a predetermined growth schedule, or (iii) a random growth schedule, so the reported gains could come entirely from EL0 and from the per-scenario tuning of w described in Appendix C. This risk is concrete: Eq. (38)-(40) choose w so that EL reaches its maximum at roughly 80% of training time, and for 6h_vs_8z both EL0 (15% of E_max) and w (900) are specially adjusted. In addition, the premise in Section 4.2 that a decreasing entropy trend indicates policy convergence is asserted without evidence, and the paper does not report how often the entropy slope was negative or how sensitive the results are to the slope test, so the entropy-based growth rule is not shown to be the cause of the observed improvements.
  3. [Section 5.2 / Figures 1-2] Statistical significance is not demonstrated. Figures 1 and 2 report median test returns and win rates over five runs, but they show no variance bands, confidence intervals, or significance tests. The abstract and Section 5.2 use phrases such as 'significantly superior performance' and single out 6h_vs_8z and Corridor as cases where QMIX is 'close to zero' while AELA-QMIX is 'significantly higher'; with only five runs and no measure of spread, this wording overstates what the reported evidence establishes. At minimum, the paper should report per-run curves or error bars and state the number of seeds used in each displayed curve.
minor comments (5)
  1. [Abstract / Section 1] The acronym is introduced inconsistently: the abstract and title use 'Adaptive Episode Length Adjustment (AELA)', while the Introduction defines 'Adaptive Episode Length Approach (AELA)'.
  2. [Section 5.1] The sentence 'The experiment is conducted over 16 episodes' is unclear, since the following sentence refers to five independent runs; this should be rephrased to specify what was averaged over what.
  3. [Section 4.2 / Definition 4] The symbol T is used for the number of time steps within an episode in Eq. (10) and also for the total training horizon in Definition 4; this overloaded notation makes the formulas hard to follow.
  4. [Figure 6] The y-axis label reads 'T est episode length' in the manuscript text; this is a typo for 'Test episode length'.
  5. [Appendix B] Assumption 1 does not specify the sign of r_g, and for the argument around Eq. (36) to work for k=T the empty sum must be handled and r_g must be assumed positive; this should be stated explicitly.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the theoretical results follow from stated definitions, and the empirical claims are not fitted predictions.

full rationale

The paper's derivation chain is self-contained and not circular. Lemma 1 and Theorem 1 formalize the definitions of dead-end and secure states (Definitions 1 and 2), with P_s(l) defined as the product of probabilities of avoiding dead ends; Theorem 1's conclusion that expected secure-state visits do not decrease when EL is shortened is an algebraic consequence of those definitions and monotonicity, not a restatement of the experimental results. Corollary 1 and Theorem 2 likewise follow from the stated regret model and Assumption 1, with Appendix B directly differentiating the constructed regret expression. The entropy-triggered growth rule (Algorithm 1 lines 22-25, Eq. 10-11) is an additional heuristic assumption, not derived from the theorems, and the paper does not ablate it against a fixed short-episode schedule; this weakens the causal attribution but is an experimental-control gap, not circularity. There are no load-bearing self-citations: cited prior work is external, and no uniqueness or convergence claim is imported from the authors' other papers. Appendix C's window-size calibration is hyperparameter tuning, not a fitted quantity that is later renamed a prediction. Thus no step reduces by construction to its own inputs.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central claim rests mostly on standard MARL machinery plus hand-tuned schedule hyperparameters. The most consequential axiom is the untested assumption that falling entropy marks the right moment to lengthen episodes. No new physical entities are introduced.

free parameters (4)
  • EL0 (initial episode length) = 25% of Emax for most scenarios, 15% for 6h_vs_8z
    Initial episode length cap chosen per scenario by hand, not derived from theory or learning progress.
  • w (window size) = 150, 300, or 900 depending on scenario
    Number of entropy samples used for the linear slope detection; tuned per scenario, with the hardest map 6h_vs_8z using 900.
  • Entropy slope threshold = 0
    The rule increases episode length when the fitted slope is negative; the threshold 0 is a design choice without analysis.
  • tau (softmax temperature) = not specified
    Used in Eq. 8 to convert Q-values to action probabilities for entropy; no value is given in the paper.
assumptions (5)
  • domain assumption Assumption 1: the goal reward is larger than the sum of rewards over any interval of time steps within an episode
    Used in Theorem 2 to guarantee that the derivative of regret with respect to P_d is positive; not verified for SMAC or MPP reward functions.
  • ad hoc to paper Entropy decrease indicates policy convergence suitable for extending episode length
    Stated in Section 4.2 and used as the trigger in Algorithm 1, without proof or ablation.
  • ad hoc to paper The regret decomposition in Appendix B assumes constant P_d and P_g and a geometric goal-reaching process
    Equation 29 models expected reward with stationary probabilities (1-P_d)^{k-1}P_g; this model is not derived from the Dec-POMDP and ignores discounting and dead-end absorption.
  • standard math P_s(l) is computed as the product of (1 - P_d(k)) for k=1..l
    This holds if P_d(k) are conditional hazards of entering a dead-end; the paper does not state the conditioning but the product form is standard.
  • domain assumption The SMAC maps used contain dead-end states as described
    The dead-end interpretation in Section 5.2 is based on visual snapshots of one trained policy, not a formal or statistical detection.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Adaptive Episode Length Adjustment for Multi-agent Reinforcement Learning." pith.science (2026). https://pith.science/paper/OF4HVW2C

@misc{pith2026250519637,
  author       = {Pith},
  title        = {Pith review of: Adaptive Episode Length Adjustment for Multi-agent Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OF4HVW2C}},
  note         = {Machine review of arXiv:2505.19637}
}
read the original abstract

In standard reinforcement learning, an episode is defined as a sequence of interactions between agents and the environment, which terminates upon reaching a terminal state or a pre-defined episode length. Setting a shorter episode length enables the generation of multiple episodes with the same number of data samples, thereby facilitating an exploration of diverse states. While shorter episodes may limit the collection of long-term interactions, they may offer significant advantages when properly managed. For example, trajectory truncation in single-agent reinforcement learning has shown how the benefits of shorter episodes can be leveraged despite the trade-off of reduced long-term interaction experiences. However, this approach remains underexplored in MARL. This paper proposes a novel MARL approach, Adaptive Episode Length Adjustment (AELA), where the episode length is initially limited and gradually increased based on an entropy-based assessment of learning progress. By starting with shorter episodes, agents can focus on learning effective strategies for initial states and minimize time spent in dead-end states. The use of entropy as an assessment metric prevents premature convergence to suboptimal policies and ensures balanced training over varying episode lengths. We validate our approach using the StarCraft Multi-agent Challenge (SMAC) and a modified predator-prey environment, demonstrating significant improvements in both convergence speed and overall performance compared to existing methods. To the best of our knowledge, this is the first study to adaptively adjust episode length in MARL based on learning progress.

Figures

Figures reproduced from arXiv: 2505.19637 by the authors.

Figure 1
Figure 1. Median test return in the MPP tasks 5.1 Modified predator-prey The modified predator-prey scenario is an extended version of the classical predator-prey problem, widely used in MARL research to evaluate agent coordination and learning capabilities. In this scenario, multiple predators collaborate to capture prey within an environment, emphasizing the need for cooperative learning. The modified predator-prey scenario… view at source ↗
Figure 2
Figure 2. Median test win rates with different SMAC scenarios [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Limited episode length during training as the experimental findings show no degradation in performance, consistent with the theoretical background discussed in Section 4.1. 5.2 StarCraft multi-agent challenge In SMAC, agents are tasked with solving micromanagement scenar￾ios where they must coordinate their actions to defeat opponent units controlled by the StarCraft II game engine. These scenarios vary in difficult… view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Number of samples with interaction steps [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Snapshot of the final policy for AELA-QMIX in 6h_vs_8z with interaction steps [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Interaction step at which an episode ends when [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: Snapshot of the final policy for QMIX in 6h_vs_8z with interaction steps [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

33 extracted references · 29 canonical work pages

  1. [1]

    Joris Dinneweth, Abderrahmane Boubezoul, René Mandiau, and Stéphane Espié

    work page

  2. [2]

    Mehdi Fatemi, Shikhar Sharma, Harm Van Seijen, and Samira Ebrahimi Kahou

    work page

  3. [3]

    Lior Fuks, Noor H Awad, Frank Hutter, and Marius Lindauer. 2019. An Evolution Strategy with Progressive Episode Lengths for Playing Games.. InIJCAI. 1234– 1240

    work page 2019

  4. [4]

    Tarun Gupta, Anuj Mahajan, Bei Peng, Wendelin Böhmer, and Shimon White- son. 2021. Uneven: Universal value exploration for multi-agent reinforcement learning. InInternational Conference on Machine Learning. PMLR, 3930–3941

    work page 2021

  5. [5]

    Maximilian Hüttenrauch, Adrian Šošić, and Gerhard Neumann. 2017. Guided deep reinforcement learning for swarm systems.arXiv preprint arXiv:1709.06011 (2017)

    arXiv 2017

  6. [6]

    Taylor W Killian, Sonali Parbhoo, and Marzyeh Ghassemi. 2023. Risk sensitive dead-end identification in safety-critical offline reinforcement learning.arXiv preprint arXiv:2301.05664(2023)

    arXiv 2023

  7. [7]

    Chuming Li, Jie Liu, Yinmin Zhang, Yuhong Wei, Yazhe Niu, Yaodong Yang, Yu Liu, and Wanli Ouyang. 2023. Ace: Cooperative multi-agent q-learning with bidirectional action-dependency. InProceedings of the AAAI conference on artificial intelligence, Vol. 37. 8536–8544

    work page 2023

  8. [8]

    Yuntao Liu, Yuan Li, Xinhai Xu, Donghong Liu, and Yong Dou. 2022. Rogc: Role-oriented graph convolution based multi-agent reinforcement learning. In 2022 IEEE International Conference on Multimedia and Expo (ICME). IEEE, 1–6

    work page 2022

Show all 33 references
  1. [9]

    Anuj Mahajan, Tabish Rashid, Mikayel Samvelyan, and Shimon Whiteson. 2019. MAVEN: multi-agent variational exploration. InProceedings of the 33rd Interna- tional Conference on Neural Information Processing Systems. 7613–7624

    2019

  2. [10]

    Debmalya Mandal, Goran Radanovic, Jiarui Gan, Adish Singla, and Rupak Ma- jumdar. 2023. Online reinforcement learning with uncertain episode lengths. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 37. 9064–9071

    2023

  3. [11]

    Nasik Muhammad Nafi, Raja Farrukh Ali, William Hsu, Kevin Duong, and Mason Vick. 2024. Policy Optimization using Horizon Regularized Advantage to Improve Generalization in Reinforcement Learning. InProceedings of the 23rd International Conference on Autonomous Agents and Multi...

    2024

  4. [12]

    Fabio Pardo, Arash Tavakoli, Vitaly Levdik, and Petar Kormushev. 2018. Time limits in reinforcement learning. InInternational Conference on Machine Learning. PMLR, 4045–4054

    2018

  5. [13]

    Riccardo Poiani, Alberto Maria Metelli, and Marcello Restelli. 2023. Truncating trajectories in Monte Carlo reinforcement learning. InInternational Conference on Machine Learning. PMLR, 27994–28042

    2023

  6. [14]

    Riccardo Poiani, Nicole Nobili, Alberto Maria Metelli, and Marcello Restelli. 2024. Truncating trajectories in Monte Carlo policy evaluation: An adaptive approach. Advances in Neural Information Processing Systems36 (2024)

    2024

  7. [15]

    Wei Qiu, Xinrun Wang, Runsheng Yu, Rundong Wang, Xu He, Bo An, Svetlana Obraztsova, and Zinovi Rabinovich. 2021. RMIX: Learning risk-sensitive policies for cooperative reinforcement learning agents. InAdvances in Neural Information Processing Systems, Vol. 34. 23049–23062

    2021

  8. [16]

    Tabish Rashid, Gregory Farquhar, Bei Peng, and Shimon Whiteson. 2020. Weighted qmix: Expanding monotonic value function factorisation for deep multi-agent reinforcement learning. InAdvances in neural information processing systems, Vol. 33. 10199–10210

    2020

  9. [17]

    Tabish Rashid, Mikayel Samvelyan, Christian Schroeder, Gregory Farquhar, Jakob Foerster, and Shimon Whiteson. 2018. Qmix: Monotonic value function factori- sation for deep multi-agent reinforcement learning. InInternational Conference on Machine Learning. PMLR, 4295–4304

    2018

  10. [18]

    Mikayel Samvelyan, Tabish Rashid, Christian Schroeder de Witt, Gregory Far- quhar, Nantas Nardelli, Tim GJ Rudner, Chia-Man Hung, Philip HS Torr, Jakob Foerster, and Shimon Whiteson. 2019. The StarCraft Multi-Agent Challenge. InProceedings of the 18th International Conference ...

    2019

  11. [19]

    Siqi Shen, Chennan Ma, Chao Li, Weiquan Liu, Yongquan Fu, Songzhu Mei, Xinwang Liu, and Cheng Wang. 2023. RiskQ: risk-sensitive multi-agent rein- forcement learning value factorization.Advances in Neural Information Processing Systems36 (2023), 34791–34825

    2023

  12. [20]

    Kyunghwan Son, Daewoo Kim, Wan Ju Kang, David Earl Hostallero, and Yung Yi. 2019. Qtran: Learning to factorize with transformation for cooperative multi- agent reinforcement learning. InInternational Conference on Machine Learning. PMLR, 5887–5896

    2019

  13. [21]

    Kyunghwan Son, Junsu Kim, Sungsoo Ahn, Roben D Delos Reyes, Yung Yi, and Jinwoo Shin. 2022. Disentangling Sources of Risk for Distributional Multi-Agent Reinforcement Learning. InInternational Conference on Machine Learning. PMLR, 20347–20368

    2022

  14. [22]

    Wei-Fang Sun, Cheng-Kuang Lee, and Chun-Yi Lee. 2021. DFAC framework: Factorizing the value function via quantile mixture for multi-agent distributional Q-learning. InInternational Conference on Machine Learning. PMLR, 9945–9954

    2021

  15. [23]

    Peter Sunehag, Guy Lever, Audrunas Gruslys, Wojciech Marian Czarnecki, Vini- cius Zambaldi, Max Jaderberg, Marc Lanctot, Nicolas Sonnerat, Joel Z Leibo, Karl Tuyls, et al. 2018. Value-Decomposition Networks For Cooperative Multi- Agent Learning Based On Team Reward. InProceedi...

    2018

  16. [24]

    2018.Reinforcement learning: An intro- duction

    Richard S Sutton and Andrew G Barto. 2018.Reinforcement learning: An intro- duction. MIT press

    2018

  17. [25]

    Jianhao Wang, Zhizhou Ren, Terry Liu, Yang Yu, and Chongjie Zhang. 2020. QPLEX: Duplex Dueling Multi-Agent Q-Learning. InInternational Conference on Learning Representations

    2020

  18. [26]

    Tonghan Wang, Heng Dong, Victor Lesser, and Chongjie Zhang. 2020. ROMA: Multi-Agent Reinforcement Learning with Emergent Roles. InInternational Conference on Machine Learning. PMLR, 9876–9886

    2020

  19. [27]

    T Wang, T Gupta, B Peng, A Mahajan, S Whiteson, and C Zhang. 2021. RODE: learning roles to decompose multi- agent tasks. InProceedings of the International Conference on Learning Representations

    2021

  20. [28]

    Dayon Ye, Minji Zhang, and Yu Yang. 2015. A multi-agent framework for packet routing in wireless sensor networks.sensors15, 5 (2015), 10026–10047

    2015

  21. [29]

    Xianghua Zeng, Hao Peng, and Angsheng Li. 2023. Effective and stable role-based multi-agent collaboration by structural information principles. InProceedings of the AAAI conference on artificial intelligence, Vol. 37. 11772–11780

    2023

  22. [30]

    Xiao Zhang, Hai Zhang, Hongtu Zhou, Chang Huang, Di Zhang, Chen Ye, and Junqiao Zhao. 2023. Safe Reinforcement Learning With Dead-Ends Avoidance and Recovery.IEEE Robotics and Automation Letters9, 1 (2023), 491–498

    2023

  23. [31]

    𝐸𝐿 Í𝐸𝐿+1 𝑙=1 𝑃𝑠(𝑙)−(𝐸 𝐿+1) Í𝐸𝐿 𝑙=1𝑃𝑠(𝑙) 𝐸𝐿(𝐸𝐿+1) # (21) =𝑁 total×  𝐸𝐿 Í𝐸𝐿 𝑙=1𝑃𝑠(𝑙)+𝑃 𝑠(𝐸𝐿+1) −(𝐸 𝐿+1) Í𝐸𝐿 𝑙=1𝑃𝑠(𝑙) 𝐸𝐿(𝐸𝐿+1)  (22) =𝑁 total×

    Lulu Zheng, Jiarui Chen, Jianhao Wang, Jiamin He, Yujing Hu, Yingfeng Chen, Changjie Fan, Yang Gao, and Chongjie Zhang. 2021. Episodic multi-agent re- inforcement learning with curiosity-driven exploration. InAdvances in Neural Information Processing Systems, Vol. 34. 3757–376...

    2021

  24. [2019]

    InInterna- tional Conference on Machine Learning

    Dead-ends and secure exploration in reinforcement learning. InInterna- tional Conference on Machine Learning. PMLR, 1873–1881

  25. [2022]

    Autonomous Intelligent Systems2, 1 (2022), 1–12

    Multi-agent reinforcement learning for autonomous vehicles: a survey. Autonomous Intelligent Systems2, 1 (2022), 1–12

    2022

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.