Pith. sign in

REVIEW 5 major objections 4 minor 28 references

A multi-agent reinforcement learning framework lets drone teams navigate unseen maze layouts immediately, reaching 75% cooperative success without retraining.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 01:33 UTC pith:MSGQQ5MO

load-bearing objection A coherent modular MARL system with an interesting ω-conditioned MoE generalisation idea, but the evaluation is thin and one central module assumes global position despite the paper's no-global-positioning premise. the 5 major comments →

arxiv 2607.25754 v1 pith:MSGQQ5MO submitted 2026-07-28 cs.RO

Cooperative Multi-UAV Navigation in Complex Environments via Systematic Multi-Agent Deep Reinforcement Learning

Yu Su , Nabil Aouf This is my paper
classification cs.RO
keywords multi-agent reinforcement learningUAV navigationzero-shot generalizationlocal optima escapecurriculum learningmixture of expertssparse rewardsbehavioral cloning
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper aims to show that cooperative UAV navigation in structured environments does not have to be retrained for every new layout. It proposes a multi-agent deep RL framework in which local geometric patterns, captured by a deterministic domain parameter computed from LiDAR readings, condition a mixture-of-experts policy, allowing a maze built from known structure types to be navigated immediately. A parameter-free execution-level intervention detects local optima and steers agents out, while a dual-condition curriculum and hierarchical demonstration cloning stabilize training. If correct, the framework would make cross-scenario generalization a matter of structure recognition rather than scenario memorization.

Core claim

The central claim is that cross-scenario transfer in multi-UAV cooperative navigation is achievable when the policy is conditioned on reusable local geometric structures rather than scenario coordinates. A deterministic function maps each agent's eight-sector LiDAR readings into a four-component domain parameter ω describing dead-ends, walls, narrow entrances, and open areas; a gating network uses ω to blend expert sub-policies. The paper argues that this structure-aware gating, not merely feeding ω into the observation, is what enables a 0.750 cooperative success rate on an unseen mixed maze. It further claims that execution-level local-optima intervention — overriding policy output with a

What carries the argument

The load-bearing machinery is the structure-aware mixture-of-experts actor. A shared LSTM encodes each agent's observation history; a gating network takes the deterministic domain parameter ω ∈ [0,1]^4, computed from eight LiDAR sectors, and produces softmax weights over expert heads; the final action is a weighted fusion of expert outputs. The same ω is appended to the observation. The paper argues that because ω is computed from sensor readings and decoupled from coordinates, the policy learns to respond to structural patterns, and the gate automatically specializes experts to different structure types, allowing zero-shot transfer. The execution-level intervention module, driven by visited

Load-bearing premise

The local-optima diagnosis module updates a grid-cell visitation table from each agent's current position, while the paper motivates the task with the absence of global positioning information; and the zero-shot claim is tested only on structures drawn from the four hand-picked ω categories.

What would settle it

Run the full framework on an unseen maze whose local structures include, say, T-junctions or curved corridors that do not map cleanly to the four ω components; if the zero-shot success rate collapses toward the w/o-MoE baseline, the claimed transfer is limited to the four structure types. Reimplement the intervention module using only local sensor data and check whether the early-training success improvement survives; if not, the decisive ablation depends on an assumption the paper's problem statement denies.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • If a new maze is composed of the four trained structure types, the team should navigate it without retraining, per the 0.750 zero-shot result.
  • Removing the local-optima override prevents cooperative success from emerging in early training, so the intervention is claimed as decisive for sample efficiency.
  • Removing the MoE gating drops unseen-scenario success to 0.250 while keeping ω in the observation, indicating the gating architecture itself, not just the feature, drives generalization.
  • The dual-condition curriculum simultaneously enforces success-rate thresholds and collision-rate ceilings, and the reported collision rates suggest safety can be trained jointly with task performance.
  • Dynamic obstacle scenarios can be handled without modification, with reported success rates of 0.800 on the moving-obstacle training scenario and 0.750 on the unseen dynamic mix.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A testable extension would be to adapt the local-optima diagnosis to use only local sensor history, since the paper's motivating scenario assumes no global positioning; the current grid-cell visitation update appears to rely on global coordinates.
  • The four ω components are hand-picked for maze corridors; an automatic learned embedding of LiDAR patches could reveal whether zero-shot transfer extends to broader structure classes, which the authors list as future work.
  • The gate weight analysis suggests the gating signal could serve as an operator-facing diagnostic of which structural pattern each drone is currently exploiting, a use the paper does not explicitly emphasize.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 4 minor

Summary. Su and Aouf propose a MASAC-based framework for cooperative multi-UAV navigation in maze environments. The framework combines (i) an execution-level local-optima diagnosis and intervention module that overrides policy outputs when stagnation is detected, (ii) a hierarchical demonstration buffer with graded behavioural cloning, (iii) a safety-aware dual-condition curriculum scheduler with retrospective evaluation, and (iv) a LiDAR-derived domain parameter ω that conditions a mixture-of-experts actor for cross-scenario transfer. The paper reports evaluations on seven AirSim maze scenarios plus an unseen 'maze mix', with cooperative success rate, collision rate, mean terminal distance, ablations, and comparisons against MAPPO and a mechanism-free MASAC variant.

Significance. If the reported results are reliable, the framework would be a useful step for MARL navigation in structured environments: the deterministic, sensor-derived structure parameter is an elegant idea, and the safety-aware curriculum with retrospection is a sensible mechanism. The paper gives a complete algorithmic description, an explicit training procedure, and ablations for each component. However, the current evidence is not sufficient for acceptance. The most serious problem is an internal inconsistency: the central intervention module requires global position information that the problem statement explicitly excludes. In addition, the quantitative evaluation rests on 20 evaluation episodes, no random seeds, and unreported hyperparameters, so the headline numbers do not establish the claimed superiority. The zero-shot claim is also narrower than stated because the unseen test maze is composed of exactly the four hand-selected structural classes.

major comments (5)
  1. [Section I-A / II-D, Eq. (6)-(7), Algorithm 1] Section I-A motivates the work with 'the absence of global positioning information', and Table I contains no position coordinates. Section II-D, however, requires each agent to maintain a cell-visitation table c(x) by 'mapping its current position to the corresponding cell'; Eq. (6) uses spatial span and unique-cell ratio over this grid, Eq. (7) probes candidate directions with visit counts, and Algorithm 1 applies the override at every timestep. The full-framework results in Tables III-V use this module. No localization method is specified. As written, the central mechanism cannot run in the stated problem setting. Please either add an explicit localization assumption and revise the problem statement, or redesign the module to operate from onboard sensing only.
  2. [Section IV (Tables III-V, Figs. 5-6)] All headline numbers are point estimates from 20 evaluation episodes with no random seeds and no confidence intervals. For η=0.800 (16/20) the 95% Wilson interval is roughly [0.56, 0.94]; for η=0.750 (15/20) it is roughly [0.51, 0.91]. The main transfer comparison 0.800→0.750 is therefore within noise, and even the 0.750 vs 0.250 difference could be a seed artifact. The ablation curves in Figs. 5 and 6 are single runs. Report multiple seeds with mean±std or confidence intervals, and use a larger evaluation set.
  3. [Table II and Eq. (4), (6), (7), (10)] Section III says 'All hyperparameter settings are listed in Table II', but Table II omits the reward weights in Eq. (4) (w_d, w_p, w_c, w_n, w_l, w_s1, w_s2, w_align, w_prog, w_opt, collision penalties, arrival rewards, and c_1,c_2,c_3), the diagnosis thresholds in Eq. (6), the scoring weights in Eq. (7), and the LiDAR occlusion threshold θ in Eq. (10). Curriculum thresholds τ_η and τ_ξ are described as per-scenario but only maze 04's values are given in Section IV-A. Without these numbers the experiments are not reproducible and the safety claims cannot be audited.
  4. [Section II-G / IV-D] The four ω components in Eq. (10) are hand-constructed to correspond to four structural classes, and the unseen test maze is composed of exactly those classes. This supports a claim of transfer across global layouts within a fixed structural taxonomy, but not the broader 'zero-shot cross-scenario generalisation' phrasing of the Abstract. Either test on scenarios containing structural patterns outside the four classes, or explicitly qualify the claim. This is a claim-scope issue rather than an internal inconsistency, but it affects how the central contribution should be read.
  5. [Section IV-B] The baseline comparison is not on equal footing: MAPPO uses a 21-dimensional observation while the full framework uses 33 dimensions; no hyperparameter search is reported; and all methods are trained for a fixed 1500 episodes, which may not be enough for the baselines to converge. Standard MASAC is described only as a 'variant with all proposed mechanisms removed', but its observation dimension and whether it retains ω are not stated. Please report tuning details, learning curves for the baselines, and multiple seeds.
minor comments (4)
  1. [Section II-C, Eq. (3)] The 'total reward assigned to agent i' is written as r_t = r^team_t + sum_i r_i_t, which is ambiguous or circular. Please clarify whether agents receive a shared team reward or per-agent rewards, and write r_i,t explicitly if per-agent rewards are intended.
  2. [Section I-C] The intervention module is called 'parameter-free', but Eq. (6)-(7) contain thresholds and weights that must be tuned. The precise statement is 'no additional trainable parameters', and the text should say that.
  3. [Abstract / Section II-D] The Abstract mentions 'penalty backpropagation', but the method in Eq. (4) applies a scalar reward penalty; there is no backpropagation of penalties. Please reword to avoid a misleading technical term.
  4. [Section IV-A] The claim 'Dynamic obstacles are detected in all episodes' is not supported by a definition of a detection metric. Please specify how detection is measured and report the corresponding statistic.

Circularity Check

0 steps flagged

No circularity in the derivation chain; the main risk is an internal feasibility gap (global position used by the local-optima module), not a fitted/self-cited prediction.

full rationale

I checked each load-bearing claim against the paper's own equations and experiment design. The local-optima escape mechanism is an explicit execution-level override with hand-coded conditions (Eqs. 6-7), and its contribution is tested by ablation (Fig. 5), not inferred from its own definition. The graded behavioural cloning loss (Eqs. 8-9) and the dual-condition curriculum scheduler are mechanisms with separate reward/success criteria; Table IV and the ablations independently compare against MAPPO and MASAC. The structure-aware generalisation mechanism is a hand-defined four-dimensional ω (Eq. 10) and is explicitly scoped by the paper to 'unseen mazes composed of structural patterns encountered during training'; this limits the breadth of the zero-shot claim but is not a circular derivation, because the policy is never trained on the target layout and the reported success is an empirical outcome, not an identity. There are no load-bearing self-citations; the cited prior works are external and used as context. The most serious issue is internal consistency: Section I-A motivates the setting by 'the absence of global positioning information,' yet Section II-D requires each agent to 'map[ ] its current position to the corresponding cell' of a global grid and Eq. (7) probes cell visit counts. That is a feasibility gap in the stated problem, but it is a missing-support/correctness concern, not a reduction of a prediction to its inputs. I therefore find no circularity.

Axiom & Free-Parameter Ledger

5 free parameters · 6 axioms · 0 invented entities

The central claim rests on a stack of hand-chosen constants and domain assumptions: reward weights, diagnosis thresholds, ω construction constants, curriculum thresholds, and sim-to-real fidelity. The most consequential unstated premise is global position access for the visitation grid, plus the assumption that four hand-picked LiDAR structure categories are sufficient for zero-shot transfer.

free parameters (5)
  • Reward shaping weights = not reported
    Eqs. (4)-(5) list w_d, w_p, w_c, w_n, w_l, w_s1, w_s2, w_align, w_prog, w_opt, c1, c2, c3 and event bonuses as tuneable; no values or sensitivity analysis are given.
  • Local-optima diagnosis thresholds = not reported
    ρ0, history window W, safety distance d_min, and the blocked-direction threshold are not quantified; they determine when the override fires.
  • Domain parameter design constants = hand-set to match four structural classes
    Eq. (10) uses clip bounds 5, 3, 2 and occlusion threshold θ chosen to make ωdead, ωwall, ωnarrow, ωopen align with the four maze structure types.
  • Curriculum thresholds τ_η, τ_ξ = not reported except maze-04 pair
    Section II-F makes these independently configurable per scenario; Table III only reveals τ_η=0.75 and τ_ξ=0.40 for maze 04. Without pre-registration, passing stages can be tautological.
  • Cloning schedule and buffer quotas = β_min=0.03, β_max=0.40, λ=1/3, τ_gap=100, p_hist=0.40, K_prefill=15, M_coop=60000, M_ind=5000
    These are set by hand; no sensitivity analysis is reported for the demonstration-exploitation and curriculum mechanisms.
axioms (6)
  • domain assumption AirSim/Unreal Engine physics and sensor models are a faithful proxy for real UAV flight.
    All validation is in simulation; the conclusion explicitly acknowledges the sim-to-real gap.
  • ad hoc to paper Each agent has access to global position (or reliable localization) for the visitation grid.
    Section II-D maps the current position to a grid cell, despite Section I-A stating the problem is characterized by absence of global positioning information.
  • ad hoc to paper The four structural classes (dead-end, wall, narrow, open) completely characterize local geometry relevant for maze navigation.
    The domain parameter ω is built from exactly these categories, and the unseen test maze is also a composition of them, so other structure classes are not tested.
  • domain assumption Self-generated successful trajectories provide valid demonstration supervision for behavioral cloning.
    The demo buffers are filled by the current policy during training, not by an external expert; the paper does not justify why this differs from self-imitation learning.
  • domain assumption 8-sector LiDAR at 10 Hz is sufficient for the navigation tasks.
    The observation space in Table I is built on 8 sectors; no sensor-noise or hardware-latency ablation is provided.
  • standard math Dec-POMDP/MASAC formalization and standard RL convergence assumptions hold.
    The paper relies on standard SAC policy-gradient machinery with a centralized critic; no stability or convergence proofs are given.

pith-pipeline@v1.3.0-alltime-deepseek · 16658 in / 14270 out tokens · 150218 ms · 2026-08-01T01:33:50.963137+00:00 · methodology

0 comments
read the original abstract

Cooperative navigation of multi-agent UAVs in complex environments faces key challenges including local optima traps, sparse rewards, learning imbalance among agents, and insufficient cross-scenario generalisation. This paper proposes a multi-agent deep reinforcement learning framework that addresses these issues through coordinated exploration, demonstration exploitation, safe curriculum scheduling, and structure-aware generalisation. First, a perception mechanism combining memory of visited states, directional novelty estimates, and penalty backpropagation enables agents to proactively detect and escape local optima. Second, a hierarchical collaborative demonstration buffer with tiered behaviour cloning manages trajectories by degree of team collaboration and applies differential supervision to the actor network, improving demonstration utilisation under sparse collaborative signals. Third, a safety-aware dual-condition curriculum scheduling mechanism reviews mastered scenarios through back-testing and experience pre-filling during training, suppressing catastrophic forgetting while ensuring both task performance and flight safety. For generalisation, local geometric features computed from sensor readings are abstracted into a domain parameter, through which a structure-aware gating network and mixture-of-experts mechanism condition the policy on local structural patterns rather than scenario-specific coordinates, enabling cross-scenario transfer without exposure to the target environment. The framework is further validated under mixed static-dynamic obstacle settings, showing robust adaptability to dynamic disturbances. Simulation results confirm strong performance in collaboration success rate, navigation robustness, zero-shot cross-scenario generalisation, and dynamic environment adaptability.

Figures

Figures reproduced from arXiv: 2607.25754 by Nabil Aouf, Yu Su.

Figure 1
Figure 1. Figure 1: Overall architecture of the proposed cooperative multi-UAV navigation [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overview of training scenarios maze 01 to maze 06 (20 m × 20 m each), numbered clockwise from the top-left. Purple and green markers indicate start and goal positions respectively. Each scenario is configured with multiple start–goal pairs; during training, start and goal positions are swapped and agent assignments are exchanged across pairs to promote generalisation and prevent overfitting to specific spa… view at source ↗
Figure 3
Figure 3. Figure 3: Training scenario maze 07 with dynamic obstacles. The obstacle enclosed within the red bounding box is a moving obstacle performing lateral reciprocating motion at 1.5 m/s within a bounded region. Purple and green markers indicate start and goal positions respectively. TABLE II HYPERPARAMETER CONFIGURATION. Parameter Symbol Value Network architecture Observation history length K 16 Hidden layer dimension —… view at source ↗
Figure 4
Figure 4. Figure 4: Representative UAV flight trajectories in the unseen test scenario [PITH_FULL_IMAGE:figures/full_fig_p010_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Cooperative success rate η during training on maze 05 for all ablation variants [PITH_FULL_IMAGE:figures/full_fig_p011_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Collision rate ξ during training on maze 05 for all ablation variants. The full framework (blue) maintains a cooperative success rate of 0.30–0.45 from the earliest training episodes, sub￾stantially above all ablation variants, demonstrating that the graded behavioural cloning supervision from the hierarchical demonstration buffer provides effective early-stage guidance. After episode 600, η rises steadily… view at source ↗
Figure 7
Figure 7. Figure 7: UAV flight trajectories of the w/o MoE variant in the unseen test [PITH_FULL_IMAGE:figures/full_fig_p012_7.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

28 extracted references · 5 linked inside Pith

  1. [1]

    Curiosity-driven exploration by self-supervised prediction,

    D. Pathak, P. Agrawal, A. A. Efros, and T. Darrell, “Curiosity-driven exploration by self-supervised prediction,” inInternational conference on machine learning. PMLR, 2017, pp. 2778–2787

    2017

  2. [2]

    Exploration by random network distillation,

    Y . Burda, H. Edwards, A. Storkey, and O. Klimov, “Exploration by random network distillation,”arXiv preprint arXiv:1810.12894, 2018

    Pith/arXiv arXiv 2018

  3. [3]

    Never give up: Learning directed exploration strategies,

    A. P. Badia, P. Sprechmann, A. Vitvitskyi, D. Guo, B. Piot, S. Kap- turowski, O. Tieleman, M. Arjovsky, A. Pritzel, A. Boltet al., “Never give up: Learning directed exploration strategies,”arXiv preprint arXiv:2002.06038, 2020

    Pith/arXiv arXiv 2002

  4. [4]

    Probabilistic robotics,

    S. Thrun, “Probabilistic robotics,”Communications of the ACM, vol. 45, no. 3, pp. 52–57, 2002

    2002

  5. [5]

    Neural map: Structured memory for deep reinforcement learning,

    E. Parisotto and R. Salakhutdinov, “Neural map: Structured memory for deep reinforcement learning,”arXiv preprint arXiv:1702.08360, 2017

    Pith/arXiv arXiv 2017

  6. [6]

    Object goal navigation using goal-oriented semantic exploration,

    D. S. Chaplot, D. P. Gandhi, A. Gupta, and R. R. Salakhutdinov, “Object goal navigation using goal-oriented semantic exploration,”Advances in Neural Information Processing Systems, vol. 33, pp. 4247–4258, 2020

    2020

  7. [7]

    Multi-agent actor-critic for mixed cooperative-competitive environ- ments,

    R. Lowe, Y . I. Wu, A. Tamar, J. Harb, O. Pieter Abbeel, and I. Mordatch, “Multi-agent actor-critic for mixed cooperative-competitive environ- ments,”Advances in neural information processing systems, vol. 30, 2017

    2017

  8. [8]

    Learning to communicate with deep multi-agent reinforcement learning,

    J. Foerster, I. A. Assael, N. De Freitas, and S. Whiteson, “Learning to communicate with deep multi-agent reinforcement learning,”Advances in neural information processing systems, vol. 29, 2016

    2016

  9. [9]

    Monotonic value function factorisation for deep multi- agent reinforcement learning,

    T. Rashid, M. Samvelyan, C. S. De Witt, G. Farquhar, J. Foerster, and S. Whiteson, “Monotonic value function factorisation for deep multi- agent reinforcement learning,”Journal of Machine Learning Research, vol. 21, no. 178, pp. 1–51, 2020

    2020

  10. [10]

    Domain randomization for transferring deep neural networks from simulation to the real world,

    J. Tobin, R. Fong, A. Ray, J. Schneider, W. Zaremba, and P. Abbeel, “Domain randomization for transferring deep neural networks from simulation to the real world,” in2017 IEEE/RSJ international conference on intelligent robots and systems (IROS). IEEE, 2017, pp. 23–30

    2017

  11. [11]

    Model-agnostic meta-learning for fast adaptation of deep networks,

    C. Finn, P. Abbeel, and S. Levine, “Model-agnostic meta-learning for fast adaptation of deep networks,” inInternational conference on machine learning. PMLR, 2017, pp. 1126–1135

    2017

  12. [12]

    Causality meets locality: Provably generalizable and scalable policy learning for networked sys- tems,

    H. Liang, Y . Zhang, B. Huang, Y . Duet al., “Causality meets locality: Provably generalizable and scalable policy learning for networked sys- tems,”Advances in Neural Information Processing Systems, vol. 38, pp. 158 980–159 030, 2026

    2026

  13. [13]

    Dynamics generalisation in reinforcement learning via adaptive context-aware policies,

    M. Beukman, D. Jarvis, R. Klein, S. James, and B. Rosman, “Dynamics generalisation in reinforcement learning via adaptive context-aware policies,”Advances in Neural Information Processing Systems, vol. 36, pp. 40 167–40 203, 2023

    2023

  14. [14]

    Mixtures of experts unlock parameter scaling for deep rl,

    J. Obando-Ceron, G. Sokar, T. Willi, C. Lyle, J. Farebrother, J. Foerster, G. K. Dziugaite, D. Precup, and P. S. Castro, “Mixtures of experts unlock parameter scaling for deep rl,”arXiv preprint arXiv:2402.08609, 2024

    Pith/arXiv arXiv 2024

  15. [15]

    Unifying count-based exploration and intrinsic motivation,

    M. Bellemare, S. Srinivasan, G. Ostrovski, T. Schaul, D. Saxton, and R. Munos, “Unifying count-based exploration and intrinsic motivation,” Advances in neural information processing systems, vol. 29, 2016

    2016

  16. [16]

    A formal basis for the heuristic determination of minimum cost paths,

    P. E. Hart, N. J. Nilsson, and B. Raphael, “A formal basis for the heuristic determination of minimum cost paths,”IEEE transactions on Systems Science and Cybernetics, vol. 4, no. 2, pp. 100–107, 1968

    1968

  17. [17]

    A note on two problems in connexion with graphs,

    E. W. Dijkstra, “A note on two problems in connexion with graphs,” in Edsger Wybe Dijkstra: his life, work, and legacy, 2022, pp. 287–290

    2022

  18. [18]

    Uav path planning in 3- d constrained environments based on layered essential visibility graphs,

    L. Blasi, E. D’Amato, M. Mattei, and I. Notaro, “Uav path planning in 3- d constrained environments based on layered essential visibility graphs,” IEEE Transactions on Aerospace and Electronic Systems, vol. 59, no. 3, pp. 2359–2375, 2022

    2022

  19. [19]

    Value-decomposition networks for cooperative multi-agent learning,

    P. Sunehag, G. Lever, A. Gruslys, W. M. Czarnecki, V . Zambaldi, M. Jaderberg, M. Lanctot, N. Sonnerat, J. Z. Leibo, K. Tuylset al., “Value-decomposition networks for cooperative multi-agent learning,” arXiv preprint arXiv:1706.05296, 2017

    Pith/arXiv arXiv 2017

  20. [20]

    Learning-based navigation and collision avoidance through reinforcement for uavs,

    R. Azzam, M. Chehadeh, O. A. Hay, M. A. Humais, I. Boiko, and Y . Zweiri, “Learning-based navigation and collision avoidance through reinforcement for uavs,”IEEE Transactions on Aerospace and Electronic Systems, vol. 60, no. 3, pp. 2614–2628, 2023

    2023

  21. [21]

    Imitation and exploration: learning for vision-based communication-free multi-uav co- ordination in cluttered environments,

    Y . Wan, X. Chen, Z. Zhao, K. Wang, and J. Tang, “Imitation and exploration: learning for vision-based communication-free multi-uav co- ordination in cluttered environments,”IEEE Transactions on Aerospace and Electronic Systems, 2025

    2025

  22. [22]

    Self-attention-enhanced multi- agent deep reinforcement learning for uavs target search in obstacle- dense environments,

    M. Liu, L. Xie, L. Xiao, and S. Guo, “Self-attention-enhanced multi- agent deep reinforcement learning for uavs target search in obstacle- dense environments,”IEEE Transactions on Aerospace and Electronic Systems, 2026

    2026

  23. [23]

    Rapid decision-making strategy for uav swarms in complex adversarial environments using proximal policy optimization and transformer,

    X. Tong, J. Song, and W. Li, “Rapid decision-making strategy for uav swarms in complex adversarial environments using proximal policy optimization and transformer,”IEEE Transactions on Aerospace and Electronic Systems, 2025

    2025

  24. [24]

    A survey of robot learning from demonstration,

    B. D. Argall, S. Chernova, M. Veloso, and B. Browning, “A survey of robot learning from demonstration,”Robotics and autonomous systems, vol. 57, no. 5, pp. 469–483, 2009

    2009

  25. [25]

    Generative adversarial imitation learning,

    J. Ho and S. Ermon, “Generative adversarial imitation learning,”Ad- vances in neural information processing systems, vol. 29, 2016

    2016

  26. [26]

    F. A. Oliehoek, C. Amatoet al.,A concise introduction to decentralized POMDPs. Springer, 2016, vol. 1

    2016

  27. [27]

    Airsim: High-fidelity visual and physical simulation for autonomous vehicles,

    S. Shah, D. Dey, C. Lovett, and A. Kapoor, “Airsim: High-fidelity visual and physical simulation for autonomous vehicles,” inField and service robotics: Results of the 11th international conference. Springer, 2017, pp. 621–635

    2017

  28. [28]

    The surprising effectiveness of ppo in cooperative multi-agent games,

    C. Yu, A. Velu, E. Vinitsky, J. Gao, Y . Wang, A. Bayen, and Y . Wu, “The surprising effectiveness of ppo in cooperative multi-agent games,” Advances in neural information processing systems, vol. 35, pp. 24 611– 24 624, 2022

    2022