Pith. sign in

REVIEW 4 major objections 5 minor 52 references

Diffusion-RL Based Air Traffic Conflict Detection and Resolution Method

T0 review · 4 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A value-guided diffusion policy keeps several valid resolution maneuvers alive at once, and that flexibility is credited with cutting NMAC incidence by about 59% in dense simulated air traffic.

desk verdict Plausible application of diffusion-RL to ATM, but the multimodal-attribution claim isn't isolated, and missing code/error bars weaken the empirical core. read the letter →

arxiv 2509.03550 v1 pith:AAM4XFZ4 submitted 2025-09-02 cs.AI

Tonghe Li , Jixin Liu , Weili Zeng , Hao Jiang This is my paper
classification cs.AI
keywords airtrafficconflictdetectionandresolutiondiffusionpolicymultimodalreinforcementlearningdensity-progressivecurriculumdeepseparationassurancenearmid-aircollision
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to show that the reason deep-RL air-traffic conflict resolvers stall or collide is that their policies are unimodal: they commit to a single best maneuver and have no backup. To test this, it builds Diffusion-AC, a reinforcement-learning agent whose policy is a diffusion model that represents a distribution over 27 discrete maneuver combinations (heading, speed, flight-level) and is steered by learned Q-values toward safe, high-value actions. In a custom en-route simulator with up to 48 intruder aircraft, the agent keeps a 94.1% success rate in the densest scenario and reduces near-mid-air collisions by roughly 59% relative to the next-best baseline. The paper's point is that keeping multiple viable resolution strategies alive is what buys the safety margin, and that diffusion models are a practical way to do this in a safety-critical discrete action space.

What carries the argument

The central object is the diffusion policy: a conditional denoising model that maps a state to a distribution over 27 joint maneuvers by iteratively denoising Gaussian noise in 27-dimensional logit space, then applying a safety-masked softmax. It is trained with a value-guided teacher: a temperature-smoothed distribution derived from twin Q-critics whose conservative min-target shapes the denoising objective. The supporting mechanism is the Density-Progressive Safety Curriculum (DPSC), a twelve-stage schedule that increases traffic density from 3 to 36 intruders and linearly tightens the Loss-of-Separation threshold from 4.5 km to 10 km while keeping the feasibility mask fixed, so the policy

What would settle it

Run the learned policy against a different traffic generator (or recorded trajectories) in an independent en-route simulator and compare success and NMAC rates; if the 94.1% and 1.2% margins do not reproduce, the multimodal benefit is simulator-specific. A sharper test is to artificially block the agent's primary maneuver mode at a critical decision point and check whether the retained alternative mode actually prevents the separation loss.

Watch

Extended reading notes

Core claim

Diffusion-AC claims that a value-guided diffusion policy can resolve multi-aircraft en-route conflicts more safely than unimodal DRL baselines because it retains a multimodal action distribution. Its policy is a reverse denoising process over 27-dimensional logit vectors; each logit vector corresponds to a joint categorical distribution over heading-change, speed-change, and flight-level-step triples. A safety mask removes infeasible actions before normalization, twin Q-critics provide conservative value estimates, and a Q-weighted teacher distribution guides the denoiser to concentrate probability on high-value regions without collapsing to one mode. The empirical claim is that in the highe

Load-bearing premise

The claims are measured inside a custom simulator whose 12-airway, three-flight-level, constant-speed-intruder world is assumed to stand for real en-route air traffic; if that proxy is not faithful, the safety margins may not transfer.

Editorial extensions

If this is right

  • At 48-intruder density, NMAC rate falls from 6.4% (PPO) to 1.2% while success stays above 94%, supporting the paper's explanation that preserved second-choice maneuvers provide the safety margin.
  • Ablating value guidance drops success to 4.1%, showing that Q-value weighting is what turns the diffusion process into a learned policy rather than random search.
  • Ablating DPSC raises timeout rate from 4.7% to 9.5% and NMAC from 1.2% to 2.9%, supporting the curriculum's role in stable convergence.
  • Single-step inference is about 8.3 ms at T=10 diffusion steps, an order of magnitude slower than feed-forward baselines but still within the seconds-scale ATC decision loop.
  • Multimodal first-action distributions persist across three constrained scenarios, indicating the agent can switch strategies when one route is blocked.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not test transfer to real traffic; the observed margin may be smaller in operational en-route conditions, since the simulator is custom-built and not cross-validated against a standard benchmark or recorded trajectory data.
  • If multimodal retention is the active mechanism, the advantage should widen with traffic density and constraint asymmetry; a scaling study could isolate whether the benefit saturates or persists at even higher densities.
  • The same value-guided diffusion-policy treatment could apply to other safety-critical discrete-action domains with multiple equivalent evasive maneuvers, such as UAV deconfliction or vessel collision avoidance.
  • The paper mentions distilling the stochastic policy into decision trees or symbolic rules but does not evaluate it; whether distillation preserves the multimodal backups is an open question.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes Diffusion-AC, a diffusion-model-based actor-critic framework for air traffic conflict detection and resolution (CD&R) in a discrete 27-class action space over heading, speed, and flight-level changes. A value-guided teacher distribution is used to train the denoising policy, and a Density-Progressive Safety Curriculum (DPSC) gradually increases traffic density and tightens the LoS threshold. Empirical comparisons against PPO, TD3, Safe-DQN-X, and Rainbow-DQN in a custom 3D en-route simulator report a 94.1% success rate and 1.2% NMAC rate in the high-density scenario, with approximately 59% NMAC reduction relative to the next-best baseline. The paper also includes ablations, a hyperparameter sensitivity study, real-time latency measurements, and heatmap visualizations intended to demonstrate multimodal action distributions.

Significance. If the central claims are substantiated, the paper would make a useful contribution by introducing diffusion-based policy representation to a safety-critical ATM problem and by proposing a curriculum mechanism tailored to CD&R. The authors provide a fairly comprehensive comparison suite, a detailed MDP formulation, ablations of key components, and a real-time inference analysis. The claimed safety improvement is practically meaningful. However, the empirical core is weakened by the absence of code/data, the lack of repeated-seed statistics, the use of an unvalidated custom simulator, and, most importantly, the absence of a direct categorical softmax baseline that would isolate the role of the diffusion mechanism in the reported gains. These issues prevent the current version from supporting the causal attribution of performance to 'multimodal decision-making capability.'

major comments (4)
  1. [§5.2.2, Table 2, and §5.4] The paper's headline explanation—that the 94.1% success / 1.2% NMAC advantage 'stems from its unique multimodal decision-making capability'—is not isolated by the experiments. Equation (33) defines the policy as a masked softmax over 27 denoised logits; any MLP with 27 output logits can represent the same family of categorical distributions. No baseline uses a direct 27-way softmax: PPO uses separate continuous/discrete heads, DQN variants use greedy Q-values, and the W/O Diffusion ablation in Table 3 uses a Gaussian policy. Section 5.4's heatmaps compare only Diffusion-AC against PPO. The observed gains could therefore be due to the joint-categorical parameterization, dual-Q critics, value-guided teacher, or DPSC rather than to diffusion per se. A direct 27-way softmax actor-critic baseline and mode-count statistics are needed to support the causal attribution.
  2. [§5.2, Table 2] The main benchmark results are single-run point estimates without error bars, confidence intervals, or significance tests. The text states that 1,000 inference episodes are used for each algorithm/density, but this reflects a single training run; stochasticity in training seeds, replay sampling, and diffusion noise is not captured. The sensitivity analysis in §5.3.2 reports five runs, but the main benchmark and ablation table do not. Without repeated seeds and statistical tests, differences such as 94.1% vs. 93.3% success or 1.2% vs. 3.0% NMAC are not statistically grounded. No code or data are provided to reproduce the environment or training, which is a central reproducibility weakness for an empirical claim.
  3. [§5.1.1, §5.1.2, §5.1.5] The entire evaluation is performed in a custom simulator (12 airways, 3 flight levels, constant-speed intruders, LoS=10 km, NMAC=0.2 km, phased spawn) that is not validated against existing benchmarks such as BlueSky or against real traffic data. All conclusions, including the 94.1%/1.2% headline numbers and the multimodal validation, are therefore confined to this unvalidated environment. Additionally, the reward specification is internally inconsistent: §3.3.1 defines α_step=0.01 and LoS/NMAC penalties of 0.5/1.0, whereas §5.1.5 states a per-step penalty of -0.001 and says timeout/boundary failure is 'typically equivalent to' an LoS penalty. This inconsistency makes the training objective non-reproducible.
  4. [§5.3.1, Table 3] The W/O Value Guidance ablation is constructed by fixing τ→∞, which makes the teacher p* uniform and trains the diffusion policy to match a uniform distribution. Unsurprisingly, the resulting policy is near-random (4.1% success, 84.7% timeout). This does not isolate the role of value guidance in the diffusion architecture; it shows that removing the teacher collapses training. The W/O Diffusion ablation is also not a direct categorical baseline. These ablations should be redesigned to separate (i) the diffusion denoiser, (ii) the joint categorical parameterization, and (iii) the teacher/critics.
minor comments (5)
  1. [Throughout] Many equations are corrupted or contain missing symbols, e.g., Eqs. (3), (14), (22), (27), and (39). The manuscript needs careful proofreading and typesetting to make the mathematical formulation readable.
  2. [§5.1.6 and Table 2] The metrics 'Average Goal Reward' and 'Average Safe Reward' appear in Table 2 but are not defined in the evaluation-index section. These should be defined precisely, ideally with formulas matching Eqs. (22)–(24).
  3. [§5.2.2, Table 2] The claim of 'approximately 59%' NMAC reduction needs a clear reference baseline. From Table 2, the next-best high-density NMAC rate is Safe-DQN-X at 3.0%, giving (3.0−1.2)/3.0 = 60%; if comparing to PPO's 6.4%, the reduction is 81%. Please specify which baseline and show the calculation.
  4. [§5.4.1] The 'success' label for a first action is obtained by running a full follow-on simulation with the learned policy. This can conflate the quality of the first action with subsequent policy behavior. Please describe how the follow-on simulation is controlled and whether the same first action is evaluated under a fixed or stochastic continuation.
  5. [§5.2.3, Fig. 6] Inference latency is reported as point values without measurement variance or the number of timing trials. Adding standard deviations would strengthen the real-time comparison.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the results are empirical evaluations in a custom simulator, and the diffusion policy is a standard actor–critic construction with no self-citation chain or definitional reduction.

full rationale

The paper's central claims are empirical: Diffusion-AC is trained and evaluated in a custom CD&R simulator, and the reported success/NMAC rates are measured outcomes, not quantities derived from equations that assume the result. The policy is a masked softmax over 27 logits produced by a denoising network (Eq. 33), trained to match a Boltzmann teacher distribution constructed from dual-Q critics (Eq. 37). This is a conventional actor–critic value-guided policy update; the teacher is an evolving Q-function, not a fixed external target, so no fitted parameter is renamed as a prediction. The ablations (W/O Diffusion, W/O Dual Q, W/O DPSC, W/O Value Guidance) compare variants of the same framework and are not circular: removing value guidance collapses the teacher to uniform and performance drops, which is an expected empirical consequence rather than a tautology. The multimodal validation (Section 5.4) uses the same simulator and intentionally complex scenarios, but it conditions on successfully executed first-step actions and compares against PPO; this is a visualization/evaluation design choice, not a logical reduction of the conclusion to the input. The paper does cite prior diffusion-policy work [37] for the expressiveness guarantee, but that is an external reference, not a self-citation, and the theorem is used as motivation rather than as the sole support for the empirical results. No load-bearing step in the derivation chain is equivalent to its own inputs by construction. The skeptic's concern that the diffusion-specific causal role is not isolated against a direct 27-way softmax baseline is a scientific attribution gap, not a circularity, and therefore does not raise the circularity score.

Assumptions & free parameters 8 free parameters · 6 assumptions · 0 invented entities

The central claims rest on a large set of hand-set simulation and training parameters. No theoretical justification is given for the curriculum schedules; they are tuned on this simulator and may not transfer.

free parameters (8)
  • Diffusion steps T = 10
    Tuned; sensitivity analysis (Fig. 7a) shows T=10 balances success and latency.
  • Noise schedule beta_t = linear from 1e-4 to 2e-2
    Chosen to match training; no derivation.
  • Teacher temperature wtemp/tau = 1.0, annealed with curriculum
    Tuned via sensitivity analysis; controls multimodality vs value concentration.
  • Soft update coefficient tau_polyak = 5e-3
    Tuned; sensitivity shows moderate value yields best reward.
  • Critic hidden size = 128
    Tuned; 128-256 best in sensitivity analysis.
  • Curriculum LoS threshold schedule = d_LoS(k)=4.5+0.5(k-1) km, k=1..12
    Tuned design choice, no theoretical basis.
  • Curriculum density schedule = R(k)=k routes, A(k)=3k intruders
    Chosen to create 12 stages; no theoretical justification.
  • Reward weights = alpha1=1, alpha2=0.5, alpha_step=0.01 (Section 3.3); per-step -0.001, LoS -0.5, NMAC -1.0 (Section 5.1.5)
    Set by hand; inconsistently specified across sections.
assumptions (6)
  • domain assumption Aircraft motion follows simplified discrete kinematics with 3 discrete controls and 3 flight levels (Eq. 2).
    Load-bearing for all simulations; real aircraft have continuous dynamics and more complex constraints.
  • domain assumption Conflict separation standards: LoS 10 km horizontal at same flight level, NMAC 0.2 km (CCAR-93TM-R6).
    These specific thresholds are adopted from operational rules and used to define the reward and termination.
  • domain assumption The environment is deterministic and Markovian (Eq. 21).
    Assumed to enable the MDP formulation; real ATM has uncertainties.
  • standard math A diffusion score estimator can approximate any target action distribution to arbitrary precision (cited to [37]).
    This is a known theoretical result, not reproved here.
  • domain assumption Intruders fly at constant speed/heading along 12 predefined airways, with up to 3 per route at different flight levels.
    Defines the traffic generator; results may not generalize to other traffic patterns.
  • ad hoc to paper The safety mask is state-local and outcome-agnostic, so it does not bias the safety comparison (Section 4.2.2).
    This is a design choice that assumptions about feasibility and does not include separation geometry.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Diffusion-RL Based Air Traffic Conflict Detection and Resolution Method." pith.science (2026). https://pith.science/paper/AAM4XFZ4

@misc{pith2026250903550,
  author       = {Pith},
  title        = {Pith review of: Diffusion-RL Based Air Traffic Conflict Detection and Resolution Method},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AAM4XFZ4}},
  note         = {Machine review of arXiv:2509.03550}
}
read the original abstract

In the context of continuously rising global air traffic, efficient and safe Conflict Detection and Resolution (CD&R) is paramount for air traffic management. Although Deep Reinforcement Learning (DRL) offers a promising pathway for CD&R automation, existing approaches commonly suffer from a "unimodal bias" in their policies. This leads to a critical lack of decision-making flexibility when confronted with complex and dynamic constraints, often resulting in "decision deadlocks." To overcome this limitation, this paper pioneers the integration of diffusion probabilistic models into the safety-critical task of CD&R, proposing a novel autonomous conflict resolution framework named Diffusion-AC. Diverging from conventional methods that converge to a single optimal solution, our framework models its policy as a reverse denoising process guided by a value function, enabling it to generate a rich, high-quality, and multimodal action distribution. This core architecture is complemented by a Density-Progressive Safety Curriculum (DPSC), a training mechanism that ensures stable and efficient learning as the agent progresses from sparse to high-density traffic environments. Extensive simulation experiments demonstrate that the proposed method significantly outperforms a suite of state-of-the-art DRL benchmarks. Most critically, in the most challenging high-density scenarios, Diffusion-AC not only maintains a high success rate of 94.1% but also reduces the incidence of Near Mid-Air Collisions (NMACs) by approximately 59% compared to the next-best-performing baseline, significantly enhancing the system's safety margin. This performance leap stems from its unique multimodal decision-making capability, which allows the agent to flexibly switch to effective alternative maneuvers.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

52 extracted references · 37 canonical work pages

  1. [1]

    2018 -2019 NextGen Implementation Plan,

    Federal Aviation Administration, “2018 -2019 NextGen Implementation Plan,” Washington DC,USA, 2019

    work page 2018

  2. [2]

    Working Document for the Aviation System Block Upgrades–The Framework for Global Harmonization,

    ICAO, “Working Document for the Aviation System Block Upgrades–The Framework for Global Harmonization,” Montreal,Canada, 2013

    work page 2013

  3. [3]

    European A TM Master Plan – The Roadmap for Delivering High Performing Aviation for Europe, Edition 2015,

    SESAR, “European A TM Master Plan – The Roadmap for Delivering High Performing Aviation for Europe, Edition 2015,” Brussels, Belgium, 2015

    work page 2015

  4. [4]

    One to Any: Distributed Conflict Resolution with Deep Multi - Agent Reinforcement Learning and Long Short -Term Memory,

    Brittain, M. W., and Wei, P., “One to Any: Distributed Conflict Resolution with Deep Multi - Agent Reinforcement Learning and Long Short -Term Memory,” AIAA Scitech 2021 Forum , American Institute of Aeronautics and Astronautics, 2021. https://doi.org/10.2514/6.2021-1952

    doi:10.2514/6.2021-1952 2021

  5. [5]

    Autonomous Air Traffic Controller: A Deep Multi -Agent Reinforcement Learning Approach

    Brittain, M., and Wei, P., “Autonomous Air Traffic Controller: A Deep Multi -Agent Reinforcement Learning Approach.” https://doi.org/10.48550/arXiv.1905.01303

    arXiv doi:10.48550/arxiv.1905.01303 1905

  6. [6]

    General Multi-Agent Reinforcement Learning Integrating Adaptive Manoeuvre Strategy for Real -Time Multi-Aircraft Conflict Resolution,

    Chen, Y ., Hu, M., Yang, L., Xu, Y ., and Xie, H., “General Multi-Agent Reinforcement Learning Integrating Adaptive Manoeuvre Strategy for Real -Time Multi-Aircraft Conflict Resolution,” Transportation Research Part C: Emerging Technologies , V ol. 151, 2023, p. 104125. https://doi.org/10.1016/j.trc.2023.104125

    work page arXiv 2023

  7. [7]

    General Real -Time Three-Dimensional Multi-Aircraft Conflict Resolution Method Using Multi -Agent Reinforcement Learning,

    Chen, Y ., Xu, Y ., Yang, L., and Hu, M., “General Real -Time Three-Dimensional Multi-Aircraft Conflict Resolution Method Using Multi -Agent Reinforcement Learning,” Transportation Research Part C: Emerging Technologies , V ol. 157, 2023, p. 104367. https://doi.org/10.1016/j.trc.2023.104367

    work page arXiv 2023

  8. [8]

    Analysis of the Impact of Traffic Density on Training of Reinforcement Learning Based Conflict Resolution Methods for Drones,

    Groot, D. J., Ellerbroek, J., and Hoekstra, J. M., “Analysis of the Impact of Traffic Density on Training of Reinforcement Learning Based Conflict Resolution Methods for Drones,” Engineering Applications of Artificial Intelligence , V ol. 133, 2024, p. 1 08066. https://doi.org/10.1016/j.engappai.2024.108066

    work page arXiv 2024

Show all 52 references
  1. [9]

    Enhancing Air Traffic Control: A Transparent Deep Reinforcement Learning Framework for Autonomous Conflict Resolution,

    Wang, L., Yang, H., Lin, Y ., Yin, S., and Wu, Y ., “Enhancing Air Traffic Control: A Transparent Deep Reinforcement Learning Framework for Autonomous Conflict Resolution,” Expert Systems with Applications, V ol. 260, 2025, p. 125389. https://doi.org/10.1016/j.eswa.2024.125389

    2025

  2. [10]

    Study on the Resolution of Multi-Aircraft Flight Conflicts Based on an IDQN,

    Sui, D., Xu, W., and Zhang, K., “Study on the Resolution of Multi-Aircraft Flight Conflicts Based on an IDQN,” Chinese Journal of Aeronautics , V ol. 35, No. 2, 2022, pp. 195 –213. https://doi.org/10.1016/j.cja.2021.03.015

    2022 doi

  3. [11]

    Graph Reinforcement Learning for Multi- Aircraft Conflict Resolution,

    Li, Y ., Zhang, Y ., Guo, T., Liu, Y ., Lv, Y ., and Du, W., “Graph Reinforcement Learning for Multi- Aircraft Conflict Resolution,” IEEE Transactions on Intelligent Vehicles, V ol. 9, No. 3, 2024, pp. 4529–4540. https://doi.org/10.1109/TIV .2024.3364652

    2024

  4. [12]

    Deep Reinforcement Learning in Service of Air Traffic Controllers to Resolve Tactical Conflicts,

    Papadopoulos, G., Bastas, A., V ouros, G. A., Crook, I., Andrienko, N., Andrienko, G., and Cordero, J. M., “Deep Reinforcement Learning in Service of Air Traffic Controllers to Resolve Tactical Conflicts,” Expert Systems with Applications , V ol. 236, 2024, p. 121234. https://...

    2024

  5. [13]

    A Review of Conflict Detection and Resolution Modeling Methods,

    Kuchar, J. K., and Yang, L. C., “A Review of Conflict Detection and Resolution Modeling Methods,” IEEE Transactions on Intelligent Transportation Systems, V ol. 1, No. 4, 2000, pp. 179–189. https://doi.org/10.1109/6979.898217

  6. [14]

    Conflict Resolution for Air Traffic Management: A Study in Multiagent Hybrid Systems,

    Tomlin, C., Pappas, G. J., and Sastry, S., “Conflict Resolution for Air Traffic Management: A Study in Multiagent Hybrid Systems,” IEEE Transactions on Automatic Control, V ol. 43, No. 4, 1998, pp. 509–521. https://doi.org/10.1109/9.664154

    1998 doi

  7. [15]

    A Time -Dependent Hamilton -Jacobi Formulation of Reachable Sets for Continuous Dynamic Games,

    Mitchell, I. M., Bayen, A. M., and Tomlin, C. J., “A Time -Dependent Hamilton -Jacobi Formulation of Reachable Sets for Continuous Dynamic Games,” IEEE Transactions on Automatic Control, V ol. 50, No. 7, 2005, pp. 947–957. https://doi.org/10.1109/TAC.2005.851439

    2005

  8. [16]

    Automated Conflict Resolution for Air Traffic Control,

    Heinz Erzberger, “Automated Conflict Resolution for Air Traffic Control,” NASA Ames Research Center, NASA Ames Research Center Moffett Field, CA, United States, 2005

    2005

  9. [17]

    Automated Conflict Resolution, Arrival Management and Weather Avoidance for ATM,

    Heinz Erzberger, Todd A Lauderdale, and Yung -Cheng Chu, “Automated Conflict Resolution, Arrival Management and Weather Avoidance for ATM,” NASA Ames Research Center, NASA Ames Research Center Moffett Field, CA, United States

  10. [18]

    Automated Conflict Resolution: A Simulation Evaluation Under High Demand Including Merging Arrivals,

    Farley, T., Kupfer, M., and Erzberger, H., “Automated Conflict Resolution: A Simulation Evaluation Under High Demand Including Merging Arrivals,” 7th AIAA ATIO Conf, 2nd CEIAT Int’l Conf on Innov and Integr in Aero Sciences,17th LTA Systems Tech Conf; foll owed by 2nd TEOS For...

    2007 doi

  11. [19]

    Conflict Resolution Problems for Air Traffic Management Systems Solved with Mixed Integer Programming,

    Pallottino, L., Feron, E. M., and Bicchi, A., “Conflict Resolution Problems for Air Traffic Management Systems Solved with Mixed Integer Programming,” Trans. Intell. Transport. Syst., V ol. 3, No. 1, 2002, pp. 3–11. https://doi.org/10.1109/6979.994791

    2002

  12. [20]

    Reciprocal N-Body Collision Avoidance,

    van den Berg, J., Guy, S., Lin, M., and Manocha, D., “Reciprocal N-Body Collision Avoidance,” Springer Tracts in Advanced Robotics , V ol. 70, 2011, pp. 3–19. https://doi.org/10.1007/978-3- 642-19457-3_1

    2011 doi

  13. [21]

    BlueSky ATC Simulator Project: An Open Data and Open Source Approach

    Hoekstra, J. M., and Ellerbroek, J., “BlueSky ATC Simulator Project: An Open Data and Open Source Approach.”

  14. [22]

    BlueSky-Gym: Reinforcement Learning Environments for Air Traffic Applications

    Groot, D. J., Leto, G., Vlaskin, A., Moec, A., and Ellerbroek, J., “BlueSky-Gym: Reinforcement Learning Environments for Air Traffic Applications.”

  15. [23]

    A Deep Multi-Agent Reinforcement Learning Approach to Autonomous Separation Assurance

    Brittain, M., Yang, X., and Wei, P., “A Deep Multi-Agent Reinforcement Learning Approach to Autonomous Separation Assurance.” https://doi.org/10.48550/arXiv.2003.08353

    2003 doi arXiv

  16. [24]

    Improving Autonomous Separation Assurance through Distributed Reinforcement Learning with Attention Networks,

    Brittain, M. W., Alvarez, L. E., and Breeden, K., “Improving Autonomous Separation Assurance through Distributed Reinforcement Learning with Attention Networks,” Proceedings of the AAAI Conference on Artificial Intelligence , V ol. 38, No. 21, 2024, p p. 22857 –22863. https://...

    2024 doi

  17. [25]

    Automating the Resolution of Flight Conflicts: Deep Reinforcement Learning in Service of Air Traffic Controllers

    V ouros, G., Papadopoulos, G., Bastas, A., Cordero, J. M., and Rodrigez, R. R., “Automating the Resolution of Flight Conflicts: Deep Reinforcement Learning in Service of Air Traffic Controllers.” https://doi.org/10.48550/arXiv.2206.07403

    doi arXiv

  18. [26]

    Conflict Resolution Strategy Based on Deep Reinforcement Learning for Air Traffic Management,

    Sui, D., Ma, C., and Dong, J., “Conflict Resolution Strategy Based on Deep Reinforcement Learning for Air Traffic Management,” Aviation, V ol. 27, No. 3, 2023, pp. 177 –186. https://doi.org/10.3846/aviation.2023.19720

    2023

  19. [27]

    Review of Deep Reinforcement Learning Approaches for Conflict Resolution in Air Traffic Control,

    Wang, Z., Pan, W., Li, H., Wang, X., and Zuo, Q., “Review of Deep Reinforcement Learning Approaches for Conflict Resolution in Air Traffic Control,” Aerospace, V ol. 9, No. 6, 2022, p

    2022

  20. [28]

    Joint Autonomous Decision- Making of Conflict Resolution and Aircraft Scheduling Based on Triple-Aspect Improved Multi- Agent Reinforcement Learning,

    Huang, X., Tian, Y ., Li, J., Zhang, N., Dong, X., Lv, Y ., and Li, Z., “Joint Autonomous Decision- Making of Conflict Resolution and Aircraft Scheduling Based on Triple-Aspect Improved Multi- Agent Reinforcement Learning,” Expert Systems with Applications , V ol. 275, 2025, p...

    2025

  21. [29]

    Toward Conflict Resolution with Deep Multi-Agent Reinforcement Learning,

    Isufaj, R., Aranega Sebastia, D., and Angel Piera, M., “Toward Conflict Resolution with Deep Multi-Agent Reinforcement Learning,” Journal of Air Transportation, V ol. 30, No. 3, 2022, pp. 71–80. https://doi.org/10.2514/1.D0296

    2022 doi

  22. [30]

    Physics Informed Deep Reinforcement Learning for Aircraft Conflict Resolution,

    Zhao, P., and Liu, Y ., “Physics Informed Deep Reinforcement Learning for Aircraft Conflict Resolution,” IEEE Transactions on Intelligent Transportation Systems, V ol. 23, No. 7, 2022, pp. 8288–8301. https://doi.org/10.1109/TITS.2021.3077572

    2022

  23. [31]

    Tactical Conflict Solver Assisting Air Traffic Controllers Using Deep Reinforcement Learning,

    Sui, D., Ma, C., and Wei, C., “Tactical Conflict Solver Assisting Air Traffic Controllers Using Deep Reinforcement Learning,” Aerospace, V ol. 10, No. 2, 2023, p. 182. https://doi.org/10.3390/aerospace10020182

    2023 doi

  24. [32]

    Self -Prioritizing Multi-Agent Reinforcement Learning for Conflict Resolution in Air Traffic Control with Limited Instructions,

    Nilsson, J., Unger, J., and Eilertsen, G., “Self -Prioritizing Multi-Agent Reinforcement Learning for Conflict Resolution in Air Traffic Control with Limited Instructions,” Aerospace, V ol. 12, No. 2, 2025, p. 88. https://doi.org/10.3390/aerospace12020088

    2025 doi

  25. [33]

    Autonomous Air Traffic Separation Assurance through Machine Learning,

    Han, Y ., and Huang, X., “Autonomous Air Traffic Separation Assurance through Machine Learning,” Journal of Industrial and Management Optimization, V ol. 20, No. 10, 2024, pp. 3195–

    2024

  26. [34]

    Autonomous Conflict Resolution in Urban Air Mobility: A Deep Multi- Agent Reinforcement Learning Approach,

    Deniz, S., and Wang, Z., “Autonomous Conflict Resolution in Urban Air Mobility: A Deep Multi- Agent Reinforcement Learning Approach,” AIAA AVIATION FORUM AND ASCEND 2024 , American Institute of Aeronautics and Astronautics, 2024. https://doi.org/10.2514/6.2024-4005

    2024 doi

  27. [35]

    An Efficient Aircraft Conflict Detection and Resolution Method Based on an Improved Reinforcement Learning Framework,

    Xu, Q., Chen, Z., Li, F., Shen, Z., and Wei, W., “An Efficient Aircraft Conflict Detection and Resolution Method Based on an Improved Reinforcement Learning Framework,” International Journal of Aerospace Engineering , V ol. 2023, 2023, pp. 1 –16. https://doi.org/10.1155/2023/6643903

    2023 doi

  28. [36]

    Autonomous Aircraft Sequencing and Separation with Hierarchical Deep Reinforcement Learning,

    Brittain, M., “Autonomous Aircraft Sequencing and Separation with Hierarchical Deep Reinforcement Learning,” 2018

    2018

  29. [37]

    Policy Representation vi a Diffusion Probability Model for Reinforcement Learning

    Yang, L., Huang, Z., Lei, F., Zhong, Y ., Yang, Y ., Fang, C., Wen, S., Zhou, B., and Lin, Z., “Policy Representation vi a Diffusion Probability Model for Reinforcement Learning.” https://doi.org/10.48550/arXiv.2305.13122

    doi arXiv

  30. [38]

    Diffusion Policies as an Expressive Policy Class for Offline Reinforcement Learning

    Wang, Z., Hunt, J. J., and Zhou, M., “Diffusion Policies as an Expressive Policy Class for Offline Reinforcement Learning.” https://doi.org/10.48550/arXiv.2208.06193

    doi arXiv

  31. [39]

    Diffusion Policy: Visuomotor Policy Learning via Action Diffusion

    Chi, C., Xu, Z., Feng, S., Cousineau, E., Du, Y ., Burchfiel, B., Tedrake, R., and Song, S., “Diffusion Policy: Visuomotor Policy Learning via Action Diffusion.” https://doi.org/10.48550/arXiv.2303.04137

    doi arXiv

  32. [40]

    Is Conditional Generative Modeling All You Need for Decision -Making?

    Ajay, A., Du, Y ., Gupta, A., Tenenbaum, J., Jaakkola, T., and Agrawal, P., “Is Conditional Generative Modeling All You Need for Decision -Making?” https://doi.org/10.48550/arXiv.2211.15657

    doi arXiv

  33. [41]

    Efficient Diffusion Policies For Off line Reinforcement Learning,

    Kang, B., Ma, X., Du, C., Pang, T., and Yan, S., “Efficient Diffusion Policies For Off line Reinforcement Learning,” V ol. 36, edited by A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, 2023, pp. 67195–67212

    2023

  34. [42]

    Policy-Guided Diffusion

    Jackson, M. T., Matthews, M. T., Lu, C., Ellis, B., Whiteson, S., and Foerster, J., “Policy-Guided Diffusion.” https://doi.org/10.48550/arXiv.2404.06356

    doi arXiv

  35. [43]

    Diffusion-Based Reinforcement Learning via Q -Weighted Variational Policy Optimization

    Ding, S., Hu, K., Zhang, Z., Ren, K., Zhang, W., Yu, J., Wang, J., and Shi, Y ., “Diffusion-Based Reinforcement Learning via Q -Weighted Variational Policy Optimization.” https://doi.org/10.48550/arXiv.2405.16173

    doi arXiv

  36. [44]

    DreamFuser: Value -Guided Diffusion Policy for Offline Reinforcement Learning,

    Luo, K., XIAO, C., Huang, Z., Ling, Z., Fang, Y ., and Su, H., “DreamFuser: Value -Guided Diffusion Policy for Offline Reinforcement Learning,” 2024

    2024

  37. [45]

    ReDiffuser: Reliable Decision -Making Using a Diffuserwith Confidence Estimation

    He, N., Li, S., Li, Z., Liu, Y ., and He, Y ., “ReDiffuser: Reliable Decision -Making Using a Diffuserwith Confidence Estimation.”

  38. [46]

    Diffusion Models for Reinforcement Learning: A Survey,

    Zhu, Z., Zhao, H., He, H., Zhong, Y ., Zhang, S., Yu, Y ., and Zhang, W., “Diffusion Models for Reinforcement Learning: A Survey,” arXiv preprint arXiv:2311.01223, 2023

    2023 arXiv

  39. [47]

    Is Exploration All You Need? Effective Exploration Characteristics for Transfer in Reinforcement Learning

    Balloch, J. C., Bhagat, R., Zollicoffer, G., Jia, R., Kim, J., and Riedl, M. O., “Is Exploration All You Need? Effective Exploration Characteristics for Transfer in Reinforcement Learning.” https://doi.org/10.48550/arXiv.2404.02235

    doi arXiv

  40. [48]

    A Survey of Safe Reinforcement Learning and Constrained MDPs: A Technical Survey on Single -Agent and Multi -Agent Safety

    Kushwaha, A., Ravish, K., Lamba, P., and Kumar, P., “A Survey of Safe Reinforcement Learning and Constrained MDPs: A Technical Survey on Single -Agent and Multi -Agent Safety.” https://doi.org/10.48550/arXiv.2505.17342

    doi arXiv

  41. [49]

    Proximal Policy Optimization Algorithms

    Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O., “Proximal Policy Optimization Algorithms.” https://doi.org/10.48550/arXiv.1707.06347

    doi arXiv

  42. [50]

    Addressing Function Approximation Error in Actor- Critic Methods

    Fujimoto, S., Hoof, H. van, and Meger, D., “Addressing Function Approximation Error in Actor- Critic Methods.” https://doi.org/10.48550/arXiv.1802.09477

    doi arXiv

  43. [294]

    https://doi.org/10.3390/aerospace9060294

    doi

  44. [3204]

    https://doi.org/10.3934/jimo.2024050

    doi

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.