REVIEW 5 major objections 6 minor 1 cited by
Curriculum-Adapted Robust Reinforcement Learning for UAV Deconfliction in Adversarial Environments
T0 review · 5 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read Aligning TD-error distributions across an adversarial curriculum yields a generalization certificate: attacks whose TD-error distribution stays close to the final stage suffer bounded loss, so robustness transfers to black-box GNSS…
desk verdict The TD-space curriculum idea has legs and the simulations are worth taking seriously, but the certificate is built on a false reverse pushforward inequality, so the central claim as stated does not follow. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The machinery is a curriculum of adversarial observations acting as an interpolation path in TD-error distribution space. Adversarial states are generated by a multi-step projected-gradient-ascent procedure (Algorithm 1) with a perturbation budget $\varepsilon$ that increases stage by stage, and at each stage the adapting critic is trained by minimizing the Wasserstein-1 distance $W_1(\mathrm{TD}^{\mathrm{exp}}, \mathrm{TD}^{\mathrm{AR}})$ between the expert critic's TD-error distribution and its own (Algorithm 3); the expert critic is carried forward from the previous stage, and the curriculum stops when the TD-error gap between successive stages falls below a threshold. Catastrophic forgetting is formalized as exactly this distance, $f^\varepsilon_\pi = \inf_\pi W_1(\mathrm{TD}^{\mathrm{DDPG}}, \mathrm{TD}^\varepsilon_\pi)$. The theoretical chain that produces the certificate is: Lipschitz assumptions on rewards and value functions (Assumptions 1–2), Lemmas 1–2 bounding value-function shifts, Lemma 3's reverse pushforward inequality $W_1(P_{k+1}, P_k) \leq L_T W_1(\mathrm{TD}_{k+1}, \mathrm{TD}_k)$, and Lemma 4 bounding the adversarial-state distance, together yielding Theorem 1 (bounded TD drift) and Theorem 2 (the test-time certificate with $\mathrm{Rad}_K$ and $\delta_A$).
What would settle it
During curriculum training, record paired samples of consecutive transition distributions and their TD-error distributions and check whether $W_1(P_{k+1}, P_k) \leq L_T\,W_1(\mathrm{TD}_{k+1}, \mathrm{TD}_k)$ actually holds at every stage with a bounded constant; alternatively, sweep candidate attacks and measure $\delta_A = W_1(\mathrm{TD}(A), \mathrm{TD}(\varepsilon_K))$ against realized mission success, looking for an attack with near-zero $\delta_A$ and a large success drop. Either observation would directly contradict the certificate.
Extended reading notes
Core claim
In the paper's own terms, the discovery is that adversarially induced value-function drift, quantified as the 1-Wasserstein distance between temporal-difference (TD) error distributions, is the quantity to control if robustness is to transfer across attack domains. The authors claim in Theorem 2 that the curriculum-adapted policy $\pi^\ast_K$ satisfies, with probability $1-\delta$ over $n$ episodes, a bound of the form $\bigl| E[R(\pi^\ast_K \text{ under } A)] - \tfrac{1}{n}\sum_{i=1}^n R(\pi^\ast_K) \bigr| \leq \mathrm{Rad}_K + C\,\delta_A + O\!\bigl(\tfrac{c}{\sqrt{n}}\sqrt{\log(1/\delta)}\bigr)$, where $\delta_A = W_1\bigl(\mathrm{TD}_{\pi^\ast_K}(A), \mathrm{TD}_{\pi^\ast_K}(\varepsilon_K)\bigr)$ is the distance between the attack-induced TD-error distribution and that of the final curriculum stage, $\mathrm{Rad}_K$ is a Rademacher-complexity term, and $C$ is a problem-dependent constant. A companion bound against the original expert adds the cumulative curriculum drift $\sum_k \beta_k$, tying stagewise TD alignment to limited error propagation. The paper further claims that Wasserstein-1 is the right metric for this certificate because it stays finite on non-overlapping supports, where W2 becomes tail-sensitive and KL diverges. Theorem 1 asserts the stagewise engine of the bound: TD-error drift across consecutive curriculum stages is controlled by $m\,W_1\bigl(\mathrm{TD}_{\pi^\ast_{k+1}}(\varepsilon_{k+1}), \mathrm{TD}_{\pi^\ast_k}(\varepsilon_k)\bigr)$ with $m = L_V L_T (1+\gamma)$, and the simulation results are presented as evidence that adapting on synthetic gradient perturbations generalizes to physically distinct GNSS spoofing attacks.
Load-bearing premise
The certificate stands on Lemma 3's reverse pushforward inequality, which claims the distance between successive environments' next-state distributions is bounded by a constant times the distance between their TD-error distributions; the standard pushforward inequality runs the opposite direction, and pushforwards can erase information, so if this reverse bound fails in the target environment the Theorem 2 guarantee collapses, as do the two heuristic approximations used in Lemma 4.
Editorial extensions
If this is right
- Robustness acquired under synthetic gradient perturbations is not tied to a particular attack family: any unseen attack whose induced TD-error distribution lies within $\delta_A$ of the final curriculum stage inherits the bounded-degradation guarantee without retraining.
- Under a fixed GNSS spoofing attack the curriculum-adapted policy reaches near-perfect mission success, while DDPG and the robust RL baselines (PR-MDP, NR-MDP, adversarial meta-learning, AutoLiRPA) fall to 20–56% success.
- Under dynamic obstacle-luring spoofing the adapted policy keeps the highest episodic rewards and completes missions in up to 45% fewer steps across all tested traffic densities.
- The cumulative curriculum drift $\sum_k \beta_k$ is controlled by design, so error does not accumulate from early adaptation stages into the final policy.
- The convergence property behind Theorem 1 keeps the TD-error distribution bounded throughout adaptation, which is the paper's operational definition of mitigating catastrophic forgetting.
Reading between the lines
- A deployment-time monitor could compute $\delta_A$ online from a rolling buffer of TD errors and raise an alert when the running estimate exceeds the certified radius; the paper does not develop this monitoring use, but the certificate's structure invites it.
- Because the certificate is agnostic to the observation-space mechanism, the same TD-space curriculum could plausibly transfer to other falsified-sensor attacks such as ADS-B spoofing or inertial drift, an extension the paper does not test.
- The sensitivity analysis, which shows smaller curriculum increments ($\Delta\varepsilon = 0.25$) generalize better than coarse ones, suggests a testable monotonicity: unseen-attack generalization should improve as the stagewise Wasserstein drift is driven toward zero.
- If Lemma 3's reverse inequality proves false in some environment, the framework would need explicit regularization of the TD-error map (for example, Lipschitz or invertibility constraints) rather than relying on the inequality as an assumed property.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a curriculum-guided adversarial adaptation framework for robust reinforcement learning in UAV deconfliction. Starting from an action-robust critic trained via a two-player game with SGLD, the method generates adversarial state observations with an increasing perturbation budget ε (multi-step projected gradient ascent, Algorithm 1) and, at each curriculum stage, updates the critic to minimize the Wasserstein-1 distance between the TD-error distribution under the current adversarial observations and that of the previous stage's expert critic (Algorithm 3). The main theoretical contribution is a 'TD-space generalization certificate' (Theorem 2): with probability 1−δ over n evaluation episodes, the performance gap between the adapted policy under a previously unseen attack A and the empirical return under the final curriculum stage is bounded by a Rademacher term plus C·δ_A, where δ_A is the W1 distance between the TD-error distribution induced by A and that of the final stage. The certificate is built from Theorem 1 and Lemmas 1–4. The framework is evaluated in a simulated 3D UAV deconfliction environment under unseen fixed constant-bias GNSS spoofing and dynamic obstacle-luring spoofing attacks, reporting near-perfect mission success under fixed spoofing (vs. 20–56% for baselines), higher rewards, and up to 45% fewer completion steps under dynamic spoofing.
Significance. If the certificate were valid, this would be a valuable contribution: it would provide a principled, attack-agnostic condition under which robustness acquired from synthetic gradient-based perturbations transfers to black-box sensor-level attacks, a practically important problem for cyber-physical RL. The paper has genuine strengths: the adaptation procedure is specified in detail (Algorithms 1–3); the threat models in Section IV are realistic (constant code-phase bias and stealth-constrained gradual carry-off spoofing); results are reported as distributions over 100 test episodes; and Section V.C includes an honest limitation statement (the authors refrain from claiming generalization beyond ε = 4.0). The idea of quantifying catastrophic forgetting as W1 drift of TD-error distributions is reasonable and clearly motivated. However, the central theoretical claim is not supported: Lemma 3 (Eq.
major comments (5)
- [III, Lemma 3 (Eq. 21); proof in Appendix D.C] Lemma 3 asserts W1(P_{k+1}, P_k) ≤ L_T W1(TD^{π*}_{k+1}(ε_{k+1}), TD^{π*}_k(ε_k)), and the proof (Appendix D.C, Eqs. D.8–D.10) invokes 'a general property of optimal transport theory' of the form W1(μ,ν) ≤ L_T W1(T_#μ, T_#ν) for Lipschitz T. The standard property is the forward inequality W1(T_#μ, T_#ν) ≤ Lip(T) W1(μ,ν); the reverse direction requires T to be metrically injective (e.g., to admit a Lipschitz right-inverse), which is not assumed in Assumptions 1–2. Since the TD error is a scalar functional of high-dimensional state-action transitions, the map is many-to-one in general: taking T ≡ 0 gives W1(T_#μ, T_#ν) = 0 while W1(μ,ν) can be arbitrarily large. The failure is load-bearing: Theorem 1 uses Lemma 3 at Eq. (28) and Theorem 2 uses it again at Eq. (D.26) to replace W1(P_A, P_K) by δ_A, so the C·δ_A term in Eqs. (30)–(31) does not follow and the certificate is unsupported.
- [III, Theorem 1 (Eq. 23) and its proof (Eqs. 24–29)] The statement of Theorem 1 bounds E[TD^{π*}_{k+1}(ε_{k+1}) − TD^{π*}_k(ε_{k+1})], the difference between two policies evaluated at the same attack level ε_{k+1}, while the proof (Eq. 24) computes E[TD^{π*}_{k+1}(ε_{k+1}) − TD^{π*}_{k+1}(ε_k)], the same policy evaluated at two attack levels; the quantities differ in both the policy index and the perturbation level. The proof also introduces an absolute value at Eq. (27) that is absent from the statement. Hence Theorem 1 is not proved as stated, and the subsequent use in Appendix D.V (attributing the bound on the 'policy change term' at a fixed attack level to Theorem 1) is not justified by the demonstrated chain.
- [III, Lemma 4 (Eq. 22); proof in Appendix D.D] Lemma 4's proof rests on two approximations that the authors themselves flag with '≈': E[f(Φ'_k)] ≈ 0 and E[f(Φ'_{k+1})] ≈ d(Φ_{ε_{k+1}}, Φ_{ε_k}). These are not inequalities, and the first is not obviously true because Φ'_k ~ P_k is the transitioned state from an adversarially perturbed state, which can be far from Φ_{ε_k} even when the dynamics is smooth. Lemma 4 provides the bound d(Φ_{ε_{k+1}}, Φ_{ε_k}) ≤ W1(P_{k+1}, P_k) used at Eq. (26) of Theorem 1's proof to obtain Eq. (27), so the stagewise bound and the constant m are not established even before Lemma 3 is invoked.
- [Appendix D.V (proof of Theorem 2); Lemma 1 (Eq. 19) and its proof in Appendix D.I] Two gaps invalidate the stagewise bound in Eq. (D.32). First, the 'policy change term' in Eq. (D.31) is a return difference E[R(π*_{k+1}, ε_{k+1})] − E[R(π*_k, ε_{k+1})], whereas Theorem 1 bounds an expectation of TD-error differences; no performance-difference lemma or Lipschitz relation between returns and TD-error expectations is supplied, so the claim that this term is 'bounded by m W1(...) = m β_k' does not follow. Second, Lemma 1 compares expectations of two different value functions Q^{π*}_{k+1} and Q^{π*}_k, but the Kantorovich–Rubinstein step at Eq. (D.2) replaces both by a single Lipschitz function, thereby dropping the term E_{P_{k+1}}[Q_{k+1} − Q_k], which need not vanish between curriculum stages; Lemma 1 is valid only for a fixed critic, not in the form used here.
- [V.D (Robustness Generalization against Unseen Spoofing Attacks)] The paper states that Theorem 2 applies to the unseen attacks 'whenever this proximity is bounded' (δ_A), but δ_A is never computed for the fixed or dynamic GNSS spoofing attacks used in the experiments, and no estimator for δ_A from test trajectories is described. The certificate's hypothesis (small W1 distance between the attack-induced TD-error distribution and that of the final curriculum stage) is therefore not verified, so the results in Figures 8–13 neither instantiate nor test Theorem 2. Moreover, since Algorithm 3 (line 20) directly minimizes W1 between consecutive TD-error distributions, the certificate is largely a consistency statement between this objective and the return gap in the same metric, rather than an independent guarantee of transfer across attack mechanisms. The paper's claim (Section VI) that the framework provides a formal explanation of the observed transfer is consequently not supported by the data.
minor comments (6)
- [III, Eqs. (13)–(15) and Algorithm 2] The TD error is defined in Eqs. (14)–(15) as a product of Gaussian densities, while Algorithm 2 (step 8) computes a scalar δ²/σ² with δ = V_tar − V_adv; please clarify how the scalar TD errors, the density products, and the distributions whose W1 distance is minimized in Algorithm 3 (line 20) are related, and define σ.
- [Throughout] The manuscript contains many typos and ligature artifacts ('sufficiently', 'traffic', 'efficiency', 'behaivour', 'decondliction', 'Nnformation' in Ref. [11]); a thorough copy-edit is needed.
- [V (figures)] In the version under review, several figures (notably Figures 8–12) contain garbled characters in their labels; please regenerate the figures and verify that all text is legible in the final PDF.
- [V.B–V.D] Quantitative claims (near-perfect mission success vs. 20–56% for baselines; up to 45% step reduction) are supported only by boxplots and density plots; please add tables with means, standard deviations, and the number of training seeds for each policy to substantiate these comparisons.
- [III, Theorem 2 (Eq. 30)] Please state explicitly the probability space for the 1−δ statement (which sources of randomness are covered) and align the constant in the O(c/√n √(log(1/δ))) term with the bound in [44].
- [V.C] The text states that the curriculum regime is bounded at ε ≤ 4.0, but also reports stable forgetting at ε = 5.0 and an ablation comparing stopping conditions ε = 3.0, 4.0, 5.0 (Section V.E); please reconcile these statements and state the final ε_max used for the evaluated policies.
Circularity Check
The TD-space generalization certificate reduces to an assumed reverse-pushforward inequality, and the reported forgetting metric is the training loss.
-
other
[Appendix D.C, Lemma 3 (Eq. D.8–D.10); used at Theorem 1 Eq. (28) and Theorem 2 Appendix D.V Eq. (D.26)]
"If we consider a general property of the optimal transport theory, then we can write if T is a Lipschitz function from a metric space X→R, and μ, ν∈P(X) are probability measures on that space, then, W1(μ,ν) ≤ L_T W1(T_# μ, T_# ν) ... Hence, the above relation becomes, W1(P_{k+1}, P_k) ≤ L_T W1(TD_{π*_{k+1}}(ε_{k+1}), TD_{π*_k}(ε_k))."
The standard pushforward bound is W1(T#μ,T#ν) ≤ Lip(T)W1(μ,ν); the reverse inequality holds only for injective or metric-embedding maps. The paper neither proves nor assumes injectivity of the TD-error map, yet this reverse inequality is exactly what Theorem 1 (Eq. 28) and Theorem 2 (Eq. D.26) need to replace W1(P_A,P_K) with δ_A = W1(TD_A,TD_K). Thus the certificate's unique content — that TD-space proximity controls performance — is not derived from Assumptions 1–2; it is asserted in Lemma 3 and then restated as the bound. The 'prediction' of bounded degradation under unseen attacks is, by construction, the same Wasserstein distance that the curriculum loss (Algorithm 3, Lcrit = W1(TDexp, TDAR)) minimizes.
-
fitted input called prediction
[Section V.C, Eq. (18), and Algorithm 3 line 20]
"Update the critic by computing the following distributional loss Lcrit = 1/N * W1(TDexp, TDAR) ... The forgetting value f^ε_π of the curriculum adapted policy is consistently lower than that of the action robust framework with α = 0.1."
Equation (18) defines catastrophic forgetting f^ε_π as the Wasserstein-1 distance between TD-error distributions, and Algorithm 3 explicitly minimizes W1(TDexp, TDAR) as the adaptation loss. Reporting that f^ε_π decreases after curriculum adaptation is therefore reporting the optimized objective itself, not an independent out-of-sample metric. This supports the adaptation scheme but does not by itself evidence transfer; the external spoofing success rates are the independent evidence.
full rationale
The central claimed theoretical contribution is Theorem 2: if the test-time TD-error distribution is δ_A-close in Wasserstein distance to the final curriculum stage, then the performance gap is bounded by a Rademacher term plus C·δ_A. This is a conditional statement in the same metric that the training loop minimizes, so the real content is the proof that TD-distance controls return. That proof rests on Lemma 3, which asserts the reverse pushforward inequality W1(P_{k+1},P_k) ≤ L_T W1(TD_{k+1},TD_k) as a 'general property of optimal transport theory.' The standard property is the forward inequality; the reverse requires injectivity or a Lipschitz right-inverse, which is not assumed. Without Lemma 3, the δ_A term in Theorem 2 does not bound W1(P_A,P_K), and the certificate collapses. This is a load-bearing assumption disguised as a derived lemma, making the certificate a restatement of its own input rather than a first-principles result. Separately, the reported forgetting-value improvement is measured with the same Wasserstein TD-distance that Algorithm 3 minimizes, so that portion of the evaluation is partly the training loss. The paper does include external validation — mission success and reward under previously unseen fixed and dynamic GNSS spoofing attacks — which is genuinely independent of the TD-space training objective, and the self-citations are not load-bearing for the certificate. The overall circularity is therefore partial: the theoretical certificate reduces to an assumed inequality, while the empirical transfer claim has independent content.
Assumptions & free parameters
free parameters (5)
- alpha (action-robust mixing weight) =
0.1
- Curriculum stopping perturbation ε_max =
4.0
- Curriculum step size Δε =
0.25
- Reward weights λ_1, λ_2, λ_3 and thresholds =
not reported
- Lipschitz constants L_Q, L_T, L_V, m =
not computed
assumptions (6)
- domain assumption Reward function decreases under stronger adversarial perturbation (Assumption 1)
- domain assumption Value function is Lipschitz continuous in adversarial and transition states (Assumption 2)
- ad hoc to paper Reverse pushforward inequality for TD-error map (Lemma 3)
- ad hoc to paper Smooth dynamics approximations in Lemma 4
- standard math Rademacher complexity bound for reparameterizable policies
- domain assumption Environment dynamics from interfered fluid dynamics
Cite this review
Pith. "Pith review of Curriculum-Adapted Robust Reinforcement Learning for UAV Deconfliction in Adversarial Environments." pith.science (2026). https://pith.science/paper/EYZSECAW
@misc{pith2026250621129,
author = {Pith},
title = {Pith review of: Curriculum-Adapted Robust Reinforcement Learning for UAV Deconfliction in Adversarial Environments},
year = {2026},
howpublished = {\url{https://pith.science/paper/EYZSECAW}},
note = {Machine review of arXiv:2506.21129}
}
read the original abstract
Autonomous unmanned aerial vehicles (UAVs) increasingly rely on reinforcement learning (RL) for navigation. However, global navigation satellite system (GNSS) spoofing attacks can induce out-of-distribution observation shifts that corrupt value estimation and degrade mission performance. Existing robust RL approaches typically improve resilience against specific attack models but often fail to generalize to attacks not encountered during training. To address this limitation, we propose a curriculum-guided adaptation framework that progressively exposes a robust policy to gradient-based adversarial observation perturbations of increasing intensity while aligning temporal-difference (TD) error distributions across curriculum stages. Rather than adapting to a particular attack model, the proposed approach preserves TD-error consistency to promote transferability across attack conditions. We further derive a TD-space generalization certificate showing that if the TD-error distribution induced by a test-time attack remains sufficiently close to that of the final curriculum stage, the resulting performance degradation is bounded. The framework is evaluated in a UAV deconfliction environment with dynamic 3D obstacles under previously unseen fixed and dynamic GNSS spoofing attacks. Under fixed spoofing conditions, the curriculum-adapted policy achieved near-perfect mission success rates, compared with 20-56% for standard and robust RL baselines. Under dynamic obstacle-luring spoofing attacks, it achieved the highest episodic rewards while reducing mission completion steps by up to 45% across increasing aerial traffic densities.
Figures
Figures from the paper (14 more)
Forward citations
Cited by 1 Pith paper
-
Real-Time Bayesian Detection of Drift-Evasive GNSS Spoofing in Reinforcement Learning Based UAV Deconfliction
A Bayesian change point detector run on RL critic value estimates detects gradual GNSS spoofing of a simulated UAV with higher reported accuracy and fewer false alarms than Page-Hinkley, an LSTM autoencoder, and signa...
Reference graph
Works this paper leans on
-
[1]
Uncovering drone intentions using control physics informed machine learning,
A. Perrusquía, W. Guo, B. Fraser, and Z. Wei, “Uncovering drone intentions using control physics informed machine learning,” Communications Engineering, vol. 3, no. 1, p. 36, 2024
work page 2024
-
[2]
A survey on reinforcement learning in aviation applications,
P. Razzaghi et al., “A survey on reinforcement learning in aviation applications,” Engineering Applications of Artificial Intelligence, vol. 136, p. 108 911, 2024
work page 2024
-
[3]
Ads-b vulnerabilities and a security solution with a timestamp,
Y. Kim, J.-Y. Jo, and S. Lee, “Ads-b vulnerabilities and a security solution with a timestamp,” IEEE Aerospace and Electronic Systems Magazine, vol. 32, no. 11, pp. 52–61, 2017
work page 2017
-
[4]
Detecting ads-b spoofing attacks using deep neural networks,
X. Ying, J. Mazer, G. Bernieri, M. Conti, L. Bushnell, and R. Poovendran, “Detecting ads-b spoofing attacks using deep neural networks,” in 2019 IEEE conference on communica- tions and network security (CNS), IEEE, 2019, pp. 187–195
work page 2019
-
[5]
D. K. Panda and W. Guo, “Action robust reinforcement learning for air mobility deconfliction against conflict induced spoofing,” IEEE Transactions on Intelligent Transportation Systems, vol. 25, no. 12, pp. 21 343–21 355, 2024
work page 2024
-
[6]
Challenges and countermeasures for adversarial attacks on deep reinforcement learning,
I. Ilahi et al., “Challenges and countermeasures for adversarial attacks on deep reinforcement learning,” IEEE Transactions on Artificial Intelligence, vol. 3, no. 2, pp. 90–109, 2021
work page 2021
-
[7]
Ads-b jamming mitigation: A solution based on a multichannel receiver,
M. Leonardi, E. Piracci, and G. Galati, “Ads-b jamming mitigation: A solution based on a multichannel receiver,” IEEE Aerospace and Electronic Systems Magazine, vol. 32, no. 11, pp. 44–51, 2017
work page 2017
-
[8]
L. Shi and Y. Chi, “Distributionally robust model-based offline reinforcement learning with near-optimal sample com- plexity,” Journal of Machine Learning Research, vol. 25, no. 200, pp. 1–91, 2024
work page 2024
Show all 52 references
-
[9]
Action robust rein- forcement learning and applications in continuous control,
C. Tessler, Y. Efroni, and S. Mannor, “Action robust rein- forcement learning and applications in continuous control,” in International Conference on Machine Learning, PMLR, 2019, pp. 6215–6224
2019
-
[10]
Autonomous option in- vention for continual hierarchical reinforcement learning and planning,
R. K. Nayyar and S. Srivastava, “Autonomous option in- vention for continual hierarchical reinforcement learning and planning,” in Proceedings of the AAAI Conference on Artifi- cial Intelligence, vol. 39, 2025, pp. 19 642–19 650
2025
-
[11]
Lifelong domain adaptation via consolidated internal distribution,
M. Rostami, “Lifelong domain adaptation via consolidated internal distribution,” Advances in Neural Nnformation Pro- cessing Systems, vol. 34, pp. 11 172–11 183, 2021
2021
-
[12]
Curriculum reinforcement learning using optimal transport via gradual domain adaptation,
P. Huang, M. Xu, J. Zhu, L. Shi, F. Fang, and D. Zhao, “Curriculum reinforcement learning using optimal transport via gradual domain adaptation,” Advances in Neural Infor- mation Processing Systems, vol. 35, pp. 10 656–10 670, 2022
2022
-
[13]
Curriculum reinforcement learning from avoiding collisions to navigating among movable obstacles in diverse environments,
H.-C. Wang et al., “Curriculum reinforcement learning from avoiding collisions to navigating among movable obstacles in diverse environments,” IEEE Robotics and Automation Letters, vol. 8, no. 5, pp. 2740–2747, 2023
2023
-
[14]
A sensor fusion-based gnss spoofing attack detection framework for autonomous vehicles,
S. Dasgupta, M. Rahman, M. Islam, and M. Chowdhury, “A sensor fusion-based gnss spoofing attack detection framework for autonomous vehicles,” IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 12, pp. 23 559–23 572, 2022
2022
-
[15]
Gnss spoofing and detection,
M. L. Psiaki and T. E. Humphreys, “Gnss spoofing and detection,” Proceedings of the IEEE, vol. 104, no. 6, pp. 1258– 1270, 2016
2016
-
[16]
Robust deep reinforcement learning against adversarial perturbations on state observations,
H. Zhang et al., “Robust deep reinforcement learning against adversarial perturbations on state observations,” Advances in Neural Information Processing Systems, vol. 33, pp. 21 024– 21 037, 2020. JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 15
2020
-
[17]
Distributionally robust reinforcement learning with interactive data collec- tion: Fundamental hardness and near-optimal algorithms,
M. Lu, H. Zhong, T. Zhang, and J. Blanchet, “Distributionally robust reinforcement learning with interactive data collec- tion: Fundamental hardness and near-optimal algorithms,” Advances in Neural Information Processing Systems, vol. 37, pp. 12 528–12 580, 2024
2024
-
[18]
Near-optimal distributionally robust reinforce- ment learning with general L_p norms,
P. Clavier, L. Shi, E. Le Pennec, E. Mazumdar, A. Wierman, and M. Geist, “Near-optimal distributionally robust reinforce- ment learning with general L_p norms,” Advances in Neural Information Processing Systems, vol. 37, pp. 1750–1810, 2024
2024
-
[19]
On corruption- robustness in performative reinforcement learning,
V. Pollatos, D. Mandal, and G. Radanovic, “On corruption- robustness in performative reinforcement learning,” in Pro- ceedings of the AAAI Conference on Artificial Intelligence, vol. 39, 2025, pp. 19 939–19 947
2025
-
[20]
On reinforcement learning and distribution matching for fine-tuning language models with no catastrophic forgetting,
T. Korbak, H. Elsahar, G. Kruszewski, and M. Dymetman, “On reinforcement learning and distribution matching for fine-tuning language models with no catastrophic forgetting,” Advances in Neural Information Processing Systems, vol. 35, pp. 16 203–16 220, 2022
2022
-
[21]
Re- thinking the foundations for continual reinforcement learn- ing,
E. Elelimy, D. Szepesvari, M. White, and M. Bowling, “Re- thinking the foundations for continual reinforcement learn- ing,” arXiv preprint arXiv:2504.08161, 2025
2025 arXiv
-
[22]
Progressive neural networks,
A. A. Rusu et al., “Progressive neural networks,” arXiv preprint arXiv:1606.04671, 2016
2016 arXiv
-
[23]
Towards continual reinforcement learning: A review and perspectives,
K. Khetarpal, M. Riemer, I. Rish, and D. Precup, “Towards continual reinforcement learning: A review and perspectives,” Journal of Artificial Intelligence Research, vol. 75, pp. 1401– 1476, 2022
2022
-
[24]
Rostami, Transfer learning through embedding spaces
M. Rostami, Transfer learning through embedding spaces. CRC Press, 2021
2021
-
[25]
Forget me not: Reducing catastrophic forgetting for domain adaptation in reading comprehension,
Y. Xu, X. Zhong, A. J. J. Yepes, and J. H. Lau, “Forget me not: Reducing catastrophic forgetting for domain adaptation in reading comprehension,” in 2020 International joint con- ference on neural networks (IJCNN), IEEE, 2020, pp. 1–8
2020
-
[26]
Self-composing policies for scalable continual reinforcement learning,
M. Malagon, J. Ceberio, and J. A. Lozano, “Self-composing policies for scalable continual reinforcement learning,” arXiv preprint arXiv:2506.14811, 2025
2025 arXiv
-
[27]
On the benefit of optimal transport for curriculum reinforcement learning,
P. Klink, C. D’Eramo, J. Peters, and J. Pajarinen, “On the benefit of optimal transport for curriculum reinforcement learning,” IEEE Transactions on Pattern Analysis and Ma- chine Intelligence, vol. 46, no. 11, pp. 7191–7204, 2024
2024
-
[28]
Unsupervised domain adap- tation by backpropagation,
Y. Ganin and V. Lempitsky, “Unsupervised domain adap- tation by backpropagation,” in International conference on machine learning, PMLR, 2015, pp. 1180–1189
2015
-
[29]
Generate to adapt: Aligning domains using generative adversarial networks,
S. Sankaranarayanan, Y. Balaji, C. D. Castillo, and R. Chel- lappa, “Generate to adapt: Aligning domains using generative adversarial networks,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 8503– 8512
2018
-
[30]
Conditional ad- versarial domain adaptation,
M. Long, Z. Cao, J. Wang, and M. I. Jordan, “Conditional ad- versarial domain adaptation,” Advances in neural information processing systems, vol. 31, 2018
2018
-
[31]
Domain adaptation in reinforcement learning via latent unified state representation,
J. Xing, T. Nagata, K. Chen, X. Zou, E. Neftci, and J. L. Krichmar, “Domain adaptation in reinforcement learning via latent unified state representation,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, 2021, pp. 10 452–10 459
2021
-
[32]
Adaptive interfered fluid dy- namic system algorithm based on deep reinforcement learning framework,
Y. Zhang and H. Wang, “Adaptive interfered fluid dy- namic system algorithm based on deep reinforcement learning framework,” in International Conference on Autonomous Un- manned Systems, Springer, 2021, pp. 1388–1397
2021
-
[33]
Sense and avoid considerations for safe suas operations in urban environments,
B. Ince, V. C. Martinez, P. K. Selvam, I. Petrunin, M. Seo, and A. Tsourdos, “Sense and avoid considerations for safe suas operations in urban environments,” IEEE Aerospace and Electronic Systems Magazine, 2024
2024
-
[34]
Industrial uav-based unsupervised domain adaptive crack recognitions: From database towards real-site infrastructural inspections,
K. Liu and B. M. Chen, “Industrial uav-based unsupervised domain adaptive crack recognitions: From database towards real-site infrastructural inspections,” IEEE Transactions on Industrial Electronics, vol. 70, no. 9, pp. 9410–9420, 2022
2022
-
[35]
Decentralized autonomous navigation of a uav network for road traffic monitoring,
H. Huang, A. V. Savkin, and C. Huang, “Decentralized autonomous navigation of a uav network for road traffic monitoring,” IEEE Transactions on Aerospace and Electronic Systems, vol. 57, no. 4, pp. 2558–2564, 2021
2021
-
[36]
Robust reinforcement learning via adversarial training with langevin dynamics,
P. Kamalaruban, Y.-T. Huang, Y.-P. Hsieh, P. Rolland, C. Shi, and V. Cevher, “Robust reinforcement learning via adversarial training with langevin dynamics,” Advances in Neural Information Processing Systems, vol. 33, pp. 8127– 8138, 2020
2020
-
[37]
Finding mixed nash equilibria of generative adversarial networks,
Y.-P. Hsieh, C. Liu, and V. Cevher, “Finding mixed nash equilibria of generative adversarial networks,” in International Conference on Machine Learning, PMLR, 2019, pp. 2810– 2819
2019
-
[38]
Towards evaluating the robustness of neural networks,
N. Carlini and D. Wagner, “Towards evaluating the robustness of neural networks,” in 2017 IEEE Symposium on Security and Privacy, Ieee, 2017, pp. 39–57
2017
-
[39]
A dirichlet process mixture of robust task models for scalable lifelong reinforcement learning,
Z. Wang, C. Chen, and D. Dong, “A dirichlet process mixture of robust task models for scalable lifelong reinforcement learning,” IEEE Transactions on Cybernetics, vol. 53, no. 12, pp. 7509–7520, 2022
2022
-
[40]
Bisimulation metrics for continuous markov decision processes,
N. Ferns, P. Panangaden, and D. Precup, “Bisimulation metrics for continuous markov decision processes,” SIAM Journal on Computing, vol. 40, no. 6, pp. 1662–1714, 2011
2011
-
[41]
Scalable methods for computing state similarity in deterministic markov decision processes,
P. S. Castro, “Scalable methods for computing state similarity in deterministic markov decision processes,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, 2020, pp. 10 069–10 076
2020
-
[42]
Wasserstein gan with quadratic transport cost,
H. Liu, X. Gu, and D. Samaras, “Wasserstein gan with quadratic transport cost,” in Proceedings of the IEEE/CVF international conference on computer vision, 2019, pp. 4832– 4841
2019
-
[43]
Distributionally robust learning,
R. Chen, I. C. Paschalidis, et al., “Distributionally robust learning,” Foundations and Trends® in Optimization, vol. 4, no. 1-2, pp. 1–243, 2020
2020
-
[44]
On the gener- alization gap in reparameterizable reinforcement learning,
H. Wang, S. Zheng, C. Xiong, and R. Socher, “On the gener- alization gap in reparameterizable reinforcement learning,” in International Conference on Machine Learning, PMLR, 2019, pp. 6648–6658
2019
-
[45]
Real-time bayesian detection of drift-evasive gnss spoofing in reinforcement learning based uav deconfliction,
D. K. Panda and W. Guo, “Real-time bayesian detection of drift-evasive gnss spoofing in reinforcement learning based uav deconfliction,” arXiv preprint arXiv:2507.11173, 2025
2025 arXiv
-
[46]
Gnss jamming and spoofing threats in uav navigation: Countermeasure status and challenges,
Y. Zeng et al., “Gnss jamming and spoofing threats in uav navigation: Countermeasure status and challenges,” IEEE Communications Surveys & Tutorials, 2026
2026
-
[47]
A uav path planning method in three-dimensional urban airspace based on safe reinforcement learning,
Y. Li, X. Zhang, Y. Zhu, and Z. Gao, “A uav path planning method in three-dimensional urban airspace based on safe reinforcement learning,” in 2023 IEEE/AIAA 42nd Digital A vionics Systems Conference (DASC), IEEE, 2023, pp. 1–7
2023
-
[48]
Meta policy switching for resilient uav navigation in adversarial airspace,
D. K. Panda and W. Guo, “Meta policy switching for resilient uav navigation in adversarial airspace,” A vailable at SSRN 6755038,
-
[49]
Adaptively preconditioned stochastic gra- dient langevin dynamics,
C. A. Bhardwaj, “Adaptively preconditioned stochastic gra- dient langevin dynamics,” arXiv preprint arXiv:1906.04324, 2019
1906 arXiv
-
[50]
Adaptive gradient methods with dynamic bound of learning rate,
L. Luo, Y. Xiong, Y. Liu, and X. Sun, “Adaptive gradient methods with dynamic bound of learning rate,” in Interna- tional Conference on Learning Representations, 2018
2018
-
[51]
A sufficient condition for convergences of adam and rmsprop,
F. Zou, L. Shen, Z. Jie, W. Zhang, and W. Liu, “A sufficient condition for convergences of adam and rmsprop,” in Pro- ceedings of the IEEE/CVF Conference on computer vision and pattern recognition, 2019, pp. 11 127–11 135
2019
-
[52]
Benchmarking deep reinforcement learning for continuous control,
Y. Duan, X. Chen, R. Houthooft, J. Schulman, and P. Abbeel, “Benchmarking deep reinforcement learning for continuous control,” in International conference on machine learning, PMLR, 2016, pp. 1329–1338. JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 16 Appendix A In...
2016
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.