{"id":"0a72fa5b-dd4e-4efb-ab47-bc00c71990f8","arxiv_id":"2506.21129","paper_version":2,"verdict":"REJECT","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"A curriculum that aligns temporal-difference error distributions across increasing adversarial perturbations is claimed to make UAV policies robust to unseen GNSS spoofing attacks, with a generalization certificate.","lead":"This paper trains drone navigation policies to resist GPS spoofing attacks they never saw, by gradually exposing them to stronger fake signals while keeping their learning signal stable. The authors claim a mathematical guarantee that if a new attack looks close in a 'temporal-difference error' space, the performance loss stays bounded.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Lemma 3's reverse pushforward inequality (Eq. 21) is not generally valid; without injectivity of the TD-error map, W1(P_k,P_{k+1}) is not bounded by W1(TD_k,TD_{k+1}), so Theorem 2's δ_A certificate does not follow.","rationale":"In good faith, the paper's distinctive contribution is the TD-space generalization certificate of Theorem 2; the empirical study is suggestive, but the certificate is the advertised formal explanation for transfer to unseen spoofing attacks. That certificate collapses if Lemma 3 is unavailable. Lemma 3 asserts a reverse Lipschitz-pushforward inequality, which is mathematically false without an injectivity or metric-invertibility condition that is neither stated nor implied by the paper's assumptions. Since TD errors are scalar summaries of states, distinct transition distributions can readily share the same TD-error distribution, making W1(TD) = 0 while W1(P) > 0. Lemma 4 adds a further heuristic step, so the proof chain is broken independently. This is an internal mathematical gap, not a disagreement with consensus, and it directly undermines the central claim that bounded δ_A implies bounded performance degradation. The empirical results may still indicate practical value, but the paper's advertised formal guarantee is unsupported as written.","tokens_in":37051,"tokens_out":5944,"duration_ms":72012,"concrete_test":"Check Lemma 3 against a minimal non-injective example: let X = {0,1}, T(0) = T(1) = 0, μ = δ_0, ν = δ_1. Then W1(μ, ν) = 1 but W1(T_#μ, T_#ν) = 0, so Eq. (21) fails for every finite L_T. If the authors believe the UAV TD-error map is injective on the relevant supports, they should state and prove that assumption; otherwise the reverse-pushforward step in Appendix D.C and its uses in Theorem 1 and Theorem 2 are invalid.","verdict_should_be":"REJECT","load_bearing_attack":"Lemma 3 (Eq. 21) asserts W1(P_{k+1}, P_k) ≤ L_T W1(TD_{k+1}, TD_k). Its proof in Appendix D.C invokes the Lipschitz pushforward property, but the standard property is the forward inequality W1(T_#μ, T_#ν) ≤ Lip(T) W1(μ, ν). The reverse inequality holds only when T is injective in a metric sense, e.g. admits a Lipschitz right-inverse; no such condition appears in Assumptions 1–2. TD error is a scalar functional of a high-dimensional state, so the map P ↦ TD is many-to-one in general: two transition kernels can produce identical TD-error distributions while W1(P_k, P_{k+1}) > 0. This failure is load-bearing. Theorem 1's final step (Eq. 29) uses Lemma 3 at Eq. (28), and Theorem 2's proof uses it again at Eq. (D.26) to replace W1(P_A, P_K) with δ_A = W1(TD_A, TD_K). Without Lemma 3, the C·δ_A term does not bound the performance gap, so the certificate does not follow. In addition, Lemma 4 in Appendix D.D is explicitly heuristic, relying on approximations 'E[f(Φ'_k)] ≈ 0' and 'E[f(Φ'_{k+1})] ≈ d(...)' rather than inequalities, so Theorem 1's chain in Eqs. (26)–(27) is unsupported even before Lemma 3 is invoked.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a curriculum-guided adversarial adaptation framework for robust reinforcement learning in UAV deconfliction. Starting from an action-robust critic trained via a two-player game with SGLD, the method generates adversarial state observations with an increasing perturbation budget ε (multi-step projected gradient ascent, Algorithm 1) and, at each curriculum stage, updates the critic to minimize the Wasserstein-1 distance between the TD-error distribution under the current adversarial observations and that of the previous stage's expert critic (Algorithm 3). The main theoretical contribution is a 'TD-space generalization certificate' (Theorem 2): with probability 1−δ over n evaluation episodes, the performance gap between the adapted policy under a previously unseen attack A and the empirical return under the final curriculum stage is bounded by a Rademacher term plus C·δ_A, where δ_A is the W1 distance between the TD-error distribution induced by A and that of the final stage. The certificate is built from Theorem 1 and Lemmas 1–4. The framework is evaluated in a simulated 3D UAV deconfliction environment under unseen fixed constant-bias GNSS spoofing and dynamic obstacle-luring spoofing attacks, reporting near-perfect mission success under fixed spoofing (vs. 20–56% for baselines), higher rewards, and up to 45% fewer completion steps under dynamic spoofing.","tokens_in":37483,"tokens_out":23382,"duration_ms":229352,"significance":"If the certificate were valid, this would be a valuable contribution: it would provide a principled, attack-agnostic condition under which robustness acquired from synthetic gradient-based perturbations transfers to black-box sensor-level attacks, a practically important problem for cyber-physical RL. The paper has genuine strengths: the adaptation procedure is specified in detail (Algorithms 1–3); the threat models in Section IV are realistic (constant code-phase bias and stealth-constrained gradual carry-off spoofing); results are reported as distributions over 100 test episodes; and Section V.C includes an honest limitation statement (the authors refrain from claiming generalization beyond ε = 4.0). The idea of quantifying catastrophic forgetting as W1 drift of TD-error distributions is reasonable and clearly motivated. However, the central theoretical claim is not supported: Lemma 3 (Eq.","major_comments":[{"comment":"Lemma 3 asserts W1(P_{k+1}, P_k) ≤ L_T W1(TD^{π*}_{k+1}(ε_{k+1}), TD^{π*}_k(ε_k)), and the proof (Appendix D.C, Eqs. D.8–D.10) invokes 'a general property of optimal transport theory' of the form W1(μ,ν) ≤ L_T W1(T_#μ, T_#ν) for Lipschitz T. The standard property is the forward inequality W1(T_#μ, T_#ν) ≤ Lip(T) W1(μ,ν); the reverse direction requires T to be metrically injective (e.g., to admit a Lipschitz right-inverse), which is not assumed in Assumptions 1–2. Since the TD error is a scalar functional of high-dimensional state-action transitions, the map is many-to-one in general: taking T ≡ 0 gives W1(T_#μ, T_#ν) = 0 while W1(μ,ν) can be arbitrarily large. The failure is load-bearing: Theorem 1 uses Lemma 3 at Eq. (28) and Theorem 2 uses it again at Eq. (D.26) to replace W1(P_A, P_K) by δ_A, so the C·δ_A term in Eqs. (30)–(31) does not follow and the certificate is unsupported.","section":"III, Lemma 3 (Eq. 21); proof in Appendix D.C"},{"comment":"The statement of Theorem 1 bounds E[TD^{π*}_{k+1}(ε_{k+1}) − TD^{π*}_k(ε_{k+1})], the difference between two policies evaluated at the same attack level ε_{k+1}, while the proof (Eq. 24) computes E[TD^{π*}_{k+1}(ε_{k+1}) − TD^{π*}_{k+1}(ε_k)], the same policy evaluated at two attack levels; the quantities differ in both the policy index and the perturbation level. The proof also introduces an absolute value at Eq. (27) that is absent from the statement. Hence Theorem 1 is not proved as stated, and the subsequent use in Appendix D.V (attributing the bound on the 'policy change term' at a fixed attack level to Theorem 1) is not justified by the demonstrated chain.","section":"III, Theorem 1 (Eq. 23) and its proof (Eqs. 24–29)"},{"comment":"Lemma 4's proof rests on two approximations that the authors themselves flag with '≈': E[f(Φ'_k)] ≈ 0 and E[f(Φ'_{k+1})] ≈ d(Φ_{ε_{k+1}}, Φ_{ε_k}). These are not inequalities, and the first is not obviously true because Φ'_k ~ P_k is the transitioned state from an adversarially perturbed state, which can be far from Φ_{ε_k} even when the dynamics is smooth. Lemma 4 provides the bound d(Φ_{ε_{k+1}}, Φ_{ε_k}) ≤ W1(P_{k+1}, P_k) used at Eq. (26) of Theorem 1's proof to obtain Eq. (27), so the stagewise bound and the constant m are not established even before Lemma 3 is invoked.","section":"III, Lemma 4 (Eq. 22); proof in Appendix D.D"},{"comment":"Two gaps invalidate the stagewise bound in Eq. (D.32). First, the 'policy change term' in Eq. (D.31) is a return difference E[R(π*_{k+1}, ε_{k+1})] − E[R(π*_k, ε_{k+1})], whereas Theorem 1 bounds an expectation of TD-error differences; no performance-difference lemma or Lipschitz relation between returns and TD-error expectations is supplied, so the claim that this term is 'bounded by m W1(...) = m β_k' does not follow. Second, Lemma 1 compares expectations of two different value functions Q^{π*}_{k+1} and Q^{π*}_k, but the Kantorovich–Rubinstein step at Eq. (D.2) replaces both by a single Lipschitz function, thereby dropping the term E_{P_{k+1}}[Q_{k+1} − Q_k], which need not vanish between curriculum stages; Lemma 1 is valid only for a fixed critic, not in the form used here.","section":"Appendix D.V (proof of Theorem 2); Lemma 1 (Eq. 19) and its proof in Appendix D.I"},{"comment":"The paper states that Theorem 2 applies to the unseen attacks 'whenever this proximity is bounded' (δ_A), but δ_A is never computed for the fixed or dynamic GNSS spoofing attacks used in the experiments, and no estimator for δ_A from test trajectories is described. The certificate's hypothesis (small W1 distance between the attack-induced TD-error distribution and that of the final curriculum stage) is therefore not verified, so the results in Figures 8–13 neither instantiate nor test Theorem 2. Moreover, since Algorithm 3 (line 20) directly minimizes W1 between consecutive TD-error distributions, the certificate is largely a consistency statement between this objective and the return gap in the same metric, rather than an independent guarantee of transfer across attack mechanisms. The paper's claim (Section VI) that the framework provides a formal explanation of the observed transfer is consequently not supported by the data.","section":"V.D (Robustness Generalization against Unseen Spoofing Attacks)"}],"minor_comments":[{"comment":"The TD error is defined in Eqs. (14)–(15) as a product of Gaussian densities, while Algorithm 2 (step 8) computes a scalar δ²/σ² with δ = V_tar − V_adv; please clarify how the scalar TD errors, the density products, and the distributions whose W1 distance is minimized in Algorithm 3 (line 20) are related, and define σ.","section":"III, Eqs. (13)–(15) and Algorithm 2"},{"comment":"The manuscript contains many typos and ligature artifacts ('sufficiently', 'traffic', 'efficiency', 'behaivour', 'decondliction', 'Nnformation' in Ref. [11]); a thorough copy-edit is needed.","section":"Throughout"},{"comment":"In the version under review, several figures (notably Figures 8–12) contain garbled characters in their labels; please regenerate the figures and verify that all text is legible in the final PDF.","section":"V (figures)"},{"comment":"Quantitative claims (near-perfect mission success vs. 20–56% for baselines; up to 45% step reduction) are supported only by boxplots and density plots; please add tables with means, standard deviations, and the number of training seeds for each policy to substantiate these comparisons.","section":"V.B–V.D"},{"comment":"Please state explicitly the probability space for the 1−δ statement (which sources of randomness are covered) and align the constant in the O(c/√n √(log(1/δ))) term with the bound in [44].","section":"III, Theorem 2 (Eq. 30)"},{"comment":"The text states that the curriculum regime is bounded at ε ≤ 4.0, but also reports stable forgetting at ε = 5.0 and an ablation comparing stopping conditions ε = 3.0, 4.0, 5.0 (Section V.E); please reconcile these statements and state the final ε_max used for the evaluated policies.","section":"V.C"}],"recommendation":"reject","confidential_remarks":"Some remarks for the editor only. The empirical section is competently executed and the problem is well chosen, but the theory–experiment connection is loose: the certificate's condition δ_A is never estimated, so the theory cannot explain the empirical results even if its lemmas were correct. Because the reverse inequality in Lemma 3 is false in general and the required injectivity of the TD-error map is implausible for neural critics, I do not see a repair within the manuscript's current scope; a resubmission as an empirical robustness study, with all 'provably robust' claims removed, might be viable. Also, Section IV.A leans on an unreviewed arXiv preprint by the same group [45] for the fixed spoofing model; the model should be self-contained in a journal version."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The TD-space curriculum idea is genuinely new and the simulation results are strong enough to take seriously, but the theoretical certificate in Section III does not hold. Lemma 3's inequality is the load-bearing piece and it is simply the wrong direction. For a Lipschitz map T, the standard bound is W1(T#μ, T#ν) ≤ Lip(T) W1(μ,ν). The reverse inequality needs T to admit a Lipschitz right-inverse or similar injectivity; nothing in Assumptions 1–2 provides that. TD error is many-to-one as a function of the transition kernel, so two different kernels can easily induce the same TD-error distribution. That kills the step from Eq. (27) to Eq. (29) in Theorem 1 and the step from (D.24) to (D.26) in Theorem 2. Lemma 4 in D.D is also openly heuristic: \"E[f(Φ'_k)] ≈ 0\" and \"E[f(Φ'_{k+1})] ≈ d(...)\" are approximations, not inequalities, so even Theorem 1's earlier chain is unsupported. The certificate is the abstract's main promise, so this is a major problem, not a typo.\n\nWhat the paper does well: the framework itself is plausible and the experiments are extensive. The idea of aligning TD-error distributions across perturbation levels rather than matching observations is a useful angle, and the transfer from PGD-style training to fixed and dynamic GNSS spoofing is exactly the kind of result the community wants. The mission-success and trajectory plots are consistent with the claim that the adapted policy is more robust. The authors also include ablations and sensitivity analysis.\n\nThe soft spots beyond the theory: the curriculum stopping point ε=4.0 is selected by testing against the actual spoofing attack (Figure 14 and the ablation text), and Δε=0.25 is chosen after seeing the test rewards in Figure 15. That is tuning on the test distribution, which weakens the generalization story. Also, there is no code or data release, so reproducibility is limited to the reported numbers.\n\nOverall: if the authors presented this as a heuristic recipe with strong empirical support, I'd be interested. As it stands, the formal certificate is invalid and the generalization claims outrun the evidence. I'd send it to review rather than desk reject, because the empirical core is worth refereeing and the authors could be pushed to either fix or remove the theory. But my own verdict on this version would be reject.","headline":"The TD-space curriculum idea has legs and the simulations are worth taking seriously, but the certificate is built on a false reverse pushforward inequality, so the central claim as stated does not follow.","tokens_in":37971,"tokens_out":3226,"would_cite":false,"duration_ms":39827,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Aligning TD-error distributions across an adversarial curriculum yields a generalization certificate: attacks whose TD-error distribution stays close to the final stage suffer bounded loss, so robustness transfers to black-box GNSS…","keywords":["curriculum reinforcement learning","adversarial robustness","TD-error distribution alignment","Wasserstein distance","catastrophic forgetting","GNSS spoofing","UAV deconfliction","domain adaptation"],"falsifier":"During curriculum training, record paired samples of consecutive transition distributions and their TD-error distributions and check whether $W_1(P_{k+1}, P_k) \\leq L_T\\,W_1(\\mathrm{TD}_{k+1}, \\mathrm{TD}_k)$ actually holds at every stage with a bounded constant; alternatively, sweep candidate attacks and measure $\\delta_A = W_1(\\mathrm{TD}(A), \\mathrm{TD}(\\varepsilon_K))$ against realized mission success, looking for an attack with near-zero $\\delta_A$ and a large success drop. Either observation would directly contradict the certificate.","tokens_in":36860,"feed_emoji":"🛰️","tokens_out":16961,"duration_ms":139593,"temperature":0.7,"pith_summary":"Autonomous navigation policies trained with reinforcement learning break under GNSS spoofing because the attack pushes observations outside the training distribution, corrupting value estimates and causing mission failure. This paper argues that the right place to defend is temporal-difference (TD) error space: it proposes a curriculum that exposes a robust policy to increasingly strong gradient-based observation perturbations while minimizing the Wasserstein-1 distance between the TD-error distributions of the expert critic and the adapting critic at each stage. The central theoretical claim is a TD-space generalization certificate: if the TD-error distribution induced by a previously unseen attack stays sufficiently close to that of the final curriculum stage, the attack's performance degradation is bounded; if true, this explains why robustness learned on synthetic perturbations transfers to black-box GNSS spoofing without attack-specific retraining. The supporting experiment is a UAV deconfliction task in which the curriculum-adapted policy achieves near-perfect mission success under fixed spoofing (baselines: 20–56%) and reduces mission steps by up to 45% under dynamic obstacle-luring spoofing.","feed_headline":"Curriculum RL beats unseen GPS spoofing without retraining","feed_subtitle":"TD-error alignment gives near-perfect missions under fixed spoofing and 45% fewer steps under dynamic attacks.","key_machinery":"The machinery is a curriculum of adversarial observations acting as an interpolation path in TD-error distribution space. Adversarial states are generated by a multi-step projected-gradient-ascent procedure (Algorithm 1) with a perturbation budget $\\varepsilon$ that increases stage by stage, and at each stage the adapting critic is trained by minimizing the Wasserstein-1 distance $W_1(\\mathrm{TD}^{\\mathrm{exp}}, \\mathrm{TD}^{\\mathrm{AR}})$ between the expert critic's TD-error distribution and its own (Algorithm 3); the expert critic is carried forward from the previous stage, and the curriculum stops when the TD-error gap between successive stages falls below a threshold. Catastrophic forgetting is formalized as exactly this distance, $f^\\varepsilon_\\pi = \\inf_\\pi W_1(\\mathrm{TD}^{\\mathrm{DDPG}}, \\mathrm{TD}^\\varepsilon_\\pi)$. The theoretical chain that produces the certificate is: Lipschitz assumptions on rewards and value functions (Assumptions 1–2), Lemmas 1–2 bounding value-function shifts, Lemma 3's reverse pushforward inequality $W_1(P_{k+1}, P_k) \\leq L_T W_1(\\mathrm{TD}_{k+1}, \\mathrm{TD}_k)$, and Lemma 4 bounding the adversarial-state distance, together yielding Theorem 1 (bounded TD drift) and Theorem 2 (the test-time certificate with $\\mathrm{Rad}_K$ and $\\delta_A$).","core_discovery":"In the paper's own terms, the discovery is that adversarially induced value-function drift, quantified as the 1-Wasserstein distance between temporal-difference (TD) error distributions, is the quantity to control if robustness is to transfer across attack domains. The authors claim in Theorem 2 that the curriculum-adapted policy $\\pi^\\ast_K$ satisfies, with probability $1-\\delta$ over $n$ episodes, a bound of the form $\\bigl| E[R(\\pi^\\ast_K \\text{ under } A)] - \\tfrac{1}{n}\\sum_{i=1}^n R(\\pi^\\ast_K) \\bigr| \\leq \\mathrm{Rad}_K + C\\,\\delta_A + O\\!\\bigl(\\tfrac{c}{\\sqrt{n}}\\sqrt{\\log(1/\\delta)}\\bigr)$, where $\\delta_A = W_1\\bigl(\\mathrm{TD}_{\\pi^\\ast_K}(A), \\mathrm{TD}_{\\pi^\\ast_K}(\\varepsilon_K)\\bigr)$ is the distance between the attack-induced TD-error distribution and that of the final curriculum stage, $\\mathrm{Rad}_K$ is a Rademacher-complexity term, and $C$ is a problem-dependent constant. A companion bound against the original expert adds the cumulative curriculum drift $\\sum_k \\beta_k$, tying stagewise TD alignment to limited error propagation. The paper further claims that Wasserstein-1 is the right metric for this certificate because it stays finite on non-overlapping supports, where W2 becomes tail-sensitive and KL diverges. Theorem 1 asserts the stagewise engine of the bound: TD-error drift across consecutive curriculum stages is controlled by $m\\,W_1\\bigl(\\mathrm{TD}_{\\pi^\\ast_{k+1}}(\\varepsilon_{k+1}), \\mathrm{TD}_{\\pi^\\ast_k}(\\varepsilon_k)\\bigr)$ with $m = L_V L_T (1+\\gamma)$, and the simulation results are presented as evidence that adapting on synthetic gradient perturbations generalizes to physically distinct GNSS spoofing attacks.","pith_inferences":["A deployment-time monitor could compute $\\delta_A$ online from a rolling buffer of TD errors and raise an alert when the running estimate exceeds the certified radius; the paper does not develop this monitoring use, but the certificate's structure invites it.","Because the certificate is agnostic to the observation-space mechanism, the same TD-space curriculum could plausibly transfer to other falsified-sensor attacks such as ADS-B spoofing or inertial drift, an extension the paper does not test.","The sensitivity analysis, which shows smaller curriculum increments ($\\Delta\\varepsilon = 0.25$) generalize better than coarse ones, suggests a testable monotonicity: unseen-attack generalization should improve as the stagewise Wasserstein drift is driven toward zero.","If Lemma 3's reverse inequality proves false in some environment, the framework would need explicit regularization of the TD-error map (for example, Lipschitz or invertibility constraints) rather than relying on the inequality as an assumed property."],"forward_implications":["Robustness acquired under synthetic gradient perturbations is not tied to a particular attack family: any unseen attack whose induced TD-error distribution lies within $\\delta_A$ of the final curriculum stage inherits the bounded-degradation guarantee without retraining.","Under a fixed GNSS spoofing attack the curriculum-adapted policy reaches near-perfect mission success, while DDPG and the robust RL baselines (PR-MDP, NR-MDP, adversarial meta-learning, AutoLiRPA) fall to 20–56% success.","Under dynamic obstacle-luring spoofing the adapted policy keeps the highest episodic rewards and completes missions in up to 45% fewer steps across all tested traffic densities.","The cumulative curriculum drift $\\sum_k \\beta_k$ is controlled by design, so error does not accumulate from early adaptation stages into the final policy.","The convergence property behind Theorem 1 keeps the TD-error distribution bounded throughout adaptation, which is the paper's operational definition of mitigating catastrophic forgetting."],"supporting_citations":[{"why":"Supplies the Rademacher-complexity bound on the intrinsic generalization gap that Theorem 2's $\\mathrm{Rad}_K$ term inherits.","marker":"[44]"},{"why":"Provides the adversarial two-player game and mixed-Nash equilibrium solution used to train the action-robust expert critic.","marker":"[36]"},{"why":"Defines action-robust RL and supplies the PR-MDP and NR-MDP baselines against which the framework is benchmarked.","marker":"[9]"},{"why":"Provides the projected-gradient-attack procedure that generates the curriculum's adversarial observation perturbations.","marker":"[38]"},{"why":"Motivates curriculum learning through optimal transport and Wasserstein interpolation between task distributions.","marker":"[12]"},{"why":"Underpins Assumption 1, the Lipschitz reward condition, via bisimulation metrics for continuous Markov decision processes.","marker":"[40]"},{"why":"Underpins Assumption 2, the Lipschitz value-function condition under adversarial states.","marker":"[41]"},{"why":"Is the prior action-robust deconfliction framework this work extends and the source of the spoofing attack model used at test time.","marker":"[5]"}],"fun_headline_variants":["Curriculum RL aligns TD errors to generalize across spoofing attacks","Wasserstein-1 controls TD drift for robust RL under spoofing","UAV RL beats unseen GPS spoofing with TD-error curriculum","TD-error alignment gives RL transferable defense against spoofing"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The certificate stands on Lemma 3's reverse pushforward inequality, which claims the distance between successive environments' next-state distributions is bounded by a constant times the distance between their TD-error distributions; the standard pushforward inequality runs the opposite direction, and pushforwards can erase information, so if this reverse bound fails in the target environment the Theorem 2 guarantee collapses, as do the two heuristic approximations used in Lemma 4.","fun_headline_variants_meta":{"raw":{"variants":["Curriculum RL aligns TD errors to generalize across spoofing attacks","Wasserstein-1 controls TD drift for robust RL under spoofing","UAV RL beats unseen GPS spoofing with TD-error curriculum","TD-error alignment gives RL transferable defense against spoofing"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000359,"raw_usage":{"total_tokens":2078,"prompt_tokens":1216,"completion_tokens":862,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":832,"completion_tokens_details":{"reasoning_tokens":787}},"tokens_in":832,"tokens_out":862,"duration_ms":9762,"temperature":1.0,"reasoning_tokens":787,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T22:33:00.530351+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"During curriculum training, record paired samples of consecutive transition distributions and their TD-error distributions and check whether $W_1(P_{k+1}, P_k) \\leq L_T\\,W_1(\\mathrm{TD}_{k+1}, \\mathrm{TD}_k)$ actually holds at every stage with a bounded constant; alternatively, sweep candidate attacks and measure $\\delta_A = W_1(\\mathrm{TD}(A), \\mathrm{TD}(\\varepsilon_K))$ against realized mission success, looking for an attack with near-zero $\\delta_A$ and a large success drop. Either observation would directly contradict the certificate.","supporting_citations":[{"cited_title":"On the gener- alization gap in reparameterizable reinforcement learning,","cited_arxiv_id":null,"evidence_quote":"Supplies the Rademacher-complexity bound on the intrinsic generalization gap that Theorem 2's $\\mathrm{Rad}_K$ term inherits."},{"cited_title":"Robust reinforcement learning via adversarial training with langevin dynamics,","cited_arxiv_id":null,"evidence_quote":"Provides the adversarial two-player game and mixed-Nash equilibrium solution used to train the action-robust expert critic."},{"cited_title":"Action robust rein- forcement learning and applications in continuous control,","cited_arxiv_id":null,"evidence_quote":"Defines action-robust RL and supplies the PR-MDP and NR-MDP baselines against which the framework is benchmarked."},{"cited_title":"Towards evaluating the robustness of neural networks,","cited_arxiv_id":null,"evidence_quote":"Provides the projected-gradient-attack procedure that generates the curriculum's adversarial observation perturbations."},{"cited_title":"Curriculum reinforcement learning using optimal transport via gradual domain adaptation,","cited_arxiv_id":null,"evidence_quote":"Motivates curriculum learning through optimal transport and Wasserstein interpolation between task distributions."},{"cited_title":"Bisimulation metrics for continuous markov decision processes,","cited_arxiv_id":null,"evidence_quote":"Underpins Assumption 1, the Lipschitz reward condition, via bisimulation metrics for continuous Markov decision processes."},{"cited_title":"Scalable methods for computing state similarity in deterministic markov decision processes,","cited_arxiv_id":null,"evidence_quote":"Underpins Assumption 2, the Lipschitz value-function condition under adversarial states."},{"cited_title":"Action robust reinforcement learning for air mobility deconfliction against conflict induced spoofing,","cited_arxiv_id":null,"evidence_quote":"Is the prior action-robust deconfliction framework this work extends and the source of the spoofing attack model used at test time."}],"review_version":1}