Delay-Aware Active Triangulation with Uncertainty-Driven Multi-Agent Reinforcement Learning for Counter-UAS
Reviewed by Pith T0 review T1 audit T2 compute T3 formal T4 kernel 2026-07-08 20:00 UTCglm-5.2pith:A6XLWRU5record.jsonopen to challenge →
The pith
Stale Data Beats Clean Data for Training Drone Teams to Track Targets
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The central finding is that when training multi-agent policies for tasks requiring geometric coordination under communication delays, aligning the reward signal with the policy's actual noisy perception yields more robust behavior than rewarding against ground-truth states. Clean-state rewards create a stability-robustness tradeoff: they produce temporally stable coordination but can guide policies toward noise-fragile geometric optima (e.g., near-degenerate ray intersection angles). Perception-consistent rewards implicitly penalize such fragility because the same configuration receives different reward values depending on its sensitivity to the noise actually present. This identifies reward
What carries the argument
The paper's argument is carried by three mechanisms: (1) AoI-augmented observations, where each inter-agent message carries a directly measured staleness tag computed as current time minus message timestamp, enabling the recurrent policy to discount stale teammate information; (2) a dual-path delay architecture that separates the observation path (always noisy and delayed) from the reward path, allowing controlled comparison of perception-consistent vs. privileged rewards under identical observation conditions; and (3) multi-source analytical covariance propagation that linearizes the triangulation around pixel, pose, gimbal, and intrinsics uncertainties, yielding a closed-form 3x3 position
If this is right
- If perception-consistent rewards generalize beyond triangulation, RL practitioners training multi-agent systems under sensor noise should reconsider the common practice of using privileged clean-state rewards during training, particularly when the deployment environment involves significant observation noise.
- The finding that AoI tags provide information that recurrent memory alone cannot recover suggests that explicit freshness metadata should be a standard observation component in any multi-agent system with asynchronous, variable-latency communication, not just visual triangulation.
- The 2.8-fold degradation from angular-only covariance modeling indicates that prior single-source uncertainty models may significantly underestimate the true error budget in multi-agent perception tasks, potentially invalidating coordination strategies that were optimized against incomplete noise models.
- The stability-robustness tradeoff between perception-consistent and privileged rewards identifies a deployment-dependent design axis: missions requiring sustained long-horizon tracking may benefit from privileged rewards, while missions requiring fault tolerance benefit from perception-consistent rewards.
Load-bearing premise
The simulation's stochastic delay model uses stationary distributions for latency, staleness, and dropout. Real-world communication interference can be bursty and non-stationary, and if the actual delay distribution differs substantially from the simulated one, the learned policy may not transfer and the relative advantage of each design choice could change.
What would settle it
If, in real-flight experiments with onboard sensing and wireless networking, the perception-consistent reward policy fails to outperform the privileged-reward policy, or if AoI-augmented observations provide no measurable benefit over AoI-free recurrent policies, the paper's central design recommendations would not hold outside simulation. The paper itself acknowledges this: 'real-flight validation with onboard sensing and wireless networking remains necessary.'
Figures
read the original abstract
Multi-agent active visual triangulation enables precise 3D localization of aerial targets by coordinating mobile observers with controllable cameras. However, existing methods assume instantaneous state feedback, ignoring cumulative latency from detection, communication, and decision propagation. We present a delay-aware, uncertainty-driven multi-agent reinforcement learning framework for target localization in Counter-UAS applications. Our contributions are: (1) a Dec-POMDP formulation with Age-of-Information (AoI) augmented observations enabling delay-aware coordination -- AoI improves triangulation validity by 10.6 percentage points; (2) a controlled comparison showing that perception-consistent rewards outperform privileged clean-state rewards (0.547 m vs.0.633 m RMSE, 27% fewer track losses) -- both policies are trained through identical observation noise but differ in what they are optimized for, producing a stability-robustness tradeoff; and (3) multi-source analytical covariance propagation incorporating pixel, pose, gimbal, and intrinsics uncertainties -- restricting to angular noise alone causes 2.8-fold RMSE degradation. Experiments with MAPPO in 4096 parallel environments achieve 0.547 +- 0.217 m RMSE with 78.1% triangulation validity, while MLP policies achieve near-zero validity (0.7%), confirming recurrent memory as essential for delay compensation.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper presents a delay-aware multi-agent reinforcement learning framework for active visual triangulation in Counter-UAS applications. The authors make three contributions: (1) a Dec-POMDP formulation with Age-of-Information (AoI)-augmented observations for handling stochastic communication delays, (2) a controlled comparison between perception-consistent (noisy) and privileged (clean) reward formulations, and (3) an extension of analytical triangulation covariance propagation to incorporate multi-source uncertainties (pixel, pose, gimbal, intrinsics). The framework is validated in Isaac Sim with 4096 parallel environments using MAPPO, with ablations isolating each contribution against external baselines (Gavin et al. angular-only covariance, no-AoI, MLP, clean-reward). The perception-consistent reward policy achieves 0.547 m RMSE with 78.1% triangulation validity.
Significance. The paper addresses a practically important problem: multi-agent visual triangulation under realistic communication delays. The ablation design is methodologically sound, with each contribution isolated by changing one variable while holding others constant, N>4000 episodes per condition, and 5th-95th percentile clipping. The covariance derivation (Eqs. 14-17) is a standard first-order linearization that is parameter-free given stated noise models. The reward comparison is a controlled experiment rather than a fitted prediction. The robustness envelopes (Figs. 6-7) test delay magnitude and target speed variation beyond training conditions. The honest reporting of the stability-robustness tradeoff in the reward comparison, including the gradual RMSE drift in the noisy-reward policy, is commendable.
major comments (4)
- Section V-C, Contribution 2 interpretation: The paper attributes the perception-consistent reward's advantage to 'reward-observation alignment' — specifically, that noisy rewards penalize noise-fragile optima because 'the same configuration receives different reward values depending on its sensitivity to the noise actually present.' However, an alternative explanation is that the stochasticity in the noisy reward path acts as implicit exploration or regularization, preventing premature convergence to any fixed geometric equilibrium — regardless of whether the noise is correlated with the policy's observations. The per-timestep analysis (Fig. 5) actually supports this alternative: the clean-reward policy converges to a 'stable geometric equilibrium' with 'monotonically decreasing RMSE,' while the noisy-reward policy 'lacks a fixed attractor' and shows 'gradual RMSE increase.' This pattern
- is consistent with reward stochasticity disrupting convergence rather than specifically teaching noise-robustness. The paper does not test this alternative. A discriminating experiment would compare the perception-consistent reward against a clean-state reward with additive i.i.d. Gaussian noise matched in variance to the perception-consistent path's reward variance. If both yield similar aggregate metrics, the benefit is generic reward stochasticity rather than noise-observation correlation. Without this test, the 'reward-observation alignment as a design axis' claim (Section V-C, final paragraph) is not fully substantiated. This is load-bearing for Contribution 2's novelty framing.
- Section V-C and Table II, clean-reward row: The clean-reward policy achieves higher triangulation validity (79.1% vs. 78.1%) and visibility (86.7% vs. 84.1%) than the noisy-reward policy, yet the paper frames the noisy-reward policy as superior in 'aggregate metrics.' The RMSE advantage (0.547 vs. 0.633 m) is the primary basis for this claim, but the clean-reward policy's higher validity and visibility suggest it maintains tracking more reliably. The paper should clarify which metrics constitute 'aggregate' superiority and acknowledge that the dominance is metric-dependent rather than uniform. The current framing in the abstract ('perception-consistent rewards outperform privileged clean-state rewards') overstates a mixed result.
- Section VI, scaling beyond two agents: The paper notes that preliminary three-agent experiments show collision rates rising 9.5x. Given that multi-agent coordination is central to the paper's framing, this limitation is significant. The authors attribute it to insufficient collision-risk encoding in observations/rewards, but this suggests the reward formulation (Contribution 2) may not generalize beyond the minimal two-agent case. The paper should discuss whether the perception-consistent reward's advantage over the clean reward persists with more agents, or whether this is an artifact of the two-agent setting.
minor comments (8)
- Table II: The N values differ across conditions (4096, 4412, 4280, 4096, 6784). The MLP condition has substantially more episodes (6784), which the text explains is due to early termination, but the table caption or footnotes should clarify that N reflects completed/terminated episodes and that conditions with early termination accumulate more episodes within the same wall-clock budget.
- Eq. (3): The AoI update rule sets Delta = d upon successful reception, but d is defined as end-to-end delay. If the delay includes both staleness and latency (as described in Section IV-A), then upon reception the AoI should equal the total delay experienced, which is correct. However, the notation could be clearer — d_k is used for delay in Eq. (2) and (3) but the three-stage pipeline (staleness, latency, dropout) is described only qualitatively. A brief equation showing how d_k composes from the three stages would improve reproducibility.
- Section IV-B, Eqs. (14)-(17): The Jacobian J_theta is described as capturing sensitivity to 'nuisance parameters theta_i (pose, gimbal, intrinsics)' but the specific parameterization of theta_i is not given. For reproducibility, a brief enumeration of the components of theta_i and their noise standard deviations (referenced as matching Section IV-B) would help.
- Fig. 5 caption: The caption states 'averaged across N > 4,000 episodes' but does not specify which condition. It should clarify that this is for the noisy-reward and clean-reward comparison.
- Section III-C, Eq. (11): The reward weights (s_ctr, s_sz, lambda_ctr, lambda_sz) are listed as free parameters in the axiom ledger but their values are not reported in the paper. Including these in a table or appendix would improve reproducibility.
- Section V-A: The curriculum schedule boundaries (Phase 1: 0-60k, Phase 2: 20k-100k, Phase 3: 80k-200k) overlap. This is presumably intentional (linear ramps), but should be clarified to avoid confusion about whether phases are sequential or overlapping.
- Abstract and Section I: '0.547m vs.0.633m' should have a space after 'vs.' for consistency.
- Section V-C, 'Non-stationary delays' paragraph: The claim that 'AoI would still indicate stale teammate information' under non-stationary channels is correct in principle, but the policy was trained under stationary distributions. The paper should note that the policy's response to AoI values outside the training distribution is not validated.
Simulated Author's Rebuttal
We thank the referee for a careful and constructive review. The referee correctly identifies that the ablation design, covariance derivation, and honest reporting of tradeoffs are strengths of the paper. We address each major comment below and commit to revisions where the referee's points reveal genuine gaps in our analysis or framing.
read point-by-point responses
-
Referee: Section V-C, Contribution 2 interpretation: The paper attributes the perception-consistent reward's advantage to 'reward-observation alignment' — specifically, that noisy rewards penalize noise-fragile optima because 'the same configuration receives different reward values depending on its sensitivity to the noise actually present.' However, an alternative explanation is that the stochasticity in the noisy reward path acts as implicit exploration or regularization, preventing premature convergence to any fixed geometric equilibrium — regardless of whether the noise is correlated with the policy's observations. The per-timestep analysis (Fig. 5) actually supports this alternative: the clean-reward policy converges to a 'stable geometric equilibrium' with 'monotonically decreasing RMSE,' while the noisy-reward policy 'lacks a fixed attractor' and shows 'gradual RMSE increase.' This pattern
Authors: The referee raises a legitimate and incisive alternative hypothesis. We agree that the current manuscript does not present a discriminating experiment to distinguish 'reward-observation alignment' from 'generic reward stochasticity as regularization.' The referee's proposed control — a clean-state reward with additive i.i.d. Gaussian noise matched in variance to the perception-consistent path's reward variance — is exactly the right experiment to run. We will conduct this experiment and report the results in the revised manuscript. If the matched-variance noisy clean reward performs comparably to the perception-consistent reward, we will revise Contribution 2's framing accordingly, downgrading the claim from 'reward-observation alignment as a design axis' to 'reward stochasticity as a regularization mechanism.' If the perception-consistent reward outperforms the matched-variance control, this would support the alignment hypothesis. Either way, the current claim is overstated without this test, and we will present the result honestly. revision: yes
-
Referee: is consistent with reward stochasticity disrupting convergence rather than specifically teaching noise-robustness. The paper does not test this alternative. A discriminating experiment would compare the perception-consistent reward against a clean-state reward with additive i.i.d. Gaussian noise matched in variance to the perception-consistent path's reward variance. If both yield similar aggregate metrics, the benefit is generic reward stochasticity rather than noise-observation correlation. Without this test, the 'reward-observation alignment as a design axis' claim (Section V-C, final paragraph) is not fully substantiated. This is load-bearing for Contribution 2's novelty framing.
Authors: This is a continuation of the previous comment and we treat it as the same point. We agree that the discriminating experiment is necessary and will include it. We also note that there is a theoretical reason to expect the two hypotheses to yield different outcomes: under the perception-consistent reward, the noise injected into the reward is correlated with the noise in the policy's observations — the same measurement perturbation affects both what the policy sees and what it is rewarded for. Under i.i.d. additive reward noise, the reward perturbation is independent of the observation noise. This structural difference means that perception-consistent rewards provide gradient information about which configurations are robust to the specific noise realization the policy is currently experiencing, whereas i.i.d. noise provides no such configuration-specific signal. However, we acknowledge this is a theoretical argument, not empirical evidence, and the referee is correct that without the experiment the claim is not substantiated. We will run the experiment and revise the framing based on the results. revision: yes
-
Referee: Section V-C and Table II, clean-reward row: The clean-reward policy achieves higher triangulation validity (79.1% vs. 78.1%) and visibility (86.7% vs. 84.1%) than the noisy-reward policy, yet the paper frames the noisy-reward policy as superior in 'aggregate metrics.' The RMSE advantage (0.547 vs. 0.633 m) is the primary basis for this claim, but the clean-reward policy's higher validity and visibility suggest it maintains tracking more reliably. The paper should clarify which metrics constitute 'aggregate' superiority and acknowledge that the dominance is metric-dependent rather than uniform. The current framing in the abstract ('perception-consistent rewards outperform privileged clean-state rewards') overstates a mixed result.
Authors: The referee is correct. The clean-reward policy does achieve higher triangulation validity (79.1% vs. 78.1%) and visibility (86.7% vs. 84.1%), and our use of 'aggregate metrics' and 'outperform' in the abstract overstates what is a mixed result. The perception-consistent reward's advantages are specifically in RMSE (0.547 vs. 0.633 m), track losses (6.1 vs. 8.3 per episode), and collisions (0.155% vs. 0.251%), while the clean-reward policy maintains better tracking continuity. We will revise the abstract and Section V-C to state this explicitly: the perception-consistent reward achieves lower RMSE and fewer track losses/collisions, while the privileged reward achieves higher validity and visibility. We will replace 'outperform' with a precise enumeration of which metrics favor each formulation and acknowledge that neither dominates. The phrase 'aggregate metrics' will be removed or replaced with the specific metrics referenced. revision: yes
-
Referee: Section VI, scaling beyond two agents: The paper notes that preliminary three-agent experiments show collision rates rising 9.5x. Given that multi-agent coordination is central to the paper's framing, this limitation is significant. The authors attribute it to insufficient collision-risk encoding in observations/rewards, but this suggests the reward formulation (Contribution 2) may not generalize beyond the minimal two-agent case. The paper should discuss whether the perception-consistent reward's advantage over the clean reward persists with more agents, or whether this is an artifact of the two-agent setting.
Authors: We agree that the three-agent collision result is a significant limitation that the paper should discuss more thoroughly. To be transparent: we have not yet conducted the perception-consistent vs. clean-reward comparison with three agents, so we cannot state whether the advantage persists. The 9.5x collision rate increase was observed with the perception-consistent reward configuration only. The referee's concern that the reward formulation may not generalize is legitimate — the pairwise collision penalty (Eq. 12) does not scale gracefully to larger teams because it does not encode joint collision risk (multiple agents approaching the same region simultaneously). We will expand Section VI to explicitly state that the perception-consistent vs. clean-reward comparison has only been validated for two agents, that we cannot currently claim the advantage generalizes, and that the collision-encoding deficiency may interact with the reward formulation in ways we have not tested. We will also note this as a boundary on Contribution 2's scope rather than deferring it entirely to future work. revision: yes
Circularity Check
No significant circularity detected
full rationale
The paper's three contributions are each validated by controlled ablations against external baselines, not by self-citation chains or definitional reductions. (1) The AoI ablation (Table II, 'No AoI' row) removes AoI from observations and measures the resulting degradation—this is a standard controlled experiment, not a fitted prediction renamed as a result. (2) The perception-consistent vs. privileged reward comparison (Table II) trains two policies under identical observation noise but different reward computation paths; the reported metrics (0.547 m vs. 0.633 m RMSE) are measured outcomes, not parameters fitted to the evaluation data. The skeptic's concern that the benefit may stem from generic reward stochasticity rather than specific noise-observation correlation is a correctness/interpretation risk, not a circularity issue—the ablation itself is clean and the result is not forced by construction. (3) The multi-source covariance derivation (Eqs. 14–17) is a parameter-free analytical Jacobian propagation given stated noise models; the 2.8-fold RMSE improvement over angular-only covariance (Table II) is measured against Gavin et al.'s [5] externally published formulation, not a self-cited result. Self-citations are minimal: the paper builds on Gavin et al. [5] (different authors), Fu et al. [6] (different authors), Baisero et al. [8] (different authors), and MAPPO [18] (different authors). No cited theorem or ansatz from the present authors is used to declare a result forced. The covariance propagation is standard linearization (Eq. 14: first-order Taylor expansion), not a renamed empirical pattern. The derivation chain is self-contained against external benchmarks.
Axiom & Free-Parameter Ledger
free parameters (4)
- Reward weights (s_tri, s_ctr, s_sz, s_coll, λ_ctr, λ_sz, action reg.) =
Not numerically specified
- Noise standard deviations (pixel, pose, gimbal, intrinsics) =
Not numerically specified
- Delay parameters (sampling interval, latency, dropout probability) =
50–1000 ms range stated
- Curriculum schedule boundaries =
Phase 1: 0–60k, Phase 2: 20k–100k, Phase 3: 80k–200k
axioms (4)
- domain assumption Pinhole camera model with linear optical zoom (f_x(z) = z·f_x,0)
- domain assumption First-order linearization adequately approximates triangulation covariance
- ad hoc to paper Stationary stochastic delay model represents real C-UAS communication
- domain assumption Two-agent triangulation is the minimal representative case for multi-agent coordination
Reference graph
Works this paper leans on
-
[1]
A. Barisic, M. Ball, N. Jackson, R. McCarthy, N. Naimi, L. Strassle, J. Becker, M. Brunner, J. Fricke, L. Markovic, I. Seslar, D. Novick, J. Salton, R. Siegwart, S. Bogdan, and R. Fierro, “Multi-robot system for autonomous cooperative counter-uas missions: Design, integration, and field testing,” in2022 IEEE International Symposium on Safety, Security, an...
work page 2022
-
[2]
A review of counter-uas technologies for cooperative defensive teams of drones,
V . U. Castrillo, A. Manco, D. Pascarella, and G. Gigante, “A review of counter-uas technologies for cooperative defensive teams of drones,” Drones, vol. 6, p. 65, 3 2022
work page 2022
-
[3]
R. Hartley and A. Zisserman,Multiple view geometry in computer vision. Cambridge University Press, 2003
work page 2003
-
[4]
Pampc: Perception- aware model predictive control for quadrotors,
D. Falanga, P. Foehn, P. Lu, and D. Scaramuzza, “Pampc: Perception- aware model predictive control for quadrotors,” in2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 10 2018, pp. 1–8
work page 2018
-
[5]
Multi-agent reinforcement learning based drone guidance for n-view triangulation,
T. Gavin, S. LacroiX, and M. Bronz, “Multi-agent reinforcement learning based drone guidance for n-view triangulation,” in2024 International Conference on Unmanned Aircraft Systems (ICUAS), 2024, pp. 578–585
work page 2024
-
[6]
S. Fu, S. Chen, S. Zhao, L. Bai, H. Liang, T. Li, and Y . Yan, “Rainbow delay compensation: A multi-agent reinforcement learning framework for mitigating observation delays,” inThe Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025
work page 2025
-
[7]
Addressing signal delay in deep reinforcement learning,
W. Wang, D. Han, X. Luo, and D. Li, “Addressing signal delay in deep reinforcement learning,” inThe Twelfth International Conference on Learning Representations, 2024
work page 2024
-
[8]
Asymmetric DQN for partially observable reinforcement learning,
A. Baisero, B. Daley, and C. Amato, “Asymmetric DQN for partially observable reinforcement learning,” inProceedings of the Thirty- Eighth Conference on Uncertainty in Artificial Intelligence, ser. Pro- ceedings of Machine Learning Research, vol. 180. PMLR, 2022, pp. 107–117
work page 2022
-
[9]
Swarm-based counter uav defense system,
M. R. Brust, G. Danoy, D. H. Stolfi, and P. Bouvry, “Swarm-based counter uav defense system,”Discover Internet of Things, vol. 1, p. 2, 12 2021
work page 2021
-
[10]
On onboard lidar-based flying object detection,
M. Vrba, V . Walter, V . Pritzl, M. Pliska, T. Baca, V . Spurny, D. Hert, and M. Saska, “On onboard lidar-based flying object detection,”IEEE Transactions on Robotics, vol. 41, pp. 593–611, 2025
work page 2025
-
[11]
A. Saviolo and G. Loianno, “Nova: Navigation via object-centric visual autonomy for high-speed target tracking in unstructured gps- denied environments,”arXiv preprint arXiv:2506.18689, 7 2025
work page internal anchor Pith review Pith/arXiv arXiv 2025
-
[12]
Error modeling in stereo navigation,
L. S. M. Shafer, L. Matthies, and S. A. Shafer, “Error modeling in stereo navigation,” Tech. Rep., 1987
work page 1987
-
[13]
Propagation of uncertainty through stereo triangulation,
G. D. Leo, C. Liguori, and A. Paolillo, “Propagation of uncertainty through stereo triangulation,” in2010 IEEE Instrumentation & Mea- surement Technology Conference Proceedings. IEEE, 2010, pp. 12– 17
work page 2010
-
[14]
Robust Uncertainty-Aware Multiview Triangulation
S. H. Lee and J. Civera, “Robust uncertainty-aware multiview trian- gulation,”arXiv preprint arXiv:2008.01258, 8 2020
work page internal anchor Pith review Pith/arXiv arXiv 2008
-
[15]
O. A. Amodu, C. Jarray, R. Azlina Raja Mahmood, H. Althumali, U. Ali Bukar, R. Nordin, N. F. Abdullah, and N. Cong Luong, “Deep reinforcement learning for aoi minimization in uav-aided data collection for wsn and iot applications: A survey,”IEEE Access, vol. 12, pp. 108 000–108 040, 2024
work page 2024
-
[16]
Real-time status: How often should one update?
S. Kaul, R. Yates, and M. Gruteser, “Real-time status: How often should one update?” in2012 Proceedings IEEE INFOCOM, March 2012, pp. 2731–2735
work page 2012
-
[17]
S. Roy, A. Bisht, A. K. Das, S. Shetty, and M. S. Hossain, “Age of correlated information-optimal dynamic policy scheduling for sus- tainable green iot devices: A multi-agent deep reinforcement learning approach,”Internet of Things, vol. 26, p. 101141, 2024
work page 2024
-
[18]
The surprising effectiveness of ppo in cooperative, multi- agent games,
C. Yu, A. Velu, E. Vinitsky, J. Gao, Y . Wang, A. Bayen, and Y . Wu, “The surprising effectiveness of ppo in cooperative, multi- agent games,”Advances in Neural Information Processing Systems (NeurIPS), Datasets and Benchmarks Track, vol. 35, 11 2022
work page 2022
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.