REVIEW 4 major objections 5 minor 18 references
A MARL Based Multi-Target Tracking Algorithm Under Jamming Against Radar
T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read Adding a simulated-annealing repair to MAPPO yields the best radar tracking accuracy for a UAV swarm under jamming.
desk verdict A genuinely new radar work-mode selection problem with a plausible SA-based action repair, but the empirical comparison is invalidated by a 5000x learning-rate asymmetry in the baselines. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The performance metric is the multi-target average Cramér-Rao lower bound $LB_{T_k}$, computed from expressions (3)-(4), which depend only on UAV–target geometry and per-radar noise factors $f$; the reward (9) combines a shared reward negatively correlated with $LB_{T_k}$, a per-agent distinct reward that penalizes 'lazy' agents, and a penalty for violating the inter-UAV and UAV–target distance constraints. The distinguishing mechanism is the simulated-annealing module, whose objective (10) minimizes the negative predicted shared reward plus a large penalty for violating the mobility constraint and a constant penalty when the repaired position would fall within $d_2 + 3\sigma_{\mathrm{pred}}$ of any target; this module is triggered when the raw MAPPO action would breach constraint (8). The SA module is the component that the paper credits for both constraint satisfaction and efficient learning.
What would settle it
Measure the actual position estimation error (for example, the root-mean-square error of an extended Kalman filter) in the same simulated scenario and compare the algorithm rankings with the paper's CRLB-based $TE$ ranking; if a non-jamming target is lost because all nearby UAVs are in passive mode, or if the penalized distance constraint $d_2$ is violated in practice to improve the CRLB score, the central claim that the SA-repaired policy performs best would be refuted.
Extended reading notes
Core claim
The central claim is that a multi-agent reinforcement learning controller can handle radar jamming in multi-target tracking if each UAV's action (movement and active/passive mode) is repaired by a simulated-annealing step whenever it would place the UAV too close to a target. The paper reports that in a scenario with six UAVs, three targets, and a 50% chance each target carries a jammer, the proposed MAPPO-plus-SA algorithm achieves the best tracking error ($TE$) among MAPPO, MATD3, and MADDPG, with the plain MAPPO losing performance mainly because of constraint violations, and the off-policy algorithms failing to explore the large action space effectively. The authors conclude that the SA repair mechanism "is effective in both avoiding violation of constraints and leading to efficient learning."
Load-bearing premise
The entire evaluation rests on the assumption that the Cramér-Rao lower bound expressions (3)-(4), with noise factors $f$ treated as constants, faithfully represent the true radar tracking error under jamming and target motion.
Editorial extensions
If this is right
- UAV swarms can maintain multi-target tracking accuracy in jamming environments by jointly switching radar modes and adjusting trajectories, and the policy runs per-agent from local observations without a central coordinator.
- The simulated-annealing repair mechanism enforces a hard geometric constraint without heavy reward-shaping, suggesting it could make learned multi-agent controllers safer in other domains with similar constraints.
- The Cramér-Rao lower bound provides a geometry-driven training signal that depends only on relative positions and noise factors, so the same reward design can be reused for other sensing-motion planning problems.
- The comparison indicates that on-policy MARL with action repair is a more reliable baseline than off-policy algorithms (MATD3, MADDPG) for anti-jamming multi-target tracking in high-dimensional action spaces.
Reading between the lines
- The SA repair module acts as a safety filter over the learned policy, and the same pattern could be transferred to any MARL control task where constraint violations are rare but catastrophic, such as multi-robot collision avoidance or autonomous driving.
- Because the CRLB model treats noise factors as constants, transferring the method to real radar would require estimating those factors online; a testable extension is to feed measured noise statistics into the reward so the policy adapts to changing jammer power or target radar cross-section.
- Given the paper's rotationally symmetric training setup, a cheap validation is to re-run the simulation with targets arriving from a fixed direction and check whether tracking performance degrades.
- The paper only evaluates in simulation; hardware-in-the-loop testing with realistic work-mode switching latency and direction-of-arrival estimation errors would show whether the SA repair still improves tracking with imperfect sensors.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper studies a multi-UAV multi-target tracking scenario in which each UAV radar can operate in active or passive mode and targets may carry jammers against radar. The authors formulate the problem as a Dec-POMDP, propose a MAPPO-based algorithm augmented with a simulated annealing (SA) module that repairs constraint-violating actions, and evaluate it against MAPPO, MATD3, and MADDPG in simulation. The central claim is that the proposed algorithm is effective in avoiding constraint violations and achieves the best tracking performance, measured by TE, the geometric-mean CRLB over the episode.
Significance. If the empirical claims are confirmed, the SA action-repair mechanism is a simple and potentially transferable way to enforce safety constraints in MARL trajectory design, and the active/passive radar work-mode selection is a practically relevant scenario. The paper provides code and a demo video, uses multiple random seeds, and the SA idea is clearly presented. However, the headline comparison is currently undermined by a large hyperparameter asymmetry, and the evaluation metric is the same CRLB quantity used in the training reward, so the transferability of the reported performance gain to real tracking is not yet established.
major comments (4)
- [Section IV, Fig. 3] The reported learning rates are 5e-5 for the proposed method and MAPPO versus 1e-8 for MATD3 and MADDPG, a 5000-fold difference. At such a low rate, the off-policy baselines' updates are effectively frozen, so the observation that they 'fail to learn good decisions' is confounded. The statement that all hyperparameters were tuned to suitable values is not consistent with this choice. Please re-run the baselines with matched and individually tuned learning rates and report confidence intervals or standard deviations over seeds; alternatively, restrict the empirical claim to the MAPPO versus MAPPO+SA comparison, which does not suffer from this asymmetry.
- [Section III.A Eq. (9), Section III.B Eq. (10), Section II.B Eq. (5)] The shared reward Rs in (9) and the SA objective in (10) are both constructed from LBTk, the same average CRLB whose geometric mean defines the reported performance metric TE. Consequently, Fig. 3(b) measures how well the policy optimizes its own training objective rather than an independent tracking benchmark. Please validate the learned policy with an out-of-sample metric, for example Monte Carlo simulated radar measurements followed by a tracking filter, and discuss the assumptions under which the CRLB is a faithful proxy, specifically the constant noise factors f_{i,j,k,1}, f_{i,j,k,2}, f_{n(i),j,k}, the absence of detection loss, and Gaussian noise.
- [Section III, first paragraph] The paper states that the feasible set determined by (7) and (8) is not convex and therefore 'the optimization cannot be solved directly.' Non-convexity alone does not imply that the problem is unsolvable by direct optimization; many non-convex problems are addressed by exact or heuristic solvers. If the claim of proving difficulty is to be retained, a more precise hardness argument, such as NP-hardness or a demonstration that standard nonlinear solvers fail on this instance, is needed.
- [Section III.B, Eq. (10)] In the SA objective, the indicator I(||(Δxi,Tk, Δyi,Tk)|| < d0) appears with a negative sign inside the parentheses. Since the SA module minimizes Ri,Tk, actions satisfying the mobility constraint (6) receive an additional +L and are thereby penalized, while violating actions are favored. This is the opposite of enforcement. Please correct the sign or the inequality, and ensure the implementation matches the corrected equation; otherwise the mechanism based on SA to avoid constraint violations is not correctly specified.
minor comments (5)
- [Section II.B] The term 'Additional White Gaussian Noise' should be 'Additive White Gaussian Noise'.
- [Section III.A] There is a typo: 'posibility' should be 'possibility' in the description of the transition function P.
- [Section II.B, Eq. (4)] Please clarify that CRLB2 is the estimation CRLB of the jammer position and explain how this maps to the target position when the jammer is carried by the target.
- [Section III.B] The statement that constraint (6) is 'naturally obeyed' needs a brief explanation of how the action space or clipping guarantees ||Δpi,Tk|| ≤ d0.
- [Fig. 4 caption] The phrase '3 forks are targets' is unclear; use 'crosses' or 'stars' to describe the target markers.
Circularity Check
No significant circularity: reward and evaluation share the same CRLB objective, but this is standard RL objective alignment, not a fitted-input prediction or self-citation chain.
full rationale
The paper's reward in Eq. (9) and SA objective in Eq. (10) are both built from the CRLB quantity LBTk, and the reported metric TE is also defined from LBTk. This overlap is not circular in the sense defined here: the paper does not fit a parameter to TE and then claim to predict TE independently. Rather, it trains agents to optimize a tracking objective and evaluates them on that same objective, which is the normal and transparent RL procedure. No self-citations are load-bearing; the cited CRLB model [16], MAPPO baseline [18], and anti-collision mechanism [15] are external works with no author overlap. The claim that the proposed algorithm has the best performance is an empirical comparison under a common reward, not a derivation forced by construction. The learning-rate asymmetry between MAPPO (5e-5) and MATD3/MADDPG (1e-8) is a legitimate concern about comparison fairness, but it is a correctness/experimental-design issue, not a circularity step under the specified criteria.
Assumptions & free parameters
free parameters (4)
- CRLB noise factors f_{i,j,k,1}, f_{i,j,k,2}, f_{n(i),j,k} =
Unspecified
- Reward coefficient alpha =
Unspecified
- SA penalty magnitude L =
Unspecified
- SA temperature parameters and iteration count =
Tmax=100, Tmin=20, 20 iterations
assumptions (5)
- domain assumption CRLB formulas from [16] correctly describe the radar measurement model and its estimation lower bound.
- domain assumption Noise factors f can be treated as time-invariant constants.
- domain assumption Targets carrying jammers can be localized solely from DOA measurements of the jamming signal, and the jammer is collocated with the target.
- domain assumption A separate tracking module produces target predictions with known standard deviation sigma_pred.
- ad hoc to paper Non-convexity of the feasible set implies the optimization cannot be solved directly, justifying the heuristic approach.
Cite this review
Pith. "Pith review of A MARL Based Multi-Target Tracking Algorithm Under Jamming Against Radar." pith.science (2026). https://pith.science/paper/Q3E4FUSQ
@misc{pith2026241212547,
author = {Pith},
title = {Pith review of: A MARL Based Multi-Target Tracking Algorithm Under Jamming Against Radar},
year = {2026},
howpublished = {\url{https://pith.science/paper/Q3E4FUSQ}},
note = {Machine review of arXiv:2412.12547}
}
read the original abstract
Unmanned aerial vehicles (UAVs) have played an increasingly important role in military operations and social life. Among all application scenarios, multi-target tracking tasks accomplished by UAV swarms have received extensive attention. However, when UAVs use radar to track targets, the tracking performance can be severely compromised by jammers. To track targets in the presence of jammers, UAVs can use passive radar to position the jammer. This paper proposes a system where a UAV swarm selects the radar's active or passive work mode to track multiple differently located and potentially jammer-carrying targets. After presenting the optimization problem and proving its solving difficulty, we use a multi-agent reinforcement learning algorithm to solve this control problem. We also propose a mechanism based on simulated annealing algorithm to avoid cases where UAV actions violate constraints. Simulation experiments demonstrate the effectiveness of the proposed algorithm.
Figures
Figures from the paper (1 more)
Reference graph
Works this paper leans on
-
[15]
Madrl-based uavs trajectory design with anti-collision mecha- nism in vehicular networks,
Leonardo Spampinato, Enrico Testi, Chiara Buratti, and Riccardo Marini, “Madrl-based uavs trajectory design with anti-collision mecha- nism in vehicular networks,” in ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , 2024, pp. 12976–12980
work page 2024
-
[1]
Wireless communications with unmanned aerial vehicles: opportunities and challenges,
Yong Zeng, Rui Zhang, and Teng Joon Lim, “Wireless communications with unmanned aerial vehicles: opportunities and challenges,” IEEE Communications Magazine , vol. 54, no. 5, pp. 36–42, 2016
work page 2016
-
[2]
Multiple moving targets surveillance based on a cooperative network for multi-uav,
Jingjing Gu, Tao Su, Qiuhong Wang, Xiaojiang Du, and Mohsen Guizani, “Multiple moving targets surveillance based on a cooperative network for multi-uav,” IEEE Communications Magazine , vol. 56, no. 4, pp. 82–89, 2018
work page 2018
-
[3]
Tracking of multiple targets using optimal number of uavs,
Arun Das, Shahrzad Shirazipourazad, David Hay, and Arunabha Sen, “Tracking of multiple targets using optimal number of uavs,” IEEE Transactions on Aerospace and Electronic Systems , vol. 55, no. 4, pp. 1769–1784, 2019
work page 2019
-
[4]
Dynamic uav swarm confrontation: An imitation based on mobile adaptive net- works,
Wei Xia, Zhuoyang Zhou, Wanyue Jiang, and Yuhan Zhang, “Dynamic uav swarm confrontation: An imitation based on mobile adaptive net- works,” IEEE Transactions on Aerospace and Electronic Systems , vol. 59, no. 5, pp. 7183–7202, 2023
work page 2023
-
[5]
Detection and monitoring of people in collapsed buildings using a rotating radar on a uav,
Philipp Stockel, Patrick Wallrath, Reinhold Herschel, and Nils Pohl, “Detection and monitoring of people in collapsed buildings using a rotating radar on a uav,” IEEE Transactions on Radar Systems , vol. 2, pp. 13–23, 2024
work page 2024
-
[6]
Uav- enabled integrated sensing and communication: Tracking design and optimization,
Yifan Jiang, Qingqing Wu, Wen Chen, and Kaitao Meng, “Uav- enabled integrated sensing and communication: Tracking design and optimization,” IEEE Communications Letters , vol. 28, no. 5, pp. 1024– 1028, 2024
work page 2024
-
[7]
Uav operation time minimization for wireless-powered data collection,
Yijia Zhang, Deepak Mishra, Hassan Habibi Gharakheili, and Derrick Wing Kwan Ng, “Uav operation time minimization for wireless-powered data collection,” in ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , 2024, pp. 46–50
work page 2024
Show all 18 references
-
[8]
Dynamic uav swarm collaboration for multi-targets track- ing under malicious jamming: Joint power, path and target association optimization,
Lanhua Xiang, Fengyu Wang, Wenjun Xu, Tiankui Zhang, Miao Pan, and Zhu Han, “Dynamic uav swarm collaboration for multi-targets track- ing under malicious jamming: Joint power, path and target association optimization,” IEEE Transactions on V ehicular Technology, vol. 73, no. 4...
2024
-
[9]
Reinforcement learning for joint detection & mapping using dynamic uav networks,
Anna Guerra, Francesco Guidi, Davide Dardari, and Petar M. Djuric, “Reinforcement learning for joint detection & mapping using dynamic uav networks,” IEEE Transactions on Aerospace and Electronic Systems, pp. 1–16, 2023
2023
-
[10]
Deep reinforcement learning multi-uav trajec- tory control for target tracking,
Jiseon Moon, Savvas Papaioannou, Christos Laoudias, Panayiotis Ko- lios, and Sunwoo Kim, “Deep reinforcement learning multi-uav trajec- tory control for target tracking,” IEEE Internet of Things Journal , vol. 8, no. 20, pp. 15441–15455, 2021
2021
-
[11]
Trajectory design and power control for joint radar and communication enabled multi-uav cooperative detection systems,
Tao Zhang, Kun Zhu, Shaoqiu Zheng, Dusit Niyato, and Nguyen Cong Luong, “Trajectory design and power control for joint radar and communication enabled multi-uav cooperative detection systems,” IEEE Transactions on Communications , vol. 71, no. 1, pp. 158–172, 2023
2023
-
[12]
Multi-agent reinforcement learning aided intelligent uav swarm for target tracking,
Zhaoyue Xia, Jun Du, Jingjing Wang, Chunxiao Jiang, Yong Ren, Gang Li, and Zhu Han, “Multi-agent reinforcement learning aided intelligent uav swarm for target tracking,” IEEE Transactions on V ehicular Technology, vol. 71, no. 1, pp. 931–945, 2022
2022
-
[13]
Intelligent uav swarm cooperation for multiple targets tracking,
Longyu Zhou, Supeng Leng, Qiang Liu, and Qing Wang, “Intelligent uav swarm cooperation for multiple targets tracking,” IEEE Internet of Things Journal , vol. 9, no. 1, pp. 743–754, 2022
2022
-
[14]
Integrated sensing and communication in uav swarms for cooperative multiple targets tracking,
Longyu Zhou, Supeng Leng, Qing Wang, and Qiang Liu, “Integrated sensing and communication in uav swarms for cooperative multiple targets tracking,” IEEE Transactions on Mobile Computing , vol. 22, no. 11, pp. 6526–6542, 2023
2023
-
[16]
Cooperative target assignment and dwell allocation for multiple target tracking in phased array radar network,
Junkun Yan, Wenqiang Pu, Hongwei Liu, Shenghua Zhou, and Zheng Bao, “Cooperative target assignment and dwell allocation for multiple target tracking in phased array radar network,” Signal Processing, vol. 141, no. dec., pp. 74–83, 2017
2017
-
[17]
[springerbriefs in intelligent systems] a concise introduction to decentralized pomdps —— multia- gent systems under uncertainty,
Frans A. Oliehoek and Christopher Amato, “[springerbriefs in intelligent systems] a concise introduction to decentralized pomdps —— multia- gent systems under uncertainty,” vol. 10.1007/978-3-319-28929-8, no. Chapter 1, pp. 1–9, 2016
-
[18]
The surprising effectiveness of mappo in cooperative, multi-agent games,
Chao Yu, Akash Velu, Eugene Vinitsky, Yu Wang, and Yi Wu, “The surprising effectiveness of mappo in cooperative, multi-agent games,” 2021
2021
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.