Pith. sign in

REVIEW 4 major objections 5 minor 18 references

A MARL Based Multi-Target Tracking Algorithm Under Jamming Against Radar

T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Pith's one-line read Adding a simulated-annealing repair to MAPPO yields the best radar tracking accuracy for a UAV swarm under jamming.

desk verdict A genuinely new radar work-mode selection problem with a plausible SA-based action repair, but the empirical comparison is invalidated by a 5000x learning-rate asymmetry in the baselines. read the letter →

arxiv 2412.12547 v1 pith:Q3E4FUSQ submitted 2024-12-17 eess.SY cs.SYeess.SP

classification eess.SYcs.SYeess.SP
keywords UAVswarmmulti-targettrackingradarjammingmulti-agentreinforcementlearningMAPPOsimulatedannealingCramér-Raolowerboundconstraintsatisfaction
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

A UAV swarm tracking several targets often loses accuracy when some targets carry radar jammers, because active radar measurements are corrupted. The paper proposes letting each UAV choose between active mode (range and angle measurements) and passive mode (direction-finding on the jammer's own signal), and treating mode selection together with movement as a multi-agent control problem. The authors show the resulting constrained optimization is non-convex in its feasible set and therefore hard to solve directly, and instead train a multi-agent reinforcement learning policy based on MAPPO. To prevent the policy from violating a minimum UAV–target distance constraint, they wrap the actions in a simulated-annealing module that repairs unsafe moves while preserving the learned strategy. In simulation, this combination avoids constraint violations and achieves lower tracking error (higher $TE$) than MAPPO, MATD3, or MADDPG, which is the paper's central claim.

What carries the argument

The performance metric is the multi-target average Cramér-Rao lower bound $LB_{T_k}$, computed from expressions (3)-(4), which depend only on UAV–target geometry and per-radar noise factors $f$; the reward (9) combines a shared reward negatively correlated with $LB_{T_k}$, a per-agent distinct reward that penalizes 'lazy' agents, and a penalty for violating the inter-UAV and UAV–target distance constraints. The distinguishing mechanism is the simulated-annealing module, whose objective (10) minimizes the negative predicted shared reward plus a large penalty for violating the mobility constraint and a constant penalty when the repaired position would fall within $d_2 + 3\sigma_{\mathrm{pred}}$ of any target; this module is triggered when the raw MAPPO action would breach constraint (8). The SA module is the component that the paper credits for both constraint satisfaction and efficient learning.

What would settle it

Measure the actual position estimation error (for example, the root-mean-square error of an extended Kalman filter) in the same simulated scenario and compare the algorithm rankings with the paper's CRLB-based $TE$ ranking; if a non-jamming target is lost because all nearby UAVs are in passive mode, or if the penalized distance constraint $d_2$ is violated in practice to improve the CRLB score, the central claim that the SA-repaired policy performs best would be refuted.

Watch

Extended reading notes

Core claim

The central claim is that a multi-agent reinforcement learning controller can handle radar jamming in multi-target tracking if each UAV's action (movement and active/passive mode) is repaired by a simulated-annealing step whenever it would place the UAV too close to a target. The paper reports that in a scenario with six UAVs, three targets, and a 50% chance each target carries a jammer, the proposed MAPPO-plus-SA algorithm achieves the best tracking error ($TE$) among MAPPO, MATD3, and MADDPG, with the plain MAPPO losing performance mainly because of constraint violations, and the off-policy algorithms failing to explore the large action space effectively. The authors conclude that the SA repair mechanism "is effective in both avoiding violation of constraints and leading to efficient learning."

Load-bearing premise

The entire evaluation rests on the assumption that the Cramér-Rao lower bound expressions (3)-(4), with noise factors $f$ treated as constants, faithfully represent the true radar tracking error under jamming and target motion.

Editorial extensions

If this is right

  • UAV swarms can maintain multi-target tracking accuracy in jamming environments by jointly switching radar modes and adjusting trajectories, and the policy runs per-agent from local observations without a central coordinator.
  • The simulated-annealing repair mechanism enforces a hard geometric constraint without heavy reward-shaping, suggesting it could make learned multi-agent controllers safer in other domains with similar constraints.
  • The Cramér-Rao lower bound provides a geometry-driven training signal that depends only on relative positions and noise factors, so the same reward design can be reused for other sensing-motion planning problems.
  • The comparison indicates that on-policy MARL with action repair is a more reliable baseline than off-policy algorithms (MATD3, MADDPG) for anti-jamming multi-target tracking in high-dimensional action spaces.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The SA repair module acts as a safety filter over the learned policy, and the same pattern could be transferred to any MARL control task where constraint violations are rare but catastrophic, such as multi-robot collision avoidance or autonomous driving.
  • Because the CRLB model treats noise factors as constants, transferring the method to real radar would require estimating those factors online; a testable extension is to feed measured noise statistics into the reward so the policy adapts to changing jammer power or target radar cross-section.
  • Given the paper's rotationally symmetric training setup, a cheap validation is to re-run the simulation with targets arriving from a fixed direction and check whether tracking performance degrades.
  • The paper only evaluates in simulation; hardware-in-the-loop testing with realistic work-mode switching latency and direction-of-arrival estimation errors would show whether the SA repair still improves tracking with imperfect sensors.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. This paper studies a multi-UAV multi-target tracking scenario in which each UAV radar can operate in active or passive mode and targets may carry jammers against radar. The authors formulate the problem as a Dec-POMDP, propose a MAPPO-based algorithm augmented with a simulated annealing (SA) module that repairs constraint-violating actions, and evaluate it against MAPPO, MATD3, and MADDPG in simulation. The central claim is that the proposed algorithm is effective in avoiding constraint violations and achieves the best tracking performance, measured by TE, the geometric-mean CRLB over the episode.

Significance. If the empirical claims are confirmed, the SA action-repair mechanism is a simple and potentially transferable way to enforce safety constraints in MARL trajectory design, and the active/passive radar work-mode selection is a practically relevant scenario. The paper provides code and a demo video, uses multiple random seeds, and the SA idea is clearly presented. However, the headline comparison is currently undermined by a large hyperparameter asymmetry, and the evaluation metric is the same CRLB quantity used in the training reward, so the transferability of the reported performance gain to real tracking is not yet established.

major comments (4)
  1. [Section IV, Fig. 3] The reported learning rates are 5e-5 for the proposed method and MAPPO versus 1e-8 for MATD3 and MADDPG, a 5000-fold difference. At such a low rate, the off-policy baselines' updates are effectively frozen, so the observation that they 'fail to learn good decisions' is confounded. The statement that all hyperparameters were tuned to suitable values is not consistent with this choice. Please re-run the baselines with matched and individually tuned learning rates and report confidence intervals or standard deviations over seeds; alternatively, restrict the empirical claim to the MAPPO versus MAPPO+SA comparison, which does not suffer from this asymmetry.
  2. [Section III.A Eq. (9), Section III.B Eq. (10), Section II.B Eq. (5)] The shared reward Rs in (9) and the SA objective in (10) are both constructed from LBTk, the same average CRLB whose geometric mean defines the reported performance metric TE. Consequently, Fig. 3(b) measures how well the policy optimizes its own training objective rather than an independent tracking benchmark. Please validate the learned policy with an out-of-sample metric, for example Monte Carlo simulated radar measurements followed by a tracking filter, and discuss the assumptions under which the CRLB is a faithful proxy, specifically the constant noise factors f_{i,j,k,1}, f_{i,j,k,2}, f_{n(i),j,k}, the absence of detection loss, and Gaussian noise.
  3. [Section III, first paragraph] The paper states that the feasible set determined by (7) and (8) is not convex and therefore 'the optimization cannot be solved directly.' Non-convexity alone does not imply that the problem is unsolvable by direct optimization; many non-convex problems are addressed by exact or heuristic solvers. If the claim of proving difficulty is to be retained, a more precise hardness argument, such as NP-hardness or a demonstration that standard nonlinear solvers fail on this instance, is needed.
  4. [Section III.B, Eq. (10)] In the SA objective, the indicator I(||(Δxi,Tk, Δyi,Tk)|| < d0) appears with a negative sign inside the parentheses. Since the SA module minimizes Ri,Tk, actions satisfying the mobility constraint (6) receive an additional +L and are thereby penalized, while violating actions are favored. This is the opposite of enforcement. Please correct the sign or the inequality, and ensure the implementation matches the corrected equation; otherwise the mechanism based on SA to avoid constraint violations is not correctly specified.
minor comments (5)
  1. [Section II.B] The term 'Additional White Gaussian Noise' should be 'Additive White Gaussian Noise'.
  2. [Section III.A] There is a typo: 'posibility' should be 'possibility' in the description of the transition function P.
  3. [Section II.B, Eq. (4)] Please clarify that CRLB2 is the estimation CRLB of the jammer position and explain how this maps to the target position when the jammer is carried by the target.
  4. [Section III.B] The statement that constraint (6) is 'naturally obeyed' needs a brief explanation of how the action space or clipping guarantees ||Δpi,Tk|| ≤ d0.
  5. [Fig. 4 caption] The phrase '3 forks are targets' is unclear; use 'crosses' or 'stars' to describe the target markers.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: reward and evaluation share the same CRLB objective, but this is standard RL objective alignment, not a fitted-input prediction or self-citation chain.

full rationale

The paper's reward in Eq. (9) and SA objective in Eq. (10) are both built from the CRLB quantity LBTk, and the reported metric TE is also defined from LBTk. This overlap is not circular in the sense defined here: the paper does not fit a parameter to TE and then claim to predict TE independently. Rather, it trains agents to optimize a tracking objective and evaluates them on that same objective, which is the normal and transparent RL procedure. No self-citations are load-bearing; the cited CRLB model [16], MAPPO baseline [18], and anti-collision mechanism [15] are external works with no author overlap. The claim that the proposed algorithm has the best performance is an empirical comparison under a common reward, not a derivation forced by construction. The learning-rate asymmetry between MAPPO (5e-5) and MATD3/MADDPG (1e-8) is a legitimate concern about comparison fairness, but it is a correctness/experimental-design issue, not a circularity step under the specified criteria.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central claim depends on the fidelity of the CRLB proxy, the unspecified noise factors, and the existence of an accurate tracking predictor. No new physical entities are introduced; the SA module is an algorithmic component, not an entity. The main burden is that the evaluation metric and the training reward are the same CRLB quantity.

free parameters (4)
  • CRLB noise factors f_{i,j,k,1}, f_{i,j,k,2}, f_{n(i),j,k} = Unspecified
    Assumed constant 'without loss of generality' (Sec. II.B), but values are not reported; the simulation results depend on them, and reproduction requires the code.
  • Reward coefficient alpha = Unspecified
    In reward (9); no sensitivity analysis is given.
  • SA penalty magnitude L = Unspecified
    Called 'sufficiently large' in (10); the chosen value affects when constraint (6) is enforced.
  • SA temperature parameters and iteration count = Tmax=100, Tmin=20, 20 iterations
    Hand-chosen in Section IV; no sensitivity study is reported.
assumptions (5)
  • domain assumption CRLB formulas from [16] correctly describe the radar measurement model and its estimation lower bound.
    Used in equations (3)-(5); the noise covariance models and factors f are adopted from a phased-array radar network paper without derivation in this work.
  • domain assumption Noise factors f can be treated as time-invariant constants.
    Sec. II.B states this 'without loss of generality'; in reality RCS, gain, and bandwidth may vary with aspect angle and time.
  • domain assumption Targets carrying jammers can be localized solely from DOA measurements of the jamming signal, and the jammer is collocated with the target.
    Used in (4) and in the PM mode definition; no explicit model of jammer signal structure or multipath.
  • domain assumption A separate tracking module produces target predictions with known standard deviation sigma_pred.
    The SA module uses predicted positions and a 3*sigma_pred margin (Sec. III.B); the tracking module is not modeled or specified.
  • ad hoc to paper Non-convexity of the feasible set implies the optimization cannot be solved directly, justifying the heuristic approach.
    Sec. III: 'feasible set determined by (7) and (8) is not convex. Therefore, the optimization cannot be solved directly.' Non-convexity does not preclude specialized solvers; this is an unstated assumption about available solvers.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A MARL Based Multi-Target Tracking Algorithm Under Jamming Against Radar." pith.science (2026). https://pith.science/paper/Q3E4FUSQ

@misc{pith2026241212547,
  author       = {Pith},
  title        = {Pith review of: A MARL Based Multi-Target Tracking Algorithm Under Jamming Against Radar},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Q3E4FUSQ}},
  note         = {Machine review of arXiv:2412.12547}
}
read the original abstract

Unmanned aerial vehicles (UAVs) have played an increasingly important role in military operations and social life. Among all application scenarios, multi-target tracking tasks accomplished by UAV swarms have received extensive attention. However, when UAVs use radar to track targets, the tracking performance can be severely compromised by jammers. To track targets in the presence of jammers, UAVs can use passive radar to position the jammer. This paper proposes a system where a UAV swarm selects the radar's active or passive work mode to track multiple differently located and potentially jammer-carrying targets. After presenting the optimization problem and proving its solving difficulty, we use a multi-agent reinforcement learning algorithm to solve this control problem. We also propose a mechanism based on simulated annealing algorithm to avoid cases where UAV actions violate constraints. Simulation experiments demonstrate the effectiveness of the proposed algorithm.

Figures

Figures reproduced from arXiv: 2412.12547 by the authors.

Figure 1
Figure 1. Scenario schematic and system workflow B. Performance Indicator In the scenario described previously, UAV actions controlled by the algorithm are a i Tk = (∆x i Tk , ∆y i Tk , ui Tk ) T , where  ∆x i Tk ∆y i Tk  =  x i Tk − x i Tk−1 y i Tk − y i Tk−1  . (1) For ease of presentation, we will subsequently default to this relationship and use p i Tk to represent UAV movement. To represent the performance of target … view at source ↗
Figure 2
Figure 2. Algorithm flow IV. RESULTS In the numerical simulation, we consider N = 6 UAVs track M = 3 targets. The probability of a target carrying a jammer is set to be 0.5. Initial positions of all UAVs are fixed around the coordinate origin without loss of generality. Initial positions of targets have a fixed distribution in polar coordinates. The distance from target initial positions to the coordinate origin is sampled fr… view at source ↗
Figure 4
Figure 4. A running example of our algorithm and track all targets. We propose a MARL algorithm to control UAVs and a mechanism based on SA to avoid action from violating constraints. Simulations illustrate that the proposed algorithm can effectively improve the performance. REFERENCES [1] Yong Zeng, Rui Zhang, and Teng Joon Lim, “Wireless communications with unmanned aerial vehicles: opportunities and challenges,” IEEE Commu… view at source ↗
Figures from the paper (1 more)
Figure 3
Figure 3. Figure 3: Performance of different algorithms In the result, MATD3 and MADDPG, while converging in a short time, fail to learn good decisions due to poor exploration and mismatch between off-policy algorithms and large number of agents. Besides, effect of MAPPO is mainly limited…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

18 extracted references · 17 canonical work pages

  1. [15]

    Madrl-based uavs trajectory design with anti-collision mecha- nism in vehicular networks,

    Leonardo Spampinato, Enrico Testi, Chiara Buratti, and Riccardo Marini, “Madrl-based uavs trajectory design with anti-collision mecha- nism in vehicular networks,” in ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , 2024, pp. 12976–12980

  2. [1]

    Wireless communications with unmanned aerial vehicles: opportunities and challenges,

    Yong Zeng, Rui Zhang, and Teng Joon Lim, “Wireless communications with unmanned aerial vehicles: opportunities and challenges,” IEEE Communications Magazine , vol. 54, no. 5, pp. 36–42, 2016

  3. [2]

    Multiple moving targets surveillance based on a cooperative network for multi-uav,

    Jingjing Gu, Tao Su, Qiuhong Wang, Xiaojiang Du, and Mohsen Guizani, “Multiple moving targets surveillance based on a cooperative network for multi-uav,” IEEE Communications Magazine , vol. 56, no. 4, pp. 82–89, 2018

  4. [3]

    Tracking of multiple targets using optimal number of uavs,

    Arun Das, Shahrzad Shirazipourazad, David Hay, and Arunabha Sen, “Tracking of multiple targets using optimal number of uavs,” IEEE Transactions on Aerospace and Electronic Systems , vol. 55, no. 4, pp. 1769–1784, 2019

  5. [4]

    Dynamic uav swarm confrontation: An imitation based on mobile adaptive net- works,

    Wei Xia, Zhuoyang Zhou, Wanyue Jiang, and Yuhan Zhang, “Dynamic uav swarm confrontation: An imitation based on mobile adaptive net- works,” IEEE Transactions on Aerospace and Electronic Systems , vol. 59, no. 5, pp. 7183–7202, 2023

  6. [5]

    Detection and monitoring of people in collapsed buildings using a rotating radar on a uav,

    Philipp Stockel, Patrick Wallrath, Reinhold Herschel, and Nils Pohl, “Detection and monitoring of people in collapsed buildings using a rotating radar on a uav,” IEEE Transactions on Radar Systems , vol. 2, pp. 13–23, 2024

  7. [6]

    Uav- enabled integrated sensing and communication: Tracking design and optimization,

    Yifan Jiang, Qingqing Wu, Wen Chen, and Kaitao Meng, “Uav- enabled integrated sensing and communication: Tracking design and optimization,” IEEE Communications Letters , vol. 28, no. 5, pp. 1024– 1028, 2024

  8. [7]

    Uav operation time minimization for wireless-powered data collection,

    Yijia Zhang, Deepak Mishra, Hassan Habibi Gharakheili, and Derrick Wing Kwan Ng, “Uav operation time minimization for wireless-powered data collection,” in ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , 2024, pp. 46–50

Show all 18 references
  1. [8]

    Dynamic uav swarm collaboration for multi-targets track- ing under malicious jamming: Joint power, path and target association optimization,

    Lanhua Xiang, Fengyu Wang, Wenjun Xu, Tiankui Zhang, Miao Pan, and Zhu Han, “Dynamic uav swarm collaboration for multi-targets track- ing under malicious jamming: Joint power, path and target association optimization,” IEEE Transactions on V ehicular Technology, vol. 73, no. 4...

  2. [9]

    Reinforcement learning for joint detection & mapping using dynamic uav networks,

    Anna Guerra, Francesco Guidi, Davide Dardari, and Petar M. Djuric, “Reinforcement learning for joint detection & mapping using dynamic uav networks,” IEEE Transactions on Aerospace and Electronic Systems, pp. 1–16, 2023

  3. [10]

    Deep reinforcement learning multi-uav trajec- tory control for target tracking,

    Jiseon Moon, Savvas Papaioannou, Christos Laoudias, Panayiotis Ko- lios, and Sunwoo Kim, “Deep reinforcement learning multi-uav trajec- tory control for target tracking,” IEEE Internet of Things Journal , vol. 8, no. 20, pp. 15441–15455, 2021

  4. [11]

    Trajectory design and power control for joint radar and communication enabled multi-uav cooperative detection systems,

    Tao Zhang, Kun Zhu, Shaoqiu Zheng, Dusit Niyato, and Nguyen Cong Luong, “Trajectory design and power control for joint radar and communication enabled multi-uav cooperative detection systems,” IEEE Transactions on Communications , vol. 71, no. 1, pp. 158–172, 2023

  5. [12]

    Multi-agent reinforcement learning aided intelligent uav swarm for target tracking,

    Zhaoyue Xia, Jun Du, Jingjing Wang, Chunxiao Jiang, Yong Ren, Gang Li, and Zhu Han, “Multi-agent reinforcement learning aided intelligent uav swarm for target tracking,” IEEE Transactions on V ehicular Technology, vol. 71, no. 1, pp. 931–945, 2022

  6. [13]

    Intelligent uav swarm cooperation for multiple targets tracking,

    Longyu Zhou, Supeng Leng, Qiang Liu, and Qing Wang, “Intelligent uav swarm cooperation for multiple targets tracking,” IEEE Internet of Things Journal , vol. 9, no. 1, pp. 743–754, 2022

  7. [14]

    Integrated sensing and communication in uav swarms for cooperative multiple targets tracking,

    Longyu Zhou, Supeng Leng, Qing Wang, and Qiang Liu, “Integrated sensing and communication in uav swarms for cooperative multiple targets tracking,” IEEE Transactions on Mobile Computing , vol. 22, no. 11, pp. 6526–6542, 2023

  8. [16]

    Cooperative target assignment and dwell allocation for multiple target tracking in phased array radar network,

    Junkun Yan, Wenqiang Pu, Hongwei Liu, Shenghua Zhou, and Zheng Bao, “Cooperative target assignment and dwell allocation for multiple target tracking in phased array radar network,” Signal Processing, vol. 141, no. dec., pp. 74–83, 2017

  9. [17]

    [springerbriefs in intelligent systems] a concise introduction to decentralized pomdps —— multia- gent systems under uncertainty,

    Frans A. Oliehoek and Christopher Amato, “[springerbriefs in intelligent systems] a concise introduction to decentralized pomdps —— multia- gent systems under uncertainty,” vol. 10.1007/978-3-319-28929-8, no. Chapter 1, pp. 1–9, 2016

  10. [18]

    The surprising effectiveness of mappo in cooperative, multi-agent games,

    Chao Yu, Akash Velu, Eugene Vinitsky, Yu Wang, and Yi Wu, “The surprising effectiveness of mappo in cooperative, multi-agent games,” 2021

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.