{"id":"97e9cd6d-8648-4070-bb4c-7a185052c588","arxiv_id":"2412.12547","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A multi-agent reinforcement learning controller with a simulated-annealing action filter is shown in simulation to improve radar-based multi-target tracking by UAV swarms when some targets carry jammers.","lead":"A team of simulated drones learns to switch their radars between active and passive modes so they can keep tracking targets even when some targets jam their signals. The paper adds a simulated-annealing correction that stops drones from getting too close to targets, and reports better tracking scores than three reinforcement learning baselines.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Baseline comparison invalidated by 5000x learning-rate asymmetry: MATD3/MADDPG at 1e-8 vs MAPPO at 5e-5, so 'best performance' may not survive fair tuning.","rationale":"I read the central claim as an empirical statement about the simulated scenario. For that statement, the CRLB model fidelity is not the weakest link: CRLB is the declared objective, and the comparison is internal to the simulation. The weakest link is the unfair baseline tuning. Fig. 3 gives LR 1e-8 for MATD3/MADDPG against 5e-5 for MAPPO/SA. That difference is large enough to prevent off-policy baselines from updating meaningfully, making the 'best performance' claim untested. The SA-vs-MAPPO comparison is fair and is the more valuable result. Thus the reader's conditional verdict is appropriate, but the stated reason should emphasize the baseline learning-rate artifact rather than CRLB fidelity.","tokens_in":6733,"tokens_out":6519,"duration_ms":62937,"concrete_test":"Rerun the experiments from Fig. 3 using the released code with MATD3 and MADDPG learning rates swept over {1e-6, 1e-5, 1e-4, 1e-3} (and, if needed, their standard tuned values) while keeping all other hyperparameters, seeds, and evaluation identical to MAPPO/MAPPO+SA. Report the final TE averaged over the 10 seeds with per-seed standard deviations and paired differences. If either baseline matches or exceeds MAPPO+SA, the 'best performance' claim must be withdrawn; if they still underperform, the claim is confirmed as stated.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical claim—that the proposed MAPPO+SA has the best performance among MAPPO, MATD3, and MADDPG—rests on Fig. 3. The reported hyperparameters set the learning rate of the proposed algorithm and MAPPO to 5e-5, while MATD3 and MADDPG are run at 1e-8, a 5000-fold difference. At 1e-8, off-policy actor and critic updates are effectively frozen, so the statement that MATD3 and MADDPG 'converge in a short time but fail to learn good decisions' is confounded: they are not being trained at comparable rates. The claim that all hyperparameters were 'tuned to a suitable value' is contradicted by this choice. Because 'best performance' is part of the strongest claim, the comparison is not yet established. Note that the MAPPO-vs-MAPPO+SA comparison does not suffer from this asymmetry; if the paper restricted its claim to the benefit of the SA repair at equal learning rate, this objection would not block acceptance.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper studies a multi-UAV multi-target tracking scenario in which each UAV radar can operate in active or passive mode and targets may carry jammers against radar. The authors formulate the problem as a Dec-POMDP, propose a MAPPO-based algorithm augmented with a simulated annealing (SA) module that repairs constraint-violating actions, and evaluate it against MAPPO, MATD3, and MADDPG in simulation. The central claim is that the proposed algorithm is effective in avoiding constraint violations and achieves the best tracking performance, measured by TE, the geometric-mean CRLB over the episode.","tokens_in":7022,"tokens_out":6428,"duration_ms":57132,"significance":"If the empirical claims are confirmed, the SA action-repair mechanism is a simple and potentially transferable way to enforce safety constraints in MARL trajectory design, and the active/passive radar work-mode selection is a practically relevant scenario. The paper provides code and a demo video, uses multiple random seeds, and the SA idea is clearly presented. However, the headline comparison is currently undermined by a large hyperparameter asymmetry, and the evaluation metric is the same CRLB quantity used in the training reward, so the transferability of the reported performance gain to real tracking is not yet established.","major_comments":[{"comment":"The reported learning rates are 5e-5 for the proposed method and MAPPO versus 1e-8 for MATD3 and MADDPG, a 5000-fold difference. At such a low rate, the off-policy baselines' updates are effectively frozen, so the observation that they 'fail to learn good decisions' is confounded. The statement that all hyperparameters were tuned to suitable values is not consistent with this choice. Please re-run the baselines with matched and individually tuned learning rates and report confidence intervals or standard deviations over seeds; alternatively, restrict the empirical claim to the MAPPO versus MAPPO+SA comparison, which does not suffer from this asymmetry.","section":"Section IV, Fig. 3"},{"comment":"The shared reward Rs in (9) and the SA objective in (10) are both constructed from LBTk, the same average CRLB whose geometric mean defines the reported performance metric TE. Consequently, Fig. 3(b) measures how well the policy optimizes its own training objective rather than an independent tracking benchmark. Please validate the learned policy with an out-of-sample metric, for example Monte Carlo simulated radar measurements followed by a tracking filter, and discuss the assumptions under which the CRLB is a faithful proxy, specifically the constant noise factors f_{i,j,k,1}, f_{i,j,k,2}, f_{n(i),j,k}, the absence of detection loss, and Gaussian noise.","section":"Section III.A Eq. (9), Section III.B Eq. (10), Section II.B Eq. (5)"},{"comment":"The paper states that the feasible set determined by (7) and (8) is not convex and therefore 'the optimization cannot be solved directly.' Non-convexity alone does not imply that the problem is unsolvable by direct optimization; many non-convex problems are addressed by exact or heuristic solvers. If the claim of proving difficulty is to be retained, a more precise hardness argument, such as NP-hardness or a demonstration that standard nonlinear solvers fail on this instance, is needed.","section":"Section III, first paragraph"},{"comment":"In the SA objective, the indicator I(||(Δxi,Tk, Δyi,Tk)|| < d0) appears with a negative sign inside the parentheses. Since the SA module minimizes Ri,Tk, actions satisfying the mobility constraint (6) receive an additional +L and are thereby penalized, while violating actions are favored. This is the opposite of enforcement. Please correct the sign or the inequality, and ensure the implementation matches the corrected equation; otherwise the mechanism based on SA to avoid constraint violations is not correctly specified.","section":"Section III.B, Eq. (10)"}],"minor_comments":[{"comment":"The term 'Additional White Gaussian Noise' should be 'Additive White Gaussian Noise'.","section":"Section II.B"},{"comment":"There is a typo: 'posibility' should be 'possibility' in the description of the transition function P.","section":"Section III.A"},{"comment":"Please clarify that CRLB2 is the estimation CRLB of the jammer position and explain how this maps to the target position when the jammer is carried by the target.","section":"Section II.B, Eq. (4)"},{"comment":"The statement that constraint (6) is 'naturally obeyed' needs a brief explanation of how the action space or clipping guarantees ||Δpi,Tk|| ≤ d0.","section":"Section III.B"},{"comment":"The phrase '3 forks are targets' is unclear; use 'crosses' or 'stars' to describe the target markers.","section":"Fig. 4 caption"}],"recommendation":"major_revision","confidential_remarks":"The SA action-repair idea is worth pursuing and the paper is within the journal's scope, but the headline comparison against MATD3 and MADDPG is invalidated by the 5000x learning-rate asymmetry. The MAPPO+SA versus MAPPO comparison at equal learning rate could support a narrower claim, and the authors should reframe the contribution accordingly. The circular reward/evaluation issue is common in RL engineering papers but should be addressed with an independent validation. Please also check the sign error in Eq. (10) against the provided code before advising acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper deserves a read for its problem formulation: UAV swarm multi-target tracking where each radar chooses active or passive mode in the presence of jammers, with the passive mode positioning the jammer. That combination is new in the cited literature, and framing it as a Dec-POMDP solved by MAPPO with a simulated-annealing action-repair step is a reasonable engineering contribution. The authors also release code, which makes the work reproducible, and the SA module is a genuine twist on [15]'s detection-only mechanism.\n\nWhere the paper falls short is the central empirical claim. The learning rate for the proposed method and MAPPO is 5e-5, while MATD3 and MADDPG run at 1e-8—a 5000-fold difference. At that rate, off-policy actor and critic updates are effectively frozen, so the baselines' poor performance is confounded. The statement that all hyperparameters were 'tuned to a suitable value' is contradicted by this choice. The stress-test note is correct: the MAPPO-vs-MAPPO+SA comparison is not affected by this asymmetry, so the paper's strongest defensible claim is the benefit of the SA repair at equal learning rate, not 'best performance' against all baselines.\n\nA second soft spot is the evaluation loop. The reward in (9) and the SA objective in (10) are both built from the same CRLB quantity LBTk that defines the reported performance metric TE. That means the algorithm is being judged on how well it optimizes the exact function it was trained on, not on an independent tracking benchmark. If the CRLB model is wrong—for example under detection loss or non-Gaussian noise—the claimed advantage may not transfer. The authors acknowledge the CRLB factors are approximations, but the load-bearing nature of that approximation is understated.\n\nMinor issues: no confidence intervals are shown despite averaging over 10 seeds, and the SA guarantee of constraint satisfaction is informal, relying on temperature tuning rather than a proof.\n\nThe paper is not a lost cause. The problem is real, the code is out, and the SA mechanism is worth exploring. But the current comparison does not establish the headline claim.\n\nMy recommendation: send it to peer review, but require the authors to redo the baseline experiments with comparable learning rates, report variance, and ideally evaluate against a tracking metric that is not the training reward. This is a conditionally acceptable paper, not a reject.","headline":"A genuinely new radar work-mode selection problem with a plausible SA-based action repair, but the empirical comparison is invalidated by a 5000x learning-rate asymmetry in the baselines.","tokens_in":7523,"tokens_out":1248,"would_cite":false,"duration_ms":12382,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Adding a simulated-annealing repair to MAPPO yields the best radar tracking accuracy for a UAV swarm under jamming.","keywords":["UAV swarm","multi-target tracking","radar jamming","multi-agent reinforcement learning","MAPPO","simulated annealing","Cramér-Rao lower bound","constraint satisfaction"],"falsifier":"Measure the actual position estimation error (for example, the root-mean-square error of an extended Kalman filter) in the same simulated scenario and compare the algorithm rankings with the paper's CRLB-based $TE$ ranking; if a non-jamming target is lost because all nearby UAVs are in passive mode, or if the penalized distance constraint $d_2$ is violated in practice to improve the CRLB score, the central claim that the SA-repaired policy performs best would be refuted.","tokens_in":6549,"feed_emoji":"📡","tokens_out":9714,"duration_ms":70919,"temperature":0.7,"pith_summary":"A UAV swarm tracking several targets often loses accuracy when some targets carry radar jammers, because active radar measurements are corrupted. The paper proposes letting each UAV choose between active mode (range and angle measurements) and passive mode (direction-finding on the jammer's own signal), and treating mode selection together with movement as a multi-agent control problem. The authors show the resulting constrained optimization is non-convex in its feasible set and therefore hard to solve directly, and instead train a multi-agent reinforcement learning policy based on MAPPO. To prevent the policy from violating a minimum UAV–target distance constraint, they wrap the actions in a simulated-annealing module that repairs unsafe moves while preserving the learned strategy. In simulation, this combination avoids constraint violations and achieves lower tracking error (higher $TE$) than MAPPO, MATD3, or MADDPG, which is the paper's central claim.","feed_headline":"MARL with SA repair outperforms in jammed UAV tracking","feed_subtitle":"A simulated-annealing repair step keeps a UAV swarm's radar tracking accurate under jamming.","key_machinery":"The performance metric is the multi-target average Cramér-Rao lower bound $LB_{T_k}$, computed from expressions (3)-(4), which depend only on UAV–target geometry and per-radar noise factors $f$; the reward (9) combines a shared reward negatively correlated with $LB_{T_k}$, a per-agent distinct reward that penalizes 'lazy' agents, and a penalty for violating the inter-UAV and UAV–target distance constraints. The distinguishing mechanism is the simulated-annealing module, whose objective (10) minimizes the negative predicted shared reward plus a large penalty for violating the mobility constraint and a constant penalty when the repaired position would fall within $d_2 + 3\\sigma_{\\mathrm{pred}}$ of any target; this module is triggered when the raw MAPPO action would breach constraint (8). The SA module is the component that the paper credits for both constraint satisfaction and efficient learning.","core_discovery":"The central claim is that a multi-agent reinforcement learning controller can handle radar jamming in multi-target tracking if each UAV's action (movement and active/passive mode) is repaired by a simulated-annealing step whenever it would place the UAV too close to a target. The paper reports that in a scenario with six UAVs, three targets, and a 50% chance each target carries a jammer, the proposed MAPPO-plus-SA algorithm achieves the best tracking error ($TE$) among MAPPO, MATD3, and MADDPG, with the plain MAPPO losing performance mainly because of constraint violations, and the off-policy algorithms failing to explore the large action space effectively. The authors conclude that the SA repair mechanism \"is effective in both avoiding violation of constraints and leading to efficient learning.\"","pith_inferences":["The SA repair module acts as a safety filter over the learned policy, and the same pattern could be transferred to any MARL control task where constraint violations are rare but catastrophic, such as multi-robot collision avoidance or autonomous driving.","Because the CRLB model treats noise factors as constants, transferring the method to real radar would require estimating those factors online; a testable extension is to feed measured noise statistics into the reward so the policy adapts to changing jammer power or target radar cross-section.","Given the paper's rotationally symmetric training setup, a cheap validation is to re-run the simulation with targets arriving from a fixed direction and check whether tracking performance degrades.","The paper only evaluates in simulation; hardware-in-the-loop testing with realistic work-mode switching latency and direction-of-arrival estimation errors would show whether the SA repair still improves tracking with imperfect sensors."],"forward_implications":["UAV swarms can maintain multi-target tracking accuracy in jamming environments by jointly switching radar modes and adjusting trajectories, and the policy runs per-agent from local observations without a central coordinator.","The simulated-annealing repair mechanism enforces a hard geometric constraint without heavy reward-shaping, suggesting it could make learned multi-agent controllers safer in other domains with similar constraints.","The Cramér-Rao lower bound provides a geometry-driven training signal that depends only on relative positions and noise factors, so the same reward design can be reused for other sensing-motion planning problems.","The comparison indicates that on-policy MARL with action repair is a more reliable baseline than off-policy algorithms (MATD3, MADDPG) for anti-jamming multi-target tracking in high-dimensional action spaces."],"supporting_citations":[{"why":"provides the CRLB expressions (3)-(4) that define the tracking-performance metric $LB_{T_k}$ used in the reward and in the reported tracking error ($TE$).","marker":"[16]"},{"why":"MAPPO is the base multi-agent reinforcement learning algorithm that the proposed method extends; the SA repair module is added on top of its actions.","marker":"[18]"},{"why":"Identifies the weakness in existing MARL approaches that penalties do not guarantee constraint satisfaction, which motivates the simulated-annealing action repair in this paper.","marker":"[15]"},{"why":"Supplies the Dec-POMDP framework used to formulate the UAV swarm's joint decision problem as a multi-agent reinforcement learning problem.","marker":"[17]"}],"fun_headline_variants":["SA repair boosts MARL for jammed radar tracking","MAPPO with SA repair wins jammed radar tracking","Simulated annealing repair makes MARL tracking jam-proof","RL swarm uses annealing to beat radar jammers","Jamming? SA-repaired MARL tracks better"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The entire evaluation rests on the assumption that the Cramér-Rao lower bound expressions (3)-(4), with noise factors $f$ treated as constants, faithfully represent the true radar tracking error under jamming and target motion.","fun_headline_variants_meta":{"raw":{"variants":["SA repair boosts MARL for jammed radar tracking","MAPPO with SA repair wins jammed radar tracking","Simulated annealing repair makes MARL tracking jam-proof","RL swarm uses annealing to beat radar jammers","Jamming? SA-repaired MARL tracks better"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000627,"raw_usage":{"total_tokens":2850,"prompt_tokens":844,"completion_tokens":2006,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":460,"completion_tokens_details":{"reasoning_tokens":1931}},"tokens_in":460,"tokens_out":2006,"duration_ms":12435,"temperature":1.0,"reasoning_tokens":1931,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T13:58:12.213883+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure the actual position estimation error (for example, the root-mean-square error of an extended Kalman filter) in the same simulated scenario and compare the algorithm rankings with the paper's CRLB-based $TE$ ranking; if a non-jamming target is lost because all nearby UAVs are in passive mode, or if the penalized distance constraint $d_2$ is violated in practice to improve the CRLB score, the central claim that the SA-repaired policy performs best would be refuted.","supporting_citations":[{"cited_title":"Cooperative target assignment and dwell allocation for multiple target tracking in phased array radar network,","cited_arxiv_id":null,"evidence_quote":"provides the CRLB expressions (3)-(4) that define the tracking-performance metric $LB_{T_k}$ used in the reward and in the reported tracking error ($TE$)."},{"cited_title":"The surprising effectiveness of mappo in cooperative, multi-agent games,","cited_arxiv_id":null,"evidence_quote":"MAPPO is the base multi-agent reinforcement learning algorithm that the proposed method extends; the SA repair module is added on top of its actions."},{"cited_title":"Madrl-based uavs trajectory design with anti-collision mecha- nism in vehicular networks,","cited_arxiv_id":null,"evidence_quote":"Identifies the weakness in existing MARL approaches that penalties do not guarantee constraint satisfaction, which motivates the simulated-annealing action repair in this paper."}],"review_version":1}