{"id":"3fb9ea2f-b672-4fbf-b2fd-35a1068e23fb","arxiv_id":"2501.10924","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A PPO-based multi-agent CNN policy that jointly controls movement, existence detection, and reachability detection, with a transfer-learned estimator for unreachable targets, outperforms three baselines in simulated radiation localization.","lead":"Engineers trained a team of simulated sensing agents to search for a radioactive source, decide whether it actually exists, and determine whether it can be reached, using deep reinforcement learning. If the source is judged unreachable, a second network built by transfer learning estimates where it is from the agents' sensor maps.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Benchmark comparison in Section 5.4 is structurally biased: baselines cannot declare no-target/unreachable, so their poor cost/time on those episodes is by construction.","rationale":"The reader identified the training-time oracle reward in Eq. (5) as the weakest assumption. That is a real gap, especially because the no-target case makes min(D_t) undefined, and because no ablation shows learning without privileged feedback. However, I see the benchmark fairness issue as more load-bearing for the paper's headline claim, which is explicitly about outperforming all benchmarks. The reward oracle is a training-detail that can be remedied (e.g., by ablating reward shaping or replacing the oracle with a learned estimator), and it does not automatically invalidate the policy's test-time behavior. In contrast, the benchmark comparison directly underlies the central claim and, as written, the baselines are placed in an environment where they cannot succeed on two of the three scenario types. This is not a disagreement with the reader's concern; it is a prioritization. Both should be fixed, but if the paper provided scenario-disaggregated results and the proposed method lost on the reachable-only scenario, the 'outperforms all benchmarks' statement would need substantial revision. Therefore I maintain the CONDITIONAL verdict and propose a concrete check that would settle whether the outperformance is substantive or structural.","tokens_in":16816,"tokens_out":2367,"duration_ms":26523,"concrete_test":"Re-run the Section 5.4 evaluation scenario-by-scenario: (1) target exists and reachable only, (2) target does not exist only, and (3) target exists but unreachable only. For scenario (1), compare episode length and cost against Uniform search, DDQN, ODMTL, and a maximum-likelihood/PDF-based estimation baseline. If the proposed method does not also win on scenario (1), then the headline outperformance claim largely reduces to the fact that baseline action spaces cannot declare 'no target' or 'unreachable.' Report means and standard deviations over at least 5 independent training seeds.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim in Section 5.4 that the proposed work outperforms all benchmarks is not established because the comparison is not apples-to-apples. Uniform search, DDQN, and ODMTL are given no action to declare target non-existence or unreachability; their action spaces and reward signals only support mobility/search. The evaluation environment randomly samples one-third episodes with no target and one-third with an unreachable target (Section 5.1). In those episodes, the baselines are forced to run to the 100-step episode limit, while the proposed agent can terminate early via existence and reachability flags. Thus a large part of the gap in Fig. 9 is incurred by design, not by learned superiority. The paper itself acknowledges this: 'existing methods struggle when addressing realistic scenarios' and ODMTL does not consider target unreachability or non-existence (Section 5.4), yet these methods are still benchmarked on the mixed scenario set. The correct comparison for the general outperformance claim would be scenario-disaggregated: on the reachable-target episodes, do the baselines have comparable time and cost? The paper does not report this. Additionally, no error bars or multiple seeds are shown for Figs. 6-9, so the magnitude of the reported gap is uncertain. The claim should be narrowed to 'the only method that can handle all three scenarios,' which is true by construction, unless scenario-specific evidence is provided.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a multi-agent deep reinforcement learning method for target localization in uncertain environments, where the target may be reachable, unreachable, or absent. The method uses PPO with CNN-based actor-critic networks under centralized learning and decentralized execution, a team-shaped reward, and an action space spanning mobility, existence detection, and reachability declaration. When the target is declared unreachable, a separate deep learning model, trained via transfer learning from the MADRL actor, estimates the target location using the shared feature extractor. The approach is evaluated in a simulated radiation localization environment with varying team sizes and target strengths, and is compared against Uniform search, DDQN, and the authors' prior ODMTL method. The paper also provides a complexity analysis showing O(I) space complexity per agent for the proposed method.","tokens_in":16987,"tokens_out":5599,"duration_ms":63912,"significance":"If validated, the work would be a useful extension of MADRL-based target localization to practically important scenarios that previous methods assume away: false alarms and unreachable targets. The action-space decomposition into mobility, existence, and reachability is a natural formalization, and the use of transfer learning to share a feature extractor between the decision policy and the estimation model is a sensible design for reducing computational overhead. The complexity analysis in Section 5.5 correctly highlights the scalability advantage of distributed action/state spaces over centralized DDQN. However, the empirical evidence currently has several gaps that directly affect the central claims: the reward function is underspecified for two of the three scenarios, the benchmark comparison is structurally biased, and no statistical variability is reported. These issues are fixable, but they are load-bearing for the claimed outperformance.","major_comments":[{"comment":"The reward function is not well-defined in two of the three scenarios that the paper explicitly targets. In the no-target scenario there is no target, so the set D_t and hence min(D_t) cannot be computed; in the unreachable-target scenario, if 'unreachable' means that no path exists, the BFS shortest-path distance is also undefined. Since Section 5.1 states that each episode is randomly assigned to one of the three scenarios with equal probability, the reward signal for two-thirds of training episodes is unspecified. The paper must state the reward for correct and incorrect existence/reachability flags and for episodes with no finite path to the target.","section":"Section 4.4, Eq. (5)"},{"comment":"The benchmark comparison is not apples-to-apples. Uniform search, DDQN, and ODMTL have no actions to declare the target non-existent or unreachable, so in the no-target and unreachable-target episodes (two-thirds of the test distribution per Section 5.1) they are forced to run until the 100-step limit, while the proposed method can terminate early through its majority flags. Thus a large part of the time and cost gap in Fig. 9 is incurred by construction rather than by learned superiority. The general claim that the proposed work 'outperforms all the benchmarks' should be supported by scenario-disaggregated results (e.g., reachable-target episodes only) or narrowed to 'the only method that can handle all three scenarios.'","section":"Section 5.4, Fig. 9"},{"comment":"No error bars, confidence intervals, or multiple-seed results are reported anywhere in the evaluation. The learning curves and benchmark comparisons appear to come from single training runs or single evaluation averages, so the magnitude of the reported improvements over baselines and the differences across team sizes cannot be assessed statistically. Please report means and standard deviations over at least five independent seeds, and where appropriate include significance tests.","section":"Section 5.2, Section 5.4, Figs. 6-9"},{"comment":"The claim that the target location is 'estimated accurately' rests entirely on training and validation loss curves that 'converge to 0.' No localization error in physical units, no accuracy threshold, and no evaluation across different environment layouts is reported. Since the estimation head is a central component of the combined model, the paper should report a quantitative error metric, such as mean Euclidean distance between the estimated and true target coordinates on held-out episodes.","section":"Section 5.3, Fig. 8"}],"minor_comments":[{"comment":"The phrase 'a team-based reward functions is proposed' is ungrammatical and should read 'a team-based reward function is proposed.'","section":"Abstract"},{"comment":"The text says the five observations are reduced into 'ten optimized ones,' but the actor is described as taking 'the first 9 reduced observations' plus a CAE embedding; please clarify whether the CAE embedding is the tenth observation and how the nine are selected.","section":"Section 4.2"},{"comment":"The state-space dimensionality for the proposed method is listed as 9I, but the actor also receives the CAE embedding of dimension d=128; the table should include this term or justify its omission.","section":"Table 2"},{"comment":"The FLOPs and trainable-parameter counts are reported without specifying whether they include the estimation head and the CAE encoder; please state the exact configuration used for these numbers.","section":"Section 5.5"},{"comment":"If the direction index d_i ranges over {1,...,D}, then d_i=1 corresponds to angle 2π/D rather than 0; please clarify whether directions are indexed from 0 or the angle formula is adjusted accordingly.","section":"Eq. (4)"},{"comment":"No information is given about random seeds, environment-generator reproducibility, or code release; providing these would substantially strengthen reproducibility of the reported curves.","section":"Section 5.1"}],"recommendation":"major_revision","confidential_remarks":"The core idea is plausible and the paper is clearly written in most places, but the evaluation needs substantial revision before the central outperformance claim can be accepted. The undefined reward in Eq. (5) for no-target and unreachable-target episodes and the structurally biased benchmark comparison in Section 5.4 are the main blockers. I see no indication of misconduct; the issues are standard, fixable evaluation gaps. The heavy citation of the authors' own prior work (e.g., refs. 4, 9, 25-27, 31-32) is noticeable but consistent with this being a continuous line of research."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short take: the paper adds a genuinely new capability to multi-agent target localization—agents can declare that no target exists or that the target is unreachable—and backs it with a clean PPO/CNN pipeline plus a transfer-learned coordinate estimator. The task formulation is a real extension of the authors' own ODMTL line and of the cited literature, which all assume a reachable target. The training curves converge, the complexity analysis is honest (O(I) per agent, independent of N), and the writing is clear. That is real value.\n\nThe soft spots are in the evaluation, not the idea. Eq. (5) defines the shared reward in terms of min(D_t), the BFS distance to the true target, but in the no-target scenario there is no target to measure distance to. The paper never says what reward the agents get when they correctly flag non-existence. That is a hole in the method as stated.\n\nMore importantly, the benchmark comparison in Section 5.4 is structurally lopsided. Uniform, DDQN, and ODMTL have no action to declare non-existence or unreachability; on the one-third of episodes where the target is absent and the one-third where it is unreachable, they are forced to run to the 100-step limit while the proposed agent can terminate early via its flags. So part of the gap in Fig. 9 is by construction, not by learned superiority. The paper acknowledges that existing methods 'struggle' with these scenarios but still benchmarks them on the mixed set. The fix is scenario-disaggregated results: on reachable-target episodes alone, do the baselines come close? That number is not reported.\n\nTwo smaller points: no error bars or multiple seeds appear anywhere in Figs. 6-9, so the magnitude of the reported wins is uncertain; and the target estimation claim rests on loss curves converging to zero, with no physical error metric (meters) and no comparison to the MLE-style estimators the paper cites. The loss is on normalized coordinates, so 'converges to 0' is less impressive than it sounds.\n\nThe circularity burden is low: the results are empirical measurements, not derivations that reduce to fitted constants. The use of the authors' own ODMTL as a baseline is fine.\n\nBottom line: the paper deserves a serious referee. The task novelty is real, the method is plausible, and the gaps are fixable with seeds, scenario-disaggregated tables, a defined reward for the no-target case, and an error metric for estimation. I'd send it to review, with a request for major revisions.","headline":"A credible new task formulation for multi-agent localization under existence/reachability uncertainty, but the benchmark comparison is structurally biased and the reward function is undefined for one of the three scenarios.","tokens_in":17638,"tokens_out":2438,"would_cite":false,"duration_ms":24851,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A single multi-agent model handles reachable, unreachable, and absent targets in one shared network.","keywords":["Multi-Agent Deep Reinforcement Learning","Target localization under uncertainty","Proximal Policy Optimization","Transfer learning","Swarm robotics","False alarms","Target unreachability","Radiation localization"],"falsifier":"Retrain with Eq. (5) replaced by a reward computable from observations alone, such as one based only on the readings map and the agents' positions, then compare episode length and cost on the same three scenario types; if performance collapses, the oracle distance is doing the work. Separately, instrument the no-target scenario by running episodes with no source and recording how often the team correctly declares absence within the step limit—the paper reports no such measure, and Eq. (5) does not specify a reward for it.","tokens_in":16483,"feed_emoji":"🤖","tokens_out":6203,"duration_ms":63318,"temperature":0.7,"pith_summary":"This paper tries to establish that one multi-agent deep-reinforcement-learning framework can handle the three situations a target-localization swarm actually meets: the target is reachable, the target exists but cannot be reached, and no target exists at all (a false alarm). The authors argue that their PPO-trained CNN policies, fed with compact 2D observation maps, learn to move, declare existence, and declare reachability as one decision process, and that a transfer-learning step lets the same feature extractor estimate the coordinates of an unreachable target. The payoff claimed is faster and cheaper searches than a uniform sweep, a centralized DDQN, and an earlier MADRL baseline, at space complexity that stays fixed per agent.","feed_headline":"Swarm AI model finds targets, flags false alarms, maps blocked ones","feed_subtitle":"PPO-trained CNN agents beat uniform search, DDQN, and ODMTL on speed and cost at fixed per-agent complexity.","key_machinery":"The load-bearing mechanism is an actor-critic CNN trained with PPO under centralized learning and distributed execution: each agent runs its own copy of the actor on ten reduced observation maps (nine local and global 2D maps plus a convolutional-autoencoder embedding of the environment layout), while a centralized critic stabilizes training. A team-based shaped reward, Eq. (5), gives all agents the same feedback based on whether the BFS shortest-path distance from the team to the true target decreased and how many agents moved. Once the policy learns to flag a target as unreachable, transfer learning copies the frozen convolutional layers into a regression head that outputs $(x,y)$ coordinates, so the deployed model is one network with two output heads.","core_discovery":"The central claim is that the combined model—a shared convolutional feature extractor with one policy head for movement, detection, and reachability and one regression head for target coordinates—solves target localization under uncertainty better than the benchmarks. In Section 5.4 the paper states that the proposed work outperforms all benchmarks by achieving faster and less costly localization; Section 5.3 reports that the training and validation loss of the estimation model converge to zero, which the paper reads as accurate location estimates. The authors attribute the gains to PPO with a centralized critic, a shaped team reward that uses BFS shortest-path distances to reward progress, and transfer learning that avoids training the estimator from scratch.","pith_inferences":["The paper's own results suggest a harder test the authors did not run: training with an oracle-free reward and measuring false-alarm performance, since the claimed behavior in the absent-target scenario has no stated reward signal.","If the framework transfers as claimed to other inverse-square-law signals such as sound or heat, the same two-head architecture and observation maps should work with only the forward sensor model changed; this is a natural benchmark the paper does not provide.","The validation loss converging to zero for coordinate estimation may indicate the estimation subtask is nearly saturated by the privileged BFS-based observations; a more informative evaluation would report meter-level error distributions, not just loss curves."],"forward_implications":["A 4-agent team using the trained policy can finish reachable, unreachable, and absent-target episodes within the 100-step limit, with the strongest target taking about 7.7 steps and 12.4 movement actions on average.","Because the actor's state and action spaces are fixed per agent, adding more agents does not increase the space complexity beyond $O(I)$; the same policy copy is deployed on each agent.","Since the convolutional feature extractor is shared with the estimation head, triggering an unreachable-target estimate does not require a second network or extra per-agent parameters.","Against the reported benchmarks, the method is claimed to dominate Uniform search, centralized DDQN, and ODMTL on both episode length and movement cost under all tested target strengths."],"supporting_citations":[{"why":"Supplies the PPO clipped-surrogate objective and the policy-update procedure used to train the actor-critic networks.","marker":"[17]"},{"why":"The single-agent Double Q-learning radiation-source search used as the DDQN baseline and contrasted for scalability.","marker":"[10]"},{"why":"The earlier MADRL target-localization method (ODMTL) used as the closest benchmark; it lacks reachability and existence handling.","marker":"[4, 9]"},{"why":"Defines the uniform-search baseline that the paper compares against for episode length and cost.","marker":"[18]"},{"why":"Provide the inverse-square-law radiation physics and sensor model used to build the simulation environment.","marker":"[19, 41]"},{"why":"Supplies the centralized-learning and distributed-execution training scheme that keeps complexity independent of team size.","marker":"[37]"},{"why":"Provides generalized advantage estimation used inside the PPO updates.","marker":"[33]"}],"fun_headline_variants":["AI swarms localize targets despite false alarms and blocked paths","Multi-agent RL hunts targets, spots false alarms, guesses unreachable","PPO agents track targets under uncertainty, with transfer learning","Swarm learning pinpoints targets, even when sensors mislead","Deep RL team finds targets, flags ghosts, estimates out-of-reach"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the team reward in Eq. (5)—which feeds the true target's BFS shortest-path distance to the agents during training—is a legitimate training signal, and that the no-target scenario, whose reward the paper never defines, is handled by the same rule; if either fails, the claimed convergence and benchmark wins do not transfer to deployment.","fun_headline_variants_meta":{"raw":{"variants":["AI swarms localize targets despite false alarms and blocked paths","Multi-agent RL hunts targets, spots false alarms, guesses unreachable","PPO agents track targets under uncertainty, with transfer learning","Swarm learning pinpoints targets, even when sensors mislead","Deep RL team finds targets, flags ghosts, estimates out-of-reach"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00018,"raw_usage":{"total_tokens":1311,"prompt_tokens":962,"completion_tokens":349,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":578,"completion_tokens_details":{"reasoning_tokens":270}},"tokens_in":578,"tokens_out":349,"duration_ms":4095,"temperature":1.0,"reasoning_tokens":270,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T18:50:59.799074+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain with Eq. (5) replaced by a reward computable from observations alone, such as one based only on the readings map and the agents' positions, then compare episode length and cost on the same three scenario types; if performance collapses, the oracle distance is doing the work. Separately, instrument the no-target scenario by running episodes with no source and recording how often the team correctly declares absence within the step limit—the paper reports no such measure, and Eq. (5) does not specify a reward for it.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The single-agent Double Q-learning radiation-source search used as the DDQN baseline and contrasted for scalability."},{"cited_title":"Ziock, W","cited_arxiv_id":null,"evidence_quote":"Defines the uniform-search baseline that the paper compares against for episode length and cost."},{"cited_title":"Schulman, P","cited_arxiv_id":null,"evidence_quote":"Provides generalized advantage estimation used inside the PPO updates."}],"review_version":1}