{"id":"052dfc3e-dc77-4e9d-bc3b-fd3a9832fe56","arxiv_id":"2412.06231","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Recurrent PPO with LSTM lets one or two simulated drones find a signal-emitting target in GPS-denied indoor maps, with 93% and 86% success rates in the authors' simulator.","lead":"The paper trains single and paired simulated drones with recurrent PPO to find a signal-emitting target without GPS, and reports 93% and 86% success rates. A smart generalist might care because it points to simple-sensor, learning-based UAV search for indoor or GPS-denied settings, though the results are simulation-only.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported 'accuracy' is Section-1 entry, not localization error; without section size or final distance error, the central claim is unsubstantiated.","rationale":"The reader's conditional verdict is reasonable, and the simulator-fidelity concern is real. My stress-test pass identified a more immediate and more internally grounded issue: the success metric itself is not a localization metric. Section II-A defines three signal sections, with Section 1 entry terminating the task as 'reaching the target,' but the paper never reports the geometric extent of that section or the final error relative to the true emitter. As a result, the abstract's 'accuracy of 93%' is ambiguous: it could mean the drone entered a possibly large disk around the target, which is not equivalent to localizing the target. This concern is load-bearing because the entire empirical case rests on this number. A concrete re-analysis with per-episode final distances would settle it. The paper also has reproducibility gaps (no code, seeds, hyperparameters, or error bars) and an evident formula labeling issue (Eq. 5 calls an average-step computation an average success rate), but those are secondary. I would keep the conditional verdict: the paper's contribution is plausible but its headline claim needs re-measurement and restatement before it can be accepted as evidence of localization performance.","tokens_in":5272,"tokens_out":3678,"duration_ms":42725,"concrete_test":"For the 100 test episodes, record the final Cartesian distance between the drone and the true target position at episode termination, along with the radius or area of Section 1 in grid cells and the physical cell size in meters. If the median or 95th percentile final distance is large relative to the Section 1 radius, or relative to a practical indoor localization tolerance, then the accuracy claim must be restated as 'section-entry rate' and the step-reduction comparison re-evaluated at a matched localization-error threshold.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim states that the single-drone model achieved 93% accuracy and the two-drone model 86% accuracy, with the latter requiring fewer average steps. However, the simulator's success criterion, defined in Section II-A and Table III, is that reaching Section 1 (the highest of three discrete signal-strength sections) terminates the episode and counts as reaching the target. Thus the reported 'accuracy' is a coarse-region entry rate, not a target-localization accuracy. The paper does not report the physical size of Section 1, the grid-cell-to-meter conversion, or the final distance between the drone and the true target at episode termination. If Section 1 spans several cells, a 93% success rate could be achieved while the drone is still far from the actual emitter, and the claim that two drones reduce steps to 'locate the target' could reflect faster entry into a large region rather than more precise localization. This ambiguity directly affects the headline numerical claim and the stated application of UAV target localization.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper describes a reinforcement-learning approach to UAV target localization in a GNSS-denied indoor setting. The authors train a recurrent PPO agent with an LSTM on a discrete grid simulator; the observation includes eight surrounding obstacle states, eight map states, and a three-level signal strength. A two-agent variant adds four neighboring-drone indicators and a shared grid map. On 100 generated test environments, the single-drone model achieves a 93% success rate and the two-drone model an 86% success rate, with the two-drone model using fewer average steps (183.82 vs 162.89 at the selected checkpoints). The authors claim this demonstrates scalable decentralized target localization.","tokens_in":5416,"tokens_out":5193,"duration_ms":51120,"significance":"If the reported results are reproducible, the paper offers a useful, simple data point: a recurrent PPO policy can use coarse signal-strength sections and a visited-cell map to navigate toward a radiant target in a discrete simulator, and two decentralized agents can share a map to reduce mean steps. The train/test layout separation (training on eight generated maps, testing on ten new layouts with 100 environments) is a reasonable generalization check, and the use of a recurrent policy for partial observability is appropriate. The paper also makes a clear, falsifiable empirical claim about success rates and step counts. However, the absence of error bars, the coarse success metric, and the custom simulator limit the current significance; the contribution is more a proof-of-concept than a validated scalable framework.","major_comments":[{"comment":"The headline 'accuracy' is defined as reaching Section 1, a discrete signal-strength annulus, not as final localization error. The paper never reports the physical size of Section 1, the cell size, or the distance between the drone and the true target at termination. Consequently, a 93% 'accuracy' could correspond to entering a large region rather than localizing the emitter, and the two-drone step reduction (183.82 vs 162.89) could reflect faster region entry rather than more precise localization. This ambiguity directly affects the central claim and must be resolved by reporting section dimensions and terminal localization error.","section":"Section II-A, Table III, and Section III, Eqs. (4)-(5)"},{"comment":"The comparison between single-drone and two-drone performance rests on single evaluations at selected training iterations, with no error bars, no multiple seeds, and no statistical test. The observed differences (93% vs 86% success; 183.82 vs 162.89 steps) may be within run-to-run noise. The authors should provide multiple independent training runs with confidence intervals or a significance test, or explicitly state that these are single-run results. Without this, the 'reduced steps' and 'scalable' claims are unsupported.","section":"Section III, Tables V and VI"},{"comment":"The reward function directly encodes the success criterion: the target-reach reward is 1000, the signal-strength cost is 0-250, and success is defined as entering Section 1. This creates a risk that the reported performance reflects reward shaping rather than a learned, generalizable localization behavior. The reward weights and signal-strength boundaries are free parameters, but no ablation or sensitivity analysis is provided. An ablation with a minimally shaped reward, or at least a discussion of sensitivity to these parameters, is needed to support the generalization claim.","section":"Section II-C and Section II-F"},{"comment":"All evaluation is in a custom grid simulator with three discrete signal sections and no sensor noise, continuous dynamics, or occlusion model. The paper states the application as 'perceptually degraded environments like places without GNSS/GPS signals' and 'complex environmental conditions,' but no argument or experiment connects the simulator abstractions to those conditions. The authors should either add validation on a higher-fidelity simulator or physical platform, or substantially narrow the claims and explicitly state the simulator's limitations in the paper. This is load-bearing because the abstract's claim of 'efficient and effective localization ... in complex environmental conditions' depends on simulator fidelity.","section":"Section II-A and Section II-F"}],"minor_comments":[{"comment":"Equation (5) is labeled 'average success rate' but the formula Tq/s computes the average number of steps for successful localizations; the notation and label should be corrected.","section":"Section III, Eq. (5)"},{"comment":"The abstract reports fewer average steps for the two-drone model, and Table VI shows 162.89 at the 86% checkpoint, but the conclusion states 162.69; these numbers should be made consistent.","section":"Abstract and Section IV"},{"comment":"The text says 'training continued for several million epochs,' while Tables V and VI list 'Training Iteration' values such as 3.8 mil and 4.15 mil; clarify whether these are steps, episodes, or epochs and use consistent terminology.","section":"Section II-F and Tables V-VI"},{"comment":"The model comparison in Fig. 2 is described qualitatively ('PPO yielded the highest mean reward'), but no quantitative results or hyperparameters for DQN, DDPG, and PPO are given; adding a table would make the selection reproducible.","section":"Section II-D, Fig. 2"},{"comment":"The variable h in Eq. (4) is not defined in the text; it appears to denote the number of successful localizations and should be stated explicitly.","section":"Section III, Eq. (4)"},{"comment":"The conclusion lists 'incorporating obstacles into the UAV-RL model state space' as future work, but Section II-A already includes obstacle states (State 1) in the observation; this should be clarified.","section":"Section IV"}],"recommendation":"major_revision","confidential_remarks":"I did not identify a novelty-disclosure issue. The manuscript reads more like a workshop or conference contribution than a fully supported journal article; the main gaps are empirical rigor (seeds, error bars, metric definition) and the absence of reproducibility artifacts such as code and hyperparameters. The metric ambiguity about Section 1 is the most serious concern and should be resolved during revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the paper is a readable, small simulation study that applies an existing method (Recurrent PPO, i.e. PPO with an LSTM) to single- and two-drone target localization in a custom grid world. What it does well: the problem setup is clear, the action/observation/reward design is explained, and the authors honestly compare DQN, DDPG, and PPO before picking PPO. The decentralized two-drone variant with a shared grid map is a reasonable extension, and the reported step reduction (183.82 to 162.89) is the kind of result you could build on. It is not a new algorithm or a reusable benchmark, and the novelty is modest, but the work is coherent on its own terms.\n\nThe soft spots are real and one is load-bearing. The headline \"accuracy\" is defined as reaching Section 1, the highest of three discrete signal-strength zones, which terminates the episode. That is a coarse region-entry rate, not a localization error. The paper never reports the physical size of Section 1, the grid-cell-to-meter conversion, or the final distance between the drone and the emitter. If Section 1 spans many cells, 93% success could mean the drone stops far from the actual target. That ambiguity undercuts the central claim and the stated application. A second soft spot is reproducibility: no code, hyperparameters, seeds, or error bars, so the numbers are plausible but unverifiable. The \"scalable\" claim rests on a two-drone experiment, which is fine as a first step but not much more. And the simulator has no sensor noise, continuous dynamics, or real-world validation, so transfer to physical UAVs is untested.\n\nNone of this makes the paper incoherent. The reward shaping directly encodes the evaluation objective, but that is standard in RL and not circular in the pejorative sense. The method is sensible and the writing is direct.\n\nWho is this for? Someone wanting a quick baseline for RL-based UAV search in simulation, or a teaching example of recurrent PPO in a multi-agent grid setting. It deserves a serious referee, but only if the referee insists on the missing metric details and reproducibility items. My recommendation: do not desk reject, but send it to peer review with a clear request for the Section 1 size, final localization error, code/data, and a discussion of what \"accuracy\" physically means. Without those, the paper is a useful technical report, not a complete research claim.","headline":"A clean but thin simulation study of Recurrent PPO for UAV target search; the headline accuracy is just entry into a coarse signal region, and the paper never says how big that region is.","tokens_in":5954,"tokens_out":1611,"would_cite":false,"duration_ms":18005,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A recurrent PPO policy lets a single drone find a signal-emitting target in a GPS-denied indoor grid with 93% success, while a second drone shortens the search.","keywords":["target localization","UAV swarm","recurrent PPO","LSTM","partial observability","GPS-denied navigation","grid exploration","decentralized reinforcement learning"],"falsifier":"Deploy the trained policies unchanged on physical quadrotors in an indoor testbed with an RF beacon, walls, and clutter; if the measured success rate falls far below the simulator's 93% and 86% (for example below 50%) or the drones fail because of multipath, dropped signals, or delays, then the simulation is not a faithful proxy and the central accuracy claim does not transfer.","tokens_in":5044,"feed_emoji":"🚁","tokens_out":7917,"duration_ms":66073,"temperature":0.7,"pith_summary":"This paper tries to show that a reinforcement-learning agent with recurrent memory can find a signal-emitting target indoors without GPS, using only a small grid map and a three-level signal-strength reading. The single-drone policy reaches the target in 93% of 100 test scenarios, averaging 183.82 steps. A decentralized two-drone variant, sharing one grid map, reaches 86% success while cutting the best average to 162.89 steps. If these results hold up, the approach offers a low-hardware route to search-and-rescue and monitoring in GPS-denied, visually degraded spaces, and a starting point for scaling to larger UAV swarms.","feed_headline":"One drone finds hidden signal targets 93% of the time","feed_subtitle":"A second drone cuts average search steps from 184 to 163 while holding accuracy near 86 percent.","key_machinery":"The machinery is a Recurrent PPO policy with an LSTM layer of 256 hidden units on top of a PPO base, trained on a custom grid simulator. The observation is 17 inputs for a single drone (8 obstacle flags, 8 map-state readings, 1 signal level) and 21 for two drones (plus 4 quadrant flags for nearby drones); the action is one of eight translations. The reward system shapes behavior with +2 for exploring unknown cells, penalties for obstacles and repeat visits, a signal-strength cost up to +250, +1000 on reaching the target, and -2/-50 for proximity or collision with another drone. The map states (unknown, obstacle, traveled once/twice/thrice) are shared across drones in the multi-agent setting.","core_discovery":"The paper's central claim is that a Recurrent PPO model—a Proximal Policy Optimization policy augmented with an LSTM layer—can solve target localization in perceptually degraded grid environments from sparse observations. In a simulator with eight surrounding obstacle flags, eight surrounding map states, and a coarse signal strength, the single-drone model reaches 93% success across 100 scenario configurations built from 10 indoor-layout maps. The two-drone model, which adds four nearby-drone presence indicators and shares a common explored grid map, retains 86% success and reduces the average number of steps to 162.89 at its 86% checkpoint (the 83% checkpoint averages 130.08). The paper interprets this as evidence that decentralized coordination and shared memory of explored cells speed up search without a central controller.","pith_inferences":["The reported accuracy gap between one drone (93%) and two drones (86%) is not explained in the paper; it could reflect added collision penalties, sparse training for cooperative behavior, or simply variance across the 100 test maps, and a confidence interval on the success rates would clarify whether the gap is real.","The scaling claim is untested beyond two agents; with more drones, shared grid coverage saturates and collision penalties may outweigh the marginal speedup, so the swarm benefit is not guaranteed by these results.","Because the signal is discretized into only three levels, the problem is close to a grid search with a proximity hint; a simpler non-learned baseline (e.g., an exploration algorithm guided by the signal gradient) would quantify how much the learned policy adds beyond standard coverage search.","The reward weights (e.g., +1000 target vs +2 exploration) likely dominate behavior; ablating reward components would show whether the signal cost or exploration bonus drives the good accuracy, and would help transfer the method to new sensor modalities."],"forward_implications":["A single drone with one signal sensor and one detection sensor can learn to locate a radiant target without GPS, using only sparse local observations and a coarse signal-strength reading.","Two decentralized drones sharing a grid map can match the single-drone success rate (86% vs 93%) while using fewer average steps (162.89 vs 183.82 at the best checkpoints), showing a speed benefit from cooperation.","Adding recurrent memory (LSTM) to PPO improves handling of partial observability, preventing the drone from getting stuck in local minima in the tested environments.","The method's low sensor and computation requirements make it a plausible candidate for search-and-rescue and environmental monitoring in buildings where GNSS/GPS is unavailable.","The authors expect that larger UAV swarms, trained with the same recipe, would reduce localization time further (scalability), although this is extrapolated beyond the two-drone experiments."],"supporting_citations":[{"why":"Supplies the base PPO algorithm whose clipped surrogate objective and GAE are used for the policy updates.","marker":"[15]"},{"why":"Describes the recurrent PPO variant with an LSTM layer, including the probability ratio conditioned on observation and hidden state used in equation (3).","marker":"[16]"},{"why":"DQN is one of the baseline models compared in the model-selection step that led to choosing PPO.","marker":"[13]"},{"why":"DDPG is another baseline model in the same comparison, representing continuous-action RL.","marker":"[14]"},{"why":"A prior reinforcement-learning framework for UAV target localization that this work extends and positions itself against.","marker":"[7]"},{"why":"Provides the motivating POMDP-based approach for UAV search in GNSS-denied and visually degraded environments.","marker":"[11]"}],"fun_headline_variants":["Two drones find targets with 86% accuracy, 21 fewer steps","Decentralized PPO target search: 93% single, 86% dual","UAV pair cuts search steps to 163 from 184, 86% success","Recurrent RL: one drone 93% accurate, two drones faster","GNSS-denied targeting: two drones speed up at 7% accuracy cost"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the simulator's three-level signal sections and grid abstractions faithfully represent a real GPS-denied indoor environment, so the trained policies will transfer to physical UAVs without retraining.","fun_headline_variants_meta":{"raw":{"variants":["Two drones find targets with 86% accuracy, 21 fewer steps","Decentralized PPO target search: 93% single, 86% dual","UAV pair cuts search steps to 163 from 184, 86% success","Recurrent RL: one drone 93% accurate, two drones faster","GNSS-denied targeting: two drones speed up at 7% accuracy cost"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000875,"raw_usage":{"total_tokens":3744,"prompt_tokens":865,"completion_tokens":2879,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":481,"completion_tokens_details":{"reasoning_tokens":2775}},"tokens_in":481,"tokens_out":2879,"duration_ms":22929,"temperature":1.0,"reasoning_tokens":2775,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T19:52:48.346704+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Deploy the trained policies unchanged on physical quadrotors in an indoor testbed with an RF beacon, walls, and clutter; if the measured success rate falls far below the simulator's 93% and 86% (for example below 50%) or the drones fail because of multipath, dropped signals, or delays, then the simulation is not a faithful proxy and the central accuracy claim does not transfer.","supporting_citations":[{"cited_title":"Generalization, mayhems and limits in recurrent proximal policy optimization,","cited_arxiv_id":null,"evidence_quote":"Describes the recurrent PPO variant with an LSTM layer, including the probability ratio conditioned on observation and hidden state used in equation (3)."},{"cited_title":"Playing atari with deep reinforcement learn- ing,","cited_arxiv_id":null,"evidence_quote":"DQN is one of the baseline models compared in the model-selection step that led to choosing PPO."},{"cited_title":"Continuous control with deep reinforcement learning,","cited_arxiv_id":null,"evidence_quote":"DDPG is another baseline model in the same comparison, representing continuous-action RL."},{"cited_title":"Reinforcement learn- ing framework for uav-based target localization applications,","cited_arxiv_id":null,"evidence_quote":"A prior reinforcement-learning framework for UAV target localization that this work extends and positions itself against."},{"cited_title":"Framework for autonomous uav navigation and target detection in global-navigation-satellite-system- denied and visually degraded environments,","cited_arxiv_id":null,"evidence_quote":"Provides the motivating POMDP-based approach for UAV search in GNSS-denied and visually degraded environments."}],"review_version":1}