{"id":"3e2ac4ec-6786-4ad9-abfa-3d0ebd548815","arxiv_id":"2502.02666","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A transformer DRL policy for persistent UAV-UGV surveillance, with drones recharging on a moving ground vehicle, beats OR-Tools heuristics and the Attention Model on simulated age-period objectives.","lead":"This paper trains a transformer-based deep reinforcement learning policy to plan repeated surveillance visits by a battery-limited drone that recharges on a ground vehicle, and reports lower age-based objective values and faster runtimes than heuristic and attention-model baselines. The result matters because disaster surveillance requires continuous re-visits, and learned policies could re-plan online when new targets appear.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"DRL recharging actions ignore UGV travel time; reported objectives assume the UGV is available at any chosen rendezvous, so the headline comparison is not yet validated against physically realizable routes.","rationale":"The reader's weakest_assumption identifies the same load-bearing concern: the MDP state contains no UGV position or arrival-time information, so recharging is treated as feasible whenever the masking rules in Section V-A2 are satisfied. I agree with that identification. The issue is concrete and mechanical: the transition model in Section III-B4 advances time by UAV travel plus recharge time only, while the UGV's travel time to the rendezvous is never inserted into the timeline. The baseline heuristics do include UGV time-window constraints through Eq. 10, so the DRL policy is evaluated under a weaker, more permissive feasibility model than the baselines. If the DRL policy selects a rendezvous the UGV cannot reach in time, the UAV must either wait on the ground or the route is infeasible; either way the age periods and the objective score increase beyond the reported values. This directly affects the strongest claim of consistently lower objective values, because the reported margins over TS, for example, could shrink or reverse once UGV travel is accounted for. I would not reject the paper on this basis, because the problem is addressable with a post-hoc feasibility audit and does not necessarily invalidate the learning approach. The appropriate verdict remains CONDITIONAL, same as the reader's verdict, pending this check. Other concerns, such as lack of error bars and the tuned priority hyperparameter, are real but secondary; they do not bear as directly on whether the reported routes are executable. The proposed concrete check is the minimal experiment that would settle whether the concern lands: if the audit passes, the central claim is supported; if it fails, the comparison in Tables I-III is not a fair comparison of realizable routes.","tokens_in":25978,"tokens_out":3708,"duration_ms":39981,"concrete_test":"For each of the 30 test instances in Table I, take the DRL(10240) rollout and, for every recharging action, compute the earliest UGV arrival time at the selected rendezvous by traversing the road network at vg = 4.5 m/s from the previous rendezvous location, and compare it with the UAV arrival time in the DRL timeline. For each violation, insert the waiting time into the mission timeline, update all age periods accordingly, and recompute the score metric from Eq. 2; then re-rank DRL(10240) against TS. If the feasibility audit shows that DRL no longer achieves the lowest objective, or if more than 5% of recharge steps are infeasible, the central claim is unsubstantiated.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section III-B1 defines the MDP state as st = (pt, ft, qt), where pt is the agent's position (the UAV's), with no UGV position or arrival-time information. The transition in Section III-B4 updates mission time as t = tij + TR for a recharging action, adding UAV travel time and a fixed recharge service time but no waiting term. The masking rules in Section V-A2 filter actions by UAV fuel reachability and by whether the UAV can still reach some refuel stop next step, but they never check whether the UGV can physically reach the selected ground point by the time the UAV arrives. By contrast, the heuristic O-EVRPTW baseline enforces the UGV time-window constraint in Eq. 10 (ti >= tr_l,i), so the baseline is held to a stronger feasibility standard. The UGV route is only constructed after the DRL action sequence is generated, as stated at the start of Section V, and Section IV-A1 explicitly assumes waiting only when the UGV arrives earlier than the UAV. Consequently, the DRL timelines in Tables I-III may be computed with rendezvous points that the UGV cannot reach in time; inserting the required waiting time would increase every mission point's age period and raise the reported objective scores. Since the strongest claim is that DRL consistently produces lower objective values than baselines, and every comparison and generalization table inherits this assumption, the realizability of the UGV rendezvous is the most load-bearing condition in the paper. If the UGV cannot reach chosen rendezvous points, the reported DRL routes are not executable and the comparison against TS and other baselines is invalid.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper addresses an energy-aware persistent surveillance problem with one UAV and one UGV, where the UGV serves as a mobile recharging station on a road network. The authors formulate the problem as an MDP, propose an encoder-decoder transformer policy trained with REINFORCE, and evaluate it against the Attention Model and three OR-Tools heuristics (GLS, TS, SA) on problem instances of varying sizes and point distributions. They additionally present a Hurricane Harvey case study, a dynamic-planning extension, and a priority-weighted variant. The central claim is that the proposed DRL policy consistently yields lower values of the age-period-based score metric than all baselines, in shorter or comparable runtime, and that it generalizes to larger instances and different distributions.","tokens_in":26281,"tokens_out":6055,"duration_ms":55884,"significance":"If the central claims hold, the paper would offer a practical learning-based method for a relevant UAV-UGV coordination problem, with a plausible transformer architecture that dynamically incorporates age-period embeddings. The generalization experiments and the case study are useful additions that go beyond a plain routing benchmark. However, the significance is currently undermined by two load-bearing issues: (i) the DRL policy never checks whether the UGV can physically reach the chosen rendezvous point at the required time, so the reported DRL routes may be infeasible, and (ii) all tabular results are point estimates over 30 instances without any measure of variance or significance, making the claimed 'consistent outperformance' impossible to verify. The priority-driven extension also tunes its key hyperparameter on the very case-study scenario used for reporting, which overstates the demonstration. These issues are fixable within the manuscript's scope and do not invalidate the overall approach, but they must be addressed before the claims can be accepted.","major_comments":[{"comment":"The MDP state in Section III-B1, st = (pt, ft, qt), contains no UGV position or arrival-time information, and the transition in Section III-B4 updates mission time as t = tij + TR for a recharging action with no waiting term. The masking rules in Section V-A2 check UAV fuel reachability but never check whether the UGV can physically arrive at the selected ground point by the time the UAV lands. In contrast, the O-EVRPTW baseline enforces the UGV time-window constraint via Eq. (10) (ti >= tr_l,i). Because the UGV route is constructed only after the DRL action sequence is generated (Section V, first paragraph), the DRL timelines in Tables I-III may rely on rendezvous that are not physically realizable; inserting the required UGV travel time would increase every mission point's age period and raise the reported objective values. This is the most load-bearing point in the paper, and it must be resolved by either incorporating UGV state into the MDP or post-hoc verifying feasibility and recomputing objectives with actual waiting times.","section":"III-B1, III-B4, V-A2, Tables I-III"},{"comment":"All comparison tables report average objective values and runtimes over 30 test instances without standard deviations, confidence intervals, or significance tests. The claim that 'the proposed DRL policy consistently produces lower objective values across all problem sizes' (Section VI-B) cannot be assessed from point estimates, especially when some gaps are small (e.g., U15G5: DRL(greedy) 2.9 vs TS 3.1, roughly 7% difference) and may lie within instance-to-instance variability. The authors should report variance and perform paired statistical tests (e.g., Wilcoxon signed-rank or paired t-test) across the 30 instances for each comparison, and state the number of random seeds or evaluation runs if applicable.","section":"Tables I-III, Section VI-B"},{"comment":"The priority-driven extension introduces the increment factor F = 1 + (wk - 1)S in Eq. (32), where S is a hyperparameter that is tuned on the Hurricane Harvey case-study scenario (Figure 9 shows the objective score metric versus S, with 'Optimal value' chosen on that same scenario). Reporting the subsequent reduction in maximum age periods for high-priority points on that same scenario after tuning S constitutes tuning on the test set. To make the demonstration credible, the value of S should be fixed a priori, selected on a separate validation scenario, or justified analytically, and the reported performance should be evaluated on held-out priority scenarios.","section":"VI-F, Eqs. (31)-(32), Figure 9"},{"comment":"The dynamic-planning experiment reports only two numbers in Table IV (initial route score 3.92 and max age 378; modified routes 5.51 and 523), despite the text stating that '30 trials of dynamic planning with an average of 5 newly appeared mission points' were conducted. The claim of 'little deviations in objective score metric and maximum age period' is unsupported without the mean, standard deviation, or range across those 30 trials. The authors should report distributional information and also clarify how the 30 trials differ (random new point locations, numbers, or timing).","section":"VI-E, Table IV"}],"minor_comments":[{"comment":"The quantity computed in Eq. (27) is the relative gap to the best objective found among the compared methods, not an optimality gap with respect to a true optimum. Renaming this 'relative gap' would avoid misleading readers.","section":"Eq. (27) and Table I caption"},{"comment":"The dataset description says the road network G is fixed but 'the road network points are chosen randomly from the given network.' This is confusing because the ground points Mg are mission points, not the road network nodes; please clarify the relationship between the fixed road network, the randomly sampled ground points, and the training/test split.","section":"Section VI-A"},{"comment":"The training reward curves in Figure 5 appear to be single runs without error bands or multiple seeds; given the stochastic nature of REINFORCE, reporting mean and variance over at least a few seeds would strengthen the convergence claim.","section":"Figure 5"},{"comment":"There are minor notational inconsistencies, such as 'UA V' with a space in several places, and the symbol P is used both for the penalty in Eq. (5) and for the probability in Eq. (12). These should be harmonized.","section":"Throughout"},{"comment":"The statement 'the DRL model generally requires slightly more runtime than the AM model, except when using greedy decoding' is not consistently supported by Table I (e.g., U30G10 DRL(1024) time is 10.5 sec vs AM(1024) 9.8 sec, but U45G15 DRL(1024) time is 12.8 sec vs AM(1024) 9.5 sec, and DRL(greedy) is indeed fastest). Please revisit the wording.","section":"Section VI-B"}],"recommendation":"major_revision","confidential_remarks":"The paper is a solid engineering application of DRL to an interesting problem, but the feasibility gap for UGV coordination and the lack of statistical rigor are serious enough that the current claims are not yet established. The authors should also consider whether the comparison against the receding-horizon heuristic is entirely fair, since the heuristic optimizes a proxy objective; acknowledging this in the text would help. I would encourage the editor to seek the revised manuscript, as the issues are addressable with additional experiments and a corrected MDP or feasibility-check step."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Both the reader and I land in the same place: this is a plausible extension of your prior IROS line, with a real contribution to learning-based persistent surveillance, but the headline comparison is not yet trustworthy because the DRL policy never checks whether the UGV can actually reach the chosen rendezvous point in time. The MDP state is (pt, ft, qt) with no UGV position or arrival time; the transition adds only UAV flight time plus fixed recharge time; and the masking rules filter by fuel reachability only. The UGV route is constructed after the fact. The heuristic O-EVRPTW baseline, by contrast, enforces Eq. 10 (ti >= tr_l,i), so it is held to a stronger feasibility standard. If the UGV cannot be at the rendezvous when the UAV arrives, every reported objective in Tables I-III is optimistic and the comparison against TS and the other baselines is unfair. This is the load-bearing flaw, and it holds up on reading the paper.\n\nWhat is genuinely new: the shift from a TSP-style single-tour problem to persistent surveillance with age-period rewards, the dynamic age embeddings that let the decoder prioritize stale points, and the recharge-action masking. The training curves and the 30-instance tables show a consistent gap over the attention model and the heuristics, and the runtime advantage of greedy decoding is real. The generalization tests across sizes and distributions are a reasonable effort, and the online insertion experiment is a nice touch.\n\nThe other soft spots are secondary but worth naming: no error bars or significance tests on the 30 instances, the priority increment S is tuned on the Hurricane Harvey scenario itself, the case study is a deterministic simulation, and no code or data is released. The omission of some VRP constraints in the heuristic baseline is pointed to prior work, which is acceptable but should be spelled out.\n\nBottom line: the paper is for the multi-robot routing community, and readers will get a clean problem formulation and a training setup described in enough detail to reimplement. It deserves a serious referee, but the referee should require a revised comparison with physically realizable routes before the headline claims are accepted. I would not cite the headline numbers in my own work yet; I would cite the formulation and the dynamic-embedding idea once the feasibility gap is closed.","headline":"A plausible extension of the authors' prior line with a real runtime advantage, but the DRL-vs-baseline comparison is not yet trustworthy because the policy ignores UGV travel time to rendezvous points.","tokens_in":26817,"tokens_out":3546,"would_cite":false,"duration_ms":33684,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a transformer-based deep reinforcement learning policy can plan cooperative UAV-UGV persistent surveillance—choosing both mission-point visits and recharging rendezvous—so as to minimize the squared age-period…","keywords":["persistent surveillance","UAV-UGV cooperation","deep reinforcement learning","transformer policy","mobile recharging station","age period","disaster management","online route planning"],"falsifier":"For any route produced by the trained policy, compute the UGV's earliest feasible arrival at each rendezvous from its previous position at its speed along the road network; if any rendezvous requires waiting that is physically impossible because the UGV cannot arrive in time, the route schedule is not realizable and the policy's superiority claim is weakened.","tokens_in":25760,"feed_emoji":"🚁","tokens_out":6755,"duration_ms":64092,"temperature":0.7,"pith_summary":"The paper tries to establish that a single deep reinforcement learning policy can solve the energy-constrained UAV-UGV cooperative persistent surveillance problem: deciding which mission points the drone should fly to and where it should meet a ground vehicle for recharging, so that the time between consecutive visits to every point stays small. This matters because persistent surveillance in disaster zones requires continuous, current information, and existing approaches are either slow heuristics or planning methods limited by a short horizon. The authors claim their transformer-based policy, trained with REINFORCE, produces lower objective values than heuristic baselines and an attention-model baseline in shorter runtimes. They also claim it generalizes to larger problem sizes, to Gaussian, Rayleigh, and Exponential point distributions, and to dynamically inserted mission points.","feed_headline":"Learned route policy beats heuristics for drone-ground disaster patrols","feed_subtitle":"A transformer policy schedules visits and recharging rendezvous, cutting revisit delays across problem sizes and new points.","key_machinery":"The machinery is the age-period formulation: the score is $S = \\frac{1}{\\alpha}\\sum_{k=1}^{n}\\sum_{q=1}^{p}(a_k^q)^2$, where the age period $a_k = t - t_{\\text{last}}$ is the time since mission point $k$ was last visited, and the squared form penalizes long gaps. This score is converted into a per-step reward $r_t = -(a_t)^2$, summed and scaled by the mission period. The MDP state is $s_t = (p_t, f_t, q_t)$, with $q_t$ holding the coordinates and age periods of all mission points. The policy is an encoder-decoder transformer: the encoder embeds coordinates and a recharge-eligibility bit; the decoder builds a context vector from the current position embedding, fuel level, and graph embedding, and adds a dynamic age-period embedding to the node embeddings at every decision step. A masked softmax over compatibility scores selects the next action, with masks enforcing fuel reachability, no consecutive recharges, no immediate same-point revisits, and the ability to reach a refuel stop on the following step. Training uses REINFORCE with a greedy rollout baseline.","core_discovery":"On the paper's own terms, the central discovery is that a single transformer-based DRL policy can solve the energy-constrained UAV-UGV cooperative persistent surveillance problem end to end. From a state consisting only of the drone's position, fuel level, and each mission point's age period, the policy learns to alternate visit actions with recharging rendezvous so that the squared age-period score is minimized. The authors report that this policy yields lower objective values than the Attention Model and the heuristic baselines on all tested problem sizes (U15G5, U30G10, U45G15, and the larger U60G20 and U75G25), across Gaussian, Rayleigh, and Exponential point distributions, and in a Hurricane Harvey case study, with the sampling-decoding variant DRL(10240) achieving the minimum objective everywhere. The paper also argues that the policy mitigates the horizon effect that hurts receding-horizon heuristics, supports online insertion of new points with only a modest objective increase, and can be steered by priority weights through an age-increment factor without retraining.","pith_inferences":["Editorial inference: The MDP state has no UGV position or arrival-time information, so the strongest validation would be a simulator that enforces UGV travel; the paper's route schedules may not be physically realizable in the field.","Editorial inference: The dynamic age-period embedding is a generic mechanism: any revisit-based routing objective (patrol, inspection, restocking) could reuse it to make decisions sensitive to how stale each target is.","Editorial inference: The masking rules form a reusable safety layer for energy-constrained multi-agent reinforcement learning, guaranteeing fuel feasibility and basic transition constraints without reward shaping.","Editorial inference: The generalization to larger sizes and other distributions suggests the policy is approximating a structural rule (visit stale points, rendezvous when fuel is low) rather than memorizing instances; a direct test would be evaluating on 100-plus-point instances."],"forward_implications":["If the central claim holds, disaster-response planners can use one trained policy for a 1000-minute mission on 20 UAV points and 10 ground points, and the same policy extends to 75 UAV points and 25 ground points without retraining while still beating the evaluated baselines.","The reported generalization across Gaussian, Rayleigh, and Exponential distributions means the policy transfers to mission layouts where points cluster near the road network or spread outward, covering a range of realistic disaster topologies.","The online-planning result implies that new mission points appearing within the first 200 minutes of a 1000-minute mission can be folded into the plan at rendezvous, at the cost of a raised score metric and maximum age period.","The priority-weighted variant implies that a uniform-weight policy can be repurposed for weighted surveillance by inflating the age of high-priority points, so no retraining is needed when visit priorities change.","The reported runtime advantage means the greedy-decoding policy could be used in settings where a route must be produced in seconds rather than the hundreds of seconds taken by the heuristic baselines."],"supporting_citations":[{"why":"Attention Model baseline that the proposed policy is compared against across all problem sizes and distributions.","marker":"[38]"},{"why":"Transformer architecture on which the encoder-decoder policy network is based.","marker":"[48]"},{"why":"REINFORCE policy-gradient rule with a greedy rollout baseline used to train the policy.","marker":"[52]"},{"why":"Fuel consumption model that sets the UAV's energy constraint and maximum flight time in the experiments.","marker":"[53]"},{"why":"OR-Tools CP-SAT solver with the metaheuristic baselines (GLS, TS, SA) used for comparison.","marker":"[46]"},{"why":"Prior DRL cooperative routing work that this persistent-surveillance framework extends.","marker":"[24]"},{"why":"Establishes the UGV-as-refueling concept for a fuel-constrained UAV that the problem formulation relies on.","marker":"[10]"},{"why":"Shows the impact of recharging instances on cooperative routing efficiency, motivating joint visit-and-recharge optimization.","marker":"[23]"}],"fun_headline_variants":["DRL beats heuristics for drone-ground disaster patrols","Transformer DRL outperforms baselines for drone-ground surveillance","Energy-aware DRL solves UAV-UGV recharging problem","DRL policy cuts revisit delays with transformer patrol","DRL wins over heuristics in UAV-UGV disaster missions"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the ground vehicle can always be at the chosen rendezvous point when the drone arrives, since the policy's state does not include the UGV's position or travel time.","fun_headline_variants_meta":{"raw":{"variants":["DRL beats heuristics for drone-ground disaster patrols","Transformer DRL outperforms baselines for drone-ground surveillance","Energy-aware DRL solves UAV-UGV recharging problem","DRL policy cuts revisit delays with transformer patrol","DRL wins over heuristics in UAV-UGV disaster missions"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001022,"raw_usage":{"total_tokens":4360,"prompt_tokens":1046,"completion_tokens":3314,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":662,"completion_tokens_details":{"reasoning_tokens":3232}},"tokens_in":662,"tokens_out":3314,"duration_ms":22166,"temperature":1.0,"reasoning_tokens":3232,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T11:32:31.447818+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"For any route produced by the trained policy, compute the UGV's earliest feasible arrival at each rendezvous from its previous position at its speed along the road network; if any rendezvous requires waiting that is physically impossible because the UGV cannot arrive in time, the route schedule is not realizable and the policy's superiority claim is weakened.","supporting_citations":[{"cited_title":"Simple statistical gradient-following al- gorithms for connectionist reinforcement learning,","cited_arxiv_id":null,"evidence_quote":"REINFORCE policy-gradient rule with a greedy rollout baseline used to train the policy."},{"cited_title":"Mobile robot battery life estimation: battery energy use of an unmanned ground vehicle,","cited_arxiv_id":null,"evidence_quote":"Fuel consumption model that sets the UAV's energy constraint and maximum flight time in the experiments."},{"cited_title":"Google OR-tools,","cited_arxiv_id":null,"evidence_quote":"OR-Tools CP-SAT solver with the metaheuristic baselines (GLS, TS, SA) used for comparison."},{"cited_title":"An attention-aware deep reinforcement learning framework for uav-ugv collaborative route planning,","cited_arxiv_id":null,"evidence_quote":"Prior DRL cooperative routing work that this persistent-surveillance framework extends."},{"cited_title":"On cooperation between a fuel constrained uav and a refueling ugv for large scale mapping applications,","cited_arxiv_id":null,"evidence_quote":"Establishes the UGV-as-refueling concept for a fuel-constrained UAV that the problem formulation relies on."},{"cited_title":"Heterogenous vehicle routing: comparing parameter tuning using genetic algorithm and bayesian optimization,","cited_arxiv_id":null,"evidence_quote":"Shows the impact of recharging instances on cooperative routing efficiency, motivating joint visit-and-recharge optimization."}],"review_version":1}