{"id":"854ea001-f2d9-449b-b484-3f40d4b76b9f","arxiv_id":"2507.08429","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A multi-agent PPO variant with LSTM temporal memory and a weighted local-global value function is proposed to reduce peak age of information in laser-charged UAV-assisted IoT networks, with simulated gains up to 15.1% over MADRL baselines.","lead":"This paper combines a multi-agent reinforcement learning algorithm, MAPPO-TM, with LSTM memory to plan UAV routes and laser-charging visits in an IoT data collection network, targeting fresher information. The simulations report up to 15.1% lower peak age of information than standard multi-agent deep reinforcement learning baselines.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reward-proxy misalignment: the handcrafted reward in Eq. (11), with unspecified r_a, r_s, and weights, is never shown to track the max-type peak-AoI objective in Eq. (5a), so all reported simulation gains may be artifacts of reward shaping.","rationale":"The reader's weakest assumption is essentially the same as the concern I identify: the hand-designed reward in Eqs. (10)-(11) is assumed to faithfully encode the peak-AoI objective, even though its components are unspecified. I therefore mark agreement as 'agree.' I do not see a more basic flaw in the system model or algorithm description: the POMDP is reasonable, the LSTM-based temporal memory and dual-attention value function are coherent, and the complexity analysis is internally consistent. The NP-hardness argument via TSP is informal, since minimizing peak AoI when all IoTs must be visited is not obviously identical to finding the shortest Hamiltonian path, but the paper's contribution is empirical and this does not directly threaten the central simulation claim. The absence of code, exact hyperparameters, and error bars is serious but secondary; the decisive issue is that the quantity measured in the objective may differ from the quantity optimized by the reward. If the authors supply the missing reward terms and a direct objective-based evaluation, the claim can be settled. Because the reader's CONDITIONAL verdict already requires exactly this kind of addressable evidence, I keep the verdict unchanged rather than escalating to unverified or reject.","tokens_in":32306,"tokens_out":6839,"duration_ms":83461,"concrete_test":"Ask the authors to release the full reward specification (exact r_a(t), r_s(t), and all constants) and the trained policy checkpoints, then rerun the benchmark in the original simulator with N≥10 independent seeds. Compute the actual objective A = max over devices and time slots of a_i(t) and a defined energy-efficiency metric for MAPPO-TM, MAPPO, MATD3, and MADDPG, reporting mean and 95% CI. If MAPPO-TM's higher cumulative reward does not translate into significantly lower A, the 15.1% peak-AoI claim is an artifact of reward shaping rather than a property of the algorithm.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is an empirical statement about peak AoI, but the objective actually optimized by MAPPO-TM is a different quantity. Problem (5a) asks to minimize A = max over devices and time slots of a_i(t), a max-type, non-additive metric. The POMDP in Section IV-D instead maximizes the expected discounted sum of the handcrafted reward r_t = α r_a(t)+β r_p(t)+γ r_s(t) (Eq. 11). The paper never defines r_a(t), r_s(t), α, β, γ, r_pen1, r_pen2, or E_phi (Eq. 10), and it provides neither a proof nor a simulation-grounded argument that maximizing this additive proxy is equivalent to, or even monotonically related to, minimizing true peak AoI. All peak-AoI evidence comes from the same simulator that uses this reward, with no raw A values, error bars, or seeds, and the abstract's 'up to 15.1%' cannot be obtained from the reported 5-10% range in Section V-B2. If the proxy is misaligned, MAPPO-TM can earn higher cumulative reward than baselines while producing equal or worse real peak AoI, so the paper's headline result would be unsupported. This is the single most load-bearing concern because reward fidelity is the only bridge from the algorithm's training objective to the evaluated system objective.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies an IoT data-collection network in which multiple UAVs are recharged by laser beam directors (LBDs) while collecting data from ground IoT devices. It formulates a joint optimization problem minimizing peak Age of Information (AoI) over UAV trajectories and charging decisions, argues NP-hardness via TSP, and reformulates the problem as a POMDP. The proposed algorithm, MAPPO-TM, extends MAPPO with LSTM-based actor networks and a weighted local-plus-global value function for multi-agent coordination. Simulation comparisons against MAPPO, MATD3, and MADDPG are reported for cumulative reward, AoI reward, energy reward, peak AoI, and UAV trajectories, with the abstract claiming up to 15.1% peak-AoI reduction.","tokens_in":32678,"tokens_out":3762,"duration_ms":53327,"significance":"If the claimed results are reproducible, the paper would provide a useful application of multi-agent reinforcement learning to laser-charged UAV-assisted IoT networks, combining a standard POMDP formulation with two sensible architectural extensions: temporal memory in actors and a local-global value decomposition. The system model and notation are clearly presented, the complexity comparison in Table III is informative, and the comparison against three common MADRL baselines is appropriate. However, the central empirical claim is currently supported only by self-reported simulation plots with undisclosed reward ingredients, no error bars or seed counts, and an abstract number that cannot be traced to any figure or table. The contribution is therefore plausible but not yet verified to the standard expected for a journal publication.","major_comments":[{"comment":"The reward function is the only learning signal, but its components r_a(t) and r_s(t), the weights alpha, beta, gamma, the penalty constants r_pen1 and r_pen2, and the energy threshold E_phi are never specified numerically or even as functional forms. The text only says that r_a(t) is 'the reward associated with the AoI' and r_s(t) rewards successful collection. This is load-bearing because all reported gains are measured after optimizing this reward. The authors must define these terms completely and provide either a proof or a simulation-grounded argument that maximizing the additive reward in Eq. (11) is monotonically related to minimizing the max-type peak AoI objective in Eq. (5a). Without that bridge, a policy can earn higher cumulative reward while producing the same or worse actual peak AoI.","section":"Section IV-D, Eqs. (10)-(11)"},{"comment":"The abstract claims 'up to 15.1% reduction in peak AoI compared to conventional MADRL methods,' but Fig. 4(a) is a bar chart without numerical values, and Section V-B2 reports only a 5-10% lower peak AoI for the eta_le sweep. No figure or table in the paper displays the 15.1% figure or identifies the specific configuration that produces it. The authors need to report the exact peak AoI values for all algorithms, the corresponding standard deviations, and the precise condition (e.g., number of IoTs, eta_le value, episode index) under which 15.1% is achieved.","section":"Section V-B, Fig. 4(a) and Abstract"},{"comment":"All conclusions are drawn from the authors' own simulator without reporting the number of random seeds, error bars, or statistical significance. The shaded variance regions in Fig. 3 and the bar chart in Fig. 4(a) are described qualitatively, and the final peak AoI comparison in Section V-B2 is given only as a percentage range. Because DRL results are stochastic and the baselines are also sensitive to hyperparameters, the authors should provide per-seed results, mean and standard deviation over at least several seeds, and a statistical test (or at least non-overlapping confidence intervals) before claiming that MAPPO-TM outperforms the baselines.","section":"Section V-A and V-B"},{"comment":"The NP-hardness reduction is stated too loosely to be rigorous. The authors claim that a simplified single-UAV case 'reduces to finding the shortest path that visits each IoT exactly once, precisely the definition of TSP,' but the objective in Eq. (5a) is peak AoI, not tour length, and the simplified problem also omits how AoI evolves over time during the tour. A formal reduction should construct an instance of the peak-AoI problem whose optimal value encodes the TSP tour length, or the authors should weaken the claim to a statement that the problem 'contains TSP as a special case' with a clear mapping. This issue does not affect the algorithm itself, but it is a correctness claim in the problem formulation.","section":"Section III-F"}],"minor_comments":[{"comment":"The data rate Rf_ij(t) is called 'in bits/Hz' but the formula includes the bandwidth W and outputs bits/s; also b1 and b2 in the LoS probability expression are not defined in Table II.","section":"Eq. (1) and Table II"},{"comment":"The peak AoI definition uses the notation Q(t) without defining it, and the max should be over the IoT index i and time t; please rewrite as A = max_{i,t} a_i(t) or define Q(t) explicitly.","section":"Eq. (3)"},{"comment":"The parameters beta0 and sigma2 are listed in one table cell with the value '80 dB' for sigma2, but the value of beta0 is missing; this prevents reproduction of Eq. (1).","section":"Table IV"},{"comment":"The penalty for E_j(t)=E appears to penalize a fully charged UAV regardless of whether it is inside or outside the charging area, which may conflict with the intent to encourage leaving the charging zone; please clarify whether the penalty applies only when the UAV is inside the charging area and what r0 represents.","section":"Eq. (10)"},{"comment":"Algorithm 1 updates the critic and actor inside every time slot, whereas the clipped surrogate objective in Eq. (14) and standard PPO practice assume updates on collected trajectory batches; please specify the actual update schedule (e.g., after a rollout of length T) and how the experience buffer is sampled.","section":"Algorithm 1"},{"comment":"The mechanism in Eq. (17) is a weighted sum of two value functions with learnable scalar weights, but it is repeatedly called a 'dual-attention mechanism'; this is not attention in the usual sense, so either rename the mechanism or provide an actual attention formulation with query/key/value vectors.","section":"Section IV-F2"}],"recommendation":"major_revision","confidential_remarks":"The manuscript has a clear structure and a relevant topic, but the evaluation section is not yet at journal standard. The most urgent issues are the undeclared reward components and the untraceable 15.1% headline figure; both can be fixed in a revision by disclosing all reward terms and hyperparameters, adding raw numerical results with seeds and error bars, and aligning the abstract with the reported numbers. I would be willing to review a revised version. The NP-hardness argument and the 'dual-attention' naming also need tightening, but these are secondary."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The new thing here is the integrated problem: multiple UAVs collecting IoT data while laser-charged, with peak AoI as the objective, in both charging and non-charging areas. That specific combination is not in the cited work, and the system model is coherent. The paper also does a solid job with the related work, and the MAPPO-LSTM with a weighted local-global value function is a well-motivated algorithm for this problem. The complexity analysis is a plus.\n\nThe soft spots are real and they are in the evaluation. The abstract's 15.1% peak AoI reduction is unsupported by the presented results: Section V-B2 reports 5-10% for the ηle sweep, and Fig. 4(a) is a bar chart with no numbers, no error bars, no seed counts. The reward function in Eq. (11) is underspecified (ra, rs, α, β, γ, rpen1, rpen2, Eφ never given). More importantly, the reward is an additive proxy, while the stated objective in Eq. (5a) is a max-type peak AoI. The paper never shows that maximizing this proxy is monotonically related to minimizing true peak AoI. Because all comparisons are in the same simulator using this reward, the reported gains could be artifacts of reward shaping. The stress-test note is right.\n\nMinor issues: the IoT energy constraint (5c) appears unused, and the robustness discussion is speculative, with no simulations. These are secondary.\n\nBottom line: this is a plausible engineering contribution for a niche subfield. It is not a breakthrough, but it is a coherent new application. It deserves a serious referee, but it needs major revision: specify the reward components and weights, report actual peak AoI values with seeds and error bars, and either train on an objective more directly linked to peak AoI or empirically demonstrate the proxy is aligned. I would send it out, expecting heavy revision.","headline":"Decent system model and a plausible RL formulation, but the headline 15.1% claim is not supported by the reported results and the reward function is underspecified.","tokens_in":33205,"tokens_out":2599,"would_cite":false,"duration_ms":29633,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims a MAPPO variant with LSTM temporal memory and a dual-attention value function reduces peak age-of-information by up to 15.1% in laser-charged multi-UAV IoT networks.","keywords":["age of information","laser charging","UAV trajectory optimization","multi-agent deep reinforcement learning","MAPPO","LSTM temporal memory","IoT data collection","peak AoI"],"falsifier":"Train MAPPO-TM with the reward function replaced by the actual peak-AoI value from Eq. (5a) (or a direct measurable surrogate), and compare the resulting policies against the baselines; if the 15.1% peak-AoI reduction disappears, the reported gain is an artifact of reward shaping rather than genuine AoI minimization.","tokens_in":32134,"feed_emoji":"🚁","tokens_out":5528,"duration_ms":54196,"temperature":0.7,"pith_summary":"This paper claims that a multi-agent reinforcement learning algorithm, MAPPO-TM, can jointly optimize the flight trajectories and laser-charging decisions of multiple UAVs so as to minimize the peak age of information (AoI) in an IoT data-collection network. The system it models lets UAVs recharge wirelessly from laser beam directors while flying, avoiding returns to fixed charging stations. If the claim holds, MADRL methods can sustain information freshness in long-duration missions where energy is the bottleneck. The evidence is a set of simulations in which MAPPO-TM outperforms MAPPO, MATD3, and MADDPG on peak AoI and energy rewards, with up to a 15.1% reduction in peak AoI.","feed_headline":"Laser-charged UAV RL cuts peak data age by 15.1%","feed_subtitle":"A multi-agent PPO variant with temporal memory beats MADDPG, MATD3, and MAPPO in AoI and energy efficiency.","key_machinery":"The load-bearing mechanism is the MAPPO-TM policy architecture: an LSTM-enhanced actor network that maintains a hidden state $h_i^t = \\mathrm{LSTM}(o_i^t, h_i^{t-1})$ to capture temporal dependencies, paired with a centralized critic that blends a local value network and a global value network through learnable weights $w_l$ and $w_g$. Together these convert a high-dimensional, non-convex joint control problem (UAV trajectories plus charging decisions) into a partially observable Markov decision process that can be trained with decentralized execution. The reward function in Eqs. (10)-(11), which penalizes both low-energy and full-energy states and rewards data collection, is what couples the learning objective to the AoI and energy goals, though its numerical components are not specified.","core_discovery":"On its own terms, the paper's central discovery is that adding two mechanisms to standard MAPPO unlocks better trade-offs between information freshness and energy use in laser-charged multi-UAV IoT networks. The first is temporal memory: LSTM-based actor networks let each UAV use historical trajectories and energy states rather than only the current observation. The second is multi-agent coordination: a dual-attention value function $V_i(s_t)=w_l V_i^{\\text{local}}(o_i^t)+w_g V^{\\text{global}}(s_t)$ lets each agent dynamically weight its individual objective against the global system objective. With these additions, the learned policies keep UAVs near charging areas when energy is low, collect IoT data efficiently, and reduce the network's peak AoI, according to the reported simulations.","pith_inferences":["If the reward function in Eqs. (10)-(11) does not track the actual peak AoI, then the published numerical gain is a statement about reward optimization, not about information freshness; a direct test is to re-run the comparison with the true objective as reward.","The paper's NP-hardness argument reduces the problem to TSP by ignoring charging and collision constraints; a stronger reduction would show hardness persists with the full set of constraints.","The approach could be extended to time-varying IoT data generation, where LSTM memory might give an even larger advantage over feed-forward baselines, though the paper only tests static data volumes."],"forward_implications":["If the algorithm works as claimed, any fleet of laser-charged UAVs can be scheduled with CTDE-trained policies that run in execution time $O(TN|\\theta_a|)$ and space $O(N|\\theta_a|)$, making real-time deployment plausible.","The LSTM temporal memory should yield more stable training and lower variance in both AoI and energy rewards than feed-forward MAPPO, as the paper reports in its training curves.","The dual-attention value function should make the system robust to UAV failures, since remaining agents can reweight global versus local objectives when the fleet changes.","The 15.1% peak-AoI reduction is the headline quantitative claim, and it appears to hold across a range of laser-to-electricity conversion efficiencies $\\eta_{le}$.","The paper argues the formulated problem is NP-hard by reducing a simplified single-UAV case to the traveling salesman problem, implying that exact methods are intractable and learning-based approximations are justified."],"supporting_citations":[{"why":"Defines the age-optimal trajectory planning problem for a single UAV collecting from IoTs, which this work extends to multiple laser-charged UAVs.","marker":"[5]"},{"why":"Supplies the rotary-wing UAV propulsion power model used in the energy-consumption analysis and constraints.","marker":"[12]"},{"why":"Demonstrates deep reinforcement learning for fresh data collection in UAV-assisted IoT networks, the approach this paper generalizes to multi-agent settings.","marker":"[23]"},{"why":"Provides the distributed laser charging model that underpins the wireless charging assumptions for LBDs.","marker":"[109]"},{"why":"Provides the baseline MAPPO algorithm whose architecture the paper extends with LSTM and dual-attention mechanisms.","marker":"[124]"},{"why":"Provides the MATD3 baseline with double centralized critics used for comparison.","marker":"[125]"},{"why":"Provides the MADDPG baseline for multi-agent coordination comparison.","marker":"[126]"}],"fun_headline_variants":["Laser-charged UAVs cut peak data age 15.1% via multi-agent RL","Multi-agent PPO with memory freshens IoT data 15.1%","Laser UAVs use temporal-memory RL to slash AoI 15.1%","15.1% fresher: Laser-charged UAVs + coordinated RL","MAPPO-TM beats standard RL in laser UAV AoI by 15.1%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper assumes that the hand-designed reward function in Eqs. (10)-(11) faithfully encodes the true peak-AoI objective of Eq. (5a), but it never specifies the reward's numerical components and offers no evidence that maximizing that reward transfers to minimizing real peak AoI.","fun_headline_variants_meta":{"raw":{"variants":["Laser-charged UAVs cut peak data age 15.1% via multi-agent RL","Multi-agent PPO with memory freshens IoT data 15.1%","Laser UAVs use temporal-memory RL to slash AoI 15.1%","15.1% fresher: Laser-charged UAVs + coordinated RL","MAPPO-TM beats standard RL in laser UAV AoI by 15.1%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00021,"raw_usage":{"total_tokens":1434,"prompt_tokens":994,"completion_tokens":440,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":610,"completion_tokens_details":{"reasoning_tokens":327}},"tokens_in":610,"tokens_out":440,"duration_ms":5222,"temperature":1.0,"reasoning_tokens":327,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T18:19:29.251831+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train MAPPO-TM with the reward function replaced by the actual peak-AoI value from Eq. (5a) (or a direct measurable surrogate), and compare the resulting policies against the baselines; if the 15.1% peak-AoI reduction disappears, the reported gain is an artifact of reward shaping rather than genuine AoI minimization.","supporting_citations":[{"cited_title":"Distributed laser charging: A wireless power transfer approach,","cited_arxiv_id":null,"evidence_quote":"Provides the distributed laser charging model that underpins the wireless charging assumptions for LBDs."},{"cited_title":"The surprising effectiveness of PPO in cooperative multi-agent games,","cited_arxiv_id":null,"evidence_quote":"Provides the baseline MAPPO algorithm whose architecture the paper extends with LSTM and dual-attention mechanisms."},{"cited_title":"Multi- agent actor-critic for mixed cooperative-competitive environments,","cited_arxiv_id":null,"evidence_quote":"Provides the MADDPG baseline for multi-agent coordination comparison."}],"review_version":1}