{"id":"8bb2c609-f5f5-4fd2-a0ee-8cf7c34586f3","arxiv_id":"2507.14299","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A deep reinforcement learning controller jointly plans a UAV's trajectory and beamforming to minimize the average age of information in a sensing-and-communication system.","lead":"This paper proposes a drone that both tracks a moving target with radar and sends fresh updates to ground users, using a deep reinforcement learning controller to choose the drone's path and signal beams. The goal is to keep the age of information at users low, and simulations show the proposed controller beats three simpler baselines on average.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The AoI generation rule reduces to a scalar SNR threshold derived from the isotropic CRLB covariance in Eq. (15); if that covariance is not the true radar-error model, the entire AoI comparison shifts.","rationale":"The strongest empirical claim is the consistent AoI dominance of SAC over the three baselines. The way AoI is generated in the simulation is through a threshold on SNRp, and that threshold comes directly from the CRLB covariance in Eq. (15). If this covariance is not representative of a real radar, then the simulated AoI values are not the AoI of any real system, and the central claim does not transfer. This is a correctness issue in the core model because Eqs. (28) and (29) depend on it at every slot; it is not merely a missing baseline or a statistical-reporting gap. The objective mismatch between the discounted RL return and the undiscounted average AoI, the Kalman typo in Eq. (22), and the reward scaling inconsistency are real but either easy to fix or affect all compared schemes roughly equally; they do not go to the heart of whether the simulated ordering is physically meaningful. The reader's weakest assumption identifies the same model-dependence, so the reader's CONDITIONAL verdict remains appropriate: the paper is a plausible contribution if the sensing-noise model is accepted, but the authors should validate or relax the CRLB assumption, or the central claim should be presented as limited to that idealized model.","tokens_in":19042,"tokens_out":17825,"duration_ms":154062,"concrete_test":"Replace the isotropic CRLB covariance in Eq. (15) with a UPA-aware measurement covariance derived from the Fisher information of delay and 2D angle estimates (non-diagonal, range/angle-dependent), regenerate the simulated measurements in Eq. (14) with this R, and rerun the average-AoI sweeps in Figs. 5, 6, 10, and 11. If SAC no longer dominates all three baselines across the swept ranges, the central claim is an artifact of the idealized noise model; if the ordering persists, the concern is mitigated.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's AoI dynamics are driven by the reliability test (16) and update rule (28): a new target state is generated exactly when SNRp[n] >= SNRth, and SNRth is derived from the CRLB covariance R[n] = sigma0^2/(SNRp[n]+epsilon) I2 in Eq. (15). The same R[n] is used to synthesize the simulated measurements in Eq. (14) and in the Kalman filter update (25)-(27), so the simulation cannot reveal errors in this model. The central claim that SAC consistently achieves the lowest average AoI across sensing-accuracy requirements (Fig. 6) is therefore conditional on assumptions that are unlikely to hold jointly in a real UPA radar: (i) the Cartesian measurement error is isotropic with equal x/y variance; (ii) that variance depends only on scalar SNRp, not on range, bearing, antenna beam shape, or array geometry; and (iii) the reliability test reduces to a scalar SNR threshold. Real delay/angle estimation from a UPA produces a range- and angle-dependent Fisher information matrix; the resulting error covariance is generally non-diagonal and anisotropic, and is not captured by a single SNR value. Under such a model, the slots in which updates are generated change, the KF covariance and beam-pointing errors change, and the optimal trajectory/power trade-off in Figs. 5-8 may differ. The reported ordering among SAC, A2C, SAGS, and KF-RAND is thus only as strong as Eq. (15).","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript studies a single-UAV ISAC system in which the UAV simultaneously tracks a moving target and serves K ground users, and proposes minimizing the long-term average AoI at the users by jointly optimizing UAV trajectory, per-slot power allocation, and beam directions. The transmit waveform superimposes communication and sensing beams; the sensing measurement is modeled by a CRLB-based isotropic covariance with an SNR reliability gate, a Kalman filter estimates the target state, and regularized zero-forcing forms user beams. The joint problem is cast as a finite-horizon MDP and solved with Soft Actor-Critic, with a state containing UAV/users/target/KF uncertainty and reward equal to negative average AoI. Simulations compare SAC against A2C, an AoI-greedy scheduler, and a random KF-based scheme, reporting lower average AoI for SAC over varying SINR thresholds, sensing accuracy requirements, array sizes, and user counts.","tokens_in":19308,"tokens_out":6917,"duration_ms":76647,"significance":"If the results hold, the paper makes a useful contribution by demonstrating that a single DRL policy can coordinate trajectory, sensing power, and multiuser beamforming for freshness-critical ISAC, going beyond prior AoI-UAV studies that ignore spatial beamforming. The MDP formulation, state/action design, and complexity analysis are clear, and the use of fixed user-layout seeds across methods is a good reproducibility practice. The central claim, however, is supported only by simulation under a heavily stylized sensing model and without statistical uncertainty, and several formulation and implementation issues need to be resolved before the results can be taken as conclusive.","major_comments":[{"comment":"The central comparative claim that SAC consistently achieves the lowest average AoI is made without error bars or confidence intervals, although the text states that results are averaged over 100 Monte Carlo episodes. For curves where the SAC/A2C gap is small, the reported ordering could be within sampling noise; please report standard errors or confidence intervals and, where useful, paired differences per user layout.","section":"IV-B, Figs. 5, 6, 10, 11"},{"comment":"The training objective is the discounted return sum_{n=1}^N gamma^{n-1} r[n] with gamma=0.99 and N=60, while P1 minimizes the undiscounted time-averaged AoI. With gamma^{59} approximately 0.55, later slots are substantially down-weighted, so the learned policy optimizes a different objective than the one stated. Please either use an average-reward formulation (or gamma=1 with appropriate variance control) or provide evidence that the relative ranking is insensitive to gamma.","section":"III-B5, Eq. (37) vs II-E, Eq. (33)"},{"comment":"The KF prediction equation is written as \\hat{s}_T^-[n] = F \\hat{s}_T^-[n-1]; given the paper's notation (superscript '-' denotes a predicted value before new measurements), this makes the filter base each prediction on the previous prediction rather than the previous posterior \\hat{s}_T[n-1], so measurement updates are effectively discarded. Since the KF output steers the sensing beam in (41), this is not a purely cosmetic issue; please correct the equation and confirm that the implementation matches the corrected filter.","section":"II-C2, Eq. (22)"},{"comment":"The reliability test (16)-(17) and the AoI update rule (28) reduce the sensing process to a scalar SNR threshold derived from the isotropic CRLB covariance R[n]=sigma_0^2/(SNR_p[n]+epsilon) I_2. For a UPA, the Fisher information for delay/angle estimation is range- and angle-dependent and generally yields an anisotropic, non-diagonal error covariance, so the simulation's measurement generation (14), the KF update (25)-(27), and the gating all share the same simplified model. The AoI ordering across schemes may change under a more realistic sensing model; please add a sensitivity study (e.g., an FIM-based covariance with range/angle dependence, or at least anisotropic perturbations of R) to show the SAC advantage persists.","section":"II-C1, Eqs. (15) and (17)"},{"comment":"There is a numerical inconsistency in the default parameters: with c=3e8 m/s and B=100 MHz, Eq. (15) gives sigma_0 approximately 0.60 m, whereas Table I lists sigma_0=0.338 m. This changes SNR_th=(sigma_0/sigma_req)^2 by about 5 dB and therefore directly shifts the gating event in (28) and all AoI numbers. Please correct the equation, the table, or the simulation parameters and re-run the affected figures.","section":"Eq. (15) and Table I"}],"minor_comments":[{"comment":"Algorithm 1 sets r[n] = -sum_k Delta_k[n], while Eq. (37) defines r[n] = -\\bar{\\Delta}[n]; for fixed K these differ by a constant factor, but the two forms should be made consistent.","section":"Algorithm 1, line 15"},{"comment":"The text says the construction guarantees the terminal constraint p_T^h[N-1] = p_N; with the slot indexing used elsewhere, this should probably be p_T^h[N] = p_N, or the indexing convention should be stated explicitly.","section":"IV-A, target trajectory paragraph"},{"comment":"There are typos: 'is is well-suited' in Section III-D1 and 'we define the the long-term' near Eq. (31); please proofread the manuscript.","section":"III-D1 and II-D"},{"comment":"Figure 10 varies the UPA size but the surrounding text lists '4 x 4 UPA' among fixed parameters; this should be rephrased to avoid confusion.","section":"Fig. 10 caption and main text"},{"comment":"The baselines SAGS and KF-RAND are quite weak (single-user greedy and random allocation); adding a conventional myopic optimization baseline (e.g., trajectory-only or beam-only optimization) would make the DRL gains more informative.","section":"IV-A, compared algorithms"}],"recommendation":"major_revision","confidential_remarks":"The paper is within scope for a wireless communications/signal processing journal. The most serious issue is that the simulated sensing model is self-consistent: measurements, the KF, and the gating rule all use Eq. (15), so the simulation cannot validate that model. I would ask the authors to include a robustness check and, ideally, to share simulation code to verify reproducibility. The fixed-seed layout reuse is commendable and should be retained."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a solid simulation-driven contribution to AoI-centric UAV-ISAC, with a genuinely new combination of trajectory and spatial beamforming under an AoI objective. It deserves a serious referee, but the experimental section needs work before I'd trust the ordering across all regimes.\n\nWhat's actually new: prior AoI-UAV-ISAC papers (Zhu, Zhou, Mei, Liu) optimized trajectory or time-division protocols without multi-user spatial beamforming. This paper adds a UPA, regularized zero-forcing for the communication beams, a Kalman-filtered sensing beam, and SAC to learn trajectory and power shares. The integration is clean and the system model is detailed enough to reproduce the simulation. The complexity analysis is a nice touch. No analytical claims are fitted to data, so there is no circularity.\n\nSoft spots, in increasing order of concern. Minor: the Kalman prediction equation (22) uses the previous predicted state rather than the updated estimate; that is a typo with real consequences for the filter. Also, Algorithm 1's reward is minus the sum of AoIs while Section III-B5 says minus the average; a normalization inconsistency. More substantial: the RL objective is a discounted sum of per-slot rewards, but the problem is long-term average AoI. With gamma=0.99 and N=60, the discount is mild, but the formal mismatch should be fixed or acknowledged.\n\nThe bigger weakness is the sensing model. The AoI update rule (28) triggers only when SNRp >= SNRth, and SNRth comes from the isotropic CRLB covariance in (15). The same covariance is used to synthesize measurements and run the Kalman filter, so the simulation cannot expose errors in that model. In a real UPA radar, the error covariance is range-, angle-, and geometry-dependent, and generally anisotropic, so the exact slot pattern of target updates, and hence the AoI numbers, could shift. The comparisons are internally consistent, but transfer to any specific hardware is conditional on Eq. (15). The authors should state this as a limitation and ideally run a sensitivity check with a non-diagonal or range-dependent R.\n\nOne more: the baselines are weak. A2C, SAGS, and KF-RAND are sanity checks, but there is no comparison to prior AoI-UAV-ISAC methods or to an alternating-optimization heuristic. Without those, \"consistently lower AoI\" is shown only against weak baselines. And there are no error bars on any AoI figure, which is material when the gaps are small at loose requirements (e.g., Fig. 6 at sigma_req = 2 or 4). Code release would help reproducibility.\n\nWho this is for: researchers in UAV-ISAC and AoI-driven resource allocation who want a working DRL recipe and a clean system model to build on. This is a meaningful step, not a breakthrough. I would engage with it as a referee and ask for the sensitivity analysis, better baselines, and fixed equations before publication.","headline":"Solid simulation-driven AoI-UAV-ISAC paper with a genuinely new joint trajectory-beamforming angle; worth refereeing, but the sensing model and baseline choices need tightening.","tokens_in":19881,"tokens_out":3738,"would_cite":true,"duration_ms":440116,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that one learned policy can jointly steer a UAV and shape its beams to keep target-status information fresher at ground users than three baseline approaches, across varying SINR thresholds, sensing accuracy requirements…","keywords":["integrated sensing and communication","age of information","unmanned aerial vehicle","deep reinforcement learning","Soft Actor-Critic","beamforming","Kalman filter","trajectory optimization"],"falsifier":"In a high-fidelity channel emulator or field trial, measure the actual covariance of the radar position measurements at the SNRs the UAV experiences, and re-run the same SAC policy and baselines with the empirical $R[n]$ substituted for Eq. (15). If the average-AoI ordering between the SAC controller and the baselines reverses, or if the policy's adaptive threshold ceases to track $\\mathrm{SNR}_{\\mathrm{th}}$, the CRLB-driven update rule is the part of the model that failed.","tokens_in":18827,"feed_emoji":"🚁","tokens_out":9056,"duration_ms":91844,"temperature":0.7,"pith_summary":"This paper claims that a single learned controller can keep information fresh in a UAV-enabled integrated sensing and communication (ISAC) downlink by jointly deciding where the UAV flies and how its antenna array splits power and beams between a moving target and several ground users. The performance metric is the long-term average Age of Information (AoI): a user accumulates age each slot, and its age resets only when the radar measurement of the target is reliable enough to generate a status update and the user's downlink SINR clears its decoding threshold. The authors train a Soft Actor-Critic agent with reward equal to the negative average AoI, using a Kalman filter to aim the sensing beam and regularized zero-forcing to shape user beams. The paper's central claim is that this controller achieves consistently lower average AoI than three baselines across varying SINR thresholds, sensing accuracy requirements, antenna array sizes, and user counts. If true, that means the coupled sensing-communication trade-off can be resolved in real time by one learned policy instead of hand-decomposed optimization.","feed_headline":"Learned drone policy keeps sensing data fresher than baselines","feed_subtitle":"One SAC-trained controller beats three baselines on average age of information in every tested setting.","key_machinery":"The load-bearing mechanism is the AoI update rule in Eqs. (28)-(29), where the freshness of the target status at each user depends on two gates: the radar SNR $\\mathrm{SNR}_p[n]$ against the threshold $\\mathrm{SNR}_{\\mathrm{th}}$ for generating an update, and the communication SINR $\\Gamma_k[n]$ against $\\Gamma_{\\mathrm{th}}$ for receiving it. The radar SNR is obtained from an array-factor-extended radar equation with coherent pulse integration, and the measurement covariance that feeds the Kalman filter is taken as the CRLB expression $R[n]=\\sigma_0^2/(\\mathrm{SNR}_p[n]+\\varepsilon)I_2$. The optimization machinery is a finite-horizon Markov Decision Process whose state packs UAV position, user geometry, SINRs, per-user AoIs, Kalman-filter estimate and covariance, radar SNR, and mission progress; whose action is the continuous triple of UAV displacement, priority scores, and an adaptive threshold; and whose post-processing maps that triple to feasible powers and beam directions. Soft Actor-Critic training with twin critics, automatic entropy temperature, and a replay buffer is what keeps this continuous-action policy stable enough to learn the coupled behavior.","core_discovery":"The central discovery, stated on the paper's own terms, is that the AoI recursion couples the radar and communication gates, and that a deep reinforcement learning policy can be trained to navigate that coupling. An update is generated only when the post-detection sensing SNR passes $\\mathrm{SNR}_{\\mathrm{th}}$, and it is received by user $k$ only when the downlink SINR $\\Gamma_k[n]$ passes $\\Gamma_{\\mathrm{th}}$; otherwise the user's age increments. By letting the agent output a two-dimensional displacement, a set of per-beam priority scores, and an adaptive threshold, and then mapping these outputs through a softmax power split, regularized zero-forcing beam directions, and a Kalman-filter-steered sensing beam, the policy learns where to fly and which beams to energize. In the simulations, the resulting SAC controller holds the lowest average AoI for every tested value of the SINR threshold $\\Gamma_{\\mathrm{th}} \\in \\{0,5,10,15,20\\}$ dB, the sensing accuracy requirement $\\sigma_{\\mathrm{req}} \\in \\{0.1,0.5,1,2,4\\}$ m, the UPA size from $2\\times2$ to $6\\times6$, and the user count from 3 to 15, and its learned trajectories visibly bend away from the target toward distant users as the sensing requirement is relaxed.","pith_inferences":["The threshold-plus-priority action representation suggests an implicit schedulability rule: a user is served only when its learned score exceeds the adaptive threshold, so the policy could be distilled into a lightweight channel-aware scheduler for hardware-constrained deployments.","Because the state already tracks per-user AoI, radar SNR, and Kalman uncertainty, the same MDP could be retargeted to peak-AoI or AoI-violation constraints by swapping the reward function, which the paper does not explore.","The free-space Friis channel and perfect Doppler compensation are part of the model; in multipath or interference-rich environments the policy would need channel-state information in its state vector, and the observed performance ordering may not persist.","The reported per-slot inference cost of $O(F_\\pi + K + M^3)$ indicates the learned policy is computationally light enough for onboard execution, but the paper does not test real-time inference on actual UAV hardware."],"forward_implications":["If the central claim is right, a single trained SAC policy can act as a real-time UAV-ISAC controller, since the same agent generalizes across different SINR thresholds, sensing accuracy requirements, array sizes, and user counts in the simulations.","The learned trajectories make the sensing-communication trade-off explicit: tightening $\\sigma_{\\mathrm{req}}$ pulls the UAV close to the target and raises radar SNR, while loosening it lets the UAV drift toward users and improves their SINR.","Larger antenna arrays reduce average AoI for every scheme, and the SAC policy uses the extra spatial degrees of freedom to satisfy all users while sustaining the sensing beam.","Increasing the number of users raises average AoI for all policies, but the SAC controller keeps the lowest age even when 15 users contend for the same power budget.","The SAC controller converges faster and more stably than the on-policy A2C baseline, suggesting that off-policy entropy-regularized updates are better matched to this continuous-action problem."],"supporting_citations":[{"why":"Supplies the CRLB-based measurement covariance model $R[n]$ that drives the sensing reliability gate and hence the AoI update rule.","marker":"[23], [24]"},{"why":"Provides the superimposed waveform design that lets sensing and multi-user communication share the same transmit signal and array.","marker":"[36]–[38]"},{"why":"Supports the assumption that the sensing waveform is orthogonal to user despreading codes, so it contributes no interference to user SINR.","marker":"[40]"},{"why":"Supplies the radar range equation and coherent pulse integration model used to compute post-detection sensing SNR.","marker":"[41]"},{"why":"Provides the discrete-time Kalman filter machinery used to predict and track the moving target and to steer the sensing beam.","marker":"[42], [43]"},{"why":"Defines the Markov decision process formulation that turns the joint optimization into a sequential decision problem.","marker":"[44]"},{"why":"Supplies the Soft Actor-Critic algorithm used to train the continuous-action policy.","marker":"[45]"},{"why":"Defines the on-policy A2C baseline against which the SAC controller is compared in the simulations.","marker":"[46]"},{"why":"Establishes Age of Information as the timeliness metric and motivates its use for time-critical UAV-ISAC missions.","marker":"[12], [13]"}],"fun_headline_variants":["SAC-trained drone minimizes age of information in ISAC","RL agent cuts information staleness in UAV sensing and comms","Deep RL learns to balance sensing and communication for fresh data","SAC drone policy lowers average AoI across all tested conditions","Beamforming and trajectory learned to keep sensor data fresh"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The entire AoI ordering rests on the simulated radar measurement noise being exactly the CRLB expression $R[n]=\\sigma_0^2/(\\mathrm{SNR}_p[n]+\\varepsilon)I_2$ and on the reliability test that turns high radar SNR into a fresh status update; if real radar errors differ because of multipath, beam shape, or imperfect pulse integration, the simulated AoI ordering between schemes may not transfer.","fun_headline_variants_meta":{"raw":{"variants":["SAC-trained drone minimizes age of information in ISAC","RL agent cuts information staleness in UAV sensing and comms","Deep RL learns to balance sensing and communication for fresh data","SAC drone policy lowers average AoI across all tested conditions","Beamforming and trajectory learned to keep sensor data fresh"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000614,"raw_usage":{"total_tokens":2910,"prompt_tokens":1058,"completion_tokens":1852,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":674,"completion_tokens_details":{"reasoning_tokens":1769}},"tokens_in":674,"tokens_out":1852,"duration_ms":14822,"temperature":1.0,"reasoning_tokens":1769,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T15:59:43.437880+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"In a high-fidelity channel emulator or field trial, measure the actual covariance of the radar position measurements at the SNRs the UAV experiences, and re-run the same SAC policy and baselines with the empirical $R[n]$ substituted for Eq. (15). If the average-AoI ordering between the SAC controller and the baselines reverses, or if the policy's adaptive threshold ceases to track $\\mathrm{SNR}_{\\mathrm{th}}$, the CRLB-driven update rule is the part of the model that failed.","supporting_citations":[{"cited_title":"Next- generation multiple access for integrated sensing and communications,","cited_arxiv_id":null,"evidence_quote":"Supports the assumption that the sensing waveform is orthogonal to user despreading codes, so it contributes no interference to user SINR."},{"cited_title":"Levanon and E","cited_arxiv_id":null,"evidence_quote":"Supplies the radar range equation and coherent pulse integration model used to compute post-detection sensing SNR."},{"cited_title":"Puterman, Markov Decision Processes: Discrete Stochastic Dynamic Programming, ser","cited_arxiv_id":null,"evidence_quote":"Defines the Markov decision process formulation that turns the joint optimization into a sequential decision problem."},{"cited_title":"Soft actor-critic: Off- policy maximum entropy deep reinforcement learning with a stochastic actor,","cited_arxiv_id":null,"evidence_quote":"Supplies the Soft Actor-Critic algorithm used to train the continuous-action policy."},{"cited_title":"Asynchronous methods for deep reinforcement learning,","cited_arxiv_id":null,"evidence_quote":"Defines the on-policy A2C baseline against which the SAC controller is compared in the simulations."}],"review_version":1}