{"id":"bfa18e08-2d4c-4c57-8b8c-6cec101987a6","arxiv_id":"2505.17530","paper_version":2,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":8,"one_line_summary":"A GPS-aided CNN-GRU model predicts current and future optimal beams for UAV mmWave links, reporting top-1 accuracy above 70% and average power loss below 0.6 dB.","lead":"This paper trains a GPS-only deep learning model to predict the best current and future millimeter-wave beams for a drone connected to a base station, reporting over 70% top-1 accuracy and low power loss. A generalist reader should care because the method could cut beam search overhead in drone networks, though the evaluation protocol raises data leakage concerns.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Adjusted splitting can place the same drone trajectory in both train and test; the 70% / 0.6 dB claims are therefore not validated as generalization.","rationale":"The reader's weakest assumption identifies the same load-bearing concern. The paper's quantitative claims—70%+ Top-1 accuracy, sub-0.6 dB power loss, 93% overhead savings, and 95% beam accuracy guarantees—are all computed on the adjusted-splitting test set. If that test set is temporally contaminated, none of these claims is trustworthy. Algorithm 1 explicitly permits the same sequence index q to appear in multiple partitions, both through raw chunk splitting and through the final label-based regrouping. The method's reported advantage over sequential splitting is large (about 31–34 percentage points), which is consistent with leakage rather than with a benign balancing of label distributions. I do not see a separate fatal flaw in the architecture or preprocessing; the approach is a reasonable engineering extension of position-aided beam prediction. But the evaluation protocol is the single load-bearing pillar, and it is not secure. The concrete test above would settle the question: a sequence-disjoint re-evaluation either confirms the headline numbers or exposes them as artifacts. Since the reader's verdict of REJECT already reflects this concern, my stress-test does not change the verdict.","tokens_in":17413,"tokens_out":2909,"duration_ms":25970,"concrete_test":"Implement a sequence-disjoint variant of Algorithm 1: before any splitting, group samples by sequence index q, assign entire q groups to training/validation/test using the same 65/15/20 ratios, then retrain with the identical architecture and hyperparameters from Table I. Compare Top-1 accuracy and mean power loss against Table III. If current-beam Top-1 accuracy falls substantially (e.g., toward the 41.8% sequential-split level) or mean power loss rises above 0.6 dB, the adjusted-splitting results are largely leakage artifacts. Also report the fraction of sequence indices q present in both train and test under the original Algorithm 1; any nonzero overlap is direct evidence of temporal leakage.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—GPS-only simultaneous beam prediction and tracking with Top-1 accuracy above 70% and mean power loss below 0.6 dB—requires that the test set be a genuine held-out set. Algorithm 1 does not ensure this. In lines 11–15, raw chunks are split internally by ratios without any constraint on sequence index q, so a drone trip that straddles a chunk boundary can contribute samples to both train and test. More decisively, lines 22–25 regroup samples by beam label across the already-split pools and re-split each label group by ratios; this can freely interleave frames from the same q across train/validation/test because the only integrity requirement in Section III-B is that each constructed sample's input and output sequences share q and are consecutive. Since beam indices along one trajectory are strongly autocorrelated, the test set can contain near-duplicates of training GPS traces and beam sequences. The pre-split min-max normalization in equations (6)–(7) adds a smaller leakage channel by using test-set extrema. The 31–34 percentage-point gap between adjusted and sequential splitting in Figure 5 is exactly the signature of this problem: adjusted splitting interleaves correlated trajectory fragments, letting the model memorize trace pieces rather than learn a position-to-beam mapping. Therefore Table III's headline numbers do not establish generalization; they may only show interpolation within trips already seen during training.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes a GPS-aided deep learning model for simultaneous current and future beam prediction in UAV mmWave communication. The model uses a CNN-GRU encoder-decoder with a classifier to map a window of W=8 GPS-derived features (normalized UE coordinates and the UE-BS unit vector) to V+1=4 beam indices. The authors introduce an \"adjusted splitting\" protocol intended to balance label distributions across training, validation, and test sets, and report Top-1 accuracy above 70%, mean power loss below 0.6 dB, and about 93% training overhead savings on DeepSense6G Scenario 23. The paper also analyzes performance across UAV heights, speeds, and codebook sizes.","tokens_in":17713,"tokens_out":5660,"duration_ms":44280,"significance":"If the quantitative claims were valid, the work would offer a lightweight, GPS-only alternative to full codebook sweeping for UAV links, and the real-world evaluation on DeepSense6G would be a practical strength. The paper gives a detailed architecture, a reproducible description of training hyperparameters, and extensive diagnostic experiments by height, speed, codebook size, and overhead/reliability. However, the validity of all headline numbers rests on the data-splitting protocol and the normalization procedure, and the manuscript does not establish that the test set is a genuinely held-out set.","major_comments":[{"comment":"The adjusted splitting procedure does not constrain the sequence index q to a single partition. In the chunk loop, each chunk is split internally by ratios (lines 11–15), so a drone trip that crosses a chunk boundary can contribute frames to different partitions. In the label-based regrouping step (lines 22–25), samples for each beam label are drawn from all three already-populated partitions and split again by ratio, which can freely interleave frames from the same q across Dtrain, Dval, and Dtest. Since beam indices along a single trajectory are highly autocorrelated and adjacent frames are near-duplicates, the test set can contain GPS traces and beam sequences that are essentially copied from training. The integrity requirement in Section III-B only ensures that each constructed sample's input and output share the same q and are consecutive; it does not prevent this cross-partition leakage. Consequently, the \"Top-1 accuracy exceeding 70%\" and \"average power loss below 0.6 dB\" claims in the abstract and Table III are not validated as generalization results.","section":"III-A, Algorithm 1, lines 11–15 and 22–25"},{"comment":"Min-max normalization is applied to the entire raw data set before splitting, so the extrema used in equations (6) and (7) are computed over samples that later enter the test set. This gives the model access to test-set distribution information through the input feature g_UE,norm. To make the evaluation clean, the normalization parameters should be estimated from the training split only and then applied unchanged to the validation and test splits.","section":"II-C1, Eqs. (6)–(7), and III-A"},{"comment":"The reported 31–34 percentage point improvement of adjusted splitting over sequential splitting is confounded. Section III-A states that under sequential splitting the training set is empty for beam 0 while the test set contains many such samples, whereas the adjusted protocol guarantees balanced label proportions by construction. Therefore the gap shown in Figure 5 reflects not only the splitting strategy but also a change in label availability and a temporal-leakage artifact. The paper's claim that adjusted splitting \"enhances model performance\" requires a comparison in which both protocols preserve sequence-index integrity and both are evaluated on truly held-out trips.","section":"IV-A, Figure 5"}],"minor_comments":[{"comment":"The row labeled \"Dtest raw 2,209 (20%) 2,050 (19.36%)\" appears to report the development test set, not the raw test set; the label should likely be \"Dtest dev\" to be consistent with the preceding rows.","section":"Table II"},{"comment":"Equation (4) is incomplete: the sentence \"Using the definition of the optimal beam index provided in equation\" is followed directly by an unnumbered arg max expression with no operator or verb. The objective should be written as a complete mathematical statement.","section":"Eq. (4)"},{"comment":"The subsection numbering contains a duplicated hierarchy: \"a) 1) Min-max Normalization.\" One of the two markers should be removed.","section":"II-C1"},{"comment":"The symbol K is already used for the raw data set size in Section II-C1 and for the chunk/sample counts elsewhere; reusing it for the number of test samples in the power-loss formula creates ambiguity. Use a distinct symbol such as N_test.","section":"Eq. (17)"},{"comment":"The abstract says \"95% beam prediction accuracy guarantees,\" while Section IV-E and Figure 8(a) use the term \"Guaranteed Reliability.\" Align the terminology to avoid confusing accuracy with reliability/outage probability.","section":"Abstract and IV-E"},{"comment":"The numeric labels \"0.30 0.28 0.27 0.57\" are placed in a way that makes it unclear which bar they correspond to; adjust the labeling for unambiguous reading.","section":"Figure 5(b)"}],"recommendation":"major_revision","confidential_remarks":"This is a borderline case. The methodological flaw is fixable in principle by enforcing sequence-index integrity in the adjusted splitting and fitting normalization on the training split only, but it invalidates the central quantitative claims as submitted. If the authors' re-run cannot reproduce the headline numbers on a clean held-out test set, the paper should not be accepted. The editor may also wish to note that the manuscript states it has already been published in IEEE Access (DOI 10.1109/ACCESS.2025.3586594), which could affect the novelty assessment for this venue."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know two things about this one. The contribution is a reasonable engineering increment: a GPS-only encoder-decoder that predicts current and future beam indices simultaneously, with a sensible unit-vector plus normalized-position input. The evaluation, however, uses a splitting scheme that does not keep drone trips in a single partition, so the reported 70%+/0.6 dB results are likely inflated by temporal leakage.\n\nThe stress-test note holds up. Algorithm 1 splits raw data into chunks and then regroups by beam label without enforcing that sequence index q stays in one split. Since beams along a single trip are strongly autocorrelated, test samples can be near-duplicates of training traces from the same flight. The 31–34 point gap between adjusted and sequential splitting is exactly the signature of that problem. The pre-split min-max normalization is a smaller but real second leak, using test-set extrema.\n\nWhat is genuinely good: the feature preprocessing is carefully motivated; the CNN-GRU encoder-decoder is standard but appropriate; the ablations on input modalities and the height/speed analysis are useful; and comparing against the [29] baseline under the same (contaminated) splitting at least shows relative behavior, though the absolute numbers share the same flaw.\n\nThe central generalization claims are not validated as stated. Fixing this requires sequence-disjoint splits: all samples from one q in one partition, or at least a trip-level split as a sanity check. Releasing code would help a referee verify. The method itself is not circular, and the architecture is not the problem; the protocol is.\n\nWho is this for? Anyone working on sensing-aided beam prediction, as a cautionary example of trajectory leakage and as a starting point for a cleaner evaluation. It deserves a serious referee because the underlying idea is sensible and the flaw is fixable, but the referee should demand a re-analysis under trip-disjoint splits. I would not cite the numbers as-is.","headline":"Useful GPS-only beam prediction extension, but the adjusted-splitting protocol leaks trajectory-level temporal information into the test set, so the headline accuracy and power-loss numbers are not demonstrated.","tokens_in":18184,"tokens_out":2458,"would_cite":false,"duration_ms":23583,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"GPS-only deep learning predicts drone beams three steps ahead.","keywords":["millimeter-wave communication","beam prediction","beam tracking","UAV communications","GPS-aided deep learning","DeepSense 6G","GRU encoder-decoder","beam training overhead"],"falsifier":"Repeat the experiment holding out entire drone trips (each sequence index $q$ in only one partition) and compare Top-1 accuracy and mean power loss against the paper's numbers; if the trip-disjoint results fall well below 70% accuracy or push power loss above 0.6 dB, the central claim fails for truly unseen trajectories.","tokens_in":17233,"feed_emoji":"📡","tokens_out":6906,"duration_ms":51901,"temperature":0.7,"pith_summary":"This paper tries to show that a small neural network using only GPS position data can replace the expensive beam-sweeping step in millimeter-wave links between a drone and a base station. The model is asked to output both the current best beam and the best beams one, two, and three time steps ahead, at 60 GHz on a real-world drone dataset. The authors report Top-1 accuracy above 70% and average power loss below 0.6 dB at every prediction step, with overhead savings of about 93% because only 2–3 of 32 codebook beams need to be trained. If the claims hold, a drone's GPS receiver alone could provide lightweight beam management that keeps working when cameras or other sensors fail.","feed_headline":"Drone GPS data alone predicts mmWave beams at 70%+ accuracy","feed_subtitle":"A 0.99 MB model replaces 32-beam sweeping, halving power loss and cutting training overhead by about 93%.","key_machinery":"The load-bearing object is the combined input feature $O[t] = \\{g_{\\mathrm{UE,norm}}[t], u_{\\mathrm{UE-BS}}[t]\\}$: the drone's latitude and longitude min-max normalized to $[0,1]$, joined with the unit vector from the base station to the drone computed through an ECEF coordinate transformation. Sequences of $W=8$ such feature vectors feed a 1D convolutional feature extractor, then a GRU encoder-decoder whose final encoder hidden state seeds the decoder, and finally a two-layer classifier with softmax over the 32 codebook beams; the decoder emits $V+1=4$ beam indices, covering the current and three future steps. The adjusted splitting algorithm is a second mechanism: it splits raw data into chunks, scores candidate chunk sizes by label-distribution similarity, then further rebalances each label group to the 65/15/20 train/validation/test ratio, which the paper argues is what makes the GPS features learnable.","core_discovery":"The central discovery, stated on the paper's own terms, is that sequential GPS data—normalized drone latitude and longitude plus the unit vector pointing from the base station to the drone—carries enough information to predict the current optimal beam and three future beams simultaneously in a 60 GHz UAV scenario. This is achieved by a 1D-CNN plus GRU encoder-decoder that maps an 8-step position window to a sequence of beam indices from a 32-beam codebook. A data-splitting procedure called adjusted splitting, which rebalances labels so train, validation, and test sets mirror the original label distribution, is presented as a necessary ingredient: without it, accuracy drops by roughly 31–34 percentage points and mean power loss rises by 48–70%. The authors report that the combined position-plus-unit-vector input outperforms either feature alone, improving Top-1 accuracy by 8–14 percentage points and roughly halving power loss.","pith_inferences":["Because the adjusted splitting does not force samples from a single drone trip to stay in one partition, the reported accuracies may partly reflect the model recognizing specific trips; the natural test is a trip-disjoint split, which the paper does not report.","The unit-vector component expresses position relative to the base station, suggesting the model may transfer across base stations only after recalibrating that reference; absolute coordinates alone would not generalize.","If the GPS-only result survives a trip-disjoint test, then fusion with vision or radar should push accuracy higher where GPS is sparse, and the same architecture could extend to indoor drones using equivalent localization such as UWB or SLAM.","The preprocessing choice of dataset-wide min-max normalization means the model assumes the test set lies within the training coordinate range; deployment outside the trained area would need online normalization or recalibration."],"forward_implications":["A drone link could maintain its beam without sweeping the full 32-beam codebook, training only 2–3 beams per reliability target and saving roughly 93% of training overhead.","The same model covers both current-beam prediction and future-beam tracking, so no separate tracking network is required.","With an 8-step observation window and 3-step horizon, the system has roughly three time slots of lookahead, enough to react to the drone's motion if the position data arrives in time.","Performance degrades as codebook size grows (63–64% accuracy at 64 beams) and as speed increases, so the approach is best suited to moderate speeds and 32-beam or smaller codebooks.","Medium flight heights (40–80 m) are the hardest height range, so future tuning should target that regime before deployment."],"supporting_citations":[{"why":"Provides the DeepSense 6G Scenario 23 dataset (60 GHz drone link, GPS, and beam labels) that the model is trained and evaluated on.","marker":"[30]"},{"why":"Supplies the baseline beam-prediction/tracking models and the evaluation setup the paper compares against on the same scenario.","marker":"[29]"},{"why":"Introduces GPS-aided beam prediction with the power-loss and overhead-savings metrics this paper adopts.","marker":"[20]"},{"why":"Defines the beam-tracking task and problem formulation adopted for the DL task.","marker":"[24]"},{"why":"Defines future-beam prediction with sensor-aided inputs, the formulation adapted here.","marker":"[21]"},{"why":"Provides the empirical comparison that motivates choosing gated recurrent units (GRUs) for the encoder-decoder.","marker":"[36]"},{"why":"Gives the geodetic-to-ECEF coordinate transformation used to compute the UE-BS unit vector.","marker":"[32]"},{"why":"References and extends the ECEF conversion used in the GPS preprocessing pipeline.","marker":"[33]"}],"fun_headline_variants":["GPS data predicts drone beams at 70%+ accuracy","GPS-aided AI cuts beam sweeping by 93% for UAV mmWave","Drone GPS alone predicts mmWave beams: 70%+ accuracy","GPS + deep learning forecasts drone beams, slashes overhead 93%","UAV beam tracking from GPS: 70% accuracy, 93% less training"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the adjusted splitting yields a test set that fairly represents unseen drone flights rather than fragments of flights that already appeared in training.","fun_headline_variants_meta":{"raw":{"variants":["GPS data predicts drone beams at 70%+ accuracy","GPS-aided AI cuts beam sweeping by 93% for UAV mmWave","Drone GPS alone predicts mmWave beams: 70%+ accuracy","GPS + deep learning forecasts drone beams, slashes overhead 93%","UAV beam tracking from GPS: 70% accuracy, 93% less training"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000865,"raw_usage":{"total_tokens":3740,"prompt_tokens":926,"completion_tokens":2814,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":542,"completion_tokens_details":{"reasoning_tokens":2717}},"tokens_in":542,"tokens_out":2814,"duration_ms":15152,"temperature":1.0,"reasoning_tokens":2717,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T14:44:57.097200+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Repeat the experiment holding out entire drone trips (each sequence index $q$ in only one partition) and compare Top-1 accuracy and mean power loss against the paper's numbers; if the trip-disjoint results fall well below 70% accuracy or push power loss above 0.6 dB, the central claim fails for truly unseen trajectories.","supporting_citations":[{"cited_title":"DeepSense 6G: A Large-Scale Real- World Multi-Modal Sensing and Communication Dataset,","cited_arxiv_id":null,"evidence_quote":"Provides the DeepSense 6G Scenario 23 dataset (60 GHz drone link, GPS, and beam labels) that the model is trained and evaluated on."},{"cited_title":"Sensing-Aided 6G Drone Communications: Real-World Datasets and Demonstration,","cited_arxiv_id":null,"evidence_quote":"Supplies the baseline beam-prediction/tracking models and the evaluation setup the paper compares against on the same scenario."},{"cited_title":"Position-Aided Beam Prediction in the Real World: How Useful GPS Locations Actually are?","cited_arxiv_id":null,"evidence_quote":"Introduces GPS-aided beam prediction with the power-loss and overhead-savings metrics this paper adopts."},{"cited_title":"Computer Vision Aided Beam Tracking in A Real-World Millimeter Wave Deployment,","cited_arxiv_id":null,"evidence_quote":"Defines the beam-tracking task and problem formulation adopted for the DL task."},{"cited_title":"LiDAR Aided Future Beam Prediction in Real-World Millimeter Wave V2I Communications,","cited_arxiv_id":null,"evidence_quote":"Defines future-beam prediction with sensor-aided inputs, the formulation adapted here."},{"cited_title":"Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling,","cited_arxiv_id":null,"evidence_quote":"Provides the empirical comparison that motivates choosing gated recurrent units (GRUs) for the encoder-decoder."},{"cited_title":"A note on computation of Geodetic coordinates from geocentric (Cartesian) coordinates,","cited_arxiv_id":null,"evidence_quote":"Gives the geodetic-to-ECEF coordinate transformation used to compute the UE-BS unit vector."},{"cited_title":"Conversion of Earth-centered Earth-fixed coordinates to geode- tic coordinates,","cited_arxiv_id":null,"evidence_quote":"References and extends the ECEF conversion used in the GPS preprocessing pipeline."}],"review_version":1}