{"id":"274d1780-1724-4d0b-bae1-68e0a7481099","arxiv_id":"2507.15832","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":8,"one_line_summary":"A hybrid CNN-LSTM-attention-Adaboost model tuned by a multi-strategy improved snake optimizer claims to cut trajectory prediction MAPE to 1.35%, but the result rests on a single route, no separate test set, and a mislabeled 39.89% improvement metric.","lead":"This paper combines an improved snake optimizer with a CNN-LSTM-attention-Adaboost network to predict aircraft 4D trajectories from ADS-B data. It reports large error reductions on one China route, but the evaluation has serious methodological gaps and the headline improvement number is a loss reduction, not an accuracy gain.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Headline results are computed on the same validation set used for hyperparameter tuning; a held-out test is needed before the claimed RMSE/MAPE and 39.89% improvement can be accepted.","rationale":"I agree with the reader's weakest assumption: the evaluation protocol is the load-bearing weakness. The paper never separates the set used for hyperparameter optimization from the set used for reporting final accuracy, and Section 4.1 explicitly identifies the output as validation-set predictions. This alone is enough to undermine the headline numbers, because the improved snake optimizer is being rewarded for fitting the very set on which it is later judged. The 39.89% claim is additionally mislabeled as an accuracy improvement when Table 7 shows it is a training-loss reduction, but that point is secondary to the missing held-out test. The CEC2022 benchmark results and Wilcoxon tests are standard practice and count as some independent evidence for the optimizer on synthetic functions, but they do not validate the trajectory-prediction claim. A single held-out re-evaluation, as described in the concrete test, would settle whether the reported ranking is real or an artifact of tuning-set overfitting. Since the reader's reject verdict is supported by this concern, I do not see a basis for changing it.","tokens_in":19377,"tokens_out":5410,"duration_ms":57678,"concrete_test":"Take the 20,526 ADS-B records and, before any optimization, carve out a temporally contiguous held-out test set (e.g., the last 20% of each sampled flight or a disjoint flight). Re-run the same nine optimizer-tuned CNN-LSTM-attention-adaboost configurations with identical search ranges and loss functions, and compute all Table 6 metrics on the held-out set only. If SO-CLA-adaboost still gives RMSE near 125.4 and ranks first by comparable margins, the central claim survives; if errors inflate materially or the ranking shifts, the current results are tuning-set artifacts.","verdict_should_be":"REJECT","load_bearing_attack":"The central quantitative claim (RMSE 125.4342, MAPE 1.3504%, 39.89% improvement) rests on an evaluation protocol in which the validation set used for hyperparameter search is also the set on which final accuracy is reported. Section 4.1 states that the model output is the validation set's true values; Section 4.2 tunes batch size, learning rate, and neuron count on that same split; Tables 6 and 8 then report RMSE/MAPE/MAE/MAXAE/R2 on it. No independent test split or cross-validation is described. This is not a cosmetic issue: with a flexible hybrid network and an optimizer that directly minimizes loss on the evaluation set, the reported margin over eight alternatives can reflect selective overfitting to the tuning set rather than superior generalization. In addition, the abstract's 39.89% figure is, according to Table 7, a relative reduction in training loss (0.000905 to 0.000544), not an accuracy improvement, so the headline claim is not even backed by the metric it cites. A held-out evaluation is therefore load-bearing; without it, the reported numbers cannot be distinguished from tuning-set artifacts.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes a multi-strategy improved Snake Optimizer (SO) for tuning the hyperparameters of a CNN-LSTM-attention-Adaboost hybrid network for four-dimensional aircraft trajectory prediction from ADS-B data. The authors describe several SO modifications (good-point-set initialization, adaptive thresholds, dual mutation, and Levy/random-walk flight), benchmark the improved optimizer against eight other metaheuristics on CEC2022 functions, and then compare SO-CLA-adaboost against variants tuned by the other optimizers on a Tianjin-Xi'an ADS-B dataset. The headline results are RMSE 125.4342, MAPE 1.3504%, MAE 51.9559, MAXAE 609.4213, and R² 0.9978, with an advertised 39.89% improvement attributable to the SO module.","tokens_in":19690,"tokens_out":7453,"duration_ms":74344,"significance":"If the reported results had been obtained on a properly held-out test set, the paper would be a useful demonstration of metaheuristic hyperparameter tuning for a hybrid spatio-temporal network in trajectory prediction. The CEC2022 benchmark comparison, including Wilcoxon rank-sum tests, and the ablation study are constructive components. However, the central quantitative claims are undermined by an evaluation protocol that uses the validation set for both hyperparameter optimization and final performance reporting, by a headline metric that is a loss reduction mislabeled as accuracy, and by the absence of any repeated-run or held-out evaluation. These issues make the claimed performance gains unsubstantiated and not yet credible.","major_comments":[{"comment":"Section 4.1 states that the model output is the validation set's true values, and Section 4.2 tunes batch size, learning rate, and LSTM neuron count on that same split. Tables 6 and 8 then report RMSE, MAPE, MAE, MAXAE, and R² on this set. Because the hyperparameters are chosen by minimizing the loss on the very set used for final evaluation, the reported errors and the ranking among optimizers reflect tuning-set fit rather than generalization. No independent test split or cross-validation is described. This is a load-bearing flaw for the abstract's headline RMSE, MAPE, and R² values and for the claimed margin over the eight other optimizers; the reported numbers cannot be distinguished from tuning-set artifacts.","section":"§4.1–4.2, Tables 6 and 8"},{"comment":"The abstract's claim that 'introducing the full-strategy collaborative improvement SO algorithm improves the model's prediction accuracy by 39.89%' is not supported by the metric cited. According to Table 7, the 39.89% figure is the relative decrease in training loss from CNN-LSTM-attention-adaboost (0.000905) to SO-CNN-LSTM-attention-adaboost (0.000544), not an improvement in prediction accuracy. In addition, Table 8 mixes formats for MAPE: the LSTM, CNN-LSTM, CNN-LSTM-attention, and CLA-adaboost rows report 0.0536, 0.0478, 0.0451, and 0.0344, while the SO row reports 1.3504%, so the apparent improvement in MAPE across ablation stages is computed on incompatible scales.","section":"Abstract and Table 7"},{"comment":"All trajectory-prediction results are single runs. The paper reports no standard deviations, confidence intervals, or significance tests for the RMSE/MAPE comparisons across optimizers or across ablation stages. Given the stochasticity of neural-network training and of metaheuristic optimization, the observed differences (for example, SO RMSE of 125.4 versus WOA RMSE of 155.7) may fall within run-to-run variability. Without multiple independent runs, the claim that SO statistically outperforms the other optimizers is not established.","section":"Tables 5, 6, and 8"},{"comment":"The paper never compares the proposed multi-strategy improved SO against the original Snake Optimizer. Section 2.2 introduces good-point-set initialization, adaptive thresholds, dual mutation, and Levy/random-walk flight, but Table 2 in Section 2.3 only compares 'SO' with WOA, GWO, BSO, PSO, DE, GA, BBO, and GCO; Section 4.2 compares SO-CLA-adaboost only with other optimizer-tuned CLA-adaboost variants. Consequently, the individual and joint contributions of the proposed improvements to SO cannot be isolated. Furthermore, Section 2.3.1 states that 12 CEC2022 functions were used, while Table 1 and Table 2 list only 10 functions (F1–F10), and the Table 2 caption contains an unexplained 'PLORCS' label.","section":"§2.2–2.3 and §4.2"}],"minor_comments":[{"comment":"The text contains placeholder words '表几' and '图几' which should be replaced with the actual table and figure numbers.","section":"§4.2"},{"comment":"The logistic chaotic map X(1−X) is applied to an arbitrary position vector without specifying how variables outside the [0,1] interval are handled; the authors should either restrict the mutation to normalized coordinates or justify the operation.","section":"Eq. (32)"},{"comment":"There are spelling and caption inconsistencies, such as 'Atention' for 'Attention' and missing spaces in figure captions (e.g., 'Figure4 CNN-LSTMattentionadaboosthybridmodelframework').","section":"Throughout"},{"comment":"Reference [28] is cited as a CNN background reference, but the cited article is on vehicle trajectory extraction with convolutional neural networks; the connection should be clarified or a more appropriate CNN reference should be used.","section":"References"}],"recommendation":"reject","confidential_remarks":"The primary reported performance numbers are artifacts of tuning-set evaluation, not a presentation issue. The manuscript would need entirely new experiments with a properly held-out test set and multiple runs before its central claims could be assessed. I would not consider the current version publishable, though a resubmission with such experiments might be worth evaluating."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Hi [Name],\n\nQuick take on arXiv:2507.15832. It's a mashup of known building blocks—improved snake optimizer, CNN-LSTM-attention, Adaboost—applied to ADS-B trajectory prediction. The specific combination seems new relative to the cited literature, and the authors do put in real work benchmarking their optimizer on CEC2022 functions against eight metaheuristics, with Wilcoxon tests. That part is legitimate.\n\nThe problem is that the main performance claim is not supported. The abstract's 39.89% 'accuracy improvement' is, by the paper's own Table 7, a reduction in training loss (0.000905 to 0.000544), not an accuracy gain. More seriously, the evaluation protocol uses the validation set both for hyperparameter tuning and for reporting final error. Section 4.1 says the model outputs the validation set's true values; Section 4.2 tunes batch size, learning rate, and neurons on that same split; Tables 6 and 8 report RMSE, MAPE, MAE, MAXAE, R2 on it. There is no held-out test set anywhere. With a flexible network and an optimizer that directly minimizes loss on the evaluation set, the reported margins over eight alternatives are exactly the signature of tuning-set overfitting.\n\nAlso missing: the original (unimproved) snake optimizer is never compared on the trajectory task, so the contribution of the multi-strategy improvements is never isolated. Single-run results, no error bars. The manuscript still has placeholders like '表几' and '如图几', the test-function count is inconsistent (claims 12, lists 10), and the MAPE notation flips between decimals and percentages. None of these are fatal alone; together with the test-set problem, they mean the paper doesn't substantiate its claims.\n\nCredit where due: the arithmetic in Section 4.2 matches Table 6, the ablation shows a monotonic loss decrease as modules are added, and the CEC benchmark is standard practice. The algorithm section is readable and reasonably detailed.\n\nBottom line: this is not ready for publication. The missing independent test set and the metric mislabel are load-bearing. If the authors add a proper held-out split, report multiple runs, include the original SO as a baseline, and clean up the draft, it could become a useful applied paper. As-is, I'd desk reject; a referee would just send it back for the same reasons.\n\nBest,\n[You]","headline":"A competent assembly of known optimizer tricks and hybrid networks, but the evaluation uses the tuning set as the test set and the headline 39.89% 'accuracy' is actually a training loss reduction—so the central claims do not hold.","tokens_in":20139,"tokens_out":6993,"would_cite":false,"duration_ms":68770,"reading_group":"no","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Tuned snake optimizer predicts flight trajectories with 0.9978 R2","keywords":["four-dimensional trajectory prediction","snake optimizer","CNN-LSTM-attention","AdaBoost ensemble","hyperparameter optimization","ADS-B data","metaheuristic tuning"],"falsifier":"Run the identical pipeline with a strictly held-out test set — for example, a different flight or a different day on the same Tianjin–Xi'an route — and recompute RMSE and MAPE. If SO-CLA-adaboost's advantage over the best competing optimizer narrows below the reported 19–57% margins, or if the 39.89% ablation gain disappears, the central claim fails.","tokens_in":19206,"feed_emoji":"✈️","tokens_out":10871,"duration_ms":91602,"temperature":0.7,"pith_summary":"This paper claims that a hybrid predictor — CNN for spatial features, LSTM for temporal dynamics, attention for global context, and AdaBoost to combine weak learners — becomes more accurate when its hyperparameters are tuned by a multi-strategy improved Snake Optimizer (SO). On a real ADS-B trajectory dataset from Tianjin to Xi'an, the tuned model reaches RMSE 125.43, MAPE 1.35%, and $R^2=0.9978$, beating eight other optimizer-tuned variants including particle swarm, whale, and gray wolf optimizers. The paper also reports an ablation study attributing a 39.89% drop in training loss to the addition of the improved SO module. If these results hold, metaheuristic hyperparameter tuning would be a strong lever for spatio-temporal trajectory prediction.","feed_headline":"Tuned snake optimizer predicts flight trajectories with 0.9978 R2","feed_subtitle":"Hybrid CNN-LSTM-attention-AdaBoost tuned by the improved snake optimizer beats eight alternatives on ADS-B data.","key_machinery":"The load-bearing mechanism is the multi-strategy improved Snake Optimizer, which keeps the original SO's male/female sub-population exploration–exploitation phases and adds four enhancements: good-point-set initialization built on number-theoretic uniform distribution; periodic cosine/sine adaptive updates for the C1, C3, food-threshold, and temperature-threshold parameters; a dual mutation scheme (Cauchy then Gaussian as primary, plus head-chaotic, body-fusion, and tail-splicing mutations as auxiliary); and an adaptive flight function that shifts from Levy flight to random walk as iterations proceed. These mechanisms govern how the optimizer searches for the three hyperparameters — batch size, learning rate, and neuron count — that configure the CNN-LSTM-attention-Adaboost ensemble. The improved SO is what the paper credits for the reported accuracy gains.","core_discovery":"The central claim is that SO-CLA-adaboost, the proposed combination, outperforms all eight compared optimizer-tuned variants on every reported metric: RMSE 125.4342 versus the next-best 155.7012 (WOA), MAPE 1.3504% versus 1.9262%, MAE 51.9559 versus 64.0093, MAXAE 609.4213 versus 1274.5740, and $R^2=0.9978$ versus 0.9962. The paper further claims that the full multi-strategy improvement contributes a 39.89% loss reduction over the non-optimized CLA-adaboost, and that the improved SO stabilizes faster (16 iterations) than the comparison optimizers. On the CEC2022 benchmark suite, the improved SO ranks first on ten of the twelve tested functions.","pith_inferences":["A consequence the paper leaves implicit: because the evaluation uses the validation set for both tuning and reporting, the true generalization gap is unknown; a held-out test across different flights, dates, or weather conditions would be the natural next check.","The improved SO's components (good-point-set initialization, adaptive thresholds, Cauchy–Gaussian and head/body/tail mutations, Levy-and-random-walk flight) are generic and could transfer to other hybrid architectures, though the paper only demonstrates them on this one dataset and task.","The MAXAE spikes at altitude changes suggest that the model class itself, not just the optimizer, may need a mechanism for detecting regime shifts such as climb/descent transitions to close the remaining error.","A simple experiment could isolate whether the 39.89% ablation gain comes from the optimizer's search or from the particular hyperparameters it lands on, by comparing against random search with the same budget."],"forward_implications":["If the reported numbers hold, metaheuristic hyperparameter tuning of hybrid spatio-temporal networks becomes a practical route to high-accuracy 4D trajectory prediction.","The 39.89% loss reduction in the ablation implies that the multi-strategy improvement to SO, rather than the ensemble architecture alone, is the decisive factor in the reported accuracy.","The convergence results imply SO needs fewer iterations (16) than PSO (27), WOA (38), and GWO (43), which matters for near-real-time hyperparameter tuning.","The MAXAE figures indicate that even the best model still lags at rapid altitude changes, so the reported gains apply mainly to normal flight phases rather than abrupt maneuvers."],"supporting_citations":[{"why":"Supplies the base Snake Optimizer algorithm that this paper modifies with multi-strategy improvements.","marker":"[15]"},{"why":"Supplies the good-point-set initialization method used to seed the SO population more uniformly.","marker":"[16]"},{"why":"Supplies the enhanced snake optimizer with Cauchy mutation that informs the primary mutation scheme.","marker":"[22]"},{"why":"Supplies the Levy-flight mechanism adapted for early global exploration in the improved SO.","marker":"[23]"},{"why":"Supplies the random-walk flight mechanism used for late-stage local refinement.","marker":"[24]"},{"why":"Supplies the AdaBoost ensemble framework that combines the weak CNN-LSTM-attention learners.","marker":"[31]"},{"why":"Supplies the Attention-LSTM architecture that each weak learner is built on.","marker":"[13]"},{"why":"Supplies the basis for the ADS-B trajectory data used for the experiments.","marker":"[32]"}],"fun_headline_variants":["Snake optimizer boosts flight path predictions by 39.89%","Improved snake optimizer lifts trajectory prediction accuracy","Hybrid AI with snake optimizer nails flight paths at 0.9978 R2","Snake-optimized CNN-LSTM beats 8 rivals on ADS-B trajectories","Multi-strategy snake optimizer cuts trajectory error by 39.89%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported accuracy is measured on the same validation set that the optimizer used to choose hyperparameters, so there is no independent test set confirming the numbers generalize.","fun_headline_variants_meta":{"raw":{"variants":["Snake optimizer boosts flight path predictions by 39.89%","Improved snake optimizer lifts trajectory prediction accuracy","Hybrid AI with snake optimizer nails flight paths at 0.9978 R2","Snake-optimized CNN-LSTM beats 8 rivals on ADS-B trajectories","Multi-strategy snake optimizer cuts trajectory error by 39.89%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000476,"raw_usage":{"total_tokens":2357,"prompt_tokens":935,"completion_tokens":1422,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":551,"completion_tokens_details":{"reasoning_tokens":1329}},"tokens_in":551,"tokens_out":1422,"duration_ms":9582,"temperature":1.0,"reasoning_tokens":1329,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T15:23:14.901940+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the identical pipeline with a strictly held-out test set — for example, a different flight or a different day on the same Tianjin–Xi'an route — and recompute RMSE and MAPE. If SO-CLA-adaboost's advantage over the best competing optimizer narrows below the reported 19–57% margins, or if the 39.89% ablation gain disappears, the central claim fails.","supporting_citations":[],"review_version":1}