{"id":"cc414b0e-8a18-461b-9324-62ed7c86aa32","arxiv_id":"2412.13356","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"A hybrid WPD-SAM-BiLSTM model reports the lowest wind-speed forecast errors among 27 benchmarks on five years of hourly data from the Dumat Al-Jandal wind farm.","lead":"This paper combines wavelet packet decomposition, seasonal adjustment, and a bidirectional LSTM to forecast hourly wind speed at a Saudi wind farm, reporting lower errors than 27 comparison models. It is a case study in applied short-term forecasting, relevant to grid operators planning wind power integration in the region.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The reported test errors are suspect because §3.2 decomposes the test signal as a block, allowing future test values to leak into WPD sub-series and SAM seasonal indices; the claimed superiority over 27 benchmarks is not yet established.","rationale":"The reader's weakest assumption is exactly the load-bearing issue: the decomposition/SAM pipeline is not shown to be causal. My reading of §3.2 confirms the ambiguity: step (2) decomposes 'training and test signals' and step (3) calculates seasonal indices without specifying that only past data are used. Because WPD is a block transform with filters of length 44 at three levels, any non-causal implementation gives each reconstructed test point access to future test observations; SAM seasonal indices estimated on the full test block add another leakage path. The paper's own §2.1 identifies this protocol as the classic decomposition data-leakage error, so the burden is on the authors to demonstrate they avoided it. Hyperparameter selection on the test set (§4.1, Table 2) is a second independent threat to the reported numbers, but the decomposition leakage is more fundamental because it can invalidate every benchmark comparison, not just the final ranking. The proposed concrete test is feasible: causal implementations of WPD are standard, and the comparison metric is well defined. If the causal rerun reproduces the reported accuracy, the concern is resolved; absent that, the central claim is unsupported. I therefore keep the reader's REJECT verdict unchanged, while noting that a clean causal re-analysis could make the paper publishable in revised form.","tokens_in":16654,"tokens_out":6086,"duration_ms":56385,"concrete_test":"Re-run the 1-hour-ahead experiment with a strictly causal decomposition: for each forecast origin t, update the WPD coefficients using only observations up to t (or decompose the training set once and append new points one at a time), estimate SAM seasonal indices only from the training portion, and retrain BiLSTM with the same Table 2 hyperparameters. Compare MAE/RMSE/R2 with Table 5's WPD-SAM-BiLSTM row (0.116901/0.152297/0.995279). If the metrics degrade materially or the model no longer ranks first among the 27 models, the reported superiority is an artifact of look-ahead leakage.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim (abstract; §4.2.2) is that WPD-SAM-BiLSTM achieves MAE 0.116901 / RMSE 0.152297 / R2 0.995279 at 1-hour ahead and beats 27 benchmarks. That claim requires every test forecast to use only information available at the forecast origin. Section 3.2(2) says WPD is applied to 'the training and test signals' as blocks, and §3.2(3) says seasonal indices are calculated from the reconstructed sub-series. A level-3 db22 WPD reconstruction at time t uses observations up to roughly t+150 h (effective support ~(2^3-1)*(44-1)+1) when the test block is decomposed non-causally. Likewise, SAM seasonal indices computed on the full test block encode future values. This is exactly the data-leakage protocol the authors themselves warn against in §2.1, citing Quilty and Adamowski [39]. The paper does not specify a causal/online implementation or a validation-set split for the hyperparameters chosen in §4.1 (window size, WPD level, seasonal period 2192 h), so the reported numbers cannot be read as independent out-of-sample errors. If future test values enter the decomposition, the error metrics are not genuine forecasts and the comparison with the 27 benchmark models is unfair.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes a hybrid wind speed forecasting framework, WPD-SAM-BiLSTM, which combines Wavelet Packet Decomposition, Seasonal Adjustment Method, and Bidirectional LSTM, and applies it to five years of hourly wind speed data from the Dumat Al-Jandal wind farm in Al-Jouf, Saudi Arabia. The authors report that the proposed model outperforms 27 benchmark models across 1-, 3-, and 5-hour horizons, with headline one-hour-ahead metrics MAE 0.116901, RMSE 0.152297, and R2 0.995279. The paper also investigates the effect of data decomposition on stationary wind speed data, a topic it identifies as understudied. The central claim is the empirical superiority of the proposed model on this specific dataset.","tokens_in":16896,"tokens_out":3980,"duration_ms":37646,"significance":"If the reported results were obtained under a valid out-of-sample protocol, the paper would offer a useful case study for wind speed forecasting in Saudi Arabia and a substantive comparison of decomposition-based hybrids on stationary data. The manuscript has some strengths: it uses a public data source, applies an ADF stationarity test, includes a broad set of 27 benchmark models, and explicitly acknowledges in Section 2.1 the data-leakage risk of decomposing the full time series before splitting. However, the evaluation protocol as described does not implement the safeguards the authors themselves cite, and the reported accuracy numbers are therefore not trustworthy as out-of-sample evidence. The methodological issues are load-bearing for the central claim of superiority, so the contribution is not currently established.","major_comments":[{"comment":"The decomposition protocol appears to be non-causal and leaks future information into the test forecasts. Step (2) states that WPD is applied to 'the training and test signals' as blocks, and step (3) states that seasonal indices are calculated from the reconstructed subseries. A level-3 db22 WPD reconstruction at a forecast origin t uses observations well beyond t when the test block is transformed as a whole, and seasonal indices computed over the full test block encode future values. This is exactly the 'decompose-then-split' data-leakage problem the authors warn against in Section 2.1 with reference to Quilty and Adamowski [39]. The paper does not specify a causal or online implementation of WPD or SAM, so the reported MAE, RMSE, and R2 values cannot be interpreted as genuine out-of-sample forecast errors.","section":"Section 3.2, steps (2)-(3)"},{"comment":"The headline hyperparameters are selected on the test data, making the reported performance partly circular. Section 4.1 states that 'the proposed model was tested over different forecasting windows, WPD levels, and seasonal periods' and that the configuration 'produced the best forecasting accuracy for the proposed model; therefore, this study selected it as the proposed model's optimal configuration.' No separate validation set or nested cross-validation is described, and Table 2 lists Bayesian optimization for several BiLSTM hyperparameters without indicating which data were used for selection. Consequently, the numbers in Table 5 and the abstract are the result of test-set tuning, not an independent evaluation. This undermines the comparison with the 27 benchmark models, since the proposed model was given the advantage of test-set-based configuration search.","section":"Section 4.1 and Table 2"},{"comment":"The claim of superiority over 27 benchmarks is not supported by any uncertainty quantification. The paper reports single-run point estimates for MAE, RMSE, and R2 for each model, but no error bars, no repeated runs with different random seeds, and no statistical significance tests. Given that deep learning training is stochastic and that several benchmark differences in Table 5 are small (e.g., SWT-LSTM vs. SWT-BiLSTM at one hour), the reported ranking may reflect noise rather than systematic improvement. The authors should provide confidence intervals or significance tests, especially because the headline improvements are used to justify the practical recommendation in Section 4.2.2.","section":"Section 4.1 and Section 4.2"}],"minor_comments":[{"comment":"The text says 'Compared with the best resulting models from Table 3,' but the referenced table containing the deep learning and WPD/SAM results is Table 4; Table 3 is the ADF test results.","section":"Section 4.1.1"},{"comment":"The row labeled 'Trial/Test epochs' appears to contain a typo; it likely should read 'Trial' or 'Training epochs.'","section":"Table 2"},{"comment":"In the CEEMDAN-LSTM row, the 5-hour MAE and RMSE are both listed as 0.738797; given RMSE is generally larger than MAE for non-constant errors, this is likely a transcription error and should be corrected.","section":"Table 5"},{"comment":"The paper does not specify how multi-step (3-hour and 5-hour) forecasts are generated, i.e., whether a direct strategy or a recursive strategy is used; this information is needed for reproducibility.","section":"Section 3.2"},{"comment":"No code or reproducible implementation details are provided, and there is no data availability statement beyond citing the KAPSARC portal; sharing code would strengthen the reproducibility of the experiments.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The two principal issues—non-causal decomposition and test-set-based hyperparameter selection—require a full re-execution of the experiments under a proper causal, validation-based protocol. If the authors cannot provide such results, the paper should not be accepted. The current manuscript appears to be a published conference/journal version; the refereeing should focus on whether the arXiv version meets the standards of the target venue."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe thing to know about this paper: it claims near-perfect wind speed forecasts (MAE 0.117, R2 0.995 at one hour ahead) but the evaluation protocol is too sloppy to support that claim. The authors decompose the test signal as a block and pick hyperparameters on the test set, which is exactly the kind of data leakage they warn against in their own literature review. Treat the headline numbers as upper bounds, not trustworthy out-of-sample results.\n\nWhat's actually new: the specific cascade WPD-SAM-BiLSTM is a new combination, and the Al-Jouf dataset hasn't been used this way before. The paper also delivers a reasonably broad benchmark comparison—27 models, three horizons—and writes the whole thing clearly. The stationarity angle is a nice hook: most decomposition studies assume non-stationary wind, and they check with ADF.\n\nThe soft spots are in the evaluation. Section 3.2(2) says WPD is applied to \"the training and test signals\" without saying whether the transform is causal. A level-3 db22 wavelet has effective support reaching ~150 hours ahead, so a block decomposition of the test set can leak future information into each sub-series. The same goes for SAM seasonal indices computed on the full test block. The paper says they use walk-forward, but the protocol isn't described consistently enough to rule out leakage. Second, Section 4.1 picks window size, WPD level, and seasonal period by \"best forecasting accuracy\" on the test set. That's tuning on the evaluation data, so the comparison against 27 benchmarks is partly circular. No error bars, no significance tests, no code or data to check.\n\nI don't think the authors are being dishonest. They cite Quilty and Adamowski and explicitly warn about this leakage trap. But the paper as written does not demonstrate they avoided it. The fix is clear: decompose causally (or describe the online implementation), use a validation split for hyperparameters, and report uncertainty. If they can do that, the case study has real value for the Dumat Al-Jandal operators and for the decomposition literature. As it stands, though, the claim of superiority over 27 benchmarks is not established.\n\nFor a journal, I'd send this to peer review rather than desk reject—the method is a legitimate new combination and the flaws are fixable. But I'd ask for the re-analysis before any acceptance. For your own reading group, it's a useful example of how decomposition-based forecasting can go wrong; otherwise it's not something I'd rush to cite.","headline":"The claimed accuracy is likely inflated by test-set hyperparameter selection and a non-causal decomposition protocol; the paper's central superiority claim is not established, though the method and case study are worth a careful re-analysis.","tokens_in":17481,"tokens_out":2766,"would_cite":false,"duration_ms":26180,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"WPD-SAM-BiLSTM, a two-stage decomposition plus bidirectional LSTM, outperforms 27 benchmarks on Al-Jouf wind speed forecasting.","keywords":["wind speed forecasting","data decomposition","wavelet packet decomposition","seasonal adjustment method","bidirectional LSTM","stationary time series","Saudi Arabia","short-term forecasting"],"falsifier":"Rerun the one-hour-ahead experiment with a strictly causal implementation: at each forecast origin, recompute the wavelet packet transform and the seasonal indices using only observations up to that origin, then measure MAE and R2 on the same test period. If the causal version's one-hour MAE is substantially larger than 0.116901, the paper's headline accuracy depends on look-ahead leakage rather than on the WPD-SAM-BiLSTM pipeline itself.","tokens_in":16406,"feed_emoji":"💨","tokens_out":7677,"duration_ms":67521,"temperature":0.7,"pith_summary":"This paper proposes a wind speed forecasting pipeline that first decomposes hourly wind speed into frequency sub-bands with wavelet packet decomposition, then removes seasonal patterns from each sub-band, and finally forecasts each cleaned sub-series with a bidirectional LSTM before recombining the results. The central claim is that this WPD-SAM-BiLSTM model beats all 27 benchmark models on five years of hourly data from the Dumat Al-Jandal wind farm in Al-Jouf, Saudi Arabia, at one-, three-, and five-hour horizons. The reported best accuracy is a mean absolute error of 0.116901, a root mean square error of 0.152297, and an R-squared of 0.995279 for one-hour-ahead forecasts. The paper also claims that decomposition methods improve forecasting even when the wind series is stationary, a setting the authors say has received little attention. Accurate short-term wind speed forecasts matter because they help grid operators dispatch power, maintain stability, and integrate wind energy more safely.","feed_headline":"Hybrid pipeline tops 27 models on Saudi wind speed forecasts","feed_subtitle":"Two-stage decomposition plus BiLSTM reaches 0.995 R2 on Al-Jouf wind data.","key_machinery":"The load-bearing mechanism is the WPD-SAM-BiLSTM cascade. Wavelet packet decomposition (WPD) splits the hourly wind speed series into several frequency sub-bands, reconstructing each sub-series separately. Seasonal adjustment (SAM) then removes the seasonal component from each reconstructed sub-series and stores the seasonal indices. A single-hidden-layer bidirectional LSTM, a recurrent network that reads sequences in both time directions, forecasts each deseasonalized sub-series; the seasonal indices are added back, and all forecast sub-series are summed to produce the final wind speed forecast. The paper uses the Augmented Dickey-Fuller test to classify the data as stationary and a walk-forward training scheme so the model can adapt as new observations arrive.","core_discovery":"The paper's discovery claim is that a two-stage decomposition, wavelet packet decomposition followed by seasonal adjustment, makes a bidirectional LSTM a high-accuracy short-term wind speed forecaster for the Al-Jouf site. The authors report that WPD-SAM-BiLSTM yields MAE 0.116901, RMSE 0.152297, and R2 0.995279 at one hour ahead; MAE 0.155379, RMSE 0.223525, and R2 0.989834 at three hours; and MAE 0.257366, RMSE 0.365386, and R2 0.972849 at five hours. These are the best results among the 27 benchmark models compared in the study, and the authors further claim that applying data decomposition to stationary wind speed data substantially improves deep-learning forecasts.","pith_inferences":["A natural extension is to test whether the same WPD plus seasonal adjustment pipeline improves other deep learners on stationary wind data, not just bidirectional LSTM.","If the decomposition is implemented causally, the pipeline could run online and update forecasts whenever a new hourly observation arrives; the paper does not specify the implementation's causality.","The pipeline could be transferred to other Saudi wind sites with different wind regimes, which would show whether the WPD-SAM combination generalizes or is tuned to Al-Jouf's particular seasonality.","Comparing WPD-SAM-BiLSTM against a strictly causal, leakage-free implementation of the same idea would isolate the contribution of the hybrid decomposition from any look-ahead advantage."],"forward_implications":["Operators at the Dumat Al-Jandal wind farm can use the one-, three-, and five-hour forecasts to plan grid dispatch with mean absolute errors around 0.12 to 0.26.","Decomposition methods previously applied mainly to non-stationary wind series also improve accuracy on stationary wind data, broadening the settings where such methods are useful.","Combining wavelet packet decomposition with seasonal adjustment gives a larger accuracy gain than either decomposition alone, supporting two-stage decomposition pipelines.","Forecast error grows with horizon, so the model's reliability is highest at the shortest lead time.","Because the model combines decomposition with a deep learner, it can be adapted to other wind sites by retraining the bidirectional LSTM and recomputing the site's seasonal indices."],"supporting_citations":[{"why":"Frames the data-leakage and boundary issues in decomposition-based forecasting that motivate the paper's walk-forward design.","marker":"[8]"},{"why":"Supports the combination of decomposition methods with bidirectional LSTM for short-term wind speed forecasting.","marker":"[26]"},{"why":"Shows that hybridizing two decomposition methods with a neural network improves forecasting accuracy, a direct precedent for the two-stage WPD-SAM design.","marker":"[27]"},{"why":"Demonstrates that wavelet packet decomposition combined with neural networks reduces wind speed forecasting error, motivating the WPD stage.","marker":"[28]"},{"why":"Provides evidence that seasonal adjustment combined with recurrent models improves wind speed forecasts, motivating the SAM stage.","marker":"[29]"},{"why":"Demonstrates that seasonal adjustment improves short-term wind speed forecasting when merged with neural models, cited for SAM's advantage.","marker":"[30]"},{"why":"Defines the data-leakage problem in wavelet-based forecasting and supplies the walk-forward technique the paper adopts.","marker":"[39]"}],"fun_headline_variants":["WPD+SAM+BiLSTM wind forecast: 0.995 R2","Saudi wind speed: hybrid deep model beats 27 rivals","Two-stage decomposition + BiLSTM tops Saudi wind speed","Decompose then forecast: BiLSTM beats 27 wind models","Hybrid WPD-SAM-BiLSTM: Saudi wind forecast winner"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The evaluation assumes that the frequency-band decomposition of the test period is computed at each forecast origin using only data available up to that moment, so the reported test errors are genuine out-of-sample errors rather than results of future values leaking into the decomposition.","fun_headline_variants_meta":{"raw":{"variants":["WPD+SAM+BiLSTM wind forecast: 0.995 R2","Saudi wind speed: hybrid deep model beats 27 rivals","Two-stage decomposition + BiLSTM tops Saudi wind speed","Decompose then forecast: BiLSTM beats 27 wind models","Hybrid WPD-SAM-BiLSTM: Saudi wind forecast winner"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001049,"raw_usage":{"total_tokens":4426,"prompt_tokens":983,"completion_tokens":3443,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":599,"completion_tokens_details":{"reasoning_tokens":3348}},"tokens_in":599,"tokens_out":3443,"duration_ms":24971,"temperature":1.0,"reasoning_tokens":3348,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T13:12:43.388252+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Rerun the one-hour-ahead experiment with a strictly causal implementation: at each forecast origin, recompute the wavelet packet transform and the seasonal indices using only observations up to that origin, then measure MAE and R2 on the same test period. If the causal version's one-hour MAE is substantially larger than 0.116901, the paper's headline accuracy depends on look-ahead leakage rather than on the WPD-SAM-BiLSTM pipeline itself.","supporting_citations":[{"cited_title":"Decomposition-based wind power foreca sting models and their boundary issue: An in -depth review and comprehensive discussion on potential solutions,","cited_arxiv_id":null,"evidence_quote":"Frames the data-leakage and boundary issues in decomposition-based forecasting that motivate the paper's walk-forward design."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Demonstrates that wavelet packet decomposition combined with neural networks reduces wind speed forecasting error, motivating the WPD stage."},{"cited_title":"Medium -term wind speeds forecasting utilizing hybrid models for three different sites in Xinjiang, China,","cited_arxiv_id":null,"evidence_quote":"Provides evidence that seasonal adjustment combined with recurrent models improves wind speed forecasts, motivating the SAM stage."},{"cited_title":"A novel hybrid approach for wind speed prediction,","cited_arxiv_id":null,"evidence_quote":"Demonstrates that seasonal adjustment improves short-term wind speed forecasting when merged with neural models, cited for SAM's advantage."},{"cited_title":"Quilty and J","cited_arxiv_id":null,"evidence_quote":"Defines the data-leakage problem in wavelet-based forecasting and supplies the walk-forward technique the paper adopts."}],"review_version":1}