{"id":"e086f2fd-eeff-4815-a498-7ee3feba902c","arxiv_id":"2501.16591","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"EMGRL, an actor-critic ensemble of ARIMA, LightGBM, LSTM, and SGNN with a GNN-based spatial state embedding, is claimed to beat baselines on NREL and GEFC wind datasets.","lead":"This paper proposes EMGRL, an ensemble model that combines graph neural networks and reinforcement learning to forecast wind farm power output. It reports up to 12.89% lower error than baseline forecasting models on two public wind datasets.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table 3 (RMSE) is byte-for-byte identical to Table 2 (MAE), which is impossible for real forecasts; the reported RMSE improvements and the 12.89% headline are not supported by the tables as printed.","rationale":"The reader's stated weakest_assumption is about the geographic graph structure being the correct relational inductive bias. That is a legitimate modeling risk, but it is not the load-bearing failure. Even if the graph structure were exactly right, the paper's empirical claim would still fail because the reported tables are internally inconsistent. The reader's rationale, however, does identify the table duplication and the missing code; that is the same concern I focus on. Since the reader's verdict is already REJECT and my analysis reinforces that rejection without adding a new fatal defect, the verdict remains unchanged. The concern is not a disagreement with the ensemble methodology or with the use of GNN/RL for wind power forecasting; it is that the manuscript's numerical evidence cannot be trusted as printed. A corrected re-run on the public datasets with released code would be needed before the central claim can be evaluated. I do not manufacture additional objections: the internal inconsistency of Tables 2 and 3 is sufficient and concrete.","tokens_in":12730,"tokens_out":4847,"duration_ms":48698,"concrete_test":"Download the public NREL WIND Toolkit (2010–2011) and GEFC 2012 datasets identified in §5.1. Independently implement the four base models (ARIMA, LightGBM, LSTM, SGNN) and the EMGRL ensemble as described in §4. Compute MAE and RMSE from the raw forecasts using Eq. 9 and Eq. 10. Then check: (1) Is Table 3 identical to Table 2? (2) For every model–farm cell, does RMSE >= MAE hold? (3) Do the per-farm improvements match 9.62/10.44/6.57/6.34, and does the aggregate 12.89% appear? A cheaper internal check already settles the issue: the equality of Tables 2 and 3 violates RMSE >= MAE, so the RMSE results as printed cannot be genuine.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is empirical: EMGRL outperforms state-of-the-art baselines, with the strongest formulation being the 12.89% improvement claimed in the abstract and conclusion. The evidence for this claim is Tables 1–3. The decisive defect is that Table 3, labeled RMSE on the GEFC dataset, is numerically identical to Table 2, labeled MAE, for every model and every wind farm. This is impossible for genuine forecasts: RMSE >= MAE by Jensen's inequality, with equality only when all absolute errors are equal, which cannot hold over thousands of hourly samples. Thus at least one of these tables does not report the metric it claims. The NREL results are similarly ambiguous: Section 5.5 text says RMSE, but Table 1's caption says MAE, and Figure 11 says RMSE. The per-farm improvements quoted in the text (9.62%, 10.44%, 6.57%, 6.34%) do not match values computed from the table (8.78%, 9.45%, 6.17%, 5.97%). The headline 12.89% is not derivable from any reported comparison. No code, hyperparameters, or error bars are released, so the internal inconsistency cannot be resolved post hoc. Even if the GNN-plus-RL ensemble idea is plausible, the quantitative evidence as printed is internally inconsistent and cannot support the paper's central claim.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes EMGRL, an ensemble model for wind power forecasting that combines ARIMA, LightGBM, LSTM, and SGNN as base learners. A dilated CNN compresses each farm's time series, a GNN aggregates neighboring-farm representations, and an actor-critic reinforcement learning module outputs ensemble weights from a state embedding that includes the historical losses of the base models. The experiments on the NREL and GEFC 2012 datasets claim that EMGRL outperforms seven baselines by up to 12.89% in MAE and RMSE. The core evaluation section, however, contains internal inconsistencies that undermine the reported quantitative claims.","tokens_in":13072,"tokens_out":6254,"duration_ms":60174,"significance":"The idea of using GNN-based spatial state embeddings and actor-critic weight selection is a reasonable and potentially useful direction for wind power forecasting ensembles, and the use of two open datasets is appropriate. The paper also separates training and test years and fits the ensemble weights on training data, so the headline result is not circular. Nevertheless, the empirical validation as presented cannot be trusted: the RMSE table is identical to the MAE table, which is impossible for nonzero forecast errors, and the claimed improvement percentages do not match the numbers shown. The contribution will be significant only after the experiments are redone or correctly reported; as submitted, the central claim is unsupported.","major_comments":[{"comment":"Table 2 (labeled MAE) and Table 3 (labeled RMSE) on the GEFC dataset are byte-for-byte identical for every model and every wind farm. For any nontrivial forecast error, RMSE must be strictly greater than MAE, so at least one of these tables cannot be reporting the metric named in its caption. This invalidates the GEFC numerical evidence and any conclusion drawn from it.","section":"§5.5, Tables 2 and 3"},{"comment":"The metric for the NREL experiment is ambiguous: Table 1's caption says MAE, the text says RMSE, Figure 10 says MAE, and Figure 11 says RMSE. In addition, the improvements over SGNN quoted in the text (9.62%, 10.44%, 6.57%, 6.34% for farms A–D) do not match the values computed from Table 1 (8.78%, 9.45%, 6.17%, 5.97%). The reported improvements are therefore not reproducible from the numbers as printed.","section":"§5.5, Table 1 and Figures 10–11"},{"comment":"The headline claim of an improvement of up to 12.89% is not derivable from any reported comparison. The largest improvement in Table 1 is about 9.45% (if MAE is intended), and the GEFC improvements over the best baseline are roughly 4.9–7.7% depending on the farm. No table, equation, or calculation in the paper yields 12.89%, so the central quantitative claim lacks support.","section":"Abstract and §6"},{"comment":"No error bars, standard deviations, numbers of independent runs, or statistical significance tests are reported for any result. Because the claimed margins over SGNN and SDAE are only a few percent, these margins could easily fall within run-to-run variability, so the central comparison is not statistically supported as presented.","section":"§5.4–5.5"},{"comment":"The training algorithm is not fully specified: line 3 samples a'_t = π_θ(s,a) with an undefined action a, line 5 writes the actor update as θ = θ + α log π_θ(s_t,a_t) Q_ω(s_t,a_t), omitting the gradient operator and any baseline, and no hyperparameters, network sizes, or optimizer settings are given in Section 5.3. This makes the method irreproducible as described.","section":"§4.4, Algorithm 1"}],"minor_comments":[{"comment":"The text says the base-model losses are compressed by a dilated convolutional neural network, but Eq. (7) uses an MLP; the notation and description should be aligned.","section":"§4.2, Eq. (7)"},{"comment":"The notation t_k in Eq. (6) is not defined; the input window and features fed to the dilated CNN should be specified.","section":"§4.2, Eq. (6)"},{"comment":"The symbol a_t^2 and the line numbering in Algorithm 1 are garbled; the action should be written consistently, for example as a'_t.","section":"§4.4, Algorithm 1"},{"comment":"The introduction says all turbines within a wind farm are graph nodes, but Section 4.2 builds a graph over wind farms A–F; the node semantics should be made consistent.","section":"§1 and §4.2"},{"comment":"Several references have formatting problems, including mixed punctuation and incomplete author lists; the reference list should be checked against the journal style.","section":"References"},{"comment":"No code or data availability statement is included, and no explicit hyperparameter settings are provided, which is essential for reproducing the experimental tables.","section":"§5"}],"recommendation":"reject","confidential_remarks":"The stress-test concern lands: the duplicated tables are not a wording typo but an impossible numerical relation, and the 12.89% headline cannot be traced to any table. I see no circularity in the training/test split, so the issue is evidential rather than methodological. A revision would need to supply a completely re-run, correctly labeled experimental evaluation with error bars and code; without that, the manuscript should not be reconsidered."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague, the one thing to know: the ensemble idea is sensible, but the evidence is broken. Table 3, labeled RMSE on the GEFC dataset, is identical to Table 2, labeled MAE, for every model and every wind farm. That cannot happen with real forecasts: RMSE >= MAE, and equality would require all absolute errors to be equal, which is impossible over thousands of hourly samples. So at least one table is mislabeled. The same kind of inconsistency shows up on NREL: the text says RMSE, but Table 1 says MAE, and Figure 11 says RMSE. The quoted per-farm improvements (9.62%, 10.44%, 6.57%, 6.34%) don't match what I can compute from the table (8.78%, 9.45%, 6.17%, 5.97%), and the headline 12.89% is not derivable from any reported comparison. There are no error bars, no code, no hyperparameters. As printed, the central quantitative claim is not testable.\n\nWhat is genuinely new: the specific pipeline — a GNN over wind-farm locations to build a spatio-temporal state embedding, combined with the historical losses of the base models, and an actor-critic RL to choose ensemble weights — is a legitimate extension of prior work on RL-based ensembles (Chen et al. 2021, Jalali et al. 2021). The writing is clear about the training and prediction flow. The choice of base models (ARIMA, LightGBM, LSTM, SGNN) is reasonable, and using two public datasets is appropriate. The closest-design concern — that the ensemble weights are trained on the same loss signal that goes into the state — is not circular; the weights are fit on training data and tested on a held-out year. The reliance on a geographic graph is a modeling assumption, not a flaw.\n\nThe citation pattern looks fine; the related work covers the relevant traditional and ML forecasting literature.\n\nMy recommendation: do not accept this as-is. The idea may have merit, but the tables need to be corrected, error bars and code released, and the headline improvement re-derived. If the journal route allows, I'd desk-reject with an invitation to resubmit after that, rather than spend referee time on numbers that cannot be true as printed. Serious thinker: yes, the paper is coherent and honestly engages the literature; the flaw looks like copy-paste sloppiness, not fabrication. But the load-bearing evidence fails.","headline":"The ensemble idea is sensible, but the central tables are internally inconsistent, so the empirical claim cannot be evaluated as printed.","tokens_in":13615,"tokens_out":4067,"would_cite":false,"duration_ms":40161,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"An ensemble of GNN and RL cuts wind forecast error by up to 12.89%","keywords":["wind power forecasting","ensemble learning","graph neural network","reinforcement learning","actor-critic","state embedding","NREL dataset","GEFC dataset"],"falsifier":"Train the same EMGRL on a wind farm group where the geographically closest farms are strongly anti-correlated in power output (for example, a ridge with different wind exposures), and compare test error against an EMGRL variant with the GNN removed. If the GNN variant does not beat the no-GNN variant, the geographic-graph assumption fails. A simpler observational check: compute the sample cross-correlation of wind power between nearest-neighbor farms in the GEFC dataset; if most neighbor pairs show near-zero or negative correlation, message passing across the geographic graph cannot be the source of the reported gain.","tokens_in":12535,"feed_emoji":"💨","tokens_out":5870,"duration_ms":48283,"temperature":0.7,"pith_summary":"The paper tries to show that wind power forecasts improve when no single model is trusted: a graph neural network embeds the target wind farm's history together with neighboring farms, and an actor-critic reinforcement learning agent reweights four base models (ARIMA, LightGBM, LSTM, and SGNN) at every time step. If true, this means the question 'which model is best right now?' is as important as the question 'what is the next power value?', and both can be answered by one learned ensemble. The reported result is up to a 12.89% lower error than the best baseline on the NREL and GEFC open wind datasets.","feed_headline":"Wind forecast error cut 12.89% by GNN-RL ensemble","feed_subtitle":"A graph-embedded state plus actor-critic weights beats every single model on two open wind datasets.","key_machinery":"The state embedding is the load-bearing piece. It is built in two parallel branches: a spatio-temporal branch passes each wind farm's dilated-CNN-compressed time series through a GNN whose message-passing stages (computation, aggregation, update) let a target farm's representation absorb information from geographically neighboring farms; a model-loss branch compresses the recent prediction losses of all base models on the target farm through a multi-layer perceptron. The two vectors are concatenated into $SE_A = (STSE_A, MLE_A)$, and this state is fed into an actor-critic loop where the actor outputs weights $\\mathbf{a}_t = [W_{t,1}, \\dots, W_{t,N}]$ and the reward is derived from comparing the ensemble forecast with the observed power.","core_discovery":"The central claim is that an ensemble whose weights are chosen by reinforcement learning, and whose state representation is a spatio-temporal embedding of wind farms produced by a GNN, forecasts wind power more accurately than any single time-series model and than fixed-weight ensembles. The paper constructs a graph whose nodes are wind farms and whose edges encode geographical proximity; a dilated CNN compresses each farm's recent power sequence into a vector, the GNN mixes these vectors across neighboring farms, and the resulting embedding is concatenated with a compressed vector of the base models' recent losses. This state is fed to an actor-critic agent that outputs normalized weights for the four base models, and the weighted sum becomes the forecast. On four NREL offshore wind farms the method reports RMSE reductions of roughly 6–10% over the best single base model, and on seven GEFC wind farms MAE around 7.74% lower than the best baseline SDAE, with the overall best case reaching 12.89% improvement.","pith_inferences":["The graph edge weights are based purely on geographic distance, but wind power dependence actually propagates with weather systems; a natural testable extension is to orient or weight edges by prevailing wind direction or by lagged cross-correlation between farms.","The same actor-critic-over-base-models recipe could be transferred to other spatiotemporal forecasting domains (solar power, load forecasting, traffic) wherever base models shift in relative performance.","The 12.89% figure comes from a specific best-farm comparison; a more demanding check would be to average the improvement over all farms and test statistical significance, which the paper does not report."],"forward_implications":["If EMGRL is right, forecasters do not need to build one perfect model; they can maintain a portfolio of diverse models and learn when each one leads.","The GNN's neighborhood aggregation makes the forecast for one farm depend explicitly on other farms' recent output, so the method's accuracy should scale with the density of the wind farm network.","Because weights are re-derived at every timestep from recent losses, the ensemble can track non-stationary wind conditions and data shifts better than fixed-weight stacking.","The 12.89% best-case improvement, if reproducible, would translate into reduced reserve requirements and more reliable grid scheduling for wind operators."],"supporting_citations":[{"why":"Supplies the NREL offshore wind dataset used to train and test the model.","marker":"(Draxl et al., 2015)"},{"why":"Supplies the GEFC 2012 wind forecasting dataset and the normalization used in preprocessing.","marker":"(Li & Chiang, 2016)"},{"why":"Defines the SGNN graph-neural-network baseline and base model for wind power prediction.","marker":"(Yu et al., 2020)"},{"why":"The ARIMA baseline referenced in the base model set.","marker":"(Wang & Hu, 2015)"},{"why":"The LightGBM baseline referenced in the base model set.","marker":"(Ren et al., 2022)"},{"why":"The LSTM baseline referenced in the base model set.","marker":"(Ko et al., 2020)"},{"why":"The COBRT baseline compared in the experiments.","marker":"(Guo & Chiang, 2016)"},{"why":"The SDAE baseline compared in the experiments.","marker":"(Yan et al., 2018)"},{"why":"Provides the reinforcement learning and Markov decision process framing that the actor-critic weight policy rests on.","marker":"(Sutton et al., 1998)"}],"fun_headline_variants":["Wind forecast error drops 12.89% with GNN-RL ensemble","GNN and reinforcement learning ensemble improve wind power forecasts","Actor-critic weights boost wind prediction by up to 12.89%","Spatio-temporal GNN state guides RL to better wind forecasts","Ensemble with graph embeddings and RL outperforms single models"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that the right way to use other wind farms is to let geographically nearest farms speak to each other through the GNN; if distance is not a reliable proxy for the wind-power dependence that carries forecasting information, the entire state embedding loses its advantage.","fun_headline_variants_meta":{"raw":{"variants":["Wind forecast error drops 12.89% with GNN-RL ensemble","GNN and reinforcement learning ensemble improve wind power forecasts","Actor-critic weights boost wind prediction by up to 12.89%","Spatio-temporal GNN state guides RL to better wind forecasts","Ensemble with graph embeddings and RL outperforms single models"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000539,"raw_usage":{"total_tokens":2576,"prompt_tokens":928,"completion_tokens":1648,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":544,"completion_tokens_details":{"reasoning_tokens":1558}},"tokens_in":544,"tokens_out":1648,"duration_ms":12891,"temperature":1.0,"reasoning_tokens":1558,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T12:02:29.816446+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the same EMGRL on a wind farm group where the geographically closest farms are strongly anti-correlated in power output (for example, a ridge with different wind exposures), and compare test error against an EMGRL variant with the GNN removed. If the GNN variant does not beat the no-GNN variant, the geographic-graph assumption fails. A simpler observational check: compute the sample cross-correlation of wind power between nearest-neighbor farms in the GEFC dataset; if most neighbor pairs show near-zero or negative correlation, message passing across the geographic graph cannot be the source of the reported gain.","supporting_citations":[],"review_version":1}