{"id":"e44ae6ed-4a03-42b9-b523-222fdc86ff96","arxiv_id":"2607.14024","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Cluster-based Sequential Feature Selection (CSFS) matches sequential feature selection accuracy on wind and solar power forecasts while reducing runtime by about 21%.","lead":"This paper reports a new feature-selection method for wind and solar power forecasting that groups similar sensor readings, tests removing whole groups, and claims about 20% less computation than the standard wrapper method with similar accuracy. Real turbine and solar-plant data back the comparison.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Runtime reduction claim is only measured for target size |F'|=2 and the reported average is inconsistent; the headline 21% figure is not supported by the experimental section.","rationale":"The reader's weakest assumption focuses on the undefined non-inferiority test in §3.2. That is a genuine reproducibility gap, but the paper provides an open-source implementation, which mitigates it: the exact test is inspectable in code. The most load-bearing concern for the headline claim is the runtime reduction figure itself, which is the quantitative differentiator of CSFS. The experimental section clearly shows the runtime comparison was performed only for |F'|=2, while the abstract generalizes the result. The internal inconsistency between the per-dataset numbers and the reported average further undermines the claim. Even if the non-inferiority test were fully specified, the runtime claim would remain unsupported without measurements at other target sizes. Therefore, the paper's central claim is not fully supported by the evidence as presented, but the issue is addressable by additional experiments, so a conditional verdict is appropriate. I agree with the reader's overall CONDITIONAL assessment, but I emphasize a different specific weakness, hence 'partial'.","tokens_in":11252,"tokens_out":4761,"duration_ms":43854,"concrete_test":"Run the CSFS and SFS implementations from the provided GitHub repository on WT-S1, WT-S2, PV-S1, and PV-S2 for target sizes 5, 8, and 10 (not only 2) using the same hardware/software setup, and compute the per-dataset and overall mean runtime reduction. If the mean across all experiments is not close to 21%, or differs substantially from the |F'|=2-only value, the abstract's unqualified 'average of 21%' must be corrected.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim (abstract and conclusion) is that CSFS reduces computational cost by an average of 21% while matching SFS. The runtime evidence in §5.2 is restricted to a single target feature-set size: Fig. 5's caption states the comparison is 'for the target set feature size |F'|=2'. The abstract and conclusion present the 21% as a general average, with no such caveat. Moreover, the four per-dataset reductions in §5.2 are 19.4%, 9.3%, 25.3%, and 39.8%; their unweighted mean is 23.45%, yet §5.4 says 'on average by 20.8%' and the abstract says 21%. No explanation of the weighting or discrepancy is given. Because backward selection with a smaller target size requires many more removal iterations, cluster-level removal is likely to yield larger runtime savings at |F'|=2 than at |F'|=5/8/10; hence the reported 'average' may be artificially high. Even if the non-inferiority test is underspecified (§3.2), the released code can disambiguate it; the runtime measurement gap is in the reported evidence itself and cannot be resolved without additional experiments. This is the most load-bearing weakness for the paper's differentiator.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper reports two literature reviews (a new structured review of 90 wind-turbine power-curve modeling papers and a synthesis of an existing PV forecasting survey), identifies that feature selection is rarely used systematically in these domains, and proposes Cluster-based Sequential Feature Selection (CSFS), a wrapper-based method that clusters features, removes clusters or individual features based on a non-inferiority test, and falls back to forced removal when nothing is safely removable. The method is evaluated on four dataset variants (WT-S1, WT-S2, PV-S1, PV-S2) against backward SFS, filter methods (MI, F-value), and Random Forest feature importance, using MLP, LightGBM, and XGBoost. The main claims are that wrapper-based methods select better feature sets than simpler baselines and that CSFS achieves predictive performance comparable to SFS while reducing computational cost by an average of 21%.","tokens_in":11654,"tokens_out":2871,"duration_ms":30224,"significance":"If the claims are supported, CSFS would be a practically useful contribution: it offers a model-agnostic, open-source wrapper feature-selection method with a plausible speed/accuracy trade-off, and the two use cases are relevant to renewable-energy forecasting. The paper has concrete strengths: the implementation is publicly available, the evaluation uses leakage-aware temporal splits with validation gaps, performance estimates are bootstrapped and repeated, and the literature-review data are released. However, the headline runtime reduction is not supported by the evidence as presented, and the non-inferiority test that drives the method's safe-removal decisions is not specified in the text. These two issues are load-bearing for the central contribution, so the paper requires substantive revision before the main claims can be accepted.","major_comments":[{"comment":"The claim that CSFS reduces computational cost by an average of 21% is not supported by the reported experiments. The runtime comparison in Fig. 5 is explicitly restricted to target set size |F'|=2, yet the abstract and conclusion present the 21% figure without this caveat. Moreover, the four per-dataset reductions listed in §5.2 are 19.4%, 9.3%, 25.3%, and 39.8%; their unweighted mean is 23.45%, not 20.8% (§5.4) or 21% (abstract/conclusion). No explanation is given for this discrepancy. Because backward selection requires more iterations for smaller target sizes, cluster-level removal likely saves more time at |F'|=2 than at larger sizes, so the reported average may not generalize. The authors should report runtimes for all target sizes and either correct the headline number or clearly qualify it as applying only to |F'|=2.","section":"Abstract, §5.2, Fig. 5, §5.4, Conclusion"},{"comment":"The non-inferiority test is never defined. The text states that clusters or features are discarded 'if the removal does not negatively impact predictive performance with a high certainty,' but no margin, test statistic, confidence level, or multiple-comparison handling is given. This test is the mechanism that authorizes early removal of whole clusters and is therefore responsible for CSFS's runtime advantage over SFS. Without a precise specification, the method is not reproducible from the paper, and the reported speedup could in principle be driven entirely by a permissive threshold that removes important features. The authors should provide the full statistical procedure and, ideally, a sensitivity analysis of the margin.","section":"§3.2"},{"comment":"The overall conclusion that 'wrapper-based methods overall provide better-performing selections' is contradicted by the PV-S2 scenario. In Fig. 3, PV-S2 shows MI, F-value, and RF FI achieving RMSE around 2.7 MW, comparable to or better than the wrapper methods, and Table 1 reports that the best PV-S2 feature set was found by F-value. The text acknowledges this in passing but does not qualify the global claim or test whether wrapper-vs-baseline differences are statistically significant. Since the paper's second central claim is 'comparable predictive performance' to SFS, the authors should report per-scenario statistical comparisons (e.g., confidence intervals or paired tests) and either restrict the 'wrapper methods better' conclusion to the larger S1 variants or explain the PV-S2 result.","section":"§5.1, §5.3, Table 1"}],"minor_comments":[{"comment":"No pseudocode or formal algorithm description is provided for CSFS. The process diagram is helpful but insufficient; the safe-remove condition, cluster evaluation order, and force-remove fallback should be specified step-by-step.","section":"§3.2 / Fig. 2"},{"comment":"The 'enhanced SFS version that also uses the safe-remove shortcut' is introduced only in §5.1, while §3.2 describes SFS in its original form. This should be clarified earlier, since the comparison baseline is not the standard SFS.","section":"§5.2"},{"comment":"The statement that whole-cluster removal occurs in 29.5% of all CSFS iterations is not accompanied by a standard deviation or range across datasets and clustering methods. This figure is used to explain the runtime gains, so it should be reported per configuration.","section":"§5.4"},{"comment":"Some feature names are abbreviated inconsistently (e.g., 'Amb. Temp.', 'Turb. Int.', 'Grd. R. Inv. Ph.2 T.'), and the table does not indicate how ties or near-ties were resolved. A legend or full feature names would improve interpretability.","section":"Table 1"},{"comment":"References [22] and [23] are identical; one should be removed or replaced with the correct original source for the WRF model.","section":"References"},{"comment":"The cyclical encoding step says it is added 'for ANN models,' but the runtime comparison aggregates across all models. It would be helpful to state explicitly whether tree-based models received the same features without cyclical encoding or received raw time variables.","section":"§4.4"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a modest but real contribution — an SFS variant that removes correlated feature clusters as a block — and the empirical study is decent. The problem is that the headline number (21% average runtime saving) doesn't match the per-scenario numbers in the paper and is only measured for one target feature-set size. Also, the non-inferiority test that makes the method work is never defined.\n\nThe good parts: the two-stage cluster-then-feature idea is new as far as I know, and it is model-agnostic. The GitHub release is a plus. The evaluation covers four dataset scenarios, three models, several target sizes, and the baselines are standard. The authors also did leakage-aware temporal splits and acknowledge the greedy nature of the algorithm. The two literature reviews (90 wind papers, re-analysis of the PV survey) are useful and provided as data. The SFS baseline includes the same safe-remove shortcut, so the runtime comparison is conservative — that's fair.\n\nThe soft spots: first, the runtime claim. Fig. 5 and the text say the comparison is for |F'|=2 only. At larger target sizes, backward SFS has fewer iterations, so the advantage of cluster removal should shrink. The abstract and conclusion present 21% as a general average. The per-dataset numbers are 19.4%, 9.3%, 25.3%, 39.8%; their unweighted mean is 23.45%, §5.4 says 20.8%, abstract says 21%. No weighting or explanation is given. That's a load-bearing inconsistency. Second, §3.2 says a non-inferiority test allows 'safe and early discarding' of clusters, but gives no test statistic, margin, or confidence level. The method cannot be reimplemented from the text alone; the code may resolve it, but the paper should state it. Third, PV-S2 shows baselines performing about as well as wrapper methods, which contradicts the 'wrapper methods overall better' conclusion. The authors do note this, but the abstract and conclusion don't caveat it.\n\nThese are real but fixable issues. The core idea is plausible, and with code the non-inferiority test is checkable. I'd send this to peer review with a request for major revision: define the test, report runtime for all target sizes, and correct the average. The intended audience is practitioners in renewable-energy forecasting who want faster wrapper FS; they'll get a useful method if the numbers hold. I'd bring it to a reading group as a case study in how not to present a runtime improvement.","headline":"A promising, model-agnostic wrapper variant with a decent empirical study, but the headline 21% runtime saving is not supported by the reported numbers and the key non-inferiority test is left undefined.","tokens_in":12041,"tokens_out":2516,"would_cite":true,"duration_ms":24651,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Cluster-based feature selection matches sequential selection accuracy while cutting runtime by 21%.","keywords":["feature selection","wrapper methods","sequential feature selection","clustering","wind power prediction","solar power prediction","renewable energy","machine learning"],"falsifier":"Run CSFS and SFS on a public regression dataset with a fully specified non-inferiority test (e.g., a paired t-test with an explicit margin). If the 21% runtime reduction can only be achieved when the margin is large enough that the final feature sets omit features that materially change RMSE, while a zero margin erases the savings, then the claim of 'comparable performance at 21% lower cost' does not hold as stated.","tokens_in":11205,"feed_emoji":"⚡","tokens_out":5439,"duration_ms":39725,"temperature":0.7,"pith_summary":"The paper argues that feature selection for wind and solar power forecasting can be made substantially cheaper without sacrificing accuracy. It proposes CSFS, a wrapper method that groups correlated features into clusters and removes whole clusters at once when a statistical test says they are safe to drop. In experiments on wind turbine SCADA data and photovoltaic power data, CSFS selects feature sets whose predictive error is comparable to those found by standard sequential feature selection, but at an average 21% lower computational cost. The paper also documents, through its own review of 218 wind-power papers and a synthesis of an existing photovoltaic review, that the field rarely uses systematic feature selection despite having many available predictors.","feed_headline":"Feature selection shortcut matches accuracy at 21% lower cost","feed_subtitle":"New CSFS method clusters features to speed up wind and solar power prediction without hurting model quality.","key_machinery":"The key mechanism is the cluster-then-prune procedure in CSFS: features are partitioned into disjoint clusters, and a non-inferiority test authorizes early removal of an entire cluster if dropping it does not degrade model performance with high certainty. This is what turns many expensive single-feature evaluations into fewer cluster-level evaluations; the gain is only realized if the test is safe enough to avoid discarding informative features and permissive enough to actually skip evaluations.","core_discovery":"The central discovery is that a wrapper-based feature selection algorithm can be made faster without hurting predictive performance by operating on clusters of features rather than individual features. Specifically, the proposed Cluster-based Sequential Feature Selection (CSFS) first groups features using correlation, random grouping, or random-forest importance, then iteratively removes entire clusters when a non-inferiority test indicates the removal does not harm predictive performance, falling back to fine-grained feature-level removal only when needed. Across four dataset variants (wind and solar, with and without system-level or measurement variables), CSFS achieves RMSE comparable to","pith_inferences":["If the non-inferiority test were fully specified and made public, CSFS could be directly compared across other regression tasks; one testable prediction is that the runtime savings scale with the number of redundant features, so datasets with more correlated feature groups should see larger speedups.","The reliance on clustering means that the choice of clustering method becomes a hyperparameter; correlation-based clustering seems the most principled default, and the paper's own results hint it may be more reliable than random or importance-based grouping in small feature spaces.","The literature synthesis suggests that most existing wind and solar prediction studies leave predictive signal on the table by using only two to five features; a direct extension would be to test whether the selected ten-feature sets generalize across sites and years, not just within the single turbine and single station evaluated here."],"forward_implications":["Practitioners in renewable-energy forecasting can run feature selection much more frequently or on larger feature sets, because CSFS lowers the computational barrier.","The method is model-agnostic, so it can be plugged into standard machine-learning pipelines with the same expected speedup.","The 21% average runtime reduction is a conservative lower bound, since the comparison baseline already includes the same safe-remove shortcut at the feature level.","The feature importance rankings from the domain-level analysis give concrete guidance: ambient temperature, wind speed, turbulence intensity, and solar irradiance variables dominate the best subsets.","The finding that wrapper methods outperform filters mainly on the larger dataset variants suggests that the benefit of expensive feature selection grows with input dimensionality."],"fun_headline_variants":["Cluster-based feature selection cuts costs 21% without losing accuracy","CSFS groups features to cut 21% computing cost, matching accuracy","Renewable prediction: cluster-based feature selection saves 21% cost","Efficient wrapper feature selection: CSFS matches accuracy, 21% cheaper","Speed up wind/solar prediction: CSFS clusters features, saves 21% cost"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The whole speed advantage of CSFS rests on a non-inferiority test that the paper never actually defines—its margin, test statistic, and confidence level are unspecified—so whether the method is safe depends on an unknown quantity.","fun_headline_variants_meta":{"raw":{"variants":["Cluster-based feature selection cuts costs 21% without losing accuracy","CSFS groups features to cut 21% computing cost, matching accuracy","Renewable prediction: cluster-based feature selection saves 21% cost","Efficient wrapper feature selection: CSFS matches accuracy, 21% cheaper","Speed up wind/solar prediction: CSFS clusters features, saves 21% cost"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001076,"raw_usage":{"total_tokens":4355,"prompt_tokens":776,"completion_tokens":3579,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":520,"completion_tokens_details":{"reasoning_tokens":3481}},"tokens_in":520,"tokens_out":3579,"duration_ms":22150,"temperature":1.0,"reasoning_tokens":3481,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-02T02:58:09.418082+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run CSFS and SFS on a public regression dataset with a fully specified non-inferiority test (e.g., a paired t-test with an explicit margin). If the 21% runtime reduction can only be achieved when the margin is large enough that the final feature sets omit features that materially change RMSE, while a zero margin erases the savings, then the claim of 'comparable performance at 21% lower cost' does not hold as stated.","supporting_citations":[],"review_version":1}