{"id":"b23483a5-7c04-43c3-b252-87b9918d8b45","arxiv_id":"2507.07477","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"Machine learning models, especially boosted trees and GLM, predict Singapore daily electricity returns out-of-sample, and a correlation-penalized ensemble beats all individual models, with predictability concentrated in expansion and volatile regimes.","lead":"Using 21 years of Singapore electricity market and macroeconomic data, this paper compares machine learning models for predicting next-day electricity price moves. It reports large predictive gains for tree-based models and a correlation-penalized ensemble, with the strongest predictability during expansions and volatile periods.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Monthly feature standardization in Appendix A.3.2 uses full-month moments for out-of-sample days, importing future information into the OOS features; this directly contaminates the reported R2_OOS values and all model rankings.","rationale":"The reader's weakest-assumption analysis identifies exactly this standardization leak, and the full text supports it: Appendix A.3.2 describes monthly standardization without any carve-out for out-of-sample months, and Section 3.2's information-set restriction is incompatible with using full-month moments. This is the single most load-bearing concern because it attacks the construction of the features themselves, not merely the statistical test or benchmark choice. Every central result—the positive R2_OOS values, the 'all significant at 1%' claim in the abstract and Section 1, the top-model ranking (GLM, XGB(+H), LGBM(+H), Ensemble wp), the 'virtue of complexity' panel regression, and the regime-level predictability patterns—is downstream of the OOS predictions. If the leak is real, those predictions are not true out-of-sample forecasts, and the paper's quantitative conclusions cannot be trusted as reported. The paper does offer some independent support in the form of simulation exercises and a large, carefully described dataset, and the writing is competent, but those do not rescue the empirical claim. A rejection is warranted unless the authors can show that their implementation of monthly standardization somehow avoided future information—something the current text does not establish. The proposed concrete test would settle the matter directly by rerunning the evaluation with moments restricted to the information set available at each forecast date.","tokens_in":54745,"tokens_out":2392,"duration_ms":31433,"concrete_test":"Recompute all OOS forecasts after replacing each out-of-sample month's feature mean and standard deviation with statistics computed only from data available at the forecast origin—for example, an expanding-window mean and standard deviation using all data up to day t, or at minimum the previous calendar month's moments. Then recompute Table 1's R2_OOS columns and the one-sided DM/CW significance stars. If Ensemble wp's R2_OOS,mean drops materially from 51.92% or the 'all positive and 1% significant' claim fails, the look-ahead bias is confirmed as the source of the headline predictability.","verdict_should_be":"REJECT","load_bearing_attack":"The central claim that all out-of-sample R2 values are positive and significant at the 1% level rests on a leaky evaluation pipeline. In Appendix A.3.2, the authors standardize features with a monthly (0,1) transformation: each feature is centered by subtracting its monthly mean and scaled by its monthly standard deviation. For an out-of-sample day t, the month is the OOS calendar month, so the mean and standard deviation are computed using all days in that month, including days after t. This violates the paper's own information-set restriction in Section 3.2, which states that a prediction at time t must not use information unavailable at t. Because the feature set includes the lagged USEP, the monthly moments for the OOS month embed future realizations of the target price itself, mechanically inflating apparent predictive accuracy. The magnitude of the leak is likely large: the reported R2_OOS,mean values of roughly 33–52% are unusually high for daily return predictability, and the lagged USEP is identified as the most important feature, making the transformation's dependence on future values especially consequential. If this standardization is corrected, the reported R2 values, the model rankings, the 'all significant at 1%' statement, and the regime-concentration results built on those values would all need to be recomputed. The paper provides no code or data to verify whether a leakage-free implementation reproduces the headline numbers, so the burden falls on the described procedure, which on its face leaks.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents a large-scale comparative machine learning study for forecasting daily Singapore electricity price returns (USEP log-returns) using 619 features that combine in-market stakeholder data, domestic macro indicators, international macro factors, and interaction terms. The main text reports out-of-sample R² values for 15 individual models and 4 ensemble methods over 2018-2023, claiming that all models achieve positive and statistically significant OOS R² at the 1% level, with top individual models (GLM, XGB(+H), LGBM(+H)) reaching roughly 43% R²_OOS,mean and a correlation-penalized ensemble reaching 51.92%. The paper further reports regime-dependent predictability (bullish/volatile/high-growth periods, geopolitical risk), a trend-based decomposition of OOS performance, a group-level feature importance measure, and utility gains under a mean-variance framework.","tokens_in":55041,"tokens_out":4311,"duration_ms":49136,"significance":"If the empirical claims held, this would be a substantial contribution: the dataset is unusually rich in market-stakeholder variables, the model coverage is broad, the proposed correlation-penalized ensemble is novel and plausibly useful, and the macro-regime analysis is policy-relevant. The paper also connects to recent asset-pricing ML literature (Gu et al., Kelly et al., Shen and Xiu) and provides a large set of robustness checks. However, the manuscript does not provide code or replication data, and the evaluation pipeline contains a critical look-ahead issue in feature standardization that directly contaminates the headline OOS R² values and all results built on them. The verification burden is therefore high, and the current reported magnitudes are not credible without a causal re-estimation.","major_comments":[{"comment":"The monthly (0,1) standardization described in Appendix A.3.2 computes each feature's monthly mean and standard deviation over the entire calendar month. For an out-of-sample day t in month T, the standardization uses the month's mean and standard deviation, which are calculated using all days of that month, including days after t. This violates the paper's own information-set restriction in Section 3.2 that predictions at time t must not use information unavailable at t. Because the feature set includes the lagged USEP, the monthly moments embed future price realizations, mechanically inflating the OOS R² values in Table 1, the DM/CW test outcomes, the model rankings, the regime comparisons in Table 6, and the utility results in Section 4.4. The entire OOS evaluation must be re-run with a causal standardization scheme (e.g., expanding-window or trailing moments) before the paper's central claims can be assessed.","section":"Appendix A.3.2, Section 3.2"},{"comment":"The macro-regime analysis defines states using full-sample terciles: for example, bearish/bullish market states are based on the full-sample (2003-2023) distribution of daily returns, volatile/tranquil states on full-sample monthly variance, and high/low night-light and GPR states on full-sample distributions. Since the OOS period (2018-2023) is included in the full sample, regime membership for a given OOS day depends on future observations, importing look-ahead information into the state-specific R² calculations. State definitions must be constructed using only in-sample data (or with a recursive/expanding information set) to provide a valid decomposition of OOS predictability.","section":"Section 4.2.1, Table 6"},{"comment":"The text states that 'all R²_OOS values are positive and statistically significant at the 1% level according to the one-sided Diebold-Mariano (DM) test,' but Table 1 contradicts this: for example, OLS's R²_OOS,AR(1) is reported as 14.16***(), with no DM significance inside the parentheses, and many entries in the R²_OOS,lagprice and R²_OOS,AR(1) columns similarly lack DM stars. If the parentheses denote DM significance, the claim is false for those metrics; if the notation means something else, it must be clarified. This inconsistency is load-bearing because the paper's first listed contribution is the universal 1% significance of all R²_OOS values.","section":"Section 4.1.1, Table 1"}],"minor_comments":[{"comment":"The abstract states that 'Simulation also supports the first virtue' but does not cite the simulation appendix (Appendix C.3); consider adding a reference for readability.","section":"Abstract"},{"comment":"The text says the validation set is 'fixed' in the recursive scheme, but Appendix B.1 shows the validation period rolling forward with each OOS month; the wording should be corrected to 'rolling validation window.'","section":"Section 3.1"},{"comment":"The correlation penalty term ρ_{m,m'} is used in Equation (17) before it is explicitly defined; define it as the Pearson correlation between predictions of models m and m' immediately after the equation.","section":"Equation (17)"},{"comment":"The manuscript contains no data or code availability statement; given the NDA data note, a clear statement about what can be shared (even synthetic or de-identified data) would help future verification.","section":"General"},{"comment":"Several cross-references and appendix labels appear inconsistent (e.g., Figure A6 is referenced in the text but the figure numbering in the appendix is not fully clear); a careful pass for referencing errors is needed.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The reported OOS evaluation is contaminated by a look-ahead standardization procedure, and the macro-regime states also use full-sample information. These are fixable in principle by re-running the entire empirical pipeline, but the magnitudes and even the qualitative conclusions may change substantially. The paper's contribution would be much stronger if the authors provide code or a detailed pseudo-implementation for the OOS pipeline, especially because the data are confidential. If the authors do not re-estimate with causal standardization, the manuscript should not be accepted; with a correct re-estimation, it could become a publishable contribution."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The short version: this paper has a lot of original material—the EMC stakeholder-level dataset, the breadth of models, the correlation-penalized ensemble, and the macro-regime decomposition—but the headline numbers are not trustworthy because the feature standardization leaks future information into the out-of-sample period. That is load-bearing, not cosmetic.\n\nWhat is genuinely new: the 619-feature set combining half-hourly in-market variables (supply, demand, regulation) with domestic and international macro and their interactions is a real step beyond the small-predictor studies that dominate electricity forecasting. The simulation exercise supporting the nonlinearity virtue is well designed. The ensemble method penalizing cross-model prediction correlation is a sensible contribution, and the economic value analysis plus the attempt to link predictability to macro regimes are in the right spirit.\n\nThe soft spots. Appendix A.3.2 says features are standardized by subtracting the monthly mean and dividing by the monthly standard deviation, computed over the whole calendar month. For an out-of-sample day t, that month's moments include days after t. Since the feature set includes lagged USEP, the transformation embeds future values of the target itself. The R2 numbers in Table 1 (e.g., 51.92% for the ensemble) are implausibly high for daily return predictability, and the leak explains why. The paper's own Section 3.2 explicitly requires predictions to use only information available at time t, so the procedure violates the stated information set. This contaminates every headline result, including model rankings, regime comparisons, and the DM significance claims.\n\nA second issue: the macro-regime analysis (Section 4.2.1) classifies states using full-sample terciles of variables like night light intensity and GPR, which include the 2018–2023 test period. That is a milder but still real look-ahead.\n\nAlso, the abstract says all R2 OOS values are significant at 1% by the one-sided DM test, but Table 1 shows the AR(1) benchmark column has no DM significance for many models (empty parentheses). The text later acknowledges this, but the abstract overstates it.\n\nOn balance: the authors have done a large amount of honest descriptive work, but the evaluation pipeline is broken in a way that invalidates the quantitative claims. The paper deserves a serious referee report because the data and ensemble idea are worth salvaging, but the current version should not be accepted without a complete re-estimation with a leakage-free standardization scheme.\n\nFor you: if you work on Singapore electricity or high-dimensional forecasting, read it for the data and design, but treat Table 1 as an upper bound until the pipeline is fixed.\n\nMy recommendation: send it to peer review with a major-revision verdict, but be prepared for the headline numbers to shrink substantially.","headline":"A genuinely ambitious Singapore electricity forecasting study undone by a likely look-ahead bias in the feature pipeline; worth a careful rework, not acceptance as-is.","tokens_in":55552,"tokens_out":2522,"would_cite":false,"duration_ms":28911,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Machine learning predicts Singapore power prices up to 52% out of sample.","keywords":["machine learning","electricity price forecasting","out-of-sample predictability","ensemble methods","Singapore wholesale electricity market","macro-finance links","feature importance","regime-dependent predictability"],"falsifier":"Recompute the full prediction pipeline with each day's features standardized using only data up to that day (e.g., trailing or expanding normalization), and compare the resulting R-squared values with the reported ones; if the correlation-penalized ensemble's $R^2_{OOS,mean}$ drops substantially below 51.92% or another model overtakes it, the reported predictability is inflated by look-ahead leakage. A single out-of-sample month where the gap exceeds the DM test's margin would settle the question.","tokens_in":54510,"feed_emoji":"⚡","tokens_out":6212,"duration_ms":60060,"temperature":0.7,"pith_summary":"This paper tries to establish that machine learning models, fed with a rich set of 619 daily features from Singapore's wholesale electricity market and the macroeconomy, can predict next-day electricity returns out of sample with economically large accuracy. The headline numbers are positive and statistically significant out-of-sample R-squared values for all models, with the best three individual models (GLM, XGB+H, LGBM+H) around 43-44% and a correlation-penalized ensemble reaching about 52%. If true, this means the USEP, the market's settlement price, is substantially forecastable, enough to generate sizable gains for a mean-variance investor, and that predictability is not uniform: it clusters in expansion periods, volatile markets, and high geopolitical risk. The paper also argues the results confirm three 'virtues' of ML—capturing nonlinearity, using complexity in high dimensions, and exploiting $\\ell^2$-regularization and bagging when factors are weak.","feed_headline":"Machine learning predicts Singapore power prices up to 52%","feed_subtitle":"A 619-feature ML ensemble beats simple benchmarks, with gains concentrated in expansions and volatile markets.","key_machinery":"The load-bearing mechanism is the correlation-penalized ensemble weighting scheme, which minimizes the validation-period MSE plus a penalty $\\lambda \\sum_{m,m'} w_m w_{m'} \\rho_{m,m'}$ on pairwise prediction correlations, so that redundant highly-correlated models are downweighted. Around this sits the 619-feature design: 35 in-market variables, 7 domestic macro, 9 international macro, 560 macro times in-market interaction terms, plus the lagged price and weekday dummies. The evaluation uses a recursive expanding-window scheme with three out-of-sample $R^2$ benchmarks (lag price, AR(1), historical mean/zero) and DM/CW tests.","core_discovery":"The central discovery is that daily Singapore electricity returns ($r_{t+1}$ from USEP log prices) are predictable out of sample once one combines in-market stakeholder data (supply, demand, regulation), domestic and international macro variables, and their interactions. Using a recursive expanding-window scheme, all 15 individual models and 4 ensembles deliver positive out-of-sample $R^2$ against lag-price, AR(1), historical-mean and zero benchmarks; the top individual models reach $R^2_{OOS,mean} \\approx 43\\%$ and the correlation-penalized ensemble (Ensemble wp) reaches 51.92%. The paper traces this predictability to three ML virtues—nonlinearity capture, complexity, and $\\ell^2$/bagging in weak-factor settings—and shows it translates into economic value: a mean-variance investor would accept a risk-free rate of up to 77.15% in lieu of an XGB(+H)-based risky portfolio. Furthermore, predictability is heterogeneous across macro regimes—concentrated in expansions (high night-light intensity), volatile electricity markets, and extreme geopolitical risk periods—while feature attribution points to a supply-side-driven market with strong regulatory influence.","pith_inferences":["Editorial inference: the correlation-penalized ensemble should transfer to any forecast-combination problem with highly correlated model outputs, such as equity premium or inflation forecasting, where average pairwise correlation is typically high.","Editorial inference: if the monthly-standardization leakage is confirmed, the absolute R-squared numbers should be read as upper bounds, and the ranking across models and regimes may shift once strict causal standardization is enforced.","Editorial inference: the dominance of interaction terms suggests that separate markets will need their own stakeholder-level interaction features, so the 52% figure is unlikely to generalize to markets without similarly rich in-market data."],"forward_implications":["Singapore's USEP is forecastable enough for practical market operations, so suppliers, retailers, and regulators could use these models for production planning, hedging, and market-monitoring decisions.","The correlation-penalized ensemble beats the best individual models and stays strong before and after the 2021 energy crisis, offering a forecast-combination recipe for other high-correlation settings.","Predictability is not constant: it concentrates in expansions and volatile regimes, so a regime-aware forecasting system should expect better performance in those states.","The feature-importance results imply that the market is supply-side driven and heavily regulated even post-liberalization, which matters for policy evaluation in Singapore and comparable deregulated markets."],"supporting_citations":[{"why":"Supplies the empirical asset-pricing ML framework, the recursive expanding-window scheme, and the R-squared reduction feature importance method.","marker":"Gu et al. (2020)"},{"why":"Provides the theoretical 'virtue of complexity' that the paper tests with model-complexity panel regressions.","marker":"Kelly et al. (2024)"},{"why":"Provides the weak-factor environment results on l2-norm and bagging that the paper verifies.","marker":"Shen and Xiu (2024)"},{"why":"Supplies the economic-driven ensemble weighting based on discounted MSE and the state-dependent R-squared analysis.","marker":"Rapach et al. (2010)"},{"why":"Provides the DM test used to assess statistical significance of out-of-sample R-squared values.","marker":"Diebold and Mariano (1995)"},{"why":"Provides the MSFE-adjusted test used as robustness for comparing nested and non-nested forecasts.","marker":"Clark and West (2007)"},{"why":"Documents expansion-concentrated predictability in the U.S. bond market, which the paper's night-light finding mirrors.","marker":"Bianchi et al. (2021)"}],"fun_headline_variants":["ML beats benchmarks for Singapore electricity price forecasts","52% out-of-sample R^2: ML for Singapore power prices","Machine learning unlocks power market predictability","Macro conditions drive ML electricity price gains","ML ensembles win in Singapore power forecasting"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The central results stand or fall on the claim that each day's feature values are constructed only from information available on that day; if the monthly (0,1) standardization uses the full calendar month's mean and standard deviation, then the out-of-sample predictions peek at future data within the very month being forecast.","fun_headline_variants_meta":{"raw":{"variants":["ML beats benchmarks for Singapore electricity price forecasts","52% out-of-sample R^2: ML for Singapore power prices","Machine learning unlocks power market predictability","Macro conditions drive ML electricity price gains","ML ensembles win in Singapore power forecasting"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000318,"raw_usage":{"total_tokens":1804,"prompt_tokens":959,"completion_tokens":845,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":575,"completion_tokens_details":{"reasoning_tokens":776}},"tokens_in":575,"tokens_out":845,"duration_ms":9137,"temperature":1.0,"reasoning_tokens":776,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T18:40:09.677290+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Recompute the full prediction pipeline with each day's features standardized using only data up to that day (e.g., trailing or expanding normalization), and compare the resulting R-squared values with the reported ones; if the correlation-penalized ensemble's $R^2_{OOS,mean}$ drops substantially below 51.92% or another model overtakes it, the reported predictability is inflated by look-ahead leakage. A single out-of-sample month where the gap exceeds the DM test's margin would settle the question.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the theoretical 'virtue of complexity' that the paper tests with model-complexity panel regressions."},{"cited_title":"and Xiu, D","cited_arxiv_id":null,"evidence_quote":"Provides the weak-factor environment results on l2-norm and bagging that the paper verifies."},{"cited_title":"E., Strauss, J","cited_arxiv_id":null,"evidence_quote":"Supplies the economic-driven ensemble weighting based on discounted MSE and the state-dependent R-squared analysis."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the DM test used to assess statistical significance of out-of-sample R-squared values."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the MSFE-adjusted test used as robustness for comparing nested and non-nested forecasts."}],"review_version":1}