{"id":"bbfd24fb-5aae-4c1e-8ccf-6a133672a710","arxiv_id":"2509.05772","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"On 55 days of UK household data, SPO+ decision-focused learning with automated features gave the lowest battery-scheduling regret, though gains over no-AFE were not statistically robust.","lead":"This paper tests whether decision-focused learning can beat predict-then-optimize for home battery scheduling when only 55 days of data exist, and whether automated feature engineering helps. On one UK household dataset, SPO+ with automated features had the lowest regret, but the statistical support for the feature boost is weak.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Regret evaluation ignores demand uncertainty in the BESS constraints; SPO+ loss in Eq. 14 only handles objective-coefficient uncertainty, so the reported 67% improvement may be an evaluation artifact.","rationale":"Pith Reader identifies dataset representativeness as the weakest assumption; I agree that is a limitation, but it is not the most load-bearing issue. The stronger problem is that the paper's own evaluation metric may be inconsistent with the problem it claims to solve. The BESS optimization has two uncertain inputs: price in the objective and demand in the constraints. Eq. 14 defines regret only over cost-vector uncertainty. The paper does not explain how predicted demand is incorporated into the SPO+ loss or how a schedule computed under predicted demand is scored under actual demand. Without a recourse/simulation step, the reported regrets compare the objective value of a solution to a differently-constrained problem. This affects both training and evaluation: if SPO+'s surrogate is invalid for RHS uncertainty, the learned model is not minimizing true decision loss, and the 67% improvement could reflect a mismatch in the evaluation rather than better decisions. The authors provide code and ten runs, which is good, but the code would need to be checked to settle this. Because the manuscript as written does not provide the necessary detail, the central claim is unverified rather than clearly wrong; hence UNVERDICTED.","tokens_in":16400,"tokens_out":7721,"duration_ms":96535,"concrete_test":"Run a diagnostic on the released code/experiment: for each test day, compute the schedule from the trained SPO+(AFE) model's predicted price and demand, then simulate the battery under actual price and actual demand with a simple recourse rule (purchase any unmet demand from the grid at the actual price). Recompute normalised regret as (simulated actual cost - true optimal cost)/sum|z*|. If SPO+(AFE)'s regret rises to the level of PTO(AFE) or above, the reported advantage is an evaluation artifact rather than evidence for DFL viability.","verdict_should_be":"UNVERDICTED","load_bearing_attack":"The central claim rests on the regret numbers in Table 4. Equation 14 defines regret as c_i^T w*(c_hat) - z*(c_i), i.e., only the objective coefficient vector c is predicted. But the BESS MILP (Eqs. 1–12) has both price and demand unknown: prices enter the objective, while demand d_t appears as a right-hand side in constraint (2) and as an upper bound on battery discharge via g_t >= 0. When the model forecasts d_t, the 'optimal' schedule w*(c_hat) is computed under predicted demand, not actual demand. Evaluating that schedule with c_i^T w*(c_hat) does not give the true operating cost under actual demand: if the predicted demand is lower than actual, the schedule may discharge more than the actual demand (b_t > d_t), making (2) infeasible, and the cost expression ignores the need for recourse grid purchases. No simulation or recourse step is described in Section 3.4 or the experimental procedure. The SPO+ surrogate loss is derived under the assumption that the feasible region is fixed and only objective coefficients vary; applying it to a problem with demand uncertainty in the constraints has no stated justification. If demand errors are not properly accounted for, the reported mean regrets over the 14 test days are not valid decision costs, and the 67.16% improvement of SPO+(AFE) over PTO(AFE) could be an artifact of comparing solutions to different optimization problems.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes an AFE–DFL framework for residential battery energy storage scheduling. It couples the authors' AutoEnergy feature-engineering algorithm with three neural-network predictors (PTO, SPO+, DBB) and a MILP battery scheduler, then evaluates normalized regret on 14 held-out days from a 55-day UK property dataset. The headline result is that SPO+ with AFE achieves a mean normalized regret of 0.0672 versus 0.2046 for PTO with AFE, and that AFE improves each method by 19.42–56.48%. Non-parametric tests with Bonferroni corrections are reported.","tokens_in":16802,"tokens_out":8173,"duration_ms":90221,"significance":"The paper has clear strengths: it targets a realistic small-data regime in which DFL is rarely evaluated, uses a novel real-world dataset, ships code and data for reproducibility, and is transparent in reporting adjusted p-values. If the regret evaluation is valid, the SPO+ result would be a practically useful demonstration of decision-focused training for BESS. However, the validity of the central numerical evidence is currently undermined by an unspecified treatment of demand uncertainty in the regret computation, and the abstract's global claims are stronger than the reported statistics support. The contribution is therefore still conditional on a corrected evaluation.","major_comments":[{"comment":"The regret metric is defined only over predicted cost vectors c_hat: L_Regret(ĉ_i, c_i)=c_i^T w*(ĉ_i)−z*(c_i). But the BESS MILP has demand d_t as a right-hand-side parameter in constraint (2) and as a bound on discharge through b_t ≥ 0. The manuscript never states how predicted demand enters w*(ĉ_i) or how a schedule generated under predicted demand is evaluated against actual demand. If constraint (2) uses predicted demand, the schedule is generally infeasible for actual demand, and c_i^T w*(ĉ_i) omits the cost of recourse grid purchases. If constraint (2) uses actual demand, then demand is not actually being forecast. Either way, the regret values in Table 4 and the 67.16% improvement in §4.1 are not established. SPO+ theory assumes a fixed feasible set with uncertainty only in objective coefficients; applying it to constraint-side demand uncertainty requires explicit justification. T","section":"§3.4, Eq. (14); §2.2, constraints (1)–(12)"},{"comment":"The statement that 'DFL yields lower operating costs than PTO' is contradicted by the DBB results. In Table 4, DBB with AFE has mean regret 0.2458 versus PTO with AFE at 0.2046, and DBB without AFE has 0.3187 versus PTO without AFE at 0.2539. Thus only SPO+ supports the claim that a decision-focused method outperforms PTO. The abstract and conclusions should be qualified to SPO+ specifically or should explicitly state that DFL methods vary, with DBB underperforming PTO in this setting.","section":"Abstract; §4.1, Table 4"},{"comment":"The AFE-improvement claim, which is a stated contribution, is not statistically robust after multiple-comparison correction. The Bonferroni-adjusted within-method comparisons are not significant: SPO+ (AFE) vs SPO+ (No AFE) has adjusted p=1.000, PTO has p=0.508, and DBB has p=0.302. Moreover, SPO+ (AFE) vs PTO (AFE) has adjusted p=0.070. The 22.9–56.5% improvements are mean differences over 14 test days. The abstract's unqualified 'adding AFE further improves the performance of DFL methods' overstates the evidence. The authors should report effect sizes with confidence intervals, explicitly discuss the low power of pairwise tests on 14 days, and soften the causal wording accordingly.","section":"§4.3, Table 5; §4.2"}],"minor_comments":[{"comment":"The notation z*(c_i) and w*(ĉ_i) is ambiguous because the MILP also depends on demand d. Please define the optimization value and solution as functions of all exogenous parameters that affect the feasible region, not only the cost vector c.","section":"§3.4, Eq. (14)"},{"comment":"The description of 'nested window features' is vague. The paper refers to [28] for details, but the method should be sufficiently self-contained for a reader to know what features are generated (e.g., window lengths, aggregation functions).","section":"§2.4"},{"comment":"The 55-day dataset is from a single UK property over winter months. This limit on generalisability is acknowledged in the conclusion, but it should also appear in the abstract or introduction since it directly conditions the scope of the empirical claims.","section":"§3.1, §5"},{"comment":"The GitHub link for code and data appears only in a footnote. If this is an availability statement, it should be a labelled data-availability section in the main text.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The demand-uncertainty gap in §3.4 is the most serious issue: the reported regret numbers may be invalid because the evaluation does not describe how schedules are assessed under actual demand. This may require re-running the experiments with a proper simulator/recourse model rather than a purely textual revision. If the corrected evaluation no longer shows an SPO+ advantage, the paper should be withdrawn, not revised. The abstract's 'DFL yields lower operating costs' and AFE-improvement claims are also too strong relative to Table 4 and Table 5."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThis paper is worth a look mainly as a case study in how hard it is to validate decision-focused learning outside synthetic benchmarks. The authors compare SPO+, DBB, and PTO on a real 55-day UK household BESS dataset, with and without their AutoEnergy feature engineering. What is genuinely new: a real-world small-data evaluation of DFL, with code and data on GitHub, and an unusually honest statistical analysis that admits the AFE benefits are not robust after Bonferroni correction. The writing is clear, the limitations section is candid, and the DBB underperformance is acknowledged rather than hidden.\n\nThe headline result is that SPO+ with AFE achieves mean normalized regret 0.067 versus PTO's 0.205, a 67% improvement. DBB does worse than PTO, which the authors acknowledge. The problem is the evaluation metric. Equation 14 defines regret only in terms of the predicted cost vector, implicitly treating the feasible region as fixed. But in their MILP, demand appears as a right-hand side in the demand satisfaction constraint and as a bound on battery discharge. When the model forecasts demand, the optimal schedule is computed under predicted demand, not actual demand. The regret calculation as described does not appear to include recourse or re-optimization for the demand error. If the schedule is infeasible under true demand, the reported cost is not the true operating cost, and the 67% improvement could be an artifact of comparing solutions to different problems. The paper does not describe any simulation or recourse step. This is not a nitpick; it cuts to the central claim.\n\nThere are also smaller issues: a single 55-day dataset from one property, a 14-day test period, and AFE benefits that do not survive multiple-comparison correction. But those are disclosed, and the authors handle them with appropriate caution.\n\nBottom line: the framework is reasonable and the study is honestly conducted, but the evaluation flaw needs fixing before the numbers can be trusted. I would not cite the quantitative results as they stand. The paper does deserve a serious referee because the topic is timely and the framework is useful, but I would expect major revision.\n\nRecommendation: send to peer review, but flag the evaluation-metric issue prominently.","headline":"Interesting real-world DFL study with honest statistics, but the regret evaluation ignores demand uncertainty in the constraints, which undermines the headline cost savings.","tokens_in":17302,"tokens_out":2810,"would_cite":false,"duration_ms":32921,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Decision-focused learning plus automated features cuts battery scheduling regret by 67 percent.","keywords":["decision-focused learning","battery energy storage systems","automated feature engineering","predict-then-optimize","SPO+","regret minimization","energy arbitrage","data scarcity"],"falsifier":"A decisive test would be to run the same SPO+ with and without AFE against PTO on a broader sample—for example, a full year of data from multiple households or several tariff structures—and compute normalised regret per property. If SPO+ with AFE does not consistently beat PTO with AFE on that broader sample, or if PTO matches it under certain tariffs, the general claim that DFL is practically viable under data scarcity would fail. A cheaper check is to hold out a different 14-day window from the same property and see whether the 67 percent reduction persists or was specific to the chosen Febr","tokens_in":16344,"feed_emoji":"🔋","tokens_out":4637,"duration_ms":49825,"temperature":0.7,"pith_summary":"The paper tries to show that decision-focused learning (DFL), which trains a forecaster to minimize the cost of the downstream battery schedule rather than prediction error, works on real-world battery storage problems with very little data, and that automated feature engineering makes it work better. On a 55-day dataset from one UK property, the best DFL method (SPO+) with automated features achieved a mean normalised regret of 0.0672, about 67 percent lower than the standard predict-then-optimise (PTO) baseline with the same features (0.2046). The authors also report that adding automated features improved DFL methods by 22.9 to 56.5 percent relative to the same models without them. If true, this matters because many real energy storage deployments cannot collect years of data and lack domain expertise for hand-crafted features; the claim is that task-aligned training plus automatically generated features can still yield cheaper battery schedules.","feed_headline":"Decision-focused learning cuts battery scheduling regret by 67 percent","feed_subtitle":"A 55-day UK household test shows task-aligned training plus auto-engineered features beats predict-then-optimize.","key_machinery":"The load-bearing components are: (1) the normalised regret metric, which scales the extra cost of decisions made under imperfect forecasts by the true optimal objective, so training optimises decision quality rather than forecast error; (2) the SPO+ convex surrogate loss, which upper-bounds the regret of predicted cost vectors and enables gradient-based training through the MILP solver; (3) DBB, a differentiable black-box approximation used as a second DFL baseline; and (4) AutoEnergy, the automated feature engineering algorithm that generates temporal, lag, and rolling-window features from timestamps and historical data, reducing reliance on manual domain expertise. The optimisation layer i","core_discovery":"The central claim is that aligning prediction with the optimisation objective—training neural networks with a regret-based loss that measures the extra cost caused by imperfect forecasts—is practically viable for real-world battery energy storage system (BESS) scheduling under data scarcity, and that automated feature engineering amplifies this benefit. The authors formulate the BESS scheduling problem as a mixed-integer linear program (MILP) that minimises electricity cost subject to demand, battery capacity, charging/discharging limits, and state-of-charge constraints. They compare three methods: PTO trained with mean squared error, and two decision-focused methods, SPO+ and DBB, trained w","pith_inferences":["The 55-day, single-property evaluation makes the quantitative regret values a proof of concept rather than a reliable effect size; the ordering could shift on data from different seasons, climates, or tariff structures.","A testable extension is to run the same AFE-DFL pipeline on a multi-property or multi-market dataset and check whether SPO+ with AFE retains its edge, and whether the large regret spikes on February 22-23 are noise or systematic.","The SHAP finding that lag and rolling-window features dominate importance suggests that automatic lag selection could be a cheap, transferable fix for PTO baselines too, even before adopting a decision-focused loss.","Because the paper lists solar integration as future work, a natural follow-up is whether AFE-DFL also reduces regret when the optimisation includes renewable generation, export decisions, and battery degradation constraints."],"forward_implications":["If the claim holds, energy storage operators with small datasets can obtain cheaper charge/discharge schedules by training on downstream cost rather than on forecast error.","Automated feature engineering can substitute for manual domain-knowledge engineering in data-scarce BESS settings, since AFE improved all methods and never degraded performance in the paper's runs.","The result implies that method choice within DFL matters: SPO+ benefited strongly while DBB did not, so DFL should not be treated as a single monolithic approach.","Because AFE also improved the PTO baseline, the finding suggests that cheaper schedules can come partly from better features independent of the learning objective, and partly from aligning the loss with the decision task.","The framework is presented as adaptable to other energy management problems with uncertain prices and demand, provided the optimisation constraints and objective are reformulated."],"supporting_citations":[{"why":"Supplies the SPO+ convex surrogate loss that makes regret differentiable and upper-bounds decision error for linear and mixed-integer optimisation.","marker":"[13]"},{"why":"Supplies the differentiable black-box (DBB) method used as the second decision-focused learning baseline.","marker":"[14]"},{"why":"Supplies the PyEPO implementation used to train with decision losses and to compute the normalised regret metric.","marker":"[35]"},{"why":"Supplies the AutoEnergy algorithm that generates the automated features evaluated in the paper.","marker":"[28]"},{"why":"Introduces the decision-focused learning idea of integrating prediction and optimisation, motivating the paper's objective.","marker":"[7]"},{"why":"Defines regret-based smart predict-and-optimise and underlies the decision-focused losses used in training.","marker":"[11]"},{"why":"Provides a prior real-world DFL application to storage reserve markets, the comparison point the paper extends to a much smaller dataset.","marker":"[19]"},{"why":"Provides the predict-plus-optimise benchmark framing for renewable energy scheduling with real data, contextualising the paper's real-world evaluation.","marker":"[5]"}],"fun_headline_variants":["Auto features boost decision-focused battery scheduling","DFL plus auto features cuts battery costs up to 56%","Task-aligned training beats predict-then-optimize for BESS","Real-world test: decision-focused learning slashes storage costs","Automated features amplify decision-focused energy storage gains"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The central result rests on a single 55-day dataset from one UK household, with 41 days for training and validation and 14 consecutive days for testing; if those 14 test days are not representative of other homes, seasons, or markets, the reported regret ordering may not transfer.","fun_headline_variants_meta":{"raw":{"variants":["Auto features boost decision-focused battery scheduling","DFL plus auto features cuts battery costs up to 56%","Task-aligned training beats predict-then-optimize for BESS","Real-world test: decision-focused learning slashes storage costs","Automated features amplify decision-focused energy storage gains"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001346,"raw_usage":{"total_tokens":5339,"prompt_tokens":816,"completion_tokens":4523,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":560,"completion_tokens_details":{"reasoning_tokens":4443}},"tokens_in":560,"tokens_out":4523,"duration_ms":37783,"temperature":1.0,"reasoning_tokens":4443,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T04:57:25.208752+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A decisive test would be to run the same SPO+ with and without AFE against PTO on a broader sample—for example, a full year of data from multiple households or several tariff structures—and compute normalised regret per property. If SPO+ with AFE does not consistently beat PTO with AFE on that broader sample, or if PTO matches it under certain tariffs, the general claim that DFL is practically viable under data scarcity would fail. A cheaper check is to hold out a different 14-day window from the same property and see whether the 67 percent reduction persists or was specific to the chosen Febr","supporting_citations":[{"cited_title":"Differentiation of blackbox combinatorial solvers","cited_arxiv_id":null,"evidence_quote":"Supplies the differentiable black-box (DBB) method used as the second decision-focused learning baseline."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the PyEPO implementation used to train with decision losses and to compute the normalised regret metric."},{"cited_title":"Bowler, Direnc Pekaslan, Nicholas J","cited_arxiv_id":null,"evidence_quote":"Supplies the AutoEnergy algorithm that generates the automated features evaluated in the paper."},{"cited_title":"Melding the data-decisions pipeline: Decision-focused learning for combinatorial optimization.Proceedings of the AAAI Conference on Artificial Intelligence, 33(01):1658–1665, 2019","cited_arxiv_id":null,"evidence_quote":"Introduces the decision-focused learning idea of integrating prediction and optimisation, motivating the paper's objective."},{"cited_title":"Stuckey, and Tias Guns","cited_arxiv_id":null,"evidence_quote":"Defines regret-based smart predict-and-optimise and underlies the decision-focused losses used in training."},{"cited_title":"Aguado, and Franc ¸ois Vall´ee","cited_arxiv_id":null,"evidence_quote":"Provides a prior real-world DFL application to storage reserve markets, the comparison point the paper extends to a much smaller dataset."},{"cited_title":"Predict+optimize problem in renewable energy scheduling","cited_arxiv_id":null,"evidence_quote":"Provides the predict-plus-optimise benchmark framing for renewable energy scheduling with real data, contextualising the paper's real-world evaluation."}],"review_version":1}