{"id":"31aeeffd-7d27-41ec-88d8-6d3e91a97fa2","arxiv_id":"2505.23607","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"A three-part taxonomy (domain, contextual, behavioral) guides feature engineering that improves hour-ahead household electricity forecast accuracy by roughly 4 percentage points in MPE on three public datasets.","lead":"This paper proposes a taxonomy that divides electricity data features into domain specific, contextual, and behavioral groups, and tests whether using all three groups improves machine learning forecasts of household electricity use. It reports that taxonomy-guided features cut forecasting error by several percentage points on three public datasets, but some headline numbers in the abstract and conclusions do not match the results tables.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The headline MPE gains and SHAP shares depend on exactly how rolling/lag features are computed; Section 4.2 describes the split but not whether these features are causal, so temporal leakage could inflate all reported improvements.","rationale":"The reader identifies temporal leakage in the engineered features as the weakest assumption, and I agree. Both pillars of the central claim—the row-8-versus-row-1 MPE improvements and the SHAP feature-group contributions—are only meaningful if the rolling and lag features are causally valid. Section 4.2 explicitly addresses shuffling but is silent on how rolling/lag features are computed; Table 2 lists exactly the features where leakage would hide. The SHAP text/table mismatch is a corroborating sign that the quantitative layer was not carefully audited, but it is less load-bearing than leakage because it undermines the importance breakdown rather than the headline accuracy gain. The proposed check—recomputing all lag/rolling features from strictly past observations and re-running XGB—would settle the matter. In fairness, the datasets are standard, the algorithms are off-the-shelf, the taxonomy is well-motivated, and the qualitative finding that domain features dominate is consistent across tables; hence the appropriate verdict is conditional, not reject.","tokens_in":16891,"tokens_out":4850,"duration_ms":46018,"concrete_test":"Obtain or reimplement the feature-construction code and audit every past-consumption feature in Table 2. For each target hour t, recompute rolling average/std, consumption-23/24h-ago, consumption-a-week-ago, and yesterday-ratio features using only observations with time index strictly before t, after applying the 80:20 split; then retrain XGB on the identical split with the same hyperparameters and compare row 8 versus row 1 MPE deltas in Tables 3–5. If any of the three deltas drops below the reported 3.73/4.70/4.30 percentage points, or the SHAP domain-share ranges shift materially, the temporal-leakage concern is confirmed and the central quantitative claim must be downgraded. If the deltas and SHAP shares are reproduced under this causal construction, the claim stands.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central quantitative claim rests on comparing row 8 (all feature groups) with row 1 (raw series) in Tables 3–5 and on the SHAP group shares in Table 6. Section 4.2 says only that data are split 80:20 and not shuffled; it never states how the rolling average/std, 'consumption {23,24} hours ago', 'consumption a week ago', and 'yesterday consumption ratio' features in Table 2 are constructed relative to the split. If these features are computed on the full series before splitting, or with a centered or expanding window that includes the current or future value, then training samples can contain information from the test period, and test features can contain information that a real hour-ahead forecast would not have. That would inflate all three headline MPE improvements and make the SHAP importance percentages untrustworthy. A corroborating sign that the quantitative layer is unstable is that the SHAP percentages quoted in Section 5.2 and the abstract (e.g., domain 65.5/83.1/68.9 for HUE/UCI/REFIT) do not match Table 6 (66.5/83.8/68.2); several cells differ by more than 0.5 percentage points, and some abstract ranges do not cover the table entries. The feature-leakage question is therefore the most load-bearing: it determines whether the 3.73/4.70/4.30 percentage-point improvements are real or an artifact of the feature-construction procedure.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes a three-category taxonomy (domain-specific, contextual, behavioral) for electricity data features and uses it to engineer features for hour-ahead household electricity forecasting. Experiments on three public datasets (HUE, UCI Household, REFIT) with five machine learning models report that taxonomy-guided feature engineering improves mean percentage error (MPE) by approximately 3.73, 4.70, and 4.30 percentage points over raw time series, and a SHAP analysis attributes the largest contribution to domain-specific features (65.5%–83.1%).","tokens_in":17208,"tokens_out":4225,"duration_ms":34792,"significance":"If the reported results are reproducible, the taxonomy offers a practical, application-oriented guide for feature engineering and model selection in energy forecasting, and the SHAP-based feature-group analysis provides interpretability of model inputs. The study uses openly available datasets and transparent tabular results, which is a strength. However, the central quantitative claims—both the MPE gains and the SHAP shares—are sensitive to the feature-construction pipeline and to internal consistency of the reported numbers; these issues need to be resolved before the claims can be fully credited.","major_comments":[{"comment":"The manuscript does not state whether the rolling statistics (rolling average and standard deviation), lag features (consumption {23,24} hours ago, consumption a week ago), and 'yesterday consumption ratio/median' are computed using only data available at the forecast time (e.g., expanding windows on the training portion) or on the full series before the 80:20 split. If these features are computed with a centered window or using test-period data, the train/test separation is violated, and the headline MPE improvements in Tables 3–5 (row 8 vs row 1) and the SHAP contributions in Table 6 would be inflated. This is load-bearing because the abstract and Section 1 quantify the claimed benefit in percentage points. The authors must specify the feature-construction procedure and demonstrate that it is causal.","section":"Section 4.2, Table 2"},{"comment":"The SHAP group percentages reported in the text (e.g., HUE domain 65.5%, UCI domain 83.1%, REFIT domain 68.9%; contextual 23.6%, 10.8%, 17.9%; behavioral 10.9%, 6.1%, 13.2%) do not match the values in Table 6 (HUE domain 66.5%, contextual 23.0%, behavioral 10.5%; UCI domain 83.8%, contextual 9.9%, behavioral 6.3%; REFIT domain 68.2%, contextual 17.0%, behavioral 14.8%). The abstract's ranges (65.5%–83.1% for domain) also exclude the table's 83.8% for UCI. These inconsistencies undermine the reliability of the feature-importance analysis and must be reconciled before publication.","section":"Section 5.2, Table 6"},{"comment":"The claimed improvements of approximately 3.73, 4.70, and 4.30 percentage points compare row 8 (all feature groups) with row 1 (raw data only). Because row 1 contains only raw measurements while row 8 includes the full taxonomy-engineered feature set, the comparison conflates the effect of feature engineering with the effect of increasing input dimensionality. The paper should explicitly state that the baseline is raw univariate time series and discuss whether the improvement is due to the taxonomy guidance or simply to adding more variables.","section":"Section 5.1, Tables 3–5"},{"comment":"The manuscript says the taxonomy is 'validated' by the forecasting experiments, but the taxonomy itself is defined a priori and the experiments do not compare it against alternative feature groupings or a random baseline. The SHAP analysis is a post hoc explanation of models trained on the taxonomy-derived features, not a statistical test of the taxonomy's structure. The authors should soften the validation claim to 'demonstration' or add a control experiment (e.g., random feature grouping of equal size) to support the claim that the taxonomy specifically, rather than any feature set, is responsible for the improvements.","section":"Abstract, Section 3, Section 5"}],"minor_comments":[{"comment":"There are several typos, including 'we study of the effect' in the abstract, 'such as such as' in Section 4.1, and 'feature feature-engineered' in the Conclusions; these should be corrected.","section":"Section 1 and Section 4.1"},{"comment":"The MPE formula in Equation (1) is not the standard mean percentage error; the paper should clarify why the maximum of the absolute values is used in the denominator and define the metric precisely, as readers may otherwise misinterpret the reported values.","section":"Equation (1)"},{"comment":"The column heading 'cum. rel. SHAP contribution' is unclear; spell out 'cumulative relative SHAP contribution' and state whether the three group percentages sum to 100% for each dataset.","section":"Table 6"},{"comment":"The Conclusions state 'up to 6% MSE and up to 23% MPE improvement over using only feature-engineered domain-specific features,' which is a different baseline than the 3.73–4.30 percentage-point improvements over raw data; these two comparisons should be explicitly distinguished to avoid confusion.","section":"Section 6"}],"recommendation":"major_revision","confidential_remarks":"The discrepancy between the SHAP percentages in the text and Table 6 may indicate insufficient proofreading, but the temporal-leakage question is more serious and should be resolved by requesting the feature-engineering code or an explicit causal-window specification. If the leakage concern is confirmed, the headline improvements would be artifacts and the paper would not be publishable in its current form."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the taxonomy is a reasonable organizing device, and the three-dataset experiment is the right kind of effort, but the headline numbers are not yet defensible. You should not trust the specific MPE gains or SHAP shares until the feature-construction protocol and the numerical inconsistencies are cleaned up.\n\nWhat's new: the paper proposes a domain/contextual/behavioral split for energy forecasting features and validates it on three open datasets with several interpretable models. The taxonomy itself is not deep, but it is sensible and could serve as a useful checklist for practitioners. The SHAP-based grouping analysis is a nice touch, and using public data means someone can reproduce the experiment.\n\nSoft spots, in order of importance. First, the temporal leakage question is load-bearing. Section 4.2 describes an 80:20 split and says they don't shuffle to avoid leakage, but never says whether rolling averages, lags, and 'yesterday ratio' features are computed on the full series before splitting. If they are, the reported 3.73/4.70/4.30 point MPE improvements are inflated. The authors need to state explicitly that features are computed causally on the training portion only.\n\nSecond, the numbers don't line up. Table 6 gives SHAP contributions that differ from the abstract and Section 5.2 (e.g., HUE domain 66.5 vs 65.5, UCI 83.8 vs 83.1). The conclusion's 'up to 23% MPE improvement' is not supported by Tables 3-5; the row-8 vs row-2 differences are under one percentage point. That kind of sloppiness makes it hard to know which number is right.\n\nThird, minor: no error bars or repeated runs, which is noticeable given the small effect sizes for some comparisons.\n\nOverall: the qualitative findings—domain features dominate, extra behavioral data helps, contextual features support domain features—are consistent across datasets and likely robust. But the quantitative layer needs a careful revision. I'd send it to peer review, with the expectation that the authors fix the leakage protocol and reconcile the numbers. It's not ready as is.","headline":"Useful taxonomy and honest three-dataset effort, but the headline MPE gains and SHAP shares are not defensible until the feature-leakage protocol is clarified and the internal numbers are reconciled.","tokens_in":17753,"tokens_out":2346,"would_cite":false,"duration_ms":21424,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A three-way feature taxonomy lifts household load forecasts by up to 4.7 percentage points.","keywords":["electricity forecasting","feature taxonomy","explainable machine learning","feature importance","SHAP","smart meter data","behavioral features","contextual features"],"falsifier":"Re-run the XGB experiments on UCI Household and REFIT with each rolling average and lag feature recomputed inside a strictly expanding training window, so every feature at time t uses only observations before t; if the roughly 4.7 and 4.3 percentage-point MPE gains over raw time series largely disappear, the improvement was driven by look-ahead rather than by the taxonomy.","tokens_in":16675,"feed_emoji":"⚡","tokens_out":5490,"duration_ms":48089,"temperature":0.7,"pith_summary":"This paper argues that a structured taxonomy of energy data—splitting features into domain-specific measurements, contextual signals such as weather and building properties, and behavioral signals such as work schedules and cooking habits—should guide data model design for machine learning in electricity applications. The authors try to establish that building features this way materially improves hour-ahead household electricity forecasting compared with feeding a model raw consumption time series. On three open datasets, taxonomy-guided feature engineering improves mean percentage error by roughly 3.73, 4.70, and 4.30 percentage points for a gradient-boosted tree model. A SHAP-based importance analysis adds that domain-specific features carry most of the predictive weight (65.5–83.1 percent), with contextual features contributing 10.8–23.6 percent and behavioral features 6.1–13.2 percent. If true, the taxonomy gives practitioners a repeatable checklist for designing data models rather than relying on ad hoc feature selection.","feed_headline":"Feature design lifts load forecasts by up to 4.7 points","feed_subtitle":"Splitting energy data into domain, contextual, and behavioral features improves hour-ahead household load forecasts on three open datasets.","key_machinery":"The load-bearing object is a first-level taxonomy that assigns every candidate input to one of three groups: domain-specific features (direct electrical measurements plus derived lags and rolling statistics), contextual features (weather, time, geolocation, building properties), and behavioral features (work schedules, cooking windows, hygiene habits, encoded social activities). The taxonomy guides a feature-engineering pipeline that converts raw smart-meter series into tabular inputs, and SHAP Shapley-value attributions are used to quantify each group's contribution. The empirical work is carried by this pipeline plus the comparison protocol: an 80:20 chronological split, five algorithms, and MPE and MSE evaluation on three open datasets.","core_discovery":"The paper's central discovery is that the choice of which data to collect and engineer is itself a performance lever for explainable load forecasting. Guided by a three-category taxonomy (domain-specific, contextual, behavioral), the authors construct feature sets for three public household datasets and compare regression and time-series algorithms against the same models trained on raw consumption only. Feature engineering brings the best model's mean percentage error down by about 3.73 percentage points on HUE, 4.70 on UCI Household, and 4.30 on REFIT, and the full feature set usually outperforms any single category or pair. Feature importance analysis attributes most predictive contribution to domain-specific features, while showing that sub-metered appliance data improves the extraction of behavioral patterns and thereby forecasting accuracy.","pith_inferences":["A direct testable extension is to apply the same taxonomy-driven feature grouping to other energy tasks, such as photovoltaic generation forecasting or anomaly detection; if the taxonomy is general, the same category-level importance ordering should reappear.","The SHAP contribution percentages may partly reflect feature-count imbalance—UCI Household has 63 domain features versus 9 contextual ones—so a per-feature normalized attribution would separate signal strength from sheer input count.","Because contextual features gain importance when sub-meter-derived behavioral features are removed, one implied design strategy is to enrich contextual metadata in households where appliance-level data is unavailable.","The headline gains are reported on MPE; the tables show smaller relative improvements on MSE, so the practical benefit of the taxonomy is metric-dependent and should be weighed against the cost of collecting extra data."],"forward_implications":["Adding contextual and behavioral features to engineered domain features improves forecasts by up to 5 percent MSE and up to 3 percent MPE; combining all three groups yields up to 6 percent MSE and 23 percent MPE improvement over domain features alone.","Sub-metering data matters: removing it costs up to 13 percent MSE and 3 percent MPE in accuracy, so investing in sub-metering or non-intrusive load monitoring is a concrete route to better forecasting.","Gradient-boosted tree models fit taxonomy-derived tabular features better than time-series-native algorithms such as Prophet and N-BEATS, reinforcing competition evidence that preprocessing plus gradient-boosted ensembles performs best.","Behavioral features gain predictive value when richer sub-meter data is available, suggesting that future forecasting models will benefit from more granular household activity data.","The taxonomy can be encoded with semantic vocabularies for interoperability, pointing toward feature stores and database-like systems that reuse the same feature design across energy applications."],"supporting_citations":[{"why":"HUE dataset: supplies household energy consumption, EV battery data, weather, and building metadata used as one of the three evaluation datasets.","marker":"[55]"},{"why":"UCI Household dataset: supplies main-meter and sub-meter consumption with geolocation metadata for the forecasting evaluation.","marker":"[58]"},{"why":"REFIT dataset: supplies appliance-level sub-metering across 20 households, enabling the behavioral feature extraction the importance analysis depends on.","marker":"[50]"},{"why":"SHAP framework: provides the Shapley-value feature attribution method used to quantify per-group and per-feature contributions.","marker":"[66]"},{"why":"N-BEATS configuration: gives the deep time-series baseline architecture and settings used in the model comparison.","marker":"[65]"},{"why":"Post-COVID forecasting competition: cited to align the observed feature-engineering gains with prior large-scale forecasting findings.","marker":"[40]"},{"why":"Feature engineering study: supports the behavioral feature category by showing electricity consumption is influenced by occupant behavior.","marker":"[63]"},{"why":"Interpretable features taxonomy: motivates the need for a context-based feature taxonomy, the paper's starting point.","marker":"[15]"}],"fun_headline_variants":["Taxonomy-guided features lift load forecasts 4.7 points","Data design boosts hourly load forecasts by 4.7 points","Feature taxonomy improves explainable load forecasting","Domain, contextual, behavioral data cut forecast errors","Smart data choice lifts household load predictions"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported gains assume that lag and rolling statistics such as consumption 23 and 24 hours ago are computed only from the training portion of each household's time series, so no test-period information enters the model.","fun_headline_variants_meta":{"raw":{"variants":["Taxonomy-guided features lift load forecasts 4.7 points","Data design boosts hourly load forecasts by 4.7 points","Feature taxonomy improves explainable load forecasting","Domain, contextual, behavioral data cut forecast errors","Smart data choice lifts household load predictions"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000145,"raw_usage":{"total_tokens":1141,"prompt_tokens":870,"completion_tokens":271,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":486,"completion_tokens_details":{"reasoning_tokens":198}},"tokens_in":486,"tokens_out":271,"duration_ms":2906,"temperature":1.0,"reasoning_tokens":198,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T12:41:06.994174+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the XGB experiments on UCI Household and REFIT with each rolling average and lag feature recomputed inside a strictly expanding training window, so every feature at time t uses only observations before t; if the roughly 4.7 and 4.3 percentage-point MPE gains over raw time series largely disappear, the improvement was driven by look-ahead rather than by the taxonomy.","supporting_citations":[{"cited_title":"Makonin, Hue: The hourly usage of energy dataset for buildings in british columbia, Tech","cited_arxiv_id":null,"evidence_quote":"HUE dataset: supplies household energy consumption, EV battery data, weather, and building metadata used as one of the three evaluation datasets."},{"cited_title":"Hebrail, A","cited_arxiv_id":null,"evidence_quote":"UCI Household dataset: supplies main-meter and sub-meter consumption with geolocation metadata for the forecasting evaluation."},{"cited_title":"Murray, L","cited_arxiv_id":null,"evidence_quote":"REFIT dataset: supplies appliance-level sub-metering across 20 households, enabling the behavioral feature extraction the importance analysis depends on."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"SHAP framework: provides the Shapley-value feature attribution method used to quantify per-group and per-feature contributions."},{"cited_title":"Mostafa, B","cited_arxiv_id":null,"evidence_quote":"Post-COVID forecasting competition: cited to align the observed feature-engineering gains with prior large-scale forecasting findings."},{"cited_title":"Spichakova, J","cited_arxiv_id":null,"evidence_quote":"Feature engineering study: supports the behavioral feature category by showing electricity consumption is influenced by occupant behavior."},{"cited_title":"Zytek, I","cited_arxiv_id":null,"evidence_quote":"Interpretable features taxonomy: motivates the need for a context-based feature taxonomy, the paper's starting point."}],"review_version":1}