REVIEW 4 major objections 5 minor 11 references
On the global feature importance for interpretable and trustworthy heat demand forecasting
T0 review · 4 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read The paper claims that four global XAI methods, read together, can validate a black-box heat-demand forecasting model, with the ALE curve's resemblance to the district-heating control curve as the central evidence.
desk verdict A workmanlike XAI case study for heat demand forecasting; the feature-importance discussion is plausible, but the trustworthiness claim rests on a visual resemblance that cannot carry it. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The machinery that carries the argument is the Accumulated Local Effects (ALE) plot, a global post-hoc explanation method that estimates how small local changes in a feature shift the model prediction while conditioning on the observed data distribution; unlike partial dependence, it does not assume features are independent. The ALE curve for ambient temperature is what the paper compares with the district-heating control curve, and that visual resemblance is the stated evidence of model quality. Around it, three complementary mechanisms supply the broader picture: XGBoost's intrinsic gain, cover, and frequency importances; partial dependence plots with individual conditional expectation (ICE) lines; and SHAP values aggregated over the test set.
What would settle it
Train a deliberately wrong heat-demand model on the same data, for example a model that ignores ambient temperature or one that predicts only the daily average, and run the same four explanation methods: if its ambient-temperature ALE curve still resembles the control curve, or if the trustworthy model's resemblance disappears when evaluated on a new heating season, then the resemblance test cannot separate good from bad models.
Extended reading notes
Core claim
The central claim is that a heat-demand forecasting model trained with gradient boosting can be judged trustworthy by interpreting its global feature importances in the context of district-heating operation. Read together, the four views tell one coherent story: ambient temperature ($t_{amb}$) is the most frequently used split in the model, yet the largest accuracy gains come from transmitted energy at the same hour of the previous day, reflecting strong daily seasonality; SHAP ranks current transmitted energy and hour of day as the largest influences on predictions; and the partial-dependence curve for the previous-day energy is flat on average but shows in ICE plots that the feature matters mainly at the start of daily operation. The load-bearing piece is the ALE curve for ambient temperature, which falls almost linearly below about 9 °C and then more steeply with nonlinearities, closely following the shape of the DHS control curve that relates secondary supply water temperature to ambient temperature. The paper calls this resemblance strong evidence of the forecasting model's quality, and concludes from it, together with the expert-plausible readings of the other methods, that the otherwise black-box model is transparent enough to support trust in automated district-heating control.
Load-bearing premise
The argument stands on accepting that the visual resemblance between the ALE curve of ambient temperature and the district-heating control curve is strong evidence of forecasting-model quality; the paper provides no quantitative similarity criterion, no baseline comparison with a deliberately wrong model, and no test on an unseen heating season to establish that this resemblance could not arise from a bad model.
Editorial extensions
If this is right
- If the resemblance argument holds, operating staff can use ALE-based validation to justify deploying or retaining a black-box forecaster without a separate quantitative comparison on unseen heating seasons.
- Because every chosen method avoids feature permutation and perturbation, the same four-view suite can be applied in regulated settings where explanations must not depend on synthetic, out-of-distribution feature values.
- The complementary reading resolves apparent contradictions such as the flat partial dependence of previous-day energy despite its high gain: the feature is important only during the opening hours of daily operation, a nuance visible in ICE plots.
- The interpretive template should transfer to other district-heating substations and to other process industries that have a known control curve, allowing domain experts to validate models by checking that ALE shapes mirror engineering knowledge.
Reading between the lines
- The paper's trustworthiness conclusion could be strengthened by replacing visual resemblance with a numeric similarity score between the ALE curve and the control curve over the operating temperature range; this is an extension the paper does not perform.
- The no-permutation criterion is applied at the global level, but SHAP's model-agnostic variants sample feature coalitions and can still generate unrealistic instances; checking whether exact tree-based SHAP avoids this would test how strictly the paper's motivation carries through.
- The same suite could be run on a model trained without time-lagged demand features; if the ALE temperature curve still matched the control curve, it would show whether the ambient-temperature signal is learned directly or through correlated demand features.
- A natural next experiment is to apply the ALE-versus-control-curve check to forecast horizons beyond one hour, where ambient temperature plays a larger relative role; the paper's data are hourly and do not address this.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents a case study of global feature importance for an XGBoost heat demand forecasting model trained on four heating seasons of SCADA and meteorological data from a district heating substation. Four approaches are used: XGBoost's intrinsic gain/cover/frequency metrics, partial dependence and ICE plots, accumulated local effects, and SHAP. The author interprets the resulting importances in terms of district heating operation and argues that agreement between the ALE curve for ambient temperature and the DHS control curve constitutes strong evidence of model quality, thereby supporting the trustworthiness of the black-box forecasting model. The paper explicitly avoids permutation- and perturbation-based XAI methods and discusses complementarities and limitations of the selected techniques.
Significance. If the evidential claim were established, the paper would offer a useful demonstration template for validating forecasting models in regulated district heating operations without relying on model perturbation. The use of four complementary interpretation tools, the explicit acknowledgement of the independence assumption in PD, and the domain-grounded discussion of lagged demand and temperature features are positive features. However, the demonstration currently lacks quantitative validation, does not provide code or data, and the key conclusion rests on a qualitative visual analogy rather than a measured comparison; these gaps must be addressed before the trustworthiness claim can be accepted.
major comments (4)
- [Section 4] The conclusion that the resemblance between the ALE plot for tamb and the DHS control curve is 'strong evidence of quality of forecasting model' is not supported by the evidence presented. The control curve is the operating policy according to which supply-water temperature is a decreasing function of ambient temperature; heat demand measured under that policy will display the same decreasing dependence in any model that has learned the historical relation. A decreasing ALE(tamb) is therefore a necessary sanity check, not a validation of forecast quality. The comparison is also unit-incompatible, since the ALE y-axis is predicted heat demand while the control curve y-axis is supply-water temperature. Please add a quantitative check, for example comparing ALE(tamb) with the empirical conditional expectation E[heat demand | tamb] on a held-out season, and report test-set MAE or a similar accuracy metric against a baseline.
- [Section 2 and Abstract] Partial Dependence, Accumulated Local Effects, and SHAP are post-hoc model-agnostic interpretation methods; describing them as 'ante-hoc' is a conceptual error. The abstract and Section 2 claim as the paper's contribution an 'ante-hoc Explainable AI methodology', while the methods are applied after training to interpret a black-box model. This misclassification should be corrected throughout the manuscript, including the first paragraph of Section 2, the second paragraph of Section 2, and the conclusions, and the claimed novelty should be reframed accordingly.
- [Section 2, data split] The description of the data split ('80% of available data is used for training and the remaining 20% is used for testing') does not state whether the split is chronological or random. For a forecasting model with 23-hour lagged features, a random split can place test instances temporally before or between training instances, allowing leakage through lagged target and temperature values. Please report the exact split rule and, if the split is not chronological, repeat the analysis with a temporal hold-out; this is essential for the trustworthiness claim.
- [Sections 3.4 and 4] The statement that SHAP 'is the most reliable and robust method', and the claim that this is 'proven' by the case study, are unsupported. The Shapley-value foundation gives SHAP desirable axiomatic properties, but it does not by itself establish stability or reliability on this particular model. No bootstrap or repeated-sampling analysis, no comparison with alternative attribution methods on known ground truth, and no error bars on the reported importances are provided. Please either remove the claim or support it with quantitative evidence, such as variance of SHAP values across bootstrap samples or a comparison of SHAP-based rankings with gain and ALE rankings.
minor comments (5)
- [Figure 5] The caption of Figure 5 says 'ALE plots for the selected features in the dataset', but the figure shows a SHAP bar plot and a SHAP summary plot; the caption should be corrected.
- [Section 3.4, Eq. (2)] The SHAP formula uses f(S) as the model prediction given a subset of features, which can be read as implying model refitting or a well-defined function on arbitrary subsets; for a trained XGBoost model SHAP values are usually computed with conditional expectations. Please clarify the definition to avoid ambiguity.
- [Section 2] The hyperparameter configuration of the XGBoost model is not reported beyond the number of trees; listing the learning rate, maximum depth, and regularization parameters would improve reproducibility.
- [References] Reference [6] is cited as a Springer volume without volume, page, or DOI details; if the work is published, the full bibliographic information should be provided because the data preprocessing pipeline is essential to the present study.
- [General] No code or data availability statement is included; providing the preprocessing and analysis code, or at least a detailed data description, would allow the reported global importance plots to be reproduced.
Circularity Check
No construction-level circularity: XAI methods summarize a fitted model; the ALE-vs-control-curve resemblance is an inductive validation claim, not an equation that reduces to its own input.
full rationale
The paper's derivation chain is not circular. A gradient-boosting model is trained on SCADA and meteorological data (Section 2); the four global-importance analyses (XGBoost gain/cover/weight, PD/ICE, ALE, SHAP) are standard post-hoc summaries of the trained model, not parameters fitted and then renamed as predictions. The central trustworthiness inference in Section 4 is qualitative: "This behavior closely resembles the shape of the control curve... the resemblance is actually the strong evidence of quality of forecasting model." That is an inductive, domain-knowledge argument rather than a derivation, and no equation in the paper makes the model's quality equivalent to the ALE shape by construction. The self-citations ([1], [6]) supply prior context and the data-preprocessing pipeline, but the XAI analysis itself does not rely on those citations for its conclusions; the ALE, PD, and SHAP methods are cited to independent literature ([8]-[10]). Thus there is no fitted-input-called-prediction step, no self-citation chain that forces the conclusion, and no uniqueness theorem imported from the authors. The main weakness is evidentiary, not circular: no temporal hold-out accuracy is reported, and the ALE/control-curve comparison is unit-incompatible and qualitative, so the trustworthiness claim is under-supported. Under the hard rules, that concern belongs to correctness risk, not circularity; therefore the circularity score is 0.
Assumptions & free parameters
assumptions (4)
- domain assumption The preprocessing and feature engineering pipeline from [6] is assumed valid and transferable to this dataset.
- domain assumption Visual resemblance between the ALE curve and the DHS control curve is evidence of forecasting model quality.
- standard math Shapley value formula correctly attributes prediction contributions for the XGBoost model.
- domain assumption The trained XGBoost model on four seasons of one substation is representative of general DHS behavior.
Cite this review
Pith. "Pith review of On the global feature importance for interpretable and trustworthy heat demand forecasting." pith.science (2026). https://pith.science/paper/35GUB275
@misc{pith2026260813039,
author = {Pith},
title = {Pith review of: On the global feature importance for interpretable and trustworthy heat demand forecasting},
year = {2026},
howpublished = {\url{https://pith.science/paper/35GUB275}},
note = {Machine review of arXiv:2608.13039}
}
read the original abstract
The paper introduces the ante-hoc Explainable AI methodology to assess the global feature importance of the Machine Learning models used for heat demand forecasting in intelligent control of District Heating Systems, with motivation to facilitate their interpretability and trustworthiness, hence addressing the challenges related to adherence to communal standards, customer satisfaction and liability risks. Methodology includes use of four different approaches, namely intrinsic interpretability of Gradient Boosting method and selected post-hoc methods, namely Partial Dependence, Accumulated Local Effects and SHAP. None of the selected methods assume feature permutation or perturbations which can introduce bias due to introduction of random unrealistic values of data instances. Discussion of results is provided, including the assessment of complementarities where applicable, with specific interpretations in context of the district heating processes.
Figures
Figures from the paper (2 more)
Reference graph
Works this paper leans on
-
[1]
Milan Zdravkovi´c, Ivana ´Ciri´c, and Milan Ignjatovi´c. Explainable heat demand forecasting for the novel control strategies of district heating systems.Annual Reviews in Control, 53:405–418, 2022
work page 2022
-
[2]
J. Runge and Etienne S. A comparison of prediction and forecasting artificial intelligence models to estimate the future energy demand in a district heating system.Energy, 269:126661, 2023
work page 2023
-
[3]
Z. Wei, T. Zhang, B. Yang, Y . Dong, R. Xu, R. Wang, and X. Zhang. Prediction of residential district heating load based on machine learning: A case study.Energy, 231:120950, 2021
work page 2021
-
[4]
Long short-term memory.Neural Computation, 9(8):1735–1780, 1997
Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory.Neural Computation, 9(8):1735–1780, 1997
1997
-
[5]
M. Gong, B. Yin, J. Qin, W. Jin, Y . Peng, and W. Sheng. Gradient boosting machine for predicting return temperature of district heating system: A case study for residential buildings in tianjin.Journal of Building Engineering, 27:100950, 2020
work page 2020
-
[6]
Milan Zdravkovi´c. XGBoost “is all you need”: The Case of Forecasting Transmitted Heat Energy in District Heating Systems. InIntelligent Systems and Applications, Lecture Notes in Networks and Systems. Springer, 2024
work page 2024
-
[7]
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. “why should i trust you?”: Explaining the predictions of any classifier. InProceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 1135–1144, 2016
work page 2016
-
[8]
Friedman
Jerome H. Friedman. Greedy function approximation: A gradient boosting machine.The Annals of Statistics, 29(5):1189–1232, 2001
2001
Show all 11 references
-
[9]
Apley and Jingyu Zhu
Daniel W. Apley and Jingyu Zhu. Visualizing the effects of predictor variables in black box supervised learning models.Journal of the Royal Statistical Society: Series B, 82(4):1059–1086, 2020
2020
-
[10]
Lundberg and Su-In Lee
Scott M. Lundberg and Su-In Lee. A unified approach to interpreting model predictions.Advances in Neural Information Processing Systems, 30, 2017. Originally available as arXiv:1705.07874. 8 On the global feature importance for interpretable and trustworthy heat demand forecasting
2017 arXiv
-
[11]
Aaron Fisher, Cynthia Rudin, and Francesca Dominici. All models are wrong, but many are useful: Learning a variable’s importance by studying an entire class of prediction models simultaneously.Journal of Machine Learning Research, 20(177):1–81, 2019. Originally available as ar...
2019 arXiv
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.