Feature Importance Depends on Properties of the Data: Towards Choosing the Correct Explanations for Your Data and Decision Trees based Models

Benjamin Bosch; C\'elia Wafa Ayad; Jesse Read; Sonali Parbhoo; Thomas Bonnier

REVIEW 3 major objections 6 minor 37 references

Feature Importance Depends on Properties of the Data: Towards Choosing the Correct Explanations for Your Data and Decision Trees based Models

T0 review · 3 major / 6 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Feature-importance estimates from local explainers depend on data properties such as correlation, noise, and imbalance, so no single explanation method is reliable across all datasets.

desk verdict A well-intentioned but flawed benchmark: the ground truth for feature importance is defined against the data-generating process rather than the model being explained, so the quantitative rankings of explainers do not hold. read the letter →

arxiv 2502.07153 v1 pith:BNRUOJ7M submitted 2025-02-11 cs.LG cs.AI

Célia Wafa Ayad , Thomas Bonnier , Benjamin Bosch , Sonali Parbhoo , Jesse Read This is my paper · ORCID

classification cs.LGcs.AI

keywords ExplainabilityDecisionTreeModelsFeatureImportanceSyntheticDataGenerationLIMESHAPInterpreterLocalexplanations

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that local feature-importance explanations for decision-tree models are not universally trustworthy: which method is accurate depends on properties of the data being explained. To make this point, the authors generate synthetic binary-classification datasets from two ground-truth functions (XOR and NOT) while varying feature correlation, label noise, and class imbalance, so that the correct feature importance is known by construction. Across these datasets and four public tabular datasets, the six explainers examined—local surrogates, LIME, three SHAP variants, and Tree Interpreter—disagree in magnitude and even sign. The paper concludes with practical guidance: avoid LSurro and LIME when irrelevant variables are present, avoid Tree Interpreter under heavy label noise, and treat the SHAP variants as largely interchangeable. The value of the claim, if correct, is that practitioners can choose an explanation method from measurable data properties rather than assuming one method is always right.

What carries the argument

The machinery is a controlled synthetic-data generation framework built from the factorization $P(X_1, X_2, Y) = P(Y \mid X_1, X_2)P(X_2 \mid X_1)P(X_1)$, which lets the authors isolate three data properties: the marginal distribution of $X_1$, the conditional dependence of $X_2$ on $X_1$ (correlation $\rho$), and the label noise $\epsilon$ introduced by flipping a fraction of labels. Two logical target functions, XOR and NOT, fix the ground-truth feature importance $\phi^*_X$: for XOR both features matter equally ($0.5$ each), while for NOT only $X_1$ matters directly and $X_2$ receives importance equal to its correlation with $X_1$. Against that ground truth, the paper compares six local explainers using consistency, stability, compactness, and feature and rank agreement metrics. This setup is what lets the paper attribute differences in explanations to data properties rather than to dataset-specific accident.

What would settle it

Recompute the benchmark with ground truth defined by the learned model's actual split usage (such as Gini decrease) rather than the generative function's hand-assigned importance; if LSurro and LIME no longer overestimate irrelevant features, or the SHAP preference for one feature disappears, then the paper's method-level conclusions depend on that ground-truth choice.

Watch

Extended reading notes

Core claim

The central claim is that feature importance estimates from local explanation methods inherit the properties of the data-generating distribution $P(X_1, X_2, Y)$, and that no single explainer is reliable across all data conditions. The paper's experiments show that LSurro and LIME systematically overestimate the importance of irrelevant variables, Tree Interpreter is the most affected by label noise, the SHAP family yields very similar explanations to each other but consistently favors one feature over the other, and feature correlation inflates the importance and instability of $X_2$ in XOR datasets for SHAP explainers. From these patterns the authors infer concrete selection rules: for data with irrelevant variables avoid LSurro and LIME; for highly noisy data avoid Tree Interpreter; and for speed or adaptability prefer Sampling SHAP or Tree SHAP over Kernel SHAP. The claim is accompanied by the caveat that the study is restricted to binary classification on tabular data with two synthetic features, so the qualitative conclusions are established for that setting.

Load-bearing premise

The whole comparison stands on the hand-assigned ground truth—equal importance to both features in XOR, and $X_2$'s importance equal to its correlation $\rho$ with $X_1$ in NOT—being the right target for an explainer rather than the features the learned tree actually splits on.

Editorial extensions

If this is right

For datasets with irrelevant variables, practitioners should avoid LSurro and LIME, since both overestimate the importance of uninformative features.
For datasets with high label noise, Tree Interpreter should be avoided because it is the most unstable explainer in that setting.
Kernel SHAP, Sampling SHAP, and Tree SHAP produce near-identical explanations, so Sampling SHAP or Tree SHAP can be used interchangeably with faster computation on tree ensembles.
Feature correlation inflates the importance and instability of the correlated feature under SHAP explainers in XOR-type data, while label noise lowers feature importance across all methods.
Model hyperparameters such as tree depth appear to influence feature importance less than data properties, so data diagnostics matter more than hyperparameter tuning in explainer choice.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The two-feature design isolates mechanisms cleanly, but in higher-dimensional tabular data the surrogate-based methods may diverge even more, because local neighborhoods become harder to populate and spurious correlations multiply; this is a testable extension rather than a paper claim.
Because the failure modes are tied to measurable data properties, a natural follow-up is a meta-predictor that inspects correlation, noise, and imbalance and recommends an explainer; the reported patterns provide the training signal for such a rule.
The SHAP family's consistent favoring of one feature in these experiments suggests that Shapley attributions on correlated inputs can be misleading for causal readings of feature importance, a distinction the paper does not develop.
The same synthetic protocol could be reused for regression and multi-class targets, though the ground-truth importance definitions would need to be re-derived, so the ranking of methods in those settings should not be assumed to carry over.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

This is a reasonable empirical comparison of six local explainers on tree ensembles, but the central quantitative claims rest on a hand-defined ground truth that does not correspond to the model being explained. The qualitative phenomenon—explainers disagree and their outputs shift with noise, correlation, and imbalance—is credible and visible in the raw figures. What is actually new is a controlled two-feature synthetic setup (XOR and NOT) with varying noise and correlation, plus consistency/stability/compactness comparisons on four UCI datasets. That is a legitimate extension of existing benchmarking work like Flora et al., not a revolution.

The paper does well in scope and spirit. It compares a sensible set of explainers, including tree-specific ones, on a common tabular binary classification task. The synthetic design isolates properties one at a time, and the real-data section gives a practical feel for the problem. The message that no method is universally reliable is worth repeating.

The soft spots are serious and concentrate in Section 4. The ground truth for NOT gives X2 importance ρ even though the true decision tree never splits on X2 and Table 3 shows the fitted tree's own importance for X2 is 0 at ε=0. That is not a minor detail: the consistency and feature-agreement metrics in Figures 4–6 and Table 4 compare explainers against a target that mixes data-generating relevance with model usage. Any method that correctly ignores X2 is penalized. The ε scaling is also internally inconsistent: explainer outputs are normalized to sum to one, but the ground truth is scaled by ε, so at ε=0.5 the XOR ground truth sums to 0.5. Unless the ground truth is renormalized, the 'noise lowers importance' finding is partly an artifact. The covariance matrix for ρ=1 is singular. There are no error bars, no repeated seeds, and no code or data release, so the numbers in Tables 3–8 are hard to verify. The abstract's claim that some hyperparameters don't matter is unsupported—there is no hyperparameter experiment. These flaws do not kill the qualitative takeaway, but they do undermine the specific rankings and the practitioner guidance (like 'avoid LSurro for irrelevant variables' and 'avoid TI for noisy data').

Overall, I would call this a conditional pass, not a reject. The authors need to fix the ground truth definition, add uncertainty quantification, release artifacts, and either support or remove the hyperparameter claim. I would send it to peer review because the question is relevant and the flaws are fixable; a good referee could push them to a much stronger paper. I would not cite it in my own work until the evaluation is redone.

Referee Report

3 major / 6 minor

Summary. The paper reports an empirical study of six local feature-importance explainers (local surrogates, LIME, Kernel/Sampling/Tree SHAP, and Tree Interpreter) applied to decision trees and random forests. Using synthetic data generated from XOR and NOT functions with controlled feature correlation, label noise, and class imbalance, the authors compare each explainer's output to a hand-defined ground-truth feature importance and also report stability, compactness, consistency, and feature/rank agreement metrics. They supplement the synthetic study with experiments on four UCI datasets. The main claims are that explainers disagree substantially, that LSurro and LIME overestimate the importance of irrelevant variables, that Tree Interpreter is the most noise-sensitive, that SHAP explainers consistently favor one feature, and that practitioners should select explainers according to data properties.

Significance. The paper tackles a practically important problem: choosing among local explanation methods for tree-based models under different data conditions. Its strengths include the systematic control of data-generating mechanisms, the comparison of six methods on a common set of metrics, the use of both synthetic and real data, and explicit practitioner recommendations. If the quantitative results were reliable, the paper would be a useful reference for model interpretability. However, the validity of the quantitative rankings depends on the ground-truth definition in Section 4, which has internal inconsistencies and normalization problems. The qualitative observation that explainers disagree and respond to noise and correlation is plausible and partially supported by the stability and agreement results, but the specific rankings and recommendations require substantial rework.

major comments (3)

[Section 4, 'Ground truth feature importance'] For the NOT datasets, the true model f* is described as a depth-1 decision tree that splits only on X1, yet the ground truth assigns phi*_X2 = rho. Table 3 confirms that the learned decision tree gives phi_X2 = 0 for all rho when epsilon = 0. Because the paper's stated objective is to explain predictions of the learned model f (Abstract, Section 5), a faithful local explanation of f must assign zero importance to X2; penalizing methods that do so (Figures 4-6, Table 4) is a mismatch between the reference and the object of explanation. The recommendations in Section 6 to avoid LSurro and LIME on datasets with irrelevant variables are derived from this comparison and are therefore not established. Please redefine the ground truth as the feature importance of the fitted model (e.g., exact Shapley values of the learned tree), or explicitly change the evaluation target to 'recovery of data-generating importance' and justify why post-hoc local explainers should be judged against that target.
[Section 4, epsilon scaling] For XOR with noise epsilon, the ground truth is defined as phi*_X1 = phi*_X2 = 0.5*epsilon, so phi*_X1 + phi*_X2 = epsilon, while Section 5.2.1 states that all explainer outputs are normalized so their contributions sum to one. Comparing normalized explainer outputs to an unnormalized ground truth is not meaningful; if the ground truth is also renormalized, the factor epsilon cancels and noise cannot lower the relative importance of X1 and X2. Therefore the claims in Section 5.2.1 that 'noise lowers the importance of X1 and X2 for all the explainers' and that 'Tree Interpreter is the most affected by noise' are not supported by the current analysis. Please specify the exact normalization used for both estimates and ground truth, and recompute the noise comparisons accordingly.
[Section 5.2.1, overestimation claim] The claim that 'LSurro and LIME overestimate the importance of irrelevant variables' is ambiguous. Under the paper's own ground truth, X2 in the NOT datasets receives positive importance rho whenever rho > 0, so it is not 'irrelevant' by that definition; at rho = 0 the claim reduces to a special case. To support the practical recommendation in Section 6, report the rho = 0 results separately and define relevance explicitly relative to the fitted model f (e.g., features that never appear in any split), or use a model-based ground truth throughout.

minor comments (6)

[Table 1] The reference [17] for Tree Interpreter is actually 'A Debiased MDI Feature Importance Measure for Random Forests' (Li et al.), which is not the Tree Interpreter; please cite the correct source for Tree Interpreter (e.g., Saabas's implementation) or adjust the table.
[Table 4 caption] The caption says 'for the XOR datasets' but the table includes NOT rows; update the caption to 'XOR and NOT datasets'.
[Figures 2 and 3] Figures 2 and 3 are referenced in Section 5.1 but never described in the text; please add a sentence explaining the layout (e.g., rows/columns correspond to values of rho and epsilon).
[Tables 5-8] The column headers '# Features for 90% Accuracy' and 'Accuracy with 5 feature(%)' are unclear; specify whether 90% refers to model accuracy or to the fraction of the explainer's approximation, and define '5 feature' (e.g., five features or 5% of features).
[Section 5.1] The sentence 'we compute the feature importance scores of the learned model f on datasets with 1 000 instances' is ambiguous about whether 1,000 is the training or total sample size; please clarify and use the conventional '1,000' separator.
[Section 2] Section 2 introduces several metrics (stability, compactness, consistency, feature agreement) but does not give exact formulas or definitions of the implementation; add precise definitions (e.g., how the L2 distance and top-k feature agreement are computed) to make the experiments reproducible.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the paper is an empirical benchmarking study whose conclusions are measured from explainer outputs, not derived from its own definitions or self-citations.

full rationale

The paper performs an empirical evaluation of local feature-importance methods on synthetic and real-world datasets. Its central claims—that explainers disagree and that their estimates are sensitive to noise, correlation, and imbalance—are supported by measured feature importance outputs (Figures 4–6, Tables 3–8), not by a derivation that reduces to its inputs. The ground-truth feature importance phi* defined in Section 4 is a hand-specified benchmark target, and it is defensible to question whether that target is the right one for local explanations of the learned model (e.g., for NOT, phi*_X2 = rho even though Table 3 shows the learned decision tree gives X2 zero importance). However, a questionable or arbitrary benchmark is an evaluation-validity concern, not circularity: the explainer estimates themselves are computed independently of phi*, and the paper does not fit a parameter to a subset of data and then predict a closely related quantity. There are no load-bearing self-citations, no imported uniqueness theorems, and no renamed known results presented as new derivations. The quantitative rankings may be undermined if the ground truth is invalid, but the paper's reasoning does not become equivalent to its own inputs by construction. Therefore no specific circular step can be exhibited, and the appropriate finding is no significant circularity.

Assumptions & free parameters 2 free parameters · 6 assumptions · 0 invented entities

The paper is an empirical benchmark, so it introduces no new physical or model entities. The central internal choices are the hand-defined ground-truth importance rules (XOR 0.5/0.5, NOT 1/ρ, ε scaling) and the implicit assumption that learned models can be scored against the true model. These choices are load-bearing: all quantitative conclusions about which explainer is most consistent or faithful depend on them.

free parameters (2)

NOT ground truth weight of X2 = ρ (0, 0.1, 0.9, 1)
Hand-chosen in Section 4 to represent indirect influence, despite the true model never splitting on X2; this value drives all NOT dataset comparisons.
Noise scaling of ground truth importance = ε (0, 0.25, 0.5)
Section 4 multiplies ground truth by ε when labels are flipped; no derivation is given, and normalization makes the scaling immaterial to the reported comparisons.

assumptions (6)

ad hoc to paper For XOR datasets, ground truth feature importance is (0.5, 0.5), independent of correlation ρ and noise ε after normalization.
Section 4 defines this ground truth from the generating function rather than from the learned model; it is the scoring target, not a derived result.
ad hoc to paper For NOT datasets, ground truth feature importance is (1, ρ), where ρ is the correlation between X1 and X2.
Section 4 assigns importance ρ to X2 even though the stated true model is a depth-1 tree that splits only on X1; this conflates data-generating relevance with model usage.
ad hoc to paper When label noise ε is present, ground truth importances scale by ε: (0.5ε, 0.5ε) for XOR and (ε, ρε) for NOT.
Section 4 asserts this scaling without derivation; since comparisons are normalized, the scaling is redundant and suggests confusion about the evaluation target.
domain assumption The learned model f is a sufficient proxy for the true model f*, so explanations of f can be scored against f* ground truth.
Section 5.2.1 compares explainer outputs for learned trees to f* importance; at ε=0.5 the tree accuracy is near chance, so f and f* differ substantially.
domain assumption Gini impurity decreases converge to equal (0.5, 0.5) importance for XOR on large datasets.
Section 4 states this limit argument; Table 3 shows finite-sample deviations (DT 0.44/0.56, RF 0.69/0.31), so the limit does not describe the evaluated models.
domain assumption The covariance matrix [[1,1],[1,1]] is an acceptable bivariate normal covariance for the ρ=1.0 setting.
Section 5.1 lists this singular matrix as a parameter value; it is not positive definite, so the generated data is degenerate.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Feature Importance Depends on Properties of the Data: Towards Choosing the Correct Explanations for Your Data and Decision Trees based Models." pith.science (2026). https://pith.science/paper/BNRUOJ7M

@misc{pith2026250207153,
  author       = {Pith},
  title        = {Pith review of: Feature Importance Depends on Properties of the Data: Towards Choosing the Correct Explanations for Your Data and Decision Trees based Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BNRUOJ7M}},
  note         = {Machine review of arXiv:2502.07153}
}

read the original abstract

In order to ensure the reliability of the explanations of machine learning models, it is crucial to establish their advantages and limits and in which case each of these methods outperform. However, the current understanding of when and how each method of explanation can be used is insufficient. To fill this gap, we perform a comprehensive empirical evaluation by synthesizing multiple datasets with the desired properties. Our main objective is to assess the quality of feature importance estimates provided by local explanation methods, which are used to explain predictions made by decision tree-based models. By analyzing the results obtained from synthetic datasets as well as publicly available binary classification datasets, we observe notable disparities in the magnitude and sign of the feature importance estimates generated by these methods. Moreover, we find that these estimates are sensitive to specific properties present in the data. Although some model hyper-parameters do not significantly influence feature importance assignment, it is important to recognize that each method of explanation has limitations in specific contexts. Our assessment highlights these limitations and provides valuable insight into the suitability and reliability of different explanatory methods in various scenarios.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

37 extracted references · 34 canonical work pages

[1]

Machine Learning 45(1), 5–32 (2001)

Breiman, L.: Random Forests. Machine Learning 45(1), 5–32 (2001)

work page 2001
[2]

Agricultural and Forest Meteorology 232(January), 210–224 (2017)

Kulmala, L., Read, J., N¨ ojd, P., Rathgeber, C.B.K., Cuny, H.E., Hollm´ en, J., M¨ akinen, H.: Identifying the main drivers for the production and maturation of scots pine tracheids along a temperature gradient. Agricultural and Forest Meteorology 232(January), 210–224 (2017)

work page 2017
[3]

Ricordeau, J., Lacaille, J.: Application of Random Forests to Engine health Monitoring. (2010)

work page 2010
[4]

BMC Bioinformatics 8(1), 25 (2007)

Strobl, C., Boulesteix, A.-L., Zeileis, A., Hothorn, T.: Bias in random for- est variable importance measures: Illustrations, sources and a solution. BMC Bioinformatics 8(1), 25 (2007)

work page 2007
[5]

http://explained.ai/decision-tree-viz/index.html

Terence, P., Prince, G.: Beware Default Random Forest Importances. http://explained.ai/decision-tree-viz/index.html

work page
[6]

Molnar, C., Gruber, S., Kopper, P.: Limitations of Interpretable Machine Learning Methods, (2020)

work page 2020
[7]

In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

Ribeiro, M.T., Singh, S., Guestrin, C.: ”Why Should I Trust You?”: Explaining the Predictions of Any Classifier. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. KDD ’16, pp. 1135–1144. Association for Computing Machinery, New York, NY, USA (2016)

work page 2016
[8]

arXiv:1705.07874 [cs, stat] (2017) arXiv:1705.07874 [cs, stat]

Lundberg, S., Lee, S.-I.: A Unified Approach to Interpreting Model Predictions. arXiv:1705.07874 [cs, stat] (2017) arXiv:1705.07874 [cs, stat]

arXiv 2017

Show all 37 references

[9]

arXiv (2022)

Krishna, S., Han, T., Gu, A., Pombra, J., Jabbari, S., Wu, S., Lakkaraju, H.: The Disagreement Problem in Explainable Machine Learning: A Practitioner’s Perspective. arXiv (2022)

2022
[10]

arXiv (2022)

Attanasio, G., Pastor, E., Di Bonaventura, C., Nozza, D.: Ferret: A Framework for Benchmarking Explainers on Transformers. arXiv (2022)

2022
[11]

arXiv (2019)

Camburu, O.-M., Giunchiglia, E., Foerster, J., Lukasiewicz, T., Blunsom, P.: Can I Trust the Explainer? Verifying Post-hoc Explanatory Methods. arXiv (2019)

2019
[12]

arXiv (2021)

Bodria, F., Giannotti, F., Guidotti, R., Naretto, F., Pedreschi, D., Rinzivillo, S.: Benchmarking and Survey of Explanation Methods for Black Box Models. arXiv (2021)

2021
[13]

arXiv (2021) 18

Neely, M., Schouten, S.F., Bleeker, M.J.R., Lucic, A.: Order in the Court: Explainable AI Methods Prone to Disagreement. arXiv (2021) 18

2021
[14]

arXiv preprint arXiv:2211.08943 (2022)

Flora, M., Potvin, C., McGovern, A., Handler, S.: Comparing explanation meth- ods for traditional machine learning models part 1: An overview of current methods and quantifying their disagreement. arXiv preprint arXiv:2211.08943 (2022)

2022 arXiv
[15]

Molnar, C.: Interpretable Machine Learning, 2nd edn. (2022). https://christophm. github.io/interpretable-ml-book

2022
[16]

arXiv (2019)

Lundberg, S.M., Erion, G.G., Lee, S.-I.: Consistent Individualized Feature Attribution for Tree Ensembles. arXiv (2019)

2019
[17]

arXiv:1906.10845 [cs, stat] (2019)

Li, X., Wang, Y., Basu, S., Kumbier, K., Yu, B.: A Debiased MDI Feature Impor- tance Measure for Random Forests. arXiv:1906.10845 [cs, stat] (2019). arXiv: 1906.10845. Accessed 2019-10-18

2019 arXiv
[18]

arXiv (2018)

Alvarez-Melis, D., Jaakkola, T.S.: On the Robustness of Interpretability Methods. arXiv (2018)

2018
[19]

IEEE Transactions on Software Engineering 48(7), 2245–2261 (2022) arXiv:2202.02389 [cs]

Rajbahadur, G.K., Wang, S., Kamei, Y., Hassan, A.E.: The impact of feature importance methods on the interpretation of defect classifiers. IEEE Transactions on Software Engineering 48(7), 2245–2261 (2022) arXiv:2202.02389 [cs]

2022 arXiv
[20]

arXiv (2021)

Slack, D., Hilgard, S., Singh, S., Lakkaraju, H.: Reliable Post Hoc Explanations: Modeling Uncertainty in Explainability. arXiv (2021)

2021
[21]

arXiv (2018)

Petsiuk, V., Das, A., Saenko, K.: RISE: Randomized Input Sampling for Explanation of Black-box Models. arXiv (2018)

2018
[22]

Bodria, F., Giannotti, F., Guidotti, R., Naretto, F., Pedreschi, D., Rinzivillo, S.: Benchmarking and survey of explanation methods for black box models 37(5), 1719–1778
[23]

arXiv (2021)

Liu, Y., Khandagale, S., White, C., Neiswanger, W.: Synthetic Benchmarks for Scientific Research in Explainable Machine Learning. arXiv (2021)

2021
[24]

arXiv preprint arXiv:2106.12543 (2021)

Liu, Y., Khandagale, S., White, C., Neiswanger, W.: Synthetic benchmarks for sci- entific research in explainable machine learning. arXiv preprint arXiv:2106.12543 (2021)

2021 arXiv
[25]

arXiv (2022)

Han, T., Srinivas, S., Lakkaraju, H.: Which Explanation Should I Choose? A Function Approximation Perspective to Characterizing Post Hoc Explanations. arXiv (2022)

2022
[26]

Turb´ e, H., Bjelogrlic, M., Lovis, C., Mengaldo, G.: Evaluation of post-hoc interpretability methods in time-series classification 5(3), 250–260
[27]

In: Advances in Neural 19 Information Processing Systems, vol

Ismail, A.A., Gunady, M., Corrada Bravo, H., Feizi, S.: Benchmarking Deep Learning Interpretability in Time Series Predictions. In: Advances in Neural 19 Information Processing Systems, vol. 33, pp. 6441–6452. Curran Associates, Inc
[28]

arXiv preprint arXiv:1907.09701 (2019)

Yang, M., Kim, B.: Benchmarking attribution methods with relative feature importance. arXiv preprint arXiv:1907.09701 (2019)

2019 arXiv
[29]

arXiv preprint arXiv:2306.17844 (2023)

Zhong, Z., Liu, Z., Tegmark, M., Andreas, J.: The clock and the pizza: Two stories in mechanistic explanation of neural networks. arXiv preprint arXiv:2306.17844 (2023)

2023 arXiv
[30]

Advances in Neural Information Processing Systems 35, 5256–5268 (2022)

Han, T., Srinivas, S., Lakkaraju, H.: Which explanation should i choose? a func- tion approximation perspective to characterizing post hoc explanations. Advances in Neural Information Processing Systems 35, 5256–5268 (2022)

2022
[31]

Agarwal, C., Krishna, S., Saxena, E., Pawelczyk, M., Johnson, N., Zitnik, M., Lakkaraju, H.: OpenXAI: Towards a Transparent Evaluation of Post hoc Model Explanations, 16
[32]

arXiv (2020)

Kokhlikyan, N., Miglani, V., Martin, M., Wang, E., Alsallakh, B., Reynolds, J., Melnikov, A., Kliushkina, N., Araya, C., Yan, S., Reblitz-Richardson, O.: Captum: A Unified and Generic Model Interpretability Library for PyTorch. arXiv (2020)

2020
[33]

arXiv (2022)

Hedstr¨ om, A., Weber, L., Bareeva, D., Motzkus, F., Samek, W., Lapuschkin, S., H¨ ohne, M.M.-C.: Quantus: An Explainable AI Toolkit for Responsible Evaluation of Neural Network Explanations. arXiv (2022)

2022
[34]

Artificial Intelligence 291, 103428 (2021)

Guidotti, R.: Evaluating local explanation methods on ground truth. Artificial Intelligence 291, 103428 (2021)

2021
[35]

In: International Joint Conference on Artificial Intelligence (2023)

Le, P.Q., Nauta, M., Van Bach Nguyen, S.P., Schl¨ otterer, J., Seifert, C.: Bench- marking explainable ai-a survey on available toolkits and open challenges. In: International Joint Conference on Artificial Intelligence (2023)

2023
[36]

Nature Machine Intelligence5(3), 250–260 (2023)

Turb´ e, H., Bjelogrlic, M., Lovis, C., Mengaldo, G.: Evaluation of post-hoc inter- pretability methods in time-series classification. Nature Machine Intelligence5(3), 250–260 (2023)

2023
[37]

http://archive.ics

Dua, D., Graff, C.: UCI Machine Learning Repository (2017). http://archive.ics. uci.edu/ml 20

2017

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.

[1] [1]

Machine Learning 45(1), 5–32 (2001)

Breiman, L.: Random Forests. Machine Learning 45(1), 5–32 (2001)

work page 2001

[2] [2]

Agricultural and Forest Meteorology 232(January), 210–224 (2017)

Kulmala, L., Read, J., N¨ ojd, P., Rathgeber, C.B.K., Cuny, H.E., Hollm´ en, J., M¨ akinen, H.: Identifying the main drivers for the production and maturation of scots pine tracheids along a temperature gradient. Agricultural and Forest Meteorology 232(January), 210–224 (2017)

work page 2017

[3] [3]

Ricordeau, J., Lacaille, J.: Application of Random Forests to Engine health Monitoring. (2010)

work page 2010

[4] [4]

BMC Bioinformatics 8(1), 25 (2007)

Strobl, C., Boulesteix, A.-L., Zeileis, A., Hothorn, T.: Bias in random for- est variable importance measures: Illustrations, sources and a solution. BMC Bioinformatics 8(1), 25 (2007)

work page 2007

[5] [5]

http://explained.ai/decision-tree-viz/index.html

Terence, P., Prince, G.: Beware Default Random Forest Importances. http://explained.ai/decision-tree-viz/index.html

work page

[6] [6]

Molnar, C., Gruber, S., Kopper, P.: Limitations of Interpretable Machine Learning Methods, (2020)

work page 2020

[7] [7]

In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

Ribeiro, M.T., Singh, S., Guestrin, C.: ”Why Should I Trust You?”: Explaining the Predictions of Any Classifier. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. KDD ’16, pp. 1135–1144. Association for Computing Machinery, New York, NY, USA (2016)

work page 2016

[8] [8]

arXiv:1705.07874 [cs, stat] (2017) arXiv:1705.07874 [cs, stat]

Lundberg, S., Lee, S.-I.: A Unified Approach to Interpreting Model Predictions. arXiv:1705.07874 [cs, stat] (2017) arXiv:1705.07874 [cs, stat]

arXiv 2017