Pith. sign in

REVIEW 4 major objections 6 minor 3 cited by

Local MDI+: Local Feature Importances for Tree-Based Models

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Local MDI+ extends the MDI+ global-importance framework to individual predictions, claiming that its per-sample attributions find predictive features more faithfully and more stably than LIME, TreeSHAP, and Local MDI across twelve…

desk verdict Solid global feature-importance extension with a shaky instance-specific validation and an unbacked headline number. read the letter →

arxiv 2506.08928 v2 pith:ESPCNDNX submitted 2025-06-10 cs.LG stat.MEstat.ML

Zhongyuan Liang , Zachary T. Rewolinski , Abhineet Agarwal , Tiffany M. Tang , Bin Yu This is my paper
classification cs.LGstat.MEstat.ML
keywords localfeatureimportancetree-basedmodelsrandomforestsMDI+modelinterpretabilitycounterfactualexplanationsstabilitygradientboosting
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Local MDI+ (LMDI+) is a method for saying which features drove a single prediction made by a random forest or gradient-boosted model. It extends the global MDI+ framework, which repairs the biases of mean decrease in impurity by reinterpreting decision trees as linear regressions on a transformed node basis, into a per-sample attribution: each feature's importance for an observation is the contribution of that feature in the linearized tree representation, averaged across trees. The paper reports that LMDI+ identifies instance-specific predictive features better than LIME, TreeSHAP, and Local MDI, with an average 10% improvement in predictive performance when only top-ranked features are retained, and that its rankings are more stable across repeated model fits. It also shows the method supports counterfactual explanations with smaller feature changes and discovers homogeneous subgroups in a housing case study. If these claims hold, practitioners get local explanations for tree ensembles that are more faithful and more reproducible without perturbation-based approximations.

What carries the argument

The stump function maps each split into a three-valued basis element, and concatenating these over all splits gives the node basis. A known result, stated as Proposition 1 from MDI+, connects the mean decrease in impurity of feature k to the $R^{2}$ from regressing the response on the node-basis columns corresponding to splits on that feature, which explains MDI's biases. LMDI+ uses the same basis to turn the tree into a linear model: it appends raw features to the node basis, fits a regularized GLM on in-bag and out-of-bag samples, and reads off the per-observation attribution as the inner product of the transformed features with the fitted coefficients. The GLM shrinkage and out-of-bag evaluation are what correct the biases; the inner-product readout is what makes the importance local and stable.

What would settle it

On a synthetic dataset where each instance has a known, instance-specific set of signal features, compute LMDI+ and baseline rankings and compare them against the per-instance ground truth; if the remove-and-retrain ordering ever ranks a method that selects globally predictive features above one that selects the true local signal, the protocol is measuring global fidelity rather than local fidelity.

Watch

Extended reading notes

Core claim

The paper proposes to compute sample-specific feature importances by exploiting the known equivalence between a fitted decision tree and ordinary least squares on the basis of stump functions defined by its splits. For each tree in the ensemble, LMDI+ builds a transformed feature map that concatenates these stump functions with the raw covariates, evaluates it on both in-bag and out-of-bag samples, fits a regularized generalized linear model to the response on that map, and defines the local importance of feature k for observation x as the inner product of the transformed feature vector with the corresponding GLM coefficients; features that never split are assigned zero. Scores are averaged over trees and ranked by absolute value. The paper claims this inherits MDI+'s corrections for overfitting, correlation bias, and poor handling of smooth or additive structure while adding sample specificity, and presents experiments—synthetic DGPs, correlation robustness, twelve real-world datasets, stability, ablations, gradient boosting, counterfactuals, and a housing subgroup case study—that it says support the method's predictive-feature identification, stability, and practical utility.

Load-bearing premise

The real-data comparisons assume that the remove-and-retrain protocol—replacing all but a sample's top-ranked features with their average and retraining a random forest—actually measures how well a method identifies the features driving that sample's prediction, an assumption the paper does not independently validate.

Editorial extensions

If this is right

  • If LMDI+ identifies predictive features as claimed, local explanations for tree ensembles can be obtained without the perturbations used by LIME or the subset averaging used by TreeSHAP, reducing a known source of instability.
  • Using only the features LMDI+ ranks highest for each sample preserves more predictive performance after retraining than the same protocol with baseline methods, which matters when a practitioner must act on a short list of reasons.
  • Stable per-sample rankings across random seeds mean repeated fits of the same model are less likely to contradict each other in high-stakes settings.
  • The framework transfers to gradient-boosting ensembles, so the method is not tied to random forests.
  • Counterfactuals selected in LMDI+ importance space require smaller feature changes, and clustering on LMDI+ scores yields subgroups on which simple models fit better.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The linearized view suggests LMDI+ attributions are additive and therefore could be aggregated across samples or groups into global or subgroup-level importance statements, something the paper only hints at in its clustering use case.
  • The remove-and-retrain protocol may reward methods that find globally predictive features rather than features that drive the particular instance; a synthetic study with known per-instance ground truth would separate these two notions.
  • Because the GLM and link function are plug-in choices, the same construction could be tested with other regularized estimators for non-Gaussian responses or with interaction features appended to the node basis.
  • The stability gain may come partly from GLM shrinkage and averaging over trees; an ablation that varies only shrinkage while holding the basis fixed would isolate how much stability is due to regularization.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes Local MDI+ (LMDI+), a sample-specific extension of the global MDI+ feature importance framework. The method augments the stump basis of a decision tree with the raw feature values, fits a regularized GLM on this transformed representation (using out-of-bag samples), and defines the local importance of feature k for an instance x as the inner product of the transformed vector with the fitted GLM coefficients. The authors claim that LMDI+ identifies instance-specific predictive features better than LIME, TreeSHAP, and Local MDI, yields an average 10% improvement in predictive performance on twelve real-world datasets, produces more stable rankings across repeated random-forest fits, generalizes to gradient boosting, and enables counterfactual and subgroup-discovery use cases. The paper includes synthetic experiments with known signal features, correlation-robustness experiments, real-data remove-and-retrain evaluations, stability experiments, ablations, runtime comparisons, and two application case studies.

Significance. If the central empirical claim is established, LMDI+ would be a practically useful structure-based local importance method that avoids perturbation-based artifacts and offers improved stability over LIME and TreeSHAP. The paper's strengths include a clear algorithmic recipe, extensive experiments across multiple datasets and ensemble settings, an ablation study that isolates the contribution of each component, and a companion repository with implementation code. However, the significance is conditional: the synthetic experiments support global signal-feature recovery, while the instance-specific claim rests on an unvalidated remove-and-retrain protocol. The abstract's headline 10% figure is not substantiated by any reported number in the body, and the attribution formula itself is heuristic. These gaps are fixable but currently leave the central claim partially unsupported.

major comments (4)
  1. [Section 4.1/4.2] The synthetic experiments do not actually test instance-specificity. In Section 4.1, a fixed subset of signal features is randomly selected once per simulation, and the same response function is applied to every sample; the AUROC is then computed for each test sample against the same global binary signal-feature label vector. Section 4.2 likewise fixes signal features X1,...,X6 for all samples. These experiments therefore measure how well a method recovers globally signal features, not whether the importance scores vary correctly with the features that drive each individual prediction. To support the 'instance-specific' wording in the abstract and Section 1, the authors need either synthetic DGPs in which the signal features vary per sample, or another evaluation that isolates per-instance variation against a known oracle.
  2. [Section 4.3, Setup] The remove-and-retrain protocol is load-bearing for the claim that LMDI+ identifies instance-specific predictive features, but it is not validated for that purpose. Replacing features outside a sample's top-k with their column mean and retraining an RF can reward methods that select globally predictive features, since those features will be retained for most rows and will dominate the retrained model; the protocol does not establish that the retained set varies meaningfully across instances or that it recovers the features driving each individual prediction. A concrete test would be to construct synthetic data with per-instance signal features and verify that the protocol's performance tracks per-instance ground truth, and to compare LMDI+ against a global-only ranking baseline (e.g., global MDI+ or mean marginal importance) to show that the local component adds information beyond global feature ranks.
  3. [Abstract] The abstract claims an 'average 10% improvement in predictive performance when using only the selected features,' but no quantitative 10% figure appears anywhere in the body. Table 1 reports average ranks, Figures 3/12/13 report R2/AUROC curves, and Appendix D reports full-dataset numbers without a 10% average. The authors should either add the exact computation supporting the 10% claim (for example, average relative improvement across datasets and feature-retention levels) or qualify/remove the quantitative claim from the abstract.
  4. [Section 3.3, Step 3] The LMDI+ attribution formula is introduced as an analogy to linear models, but no formal property is established for the resulting score. In a linear model, beta_k * x_k is the exact additive contribution to the prediction, but here the GLM is fitted on an augmented basis containing stump functions and the raw feature; the inner product of the transformed vector with the GLM coefficients is not shown to satisfy local accuracy, additivity, or any consistency property relative to the tree ensemble's prediction. This is load-bearing because the empirical claims are about the quality of these scores. The authors should provide a formal statement of what the score represents, or at minimum a sanity check on synthetic data with known per-instance local contributions.
minor comments (6)
  1. [Section 3.3] The notation in Step 3 is dense and ambiguous: it is not immediately clear whether a separate GLM is fit for each feature k on the augmented basis [Psi(S^(k)), x_k], or whether a single GLM is fit on the full concatenated basis and then restricted. Please clarify the dimension of beta^(k)_lambda and the fitting procedure.
  2. [Section 5] The 'unique feature ratio' measures overlap between top-k sets across five RF fits, but it does not directly measure rank-order stability. Reporting an additional rank-correlation metric (e.g., average pairwise Kendall tau across fits) would strengthen the stability claim.
  3. [Table 4] The counterfactual comparison reports mean L1 distances without standard errors or significance tests; the gaps between methods (e.g., 7.6 versus 9.5 on House 16H) may be within noise. Please add variability measures or statistical tests.
  4. [Section 8] The statement that LMDI+ performs '40% better than the Global model' is undefined; specify the relative MSE improvement and the baseline to which it is compared.
  5. [Section 6 / Appendix F] The ablation tables report only average ranks; providing the underlying metric values (or a plot of the full curves) for at least a few datasets would make the ablation results more transparent.
  6. [Section 10] There is a typo in the Future Work paragraph: 'also supports 0choices of generalized linear models' should read 'also supports choices of generalized linear models.'

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: LMDI+ is a newly defined local attribution, its cited Proposition 1 is a parameter-free mathematical equivalence, and the evaluations use external benchmarks and held-out performance. The flagged auditability and protocol-validity concerns are not circularity.

full rationale

The paper's derivation chain is self-contained. LMDI+ is defined in Section 3.3 as a linear attribution LMDI+_k(x, S^(k), G) := tilde(Psi)^(k)(x)^T beta_hat, where beta_hat comes from a regularized GLM fit on the transformed node basis; this is a definition, not a fitted quantity renamed as a prediction. The load-bearing Proposition 1 (Agarwal et al., 2023) is a mathematical equivalence between MDI and an R^2 from OLS on stump functions; its stated assumptions concern tree structure and bootstrapped data, not the LMDI+ scoring rule, so the self-citation is real evidence rather than a circular premise. Synthetic experiments in Sections 4.1-4.2 compare scores against known DGP signal features, and the real-data evaluation in Section 4.3 uses the external remove-and-retrain protocol of Hooker et al. (2019), with preserved test data and retraining, so results are not forced by construction. Stability and ablation experiments are likewise held-out or rank-based comparisons against LIME, TreeSHAP, and Local MDI. Two concerns should be weighed in the verdict but are not circularity: (1) the abstract's 'average 10% improvement' does not appear in any body table or figure, since Table 1 reports average ranks and Figures 3/12/13 report R^2/AUROC curves, making the headline quantitative claim unauditable; and (2) the Section 4.3 protocol assumes that replacing non-top-k features with their column mean and retraining measures instance-specific predictivity, but it may reward globally predictive features, so the instance-specific interpretation rests on an unvalidated measurement assumption. Neither concern involves a quantity defined in terms of the target claim, so the circularity score remains 0.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central claim rests on two prior mathematical results (the tree-OLS equivalence and the MDI-R2 connection), on a heuristic attribution formula introduced in this paper, and on an unvalidated evaluation protocol. The GLM regularization parameters and random forest hyperparameters are selected by cross-validation or fixed by the authors, but they are not fit to the evaluation metrics and the main ranking of methods is shown to be robust to them. No new physical or conceptual entities are introduced beyond the transformed feature map.

free parameters (2)
  • ElasticNet penalty and l1_ratio (LMDI+ GLM) = Selected by ElasticNetCV/LogisticRegressionCV with l1_ratios in {0.1, 0.5, 0.99} and cv=3
    Internal hyperparameters of the regularized GLM step. They are chosen by cross-validation on the regression objective, not by fitting to the feature-importance evaluation metrics, so they are standard modeling choices rather than deliberately tuned constants.
  • Random forest hyperparameters = n_estimators=100, min_samples_leaf=5 (regression) or 1 (classification), max_features=0.33 (regression) or sqrt…
    Fixed model configuration; the ablation study varies some of these and finds the ranking unchanged, so the central conclusions are not sensitive to them.
assumptions (4)
  • domain assumption Proposition 1 (Agarwal et al. 2023): MDIk is proportional to R^2 of OLS on the stump basis restricted to splits on feature k.
    The paper uses this connection to justify building a local attribution from a linear model on the transformed node basis. It is cited as a result from the authors' own prior work, not re-derived or machine-checked here.
  • standard math Klusowski and Tian (2023): decision tree predictions are equivalent to OLS predictions on the stump (node) basis.
    This equivalence is the foundation for the linear interpretation of trees. It is taken as given from the cited paper.
  • ad hoc to paper For each feature k, the inner product of the transformed vector Psi_tilde^(k)(x) with the fitted GLM coefficients beta^(k)_lambda gives the local contribution of feature k to the prediction.
    This is the core modeling choice of LMDI+. The paper provides no derivation or formal justification that this decomposition is faithful; it is validated only empirically (Section 3.3, Step 3).
  • domain assumption The remove-and-retrain evaluation with per-sample feature masking and mean imputation measures local predictive importance.
    The validity of the real-data comparisons rests on this measurement assumption, which is not justified in the paper (Section 4.3).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Local MDI+: Local Feature Importances for Tree-Based Models." pith.science (2026). https://pith.science/paper/ESPCNDNX

@misc{pith2026250608928,
  author       = {Pith},
  title        = {Pith review of: Local MDI+: Local Feature Importances for Tree-Based Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ESPCNDNX}},
  note         = {Machine review of arXiv:2506.08928}
}
read the original abstract

Tree-based ensembles such as random forests remain the go-to for tabular data over deep learning models due to their prediction performance and computational efficiency. These advantages have led to their widespread deployment in high-stakes domains, where interpretability is essential for ensuring trustworthy predictions. This has motivated the development of popular local feature importance methods such as LIME and TreeSHAP. However, these approaches rely on approximations that ignore the model's internal structure and instead depend on potentially unstable perturbations. These issues are addressed in the global setting by MDI+, a global feature importance method which combines tree-based and linear feature importances by exploiting an equivalence between decision trees and least squares on a transformed node basis. However, the global MDI+ scores are not able to explain predictions when faced with heterogeneous individual characteristics. To address this gap, we propose Local MDI+ (LMDI+), a novel extension of the MDI+ framework that quantifies feature importances for each particular sample. Across twelve real-world benchmark datasets, LMDI+ outperforms existing baselines at identifying instance-specific predictive features, yielding an average 10% improvement in predictive performance when using only the selected features. It further demonstrates greater stability by consistently producing similar instance-level feature importance rankings across repeated model fits with different random seeds. Ablation experiments show that each component of LMDI+ contributes to these gains, and that the improvements extend beyond random forests to gradient boosting models. Finally, we show that LMDI+ enables local interpretability use cases by identifying closely matched counterfactuals for each classification benchmark and discovering homogeneous subgroups in a housing dataset case study.

Figures

Figures reproduced from arXiv: 2506.08928 by the authors.

Figure 1
Figure 1. LMDI+ consistently achieves higher AUROC (±1 SE) across different datasets, response functions and SNRs, demonstrating its superior ability to distinguish signal features from non-signal features. Results are averaged over 30 runs. For classification, noise levels are adjusted by randomly flipping {0%, 5%, 10%, 15%} of the labels. Sample sizes are varied over {300, 500, 1000}. Evaluation. We evaluate the task based … view at source ↗
Figure 2
Figure 2. We show the average per sample ranks (±1 SE) of each feature group under various levels of correlation ρ. In the presence of extreme correlation, only LMDI+ continues to rank the signal features as most important. Results are averaged over 50 runs. important than signal features in the presence of strong correlation. However, the use of regularized GLMs and OOB samples helps LMDI+ rank signal features as the most im… view at source ↗
Figure 3
Figure 3. LMDI+ achieves higher R2 and AUROC after retraining on masked training data, demonstrating its superior ability to identify predictive features. Results are averaged over 20 runs. 5 LMDI+ Produces More Stable Feature Importance Rankings Trustworthy interpretations require robustness to choices made during the modeling process (Murdoch et al., 2019; Yu and Kumbier, 2020; Burger et al., 2023). A well-known limitation … view at source ↗
Figures from the paper (15 more)
Figure 4
Figure 4. Figure 4: LMDI+ selects the fewest unique features, indicating the most consistent feature rankings across repeated RF fits. Results are averaged over 15 runs. 9 [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: We show the MSE (±1SE) of linear regressions fit on each cluster. We observe a steep drop until k = 4, after which MSE levels off. 9 Runtime Analysis In this section, we compare the runtime of LMDI+ to that of other LFI methods. The computation time for obtaining LMDI+…
Figure 6
Figure 6. Figure 6: Across different sample sizes and signal-to-noise ratios in linear response function, LMDI [PITH_FULL_IMAGE:figures/full_fig_p019_6.png]
Figure 7
Figure 7. Figure 7: Across different sample sizes and signal-to-noise ratios in interaction response function, LMDI [PITH_FULL_IMAGE:figures/full_fig_p020_7.png]
Figure 8
Figure 8. Figure 8: Across different sample sizes and signal-to-noise ratios in linear + LSS response function, LMDI [PITH_FULL_IMAGE:figures/full_fig_p021_8.png]
Figure 9
Figure 9. Figure 9: Across different sample sizes and signal-to-noise ratios in logistic response function, LMDI [PITH_FULL_IMAGE:figures/full_fig_p022_9.png]
Figure 10
Figure 10. Figure 10: Across different sample sizes and signal-to-noise ratios in logistic interaction response function, [PITH_FULL_IMAGE:figures/full_fig_p023_10.png]
Figure 11
Figure 11. Figure 11: Across different sample sizes and signal-to-noise ratios in logistic linear + LSS response function, [PITH_FULL_IMAGE:figures/full_fig_p024_11.png]
Figure 12
Figure 12. Figure 12: For regression benchmark datasets, LMDI+ achieves higher test R2 after retraining on the masked training data selected by its feature ranking, demonstrating its ability to identify more signal features than baseline methods. Results are averaged over 20 runs with diff…
Figure 13
Figure 13. Figure 13: For classification benchmark datasets, LMDI [PITH_FULL_IMAGE:figures/full_fig_p025_13.png]
Figure 14
Figure 14. Figure 14: Across all regression datasets, LMDI+ yields more stable feature rankings as it identifies a smaller number of unique features when selecting the top important features across different random forest fits on the same data. Performance is reported as the average on the…
Figure 15
Figure 15. Figure 15: Across all classification datasets, LMDI [PITH_FULL_IMAGE:figures/full_fig_p028_15.png]
Figure 16
Figure 16. Figure 16: Distribution of coordinate-wise distances between an observation and its counterfactual. When [PITH_FULL_IMAGE:figures/full_fig_p030_16.png]
Figure 17
Figure 17. Figure 17: Homes in Miami-Dade County, location determined by latitude and longitude. We witness clear [PITH_FULL_IMAGE:figures/full_fig_p031_17.png]
Figure 18
Figure 18. Figure 18: We observe that the distributions of living area and home price differ drastically between clusters, [PITH_FULL_IMAGE:figures/full_fig_p032_18.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Kernel of Partition Paths: A Unified Representation for Tree Ensembles

    stat.ML 2026-06 unverdicted novelty 7.0 of 10

    KPP is a new representation for tree ensembles that indexes features by nodes with a path metric, yielding a non-diagonal Gram matrix that unifies prediction, exact additive attribution, deterministic Lipschitz robust...

  2. OC-Distill: Ontology-aware Contrastive Learning with Cross-Modal Distillation for ICU Risk Prediction

    cs.LG 2026-04 unverdicted novelty 6.0 of 10

    Ontology-aware contrastive pretraining plus note-to-vitals distillation improves MIMIC ICU risk and length-of-stay prediction using only vital signs at inference.

  3. Provable Recovery of Locally Important Signed Features and Interactions from Random Forest

    stat.ML 2025-12 conditional novelty 6.0 of 10

    Under a Locally Spike Sparse model and idealized Random Forest assumptions, LocalLSSFind provably recovers the signed features and interactions driving a single test prediction.

Reference graph

Works this paper leans on

4 extracted references · 3 canonical work pages · cited by 3 Pith papers

  1. [1]

    MDI regresses only on in-bag samplesΨ(X∗;S)

    Obtain enhanced representation.Each tree in an RF is fit on a bootstrapped datasetD∗ = (X∗,y∗). MDI regresses only on in-bag samplesΨ(X∗;S). MDI+ instead appends the raw featurexk∈R n to the feature map consisting of both in-bag and out-of-bag samples, yielding the transformed representation ˜Ψ(k) (X) = ˜Ψ ( X;S (k)) = [Ψ ( X;S (k)) ,xk]

    work page

  2. [2]

    Fit regularized GLM.Instead of using OLS, fit a regularized GLMG with link functiong and penalty λby regressing responseyon the transformed data ˜Ψ(X) = ˜Ψ(X;S)

    work page

  3. [3]

    Make partial model predictions.Let ¯˜Ψ(j) denote the vector of average values for features in˜Ψ(j)(X). Fork= 1,...,p, we then define the partial model predictions for each samplexi to be ˆy(k) i =g−1 ([¯˜Ψ(1),..., ¯˜Ψ(k−1), ˜Ψ(k)(xi), ¯˜Ψ(k+1),..., ¯˜Ψ(p) ] ˆβ−i,λ +αλ ) ,(5) where ˆβ−i,λ is the LOO coefficient vector learned without samplexi. 4.Evaluate p...

    work page 2013

  4. [1995]

    why should i trust you?

    Springer Berlin Heidelberg. ISBN 978-3-540-49195-8. Leo Breiman. Random forests.Machine Learning, 45(1):5–32, 2001. Yury Gorishniy, Ivan Rubachev, Valentin Khrulkov, and Artem Babenko. Revisiting deep learning models for tabular data.Advances in neural information processing systems, 34:18932–18943, 2021. Ravid Shwartz-Ziv and Amitai Armon. Tabular data: ...

    arXiv 2001

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.