Pith. sign in

REVIEW 3 major objections 5 minor 51 references

Decorrelated feature importance from local sample weighting

T0 review · 3 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that local sample weighting, which decorrelates each target feature from the rest before evaluating it, makes feature importance scores reflect true signal features under feature correlation, and can often improve out-of-di

desk verdict Local reweighting is a genuine new idea with broad simulations, but the missing link between split-level weights and the final importance metric leaves the headline improvement partly uninterpreted. read the letter →

arxiv 2508.06337 v1 pith:N3EVJTRH submitted 2025-08-08 stat.ML cs.LGstat.ME

Benedikt Fröhlich , Alison Durst , Merle Behr This is my paper · ORCID
classification stat.MLcs.LGstat.ME MSC 62G0862H2068T07
keywords featureimportancecorrelationsampleweightingpropensityscorerandomforestneuralnetworksinterpretabilityout-of-distribution
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes local sample weighting (losaw), a training-time reweighting scheme that decorrelates a target feature from the other features before the model evaluates its importance. The scheme is motivated by inverse probability weighting: each observation is weighted by the inverse stabilized propensity score of the target feature, creating a pseudo-population in which that feature is independent of the rest. Because noise features that only matter through correlation have constant marginal effect in this pseudo-population, feature importance scores computed from the reweighted data reflect the true signal features rather than correlated noise. The method comes with a natural tuning parameter, the minimum relative effective sample size eta, which controls an interpretation-prediction tradeoff. The authors integrate losaw into random forest split selection and into neural network mini-batch training, and show in simulations that the resulting feature importances separate signal from noise far better, with little or no loss in predictive accuracy.

What carries the argument

Inverse stabilized propensity score weighting: for a target feature X_p, each observation receives weight w_p(x) = P(X_p=x_p)/P(X_p=x_p | X_{−p}=x_{−p}), which is exactly the inverse of the stabilized propensity score from causal inference. The central identity is that the weighted pseudo-population follows the product distribution P_p ⊗ P_{−p}, in which X_p is independent of all other features. The weights are estimated by fitting a regression (continuous) or multinomial logistic model (discrete) predicting X_p from an adjustment set, and are then capped and redistributed to guarantee a minimum relative effective sample size eta via the Kish effective-sample-size formula; eta is the knob th

What would settle it

Train losawRF on data where the target feature depends nonlinearly on confounders (e.g., X_p = $X_1^{2}$ + X_2 + ε) while the response depends on X_1 and X_2, with the propensity model kept as linear regression; if noise features correlated with X_p still receive high feature importance, the decorrelation guarantee fails. Alternatively, compute the empirical correlation between X_p and each adjustment feature in the weighted sample and check whether it is actually near zero.

Watch

Extended reading notes

Core claim

The central claim is that locally reweighting the training sample so that, for each target feature, the feature is independent of the remaining features corrects a known failure mode of feature importance statistics: under feature correlation importance is redistributed from signal features to correlated noise features, sometimes making a pure noise feature the top-ranked one. The paper proves that weighting by the inverse stabilized propensity score w_p(x) = P(X_p=x_p)/P(X_p=x_p | X_{−p}=x_{−p}) makes the weighted population follow the product distribution P_p ⊗ P_{−p}, and that under this distribution every noise feature has a constant marginal effect function. It then integrates these wei

Load-bearing premise

The reweighting only decorrelates the target feature if the estimated propensity scores are correct, which requires the adjustment feature set to contain the true confounders and the chosen regression/logistic model to be correctly specified; if those fail, the weighted pseudo-population is still correlated and the importance gains lose their basis.

Editorial extensions

If this is right

  • Under correlated features, MDI and gradient-based feature importances from losawRF/losawGD separate signal from noise reliably, where classical RF/CNN importances fail (e.g., pr-AUC 0.417 to 0.999 for f3, P=100 discrete, N=5000).
  • The method generalizes beyond RF and CNN: any tree-based split selection and any mini-batch gradient learner can incorporate the same local weighting.
  • Predictions on data with independent features (a distribution shift) are often more accurate for losawRF than for RF, while in-distribution R² is maintained within 0.018.
  • The eta parameter gives practitioners a principled dial between interpretability and predictive performance, analogous to a bias-variance tradeoff.
  • losaw can be combined with existing debiasing approaches for feature importance (e.g., conditional permutation importance) rather than only replacing them.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct testable extension: use losaw-style weighting with any model-agnostic importance (e.g., LOCO or Shapley values) by computing weights per feature on a held-out validation set, without retraining; the decorrelation guarantee suggests these would inherit the same improvement.
  • The initial adjustment-feature selection via RF MDI is the weak point: since MDI itself is biased under correlation, the chosen adjustment set may omit true confounders; an iterative or correlation-matrix-based selection could make the method more robust.
  • The theory covers regression with additive independent noise; extending to classification and to heteroscedastic or non-additive noise would require re-deriving the marginal-effect argument, since the conditional-mean argument may no longer hold.
  • In high-dimensional settings with P >> N, the per-feature propensity models are unstable; the paper's proposal to restrict to Q adjustment features is necessary, but the choice of Q and the correlation threshold 0.1 are ad hoc and could be tuned per dataset.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes local sample weighting (losaw), a training-time reweighting scheme that decorrelates a target feature from the remaining features, motivated by inverse probability weighting in causal inference. It is integrated into random forests (losawRF, via feature-specific weights in split selection) and into mini-batch gradient descent for neural networks (losawGD, via sampling mini-batches from weighted populations). A tuning parameter η controls the minimum relative effective sample size, providing an interpretation-prediction tradeoff. The theoretical motivation is that under population losaw weights the target feature is independent of the other features, and Lemma 2.6 shows that noise features have a constant marginal effect function f_eff. The paper reports simulation studies across seven regression functions, continuous and discrete features, several dimensions and sample sizes, and compares losawRF vs RF and losawGD vs standard CNN training. Main claims are that losaw improves feature importance scores under feature correlation and often improves out-of-distribution R² while maintaining in-distribution accuracy.

Significance. If the claims hold, the paper contributes a practical, training-integrated method for decorrelated feature importance, with a natural tuning parameter and potential extension to many ML models. The simulation design is broad and the code is made available. The paper also contains correct proofs for the weight-redistribution algorithm and for complexity bounds. However, the key evidence is simulation-only, and the central comparison conflates the reweighting scheme with a change in the feature-importance aggregation. The lack of a theoretical link from the proposed FI estimator to the decorrelation target, together with the absence of uncertainty quantification, currently weakens the strength of the conclusions.

major comments (3)
  1. [Definition 4.6 / Section 4.5] The losawRF feature importance metric is not shown to estimate the marginal effect function f_eff, and it differs from standard MDI by the extra MSE_k factor. In the uniform-weight limit it reduces to standard MDI, but simulations use η=0.25, so both the split selection and the FI aggregation change. The paper motivates MSE_k as a scaling device to make contributions comparable to standard MDI, but this factor changes the relative contributions of nodes/features whenever the weighted MSE used in split selection differs from the node response variance. Thus the impressive pr-AUC gains (e.g., Table 3, f3: 0.417 to 0.999) could be driven partly by the metric change rather than by decorrelation. Please provide ablations: (i) standard MDI applied to losawRF trees, (ii) Definition 4.6 applied to standard RF trees, (iii) η=1 (uniform weights) comparison, and/or a theoretical statement connectin
  2. [Remark 4.9 / Section 4.10] The adjustment-feature set for propensity estimation is selected using initial RF MDI and a correlation threshold. This selection can fail precisely in the settings the method targets: when a noise feature is highly correlated with multiple signal features, RF MDI often ranks that noise feature first (see Example 1.1), so the top-Q set may omit true confounders. Since the decorrelation property that underlies Lemma 2.6 requires a sufficient adjustment set, this is a load-bearing assumption. The manuscript does not analyze this failure mode or provide sensitivity analyses (varying Q, threshold, or using an oracle adjustment set). Please report such robustness checks or state conditions under which the initial-MDI selection is valid.
  3. [Section 4.10 / Section 5.3] All simulation results are averages over Monte Carlo runs without standard errors or confidence intervals. Some conclusions rely on small differences (e.g., Table 5, R²_ind for f9: 0.106 vs 0.068; FIgap differences of about 0.02). Without uncertainty quantification, it is difficult to assess whether the reported improvements are systematic. Please report MC standard deviations, confidence intervals, or paired-comparison statistics, at least for the primary metrics (pr-AUC, R²_ind).
minor comments (5)
  1. [Definition 3.1 / Algorithm 3.2] The parameter η is introduced as η∈[0,1] in Definition 3.1 but Algorithm 3.2 and Equation (2) use η∈(0,1]; please make the domain consistent.
  2. [Section 4.10] There is a typo: 'Them try hyperparameter' should be 'The m_try hyperparameter'.
  3. [Appendix 9.14] Figure 5 caption says input shape (1000,1,1), while the text says 'tabular data with 3000 features'. This is inconsistent and should be fixed.
  4. [Remark 9.10] 'We therefore advice to restrict' should be 'advise'.
  5. [Figure 2] The right panel shows '95% coverage bands' but the method for constructing these bands is not described in the caption or text; please clarify how they are computed.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the reweighting derivation is definitional and the FI metric reduces to standard MDI in the uniform limit; simulation gains are empirical.

full rationale

The paper's central derivation is self-contained. The population losaw weights (Def. 2.1) are constructed so that the weighted pseudo-population equals P_p⊗P_-p; this is a definitional identity, not a prediction. Lemma 2.6 is a proved statement that noise features have constant marginal effect under the product distribution, and it is not equivalent to the input by construction. The losawRF split criterion (Def. 4.2) is a new algorithm, and the FI metric (Def. 4.6) reduces to standard MDI in the uniform-weight limit because Δ_rel = Δ/MSE and multiplying by MSE·N gives Δ·N. Thus the reported pr-AUC gains are not baked into the metric by definition; they are empirical comparisons on simulated data with known signal features. The only mildly self-referential step is Remark 4.9, where adjustment features for propensity estimation are selected using an initial RF MDI; however, this is a heuristic preprocessing choice, not a fitted parameter later reported as a prediction, and the final FI is not statistically forced by that initial MDI. The citation [36] with an overlapping author is used only to name LSS simulation models and is not load-bearing. No step in the derivation chain reduces to its own inputs or to an unverified self-citation, so no circularity is found.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The method does not introduce new entities; the pseudo-population is a standard IPW construct, and losaw is an algorithmic procedure. The main extra commitments are the propensity-model assumptions and the MDI-based adjustment-feature heuristic, both of which could fail silently under correlation-induced bias.

free parameters (3)
  • eta (minimal relative effective sample size) = 0.25 (losawRF simulation), 0.2 (losawGD simulation)
    User-chosen tuning parameter controlling interpretation-prediction tradeoff; no data-driven selection rule provided, and results vary with it (Figure 2).
  • Q (number of adjustment features) = 10 in losawRF sim, P=1000 in losawGD sim
    Number of covariates used in propensity estimation; chosen by heuristic in Remark 4.9, affects which confounders enter the weights.
  • correlation threshold for adjustment features = 0.1
    Used to filter adjustment features by absolute correlation with the target feature; hand-picked, not tuned or justified.
assumptions (5)
  • domain assumption Additive response model Y=f(X)+epsilon with E[epsilon|X]=0 and epsilon independent of X
    Stated in Section 2; central to defining signal/noise and marginal effects.
  • domain assumption Positivity/overlap: conditional density of X_p given X_{-p} is positive wherever the marginal density is positive
    Implicit in Definition 2.1; inverse probability weights are only well-defined and stable under this condition; not discussed or checked.
  • domain assumption Propensity models are correctly specified (linear regression with normal residuals for continuous, multinomial logistic for discrete)
    Section 4.10; if misspecified, the weighted sample does not achieve independence, degrading decorrelation.
  • ad hoc to paper Initial RF MDI top-Q plus correlation threshold selects a sufficient adjustment set
    Remark 4.9; this heuristic assumes the initial MDI, which is itself biased under correlation, still identifies the relevant confounders.
  • ad hoc to paper Greedy weighted impurity splits approximate the marginal effect function f_eff
    Section 2.2 and 4 present this as motivation for MDI-type FI, but no theorem links the finite-sample greedy tree procedure to f_eff.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Decorrelated feature importance from local sample weighting." pith.science (2026). https://pith.science/paper/N3EVJTRH

@misc{pith2026250806337,
  author       = {Pith},
  title        = {Pith review of: Decorrelated feature importance from local sample weighting},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/N3EVJTRH}},
  note         = {Machine review of arXiv:2508.06337}
}
read the original abstract

Feature importance (FI) statistics provide a prominent and valuable method of insight into the decision process of machine learning (ML) models, but their effectiveness has well-known limitations when correlation is present among the features in the training data. In this case, the FI often tends to be distributed among all features which are in correlation with the response-generating signal features. Even worse, if multiple signal features are in strong correlation with a noise feature, while being only modestly correlated with one another, this can result in a noise feature having a distinctly larger FI score than any signal feature. Here we propose local sample weighting (losaw) which can flexibly be integrated into many ML algorithms to improve FI scores in the presence of feature correlation in the training data. Our approach is motivated from inverse probability weighting in causal inference and locally, within the ML model, uses a sample weighting scheme to decorrelate a target feature from the remaining features. This reduces model bias locally, whenever the effect of a potential signal feature is evaluated and compared to others. Moreover, losaw comes with a natural tuning parameter, the minimum effective sample size of the weighted population, which corresponds to an interpretation-prediction-tradeoff, analog to a bias-variance-tradeoff as for classical ML tuning parameters. We demonstrate how losaw can be integrated within decision tree-based ML methods and within mini-batch training of neural networks. We investigate losaw for random forest and convolutional neural networks in a simulation study on settings showing diverse correlation patterns. We found that losaw improves FI consistently. Moreover, it often improves prediction accuracy for out-of-distribution, while maintaining a similar accuracy for in-distribution test data.

Figures

Figures reproduced from arXiv: 2508.06337 by the authors.

Figure 1
Figure 1. Mean decrease in impurity (MDI) feature importance scores of random forest [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Left: local sample weighting (losaw) with tuning parameter [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Illustration of losawRF on two features. At each node sample, feature-specific [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Gradient descent training with losaw, depicted schematically. In each gradient [PITH_FULL_IMAGE:figures/full_fig_p024_4.png]
Figure 5
Figure 5. Figure 5: Architecture of the NN model used in this study in both training algorithms. The [PITH_FULL_IMAGE:figures/full_fig_p042_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

51 extracted references · 28 canonical work pages

  1. [1]

    Importance of interpretability in healthcare,

    A. Vellido, “Importance of interpretability in healthcare,”Neural Computing and Applications, vol. 32, no. 24, pp. 18 069–18 083, 2020.doi:10.1007/s00521- 019- 04051-w

    doi:10.1007/s00521- 2020

  2. [2]

    On the impor- tance of interpretable machine learning predictions to inform clinical decision making in oncology,

    S.-C. Lu, C. L. Swisher, C. Chung, D. Jaffray, and C. Sidey-Gibbons, “On the impor- tance of interpretable machine learning predictions to inform clinical decision making in oncology,”Frontiers in Oncology, vol. 13, 2023.doi:10.3389/fonc.2023.1129380

    work page arXiv 2023

  3. [3]

    Visualization of neural networks using saliency maps,

    N. Morch et al., “Visualization of neural networks using saliency maps,” inProceed- ings of ICNN’95 - International Conference on Neural Networks, vol. 4, IEEE, 1995, pp. 2085–2090.doi:10.1109/ICNN.1995.488997 29 REFERENCES REFERENCES

    work page arXiv 1995

  4. [4]

    Hastie, R

    T. Hastie, R. Tibshirani, and J. Friedman,The Elements of Statistical Learning (Springer Series in Statistics), 2nd ed. New York, NY: Springer, 2009.doi:10.1007/ 978-0-387-84858-7

    work page 2009

  5. [5]

    Explaining prediction models and individual predic- tions with feature contributions,

    E. ˇStrumbelj and I. Kononenko, “Explaining prediction models and individual predic- tions with feature contributions,”Knowledge and Information Systems, vol. 41, no. 3, pp. 647–665, 2014.doi:10.1007/s10115-013-0679-x

    doi:10.1007/s10115-013-0679-x 2014

  6. [6]

    Random Forests,

    L. Breiman, “Random Forests,”Machine Learning, vol. 45, no. 1, pp. 5–32, 2001.doi: 10.1023/A:1010933404324

    doi:10.1023/a:1010933404324 2001

  7. [7]

    Distribution-Free Predictive Inference for Regression,

    J. Lei, G. Max, R. Alessandro, T. Ryan J., and L. Wasserman, “Distribution-Free Predictive Inference for Regression,”Journal of the American Statistical Association, vol. 113, no. 523, pp. 1094–1111, 2018.doi:10.1080/01621459.2017.1307116

    arXiv 2018

  8. [8]

    Bias in random forest variable importance measures: Illustrations, sources and a solution,

    C. Strobl, A.-L. Boulesteix, A. Zeileis, and T. Hothorn, “Bias in random forest variable importance measures: Illustrations, sources and a solution,”BMC Bioinformatics, vol. 8, no. 25, 2007.doi:10.1186/1471-2105-8-25

    doi:10.1186/1471-2105-8-25 2007

Show all 51 references
  1. [9]

    Disentangling Interactions and Depen- dencies in Feature Attribution,

    G. K¨ onig, E. G¨ unther, and U. von Luxburg, “Disentangling Interactions and Depen- dencies in Feature Attribution,”arXiv preprint, 2024.doi:10.48550/arXiv.2410. 23772

    2024 doi

  2. [10]

    Do little interactions get lost in dark random forests?

    M. N. Wright, A. Ziegler, and I. R. K¨ onig, “Do little interactions get lost in dark random forests?”BMC Bioinformatics, vol. 17, no. 145, 2016.doi:10.1186/s12859- 016-0995-8

    2016 doi

  3. [11]

    On the trustworthiness of tree ensemble explainability methods,

    A. Yasodhara, A. Asgarian, D. Huang, and P. Sobhani, “On the trustworthiness of tree ensemble explainability methods,” inMachine Learning and Knowledge Extraction, Springer International Publishing, 2021, pp. 293–308.doi:10 . 1007 / 978 - 3 - 030 - 84060-0_19

    2021

  4. [12]

    Conditional vari- able importance for random forests,

    C. Strobl, A.-L. Boulesteix, T. Kneib, T. Augustin, and A. Zeileis, “Conditional vari- able importance for random forests,”BMC Bioinformatics, vol. 9, no. 307, 2008.doi: 10.1186/1471-2105-9-307

    2008 doi

  5. [13]

    Correlation and variable importance in random forests,

    B. Gregorutti, B. Michel, and P. Saint-Pierre, “Correlation and variable importance in random forests,”Statistics and Computing, vol. 27, no. 3, pp. 659–678, 2017.doi: 10.1007/s11222-016-9646-1

    2017 doi

  6. [14]

    Breiman, F

    L. Breiman, F. H. Jerome, O. A. Richard, and S. J. Charles,Classification and re- gression trees, 1st ed. Boca Raton, Fla.: Chapman & Hall/CRC, 1984.doi:10.1201/ 9781315139470

    1984

  7. [15]

    Stable learning establishes some common ground between causal inference and machine learning,

    P. Cui and S. Athey, “Stable learning establishes some common ground between causal inference and machine learning,”Nature Machine Intelligence, vol. 4, no. 2, pp. 110– 115, 2022.doi:10.1038/s42256-022-00445-z

    2022 doi

  8. [16]

    Stable Learning via Differ- entiated Variable Decorrelation,

    Z. Shen, P. Cui, J. Liu, T. Zhang, B. Li, and Z. Chen, “Stable Learning via Differ- entiated Variable Decorrelation,” inProceedings of the 26th ACM SIGKDD Interna- tional Conference on Knowledge Discovery & Data Mining, ser. KDD ’20, Virtual Event, CA, USA: Association for Com...

    2020

  9. [17]

    Stable Learning via Sample Reweighting,

    Z. Shen, P. Cui, T. Zhang, and K. Kunag, “Stable Learning via Sample Reweighting,” Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, no. 4, pp. 5692– 5699, 2020.doi:10.1609/aaai.v34i04.6024

    2020 doi

  10. [18]

    Stable Learning via Sparse Variable Independence,

    H. Yu et al., “Stable Learning via Sparse Variable Independence,”Proceedings of the AAAI Conference on Artificial Intelligence, vol. 37, no. 9, pp. 10 998–11 006, 2023. doi:10.1609/aaai.v37i9.26303

    2023 doi

  11. [19]

    Stable Learning via Triplex Learning,

    S. Yang, T. Jiang, Q. Dang, L. Gu, and X. Wu, “Stable Learning via Triplex Learning,” IEEE Transactions on Artificial Intelligence, vol. 5, no. 10, pp. 5267–5276, 2024.doi: 10.1109/TAI.2024.3404411

    2024

  12. [20]

    Stable Prediction with Model Mis- specification and Agnostic Distribution Shift,

    K. Kuang, R. Xiong, P. Cui, S. Athey, and B. Li, “Stable Prediction with Model Mis- specification and Agnostic Distribution Shift,” inProceedings of the AAAI Conference on Artificial Intelligence, vol. 34, 2020, pp. 4485–4492.doi:10.1609/aaai.v34i04. 5876

    2020 doi

  13. [21]

    Propensity Score Stratification Methods for Continuous Treatments,

    D. W. Brown, T. J. Greene, M. D. Swartz, A. V. Wilkinson, and S. M. DeSantis, “Propensity Score Stratification Methods for Continuous Treatments,”Statistics in medicine, vol. 40, no. 5, pp. 1189–1203, 2021.doi:10.1002/sim.8835

    2021 doi

  14. [22]

    Marginal Structural Models and Causal Inference in Epidemiology,

    J. M. Robins, M. ´A. Hern´ an, and B. Brumback, “Marginal Structural Models and Causal Inference in Epidemiology,”Epidemiology, vol. 11, no. 5, pp. 550–560, 2000. doi:10.1097/00001648-200009000-00011

    doi

  15. [23]

    Decorrelated variable importance,

    I. Verdinelli and L. Wasserman, “Decorrelated variable importance,”Journal of Ma- chine Learning Research, vol. 25, no. 1, 7:412–7:438, 2024.doi:10.48550/arXiv. 2111.10853

    2024 doi arXiv

  16. [24]

    Learning generalizable agents via saliency-guided features decorrela- tion,

    S. Huang et al., “Learning generalizable agents via saliency-guided features decorrela- tion,”Advances in Neural Information Processing Systems, vol. 36, pp. 39 363–39 381, 2023.doi:10.48550/arXiv.2310.05086

    2023 doi arXiv

  17. [25]

    A theoretical analysis on independence- driven importance weighting for covariate-shift generalization,

    R. Xu, X. Zhang, Z. Shen, T. Zhang, and P. Cui, “A theoretical analysis on independence- driven importance weighting for covariate-shift generalization,” inProceedings of the 39th International Conference on Machine Learning, vol. 162, PMLR, 2022, pp. 24 803– 24 829.doi:10.485...

    2022 doi arXiv

  18. [26]

    Invariant Random Forest: Tree-Based Model Solution for OOD Generalization,

    Y. Liao, Q. Wu, and X. Yan, “Invariant Random Forest: Tree-Based Model Solution for OOD Generalization,”Proceedings of the AAAI Conference on Artificial Intelli- gence, vol. 38, no. 12, pp. 13 772–13 781, 2024.doi:10.1609/aaai.v38i12.29283

    2024 doi

  19. [27]

    Deep Stable Learning for Out-Of-Distribution Generalization,

    X. Zhang, P. Cui, R. Xu, L. Zhou, Y. He, and Z. Shen, “Deep Stable Learning for Out-Of-Distribution Generalization,” in2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), IEEE Computer Society, 2021, pp. 5368– 5378.doi:10.1109/CVPR46437.2021.00533

    2021

  20. [28]

    Panning for gold: ‘model-X’ knockoffs for high dimensional controlled variable selection,

    E. Cand` es, Y. Fan, L. Janson, and J. Lv, “Panning for gold: ‘model-X’ knockoffs for high dimensional controlled variable selection,”Journal of the Royal Statistical Society: Series B (Statistical Methodology), vol. 80, no. 3, pp. 551–577, 2018.doi: 10.1111/rssb.12265 31 REFE...

    2018 doi

  21. [29]

    A Novel Random Forest Variant Based on Intervention Correlation Ratio,

    T. Zhang, T. Li, Z. Xue, X. Lu, and L. Gao, “A Novel Random Forest Variant Based on Intervention Correlation Ratio,”IEEE Transactions on Emerging Topics in Com- putational Intelligence, vol. 8, no. 3, pp. 2541–2553, 2024.doi:10.1109/TETCI.2024. 3369320

    2024 doi

  22. [30]

    Training Diagonal Linear Networks with Stochastic Sharpness-Aware Minimization,

    G. Clara, S. Langer, and J. Schmidt-Hieber, “Training Diagonal Linear Networks with Stochastic Sharpness-Aware Minimization,”arXiv preprint, 2025.doi:10 . 48550 / arXiv.2503.11891

    2025 doi arXiv

  23. [31]

    Sharpness-aware minimization for efficiently improving generalization,

    P. Foret, A. Kleiner, H. Mobahi, and B. Neyshabur, “Sharpness-aware minimization for efficiently improving generalization,”arXiv preprint, 2020.doi:10.48550/arXiv. 2010.01412

    2020 doi arXiv

  24. [32]

    Simplifying neural nets by discovering flat min- ima,

    S. Hochreiter and J. Schmidhuber, “Simplifying neural nets by discovering flat min- ima,”Advances in neural information processing systems, vol. 7, pp. 529–536, 1994. doi:10.5555/2998687.2998753

    1994

  25. [33]

    Testing conditional independence in supervised learning algorithms,

    D. S. Watson and M. N. Wright, “Testing conditional independence in supervised learning algorithms,”Machine Learning, vol. 110, no. 8, pp. 2107–2129, 2021.doi: 10.1007/s10994-021-06030-6

    2021 doi

  26. [34]

    Survey sampling,

    L. Kish, “Survey sampling,”Social Forces, vol. 45, no. 1, pp. 132–133, 1966.doi: 10.1093/sf/45.1.132-a

    1966 doi

  27. [35]

    Iterative random forests to discover predictive and stable high-order interactions,

    S. Basu, K. Kumbier, J. B. Brown, and B. Yu, “Iterative random forests to discover predictive and stable high-order interactions,”Proceedings of the National Academy of Sciences, vol. 115, no. 8, pp. 1943–1948, 2018, Publisher: Proceedings of the National Academy of Sciences.d...

    1943 doi

  28. [36]

    Provable boolean interaction recovery from tree ensemble obtained via random forests,

    M. Behr, Y. Wang, X. Li, and B. Yu, “Provable boolean interaction recovery from tree ensemble obtained via random forests,”Proceedings of the National Academy of Sciences, vol. 119, no. 22, e2118636119, 2022.doi:10.1073/pnas.2118636119

    2022 doi

  29. [37]

    Scikit-learn: Machine learning in Python,

    F. Pedregosa et al., “Scikit-learn: Machine learning in Python,”Journal of Machine Learning Research, vol. 12, pp. 2825–2830, 2011.doi:10.5555/1953048.2078195

    2011

  30. [38]

    Large-scale machine learning with stochastic gradient descent,

    L. Bottou, “Large-scale machine learning with stochastic gradient descent,” inPro- ceedings of COMPSTAT’2010, Y. Lechevallier and G. Saporta, Eds., Heidelberg: Physica-Verlag HD, 2010, pp. 177–186.doi:10.1007/978-3-7908-2604-3_16

    2010 doi

  31. [39]

    Deep inside convolutional networks: Vi- sualising image classification models and saliency maps,

    K. Simonyan, A. Vedaldi, and A. Zisserman, “Deep inside convolutional networks: Vi- sualising image classification models and saliency maps,” inInternational Conference on Learning Representations, ICLR 2014, 2014.doi:10.48550/arXiv.1312.6034

    2014 doi arXiv

  32. [40]

    Tensorflow: A system for large-scale machine learning,

    M. Abadi et al., “Tensorflow: A system for large-scale machine learning,” in12th USENIX symposium on operating systems design and implementation (OSDI 16), 2016, pp. 265–283.doi:10.48550/arXiv.1605.08695

    2016 doi arXiv

  33. [41]

    Gradient-based learning applied to document recognition,

    Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,”Proceedings of the IEEE, vol. 86, no. 11, pp. 2278–2324, 1998. doi:10.1109/5.726791

    1998 doi

  34. [42]

    Goodfellow, Y

    I. Goodfellow, Y. Bengio, A. Courville, and Y. Bengio,Deep learning. MIT press Cambridge, 2016, vol. 1. 32 REFERENCES REFERENCES

    2016

  35. [43]

    Adam: A method for stochastic optimization,

    K. D. P. and J. Ba, “Adam: A method for stochastic optimization,” in3rd Interna- tional Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015.doi:10.48550/arXiv.1412. 6980

    2015 doi

  36. [44]

    Model-Agnostic Confidence Intervals for Fea- ture Importance: A Fast and Powerful Approach Using Minipatch Ensembles,

    L. Gan, L. Zheng, and G. I. Allen, “Model-Agnostic Confidence Intervals for Fea- ture Importance: A Fast and Powerful Approach Using Minipatch Ensembles,”arXiv preprint, 2022.doi:10.48550/arXiv.2206.02088

    2022 doi

  37. [45]

    C. M. Bishop,Pattern Recognition and Machine Learning (Information Science and Statistics). Berlin, Heidelberg: Springer-Verlag, 2006, vol. 1

    2006

  38. [46]

    Optimization methods for large-scale ma- chine learning,

    L. Bottou, F. E. Curtis, and J. Nocedal, “Optimization methods for large-scale ma- chine learning,”SIAM review, vol. 60, no. 2, pp. 223–311, 2018.doi:10 . 1137 / 16M1080173 33 9 APPENDIX 9 Appendix 9.1 Details on split point selection algorithm The split point selection algori...

    2018

  39. [47]

    Moreover, the runtime complexity of the algorithm is given byO(KN k)

    The split pointxcomputed by Algorithm 9.4 satisfies x= arg max x∈{x1,...,xK } ∆rel w (x, p). Moreover, the runtime complexity of the algorithm is given byO(KN k). Proof.For both parts, it follows directly from Lemma 9.2 that the value ∆ temp computed in the respective for-loop...

  40. [48]

    In combination with a sorting algorithm (which is ofO(N k logN k) complexity), we can compute the optimal splitting point in aO(N k logN k) time

    The updates within the for-loop are ofO(1), so in combination with the computation ofSandTin the beginning, the entire algorithm has a complexity ofO(N k). In combination with a sorting algorithm (which is ofO(N k logN k) complexity), we can compute the optimal splitting point...

  41. [49]

    Likewise, in the for-loop, the setLand left child weightWcan be computed in O(Nk) time, whereas the remaining steps are of constant complexity

    The sumsSandTcan be computed inO(N k) time by iterating over the sample. Likewise, in the for-loop, the setLand left child weightWcan be computed in O(Nk) time, whereas the remaining steps are of constant complexity. As this loop consists ofKiterations, we end up with a comple...

  42. [50]

    We therefore advice to restrict to chooseηwithin the interval(0, 2 3 ]for applications of losaw

    Forη= 1all sample weights are uniform and losawRF is identical to RF but requires higher computational cost to include the estimation of propensities and the weight modifi- cation algorithms. We therefore advice to restrict to chooseηwithin the interval(0, 2 3 ]for application...

  43. [51]

    Each of these linear models requires the covariance matrix of theQadjustment features, so it suffices to compute this matrix a single time inO(Q 2Nk) complexity

    For the estimation of propensity scores with continuous features, a linear regression model has to be fitted for each available splitting feature. Each of these linear models requires the covariance matrix of theQadjustment features, so it suffices to compute this matrix a sin...

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.