Pith. sign in

REVIEW 5 major objections 5 minor 81 references

Reliable and scalable variable importance estimation via warm-start and early stopping

T0 review · 5 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that variable importance can be estimated from one full model fit plus short warm-started, early-stopped updates, without retraining per feature subset.

desk verdict A practical warm-start early-stopping scheme for variable importance with a serious theoretical attempt, but the headline guarantee only holds under an RKHS membership assumption that the verification simulation is built to satisfy. read the letter →

arxiv 2412.01120 v2 pith:AC2C2RAJ submitted 2024-12-02 stat.ML cs.LG

Zexuan Sun , Garvesh Raskutti This is my paper · ORCID
classification stat.MLcs.LG
keywords variableimportanceearlystoppingwarm-startShapleyvaluesneuraltangentkernelgradientboostingdecisiontreesiterativeupdatemethods
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Variable importance asks how much a set of features contributes to prediction, but answering it usually means retraining the model once per feature subset, which is prohibitive when there are many variables. This paper claims that for any gradient-based learner whose updates can be written as an iterative kernel update equation, covering neural networks and gradient-boosted decision trees, one can train the full model once and then, for each subset, initialize from that full model with the features dropped and stop after a few gradient steps. The authors prove that the early-stopped model tracks the model that would have been trained on the reduced data, with squared error $O(N^{-1/2})$ in sample size, matching the precision of full retraining while using far fewer iterations. If the claim holds, variable importance and Shapley values become computationally feasible in high dimensions without sacrificing accuracy.

What carries the argument

The load-bearing object is the iterative kernel update equation (14), $f_{\tau+1}(X^{(I)}) = (I - \epsilon K_\tau^{(I)}) f_\tau(X^{(I)}) + \epsilon K_\tau^{(I)} Y$, which rewrites both neural-network gradient descent (via the neural tangent kernel) and gradient-boosted trees (via a tree-induced kernel) as a linear recursion on the empirical kernel matrix. Lemma 1 decomposes the squared error into three terms: bias from not running long enough, variance from noise, and a difference term $D_\tau^2$ caused by the kernel evolving during training. The stopping rule is formalized through the local empirical Rademacher complexity $\widehat{R}_K(\varrho)$, which sets the maximal horizon $\widehat{T}_{\max}$; the optimum $\widehat{T}_{\mathrm{op}}$ balances the decreasing bias and variance with the non-decreasing difference term. Warm-start enters by reparameterizing the recursion around the dropout error $e^{(I)} := f_{0,-I}(X^{(I)}) - f_N^c(X^{(I)})$, so the analysis reduces to early stopping from zero on the shifted target $f_{0,-I} - f_N^c$.

What would settle it

Construct a regression where the reduced target $f_{0,-I}$ is deliberately outside the span of the reduced-feature kernel, for example a function with an interaction between a dropped and a kept feature, and measure $\|f_{\widehat{T}} - f_{0,-I}\|_N^2$ as $N$ grows; if the error fails to decay at the claimed $O(N^{-1/2})$ rate, the scope of Theorem 1 is falsified.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central claim is Theorem 1: under Assumptions 4.1–4.5, a gradient descent or gradient boosting model started from the full-model fit $f_N^c$ and stopped at a data-dependent time $\widehat{T}_{\mathrm{op}}$ satisfies $\|f_{\widehat{T}_{\mathrm{op}}} - f_{0,-I}\|_N^2 \le O(N^{-1/2})$ with high probability, with an analogous $L^2(P)$ bound in Theorem 2. The estimator built from this reduced-model fit, $\widehat{VII}$, therefore achieves the same accuracy as retraining while starting from the dropout solution and moving only a few steps toward the dropped-feature target. For neural networks the same bound follows by bounding kernel drift and linearization error in terms of width, and for gradient-boosted trees by taking the random-strength parameter large enough; in the neural-network case the VI estimator is additionally asymptotically normal when the truth is nonzero, yielding Wald-type confidence intervals.

Load-bearing premise

The proof assumes that both the full model fit and the true reduced-model regression function lie exactly in the span of the kernel built from the remaining features; the paper states this is purely for theoretical convenience, but if the true function has components outside that span, the claimed $O(N^{-1/2})$ rates need not follow.

Editorial extensions

If this is right

  • Variable importance for every feature subset can be obtained from one full training run plus a short warm-started run per subset, reducing the cost of model-agnostic interpretability from many retrainings to roughly one training run plus short updates.
  • Shapley-value estimation inherits the same speedup: the paper reports a 2–3 times faster runtime than retraining on its logistic-model simulation while matching retrain accuracy.
  • For sufficiently wide ReLU networks, $\widehat{VII}$ is asymptotically normal when the true variable importance is nonzero, so Wald-type confidence intervals can be constructed without additional model fits.
  • For gradient-boosted trees with large random strength, the same guarantees hold, so tree ensembles can use the accelerated estimator rather than per-subset retraining.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural testable extension is to compare the practical hold-out stopping rule used in the experiments with the theoretically derived $\widehat{T}_{\mathrm{op}}$ on the same problems; the theory motivates the rule, but the paper does not quantify the gap between them.
  • The same warm-start-plus-early-stopping mechanism should transfer to any iterative predictor with a tractable tangent kernel, such as convolutional or attention-based networks, since the proof only requires the update to be kernel-like and the kernel drift to be controllable; this transfer is an inference, not a claim in the paper.
  • Because dropout overestimates variable importance under feature correlation in the paper's simulations, replacing dropout with one or two warm-started gradient steps may offer a cheap correction in existing interpretation pipelines.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes a warm-start plus early-stopping method for estimating variable importance without retraining a model for every dropped-feature subset. Starting from the fully trained model, the method runs gradient descent or gradient boosting on the feature-dropped data for a short time, using a stopping rule that is motivated by early-stopping theory for kernel methods. The main theoretical claims are Theorem 1, an O(N^{-1/2}) empirical-norm error bound for the stopped model under Assumptions 4.1-4.5, and Theorem 2, an analogous population-norm bound. The results are specialized to wide neural networks through the neural tangent kernel and to a simplified gradient-boosted decision tree algorithm, and an asymptotic normality result is claimed for VI estimation with neural networks. Simulations and a gas-turbine data example compare the method with dropout and retraining baselines.

Significance. The proposed computational strategy is valuable: if the theoretical guarantee held in realistic settings, variable importance could be estimated at the cost of one full training run plus short warm-start runs, with accuracy close to full retraining. The paper has concrete strengths: Lemma 1 gives a clean error decomposition into bias, variance, and kernel-drift terms; the connection to the early-stopping framework of Raskutti et al. and to the GBDT kernel of Ustimenko et al. is explicit; and the authors provide code and reproducible experiments. However, the significance is conditional because the main theorems require the target and the full model to lie in the dropped-feature RKHS, rely on an oracle stopping time that uses unknown quantities, and leave the width requirement for neural networks unquantified. These gaps mean the practical reliability claim is not yet established by the theory.

major comments (5)
  1. [§4.2, Assumption 4.1 and Lemma 1] The O(N^{-1/2}) bound in Theorem 1 requires both f_N^c and f0,-I to lie in span{K(I)(·, X_-I)}. This assumption is exactly what makes the null-space bias term 2∑_{j=r+1}^N (ζ*_jj)^2 in Lemma 1 vanish; the paper calls it 'purely theoretical convenience' and Section 5.1/D.3 deliberately constructs f0 and computes C_H under this membership. For a real regression function outside this span the null-space bias need not decay, so the stated guarantee does not cover the misspecified setting that the method is intended for. The manuscript should either prove a misspecification bias bound or explicitly restrict the reliability claim to the RKHS-membership case.
  2. [§6.1.2, proof of Theorem 2] The population-norm proof contains the sentence 'Here we assume that ∥f^δ_bTop(·)∥_2^2 can be upper bounded by O(1/√N)' after Lemma 4. This is the key bound needed to control the varying-kernel term in the random-design case, and it is assumed rather than proved in the general framework. The later Lemmas 5 and 7 supply such bounds only for the NN and GBDT examples, so Theorem 2 as stated for general kernel-based gradient methods is not established by the proof. This is load-bearing for the random-design claim.
  3. [§5, Algorithm 1, and Theorem 1] The theoretical stopping rule bTop/bTmax depends on C_H and σ, which are unknown in practice (eqs. (21)-(23)), while Algorithm 1 uses a validation-patience rule. No theorem or quantitative analysis connects the patience-based stopping time to the oracle bTop, so the empirical accuracy claims in Sections 5.2-5.7 do not follow from Theorem 1. The manuscript should add an explicit approximation result for the practical stopping rule or clearly label the empirical results as heuristic.
  4. [§4.3, Corollaries 1-2] The width m is only shown to exist, with no stated rate relative to N; the text says 'We do not specify a particular rate for m in relation to N.' To use the corollaries one needs a checkable overparameterization condition (e.g., m ≳ N^a) under which the linearization and NTK stability bounds hold; as written, the result is consistent with m depending on N in a way that the finite-width networks in the experiments (m=2048) do not satisfy. This makes the theorem's applicability to the implemented networks unclear.
  5. [§4.5, Corollary 5] The asymptotic normality result is proved only for the 0-bias ReLU network setting of Corollary 2, but the text then says 'For general networks, even though we do not provide rigorous theoretical results, we should expect the same asymptotical normality to hold,' and the CI experiments (Section 5.6) use general networks and GBDT. This is an extrapolation beyond the proven statement; either prove the extension or restrict the CI claim accordingly.
minor comments (5)
  1. [Keywords] The keyword list contains 'wart-start'; this should be 'warm-start'.
  2. [§3.1] The display defining val(S∪{j})−val(S) is malformed ('V ISj'); fix the formatting so the difference is clearly equal to VI_S^j.
  3. [§6.1.2, Lemma 4] The recursion in eq. (58) uses K(I)_τ(·, X(I)), while the induction display in eq. (84) uses K(I)(·, X(I)); reconcile the two definitions because the claimed induction depends on which kernel is used.
  4. [§5.1, Figure 1] The text says the observed population bound is 'closer to or even faster than O(N^{-1})' while Theorem 1/2 state O(N^{-1/2}); either report the observed slope as a heuristic or align the claim with the proven rate.
  5. [Appendix D.5] The cross-reference to 'Section 5.4' should point to the gas-turbine application (Section 5.7), since Section 5.4 is the LazyVI comparison.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the VI error bound is derived from explicit kernel early-stopping assumptions and is not a restatement of the estimator's inputs.

full rationale

I walked the derivation chain. Lemma 1 (Eq. 18) decomposes the warm-start error into bias, variance, and kernel-drift terms after rewriting the update as an iterative kernel equation; the bias term is expressed in terms of the shifted target f0,-I - f_c_N, which is the actual object being estimated, not a fitted quantity relabeled as a prediction. Theorem 1 then bounds these terms using the Raskutti et al. (2014) early-stopping machinery: the critical radius relation bR_K(bϱ_N) = bϱ_N^2 C_H^2/(2eσ) and Assumption 4.5 give bϱ_N^2 = O(N^{-1/2}), so the O(N^{-1/2}) rate is derived, not assumed in the stopping rule. The stopping time bTop minimizes a genuine bias-variance trade-off plus the kernel-drift term; it does not encode the final error by construction. The main caveats are scope limitations rather than circularity: Assumption 4.1, which requires f_c_N and f0,-I to lie in span{K(I)(·, X_-I)}, is explicitly called 'purely theoretical convenience' and is deliberately satisfied in the Section 5.1 simulation; Section D.3 further approximates C_H by assuming that membership. Likewise, the proof of Theorem 2 contains an explicit unproved assertion that ∥f^δ_bTop(·)∥_2^2 = O(N^{-1/2}), which is a missing argument, not a reduction of the conclusion to the hypothesis. The reliance on Raskutti et al. (2014) is a self-citation lineage because Raskutti is a co-author, but that prior work is an independent published early-stopping theorem with its own stated assumptions and does not assume the VI result of this paper; it supplies the bias-variance lemmas used here. I therefore find no equation that reduces to its own input and no fitted constant renamed as a prediction, so the central derivation is self-contained under its stated assumptions.

Assumptions & free parameters 4 free parameters · 9 assumptions · 0 invented entities

The central claim rests on strong domain assumptions: the target must lie in the RKHS of the dropped-feature kernel, the NTK matrix must be full rank, and the theoretical stopping rule requires oracle knowledge of the Hilbert-norm gap and noise variance. The GBDT guarantees additionally require very large random-strength values not used in the experiments. No new physical or algorithmic entities are postulated.

free parameters (4)
  • GBDT random strength beta = 10,000 in experiments; theory requires beta >= N^{5/4} or N^{15/4}
    Controls the difference between the evolving tree kernel and the stationary kernel; the proof of Corollaries 3 and 4 bounds the difference term only for such large beta. The experimental choice beta=10,000 with N=5000 violates the Corollary 4 condition.
  • Network width m = 2048 in experiments
    The NTK linearization error is O(m^{-1/2}) only for m at least some unspecified M; the paper does not quantify M, so the experiments choose 2048 by convention.
  • GBDT tree depth d = 2 in simulations, 8 in real data
    Bounds tr(K(I)) <= 2d and the finite-rank kernel class; chosen as a standard GBDT hyperparameter.
  • Validation patience P and split fraction q = q=0.75; P not specified
    Algorithm 1 uses a hold-out patience rule that is not analyzed by the theorems; P is a free heuristic parameter.
assumptions (9)
  • ad hoc to paper Assumption 4.1: f_N^c and f0,-I belong to H, i.e. span{K(I)(., X_-I)}
    Stated for theoretical convenience; if the target regression function is outside this span, the bias term in Lemma 1 does not vanish and Theorems 1 and 2 do not apply.
  • domain assumption Assumption 4.2: data lie in a closed bounded subset of R^{p+1}
    Used to ensure bounded NTK entries and kernel trace bounds.
  • domain assumption Assumption 4.3: noise variables are independent, zero-mean, sub-Gaussian
    Controls the variance term in the error decomposition via concentration inequalities from Raskutti et al. 2014.
  • domain assumption Assumption 4.4: dropout error satisfies ||Y - f_N^c(X(I))||_2 = O(sqrt(N))
    Bounds the initial error at warm start; the paper sketches why it holds for NNs but states it as an assumption.
  • domain assumption Assumptions 4.5 and 4.6: trace or eigenvalue sum of the kernel is O(1)
    Gives the critical radius decay needed for the O(N^{-1/2}) rate; it holds for finite-rank and polynomial-decay kernels.
  • domain assumption Assumption 4.7: empirical NTK matrix is full rank with 0 < lambda_min <= lambda_max < infinity
    Needed for the NTK convergence and linearization results; may fail for some architectures or data.
  • ad hoc to paper Assumption 4.9: full model f_N^c has the same architecture and training initialization as the reduced model
    Warm-start analysis treats f_N^c as a near-random NTK initialization; not guaranteed in practice.
  • ad hoc to paper Oracle stopping rule: bTop and bTmax require CH = ||f_N^c - f0,-I||_H and noise variance sigma
    The theoretical stopping time is not computable from data; the experiments use validation-based patience, so the proven rates do not directly cover the implemented algorithm.
  • domain assumption ReLU no-bias network with normalized inputs on the sphere for Corollary 2
    The population rate N^{-p/(p+1)} depends on known eigenvalue decays for zero-bias ReLU NTK on the hypersphere.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Reliable and scalable variable importance estimation via warm-start and early stopping." pith.science (2026). https://pith.science/paper/AC2C2RAJ

@misc{pith2026241201120,
  author       = {Pith},
  title        = {Pith review of: Reliable and scalable variable importance estimation via warm-start and early stopping},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AC2C2RAJ}},
  note         = {Machine review of arXiv:2412.01120}
}
read the original abstract

As opaque black-box predictive models become more prevalent, the need to develop interpretations for these models is of great interest. The concept of variable importance and Shapley values are interpretability measures that applies to any predictive model and assesses how much a variable or set of variables improves prediction performance. When the number of variables is large, estimating variable importance presents a significant computational challenge because re-training neural networks or other black-box algorithms requires significant additional computation. In this paper, we address this challenge for algorithms using gradient descent and gradient boosting (e.g. neural networks, gradient-boosted decision trees). By using the ideas of early stopping of gradient-based methods in combination with warm-start using the dropout method, we develop a scalable method to estimate variable importance for any algorithm that can be expressed as an iterative kernel update equation. Importantly, we provide theoretical guarantees by using the theory for early stopping of kernel-based methods for neural networks with sufficiently large (but not necessarily infinite) width and gradient-boosting decision trees that use symmetric trees as a weaker learner. We also demonstrate the efficacy of our methods through simulations and a real data example which illustrates the computational benefit of early stopping rather than fully re-training the model as well as the increased accuracy of our approach.

Figures

Figures reproduced from arXiv: 2412.01120 by the authors.

Figure 3
Figure 3. 14 [PITH_FULL_IMAGE:figures/full_fig_p014_3.png] view at source ↗
Figure 1
Figure 1. Log-log plot comparing theoretical bounds. [PITH_FULL_IMAGE:figures/full_fig_p015_1.png] view at source ↗
Figure 2
Figure 2. VI estimation comparison for correlated linear model. [PITH_FULL_IMAGE:figures/full_fig_p015_2.png] view at source ↗
Figures from the paper (7 more)
Figure 3
Figure 3. Figure 3: Distribution of computation time vs. normalized estimation error relative to retrain for the VI of [PITH_FULL_IMAGE:figures/full_fig_p016_3.png]
Figure 4
Figure 4. Figure 4: Distribution of computation time vs. normalized estimation error relative to retrain for the VI of [PITH_FULL_IMAGE:figures/full_fig_p016_4.png]
Figure 5
Figure 5. Figure 5: Shapley value estimation for logisitic model. [PITH_FULL_IMAGE:figures/full_fig_p017_5.png]
Figure 6
Figure 6. Figure 6: Wald type CI coverage experiment. 5.7 Predicting flue gas emissions Assessing variable importance in predicting CO and NOx emissions from gas turbines is crucial for optimizing efficiency, reducing environmental impact, and ensuring compliance. It helps operators adjus…
Figure 7
Figure 7. Figure 7: Flue gas emissions application. 6 Proofs We present the proofs of our primary results in this section. The key steps for each proof are outlined in the main text, while the more technical details are included in the appendix. 6.1 General algorithm 6.1.1 Proof of Theore…
Figure 8
Figure 8. Figure 8: Correlation heat map. (a) VI estimation (b) Shapley value estimation [PITH_FULL_IMAGE:figures/full_fig_p038_8.png]
Figure 9
Figure 9. Figure 9: Flue gas emissions variable importance estimation results using GBDT. [PITH_FULL_IMAGE:figures/full_fig_p038_9.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

81 extracted references · 58 canonical work pages

  1. [1]

    Why Are We Using Black Box Models in AI When We Don t Need To ? A Lesson From an Explainable AI Competition

    Cynthia Rudin and Joanna Radin. Why Are We Using Black Box Models in AI When We Don t Need To ? A Lesson From an Explainable AI Competition . Harvard Data Science Review, 1 0 (2), nov 22 2019

    2019

  2. [2]

    A survey of methods for explaining black box models

    Riccardo Guidotti, Anna Monreale, Salvatore Ruggieri, Franco Turini, Fosca Giannotti, and Dino Pedreschi. A survey of methods for explaining black box models. ACM Computing Surveys (CSUR), 51 0 (5), aug 2018. ISSN 0360-0300. doi:10.1145/3236009

    doi:10.1145/3236009 2018

  3. [3]

    Allen, Luqin Gan, and Lili Zheng

    Genevera I. Allen, Luqin Gan, and Lili Zheng. Interpretable machine learning for discovery: Statistical challenges & opportunities. Technical report, 2024

    work page 2024

  4. [4]

    N. Liu M. Du and X. Hu. Techniques for interpretable machine learning. Communication (ACM), 63(1): 0 68–77, 2019

    work page 2019

  5. [5]

    C. Molnar. Interpretable Machine Learning: A Guide For Making Black Box Models Explainable. 2nd Ed, 2022 a

    work page 2022

  6. [6]

    W. J. Murdoch, C. Singh, R. Abbassi-Asi K. Kumbier, , and B. Yu. Interpretable machine learning: definitions, methods, and applications. Proceedings of National Academy of Sciences, 116(44): 0 22071--22080, 2019

    work page 2019

  7. [7]

    Learning important features through propagating activation differences

    Avanti Shrikumar, Peyton Greenside, and Anshul Kundaje. Learning important features through propagating activation differences. In Doina Precup and Yee Whye Teh, editors, Proceedings of the 34th International Conference on Machine Learning, volume 70 of Proceedings of Machine Learning Research, pages 3145--3153. PMLR, 06--11 Aug 2017 a

    work page 2017

  8. [8]

    Axiomatic attribution for deep networks

    Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks. In Doina Precup and Yee Whye Teh, editors, Proceedings of the 34th International Conference on Machine Learning, volume 70 of Proceedings of Machine Learning Research, pages 3319--3328. PMLR, 06--11 Aug 2017 a . URL https://proceedings.mlr.press/v70/sundararajan17a.html

    work page 2017

Show all 81 references
  1. [9]

    Learning important features through propagating activation differences

    Avanti Shrikumar, Peyton Greenside, and Anshul Kundaje. Learning important features through propagating activation differences. In Proceedings of the 34th International Conference on Machine Learning, 2017 b

    2017

  2. [10]

    Axiomatic attribution for deep networks

    Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks. In Proceedings of the 34th International Conference on Machine Learning, 2017 b

    2017

  3. [11]

    Nonparametric variable importance using an augmented neural network with multi-task learning

    Jean Feng, Brian Williamson, Noah Simon, and Marco Carone. Nonparametric variable importance using an augmented neural network with multi-task learning. In Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Resear...

    2018

  4. [12]

    Dropout feature ranking for deep learning models, 2018

    Chun-Hao Chang, Ladislav Rampasek, and Anna Goldenberg. Dropout feature ranking for deep learning models, 2018

    2018

  5. [13]

    Morgan and H

    N. Morgan and H. Bourlard. Generalization and parameter estimation in feedforward nets: Some experiments. In Proceedings of Neural Information Processing Systems, 1989

    1989

  6. [14]

    Wainwright, and Bin Yu

    Garvesh Raskutti, Martin J. Wainwright, and Bin Yu. Early stopping and non-parametric regression: An optimal data-dependent stopping rule. Journal of Machine Learning Research, 15: 0 335--366, 2014

    2014

  7. [15]

    Boosting with the l2 loss

    Peter Bühlmann and Bin Yu. Boosting with the l2 loss. Journal of the American Statistical Association, 98 0 (462): 0 324--339, 2003. doi:10.1198/016214503000125

    2003 doi

  8. [16]

    Boosting with early stopping: Convergence and consistency

    Tong Zhang and Bin Yu. Boosting with early stopping: Convergence and consistency . The Annals of Statistics, 33 0 (4): 0 1538 -- 1579, 2005. doi:10.1214/009053605000000255

    2005 doi

  9. [17]

    Wainwright1

    Yuting Wei, Fanny Yang, and Martin J. Wainwright1. Early stopping for kernel boosting algorithms: A general analysis with localized complexities. In 31st Conference on Neural Information Processing Systems, Long Beach, CA, USA, 2017

    2017

  10. [18]

    Lazy Estimation of VI for Large NNs

    Yue Gao, Abby Stevens, Garvesh Raskutti, and Rebecca Willett. Lazy Estimation of VI for Large NNs . In Proceedings of the 39th International Conference on Machine Learning, Baltimore, Maryland, USA, 2022

    2022

  11. [19]

    R. S. Anderssen and P. M. Prenter. A formal comparison of methods proposed for the numerical solution of first kind integral equations. The Journal of the Australian Mathematical Society. Series B. Applied Mathematics, 22 0 (4): 0 488–500, 1981. doi:10.1017/S0334270000002824

    1981 doi

  12. [20]

    Theory and methods related to the singular-function expansion and landweber's iteration for integral equations of the first kind

    Otto Neall Strand. Theory and methods related to the singular-function expansion and landweber's iteration for integral equations of the first kind. SIAM Journal on Numerical Analysis, 11 0 (4): 0 798--825, 1974. ISSN 00361429

    1974

  13. [21]

    Process consistency for adaboost

    Wenxin Jiang. Process consistency for adaboost. The Annals of Statistics, 32 0 (1): 0 13--29, 2004. ISSN 00905364

    2004

  14. [22]

    On early stopping in gradient descent learning

    Yuan Yao, Lorenzo Rosasco, and Andrea Caponnetto. On early stopping in gradient descent learning. Constructive Approximation, 26 0 (2), 2007

    2007

  15. [23]

    Boosting algorithms: Regularization, prediction and model fitting

    Peter B \"u hlmann and Torsten Hothorn. Boosting algorithms: Regularization, prediction and model fitting. Statistical Science, 22 0 (4): 0 477 -- 505, 2007. doi:10.1214/07-STS242. URL https://doi.org/10.1214/07-STS242

    2007 doi

  16. [24]

    E. D. Vito, S. Pereverzyev, and L. Rosasco. Adaptive kernel methods using the balancing principle. Foundations of Computational Mathematics, 10: 0 455–479, 2010

    2010

  17. [25]

    Neural tangent kernel: Convergence and generalization in neural networks

    Arthur Jacot, Franck Gabriel, and Clément Hongler. Neural tangent kernel: Convergence and generalization in neural networks. In 32nd Conference on Neural Information Processing Systems, Montréal , Canada, 2018

    2018

  18. [26]

    Du, Wei Hu, Zhiyuan Li, Ruslan Salakhutdinov, and Ruosong Wang

    Sanjeev Arora, Simon S. Du, Wei Hu, Zhiyuan Li, Ruslan Salakhutdinov, and Ruosong Wang. On exact computation with an infinitely wide neural net. In 33rd Conference on Neural Information Processing Systems, Vancouver, Canada, 2019

    2019

  19. [27]

    A convergence theory for deep learning via over-parameterization

    Zeyuan Allen-Zhu, Yuanzhi Li, and Zhao Song. A convergence theory for deep learning via over-parameterization. In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Lea...

    2019

  20. [28]

    Du, Jason D

    Simon S. Du, Jason D. Lee, Haochuan Li, Liwei Wang, and Xiyu Zhai. Gradient descent finds global minima of deep neural networks. In Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, 2019 a

    2019

  21. [29]

    Du, Xiyu Zhai, Barnabas Poczos, and Aarti Singh

    Simon S. Du, Xiyu Zhai, Barnabas Poczos, and Aarti Singh. Gradient descent provably optimizes over-parameterized neural networks. In In Proceedings of the International Conference on Learning Representations (ICLR), 2019 b

    2019

  22. [30]

    Schoenholz, Yasaman Bahri, Roman Novak, Jascha Sohl-Dickstein, and Jeffrey Pennington

    Jaehoon Lee, Lechao Xiao, Samuel S. Schoenholz, Yasaman Bahri, Roman Novak, Jascha Sohl-Dickstein, and Jeffrey Pennington. Wide neural networks of any depth evolve as linear models under gradient descent. In 33rd Conference on Neural Information Processing Systems, Vancouver, ...

    2019

  23. [31]

    Scaling limits of wide neural networks with weight sharing: Gaussian process behavior, gradient independence, and neural tangent kernel derivation, 2020

    Greg Yang. Scaling limits of wide neural networks with weight sharing: Gaussian process behavior, gradient independence, and neural tangent kernel derivation, 2020. URL https://arxiv.org/abs/1902.04760

    2020 arXiv

  24. [32]

    Gradient boosting performs gaussian process inference

    Aleksei Ustimenko, Artem Beliakov, and Liudmila Prokhorenkova. Gradient boosting performs gaussian process inference. In 11th Conference on International Conference on Learning Representations, Kigali, Rwanda, 2023

    2023

  25. [33]

    Relative importance for linear regression in r: The package relaimpo

    Groemping Ulrike. Relative importance for linear regression in r: The package relaimpo. Journal of Statistical Software, 17 0 (1): 0 1–27, 2006. doi:10.18637/jss.v017.i01. URL https://www.jstatsoft.org/index.php/jss/article/view/v017i01

    2006 doi

  26. [34]

    Nathans, Frederick L

    Laura L. Nathans, Frederick L. Oswald, and Kim Nimon. Interpreting multiple linear regression: A guidebook of variable importance. Practical Assessment, Research & Evaluation, 17 0 (9), 2012

    2012

  27. [35]

    Michael D. McKay. Nonparametric variance-based methods of assessing uncertainty importance. Reliability Engineering & System Safety, 57 0 (3): 0 267--279, 1997. doi:https://doi.org/10.1016/S0951-8320(97)00039-2

    1997 doi

  28. [36]

    Williamson, Peter B

    Brian D. Williamson, Peter B. Gilbert, Marco Carone, and Noah Simon. Nonparametric variable importance assessment using machine learning techniques. Biometrics, 77 0 (1): 0 9--22, 2021

    2021

  29. [37]

    Williamson, Peter B

    Brian D. Williamson, Peter B. Gilbert, Noah R. Simon, and Marco Carone. A general framework for inference on algorithm-agnostic variable importance. Journal of the American Statistical Association, 118 0 (543): 0 1645--1658, 2023. doi:10.1080/01621459.2021.2003200

    2023

  30. [38]

    Variable importance in binary regression trees and forests

    Hemant Ishwaran. Variable importance in binary regression trees and forests . Electronic Journal of Statistics, 1: 0 519 -- 537, 2007. doi:10.1214/07-EJS039

    2007 doi

  31. [39]

    Variable importance assessment in regression: Linear regression versus random forest

    Ulrike Grömping. Variable importance assessment in regression: Linear regression versus random forest. The American Statistician, 63 0 (4): 0 308--319, 2009. doi:10.1198/tast.2009.08199

    2009 arXiv

  32. [40]

    Bias in random forest variable importance measures: Illustrations, sources and a solution

    Carolin Strobl, Anne-Laure Boulesteix, Achim Zeileis, and Torsten Hothorn. Bias in random forest variable importance measures: Illustrations, sources and a solution. BMC Bioinformatics, 8, 2007

    2007

  33. [41]

    On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation

    Sebastian Bach, Alexander Binder, Grégoire Montavon, Frederick Klauschen, Klaus-Robert Müller, and Wojciech Samek. On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PLOS ONE, 10 0 (7): 0 1--46, 07 2015. doi:10.1371/journal.pone...

    2015 doi

  34. [42]

    L. S. Shapley. 17. A Value for n-Person Games, pages 307--318. Princeton University Press, Princeton, 1953. ISBN 9781400881970. doi:doi:10.1515/9781400881970-018. URL https://doi.org/10.1515/9781400881970-018

    1953 doi

  35. [43]

    Algorithmic transparency via quantitative input influence: Theory and experiments with learning systems

    Anupam Datta, Shayak Sen, and Yair Zick. Algorithmic transparency via quantitative input influence: Theory and experiments with learning systems. In 2016 IEEE Symposium on Security and Privacy (SP), pages 598--617, 2016. doi:10.1109/SP.2016.42

    2016 doi

  36. [44]

    A unified approach to interpreting model predictions

    Scott Lundberg and Su-In Lee. A unified approach to interpreting model predictions. arXiv preprint arXiv:1705.07874, 2017

    2017 arXiv

  37. [45]

    Understanding global feature contributions with additive importance measures

    Ian Covert, Scott Lundberg, and Su-In Lee. Understanding global feature contributions with additive importance measures. In Advances in Neural Information Processing Systems,, 2020

    2020

  38. [46]

    Williamson and Jean Feng

    Brian D. Williamson and Jean Feng. Efficient nonparametric statistical inference on population feature importance using shapley values. In Proceedings of the 37th International Conference on Machine Learning, Online, 2020

    2020

  39. [47]

    Lundberg, Hugh Chen Gabriel Erion, Alex DeGrave, Jordan M

    Scott M. Lundberg, Hugh Chen Gabriel Erion, Alex DeGrave, Jordan M. Prutkin, Bala Nair, Ronit Katz, Jonathan Himmelfarb, Nisha Bansal, and Su-In Lee. From local explanations to global understanding with explainable ai for trees. Nature Machine Intelligence, 2: 0 56--67, 2020

    2020

  40. [48]

    Covert, Scott M

    Hugh Chen, Ian C. Covert, Scott M. Lundberg, and Su-In Lee. Explaining a series of models by propagating shapley values. Nature Communications, 13 0 (4512), 2022

    2022

  41. [49]

    Marco Ancona, Cengiz Öztireli, and Markus H. Gross. Explaining Deep Neural Networks with a Polynomial Time Algorithm for Shapley Value Approximation . In Proceedings of the 36th International Conference on Machine Learning , pages 272--281. PMLR , 2019

    2019

  42. [50]

    Rui Wang, Xiaoqian Wang, and David I. Inouye. Shapley explanation networks. In International Conference on Learning Representations, 2021

    2021

  43. [51]

    Covert, Scott M

    Hugh Chen, Ian C. Covert, Scott M. Lundberg, and Su-In Lee. Algorithms to estimate shapley value feature attributions. Nature Machine Intelligence, 5: 0 590--601, 2023

    2023

  44. [52]

    Floodgate: inference for model-free variable importance, 2022

    Lu Zhang and Lucas Janson. Floodgate: inference for model-free variable importance, 2022. URL https://arxiv.org/abs/2007.01283

    2022 arXiv

  45. [53]

    All models are wrong, but many are useful: Learning a variable’s importance by studying an entire class of prediction models simultaneously

    Aaron Fisher, Cynthia Rudin, and Francesca Dominici. All models are wrong, but many are useful: Learning a variable’s importance by studying an entire class of prediction models simultaneously. Journal of Machine Learning Research, 20: 0 1--81, 2019

    2019

  46. [54]

    Model class reliance for random forests

    Gavin Smith, Roberto Mansilla, and James Goulding. Model class reliance for random forests. In Advances in Neural Information Processing Systems, volume 33, 2020

    2020

  47. [55]

    Tibshirani, and Larry Wasserman

    Jing Lei, Max G’Sell, Alessandro Rinaldo, Ryan J. Tibshirani, and Larry Wasserman. Distribution-free predictive inference for regression. Journal of the American Statistical Association, 113 0 (523): 0 1094--1111, 2018. doi:10.1080/01621459.2017.1307116

    2018

  48. [56]

    Bootstrapping and sample splitting for high-dimensional, assumption-lean inference

    Alessandro Rinaldo, Larry Wasserman, and Max G’Sell. Bootstrapping and sample splitting for high-dimensional, assumption-lean inference . The Annals of Statistics, 47 0 (6): 0 3438 -- 3469, 2019. doi:10.1214/18-AOS1784. URL https://doi.org/10.1214/18-AOS1784

    2019 doi

  49. [57]

    The hardness of conditional independence testing and the generalised covariance measure

    Rajen D Shah and Jonas Peters. The hardness of conditional independence testing and the generalised covariance measure. The Annals of Statistics, 48 0 (3): 0 1514–1538, 2020

    2020

  50. [58]

    On Azadkia–Chatterjee’s conditional dependence coefficient

    Hongjian Shi, Mathias Drton, and Fang Han. On Azadkia–Chatterjee’s conditional dependence coefficient . Bernoulli, 30 0 (2): 0 851 -- 877, 2024. doi:10.3150/22-BEJ1529. URL https://doi.org/10.3150/22-BEJ1529

    2024 doi

  51. [59]

    Testing conditional independence in supervised learning algorithms

    David S Watson and Marvin N Wright. Testing conditional independence in supervised learning algorithms. Machine Learning, 110 0 (8): 0 2107–2129, 2021

    2021

  52. [60]

    Panning for gold: ‘model-x’ knockoffs for high dimensional controlled variable selection

    Emmanuel Candes, Yingying Fan, Lucas Janson, and Jinchi Lv. Panning for gold: ‘model-x’ knockoffs for high dimensional controlled variable selection. Journal of the Royal Statistical Society Series B: Statistical Methodology, 80 0 (3): 0 551--577, 01 2018. ISSN 1369-7412. doi:...

    2018 doi

  53. [61]

    Functions of positive and negative type and their connection with the theory of integral equations

    James Mercer. Functions of positive and negative type and their connection with the theory of integral equations. Philosophical Transactions of the Royal Society A, 209: 0 415--446, 1909

    1909

  54. [62]

    Random matrix approximation of spectra of integral operators

    Vladimir Koltchinskii and Evarist Gin \'e . Random matrix approximation of spectra of integral operators . Bernoulli, 6 0 (1): 0 113 -- 167, 2000

    2000

  55. [63]

    Owen and Cl\' e mentine Prieur

    Art B. Owen and Cl\' e mentine Prieur. On shapley value for measuring importance of dependent inputs. SIAM/ASA Journal on Uncertainty Quantification, 5 0 (1): 0 986--1002, 2017. doi:10.1137/16M1097717

    2017 doi

  56. [64]

    Interpretable Machine Learning

    Christoph Molnar. Interpretable Machine Learning. 2 edition, 2022 b . URL https://christophm.github.io/interpretable-ml-book

    2022

  57. [65]

    Mendelson

    S. Mendelson. Geometric parameters of kernel machines. In Proceedings of COLT , pages 29--43, 2002

    2002

  58. [66]

    Progressive growing of gans for improved quality, stability, and variation

    Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. Progressive growing of gans for improved quality, stability, and variation. In 6th Conference on International Conference on Learning Representations, Vancouver, Canada, 2018

    2018

  59. [67]

    Park, Jascha Sohl-Dickstein, Quoc V

    Daniel S. Park, Jascha Sohl-Dickstein, Quoc V. Le, and Samuel L. Smith. The effect of network width on stochastic gradient descent and generalization: an empirical study. In International Conference on Machine Learning, CA, USA, 2019

    2019

  60. [68]

    Gradient descent finds global minima of deep neural networks

    Simon S Du, Jason D Lee, Haochuan Li, Liwei Wang, and Xiyu Zhai. Gradient descent finds global minima of deep neural networks. In International Conference on Machine Learning, CA, USA, 2019 c

    2019

  61. [69]

    On the inductive bias of neural tangent kernels

    Bietti, Alberto, Mairal, and Julien. On the inductive bias of neural tangent kernels. In Advances in Neural Information Processing Systems, volume 32, 2019

    2019

  62. [70]

    Breaking the curse of dimensionality with convex neural networks

    Francis Bach. Breaking the curse of dimensionality with convex neural networks. Journal of Machine Learning Research, 18 0 (19): 0 1--53, 2017. URL http://jmlr.org/papers/v18/14-546.html

    2017

  63. [71]

    Deep neural tangent kernel and laplace kernel have the same rkhs

    Lin Chen and Sheng Xu. Deep neural tangent kernel and laplace kernel have the same rkhs. In 9th Conference on International Conference on Learning Representations, Virtual, 2021

    2021

  64. [72]

    On the similarity between the laplace and neural tangent kernels

    Amnon Geifman, Abhay Yadav, Meirav Galun Yoni Kasten, David Jacobs, and Ronen Basri. On the similarity between the laplace and neural tangent kernels. In Advances in Neural Information Processing Systems, 2020

    2020

  65. [73]

    Deep equals shallow for relu networks in kernel regimes

    Alberto Bietti and Francis Bach. Deep equals shallow for relu networks in kernel regimes. In 9th Conference on International Conference on Learning Representations, Virtual, 2021

    2021

  66. [74]

    On the approximate realization of continuous mappings by neural networks

    Funahashi and Ken-Ichi. On the approximate realization of continuous mappings by neural networks. Neural Networks, 2 0 (3): 0 183 -- 192, 1989. doi:10.1016/0893-6080(89)90003-8

    1989 doi

  67. [75]

    Approximation capabilities of multilayer feedforward networks

    Kurt Hornik. Approximation capabilities of multilayer feedforward networks. Neural Networks, 4 0 (2): 0 251 -- 257, 1991. doi:10.1016/0893-6080(91)90009-T

    1991 doi

  68. [76]

    Friedman

    Jerome H. Friedman. Greedy function approximation: A gradient boosting machine. The Annals of Statistics, 29 0 (5): 0 1189 -- 1232, 2001. doi:10.1214/aos/1013203451

    2001

  69. [77]

    Gas Turbine CO and NOx Emission Data Set

    Gas Turbine Dataset . Gas Turbine CO and NOx Emission Data Set . UCI Machine Learning Repository, 2019. DOI : https://doi.org/10.24432/C5WC95

    2019 doi

  70. [78]

    Data analysis and symbolic regression models for predicting co and nox emissions from gas turbines

    Olga Kochueva and Kirill Nikolskii. Data analysis and symbolic regression models for predicting co and nox emissions from gas turbines. Computation, 9 0 (12), 2021. ISSN 2079-3197. doi:10.3390/computation9120139. URL https://www.mdpi.com/2079-3197/9/12/139

    2021 doi

  71. [79]

    Abdul Karim Miah, and Md Azazul Haque

    Kazi Ekramul Hoque, Tahiya Hossain, ABM Mominul Haque, Md. Abdul Karim Miah, and Md Azazul Haque. NOx Emission Predictions in Gas Turbines Through Integrated Data-Driven Machine Learning Approaches . Journal of Energy Resources Technology, 146 0 (7): 0 071201, 04 2024. ISSN 01...

    2024 doi

  72. [80]

    Pytorch lightning, 2019

    William Falcon and the PyTorch Lightning team. Pytorch lightning, 2019. URL https://www.pytorchlightning.ai/

    2019

  73. [81]

    Catboost: gradient boosting with categorical features support, 2018

    Liudmila Prokhorenkova, Gleb Gusev, Aleksandr Vorobev, Anna Veronika Dorogush, and Andrey Gulin. Catboost: gradient boosting with categorical features support, 2018. URL https://catboost.ai/

    2018

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.