REVIEW 3 major objections 4 minor 65 references
Lazy training reduces deep-network feature-importance testing to a z-test.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-01 03:57 UTC pith:6LY73U2J
load-bearing objection A plausible and well-motivated extension of lazy-training variable importance to binary classification, but the central theorem is not fully established because the Donsker condition (R3) is verified only for fixed network width W, which grows with n. the 3 major comments →
Variable Importance Identification Through Lazy Training for Binary Classification
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The central claim is Theorem 25: for any candidate feature set S, under the null hypothesis and with W = o(sqrt(n)/log n) network weights, the estimated variable-importance difference psi-hat minus psi-0 equals a sample average of influence-function contrasts plus op(n^{-1/2}). If this holds, the lazy variable-importance statistic is asymptotically normal with a variance that can be estimated from the same fitted network, so a valid z-test for feature importance can be run without refitting deep networks for each feature set. The paper's key claimed improvement is the op(n^{-1/2}) remainder rather than the Op(n^{-1/2}) rate achieved in prior work, attributed to the use of local Rademacher co
What carries the argument
The neural tangent kernel (NTK) matrix K_{-S}, formed from the gradients of the trained network evaluated at inputs with the candidate feature subset replaced by its mean. The lazy-regime linearization h_theta approximately equal to h_theta-f plus the gradient inner product with (theta - theta-f) turns each feature-removal fit into a ridge-regularized logistic regression in the RKHS of the NTK, solved by Newton-Raphson. The convergence argument rides on the local Rademacher complexity of the RKHS ball H_B, whose fixed point is controlled by Assumption (A1): the eigenvalues mu_j of K_{-S}/n decay as mu_j <= c j^{-alpha} with alpha > 1. That eigenvalue decay is what converts the estimation err
Load-bearing premise
The load-bearing premise, which the authors state they cannot currently prove for deep ReLU networks, is that the neural tangent kernel matrix at the fitted parameters has eigenvalues decaying as mu_j <= c j^{-alpha} with alpha > 1; without this spectral decay, the local-Rademacher rate and the op(n^{-1/2}) remainder collapse.
What would settle it
Compute the sorted eigenvalues of the NTK matrix K_{-S} for a deep ReLU network at converged parameters on a moderate-size nonlinear classification problem and fit the log-log slope of the tail. A slope with alpha <= 1, or a heavy tail that persists as network width grows, would invalidate the local-Rademacher rate and the claimed op(n^{-1/2}) result.
If this is right
- Feature-importance inference for binary deep classifiers requires only one full-network training pass; every candidate feature subset is handled by a cheap linearized logistic regression.
- Under the null hypothesis, the LazyVI test statistic is asymptotically standard normal, enabling p-values and confidence intervals without retraining or permutation tests.
- The required regime is underparameterized: the number of network weights must grow slower than sqrt(n)/log n.
- The likelihood-based setup extends to other exponential-family outcomes whenever the negative log-likelihood satisfies the convexity and Lipschitz conditions used in the proofs.
Where Pith is reading between the lines
- If Assumption (A1) transfers from initialization to fitted parameters, the same framework may extend to overparameterized networks; the paper identifies this as an open bottleneck rather than a proven result.
- A natural stress test is to run LazyVI on nonlinear simulations with eigenvalue decay alpha close to 1, where the local-Rademacher rate becomes borderline and the z-test approximation should visibly degrade.
- The n-by-n NTK matrix is the computational bottleneck; Nystrom or block approximations to K_{-S} could plausibly scale the method to biobank-sized data, a direction the paper mentions but does not implement.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes LazyVI, a variable-importance testing procedure for binary classification with deep ReLU networks under the lazy-training / neural tangent kernel regime. The predictiveness measure is the expected logistic likelihood, and the estimator uses one fitted network plus a ridge-penalized linearized (NTK) fit for each feature subset, avoiding retraining. The main theoretical claim (Theorem 25) is that, under an eigenvalue decay condition on the NTK at the fitted parameters (A1), a regularization-rate condition (A2), and W = o(sqrt(n)/log n), the variable-importance estimator is asymptotically linear with an op(n^{-1/2}) remainder, yielding a valid z-test. The paper also reports simulations, MNIST experiments, an ADNI gene-association application, and a software implementation. The appendices contain detailed proofs of local Rademacher bounds, NTK-RKHS estimates, and the verification of the variable-importance framework conditions.
Significance. If Theorem 25 were established, it would be a practically useful result: it would provide first-order inference for feature importance in binary ReLU classifiers without retraining for each feature subset, improving on the Op(n^{-1/2}) rate in prior lazy-training work. The paper is also transparently written: it ships code, gives explicit conditions, includes large-scale simulation support, and lays out the local-Rademacher machinery in detail. These are genuine strengths. However, the central theoretical claim is not supported as written. The verification of Williamson et al.'s condition (R3) is incomplete because the relevant function class grows with W_n, and the main rate improvement rests on an unproven spectral assumption on the NTK at the fitted parameters. These are load-bearing gaps, not presentation issues.
major comments (3)
- [Appendix E.3, Condition (R3)] The proof defines D = {Z ↦ ℓ(f(X),Y) − ℓ(f0(X),Y) − E_P0[ℓ(f(X),Y) − ℓ(f0(X),Y)] : f ∈ F} and bounds log N(ε,F,||·||_sup) ≤ W log(1 + C/ε) via Theorem 36. For fixed W this would give a finite entropy integral and a Donsker class. But Theorem 25 assumes W = W_n = o(√n/log n), so W_n grows with n. Condition (R3) of Williamson et al. requires a single fixed P0-Donsker class D0 such that P0(g_n ∈ D0) → 1. The proof never supplies such a fixed class: D_n changes with n, and its entropy integral is O(√W_n), which diverges as n → ∞. Thus the empirical process indexed by D_n need not converge to a tight Gaussian limit, and the op(n^{-1/2}) expansion in Theorem 25 does not follow. This gap is independent of Assumption (A1); even granting A1, the Donsker verification is incomplete.
- [§3.2, Assumption (A1)] The faster rate and Theorem 25 depend critically on Assumption (A1): the eigenvalues of the NTK matrix K_{-S} at the fitted parameters decay as μ_j ≤ c j^{-α} with α > 1. The paper states explicitly in §3.2: 'we are currently unable to provide a theoretical proof that Assumption (A1) holds for the class of deep ReLU neural networks' and extrapolates from NTK results at initialization. Since Lemma 23, Theorem 24, and the resulting o(n^{-1/4}) rates used for condition (R1) all rely on this spectral assumption, the paper's central claim is conditional on an unproved structural conjecture. The empirical check in §4.2 is a single simulation setup and does not establish the assumption for general ReLU networks. The abstract's statement that the method relies on 'only a minimal set of assumptions' overstates the situation when a key assumption is conjectural.
- [§E.3, Condition (R1)] The proof verifies condition (R1) using L2(P) convergence rates from Theorem 15 and Theorem 24. However, Theorem 7's condition (R1) is stated in terms of the function-class norm ||·||_F of F, and the paper never specifies which norm ||·||_F represents. If the authors intend ||·||_F to be the L2(P) norm, this should be stated explicitly and used consistently in conditions (D1) and (D2). If a different norm is intended, the L2(P) bounds do not establish the required function-class-norm convergence. This ambiguity is load-bearing because (R1) is one of the three random conditions needed for the asymptotic linear expansion.
minor comments (4)
- [§4.1.3] The runtime comparison sentence contains a typo: 'LazyVInonlinear required approximately 21,534 seconds in the linear setting and 33,766 seconds in the linear setting.' The second 'linear setting' should presumably read 'nonlinear setting.'
- [Lemma 34] The class of functions defined in equation (55) is denoted M, but the lemma statement uses M before its definition; the notation is confusing. Also, the covering-number bound should make explicit the dependence on the Lipschitz constant of the activation, which is stated but not reflected in the constants.
- [§4.3.3] Two consecutive paragraphs give conflicting summaries: the first says 'both halves were significant with p<0.001' and identifies region 10 as most important, while the second says the top half had p≈0.388 and the Bottom-Left quadrant was not significant. The text should be reconciled.
- [Algorithm 1] The line 'Ensure::' contains a double colon typo.
Circularity Check
No circular reduction found: Theorem 25 is a conditional application of Williamson et al.'s external framework, and the paper's own caveats identify unproved assumptions, not circularity.
full rationale
The derivation chain is not circular. Theorem 25 is obtained by verifying conditions (D1), (D2), (R1), (R2), (R3) from Williamson et al. (2023); the central conclusion is an asymptotic linear expansion of the same form as that framework's Theorem 1, but it is derived from stated assumptions rather than being assumed. Assumption (A1) on NTK eigenvalue decay is an explicit input, and the paper candidly states it cannot prove it for deep ReLU networks at fitted parameters: 'Although we are currently unable to provide a theoretical proof that Assumption (A1) holds...' Assumption (A2) fixes the regularization order; these are assumptions, not fitted constants renamed as predictions. The local Rademacher rates in Lemmas 23-24 conditionally use (A1) to produce op(n^{-1/2}); that is a derivation from assumptions, not an equivalence by construction. The self-citations (Shen et al.; Shen and Wang; Shen and Espinoza) appear in related-work or analogous-condition contexts and are not load-bearing. The manuscript itself flags the real technical bottleneck in Section 5: the entropy bound 'yields a growth rate of order O(log 1/epsilon), but at the cost of dependence on the total number of network parameters.' This corresponds to the incomplete Donsker verification under W_n -> infinity in condition (R3), but an incomplete proof or unproved assumption is a correctness risk, not circularity. No step in the paper reduces its claimed prediction to its own inputs by definition or by self-citation.
Axiom & Free-Parameter Ledger
free parameters (2)
- Ridge penalty lambda =
selected by 3-fold CV from 10 values in [1e-4, 1e-2.25] in simulations, MNIST, and ADNI
- Network architecture and training hyperparameters =
two hidden layers of 50 ReLU neurons, L-BFGS learning rate 5e-4, early stopping patience 10
axioms (6)
- domain assumption (X,Y) are iid from P0, X is bounded by kappa, Y is Bernoulli, and logit(P(Y=1|X)) belongs to the network class F.
- domain assumption The deep ReLU network class has fixed weight bounds kappa_i, b_i with b_i >= kappa_i.
- ad hoc to paper Assumption (A1): eigenvalues of (1/n)K_{-S} decay as mu_j <= c j^{-alpha} with alpha > 1, where K_{-S} is the neural tangent kernel at the fitted parameters.
- ad hoc to paper Assumption (A2): lambda = (1 or ||K_{-S}||_op/4) * xi_n with xi_n = omega(n^{-(alpha-1)/(2(alpha+1))} or n^{-1/4}).
- domain assumption W = o(sqrt(n)/log n) and kappa_i, b_i are fixed constants.
- standard math Background empirical-process tools: Bartlett et al. local Rademacher bounds, Ossiander's entropy-integral Donsker criterion, Hsu et al. concentration for quadratic forms, Ledoux-Talagrand contraction.
read the original abstract
Deep neural networks have been widely used in many applications (e.g., computer vision and natural language processing); however, understanding their explainability remains a challenging task. Recently, substantial research has been devoted to improving the explainability of deep neural networks, with most of this work focusing on the regression framework. In this paper, we instead focus on the binary classification framework and adopt a variable-importance framework combined with the idea of lazy training to propose an efficient algorithm for identifying important features. From a theoretical perspective, our method relies on only a minimal set of assumptions and achieves well-controlled error rates. The validity of the proposed method and algorithm is examined through extensive simulation studies and real-data applications.
Figures
Reference graph
Works this paper leans on
-
[1]
Electronic Communications in Probability , volume=
A tail inequality for quadratic forms of subgaussian random vectors , author=. Electronic Communications in Probability , volume=
-
[2]
The collected works of Wassily Hoeffding , pages=
Probability inequalities for sums of bounded random variables , author=. The collected works of Wassily Hoeffding , pages=. 1994 , publisher=
1994
-
[3]
Monatshefte f
A trace inequality of John von Neumann , author=. Monatshefte f. 1975 , publisher=
1975
-
[4]
2024 , publisher=
Learning theory from first principles , author=. 2024 , publisher=
2024
-
[5]
International Conference on Machine Learning , pages=
Lazy estimation of variable importance for large neural networks , author=. International Conference on Machine Learning , pages=. 2022 , organization=
2022
-
[6]
Bartlett and Olivier Bousquet and Shahar Mendelson , title =
Peter L. Bartlett and Olivier Bousquet and Shahar Mendelson , title =. The Annals of Statistics , number =
-
[7]
2019 , publisher=
High-dimensional statistics: A non-asymptotic viewpoint , author=. 2019 , publisher=
2019
-
[8]
Transactions of the American mathematical society , volume=
Theory of reproducing kernels , author=. Transactions of the American mathematical society , volume=
-
[9]
2018 , publisher=
Foundations of machine learning , author=. 2018 , publisher=
2018
-
[10]
2013 , publisher=
Probability in Banach Spaces: isoperimetry and processes , author=. 2013 , publisher=
2013
-
[11]
2006 , publisher =
Numerical Optimization , author =. 2006 , publisher =
2006
-
[12]
Journal of machine learning research , volume=
Rademacher and gaussian complexities: Risk bounds and structural results , author=. Journal of machine learning research , volume=
-
[13]
Journal of the American Statistical Association , volume=
A general framework for inference on algorithm-agnostic variable importance , author=. Journal of the American Statistical Association , volume=. 2023 , publisher=
2023
-
[14]
Advances in neural information processing systems , volume=
Spectrally-normalized margin bounds for neural networks , author=. Advances in neural information processing systems , volume=
-
[15]
Proceedings of the 27th international conference on machine learning (ICML-10) , pages=
Rectified linear units improve restricted boltzmann machines , author=. Proceedings of the 27th international conference on machine learning (ICML-10) , pages=
-
[16]
Econometrica , volume=
Deep neural networks for estimation and inference , author=. Econometrica , volume=. 2021 , publisher=
2021
-
[17]
IEEE transactions on Information Theory , volume=
Improving the sample complexity using global data , author=. IEEE transactions on Information Theory , volume=. 2002 , publisher=
2002
-
[18]
Journal of Statistical Theory and Practice , volume=
Consistency and Rate of Convergence for Deep ReLU Neural Networks , author=. Journal of Statistical Theory and Practice , volume=. 2025 , publisher=
2025
-
[19]
Lecture notes for machine learning theory (CS229M/STATS214) , author=
-
[20]
High-Dimensional Probability: An Introduction with Applications in Data Science , publisher=
Vershynin, Roman , year=. High-Dimensional Probability: An Introduction with Applications in Data Science , publisher=
-
[21]
Journal of Statistical Mechanics: Theory and Experiment , volume=
Six lectures on linearized neural networks , author=. Journal of Statistical Mechanics: Theory and Experiment , volume=. 2024 , publisher=
2024
-
[22]
IEEE Journal on Selected Areas in Information Theory , volume=
Toward moderate overparameterization: Global convergence guarantees for training shallow neural networks , author=. IEEE Journal on Selected Areas in Information Theory , volume=. 2020 , publisher=
2020
-
[23]
arXiv preprint arXiv:1811.08888 , year=
Stochastic gradient descent optimizes over-parameterized deep relu networks , author=. arXiv preprint arXiv:1811.08888 , year=
-
[24]
Journal of nonparametric statistics , volume=
Asymptotic properties of neural network sieve estimators , author=. Journal of nonparametric statistics , volume=. 2023 , publisher=
2023
-
[25]
Journal of Machine Learning Research , volume=
Significance tests for neural networks , author=. Journal of Machine Learning Research , volume=
-
[26]
The Annals of Statistics , volume =
Schmidt-Hieber, Johannes , title =. The Annals of Statistics , volume =
-
[27]
Weak convergence and empirical processes: with applications to statistics , year=
-
[28]
The Annals of Probability , pages=
A central limit theorem under metric entropy with L 2 bracketing , author=. The Annals of Probability , pages=. 1987 , publisher=
1987
-
[29]
2025 , eprint=
Towards Unified Attribution in Explainable AI, Data-Centric AI, and Mechanistic Interpretability , author=. 2025 , eprint=
2025
-
[30]
arXiv preprint arXiv:1806.07421 , year=
Rise: Randomized input sampling for explanation of black-box models , author=. arXiv preprint arXiv:1806.07421 , year=
-
[31]
Advances in neural information processing systems , volume=
A unified approach to interpreting model predictions , author=. Advances in neural information processing systems , volume=
-
[32]
arXiv preprint arXiv:1706.03825 , year=
Smoothgrad: removing noise by adding noise , author=. arXiv preprint arXiv:1706.03825 , year=
-
[33]
arXiv preprint arXiv:1312.6034 , year=
Deep inside convolutional networks: Visualising image classification models and saliency maps , author=. arXiv preprint arXiv:1312.6034 , year=
-
[34]
Statistics & probability letters , volume=
A goodness-of-fit test based on neural network sieve estimators , author=. Statistics & probability letters , volume=. 2021 , publisher=
2021
-
[35]
Frontiers in Systems Biology , volume=
An exploration of testing genetic associations using goodness-of-fit statistics based on deep ReLU neural networks , author=. Frontiers in Systems Biology , volume=. 2024 , publisher=
2024
-
[36]
Advances in neural information processing systems , volume=
On lazy training in differentiable programming , author=. Advances in neural information processing systems , volume=
-
[37]
Proceedings of the AAAI Conference on Artificial Intelligence , volume=
Permutation-based hypothesis testing for neural networks , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=
-
[38]
arXiv preprint arXiv:2212.08255 , year=
A sieve quasi-likelihood ratio test for neural networks with applications to genetic association studies , author=. arXiv preprint arXiv:2212.08255 , year=
-
[39]
IEEE transactions on neural networks and learning systems , volume=
Significance tests of feature relevance for a black-box learner , author=. IEEE transactions on neural networks and learning systems , volume=. 2022 , publisher=
2022
-
[40]
1999 , publisher=
Generalized linear models , author=. 1999 , publisher=
1999
-
[41]
Advances in Neural Information Processing Systems , volume=
On the inductive bias of neural tangent kernels , author=. Advances in Neural Information Processing Systems , volume=
-
[42]
arXiv preprint arXiv:2009.14397 , year=
Deep equals shallow for ReLU networks in kernel regimes , author=. arXiv preprint arXiv:2009.14397 , year=
Pith/arXiv arXiv 2009
-
[43]
Journal of Machine Learning Research , volume=
On the eigenvalue decay rates of a class of neural-network related kernel functions defined on general domains , author=. Journal of Machine Learning Research , volume=
-
[44]
arXiv preprint arXiv:1810.02054 , year=
Gradient descent provably optimizes over-parameterized neural networks , author=. arXiv preprint arXiv:1810.02054 , year=
-
[45]
Proceedings of the IEEE , volume=
Gradient-based learning applied to document recognition , author=. Proceedings of the IEEE , volume=. 2002 , publisher=
2002
-
[46]
Neural networks , volume=
Multilayer feedforward networks are universal approximators , author=. Neural networks , volume=. 1989 , publisher=
1989
-
[47]
Mathematics of control, signals and systems , volume=
Approximation by superpositions of a sigmoidal function , author=. Mathematics of control, signals and systems , volume=. 1989 , publisher=
1989
-
[48]
Neural networks , volume=
Error bounds for approximations with deep ReLU networks , author=. Neural networks , volume=. 2017 , publisher=
2017
-
[49]
Advances in neural information processing systems , volume=
The phase diagram of approximation rates for deep neural networks , author=. Advances in neural information processing systems , volume=
-
[50]
Advances in neural information processing systems , volume=
Attention is all you need , author=. Advances in neural information processing systems , volume=
-
[51]
Connectionism in perspective , pages=
Generalization and network design strategies , author=. Connectionism in perspective , pages=. 1989 , publisher=
1989
-
[52]
Neural computation , volume=
Long short-term memory , author=. Neural computation , volume=. 1997 , publisher=
1997
-
[53]
Remarques sur un r
Pisier, Gilles , journal=. Remarques sur un r
-
[54]
Journal of Machine Learning Research , volume=
Statistical analysis of some multi-category large margin classification methods , author=. Journal of Machine Learning Research , volume=
-
[55]
Conference on learning theory , pages=
Size-independent sample complexity of neural networks , author=. Conference on learning theory , pages=. 2018 , organization=
2018
-
[56]
Nature , volume=
The UK Biobank resource with deep phenotyping and genomic data , author=. Nature , volume=. 2018 , publisher=
2018
-
[57]
Neuron , volume=
Alzheimer’s disease genetics: from the bench to the clinic , author=. Neuron , volume=. 2014 , publisher=
2014
-
[58]
Nature neuroscience , volume=
The multiplex model of the genetics of Alzheimer’s disease , author=. Nature neuroscience , volume=. 2020 , publisher=
2020
-
[59]
Biochimica et Biophysica Acta (BBA)-Molecular Basis of Disease , volume=
Linking cell-cycle dysfunction in Alzheimer's disease to a failure of synaptic plasticity , author=. Biochimica et Biophysica Acta (BBA)-Molecular Basis of Disease , volume=. 2007 , publisher=
2007
-
[60]
Alzheimer's & Dementia , volume=
Genome-wide transcriptome analysis of A deposition on PET in a Korean cohort , author=. Alzheimer's & Dementia , volume=. 2024 , publisher=
2024
-
[61]
Proceedings of the National Academy of Sciences , volume=
PIP30/FAM192A is a novel regulator of the nuclear proteasome activator PA28 , author=. Proceedings of the National Academy of Sciences , volume=. 2018 , publisher=
2018
-
[62]
Aging Cell , volume=
Tau inhibits PKA by nuclear proteasome-dependent PKAR2 elevation with suppressed CREB/GluA1 phosphorylation , author=. Aging Cell , volume=. 2020 , publisher=
2020
-
[63]
Advances in neural information processing systems , volume=
Neural tangent kernel: Convergence and generalization in neural networks , author=. Advances in neural information processing systems , volume=
-
[64]
Neurobiology of disease , volume=
MAP kinase signaling cascade dysfunction specific to Alzheimer's disease in fibroblasts , author=. Neurobiology of disease , volume=. 2002 , publisher=
2002
-
[65]
Journal of Alzheimer’s Disease , volume=
Application of weighted gene co-expression network analysis to explore the key genes in Alzheimer’s disease , author=. Journal of Alzheimer’s Disease , volume=. 2018 , publisher=
2018
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.