Pith. sign in

REVIEW 2 major objections 6 minor 129 references

In the high-dimensional regime n comparable to d, leave-one-out influences of training samples on convex M-estimators converge to an explicit limiting distribution given by a nonlinear pushforward of a four-dimensional Gaussian.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-13 04:23 UTC pith:RAUVKMDQ

load-bearing objection Solid, first sharp asymptotics for leave-one-out influences in the n~d regime; the math holds under standard assumptions and the active-learning link is a clean payoff. the 2 major comments →

arxiv 2607.09250 v1 pith:RAUVKMDQ submitted 2026-07-10 stat.ML cs.LG

Influence Diagnostics in High-dimensional M-estimation: Precise Asymptotics

Hugo Cui This is my paper
classification stat.ML cs.LG MSC 62J0762F1260B20
keywords high-dimensional M-estimationleave-one-out influenceDFBETAresolvent equationsactive learningGaussian designproportional asymptotics
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Classical leave-one-out influence diagnostics become random and interdependent when the number of samples n is comparable to the dimension d. For convex ridge-regularized M-estimation under isotropic Gaussian design, the paper proves that the distribution of these influences (on test error and on parameter change) converges to a sharply characterized limiting measure. That measure is the pushforward of a low-dimensional Gaussian whose covariance is built from a handful of summary statistics of the estimator and its Hessian; the statistics themselves solve self-consistent resolvent equations. The same characterization shows that points with large influence tend to lie near the decision boundary, supplying a rigorous footing for a common active-learning heuristic. The results therefore give both an exact asymptotic description of sample importance and a practical geometric cue for data selection in high dimensions.

Core claim

In the proportional high-dimensional limit n, d → ∞ with fixed ratio α = n/d, the marginal law of the leave-one-out test-error influence of a random training point converges weakly to the pushforward φ_IF ♯ N(0_4, Q), where φ_IF is an explicit nonlinear map built from the proximal residual and the partial derivatives of the test-error functional, and the 4 × 4 covariance Q is assembled from the resolvent moments Q^(k) and V^(k) of the leave-one-out Hessian.

What carries the argument

The four-dimensional Gaussian N(0_4, Q) whose covariance blocks are the Cauchy integrals of the resolvents Ω(z) and V(z); these resolvents satisfy closed self-consistent equations that involve only the proximal residual of the loss and the design ratio α, and the influence map φ_IF simply evaluates a first-order expansion of the test error along the leave-one-out correction.

Load-bearing premise

The loss must be strongly convex with derivatives up to order four growing at most polylogarithmically, and the covariates must be exactly isotropic Gaussian (or elliptical in the conjectural extension).

What would settle it

Generate synthetic Gaussian data for logistic or ridge regression at moderate n = d = 1000–2000, compute the empirical histogram of leave-one-out test-error influences, and check whether it matches the predicted density φ_IF ♯ N(0_4, Q) within sampling error; a systematic mismatch would falsify the claimed weak convergence.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • The average influence of a training point is a non-monotonic function of the sample complexity α = n/d and is maximized at intermediate α.
  • Conditional on margin, the mean influence is largest for points lying near the current decision boundary and inside the disagreement region with the ground-truth separator.
  • DFBETA influences concentrate even more strongly: their empirical distribution converges in probability to a simple one-dimensional pushforward of a two-dimensional Gaussian.
  • In the population limit α → ∞ the high-dimensional law recovers the classical influence-function χ² law, linking the two asymptotic regimes.
  • The same geometric picture (high influence near the boundary) is observed qualitatively on real image data after a neural feature map, suggesting the heuristic remains useful beyond pure Gaussians.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Because the limiting law is fully determined by a few scalar summary statistics, one can in principle estimate those statistics once from the full-data fit and then rank every training point’s influence without repeated leave-one-out retraining.
  • The same resolvent machinery should extend, with only technical changes, to subset-influence diagnostics that delete blocks of size o(n), provided the blocks remain sparse relative to the Hessian spectrum.
  • The observed concentration of influence near the margin supplies a quantitative justification for margin-based active learning even when labels are expensive and the model is still far from the population risk minimizer.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 6 minor

Summary. The paper studies leave-one-out influence diagnostics for ridge-regularized convex M-estimation under isotropic Gaussian design in the proportional high-dimensional regime n ≍ d. The central results are: (i) weak convergence of the marginal law of the leave-one-out test-error influence IF_i to the pushforward φ_IF ♯ N(0_4, Q), where φ_IF is an explicit nonlinear map built from resolvent summary statistics Q^(k), V^(k) characterized by self-consistent equations (Theorem 2.1); (ii) concentration in probability of the empirical distribution of DFBETAs to a related two-dimensional pushforward (Proposition 2.2). The proofs combine leave-one-out expansions, Efron–Stein and Lipschitz concentration of resolvents, and Gaussian residual limits. Section 3 then conditions the limiting law on geometric descriptors (margin, ground-truth projection) and argues that influential points tend to lie near the decision boundary, linking the asymptotics to a standard active-learning heuristic, with supporting synthetic and real-data experiments.

Significance. Classical influence-function theory is low-dimensional; the high-dimensional regime n ≍ d is the relevant one for many modern estimators, yet the joint dependence of each sample’s influence on the full training set has remained largely uncharacterized. The paper supplies a sharp, closed-form limiting description for two standard diagnostics under the standard Gaussian-design M-estimation model, with fully explicit summary-statistic equations and careful treatment of the 1/n versus 1/(n−1) normalization. The technical development (leave-one-out approximation, pointwise and integral concentration of resolvents, deterministic equivalents) is self-contained and builds cleanly on the El Karoui / Donoho–Montanari / Thrampoulidis line. The active-learning discussion is appropriately cautious (“evidence,” “making contact”) and is backed by both the conditional limiting law and real-data visualizations. Explicit characterizations, numerical histogram matches, and a clearly labeled conjectural extension to elliptical/noisy designs are genuine strengths.

major comments (2)
  1. Abstract and §2.3 state that “the distribution of the influences across the training set converges to a limiting measure.” Theorem 2.1 only establishes weak convergence of the marginal law of IF_{i_n} for a sequence of indices (plus an O(polylog n / n^{1/4}) error for Lipschitz test functions). Empirical-measure concentration is proved only for DFBETA (Proposition 2.2) and is explicitly left as a conjecture for IF. The abstract/introduction wording should be aligned with the theorem statements, or the stronger empirical claim for IF should be flagged as conjectural in the same places where the main result is announced.
  2. Assumption A.1 requires strong convexity and derivatives up to order four bounded by O(polylog n). The square-loss case (Remark 2.3, Fig. 2) is recovered only by a mollification argument that is not written out. Because the square-loss formulae are used for intuition and for the population-limit comparison with classical influence functions (Remark 2.4), a short, self-contained justification that the limiting distribution is continuous under the mollification (or a direct argument for quadratic loss) would make that part of the paper fully rigorous rather than heuristic.
minor comments (6)
  1. The encoding of arrows and asymptotics in the provided text (e.g., “n/∫hortrightarrow∞”) is garbled; ensure the arXiv/source version uses standard LaTeX arrows throughout.
  2. Notation for leave-one-out objects is dense (ˆw_{(i)}, ˆw_{\i}, ˜w_i, H_{(i)}, H_{\i}, ˜H_i). A short “notation table” early in §2 or Appendix A would help readers navigate Appendices B–D.
  3. Fig. 1 (right) and Conjecture F.1: the real-data histogram is compared to the conjectural elliptical/noisy formula. State more clearly in the caption that the red curve is not covered by Theorem 2.1.
  4. Section 3.2: the conditional densities ν_{IF|ω} are obtained by conditioning the four-dimensional Gaussian and pushing forward; a one-line formula for the conditional mean μ_{IF|ω} (or a pointer to the corresponding Gaussian conditional) would make the insets of Figs. 3–4 easier to reproduce.
  5. Appendix G: the MNIST / chest X-ray protocol synthesizes points in span(β̂, ŵ) with oracle labels sign(⟨β, z⟩). A brief remark that this is an idealized probe of the (β, ŵ) plane, not a practical active-learning algorithm, would avoid over-interpretation.
  6. Typos / polish: “succintly” → “succinctly” (p. 10); “on the other hand is the understanding” (p. 3); occasional missing articles. None affect correctness.

Circularity Check

0 steps flagged

No significant circularity: the limiting influence measure is derived from leave-one-out expansions, resolvent identities and Gaussian residual limits, without fitted parameters or load-bearing self-citations.

full rationale

The central claim (Theorem 2.1) follows a self-contained chain: the leave-one-out identity (8) from El Karoui, concentration of resolvents/summary statistics Q^(k), V^(k) via Efron-Stein and Lipschitz arguments (Appendices B–C), asymptotic Gaussianity of residuals (Lemma C.2), and the explicit pushforward map φ_IF built from those statistics (Lemma D.5 + Theorem D.6). The self-consistent equations (14)–(16) for the resolvents are closed by Woodbury and trace identities under the stated assumptions; they are not fitted to influence histograms. Numerical comparisons (Figs. 1–2) are validation, not inputs. Active-learning remarks are post-hoc conditionings of the same limiting measure. Citations to prior high-dimensional M-estimation (El Karoui, Donoho–Montanari, Thrampoulidis et al.) and minor self-citations (Cui–Lu) supply standard tools, not the target result. No step reduces by construction to its own input.

Axiom & Free-Parameter Ledger

0 free parameters · 4 axioms · 0 invented entities

The central claim rests on standard high-dimensional assumptions (Gaussian or elliptical design, strong convexity and derivative bounds on the loss) that are classical in the exact-asymptotics literature, plus the technical leave-one-out and resolvent machinery developed in the appendices. No free parameters are fitted to the influence histograms; all summary statistics are determined by self-consistent equations. No new physical or statistical entities are postulated.

axioms (4)
  • domain assumption Covariates are i.i.d. isotropic Gaussian (or, conjecturally, elliptical with random scale).
    Used for residual Gaussianity (Lemma C.2) and all concentration arguments; stated in §2.1 and extended in Appendix F.
  • domain assumption Loss ℓ is strongly convex in its first argument with derivatives up to order four bounded by O(polylog n).
    Assumption A.1; required for leave-one-out expansions, Hessian invertibility and Lipschitz control of resolvents.
  • domain assumption Test-error functional E has bounded first and second derivatives on a domain containing the summary statistics.
    Assumption A.4; needed to transfer concentration from parameter differences to influence values.
  • standard math Ridge regularization λ > 0 is fixed and positive.
    Ensures uniform positive-definiteness of Hessians and existence of a deterministic contour Γ enclosing the spectrum.

pith-pipeline@v1.1.0-grok45 · 66391 in / 2594 out tokens · 31287 ms · 2026-07-13T04:23:26.825387+00:00 · methodology

0 comments
read the original abstract

The impact of a given training point on a statistical model is classically measured through its leave-one-out influence, which quantifies the effect of its removal from the training set on the model accuracy. While the statistics of leave-one-out influences are well understood in the low-dimensional, large sample limit $n\to \infty, d=O(1)$, they become more intricate in high dimensions, as the influence of a given sample develops non-trivial dependencies on all other training samples. For convex M-estimation under Gaussian design, in the high-dimensional limit $n\asymp d$, we show that the distribution of the influences across the training set converges to a limiting measure which we sharply characterize. Building on these results, we provide evidence that influential samples tend to lie close to the decision boundary, thereby making contact with a standard data selection heuristic in active learning.

Figures

Figures reproduced from arXiv: 2607.09250 by Hugo Cui.

Figure 1
Figure 1. Figure 1: Empirical distribution νˆIF of the leave-one-out test error influences IFi (5) across the training set. (left) Logistic regression ℓ(z, y) = ln 1 + exp(−yz)  for binary classification (y = sign(⟨β, x⟩)), α = 2, λ = 0.05. The blue histogram represents numerical experiments on synthetic Gaussian data, in dimension d = 2000. The red solid line indicates the limiting distribution φIF♯N (04, Q) (9) characteriz… view at source ↗
Figure 2
Figure 2. Figure 2: Empirical distribution νˆIF of the leave-one-out test error influences IFi (5) across the training set, for ridge regression (ℓ(z, y) = 1/2(y − z) 2 ), assuming a linear model y = ⟨β, x⟩+N (0, δ2 ) (1). (left) Histograms of νˆIF obtained from numerical experiments in dimension d = 1000, contrasted with the limiting distribution φIF♯N (04, Q) (9) characterized in the extension of Theorem 2.1 stated in Conje… view at source ↗
Figure 3
Figure 3. Figure 3: Logistic regression ℓ(y, z) = ln 1 + exp{−yz}  with λ = 0.05, binary classification task. (left) Marginal distribution νIF|ω of the test error influence IFi (5) conditional on the margin D xi , wˆ(i) E = ω, for α = 2. Lines are obtained from conditioning the limiting density φIF ♯ N (04, Q | ω) (9) characterized in Theorem 2.1. Inset: mean influence µIF|ω conditional on the margin being ω, as a function o… view at source ↗
Figure 4
Figure 4. Figure 4: Reproducing the plots of Fig [PITH_FULL_IMAGE:figures/full_fig_p017_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Test-error influence IF (5) for the odd versus even classification task on MNIST digits [LeCun et al., 1998] (left), and for the pneumonia diagnosis task on chest X-rays images [Kermany et al., 2018] (right). A 3−layer, ReLU-activated neural network feature map was trained on a first held-out dataset, and the population readout β estimated from retraining on another held-out dataset. The weights wˆ are est… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

129 extracted references · 19 linked inside Pith

  1. [1]

    2011 IEEE International Symposium on Information Theory Proceedings , pages=

    Generalized approximate message passing for estimation with random linear mixing , author=. 2011 IEEE International Symposium on Information Theory Proceedings , pages=. 2011 , organization=

    2011

  2. [2]

    Advances in Neural Information Processing Systems , volume=

    Generalization error in high-dimensional perceptrons: Approaching bayes error with convex optimization , author=. Advances in Neural Information Processing Systems , volume=

  3. [3]

    2009 , publisher=

    Active learning literature survey , author=. 2009 , publisher=

    2009

  4. [4]

    Journal of chemical information and computer sciences , volume=

    Active learning with support vector machine applied to gene expression data for cancer classification , author=. Journal of chemical information and computer sciences , volume=. 2004 , publisher=

    2004

  5. [5]

    Advances in Neural Information Processing Systems , volume=

    Universality laws for gaussian mixtures in generalized linear models , author=. Advances in Neural Information Processing Systems , volume=

  6. [6]

    Conference on Learning Theory , pages=

    Universality of empirical risk minimization , author=. Conference on Learning Theory , pages=. 2022 , organization=

    2022

  7. [7]

    IEEE Transactions on Information Theory , volume=

    Universality laws for high-dimensional learning with random features , author=. IEEE Transactions on Information Theory , volume=. 2022 , publisher=

    2022

  8. [8]

    International conference on machine learning , pages=

    The role of regularization in classification of high-dimensional noisy gaussian mixture , author=. International conference on machine learning , pages=. 2020 , organization=

    2020

  9. [9]

    Journal of Statistical Mechanics: Theory and Experiment , volume=

    High-dimensional learning of narrow neural networks , author=. Journal of Statistical Mechanics: Theory and Experiment , volume=. 2025 , publisher=

    2025

  10. [10]

    Journal of Machine Learning Research , volume=

    Kymatio: Scattering transforms in python , author=. Journal of Machine Learning Research , volume=

  11. [11]

    Probability Theory and Related Fields , volume=

    On the impact of predictor geometry on the performance on high-dimensional ridge-regularized generalized robust regression estimators , author=. Probability Theory and Related Fields , volume=. 2018 , publisher=

    2018

  12. [12]

    arXiv preprint arXiv:1311.2445 , year=

    Asymptotic behavior of unregularized and ridge-regularized high-dimensional robust regression estimators: rigorous results , author=. arXiv preprint arXiv:1311.2445 , year=

    Pith/arXiv arXiv

  13. [13]

    The Annals of Statistics , pages=

    The jackknife estimate of variance , author=. The Annals of Statistics , pages=. 1981 , publisher=

    1981

  14. [14]

    Communications in Mathematical Physics , volume=

    Free states of the canonical anticommutation relations , author=. Communications in Mathematical Physics , volume=. 1970 , publisher=

    1970

  15. [15]

    arXiv preprint arXiv:2509.25153 , year=

    High-dimensional analysis of single-layer attention for sparse-token classification , author=. arXiv preprint arXiv:2509.25153 , year=

    arXiv

  16. [16]

    cell , volume=

    Identifying medical diagnoses and treatable diseases by image-based deep learning , author=. cell , volume=. 2018 , publisher=

    2018

  17. [17]

    Advances in Neural Information Processing Systems , volume=

    Beyond neural scaling laws: beating power law scaling via data pruning , author=. Advances in Neural Information Processing Systems , volume=

  18. [18]

    arXiv preprint arXiv:2309.14563 , year=

    Towards a statistical theory of data selection under weak supervision , author=. arXiv preprint arXiv:2309.14563 , year=

    Pith/arXiv arXiv

  19. [19]

    Journal of the American Statistical Association , volume=

    Optimal subsampling for large sample logistic regression , author=. Journal of the American Statistical Association , volume=. 2018 , publisher=

    2018

  20. [20]

    Journal of Machine Learning Research , volume=

    Asymptotic analysis of sampling estimators for randomized numerical linear algebra algorithms , author=. Journal of Machine Learning Research , volume=

  21. [21]

    Proceedings of the fifth annual workshop on Computational learning theory , pages=

    Query by committee , author=. Proceedings of the fifth annual workshop on Computational learning theory , pages=

  22. [22]

    arXiv preprint arXiv:1912.07242 , year=

    More data can hurt for linear regression: Sample-wise double descent , author=. arXiv preprint arXiv:1912.07242 , year=

    Pith/arXiv arXiv 1912

  23. [23]

    Proceedings of the National Academy of Sciences , volume=

    Reconciling modern machine-learning practice and the classical bias--variance trade-off , author=. Proceedings of the National Academy of Sciences , volume=. 2019 , publisher=

    2019

  24. [24]

    Journal of Statistical Mechanics: Theory and Experiment , volume=

    Scaling description of generalization with number of parameters in deep learning , author=. Journal of Statistical Mechanics: Theory and Experiment , volume=. 2020 , publisher=

    2020

  25. [25]

    arXiv preprint arXiv:2511.03492 , year=

    Why Less is More (Sometimes): A Theory of Data Curation , author=. arXiv preprint arXiv:2511.03492 , year=

    arXiv

  26. [26]

    arXiv preprint arXiv:2406.11485 , year=

    Active clustering with bandit feedback , author=. arXiv preprint arXiv:2406.11485 , year=

    Pith/arXiv arXiv

  27. [27]

    Journal of Machine Learning Research , volume=

    Optimal subsampling for high-dimensional partially linear models via machine learning methods , author=. Journal of Machine Learning Research , volume=

  28. [28]

    ACM computing surveys (CSUR) , volume=

    A survey of deep active learning , author=. ACM computing surveys (CSUR) , volume=. 2021 , publisher=

    2021

  29. [29]

    Journal of machine learning research , volume=

    Support vector machine active learning with applications to text classification , author=. Journal of machine learning research , volume=

  30. [30]

    International conference on machine learning , pages=

    Convergence of uncertainty sampling for active learning , author=. International conference on machine learning , pages=. 2022 , organization=

    2022

  31. [31]

    International conference on computational learning theory , pages=

    Analysis of perceptron-based active learning , author=. International conference on computational learning theory , pages=. 2005 , organization=

    2005

  32. [32]

    Proceedings of the AAAI Conference on Artificial Intelligence , volume=

    Noise-adaptive margin-based active learning and lower bounds under tsybakov noise condition , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=

  33. [33]

    Conference on Learning Theory , pages=

    Active and passive learning of linear separators under log-concave distributions , author=. Conference on Learning Theory , pages=. 2013 , organization=

    2013

  34. [34]

    SIAM journal on control and optimization , volume=

    Monotone operators and the proximal point algorithm , author=. SIAM journal on control and optimization , volume=. 1976 , publisher=

    1976

  35. [35]

    Information and Inference: A Journal of the IMA , volume=

    Alternating minimization for generalized rank-1 matrix sensing: sharp predictions from a random initialization , author=. Information and Inference: A Journal of the IMA , volume=. 2024 , publisher=

    2024

  36. [36]

    arXiv preprint arXiv:2510.12249 , year=

    Optimal Regularization for Performative Learning , author=. arXiv preprint arXiv:2510.12249 , year=

    Pith/arXiv arXiv

  37. [37]

    Lou et al

    Hyperparameter tuning via trajectory predictions: stochastic prox-linear methods in matrix sensing: M. Lou et al. , author=. Mathematical Programming , pages=. 2025 , publisher=

    2025

  38. [38]

    Advances in Neural Information Processing Systems , volume=

    Precise asymptotics of reweighted least-squares algorithms for linear diagonal networks , author=. Advances in Neural Information Processing Systems , volume=

  39. [39]

    Mathematical and Scientific Machine Learning , pages=

    Solvable model for inheriting the regularization through knowledge distillation , author=. Mathematical and Scientific Machine Learning , pages=. 2022 , organization=

    2022

  40. [40]

    arXiv preprint arXiv:2501.16226 , year=

    The effect of optimal self-distillation in noisy gaussian mixture model , author=. arXiv preprint arXiv:2501.16226 , year=

    arXiv

  41. [41]

    arXiv preprint arXiv:2205.07739 , year=

    The role of pseudo-labels in self-training linear classifiers on high-dimensional gaussian mixture data , author=. arXiv preprint arXiv:2205.07739 , year=

    Pith/arXiv arXiv

  42. [42]

    Journal of Statistical Mechanics: Theory and Experiment , volume=

    Asymptotic dynamics of alternating minimization for bilinear regression , author=. Journal of Statistical Mechanics: Theory and Experiment , volume=. 2025 , publisher=

    2025

  43. [43]

    Physical review letters , volume=

    Phase diagram of coupled glassy systems: A mean-field study , author=. Physical review letters , volume=. 1997 , publisher=

    1997

  44. [44]

    Probability Theory and Related Fields , volume=

    High dimensional robust m-estimation: Asymptotic variance via approximate message passing , author=. Probability Theory and Related Fields , volume=. 2016 , publisher=

    2016

  45. [45]

    IEEE Transactions on Information Theory , volume=

    Precise error analysis of regularized M -estimators in high dimensions , author=. IEEE Transactions on Information Theory , volume=. 2018 , publisher=

    2018

  46. [46]

    Advances in neural information processing systems , volume=

    Analysis of a greedy active learning strategy , author=. Advances in neural information processing systems , volume=

  47. [47]

    International Conference on Computational Learning Theory , pages=

    Margin based active learning , author=. International Conference on Computational Learning Theory , pages=. 2007 , organization=

    2007

  48. [48]

    Proceedings of the 26th annual international conference on machine learning , pages=

    Robust bounds for classification via selective sampling , author=. Proceedings of the 26th annual international conference on machine learning , pages=

  49. [49]

    , author=

    Robust Selective Sampling from Single and Multiple Teachers. , author=. COLT , pages=

  50. [50]

    Proceedings of the 28th international conference on machine learning: Bellevue, Washington, USA, june 28

    Better algorithms for selective sampling , author=. Proceedings of the 28th international conference on machine learning: Bellevue, Washington, USA, june 28. july 2, 2011 , pages=. 2011 , organization=

    2011

  51. [51]

    arXiv preprint arXiv:2408.13690 , year=

    Understanding uncertainty-based active learning under model mismatch , author=. arXiv preprint arXiv:2408.13690 , year=

    Pith/arXiv arXiv

  52. [52]

    Advances in Neural Information Processing Systems , volume=

    How to select which active learning strategy is best suited for your specific problem and budget , author=. Advances in Neural Information Processing Systems , volume=

  53. [53]

    arXiv preprint arXiv:2202.02794 , year=

    Active learning on a budget: Opposite strategies suit high and low budgets , author=. arXiv preprint arXiv:2202.02794 , year=

    Pith/arXiv arXiv

  54. [54]

    2014 International joint conference on neural networks (IJCNN) , pages=

    A new active labeling method for deep learning , author=. 2014 International joint conference on neural networks (IJCNN) , pages=. 2014 , organization=

    2014

  55. [55]

    European conference on machine learning , pages=

    Margin-based active learning for structured output spaces , author=. European conference on machine learning , pages=. 2006 , organization=

    2006

  56. [56]

    IEEE Transactions on Knowledge and Data Engineering , volume=

    A survey on data collection for machine learning: a big data-ai integration perspective , author=. IEEE Transactions on Knowledge and Data Engineering , volume=. 2019 , publisher=

    2019

  57. [57]

    arXiv preprint arXiv:2406.13581 , year=

    Optimal constants in concentration inequalities on the sphere and in the Gauss space , author=. arXiv preprint arXiv:2406.13581 , year=

    arXiv

  58. [58]

    Physical Review E , volume=

    Large deviations of semisupervised learning in the stochastic block model , author=. Physical Review E , volume=. 2022 , publisher=

    2022

  59. [59]

    ICML 2024 workshop on theoretical foundations of foundation models , year=

    Beyond model collapse: Scaling up with synthesized data requires reinforcement , author=. ICML 2024 workshop on theoretical foundations of foundation models , year=

    2024

  60. [60]

    Israel Journal of Mathematics , volume=

    Some inequalities for Gaussian processes and applications , author=. Israel Journal of Mathematics , volume=. 1985 , publisher=

    1985

  61. [61]

    Journal of computer and system sciences , volume=

    A decision-theoretic generalization of on-line learning and an application to boosting , author=. Journal of computer and system sciences , volume=. 1997 , publisher=

    1997

  62. [62]

    arXiv preprint arXiv:2507.19611 , year=

    State evolution beyond first-order methods I: Rigorous predictions and finite-sample guarantees , author=. arXiv preprint arXiv:2507.19611 , year=

    Pith/arXiv arXiv

  63. [63]

    arXiv preprint arXiv:2509.22341 , year=

    Preventing Model Collapse Under Overparametrization: Optimal Mixing Ratios for Interpolation Learning and Ridge Regression , author=. arXiv preprint arXiv:2509.22341 , year=

    arXiv

  64. [64]

    arXiv preprint arXiv:2502.15588 , year=

    Improving the Scaling Laws of Synthetic Data with Deliberate Practice , author=. arXiv preprint arXiv:2502.15588 , year=

    Pith/arXiv arXiv

  65. [65]

    Machine Learning: Science and Technology , volume=

    Large deviations in the perceptron model and consequences for active learning , author=. Machine Learning: Science and Technology , volume=. 2021 , publisher=

    2021

  66. [66]

    Foundations and Trends in Machine Learning , volume=

    Theory of active learning , author=. Foundations and Trends in Machine Learning , volume=

  67. [67]

    , author=

    Randnla: Randomized Numerical Linear Algebra Randomization offers new benefits for large-scale linear algebra computations. , author=. Communications of the ACM , volume=

  68. [68]

    Proceedings of the seventeenth annual ACM-SIAM symposium on Discrete algorithm , pages=

    Sampling algorithms for l 2 regression and applications , author=. Proceedings of the seventeenth annual ACM-SIAM symposium on Discrete algorithm , pages=

  69. [69]

    The Journal of Machine Learning Research , volume=

    A statistical perspective on algorithmic leveraging , author=. The Journal of Machine Learning Research , volume=. 2015 , publisher=

    2015

  70. [70]

    arXiv preprint arXiv:2601.23031 , year=

    Asymptotic Theory of Iterated Empirical Risk Minimization, with Applications to Active Learning , author=. arXiv preprint arXiv:2601.23031 , year=

    arXiv

  71. [71]

    Information and Inference: A Journal of the IMA , volume=

    Hitting the high-dimensional notes: An ode for sgd learning dynamics on glms and multi-index models , author=. Information and Inference: A Journal of the IMA , volume=. 2024 , publisher=

    2024

  72. [72]

    Wiley Series in Probability and Mathematical Statistics , year=

    Regression diagnostics: Identifying influential data and sources of collinearity , author=. Wiley Series in Probability and Mathematical Statistics , year=

  73. [73]

    Comptes Rendus

    On the distribution of the product of correlated normal random variables , author=. Comptes Rendus. Math

  74. [74]

    Journal of Statistical Mechanics: Theory and Experiment , volume=

    High-dimensional robust regression under heavy-tailed data: asymptotics and universality , author=. Journal of Statistical Mechanics: Theory and Experiment , volume=. 2024 , publisher=

    2024

  75. [75]

    , author=

    On Limit of the Largest Eigenvalue of the Large Dimensional Sample Covariance Matrix. , author=

  76. [76]

    , author=

    Introduction to the non-asymptotic analysis of random matrices. , author=

  77. [77]

    2018 , publisher=

    High-dimensional probability: An introduction with applications in data science , author=. 2018 , publisher=

    2018

  78. [78]

    arXiv preprint arXiv:2109.02644 , year=

    Spectral properties of sample covariance matrices arising from random matrices with independent non identically distributed columns , author=. arXiv preprint arXiv:2109.02644 , year=

    Pith/arXiv arXiv

  79. [79]

    Journal of the american statistical association , volume=

    The influence curve and its role in robust estimation , author=. Journal of the american statistical association , volume=. 1974 , publisher=

    1974

  80. [80]

    2007 , publisher=

    Random fields and geometry , author=. 2007 , publisher=

    2007

Showing first 80 references.