Pith. sign in

REVIEW 3 major objections 5 minor 118 references

Beyond Universal Approximation Theorems: Algorithmic Uniform Approximation by Neural Networks Trained with Noisy Data

T0 review · 3 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper gives an explicit randomized training algorithm for a transformer that uniformly recovers a continuous target function from finitely many noisy samples, with minimax-optimal trained parameter counts.

desk verdict A promising algorithmic UAT pipeline and a new symmetry framework, but the main theorem's probability expression is not a probability and the abstract overstates the scope; the central claims as stated do not hold. read the letter →

arxiv 2509.00924 v1 pith:7MPDYKOJ submitted 2025-08-31 stat.ML cs.LGcs.NAcs.NEmath.NAmath.PR

Anastasis Kratsios , Tin Sum Cheng , Daniel Roy This is my paper
classification stat.MLcs.LGcs.NAcs.NEmath.NAmath.PR MSC 68T0768Q3268T0541A65
keywords universalapproximationalgorithmictrainingnoisydatatransformersrandomfeaturescombinatorialsymmetryfine-tuningminimaxoptimality
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to close the gap between universal approximation theorems and actual training by giving an explicit, randomized training algorithm for a transformer that uniformly approximates a target function from finitely many noisy samples on [0,1]^d. The construction runs through three phases: local averaging to denoise the data, an attention layer that clusters inputs sharing similar target values, and a frozen random ReLU encoder followed by ordinary least squares on the final layer. The authors prove that, for target functions satisfying a combinatorial symmetry condition, the trained network attains minimax-optimal numbers of trainable parameters (up to logarithmic factors), exactly interpolates the denoised data, and has controlled Lipschitz regularity. They also show that fine-tuning on related out-of-distribution tasks can be done with sub-linear parameter counts when the new task shares the same symmetries.

What carries the argument

The load-bearing machinery is the combination of the attention layer and the random feature encoder. The attention layer uses max-temperature softmax with an l-infinity alignment score, which routes each input to the nearest cluster representative and makes the learned representation depend on the domain structure. The encoder is a frozen two-layer ReLU MLP with a random unactivated projection, designed so that the resulting deep feature matrix is well-conditioned with high probability; this well-conditioning is what guarantees a unique ridgeless regression solution, interpolation of the denoised data, and Lipschitz regularity of the final network.

What would settle it

Take a continuous target on [0,1]^d whose values on two adjacent grid cells differ by less than epsilon but are not equal, run Algorithms 1-3 on sufficiently many noisy samples, and check whether the trained network uniformly approximates the true function to tolerance below epsilon. The theorem predicts failure at this separation threshold, whereas the abstract's 'any continuous function' claim would require success.

Watch

Extended reading notes

Core claim

The central claim is Theorem 5.1: under Assumption A.1, for an (epsilon,S)-symmetric task, the transformer trained by Algorithms 1, 2, and 3 uniformly approximates the ground truth function with high probability, exactly interpolates the denoised training data, and has bounded Lipschitz regularity. The proof works by first denoising and compressing the noisy dataset, then using a max-temperature attention mechanism to cluster the domain into regions whose target values are separated, and finally regressing on a well-conditioned random deep feature matrix. The authors argue this is the first algorithmic uniform approximation theorem that handles noisy training data without decoupling approxim

Load-bearing premise

The target function's distinct values must be separated by at least epsilon on the q-grid, so the clustering step can tell them apart; if two genuinely different values are too close, the algorithm's uniform approximation guarantee no longer follows.

Editorial extensions

If this is right

  • If Theorem 5.1 holds, a trained transformer can recover the ground truth uniformly from noisy data, not merely match in-sample loss.
  • The trained parameter count is minimax-optimal up to logarithmic factors, so the construction cannot be substantially improved without additional structure.
  • When an out-of-distribution task shares the same combinatorial symmetries, fine-tuning only the final layer uses O(epsilon^{-r}) parameters and achieves O(1/N^r) sample complexity, which is dimension-free when the symmetry condition holds.
  • Under noiseless data, the algorithm gives an explicit polynomial-time procedure to construct a three-layer MLP that memorizes any finite regression dataset and also serves as a uniform approximator.
  • The denoising preprocessing step, Algorithm 1, is presented as independently useful for other deep learning pipelines and theoretical analyses.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The abstract promises uniform approximation of any continuous function, but the theorem actually requires the target's distinct values on the q-grid to be epsilon-separated; without that separation, the clustering step can merge regions with genuinely different target values, and the stated guarantee no longer follows.
  • A concrete test of the separation threshold would be to run Algorithms 1-3 on a continuous function where two grid cells have values differing by less than epsilon but not exactly equal, and measure whether the uniform error stays below epsilon.
  • The paper's 1D experiments suggest the attention mechanism captures high-frequency detail early, but a systematic comparison over higher dimensions and multiple random seeds would test whether this behavior is robust and not an artifact of the specific benchmark functions.
  • The combinatorial symmetry condition may be checked directly from data before training, unlike smoothness assumptions, which could make it a practical diagnostic for when this style of algorithmic approximation is applicable.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes a three-phase, architecture-specific training procedure for a transformer with max-temperature attention, a randomly initialized ReLU MLP encoder, and a linear readout. Given N noisy samples from a target function, Algorithm 1 denoises and compresses the data by cube averaging, Algorithm 2 initializes the attention keys/values via clustering, and Algorithm 3 specifies the random encoder initialization; the final layer is fit by ridgeless linear regression. Theorem 5.1 claims that, for (ε,S)-symmetric target functions satisfying Assumption A.1, the trained transformer uniformly approximates the ground truth with high probability, exactly interpolates the denoised data, and has Lipschitz regularity. The paper further claims minimax-optimal trainable parameter count and sub-linear fine-tuning complexity for out-of-distribution tasks sharing the same combinatorial symmetries.

Significance. If the main theorem were correct, this would be a valuable step toward an algorithmic, noise-aware universal approximation theory: it provides an explicit construction with a concrete training algorithm, analyzes a realistic architecture including attention, and compares parameter counts with classical minimax UATs. The denoising and clustering analysis is nontrivial and the random feature conditioning arguments are of independent interest. However, the central probability statement in Theorem 5.1 is invalid as written, and the scope of the theorem is substantially narrower than the abstract's claim of approximating any continuous function. The secondary sub-linear rate claim is also internally inconsistent. The paper contains promising ideas, but the advertised results are not currently supported.

major comments (3)
  1. [Theorem 5.1 and Appendix B.3] The quantity δ(κ,N) defined in Theorem 5.1 is not a probability. It is a product of terms including κ^{2+α/2}, which grows super-polynomially; hence 1−δ(κ,N) is negative for all nontrivial parameter regimes and the statement 'with probability at-least 1−δ(κ,N)' is vacuous. The proof never derives this expression; it instead assembles separate high-probability events from Proposition 4.1, Proposition 4.2, and Proposition B.4, with failure probabilities δ, δ, and 2e^{−κ}+2κ^{−3−20(1+α)}. The correct theorem statement should combine these as, e.g., 1−δ−2e^{−κ}−2κ^{−3−20(1+α)} (or a union-bound version), not as the singular-value lower bound itself. As stated, the main theorem's high-probability uniform approximation and interpolation guarantee is unsupported.
  2. [Abstract, Section 1, Definition 3.1, Theorem 5.1] The abstract and introduction claim uniform approximation of 'any continuous function on [0,1]^d' from noisy data, but Theorem 5.1 applies only to (ε,S)-symmetric functions at scale q with ε-separated centroid values. This assumption is load-bearing: Proposition 4.2 and Lemma B.3 require that cubes with distinct ground-truth values have denoised values separated by more than ε/8; without it, Algorithm 2 merges regions with different target values and the stated uniform approximation guarantee fails. No theorem or proof is provided that reduces a general continuous f to the symmetric class, and the informal statement in Section 3.1 that 'there is no loss in generality' is not backed by a formal result. The scope of the main result must either be narrowed in the abstract and title, or a rigorous approximation argument for general continuous targets must be added.
  3. [Corollary 5.1 and Section 5.2] The claimed sub-linear learning rate is internally inconsistent. The corollary assumes 0<r<1 and states an error bound that, as printed, reads '≲ 1/\tilde N^r < 1/\tilde N'; for 0<r<1 the inequality is reversed, so the statement is false. The condition (24), ln(κ/C)/ln(q) ≤ r, relates κ to the quantization scale q, not to the sample size \tilde N, and the proof is merely a restatement of the assumption. The earlier claim in Section 1.3 that 'a sample complexity of O(1/N^r) ≪ O(1/N)' is also mathematically incorrect for 0<r<1. The secondary OOD fine-tuning contribution therefore lacks a valid quantitative rate statement as written.
minor comments (5)
  1. [Section 2.2] Typo: 'mportantly' should be 'Importantly'.
  2. [Theorem 5.2 proof] The proof refers to 'Theorem 4.1' when the intended reference is Proposition 4.1.
  3. [Algorithm 2] The definition of the values matrix V is garbled: 'V ← (1/√d x)_{x∈D⋆} (I_{x_n∈c})_{N,c∈C,n=1}' is unclear as to dimensions and indexing. Please rewrite.
  4. [Lemma B.1] In the sub-Gaussian tail bound, the displayed probability is for the empirical mean of the noise variables, but the text writes 'X_n' instead of 'ε_n' in the last step.
  5. [Throughout] There are many grammar and typographical errors (e.g., 'noising training data', 'an (ε,S)-symmetric task', 'retroactively couplet'). A careful proofreading pass is needed.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; the construction is explicit and the main approximation bound is a standard discretization estimate (with a separate, non-circular probability-validity flaw in Theorem 5.1).

full rationale

After walking the derivation chain — Algorithms 1–3, Propositions 4.1–4.3, Theorems 5.1–5.2, and the appendix proofs — the central construction is explicit and not circular. Algorithm 1 averages noisy labels within cubes; Proposition 4.1 gives a genuine concentration bound for this averaging, and Lemma B.3/Proposition 4.2 use the (ε,S)-symmetry and ε-separation assumptions to show that clustering on denoised values recovers the true symmetry classes. Algorithm 3's well-conditioning guarantee (Proposition 4.3 / Proposition B.4) is proved from non-linear random matrix results external to the paper (Vershynin [110]), and the final layer is the actual ordinary least-squares solution. The uniform approximation bound in Theorem 5.2 is a standard cube-discretization estimate: for x in a cube Q of cluster c, |f(x) − βE(x)| ≤ |f(x) − f(Q̄)| + |f(Q̄) − y_c| ≤ ω(√d/q) + 1/q. No equation in this chain is equivalent to its input by construction: the fitted values y_c are not the same as the function values being predicted, and the separation/symmetry conditions are hypotheses, not conclusions. Self-citations appear ([2] for a covering-number bound, [52] for minimax approximation benchmarks, [64,65] for context), but none is load-bearing: the well-conditioning proof uses external results [110], and the minimax comparison is ultimately against the external lower bound [101, Theorem 2.4]. I also flag, as a correctness issue rather than a circularity issue, that Theorem 5.1's displayed δ(κ,N) is not a probability for admissible parameters: the factor (κ^{2+α/2} − C1) grows, so the claimed success probability '1 − δ(κ,N)' is not a meaningful high-probability guarantee. The proof appears to conflate a denoising failure probability with spectral lower bounds. This invalidates the stated probability guarantee, but it does not make the derivation circular.

Assumptions & free parameters 4 free parameters · 6 assumptions · 1 invented entities

The construction rests on standard probabilistic and linear-algebraic tools, but the paper adds a strong structural assumption, the epsilon-symmetric separation condition, that is not mentioned in the abstract. The free parameters q, kappa, F are chosen by the algorithm/user, and the hidden width F is required to be super-polynomial in the number of clusters, which undermines the practical reading of 'minimax-optimal'. No genuinely new physical or external entity is postulated; the combinatorial symmetry is a formal concept with independent empirical support.

free parameters (4)
  • quantization scale q = chosen with q >= L/epsilon or q >= sqrt(d)/omega^{-1}(sqrt(d)/epsilon)
    q sets the diameter of the cubes used in Algorithm 1 and the coarsening error omega(sqrt(d)/q)+1/q. It is a user-chosen hyperparameter, not data-fitted, but it controls both approximation accuracy and sample complexity.
  • accuracy tolerance epsilon = target approximation error, chosen by user
    epsilon enters Assumption 4.1, the clustering threshold epsilon/8 in Algorithm 2, and the separation condition in Definition 3.1. It is not learned from data.
  • number of symmetries/clusters kappa = assumed to satisfy kappa <= q^d and to equal the number of distinct output clusters
    In Theorem 5.1, kappa is the number of symmetry classes of the target. For a general function it would be the number of clusters produced by Algorithm 2, which can be as large as q^d. The sub-linear fine-tuning claims depend on kappa being small, which is assumed, not derived.
  • hidden width F = chosen with F > kappa^{5(1+alpha)} and F >= Delta^{C_1^2(1+alpha)}
    F is a free architectural hyperparameter required to be super-polynomial in kappa and q to guarantee well-conditioning of the random feature matrix. It is a genuine free parameter of the construction and is not counted in the paper's 'trainable parameters' total, which is a source of overclaim.
assumptions (6)
  • domain assumption Measurement noise is centered (C, sigma^2)-sub-Gaussian (Assumption 3.1)
    Invoked in Lemma B.1 and Proposition 4.1 to control the averaging error in Algorithm 1. The whole denoising step depends on this tail condition.
  • domain assumption Sampling distribution P_X is non-atomic (Assumption 4.1 / A.1)
    Used to guarantee that the boundary 'bad set' G has zero measure, allowing the essential-supremum formulation in Theorem 5.2.
  • domain assumption Target function is L-Lipschitz or uniformly continuous with modulus omega (Setting A.1)
    The uniform approximation error is expressed in terms of omega(sqrt(d)/q) + 1/q, so some continuity is required. The abstract says 'any continuous', which is the general setting, but the proof explicitly uses the modulus.
  • ad hoc to paper Target is (epsilon,S)-symmetric at scale q with epsilon-separated centroid values (Definition 3.1)
    This is the paper's novel structural assumption. It is not derived and is load-bearing: the clustering in Algorithm 2 and the identifiability property in Proposition 4.2 require the centroid values to be separated by epsilon. The abstract omits this restriction.
  • standard math Random matrix concentration lemmas from Vershynin (2020) and Gordon's theorem (used in Propositions B.2-B.4)
    The well-conditioning of random ReLU features is imported from [110] with a relaxed norm version proven in Appendix C. This is standard high-dimensional probability, not specific to this paper.
  • standard math Moore-Penrose pseudo-inverse and spectral inequalities for least squares (used in Section 2.4 and Corollary B.1)
    The unique solution of the OLS problem and the bound on the operator norm of beta are standard linear algebra facts.
invented entities (1)
  • Combinatorial symmetry (S-symmetric task, Definition 3.1) independent evidence
    purpose: Defines task similarity and enables the sub-linear fine-tuning claims: a new task is similar if it shares the same domain partitioning into symmetry classes, allowing the attention layer to be frozen and only the readout retrained.
    The paper presents empirical evidence from MNIST (Figures 3-5) that such repeated-value structures occur in real image data, and Proposition 3.1 argues they are unavoidable by Ramsey theory. This gives the entity an empirical handle outside the paper's own formalism, although the theory does not predict specific datasets.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Beyond Universal Approximation Theorems: Algorithmic Uniform Approximation by Neural Networks Trained with Noisy Data." pith.science (2026). https://pith.science/paper/7MPDYKOJ

@misc{pith2026250900924,
  author       = {Pith},
  title        = {Pith review of: Beyond Universal Approximation Theorems: Algorithmic Uniform Approximation by Neural Networks Trained with Noisy Data},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7MPDYKOJ}},
  note         = {Machine review of arXiv:2509.00924}
}
abstract

At its core, machine learning seeks to train models that reliably generalize beyond noisy observations; however, the theoretical vacuum in which state-of-the-art universal approximation theorems (UATs) operate isolates them from this goal, as they assume noiseless data and allow network parameters to be chosen freely, independent of algorithmic realism. This paper bridges that gap by introducing an architecture-specific randomized training algorithm that constructs a uniform approximator from $N$ noisy training samples on the $d$-dimensional cube $[0,1]^d$. Our trained neural networks attain the minimax-optimal quantity of \textit{trainable} (non-random) parameters, subject to logarithmic factors which vanish under the idealized noiseless sampling assumed in classical UATs. Additionally, our trained models replicate key behaviours of real-world neural networks, absent in standard UAT constructions, by: (1) exhibiting sub-linear parametric complexity when fine-tuning on structurally related and favourable out-of-distribution tasks, (2) exactly interpolating the training data, and (3) maintaining reasonable Lipschitz regularity (after the initial clustering attention layer). These properties bring state-of-the-art UATs closer to practical machine learning, shifting the central open question from algorithmic implementability with noisy samples to whether stochastic gradient descent can achieve comparable guarantees.

Figures

Figures reproduced from arXiv: 2509.00924 by the authors.

Figure 1
Figure 1. The structure of the three-phase transformers trained with Algorithms 1, 2, and 3: Phase 0 – Denoising (Algorithm 1): In the preliminary denoising phase, an approximately noise-free dataset is formed by locally averaging the original noisy samples, allowing downstream training to use near-true function values. This step can be omitted if no measurement noise is present, as in universal approximation theorems; e.g. [… view at source ↗
Figure 2
Figure 2. Coarsening of a typical greyscale image into [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. ℏ = 10. 2 5 8 10 13 16 19 21 24 27 Domain Scale q - Over Images 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 R atio: q 2 Normalized Distinct Rounded Block Colors per MNIST Image [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (5 more)
Figure 6
Figure 6. Figure 6: A function with a (ϵ, 3)-combinatorial symmetry at scale 2. Each symmetry class S1, S2, and S3 is illustrated by one of the three distinct colours. The values of the centre of each cube under the real-valued symmetric function f is illustrated by the coloured bubble ly…
Figure 7
Figure 7. Figure 7: In this figure, the first task (left) assigns three distinct values, represented by different colours, to four cubes, while the second task (right) assigns three different values to the same four cubes, maintaining the same pattern of repeated values; that is, in both …
Figure 8
Figure 8. Figure 8: A comparison between random feature models (RFMs: [PITH_FULL_IMAGE:figures/full_fig_p017_8.png]
Figure 9
Figure 9. Figure 9: Comparison: A typical (100 epochs) trained (ADAM) 2-layer MLP vs. B) randomly initialized transformer with one attention head and 2 activated MLP layers and one trained final MLP output layer. Our examples across two challenging functions, illustrated in [PITH_FULL_IM…
Figure 10
Figure 10. Figure 10: Comparison: A typical (100 epochs) trained (ADAM) 2-layer MLP vs. B) randomly initialized transformer with one attention head and 2 activated MLP layers and one trained final MLP output layer. Conclusion: Even at initialization, one attention head can almost match the…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

118 extracted references · 64 canonical work pages

  1. [1]

    Weighted Sobolev Approximation Rates for Neural Networks on Unbounded Domains

    Ahmed Abdeljawad and Thomas Dittrich. Weighted Sobolev Approximation Rates for Neural Networks on Unbounded Domains. arXiv preprint arXiv:2411.04108, 2024

    arXiv 2024

  2. [2]

    Designing universal causal deep learning models: The geometric (hyper) transformer.Mathematical Finance, 34(2):671–735, 2024

    Beatrice Acciaio, Anastasis Kratsios, and Gudmund Pammer. Designing universal causal deep learning models: The geometric (hyper) transformer.Mathematical Finance, 34(2):671–735, 2024

    2024

  3. [3]

    Slic superpixels compared to state-of-the-art superpixel methods.IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(11):2274–2282, 2012

    Radhakrishna Achanta, Appu Shaji, Kevin Smith, Aurélien Lucchi, Pascal Fua, and Sabine Süsstrunk. Slic superpixels compared to state-of-the-art superpixel methods.IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(11):2274–2282, 2012

    2012

  4. [4]

    Deep Neural Networks Are Effective At Learning High-Dimensional Hilbert-Valued Functions From Limited Data

    Ben Adcock, Simone Brugiapaglia, Nick Dexter, and Sebastian Moraga. Deep neural networks are effective at learning high-dimensional hilbert-valued functions from limited data.arXiv preprint arXiv:2012.06081, 2020

    work page Pith review arXiv 2012

  5. [5]

    Scale-sensitive dimensions, uniform convergence, and learnability.Journal of the ACM (JACM), 44(4):615–631, 1997

    Noga Alon, Shai Ben-David, Nicolo Cesa-Bianchi, and David Haussler. Scale-sensitive dimensions, uniform convergence, and learnability.Journal of the ACM (JACM), 44(4):615–631, 1997

    1997

  6. [6]

    Scale-sensitive dimensions, uniform convergence, and learnability.J

    Noga Alon, Shai Ben-David, Nicolò Cesa-Bianchi, and David Haussler. Scale-sensitive dimensions, uniform convergence, and learnability.J. ACM, 44(4):615–631, 1997

    1997

  7. [7]

    On the properties of variational approximations of gibbs posteriors

    Pierre Alquier, James Ridgway, and Nicolas Chopin. On the properties of variational approximations of gibbs posteriors. Journal of Machine Learning Research, 17(236):1–41, 2016

    2016

  8. [8]

    Some fine properties of sets of finite perimeter in ahlfors regular metric measure spaces

    Luigi Ambrosio. Some fine properties of sets of finite perimeter in ahlfors regular metric measure spaces. Advances in Mathematics, 159(1):51–67, 2001

    2001

Show all 118 references
  1. [9]

    On a theory of learning with similarity functions

    Maria-Florina Balcan and Avrim Blum. On a theory of learning with similarity functions. InProceedings of the 23rd international conference on Machine learning, pages 73–80, 2006

    2006

  2. [10]

    Universal approximation bounds for superpositions of a sigmoidal function

    Andrew R Barron. Universal approximation bounds for superpositions of a sigmoidal function. IEEE Transactions on Information theory, 39(3):930–945, 1993

    1993

  3. [11]

    Spectrally-normalized margin bounds for neural networks

    Peter L Bartlett, Dylan J Foster, and Matus J Telgarsky. Spectrally-normalized margin bounds for neural networks. Advances in neural information processing systems, 30, 2017

    2017

  4. [12]

    Nearly-tight vc-dimension and pseudodimension bounds for piecewise linear neural networks.Journal of MachineLearning Research, 20(63):1– 17, 2019

    Peter L Bartlett, Nick Harvey, Christopher Liaw, and Abbas Mehrabian. Nearly-tight vc-dimension and pseudodimension bounds for piecewise linear neural networks.Journal of MachineLearning Research, 20(63):1– 17, 2019. 35

    2019

  5. [13]

    Vapnik-chervonenkis dimension of neural nets.The handbook of brain theory and neural networks, pages 1188–1192, 2003

    Peter L Bartlett and Wolfgang Maass. Vapnik-chervonenkis dimension of neural nets.The handbook of brain theory and neural networks, pages 1188–1192, 2003

    2003

  6. [14]

    On numerical computation for the distribution of the convolution ofN independent rectified Gaussian variables.J

    Maxime Beauchamp. On numerical computation for the distribution of the convolution ofN independent rectified Gaussian variables.J. SFdS, 159(1):88–111, 2018

    2018

  7. [15]

    Learnability and the vapnik-chervonenkis dimension.Journal of the ACM (JACM), 36(4):929–965, 1989

    Anselm Blumer, Andrzej Ehrenfeucht, David Haussler, and Manfred K Warmuth. Learnability and the vapnik-chervonenkis dimension.Journal of the ACM (JACM), 36(4):929–965, 1989

    1989

  8. [16]

    Optimal approximation with sparsely connected deep neural networks.SIAM Journal on Mathematics of Data Science, 1(1):8–45, 2019

    Helmut Bolcskei, Philipp Grohs, Gitta Kutyniok, and Philipp Petersen. Optimal approximation with sparsely connected deep neural networks.SIAM Journal on Mathematics of Data Science, 1(1):8–45, 2019

    2019

  9. [17]

    Practical existence theorems for deep learning approximation in high dimensions

    Simone Brugiapaglia. Practical existence theorems for deep learning approximation in high dimensions. In 15th International Conference on Sampling Theory and Applications, 2024

    2024

  10. [18]

    Physics-informed deep learning and compressive collocation for high-dimensional diffusion-reaction equations: practical existence theory and numerics

    Simone Brugiapaglia, Nick Dexter, Samir Karam, and Weiqi Wang. Physics-informed deep learning and compressive collocation for high-dimensional diffusion-reaction equations: practical existence theory and numerics. arXiv preprint arXiv:2406.01539, 2024

    2024

  11. [19]

    Physics-informed neural networks (pinns) for fluid mechanics: A review.Acta Mechanica Sinica, 37(12):1727–1738, 2021

    Shengze Cai, Zhiping Mao, Zhicheng Wang, Minglang Yin, and George Em Karniadakis. Physics-informed neural networks (pinns) for fluid mechanics: A review.Acta Mechanica Sinica, 37(12):1727–1738, 2021

    2021

  12. [20]

    Dimension-free log-sobolev inequalities for mixture distributions

    Hong-Bin Chen, Sinho Chewi, and Jonathan Niles-Weed. Dimension-free log-sobolev inequalities for mixture distributions. Journal of Functional Analysis, 281(11):109236, 2021

    2021

  13. [21]

    Characterizing overfitting in kernel ridgeless regression through the eigenspectrum

    Tin Sum Cheng, Aurelien Lucchi, Anastasis Kratsios, and David Belius. Characterizing overfitting in kernel ridgeless regression through the eigenspectrum. In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenka...

    2024

  14. [22]

    A comprehensive analysis on the learning curve in kernel ridge regression.Advancesin Neural Information Processing Systems, 37:24659–24723, 2024

    Tin Sum Cheng, Aurelien Lucchi, Anastasis Kratsios, and David Belius. A comprehensive analysis on the learning curve in kernel ridge regression.Advancesin Neural Information Processing Systems, 37:24659–24723, 2024

    2024

  15. [23]

    A theoretical analysis of the test error of finite-rank kernel ridge regression

    Tin Sum Cheng, Aurelien Lucchi, Anastasis Kratsios, Ivan Dokmanić, and David Belius. A theoretical analysis of the test error of finite-rank kernel ridge regression. InThirty-seventh Conference on Neural Information Processing Systems, 2023

    2023

  16. [24]

    Efficient approximation of high-dimensional functions with neural networks.IEEE Transactions on Neural Networks and Learning Systems, 33(7):3079– 3093, 2021

    Patrick Cheridito, Arnulf Jentzen, and Florian Rossmannek. Efficient approximation of high-dimensional functions with neural networks.IEEE Transactions on Neural Networks and Learning Systems, 33(7):3079– 3093, 2021

    2021

  17. [25]

    Optimal stable nonlinear approximation

    Albert Cohen, Ronald DeVore, Guergana Petrova, and Przemyslaw Wojtaszczyk. Optimal stable nonlinear approximation. Foundations of Computational Mathematics, 22(3):607–648, 2022

    2022

  18. [26]

    An improved uniform convergence bound with fat-shattering dimension

    Roberto Colomboni, Emmanuel Esposito, and Andrea Paudice. An improved uniform convergence bound with fat-shattering dimension. Information Processing Letters, 188:106539, 2025

    2025

  19. [27]

    Scientific machine learning through physics–informed neural networks: Where we are and what’s next

    Salvatore Cuomo, Vincenzo Schiano Di Cola, Fabio Giampaolo, Gianluigi Rozza, Maziar Raissi, and Francesco Piccialli. Scientific machine learning through physics–informed neural networks: Where we are and what’s next. Journal of Scientific Computing, 92(3):88, 2022

    2022

  20. [28]

    Approximation by superpositions of a sigmoidal function.Mathematics of Control, Signals and Systems, 2(4):303–314, 1989

    George Cybenko. Approximation by superpositions of a sigmoidal function.Mathematics of Control, Signals and Systems, 2(4):303–314, 1989

    1989

  21. [29]

    Nonlinear approxima- tion and (deep) relu networks.Constructive Approximation, 55(1):127–172, 2022

    Ingrid Daubechies, Ronald DeVore, Simon Foucart, Boris Hanin, and Guergana Petrova. Nonlinear approxima- tion and (deep) relu networks.Constructive Approximation, 55(1):127–172, 2022

    2022

  22. [30]

    Rogue waves and large deviations in deep sea

    Giovanni Dematteis, Tobias Grafke, and Eric Vanden-Eijnden. Rogue waves and large deviations in deep sea. Proceedings of the National Academy of Sciences, 115(5):855–860, 2018. Large-deviation and extreme value analysis for rare rogue-wave events in ocean dynamics. 36

    2018

  23. [31]

    Ronald A. DeVore. Nonlinear approximation. InActa numerica, 1998, volume 7 ofActa Numer., pages 51–150. Cambridge Univ. Press, Cambridge, 1998

    1998

  24. [32]

    Wavelet compression and nonlinear n-widths

    Ronald A DeVore, George Kyriazis, Dany Leviatan, and Vladimir M Tikhomirov. Wavelet compression and nonlinear n-widths. Adv. Comput. Math., 1(2):197–214, 1993

    1993

  25. [33]

    Learning multivariate log-concave distributions

    Ilias Diakonikolas, Daniel M Kane, and Alistair Stewart. Learning multivariate log-concave distributions. In Conference on Learning Theory, pages 711–727. PMLR, 2017

    2017

  26. [34]

    Memorization with neural nets: Going beyond the worst case

    Sjoerd Dirksen, Patrick Finke, and Martin Genzel. Memorization with neural nets: Going beyond the worst case. Journal of Machine Learning Research, 25(347):1–38, 2024

    2024

  27. [35]

    Computing nonvacuous generalization bounds for deep (stochastic) neural networks with many more parameters than training data.arXiv preprint arXiv:1703.11008, 2017

    Gintare Karolina Dziugaite and Daniel M Roy. Computing nonvacuous generalization bounds for deep (stochastic) neural networks with many more parameters than training data.arXiv preprint arXiv:1703.11008, 2017

    2017 arXiv

  28. [36]

    Vc dimension of graph neural networks with pfaffian activation functions.Neural Networks, 182:106924, 2025

    Giuseppe Alessio D’Inverno, Monica Bianchini, and Franco Scarselli. Vc dimension of graph neural networks with pfaffian activation functions.Neural Networks, 182:106924, 2025

    2025

  29. [37]

    On the efficiency of erm in feature learning

    Ayoub El Hanchi, Chris J Maddison, and Murat A Erdogdu. On the efficiency of erm in feature learning. Advances in Neural Information Processing Systems, 37:98596–98624, 2024

    2024

  30. [38]

    Springer, 1997

    Paul Embrechts, Claudia Klüppelberg, and Thomas Mikosch.Modelling Extremal Events: for Insurance and Finance. Springer, 1997. Foundational text bridging EVT and finance/insurance

    1997

  31. [39]

    Benefits of additive noise in composing classes with bounded capacity

    Alireza Fathollah Pour and Hassan Ashtiani. Benefits of additive noise in composing classes with bounded capacity. Advances in Neural Information Processing Systems, 35:32709–32722, 2022

    2022

  32. [40]

    Sum-of-squares proofs of logarithmic sobolev inequalities on finite markov chains

    Oisín Faust and Hamza Fawzi. Sum-of-squares proofs of logarithmic sobolev inequalities on finite markov chains. IEEE Transactions on Information Theory, 70(2):803–819, 2023

    2023

  33. [41]

    Efficient graph-based image segmentation.International Journal of Computer Vision, 59(2):167–181, 2004

    Pedro F Felzenszwalb and Daniel P Huttenlocher. Efficient graph-based image segmentation.International Journal of Computer Vision, 59(2):167–181, 2004

    2004

  34. [42]

    Sample compression, learnability, and the vapnik-chervonenkis dimension

    Sally Floyd and Manfred Warmuth. Sample compression, learnability, and the vapnik-chervonenkis dimension. Machine learning, 21(3):269–304, 1995

    1995

  35. [43]

    A practical existence theorem for reduced order models based on convolutional autoencoders.Foundations of Data Science, 7(1):72–98, 2025

    Nicola Rares Franco and Simone Brugiapaglia. A practical existence theorem for reduced order models based on convolutional autoencoders.Foundations of Data Science, 7(1):72–98, 2025

    2025

  36. [44]

    Benign overfitting without linearity: Neural network classifiers trained by gradient descent for noisy linear data

    Spencer Frei, Niladri S Chatterji, and Peter Bartlett. Benign overfitting without linearity: Neural network classifiers trained by gradient descent for noisy linear data. InConference on Learning Theory, pages 2668–2703. PMLR, 2022

    2022

  37. [45]

    On the approximate realization of continuous mappings by neural networks.Neural networks, 2(3):183–192, 1989

    Ken-Ichi Funahashi. On the approximate realization of continuous mappings by neural networks.Neural networks, 2(3):183–192, 1989

    1989

  38. [46]

    Scaling description of generalization with number of parameters in deep learning

    Mario Geiger, Stefano Spigler, Stephane d’Ascoli, Levent Sagun, Giulio Biroli, and Matthieu Wyart. Scaling description of generalization with number of parameters in deep learning. InInternational Conference on Machine Learning, pages 3647–3657. PMLR, 2020

    2020

  39. [47]

    Random feature neural networks learn black-scholes type pdes without curse of dimensionality

    Lukas Gonon. Random feature neural networks learn black-scholes type pdes without curse of dimensionality. Journal of Machine Learning Research, 24(189):1–51, 2023

    2023

  40. [48]

    Risk bounds for reservoir computing.Journal of Machine Learning Research, 21(240):1–61, 2020

    Lukas Gonon, Lyudmila Grigoryeva, and Juan-Pablo Ortega. Risk bounds for reservoir computing.Journal of Machine Learning Research, 21(240):1–61, 2020

    2020

  41. [49]

    A new proof of szemerédi’s theorem.Geometric & Functional Analysis GAFA, 11(3):465– 588, 2001

    William T Gowers. A new proof of szemerédi’s theorem.Geometric & Functional Analysis GAFA, 11(3):465– 588, 2001

    2001

  42. [50]

    Universal function approximation by deep neural nets with bounded width and relu activations

    Boris Hanin. Universal function approximation by deep neural nets with bounded width and relu activations. Mathematics, 7(10):992, 2019. 37

    2019

  43. [51]

    Probability inequalities for sums of bounded random variables.J

    Wassily Hoeffding. Probability inequalities for sums of bounded random variables.J. Amer. Statist. Assoc., 58:13–30, 1963

    1963

  44. [52]

    Bridging the gap between approximation and learning via optimal approximation by relu mlps of maximal regularity.arXiv preprint arXiv:2409.12335, 2024

    Ruiyang Hong and Anastasis Kratsios. Bridging the gap between approximation and learning via optimal approximation by relu mlps of maximal regularity.arXiv preprint arXiv:2409.12335, 2024

    2024 arXiv

  45. [53]

    Horn and Charles R

    Roger A. Horn and Charles R. Johnson.Matrix analysis. Cambridge University Press, Cambridge, second edition, 2013

    2013

  46. [54]

    Multilayer feedforward networks are universal approximators

    Kurt Hornik, Maxwell Stinchcombe, and Halbert White. Multilayer feedforward networks are universal approximators. Neural Networks, 2(5):359–366, 1989

    1989

  47. [55]

    Universal approximation of an unknown mapping and its derivatives using multilayer feedforward networks.Neural networks, 3(5):551–560, 1990

    Kurt Hornik, Maxwell Stinchcombe, and Halbert White. Universal approximation of an unknown mapping and its derivatives using multilayer feedforward networks.Neural networks, 3(5):551–560, 1990

    1990

  48. [56]

    Instance-dependent generalization bounds via optimal transport.Journal of Machine Learning Research, 24(349):1–51, 2023

    Songyan Hou, Parnian Kassraie, Anastasis Kratsios, Andreas Krause, and Jonas Rothfuss. Instance-dependent generalization bounds via optimal transport.Journal of Machine Learning Research, 24(349):1–51, 2023

    2023

  49. [57]

    Learning image priors through patch-based diffusion models for solving inverse problems.Advancesin Neural Information Processing Systems, 37:1625–1660, 2024

    Jason Hu, Bowen Song, Xiaojian Xu, Liyue Shen, and Jeffrey A Fessler. Learning image priors through patch-based diffusion models for solving inverse problems.Advancesin Neural Information Processing Systems, 37:1625–1660, 2024

    2024

  50. [58]

    Neural tangent kernel: Convergence and generalization in neural networks

    Arthur Jacot, Franck Gabriel, and Clément Hongler. Neural tangent kernel: Convergence and generalization in neural networks. InAdvances in Neural Information Processing Systems (NeurIPS), volume 31, 2018

    2018

  51. [59]

    Deep neural networks with relu-sine-exponential activations break curse of dimensionality in approximation on hölder class

    Yuling Jiao, Yanming Lai, Xiliang Lu, Fengru Wang, Jerry Zhijian Yang, and Yuanyuan Yang. Deep neural networks with relu-sine-exponential activations break curse of dimensionality in approximation on hölder class. SIAM Journal on Mathematical Analysis, 55(4):3635–3649, 2023

    2023

  52. [60]

    Universal approximation with deep narrow networks

    Patrick Kidger and Terry Lyons. Universal approximation with deep narrow networks. InConference on learning theory, pages 2306–2327. PMLR, 2020

    2020

  53. [61]

    Provable memorization capacity of transformers

    Junghwan Kim, Michelle Kim, and Barzan Mozafari. Provable memorization capacity of transformers. InThe Eleventh International Conference on Learning Representations, 2023

    2023

  54. [62]

    Benignoverfittingintwo-layerreluconvolutional neural networks

    YiwenKou, ZixiangChen, YuanzhouChen, andQuanquanGu. Benignoverfittingintwo-layerreluconvolutional neural networks. InInternational conference on machine learning, pages 17615–17659. PMLR, 2023

    2023

  55. [63]

    Neural operator: Learning maps between function spaces with applications to pdes

    Nikola Kovachki, Zongyi Li, Burigede Liu, Kamyar Azizzadenesheli, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. Neural operator: Learning maps between function spaces with applications to pdes. Journal of Machine Learning Research, 24(89):1–97, 2023

    2023

  56. [64]

    Small transformers compute universal metric embeddings

    Anastasis Kratsios, Valentin Debarnot, and Ivan Dokmanić. Small transformers compute universal metric embeddings. Journal of Machine Learning Research, 24(170):1–48, 2023

    2023

  57. [65]

    Is in-context universality enough? mlps are also universal in-context

    Anastasis Kratsios and Takashi Furuya. Is in-context universality enough? mlps are also universal in-context. arXiv preprint arXiv:2502.03327, 2025

    2025 arXiv

  58. [66]

    Universal approximation theorems for differentiable geometric deep learning

    Anastasis Kratsios and Léonie Papon. Universal approximation theorems for differentiable geometric deep learning. Journal of Machine Learning Research, 23(196):1–73, 2022

    2022

  59. [67]

    Sur l’intégration des fonctions discontinues

    Henri Lebesgue. Sur l’intégration des fonctions discontinues. In Annales scientifiques de l’École normale supérieure, volume 27, pages 361–450, 1910

    1910

  60. [68]

    Oracle inequalities for high-dimensional prediction.Bernoulli, 25(2):1225–1255, 2019

    Johannes Lederer, Lu Yu, and Irina Gaynanova. Oracle inequalities for high-dimensional prediction.Bernoulli, 25(2):1225–1255, 2019

    2019

  61. [69]

    Springer-Verlag, Berlin, 1991

    Michel Ledoux and Michel Talagrand.Probability in Banach spaces, volume 23 ofErgebnisse der Mathematik und ihrer Grenzgebiete (3) [Results in Mathematics and Related Areas (3)]. Springer-Verlag, Berlin, 1991. Isoperimetry and processes

    1991

  62. [70]

    Lower bounds on the vc-dimension of smoothly parametrized function classes

    Wee Sun Lee, Peter L Bartlett, and Robert C Williamson. Lower bounds on the vc-dimension of smoothly parametrized function classes. InProceedings of the seventh annual conference on Computational learning theory, pages 362–367, 1994. 38

    1994

  63. [71]

    Higher-order transformer derivative estimates for explicit pathwise learning guarantees.arXiv preprint arXiv:2405.16563, 2024

    Yannick Limmer, Anastasis Kratsios, Xuwei Yang, Raeid Saqur, and Blanka Horvath. Higher-order transformer derivative estimates for explicit pathwise learning guarantees.arXiv preprint arXiv:2405.16563, 2024

    2024 arXiv

  64. [72]

    Network in network.arXiv preprint arXiv:1312.4400, 2013

    Min Lin, Qiang Chen, and Shuicheng Yan. Network in network.arXiv preprint arXiv:1312.4400, 2013

    2013 arXiv

  65. [73]

    Relating data compression and learnability, 1986

    Nick Littlestone and Manfred Warmuth. Relating data compression and learnability, 1986

    1986

  66. [74]

    Citeseer, 1996

    George G Lorentz, Manfred von Golitschek, and Yuly Makovoz.Constructive approximation: advanced problems, volume 304. Citeseer, 1996

    1996

  67. [75]

    Deep network approximation for smooth functions

    Jianfeng Lu, Zuowei Shen, Haizhao Yang, and Shijun Zhang. Deep network approximation for smooth functions. SIAM J. Math. Anal., 53(5):5465–5506, 2021

    2021

  68. [76]

    Learning nonlinear operators via deeponet based on the universal approximation theorem of operators.Nature machineintelligence, 3(3):218–229, 2021

    Lu Lu, Pengzhan Jin, Guofei Pang, Zhongqiang Zhang, and George Em Karniadakis. Learning nonlinear operators via deeponet based on the universal approximation theorem of operators.Nature machineintelligence, 3(3):218–229, 2021

    2021

  69. [77]

    Superpixel-based image segmentation using convex optimization

    Aurélien Lucchi, Karel Smith, Radhakrishna Achanta, Pascal Fua, and Sabine Süsstrunk. Superpixel-based image segmentation using convex optimization. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 2089–2096. IEEE, 2011

  70. [78]

    Memory capacity of two layer neural networks with smooth activations

    Liam Madden and Christos Thrampoulidis. Memory capacity of two layer neural networks with smooth activations. SIAM Journal on Mathematics of Data Science, 6(3):679–702, 2024

    2024

  71. [79]

    Concentration Inequalities and Model Selection

    Pascal Massart. Concentration Inequalities and Model Selection. Springer, 2007

    2007

  72. [80]

    The generalization error of random features regression: Precise asymptotics and the double descent curve.Communications on Pure and Applied Mathematics, 75(4):667–766, 2022

    Song Mei and Andrea Montanari. The generalization error of random features regression: Precise asymptotics and the double descent curve.Communications on Pure and Applied Mathematics, 75(4):667–766, 2022

    2022

  73. [81]

    Understanding the dynamics of the frequency bias in neural networks, 2024

    Juan Molina, Mircea Petrache, Francisco Sahli Costabal, and Matías Courdurier. Understanding the dynamics of the frequency bias in neural networks, 2024

    2024

  74. [82]

    Sample compression schemes for vc classes.Journal of the ACM (JACM), 63(3):1–10, 2016

    Shay Moran and Amir Yehudayoff. Sample compression schemes for vc classes.Journal of the ACM (JACM), 63(3):1–10, 2016

    2016

  75. [83]

    Simplicity bias in 1-hidden layer neural networks

    Depen Morwani, Jatin Batra, Prateek Jain, and Praneeth Netrapalli. Simplicity bias in 1-hidden layer neural networks. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, editors,Advances in Neural Information Processing Systems, volume 36, pages 8048–8075. ...

    2023

  76. [84]

    Robust feature learning for multi-index models in high dimensions

    Alireza Mousavi-Hosseini, Adel Javanmard, and Murat A Erdogdu. Robust feature learning for multi-index models in high dimensions. InNeurIPS 2024 Workshop on Mathematics of Modern Machine Learning, 2024

    2024

  77. [85]

    Universal approximation property of random neural networks.arXiv preprint arXiv:2312.08410, 2023

    Ariel Neufeld and Philipp Schmocker. Universal approximation property of random neural networks.arXiv preprint arXiv:2312.08410, 2023

    2023 arXiv

  78. [86]

    A PAC-bayesian approach to spectrally- normalized margin bounds for neural networks

    Behnam Neyshabur, Srinadh Bhojanapalli, and Nathan Srebro. A PAC-bayesian approach to spectrally- normalized margin bounds for neural networks. InInternational Conference on Learning Representations, 2018

    2018

  79. [87]

    Fast samplers for inverse problems in iterative refinement models

    Kushagra Pandey, Ruihan Yang, and Stephan Mandt. Fast samplers for inverse problems in iterative refinement models. Advances in Neural Information Processing Systems, 37:26872–26914, 2024

    2024

  80. [88]

    Optimal approximation of piecewise smooth functions using deep relu neural networks.Neural Networks, 108:296–330, 2018

    Philipp Petersen and Felix Voigtlaender. Optimal approximation of piecewise smooth functions using deep relu neural networks.Neural Networks, 108:296–330, 2018

    2018

  81. [89]

    Mathematical theory of deep learning.arXiv preprint, 2024

    Philipp Petersen and Jakob Zech. Mathematical theory of deep learning.arXiv preprint, 2024

    2024

  82. [90]

    Lipschitz widths.ConstructiveApproximation, 57(2):759–805, 2023

    Guergana Petrova and Przemysław Wojtaszczyk. Lipschitz widths.ConstructiveApproximation, 57(2):759–805, 2023

    2023

  83. [91]

    n-widths in approximation theory, volume 7 of Ergebnisse der Mathematik und ihrer Grenzgebiete (3) [Results in Mathematics and Related Areas (3)]

    Allan Pinkus. n-widths in approximation theory, volume 7 of Ergebnisse der Mathematik und ihrer Grenzgebiete (3) [Results in Mathematics and Related Areas (3)]. Springer-Verlag, Berlin, 1985. 39

    1985

  84. [92]

    I-theory on depth vs width: hierarchical function composition

    Tomaso Poggio, Fabio Anselmi, and Lorenzo Rosasco. I-theory on depth vs width: hierarchical function composition. Technical report, Center for Brains, Minds and Machines (CBMM), 2015

    2015

  85. [93]

    Compositional sparsity of learnable functions.Bulletin of the American Mathematical Society, 61(3):438–456, 2024

    Tomaso Poggio and Maia Fraser. Compositional sparsity of learnable functions.Bulletin of the American Mathematical Society, 61(3):438–456, 2024

    2024

  86. [94]

    Random features for large-scale kernel machines

    Ali Rahimi and Benjamin Recht. Random features for large-scale kernel machines. InAdvances in Neural Information Processing Systems, volume 20, 2007

    2007

  87. [95]

    Learning a classification model for segmentation

    Xiaofeng Ren and Jitendra Malik. Learning a classification model for segmentation. InProceedings Ninth IEEE International Conference on Computer Vision, volume 1, pages 10–17. IEEE, 2003

    2003

  88. [96]

    Generating rectifiable measures through neural networks

    Erwin Riegler, Alex Bühler, Yang Pan, and Helmut Bölcskei. Generating rectifiable measures through neural networks. arXiv preprint arXiv:2412.05109, 2024

    2024 arXiv

  89. [97]

    Nonparametric regression via deep neural networks.The Annals of Statistics, 48(4):1875–1897, 2020

    Johannes Schmidt-Hieber. Nonparametric regression via deep neural networks.The Annals of Statistics, 48(4):1875–1897, 2020

    2020

  90. [98]

    Nonlocal techniques for the analysis of deep relu neural network approximations.arXiv preprint arXiv:2504.04847, 2025

    Cornelia Schneider, Mario Ullrich, and Jan Vybiral. Nonlocal techniques for the analysis of deep relu neural network approximations.arXiv preprint arXiv:2504.04847, 2025

    2025 arXiv

  91. [99]

    A multivariate Riesz basis of ReLU neural networks.Appl

    Cornelia Schneider and Jan Vybíral. A multivariate Riesz basis of ReLU neural networks.Appl. Comput. Harmon. Anal., 68:Paper No. 101605, 16, 2024

    2024

  92. [100]

    Pac-bayesian generalisation error bounds for gaussian process classification

    Matthias Seeger. Pac-bayesian generalisation error bounds for gaussian process classification. InInternational Conference on Machine Learning (ICML), pages 226–233, 2002

    2002

  93. [101]

    Optimal approximation rate of ReLU networks in terms of width and depth.J

    Zuowei Shen, Haizhao Yang, and Shijun Zhang. Optimal approximation rate of ReLU networks in terms of width and depth.J. Math. Pures Appl. (9), 157:101–135, 2022

    2022

  94. [102]

    Learning theory of distribution regression with neural networks

    Zhongjie Shi, Zhan Yu, and Ding-Xuan Zhou. Learning theory of distribution regression with neural networks. Constructive Approximation, pages 1–44, 2025

    2025

  95. [103]

    Adaptivity of deep reLU network for learning in besov and mixed smooth besov spaces: optimal rate and curse of dimensionality

    Taiji Suzuki. Adaptivity of deep reLU network for learning in besov and mixed smooth besov spaces: optimal rate and curse of dimensionality. InInternational Conference on Learning Representations, 2019

    2019

  96. [104]

    Trenberth, John T

    Kevin E. Trenberth, John T. Fasullo, and Theodore G. Shepherd. Attribution of climate extreme events. Nature Climate Change, 5:725–730, 2015. Perspective on framing attribution of climate extremes

    2015

  97. [105]

    Benign overfitting in ridge regression.Journal of Machine Learning Research, 24(123):1–76, 2023

    Alexander Tsigler and Peter L Bartlett. Benign overfitting in ridge regression.Journal of Machine Learning Research, 24(123):1–76, 2023

    2023

  98. [106]

    Optimal aggregation of classifiers in statistical learning.The Annals of Statistics, 32(1):135–166, 2004

    Alexander B Tsybakov. Optimal aggregation of classifiers in statistical learning.The Annals of Statistics, 32(1):135–166, 2004

    2004

  99. [107]

    A. W. van der Vaart and Jon A. Wellner.Weak convergence and empirical processes—with applications to statistics. Springer Series in Statistics. Springer, Cham, second edition, [2023]©2023

    2023

  100. [108]

    On the optimal memorization power of reLU neural networks

    Gal Vardi, Gilad Yehudai, and Ohad Shamir. On the optimal memorization power of reLU neural networks. In International Conference on Learning Representations, 2022

    2022

  101. [109]

    Cambridge university press, 2018

    Roman Vershynin.High-dimensional probability: An introduction with applications in data science, volume 47. Cambridge university press, 2018

    2018

  102. [110]

    Memory capacity of neural networks with threshold and rectified linear unit activations

    Roman Vershynin. Memory capacity of neural networks with threshold and rectified linear unit activations. SIAM Journal on Mathematics of Data Science, 2(4):1004–1033, 2020

    2020

  103. [111]

    A general framework for the practical disintegration of pac-bayesian bounds.Machine Learning, 113(2):519–604, 2024

    Paul Viallard, Pascal Germain, Amaury Habrard, and Emilie Morvant. A general framework for the practical disintegration of pac-bayesian bounds.Machine Learning, 113(2):519–604, 2024

    2024

  104. [112]

    A scalable generative model for dynamical system reconstruction from neuroimaging data.Advancesin Neural Information Processing Systems, 37:80328–80362, 2024

    Eric Volkmann, Alena Brändle, Daniel Durstewitz, and Georgia Koppe. A scalable generative model for dynamical system reconstruction from neuroimaging data.Advancesin Neural Information Processing Systems, 37:80328–80362, 2024. 40

    2024

  105. [113]

    Error bounds for approximations with deep relu networks.Neural networks, 94:103–114, 2017

    Dmitry Yarotsky. Error bounds for approximations with deep relu networks.Neural networks, 94:103–114, 2017

    2017

  106. [114]

    Optimal approximation of continuous functions by very deep relu networks

    Dmitry Yarotsky. Optimal approximation of continuous functions by very deep relu networks. InConference on learning theory, pages 639–649. PMLR, 2018

    2018

  107. [115]

    Elementary superexpressive activations

    Dmitry Yarotsky. Elementary superexpressive activations. InInternational conference on machine learning, pages 11932–11940. PMLR, 2021

    2021

  108. [116]

    The phase diagram of approximation rates for deep neural networks

    Dmitry Yarotsky and Anton Zhevnerchuk. The phase diagram of approximation rates for deep neural networks. Advances in neural information processing systems, 33:13005–13015, 2020

    2020

  109. [117]

    Deep network approximation: Beyond ReLU to diverse activation functions.Journal of Machine Learning Research, 25(35):1–39, 2024

    Shijun Zhang, Jianfeng Lu, and Hongkai Zhao. Deep network approximation: Beyond ReLU to diverse activation functions.Journal of Machine Learning Research, 25(35):1–39, 2024

    2024

  110. [118]

    Deep network approximation: Achieving arbitrary accuracy with fixed number of neurons.Journal of Machine Learning Research, 23(276):1–60, 2022

    Shijun Zhang, Zuowei Shen, and Haizhao Yang. Deep network approximation: Achieving arbitrary accuracy with fixed number of neurons.Journal of Machine Learning Research, 23(276):1–60, 2022. 41

    2022

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.