Pith. sign in

REVIEW 2 major objections 5 minor 83 references

Do Neural Networks Really Beat the Curse of Dimensionality? A Bit-Complexity View

T0 review · 2 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Measured in bits rather than parameters, neural networks do not beat classical approximation methods.

desk verdict Read this as a framework and case studies, not as a proof of the abstract's universal claim: the deep-net bit rate is a non-tight upper bound, so the paper needs a lower bound or a qualification before the main conclusion holds. read the letter →

arxiv 2608.01357 v1 pith:THZGDIAK submitted 2026-08-02 cs.LG

Tong Mao , Jinchao Xu This is my paper · ORCID
classification cs.LG MSC 41A4641A2565D1568T07
keywords bitcomplexitymetricentropycurseofdimensionalitydeepReLUnetworksapproximationratesfiniteprecisionBarronspacesuperconvergence
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that counting parameters overstates what neural networks achieve, because real computation stores parameters in finitely many bits. It builds a unified framework in which the minimum worst-case error achievable with $m$ bits is exactly the metric entropy of the target function class, then converts parameter-based rates for polynomials, finite elements, sparse grids, kernels, shallow networks, and deep ReLU networks into bit-based rates. The central finding is that deep ReLU superconvergence, $O(N^{-2r/d})$ on $H^r$ in parameter count, becomes only $O(N_b^{-2r/3d})$ in bits, while the metric-entropy lower bound is $N_b^{-r/d}$; the apparent advantage is eaten up by the extra precision needed. A sympathetic reader should care because this reframes the curse of dimensionality as a curse of bit complexity: if the paper is right, the real limit on approximation is the finite-precision budget, and many apparent neural-network gains are artifacts of comparing function classes with different complexity.

What carries the argument

The load-bearing object is metric entropy, $\epsilon_m(K)_X$, the smallest radius such that $K$ can be covered by $2^m$ balls in $X$. Proposition 2 identifies it exactly with the best worst-case approximation error attainable with an $m$-bit code, so entropy is bit-complexity, not merely a bound on it. The lower-bound engine is Lemma 2, which compares the entropy of the target class with the entropy of the approximating class to force a minimum approximation error. For deep ReLU networks the crucial quantitative input is Lemma 1, an $O((n+1)(W+1)^{n+1} M^{n+1})$ Lipschitz bound on the reconstruction map; because the weights in Theorem 10 are bounded by $M=2^{C_4 n}$, this bound grows exponentially in depth and forces a polynomial-in-$N$ number of bits per parameter, degrading the bit rate from $N^{-2r/d}$ to $N_b^{-2r/3d}$.

What would settle it

Compute the metric entropy of the deep ReLU class $\Sigma_{W,n}(M)$ from Theorem 10 with $M=2^{C_4 n}$ directly, or try to encode its parameters with a correlated, shared-weight, or recursive scheme; if either shows that $m$ bits can achieve error below the rate $N_b^{-2r/3d}$ on $H^r$, then Corollary 8's bound is not tight and the bit-based loss of superconvergence would be called into question. A simpler empirical version: train finite-precision deep ReLU networks on a known $H^r$ function and plot error against total bits; if the slope beats $N_b^{-r/d}$, the entropy lower bound or the experimental setup is the culprit.

Watch

Extended reading notes

Core claim

The paper's discovery is a re-evaluation of approximation rates under finite precision. Its formal claim is that when the cost of an approximant is measured in binary bits instead of parameter count, no method fundamentally exceeds the approximation order of classical approaches such as polynomials and finite elements. The mechanism is metric entropy: for any compact function class $K$, the minimal worst-case error achievable with $m$ bits equals the entropy number $\epsilon_m(K)_X$, so entropy acts as a universal lower bound. On Sobolev spaces $H^r(\Omega)$ the paper shows that deep ReLU networks, which reach the superconvergent parameter rate $N^{-2r/d}$ in Theorem 10, have a bit-based rate only $N_b^{-2r/3d}$ (Corollary 8), strictly worse than the entropy lower bound $N_b^{-r/d}$; therefore the superconvergence is a parameter-count artifact, not an architectural superiority. It also claims that dimension-independent rates for Barron and spectral Barron spaces reflect the small metric entropy of those target classes rather than a special power of the network.

Load-bearing premise

The argument's load-bearing premise is that a network's parameters must be treated as independent numbers, each stored with the same number of bits set by a worst-case sensitivity bound; if deep ReLU parameters can be encoded more compactly than that bound suggests, the claimed disappearance of superconvergence in bit cost would not hold.

Editorial extensions

If this is right

  • Classical methods—polynomials, finite elements, kernels, and sparse grids—keep their parameter-based rates up to logarithmic factors after quantization: roughly $(N_b/\log N_b)^{-r/d}$ for Sobolev classes.
  • Deep ReLU superconvergence does not transfer to bits: the same network that achieves $N^{-2r/d}$ in parameters needs so many bits per parameter that its bit rate drops to $N_b^{-2r/3d}$, slower than the $N_b^{-r/d}$ entropy bound.
  • Dimension-independent rates of shallow networks on Barron spaces should be read as statements about the small entropy of Barron classes, not about an architectural escape from the curse of dimensionality.
  • Metric entropy provides a universal benchmark: no method can beat the entropy rate of the target class, and methods whose binary encodings carry extra redundancy are provably suboptimal in bits.
  • In finite-precision practice, deep networks may need precision that grows with network size to deliver their theoretical parameter rates, so bit budgets, not parameter counts, determine which method wins.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • This suggests a testable engineering rule: for a fixed bit budget, compare methods on the same target class by measuring bits-to-error directly; if a deep network's empirical bit-rate falls below the entropy lower bound of the class, the bound or the experiment is wrong.
  • The paper's bit-rate for deep networks relies on independent per-parameter quantization; correlated encodings such as weight sharing or low-rank factorizations could in principle compress deep ReLU parameters below that cost, potentially reviving some superconvergence in bit terms—Remark 2 admits this open possibility.
  • The same entropy-based machinery could be applied to modern architectures such as transformers or tensor networks to check whether their advertised rates survive finite precision, a direction the paper does not pursue.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. This paper develops a bit-complexity framework for approximation theory, replacing parameter count by the number of bits required to encode approximants and linking it to metric entropy. The authors analyze classical methods (Legendre polynomials, finite elements, kernel interpolants, sparse grids) and neural networks (shallow ReLU^k, linearized shallow, spectral Barron, deep ReLU). For Sobolev classes they show bit-based rates of the form (N_b/log N_b)^{-r/d} for classical methods, against a metric-entropy lower bound of N_b^{-r/d}. For Barron spaces, shallow networks match the entropy bound up to logarithms. For deep ReLU networks on H^r, they use a Lipschitz bound (Lemma 1) to convert the parameter rate N^{-2r/d} into the bit rate N_b^{-2r/3d}, which is slower than the entropy lower bound. The paper concludes that no method fundamentally exceeds classical approximation order in bits, and that deep-network superconvergence is a parameter-count artifact offset by bit cost.

Significance. The paper's core framework is valuable: Proposition 2 cleanly identifies metric entropy with the optimal worst-case error under a fixed bit budget, and Proposition 3 provides a general conversion from parameter-based to bit-based rates. The bit-based analyses of classical methods are correct and reveal genuine log-factor gaps relative to entropy lower bounds. The treatment of Barron and spectral Barron spaces correctly attributes dimension-independent rates to the small metric entropy of the target classes. The main weakness is that the deep-network superconvergence discussion is supported only by a sufficient upper bound for one known construction, not by a lower bound on the bit complexity of the deep-network function class, so the paper's central explanatory claim about superconvergence is not fully established.

major comments (2)
  1. [§4.4, Corollary 8, Remark 2, §6] The conclusion that deep superconvergence is a parameter-based phenomenon offset by bit cost is not established by the results presented. Corollary 8 bounds the error of a specific uniform quantization of the [61] construction whose weights satisfy M = 2^{C_4 n}; this is an upper bound on the achievable error of one encoding scheme, not a lower bound on the bit complexity of the class Σ_{W,n}(M). The lower bound quoted in Corollary 8 is the metric entropy of the target space H^r, which applies to all methods and therefore cannot attribute any gap to the deep-network architecture. Because Remark 2 explicitly concedes that a refined construction with poly(n)-bounded weights would yield the near-optimal bit rate (N_b/log N_b)^{-r/d}, the paper's strong claim in Section 6 requires either a matching lower bound on ε_m(Σ_{W,n}(M)) or a rephrasing that limits the conclusion to the specific construction analyzed. This issue is load-bearing for the paper's central explanatory claim about superconvergence.
  2. [Abstract and Section 6] The universal claim that 'no method fundamentally exceeds the approximation order achieved by classical approaches' is stated as a conclusion of the bit-based analysis, but it is actually a direct consequence of the metric-entropy lower bounds (Proposition 2 together with Theorems 1 and 2), independent of the deep-network calculation. The deep-network upper bound in Corollary 8 is consistent with this claim but does not support it, since a faster bit rate for deep networks would not violate the entropy lower bound. The authors should state explicitly that the universal claim follows from the entropy lower bounds, and should present the deep-network analysis as a sufficient (not necessary) bit cost for one known construction rather than as the basis for the universal statement.
minor comments (5)
  1. [§5.1.1, Eq. (5.3)] The comparison of the bit-based lower bound (N log N)^{-r/d} with the parameter rate N^{-r/d} is ambiguous; the authors should state explicitly that the polynomial method with N parameters requires N_b ≃ N log N bits, so the demonstrated suboptimality is in the bit-based sense, not in the parameter-based sense.
  2. [§5.1.2] The same ambiguity appears in the Barron-space example, where the lower bound (N log N)^{−β} is compared to the parameter rate N^{−β}; please add the explicit bit-count conversion for clarity.
  3. [Introduction, Eq. (1.5), and Theorem 10] The introduction states a construction with width 25d+31, while Theorem 10 uses W = d3^{d+3}; please reconcile the constants or clarify that these refer to different constructions with the same rate.
  4. [Corollary 8, proof] The proof sets τ = C_5 N^2 without stating the dependencies of C_5; please note that C_5 depends on d, r, and W and that N_b = τ N, making the relation N_b ≃ N^3 explicit.
  5. [§5.2, item 1] The notation L2(T^d) appears where Ω is used elsewhere; please correct the domain in that item.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the bit-complexity framework is derived from definitions and external entropy/rate theorems; the deep-network bit-rate caveat in Remark 2 is a non-tight upper bound, not a circular step.

full rationale

The paper's core derivation is self-contained in the sense required by the circularity check. Proposition 2 proves the equivalence between metric entropy and the minimal m-bit encoding error directly from the definition of the dyadic entropy number; this is a mathematical identity, not a conclusion that presupposes the paper's main claims. The classical parameter rates in Theorems 3-6 are cited from standard approximation theory, and the bit-level conversions in Corollaries 1-4 follow from Proposition 3 using explicit Lipschitz bounds and explicit bit budgets. Those conversions are algebraically derived in the paper, not borrowed from the conclusions. For neural networks, the parameter rates in Theorems 7-10 come from cited published works, including several by the same research group (e.g., [66], [38], [40]); however, those cited results do not assume the present paper's conclusion that bit complexity offsets neural-network advantages, and they are independently stated approximation theorems with their own assumptions. Thus they count as genuine external inputs rather than circular self-citation. No fitted parameter is renamed as a prediction, no uniqueness theorem is imported from the authors' prior work, and no ansatz is smuggled in via citation. The main caveat is explicitly acknowledged in Remark 2: the bound M = 2^{C_4 n} is a sufficient condition and a refined deep-network construction could improve the bit rate. Corollary 8 therefore gives an upper bound for one encoding scheme of the [61] construction, not a lower bound on the bit complexity of the deep-network function class; the comparison with the metric-entropy lower bound for H^r does not by itself close the gap to N_b^{-r/d}. This is a logical or completeness limitation of the paper's strong conclusion, but it is not circularity: the derivation does not reduce to its own inputs by definition, and the paper itself flags the missing tightness. Accordingly, no circular step is identified, and the score reflects only the minor presence of same-group citations used as non-circular external inputs.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

No free parameters or invented entities. The analysis rests on known entropy and approximation results, plus a specific (and acknowledged as non-tight) Lipschitz bound for deep networks.

assumptions (3)
  • standard math Known metric entropy estimates for Sobolev spaces (Theorem 1) and Barron spaces (Theorem 2) are assumed from prior literature.
    Used to set the fundamental lower bounds in Corollaries 1-8.
  • domain assumption Known parameter-based approximation rates for polynomials, finite elements, kernels, sparse grids, and neural networks are imported from cited papers (Theorems 3-10).
    These rates are the inputs to Proposition 3, which converts them to bit-based rates.
  • domain assumption Parameters must be quantized independently to tau bits each, with no structured compression across parameters.
    If parameters could be encoded jointly or with side information, the bit rates might improve. Remark 2 partially acknowledges this for deep networks.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Do Neural Networks Really Beat the Curse of Dimensionality? A Bit-Complexity View." pith.science (2026). https://pith.science/paper/THZGDIAK

@misc{pith2026260801357,
  author       = {Pith},
  title        = {Pith review of: Do Neural Networks Really Beat the Curse of Dimensionality? A Bit-Complexity View},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/THZGDIAK}},
  note         = {Machine review of arXiv:2608.01357}
}
read the original abstract

Traditional approximation theory measures convergence rates in terms of the number of parameters or degrees of freedom. However, practical computation operates under finite precision: parameters must be encoded using a finite number of bits. Therefore, approximation efficiency should be evaluated in terms of computational bit complexity, which is intrinsically connected to the metric entropy of the underlying function class. In this work, we develop a unified approximation framework based on binary encoding and metric entropy. We analyze classical methods (including polynomial approximation, sparse grids, and finite elements) as well as shallow and deep neural networks, and compare their approximation rates for function classes with comparable metric entropy. We observe that, when evaluated in terms of bits, most classical methods are in general suboptimal relative to the intrinsic limits dictated by metric entropy, while neural network methods may exhibit different behaviors. We show that when complexity is measured in bits rather than parameters, no method fundamentally exceeds the approximation order achieved by classical approaches. Our results also indicate that many seeming advantages of neural networks, including dimension-independent rates and superconvergence phenomena, stem from differences in function class complexity rather than intrinsic architectural superiority. In this sense, the traditional curse of dimensionality can be misleading; the fundamental limitation is instead a curse of bit complexity, governed by metric entropy.

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

83 extracted references · 72 canonical work pages

  1. [61]

    Z. Shen, H. Yang, and S. Zhang, Optimal approximation rate of ReLU networks in terms of width and depth,Journal de Math´ ematiques Pures et Appliqu´ ees,157(2022), 101–135

    work page 2022

  2. [1]

    Abramowitz and I

    M. Abramowitz and I. A. Stegun,Handbook of Mathematical Functions: With Formulas, Graphs, and Mathematical Tables, vol. 55, US Government Printing Office, 1964

    1964

  3. [2]

    Arcang´ eli, M

    R. Arcang´ eli, M. C. L´ opez de Silanes, and J. J. Torrens, An extension of a bound for functions in Sobolev spaces, with applications to (m,s)-spline interpolation and smoothing,Numerische Mathematik,107(2) (2007), 181–211

    2007

  4. [3]

    V. I. Arnold, The representation of functions of several variables,Mat. Prosvesc.,3(1958), 41–61

    1958

  5. [4]

    A. R. Barron, Complexity regularization with application to artificial neural networks, in Nonparametric Functional Estimation and Related Topics, Springer, 1991, 561–576

    work page 1991

  6. [5]

    A. R. Barron, A. Cohen, W. Dahmen, and R. A. DeVore, Approximation and learning by greedy algorithms,Annals of Statistics,36(1) (2008), 64–94

    work page 2008

  7. [6]

    P. Bartlett, The sample complexity of pattern classification with neural networks: the size of the weights is more important than the size of the network,IEEE Transactions on Information Theory,44(2) (1998), 525–536

    work page 1998

  8. [7]

    P. L. Bartlett and S. Mendelson, Rademacher and Gaussian complexities: risk bounds and structural results,Journal of Machine Learning Research,3(2002), 463–482

    work page 2002

Show all 83 references
  1. [8]

    Bathe,Finite Element Procedures, Klaus-Jurgen Bathe, 2006

    K.-J. Bathe,Finite Element Procedures, Klaus-Jurgen Bathe, 2006

    2006

  2. [9]

    Bengio, Learning deep architectures for AI,Foundations and Trends in Machine Learning, 2(1) (2009), 1–127

    Y. Bengio, Learning deep architectures for AI,Foundations and Trends in Machine Learning, 2(1) (2009), 1–127

    2009

  3. [10]

    Bertoluzza, R

    S. Bertoluzza, R. H. Nochetto, A. Quarteroni, K. G. Siebert, and A. Veeser, Primer of adaptive finite element methods, inMultiscale and Adaptivity: Modeling, Numerics and Applications, CIME Summer School, Cetraro, Italy, 2009, Springer, 2012, 125–225. 29

    2009

  4. [11]

    S. C. Brenner,The Mathematical Theory of Finite Element Methods, Springer, 2008

    2008

  5. [12]

    Bungartz and M

    H.-J. Bungartz and M. Griebel, Sparse grids,Acta Numerica,13(2004), 147–269

    2004

  6. [13]

    D.-R. Chen, Q. Wu, Y. Ying, and D.-X. Zhou, Support vector machine soft margin classifiers: error analysis,Journal of Machine Learning Research,5(2004), 1143–1175

    2004

  7. [14]

    P. G. Ciarlet,The Finite Element Method for Elliptic Problems, SIAM, 2002

    2002

  8. [15]

    Cucker and S

    F. Cucker and S. Smale, On the mathematical foundations of learning,Bulletin of the American Mathematical Society,39(1) (2002), 1–49

    2002

  9. [16]

    Cucker and D

    F. Cucker and D. X. Zhou,Learning Theory: An Approximation Theory Viewpoint, vol. 24, Cambridge University Press, 2007

    2007

  10. [17]

    P. J. Davis,Interpolation and Approximation, Dover Publications, 1975

    1975

  11. [18]

    R. A. DeVore, R. Howard, and C. Micchelli, Optimal nonlinear approximation,Manuscripta Mathematica,63(1989), 469–478

    1989

  12. [19]

    R. A. DeVore and G. G. Lorentz,Constructive Approximation, vol. 303, Springer Science & Business Media, 1993

    1993

  13. [20]

    R. A. DeVore and V. N. Temlyakov, Some remarks on greedy algorithms,Advances in Com- putational Mathematics,5(1) (1996), 173–187

    1996

  14. [21]

    Eldan and O

    R. Eldan and O. Shamir, The power of depth for feedforward neural networks, inConference on Learning Theory, PMLR, 2016, 907–940

    2016

  15. [22]

    H. Feng, S. Huang, and D.-X. Zhou, Generalization analysis of CNNs for classification on spheres,IEEE Transactions on Neural Networks and Learning Systems, 2021

    2021

  16. [23]

    Harvey, C

    N. Harvey, C. Liaw, and A. Mehrabian, Nearly-tight VC-dimension bounds for piecewise linear neural networks, inConference on Learning Theory, PMLR, 2017, 1064–1068

    2017

  17. [24]

    J. He, T. Mao, and J. Xu, Expressivity and approximation properties of deep neural networks with ReLUk activation,arXiv preprint arXiv:2312.16483, 2023

    2023 arXiv

  18. [25]

    Y. Jiao, Y. Lai, X. Lu, and Z. Yang, Deep neural networks with ReLU-sine-exponential acti- vations break curse of dimensionality on H¨ older class,arXiv preprint arXiv:2103.00542, 2021

    2021 arXiv

  19. [26]

    Jin,The Finite Element Method in Electromagnetics, John Wiley & Sons, 2015

    J.-M. Jin,The Finite Element Method in Electromagnetics, John Wiley & Sons, 2015

    2015

  20. [27]

    L. K. Jones, A simple lemma on greedy approximation in Hilbert space and convergence rates for projection pursuit regression and neural network training,The Annals of Statistics,20(1) (1992), 608–613

    1992

  21. [28]

    Karvonen, G

    T. Karvonen, G. Santin, and T. Wenzel, General superconvergence for kernel-based approxi- mation,arXiv preprint arXiv:2505.11435, 2025

    2025 arXiv

  22. [29]

    J. M. Klusowski and A. R. Barron, Approximation by combinations of ReLU and squared ReLU ridge functions withℓ 1 andℓ 0 controls,IEEE Transactions on Information Theory, 64(12) (2018), 7649–7656. 30

    2018

  23. [30]

    A. N. Kolmogorov, On the representation of continuous functions of several variables as super- positions of functions of smaller number of variables,Soviet Math. Dokl.,108(1956), 179–182

    1956

  24. [31]

    A. N. Kolmogorov and V. A. Uspenskii, On the definition of an algorithm,Uspekhi Matem- aticheskikh Nauk,13(4) (1958), 3–28

    1958

  25. [32]

    Koltchinskii, Rademacher penalties and structural risk minimization,IEEE Transactions on Information Theory,47(5) (2001), 1902–1914

    V. Koltchinskii, Rademacher penalties and structural risk minimization,IEEE Transactions on Information Theory,47(5) (2001), 1902–1914

    2001

  26. [33]

    K˚ urkov´ a and M

    V. K˚ urkov´ a and M. Sanguineti, Bounds on rates of variable basis and neural network approx- imation,IEEE Transactions on Information Theory,47(6) (2001), 2659–2665

    2001

  27. [34]

    K˚ urkov´ a and M

    V. K˚ urkov´ a and M. Sanguineti, Comparison of worst case errors in linear and neural network approximation,IEEE Transactions on Information Theory,48(1) (2002), 264–275

    2002

  28. [35]

    S. R. Kulkarni et al., On metric entropy, Vapnik–Chervonenkis dimension, and learnability for a class of distributions, 1989

    1989

  29. [36]

    Lewicki and G

    G. Lewicki and G. Marino, Approximation of functions of finite variation by superpositions of a sigmoidal function,Applied Mathematics Letters,17(10) (2004), 1147–1152

    2004

  30. [37]

    Q. Lin, H. Xie, and J. Xu, Lower bounds of the discretization error for piecewise polynomials, Mathematics of Computation,83(285) (2014), 1–13

    2014

  31. [38]

    X. Liu, T. Mao, and J. Xu, Integral representations of Sobolev spaces via ReLU k activation function and optimal error estimates for linearized networks,arXiv preprint arXiv:2505.00351, 2025

    2025 arXiv

  32. [39]

    G. G. Lorentz, M. von Golitschek, and Y. Makovoz,Constructive Approximation: Advanced Problems, vol. 304, Springer, 1996

    1996

  33. [40]

    L. Ma, J. W. Siegel, and J. Xu, Uniform approximation rates and metric entropy of shallow neural networks,Research in the Mathematical Sciences,9(3) (2022), 46

    2022

  34. [41]

    Maiorov and A

    V. Maiorov and A. Pinkus, Lower bounds for approximation by MLP neural networks,Neuro- computing,25(1–3) (1999), 81–91

    1999

  35. [42]

    Mao and D.-X

    T. Mao and D.-X. Zhou, Approximation of functions from Korobov spaces by deep convolu- tional neural networks,Advances in Computational Mathematics,48(6) (2022), 84

    2022

  36. [43]

    H. N. Mhaskar, On the representation of smooth functions by radial basis functions,Journal of Approximation Theory,127(1) (2004), 1–16

    2004

  37. [44]

    H. N. Mhaskar, Eignets for function approximation on manifolds,Applied and Computational Harmonic Analysis,29(1) (2010), 63–87

    2010

  38. [45]

    H. N. Mhaskar, Kernel-based analysis of massive data,Frontiers in Applied Mathematics and Statistics,6(2020), 30

    2020

  39. [46]

    H. N. Mhaskar and C. A. Micchelli, Dimension-independent bounds on the degree of approxi- mation by neural networks,IBM Journal of Research and Development,38(3) (1994), 277–284. 31

    1994

  40. [47]

    H. N. Mhaskar, F. J. Narcowich, and J. D. Ward, Approximation properties of zonal function networks using scattered data on the sphere,Advances in Computational Mathematics,11 (1999), 121–137

    1999

  41. [48]

    Moaveni,Finite Element Analysis: Theory and Application with ANSYS, 3rd ed., Pearson Education India, 2011

    S. Moaveni,Finite Element Analysis: Theory and Application with ANSYS, 3rd ed., Pearson Education India, 2011

    2011

  42. [49]

    Mohri, A

    M. Mohri, A. Rostamizadeh, and A. Talwalkar,Foundations of Machine Learning, 2nd ed., MIT Press, 2018

    2018

  43. [50]

    Montanelli and Q

    H. Montanelli and Q. Du, New error bounds for deep ReLU networks using sparse grids,SIAM Journal on Mathematics of Data Science,1(1) (2019), 78–92

    2019

  44. [51]

    F. J. Narcowich, J. D. Ward, and H. Wendland, Sobolev bounds on functions with scattered zeros, with applications to radial basis function surface fitting,Mathematics of Computation, 74(250) (2005), 743–763

    2005

  45. [52]

    F. J. Narcowich, J. D. Ward, and H. Wendland, Sobolev error estimates and a Bernstein in- equality for scattered data interpolation via radial basis functions,Constructive Approximation, 24(2) (2006), 175–186

    2006

  46. [53]

    P. P. Petrushev, Approximation by ridge functions and neural networks,SIAM Journal on Mathematical Analysis,30(1) (1998), 155–189

    1998

  47. [54]

    Poggio, H

    T. Poggio, H. Mhaskar, L. Rosasco, B. Miranda, and Q. Liao, Why and when can deep-but not shallow-networks avoid the curse of dimensionality: a review,International Journal of Automation and Computing,14(5) (2017), 503–519

    2017

  48. [55]

    Reddy,An Introduction to the Finite Element Method, 1993

    J. Reddy,An Introduction to the Finite Element Method, 1993

    1993

  49. [56]

    Schaback, Improved error bounds for scattered data interpolation by radial basis functions, Mathematics of Computation,68(225) (1999), 201–216

    R. Schaback, Improved error bounds for scattered data interpolation by radial basis functions, Mathematics of Computation,68(225) (1999), 201–216

    1999

  50. [57]

    Schaback, Superconvergence of kernel-based interpolation,Journal of Approximation The- ory,235(2018), 1–19

    R. Schaback, Superconvergence of kernel-based interpolation,Journal of Approximation The- ory,235(2018), 1–19

    2018

  51. [58]

    Z. Shen, H. Yang, and S. Zhang, Deep network approximation characterized by number of neurons,arXiv preprint arXiv:1906.05497, 2019

    1906 arXiv

  52. [59]

    Z. Shen, H. Yang, and S. Zhang, Deep network with approximation error being reciprocal of width to power of square root of depth,arXiv preprint arXiv:2006.12231, 2020

    2006 arXiv

  53. [60]

    Z. Shen, H. Yang, and S. Zhang, Neural network approximation: three hidden layers are enough,Neural Networks,141(2021), 160–173

    2021

  54. [62]

    J. W. Siegel, Optimal approximation of zonoids and uniform approximation by shallow neural networks,arXiv preprint arXiv:2307.15285, 2023. 32

    2023 arXiv

  55. [63]

    J. W. Siegel, Optimal approximation rates for deep ReLU neural networks on Sobolev and Besov spaces,Journal of Machine Learning Research,24(357) (2023), 1–52

    2023

  56. [64]

    J. W. Siegel and J. Xu, Approximation rates for neural networks with general activation functions,Neural Networks,128(2020), 313–321

    2020

  57. [65]

    J. W. Siegel and J. Xu, High-order approximation rates for shallow neural networks with cosine and ReLUk activation functions,Applied and Computational Harmonic Analysis,58(2022), 1–26

    2022

  58. [66]

    J. W. Siegel and J. Xu, Sharp bounds on the approximation rates, metric entropy, andn-widths of shallow neural networks,Foundations of Computational Mathematics, 2022, 1–57

    2022

  59. [67]

    J. W. Siegel and J. Xu, Characterization of the variation spaces corresponding to shallow neural networks,Constructive Approximation,57(3) (2023), 1109–1132

    2023

  60. [68]

    Sloan and V

    I. Sloan and V. Kaarnioja, Doubling the rate: improved error bounds for orthogonal projection with application to interpolation,BIT Numerical Mathematics, 2025

    2025

  61. [69]

    Stoer, R

    J. Stoer, R. Bulirsch, R. Bartels, W. Gautschi, and C. Witzgall,Introduction to Numerical Analysis, Springer, 1980

    1980

  62. [70]

    S¨ uli and D

    E. S¨ uli and D. F. Mayers,An Introduction to Numerical Analysis, Cambridge University Press, 2003

    2003

  63. [71]

    Szeg˝ o,Orthogonal Polynomials, vol

    G. Szeg˝ o,Orthogonal Polynomials, vol. 23, American Mathematical Society, 1939

    1939

  64. [72]

    V. N. Temlyakov, Greedy approximation,Acta Numerica,17(2008), 235–409

    2008

  65. [73]

    A. F. Timan,Theory of Approximation of Functions of a Real Variable, Elsevier, 2014

    2014

  66. [74]

    V. N. Vapnik and A. Y. Chervonenkis, On the uniform convergence of relative frequencies of events to their probabilities, inMeasures of Complexity: Festschrift for Alexey Chervonenkis, Springer, 2015, 11–30

    2015

  67. [75]

    Wendland and C

    H. Wendland and C. Rieger, Approximate interpolation with applications to selecting smooth- ing parameters,Numerische Mathematik,101(4) (2005), 729–748

    2005

  68. [76]

    Wenzel, Sharp inverse statements for kernel approximation: superconvergence and satura- tion,arXiv preprint arXiv:2601.01808, 2026

    T. Wenzel, Sharp inverse statements for kernel approximation: superconvergence and satura- tion,arXiv preprint arXiv:2601.01808, 2026

    2026

  69. [77]

    Xu, Iterative methods by space decomposition and subspace correction,SIAM Review,34(4) (1992), 581–613

    J. Xu, Iterative methods by space decomposition and subspace correction,SIAM Review,34(4) (1992), 581–613

    1992

  70. [78]

    Xu, The finite neuron method and convergence analysis,arXiv preprint arXiv:2010.01458, 2020

    J. Xu, The finite neuron method and convergence analysis,arXiv preprint arXiv:2010.01458, 2020

    2010 arXiv

  71. [79]

    Y. Yang, Y. Wu, H. Yang, and Y. Xiang, Nearly optimal approximation rates for deep super ReLU networks on Sobolev spaces,arXiv preprint arXiv:2310.10766, 2023

    2023 arXiv

  72. [80]

    Yang and D.-X

    Y. Yang and D.-X. Zhou, Optimal rates of approximation by shallow ReLU k neural networks and applications to nonparametric regression,arXiv preprint arXiv:2304.01561, 2023. 33

    2023 arXiv

  73. [81]

    Yarotsky, Optimal approximation of continuous functions by very deep ReLU networks, in Conference on Learning Theory, PMLR, 2018, 639–649

    D. Yarotsky, Optimal approximation of continuous functions by very deep ReLU networks, in Conference on Learning Theory, PMLR, 2018, 639–649

    2018

  74. [82]

    Yarotsky and A

    D. Yarotsky and A. Zhevnerchuk, The phase diagram of approximation rates for deep neural networks,Advances in Neural Information Processing Systems,33(2020), 13005–13015

    2020

  75. [83]

    Zhang, Z

    S. Zhang, Z. Shen, and H. Yang, Deep network approximation: achieving arbitrary accuracy with fixed number of neurons,Journal of Machine Learning Research,23(276) (2022), 1–60. 34

    2022

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.