Pith. sign in

REVIEW 2 major objections 3 minor 35 references

On best approximation by multivariate ridge functions with applications to generalized translation networks

T0 review · 2 major / 3 minor · reviewed 2026-08-11 · deepseek-v4-flash

Pith's one-line read The paper establishes that n sums of ℓ-variate ridge functions approximate r-times-differentiable functions on the ball of R^d with sharp error n^{-r/(d-ℓ)}, and transfers this order to generalized translation and complex-valued networks.

desk verdict Sharp multivariate ridge rates with an honest fix of Maiorov's gap; the only real risk is an imported Cesaro-boundedness theorem at the core of the lower bound. read the letter →

arxiv 2412.08453 v3 pith:MBYRTCKA submitted 2024-12-11 math.FA cs.LGstat.ML

classification math.FAcs.LGstat.ML MSC 41A3041A2541A6346E3568T07
keywords multivariateridgefunctionsSobolevoptimalapproximationratesgeneralizedtranslationnetworkscomplex-valuedneuralquasi-projectionoperatorssignpatterncountingsharpbounds
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proves a sharp approximation theorem: any L^p-Sobolev function of regularity r on the unit ball of R^d is approximated, up to error of order $n^{{-r/(d-ℓ)}}$, by a sum of n ridge functions of ℓ variables, and no such sum can do asymptotically better. The lower bound holds even in the hardest regime, approximating L^∞-Sobolev targets with error measured in $L^{1}$, while the upper bound is achieved using only polynomial ridge functions with fixed matrices. This settles the multivariate generalization of the classical univariate ridge-function results. As consequences, generalized translation networks with activation dimension ℓ attain exactly $n^{{-r/(d-ℓ)}}$, improving as ℓ grows, and a constructed smooth complex activation function lets complex-valued shallow networks reach $n^{{-r/(2d-2)}}$, better than the real-network benchmark $n^{{-r/(2d-1)}}$.

What carries the argument

The argument rides on a quasi-projection operator Pr_s: $L^{1}$(B_d) → P_{2s-1}(B_d) that fixes polynomials of degree up to s and, unlike the orthogonal projection, is uniformly bounded in $L^{1}$ for all s (Proposition 2.7). This operator transfers the lower bound from the projected ridge class Pr_s(R*_{n,d,ℓ}) to the actual class R*_{n,d,ℓ} through Lemma 3.8. The lower bound itself is built from sign-set counting: after separating variables in the inner product ⟨ρ(A·), P⟩ (Lemma 3.5), the sign patterns arising from projected ridge functions are few enough (Lemma 3.1) that a standard volume argument (Lemma 3.2) finds a Sobolev bump function far from every such pattern.

What would settle it

Compute sup_s ||Pr_s||_{$L^{1}$(B_d)→$L^{1}$(B_d)} for the quasi-projection of Section 2.3: if the norms are unbounded in s, the transfer lemma (Lemma 3.8) fails and with it the lower bound; conversely, an explicit Sobolev function of regularity r approximated by n ℓ-variate ridge sums at error much smaller than $n^{{-r/(d-ℓ)}}$ would refute Theorem 1.1 directly.

Watch

Extended reading notes

Core claim

The central discovery is that the approximation order of Sobolev functions by sums of ℓ-variate ridge functions is governed only by the co-dimension d−ℓ of the ridge subspace: an asymptotic error of order $n^{{-r/(d-ℓ)}}$ is simultaneously necessary and sufficient. The lower bound holds for every p,q ∈ [1,∞], the matching upper bound for 1 ≤ q ≤ p ≤ ∞, and the upper bound is attainable with polynomial ridge functions and a fixed choice of matrices. The proof also closes a gap in the univariate case by replacing the orthogonal projection onto the polynomials of degree at most s, which is not uniformly bounded in $L^{1}$, with a quasi-projection onto P_{2s-1}(B_d) that is uniformly $L^{1}$-bounded. The same rate then transfers to generalized translation networks and, through the Wirtinger calculus, to complex-valued neural networks.

Load-bearing premise

The lower bound rests on the quasi-projection operators being uniformly bounded in the $L^{1}$ norm for every polynomial degree, a fact whose proof invokes a Cesàro-mean theorem from a cited textbook.

Editorial extensions

If this is right

  • Generalized translation networks with activation dimension ℓ approximate L^p-Sobolev functions of regularity r at the sharp rate n^{-r/(d-ℓ)}, so raising ℓ strictly improves the achievable rate (Theorem 1.3).
  • For each ℓ there is a single smooth activation function τ: R^ℓ → R for which the optimal rate is achieved with fixed weight matrices, whatever d, p, q and r (Theorem 1.3(2)).
  • Shallow complex-valued networks attain n^{-r/(2d-2)}, beating the real benchmark n^{-r/(2d-1)} obtained by identifying C^d with R^{2d} (Theorem 1.4).
  • The L^1-norm lower bound repairs the gap in the earlier univariate proof by replacing orthogonal projection with a uniformly L^1-bounded quasi-projection (Appendix A).
  • The upper bound needs no smoothness beyond a Jackson-type polynomial approximation inequality, so the same rate holds for any function class satisfying Proposition 2.5 (Remark 4.3).

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The exponent −r/(d−ℓ) suggests a co-dimension law: the approximation difficulty in R^d is set by the dimension of the space orthogonal to the ridge directions, and one could test whether other structured model classes (tensor trains, subspace networks, dictionary models) obey an analogous law with their own intrinsic co-dimension.
  • Because only the Jackson inequality enters the upper bound, the n^{-r/(d-ℓ)} rate plausibly extends to Besov or other smoothness scales with the same regularity parameter r, a direct extrapolation of Remark 4.3.
  • The two-step lower-bound scheme (cardinality bound for sign patterns, then a volume argument on separated cubes) is transferable and should yield sharp rates for other constrained dictionaries whose projected parameter sets have small dimension.
  • One could empirically test the constructed piecewise activation for small dimensions (for instance d=3, ℓ=2, r=2, where the predicted rate is n^{-2}): a materially faster decay would point to a hidden structural assumption, while a slower decay would suggest the constant prefactor is large.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 3 minor

Summary. The paper studies approximation of Sobolev functions on the unit ball B_d by sums of n multivariate ridge functions x ↦ ρ(Ax) with A ∈ R^{ℓ×d}. The main results (Theorems 1.1 and 1.2) establish matching lower and upper bounds of order n^{-r/(d-ℓ)} for the worst-case L^q error over the unit ball of W^{r,p}(B_d), under the assumptions that the lower bound holds for all p,q∈[1,∞] and the upper bound for 1≤q≤p≤∞. The lower-bound proof uses a volumetric/sign-pattern argument together with a family of quasi-projection operators P_r^s that are uniformly bounded on L^1, and the upper bound is obtained by expressing polynomials as sums of ridge polynomials with fixed matrices. The paper also applies these results to generalized translation networks and complex-valued neural networks, obtaining optimal rates n^{-r/(d-ℓ)} and n^{-r/(2d-2)}, and it identifies and repairs a gap in the earlier proof for univariate ridge functions in [18].

Significance. If the external hypotheses on which the proof depends are verified, this paper solves the open problem of sharp approximation rates for multivariate ridge functions and provides the first optimal rates for shallow generalized translation networks and complex-valued neural networks with general activation functions. The paper is careful and detailed, with explicit constants, and its replacement of the orthogonal projection in [18] by uniformly bounded quasi-projections is a genuinely new methodological step that also corrects a known gap. The results are sharp and falsifiable, and the applications to neural networks are concrete. The principal caveat is that the central lower bound rests on an imported uniform-L^1 boundedness estimate for Cesàro means, whose hypotheses are not fully checked in the manuscript.

major comments (2)
  1. [Section 2.3, Proposition 2.7(3) and Lemma 3.8] The uniform L^1 bound for the quasi-projections P_r^s is the linchpin of the lower bound. The proof of Proposition 2.7(3) derives the estimate ∥P_r^s f∥_{L^1} ≤ C∥f∥_{L^1} from the uniform-in-k bound ∥S^σ_k(f)∥_{L^1} ≤ C_3∥f∥_{L^1}, which is imported from [4, Theorem 11.4.1] with the parameter choice κ=(0,…,0,1/2)∈R^{d+1} and σ>d/2. The manuscript does not state the hypotheses of that theorem, and it explicitly notes that the corresponding proposition is stated without proof in [4]. Since Lemma 3.8 transfers the lower bound from P_r^s(R*_{n,d,ℓ}) to R*_{n,d,ℓ}, the whole lower bound of Theorem 1.1 collapses if this estimate fails (for instance, if the theorem requires σ>d/2+κ_{d+1} or excludes p=1). The authors should either provide a self-contained proof of the uniform L^1 bound or quote [4, Theorem 11.4.1] in full and verify that the chosen parameters satisfy all of its assumptions.
  2. [Section 3.2, Lemma 3.1 and Lemma 3.3] The sign-set cardinality bound in Lemma 3.1, which is essential for the entropy lower bound, relies on Lemma 3.3, imported verbatim from [17, Lemma 3] without proof. Similarly, the upper bound in Section 4 depends on [26, Corollary 5.12]. These are external results at load-bearing steps of the argument. Given that the paper's contribution is precisely a careful and self-contained repair of a gap in [18], the reader needs to be able to verify that these external lemmas apply. The authors should either include proofs of these lemmas or state them with all hypotheses and exact reference locations, so that no hidden condition (e.g., on the degree bounds or on N+K) is left unchecked.
minor comments (3)
  1. [Section 3.3, Proposition 3.7] In the displayed chain of inequalities after equation (3.31), the factor should be (2ϑ)^{-r}, not (2ϑ)^r. The subsequent conclusion ∥f_{ε*}−P∥_{L^1} ≥ a·c_2·c_3·2^{-r}·m^{-r/d} is consistent with the negative exponent, so this appears to be a typo rather than a mathematical error.
  2. [Section 2.3, proof of Proposition 2.7(3)] The constant C(d) is written as C = C_1C_2C_3·2^{σ+1}, but C_3 depends on d and σ, and σ is chosen depending on d; this is fine, but the notation could be made clearer by writing C = C(d,σ) and then noting that σ is fixed by d.
  3. [Appendix A] The equation numbering in the discussion of the gap in [18] is slightly confusing: the manuscript refers to 'Equation (14)' of [18] but then labels its own displayed equation as (A.4). This could be clarified by explicitly saying that (A.4) is the corresponding statement in the notation of the present paper.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the sharp ridge-function bounds are derived from external harmonic-analysis and polynomial-representation results, with no fitted constant or self-citation chain determining the target rates.

full rationale

The central claims (Theorems 1.1 and 1.2) are not circular. The lower bound in Section 3 is obtained by a counting argument on sign patterns (Lemma 3.1, built on Lemmas 3.3 and 3.5) combined with the quasi-projection transfer in Lemma 3.8. The uniform L1 bound needed there, Proposition 2.7(3), is imported from the external monograph [4, Theorem 11.4.1] after the paper makes the explicit parameter choice kappa=(0,...,0,1/2) and sigma>d/2; it is not a self-citation, it is not fitted to the target rate, and the proof does not define the quasi-projection in terms of the lower bound. Whether [4, Theorem 11.4.1] really applies at this parameter boundary is a correctness question, not a circularity question. The upper bound in Section 4 uses the external Jackson estimate Proposition 2.5 (from [21]) and the polynomial ridge representation from [26]; these assumptions do not include n^{-r/(d-ell)}. The GTN and CVNN applications are deductions from Theorems 1.1 and 1.2 plus two standalone constructions. The re-used activation function from [9, Lemma F.4] is a parameter-free construction whose statement contains no approximation rate, and the dimension bound from [9, Lemma F.1] is an elementary counting estimate. Neither source imports the conclusion being proved. No equation in the derivation is equal by construction to the target rate, and no parameter is fitted to a subset of the data and then renamed as a prediction.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

All constants in the proofs are existential and are not fitted to data; there are no free parameters in the statements of the theorems. The central claim rests on standard approximation-theoretic tools (Jackson bounds, Cesaro means, polynomial ridge representations) and on two combinatorial and analytic lemmas cited from the literature. The construction of the complex activation function is taken from the authors' prior paper [9] but is not the target result. No new physical or mathematical entities are postulated.

assumptions (5)
  • standard math Jackson-type bound: for every f in B(W^{r,p}_d) and s in N there exists P in P_s(B_d) with ||f-P||_{L^q} <= C s^{-r} for 1 <= q <= p <= infty.
    Stated as Proposition 2.5, cited from Mhaskar [21, Equation (2.10)]; used in both upper (Theorem 4.2) and lower (Theorem 3.9) bounds.
  • standard math Uniform L^1 boundedness of the Cesaro means S^sigma_k on L^1(B_d), with a constant independent of k, for sigma > d/2.
    Invoked in the proof of Proposition 2.7(3) via [4, Theorem 11.4.1] with kappa = (0,...,0,1/2); this is the key tool behind the quasi-projection bound and is unproved in the paper.
  • standard math Sign-pattern counting bound: the number of sign patterns of m polynomials of degree at most s in N variables with K linear combinations is at most (4s)^N (N+K+1)^{N+2} (2em/(N+K))^{N+K}.
    Stated as Lemma 3.3, cited from Maiorov [17, Lemma 3]; used to bound the sign set of ridge-function evaluations in Lemma 3.1.
  • standard math Polynomial ridge representation: if P_h^s(R^{d-ell+1}) is spanned by (a_i^T x)^s for i = 1..n, then P_s(R^{d-ell+1}) is spanned by (a_i^T x)^k for 0 <= k <= s, with a_i fixed.
    Used in Proposition 4.1 via Pinkus [26, Proposition 5.9 and Corollary 5.12] to realize the Jackson polynomial as a sum of n polynomial ridge functions.
  • standard math Existence of the complex activation function phi from [9, Lemma F.4] whose shifts reproduce a dense set of polynomials in z and zbar on B_1(C).
    Used in Theorem 5.9 for the CVNN upper bound; the construction is from a prior paper by two of the present authors, but is a standalone technical lemma, not the target rate result.

how reviews work

0 comments
Cite this review

Pith. "Pith review of On best approximation by multivariate ridge functions with applications to generalized translation networks." pith.science (2026). https://pith.science/paper/MBYRTCKA

@misc{pith2026241208453,
  author       = {Pith},
  title        = {Pith review of: On best approximation by multivariate ridge functions with applications to generalized translation networks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MBYRTCKA}},
  note         = {Machine review of arXiv:2412.08453}
}
abstract

In this paper, we prove sharp upper and lower bounds for the approximation of Sobolev functions by sums of multivariate ridge functions, i.e., for approximation by functions of the form $\mathbb{R}^d \ni x \mapsto \sum_{k=1}^n \varrho_k(A_k x) \in \mathbb{R}$ with $\varrho_k : \mathbb{R}^\ell \to \mathbb{R}$ and $A_k \in \mathbb{R}^{\ell \times d}$. We show that the order of approximation asymptotically behaves as $n^{-r/(d-\ell)}$, where $r$ is the regularity (order of differentiability) of the Sobolev functions to be approximated. Our lower bound even holds when approximating $L^\infty$-Sobolev functions of regularity $r$ with error measured in $L^1$, while our upper bound applies to the approximation of $L^p$-Sobolev functions in $L^p$ for any $1 \leq p \leq \infty$. These bounds generalize well-known results regarding the approximation properties of univariate ridge functions to the multivariate case. We use our results to obtain sharp asymptotic bounds for the approximation of Sobolev functions using generalized translation networks and complex-valued neural networks.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

35 extracted references · 32 canonical work pages

  1. [4]

    Dai and Y

    F. Dai and Y. Xu.Approximation Theory and Harmonic Analysis on Spheres and Balls. Springer Mono- graphs in Mathematics. Springer New York, 2013.isbn: 9781461466604

  2. [18]

    Best approximation by ridge functions inLp-spaces

    V. Maiorov. “Best approximation by ridge functions inLp-spaces.” In:Ukrainian Mathematical Journal 62 (2010), pp. 452–466

  3. [1]

    Nearly-tight VC-dimension and pseudodimension bounds for piecewise linear neural networks

    P. L. Bartlett, N. Harvey, C. Liaw, and A. Mehrabian. “Nearly-tight VC-dimension and pseudodimension bounds for piecewise linear neural networks.” In:Journal of Machine Learning Research20.63 (2019), pp. 1–17

  4. [2]

    Bassey, L

    J. Bassey, L. Qian, and X. Li.A Survey of Complex-Valued Neural Networks. 2021. arXiv:2101.12249

  5. [3]

    Ridgelets: estimating with ridge functions

    E. J. Candès. “Ridgelets: estimating with ridge functions.” In: The Annals of Statistics 31.5 (2003), pp. 1561–1599

  6. [5]

    L. C. Evans and R. F. Gariepy. Measure theory and fine properties of functions. Studies in advanced mathematics. Boca Raton: CRC Press, 1992. 268 pp.isbn: 978-0-8493-7157-8

  7. [6]

    G. Folland. Real Analysis: Modern Techniques and Their Applications. Wiley, 1984.isbn: 9780471809586

  8. [7]

    Robust and resource efficient identification of shallow neural networks by fewest samples

    M. Fornasier, J. Vybíral, and I. Daubechies. “Robust and resource efficient identification of shallow neural networks by fewest samples.” In:Information and Inference: A Journal of the IMA10.2 (Jan. 2021), pp. 625–695.doi: 10.1093/imaiai/iaaa036

Show all 35 references
  1. [8]

    Geuchen, T

    P. Geuchen, T. Jahn, and H. Matt. Universal approximation with complex-valued deep narrow neural networks. 2024. arXiv:2305.16910. Accepted for publication inConstructive Approximation

  2. [9]

    Optimal approximation using complex-valued neural networks

    P. Geuchen and F. Voigtlaender. “Optimal approximation using complex-valued neural networks.” In: Advances in Neural Information Processing Systems36 (2024)

  3. [10]

    On the best approximation by ridge functions in the uniform norm

    Y. Gordon, V. Maiorov, M. Meyer, and S. Reisner. “On the best approximation by ridge functions in the uniform norm.” In:Constructive approximation18 (2001), pp. 61–85

  4. [11]

    Ismailov

    V. Ismailov. Ridge Functions and Applications in Neural Networks. Mathematical Surveys and Mono- graphs. American Mathematical Society, 2021.isbn: 9781470467654

  5. [12]

    A review of some results on ridge function approximation

    V. E. Ismailov. “A review of some results on ridge function approximation.” In: Azerb. J. Math 3.1 (2013), pp. 3–51

  6. [13]

    Kaup and B

    L. Kaup and B. Kaup.Holomorphic Functions of Several Variables: An Introduction to the Fundamental Theory. De Gruyter, Dec. 1983.isbn: 978-3-11-004150-7.doi: 10.1515/9783110838350

  7. [14]

    Complex-valued neural networks: A comprehensive survey

    C. Lee, H. Hasegawa, and S. Gao. “Complex-valued neural networks: A comprehensive survey.” In: IEEE/CAA Journal of Automatica Sinica9.8 (2022), pp. 1406–1426

  8. [15]

    Fundamentality of ridge functions

    V. Y. Lin and A. Pinkus. “Fundamentality of ridge functions.” In:Journal of Approximation Theory75.3 (1993), pp. 295–311

  9. [16]

    Luenberger

    D. Luenberger. Optimization by Vector Space Methods. Professional Series. Wiley, 1997.isbn: 978-0- 4711-8117-0

  10. [17]

    On Best Approximation by Ridge Functions

    V. Maiorov. “On Best Approximation by Ridge Functions.” In:Journal of Approximation Theory99.1 (July 1999), pp. 68–94.doi: 10.1006/jath.1998.3304

  11. [19]

    Lower bounds for approximation by MLP neural networks

    V. Maiorov and A. Pinkus. “Lower bounds for approximation by MLP neural networks.” In:Neurocom- puting 25.1-3 (1999), pp. 81–91

  12. [20]

    Onachoiceofsamplingnodesforoptimalapproximationofsmoothfunctions by generalized translation networks

    H.MhaskarandJ.Prestin.“Onachoiceofsamplingnodesforoptimalapproximationofsmoothfunctions by generalized translation networks.” In:Fifth International Conference on Artificial Neural Networks (Conf. Publ. No. 440). 1997, pp. 210–215.doi: 10.1049/cp:19970728

  13. [21]

    Neural networks for optimal approximation of smooth and analytic functions

    H. N. Mhaskar. “Neural networks for optimal approximation of smooth and analytic functions.” In: Neural computation8.1 (1996), pp. 164–177

  14. [22]

    Degree of approximation by neural and translation networks with a single hidden layer

    H. N. Mhaskar and C. A. Micchelli. “Degree of approximation by neural and translation networks with a single hidden layer.” In:Advances in applied mathematics16.2 (1995), pp. 151–183

  15. [23]

    Easy proof of the Jacobian for then-dimensional polar coordinates

    A. Muleshkov and T. Nguyen. “Easy proof of the Jacobian for then-dimensional polar coordinates.” In: Pi Mu Epsilon Journal14.4 (2016), pp. 269–273

  16. [24]

    Optimal approximation of piecewise smooth functions using deep ReLU neural networks

    P. Petersen and F. Voigtlaender. “Optimal approximation of piecewise smooth functions using deep ReLU neural networks.” In:Neural Networks108 (2018), pp. 296–330

  17. [25]

    Approximating by ridge functions

    A. Pinkus. “Approximating by ridge functions.” In:Surface fitting and multiresolution methods(1997), pp. 279–292

  18. [26]

    A. Pinkus. Ridge functions. Cambridge: Cambridge University Press, 2016.isbn: 978-1-107-12439-4

  19. [27]

    Prudnikov, Y

    A. Prudnikov, Y. Brychkov, and O. Marichev.Integrals and Series: Volume 1: Elementary Functions. Taylor & Francis, 1986.isbn: 9782881240973. 48 REFERENCES

  20. [28]

    Depth-width tradeoffs in approximating natural functions with neural net- works

    I. Safran and O. Shamir. “Depth-width tradeoffs in approximating natural functions with neural net- works.” In:International conference on machine learning. PMLR. 2017, pp. 2979–2987

  21. [29]

    Deep Network Approximation Characterized by Number of Neurons

    Z. Shen, H. Yang, and S. Zhang. “Deep Network Approximation Characterized by Number of Neurons.” In: Communications in Computational Physics28.5 (2020), pp. 1768–1811.doi: https://doi.org/10. 4208/cicp.OA-2020-0149

  22. [30]

    Optimal approximation rate of ReLU networks in terms of width and depth

    Z. Shen, H. Yang, and S. Zhang. “Optimal approximation rate of ReLU networks in terms of width and depth.” In:Journal de Mathématiques Pures et Appliquées157 (2022), pp. 101–135.doi: https: //doi.org/10.1016/j.matpur.2021.07.009

  23. [31]

    Benefits of depth in neural networks

    M. Telgarsky. “Benefits of depth in neural networks.” In:Conference on learning theory. PMLR. 2016, pp. 1517–1539

  24. [32]

    Vershynin

    R. Vershynin. High-dimensional probability: an introduction with applications in data science. Cambridge University Press, 2018. 284 pp.isbn: 978-1-108-41519-4

  25. [33]

    The universal approximation theorem for complex-valued neural networks

    F. Voigtlaender. “The universal approximation theorem for complex-valued neural networks.” In:Applied and computational harmonic analysis64 (2023), pp. 33–61

  26. [34]

    The phase diagram of approximation rates for deep neural networks

    D. Yarotsky and A. Zhevnerchuk. “The phase diagram of approximation rates for deep neural networks.” In: Advances in neural information processing systems33 (2020), pp. 13005–13015

  27. [35]

    Deep network approximation: Achieving arbitrary accuracy with fixed number of neurons

    S. Zhang, Z. Shen, and H. Yang. “Deep network approximation: Achieving arbitrary accuracy with fixed number of neurons.” In:Journal of Machine Learning Research23.276 (2022), pp. 1–60. (P. Geuchen)Mathematical Institute for Machine Learning and Data Science (MIDS), Catholic Un...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.