REVIEW 2 major objections 3 minor 35 references
On best approximation by multivariate ridge functions with applications to generalized translation networks
T0 review · 2 major / 3 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read The paper establishes that n sums of ℓ-variate ridge functions approximate r-times-differentiable functions on the ball of R^d with sharp error n^{-r/(d-ℓ)}, and transfers this order to generalized translation and complex-valued networks.
desk verdict Sharp multivariate ridge rates with an honest fix of Maiorov's gap; the only real risk is an imported Cesaro-boundedness theorem at the core of the lower bound. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The argument rides on a quasi-projection operator Pr_s: $L^{1}$(B_d) → P_{2s-1}(B_d) that fixes polynomials of degree up to s and, unlike the orthogonal projection, is uniformly bounded in $L^{1}$ for all s (Proposition 2.7). This operator transfers the lower bound from the projected ridge class Pr_s(R*_{n,d,ℓ}) to the actual class R*_{n,d,ℓ} through Lemma 3.8. The lower bound itself is built from sign-set counting: after separating variables in the inner product ⟨ρ(A·), P⟩ (Lemma 3.5), the sign patterns arising from projected ridge functions are few enough (Lemma 3.1) that a standard volume argument (Lemma 3.2) finds a Sobolev bump function far from every such pattern.
What would settle it
Compute sup_s ||Pr_s||_{$L^{1}$(B_d)→$L^{1}$(B_d)} for the quasi-projection of Section 2.3: if the norms are unbounded in s, the transfer lemma (Lemma 3.8) fails and with it the lower bound; conversely, an explicit Sobolev function of regularity r approximated by n ℓ-variate ridge sums at error much smaller than $n^{{-r/(d-ℓ)}}$ would refute Theorem 1.1 directly.
Extended reading notes
Core claim
The central discovery is that the approximation order of Sobolev functions by sums of ℓ-variate ridge functions is governed only by the co-dimension d−ℓ of the ridge subspace: an asymptotic error of order $n^{{-r/(d-ℓ)}}$ is simultaneously necessary and sufficient. The lower bound holds for every p,q ∈ [1,∞], the matching upper bound for 1 ≤ q ≤ p ≤ ∞, and the upper bound is attainable with polynomial ridge functions and a fixed choice of matrices. The proof also closes a gap in the univariate case by replacing the orthogonal projection onto the polynomials of degree at most s, which is not uniformly bounded in $L^{1}$, with a quasi-projection onto P_{2s-1}(B_d) that is uniformly $L^{1}$-bounded. The same rate then transfers to generalized translation networks and, through the Wirtinger calculus, to complex-valued neural networks.
Load-bearing premise
The lower bound rests on the quasi-projection operators being uniformly bounded in the $L^{1}$ norm for every polynomial degree, a fact whose proof invokes a Cesàro-mean theorem from a cited textbook.
Editorial extensions
If this is right
- Generalized translation networks with activation dimension ℓ approximate L^p-Sobolev functions of regularity r at the sharp rate n^{-r/(d-ℓ)}, so raising ℓ strictly improves the achievable rate (Theorem 1.3).
- For each ℓ there is a single smooth activation function τ: R^ℓ → R for which the optimal rate is achieved with fixed weight matrices, whatever d, p, q and r (Theorem 1.3(2)).
- Shallow complex-valued networks attain n^{-r/(2d-2)}, beating the real benchmark n^{-r/(2d-1)} obtained by identifying C^d with R^{2d} (Theorem 1.4).
- The L^1-norm lower bound repairs the gap in the earlier univariate proof by replacing orthogonal projection with a uniformly L^1-bounded quasi-projection (Appendix A).
- The upper bound needs no smoothness beyond a Jackson-type polynomial approximation inequality, so the same rate holds for any function class satisfying Proposition 2.5 (Remark 4.3).
Reading between the lines
- The exponent −r/(d−ℓ) suggests a co-dimension law: the approximation difficulty in R^d is set by the dimension of the space orthogonal to the ridge directions, and one could test whether other structured model classes (tensor trains, subspace networks, dictionary models) obey an analogous law with their own intrinsic co-dimension.
- Because only the Jackson inequality enters the upper bound, the n^{-r/(d-ℓ)} rate plausibly extends to Besov or other smoothness scales with the same regularity parameter r, a direct extrapolation of Remark 4.3.
- The two-step lower-bound scheme (cardinality bound for sign patterns, then a volume argument on separated cubes) is transferable and should yield sharp rates for other constrained dictionaries whose projected parameter sets have small dimension.
- One could empirically test the constructed piecewise activation for small dimensions (for instance d=3, ℓ=2, r=2, where the predicted rate is n^{-2}): a materially faster decay would point to a hidden structural assumption, while a slower decay would suggest the constant prefactor is large.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper studies approximation of Sobolev functions on the unit ball B_d by sums of n multivariate ridge functions x ↦ ρ(Ax) with A ∈ R^{ℓ×d}. The main results (Theorems 1.1 and 1.2) establish matching lower and upper bounds of order n^{-r/(d-ℓ)} for the worst-case L^q error over the unit ball of W^{r,p}(B_d), under the assumptions that the lower bound holds for all p,q∈[1,∞] and the upper bound for 1≤q≤p≤∞. The lower-bound proof uses a volumetric/sign-pattern argument together with a family of quasi-projection operators P_r^s that are uniformly bounded on L^1, and the upper bound is obtained by expressing polynomials as sums of ridge polynomials with fixed matrices. The paper also applies these results to generalized translation networks and complex-valued neural networks, obtaining optimal rates n^{-r/(d-ℓ)} and n^{-r/(2d-2)}, and it identifies and repairs a gap in the earlier proof for univariate ridge functions in [18].
Significance. If the external hypotheses on which the proof depends are verified, this paper solves the open problem of sharp approximation rates for multivariate ridge functions and provides the first optimal rates for shallow generalized translation networks and complex-valued neural networks with general activation functions. The paper is careful and detailed, with explicit constants, and its replacement of the orthogonal projection in [18] by uniformly bounded quasi-projections is a genuinely new methodological step that also corrects a known gap. The results are sharp and falsifiable, and the applications to neural networks are concrete. The principal caveat is that the central lower bound rests on an imported uniform-L^1 boundedness estimate for Cesàro means, whose hypotheses are not fully checked in the manuscript.
major comments (2)
- [Section 2.3, Proposition 2.7(3) and Lemma 3.8] The uniform L^1 bound for the quasi-projections P_r^s is the linchpin of the lower bound. The proof of Proposition 2.7(3) derives the estimate ∥P_r^s f∥_{L^1} ≤ C∥f∥_{L^1} from the uniform-in-k bound ∥S^σ_k(f)∥_{L^1} ≤ C_3∥f∥_{L^1}, which is imported from [4, Theorem 11.4.1] with the parameter choice κ=(0,…,0,1/2)∈R^{d+1} and σ>d/2. The manuscript does not state the hypotheses of that theorem, and it explicitly notes that the corresponding proposition is stated without proof in [4]. Since Lemma 3.8 transfers the lower bound from P_r^s(R*_{n,d,ℓ}) to R*_{n,d,ℓ}, the whole lower bound of Theorem 1.1 collapses if this estimate fails (for instance, if the theorem requires σ>d/2+κ_{d+1} or excludes p=1). The authors should either provide a self-contained proof of the uniform L^1 bound or quote [4, Theorem 11.4.1] in full and verify that the chosen parameters satisfy all of its assumptions.
- [Section 3.2, Lemma 3.1 and Lemma 3.3] The sign-set cardinality bound in Lemma 3.1, which is essential for the entropy lower bound, relies on Lemma 3.3, imported verbatim from [17, Lemma 3] without proof. Similarly, the upper bound in Section 4 depends on [26, Corollary 5.12]. These are external results at load-bearing steps of the argument. Given that the paper's contribution is precisely a careful and self-contained repair of a gap in [18], the reader needs to be able to verify that these external lemmas apply. The authors should either include proofs of these lemmas or state them with all hypotheses and exact reference locations, so that no hidden condition (e.g., on the degree bounds or on N+K) is left unchecked.
minor comments (3)
- [Section 3.3, Proposition 3.7] In the displayed chain of inequalities after equation (3.31), the factor should be (2ϑ)^{-r}, not (2ϑ)^r. The subsequent conclusion ∥f_{ε*}−P∥_{L^1} ≥ a·c_2·c_3·2^{-r}·m^{-r/d} is consistent with the negative exponent, so this appears to be a typo rather than a mathematical error.
- [Section 2.3, proof of Proposition 2.7(3)] The constant C(d) is written as C = C_1C_2C_3·2^{σ+1}, but C_3 depends on d and σ, and σ is chosen depending on d; this is fine, but the notation could be made clearer by writing C = C(d,σ) and then noting that σ is fixed by d.
- [Appendix A] The equation numbering in the discussion of the gap in [18] is slightly confusing: the manuscript refers to 'Equation (14)' of [18] but then labels its own displayed equation as (A.4). This could be clarified by explicitly saying that (A.4) is the corresponding statement in the notation of the present paper.
Circularity Check
No significant circularity: the sharp ridge-function bounds are derived from external harmonic-analysis and polynomial-representation results, with no fitted constant or self-citation chain determining the target rates.
full rationale
The central claims (Theorems 1.1 and 1.2) are not circular. The lower bound in Section 3 is obtained by a counting argument on sign patterns (Lemma 3.1, built on Lemmas 3.3 and 3.5) combined with the quasi-projection transfer in Lemma 3.8. The uniform L1 bound needed there, Proposition 2.7(3), is imported from the external monograph [4, Theorem 11.4.1] after the paper makes the explicit parameter choice kappa=(0,...,0,1/2) and sigma>d/2; it is not a self-citation, it is not fitted to the target rate, and the proof does not define the quasi-projection in terms of the lower bound. Whether [4, Theorem 11.4.1] really applies at this parameter boundary is a correctness question, not a circularity question. The upper bound in Section 4 uses the external Jackson estimate Proposition 2.5 (from [21]) and the polynomial ridge representation from [26]; these assumptions do not include n^{-r/(d-ell)}. The GTN and CVNN applications are deductions from Theorems 1.1 and 1.2 plus two standalone constructions. The re-used activation function from [9, Lemma F.4] is a parameter-free construction whose statement contains no approximation rate, and the dimension bound from [9, Lemma F.1] is an elementary counting estimate. Neither source imports the conclusion being proved. No equation in the derivation is equal by construction to the target rate, and no parameter is fitted to a subset of the data and then renamed as a prediction.
Assumptions & free parameters
assumptions (5)
- standard math Jackson-type bound: for every f in B(W^{r,p}_d) and s in N there exists P in P_s(B_d) with ||f-P||_{L^q} <= C s^{-r} for 1 <= q <= p <= infty.
- standard math Uniform L^1 boundedness of the Cesaro means S^sigma_k on L^1(B_d), with a constant independent of k, for sigma > d/2.
- standard math Sign-pattern counting bound: the number of sign patterns of m polynomials of degree at most s in N variables with K linear combinations is at most (4s)^N (N+K+1)^{N+2} (2em/(N+K))^{N+K}.
- standard math Polynomial ridge representation: if P_h^s(R^{d-ell+1}) is spanned by (a_i^T x)^s for i = 1..n, then P_s(R^{d-ell+1}) is spanned by (a_i^T x)^k for 0 <= k <= s, with a_i fixed.
- standard math Existence of the complex activation function phi from [9, Lemma F.4] whose shifts reproduce a dense set of polynomials in z and zbar on B_1(C).
Cite this review
Pith. "Pith review of On best approximation by multivariate ridge functions with applications to generalized translation networks." pith.science (2026). https://pith.science/paper/MBYRTCKA
@misc{pith2026241208453,
author = {Pith},
title = {Pith review of: On best approximation by multivariate ridge functions with applications to generalized translation networks},
year = {2026},
howpublished = {\url{https://pith.science/paper/MBYRTCKA}},
note = {Machine review of arXiv:2412.08453}
}
abstract
In this paper, we prove sharp upper and lower bounds for the approximation of Sobolev functions by sums of multivariate ridge functions, i.e., for approximation by functions of the form $\mathbb{R}^d \ni x \mapsto \sum_{k=1}^n \varrho_k(A_k x) \in \mathbb{R}$ with $\varrho_k : \mathbb{R}^\ell \to \mathbb{R}$ and $A_k \in \mathbb{R}^{\ell \times d}$. We show that the order of approximation asymptotically behaves as $n^{-r/(d-\ell)}$, where $r$ is the regularity (order of differentiability) of the Sobolev functions to be approximated. Our lower bound even holds when approximating $L^\infty$-Sobolev functions of regularity $r$ with error measured in $L^1$, while our upper bound applies to the approximation of $L^p$-Sobolev functions in $L^p$ for any $1 \leq p \leq \infty$. These bounds generalize well-known results regarding the approximation properties of univariate ridge functions to the multivariate case. We use our results to obtain sharp asymptotic bounds for the approximation of Sobolev functions using generalized translation networks and complex-valued neural networks.
Reference graph
Works this paper leans on
- [4]
-
[18]
Best approximation by ridge functions inLp-spaces
V. Maiorov. “Best approximation by ridge functions inLp-spaces.” In:Ukrainian Mathematical Journal 62 (2010), pp. 452–466
work page 2010
-
[1]
Nearly-tight VC-dimension and pseudodimension bounds for piecewise linear neural networks
P. L. Bartlett, N. Harvey, C. Liaw, and A. Mehrabian. “Nearly-tight VC-dimension and pseudodimension bounds for piecewise linear neural networks.” In:Journal of Machine Learning Research20.63 (2019), pp. 1–17
work page 2019
- [2]
-
[3]
Ridgelets: estimating with ridge functions
E. J. Candès. “Ridgelets: estimating with ridge functions.” In: The Annals of Statistics 31.5 (2003), pp. 1561–1599
work page 2003
-
[5]
L. C. Evans and R. F. Gariepy. Measure theory and fine properties of functions. Studies in advanced mathematics. Boca Raton: CRC Press, 1992. 268 pp.isbn: 978-0-8493-7157-8
work page 1992
-
[6]
G. Folland. Real Analysis: Modern Techniques and Their Applications. Wiley, 1984.isbn: 9780471809586
work page 1984
-
[7]
Robust and resource efficient identification of shallow neural networks by fewest samples
M. Fornasier, J. Vybíral, and I. Daubechies. “Robust and resource efficient identification of shallow neural networks by fewest samples.” In:Information and Inference: A Journal of the IMA10.2 (Jan. 2021), pp. 625–695.doi: 10.1093/imaiai/iaaa036
Show all 35 references
-
[8]
Geuchen, T
P. Geuchen, T. Jahn, and H. Matt. Universal approximation with complex-valued deep narrow neural networks. 2024. arXiv:2305.16910. Accepted for publication inConstructive Approximation
2024 arXiv
-
[9]
Optimal approximation using complex-valued neural networks
P. Geuchen and F. Voigtlaender. “Optimal approximation using complex-valued neural networks.” In: Advances in Neural Information Processing Systems36 (2024)
2024
-
[10]
On the best approximation by ridge functions in the uniform norm
Y. Gordon, V. Maiorov, M. Meyer, and S. Reisner. “On the best approximation by ridge functions in the uniform norm.” In:Constructive approximation18 (2001), pp. 61–85
2001
-
[11]
Ismailov
V. Ismailov. Ridge Functions and Applications in Neural Networks. Mathematical Surveys and Mono- graphs. American Mathematical Society, 2021.isbn: 9781470467654
2021
-
[12]
A review of some results on ridge function approximation
V. E. Ismailov. “A review of some results on ridge function approximation.” In: Azerb. J. Math 3.1 (2013), pp. 3–51
2013
-
[13]
Kaup and B
L. Kaup and B. Kaup.Holomorphic Functions of Several Variables: An Introduction to the Fundamental Theory. De Gruyter, Dec. 1983.isbn: 978-3-11-004150-7.doi: 10.1515/9783110838350
1983 doi
-
[14]
Complex-valued neural networks: A comprehensive survey
C. Lee, H. Hasegawa, and S. Gao. “Complex-valued neural networks: A comprehensive survey.” In: IEEE/CAA Journal of Automatica Sinica9.8 (2022), pp. 1406–1426
2022
-
[15]
Fundamentality of ridge functions
V. Y. Lin and A. Pinkus. “Fundamentality of ridge functions.” In:Journal of Approximation Theory75.3 (1993), pp. 295–311
1993
-
[16]
Luenberger
D. Luenberger. Optimization by Vector Space Methods. Professional Series. Wiley, 1997.isbn: 978-0- 4711-8117-0
1997
-
[17]
On Best Approximation by Ridge Functions
V. Maiorov. “On Best Approximation by Ridge Functions.” In:Journal of Approximation Theory99.1 (July 1999), pp. 68–94.doi: 10.1006/jath.1998.3304
1999
-
[19]
Lower bounds for approximation by MLP neural networks
V. Maiorov and A. Pinkus. “Lower bounds for approximation by MLP neural networks.” In:Neurocom- puting 25.1-3 (1999), pp. 81–91
1999
-
[20]
Onachoiceofsamplingnodesforoptimalapproximationofsmoothfunctions by generalized translation networks
H.MhaskarandJ.Prestin.“Onachoiceofsamplingnodesforoptimalapproximationofsmoothfunctions by generalized translation networks.” In:Fifth International Conference on Artificial Neural Networks (Conf. Publ. No. 440). 1997, pp. 210–215.doi: 10.1049/cp:19970728
1997 doi
-
[21]
Neural networks for optimal approximation of smooth and analytic functions
H. N. Mhaskar. “Neural networks for optimal approximation of smooth and analytic functions.” In: Neural computation8.1 (1996), pp. 164–177
1996
-
[22]
Degree of approximation by neural and translation networks with a single hidden layer
H. N. Mhaskar and C. A. Micchelli. “Degree of approximation by neural and translation networks with a single hidden layer.” In:Advances in applied mathematics16.2 (1995), pp. 151–183
1995
-
[23]
Easy proof of the Jacobian for then-dimensional polar coordinates
A. Muleshkov and T. Nguyen. “Easy proof of the Jacobian for then-dimensional polar coordinates.” In: Pi Mu Epsilon Journal14.4 (2016), pp. 269–273
2016
-
[24]
Optimal approximation of piecewise smooth functions using deep ReLU neural networks
P. Petersen and F. Voigtlaender. “Optimal approximation of piecewise smooth functions using deep ReLU neural networks.” In:Neural Networks108 (2018), pp. 296–330
2018
-
[25]
Approximating by ridge functions
A. Pinkus. “Approximating by ridge functions.” In:Surface fitting and multiresolution methods(1997), pp. 279–292
1997
-
[26]
A. Pinkus. Ridge functions. Cambridge: Cambridge University Press, 2016.isbn: 978-1-107-12439-4
2016
-
[27]
Prudnikov, Y
A. Prudnikov, Y. Brychkov, and O. Marichev.Integrals and Series: Volume 1: Elementary Functions. Taylor & Francis, 1986.isbn: 9782881240973. 48 REFERENCES
1986
-
[28]
Depth-width tradeoffs in approximating natural functions with neural net- works
I. Safran and O. Shamir. “Depth-width tradeoffs in approximating natural functions with neural net- works.” In:International conference on machine learning. PMLR. 2017, pp. 2979–2987
2017
-
[29]
Deep Network Approximation Characterized by Number of Neurons
Z. Shen, H. Yang, and S. Zhang. “Deep Network Approximation Characterized by Number of Neurons.” In: Communications in Computational Physics28.5 (2020), pp. 1768–1811.doi: https://doi.org/10. 4208/cicp.OA-2020-0149
2020
-
[30]
Optimal approximation rate of ReLU networks in terms of width and depth
Z. Shen, H. Yang, and S. Zhang. “Optimal approximation rate of ReLU networks in terms of width and depth.” In:Journal de Mathématiques Pures et Appliquées157 (2022), pp. 101–135.doi: https: //doi.org/10.1016/j.matpur.2021.07.009
2022 doi
-
[31]
Benefits of depth in neural networks
M. Telgarsky. “Benefits of depth in neural networks.” In:Conference on learning theory. PMLR. 2016, pp. 1517–1539
2016
-
[32]
Vershynin
R. Vershynin. High-dimensional probability: an introduction with applications in data science. Cambridge University Press, 2018. 284 pp.isbn: 978-1-108-41519-4
2018
-
[33]
The universal approximation theorem for complex-valued neural networks
F. Voigtlaender. “The universal approximation theorem for complex-valued neural networks.” In:Applied and computational harmonic analysis64 (2023), pp. 33–61
2023
-
[34]
The phase diagram of approximation rates for deep neural networks
D. Yarotsky and A. Zhevnerchuk. “The phase diagram of approximation rates for deep neural networks.” In: Advances in neural information processing systems33 (2020), pp. 13005–13015
2020
-
[35]
Deep network approximation: Achieving arbitrary accuracy with fixed number of neurons
S. Zhang, Z. Shen, and H. Yang. “Deep network approximation: Achieving arbitrary accuracy with fixed number of neurons.” In:Journal of Machine Learning Research23.276 (2022), pp. 1–60. (P. Geuchen)Mathematical Institute for Machine Learning and Data Science (MIDS), Catholic Un...
2022
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.