REVIEW 4 minor 1 cited by
Approximation and learning of anisotropic and mixed smooth functions by deep ReLU neural networks
T0 review · 0 major / 4 minor · reviewed 2026-08-02 · deepseek-v4-flash
Pith's one-line read Deep ReLU networks approximate anisotropic and mixed-smooth functions at rates set by the harmonic mean of the coordinate smoothnesses.
desk verdict Fully-connected ReLU nets get super rates for anisotropic and mixed Besov spaces, and the main proofs hold up; the abstract overclaims minimaxity for the composition model and one lower bound has an unshown calculation. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The proof rests on an adaptive multiscale anisotropic grid: at scale $\ell$, each coordinate direction j is partitioned into about $b^{\ell \tilde{s}/s_j}$ intervals, so the number of rectangles grows like $b^\ell$ regardless of dimension. The averaged modulus of smoothness converts local Whitney approximation errors on these rectangles into a global coefficient bound $||a_\ell||_q \lesssim b^{(1/q-\tilde{s})\ell}$, which controls the sparse coefficients fed to a network interpolation lemma for discretized data. A base-prime trick constructs several approximants on slightly shifted grids and uses an order-statistic network to select the best one, removing the 'trifling' boundary region. The width/de
What would settle it
Numerically compute the ratio of the averaged modulus $\tilde{\omega}(f,t)$ to the ordinary modulus $\omega(f,t)$ on a sequence of rectangles with increasingly extreme side ratios, with $t$ just below $\delta_j/(4k^2)$, for a function such as $f(x)=x_1^{s_1}x_2^{s_2}$ with widely separated $s_1,s_2$. If the ratio grows without bound as the side ratio grows, Lemma 5.9 fails and the rate is not supported. Alternatively, compute the coefficient bound $||a_\ell||_q$ for such an $f$ and check whether it really follows $b^{\ell/q - \tilde{s}\ell}$ or picks up a factor growing with $s_1/s_2$.
Extended reading notes
Core claim
The central claim is that the 'super approximation rate' known for isotropic Besov functions extends to anisotropic and mixed smoothness without sparse architectures. For anisotropic Besov space $B^s_q([0,1]^d)$ with mean smoothness $\tilde{s} = (\sum_j s_j^{-1})^{-1}$, if $\tilde{s} > 1/q - 1/p$, the network class $NN(W,L)$ satisfies $\sup_{f: ||f||\le 1} \inf_g ||f-g||_{L^p} \le C (WL)^{-2\tilde{s}}$. For mixed-smooth Besov space $MB^s_q$, if $s > 1/q - 1/p$, the rate is $(WL)^{-2s} (\log W \log L)^{(d-1)(2s+1)}$. These rates are optimal up to logarithmic factors, and the same machinery yields rates for compositions of anisotropic Besov functions and for least-squares learning.
Load-bearing premise
The whole argument leans on a single estimate asserting that an averaged smoothness measure stays comparable to the usual modulus of smoothness with a constant that does not blow up as rectangles become long and thin; if that constant actually depends on the coordinate smoothness parameters, the coefficient bounds and the final $(WL)^{-2\tilde{s}}$ rate collapse.
Editorial extensions
If this is right
- Anisotropic Besov approximation at rate (WL)^{-2\tilde{s}} is optimal up to a logarithmic factor; with bounded width the rate becomes L^{-2\tilde{s}}.
- Mixed-smooth Besov approximation reaches (WL)^{-2s} up to logs, improving on sparse-network results and matching lower bounds up to logarithmic factors.
- Fully-connected ReLU networks — no sparsity constraints — attain minimax regression rates n^{-2\tilde{s}/(2\tilde{s}+1)} (and the mixed-smooth analogue) up to logs.
- Compositions of anisotropic Besov functions are approximated at a rate determined by the smoothness index s^* = \min_m \tilde{s}_m \prod_{k>m} \gamma_k, near-optimal when all q_m = \infty.
- Existing isotropic Sobolev, Hölder, and Besov bounds appear as the special case s = (s_0,\ldots,s_0), giving \tilde{s} = s_0/d.
Reading between the lines
- If the constants in the key coefficient estimate turn out to depend badly on the dimension d or on the spread of the s_j, the practical benefit in high dimensions would shrink even though the formal rate is dimension-free; the paper leaves this dependence unquantified.
- The same anisotropic-grid averaging argument may extend to smoothness classes with mixed coordinate interactions beyond product Besov spaces, such as Triebel–Lizorkin spaces with dominating mixed smoothness.
- A direct testable corollary: for additive models or tensor-product functions listed in Remark 2.1, the network approximation error should behave as (WL)^{-2s} with logarithmic factors; this can be checked numerically with small networks.
- The technique suggests a recipe for other univariate nonlinear approximators: any basis that yields Whitney estimates and averaged-modulus sum inequalities can be converted into a deep ReLU approximation bound.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper studies approximation and learning of anisotropic and mixed-smooth Besov functions by fully-connected deep ReLU networks. The central upper bounds are Theorem 3.1, which gives an Lp approximation rate of order (WL)^{-2\tilde{s}} for anisotropic Besov spaces under \tilde{s} > 1/q - 1/p, and Theorem 3.3, which gives the rate (WL)^{-2s}(\log W \log L)^{(d-1)(2s+1)} for mixed-smooth Besov spaces under s > 1/q - 1/p. Theorem 3.5 extends the construction to compositions of anisotropic Besov functions. Theorems 3.2, 3.4 and 3.6 provide pseudo-dimension/entropy-based lower bounds showing that these rates are optimal up to logarithmic factors. Section 4 applies the approximation bounds to least-squares regression and derives minimax-optimal learning rates, up to logarithms, for the corresponding smoothness classes. The proofs are built on anisotropic multiresolution partitions, averaged moduli of smoothness, piecewise-polynomial decompositions, and the interpolation networks from Yang (2025).
Significance. If the results stand, they are a substantial advance: they show that the good approximation rates of deep ReLU networks for anisotropic and mixed smoothness do not require sparse architectures, and they provide explicit width/depth trade-offs. The extension from isotropic to anisotropic approximations is nontrivial, and the use of averaged moduli to sum local Whitney estimates is well suited to the problem. The paper is also careful about constants and about the distinction between the parallel and sequential summation of networks. The lower-bound arguments are standard but are adapted cleanly to the anisotropic and mixed cases. Overall, the central claims of Theorems 3.1 and 3.3 appear sound; the issues I found are local and fixable.
minor comments (4)
- [Lemma 5.9] The statement requires 0 < t ≤ δ_j/(4k), but the proof only establishes the equivalence for 0 < t ≤ δ_j/(4k^2) (see the constraint 'h ≤ t/2 ≤ δ_j/(8k^2)' and the final line 't ≤ δ_j/(4k^2)'). In Lemma 5.10 the argument applies the lemma with t = λ b^{-ℓ_j}, λ = min_j 1/(5k_j), which satisfies the stated 4k condition but not the proof's 4k^2 condition when k_j > 1. This mismatch should be fixed explicitly: either complete the proof for the stated 4k condition or, alternatively, replace λ by min_j 1/(5k_j^2) in Lemma 5.10; the rest of the estimates are unaffected since the new constants still depend only on p,q,s,d,b.
- [Theorem 3.6 proof] The sentence 'through a cumbersome calculation, one can verify that x^{eγ_m} ∈ B^{s_m}_{q_m}([0,1])' is an omitted proof of a claim used to construct the lower-bound family. The endpoint case eγ_m = s_m - 1/q_m with r = ∞ is plausible via the critical embedding, but the calculation should be written out, especially since Theorem 3.6 supports the claimed near-optimality of Theorem 3.5.
- [Proposition 5.13] The application of Corollary 5.2 to the N_{ℓ*,d} networks G_i is terse. A single application of the corollary gives only one of the two polynomial factors: width O(N b^α) with depth O(b^β), or width O(b^α) with depth O(N b^β). To obtain simultaneously W ≤ C α^{d-1} b^α and L ≤ C β^{d-1} b^β, the networks must be grouped into a two-dimensional grid (N_1 × N_2 with N_1 N_2 ≥ N_{ℓ*,d}). Please spell out this grouping.
- [Typos and notation] Several typos should be corrected: 'supper approximation' → 'super approximation'; 'extent this result' → 'extend this result'; 'there exits f' → 'there exists f'. In Section 3.2, after Theorem 3.4, 'the upper bound in Theorem 3.1' should be 'Theorem 3.3'. In the proof of Theorem 3.5, g_m should map to R^{d_{m+1}} rather than R^{d_m}, and the width bound should accordingly use max_{2≤m≤M} d_{m+1} W_0.
Circularity Check
No significant circularity: the super-approximation results are derived from Besov-space definitions and explicit ReLU network constructions, not from their stated rates.
full rationale
The central bounds (Theorems 3.1 and 3.3) are obtained by a constructive chain: Whitney/Brudnyi approximation (Lemma 5.8), averaged-modulus estimates (Lemma 5.9), multiscale coefficient decompositions of anisotropic and mixed Besov functions (Lemmas 5.10 and 5.12), and explicit ReLU network interpolation of piecewise polynomials (Lemma 5.7). The target rate (WL)^{-2\tilde{s}} or (WL)^{-2s} appears only as the final exponent after balancing width and depth, not as an input. The paper's self-citations to Yang (2025) supply technical building blocks (product approximation, coefficient interpolation, the isotropic base case); those lemmas have stated assumptions that do not include the present anisotropic or mixed-smoothness conclusions, and the anisotropic/mixed analysis itself is carried out in this manuscript. Lower bounds (Theorems 3.2, 3.4) use pseudo-dimension and metric entropy arguments rather than the upper-bound statements. The proof of Lemma 5.9 contains a minor constant typo (4k vs 4k^2), and Theorem 3.6's proof defers a 'cumbersome calculation,' but neither makes the derivation circular. No fitted parameter is relabeled as a prediction, and no uniqueness theorem is imported from the authors.
Assumptions & free parameters
free parameters (1)
- epsilon in definition of gamma_m =
arbitrary epsilon>0, can be taken arbitrarily small
assumptions (6)
- standard math Whitney-type polynomial approximation on rectangles (Lemma 5.8, Brudnyi 1970 and Dahmen et al. 1980)
- standard math Pseudo-dimension bounds for piecewise-linear ReLU networks (Bartlett et al. 2019, equation (5.17))
- standard math Metric entropy estimates for mixed Besov spaces (Vybral 2006 and Dung et al. 2018, equation (5.22))
- standard math Oracle inequality for truncated least-squares estimators (Kohler and Langer 2021, Lemma 18)
- standard math Besov embedding and Sobolev embedding facts, including B^s_{\infty,1} embedded in B^s_{q,r} and anisotropic Besov embedded in Holder-Zygmund spaces
- domain assumption Exponential moment condition (4.1) and bounded regression function in the learning theorems
Cite this review
Pith. "Pith review of Approximation and learning of anisotropic and mixed smooth functions by deep ReLU neural networks." pith.science (2026). https://pith.science/paper/LAK3ZVV2
@misc{pith2026260531152,
author = {Pith},
title = {Pith review of: Approximation and learning of anisotropic and mixed smooth functions by deep ReLU neural networks},
year = {2026},
howpublished = {\url{https://pith.science/paper/LAK3ZVV2}},
note = {Machine review of arXiv:2605.31152}
}
abstract
This paper studies how efficiently deep ReLU neural networks can approximate and learn smooth functions. When the error is measured in $L^p([0,1]^d)$ norm and the approximator is a network with width $W$ and depth $L$, recent works have proven the supper approximation rate $\mathcal{O}((WL)^{-2s/d})$ for Besov space $\mathcal{B}^s_{q,r}([0,1]^d)$ under the Sobolev embedding condition $s/d>1/q-1/p$. In order to overcome the curse of dimensionality in this rate, we extent this result to anisotropic and mixed smooth function classes. We establish the approximation rate $\mathcal{O}((WL)^{-2\tilde{s}})$ for anisotropic Besov space $\mathcal{B}^{\boldsymbol{s}}_{q,r}([0,1]^d)$ with anisotropic smoothness $\boldsymbol{s}=(s_1,\dots,s_d)$ under the embedding condition $\tilde{s} > 1/q-1/p$, where the mean smoothness $\tilde{s} = (\sum_{i=1}^d s_i^{-1})^{-1}$. For mixed smooth Besov space $\mathcal{MB}^s_{q,r}([0,1]^d)$ with mixed smoothness $s>1/q-1/p$, we show that the approximation rate $\mathcal{O}((WL)^{-2s})$ holds up to logarithmic factors. Using these results, we also derive approximation bounds for the composition of anisotropic Besov functions. As an application, it is shown that deep ReLU neural networks can achieve minimax optimal rates up to logarithmic factors for a wide range of smooth function classes.
Forward citations
Cited by 1 Pith paper
-
Approximation of Analytic Functions by ReLU Neural Networks with Adjustable Depth and Width
ReLU nets of width N and depth L approximate analytic functions at rate O(N^{-C L^tau}) with tau depending on N-L scaling, so depth dominates width.
Reference graph
Works this paper leans on
-
[2015]
Higher order approximation rates for ReLU CNNs in Korobov spaces.arXiv: 2501.11275,
Yuwen Li and Guozhi Zhang. Higher order approximation rates for ReLU CNNs in Korobov spaces.arXiv: 2501.11275,
-
[2022]
Marco Signoretto, Lieven De Lathauwer, and Johan A. K. Suykens. Learning tensors in reproducing kernel Hilbert spaces with multilinear spectral penalties.arXiv: 1310.4977,
Reviewed August 2, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.