{"id":"bac26e83-a24a-40fd-8fde-188df09c3bd7","arxiv_id":"1908.00695","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Deep ReLU networks approximate beta-Hoelder functions on a d*-dimensional manifold with O(epsilon^{-d*/beta} log(1/epsilon)) nonzero parameters, and empirical risk minimization achieves risk n^{-2 beta/(2 beta + d*)} log^3(n).","lead":"This paper proves a mathematical guarantee: deep ReLU networks approximate smooth functions on a low-dimensional manifold with accuracy and network size that scale with the manifold's dimension, not the high-dimensional space around it. It also shows the resulting regression estimator achieves near-optimal prediction error, supporting why deep learning works on data like images that live on lower-dimensional structures.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Composition step (5.6) is not justified as written: Lemma 6's K-factor is dropped and Lemma 2 is cited outside its stated β≥1 range; both are patchable without changing the rate.","rationale":"I read the paper charitably: the chart-direction typo is resolved by ψ_j(V_j) and (3.2), the unquantified constants are standard existential constants, and dependence on Theorem 1 of [26] is acceptable. Definition 1 is explicit, and for C∞ embedded submanifolds it can be realized by charts that are restrictions of linear projections, so I do not treat it as a fatal restriction. The most load-bearing soft spot is instead the composition chain that converts the chart, partition-of-unity, and target-function approximations into the final bound on M. Two concrete proof gaps appear there: the dropped K-factor from Lemma 6 in (5.6), and the use of Lemma 2 for β<1 outside its stated range. Both are patchable without changing the η^{-d*/β} rate, so they do not warrant rejection; they do justify keeping the conditional verdict and asking for a corrected, self-contained write-up of (5.2)–(5.7).","tokens_in":18779,"tokens_out":37981,"duration_ms":402957,"concrete_test":"Re-derive the composition bound leading to (5.6) with explicit constants: replace the first term by K‖~ψ_j−ψ_j‖^{β∧1}, then recompute the required N1 exponent from (5.2) with error target (η/(4rK))^{1/(β∧1)} and verify the final constants still yield total error η. Separately, prove the 0<β<1 case of the composition f∘ψ_j^{-1} directly using Lipschitzness of ψ_j^{-1} and β-Hölder continuity of f. If both repairs succeed with only redefined constants, the central claim stands; if not, the proof of Theorem 2 is incomplete in a rate-relevant way.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central charting construction in Theorem 2 depends on the estimate leading to (5.6). Lemma 6 states ‖h1∘h0−~h1∘~h0‖ ≤ K‖h0−~h0‖^{β∧1} + ‖h1−~h1‖, where K is the Hölder constant of h1=f∘ψ_j^{-1} on the δ'-neighborhood. In (5.6) this is applied with the first error term written as ‖~ψ_j−ψ_j‖^{β∧1}, omitting K. Since Theorem 2 takes a supremum over f∈C^β_d(R^d,K), K need not be ≤1, so the written inequality does not yield the claimed η/(2r) bound. The repair is to target (η/(4rK))^{1/(β∧1)} in (5.2) instead of (η/(4r))^{1/(β∧1)}; this changes only the existential constants, not the rate. Separately, the proof that f∘ψ_j^{-1}∈C^β_{d*} invokes Lemma 2, which is stated and proved only for β≥1, while Theorem 2 allows all β>0. For 0<β<1 the composition is still β-Hölder because ψ_j^{-1} is Lipschitz, but the proof as written does not cover that case. These are concrete proof gaps, not counterexamples to the intrinsic-dimension rate; the high-level construction and the stated rates remain credible.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper studies sup-norm approximation of β-Hölder functions on a compact d*-dimensional manifold M⊂R^d by sparsely connected deep ReLU networks. The main result, Theorem 2, states that for manifolds with 'smooth local coordinates' (Definition 1), any f∈C^β_d(R^d,K) with |f|≤1 can be approximated on M to error η by a network with depth O(log(1/η)), width O(η^{-d*/β}), and sparsity O(η^{-d*/β} log(1/η)), with constants depending on d, d*, β, and K. The proof combines a partition of unity, chart-wise approximations via the author's earlier Euclidean ReLU approximation theorem, and ReLU network composition and multiplication lemmas. Theorem 3 converts this into an oracle-inequality risk bound of order n^{-2β/(2β+d*)} log^3 n for the empirical risk minimizer.","tokens_in":18983,"tokens_out":30623,"duration_ms":279390,"significance":"If the stated result is correct, it is a valuable contribution: it shows that the intrinsic dimension d*, rather than the ambient dimension d, controls ReLU approximation complexity on manifolds, up to logarithmic factors, while keeping all network parameters bounded by one. The construction via smooth local charts and a partition of unity is natural, and the modular network calculus (parallelization, composition, multiplication) makes the proof transparent and reusable. The statistical rate for the ERM is the expected dimension-adapted nonparametric rate and supports the view that deep networks adapt to low-dimensional structure. The main caveats are that the theorem requires a strong chart-regularity assumption and ambient Hölder smoothness, and that the proof as written contains several localized gaps, identified below, which are patchable without changing the rates.","major_comments":[{"comment":"The first inequality in (5.6) applies Lemma 6 but drops the multiplicative Hölder constant K. Lemma 6 states ||h1∘h0−~h1∘~h0|| ≤ K||h0−~h0||^{β∧1} + ||h1−~h1||, and in the application h1 = (f∘ψ_j^{-1}, -f∘ψ_j^{-1}) has a Hölder constant that need not be ≤1 because Theorem 2 takes a supremum over f∈C^β_d(R^d,K). As written, the bound ||~ψ_j−ψ_j||^{β∧1} ≤ η/(4r) does not imply the displayed η/(2r) term. The gap is patchable by targeting (η/(4rK'))^{1/(β∧1)} in (5.2), where K' is a uniform Hölder constant for the composed maps over the class; this changes only the constants c, C, C' and not the η-rate.","section":"Section 5, Eq. (5.6)"},{"comment":"Lemma 2 is stated and proved only for β≥1, but the proof of Theorem 2 invokes it without restriction to conclude that f∘ψ_j^{-1}∈C^β_{d*}, and the proof of Lemma 3 invokes it for arbitrary β>0 when forming τ_j. For 0<β<1 the conclusion is true in this setting because the maps being composed are Lipschitz (ψ_j^{-1} by Definition 1, and the function G in Lemma 3 is C^∞), so the relevant compositions are β-Hölder; however, this argument is not what is written. Since Theorem 2 claims all β>0, this is a load-bearing proof gap and should be closed by an explicit Lipschitz-composition argument or by stating a suitable β<1 variant of Lemma 2.","section":"Section 5, Theorem 2 proof and Lemma 3 proof"},{"comment":"The approximation bound for ~f∘ψ_j^{-1} is stated on ψ_j(V_j^{δ/2}∩M)^{δ'/2}, while the composition bound (5.6) is applied on (V_j^{-δ})^{δ'}. The set V_j^{δ/2} is not defined, and no inclusion between these two sets is established; without such an inclusion, the second term in (5.6) is not controlled by (5.3). This is patchable by stating (5.3) on ψ_j(V_j^{-δ})^{δ'} directly, with δ' taken from Lemma 5, but as written the proof of the main error estimate is incomplete.","section":"Section 5, displays (5.3) and (5.6)"}],"minor_comments":[{"comment":"Definition 1 as printed says ψ_j: R^{d*}→V_j and ψ_j∈C^γ_d(V_j), which is inconsistent with the later usage ψ_j(V_j)⊂R^{d*} throughout the proof of Theorem 2. The intended statement is presumably ψ_j: V_j→R^{d*} with ψ_j∈C^γ_{d*}(V_j) and ψ_j^{-1}∈C^γ_{d*}(ψ_j(V_j)); please correct the domains and codomains.","section":"Definition 1"},{"comment":"There is a typo in the proof of Lemma 2: 'Th coefficient' should read 'The coefficient'.","section":"Lemma 2 proof"},{"comment":"In the paragraph bounding the network approximating τ_j, the sparsity bound is written as s1≤142d*(d+γ3+1)^{3+d}N3(m3+6); this should be s3, consistently with the notation used for the other nets.","section":"Theorem 2 proof, after (5.4)"},{"comment":"The theorem assumes f∈C^β_d(R^d,K), which is stronger than intrinsic smoothness on the manifold. The proof actually uses only the regularity of f∘ψ_j^{-1} on the charts together with boundedness of f, so the statement could be weakened; as written, the ambient assumption should be acknowledged as a limitation.","section":"Theorem 2 and Section 4"},{"comment":"The ERM analysis restricts the network class to functions mapping [0,1]^d to [-1,1], but Theorem 2 does not guarantee that the constructed approximant lies in that class. A clipping or truncation step is needed to make the oracle inequality apply; this is standard and should not change the rate, but it should be stated.","section":"Section 4"}],"recommendation":"major_revision","confidential_remarks":"The paper leans heavily on the author's earlier work [26] for the Euclidean ReLU approximation engine; at the time of this arXiv version the reference is 'to appear', so the editors may want to verify that it is now published or otherwise publicly available with the exact theorem used. The proof gaps identified above are local and patchable, and the claimed rates appear credible; I do not see a reason to doubt the main approximation result. I recommend major revision so that the authors can repair the composition estimate, the β<1 usage of Lemma 2, and the set mismatch in (5.3)/(5.6), and carefully proofread the notation in Definition 1 and the surrounding displays."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe thing you should know first: this paper is the first in the ReLU approximation line to get rates that scale with the intrinsic manifold dimension d* rather than the ambient dimension d, up to logs. The construction—local coordinate charts, a refined partition of unity, and the ReLU multiplication network—is genuinely new and the high-level error accounting is coherent. The statistical risk bound n^{-2β/(2β+d*)} up to logs follows naturally. This is a real step forward.\n\nCredit where due: the paper engages honestly with the concurrent Nakada–Imaizumi work, uses only networks with weights bounded by one, which is a nontrivial ReLU-specific feature, and the proof strategy is transparent even when the details are heavy. The constants in Theorem 2 are explicitly unquantified, so that is a weakness in the statement rather than a hidden flaw.\n\nNow the soft spots. The stress-test note is right on both counts. First, in the composition step leading to (5.6), Lemma 6 gives a factor K on the chart error term ||ψ_j − ~ψ_j||^{β∧1}. The displayed inequality drops that K. Since the theorem is uniform over f ∈ C^β_d(R^d, K), K need not be ≤ 1, so the written bound does not yield η/(2r). This is patchable by targeting (η/(4rK))^{1/(β∧1)} in (5.2); only the existential constants change, not the rate. Second, the claim that f∘ψ_j^{-1} ∈ C^β_{d*} cites Lemma 2, which is stated and proved only for β ≥ 1, while Theorem 2 allows all β>0. For 0<β<1 the composition is still β-Hölder because ψ_j^{-1} is Lipschitz, but the proof as written does not cover that case. Again a patch, not a catastrophe.\n\nTwo further caveats. The notation for the coordinate maps is self-contradictory at one point: the sentence \"ψ_j: R^{d*} → V_j\" is reversed relative to Definition 1 and every subsequent use. That is a typo, but it needs fixing. More substantively, Definition 1 is strong: it requires charts that are γ-Hölder for every γ>0, i.e., essentially C∞-smooth. That is fine for a smooth manifold, but the paper should say so and discuss whether finite smoothness suffices. The assumption that f is β-Hölder on all of R^d is also stronger than intrinsic smoothness on the manifold; the paper states it, but not loudly.\n\nWho is this for? Anyone working on approximation or nonparametric regression with low-dimensional structure. Does it deserve a serious referee? Yes. The central claim is probably true and the paper is a real contribution. The referee should ask for the two proof repairs, the notation fix, and a more explicit discussion of assumptions. I would be surprised if the intrinsic-dimension rate itself is wrong.","headline":"Genuinely new intrinsic-dimension rates for ReLU nets on manifolds; the high-level proof is credible, but the written composition step has two patchable gaps and the main assumption is stronger than advertised.","tokens_in":19577,"tokens_out":5364,"would_cite":true,"duration_ms":50126,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T07","41A25","62G08"],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper proves that a deep ReLU network with all weights bounded by one can approximate any $\\beta$-Hölder function on a compact $d^*$-dimensional manifold using $O(\\eta^{-d^*/\\beta}\\log(1/\\eta))$ nonzero parameters, so the ambient…","keywords":["manifolds","neural networks","ReLU activation function","approximation rates","estimation risk","intrinsic dimension","Hölder functions","sparse networks"],"falsifier":"Take a compact $C^k$ manifold with $k<\\infty$ (for example a two-dimensional cone-like surface in $\\mathbb{R}^3$) and a $\\beta$-Hölder ambient function, and compute the minimal sup-norm error achievable by ReLU networks with $O(\\eta^{-d^*/\\beta}\\log(1/\\eta))$ nonzero parameters as $\\eta\\to0$; if the error does not go to zero, the smooth-local-coordinates hypothesis is necessary rather than technical.","tokens_in":18488,"feed_emoji":"🧠","tokens_out":11169,"duration_ms":108471,"temperature":0.7,"pith_summary":"The paper establishes a worst-case approximation guarantee: for any compact $d^*$-dimensional manifold embedded in $\\mathbb{R}^d$ that has smooth local coordinates, every $\\beta$-Hölder function valued in $[-1,1]$ can be approximated on the manifold to uniform error $\\eta$ by a deep ReLU network with $O(\\eta^{-d^*/\\beta}\\log(1/\\eta))$ nonzero parameters. The exponent is governed by the manifold's intrinsic dimension rather than the ambient dimension, which is why the result circumvents the curse of dimensionality when $d^* \\ll d$. The proof decomposes the target as a partition-of-unity sum of functions pulled back through local coordinate charts, approximates each ingredient separately, and recombines them inside one bounded-weight ReLU network. A statistical consequence is that empirical risk minimization over such sparse networks attains prediction risk $O(n^{-2\\beta/(2\\beta+d^*)}\\log^3 n)$ under Gaussian noise with design on an unknown manifold.","feed_headline":"Sparse ReLU nets match manifold dimension, not ambient dimension","feed_subtitle":"ReLU networks approximate Hölder functions on a d*-dimensional manifold with O(η^{-d*/β} log(1/η)) nonzero parameters.","key_machinery":"The machinery has three parts. First, Definition 1 (smooth local coordinates): the manifold admits charts $V_j$ with maps $\\psi_j$ and inverses $\\psi_j^{-1}$ that are $\\gamma$-Hölder with respect to the ambient Euclidean norm for every $\\gamma>0$; this is what lets $f\\circ\\psi_j^{-1}$ inherit $\\beta$-smoothness on $\\mathbb{R}^{d^*}$ and lets local Taylor approximation survive composition. Second, a base approximation theorem (Theorem 1) for Hölder functions on bounded Euclidean domains, producing bounded-weight ReLU networks from localized Taylor polynomials and grid weighting. Third, the glue: the ReLU identity $\\sigma\\circ\\sigma=\\sigma$ to pad sub-network depths, parallelization to compute all pieces, and a bounded-weight multiplication sub-network to form $\\tau_j\\cdot(f\\circ\\psi_j^{-1})\\circ\\psi_j$ before summing over charts. The finite partition of unity with Hölder regularity and a positive margin from the chart boundaries (Lemma 3) is what turns the local estimates into one global network.","core_discovery":"The central claim is Theorem 2. For a compact $d^*$-dimensional manifold $M\\subset\\mathbb{R}^d$ with smooth local coordinates, there exist constants $c,C,C'>0$ such that for every $0<\\eta\\le 1/2$, every depth $L\\ge c\\log(1/\\eta)$, every width $p>C\\eta^{-d^*/\\beta}$, and every sparsity $s\\ge C'L\\eta^{-d^*/\\beta}$, any $\\beta$-Hölder function $f:\\mathbb{R}^d\\to[-1,1]$ can be matched on $M$ by a network in $F(L,(d\\sim p\\sim 1),s)$ to sup-norm error at most $\\eta$. All network weights are bounded in absolute value by one. The proof realizes the target through the identity $\\sum_{j=1}^r \\tau_j(x)(f\\circ\\psi_j^{-1})(\\psi_j(x))=f(x)$ on $M$, with $(\\psi_j)$ local coordinate charts and $(\\tau_j)$ a smooth partition of unity; each of the chart maps, the pullbacks $f\\circ\\psi_j^{-1}$ on $\\mathbb{R}^{d^*}$, and the partition functions is approximated by a separate ReLU sub-network, and the sub-networks are synchronized in depth and combined using ReLU-specific composition and multiplication lemmas.","pith_inferences":["If the chart smoothness assumption were weakened to finite Hölder order $k$, one would expect the available rate to degrade to roughly $\\eta^{-d^*/(\\beta\\wedge(k-1))}$ or to require $\\beta