{"id":"c8403c0e-abaf-4429-bd64-a3360b2eb730","arxiv_id":"2504.15110","paper_version":3,"verdict":"REJECT","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":0,"one_line_summary":"Residual KANs are claimed to approximate and learn Besov functions at optimal rates, but the proof's width count is invalid in dimension d.","lead":"This paper proves approximation and sample-complexity guarantees for residual Kolmogorov-Arnold networks in Besov norms. It claims optimal rates on Lipschitz and fractal domains, but a counting error in the main proof undermines the stated complexity.","discovery_kind":"first_principles","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Lemma 3 Step 4 counts the d-dimensional dyadic grid as if it were one-dimensional, so the width and parameter rates in Theorem 1 are not established.","rationale":"The reader's weakest_assumption identifies exactly the same load-bearing error: Lemma 3 Step 4 assumes the d-dimensional index set Λ(k) has at most 2^k elements, whereas the cardinality of dyadic cubes at scale k in [0,1]^d is 2^{kd}. This is not a stylistic or technical quibble; it is the step that converts the truncated spline expansion into the stated width O(ε^{1/(α-s)}) and parameter O(d^2 ε^{1/(α-s)}) bounds. Fixing the count introduces a factor d in the exponent, destroying the dimension-free optimality claim that the abstract and Section 3.1 emphasize. I agree with the reader's REJECT verdict: the proof of the central theorem has a concrete false estimate, and the claim as stated is unsupported. The suggested concrete test is straightforward and would settle the issue definitively by recomputing the counting argument; no additional experiments or numerical checks are needed to expose the gap.","tokens_in":22108,"tokens_out":3753,"duration_ms":36489,"concrete_test":"Recompute Lemma 3 Step 4 with the correct d-dimensional cardinality: replace ∑_{k=0}^K 2^k by ∑_{k=0}^K 2^{kd} in both the width and parameter counts, then substitute K = ⌈log_2 ε^{1/(α-s)} - 1⌉. If the width bound becomes (d+4)(2^{d(K+1)}-1)/(2^d-1) and the parameter bound becomes (d^2+35d-35)(2^{d(K+1)}-1)/(2^d-1), the rates are ε^{d/(α-s)}, so Theorem 1's dimension-free exponent fails. A complementary check is to verify whether any Besov decay estimate in Step 1 or Step 3 bounds |Λ(k)| by 2^k; it does not, and the proof supplies no such sparsity condition.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central approximation-rate claim rests on Lemma 3, and Step 4 of its proof contains a false count. At scale k, the spline expansion in Step 1 runs over dyadic cubes D_k ⊂ [0,1]^d with side length 2^{-k}; there are 2^{kd} such cubes, not 2^k. The index set Λ(k) = {j ∈ 2^{-k}Z^d : β_j ≠ 0} is defined as the set of nonzero coefficients, and nothing in the Besov decay estimate (15)–(16) bounds its cardinality by 2^k; an arbitrary f ∈ B^s_{p,q}([0,1]^d) can have coefficients supported on a full d-dimensional dyadic lattice, i.e., up to 2^{kd} nonzero coefficients at scale k. Step 4 nevertheless replaces ∑_{j∈Λ(k)} with the one-dimensional count 2^k, writing width ≤ (d+4)∑_{k=1}^K 2^k = (d+4)(2^{K+1}-2) and an analogous parameter count. With the correct bound, the sums become ∑_{k=1}^K 2^{kd} = (2^{d(K+1)}-2^d)/(2^d-1), and substituting K = ⌈log_2 ε^{1/(α-s)} - 1⌉ gives width and parameter counts of order ε^{d/(α-s)} = ε^{-d/(s-α)}, with the dimension d in the exponent. This removes the claimed dimension-free rate, and since the same count is used in Lemmas 4 and 5 and hence in Theorem 1, the main approximation theorem and the abstract's central claim do not follow from the submitted proof. Additional issues, such as the undefined target regularity r in Theorem 1 and the undefined smoothing layer in Definition 1, reinforce that the proof is not yet internally consistent, but the dyadic count is the load-bearing failure.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies residual Kolmogorov-Arnold networks (Res-KANs), whose activations are cardinal B-splines plus ReLU and ReLU², with residual connections, and analyzes their approximation and learning properties. The main approximation claim (Theorem 1) is that for any f ∈ B^s_{p,q}(X) on a bounded extension domain or an Ahlfors-regular fractal domain, there is a Res-KAN fhat with width O(ε^{1/(α⋆-s)}), depth O(d), and O(d² ε^{1/(α⋆-s)}) nonzero parameters such that ∥f - fhat|X∥_{B^α_{p,q}(X)} < ε for any α < s. The proof builds on an exact d-fold multiplication gadget (Lemma 1), emulates tensor-product cardinal B-splines (Lemma 2), invokes a spline characterization of Besov spaces (Lemma 3), and then transfers the result to Lipschitz and fractal domains via extension operators (Lemmas 4 and 5). The paper also proves a pseudodimension bound for Res-KANs (Lemma 6) and derives a PAC sample-complexity bound for noiseless learning (Theorem 2), and it reports toy numerical experiments showing that residual connections do not degrade accuracy.","tokens_in":22437,"tokens_out":8776,"duration_ms":74791,"significance":"The architecture-level construction is a strength: Lemma 1 gives an explicit residual-KAN implementation of d-fold multiplication with exact parameter counts, and Lemma 2's exact emulation of cardinal B-splines is a clean and useful observation. If Theorem 1 were proved, the paper would provide a natural spline-based explanation of KAN approximation in weaker Besov norms, with derivative control relevant to PDE applications, and would place KANs on the same footing as ReLU networks in constructive approximation theory. The numerical sanity checks are appropriate for a theory paper. However, the central approximation-rate claim rests on a coefficient-counting argument that is false in dimension d > 1, and the advertised dimension-free rates are therefore not established; the remaining statements also contain internal inconsistencies that prevent the theorem from being read as a valid mathematical claim.","major_comments":[{"comment":"Step 4 estimates ∑_{k=1}^{K}∑_{j∈Λ(k)}(d+4) ≤ (d+4)∑_{k=1}^{K}2^{k}, treating the index set Λ(k) as if it had at most 2^k elements. At scale k, Λ(k) indexes the nonzero spline coefficients over dyadic cubes in [0,1]^d; a general f ∈ B^s_{p,q}([0,1]^d) can have coefficients on a full d-dimensional dyadic lattice, so |Λ(k)| can be as large as 2^{kd}. The decay estimate (16) bounds the weighted ℓ_p sums of the coefficients, but it does not bound the support size. With the correct cardinality, the width and parameter bounds become O(2^{dK}), and substituting K ∼ log_2 ε^{1/(α-s)} yields O(ε^{-d/(s-α)}) rather than O(ε^{-1/(s-α)}). Since Lemma 4, Lemma 5, and Theorem 1 inherit the same count, the claimed dimension-free approximation rates are not proved.","section":"Appendix A, Lemma 3, Step 4"},{"comment":"The statement is internally inconsistent. It introduces α⋆ = α or α⋆ = α − (n−d)/p, then asserts f ∈ B^{α⋆}_{p,q}(X), whereas the proof in Lemmas 4 and 5 starts from f ∈ B^s_{p,q}(X) and measures the error in B^α_{p,q} (resp. B^{α−(n−d)/p}_{p,q}) norms. It also states that the approximant belongs to B^r_{p,q}(X) for an r that is never defined. The theorem must specify the source regularity s, the target norm, and the role of α⋆ precisely.","section":"Theorem 1"},{"comment":"Section 2.2 promises that a smoothing layer is added at the output so that \"any function implemented by our smoothed residual KANs is necessarily smooth\", but Definition 1 contains no such layer: it only imposes the sparsity rule (6) on the first I+1 rows of β. The networks used in Lemmas 1 and 2 explicitly activate ReLU and ReLU², which are not smooth, so the constructed approximants are not C∞ and the assertion fhat ∈ B^r_{p,q}(X) in Theorem 1 is unsupported. Either the smoothing layer must be formally added to Definition 1 and its effect on the approximation analysis justified, or the smoothness requirement must be changed.","section":"Definition 1 and Section 2.2"},{"comment":"The abstract and introduction call the sample-complexity result \"dimension-free\", but Theorem 2 states N ∈ O(ε^{-2-(d+1)/α} (ln(1/ε))² + ε^{-2} ln(1/δ)), with the dimension d appearing in the exponent. This is not dimension-free in the standard sense, and the claim should be replaced by a precise statement of what does not depend on d.","section":"Theorem 2 and Abstract"}],"minor_comments":[{"comment":"The symbol S in the normalized best-polynomial approximation energy E_k(f,Q)_{L_p(X)} is never defined; presumably it should be X (or Q∩X).","section":"Equation (9)"},{"comment":"In the statement of Theorem 2, \"each centered Borel probability measure Psmpl ∈ P(R)\" should read Perr ∈ P(R).","section":"Theorem 2"},{"comment":"The index set in the sequence of networks is written as j ∈ Λ(j); it should be j ∈ Λ(k).","section":"Lemma 3, Step 2"},{"comment":"The extension operator is written as E: B^{s-(n-d)/p}_{p,q}(X) → B^s_{p,q}(R^d), which appears to have domain and codomain reversed relative to its use on f ∈ B^s_{p,q}(X); please check the cited [20, Theorem 6.1].","section":"Lemma 5 proof"},{"comment":"In the fractal Besov definition, the integrand uses ∥E_k(f,Q(·,τ))∥_{L_u(X)} but the earlier display (9) uses Q∩S; the notation should be harmonized.","section":"Definition 2"},{"comment":"The text refers to Figures 2–4, but in the compiled manuscript the captions and axis labels are garbled; please ensure they render correctly.","section":"Figures 2–4"}],"recommendation":"reject","confidential_remarks":"The reject recommendation is driven by the false dyadic count in Lemma 3. If the authors correct the count, the rates become dimension-dependent; they should then carefully compare with classical nonlinear approximation results and adjust the abstract's \"optimal\" and \"dimension-free\" claims. The pseudodimension analysis (Lemma 6) appears independent of the approximation theorem and might be salvageable."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the main theorem is not supported. The proof of Lemma 3, Step 4, counts the d-dimensional dyadic coefficient set Λ(k) as if it had at most 2^k elements; in reality it can have up to 2^{kd}. That count is what produces the width O(ε^{1/(α-s)}) and parameter count O(d² ε^{1/(α-s)}). With the correct count, you get ε^{d/(α-s)} and the dimension enters the exponent. Since the same step feeds Lemmas 4 and 5 and Theorem 1, the advertised rates and the 'dimension-free' phrasing don't follow. The stress-test note is accurate.\n\nWhat's actually good: the paper gives an explicit, exact construction of cardinal B-spline tensor products inside Res-KANs, with clean parameter counts for the multiplication gadget. That is a real contribution and could be reused. The idea of approximating in weaker Besov norms (not just Lp or uniform) is timely for PDE applications, and the pseudodimension analysis is a reasonable transfer of known VC/fat-shattering techniques. The experiments are honest sanity checks, nothing more.\n\nOther soft spots: Theorem 1 states a target regularity r that never appears in the statement, and Definition 1's smoothing layer is not formally defined. These are secondary and fixable; the dyadic count is not.\n\nIs the paper a serious thinker? Yes — the overall strategy is clear and the literature engagement is honest. The error is a concrete miscalculation, not hand-waving. But it is load-bearing.\n\nRecommendation: if this lands on my desk, I'd send it to a referee who knows Besov/wavelet theory in the hope that revision could correct the rates — the spline-implementation part deserves to be preserved. But as submitted, the central claims should not be accepted. If the journal desk-rejects on the basis of the counting error, that's defensible too; the onus is on the authors to fix the bound and re-state the theorem.","headline":"The paper's Besov approximation and sample-complexity results are undone by a false dyadic-counting step in Lemma 3, though the spline-implementation machinery is genuinely useful.","tokens_in":23025,"tokens_out":1986,"would_cite":false,"duration_ms":17944,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["41A25","41A15","46E35","68T07"],"pacs":[],"model":"deepseek-v4-flash","headline":"Residual KANs approximate Besov functions at optimal rates and learn them with dimension-free sample complexity, this paper claims.","keywords":["Kolmogorov-Arnold networks","Besov spaces","approximation rates","residual connections","pseudodimension","sample complexity","spline activations","fractal domains"],"falsifier":"Take $d=2$ and a target function whose spline projection has nonzero coefficients on every dyadic square at every scale, for instance the tensor exponential $e^{x_1+x_2}$ in the B-spline basis. At scale $k$ there are $2^{2k}$ coefficients, not $2^k$, so repeating Lemma 3's truncation with the correct count replaces the claimed $\\varepsilon^{1/(\\alpha-s)}$ width and parameter rates by expressions involving $\\varepsilon^{2/(\\alpha-s)}$; observing this in the paper's own Step 4 calculation would settle the discrepancy.","tokens_in":2097,"feed_emoji":"🧮","tokens_out":2208,"duration_ms":102801,"temperature":0.7,"pith_summary":"This paper claims that residual Kolmogorov–Arnold networks, meaning KANs with trainable spline activations plus a skip connection, can approximate every function in a Besov space, a class defined by quantified smoothness, to any desired accuracy in any weaker Besov norm. The stated cost is width $O(\\varepsilon^{1/(\\alpha_\\star-s)})$, depth $O(d)$, and $O(d^2\\varepsilon^{1/(\\alpha_\\star-s)})$ nonzero parameters, where $\\alpha_\\star$ accounts for the domain type. The paper further claims a pseudodimension bound for Res-KANs that yields sample complexity $N=O(\\varepsilon^{-2-(d+1)/\\alpha}\\log^2(1/\\varepsilon)+\\varepsilon^{-2}\\log(1/\\delta))$ for learning unit-Besov functions from $N$ i.i.d. samples with additive centered noise. If correct, KANs match the optimal approximation and learning rates already established for ReLU MLPs and transformers, so any practical KAN advantage would have to come from optimization rather than expressive power.","feed_headline":"KANs match optimal Besov approximation and learning rates","feed_subtitle":"New proof gives residual KANs the same approximation and sample-complexity guarantees as MLPs and transformers.","key_machinery":"The load-bearing object is the residual KAN layer $L(x)=\\sigma_{\\beta:I}(Ax+b)+Gx$, with a diagonal gating matrix $G$ and activations drawn from cardinal B-splines $N_I$ plus $\\operatorname{ReLU}$ and $\\operatorname{ReLU}^2$. The identity $q(t)=\\operatorname{ReLU}(t)^2+\\operatorname{ReLU}(-t)^2=t^2$ yields an exact $d$-fold multiplication gadget, and that gadget lets the network reproduce the tensor-product splines $N_{j,k:I}(x)=N_I(2^kx-j)$ exactly. With these splines as building blocks, the proof invokes the Besov-characterization theorem that expresses $B^s_{p,q}$ membership as decay of spline coefficients, truncates the expansion, and uses extension operators for $(\\epsilon,\\delta)$-domains and for Ahlfors-regular sets to reach the general domains. The learning guarantee runs through a fat-shattering-dimension bound obtained by viewing Res-KANs as piecewise-polynomial feedforward networks and applying standard VC-dimension and covering-number bounds.","core_discovery":"The paper's central result, Theorem 1, states that for any $f\\in B^{\\alpha_\\star}_{p,q}(\\mathcal{X})$ with $\\mathcal{X}$ a bounded Lipschitz domain or an Ahlfors $n$-regular fractal subset of $[0,1]^d$, and for any target error $\\varepsilon>0$, there is a Res-KAN $\\hat{f}$ whose restriction to $\\mathcal{X}$ is $\\varepsilon$-close to $f$ in the weaker Besov norm $B^\\alpha_{p,q}(\\mathcal{X})$, with the stated width, depth, and parameter counts. On fractal domains the smoothness parameter shifts to $\\alpha_\\star=\\alpha-(n-d)/p$. Theorem 2 complements this by bounding the pseudodimension of the Res-KAN hypothesis class and deriving the quoted sample complexity for learning in the unit ball of a Besov space on a Lipschitz domain. The proof mechanism is explicit: a square gadget $q(t)=\\operatorname{ReLU}(t)^2+\\operatorname{ReLU}(-t)^2=t^2$ builds an exact $d$-fold multiplier, which then reproduces tensor-product cardinal B-splines exactly, and extension operators transport the unit-cube construction to regular and fractal domains.","pith_inferences":["Editorial: The theorem's stated width and parameter counts rest on the proof's assumption that each scale $k$ contributes at most $2^k$ spline coefficients; a dense $d$-dimensional expansion would contribute up to $2^{kd}$, so a corrected count would change the quoted $\\varepsilon^{1/(\\alpha_\\star-s)}$ factors unless sparsity is imposed.","Editorial: The construction relies on exact reproduction of B-splines through ReLU-polynomial identities, so the same proof template would likely extend to any activation basis admitting an exact ReLU-polynomial representation; this points to a testable comparison of KAN variants with non-spline bases.","Editorial: A direct empirical check of the paper's negative message would be to compare training trajectories of KANs and MLPs on the same Besov regression tasks, measuring whether any gap in final error comes from optimization rather than from the function class."],"forward_implications":["Res-KANs achieve the optimal approximation rates in Besov norms on Lipschitz and fractal domains, putting them on par with ReLU MLPs and transformers from a constructive approximation standpoint.","Because the error is measured in the weaker Besov norm $B^\\alpha_{p,q}$, the approximant also captures $f$'s behavior in lower-order derivatives up to the allowed smoothness, which is useful in PDE and physics-informed settings.","The pseudodimension bound implies a sample complexity for learning unit-Besov functions that is independent of network width and depth and only mildly dimension-dependent in the smoothness exponent, so in the high-smoothness regime the bound does not suffer the full curse of dimensionality.","If these guarantees hold, KANs offer no theoretical advantage over MLPs or transformers in approximation or generalization; the paper concludes that any observed empirical benefit must come from optimization dynamics or inductive bias.","Residual connections do not degrade the approximation or learning rates, so the Res-KAN build can replace the standard KAN without sacrificing these guarantees."],"supporting_citations":[{"why":"Supplies the spline-coefficient decay characterization of Besov spaces that the truncation argument uses.","marker":"[13]"},{"why":"Gives the Besov-space characterization on domains that underpins the unit-cube multiresolution expansion.","marker":"[14]"},{"why":"Provides the cardinal B-spline formula in terms of ReLU powers that the Res-KAN activations implement exactly.","marker":"[32]"},{"why":"Provides the extension operator for $(\\epsilon,\\delta)$-domains used to move from the unit cube to Lipschitz domains.","marker":"[38]"},{"why":"Supplies trace and extension results on Ahlfors-regular sets used for the fractal-domain case.","marker":"[20]"},{"why":"Relates fat-shattering dimensions to uniform convergence, giving the sample-complexity theorem its statistical backbone.","marker":"[2]"},{"why":"Provides the pseudodimension and VC-dimension tools used to bound the fat-shattering dimension of Res-KANs.","marker":"[3]"},{"why":"Connects fat-shattering dimension to covering numbers, used in the generalization-bound proof.","marker":"[5]"}],"fun_headline_variants":["Residual KANs match optimal Besov approximation and sample rates","KANs get optimal Besov rates and dimension-free sample complexity","Proof: KANs hit optimal rates in weaker Besov norms","Exact rates for residual KANs in Besov spaces with sample bounds"],"cache_read_input_tokens":24960,"weakest_assumption_plain":"At each refinement level, the proof assumes at most $2^k$ building blocks are available, but in $d$ dimensions a full grid has $2^{kd}$ small cubes; the theorem's stated network sizes follow exactly from that count.","fun_headline_variants_meta":{"raw":{"variants":["Residual KANs match optimal Besov approximation and sample rates","KANs get optimal Besov rates and dimension-free sample complexity","Proof: KANs hit optimal rates in weaker Besov norms","Exact rates for residual KANs in Besov spaces with sample bounds"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000242,"raw_usage":{"total_tokens":1563,"prompt_tokens":1019,"completion_tokens":544,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":635,"completion_tokens_details":{"reasoning_tokens":467}},"tokens_in":635,"tokens_out":544,"duration_ms":5333,"temperature":1.0,"reasoning_tokens":467,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T11:33:35.472326+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take $d=2$ and a target function whose spline projection has nonzero coefficients on every dyadic square at every scale, for instance the tensor exponential $e^{x_1+x_2}$ in the B-spline basis. At scale $k$ there are $2^{2k}$ coefficients, not $2^k$, so repeating Lemma 3's truncation with the correct count replaces the claimed $\\varepsilon^{1/(\\alpha-s)}$ width and parameter rates by expressions involving $\\varepsilon^{2/(\\alpha-s)}$; observing this in the paper's own Step 4 calculation would settle the discrepancy.","supporting_citations":[{"cited_title":"A., and Popov, V","cited_arxiv_id":null,"evidence_quote":"Supplies the spline-coefficient decay characterization of Besov spaces that the truncation argument uses."},{"cited_title":"A., and Sharpley, R","cited_arxiv_id":null,"evidence_quote":"Gives the Besov-space characterization on domains that underpins the unit-cube multiresolution expansion."},{"cited_title":"N., and Micchelli, C","cited_arxiv_id":null,"evidence_quote":"Provides the cardinal B-spline formula in terms of ReLU powers that the Res-KAN activations implement exactly."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the extension operator for $(\\epsilon,\\delta)$-domains used to move from the unit cube to Lipschitz domains."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies trace and extension results on Ahlfors-regular sets used for the fractal-domain case."},{"cited_title":"Scale-sensitive dimensions, uniform convergence, and learnability","cited_arxiv_id":null,"evidence_quote":"Relates fat-shattering dimensions to uniform convergence, giving the sample-complexity theorem its statistical backbone."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the pseudodimension and VC-dimension tools used to bound the fat-shattering dimension of Res-KANs."},{"cited_title":"L., Kulkarni, S","cited_arxiv_id":null,"evidence_quote":"Connects fat-shattering dimension to covering numbers, used in the generalization-bound proof."}],"review_version":1}