Pith. sign in

REVIEW 4 major objections 6 minor 1 cited by

Approximation Rates in Besov Norms and Sample-Complexity of Kolmogorov-Arnold Networks with Residual Connections

T0 review · 4 major / 6 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Residual KANs approximate Besov functions at optimal rates and learn them with dimension-free sample complexity, this paper claims.

desk verdict The paper's Besov approximation and sample-complexity results are undone by a false dyadic-counting step in Lemma 3, though the spline-implementation machinery is genuinely useful. read the letter →

arxiv 2504.15110 v3 pith:IP74UOYX submitted 2025-04-21 cs.LG cs.NAcs.NEmath.FAmath.NAstat.ML

Anastasis Kratsios , Bum Jun Kim , Takashi Furuya This is my paper · ORCID
classification cs.LGcs.NAcs.NEmath.FAmath.NAstat.ML MSC 41A2541A1546E3568T07
keywords Kolmogorov-ArnoldnetworksBesovspacesapproximationratesresidualconnectionspseudodimensionsamplecomplexitysplineactivationsfractaldomains
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that residual Kolmogorov–Arnold networks, meaning KANs with trainable spline activations plus a skip connection, can approximate every function in a Besov space, a class defined by quantified smoothness, to any desired accuracy in any weaker Besov norm. The stated cost is width $O(\varepsilon^{1/(\alpha_\star-s)})$, depth $O(d)$, and $O(d^2\varepsilon^{1/(\alpha_\star-s)})$ nonzero parameters, where $\alpha_\star$ accounts for the domain type. The paper further claims a pseudodimension bound for Res-KANs that yields sample complexity $N=O(\varepsilon^{-2-(d+1)/\alpha}\log^2(1/\varepsilon)+\varepsilon^{-2}\log(1/\delta))$ for learning unit-Besov functions from $N$ i.i.d. samples with additive centered noise. If correct, KANs match the optimal approximation and learning rates already established for ReLU MLPs and transformers, so any practical KAN advantage would have to come from optimization rather than expressive power.

What carries the argument

The load-bearing object is the residual KAN layer $L(x)=\sigma_{\beta:I}(Ax+b)+Gx$, with a diagonal gating matrix $G$ and activations drawn from cardinal B-splines $N_I$ plus $\operatorname{ReLU}$ and $\operatorname{ReLU}^2$. The identity $q(t)=\operatorname{ReLU}(t)^2+\operatorname{ReLU}(-t)^2=t^2$ yields an exact $d$-fold multiplication gadget, and that gadget lets the network reproduce the tensor-product splines $N_{j,k:I}(x)=N_I(2^kx-j)$ exactly. With these splines as building blocks, the proof invokes the Besov-characterization theorem that expresses $B^s_{p,q}$ membership as decay of spline coefficients, truncates the expansion, and uses extension operators for $(\epsilon,\delta)$-domains and for Ahlfors-regular sets to reach the general domains. The learning guarantee runs through a fat-shattering-dimension bound obtained by viewing Res-KANs as piecewise-polynomial feedforward networks and applying standard VC-dimension and covering-number bounds.

What would settle it

Take $d=2$ and a target function whose spline projection has nonzero coefficients on every dyadic square at every scale, for instance the tensor exponential $e^{x_1+x_2}$ in the B-spline basis. At scale $k$ there are $2^{2k}$ coefficients, not $2^k$, so repeating Lemma 3's truncation with the correct count replaces the claimed $\varepsilon^{1/(\alpha-s)}$ width and parameter rates by expressions involving $\varepsilon^{2/(\alpha-s)}$; observing this in the paper's own Step 4 calculation would settle the discrepancy.

Watch

Extended reading notes

Core claim

The paper's central result, Theorem 1, states that for any $f\in B^{\alpha_\star}_{p,q}(\mathcal{X})$ with $\mathcal{X}$ a bounded Lipschitz domain or an Ahlfors $n$-regular fractal subset of $[0,1]^d$, and for any target error $\varepsilon>0$, there is a Res-KAN $\hat{f}$ whose restriction to $\mathcal{X}$ is $\varepsilon$-close to $f$ in the weaker Besov norm $B^\alpha_{p,q}(\mathcal{X})$, with the stated width, depth, and parameter counts. On fractal domains the smoothness parameter shifts to $\alpha_\star=\alpha-(n-d)/p$. Theorem 2 complements this by bounding the pseudodimension of the Res-KAN hypothesis class and deriving the quoted sample complexity for learning in the unit ball of a Besov space on a Lipschitz domain. The proof mechanism is explicit: a square gadget $q(t)=\operatorname{ReLU}(t)^2+\operatorname{ReLU}(-t)^2=t^2$ builds an exact $d$-fold multiplier, which then reproduces tensor-product cardinal B-splines exactly, and extension operators transport the unit-cube construction to regular and fractal domains.

Load-bearing premise

At each refinement level, the proof assumes at most $2^k$ building blocks are available, but in $d$ dimensions a full grid has $2^{kd}$ small cubes; the theorem's stated network sizes follow exactly from that count.

Editorial extensions

If this is right

  • Res-KANs achieve the optimal approximation rates in Besov norms on Lipschitz and fractal domains, putting them on par with ReLU MLPs and transformers from a constructive approximation standpoint.
  • Because the error is measured in the weaker Besov norm $B^\alpha_{p,q}$, the approximant also captures $f$'s behavior in lower-order derivatives up to the allowed smoothness, which is useful in PDE and physics-informed settings.
  • The pseudodimension bound implies a sample complexity for learning unit-Besov functions that is independent of network width and depth and only mildly dimension-dependent in the smoothness exponent, so in the high-smoothness regime the bound does not suffer the full curse of dimensionality.
  • If these guarantees hold, KANs offer no theoretical advantage over MLPs or transformers in approximation or generalization; the paper concludes that any observed empirical benefit must come from optimization dynamics or inductive bias.
  • Residual connections do not degrade the approximation or learning rates, so the Res-KAN build can replace the standard KAN without sacrificing these guarantees.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial: The theorem's stated width and parameter counts rest on the proof's assumption that each scale $k$ contributes at most $2^k$ spline coefficients; a dense $d$-dimensional expansion would contribute up to $2^{kd}$, so a corrected count would change the quoted $\varepsilon^{1/(\alpha_\star-s)}$ factors unless sparsity is imposed.
  • Editorial: The construction relies on exact reproduction of B-splines through ReLU-polynomial identities, so the same proof template would likely extend to any activation basis admitting an exact ReLU-polynomial representation; this points to a testable comparison of KAN variants with non-spline bases.
  • Editorial: A direct empirical check of the paper's negative message would be to compare training trajectories of KANs and MLPs on the same Besov regression tasks, measuring whether any gap in final error comes from optimization rather than from the function class.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper studies residual Kolmogorov-Arnold networks (Res-KANs), whose activations are cardinal B-splines plus ReLU and ReLU², with residual connections, and analyzes their approximation and learning properties. The main approximation claim (Theorem 1) is that for any f ∈ B^s_{p,q}(X) on a bounded extension domain or an Ahlfors-regular fractal domain, there is a Res-KAN fhat with width O(ε^{1/(α⋆-s)}), depth O(d), and O(d² ε^{1/(α⋆-s)}) nonzero parameters such that ∥f - fhat|X∥_{B^α_{p,q}(X)} < ε for any α < s. The proof builds on an exact d-fold multiplication gadget (Lemma 1), emulates tensor-product cardinal B-splines (Lemma 2), invokes a spline characterization of Besov spaces (Lemma 3), and then transfers the result to Lipschitz and fractal domains via extension operators (Lemmas 4 and 5). The paper also proves a pseudodimension bound for Res-KANs (Lemma 6) and derives a PAC sample-complexity bound for noiseless learning (Theorem 2), and it reports toy numerical experiments showing that residual connections do not degrade accuracy.

Significance. The architecture-level construction is a strength: Lemma 1 gives an explicit residual-KAN implementation of d-fold multiplication with exact parameter counts, and Lemma 2's exact emulation of cardinal B-splines is a clean and useful observation. If Theorem 1 were proved, the paper would provide a natural spline-based explanation of KAN approximation in weaker Besov norms, with derivative control relevant to PDE applications, and would place KANs on the same footing as ReLU networks in constructive approximation theory. The numerical sanity checks are appropriate for a theory paper. However, the central approximation-rate claim rests on a coefficient-counting argument that is false in dimension d > 1, and the advertised dimension-free rates are therefore not established; the remaining statements also contain internal inconsistencies that prevent the theorem from being read as a valid mathematical claim.

major comments (4)
  1. [Appendix A, Lemma 3, Step 4] Step 4 estimates ∑_{k=1}^{K}∑_{j∈Λ(k)}(d+4) ≤ (d+4)∑_{k=1}^{K}2^{k}, treating the index set Λ(k) as if it had at most 2^k elements. At scale k, Λ(k) indexes the nonzero spline coefficients over dyadic cubes in [0,1]^d; a general f ∈ B^s_{p,q}([0,1]^d) can have coefficients on a full d-dimensional dyadic lattice, so |Λ(k)| can be as large as 2^{kd}. The decay estimate (16) bounds the weighted ℓ_p sums of the coefficients, but it does not bound the support size. With the correct cardinality, the width and parameter bounds become O(2^{dK}), and substituting K ∼ log_2 ε^{1/(α-s)} yields O(ε^{-d/(s-α)}) rather than O(ε^{-1/(s-α)}). Since Lemma 4, Lemma 5, and Theorem 1 inherit the same count, the claimed dimension-free approximation rates are not proved.
  2. [Theorem 1] The statement is internally inconsistent. It introduces α⋆ = α or α⋆ = α − (n−d)/p, then asserts f ∈ B^{α⋆}_{p,q}(X), whereas the proof in Lemmas 4 and 5 starts from f ∈ B^s_{p,q}(X) and measures the error in B^α_{p,q} (resp. B^{α−(n−d)/p}_{p,q}) norms. It also states that the approximant belongs to B^r_{p,q}(X) for an r that is never defined. The theorem must specify the source regularity s, the target norm, and the role of α⋆ precisely.
  3. [Definition 1 and Section 2.2] Section 2.2 promises that a smoothing layer is added at the output so that "any function implemented by our smoothed residual KANs is necessarily smooth", but Definition 1 contains no such layer: it only imposes the sparsity rule (6) on the first I+1 rows of β. The networks used in Lemmas 1 and 2 explicitly activate ReLU and ReLU², which are not smooth, so the constructed approximants are not C∞ and the assertion fhat ∈ B^r_{p,q}(X) in Theorem 1 is unsupported. Either the smoothing layer must be formally added to Definition 1 and its effect on the approximation analysis justified, or the smoothness requirement must be changed.
  4. [Theorem 2 and Abstract] The abstract and introduction call the sample-complexity result "dimension-free", but Theorem 2 states N ∈ O(ε^{-2-(d+1)/α} (ln(1/ε))² + ε^{-2} ln(1/δ)), with the dimension d appearing in the exponent. This is not dimension-free in the standard sense, and the claim should be replaced by a precise statement of what does not depend on d.
minor comments (6)
  1. [Equation (9)] The symbol S in the normalized best-polynomial approximation energy E_k(f,Q)_{L_p(X)} is never defined; presumably it should be X (or Q∩X).
  2. [Theorem 2] In the statement of Theorem 2, "each centered Borel probability measure Psmpl ∈ P(R)" should read Perr ∈ P(R).
  3. [Lemma 3, Step 2] The index set in the sequence of networks is written as j ∈ Λ(j); it should be j ∈ Λ(k).
  4. [Lemma 5 proof] The extension operator is written as E: B^{s-(n-d)/p}_{p,q}(X) → B^s_{p,q}(R^d), which appears to have domain and codomain reversed relative to its use on f ∈ B^s_{p,q}(X); please check the cited [20, Theorem 6.1].
  5. [Definition 2] In the fractal Besov definition, the integrand uses ∥E_k(f,Q(·,τ))∥_{L_u(X)} but the earlier display (9) uses Q∩S; the notation should be harmonized.
  6. [Figures 2–4] The text refers to Figures 2–4, but in the compiled manuscript the captions and axis labels are garbled; please ensure they render correctly.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the derivation reduces to external spline-characterization and extension theorems, not to its own outputs.

full rationale

I walked the derivation chain from Theorem 1 back to its inputs. Lemma 3 constructs an approximator by (i) proving in Lemmas 1–2 that a Res-KAN can exactly realize d-fold multiplication and the tensor-product cardinal B-splines N_{j,k:I}, and (ii) invoking the external spline characterization of Besov spaces in [13, Corollary 5.3] and the coefficient-decay estimates in [42] to truncate the spline expansion and bound the tail in B^α_{p,q}. The parameters of the approximator are exactly the coefficients β^f of that external spline expansion, so the theorem is a consequence of a known external rate plus an implementability lemma; no parameter is fitted to data and then reported as a prediction, and no target result is baked into the definition of the Res-KAN class. Lemmas 4 and 5 import bounded extension operators from [38,40,36,20], again external results; the learning bounds import pseudodimension, fat-shattering, and covering-number bounds from [3,4,5,43,2]. Self-citations ([1], [7], [15], [27]) appear only as background or as a definitional convention for the gated residual layer, not as the load-bearing step of Theorems 1–2; hence none of the enumerated circularity patterns is exhibited. Separate correctness issues exist—most prominently, Lemma 3 Step 4 replaces the d-dimensional dyadic sum over j∈Λ(k) by 2^k, although Λ(k) can contain up to 2^{kd} indices; Theorem 1 also uses an undefined target regularity r, and Definition 1 promises a smoothing layer that representation (5) does not explicitly include—but these are mathematical-validity concerns, not circularity, and they do not raise the circularity score.

Assumptions & free parameters 0 free parameters · 6 assumptions · 0 invented entities

No parameters are fitted to data; the theory uses known spline, extension, and covering-number results. The only introduced object is the Res-KAN architecture itself, which is a variant of existing networks. The smoothing layer mentioned in Section 2.2 is never defined and does not appear in the formal class Res-KAN, so this is an unflagged construction gap rather than an invented entity.

assumptions (6)
  • standard math Spline characterization of Besov spaces: f belongs to B^s_{p,q}([0,1]^d) if and only if f has a spline expansion whose coefficient quasi-norm in (14) is finite.
    Invoked in Lemma 3 through [13, Corollary 5.3] and [42, Theorem 4.22].
  • standard math There is a bounded linear extension operator from Besov spaces on (epsilon,delta)-domains to Besov spaces on R^d.
    Used in Lemma 4 to transfer approximation from the cube to general Lipschitz and (epsilon,delta)-domains via [36] and [38].
  • standard math Whitney-type extension and trace theorems on Ahlfors n-regular sets: extensions map B^{s-(n-d)/p}_{p,q}(X) into B^s_{p,q}(R^d), and restrictions of Besov functions on R^d lie in Besov spaces on X with smoothness reduced by (n-d)/p.
    Used in Lemma 5 with [20, Theorem 6.1 and Theorem 4.1].
  • standard math The unit ball of B^alpha_{p,q}(X times R) has covering number in the uniform norm of order epsilon^{-(d+1)/alpha}.
    Used in Lemma 7 via [43, Theorem 2.7.4] to derive sample complexity.
  • standard math VC-dimension and fat-shattering bounds hold for feedforward piecewise-polynomial networks in terms of weights and layers.
    Used in Lemma 6 via [4, Theorem 2.1] and [3, Theorem 14.1].
  • domain assumption The domain X is an (epsilon,delta)-domain or an Ahlfors n-regular set with d-1 < n < d, and the target functions lie in the corresponding Besov spaces on X.
    Theorem 1 states these as hypotheses, and the proof depends on extension and restriction theorems that require them.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Approximation Rates in Besov Norms and Sample-Complexity of Kolmogorov-Arnold Networks with Residual Connections." pith.science (2026). https://pith.science/paper/IP74UOYX

@misc{pith2026250415110,
  author       = {Pith},
  title        = {Pith review of: Approximation Rates in Besov Norms and Sample-Complexity of Kolmogorov-Arnold Networks with Residual Connections},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IP74UOYX}},
  note         = {Machine review of arXiv:2504.15110}
}
abstract

Inspired by the Kolmogorov-Arnold superposition theorem, Kolmogorov-Arnold Networks (KANs) have recently emerged as an improved backbone for most deep learning frameworks, promising more adaptivity than their multilayer perceptron (MLP) predecessor by allowing for trainable spline-based activation functions. In this paper, we probe the theoretical foundations of the KAN architecture by showing that it can optimally approximate any Besov function in $B^{s}_{p,q}(\mathcal{X})$ on a bounded open, or even fractal, domain $\mathcal{X}$ in $\mathbb{R}^d$ at the optimal approximation rate with respect to any weaker Besov norm $B^{\alpha}_{p,q}(\mathcal{X})$; where $\alpha < s$. We complement our approximation result with a statistical guarantee by bounding the pseudodimension of the relevant class of Res-KANs. As an application of the latter, we directly deduce a dimension-free estimate on the sample complexity of a residual KAN model when learning a function of Besov regularity from $N$ i.i.d. noiseless samples, showing that KANs can learn the smooth maps which they can approximate.

Figures

Figures reproduced from arXiv: 2504.15110 by the authors.

Figure 1
Figure 1. The cardinal B-splines of orders I = 0, 1, and 2. As shown, for example, in [32, Equation (4.28)], for any I ∈ N+, the cardinal B-spline of order I with knots on 0, . . . , I + 1 is given by NI (x) = X I+1 j=0 (−1)j [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Evolution of training loss, test prediction, and derivative approximation quality [PITH_FULL_IMAGE:figures/full_fig_p020_2.png] view at source ↗
Figure 3
Figure 3. Both the KAN and Res-KAN build offer similar approximation efficacy across [PITH_FULL_IMAGE:figures/full_fig_p021_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Two challenging one-dimensional functions to learn - using ReLU and ReLU [PITH_FULL_IMAGE:figures/full_fig_p021_4.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Model Risk in Machine-Learning Distributional IV Estimation

    econ.EM 2025-06 reject novelty 4.0 of 10

    Using random forests versus Kolmogorov-Arnold networks for nuisance functions in a distributional IV estimator can reverse the estimated effect curve, so model choice is a substantive decision.

Reference graph

Works this paper leans on

54 extracted references · 49 canonical work pages · cited by 1 Pith paper

  1. [1]

    Mathematical Finance 34 , 2 (2024), 671– 735

    Acciaio, B., Kratsios, A., and Pammer, G.Designing universal causal deep learning 21 REFERENCES REFERENCES models: The geometric (hyper) transformer. Mathematical Finance 34 , 2 (2024), 671– 735

    work page 2024

  2. [2]

    Scale-sensitive dimensions, uniform convergence, and learnability

    Alon, N., Ben-David, S., Cesa-Bianchi, N., and Haussler, D. Scale-sensitive dimensions, uniform convergence, and learnability. J. ACM 44 , 4 (1997), 615–631

    work page 1997

  3. [3]

    Anthony, M., and Bartlett, P. L. Neural network learning: Theoretical founda- tions. cambridge university press, 2009

    work page 2009

  4. [4]

    Almost linear vc dimension bounds for piecewise polynomial networks

    Bartlett, P., Maiorov, V., and Meir, R. Almost linear vc dimension bounds for piecewise polynomial networks. Advances in neural information processing systems 11 (1998)

    work page 1998

  5. [5]

    L., Kulkarni, S

    Bartlett, P. L., Kulkarni, S. R., and Posner, S. E. Covering numbers for real- valued function classes. IEEE Trans. Inform. Theory 43 , 5 (1997), 1721–1724

    work page 1997

  6. [6]

    Simultaneous approximation of a smooth function and its derivatives by deep neural networks with piecewise-polynomial activations

    Belomestny, D., Naumov, A., Puchkin, N., and Samsonov, S. Simultaneous approximation of a smooth function and its derivatives by deep neural networks with piecewise-polynomial activations. Neural Networks 161 (2023), 242–253

    work page 2023

  7. [7]

    Borde, H. S. d. O., Lukoianov, A., Kratsios, A., Bronstein, M., and Dong, X. Scalable message passing neural networks: No need for attention in large graph representation learning. arXiv preprint arXiv:2411.00835 (2024)

    arXiv 2024

  8. [8]

    Extending sobolev func- tions with partially vanishing traces from locally (ε,δ)-domains and applications to mixed boundary problems

    Brewster, K., Mitrea, D., Mitrea, I., and Mitrea, M. Extending sobolev func- tions with partially vanishing traces from locally (ε,δ)-domains and applications to mixed boundary problems. Journal of Functional Analysis 266 , 7 (2014), 4314–4421

    work page 2014

Show all 54 references
  1. [9]

    Efficient approximation of high- dimensional functions with neural networks

    Cheridito, P., Jentzen, A., and Rossmannek, F. Efficient approximation of high- dimensional functions with neural networks. IEEE Transactions on Neural Networks and Learning Systems 33, 7 (2021), 3079–3093

    2021

  2. [10]

    K., and Wang, J.-Z

    Chui, C. K., and Wang, J.-Z. A general framework of compactly supported splines and wavelets. J. Approx. Theory 71 , 3 (1992), 263–304

    1992

  3. [11]

    A practical guide to splines , revised ed., vol

    de Boor, C. A practical guide to splines , revised ed., vol. 27 of Applied Mathematical Sciences. Springer-Verlag, New York, 2001

    2001

  4. [12]

    On the approximation of functions by tanh neural networks

    De Ryck, T., Lanthaler, S., and Mishra, S. On the approximation of functions by tanh neural networks. Neural Networks 143 (2021), 732–750

    2021

  5. [13]

    A., and Popov, V

    DeVore, R. A., and Popov, V. A. Interpolation of approximation spaces. In Con- structive theory of functions (Varna, 1987) . Publ. House Bulgar. Acad. Sci., Sofia, 1988, pp. 110–119

    1987

  6. [14]

    A., and Sharpley, R

    DeVore, R. A., and Sharpley, R. C. Besov spaces on domains in Rd. Trans. Amer. Math. Soc. 335 , 2 (1993), 843–864

    1993

  7. [15]

    Simultaneously solving fbsdes with neural operators of logarithmic depth, constant width, and sub-linear rank

    Furuya, T., and Kratsios, A. Simultaneously solving fbsdes with neural operators of logarithmic depth, constant width, and sub-linear rank. arXiv preprint arXiv:2410.14788 (2024)

    2024

  8. [16]

    Approxima- tion spaces of deep neural networks

    Gribonval, R., Kutyniok, G., Nielsen, M., and Voigtlaender, F. Approxima- tion spaces of deep neural networks. Constructive approximation 55 , 1 (2022), 259–367. 22 REFERENCES REFERENCES

    2022

  9. [17]

    Deep residual learning for image recognition

    He, K., Zhang, X., Ren, S., and Sun, J. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition(2016), pp. 770–778

    2016

  10. [18]

    Minimum width for deep, narrow mlp: A diffeomorphism approach

    Hwang, G. Minimum width for deep, narrow mlp: A diffeomorphism approach. arXiv preprint arXiv:2308.15873 (2023)

    2023 arXiv

  11. [19]

    Smoothness spaces of higher order on lower dimen- sional subsets of the Euclidean space

    Ihnatsyeva, L., and Korte, R. Smoothness spaces of higher order on lower dimen- sional subsets of the Euclidean space. Math. Nachr. 288 , 11-12 (2015), 1303–1316

    2015

  12. [20]

    Ihnatsyeva, L., and V¨ah¨akangas, A. V. Characterization of traces of smooth func- tions on Ahlfors regular sets. J. Funct. Anal. 265 , 9 (2013), 1870–1915

    2013

  13. [21]

    Z., and Yang, Y

    Jiao, Y., Lai, Y., Lu, X., Wang, F., Yang, J. Z., and Yang, Y. Deep neural networks with relu-sine-exponential activations break curse of dimensionality in approxi- mation on h¨ older class.SIAM Journal on Mathematical Analysis 55, 4 (2023), 3635–3649

    2023

  14. [22]

    Jones, P. W. Quasiconformal mappings and extendability of functions in Sobolev spaces. Acta Mathematica 147 (1981), 71–88

    1981

  15. [23]

    Sur le th´ eor` eme de superposition de Kolmogorov

    Kahane, J.-P. Sur le th´ eor` eme de superposition de Kolmogorov. J. Approximation Theory 13 (1975), 229–234

    1975

  16. [24]

    Polynomial bounds for vc dimension of sigmoidal and general pfaffian neural networks

    Karpinski, M., and Macintyre, A. Polynomial bounds for vc dimension of sigmoidal and general pfaffian neural networks. Journal of Computer and System Sciences 54 , 1 (1997), 169–176

    1997

  17. [25]

    Universal approximation with deep narrow networks

    Kidger, P., and Lyons, T. Universal approximation with deep narrow networks. In Conference on learning theory (2020), PMLR, pp. 2306–2327

    2020

  18. [26]

    Transformers are minimax optimal nonpara- metric in-context learners

    Kim, J., Nakamaki, T., and Suzuki, T. Transformers are minimax optimal nonpara- metric in-context learners. In ICML 2024 Workshop on In-Context Learning (2024)

    2024

  19. [27]

    Universal approximation theorems for differentiable geometric deep learning

    Kratsios, A., and Papon, L. Universal approximation theorems for differentiable geometric deep learning. The Journal of Machine Learning Research 23 , 1 (2022), 8896– 8968

    2022

  20. [28]

    Resnet with one-neuron hidden layers is a universal approx- imator

    Lin, H., and Jegelka, S. Resnet with one-neuron hidden layers is a universal approx- imator. Advances in neural information processing systems 31 (2018)

    2018

  21. [29]

    Y., and Tegmark, M

    Liu, Z., Wang, Y., Vaidya, S., Ruehle, F., Halverson, J., Soljacic, M., Hou, T. Y., and Tegmark, M. KAN: Kolmogorov–arnold networks. In The Thirteenth International Conference on Learning Representations (2025)

    2025

  22. [30]

    Mallat, S. G. Multiresolution approximations and wavelet orthonormal bases of l2. Transactions of the American mathematical society 315 , 1 (1989), 69–87

    1989

  23. [31]

    W., and Xu, J

    Mao, T., Siegel, J. W., and Xu, J. Approximation rates for shallow ReLUk neural networks on sobolev spaces via the radon transform. arXiv preprint arXiv:2408.10996 (2024)

    2024

  24. [32]

    N., and Micchelli, C

    Mhaskar, H. N., and Micchelli, C. A. Approximation by superposition of sigmoidal and radial basis functions. Adv. in Appl. Math. 13 , 3 (1992), 350–373

    1992

  25. [33]

    N., and Poggio, T

    Mhaskar, H. N., and Poggio, T. An analysis of training and generalization errors in shallow and deep networks. Neural Networks 121 (2020), 229–241. 23 REFERENCES REFERENCES

    2020

  26. [34]

    In International Conference on Learning Representations (2021)

    Park, S., Yun, C., Lee, J., and Shin, J.Minimum width for universal approximation. In International Conference on Learning Representations (2021)

    2021

  27. [35]

    H., Balestriero, R., and Baraniuk, R

    Riedi, R. H., Balestriero, R., and Baraniuk, R. G. Singular value perturbation and deep network optimization. Constructive Approximation 57 , 2 (2023), 807–852

    2023

  28. [36]

    Rogers, L. G. Degree-independent sobolev extension on locally uniform domains. Jour- nal of Functional Analysis 235 , 2 (2006), 619–665

    2006

  29. [37]

    Recurrent fourier-kolmogorov arnold networks for photovoltaic power forecasting

    Rong, D., Lin, Z., and Xie, G. Recurrent fourier-kolmogorov arnold networks for photovoltaic power forecasting. Scientific Reports 15, 1 (2025), 4684

    2025

  30. [38]

    Rychkov, V. S. On restrictions and extensions of the Besov and Triebel-Lizorkin spaces with respect to Lipschitz domains. J. London Math. Soc. (2) 60 , 1 (1999), 237–257

    1999

  31. [39]

    Glu variants improve transformer.arXiv preprint arXiv:2002.05202 (2020)

    Shazeer, N. Glu variants improve transformer.arXiv preprint arXiv:2002.05202 (2020)

    2020 arXiv

  32. [40]

    New estimates of Rychkov’s universal extension operator for Lipschitz domains and some applications

    Shi, Z., and Yao, L. New estimates of Rychkov’s universal extension operator for Lipschitz domains and some applications. Math. Nachr. 297 , 4 (2024), 1407–1443

    2024

  33. [41]

    Adaptivity of deep relu network for learning in besov and mixed smooth besov spaces: optimal rate and curse of dimensionality

    Suzuki, T. Adaptivity of deep relu network for learning in besov and mixed smooth besov spaces: optimal rate and curse of dimensionality. In ICLR (2019)

    2019

  34. [42]

    Theory of function spaces

    Triebel, H. Theory of function spaces. III , vol. 100 of Monographs in Mathematics . Birkh¨ auser Verlag, Basel, 2006

    2006

  35. [43]

    W., Wellner, J

    Van Der Vaart, A. W., Wellner, J. A., van der Vaart, A. W., and Wellner, J. A. Weak convergence. Springer, 1996

    1996

  36. [44]

    N., Kaiser, L., and Polosukhin, I

    Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., and Polosukhin, I. Attention is all you need. Advances in neural infor- mation processing systems 30 (2017)

    2017

  37. [45]

    Don’t fear peculiar activation functions: Euaf and beyond

    Wang, Q., Zhang, S., Zeng, D., Xie, Z., Guo, H., Zeng, T., and Fan, F.-L. Don’t fear peculiar activation functions: Euaf and beyond. Neural Networks (2025), 107258

    2025

  38. [46]

    W., Liu, Z., and Hou, T

    Wang, Y., Siegel, J. W., Liu, Z., and Hou, T. Y.On the expressiveness and spectral bias of KANs. In The Thirteenth International Conference on Learning Representations (2025)

    2025

  39. [47]

    Graph attention and kolmogorov–arnold network based smart grids intrusion detection

    Wu, Y., Zang, Z., Zou, X., Luo, W., Bai, N., Xiang, Y., Li, W., and Dong, W. Graph attention and kolmogorov–arnold network based smart grids intrusion detection. Scientific Reports 15, 1 (2025), 8648

    2025

  40. [48]

    Optimal rates of approximation by shallow ReLU k neu- ral networks and applications to nonparametric regression

    Yang, Y., and Zhou, D.-X. Optimal rates of approximation by shallow ReLU k neu- ral networks and applications to nonparametric regression. Constructive Approximation (2024), 1–32

    2024

  41. [49]

    Optimal approximation of continuous functions by very deep relu net- works

    Yarotsky, D. Optimal approximation of continuous functions by very deep relu net- works. In Conference on learning theory (2018), PMLR, pp. 639–649

    2018

  42. [50]

    Elementary superexpressive activations

    Yarotsky, D. Elementary superexpressive activations. In International conference on machine learning (2021), PMLR, pp. 11932–11940. 24 REFERENCES REFERENCES

    2021

  43. [51]

    Deep network approximation: Beyond relu to diverse activation functions

    Zhang, S., Lu, J., and Zhao, H. Deep network approximation: Beyond relu to diverse activation functions. Journal of Machine Learning Research 25 , 35 (2024), 1–39

    2024

  44. [52]

    Deep network approximation: Achieving arbitrary accuracy with fixed number of neurons

    Zhang, S., Shen, Z., and Yang, H. Deep network approximation: Achieving arbitrary accuracy with fixed number of neurons. Journal of Machine Learning Research 23 , 276 (2022), 1–60

    2022

  45. [53]

    Neural network architecture beyond width and depth

    Zhang, S., Shen, Z., and Yang, H. Neural network architecture beyond width and depth. Advances in Neural Information Processing Systems 35 (2022), 5669–5681

    2022

  46. [54]

    Physics-informed neu- ral networks with hybrid kolmogorov-arnold network and augmented lagrangian function for solving partial differential equations

    Zhang, Z., Wang, Q., Zhang, Y., Shen, T., and Zhang, W. Physics-informed neu- ral networks with hybrid kolmogorov-arnold network and augmented lagrangian function for solving partial differential equations. Scientific Reports 15, 1 (2025), 10523. 25

    2025

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.