Pith. sign in

REVIEW 3 major objections 2 minor 1 cited by

The paper establishes a new C^1 approximation theorem: rational functions and rational neural networks of moderate degree can approximate a smooth function together with its first derivatives at rates that match or improve on polynomial app

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

C^1 approximation of Hölder functions by rational functions and rational neural networks is proven with rate N^{-(β-1)}, via a new rational approximation of ReQU.

T0 review reviewed 2026-08-05 challenge →

load-bearing objection Genuinely useful ReQU approximation result, but the main theorem misstates width dependence: width is Θ(N^d), not O(N). the 3 major comments →

arxiv 2508.19672 v1 pith:C3MHRSUX submitted 2025-08-27 cs.LG cs.ITcs.NAmath.ITmath.NA

$\mathcal{C}^1$-approximation with rational functions and rational neural networks

Erion Morina , Martin Holler This is my paper
classification cs.LG cs.ITcs.NAmath.ITmath.NA MSC 33F0541A2041A2526C15
keywords rational functionsrational neural networksC^1 approximationhigher-order approximationReQU activationNewman polynomialsEQL^div architecturephysical law learning
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper shows that functions with Hölder smoothness β>2 on [0,1]^d can be approximated, together with their first derivatives, by rational functions and by rational neural networks. The rational approximants have degree roughly N^{d+ε} and achieve C^1 error O(N^{−(β−1)}); the networks have width O(N), constant depth, and rational activation degree O(N^ε) at the same rate. The engine is a new rational approximation of the ReQU activation: Newman-polynomial-based rational functions of type (n+1,n−1) approximate ReQU with value error O(e^{−√n}) and derivative error O(n^{−(K−2)/2}) for any chosen K≥3. Because the underlying spline/ReQU network already attains higher-order approximation rates, replacing each ReQU activation by such rational functions transfers the rate. The result also yields C^1 approximation for the EQL^div and ParFam symbolic-regression architectures used in physical law learning.

Core claim

Corollary 6 is the key new technical result: the rectified quadratic unit ReQU(x)=max(x,0)^2 is approximated in C^1([−1,1]) by rational functions of type (n+1,n−1) with error O(e^{−√n}) for values and O(n^{−(K−2)/2}) for derivatives, for any prescribed K≥3. The approximants are R_n(x)=x^2(P_n(x)−P_n(−x))/(P_n(x)+P_n(−x)), with Newman polynomials P_n(x)=∏_{i=0}^{n−1}(x+e^{−i/√n}); this rational sign-function approximates x^2 sgn(x), and ReQU is half of x^2 plus x^2 sgn(x). The paper then replaces ReQU activations in the spline-based ReQU networks of [2], yielding constant-depth rational neural networks of width O(N) and rational degree O(N^ε) that approximate every f∈H^β([0,1]^d), β>2, in C^1

What carries the argument

Newman-polynomial rational approximants. The paper constructs R_n(x)=x^2(P_n(x)−P_n(−x))/(P_n(x)+P_n(−x)) using Newman polynomials P_n(x)=∏_{i=0}^{n−1}(x+e^{−i/√n}). The exponentially spaced roots make the ratio (P_n(x)−P_n(−x))/(P_n(x)+P_n(−x)) approximate the sign function, so x^2 times it approximates x^2 sgn(x), and ReQU is recovered as half the sum of x^2 and x^2 sgn(x). The derivative estimate O(n^{−(K−2)/2}) follows by controlling x^2 r_n′(x) through the sum over the Newman roots, using a density lemma for those roots. Inserting this rational C^1 approximant into the B-spline representation of the ReQU network from [2] transfers the higher-order approximation rate to rational function

Load-bearing premise

The proof inherits the tensor-product spline approximation bound from [2] (equation (16)): every function in the Hölder class H^β with β>2 is within c N^{−(β−1)} of a spline in C^1, with bounded spline coefficients; if that bound is false or requires stronger smoothness, the main rates do not follow.

What would settle it

Take n=10^4 and K=3, compute R_n from Lemma 5 explicitly, and evaluate max_x |ReQU′(x)−R_n′(x)| on a fine grid covering [−1,1]. If the maximum does not shrink like n^{−1/2} while the value error remains O(e^{−√n}), the core derivative estimate fails. Alternatively, for a fixed smooth f with known spline error, construct the network of Theorem 11 with M=N^ε and check whether the C^1 error decays as N^{−(β−1)}; any slower exponent would show the transfer step, not merely the ReQU lemma, is wrong.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • For every f in the Hölder class H^β([0,1]^d) with β>2, rational functions of degree O(N^{d+ε}) approximate f and its first derivatives uniformly with error O(N^{−(β−1)}), so rational approximation preserves first-order smoothness at a rate comparable to spline approximation.
  • Rational neural networks with width O(N), constant depth, and rational activation degree O(N^ε) achieve the same C^1 rate, showing that using rational activations does not sacrifice first-order accuracy.
  • For ReQU itself, the rational approximants attain derivative error of arbitrarily high polynomial order, whereas polynomial C^1 approximants are limited to rate n^{−1}; this gap is what makes the rational transfer possible.
  • The EQL^div and ParFam symbolic-regression architectures inherit C^1 approximation: under mild regularity on the activation functions and their inverses, they can match both function values and gradients, which is directly relevant to physical law learning.
  • If the activation inverses satisfy Hölder regularity with β>2, the pointwise C^1 approximation holds; under global Lipschitz assumptions, the approximation is uniform on [0,1]^d.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The construction suggests a general transfer principle: any higher-order universal approximator whose activation can be C^1-approximated by rationals likely inherits rational C^1 approximation at the same rate; the authors note this for other Sobolev and ReLU/ReQU results, and extending it to W^{m,p} norms would be a natural next step.
  • If the same Newman-polynomial mechanism can control higher derivatives of x^2 r_n(x), the argument would extend to C^k approximation for k≥2, which the paper explicitly leaves open.
  • The symbolic-regression result in Lemma 16 currently proves pointwise convergence by effectively using one rational layer to cancel the activations; a rate-preserving version would need to exploit the non-rational activations directly, and the paper's Remark 19 flags this as the weak point of the proof strategy.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 2 minor

Summary. The paper studies C^1-approximation by rational functions and rational neural networks. In Section 2, it constructs Newman-polynomial-based rational approximations of the ReQU activation: Corollary 6 states rational functions of type (n+1,n-1) approximating ReQU in C^1([-1,1]) with error O(e^{-√n}) in the function value and O(n^{-(K-2)/2}) in the derivative, for every fixed K≥3. In Section 3, the paper combines this ReQU approximation with the tensor-product-spline/ReQU-network results of Belomestny et al. [2]. Theorem 11 claims rational neural networks of width O(N), constant depth, and activation degree O(N^ε) approximate every f∈H^β_p([0,1]^d) in C^1 with error O(N^{-(β-1)}). Corollary 12 transfers this to a rational-function approximation result with degree O(N^{d+ε}). Finally, Lemma 16 and Corollary 18 derive pointwise C^1-approximation statements for the ParFam and EQL^÷-type symbolic-regression architectures by composing rational approximations of the inverse activations.

Significance. The genuinely new technical contribution is Lemma 5 / Corollary 6: a careful, constructive proof that ReQU can be approximated in C^1 by rational functions with exponential uniform error and arbitrarily high polynomial derivative rate. This part appears sound and is a useful addition to the rational-approximation literature. If Theorem 11 were correct as stated, the paper would also show a dimension-robust width-rate trade-off for rational neural networks. Unfortunately, the proof establishes only a different statement: the constructed width is Θ(N^d), so the claimed O(N) width with O(N^{-(β-1)}) error is unsupported. With an honest restatement, the paper still contains a valid C^1-approximation transfer from ReQU spline networks to rational networks, but with a dimension-dependent rate in terms of width.

major comments (3)
  1. [Theorem 11 and its proof (§3.1, design of rational neural network)] The theorem states that the rational neural network has width of order N and error O(N^{-(β-1)}). But the construction in the proof is layer-by-layer with widths (N+1)^d, (N+5)^d, (N+3)^d, and (N+m+1)^d, and the output layer sums over (⌊β⌋+N)^d tensor-product coefficients. The paper explicitly assigns the network width (N+max(4,⌊β⌋)+1)^d. Hence the width is Θ(N^d), not O(N). The N in the error estimate is a per-dimension knot parameter in the spline bound (16), not the network width. If W denotes the network width, the proof yields, after substituting W=N^d, an error O(W^{-(β-1)/d}) up to the N^ε factor, not O(N^{-(β-1)}). This is a load-bearing quantitative discrepancy and the main claim of Theorem 11 is therefore not proven as stated.
  2. [Equations (13)–(16), proof of Theorem 11] The quoted result of [2] is stated in (13) as error cN^{(l-β)/d}, so for l=1 it gives C^1 error cN^{-(β-1)/d}. The proof then uses the spline bound (16) with cN^{-(β-1)}, silently changing the interpretation of N from a width parameter to a per-dimension knot count. If (16) is meant as a per-dimension spline bound, that N is not the network width, and the subsequent layer widths must be expressed in terms of N^d. If, instead, N is meant to remain the width, then (16) should contain the factor 1/d. The two roles of N must be separated; otherwise the rates in Theorem 11 and in the derived Corollary 12 are not justified.
  3. [Lemma 5 / Corollary 6 (type count)] The stated type (n+1,n-1) appears inconsistent with the construction. For example, for n=2, P_2(x)+P_2(-x)=2(x^2+ξ) and P_2(x)-P_2(-x)=2(1+ξ)x, so R_2(x)=x^2 r_2(x) has numerator degree 3 and denominator degree 2, i.e. type (3,2), not (3,1). More generally, the parity of n changes the degree of P_n(x)+P_n(-x). Since the remainder of the paper only uses the order n of the degree, this does not affect the main rates, but the type statement in Lemma 5 and Corollary 6 should be corrected.
minor comments (2)
  1. [Section 3.2 / Lemma 16] The abstract and introduction describe 'C^1-approximation results' for EQL^÷ and ParFam, but Lemma 16 only establishes pointwise convergence, not uniform approximation, and no rates are given. Moreover, the proof cancels the activation functions by approximating their inverses; this is acknowledged in Remark 19. The claims in the abstract should be phrased more cautiously.
  2. [Introduction / Section 3.1] There are several typographical infelicities: 'with with respect to' in the Introduction, and 'We no consider' at the start of Section 3.1. In Definition 10 the indexing of the coefficients is difficult to parse and could be simplified.

Circularity Check

0 steps flagged

No significant circularity; central results are derived from external Newman and [2] results, not from the paper's own conclusions.

full rationale

The main derivation chain is self-contained against external benchmarks. Corollary 6 rests on Lemma 5, which uses Newman's external rational approximation of |x| from [19] (quoted as 'The result in [19] shows that ||x| - x^{-1} R_n(x)||_{C([-1,1])} <= 3 e^{-sqrt n}'), and the derivative estimate is proved directly via Newman-polynomial estimates, not assumed. Theorem 11 uses the tensor-product spline bound (16) from [2], by different authors, and then approximates the ReQU activations in C^1 using Corollary 6; the error is controlled by explicit product and recursion estimates (equations (21)-(30)). This is a genuine transfer of an external spline result and an independently proved ReQU approximation, not a self-citation or fitted-input prediction. Corollary 12 follows from Theorem 11, and Lemma 16 follows from Corollary 12, but these are direct consequences rather than circular premises. The only self-citation is [13] in the introduction ('In [13], which deals with uniqueness in learning PDE based physical laws, a first order approximability condition is required...'), and it is purely motivational, not load-bearing for any proof. The reader's flagged width/rate discrepancy is a potential correctness issue, not a circularity: the stated width O(N) versus the constructed width (N+4)^d does not make any result equivalent to its own inputs. No fitted parameters are renamed as predictions, and no uniqueness theorem is imported from the authors' prior work. Therefore the derivation is not circular; the score reflects only the presence of one minor, non-load-bearing self-citation.

Axiom & Free-Parameter Ledger

0 free parameters · 4 axioms · 0 invented entities

The proofs rely on standard results in rational approximation and spline theory. No new physical or mathematical entities are posited, and no parameters are fitted to data.

axioms (4)
  • standard math Newman's rational approximation of |x|: || |x| - x^{-1} R_n(x)||_{C([-1,1])} <= 3 e^{-sqrt(n)}
    Invoked in Lemma 5 to bound the value approximation; a classical external result from [19].
  • domain assumption Tensor-product spline approximation bound from [2, Theorem 2]: for f in H^β_p([0,1]^d,H), there exist splines S^{⌊β⌋,N}_f with ||f - S||_{C^1} <= c N^{-(β-1)}
    External approximation-theoretic result used as the starting point of Theorem 11; not proven in this paper.
  • standard math B-spline recursion formula (18) and the representation of quadratic B-splines via ReQU
    Standard spline theory, quoted from [2] and used to design the rational network.
  • standard math Positivity of P_n(x)+P_n(-x) on R, ensuring rational functions are globally defined
    Derived in Remark 8 from the even/odd structure of Newman polynomials; needed for Definition 1.

reviewed 2026-08-05 · how reviews work

0 comments
Cite this review

Pith. "Pith review of $\mathcal{C}^1$-approximation with rational functions and rational neural networks." pith.science (2026). https://pith.science/paper/C3MHRSUX

@misc{pith2026250819672,
  author       = {Pith},
  title        = {Pith review of: $\mathcalC^1$-approximation with rational functions and rational neural networks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/C3MHRSUX}},
  note         = {Machine review of arXiv:2508.19672}
}
Share X Bluesky LinkedIn Reddit HN
abstract

We show that suitably regular functions can be approximated in the $\mathcal{C}^1$-norm both with rational functions and rational neural networks, including approximation rates with respect to width and depth of the network, and degree of the rational functions. As consequence of our results, we further obtain $\mathcal{C}^1$-approximation results for rational neural networks with the $\text{EQL}^\div$ and ParFam architecture, both of which are important in particular in the context of symbolic regression for physical law learning.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Symbolic recovery of PDEs from measurement data

    cs.LG 2026-02 unverdicted novelty 7.0

    Symbolic rational-function networks recover an admissible PDE from noiseless complete measurements and select the regularization-minimizing parameterization within the architecture.

Reference graph

Works this paper leans on

27 extracted references · 25 canonical work pages · cited by 1 Pith paper · 1 internal anchor

  1. [1]

    A. C. Antoulas and B. D. Q. Anderson. On the scalar rational interpolation problem. IMA Journal of Mathematical Control and Information , 3(2-3):61– 88, 09 1986

    work page 1986

  2. [2]

    Simultaneous approximation of a smooth function and its derivatives by deep neural networks with piecewise-polynomial activations

    Denis Belomestny, Alexey Naumov, Nikita Puchkin, and Sergey Samsonov. Simultaneous approximation of a smooth function and its derivatives by deep neural networks with piecewise-polynomial activations. Neural Networks , 161:242–253, 2023

    work page 2023

  3. [3]

    Bengio, P

    Y. Bengio, P. Simard, and P. Frasconi. Learning long-term dependencies with gradient descent is difficult. IEEE Transactions on Neural Networks , 5(2):157–166, 1994

    work page 1994

  4. [4]

    Rational neural net- works

    Nicolas Boull´ e, Yuji Nakatsukasa, and Alex Townsend. Rational neural net- works. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 14243–14253. Curran Associates, Inc., 2020

    work page 2020

  5. [5]

    From ESPRIT to ESPIRA: estimation of signal parameters by iterative rational approximation

    Nadiia Derevianko, Gerlind Plonka, and Markus Petz. From ESPRIT to ESPIRA: estimation of signal parameters by iterative rational approximation. IMA Journal of Numerical Analysis , 43(2):789–827, 02 2022

    work page 2022

  6. [6]

    Neural network approx- imation

    Ronald DeVore, Boris Hanin, and Guergana Petrova. Neural network approx- imation. Acta Numerica, 30:327–444, 2021

    work page 2021

  7. [7]

    A descent algorithm for the optimal control of ReLU neural network informed PDEs based on approximate directional derivatives.SIAM Journal on Optimization, 34(3):2314–2349, 2024

    Guozhi Dong, Michael Hinterm¨ uller, and Kostas Papafitsoros. A descent algorithm for the optimal control of ReLU neural network informed PDEs based on approximate directional derivatives.SIAM Journal on Optimization, 34(3):2314–2349, 2024

    work page 2024

  8. [8]

    First-order conditions for the optimal control of learning- informed nonsmooth PDEs

    Guozhi Dong, Michael Hinterm¨ uller, Kostas Papafitsoros, and Kathrin V¨ olkner. First-order conditions for the optimal control of learning- informed nonsmooth PDEs. Numerical Functional Analysis and Optimization, 46(7):505–539, 2025. 24

    work page 2025

  9. [9]

    Driscoll, Yuji Nakatsukasa, and Lloyd N

    Tobin A. Driscoll, Yuji Nakatsukasa, and Lloyd N. Trefethen. AAA ratio- nal approximation on a continuum. SIAM Journal on Scientific Computing , 46(2):A929–A952, 2024

    work page 2024

  10. [10]

    Deep neural network approximation theory

    Dennis Elbr¨ achter, Dmytro Perekrestenko, Philipp Grohs, and Helmut B¨ olcskei. Deep neural network approximation theory. Transactions on In- formation Theory, 67:2581–2623, 2021

    work page 2021

  11. [11]

    Approximation spaces of deep neural networks

    R´ emi Gribonval, Gitta Kutyniok, Morten Nielsen, and Felix Voigtlaender. Approximation spaces of deep neural networks. Constructive Approximation, 55:259–367, 2022

    work page 2022

  12. [12]

    Error bounds for ap- proximations with deep ReLU neural networks in W s,p norms

    Ingo G¨ uhring, Gitta Kutyniok, and Philipp Petersen. Error bounds for ap- proximations with deep ReLU neural networks in W s,p norms. Analysis and Applications, 18:803–859, 2020

    work page 2020

  13. [13]

    On uniqueness in structured model learning

    Martin Holler and Erion Morina. On uniqueness in structured model learning. ArXiv preprint arXiv:2410.22009 , 2024

    arXiv 2024

  14. [14]

    Semiglobal optimal feedback stabilization of autonomous systems via deep neural network approximation.ESAIM: COCV, 27:16, 2021

    Karl Kunisch and Daniel Walter. Semiglobal optimal feedback stabilization of autonomous systems via deep neural network approximation.ESAIM: COCV, 27:16, 2021

    work page 2021

  15. [15]

    Extrapolation and learning equa- tions

    Christoph Lampert and Georg Martius. Extrapolation and learning equa- tions. In 5th International Conference on Learning Representations , ICLR 2017 - Workshop Track Proceedings. International Conference on Learning Representations, 24–26 Apr 2017

    work page 2017

  16. [16]

    H. N. Mhaskar. Neural networks for optimal approximation of smooth and analytic functions. Neural Computation, 8:164–177, 1996

    work page 1996

  17. [17]

    Yuji Nakatsukasa and Roland W. Freund. Computing fundamental matrix decompositions accurately via the matrix sign function in two iterations: The power of Zolotarev’s functions. SIAM Review, 58(3):461–493, 2016

    work page 2016

  18. [18]

    Trefethen

    Yuji Nakatsukasa, Olivier S` ete, and Lloyd N. Trefethen. The AAA algo- rithm for rational approximation. SIAM Journal on Scientific Computing , 40(3):A1494–A1522, 2018

    work page 2018

  19. [19]

    D. J. Newman. Rational approximation to |x|. Michigan Mathematical Jour- nal, 11(1):11 – 14, 1964

    work page 1964

  20. [20]

    P. P. Petrushev and Vasil Atanasov Popov. Rational Approximation of Real Functions. Encyclopedia of Mathematics and its Applications. Cambridge University Press, 1988. 25

    work page 1988

  21. [21]

    Learning equa- tions for extrapolation and control

    Subham Sahoo, Christoph Lampert, and Georg Martius. Learning equa- tions for extrapolation and control. In Jennifer Dy and Andreas Krause, editors, Proceedings of the 35th International Conference on Machine Learn- ing, volume 80 ofProceedings of Machine Learning Research, pages 4442–4450. PMLR, 10–15 Jul 2018

    work page 2018

  22. [22]

    ParFam -- (Neural Guided) Symbolic Regression Based on Continuous Global Optimization

    Philipp Scholl, Katharina Bieker, Hillary Hauger, and Gitta Kutyniok. ParFam–(Neural guided) symbolic regression based on continuous global op- timization. ArXiv preprint arXiv:2310.05537 , 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  23. [23]

    Neural networks and rational functions

    Matus Telgarsky. Neural networks and rational functions. In Doina Precup and Yee Whye Teh, editors, Proceedings of the 34th International Confer- ence on Machine Learning , volume 70 of Proceedings of Machine Learning Research, pages 3387–3393. PMLR, 06–11 Aug 2017

    work page 2017

  24. [24]

    Trefethen and Yuji Nakatsukasa

    Lloyd N. Trefethen and Yuji Nakatsukasa. Rational approximation of xn. Proceedings of the American Mathematical Society, 146:5219–5224, 2018

    work page 2018

  25. [25]

    Trefethen and Heather D

    Lloyd N. Trefethen and Heather D. Wilber. Computation of Zolotarev rational functions. SIAM Journal on Scientific Computing , 47(4):A2205–A2220, 2025

    work page 2025

  26. [26]

    Splines, rational functions and neu- ral networks

    Robert C Williamson and Peter Bartlett. Splines, rational functions and neu- ral networks. In J. Moody, S. Hanson, and R.P. Lippmann, editors, Advances in Neural Information Processing Systems , volume 4. Morgan-Kaufmann, 1991

    work page 1991

  27. [27]

    Nearly optimal approximation rates for deep super ReLU networks on Sobolev spaces

    Yahong Yang, Yue Wu, Haizhao Yang, and Yang Xiang. Nearly optimal approximation rates for deep super ReLU networks on Sobolev spaces. ArXiv preprint arXiv:2310.10766, 2025. 26

    Pith/arXiv arXiv 2025

This paper was first reviewed by deepseek-v4-flash on August 5, 2026.