{"id":"445ada50-519b-428f-9225-54dcf013b1f0","arxiv_id":"2505.18131","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Training in a B-spline KAN basis is equivalent to preconditioned gradient descent on a multichannel ReLU MLP, and geometric refinement plus trainable knots accelerate and improve training.","lead":"This paper shows that KANs with B-spline activations are algebraically equivalent to multichannel MLPs, and that training in the spline basis acts as preconditioned gradient descent in the ReLU basis. It adds a cheap multilevel grid-refinement scheme and trainable knot positions that improve accuracy on regression and physics-informed benchmarks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The deep-network speedup claim depends on an unproven extrapolation from linear preconditioning, and the supporting NTK bound (Theorem 3.2) contains a scaling error in Appendix B that leaves the proof of the stated bound invalid as written.","rationale":"The paper's strongest and most secure contribution is the exact algebraic relationship between KANs and multichannel MLPs, including the change-of-basis matrix and the per-layer preconditioning identity. That part is internally consistent and does not require the deep extension to be true. The load-bearing weakness is the step from 'this is exactly preconditioned gradient descent' to 'therefore training is expedited in deep nonlinear networks.' The linear least-squares conditioning result (Proposition 3.1) is a shallow analysis; the NTK result (Theorem 3.2) is the only quantitative bridge to deep networks, but its proof contains the scaling error the reader identified in Eqs. (49)-(50), and the theorem's conclusion is also insufficient because a spectral-radius bound does not control conditioning. I agree with the reader's conditional verdict: the central equivalence and refinement method are likely correct and valuable, but the theoretical support for the headline speedup claim is not yet solid. The experiments, while suggestive, are few in number, use small networks, and are not backed by released code or a direct Hessian-conditioning measurement. The concrete test above would either validate or refute the mechanism underlying the central claim; until then, the present wording overstates what has been established.","tokens_in":19336,"tokens_out":14963,"duration_ms":120543,"concrete_test":"Independently re-derive Theorem 3.2 from Eqs. (18), (31), and (42) with consistent powers of h = 1/n. Then for a fixed small depth (e.g., two hidden layers), compute the spectral radii of JJ^T and JAA^TJ^T at a random initialization for increasing knot counts n = 8, 16, 32 and for r = 2 and r = 3. If the ratio ρ(NTK_S)/ρ(NTK_R) grows with n or is not bounded by 4, the theorem as stated is false and the theoretical support for the deep preconditioning claim must be withdrawn. A complementary check is to measure the Hessian condition number of a small deep ReLU MLP before and after applying the block-diagonal preconditioner P = AA^T; if the preconditioned Hessian is not better conditioned, the proposed mechanism does not explain the reported speedups.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The algebraic equivalence in Eqs. (12)-(14) is exact for full-batch gradient descent for any differentiable loss: training a KAN in spline weights is equivalent to preconditioned gradient descent in ReLU weights with block-diagonal P = AA^T. What is load-bearing is the further assertion that this block-diagonal preconditioner improves conditioning and convergence for deep nonlinear networks. The paper's only quantitative support is Theorem 3.2, but its proof in Appendix B has a scaling inconsistency. Eq. (49) introduces a factor n^{r-1} that is not justified by the preceding definitions; Eq. (47) normalizes eA so this factor should cancel, and if it does not cancel, the bound in Eq. (50) grows with n instead of being ≤4. Even accepting Eq. (50) as is, ||eA|| ≤ 4 yields ρ(NTK_S) ≤ 16 ρ(NTK_R) via sub-multiplicativity, not the stated 4. More fundamentally, a bound on the largest eigenvalue does not establish better conditioning; the convergence benefit depends on the full spectrum. Section 3 explicitly concedes that rigorously proving the deep-network statement is difficult, so the central 'expedited training' claim rests on three small experiments rather than on the theory advertised.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper establishes an exact algebraic relationship between KANs built on a B-spline basis and multichannel MLPs with ReLU^{r-1} activations whose biases are fixed to the spline knots. The central identity (Eqs. 12-14) shows that full-batch gradient descent on the spline weights is equivalent to preconditioned gradient descent on the ReLU weights with block-diagonal preconditioner P = AA^T. The paper then argues that this preconditioning improves conditioning and convergence, proposes a geometric refinement scheme for multilevel KAN training using exact injection operators, and introduces a trainable free-knot spline parameterization based on softmax-normalized knot spacings. Experiments on two regression problems and a physics-informed neural network (PINN) report improved accuracy from the spline basis, multilevel refinement, and free knots. The main theoretical support for the deep-network speedup claim is Theorem 3.2, which asserts a spectral-radius bound for the neural tangent kernel under the basis change; the proof of this theorem appears in Appendix B.","tokens_in":19557,"tokens_out":11059,"duration_ms":91543,"significance":"If the central claims are correct, the paper provides a clean, non-circular explanation of KAN training speedups: KANs in a B-spline basis are exactly multichannel MLPs in a better-conditioned parameterization. The equivalence in Section 2 and the preconditioning identity in Section 3 are direct algebraic identities that can be verified independently, and the proposed geometric refinement and free-knot parameterizations are concrete, executable algorithmic contributions. The paper does not adapt existing results by curve-fitting; the derivations are self-contained and cite the relevant spline and Toeplitz literature. However, the advertised theoretical guarantee for deep nonlinear networks rests on a proof that contains a scaling inconsistency, and the experiments use an optimizer different from the one analyzed in the equivalence, so the theoretical portion needs correction before the paper's claims can be fully accepted.","major_comments":[{"comment":"The generating function for the normalized change-of-basis matrix eA[r] contains a spurious factor n^{r-1}. By Eq. (47), eA[r] = A[r]/h^{1-r}, and with h = 1/n, Corollary B.3 gives eA[r] = (A[1])^r/(r-1)!, which has Fourier symbol (1-e^{ix})^r/(r-1)! and operator norm at most 2^r/(r-1)!. The last equality in Eq. (49), n^{r-1}(1-e^{ix})^r/(r-1)!, is instead the symbol of the unscaled A[r]. As written, this makes Eq. (50) internally inconsistent: the n^{r-1} factor would grow with n, contradicting the claimed uniform bound 2^r/(r-1)! ≤ 4. Consequently, the proof of Theorem 3.2 is invalid as written.","section":"Appendix B, Eq. (49)"},{"comment":"Even after removing the spurious n^{r-1} factor, the proof does not establish the stated constant 4. The argument bounds the singular values of eA, but the matrix appearing in NTK_S = JAA^TJ^T is the unscaled change-of-basis matrix A; if one instead works with the scaled basis, the Jacobian J changes by the same scaling, and this change is not tracked. Submultiplicativity gives ρ(NTK_S) = ||JA||^2 ≤ ||J||^2 ||A||^2, so with ||A|| ≤ 4 one obtains ρ(NTK_S) ≤ 16 ρ(NTK_R), not the stated factor 4, unless an additional argument is supplied.","section":"Theorem 3.2 / Appendix B, after Eq. (50)"},{"comment":"The step from the linear least-squares problem to deep nonlinear networks is asserted rather than proved. The text states that the preconditioner is 'applied block-wise independently for each layer, thereby improving the conditioning and convergence' and then concedes that rigorously proving the deep-network statement is difficult. Since Theorem 3.2 is the only quantitative support for this extrapolation, and its proof is currently invalid, the theoretical explanation of expedited training in deep networks is incomplete. The authors should either provide a corrected proof of a suitable statement or explicitly reframe the deep-network conditioning claim as an empirical observation supported by the experiments.","section":"Section 3, paragraph after Eq. (14)"},{"comment":"The theoretical equivalence in Eqs. (12)-(14) is derived for gradient descent, but the experiments in Section 6 use L-BFGS. It is not shown that the preconditioning result transfers to L-BFGS updates, and the paper does not describe how 'equivalent amounts of work (FLOPs)' are counted. This mismatch makes it difficult to attribute the observed speedups specifically to the preconditioning mechanism, as opposed to other differences between the architectures and optimizers. The authors should address the optimizer mismatch or qualify the claim that the experiments validate the gradient-descent preprocessing analysis.","section":"Section 6 and Section 3"}],"minor_comments":[{"comment":"The phrase 'low-rank factorization' for the restriction W^{(ℓ)}_{qpi} = δ_{i=p} W^{(ℓ)}_{qp} is inaccurate; this is a sparsity/selection pattern, not a low-rank factorization.","section":"Section 2, Eq. (9)"},{"comment":"The column header 'FREE# MSE' is garbled and should be split into 'FREE KNOTS?' and '# MSE' or equivalent; several other table headers are crowded and would benefit from clearer formatting.","section":"Tables 1-3"},{"comment":"There are minor typos: 'we us weight-regularized Adam' should read 'we use weight-regularized Adam', and 'ranging form 0.0001 to 0.001' should read 'ranging from'.","section":"Appendix D"},{"comment":"The batch-size condition should be stated as a ceiling: the required batch size is the smallest integer greater than or equal to log(1-τ)/log(P(x∉[t_i,t_{i+1}])).","section":"Appendix C.1, Eq. (51)"},{"comment":"The sentence 'the dominant eigenvalue is the same order for both bases' is stronger than the inequality that follows; the inequality only provides an upper bound on one spectral radius, not order equivalence in both directions.","section":"Theorem 3.2"},{"comment":"The proof applies Tilli's theorem for finite block-Toeplitz matrices, but the finite change-of-basis matrices have boundary rows that are not exact Toeplitz; the authors should justify that the theorem's hypotheses cover their finite matrices or provide a direct norm bound for the banded matrices.","section":"Appendix B, Theorem B.5"}],"recommendation":"major_revision","confidential_remarks":"The algebraic equivalence in Sections 2-3 is the strongest part of the paper and is likely correct; I see no circularity in the derivation. The main risk is overclaiming theoretical support for the deep-network speedup: the proof of Theorem 3.2 has a scaling error, and the step from linear least squares to deep nonlinear networks is asserted rather than proved. These issues are localizable and potentially fixable by correcting the proof or re-scoping the claim, so I would not reject the paper, but the manuscript needs meaningful revision before publication. The empirical section would also be more convincing if the FLOP counting and the L-BFGS/GD mismatch were addressed."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Alex — quick read of arXiv:2505.18131. The core identity is right and worth knowing: training a B-spline KAN is exactly preconditioned gradient descent in the ReLU-basis weights of an equivalent multichannel MLP, with block-diagonal preconditioner AA^T (Eqs. 12-14). That holds for any differentiable loss, full-batch GD, and gives a clean optimization explanation for why the spline basis trains faster. The nested-grid refinement using multigrid prolongation/restriction is a genuinely useful practical contribution; it avoids the expensive least-squares grid transfer in KAN 1.5. The softmax free-knot parameterization is simple and sensible. These parts are solid, and the authors properly credit He et al. and Qiu et al. for the KAN/MLP equivalence rather than claiming it as new.\n\nThe soft spots are in the theory section. Theorem 3.2's proof in Appendix B has a real scaling error: Eq. (49) introduces a spurious n^{r-1} in the generating function of eA[r], which should be (1-e^{ix})^r/(r-1)!, not n^{r-1} times that. And even accepting ||eA[r]|| ≤ 4, submultiplicativity gives ρ(JAA^T J^T) ≤ ||J||^2||A||^2 ≤ 16 ρ(JJ^T), not 4. The stated bound is off by a factor of ||A||; it can likely be repaired to 16ρ, which preserves the qualitative claim, but the proof as written is invalid. Separately, the step from linear least-squares conditioning to deep nonlinear networks is explicitly conceded to be heuristic: the block-wise preconditioner argument is plausible and the three small experiments support it, but the paper does not prove it. That's okay if the claim is softened, but \"dramatically accelerates\" is currently backed by three benchmarks with no released code and no direct comparison to KAN 1.5's grid extension.\n\nMy bottom line: this is a solid, honest paper with one correct central identity and a practically useful refinement scheme. The theory needs fixing and the experiments need widening before the strong claims hold, but the work deserves a serious referee rather than a desk reject. I'd cite it for the preconditioning identity and the multilevel transfer idea.","headline":"A correct and useful algebraic reading of KANs as preconditioned multichannel MLPs, with a practical multilevel refinement scheme, but the theoretical spectral bound has a repairable scaling error and the deep-net speedup claim outruns the evidence.","tokens_in":20143,"tokens_out":4107,"would_cite":true,"duration_ms":32169,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["65D07","68T07","65F35"],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper establishes that a spline-basis KAN layer is exactly a multichannel MLP with biases fixed to the spline knots, and that training the KAN is preconditioned gradient descent in the ReLU basis, which together with geometric…","keywords":["Kolmogorov-Arnold networks","multichannel MLPs","B-splines","preconditioned gradient descent","geometric refinement","free-knot splines","spectral bias","physics-informed neural networks"],"falsifier":"Run a deep multichannel ReLU MLP and its spline-basis counterpart on the same task and compare per-layer Hessian condition numbers after the change of basis: if any layer's condition number is not improved, or if spline-basis gradient descent and preconditioned ReLU gradient descent do not follow the same trajectory, the layer-wise preconditioning claim fails. Independently, recompute the uniform-knot bound in Appendix B: with $\\tilde A^{[r]}=h^{r-1}A^{[r]}$ and generating function $F_r(x)=\\frac{r}{(r-1)!}(1-e^{ix})^r$ (Eqs. 49-50), the claimed uniform bound $\\|\\tilde A^{[r]}\\|\\le 2^r/(r-1)!\\le 4$ appears to drop factors of $h$; explicit spectral norm computation for $r=3,4$ and $n\\to\\infty$ would settle whether $\\rho(\\mathrm{NTK}_S^{[r]})\\le 4\\rho(\\mathrm{NTK}_R)$ actually holds for all $r$.","tokens_in":19092,"feed_emoji":"⚡","tokens_out":12727,"duration_ms":111929,"temperature":0.7,"pith_summary":"The paper's central claim is that the training advantage of spline-based Kolmogorov-Arnold networks over plain multilayer perceptrons is not a matter of expressive power but of optimization geometry. Because the B-spline basis and the truncated-power (ReLU) basis span the same spline space, any KAN layer can be rewritten as a multichannel MLP whose extra bias channels are fixed at the spline knots. Gradient descent on the KAN's spline weights is then exactly gradient descent on the MLP's ReLU weights multiplied by a fixed preconditioner $A A^\\top$, where $A$ is the change-of-basis matrix between the two bases. The paper argues that this preconditioning improves the conditioning of the loss landscape, that the local support of splines tames the spectral bias that slows ReLU MLPs, and that nested spline spaces allow cheap geometric refinement during training. On regression, XOR, and physics-informed Burgers benchmarks, the spline-basis and multilevel-trained models reach lower loss for the same training work, and trainable free-knot splines help on nonsmooth problems.","feed_headline":"Spline basis turns MLP training into preconditioned descent","feed_subtitle":"KAN layers are multichannel ReLU MLPs; the spline change of basis preconditioner speeds gradient descent.","key_machinery":"The load-bearing object is the change-of-basis matrix $A$ between the B-spline basis and the $\\mathrm{ReLU}^{\\,r-1}$ truncated-power basis of the spline space $S_r(T)$: for uniform knots $A^{[r]}=\\frac{h^{1-r}}{(r-1)!}(A^{[1]})^r$ with $A^{[1]}$ bidiagonal, and for nonuniform knots it is built recursively by $A^{[r]}_{ij}=\\frac{1}{t_{i+r-1}-t_i}A^{[r-1]}_{ij}-\\frac{1}{t_{i+r}-t_{i+1}}A^{[r-1]}_{i+1,j}$. It carries the argument because it converts KAN spline weights into multichannel MLP ReLU weights, relates the Hessians by congruence $H_{\\mathrm{spline}} = A H_{\\mathrm{ReLU}} A^\\top$, and makes gradient descent in the spline basis equal to preconditioned gradient descent in the ReLU basis with preconditioner $A A^\\top$. The same matrix, in uniform form, is what the NTK spectral-radius bound $\\rho(\\mathrm{NTK}_S^{[r]})\\le 4\\rho(\\mathrm{NTK}_R)$ uses, and its banded structure keeps the change of basis cheap ($O(nr)$ per layer).","core_discovery":"On the paper's own terms, the discovery is an exact structural equivalence with a practical payoff. Let $T=\\{t_i\\}$ be the spline knots and let $S_r(T)$ be the space of order-$r$ splines on an interval; the B-spline basis $B_S=\\{b_i^{[r-1]}\\}$ and the truncated-power basis $B_R=\\{\\mathrm{ReLU}(x-t_i)^{r-1}\\}$ are both bases of $S_r(T)$, so there is an invertible, banded matrix $A$ with $\\tilde W^{(\\ell)} \\times_3 A = W^{(\\ell)}$ mapping each KAN layer's spline weights to ReLU weights. A KAN layer $\\sum_{p,i} \\tilde W^{(\\ell)}_{qpi} b_i(x_p^{(\\ell)})$ is therefore a multichannel MLP layer $\\sum_{p,i} W^{(\\ell)}_{qpi}\\mathrm{ReLU}(x_p^{(\\ell)}-t_i)^{r-1}$, with one channel per knot and biases fixed to the knots. Minimizing the same loss in the two parameterizations is equivalent, but gradient descent in the spline basis is the preconditioned update $W_{k+1}=W_k-\\eta A A^\\top \\nabla \\mathcal{L}(W_k)$, and in the linear least-squares case the spline Hessian condition number is bounded independently of grid size while the ReLU Hessian condition number grows without bound. The paper concludes from this that KANs inherit MLP approximation power while enjoying a better-conditioned optimization problem, that hierarchical multilevel refinement is an exact injection that preserves progress, and that parameterizing knot distances with a softmax makes free-knot splines trainable by backpropagation.","pith_inferences":["If the equivalence is exact, the practical implication is broader than KANs: one could wrap any MLP optimizer in the fixed preconditioner $A A^\\top$ and reproduce KAN-style training dynamics on standard architectures, making the speedup a drop-in optimization change rather than a new architecture.","The paper's layer-wise decoupling of the preconditioning argument suggests a diagnostic: measure per-layer Hessian or NTK conditioning in a deep ReLU MLP before and after the spline change of basis; if some layers do not improve, the benefit may be confined to particular depths or tasks rather than universal.","The free-knot softmax parameterization points toward a unified view of adaptive KAN grids and trainable MLP biases: both are the same mechanism, so adaptivity results from one setting should transfer to the other."],"forward_implications":["Any standard multichannel ReLU MLP can be trained with KAN-like speedups by fixing biases at spline knots and preconditioning each layer's gradient by $A A^\\top$, without changing the architecture's function class.","Geometric refinement via nested spline grids is an exact injection: transferring a trained coarse KAN to a finer grid preserves the learned function, so multilevel training never throws away progress and costs little enough to run during training.","Trainable knots parameterized by softmax-spaced distances let a KAN adapt its grid to nonsmooth features, improving accuracy where fixed uniform grids fail.","The bounded spline Gram condition number and the NTK spectral-radius comparison imply training updates are spread across localized modes instead of concentrating in the dominant eigen-direction, reversing the spectral-bias failure mode of ReLU MLPs.","On equal-FLOP comparisons, spline-basis and multilevel KANs reach lower training loss than comparable MLPs on regression, XOR, and a physics-informed Burger's equation problem."],"supporting_citations":[{"why":"Defines the spline space and proves the equivalence of the B-spline and truncated-power bases that the whole argument builds on.","marker":"Chui (1988)"},{"why":"Provides the B-spline recursion and nonuniform-spline conditioning facts used for the change-of-basis matrix and Proposition 3.1.","marker":"De Boor (1978)"},{"why":"Supplies the bounded spline and unbounded ReLU Gram condition-number results and spectral-bias analysis that Proposition 3.1 and Theorem 3.2 extend.","marker":"Hong et al. (2022)"},{"why":"Introduces KANs and the original grid-refinement procedure that the paper's geometric multilevel refinement is designed to replace.","marker":"Liu et al. (2024b)"},{"why":"Provides the restriction and prolongation grid-transfer operators used for exact geometric refinement of nested spline grids.","marker":"Höllig (2003)"},{"why":"Supplies multigrid transfer operators that make the hierarchical refinement cheap enough to run during training.","marker":"Hackbusch (2013)"},{"why":"Contributes the relative-distance parameterization that keeps trainable free knots strictly ordered in the softmax construction.","marker":"Actor et al. (2024)"},{"why":"Defines the tensor mode product and vectorization conventions used to write the change of basis and preconditioned update.","marker":"Kolda & Bader (2009)"}],"fun_headline_variants":["KANs precondition MLP training via spline basis","Spline basis gives multichannel MLPs a speed boost","Free-knot splines: a preconditioner for MLP training","Why KANs train faster: preconditioned descent in splines","Geometric refinement speeds up multichannel MLPs"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing assumption is that the conditioning improvement proven for a one-layer linear least-squares problem still holds in deep nonlinear networks when the spline change of basis is applied independently layer by layer; the paper does not prove this, noting that a rigorous statement is difficult, and instead offers three small experiments.","fun_headline_variants_meta":{"raw":{"variants":["KANs precondition MLP training via spline basis","Spline basis gives multichannel MLPs a speed boost","Free-knot splines: a preconditioner for MLP training","Why KANs train faster: preconditioned descent in splines","Geometric refinement speeds up multichannel MLPs"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000361,"raw_usage":{"total_tokens":2046,"prompt_tokens":1139,"completion_tokens":907,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":755,"completion_tokens_details":{"reasoning_tokens":822}},"tokens_in":755,"tokens_out":907,"duration_ms":6840,"temperature":1.0,"reasoning_tokens":822,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T14:35:09.169042+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a deep multichannel ReLU MLP and its spline-basis counterpart on the same task and compare per-layer Hessian condition numbers after the change of basis: if any layer's condition number is not improved, or if spline-basis gradient descent and preconditioned ReLU gradient descent do not follow the same trajectory, the layer-wise preconditioning claim fails. Independently, recompute the uniform-knot bound in Appendix B: with $\\tilde A^{[r]}=h^{r-1}A^{[r]}$ and generating function $F_r(x)=\\frac{r}{(r-1)!}(1-e^{ix})^r$ (Eqs. 49-50), the claimed uniform bound $\\|\\tilde A^{[r]}\\|\\le 2^r/(r-1)!\\le 4$ appears to drop factors of $h$; explicit spectral norm computation for $r=3,4$ and $n\\to\\infty$ would settle whether $\\rho(\\mathrm{NTK}_S^{[r]})\\le 4\\rho(\\mathrm{NTK}_R)$ actually holds for all $r$.","supporting_citations":[],"review_version":1}