REVIEW 3 major objections 7 minor 1 cited by
Free-Knots Kolmogorov-Arnold Network: On the Analysis of Spline Knots and Advancing Stability
T0 review · 3 major / 7 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read This paper claims that freeing B-spline knots per neuron, sharing weights across neuron groups, and widening the grid range makes Kolmogorov-Arnold Networks outperform MLPs and other KAN variants while keeping trainable parameters at MLP…
desk verdict The free-knots idea is worth trying, but the regularizer that supposedly stabilizes training is identically zero as written, and the theoretical bound is not established. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the free-knot B-spline activation: each neuron keeps its own grid $G^* = \mathrm{Sort}(G + b_g)$, where $b_g$ is a small learnable per-grid-point bias, and the grid is sorted to preserve B-spline order. Neuron grouping shares one grid among $h$ neurons and shares the linear-combination weight $A$ between spline and SiLU, decoupling the parameter count from the grid size. The theoretical engine is Theorem 4.3's tight bound on spline knots, which depends only on grid size $G$, spline order $K$, and depth $L$; it justifies freeing knots from the shared grid. A second-derivative regularizer $\lambda \sum_i \sum_j \partial^2 c_{j,i}/\partial x^2$ and an expanded grid range are introduced as stability mechanisms.
What would settle it
Run FR-KAN with and without the second-derivative term while keeping all other settings fixed: if the training curves and final accuracy are indistinguishable, the regularizer is not doing the stated smoothing work. A direct inspection of the released code for $\partial^2 c_{j,i}/\partial x^2$—if it is identically zero—would settle the same point.
Extended reading notes
Core claim
The paper's central claim is that a KAN whose B-spline knots are learnable per neuron—shifted by a small trainable bias and then re-sorted—has more expressive power than the original fixed-grid KAN, and that this gain can be obtained without the original's heavy parameter cost. FR-KAN groups neurons and shares the spline and SiLU weights, reducing the trainable-parameter count from $O(d_{\rm in} d_{\rm out}(G+K+1))$ to $O(d_{\rm in} d_{\rm out} + h(G+K))$, i.e. MLP scale. The paper derives a tight knot-count bound: a fixed-grid KAN has at most $(G+K)+\prod_{l=1}^{L} G(G-1)$ knots, while free-knot KAN has up to $h$ times that, with $h$ the number of neuron groups. In the reported experiments, FR-KAN outperforms ReLU-MLP, original KAN, Fourier-KAN, Rational-KAN, and RBF-KAN on most of the eight evaluated datasets while matching MLP's parameter scale.
Load-bearing premise
The stability mechanism assumes each spline coefficient $c_{j,i}$ is a function of the input $x$, so that penalizing its second derivative smooths the activation; in the implementation these coefficients are learned constants, making the penalty zero and leaving the wider grid range as the only stated source of stability.
Editorial extensions
If this is right
- If the knot bound and free-grid gains hold, KANs can be deployed in standard deep-learning pipelines without the customary parameter blow-up.
- The bound implies that increasing width alone cannot create new knots in a fixed-grid KAN; depth, grid size, or free knot shifts are needed to grow expressive capacity.
- Weight-sharing across neuron groups plus per-group free grids offers a reusable recipe for scaling other spline-based KAN variants.
- Reported results indicate spline-based KANs remain competitive with kernel-based KANs when the grid is learnable, on both classification and symbolic regression.
- A wider grid range removes the NaN-loss failure observed with narrow grids, making KAN training sufficiently stable for deeper stacks.
Reading between the lines
- Because the regularizer penalizes $\partial^2 c_{j,i}/\partial x^2$ but the spline weights $c_{j,i}$ are trained constants, the penalty is identically zero as written; an ablation with and without $\lambda$ would reveal whether any stability gain actually comes from this term or only from the wider grid range.
- A natural next step not explored in the paper is data-dependent initialization of the grid shift $b_g$ (for instance, placing knots at input quantiles) instead of uniform random, which could sharpen the approximation benefit.
- The knot-count bound suggests a concrete architecture rule—raise grid size or depth before width when using fixed-grid KANs—that could be tested on larger vision and language benchmarks.
- If the parameter savings transfer to convolutional or transformer KAN hybrids, the same grouping-plus-free-grid recipe may provide a low-cost route to scaling KANs beyond the MLP-style settings evaluated here.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper studies Kolmogorov-Arnold Networks from the viewpoint of B-spline knots. It derives claimed upper bounds on the number of knots in fixed-grid KANs (Theorem 4.3) and in the proposed Free-Knot KAN (Theorem 5.1). To reduce the parameter cost of KANs, it introduces neuron grouping and weight sharing, and to improve training stability it proposes a second-derivative regularizer plus an enlarged grid range. The method is evaluated on image, text, multimodal, time-series, and function-approximation benchmarks, reporting accuracy/RMSE improvements over MLP and several KAN variants.
Significance. If the claims were fully substantiated, FR-KAN would be a practically useful KAN variant: it promises MLP-scale parameter counts and better stability while retaining the expressive advantages of spline-based activations. The paper ships code and covers a broad set of benchmarks, which is a strength. However, two load-bearing elements are currently not supported by the manuscript text: the regularization term in Eq. (8) is identically zero as written, and the knot-count bound in Theorem 4.3 is internally inconsistent and only sketched in the appendix. These issues directly affect contributions (C1) and (C3), so the significance cannot be assessed from the present version.
major comments (3)
- [Section 5.3, Eq. (8)] The regularization term λ Σ_i Σ_j ∂²c_{j,i}/∂x² is identically zero because c_{j,i} is defined throughout the paper (Eqs. (1) and (5)) as a learned scalar combination weight for the B-spline basis, not as a function of the input x. Every second derivative with respect to x therefore vanishes, so the claimed C2-smoothing mechanism is absent as written. The stability improvements attributed to this regularizer in Section 5.3, Figure 4, and contribution (C3) are unsupported. A valid penalty would act on the second derivative of the activation itself, for example λ∫(Σ_j c_j B''_j(x))² dx, but no such term appears in the paper.
- [Section 4.2, Theorem 4.3 and Appendix B] The asserted tight bound is inconsistent with the paper's own Lemma 4.1 and does not follow from the stated recurrence. Lemma 4.1 gives exactly G+K knots for a single-layer fixed-grid KAN, but Eq. (3) with L=1 yields the upper bound G+K+G(G−1). Moreover, the recurrence in Eq. (4) is additive in the previous-layer knot count m_{l−1}, so after L layers it gives at most about (L−1)G(G−1) additional knots, not the multiplicative product ∏_{l=1}^L G(G−1) in Eq. (3). The tightness argument in Appendix B constructs only one sawtooth example with K=1 and randomly initialized layer-2 weights; it does not establish that the bound is reachable for general L and K. Since Theorem 4.3 is contribution (C1) and motivates the proposed free-grid design, it must be corrected or removed.
- [Section 5.1-5.2, Eqs. (5)-(6)] The parameter-count claim O(d_in·d_out + h(G+K)) is not supported by the free-grid description. Section 5.2 states that each neuron n_i receives a different shift b_g on the grid, which would introduce O(d_in·d_out·G) additional parameters and would also be incompatible with sharing the spline combination weights c_j across a group. If the shift is instead per group, the text should say so explicitly. This ambiguity directly affects the central claim that FR-KAN matches the parameter scale of MLPs.
minor comments (7)
- [Abstract and Section 2] The abstract says "new a training strategy" and "The promising results demonstrates"; these should be corrected. Contribution (C4) states 9 datasets while the abstract and Section 6.1 list 8; use a consistent count.
- [Section 6.1] The dataset enumeration mentions image, text, multimodal, and function approximation, but the experiments also include the time-series dataset ETTh1 (Figure 2); ETTh1 should be listed explicitly in Section 6.1.
- [Section 6.1] The text says three independent runs use the random seed "2024"; please clarify how the runs are independent if the same seed is used for all of them.
- [Eq. (5)] The notation B_j(x_i)^{j/h} is undefined and appears to be a typo; the intended indexing of the shared grid within the h groups should be defined precisely.
- [Sections 3 and 4] There are several terminology slips: "Universal Approximation Theorem(URT)" should be "UAT", "Kolmogorov-Arnold Representation Theorem(KAT)" should be "KART" or "Kolmogorov-Arnold theorem", and "two steams" should be "two streams".
- [Table 2] In row I.12.11, the formula "q(Ef + B⊑sinθ)" contains a garbled symbol; please correct the typesetting.
- [Section 5.2 and Section 8] The choices Z=8, grid range [-10,10], and grid size G=20 are each justified by empirical results on the same datasets used for final evaluation; a separate validation procedure or an ablation on held-out data would strengthen the claim that these settings are not overfit to the reported benchmarks.
Circularity Check
The C2 regularizer in Eq. (8) differentiates learned constant spline weights with respect to x, making the smoothing mechanism identically zero by definition, and the Appendix B proof of the knot upper bound restates its own formula as an assumption. The external benchmark comparisons are otherwise not circular.
-
self definitional
[Section 5.3, Eq. (8), with Eqs. (1) and (5)]
"We propose a novel regularization term that constraint the second-derivatives of cj w.r.t input data to smooth oscillation. ... denoted cj,i as spline combination weight of neuron ni at segment gj, our optimization objective can be expressed as: L(x) = J(x) + λ Σ_i Σ_j ∂²c_{j,i}/∂x² (8)"
Eq. (1) defines c_j as 'the weight of combination of each spline' in φ(x_i) = A_b Σ c_j B_j(x_i) + A_s SiLU(x_i), and Eq. (5) keeps these as learned scalar coefficients. They are not functions of the input x, so ∂²c_{j,i}/∂x² is identically zero by construction. Substituting this definition into Eq. (8) gives L(x) = J(x) + λ·0, so the regularizer contributes nothing to the objective. The claimed C2-continuity training strategy, the oscillation reduction in Section 5.3, and the Section 9 statement that function-approximation wins come 'due to the smooth regularizer' therefore rest on a term that vanishes under the paper's own definitions.
-
other
[Appendix B, proof of Theorem 4.3, Eqs. (3)-(4)]
"The knots can be generated only if △yi > △g. For each grid segment, one can generate unique knots at most G − 1 ... To reach the maximum new knots in segment gi, we can assume the KAN can obtain ⌊∆yi/∆gi⌋ additional knots for each segment. Then, we can obtain the knots upper bound ... as follows."
The theorem statement gives N_k = m_{l−1} + Σ_{i=1}^{G−1} ⌊Δy_i/Δg⌋ as its conclusion (Eq. 4). The proof then assumes exactly that per-segment count ('we can assume the KAN can obtain ⌊Δy_i/Δg_i⌋ additional knots') and derives the product bound by capping each term at G−1. The claimed upper bound is therefore an assumption restated in theorem form rather than a consequence of the B-spline recursion or of Lemma 4.2; the 'tightness' construction illustrates the assumed sawtooth configuration but does not break the circularity.
full rationale
The paper's strongest empirical claim—that FR-KAN outperforms MLP and KAN variants on external benchmarks—is not circular: the comparisons use independent baselines and public datasets, and no fitted parameter is renamed as an out-of-sample prediction. The same-dataset choices of Z=8 ('setZ = 8 gives us best performance') and G=20 ('the sweet spot') weaken external validity but are tuning choices rather than circular derivations. The circularity lies in two theoretical or mechanistic claims. First, the C2-smoothing regularizer of Eq. (8) differentiates the constant spline-combination weights c_{j,i} with respect to the input, so it is identically zero under the paper's own definitions; the stability and function-approximation benefits attributed to it in Sections 5.3, 7, and 9 reduce by construction. Second, the proof of the knot upper bound in Appendix B assumes the per-segment floor formula that Theorem 4.3 states as its result. No load-bearing self-citation chain is present. Because the range-expansion heuristic in Section 8 is an independent mechanism and the benchmark results are external, the circularity is partial rather than total.
Assumptions & free parameters
free parameters (4)
- Grid shift initialization scale Z =
8
- Grid range [a,b] =
[-10,10]
- Grid size G =
20
- Regularization coefficient lambda =
not reported
assumptions (3)
- standard math B-spline recurrence and partition of unity (Eq. 15-16)
- domain assumption Kolmogorov-Arnold representation theorem justifies univariate activations
- ad hoc to paper New knots in a KAN layer are generated only when Delta y_i >= Delta g, and at most G-1 per segment
Cite this review
Pith. "Pith review of Free-Knots Kolmogorov-Arnold Network: On the Analysis of Spline Knots and Advancing Stability." pith.science (2026). https://pith.science/paper/YI43EFS5
@misc{pith2026250109283,
author = {Pith},
title = {Pith review of: Free-Knots Kolmogorov-Arnold Network: On the Analysis of Spline Knots and Advancing Stability},
year = {2026},
howpublished = {\url{https://pith.science/paper/YI43EFS5}},
note = {Machine review of arXiv:2501.09283}
}
abstract
Kolmogorov-Arnold Neural Networks (KANs) have gained significant attention in the machine learning community. However, their implementation often suffers from poor training stability and heavy trainable parameter. Furthermore, there is limited understanding of the behavior of the learned activation functions derived from B-splines. In this work, we analyze the behavior of KANs through the lens of spline knots and derive the lower and upper bound for the number of knots in B-spline-based KANs. To address existing limitations, we propose a novel Free Knots KAN that enhances the performance of the original KAN while reducing the number of trainable parameters to match the trainable parameter scale of standard Multi-Layer Perceptrons (MLPs). Additionally, we introduce new a training strategy to ensure $C^2$ continuity of the learnable spline, resulting in smoother activation compared to the original KAN and improve the training stability by range expansion. The proposed method is comprehensively evaluated on 8 datasets spanning various domains, including image, text, time series, multimodal, and function approximation tasks. The promising results demonstrates the feasibility of KAN-based network and the effectiveness of proposed method.
Figures
Figures from the paper (2 more)
Forward citations
Cited by 1 Pith paper
-
Clifford Kolmogorov-Arnold Networks
ClKAN extends complex-valued KANs to arbitrary Clifford algebras, and using scrambled Sobol-sequence grids cuts the parameter count in higher-dimensional spaces.
Reference graph
Works this paper leans on
-
[1]
Balestriero, R. and Baraniuk, R. Mad max: Affine spline insights into deep learning. arXiv preprint arXiv:1805.06576,
-
[4]
Chen, K. K. The upper bound on knots in neural networks. arXiv preprint arXiv:1611.09448,
-
[8]
P., Lyu, Y ., Fan, X., Wu, Z., Cheng, Y ., Wu, J., Chen, L., Wu, P., Lee, M
Liang, P. P., Lyu, Y ., Fan, X., Wu, Z., Cheng, Y ., Wu, J., Chen, L., Wu, P., Lee, M. A., Zhu, Y ., et al. Multibench: Multiscale benchmarks for multimodal representation learning. Advances in neural information processing systems, 2021(DB1):1,
work page 2021
-
[12]
Yang, X. and Wang, X. Kolmogorov-arnold transformer. arXiv preprint arXiv:2409.10594,
-
[13]
Zhang, F. and Zhang, X. Graphkan: Enhancing feature extraction with graph kolmogorov arnold networks.arXiv preprint arXiv:2406.13597,
-
[2002]
SS, S., AR, K., KP, A., et al. Chebyshev polynomial-based kolmogorov-arnold networks: An efficient architecture for nonlinear function approximation. arXiv preprint arXiv:2405.07200,
-
[2011]
Genet, R. and Inzirillo, H. A temporal kolmogorov-arnold transformer for time series forecasting. arXiv preprint arXiv:2406.02486,
-
[2013]
U-kan makes strong backbone for medi- cal image segmentation and generation
Li, C., Liu, X., Li, W., Wang, C., Liu, H., Liu, Y ., Chen, Z., and Yuan, Y . U-kan makes strong backbone for medi- cal image segmentation and generation. arXiv preprint arXiv:2406.02918,
Show all 14 references
-
[2016]
9 Free-Knots Kolmogorov-Arnold Network: On the Analysis of Spline Knots and Advancing Stability A
URL https://arxiv.org/abs/1509.01626. 9 Free-Knots Kolmogorov-Arnold Network: On the Analysis of Spline Knots and Advancing Stability A. Proof Lemma 4.2 Next, let us prove Lemma 4.2 the knot upper bound of ReLU network. Given an arbitrary initialized fully connected Multi-laye...
-
[2018]
D., Tepsich, A
Bodner, A. D., Tepsich, A. S., Spolski, J. N., and Pourteau, S. Convolutional kolmogorov-arnold networks. arXiv preprint arXiv:2406.13155,
-
[2019]
doi: 10.1038/ s41597-019-0103-9
ISSN 2052-4463. doi: 10.1038/ s41597-019-0103-9. URL https://doi.org/10. 1038/s41597-019-0103-9 . Hornik, K., Stinchcombe, M., and White, H. Multilayer feedforward networks are universal approximators. Neu- ral networks, 2(5):359–366,
-
[2020]
Xu, J., Chen, Z., Li, J., Yang, S., Wang, W., Hu, X., and Ngai, E. C.-H. Fourierkan-gcf: Fourier kolmogorov- arnold network–an effective and efficient feature transfor- mation for graph collaborative filtering. arXiv preprint arXiv:2406.01034,
-
[2021]
Y ., and Tegmark, M
Liu, Z., Wang, Y ., Vaidya, S., Ruehle, F., Halver- son, J., Solja ˇci´c, M., Hou, T. Y ., and Tegmark, M. Kan: Kolmogorov-arnold networks. arXiv preprint arXiv:2404.19756,
-
[2024]
and Chen, H
Bozorgasl, Z. and Chen, H. Wav-kan: Wavelet kolmogorov- arnold networks. arXiv preprint arXiv:2405.12832,
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.