{"id":"1ce85f44-3c9a-43e9-9a17-7f6ccb54f77c","arxiv_id":"2501.09283","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"A free-knot variant of Kolmogorov-Arnold networks reports higher accuracy with fewer parameters than fixed-grid KAN, but its central smoothing regularizer is mathematically inert and the knot bound proof is not rigorous.","lead":"This paper proposes FR-KAN, a Kolmogorov-Arnold network variant that lets each neuron shift its spline knot positions and uses a larger grid range to reduce training instability. It claims FR-KAN matches MLP parameter counts and outperforms standard KAN and MLP on eight datasets.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. (8)'s C2 regularizer differentiates the constant spline coefficients c_{j,i} with respect to x, so it is identically zero; the proposed smoothing/stability mechanism is a no-op as written.","rationale":"The reader's weakest-assumption diagnosis is correct and is the most load-bearing point I found. The central claim of the paper includes a training strategy that 'ensures C2 continuity' and improves training stability; the only formal mechanism offered for that is Eq. (8), and that mechanism vanishes because the c_{j,i} are weights, not functions of x. This is not a minor typo: it removes the purported smoothing regularizer entirely, so the stability comparisons in Figure 4 cannot be attributed to it. The parameter-reduction and free-grid ideas are independent and may retain some value, but the abstract and Section 9 explicitly credit the smooth regularizer for the stability results, so the main robustness claim is unsupported. A secondary concern is that Theorem 4.3's bound is not tight for L=1, since Lemma 4.1 gives G+K knots while the theorem's formula adds G(G-1) extra knots for a single layer; I did not make this the primary attack because the regularizer no-op is more directly tied to the paper's headline stability claim. The empirical results are not formal verification, and no machine-checked or parameter-free derivation is provided; however, my recommendation is unchanged because the reader already reached REJECT and the identified flaw is confirmed rather than weakened by a second pass.","tokens_in":13278,"tokens_out":4106,"duration_ms":41755,"concrete_test":"Instrument the released FR-KAN code and log the value of the regularizer term from Eq. (8) at every optimizer step for the STL10 configuration of Figure 4; if the logged value is identically 0.0, the term is a no-op. Then run the same experiment with lambda=0 and with a corrected penalty lambda * sum_i sum_j ||sum_k c_{k,i} B''_k(x)||^2 over a small sweep of lambda; if the accuracy/stability curves are unchanged when the proposed term is zeroed and only change with the corrected penalty, the paper's stability contribution is not realized by Eq. (8).","verdict_should_be":"UNCHANGED","load_bearing_attack":"In Section 5.3, Eq. (8) defines the regularization term as lambda * sum_i sum_j partial^2 c_{j,i} / partial x^2, where c_{j,i} is the \"spline combination weight of neuron n_i at segment g_j\". Throughout the paper these c_{j,i} are learned scalar parameters of the B-spline linear combination (Eqs. (1) and (5)); they do not depend on the input x. Hence every second derivative is exactly zero and the regularization term contributes nothing to the optimization objective. The claimed C2-continuity training strategy and the smoothing/stability improvements attributed to it in Section 5.3, Figure 4, and Section 9 are therefore unsupported by this mechanism. A correct penalty would act on the second derivative of the spline activation itself, for instance ||sum_j c_{j,i} B''_j(x)||^2, but no such term is present. This is an internal inconsistency in the paper's own formalism, not a disagreement with external consensus, and it directly undermines contribution (C3) and the abstract's stability claim. The range-expansion heuristic may still help, but the regularizer, which is presented as the way to explicitly reduce inflection points and smooth oscillation, is absent as written.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies Kolmogorov-Arnold Networks from the viewpoint of B-spline knots. It derives claimed upper bounds on the number of knots in fixed-grid KANs (Theorem 4.3) and in the proposed Free-Knot KAN (Theorem 5.1). To reduce the parameter cost of KANs, it introduces neuron grouping and weight sharing, and to improve training stability it proposes a second-derivative regularizer plus an enlarged grid range. The method is evaluated on image, text, multimodal, time-series, and function-approximation benchmarks, reporting accuracy/RMSE improvements over MLP and several KAN variants.","tokens_in":13566,"tokens_out":8045,"duration_ms":79025,"significance":"If the claims were fully substantiated, FR-KAN would be a practically useful KAN variant: it promises MLP-scale parameter counts and better stability while retaining the expressive advantages of spline-based activations. The paper ships code and covers a broad set of benchmarks, which is a strength. However, two load-bearing elements are currently not supported by the manuscript text: the regularization term in Eq. (8) is identically zero as written, and the knot-count bound in Theorem 4.3 is internally inconsistent and only sketched in the appendix. These issues directly affect contributions (C1) and (C3), so the significance cannot be assessed from the present version.","major_comments":[{"comment":"The regularization term λ Σ_i Σ_j ∂²c_{j,i}/∂x² is identically zero because c_{j,i} is defined throughout the paper (Eqs. (1) and (5)) as a learned scalar combination weight for the B-spline basis, not as a function of the input x. Every second derivative with respect to x therefore vanishes, so the claimed C2-smoothing mechanism is absent as written. The stability improvements attributed to this regularizer in Section 5.3, Figure 4, and contribution (C3) are unsupported. A valid penalty would act on the second derivative of the activation itself, for example λ∫(Σ_j c_j B''_j(x))² dx, but no such term appears in the paper.","section":"Section 5.3, Eq. (8)"},{"comment":"The asserted tight bound is inconsistent with the paper's own Lemma 4.1 and does not follow from the stated recurrence. Lemma 4.1 gives exactly G+K knots for a single-layer fixed-grid KAN, but Eq. (3) with L=1 yields the upper bound G+K+G(G−1). Moreover, the recurrence in Eq. (4) is additive in the previous-layer knot count m_{l−1}, so after L layers it gives at most about (L−1)G(G−1) additional knots, not the multiplicative product ∏_{l=1}^L G(G−1) in Eq. (3). The tightness argument in Appendix B constructs only one sawtooth example with K=1 and randomly initialized layer-2 weights; it does not establish that the bound is reachable for general L and K. Since Theorem 4.3 is contribution (C1) and motivates the proposed free-grid design, it must be corrected or removed.","section":"Section 4.2, Theorem 4.3 and Appendix B"},{"comment":"The parameter-count claim O(d_in·d_out + h(G+K)) is not supported by the free-grid description. Section 5.2 states that each neuron n_i receives a different shift b_g on the grid, which would introduce O(d_in·d_out·G) additional parameters and would also be incompatible with sharing the spline combination weights c_j across a group. If the shift is instead per group, the text should say so explicitly. This ambiguity directly affects the central claim that FR-KAN matches the parameter scale of MLPs.","section":"Section 5.1-5.2, Eqs. (5)-(6)"}],"minor_comments":[{"comment":"The abstract says \"new a training strategy\" and \"The promising results demonstrates\"; these should be corrected. Contribution (C4) states 9 datasets while the abstract and Section 6.1 list 8; use a consistent count.","section":"Abstract and Section 2"},{"comment":"The dataset enumeration mentions image, text, multimodal, and function approximation, but the experiments also include the time-series dataset ETTh1 (Figure 2); ETTh1 should be listed explicitly in Section 6.1.","section":"Section 6.1"},{"comment":"The text says three independent runs use the random seed \"2024\"; please clarify how the runs are independent if the same seed is used for all of them.","section":"Section 6.1"},{"comment":"The notation B_j(x_i)^{j/h} is undefined and appears to be a typo; the intended indexing of the shared grid within the h groups should be defined precisely.","section":"Eq. (5)"},{"comment":"There are several terminology slips: \"Universal Approximation Theorem(URT)\" should be \"UAT\", \"Kolmogorov-Arnold Representation Theorem(KAT)\" should be \"KART\" or \"Kolmogorov-Arnold theorem\", and \"two steams\" should be \"two streams\".","section":"Sections 3 and 4"},{"comment":"In row I.12.11, the formula \"q(Ef + B⊑sinθ)\" contains a garbled symbol; please correct the typesetting.","section":"Table 2"},{"comment":"The choices Z=8, grid range [-10,10], and grid size G=20 are each justified by empirical results on the same datasets used for final evaluation; a separate validation procedure or an ablation on held-out data would strengthen the claim that these settings are not overfit to the reported benchmarks.","section":"Section 5.2 and Section 8"}],"recommendation":"major_revision","confidential_remarks":"The stress-test concern about Eq. (8) is valid and is reflected in my first major comment: the regularization term is a no-op as written. My second major comment identifies an internal inconsistency between Eq. (3) and Eq. (4) that the authors need to resolve. The code release is a positive, and it may be worth asking the authors to confirm what regularization term is actually implemented, since the text and the implementation may differ. The hyperparameter selection on the evaluation datasets (Z=8, grid range [-10,10], G=20) is a further risk for the validity of the reported gains; a proper validation split or systematic ablation would be needed before the experimental claims can be regarded as reliable."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The free-knots twist is real and the empirical coverage is better than most KAN variants, but the paper's central stability mechanism is a no-op as written. Eq. (8) differentiates the spline coefficients c_{j,i} with respect to x; those coefficients are learned constants, so the regularizer is identically zero. That kills the C2-smoothing claim and most of the stability narrative in Section 5.3 and Figure 4.\n\nWhat's actually new: a learnable shift per neuron on the B-spline grid, with sorting to preserve knot order, plus neuron grouping and weight sharing to bring parameter count down to MLP scale. That combination is not in the cited prior work, and it's a plausible, cheap improvement. The parameter-matched comparisons across image, text, time series, multimodal, and Feynman function fitting are more thorough than what most KAN papers do, and FR-KAN is frequently ahead. If the stability mechanism were real, this would be a solid incremental contribution.\n\nSoft spots: the regularizer flaw is load-bearing. A correct penalty would involve something like ||sum c_j B''_j(x)||^2; no such term appears. The range-expansion heuristic (larger grid interval) might still help, but that's separate and not enough to support the stability claims. The knot bound (Theorem 4.3) is intuitive but the proof is a sketch: the tightness construction uses K=1 and a specially constructed c_j, and the L=1 case doesn't support the claimed bound. Experiments use one seed, three runs, no significance tests, and key hyperparameters (Z=8, grid range [-10,10], G=20) were tuned on the same datasets used for final evaluation—so the gains could partly reflect selection. These are real limitations but secondary; the regularizer alone is fatal to the paper as written.\n\nWho this is for: KAN-variant researchers who want a quick idea to try. The free-grid trick is worth stealing. But the paper doesn't deliver the theory or the stability mechanism it advertises.\n\nRecommendation: I'd send it to peer review rather than desk reject—the core idea is concrete and the experiments are broad enough that a reviewer can give useful feedback. But my own verdict would be reject or major revision: the regularizer must be fixed or removed with claims adjusted, the bound proof needs to be made correct or explicitly conjectural, and the experiments need significance testing or at least more seeds.","headline":"The free-knots idea is worth trying, but the regularizer that supposedly stabilizes training is identically zero as written, and the theoretical bound is not established.","tokens_in":14099,"tokens_out":2504,"would_cite":false,"duration_ms":41188,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that freeing B-spline knots per neuron, sharing weights across neuron groups, and widening the grid range makes Kolmogorov-Arnold Networks outperform MLPs and other KAN variants while keeping trainable parameters at MLP…","keywords":["Kolmogorov-Arnold networks","B-spline","free knots","spline knot bound","parameter efficiency","training stability","second derivative regularization","function approximation"],"falsifier":"Run FR-KAN with and without the second-derivative term while keeping all other settings fixed: if the training curves and final accuracy are indistinguishable, the regularizer is not doing the stated smoothing work. A direct inspection of the released code for $\\partial^2 c_{j,i}/\\partial x^2$—if it is identically zero—would settle the same point.","tokens_in":13069,"feed_emoji":"🧮","tokens_out":8271,"duration_ms":73220,"temperature":0.7,"pith_summary":"This paper argues that the fixed, shared B-spline grid is the main bottleneck of Kolmogorov-Arnold Networks, and that freeing the knot positions per neuron—while sharing weights across neuron groups—makes KANs both more accurate and as parameter-efficient as ordinary MLPs. It derives a tight upper bound on the number of spline knots a KAN can produce, showing the count depends on grid size, spline order, and depth, not on width. To stabilize training, it adds a second-derivative regularizer intended to smooth activations and recommends a wider grid range than the usual [-1,1]. Across eight datasets spanning image, text, time-series, multimodal, and symbolic-regression tasks, the proposed FR-KAN is reported to beat ReLU-MLP and several KAN variants at matched parameter budgets.","feed_headline":"Free-knot KANs beat MLPs with MLP-scale parameters","feed_subtitle":"Per-neuron spline knots plus shared weights cut KAN parameters to MLP level and lift accuracy on image, text, and symbolic-regression…","key_machinery":"The load-bearing object is the free-knot B-spline activation: each neuron keeps its own grid $G^* = \\mathrm{Sort}(G + b_g)$, where $b_g$ is a small learnable per-grid-point bias, and the grid is sorted to preserve B-spline order. Neuron grouping shares one grid among $h$ neurons and shares the linear-combination weight $A$ between spline and SiLU, decoupling the parameter count from the grid size. The theoretical engine is Theorem 4.3's tight bound on spline knots, which depends only on grid size $G$, spline order $K$, and depth $L$; it justifies freeing knots from the shared grid. A second-derivative regularizer $\\lambda \\sum_i \\sum_j \\partial^2 c_{j,i}/\\partial x^2$ and an expanded grid range are introduced as stability mechanisms.","core_discovery":"The paper's central claim is that a KAN whose B-spline knots are learnable per neuron—shifted by a small trainable bias and then re-sorted—has more expressive power than the original fixed-grid KAN, and that this gain can be obtained without the original's heavy parameter cost. FR-KAN groups neurons and shares the spline and SiLU weights, reducing the trainable-parameter count from $O(d_{\\rm in} d_{\\rm out}(G+K+1))$ to $O(d_{\\rm in} d_{\\rm out} + h(G+K))$, i.e. MLP scale. The paper derives a tight knot-count bound: a fixed-grid KAN has at most $(G+K)+\\prod_{l=1}^{L} G(G-1)$ knots, while free-knot KAN has up to $h$ times that, with $h$ the number of neuron groups. In the reported experiments, FR-KAN outperforms ReLU-MLP, original KAN, Fourier-KAN, Rational-KAN, and RBF-KAN on most of the eight evaluated datasets while matching MLP's parameter scale.","pith_inferences":["Because the regularizer penalizes $\\partial^2 c_{j,i}/\\partial x^2$ but the spline weights $c_{j,i}$ are trained constants, the penalty is identically zero as written; an ablation with and without $\\lambda$ would reveal whether any stability gain actually comes from this term or only from the wider grid range.","A natural next step not explored in the paper is data-dependent initialization of the grid shift $b_g$ (for instance, placing knots at input quantiles) instead of uniform random, which could sharpen the approximation benefit.","The knot-count bound suggests a concrete architecture rule—raise grid size or depth before width when using fixed-grid KANs—that could be tested on larger vision and language benchmarks.","If the parameter savings transfer to convolutional or transformer KAN hybrids, the same grouping-plus-free-grid recipe may provide a low-cost route to scaling KANs beyond the MLP-style settings evaluated here."],"forward_implications":["If the knot bound and free-grid gains hold, KANs can be deployed in standard deep-learning pipelines without the customary parameter blow-up.","The bound implies that increasing width alone cannot create new knots in a fixed-grid KAN; depth, grid size, or free knot shifts are needed to grow expressive capacity.","Weight-sharing across neuron groups plus per-group free grids offers a reusable recipe for scaling other spline-based KAN variants.","Reported results indicate spline-based KANs remain competitive with kernel-based KANs when the grid is learnable, on both classification and symbolic regression.","A wider grid range removes the NaN-loss failure observed with narrow grids, making KAN training sufficiently stable for deeper stacks."],"supporting_citations":[{"why":"Defines the original KAN architecture, B-spline activation, and Feynman evaluation protocol that FR-KAN extends and compares against.","marker":"(Liu et al., 2024)"},{"why":"Supplies the ReLU-MLP spline-knot upper bound that Lemma 4.2 and Theorem 4.3 adapt to KANs.","marker":"(Chen, 2016)"},{"why":"Provides the fact that a ReLU neuron creates or destroys a knot depending on input sign, which Lemma 4.1 uses.","marker":"(Pinkus, 1999)"},{"why":"Contributes the observation that KANs are hard to scale and the neuron-grouping idea that motivates shared-weight free-grid design.","marker":"(Yang & Wang, 2024)"},{"why":"Frames MLPs as affine spline operators, the perspective the paper applies to KANs throughout.","marker":"(Balestriero & Baraniuk, 2018)"},{"why":"Supplies the CIFAR-10 and CIFAR-100 benchmark datasets used in the controlled parameter comparisons.","marker":"(Krizhevsky et al., 2009)"}],"fun_headline_variants":["Free-knot KAN matches MLP parameters with better accuracy","FR-KAN: MLP-scale parameters, superior performance","Free knots cut KAN params to MLP level, boost accuracy","Learnable spline knots shrink KAN to MLP parameter count","KAN with free knots beats MLP at equal parameter count"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The stability mechanism assumes each spline coefficient $c_{j,i}$ is a function of the input $x$, so that penalizing its second derivative smooths the activation; in the implementation these coefficients are learned constants, making the penalty zero and leaving the wider grid range as the only stated source of stability.","fun_headline_variants_meta":{"raw":{"variants":["Free-knot KAN matches MLP parameters with better accuracy","FR-KAN: MLP-scale parameters, superior performance","Free knots cut KAN params to MLP level, boost accuracy","Learnable spline knots shrink KAN to MLP parameter count","KAN with free knots beats MLP at equal parameter count"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000191,"raw_usage":{"total_tokens":1375,"prompt_tokens":1009,"completion_tokens":366,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":625,"completion_tokens_details":{"reasoning_tokens":279}},"tokens_in":625,"tokens_out":366,"duration_ms":4010,"temperature":1.0,"reasoning_tokens":279,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T20:06:03.329644+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run FR-KAN with and without the second-derivative term while keeping all other settings fixed: if the training curves and final accuracy are indistinguishable, the regularizer is not doing the stated smoothing work. A direct inspection of the released code for $\\partial^2 c_{j,i}/\\partial x^2$—if it is identically zero—would settle the same point.","supporting_citations":[],"review_version":1}