Pith. sign in

REVIEW 3 major objections 6 minor 13 references

Kolmogorov-Arnold network training can be parallelized by training copies on disjoint data subsets and merging their parameters, yielding near-linear speedups and a fixed-point FPGA implementation that trains 1,024 records in about 0.06 ms.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

T0 review · deepseek-v4-flash

2026-08-03 14:50 UTC pith:KVTCTND5

load-bearing objection Solid engineering results on parallelizing NK-KAN training, but the FPGA 'training' claim is unproven and the headline speedup is inflated. the 3 major comments →

arxiv 2512.18921 v5 pith:KVTCTND5 submitted 2025-12-21 cs.LG

Concurrent training methods for Kolmogorov-Arnold networks: Disjoint datasets and FPGA implementation

classification cs.LG
keywords Kolmogorov-Arnold networksNewton-Kaczmarz methodparallel trainingdisjoint datasetsmodel mergingfield-programmable gate arraysfixed-point arithmeticpiecewise-linear basis functions
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper sets out to remove the sequential bottleneck in Newton-Kaczmarz training for Kolmogorov-Arnold networks, where every parameter update depends on the previous step. It proposes three mechanisms: pre-training on groups of model addends, concurrent training on disjoint data subsets followed by merging models through parameter averaging, and a division-free fixed-point variant for FPGAs. On standard laptop hardware, the combined pre-training and disjoint-data approach is reported to be about 30 times faster than the sequential baseline and about seven times faster than GPU-accelerated competitors, with near-linear strong scaling up to six threads. An FPGA implementation is reported to process 1,024 records in roughly 0.06 ms. A sympathetic reader would take the main claim to be that this training algorithm is no longer inherently sequential, and that its natural parallelism is usable on ordinary CPUs and on simple FPGA boards.

Core claim

The central claim is that Newton-Kaczmarz training of KANs, although formulated as a sequence of dependent updates, can be parallelized effectively. Training several copies of the same model on disjoint subsets of the data and then averaging every parameter across copies preserves predictive accuracy closely enough that the total work scales almost linearly with the number of threads. A new pre-training step splits the model's addends into groups, trains those groups concurrently, and assembles them with rescaling, which the authors report gives a further substantial speedup. For FPGAs, rescaling the inputs and choosing the node spacing as a power of two replaces division with binary shifts

What carries the argument

The Newton-Kaczmarz layer update rule: for each training record, the layer parameters are updated by damped gradient steps such as G_new = G_old + μ(z_i - \hat z_i)(1-f) and G_new = G_old + μ(z_i - \hat z_i)f, with residuals propagated to the preceding layer via the chain rule. This update is linear in the parameters and local to each basis function, which makes three mechanisms natural: pre-training on groups of addends, concurrent training on disjoint batches with merging by averaging, and fixed-point arithmetic where power-of-two scalings turn divisions into shifts. The merging step carries the argument: averaging all parameters of independently trained copies is asserted to approximate t

Load-bearing premise

The entire speedup story rests on the assumption that averaging the parameters of models trained on different data subsets closely reproduces the model that would have been trained sequentially; the paper reports this holds only approximately, with accuracy dropping as the number of threads increases, and provides no theoretical guarantee.

What would settle it

Run the same KAN on a fixed dataset both sequentially and with, say, six threads using disjoint batches and the same total number of records, then compare validation Pearson correlation; if the merged model's correlation falls clearly below the sequential model's and additional rounds cannot recover it within the sequential run time, the near-linear speedup claim collapses. For the FPGA part, measure the cycle count for a model with more layers: if the number of cycles per record grows with layer size rather than remaining a small fixed constant, the stated 'small fixed number of sequential st

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If the claims hold, KAN training on a laptop CPU becomes faster than GPU-accelerated training of conventional neural networks on the same examples, by up to roughly seven times when concurrency is used.
  • The pre-training and disjoint-merge strategies are described as independent of the basis-function choice, so the speedups should transfer to spline-based or other KAN variants; only the FPGA variant is specific to piecewise-linear functions.
  • The reported accuracy loss from merging is modest (about one to two percent Pearson correlation with six threads) and can be offset by more rounds, at the cost of reducing speedup, giving a tunable trade-off.
  • FPGA training at about 0.06 ms per 1,024 records would make on-device training practical for small models, not merely inference, and the fixed-point design avoids division entirely.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The parameter-averaging merge is the load-bearing simplification; an editorial extension would be to test whether the accuracy loss with thread count follows a predictable curve from the loss landscape's curvature, so the optimal batch count could be chosen without exhaustive runs.
  • The pre-training idea is a form of model parallelism and could be combined with the data-parallel disjoint training to scale further, a combination the paper does not test.
  • The fixed-point FPGA training suggests a two-stage pipeline—train on the device in fixed point, then refine in floating point—which the paper leaves implicit and which could be benchmarked directly.
  • Because the disjoint batches are random splits of one dataset, the method's behavior under non-identically-distributed splits (as in federated settings) remains untested; the merge-by-averaging mechanism may behave differently there.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes three concurrency-oriented enhancements to the Newton-Kaczmarz (NK) training algorithm for Kolmogorov-Arnold networks (KANs). The first is a pre-training procedure that trains groups of addends separately and then assembles them with scaling. The second is training on disjoint subsets of the data in parallel, followed by merging the resulting models by averaging all parameters. The third is a fixed-point, division-free FPGA implementation of the training loop, claimed to be the first on-device KAN training implementation. The authors report CPU/GPU comparisons against MATLAB, FastKAN, and Keras on a determinant-prediction task (Det4) and a tetrahedron-area task (Tetra), strong and weak scaling tests on a laptop and an HPC cluster, respectively, and an FPGA run on two norm-prediction datasets. The central claims are substantial training speedups (up to ~30× over baseline CPU tests and ~7× over GPU baselines) and near-linear parallel scaling, with source code made available.

Significance. If the claims are fully validated, the paper would make a useful contribution to the growing KAN literature: the pre-training and disjoint-merge strategies are simple and potentially portable to other basis choices, and an actual FPGA training implementation would be a notable step beyond the existing FPGA-inference work. The strong scaling results on an HPC node (Table 4) are encouraging and, together with the released C++ and RTL code, give the paper a reproducible-engineering character that is valuable. However, the significance is bounded by two gaps. First, the FPGA section reports only cycle counts and wall-clock time, with no accuracy, loss, or comparison to a fixed-point software reference, so the paper's most distinctive claim—on-device training—is not actually demonstrated. Second, the merge-by-averaging premise shows a measurable accuracy drop that is acknowledged but not explained; the paper's own Table 3 indicates that restoring accuracy requires roughly doubling the compute, reducing the real-world speedup to about 2.4×. These issues are local and fixable with additional experiments or recalibrated claims, but they currently prevent the paper from supporting its headl

major comments (3)
  1. [§4.5] The FPGA validation is missing the essential evidence for 'training'. The section reports only that 1024 records were 'processed' in 3 cycles per record over 2 epochs, totaling 6144 cycles at 100 MHz, giving 0.06144 ms. No validation accuracy, loss, or any error metric is reported, and no comparison to a fixed-point software reference is made. The text also states that 'the same number of latches is used for both the 4-function and 16-function models', which suggests a structural claim but does not show that parameter updates occur or that the residual decreases. Consequently, the Abstract's 'implemented and tested directly on the device' and the Introduction's 'first implementation of KANs' training on FPGAs' are not supported by the reported results. Please add at least (a) a convergence/accuracy measure on the two norm datasets, (b) a comparison with a fixed-point C++ or MATLAB implem
  2. [§4.1, Table 1] The 'approximately 30 times' speedup summary is not supported by the authors' own baseline. From Table 1, the C++ sequential time is 6.57 s and the disjoints+pre-training Linux time is 0.70 s, which is a 9.4× speedup, not ~30×. The ~30× figure appears to be relative to the MATLAB CPU time (32.7 s), which is a different baseline and is not labelled as such in the Abstract or Section 4.1. The Conclusion's 'up to a 7 times speedup' refers to yet another comparison (GPU time). Please state explicitly for each speedup claim which baseline is used and avoid conflating CPU/GPU or MATLAB/FastKAN/Keras baselines. The underlying speedup over the sequential C++ implementation is still substantial, but the current wording is misleading.
  3. [§3.2 and Table 3] The central hypothesis that disjoint-batch training with averaging gives near-ideal parallel speedup is only partially supported. Table 3 shows Pearson correlation dropping from 96.8% (1 thread) to 94.5% (6 threads) at the same total workload, and the text concedes that 20 rounds with 6 threads restores 97.0% but requires twice the computational work, yielding a 2.4× speedup over 1 thread. This discrepancy is not merely a scaling artifact; it reflects the fact that parallel threads update from a stale model, as the authors note. The paper currently provides no quantitative model or theoretical bound for this accuracy drop, nor a principled rule for choosing the number of rounds/batches. Given that the speedup claim is the main practical contribution of strategies (ii), this gap should be addressed, at least with a more careful discussion of the accuracy/compute trade-off and a statement
minor comments (6)
  1. [§3.2] Typo: 'model coping/merging' should read 'model copying/merging'.
  2. [§4.5] The phrase 'same number of latches' is unclear—do you mean the same number of hardware resources/registers? Please clarify the resource utilization and the meaning of 'latches' in this context.
  3. [Table 2] The header row appears to have an extra entry: it lists 'Pearson (%) time (s)' followed by four accuracy columns and one time column. Please align the columns and label the four outputs (e.g., Face 1–4).
  4. [§4.4, Table 4] The definition of efficiency E = t(1)/t(n) is non-standard for weak scaling, where ideal efficiency is 100% when time is constant. The text explains this correctly, but the formula should be stated as E = t(1)/t(n) with t(n) the time for n times the work, not 't(n)' as currently implied; please make the notation explicit in the text.
  5. [§3.1] The pre-training procedure is described for the two-layer and three-layer cases, but the scaling factor p/v is introduced without a derivation. Since this is a central component of strategy (i), a short derivation or reference to a prior derivation would improve reproducibility.
  6. [§4.1] The discussion of FastKAN's default configuration is lengthy and somewhat editorial ('Even first-year undergraduate students are taught...'). While the parameter-count mismatch is a valid concern, the tone could be more neutral and the point more concise.

Circularity Check

0 steps flagged

No circularity found: the paper's contributions are empirical engineering results, not quantities derived from fitted inputs or from a self-citation chain.

full rationale

The paper's core claims are measured outcomes: training-time comparisons (Tables 1 and 2), strong/weak scaling measurements (Tables 3 and 4), and FPGA cycle counts (Section 4.5). None of these are derived by construction from fitted parameters or from equations equivalent to their inputs. The merge-by-averaging strategy in Section 3.2 is explicitly called a hypothesis and is tested experimentally, with the accuracy loss openly reported in Table 3. The pre-training procedure in Section 3.1 is a training-scheme proposal, not a prediction derived from a fitted quantity. The self-citations to the authors' prior work [3,5] motivate the underlying NK-KAN training method, but the paper re-establishes the baseline independently against external packages (MATLAB, FastKAN, Keras) in Table 1, so the efficiency premise does not rest solely on self-citation. The FPGA section reports cycle counts but no validation accuracy; this is a missing-validation/correctness concern, not a circularity, since no claimed equation reduces to a fitted or self-cited quantity.

Axiom & Free-Parameter Ledger

3 free parameters · 6 axioms · 0 invented entities

The central experimental claims rest on four domain assumptions: Kaczmarz convergence inherited from [3], the merge-by-averaging hypothesis (Section 3.2), batch representativeness, and fixed-point fidelity (Section 3.3.1). The merge hypothesis is the most fragile: Table 3 shows a real accuracy penalty (2.3 points at 6 threads) that is only partly compensated by extra compute. No convergence theory supports any of the three strategies; the paper is purely empirical. Free parameters are limited to manual hyperparameters (learning rate, batch/round counts, architecture), none fitted to data.

free parameters (3)
  • learning rate / scaling factor µ = 1/64 on FPGA (Section 3.3.1); not tabulated for CPU runs
    Step size in Eqs. (5)-(6); chosen by hand. The authors state that 1/64 'instead of 0.02 does not lead to any noticeable degradation', which is an empirical claim without reported accuracy numbers.
  • batch size and number of rounds (threads) = 50K down to 3K records; 10-40 rounds (Table 3); 50K-100K records, 10-200 batches (Table 4)
    Tuning knobs for the disjoint-merge strategy; the accuracy-efficiency tradeoff is explored empirically, with no selection rule or theory given (Sections 3.2, 4.3-4.4).
  • network architecture (blocks per layer, points per function) = e.g. 70 blocks, 3 pts/function -> 5,460 params (Det4); 160/3 + 1/30 -> 16,800 (Det5); 70/3, 30/18, 4/22 (Tetra)
    Manual configuration in Section 4. Comparisons with MLP baselines use very different parameter counts (621-124K), so part of the relative speed is architectural.
axioms (6)
  • standard math Kolmogorov-Arnold representability: the target function can be expressed as Eq. (1) and a layered KAN of form (2) with piecewise-linear g_i,j.
    Justifies the KAN architecture (Eqs. 1-2, Section 2); standard approximation/representation theorem.
  • domain assumption Newton-Kaczmarz iterative updates (Eqs. 5-7) converge, and the chain-rule residual backpropagation (Eq. 7) yields correct layer targets.
    Convergence is cited to the authors' prior paper [3] (Machine Learning 2025) and not re-proven here; the training loop depends on it.
  • domain assumption Parameter averaging of models trained on disjoint batches yields a merged model with accuracy close to the sequential model.
    Section 3.2: 'the models are merged into a single model simply by computing the mean value of every parameter.' Table 3 quantifies the validity gap (96.8% -> 94.5% at 6 threads); no convergence theory is offered.
  • domain assumption Each disjoint batch is 'sufficiently representative of the modelled system' so that averaging works.
    Section 3.2 ('One should choose the number of batches and the batch size'); restricts the method to large datasets with i.i.d.-like structure. The Det4/Det5/Tetra synthetic datasets satisfy this by construction; real data may not.
  • domain assumption Fixed-point integer arithmetic with power-of-2 domain scaling preserves training dynamics close enough to floating point.
    Section 3.3.1 replaces division by shifts and uses a 1/64 learning rate; no accuracy comparison between fixed-point FPGA training and float training is reported (Section 4.5).
  • ad hoc to paper Pre-trained sub-models assembled with scaling by p/v approximate the original model (Eq. 8).
    Section 3.1: trained sub-models each approximate target z and are then assembled 'with the appropriate scaling (by p/v)'. The scaling is not derived; it relies on additivity of the block outputs.

pith-pipeline@v1.3.0-alltime-deepseek · 9552 in / 22816 out tokens · 205610 ms · 2026-08-03T14:50:45.012326+00:00 · methodology

0 comments
read the original abstract

The present paper introduces concurrency-driven enhancements to the training algorithm for the Kolmogorov-Arnold networks (KANs) that is based on the Newton-Kaczmarz (NK) method. Prior research shows that KANs trained using the NK-based approach outperform classical neural networks (multilayer perceptrons - MLPs) both in terms of accuracy and training time. Up to now, the fundamental limitation of the algorithm has been the sequential computation of the updates - each update depends on the results of the previous step, obstructing parallelisation; even though parallelisation of some parts of the algorithm, such as the evaluation of the basis functions, has already been proposed and tested. However, substantial acceleration is achievable. Three complementary concurrency-driven novel strategies are proposed in the present paper: (i) a pre-training procedure tailored to the NK updates' structure, (ii) training on disjoint subsets of data together with models' merging, and (iii) a division-free customisation of the algorithm for field-programmable gate arrays (FPGAs), which is implemented and tested directly on the device. Computational experiments are used to assess the scalability and to compare the authors' KAN implementation with popular MLP packages. All presented experimental results are fully reproducible, with the complete source codes available online.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

13 extracted references · 5 linked inside Pith

  1. [1]

    Z. Liu, Y. Wang, S. Vaidya, F. Ruehle, J. Halverson, M. Soljaˇ ci´ c, T. Y. Hou, and M. Tegmark. KAN: Kolmogorov-Arnold Networks. arXiv:2404.19756, 2024

  2. [2]

    Igelnik and N

    B. Igelnik and N. Parikh. Kolmogorov’s spline network.IEEE Transactions on Neural Networks, 14(4):725–733, 2003

  3. [3]

    Poluektov and A

    M. Poluektov and A. Polar. Construction of the Kolmogorov-Arnold networks using the Newton- Kaczmarz method.Machine Learning, 114(8):185, 2025

  4. [4]

    Montanelli and H

    H. Montanelli and H. Yang. Error bounds for deep ReLU networks using the Kolmogorov-Arnold superposition theorem.Neural Networks, 129:1–6, 2020

  5. [5]

    Polar and M

    A. Polar and M. Poluektov. A deep machine learning algorithm for construction of the Kolmogorov- Arnold representation.Engineering Applications of Artificial Intelligence, 99:104137, 2021

  6. [6]

    Kolmogorov–arnold networks are radial basis function networks

    Ziyao Li. Kolmogorov–arnold networks are radial basis function networks. 2024

  7. [7]

    Vaca-Rubio, Luis Blanco, Roberto Pereira, Marius Caus, and Abdullah Aydeger

    Engin Zeydan, Cristian J. Vaca-Rubio, Luis Blanco, Roberto Pereira, Marius Caus, and Abdullah Aydeger. F-kans: Federated kolmogorov-arnold networks.arXiv preprint, abs/2407.20100, 2024. arXiv:2407.20100

  8. [8]

    Evaluating federated kolmogorov-arnold networks on non-iid data.arXiv preprint, abs/2410.08961, 2024

    Arthur Mendonca Sasse and Claudio Miceli de Farias. Evaluating federated kolmogorov-arnold networks on non-iid data.arXiv preprint, abs/2410.08961, 2024. arXiv:2410.08961

  9. [9]

    Enhancing federated learning with kolmogorov- arnold networks: A comparative study across diverse aggregation strategies.arXiv preprint, abs/2505.07629, 2025

    Yizhou Ma, Zhuoqin Yang, and Luis-Daniel Ibanez. Enhancing federated learning with kolmogorov- arnold networks: A comparative study across diverse aggregation strategies.arXiv preprint, abs/2505.07629, 2025. arXiv:2505.07629

  10. [10]

    A unified benchmark of federated learning with kolmogorov-arnold networks for medical imaging.arXiv preprint, abs/2504.19639, 2025

    Youngjoon Lee, Jinu Gong, and Joonhyuk Kang. A unified benchmark of federated learning with kolmogorov-arnold networks for medical imaging.arXiv preprint, abs/2504.19639, 2025. arXiv:2504.19639

  11. [11]

    KANELE: Kolmogorov-Arnold Networks for Efficient LUT-based Evaluation.arXiv preprint arXiv:2512.12850, 2025

    Duc Hoang, Aarush Gupta, and Philip Harris. KANELE: Kolmogorov-Arnold Networks for Efficient LUT-based Evaluation.arXiv preprint arXiv:2512.12850, 2025

  12. [12]

    V. V. Krylov. Models of sampled data systems having an infinite-dimensional state space.Automation and Remote Control, 40(5):694–699, 1979

  13. [13]

    T. J. Hastie and R. J. Tibshirani.Generalized additive models. Chapman and Hall/CRC, 1990. 1http://openkan.org/downloadskan/matlabscripts.zip 2https://github.com/andrewpolar/kankan-8 3https://github.com/andrewpolar/meanmodel 4https://github.com/andrewpolar/fpga 5http://openkan.org 9