REVIEW 4 major objections 5 minor 39 references
"KAN you hear me?" Exploring Kolmogorov-Arnold Networks for Spoken Language Understanding
T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read KAN layers can replace linear layers in spoken-language understanding models, with a single KAN between two linear layers giving the best results.
desk verdict First systematic KAN-for-SLU study with real configuration search; the headline gains are partly selection effects, but the paper is honest and worth refereeing. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The key object is the KAN layer, a neural layer whose learnable univariate functions are implemented as a weighted combination of a basis function (Swish) and B-splines, per the efficient approximation in Eq. (2). The Kolmogorov-Arnold representation theorem motivates the layer: any multivariate continuous function can be written as sums of composed univariate functions, so KANs aim to learn these univariate pieces directly. The paper's central design choice is the FKF configuration, which places exactly one KAN layer between two linear feed-forward layers; this embedding lets linear layers perform coarse projection while the KAN captures localized nonlinear structure. The authors also use a recomputation trick that avoids expanding the input tensor to shape $(B, d_{out}, d_{in})$, turning the KAN computation into matrix multiplication and keeping memory and speed close to the linear baseline.
What would settle it
Retrain the FFF, FFK, and FKF configurations on the same five datasets with at least ten random seeds and compare F1 distributions; if the FKF advantage over FFF overlaps zero within confidence intervals on the transformer backbones, the central claim of superior performance would collapse to 'comparable,' and on datasets where the point estimate favors FFF it would disappear.
Extended reading notes
Core claim
The central discovery is that KAN layers are a viable drop-in replacement for linear layers in SLU classification heads, provided they are embedded rather than used alone. Across five datasets and two backbone families, the configuration with one KAN layer sandwiched between two feed-forward layers (FKF) achieves the best or near-best F1 macro scores, beating the all-linear baseline on FSC, Timers and Such, ITALIC, and SPEECH-MASSIVE, and matching it on SLURP. A KAN-only head performs worse than the linear baseline while adding parameters, showing that the benefit comes from combining KAN nonlinearity with linear projections. The authors also show that replacing the final-layer attention patterns changes which words drive predictions, making errors more human-plausible in their examples.
Load-bearing premise
The load-bearing premise is that the reported F1 differences between the FKF configuration and the all-linear baseline are stable across random initializations and not just noise; the paper does not state how many training runs each number averages over.
Editorial extensions
If this is right
- If the FKF configuration is adopted, SLU systems can gain small F1 improvements without increasing model size or training time.
- The FKF pattern transfers from 2D-CNNs to wav2vec 2.0 and XLS-R transformers, suggesting the placement finding is architecture-agnostic.
- B-spline basis functions are the best among the tested approximations for this task, so future KAN-SLU work should start with B-splines.
- The attention analysis indicates that KAN-equipped heads attend to more semantically relevant words, which could make errors easier to interpret.
Reading between the lines
- If the central claim holds, a natural next step is to test FKF on larger SLU benchmarks or on out-of-distribution speech to see whether the small gain persists when the classifier head has more capacity.
- The observed shift in word-level attention could be studied as a regularizing side effect; one could test whether FKF improves robustness to background noise or mispronunciations.
- The paper's comparisons are on intent classification only; the same dense-block substitution could be tried on slot filling or dialogue act classification, where the head structure is similar.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper investigates whether Kolmogorov-Arnold Network (KAN) layers can replace linear layers in the classification heads of Spoken Language Understanding (SLU) models. It first searches over five FF/KAN configurations on a 2D-CNN trained on FSC and Timers and Such, identifies FKF (a KAN layer between two linear layers) as the best configuration, and ablates the KAN hidden size and approximation function. It then transfers FKF to wav2vec 2.0 and XLS-R and reports F1-macro results on five SLU datasets, followed by a qualitative analysis of word-level explanations on Timers and Such. The central empirical claim is that KAN layers can effectively replace linear layers with comparable or superior performance in most cases.
Significance. The paper provides a systematic configuration search for KAN placement in SLU classification heads, releases code, and includes ablations over hidden size and approximation function. If the reported gains are reproducible, the result is a modest but useful architectural substitution for SLU models. The main limitation is statistical: the headline transformer comparison reuses the datasets used for model selection, no seed count or significance testing is reported, and most cross-dataset differences are within one standard deviation of the baseline. The paper also contains an incorrect definition of the KAN layer in Eq. (1).
major comments (4)
- [§5.2, Table 4] The claim that FKF 'generalizes well' across transformer architectures is not statistically supported by the reported numbers. FKF was selected using the 2D-CNN on FSC and Timers and Such (Tables 1–3), and Table 4 reuses those two datasets; the largest FKF-vs-FFF deltas (+0.005 and +0.016 F1) occur there. On the datasets not used in selection (SLURP, ITALIC, SPEECH-MASSIVE de-DE and fr-FR), the deltas are -0.008, +0.002, +0.008, and +0.007, all within one standard deviation of the FFF baseline. The paper neither states the number of seeds nor reports paired significance tests, so these differences cannot be distinguished from run-to-run noise. Please report seed counts and significance tests, or explicitly cast FSC and Timers and Such as selection data and evaluate the generalization claim only on the untouched datasets.
- [§5.2, Table 4; §4] The exact KAN configuration transferred to the transformer models is not specified. Section 5.1 shows that FKF performance depends strongly on the hidden-size design (fixed vs variable) and hidden size (Table 2), with the best 2D-CNN result at FKF(b) with 512 hidden units (0.631 accuracy vs 0.555 for FFF). The model sizes in Table 4 (e.g., 95.4M vs 94.6M for wav2vec2) suggest a much smaller KAN head, but the paper never states which variant, (a) or (b), and which hidden size were used in the transformer experiments. Please specify the exact architecture and justify the choice relative to the ablation.
- [§5.3, Fig. 3] The attention analysis is based on a single example and the statement that the 'pattern holds across other misclassified samples' is not accompanied by any quantitative aggregation, error counts, or multi-run evidence. This overstates the reliability of the explanation-level conclusion. Please provide aggregate statistics (e.g., the number of corrected cases and the proportion that follow the stated pattern) or soften the claim to an anecdotal observation.
- [Tables 1–4] All tables report standard deviations but no number of seeds or runs. The abstract's 'comparable or superior performance in most cases' is a claim about average performance; without knowing whether the reported values are single runs or averages over N seeds, the standard deviations cannot be interpreted and the claim is not falsifiable from the reported numbers. Add the number of runs to every table caption and, ideally, report paired significance tests between configurations.
minor comments (5)
- [§2, Eq. (1)] The displayed formula for a KAN layer appears to use products over input dimensions; a KAN layer is a sum of univariate functions. The equation should read L(x) = sum over i of phi_{i,o}(x_i), not a product.
- [§5.2, first paragraph] The text says FFK and FKF 'consistently outperform FFF in most datasets,' but on SLURP both FFK and FKF have lower F1 than FFF (0.534 and 0.531 vs 0.539). Please rephrase to reflect the actual pattern.
- [§4, Models] The footnote markers in 'wav2vec 2.02' and 'XLS-R3' render poorly in the manuscript; use proper citation or inline references.
- [Table 2, §5.1] The notation FKF(a) and FKF(b) is introduced only in the table caption; define these two variants in the text before referring to them in the hidden-size analysis.
- [Introduction, last bullet] The phrase 'without increasing model size or training time' is not exact: FKF is 6.3M vs 6.2M in Table 1 and 95.4M vs 94.6M in Table 4. Please say 'without substantially increasing model size or training time.'
Circularity Check
No circularity: the KAN-vs-linear comparison is an empirical benchmark study; the FKF selection on FSC/Timers and reuse of those datasets in the transformer table is a selection-bias concern, not a circular reduction.
full rationale
The paper does not contain a derivation chain in which an output is defined in terms of the quantity it claims to predict. Its central claim—that a KAN layer placed between two linear layers (FKF) gives comparable or superior SLU performance—is supported by benchmark experiments on external datasets (FSC, Timers and Such, SLURP, ITALIC, Speech-MASSIVE). The FKF configuration is selected from the 2D-CNN results in Table 1 on FSC and Timers and Such, and the same two datasets reappear in the transformer results of Table 4. This reuse weakens those two rows as independent evidence, but it is not circularity: the transformer F1 values are not determined by the CNN-based selection, and the paper additionally reports three datasets (SLURP, ITALIC and the two Speech-MASSIVE splits) that were not used in any selection step. Self-citations (e.g., the ITALIC dataset [29] and the word-level explanation method [31]) are used as data sources or analysis tools, not as load-bearing justification for the main result. The approximation-function choice (B-spline) is likewise selected on FSC and then reused, but again this is hyperparameter selection rather than a fitted parameter renamed as a prediction. Statistical concerns about seed counts and small differences belong to correctness risk, not circular reasoning. Therefore no step qualifies as circular under the stated definitions.
Assumptions & free parameters
free parameters (4)
- KAN hidden/output size in FKF(b) =
512 chosen as best on FSC
- Approximation function within KAN layers =
B-Spline selected as best on FSC
- FKF as the chosen architecture =
Architecture selected on FSC and Timers and Such
- Number of training epochs and early stopping =
Max 20 epochs for CNN, max 50 for transformers, early stopping patience 5
assumptions (4)
- standard math The Kolmogorov-Arnold representation theorem justifies KAN layers as universal approximators of continuous functions.
- domain assumption Pretrained wav2vec 2.0 and XLS-R features are adequate frozen or fine-tuned representations for SLU on all five datasets.
- domain assumption The explanation method of Pastor et al. [31] assigns word-level relevance scores that are meaningful for comparing attention patterns.
- domain assumption Reported differences in Table 4 are treated as meaningful even though no variance or seed count is reported.
Cite this review
Pith. "Pith review of "KAN you hear me?" Exploring Kolmogorov-Arnold Networks for Spoken Language Understanding." pith.science (2026). https://pith.science/paper/BKX6RAEM
@misc{pith2026250520176,
author = {Pith},
title = {Pith review of: "KAN you hear me?" Exploring Kolmogorov-Arnold Networks for Spoken Language Understanding},
year = {2026},
howpublished = {\url{https://pith.science/paper/BKX6RAEM}},
note = {Machine review of arXiv:2505.20176}
}
read the original abstract
Kolmogorov-Arnold Networks (KANs) have recently emerged as a promising alternative to traditional neural architectures, yet their application to speech processing remains under explored. This work presents the first investigation of KANs for Spoken Language Understanding (SLU) tasks. We experiment with 2D-CNN models on two datasets, integrating KAN layers in five different configurations within the dense block. The best-performing setup, which places a KAN layer between two linear layers, is directly applied to transformer-based models and evaluated on five SLU datasets with increasing complexity. Our results show that KAN layers can effectively replace the linear layers, achieving comparable or superior performance in most cases. Finally, we provide insights into how KAN and linear layers on top of transformers differently attend to input regions of the raw waveforms.
Figures
Reference graph
Works this paper leans on
-
[1]
Introduction Kolmogorov-Arnold Networks (KANs) have recently emerged as an alternative to traditional neural architectures, offering ad- vantages in modeling complex, nonlinear relationships through learnable activation functions [1]. Inspired by the Kolmogorov- Arnold representation theorem [2–4], KANs replace fixed acti- vation functions with learnable ...
-
[2]
"KAN you hear me?" Exploring Kolmogorov-Arnold Networks for Spoken Language Understanding
Preliminaries A KAN layer L(x) models continuous functions by decompos- ing them into learnable univariate transformations. Formally, for an input vector x ∈ RI and output dimension O, it can be expressed as: L(x) = Φ ◦ x = IY i=1 ϕi,1(xi) · · · IY i=1 ϕi,O(xi) (1) where Φ is a matrix of learnable scalar functions ϕi,o(·), with i indexing input dimensions...
work page Pith review arXiv 2025
-
[3]
The baseline architecture (FFF in Fig
KAN in SLU We first investigate the impact of architectural modifications on the dense classification block of a 2D-CNN, and then extend our analysis to transformer-based models. The baseline architecture (FFF in Fig. 1) employs a 2D- CNN feature extractor followed by a dense block comprising a 3-layer Multi-Layer Perceptron (MLP). Each Feed-Forward (FF) ...
-
[4]
Experimental Setup Datasets. We evaluate our approach on five publicly avail- able intent classification datasets: FSC [19], Timers and Such [20], and SLURP [28] for English, ITALIC [29] for Italian, and S PEECH -MASSIVE [30] for German and French. The first two datasets are relatively simple, containing 31 and 4 intents, respectively. In contrast, SLURP,...
-
[5]
Results In the following, we present the wide variety of experiments performed to explore the behavior of KAN network integration. 5.1. Experimental results on 2D-CNN Table 1 shows the results on the 2D-CNN. FFF serves as the baseline with only FF layers, providing a reference point for evaluating KAN-based configurations. KAN, which fully re- places FF l...
-
[6]
Conclusion This work explores the integration of KAN layers in SLU tasks, demonstrating their effectiveness across different model archi- tectures and languages. Our experiments show that strategic placement of KAN layers between FF layers achieves optimal performance while maintaining computational efficiency com- parable to traditional approaches. Multi...
-
[7]
Acknowledgments This work is partially supported by the FAIR - Future Artifi- cial Intelligence Research (PIANO NAZIONALE DI RIPRESA E RESILIENZA (PNRR) – MISSIONE 4 COMPONENTE 2, INVESTIMENTO 1.3 – D.D. 1555 11/10/2022, PE00000013) and the spoke “FutureHPC & BigData” of the ICSC - Cen- tro Nazionale di Ricerca in High-Performance Computing, Big Data and ...
work page 2022
-
[8]
KAN: Kolmogorov–arnold networks,
Z. Liu, Y . Wang, S. Vaidya, F. Ruehle, J. Halverson, M. Sol- jacic, T. Y . Hou, and M. Tegmark, “KAN: Kolmogorov–arnold networks,” in The Thirteenth International Conference on Learn- ing Representations, 2025
work page 2025
Show all 39 references
-
[9]
A. N. Kolmogorov, On the representation of continuous functions of several variables by superpositions of continuous functions of a smaller number of variables. American Mathematical Society, 1961
1961
-
[10]
On the representation of continuous functions of many variables by superposition of continuous functions of one vari- able and addition,
——, “On the representation of continuous functions of many variables by superposition of continuous functions of one vari- able and addition,” in Doklady Akademii Nauk , vol. 114, no. 5. Russian Academy of Sciences, 1957, pp. 953–956
1957
-
[11]
On a constructive proof of kol- mogorov’s superposition theorem,
J. Braun and M. Griebel, “On a constructive proof of kol- mogorov’s superposition theorem,” Constructive approximation, vol. 30, pp. 653–675, 2009
2009
-
[12]
Kan or mlp: A fairer comparison,
R. Yu, W. Yu, and X. Wang, “Kan or mlp: A fairer comparison,” arXiv preprint arXiv:2407.16674, 2024
2024 arXiv
-
[13]
A benchmarking study of kolmogorov-arnold networks on tabular data,
E. Poeta, F. Giobergia, E. Pastor, T. Cerquitelli, and E. Baralis, “A benchmarking study of kolmogorov-arnold networks on tabular data,” in2024 IEEE 18th International Conference on Application of Information and Communication Technologies (AICT), 2024
2024
-
[14]
Suitability of kans for computer vision: A preliminary investigation,
B. Azam and N. Akhtar, “Suitability of kans for computer vision: A preliminary investigation,” arXiv preprint arXiv:2406.09087 , 2024
2024 arXiv
-
[15]
Kan you see it? kans and sentinel for effective and explainable crop field segmentation,
D. Rege Cambrin, E. Poeta, E. Pastor, T. Cerquitelli, E. Baralis, and P. Garza, “Kan you see it? kans and sentinel for effective and explainable crop field segmentation,” inComputer Vision – ECCV 2024 Workshops. Cham: Springer Nature Switzerland, 2025, pp. 115–131
2024
-
[16]
U-kan makes strong backbone for medical image segmentation and generation,
C. Li, X. Liu, W. Li, C. Wang, H. Liu, Y . Liu, Z. Chen, and Y . Yuan, “U-kan makes strong backbone for medical image segmentation and generation,” arXiv preprint arXiv:2406.02918, 2024
2024 arXiv
-
[17]
Kolmogorov-arnold networks for time series: Bridging predictive power and interpretability,
K. Xu, L. Chen, and S. Wang, “Kolmogorov-arnold networks for time series: Bridging predictive power and interpretability,”arXiv preprint arXiv:2406.02496, 2024
2024 arXiv
-
[18]
Kan4tsf: Are kan and kan-based models effective for time series forecasting?
X. Han, X. Zhang, Y . Wu, Z. Zhang, and Z. Wu, “Kan4tsf: Are kan and kan-based models effective for time series forecasting?” arXiv preprint arXiv:2408.11306, 2024
2024 arXiv
-
[19]
Tur and R
G. Tur and R. De Mori, Spoken language understanding: Systems for extracting semantic information from speech. John Wiley & Sons, 2011
2011
-
[20]
A con- trastive learning approach to mitigate bias in speech models,
A. Koudounas, F. Giobergia, E. Pastor, and E. Baralis, “A con- trastive learning approach to mitigate bias in speech models,” in Proc. INTERSPEECH 2024, 2024, pp. 827–831
2024
-
[21]
Towards end-to-end spoken language understanding,
D. Serdyuk, Y . Wang, C. Fuegen, A. Kumar, B. Liu, and Y . Ben- gio, “Towards end-to-end spoken language understanding,” in ICASSP, 2018
2018
-
[22]
Prioritizing data acquisition for end-to-end speech model improvement,
A. Koudounas, E. Pastor, G. Attanasio, L. de Alfaro, and E. Bar- alis, “Prioritizing data acquisition for end-to-end speech model improvement,” in ICASSP 2024 - 2024 IEEE International Con- ference on Acoustics, Speech and Signal Processing (ICASSP) , 2024, pp. 7000–7004
2024
-
[23]
Spoken language understanding on the edge,
A. Saade, J. Dureau, D. Leroy, F. Caltagirone, A. Coucke, A. Ball, C. Doumouro, T. Lavril, A. Caulier, T. Bluche, T. Gisselbrecht, and M. Primet, “Spoken language understanding on the edge,” in 2019 Fifth Workshop on Energy Efficient Machine Learning and Cognitive Computing - ...
2019
-
[24]
Effective integration of kan for keyword spotting,
A. Xu, B. Zhang, S. Kong, Y . Huang, Z. Yang, S. Srivastava, and M. Sun, “Effective integration of kan for keyword spotting,” in ICASSP 2025 - 2025 IEEE International Conference on Acous- tics, Speech and Signal Processing (ICASSP), 2025, pp. 1–5
2025
-
[25]
An investigation on the potential of kan in speech enhancement,
H. Li, Y . Hu, C. Chen, and E. S. Chng, “An investigation on the potential of kan in speech enhancement,” arXiv preprint arXiv:2412.17778, 2024
2024 arXiv
-
[26]
Speech model pre-training for end-to-end spoken language understanding,
L. Lugosch, M. Ravanelli, P. Ignoto, V . S. Tomar, and Y . Ben- gio, “Speech model pre-training for end-to-end spoken language understanding,” in Proc. INTERSPEECH, 2019
2019
-
[27]
Timers and such: A practical benchmark for spoken language un- derstanding with numbers,
L. Lugosch, P. Papreja, M. Ravanelli, A. HEBA, and T. Parcollet, “Timers and such: A practical benchmark for spoken language un- derstanding with numbers,” inNeurIPS Datasets and Benchmarks Track, 2021
2021
-
[28]
Introduction to radial basis function networks,
M. J. Orr et al., “Introduction to radial basis function networks,” 1996
1996
-
[29]
Kolmogorov-arnold networks are radial basis function net- works,
Z. Li, “Kolmogorov-arnold networks are radial basis function net- works,” arXiv preprint arXiv:2405.06721, 2024
2024 arXiv
-
[30]
Fasterkan,
A. Delis, “Fasterkan,” https://github.com/AthanasiosDelis/ faster-kan/, 2024
2024
-
[31]
Properties of chebyshev polynomials,
N. Karjanto, “Properties of chebyshev polynomials,” arXiv preprint arXiv:2002.01342, 2020
2002 arXiv
-
[32]
Kolmogorov-arnold transformer,
X. Yang and X. Wang, “Kolmogorov-arnold transformer,” arXiv preprint arXiv:2409.10594, 2024
2024 arXiv
-
[33]
wav2vec 2.0: A framework for self-supervised learning of speech representa- tions,
A. Baevski, Y . Zhou, A. Mohamed, and M. Auli, “wav2vec 2.0: A framework for self-supervised learning of speech representa- tions,” in NeurIPS, 2020
2020
-
[34]
Xls-r: Self- supervised cross-lingual speech representation learning at scale,
A. Babu, C. Wang, A. Tjandra, K. Lakhotia, Q. Xu, N. Goyal, K. Singh, P. von Platen, Y . Saraf, J. Pino et al. , “Xls-r: Self- supervised cross-lingual speech representation learning at scale,” Interspeech, 2022
2022
-
[35]
SLURP: A spoken language understanding resource package,
E. Bastianelli, A. Vanzo, P. Swietojanski, and V . Rieser, “SLURP: A spoken language understanding resource package,” in EMNLP, 2020
2020
-
[36]
ITALIC: An Italian In- tent Classification Dataset,
A. Koudounas, M. La Quatra, L. Vaiani, L. Colomba, G. Attana- sio, E. Pastor, L. Cagliero, and E. Baralis, “ITALIC: An Italian In- tent Classification Dataset,” in Proc. INTERSPEECH 2023, 2023, pp. 2153–2157
2023
-
[37]
Speech-massive: A multilingual speech dataset for slu and be- yond,
B. Lee, I. Calapodescu, M. Gaido, M. Negri, and L. Besacier, “Speech-massive: A multilingual speech dataset for slu and be- yond,” in Proc. INTERSPEECH, 2024, pp. 817–821
2024
-
[38]
Explaining speech classification models via word-level au- dio segments and paralinguistic features,
E. Pastor, A. Koudounas, G. Attanasio, D. Hovy, and E. Bar- alis, “Explaining speech classification models via word-level au- dio segments and paralinguistic features,” in Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguisti...
2024
-
[39]
Towards faithfully interpretable NLP systems: How should we define and evaluate faithfulness?
A. Jacovi and Y . Goldberg, “Towards faithfully interpretable NLP systems: How should we define and evaluate faithfulness?” in ACL, 2020
2020
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.