Pith. sign in

REVIEW 4 major objections 4 minor 1 cited by

KAE: Kolmogorov-Arnold Auto-Encoder for Representation Learning

T0 review · 4 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that replacing autoencoder layers with polynomial Kolmogorov-Arnold layers lowers reconstruction error and improves retrieval, classification, and denoising.

desk verdict Plausible gains from polynomial KAN autoencoders, but the paper's own equation doesn't define the layer it evaluates. read the letter →

arxiv 2501.00420 v1 pith:YV7RBGUQ submitted 2024-12-31 cs.LG

Fangchen Yu , Ruilizhen Hu , Yidong Lin , Yuqi Ma , Zhenghao Huang , Wenye Li This is my paper · ORCID
classification cs.LG
keywords Kolmogorov-Arnoldnetworkautoencoderpolynomialactivationfunctionrepresentationlearningimagereconstructionsimilaritysearchdenoising
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper is trying to establish that a Kolmogorov-Arnold Auto-Encoder (KAE) whose KAN layers use learnable polynomial activations is a better representation learner than a standard MLP autoencoder and than KAN variants using B-spline, Fourier, or wavelet functions. The authors argue that the flexibility of low-order polynomials ($p=2$, $p=3$) lets the encoder and decoder stay near-inverses while capturing nonlinear structure. They support this with experiments on MNIST, FashionMNIST, CIFAR10, and CIFAR100, reporting lower reconstruction MSE, higher similarity-search recall, higher nearest-neighbor classification accuracy, and lower denoising error than the baselines. The paper positions KAE as a practical, parameter-lean alternative for representation learning tasks.

What carries the argument

The central object is the polynomial KAE layer defined in Eq. (3) of the paper: $\text{KAE}(x)=\sigma(h(x)+b)$, with $h(x)=c_0+c_1x+c_2x^2+\cdots+c_px^p$, where $p$ is an order hyperparameter and $\sigma$ is the sigmoid. It is used in both encoder and decoder of a $d_{\text{in}}$-$d_{\text{latent}}$-$d_{\text{out}}$ autoencoder trained with MSE. The layer is meant to do the work of the fixed $Wx+b$ in an MLP: the polynomial terms supply learnable nonlinearity, and the constant term $c_0$ plays the role of a flexible bias. The paper's claim is that keeping $p$ small balances expressivity with stability and preserves the encoder-decoder inversion property.

What would settle it

Inspect the released code's forward pass for a single KAE layer: feed an input with one nonzero entry and check whether every output coordinate changes. A true KAN layer would mix all coordinates, while an element-wise polynomial would leave unrelated output coordinates unchanged, which would contradict the paper's explanation of its gains.

Watch

Extended reading notes

Core claim

The central claim is that the KAE architecture, defined by replacing the fixed linear-plus-activation layer $y=\sigma(Wx+b)$ with KAN layers whose learnable activations are polynomials $h(x)=c_0+c_1x+c_2x^2+\cdots+c_px^p$ and then applying $\sigma$, reconstructs data with lower error and yields latents that better preserve neighborhoods, class structure, and clean signal. The empirical finding is that KAE with $p=2$ or $p=3$ consistently achieves the best reconstruction and denoising MSE on all four datasets, best Recall@10 with $p=2$, and best classification accuracy with $p=3$. The paper also claims that KAE reaches these results while using fewer parameters than the B-spline KAN and FourierKAN baselines.

Load-bearing premise

The paper's central result depends on the KAE layer actually mixing input coordinates the way a Kolmogorov-Arnold layer should; as written, the formula looks like it only transforms each coordinate separately, and the paper never says which behavior the code implements.

Editorial extensions

If this is right

  • With quadratic or cubic polynomial activations, KAE produces the lowest reconstruction error on MNIST, FashionMNIST, CIFAR10, and CIFAR100 at latent dimensions 16 and 32, roughly halving the MSE of the sigmoid autoencoder.
  • The same trained encoder gives better similarity-search recall: KAE with $p=2$ raises Recall@10 over the standard AE by about 0.13 to 0.24 across the four datasets.
  • Nearest-neighbor classification on the latent space improves by about 5 to 9 percentage points on MNIST and FashionMNIST and 5 to 7 points on CIFAR10 and CIFAR100 when using KAE with $p=3$.
  • KAE with $p=2$ or $p=3$ also reports the lowest denoising error under both Gaussian and salt-and-pepper noise, so the architecture doubles as a denoiser without extra components.
  • The paper's capacity analysis shows KAE reaching these results with 75,000 to 101,000 parameters, fewer than the 250,000 to 251,000 used by B-spline KAN and FourierKAN, and converging within roughly 10 epochs.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If KAE is a genuine KAN layer, the paper's success with low-order polynomials suggests that expensive spline grids are not necessary for autoencoder-style reconstruction, and the same polynomial recipe could transfer to variational or convolutional autoencoder variants.
  • The reported order dependence ($p=2$ for retrieval, $p=3$ for classification) points to a tunable trade-off between smooth distance preservation and sharp class boundaries; sweeping $p$ on other datasets would test whether the polynomial degree acts as a prior on the latent metric's smoothness.
  • A natural next experiment, not in the paper, is to test KAE on higher-resolution color images or structured data where the latent dimension is much smaller than the input; if the distance-preservation gains persist, a polynomial-KAN autoencoder could serve as a general-purpose embedding model.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes a Kolmogorov-Arnold Auto-Encoder (KAE) that replaces the linear transformations in standard autoencoder layers with learnable polynomial functions, motivated by the Kolmogorov-Arnold representation theorem and by KANs. The authors report experiments on MNIST, FashionMNIST, CIFAR10, and CIFAR100 showing that KAE with polynomial order p=2 or p=3 improves reconstruction MSE, retrieval recall, nearest-neighbor classification accuracy, and denoising error relative to a sigmoid autoencoder and to KAN, FourierKAN, and WavKAN baselines. The paper also includes a capacity and convergence analysis. The central claim is that KAE is a useful drop-in alternative for representation learning.

Significance. If the architecture were precisely specified and the reported results were reproducible, KAE would be a useful lightweight alternative to standard autoencoders, and the paper has strengths: experiments are repeated over ten seeds, four benchmark datasets are used, several downstream tasks are evaluated, and a code repository is referenced. However, the formal definition of the core KAE layer in Eq. (3) is internally inconsistent, and the paper never specifies the operation that maps the input dimension to the latent dimension. Because every table in Section 4 depends on this unspecified operation, the significance of the empirical claims cannot currently be assessed.

major comments (4)
  1. [3.3, Eq. (3)] The core layer definition is dimensionally inconsistent and, taken literally, is not a KAN layer. With c0 ∈ R^{dout×din}, the term c0 * 1_din is a dout-vector, while each term c_i * x^i is a din-vector, so the sum is undefined unless dout=din and c0 acts diagonally. If c0 is diagonal, however, output coordinate i depends only on input coordinate i and the layer reduces to a per-coordinate polynomial map with no mixing of input dimensions. That contradicts the standard KAN layer in Eq. (2), where each output sums learnable univariate functions over all input coordinates. Since Eq. (3) is the only formal specification of the proposed layer, and all results in Tables 2–5 depend on it, the paper must state the exact operation, correct Eq. (3), and confirm that the released code implements that corrected operation.
  2. [3.3] The text states that f and g are 'designed to be invertible,' but an encoder mapping R^{dinput} to R^{dlatent} with dlatent < dinput cannot be invertible as a function on the input space. This claim is incompatible with the bottleneck autoencoder used in Section 4 and should be replaced with a statement about reconstruction fidelity rather than invertibility.
  3. [4.1] The experimental protocol reports 'the best-performing configuration' among four combinations of learning rate and weight decay without stating the selection criterion. Because the polynomial order p is also chosen by observed performance, this creates a multiple-comparisons concern. The authors should either report results for all four configurations, describe a validation-based selection rule, or state explicitly which configuration was chosen for every model reported in Tables 2–5.
  4. [4.1] The baseline set is too narrow to support the strong claims in the abstract and conclusion. The only standard autoencoder is a single shallow sigmoid AE, and training is limited to 10 epochs. The convergence analysis in Fig. 3 suggests that some baselines, particularly WavKAN and AE, are still improving at 50 epochs, so the comparison may be biased by a short training budget. Comparisons with deeper MLP autoencoders, variational autoencoders, or denoising autoencoders would be needed to show that the reported gains come from the polynomial KAN-style layer rather than from weaker baselines.
minor comments (4)
  1. [3.3] The notation c_i * x^i is ambiguous because x^i could mean coordinate-wise exponentiation, a tensor product, or a matrix product; the authors should define the operation over input coordinates explicitly and state the tensor shapes of c_i.
  2. [2.1, Fig. 1] The figure caption contains placeholder question marks in the formulas for AE and KAE; the figure should be regenerated with the actual equations.
  3. [References] Several references are incomplete or inconsistently formatted, including the entry for Kolmogorov's paper and the entries for Kingma 2013 and Kingma 2014; the reference list should be checked against the publisher's style.
  4. [4.4, Fig. 3] The caption for Fig. 3 lists 'CIFAR100' twice in the subplot labels, and one of the labels should presumably be 'CIFAR10'; the figure needs to be corrected.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: KAE is an empirical architecture paper; the KAT theorem is motivational only, no prediction is derived from fitted parameters, and there are no load-bearing self-citations.

full rationale

The paper's central claims are empirical comparisons of KAE against AE/KAN baselines on reconstruction and downstream tasks. The Kolmogorov-Arnold theorem in Eq. (1) is used only as motivation ('inspired by', 'motivates the design'), and no subsequent derivation assumes the theorem's conclusion as an input. Eq. (3) defines a KAN-style polynomial layer: if c_i are interpreted as dout×din matrices acting on elementwise powers of x, each output coordinate is a sum over input coordinates of polynomial univariate functions, matching the KAN layer definition in Eq. (2). The polynomial order p is treated as a hyperparameter and chosen empirically, which is standard model selection rather than fitting a parameter to the target metric and calling it a prediction. The paper contains no self-citations: the KAN, FourierKAN, WavKAN, and autoencoder references are all external prior work, and no uniqueness theorem or ansatz is imported from the authors' own prior papers. The methodological weakness noted by the skeptic—the ambiguous notation in Eq. (3) and the claim that f and g are 'invertible' despite an information-bottleneck dlatent < dinput—is an implementation/description risk, not circular reasoning, because the reported gains are not forced by construction or by a self-referential chain. Therefore the derivation chain is self-contained with respect to circularity.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

No new physical or mathematical entities are introduced. The main free parameter is the polynomial order p, selected by hand; the architecture relies on the unverified assumption that polynomial KAN layers yield approximate inverse maps, and the Kolmogorov-Arnold theorem is used only as motivation.

free parameters (2)
  • Polynomial order p = 1, 2, 3; best p=2 for retrieval, p=3 for classification
    The paper reports task-dependent best values, showing p is selected by empirical performance rather than derived.
  • Activation function sigma = Sigmoid
    Chosen as the fixed nonlinearity in Eq. (3); no ablation or justification is provided, but it is a constant choice.
assumptions (3)
  • standard math Kolmogorov-Arnold representation theorem
    Invoked in Sections 2.2.1 and 3.2 but not used to derive the architecture; the theorem only motivates learnable univariate functions.
  • domain assumption Encoder/decoder approximate inversion
    Section 3.3 assumes polynomial layers can make f and g approximate inverses; no proof is given, and for dimension reduction exact inversion is impossible.
  • ad hoc to paper KAE layer mixes dimensions
    Eq. (3) as written is element-wise, but the empirical results require a KAN-style summation over inputs; the intended operation is never stated.

how reviews work

0 comments
Cite this review

Pith. "Pith review of KAE: Kolmogorov-Arnold Auto-Encoder for Representation Learning." pith.science (2026). https://pith.science/paper/YV7RBGUQ

@misc{pith2026250100420,
  author       = {Pith},
  title        = {Pith review of: KAE: Kolmogorov-Arnold Auto-Encoder for Representation Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YV7RBGUQ}},
  note         = {Machine review of arXiv:2501.00420}
}
read the original abstract

The Kolmogorov-Arnold Network (KAN) has recently gained attention as an alternative to traditional multi-layer perceptrons (MLPs), offering improved accuracy and interpretability by employing learnable activation functions on edges. In this paper, we introduce the Kolmogorov-Arnold Auto-Encoder (KAE), which integrates KAN with autoencoders (AEs) to enhance representation learning for retrieval, classification, and denoising tasks. Leveraging the flexible polynomial functions in KAN layers, KAE captures complex data patterns and non-linear relationships. Experiments on benchmark datasets demonstrate that KAE improves latent representation quality, reduces reconstruction errors, and achieves superior performance in downstream tasks such as retrieval, classification, and denoising, compared to standard autoencoders and other KAN variants. These results suggest KAE's potential as a useful tool for representation learning. Our code is available at \url{https://github.com/SciYu/KAE/}.

Figures

Figures reproduced from arXiv: 2501.00420 by the authors.

Figure 1
Figure 1. Model Comparison of AE, KAN, and KAE. Autoencoders (Hinton & Salakhutdinov, 2006), as a framework of unsupervised learning, aim to learn a compressed representation of input data while minimizing reconstruction error. Traditionally, autoencoders rely on MLPs to achieve this task, where each layer applies a fixed non-linearity. However, this may limit the ability of the network to capture complex structures in the da… view at source ↗
Figure 2
Figure 2. Recall@N of Similarity Search Across Datasets for Different Latent Dimensions. 4.3.2 IMAGE CLASSIFICATION We further applied the learned latent representations to image classification using a nearest neighbor classifier. For each of the 10,000 test samples, the predicted label was assigned based on the nearest sample in the latent space with the smallest Euclidean distance. We then compared the predicted labels with… view at source ↗
Figure 3
Figure 3. Convergence Analysis of Test Loss Across Datasets for Latent Dimension dlatent = 16. 20 30 40 50 60 Training Time for 10 Epochs (sec) 0.6 0.7 0.8 0.9 1.0 Classification Accuracy 25K 103K 205K 250K 251K 101K 50K 76K101K AE-T AE-S AE-B KAN FourierKAN WavKAN KAE (p=1) KAE (p=2) KAE (p=3) Model Capacity Bubble size represents #Param. AE-T (784-16-784) AE-S (784-64-16-64-784) AE-B (784-128-16-128-784) KAN FourierKAN WavK… view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Model Capacity Analysis on the MNIST Dataset with Latent Dimension dlatent = 16. Bubble size represents the number of learnable parameters. For AE models, T = Tiny, S = Small, and B = Base. Model Capacity Analysis. As shown in [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Nonlinear Factor Decomposition via Kolmogorov-Arnold Networks: A Spectral Approach to Asset Return Analysis

    q-fin.ST 2026-03 conditional novelty 4.5 of 10

    KAN-PCA is a KAN-encoder/linear-decoder autoencoder that strictly contains classical PCA and slightly raises in-sample explained variance on 20 stocks without an out-of-sample edge.

Reference graph

Works this paper leans on

13 extracted references · 2 canonical work pages · cited by 1 Pith paper

  1. [10]

    Fashion-mnist: a novel image dataset for benchmark- ing machine learning algorithms

    Han Xiao, Kashif Rasul, and Roland V ollgraf. Fashion-mnist: a novel image dataset for benchmark- ing machine learning algorithms. arXiv preprint arXiv:1708.07747,

    arXiv

  2. [13]

    Self pre- training with masked autoencoders for medical image classification and segmentation

    Lei Zhou, Huidong Liu, Joseph Bae, Junjun He, Dimitris Samaras, and Prateek Prasanna. Self pre- training with masked autoencoders for medical image classification and segmentation. In 2023 IEEE 20th International Symposium on Biomedical Imaging, pp. 1–6. IEEE,

    work page 2023

  3. [1998]

    U-kan makes strong backbone for medical image segmentation and generation

    Chenxin Li, Xinyu Liu, Wuyang Li, Cheng Wang, Hengyu Liu, and Yixuan Yuan. U-kan makes strong backbone for medical image segmentation and generation. arXiv preprint arXiv:2406.02918,

    arXiv

  4. [2006]

    Leveraging fourierkan classification head for pre- trained transformer-based text classification

    Abdullah Al Imran and Md Farhan Ishmam. Leveraging fourierkan classification head for pre- trained transformer-based text classification. arXiv preprint arXiv:2408.08803,

    arXiv

  5. [2009]

    Kolmogorov-arnold network for satellite image classification in remote sensing

    Minjong Cheon. Kolmogorov-arnold network for satellite image classification in remote sensing. arXiv preprint arXiv:2406.00600,

    arXiv

  6. [2013]

    Adam: A method for stochastic optimization

    Diederik P Kingma. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980,

    arXiv

  7. [2017]

    Fourierkan-gcf: Fourier kolmogorov-arnold network–an effective and efficient feature transfor- mation for graph collaborative filtering

    Jinfeng Xu, Zheyu Chen, Jinze Li, Shuo Yang, Wei Wang, Xiping Hu, and Edith C-H Ngai. Fourierkan-gcf: Fourier kolmogorov-arnold network–an effective and efficient feature transfor- mation for graph collaborative filtering. arXiv preprint arXiv:2406.01034,

    arXiv

  8. [2018]

    Kolmogorov-arnold net- work autoencoders

    Mohammadamin Moradi, Shirin Panahi, Erik Bollt, and Ying-Cheng Lai. Kolmogorov-arnold net- work autoencoders. arXiv preprint arXiv:2410.02077,

    arXiv

Show all 13 references
  1. [2019]

    Kolmogorov-arnold transformer

    Xingyi Yang and Xinchao Wang. Kolmogorov-arnold transformer. arXiv preprint arXiv:2409.10594,

    arXiv

  2. [2020]

    Kan 2.0: Kolmogorov-arnold networks meet science

    Ziming Liu, Pingchuan Ma, Yixuan Wang, Wojciech Matusik, and Max Tegmark. Kan 2.0: Kolmogorov-arnold networks meet science. arXiv preprint arXiv:2408.10205, 2024a. 10 Preprint Version Ziming Liu, Yixuan Wang, Sachin Vaidya, Fabian Ruehle, James Halverson, Marin Solja ˇci´c, Th...

    arXiv

  3. [2021]

    Auto-encoding variational bayes

    Diederik P Kingma. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114,

    arXiv

  4. [2023]

    Medical image denoising using convolutional denoising autoencoders

    Lovedeep Gondara. Medical image denoising using convolutional denoising autoencoders. In 2016 IEEE 16th International Conference on Data Mining Workshops, pp. 241–246. IEEE,

    2016

  5. [2024]

    Wav-kan: Wavelet kolmogorov-arnold networks

    Zavareh Bozorgasl and Hao Chen. Wav-kan: Wavelet kolmogorov-arnold networks. arXiv preprint arXiv:2405.12832,

    arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.