{"id":"d5078f17-498e-435f-a88a-8c478ad66181","arxiv_id":"2501.00420","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"KAE, an autoencoder with polynomial KAN layers, achieves lower reconstruction error and better downstream task performance than standard autoencoders and other KAN variants on four image benchmarks.","lead":"This paper proposes KAE, an autoencoder that replaces standard neural layers with Kolmogorov-Arnold layers using learnable polynomial activations. The authors report improved reconstruction, retrieval, classification, and denoising on MNIST, FashionMNIST, CIFAR10, and CIFAR100 compared with standard and KAN-based autoencoders.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. (3) is dimensionally inconsistent: as written it defines an elementwise polynomial map, not a KAN layer, so every reported improvement depends on an unspecified implementation.","rationale":"The reader's CONDITIONAL verdict identifies Eq. (3) as the weakest assumption. My stress-test agrees and sharpens the point: the equation is not merely ambiguous; it is dimensionally inconsistent and, taken literally, defines an elementwise map that cannot serve as the KAN layer in an autoencoder with a bottleneck. The reported gains in reconstruction, retrieval, classification, and denoising all depend on this layer. If the code implements true dimension-mixing polynomials, then Eq. (3) must be corrected and the experiments re-verified against the corrected definition. If the code implements elementwise polynomials, the paper's central claim is unsupported by its own architecture. The Jacobian check on the released code is decisive because it directly reveals whether the implemented layer mixes input coordinates. I do not move the verdict to REJECT because the code URL provides a concrete path to resolution; I keep the reader's CONDITIONAL stance, meaning the paper should not be accepted until the architecture is clarified and at least the headline reconstruction results are re-run with the clarified layer.","tokens_in":12538,"tokens_out":4409,"duration_ms":44811,"concrete_test":"Using the released repository (github.com/SciYu/KAE), instantiate the KAE layer with din=4, dout=2, order=2 and a random input vector. Numerically compute the Jacobian J_{k,j} = d(output_k)/d(input_j) by finite differences. If all off-diagonal entries are exactly zero for all tested inputs, the layer is elementwise and does not mix dimensions, contradicting the KAN definition and the central claim. If off-diagonal entries are nonzero, inspect the forward function to identify the actual mixing operation and rewrite Eq. (3) with correct tensor shapes and summation indices.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim requires KAE layers to behave like KAN layers: each output coordinate should be a sum of learnable univariate functions of all input coordinates, as in Eq. (2). However, Eq. (3) in Section 3.3 defines KAE(x) = sigma((c0 * 1_din + c1*x + c2*x^2 + ... + cp*x^p) + b), with 1_din an all-ones vector. As written, this is a per-coordinate polynomial transform: output coordinate i depends only on input coordinate i, with no summation over input indices j. The expression is also dimensionally inconsistent: c0 is said to be in R^{dout x din}, but c0 * 1_din is then a dout-vector, while each c_i*x^i term is a din-vector, so the sum is undefined unless dout = din and c0 acts diagonally. The text further claims f and g are 'designed to be invertible' (Section 3.3), which is incompatible with a bottleneck dlatent < dinput. Thus the manuscript does not specify the operation that maps dinput to dlatent. If the released code uses a true KAN-style sum, Eq. (3) is wrong; if the code follows Eq. (3), KAE is not a KAN layer and the reported representation-learning gains need a different explanation. Every table in Section 4 depends on this unspecified operation, making this the load-bearing risk.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a Kolmogorov-Arnold Auto-Encoder (KAE) that replaces the linear transformations in standard autoencoder layers with learnable polynomial functions, motivated by the Kolmogorov-Arnold representation theorem and by KANs. The authors report experiments on MNIST, FashionMNIST, CIFAR10, and CIFAR100 showing that KAE with polynomial order p=2 or p=3 improves reconstruction MSE, retrieval recall, nearest-neighbor classification accuracy, and denoising error relative to a sigmoid autoencoder and to KAN, FourierKAN, and WavKAN baselines. The paper also includes a capacity and convergence analysis. The central claim is that KAE is a useful drop-in alternative for representation learning.","tokens_in":12801,"tokens_out":3426,"duration_ms":36083,"significance":"If the architecture were precisely specified and the reported results were reproducible, KAE would be a useful lightweight alternative to standard autoencoders, and the paper has strengths: experiments are repeated over ten seeds, four benchmark datasets are used, several downstream tasks are evaluated, and a code repository is referenced. However, the formal definition of the core KAE layer in Eq. (3) is internally inconsistent, and the paper never specifies the operation that maps the input dimension to the latent dimension. Because every table in Section 4 depends on this unspecified operation, the significance of the empirical claims cannot currently be assessed.","major_comments":[{"comment":"The core layer definition is dimensionally inconsistent and, taken literally, is not a KAN layer. With c0 ∈ R^{dout×din}, the term c0 * 1_din is a dout-vector, while each term c_i * x^i is a din-vector, so the sum is undefined unless dout=din and c0 acts diagonally. If c0 is diagonal, however, output coordinate i depends only on input coordinate i and the layer reduces to a per-coordinate polynomial map with no mixing of input dimensions. That contradicts the standard KAN layer in Eq. (2), where each output sums learnable univariate functions over all input coordinates. Since Eq. (3) is the only formal specification of the proposed layer, and all results in Tables 2–5 depend on it, the paper must state the exact operation, correct Eq. (3), and confirm that the released code implements that corrected operation.","section":"3.3, Eq. (3)"},{"comment":"The text states that f and g are 'designed to be invertible,' but an encoder mapping R^{dinput} to R^{dlatent} with dlatent < dinput cannot be invertible as a function on the input space. This claim is incompatible with the bottleneck autoencoder used in Section 4 and should be replaced with a statement about reconstruction fidelity rather than invertibility.","section":"3.3"},{"comment":"The experimental protocol reports 'the best-performing configuration' among four combinations of learning rate and weight decay without stating the selection criterion. Because the polynomial order p is also chosen by observed performance, this creates a multiple-comparisons concern. The authors should either report results for all four configurations, describe a validation-based selection rule, or state explicitly which configuration was chosen for every model reported in Tables 2–5.","section":"4.1"},{"comment":"The baseline set is too narrow to support the strong claims in the abstract and conclusion. The only standard autoencoder is a single shallow sigmoid AE, and training is limited to 10 epochs. The convergence analysis in Fig. 3 suggests that some baselines, particularly WavKAN and AE, are still improving at 50 epochs, so the comparison may be biased by a short training budget. Comparisons with deeper MLP autoencoders, variational autoencoders, or denoising autoencoders would be needed to show that the reported gains come from the polynomial KAN-style layer rather than from weaker baselines.","section":"4.1"}],"minor_comments":[{"comment":"The notation c_i * x^i is ambiguous because x^i could mean coordinate-wise exponentiation, a tensor product, or a matrix product; the authors should define the operation over input coordinates explicitly and state the tensor shapes of c_i.","section":"3.3"},{"comment":"The figure caption contains placeholder question marks in the formulas for AE and KAE; the figure should be regenerated with the actual equations.","section":"2.1, Fig. 1"},{"comment":"Several references are incomplete or inconsistently formatted, including the entry for Kolmogorov's paper and the entries for Kingma 2013 and Kingma 2014; the reference list should be checked against the publisher's style.","section":"References"},{"comment":"The caption for Fig. 3 lists 'CIFAR100' twice in the subplot labels, and one of the labels should presumably be 'CIFAR10'; the figure needs to be corrected.","section":"4.4, Fig. 3"}],"recommendation":"major_revision","confidential_remarks":"The central issue for the editor is that the manuscript's only formal definition of the proposed layer, Eq. (3), is dimensionally inconsistent and does not obviously perform dimension mixing. The GitHub repository is referenced, so I recommend that the authors be asked to clarify the exact operation and to state explicitly whether the released code follows the corrected equation or a different implementation. If the code follows Eq. (3) literally, the empirical gains would need a different explanation, and the paper's central claim would be unsupported."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the empirical claim is plausible and the comparison is broad, but the paper doesn't specify its own architecture. Eq. (3) is not a KAN layer as printed, and it's dimensionally inconsistent. That's the load-bearing problem.\n\nWhat's new: concurrent Moradi et al. already applied KAN to autoencoders with B-splines; this paper swaps in polynomial activations and isolates that choice. It's incremental, but useful. The experiments cover reconstruction, retrieval, classification, and denoising on four datasets, with 10 seeds and two latent dimensions, and there's a parameter-efficiency plot. The gains over KAN, FourierKAN, and WavKAN are large and internally consistent, and the code is linked. Credit where due.\n\nThe soft spots: Eq. (3) defines KAE(x) = sigma((c0*1_din + c1*x + ... + cp*x^p) + b). With c0 a dout×din matrix, c0*1_din is a dout-vector and each c_i*x^i is a din-vector; the sum is undefined unless dout=din and c0 acts diagonally. The text also claims f and g are designed invertible, which is impossible with a bottleneck dlatent < dinput. So the paper never defines the dinput→dlatent map. If the code implements a true KAN-style sum over input coordinates, the equation is wrong; if it follows the equation, the method is not a KAN layer and the reported representation-learning gains need a different explanation. This is not a minor typo; every table in Section 4 depends on this operation.\n\nThe evaluation protocol also has holes: the best of four hyperparameter configurations is reported without saying whether selection was on validation or test, training runs only 10 epochs, and the main baseline is a single shallow sigmoid autoencoder. On CIFAR, that AE is weak, so part of the gain is a weak baseline. The Kolmogorov-Arnold theorem is stated but only used as motivation; that's fine, but the paper shouldn't imply more.\n\nWho this is for: people working on KAN variants for unsupervised representation learning. It deserves a serious referee, but only with a requirement to fix Eq. (3), clarify the selection protocol, and add a stronger MLP baseline. I'd send it out, and I'd check the code myself.","headline":"Plausible gains from polynomial KAN autoencoders, but the paper's own equation doesn't define the layer it evaluates.","tokens_in":13347,"tokens_out":5245,"would_cite":false,"duration_ms":47840,"reading_group":"maybe","serious_thinker":"unclear","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that replacing autoencoder layers with polynomial Kolmogorov-Arnold layers lowers reconstruction error and improves retrieval, classification, and denoising.","keywords":["Kolmogorov-Arnold network","autoencoder","polynomial activation function","representation learning","image reconstruction","similarity search","image denoising"],"falsifier":"Inspect the released code's forward pass for a single KAE layer: feed an input with one nonzero entry and check whether every output coordinate changes. A true KAN layer would mix all coordinates, while an element-wise polynomial would leave unrelated output coordinates unchanged, which would contradict the paper's explanation of its gains.","tokens_in":12335,"feed_emoji":"🧠","tokens_out":13946,"duration_ms":118619,"temperature":0.7,"pith_summary":"The paper is trying to establish that a Kolmogorov-Arnold Auto-Encoder (KAE) whose KAN layers use learnable polynomial activations is a better representation learner than a standard MLP autoencoder and than KAN variants using B-spline, Fourier, or wavelet functions. The authors argue that the flexibility of low-order polynomials ($p=2$, $p=3$) lets the encoder and decoder stay near-inverses while capturing nonlinear structure. They support this with experiments on MNIST, FashionMNIST, CIFAR10, and CIFAR100, reporting lower reconstruction MSE, higher similarity-search recall, higher nearest-neighbor classification accuracy, and lower denoising error than the baselines. The paper positions KAE as a practical, parameter-lean alternative for representation learning tasks.","feed_headline":"Polynomial KAN autoencoder beats AE on four image datasets","feed_subtitle":"Quadratic and cubic activation functions lift retrieval, classification, and denoising with fewer parameters.","key_machinery":"The central object is the polynomial KAE layer defined in Eq. (3) of the paper: $\\text{KAE}(x)=\\sigma(h(x)+b)$, with $h(x)=c_0+c_1x+c_2x^2+\\cdots+c_px^p$, where $p$ is an order hyperparameter and $\\sigma$ is the sigmoid. It is used in both encoder and decoder of a $d_{\\text{in}}$-$d_{\\text{latent}}$-$d_{\\text{out}}$ autoencoder trained with MSE. The layer is meant to do the work of the fixed $Wx+b$ in an MLP: the polynomial terms supply learnable nonlinearity, and the constant term $c_0$ plays the role of a flexible bias. The paper's claim is that keeping $p$ small balances expressivity with stability and preserves the encoder-decoder inversion property.","core_discovery":"The central claim is that the KAE architecture, defined by replacing the fixed linear-plus-activation layer $y=\\sigma(Wx+b)$ with KAN layers whose learnable activations are polynomials $h(x)=c_0+c_1x+c_2x^2+\\cdots+c_px^p$ and then applying $\\sigma$, reconstructs data with lower error and yields latents that better preserve neighborhoods, class structure, and clean signal. The empirical finding is that KAE with $p=2$ or $p=3$ consistently achieves the best reconstruction and denoising MSE on all four datasets, best Recall@10 with $p=2$, and best classification accuracy with $p=3$. The paper also claims that KAE reaches these results while using fewer parameters than the B-spline KAN and FourierKAN baselines.","pith_inferences":["If KAE is a genuine KAN layer, the paper's success with low-order polynomials suggests that expensive spline grids are not necessary for autoencoder-style reconstruction, and the same polynomial recipe could transfer to variational or convolutional autoencoder variants.","The reported order dependence ($p=2$ for retrieval, $p=3$ for classification) points to a tunable trade-off between smooth distance preservation and sharp class boundaries; sweeping $p$ on other datasets would test whether the polynomial degree acts as a prior on the latent metric's smoothness.","A natural next experiment, not in the paper, is to test KAE on higher-resolution color images or structured data where the latent dimension is much smaller than the input; if the distance-preservation gains persist, a polynomial-KAN autoencoder could serve as a general-purpose embedding model."],"forward_implications":["With quadratic or cubic polynomial activations, KAE produces the lowest reconstruction error on MNIST, FashionMNIST, CIFAR10, and CIFAR100 at latent dimensions 16 and 32, roughly halving the MSE of the sigmoid autoencoder.","The same trained encoder gives better similarity-search recall: KAE with $p=2$ raises Recall@10 over the standard AE by about 0.13 to 0.24 across the four datasets.","Nearest-neighbor classification on the latent space improves by about 5 to 9 percentage points on MNIST and FashionMNIST and 5 to 7 points on CIFAR10 and CIFAR100 when using KAE with $p=3$.","KAE with $p=2$ or $p=3$ also reports the lowest denoising error under both Gaussian and salt-and-pepper noise, so the architecture doubles as a denoiser without extra components.","The paper's capacity analysis shows KAE reaching these results with 75,000 to 101,000 parameters, fewer than the 250,000 to 251,000 used by B-spline KAN and FourierKAN, and converging within roughly 10 epochs."],"supporting_citations":[{"why":"Defines the standard autoencoder baseline whose fixed-activation MLP layers KAE replaces.","marker":"(Hinton & Salakhutdinov, 2006)"},{"why":"Introduces KAN layers with B-spline learnable activations, the architecture KAE adapts and the primary KAN baseline.","marker":"(Liu et al., 2024b;a)"},{"why":"States the representation theorem that motivates KAN-style learnable univariate functions in KAE.","marker":"(Kolmogorov, 1961)"},{"why":"Concurrent KAN autoencoder using B-spline functions and a KAN-ReLU-Dense structure that KAE contrasts with its polynomial-only design.","marker":"(Moradi et al., 2024)"},{"why":"Supplies the FourierKAN activation baseline used in the comparison.","marker":"(Xu et al., 2024)"},{"why":"Supplies the WavKAN baseline with Mexican-hat wavelet activations.","marker":"(Bozorgasl & Chen, 2024)"},{"why":"Provides the MNIST dataset used in the reconstruction and downstream evaluations.","marker":"(LeCun et al., 1998)"},{"why":"Provides the FashionMNIST dataset used in the evaluations.","marker":"(Xiao et al., 2017)"},{"why":"Provides the CIFAR10 and CIFAR100 datasets used in the evaluations.","marker":"(Krizhevsky & Hinton, 2009)"}],"fun_headline_variants":["KAE: Polynomial KAN autoencoder beats AE on four image benchmarks","KAE: Polynomial KAN autoencoder tops AE in retrieval, classification, denoising","KAE: Polynomial KAN autoencoder wins with fewer parameters on four datasets","KAE: KAN autoencoder with polynomial edges outperforms AE on four datasets"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper's central result depends on the KAE layer actually mixing input coordinates the way a Kolmogorov-Arnold layer should; as written, the formula looks like it only transforms each coordinate separately, and the paper never says which behavior the code implements.","fun_headline_variants_meta":{"raw":{"variants":["KAE: Polynomial KAN autoencoder beats AE on four image benchmarks","KAE: Polynomial KAN autoencoder tops AE in retrieval, classification, denoising","KAE: Polynomial KAN autoencoder wins with fewer parameters on four datasets","KAE: KAN autoencoder with polynomial edges outperforms AE on four datasets"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001748,"raw_usage":{"total_tokens":6869,"prompt_tokens":878,"completion_tokens":5991,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":494,"completion_tokens_details":{"reasoning_tokens":5905}},"tokens_in":494,"tokens_out":5991,"duration_ms":38817,"temperature":1.0,"reasoning_tokens":5905,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T22:51:20.900845+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Inspect the released code's forward pass for a single KAE layer: feed an input with one nonzero entry and check whether every output coordinate changes. A true KAN layer would mix all coordinates, while an element-wise polynomial would leave unrelated output coordinates unchanged, which would contradict the paper's explanation of its gains.","supporting_citations":[],"review_version":1}