Generative Kernel Spectral Clustering

David Winant; Johan A. K. Suykens; Sonny Achten

REVIEW 5 major objections 5 minor 10 references

Generative Kernel Spectral Clustering

T0 review · 5 major / 5 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Clustering that can generate and exaggerate its own cluster images.

desk verdict A plausible incremental extension of KSC with a generative decoder, but the claim that extrapolated traversals exaggerate cluster features rests on unexamined out-of-distribution decoding and on image-only evidence. read the letter →

arxiv 2502.02185 v1 pith:HRFSHGZS submitted 2025-02-04 cs.LG

David Winant , Sonny Achten , Johan A. K. Suykens This is my paper · ORCID

classification cs.LG

keywords kernelspectralclusteringgenerativemodelinginterpretablelatentspacetraversaldeepreconstructionlossexplainableAI

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes GenKSC, a clustering model that combines kernel spectral clustering with a learned encoder-decoder so that clusters are not only assigned but also visualized. The model trains a feature map and an inverse map together, adding reconstruction and cluster-direction losses to the spectral variance objective. A user can pick a point in the score space, decode it, and obtain an image that represents or exaggerates a cluster's distinguishing features. The authors demonstrate on MNIST and FashionMNIST subsets that traversals along cluster directions reveal what the clustering is based on. The point of the work is to close the gap between deep clustering accuracy and interpretability.

What carries the argument

The central object is the score-vector latent space $e_i = U^\top \varphi(x_i; \theta_\varphi)$ together with the inverse map $\psi(U e; \theta_\psi)$. The training objective combines a negative degree-weighted variance term that creates the spectral clustering structure, a feature-map regularization, an autoencoding reconstruction loss, and a cosine-distance cluster loss that aligns the first $k-1$ score components toward $k$ simplex vertices. This combination is what makes the latent directions stable enough that decoding far extrapolations along a cluster direction stays cluster-specific.

What would settle it

Take a dataset with known labels, generate decoded images at increasing distances along a cluster direction, and classify them with a pretrained classifier; the central claim fails if class identity collapses or shifts before characteristic exaggeration appears.

Watch

Extended reading notes

Core claim

GenKSC shows that the line-structured eigenspace of kernel spectral clustering can be turned into a generative, explorable latent space. The feature map $\varphi$ and approximate inverse $\psi$ are learned along with the projection $U$, and the objective augments weighted variance maximization with a reconstruction term and a cosine cluster loss that pushes score vectors onto predefined directions, namely the vertices of a regular simplex. Because $U$ is orthonormal, one can select any score $e^*$ and decode it as $x^* = \psi(U e^*)$; following a cluster direction beyond the data range generates images that exaggerate the cluster's characteristic features, while higher latent components expose intra-cluster variation.

Load-bearing premise

The model's decoder, trained on ordinary training images, has to keep producing meaningful cluster-specific images when asked to decode points far beyond the training range along a cluster direction.

Editorial extensions

If this is right

Users can generate a representative image for a cluster by decoding a point on that cluster's direction, giving a direct visual definition of each cluster.
Extrapolating beyond the farthest training point along a cluster direction exaggerates the cluster's defining traits, so cluster boundaries and failure modes become visible.
Intra-cluster variation can be explored by moving along higher latent dimensions beyond the first $k-1$ directions.
The same loss combination could be reused in semi-supervised or supervised settings, where cluster labels from another model are used to build an interpretable latent space.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Traversal distance could serve as a cluster-quality diagnostic: if extreme extrapolations lose class identity or bleed into another cluster, the clusters are not cleanly separated.
The generation step could be turned into a counterfactual tool, interpolating between cluster codes to show which features must change to move an input from one cluster to another.
Quantifying generation with classifier agreement or distribution distances would test whether visual exaggeration is actually class-preserving, since the paper's evidence is visual only.
The method might extend to non-image tabular data, where cluster directions would be less visual but could still be decoded as synthetic examples; this extension is untested.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague,

Quick read of Winant, Achten, and Suykens, "Generative Kernel Spectral Clustering." The short version: it's a legitimate incremental contribution — KSC with a learned feature map, a reconstruction loss, and fixed simplex cluster codes, trained jointly so you can decode along cluster directions. On MNIST012 and FashionMNIST the traversals look reasonable. But the paper's one big claim, that extrapolating in latent space "emphasizes or exaggerates" cluster-defining features, is supported only by eyeballing images, and the mechanism that would make it true is never examined.

The synthesis is genuinely new relative to what they cite: KSC gives the line structure, Stiefel-RKM gives the encoder-decoder setup, ClusterGAN gives the clustering-in-latent-space idea, and nobody combines them with decoding along cluster directions. The writing is honest about the lineage. The math is mostly standard — the "equivalence" between problems (1) and (2) is stated laconically, but it's the standard weighted-kernel-PCA stationarity argument from Alzate and Suykens, so I'd call that a presentation gap, not an error.

The real problems, in order. First, the stress-test note is right: ψ is trained only on projected training features U U^T φ(x_i). When you decode U e* for e* beyond the training range, nothing supervises the decoder. No Lipschitz penalty, no prior, no adversarial loss. So the "exaggeration" result could be decoder artifact, and the paper gives no quantitative check that the generated points actually belong to the target cluster or that the direction aligns with discriminative features. Second, there is no quantitative evaluation at all — no clustering accuracy, NMI, ARI, no baselines (KSC, ClusterGAN, plain autoencoder), no error bars. For a paper whose claims are "well-defined clusters" and "interpretable representations," the evidence is images and scatter plots only. Third, the ||φ||² term in Eq. (3) appears with no explanation; it's doing real work (the weighted variance term is scale-invariant in φ, so the regularizer picks a scale) but the paper doesn't say so. That's fixable in a sentence.

The circularity concern about fixed simplex codes is present but mild: the partition still comes from the data; the codes just kill the rotation ambiguity. I wouldn't hang a rejection on that.

Who's this for? People working on interpretable clustering or the RKM/KSC line. It's an honest, buildable-on idea with thin evidence. It deserves a serious referee, and my read is a conditional: require quantitative evaluation, one baseline, and either a regularity argument for the decoder or a tempered claim about extrapolation.

Referee Report

5 major / 5 minor

Summary. The paper proposes Generative Kernel Spectral Clustering (GenKSC), a model that combines kernel spectral clustering with a generative autoencoder-like architecture. The method trains a parametric feature map and an approximate inverse map by minimizing a weighted variance objective together with a reconstruction loss and a cosine-distance cluster loss whose target directions are fixed as vertices of a regular simplex. After training, new points are generated by decoding extrapolated score-vector locations, producing traversals intended to exaggerate cluster-specific features. Experiments on a three-digit subset of MNIST and on FashionMNIST present qualitative latent-space traversals, and the authors claim that these traversals reveal and exaggerate distinctive cluster characteristics.

Significance. If the central claim holds, GenKSC would make a useful contribution by coupling spectral-clustering-style embeddings with generative decoding, yielding explorable latent spaces that are both clustering-oriented and interpretable. A notable strength is the explicit combination of a weighted-variance KSC objective with learned deep feature maps, an idea with clear potential for interpretable deep clustering. However, the current evidence is largely qualitative: the manuscript contains no quantitative clustering metrics, no baseline comparisons, no error bars, and no validation that extrapolated traversals remain faithful to the target cluster. The significance of the method therefore remains plausible but unestablished at the level expected for a journal publication.

major comments (5)

[Section 3.3, Eq. (3)] The objective includes the term ||phi(x_i)||_2^2 described only as "regularization on feature representations." This term is not a constant: D_ii itself depends on phi through K, so the term modifies the weighted variance problem in a nontrivial way, and no justification or practical role for it is given. The authors should either derive this term from a principled constraint or remove it, and they should explain why no weighting hyperparameter is attached to it.
[Section 3.1, after Eq. (2)] The equivalence between the eigendecomposition in Eq. (1) and the weighted PCA formulation in Eq. (2) is asserted in a single sentence referring to stationarity conditions, but no derivation is provided. Because the paper then uses a learned feature map and adds reconstruction and cluster losses, it is important to state precisely under which conditions the equivalence holds and how the centering assumption in footnote 1 affects it. A short derivation or a precise reference to the corresponding primal-dual result is needed.
[Section 3.2, cluster loss] The cluster directions are defined a priori as vertices of a regular simplex, and the cluster loss minimizes the cosine distance to these fixed codes. Consequently the "traversals along cluster directions" in Figs. 2 and 3 follow predefined directions rather than directions discovered from the data. The interpretability claim is therefore partly circular: the model is forced to align with the fixed codes, and then those same codes are used to interpret the traversals. To support the claim, the authors should demonstrate that the resulting cluster assignments are meaningful, for example by reporting clustering accuracy or NMI with respect to known labels and by comparing with a standard KSC baseline.
[Section 3.3 and Section 4.2] The inverse map psi is trained only through L_rec on the projected training points U U^T phi(x_i). The generation step x* = psi(U e*) evaluates psi at points that can lie far beyond the convex hull of the training projections, and no mechanism (e.g., Lipschitz regularization, adversarial training, or density modeling) constrains psi in that region. The claim that extrapolated traversals "exaggerate distinctive cluster features" therefore rests on an unvalidated out-of-distribution generalization assumption. The authors should provide quantitative evidence that extrapolated images are classified as the intended cluster, or otherwise measure how cluster fidelity degrades as the traversal moves outward.
[Section 4, Experiments] The experiments report no quantitative clustering metrics, no error bars, and no baseline comparisons. For a clustering paper, this is a serious omission: the reader cannot assess whether GenKSC produces well-defined clusters in any measurable sense. In addition, Section 4.1 states that FashionMNIST is trained with k=10, while Fig. 3 reports traversals for "6 clusters" and Section 4.1 also mentions hyperparameter tuning based on an "average membership strength criterion" that is not defined. These inconsistencies and omissions should be addressed with a full experimental evaluation.

minor comments (5)

[Abstract and Section 1] There is a typographical line-break artifact in the abstract: "interpretabili ty" should be "interpretability." Please proofread the text.
[Section 3.1, notation] The relation e_i = D_ii h_i Lambda is stated without defining the dimensions of the score vectors and embeddings clearly. In particular, the later notation e* in Section 3.3 and the operation U e* should be explained more carefully, since U maps from feature space to score space while U e* maps from score space back to feature space; a consistent convention would help the reader.
[Section 3.2, cluster loss] The cluster loss is applied only to the first k-1 components of the score vectors, while the model allows s > k-1. The paper should discuss why the remaining components are excluded and what role they play in the reconstruction and in the generation of intra-cluster variations.
[Section 4.1, hyperparameters] The hyperparameter selection for FashionMNIST relies on an "average membership strength criterion" referenced as in [8], but no definition is given in the text. A one-sentence explanation would make the experimental setup self-contained.
[References] Footnote 2 points to reference [9] for a detailed comparison of primal and dual formulations, but the citation appears after the formula and the connection to the present paper is not elaborated. Please clarify the intended reference and its relevance.

Circularity Check

1 steps flagged · score 3.0 of 10

Cluster directions are fixed simplex inputs to the loss, making the traversals partly self-referential; otherwise the derivation is self-contained.

self definitional [Section 3.2 (cluster loss) and Section 3.3 (generation via x* = psi(U e*))]
"We predefine k directions for these lines using cluster codes ({sc}kc=1) and minimize the cosine distance of each representation to its closest cluster code. ... After training, a new point e* in the score variable space can be selected by targeting a specific cluster center to generate a representative datapoint, or sampled randomly to explore the latent space. The corresponding datapoint is then computed as x* = ψ(U e*)."

The cluster directions used for generation are exactly the fixed simplex vertices s_c that define the cluster loss Lcl. The loss pulls each score vector e_i toward s_c, and generation sets e* along that same s_c. Thus the 'cluster direction' is an input to the training objective, not an independent property discovered from the data. The claim that traversals 'give an indication on which features the model has clustered the data' is therefore partly guaranteed by construction: the generated points lie on the same direction that the loss already forces the clusters to occupy. The semantic content of the images remains data-dependent, so this is partial self-referentiality rather than full circularity.

full rationale

The paper's main derivation is not circular in a statistically forced sense: the optimization problem (3) combines a weighted variance term, a reconstruction loss, and a cosine cluster loss, and the training dynamics could in principle fail to produce meaningful clusters. The extrapolation claim rests on the untested assumption that the inverse map psi generalizes beyond the projected training points U U^T phi(x_i); that is a generalization risk and a correctness concern, not a circularity. The cited prior work [4, 7, 8] provides published components (inverse mapping, KSC formulation, membership criterion) rather than an unverified self-citation chain. The one mild circular element is that the 'cluster directions' are predefined as simplex vertices in the loss and then used as the directions for latent traversal, so the interpretive claim about those directions is partly self-referential. Because the actual visual features revealed by the traversals are learned and data-dependent, the central claim retains independent content; hence a moderate score of 3.

Assumptions & free parameters 4 free parameters · 3 assumptions · 1 invented entities

The central claim rests on the model design choices: the fixed simplex codes, the reconstruction loss, and the regularization. These are not independently verified against external benchmarks; they are postulated to yield interpretable clusters.

free parameters (4)

eta_rec (reconstruction loss weight) = 1 (MNIST012), 0.001 (FashionMNIST)
Tuned based on average membership strength criterion; balances reconstruction and clustering.
eta_cl (cluster loss weight) = 1 (MNIST012), 0.008 (FashionMNIST)
Tuned similarly; balances cluster alignment.
warm-up epochs before cluster loss = 10 (MNIST012), 32 (FashionMNIST)
Chosen to let representations develop before clustering; affects final clusters.
latent dimension s = 10 (MNIST012), 40 (FashionMNIST)
Set manually; must be at least k-1, but larger values may improve generation.

assumptions (3)

domain assumption The regular simplex cluster codes provide maximal angular separation and are an appropriate target for learning interpretable cluster directions.
Introduced in Section 3.2; without this, the cluster loss has no reason to produce interpretable directions.
domain assumption The reconstruction loss with U U^T projection ensures that traversals in the score space correspond to meaningful variations in the data space.
Relies on the encoder-decoder generalizing beyond training points, see Section 3.2.
standard math The stationarity conditions of problem (2) give the KSC eigenvalue problem (1) as asserted, via reference [9].
This is a known result for fixed feature maps, but it is not proven in the text; it is used to motivate the model.

invented entities (1)

Fixed cluster codes (vertices of a regular simplex)
purpose: Define the k latent directions used for cluster assignment and for traversal-based visualization
These vectors are chosen a priori, not learned from data; their semantic meaning is not validated externally.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Generative Kernel Spectral Clustering." pith.science (2026). https://pith.science/paper/HRFSHGZS

@misc{pith2026250202185,
  author       = {Pith},
  title        = {Pith review of: Generative Kernel Spectral Clustering},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HRFSHGZS}},
  note         = {Machine review of arXiv:2502.02185}
}

read the original abstract

Modern clustering approaches often trade interpretability for performance, particularly in deep learning-based methods. We present Generative Kernel Spectral Clustering (GenKSC), a novel model combining kernel spectral clustering with generative modeling to produce both well-defined clusters and interpretable representations. By augmenting weighted variance maximization with reconstruction and clustering losses, our model creates an explorable latent space where cluster characteristics can be visualized through traversals along cluster directions. Results on MNIST and FashionMNIST datasets demonstrate the model's ability to learn meaningful cluster representations.

Figures

Figures reproduced from arXiv: 2502.02185 by the authors.

**Figure 2.** Generated images along indicated cluster directions of the fir [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

**Figure 3.** Latent space traversals for the FashionMNIST dataset. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

10 extracted references · 9 canonical work pages

[1]

Yu, and Lifang He

Yazhou Ren, Jingyu Pu, Zhimeng Yang, Jie Xu, Guofeng Li, X iaorong Pu, Philip S. Yu, and Lifang He. Deep Clustering: A Comprehensive Survey. IEEE Transactions on Neural Networks and Learning Systems , pages 1–21, 2024. Early Access

work page 2024
[2]

Interpretable Clus- tering: A Survey, 2024

Lianyu Hu, Mudi Jiang, Junjie Dong, Xinying Liu, and Zeng you He. Interpretable Clus- tering: A Survey, 2024. arXiv:2409.00743 [cs]

arXiv 2024
[3]

Johan A. K. Suykens. Deep Restricted Kernel Machines Usi ng Conjugate Feature Duality. Neural Computation , 29(8):2123–2163, 2017

work page 2017
[4]

Arun Pandey, Micha¨ el Fanuel, Joachim Schreurs, and Johan A. K. Suykens. Disentangled Representation Learning and Generation With Manifold Opti mization. Neural Compu- tation, 34(10):2009–2036, 09 2022

work page 2009
[5]

Kingma and Max W elling

Diederik P. Kingma and Max W elling. Auto-encoding Varia tional Bayes. In International Conference on Learning Representations , 2014

work page 2014
[6]

ClusterGAN: Latent Space Clustering in Generative Adversarial Network s

Sudipto Mukherjee, Himanshu Asnani, Eugene Lin, and Sre eram Kannan. ClusterGAN: Latent Space Clustering in Generative Adversarial Network s. Proceedings of the AAAI Conference on Artiﬁcial Intelligence , 33(01):4610–4617, Jul. 2019

work page 2019
[7]

Carlos Alzate and Johan A. K. Suykens. Multiway Spectral Clustering with Out-of-sample Extensions through W eighted Kernel PCA. IEEE Transactions on Pattern Analysis and Machine Intelligence , 32(2):335–347, 2010

work page 2010
[8]

Rocco Langone, Raghvendra Mall, and Johan A. K. Suykens. Soft kernel spectral clus- tering. In The 2013 International Joint Conference on Neural Networks (IJCNN), pages 1–8. IEEE, 2013

work page 2013

Show all 10 references

[9]

Sonny Achten, Arun Pandey, Hannes De Meulemeester, Bart De Moor, and Johan A. K. Suykens. Duality in Multi-View Restricted Kernel Machines . ICML W orkshop on Duality for Modern Machine Learning, 2023. arXiv:2305.17251 [cs]

2023 arXiv
[10]

Eﬃcient Riemann ian Optimization on the Stiefel Manifold via the Cayley Transform

Jun Li, Fuxin Li, and Sinisa Todorovic. Eﬃcient Riemann ian Optimization on the Stiefel Manifold via the Cayley Transform. In International Conference on Learning Represen- tations, 2019

2019

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.

[1] [1]

Yu, and Lifang He

Yazhou Ren, Jingyu Pu, Zhimeng Yang, Jie Xu, Guofeng Li, X iaorong Pu, Philip S. Yu, and Lifang He. Deep Clustering: A Comprehensive Survey. IEEE Transactions on Neural Networks and Learning Systems , pages 1–21, 2024. Early Access

work page 2024

[2] [2]

Interpretable Clus- tering: A Survey, 2024

Lianyu Hu, Mudi Jiang, Junjie Dong, Xinying Liu, and Zeng you He. Interpretable Clus- tering: A Survey, 2024. arXiv:2409.00743 [cs]

arXiv 2024

[3] [3]

Johan A. K. Suykens. Deep Restricted Kernel Machines Usi ng Conjugate Feature Duality. Neural Computation , 29(8):2123–2163, 2017

work page 2017

[4] [4]

Arun Pandey, Micha¨ el Fanuel, Joachim Schreurs, and Johan A. K. Suykens. Disentangled Representation Learning and Generation With Manifold Opti mization. Neural Compu- tation, 34(10):2009–2036, 09 2022

work page 2009

[5] [5]

Kingma and Max W elling

Diederik P. Kingma and Max W elling. Auto-encoding Varia tional Bayes. In International Conference on Learning Representations , 2014

work page 2014

[6] [6]

ClusterGAN: Latent Space Clustering in Generative Adversarial Network s

Sudipto Mukherjee, Himanshu Asnani, Eugene Lin, and Sre eram Kannan. ClusterGAN: Latent Space Clustering in Generative Adversarial Network s. Proceedings of the AAAI Conference on Artiﬁcial Intelligence , 33(01):4610–4617, Jul. 2019

work page 2019

[7] [7]

Carlos Alzate and Johan A. K. Suykens. Multiway Spectral Clustering with Out-of-sample Extensions through W eighted Kernel PCA. IEEE Transactions on Pattern Analysis and Machine Intelligence , 32(2):335–347, 2010

work page 2010

[8] [8]

Rocco Langone, Raghvendra Mall, and Johan A. K. Suykens. Soft kernel spectral clus- tering. In The 2013 International Joint Conference on Neural Networks (IJCNN), pages 1–8. IEEE, 2013

work page 2013