REVIEW 2 major objections 5 minor 32 references
N2D: (Not Too) Deep Clustering via Clustering the Local Manifold of an Autoencoded Embedding
T0 review · 2 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read A three-stage pipeline—autoencoder, UMAP, then GMM—matches or beats deep clustering networks on several benchmarks.
desk verdict Simple, honest empirical pipeline (AE + UMAP + GMM) with a clean ablation, but single-run results and post hoc UMAP selection keep the state-of-the-art claim from being fully settled. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The carrying mechanism is the composed pipeline $C = F_C(F_M(F_A(X)))$: an autoencoder $F_A$, a manifold learner $F_M$, and a shallow clustering step $F_C$. The decisive component is UMAP applied to the bottleneck of $F_A$; UMAP models the data's local manifold as a fuzzy topological structure and finds a low-dimensional layout whose fuzzy topology is as close as possible to the original, using a cross-entropy objective. In N2D its target dimension is set to the cluster count and its neighbourhood size is 20, so the embedding retains local distances while being compact enough for a GMM to partition. The autoencoder uses a fully connected $d$–500–500–2000–$c$ architecture trained with Adam for 1000 epochs; the final GMM allows each component its own covariance.
What would settle it
Run N2D on MNIST while sweeping UMAP's neighbour count from 5 to 100 and minimum distance from 0 to 0.5; if accuracy moves more than a few points away from the reported 0.979 under ordinary settings, or varies across seeds, the reported advantage is not robust.
Extended reading notes
Core claim
The central discovery is that local manifold learning inside an already-learned autoencoder embedding is enough for high-quality clustering. In N2D, the autoencoder is trained only to reconstruct the input; UMAP then maps the encoder's output into $c$ dimensions (the number of clusters), with 20 neighbours and zero minimum distance; a GMM with $c$ full-covariance components produces the final labels. Across six datasets, this yields accuracies of 0.979 on MNIST, 0.948 on MNIST-test, 0.958 on USPS, 0.672 on Fashion, 0.885 on pendigits, and 0.801 on HAR, placing N2D in the top three on five of the six datasets and best on Fashion, pendigits, and HAR. The paper attributes the effect to UMAP's combination of local focus with better global-structure preservation than t-SNE, which makes the re-embedded space more clusterable than either the raw data or the autoencoder bottleneck alone.
Load-bearing premise
The result rests on the assumption that UMAP's fixed defaults—20 neighbours, zero minimum distance, and an embedding dimension equal to the known number of clusters—are suitable for every dataset, with no sensitivity analysis or repeated seeded runs reported.
Editorial extensions
If this is right
- Joint clustering losses are not necessary for strong clustering on the tested benchmarks; a representation learned purely by reconstruction suffices when followed by a local manifold re-embedding.
- Deep clustering can be made substantially simpler and cheaper: N2D clusters MNIST and Fashion in about 18 minutes on a single modern GPU, compared with jointly trained clustering networks.
- The same recipe transfers from images to time-series, achieving best accuracy on pendigits and HAR, so the effect is not specific to visual data.
- The component ablation identifies UMAP as the driver of the gains: on HAR, adding UMAP after the autoencoder improves accuracy by about 25 percentage points over either stage alone.
Reading between the lines
- A plausible reading of the ablation is that clusterability of an autoencoded space is governed by its local geometry; if so, other local manifold learners that preserve global structure might substitute for UMAP with similar gains.
- The fixed UMAP defaults raise a testable extension: sweeping neighbourhood size and minimum distance per dataset could show whether the reported gains are concentrated at narrow parameter ranges or spread broadly.
- Because several baselines are taken from published numbers, a controlled comparison that trains all methods with the same encoder and training budget would clarify how much of the advantage comes from the pipeline versus from experimental setting.
- The authors note ASPC-DA's edge often comes from data augmentation; applying augmentation inside N2D is a natural next step that the paper itself flags.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes N2D, a clustering pipeline that first trains an autoencoder on the raw data, then applies UMAP as a local manifold learner to the autoencoded embedding, and finally clusters the resulting embedding with a Gaussian Mixture Model. The authors evaluate the contribution of each component on six datasets (MNIST, MNIST-test, USPS, Fashion, pendigits, HAR), showing that UMAP on the autoencoded embedding yields higher ACC and NMI than clustering the raw data, the autoencoded embedding alone, or embeddings produced by Isomap and t-SNE. They then compare N2D with shallow and deep clustering baselines, reporting top-3 performance on five of six datasets and the best accuracy and NMI on Fashion, pendigits, and HAR. The paper argues that replacing the joint deep clustering network with a decoupled autoencoder + manifold learning + shallow clustering pipeline is an effective and simple approach.
Significance. If the empirical claims are robust, N2D offers a practical and simple alternative to joint deep clustering methods, potentially reducing the need for complex clustering-specific losses and networks. The paper provides a clear component-wise ablation and makes code and weights publicly available, which supports reproducibility. The central finding—that a locally focused manifold learner on an autoencoded embedding produces a more clusterable space—is falsifiable and could guide future work. However, the strength of the claims currently exceeds the strength of the empirical evidence, because all reported N2D results come from single runs without variance estimates and the UMAP configuration is fixed without sensitivity analysis.
major comments (2)
- [Section IV-D, Table II] The central claim that N2D is competitive with or outperforms state-of-the-art deep clustering algorithms rests solely on single-run accuracy and NMI values. The pipeline is stochastic in three stages: autoencoder weight initialization and Adam training (Section IV-C), UMAP's low-dimensional layout optimization (Section III-D), and GMM initialization (Section IV-C). For example, the Fashion ACC margin of 0.672 over ClusterGAN's 0.630 and the HAR ACC margin of 0.801 over IDEC's 0.642 are reported without standard deviations, so the reader cannot determine whether these margins are robust or within run-to-run noise. Please report results across multiple seeds (at least 5) with means and standard deviations for N2D, and, where feasible, for the baselines computed by the authors, so the comparative claims can be statistically assessed.
- [Section III-E and Section IV-E] The UMAP hyperparameters (n_neighbors=20, min_dist=0, embedding dimension equal to the number of clusters) are fixed as defaults, and UMAP is selected as the best manifold learner based on its performance on the same six datasets used for the headline comparison in Table II. As the authors themselves note in Section III-E, there is generally no ability to cross-validate hyperparameters in the unsupervised setting, but the current design effectively chooses the manifold learning method after observing test-set outcomes. This creates a risk that the reported gains are configuration-specific rather than intrinsic to the N2D framework. Please provide a sensitivity analysis (e.g., n_neighbors in {5, 10, 20, 50}, min_dist in {0, 0.1, 0.5}) and/or a model-selection procedure that does not use the ground-truth labels, to demonstrate that the advantages of UMAP within N2D are stable across reasonable parameter choices.
minor comments (5)
- [Abstract] The phrase 'discovering higher quality discovering clusters' appears to contain a redundancy; it should likely read 'discovering higher quality clusters.'
- [Section II] The word 'artitional' is a typo and should be 'partitional'.
- [Figure 1 and Section IV-D] Figure 1 uses a 2-D embedding for visualization while the clustering experiments use a c-dimensional embedding. The caption and text explain this difference, but the figure legend or caption should make explicit that the visual separation shown is not necessarily what the GMM sees in the higher-dimensional space.
- [Section IV-C] The autoencoder architecture d-500-500-2000-c is taken from previous work, but for the time-series datasets (pendigits, HAR) the input dimensionality is much lower than for images; a sentence justifying the use of the same architecture for these datasets would improve the presentation.
- [Conclusion] The phrase 'datasets datasets' in the final sentence contains a duplicated word and should be corrected.
Circularity Check
No significant circularity; the empirical pipeline is self-contained and no claimed result reduces to its own inputs.
full rationale
N2D is an empirical pipeline (autoencoder, UMAP, GMM) with no analytic derivation whose output is definitionally equal to an input. The autoencoder, UMAP, and GMM are standard off-the-shelf methods with fixed hyperparameters (Section IV-C: n_neighbors=20, min_dist=0, embedding dimension equal to the number of clusters; GMM with c full-covariance components). The headline comparisons in Tables I and II are external benchmark evaluations, not quantities reconstructed from fitted parameters. The main methodological concern is that UMAP was selected as the manifold learner after observing its performance on the same six datasets used for the final comparison (Sections III-E and IV-E), and the reported scores lack variance estimates; however, this is a model-selection and statistical-reporting issue, not circularity of the kind where an equation reduces to itself or a fitted input is renamed as a prediction. There is also no load-bearing self-citation: the method relies on standard references (autoencoders, UMAP, GMM, and comparison algorithms) without invoking the authors' own prior results as proof of correctness. Accordingly the derivation chain, such as it is, is self-contained rather than circular.
Assumptions & free parameters
free parameters (8)
- UMAP n_neighbors =
20
- UMAP min_dist =
0
- UMAP embedding dimension =
c (number of ground-truth clusters)
- GMM number of components =
c
- Autoencoder architecture dimensions =
d-500-500-2000-c
- Autoencoder training epochs =
1000
- Isomap n_neighbors =
5
- t-SNE perplexity =
not reported
assumptions (3)
- domain assumption The autoencoded embedding satisfies UMAP's assumptions: uniform distribution on a Riemannian manifold, locally constant metric, and local connectivity.
- domain assumption The true number of clusters c is known a priori and is used to set UMAP dimensionality and GMM components.
- domain assumption An autoencoder trained only with reconstruction loss retains enough class-discriminative information for manifold learning to expose clusters.
Cite this review
Pith. "Pith review of N2D: (Not Too) Deep Clustering via Clustering the Local Manifold of an Autoencoded Embedding." pith.science (2026). https://pith.science/paper/TTUA7TVO
@misc{pith2026190805968,
author = {Pith},
title = {Pith review of: N2D: (Not Too) Deep Clustering via Clustering the Local Manifold of an Autoencoded Embedding},
year = {2026},
howpublished = {\url{https://pith.science/paper/TTUA7TVO}},
note = {Machine review of arXiv:1908.05968}
}
read the original abstract
Deep clustering has increasingly been demonstrating superiority over conventional shallow clustering algorithms. Deep clustering algorithms usually combine representation learning with deep neural networks to achieve this performance, typically optimizing a clustering and non-clustering loss. In such cases, an autoencoder is typically connected with a clustering network, and the final clustering is jointly learned by both the autoencoder and clustering network. Instead, we propose to learn an autoencoded embedding and then search this further for the underlying manifold. For simplicity, we then cluster this with a shallow clustering algorithm, rather than a deeper network. We study a number of local and global manifold learning methods on both the raw data and autoencoded embedding, concluding that UMAP in our framework is best able to find the most clusterable manifold in the embedding, suggesting local manifold learning on an autoencoded embedding is effective for discovering higher quality discovering clusters. We quantitatively show across a range of image and time-series datasets that our method has competitive performance against the latest deep clustering algorithms, including out-performing current state-of-the-art on several. We postulate that these results show a promising research direction for deep clustering. The code can be found at https://github.com/rymc/n2d
Figures
Reference graph
Works this paper leans on
-
[1]
H. Abdi and L. J. Williams, “Principal component analysis,” WIREs Comput. Stat., vol. 2, no. 4, pp. 433–459, Jul. 2010. [Online]. Available: https://doi.org/10.1002/wics.101
doi:10.1002/wics.101 2010
-
[2]
Deep clustering for unsupervised learning of visual features,
M. Caron, P. Bojanowski, A. Joulin, and M. Douze, “Deep clustering for unsupervised learning of visual features,” in Proceedings of the European Conference on Computer Vision (ECCV) , 2018, pp. 132–149
work page 2018
-
[3]
Deep adaptive image clustering,
J. Chang, L. Wang, G. Meng, S. Xiang, and C. Pan, “Deep adaptive image clustering,” in Proceedings of the IEEE International Conference on Computer Vision , 2017, pp. 5879–5887
work page 2017
-
[4]
M. Ester, H.-P. Kriegel, J. Sander, and X. Xu, “A density-based algorithm for discovering clusters a density-based algorithm for discovering clusters in large spatial databases with noise,” in Proceedings of the Second International Conference on Knowledge Discovery and Data Mining, ser. KDD’96. AAAI Press, 1996, pp. 226–231. [Online]. Available: http://d...
-
[5]
Deep clustering via joint convolutional autoencoder embedding and relative entropy minimization,
K. Ghasedi Dizaji, A. Herandi, C. Deng, W. Cai, and H. Huang, “Deep clustering via joint convolutional autoencoder embedding and relative entropy minimization,” inProceedings of the IEEE International Conference on Computer Vision , 2017, pp. 5736–5745
work page 2017
-
[6]
Adaptive self-paced deep clustering with data augmentation,
X. Guo, X. Liu, E. Zhu, X. Zhu, M. Li, X. Xu, and J. Yin, “Adaptive self-paced deep clustering with data augmentation,” IEEE Transactions on Knowledge and Data Engineering , pp. 1–1, 2019
work page 2019
-
[7]
Improved deep embedded clustering with local structure preservation,
X. Guo, L. Gao, X. Liu, and J. Yin, “Improved deep embedded clustering with local structure preservation,” in Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI-17, 2017, pp. 1753–1759. [Online]. Available: https://doi.org/10. 24963/ijcai.2017/243
work page 2017
-
[8]
Word re-embedding via manifold dimensionality retention,
S. Hasan and E. Curry, “Word re-embedding via manifold dimensionality retention,” in Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing . Copenhagen, Denmark: Association for Computational Linguistics, Sep. 2017, pp. 321–326. [Online]. Available: https://www.aclweb.org/anthology/D17-1033
work page 2017
Show all 32 references
-
[9]
Word embeddings as metric recovery in semantic spaces,
T. B. Hashimoto, D. Alvarez-Melis, and T. S. Jaakkola, “Word embeddings as metric recovery in semantic spaces,” Transactions of the Association for Computational Linguistics , vol. 4, pp. 273–286,
-
[10]
Independent component analysis: algorithms and applications,
A. Hyv ¨arinen and E. Oja, “Independent component analysis: algorithms and applications,” Neural networks, vol. 13, no. 4-5, pp. 411–430, 2000
2000
-
[11]
Deep clustering: On the link between discriminative models and k-means,
M. Jabi, M. Pedersoli, A. Mitiche, and I. B. Ayed, “Deep clustering: On the link between discriminative models and k-means,” arXiv preprint arXiv:1810.04246, 2018
2018 arXiv
-
[12]
Data clustering: a review,
A. K. Jain, M. N. Murty, and P. J. Flynn, “Data clustering: a review,” ACM computing surveys (CSUR) , vol. 31, no. 3, pp. 264–323, 1999
1999
-
[13]
Variational deep embedding: An unsupervised and generative approach to clustering,
Z. Jiang, Y . Zheng, H. Tan, B. Tang, and H. Zhou, “Variational deep embedding: An unsupervised and generative approach to clustering,” in Proceedings of the 26th International Joint Conference on Artificial Intelligence, ser. IJCAI’17. AAAI Press, 2017, pp. 1965–1972. [Online]...
2017
-
[14]
Hierarchical clustering schemes,
S. C. Johnson, “Hierarchical clustering schemes,” Psychometrika, vol. 32, no. 3, pp. 241–254, 1967
1967
-
[15]
Adam: A method for stochastic optimization,
D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980 , 2014
2014 arXiv
-
[16]
Discriminatively boosted image clustering with fully convolutional auto-encoders,
F. Li, H. Qiao, and B. Zhang, “Discriminatively boosted image clustering with fully convolutional auto-encoders,”Pattern Recognition, vol. 83, pp. 161–173, 2018
2018
-
[17]
Least squares quantization in pcm,
S. P. Lloyd, “Least squares quantization in pcm,” IEEE Transactions on Information Theory, vol. 28, pp. 129–137, 1982
1982
-
[18]
Visualizing data using t-sne,
L. v. d. Maaten and G. Hinton, “Visualizing data using t-sne,” Journal of machine learning research , vol. 9, no. Nov, pp. 2579–2605, 2008
2008
-
[19]
Umap: Uniform manifold approximation and projection for dimension reduction,
L. McInnes, J. Healy, and J. Melville, “Umap: Uniform manifold approximation and projection for dimension reduction,” arXiv preprint arXiv:1802.03426, 2018
2018 arXiv
-
[20]
Clustergan : Latent space clustering in generative adversarial networks,
S. Mukherjee, H. Asnani, E. Lin, and S. Kannan, “Clustergan : Latent space clustering in generative adversarial networks,” The Thirty-Third AAAI Conference on Artificial Intelligence (AAAI) , 2019
2019
-
[21]
Rectified linear units improve restricted boltzmann machines,
V . Nair and G. E. Hinton, “Rectified linear units improve restricted boltzmann machines,” in Proceedings of the 27th International Conference on International Conference on Machine Learning , ser. ICML’10. USA: Omnipress, 2010, pp. 807–814. [Online]. Available: http://dl.acm.o...
-
[22]
On spectral clustering: Anal- ysis and an algorithm,
A. Y . Ng, M. I. Jordan, and Y . Weiss, “On spectral clustering: Anal- ysis and an algorithm,” in Advances in neural information processing systems, 2002, pp. 849–856
2002
-
[23]
Gaussian mixture models,
D. A. Reynolds, “Gaussian mixture models,” in Encyclopedia of Bio- metrics, 2009
2009
-
[24]
Nonlinear dimensionality reduction by locally linear embedding,
S. T. Roweis and L. K. Saul, “Nonlinear dimensionality reduction by locally linear embedding,” science, vol. 290, no. 5500, pp. 2323–2326, 2000
2000
-
[25]
A global geometric framework for nonlinear dimensionality reduction,
J. B. Tenenbaum, V . De Silva, and J. C. Langford, “A global geometric framework for nonlinear dimensionality reduction,” science, vol. 290, no. 5500, pp. 2319–2323, 2000
2000
-
[26]
Recent advances in autoencoder-based representation learning,
M. Tschannen, O. Bachem, and M. Lucic, “Recent advances in autoencoder-based representation learning,” 3rd workshop on Bayesian Deep Learning (NeurIPS 2018) , 2018
2018
-
[27]
Locally embedding autoencoders: A semi-supervised manifold learning approach of document representation,
C. Wei, S. Luo, X. Ma, H. Ren, J. Zhang, and L. Pan, “Locally embedding autoencoders: A semi-supervised manifold learning approach of document representation,” PLOS ONE , vol. 11, no. 1, pp. 1–20, 01
-
[28]
Unsupervised deep embedding for clustering analysis,
J. Xie, R. Girshick, and A. Farhadi, “Unsupervised deep embedding for clustering analysis,” in Proceedings of The 33rd International Conference on Machine Learning , ser. Proceedings of Machine Learning Research, M. F. Balcan and K. Q. Weinberger, Eds., vol. 48. New York, New ...
2016
-
[29]
Available: https://doi.org/10.1371/journal.pone.0146672
[Online]. Available: https://doi.org/10.1371/journal.pone.0146672
-
[30]
Joint unsupervised learning of deep representations and image clusters,
J. Yang, D. Parikh, and D. Batra, “Joint unsupervised learning of deep representations and image clusters,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2016
2016
-
[31]
Towards k- means-friendly spaces: Simultaneous deep learning and clustering,
B. Yang, X. Fu, N. D. Sidiropoulos, and M. Hong, “Towards k- means-friendly spaces: Simultaneous deep learning and clustering,” in Proceedings of the 34th International Conference on Machine Learning- Volume 70. JMLR. org, 2017, pp. 3861–3870
2017
-
[2016]
Available: https://transacl.org/ojs/index.php/tacl/article/ view/809
[Online]. Available: https://transacl.org/ojs/index.php/tacl/article/ view/809
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.