pith. sign in

arxiv: 2604.13307 · v1 · submitted 2026-04-14 · 💻 cs.CV · cs.LG

Deep Spatially-Regularized and Superpixel-Based Diffusion Learning for Unsupervised Hyperspectral Image Clustering

Vutichart Buranasiri , James M. Murphy This is my paper

Pith reviewed 2026-05-10 15:27 UTC · model grok-4.3

classification 💻 cs.CV cs.LG
keywords hyperspectral image clusteringdiffusion learningmasked autoencodersuperpixel segmentationVision Transformerunsupervised representation learningdata manifold
0
0 comments X

The pith

DS²DL builds diffusion graphs in a masked autoencoder latent space to better reflect hyperspectral data geometry and raise clustering accuracy.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper proposes an unsupervised method for clustering hyperspectral images by first learning a compressed latent representation through a masked autoencoder with a Vision Transformer backbone. This pretraining step uses spatial context and long-range spectral correlations while training efficiently on only a small subset of pixels. The method then applies entropy rate superpixel segmentation and constructs a spatially regularized diffusion graph using Euclidean and diffusion distances computed in the latent space rather than the original image data. The central argument is that these latent distances more faithfully capture the intrinsic manifold geometry, which in turn produces more accurate pixel labels and higher-quality clusters.

Core claim

The paper claims that by learning a denoised latent representation of the hyperspectral image via an unsupervised masked autoencoder with Vision Transformer backbone, the DS²DL algorithm can construct spatially regularized diffusion graphs using distances in this compressed space that more faithfully reflect the intrinsic geometry of the underlying data manifold, thereby improving labeling accuracy and clustering quality over methods that operate directly in the original hyperspectral space.

What carries the argument

The unsupervised masked autoencoder (UMAE) with Vision Transformer backbone that produces a latent representation, followed by entropy rate superpixel segmentation and construction of a spatially regularized diffusion graph using latent-space distances.

Load-bearing premise

The distances derived from the latent representation learned by the unsupervised masked autoencoder more faithfully reflect the intrinsic geometry of the data manifold than distances computed in the original hyperspectral image space.

What would settle it

Clustering experiments on the Botswana or KSC datasets that yield equal or lower accuracy when the diffusion graph is built with latent-space distances instead of original-space distances.

Figures

Figures reproduced from arXiv: 2604.13307 by James M. Murphy, Vutichart Buranasiri.

Figure 1
Figure 1. Figure 1: Top: Botswana; Bottom: KSC. 6 [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
read the original abstract

An unsupervised framework for hyperspectral image (HSI) clustering is proposed that incorporates masked deep representation learning with diffusion-based clustering, extending the Spatially-Regularized Superpixel-based Diffusion Learning ($S^2DL$) algorithm. Initially, a denoised latent representation of the original HSI is learned via an unsupervised masked autoencoder (UMAE) model with a Vision Transformer backbone. The UMAE takes spatial context and long-range spectral correlations into account and incorporates an efficient pretraining process via masking that utilizes only a small subset of training pixels. In the next stage, the entropy rate superpixel (ERS) algorithm is used to segment the image into superpixels, and a spatially regularized diffusion graph is constructed using Euclidean and diffusion distances within the compressed latent space instead of the HSI space. The proposed algorithm, Deep Spatially-Regularized Superpixel-based Diffusion Learning ($DS^2DL$), leverages more faithful diffusion distances and subsequent diffusion graph construction that better reflect the intrinsic geometry of the underlying data manifold, improving labeling accuracy and clustering quality. Experiments on Botswana and KSC datasets demonstrate the efficacy of $DS^2DL$.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The manuscript proposes DS²DL, an extension of the prior S²DL algorithm for unsupervised hyperspectral image (HSI) clustering. It first learns a denoised latent representation of the input HSI using an unsupervised masked autoencoder (UMAE) with a Vision Transformer backbone that incorporates spatial context and long-range spectral correlations via efficient masking-based pretraining on a small subset of pixels. Entropy rate superpixels (ERS) are then computed, followed by construction of a spatially regularized diffusion graph that employs Euclidean and diffusion distances computed in the compressed latent space (rather than raw HSI space). The central claim is that this yields more faithful diffusion distances reflecting the intrinsic data manifold geometry, thereby improving labeling accuracy and clustering quality, with efficacy demonstrated on the Botswana and KSC datasets.

Significance. If the core claim is substantiated, the work could advance unsupervised HSI clustering by showing how deep masked representation learning can be integrated with diffusion methods to better capture manifold structure while maintaining computational efficiency through masking. The logical extension of S²DL and the use of ViT for spectral-spatial modeling are constructive elements. However, absent direct validation of the manifold-fidelity assumption, the significance remains provisional.

major comments (2)
  1. [Abstract and Experiments] Abstract and Experiments section: The headline claim that the UMAE latent space 'leverages more faithful diffusion distances and subsequent diffusion graph construction that better reflect the intrinsic geometry of the underlying data manifold' is load-bearing for the contribution, yet no supporting analysis is provided—no comparison of diffusion distance matrices, no manifold quality metrics (trustworthiness, continuity, or geodesic error), and no ablation that isolates the latent-space distances while holding ERS superpixels and graph construction fixed. Gains could therefore arise from dimensionality reduction or denoising alone rather than superior geometry capture.
  2. [Method] Method section: The description of how diffusion distances are computed within the UMAE latent space (versus the original HSI space) lacks explicit equations for the diffusion operator, transition probabilities, or the precise form of spatial regularization applied to the graph; without these, it is impossible to verify that the latent-space construction is indeed more faithful to the manifold or to reproduce the claimed improvements.
minor comments (2)
  1. [Abstract] Abstract: The acronym UMAE is introduced without immediate expansion (though later defined as unsupervised masked autoencoder); similarly, ensure ERS is expanded on first use for reader clarity.
  2. [Title and Abstract] Title and abstract: Minor inconsistency in phrasing—the title includes 'and Superpixel-Based' while the abstract expansion of DS²DL omits explicit mention of superpixels in the acronym definition, though the method clearly incorporates them.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive and detailed feedback. We address each major comment below and will revise the manuscript accordingly to improve clarity, reproducibility, and substantiation of our claims.

read point-by-point responses

  1. Referee: [Abstract and Experiments] Abstract and Experiments section: The headline claim that the UMAE latent space 'leverages more faithful diffusion distances and subsequent diffusion graph construction that better reflect the intrinsic geometry of the underlying data manifold' is load-bearing for the contribution, yet no supporting analysis is provided—no comparison of diffusion distance matrices, no manifold quality metrics (trustworthiness, continuity, or geodesic error), and no ablation that isolates the latent-space distances while holding ERS superpixels and graph construction fixed. Gains could therefore arise from dimensionality reduction or denoising alone rather than superior geometry capture.

    Authors: We agree that direct supporting analysis would strengthen the central claim. In the revised manuscript we will add a dedicated ablation that isolates the effect of computing Euclidean and diffusion distances in the UMAE latent space versus the original HSI space while keeping the ERS superpixel segmentation and graph-construction procedure identical. We will also include a side-by-side comparison of the resulting diffusion distance matrices (or their key statistics) on the Botswana and KSC datasets and report the corresponding clustering metrics. Although classical manifold-quality metrics such as trustworthiness are less commonly applied to diffusion distances, we will evaluate their suitability and, if appropriate, include them or substitute alternative indicators of improved manifold fidelity. revision: yes

  2. Referee: [Method] Method section: The description of how diffusion distances are computed within the UMAE latent space (versus the original HSI space) lacks explicit equations for the diffusion operator, transition probabilities, or the precise form of spatial regularization applied to the graph; without these, it is impossible to verify that the latent-space construction is indeed more faithful to the manifold or to reproduce the claimed improvements.

    Authors: We acknowledge the omission. The revised Method section will contain the complete mathematical formulation: the diffusion operator and its transition probabilities defined on the latent-space affinity graph, the precise expression for the diffusion distance, and the explicit spatial-regularization term that incorporates superpixel adjacency. These additions will enable verification that the latent-space construction better respects the data manifold and will facilitate exact reproduction of the reported results. revision: yes

Circularity Check

0 steps flagged

No significant circularity; central improvement is an asserted property of the proposed architecture rather than a reduction to fitted inputs or self-citations

full rationale

The paper describes an algorithmic pipeline that first trains an unsupervised masked autoencoder to obtain a latent representation, then applies entropy-rate superpixels and constructs a diffusion graph using distances in that latent space. The claim that these distances are 'more faithful' and 'better reflect the intrinsic geometry' is presented as a consequence of the design choice (UMAE + ViT + masking) rather than derived from any equation that equates the output to its own inputs by construction. No self-definitional loops, fitted parameters renamed as predictions, or load-bearing uniqueness theorems imported via self-citation appear in the provided derivation chain. The extension of S²DL is a straightforward modular replacement of the distance metric and is not used to justify the metric's superiority. The overall method is therefore self-contained as a proposed procedure whose validity rests on empirical results rather than tautological re-expression of its components.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The central claim rests on the domain assumption that the learned latent space better captures manifold geometry; no free parameters or invented entities are explicitly introduced in the abstract.

axioms (1)
  • domain assumption The latent representation from the unsupervised masked autoencoder captures the intrinsic geometry of the hyperspectral data manifold more faithfully than the original high-dimensional space.
    Invoked to justify constructing the diffusion graph in latent space rather than raw HSI space.

pith-pipeline@v0.9.0 · 5507 in / 1230 out tokens · 33995 ms · 2026-05-10T15:27:59.069109+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

30 extracted references · 30 canonical work pages · 1 internal anchor

  1. [1]

    Self-supervised learning in remote sensing: A review.IEEE Geoscience and Remote Sensing Magazine, 10(4):213–247, 2022

    Yi Wang, Conrad M Albrecht, Nassim Ait Ali Braham, Lichao Mou, and Xiao Xiang Zhu. Self-supervised learning in remote sensing: A review.IEEE Geoscience and Remote Sensing Magazine, 10(4):213–247, 2022

    work page 2022

  2. [2]

    Deep learning for urban land use category classification: A review and experimental assessment.Remote Sensing of Environment, 311:114290, 2024

    Ziming Li, Bin Chen, Shengbiao Wu, Mo Su, Jing M Chen, and Bing Xu. Deep learning for urban land use category classification: A review and experimental assessment.Remote Sensing of Environment, 311:114290, 2024

    work page 2024

  3. [3]

    A comprehensive survey for hyperspectral image classification: The evolution from conventional to transformers and mamba models.Neurocomputing, page 130428, 2025

    Muhammad Ahmad, Salvatore Distefano, Adil Mehmood Khan, Manuel Mazzara, Chenyu Li, Hao Li, Jagannath Aryal, Yao Ding, Gemine Vivone, and Danfeng Hong. A comprehensive survey for hyperspectral image classification: The evolution from conventional to transformers and mamba models.Neurocomputing, page 130428, 2025

    work page 2025

  4. [4]

    Hyperspectral remote sensing scenes

    Grupo de Inteligencia Computacional (GIC). Hyperspectral remote sensing scenes. https://www.ehu.eus/ ccwintco/index.php/Hyperspectral_Remote_Sensing_Scenes, 2021. Collected by M. Graña, M. A. Veganzons, and B. Ayerdi. Last modified 12 July 2021. Accessed 30 Dec 2025

    work page 2021

  5. [5]

    Y .-Q. Zhao, L. Zhang, and S. G. Kong. Band-subset-based clustering and fusion for hyperspectral imagery classification.IEEE Transactions on Geoscience and Remote Sensing, 49(2):747–756, Feb. 2011

    work page 2011

  6. [6]

    Unsupervised clustering and active learning of hyperspectral images with nonlinear diffusion.IEEE Transactions on Geoscience and Remote Sensing, 57(3):1829–1845, 2018

    James M Murphy and Mauro Maggioni. Unsupervised clustering and active learning of hyperspectral images with nonlinear diffusion.IEEE Transactions on Geoscience and Remote Sensing, 57(3):1829–1845, 2018

    work page 2018

  7. [7]

    Learning by unsupervised nonlinear diffusion.Journal of Machine Learning Research, 20(160):1–56, 2019

    Mauro Maggioni and James M Murphy. Learning by unsupervised nonlinear diffusion.Journal of Machine Learning Research, 20(160):1–56, 2019

    work page 2019

  8. [8]

    Spectral–spatial diffusion geometry for hyperspectral image clustering

    James M Murphy and Mauro Maggioni. Spectral–spatial diffusion geometry for hyperspectral image clustering. IEEE Geoscience and Remote Sensing Letters, 17(7):1243–1247, 2019

    work page 2019

  9. [9]

    Multiscale clustering of hyperspectral images through spectral-spatial diffusion geometry

    Sam L Polk and James M Murphy. Multiscale clustering of hyperspectral images through spectral-spatial diffusion geometry. In2021 IEEE International Geoscience and Remote Sensing Symposium IGARSS, pages 4688–4691. IEEE, 2021

    work page 2021

  10. [10]

    Unsupervised detection of ash dieback disease (hymenoscyphus fraxineus) using diffusion-based hyperspectral image clustering

    Sam L Polk, Aland HY Chan, Kangning Cui, Robert J Plemmons, David A Coomes, and James M Murphy. Unsupervised detection of ash dieback disease (hymenoscyphus fraxineus) using diffusion-based hyperspectral image clustering. InIGARSS 2022-2022 IEEE International Geoscience and Remote Sensing Symposium, pages 2287–2290. IEEE, 2022

    work page 2022

  11. [11]

    Unsupervised diffusion and volume maximization-based clustering of hyperspectral images.Remote Sensing, 15(4):1053, 2023

    Sam L Polk, Kangning Cui, Aland HY Chan, David A Coomes, Robert J Plemmons, and James M Murphy. Unsupervised diffusion and volume maximization-based clustering of hyperspectral images.Remote Sensing, 15(4):1053, 2023. 7 APREPRINT- APRIL16, 2026

    work page 2023

  12. [12]

    Polk, Yinyi Lin, Hongsheng Zhang, James M

    Kangning Cui, Ruoning Li, Sam L. Polk, Yinyi Lin, Hongsheng Zhang, James M. Murphy, Robert J. Plem- mons, and Raymond H. Chan. Superpixel-based and spatially regularized diffusion learning for unsupervised hyperspectral image clustering.IEEE Transactions on Geoscience and Remote Sensing, 62:1–18, 2024

    work page 2024

  13. [13]

    Deep diffusion processes for active learning of hyperspectral images

    Abiy Tasissa, Duc Nguyen, and James M Murphy. Deep diffusion processes for active learning of hyperspectral images. In2021 IEEE International Geoscience and Remote Sensing Symposium IGARSS, pages 3665–3668. IEEE, 2021

    work page 2021

  14. [14]

    Deep convolutional neural networks for hyperspectral image classification.Journal of Sensors, 2015(1):258619, 2015

    Wei Hu, Yangyu Huang, Li Wei, Fan Zhang, and Hengchao Li. Deep convolutional neural networks for hyperspectral image classification.Journal of Sensors, 2015(1):258619, 2015

    work page 2015

  15. [15]

    Convolutional neural networks for hyperspectral image classification

    Shiqi Yu, Sen Jia, and Chunyan Xu. Convolutional neural networks for hyperspectral image classification. Neurocomputing, 219:88–98, 2017

    work page 2017

  16. [16]

    Graph convolutional subspace clustering: A robust subspace clustering framework for hyperspectral image.IEEE Transactions on Geoscience and Remote Sensing, 59(5):4191–4202, 2020

    Yaoming Cai, Zijia Zhang, Zhihua Cai, Xiaobo Liu, Xinwei Jiang, and Qin Yan. Graph convolutional subspace clustering: A robust subspace clustering framework for hyperspectral image.IEEE Transactions on Geoscience and Remote Sensing, 59(5):4191–4202, 2020

    work page 2020

  17. [17]

    Yang, Y .-Q

    J. Yang, Y .-Q. Zhao, and J. C.-W. Chan. Learning and transferring deep joint spectral–spatial features for hyperspectral classification.IEEE Transactions on Geoscience and Remote Sensing, 55(8):4729–4742, Aug. 2017

    work page 2017

  18. [18]

    Unsupervised feature learning using recurrent neural nets for segmenting hyperspectral images.IEEE Geoscience and Remote Sensing Letters, 18(12):2142–2146, 2020

    Lukasz Tulczyjew, Michal Kawulok, and Jakub Nalepa. Unsupervised feature learning using recurrent neural nets for segmenting hyperspectral images.IEEE Geoscience and Remote Sensing Letters, 18(12):2142–2146, 2020

    work page 2020

  19. [19]

    Spectral- former: Rethinking hyperspectral image classification with transformers.IEEE Transactions on Geoscience and Remote Sensing, 60:1–15, 2021

    Danfeng Hong, Zhu Han, Jing Yao, Lianru Gao, Bing Zhang, Antonio Plaza, and Jocelyn Chanussot. Spectral- former: Rethinking hyperspectral image classification with transformers.IEEE Transactions on Geoscience and Remote Sensing, 60:1–15, 2021

    work page 2021

  20. [20]

    Sdst: Self-supervised double-structure transformer for hyperspectral images clustering.IEEE Transactions on Geoscience and Remote Sensing, 62:1–14, 2024

    Fulin Luo, Yi Liu, Yule Duan, Tan Guo, Lefei Zhang, and Bo Du. Sdst: Self-supervised double-structure transformer for hyperspectral images clustering.IEEE Transactions on Geoscience and Remote Sensing, 62:1–14, 2024

    work page 2024

  21. [21]

    Learning long-term dependencies with gradient descent is difficult.IEEE Transactions on Neural Networks, 5(2):157–166, March 1994

    Yoshua Bengio, Patrice Simard, and Paolo Frasconi. Learning long-term dependencies with gradient descent is difficult.IEEE Transactions on Neural Networks, 5(2):157–166, March 1994

    work page 1994

  22. [22]

    Masked auto-encoding spectral– spatial transformer for hyperspectral image classification.IEEE Transactions on Geoscience and Remote Sensing, 60:1–16, 2022

    Damian Ibañez, Ruben Fernandez-Beltran, Filiberto Pla, and Naoto Yokoya. Masked auto-encoding spectral– spatial transformer for hyperspectral image classification.IEEE Transactions on Geoscience and Remote Sensing, 60:1–16, 2022

    work page 2022

  23. [23]

    An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale.arXiv preprint arXiv:2010.11929, 2020

    work page internal anchor Pith review Pith/arXiv arXiv 2010

  24. [24]

    Entropy rate superpixel segmentation

    Ming-Yu Liu, Oncel Tuzel, Srikumar Ramalingam, and Rama Chellappa. Entropy rate superpixel segmentation. InCVPR 2011, pages 2097–2104. IEEE, 2011

    work page 2011

  25. [25]

    Geometric diffusions as a tool for harmonic analysis and structure definition of data: Diffusion maps

    Ronald R Coifman, Stephane Lafon, Ann B Lee, Mauro Maggioni, Boaz Nadler, Frederick Warner, and Steven W Zucker. Geometric diffusions as a tool for harmonic analysis and structure definition of data: Diffusion maps. Proceedings of the national academy of sciences, 102(21):7426–7431, 2005

    work page 2005

  26. [26]

    Diffusion maps.Applied and computational harmonic analysis, 21(1):5–30, 2006

    Ronald R Coifman and Stéphane Lafon. Diffusion maps.Applied and computational harmonic analysis, 21(1):5–30, 2006

    work page 2006

  27. [27]

    Yuval Eldar, Michael Lindenbaum, Moshe Porat, and Yehoshua Y . Zeevi. The farthest point strategy for progressive image sampling.IEEE Transactions on Image Processing, 6(9):1305–1315, 1997

    work page 1997

  28. [28]

    Iterative active learning with diffusion geometry for hyperspectral images

    James M Murphy and Mauro Maggioni. Iterative active learning with diffusion geometry for hyperspectral images. In2018 9th Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing (WHISPERS), pages 1–5. IEEE, 2018

    work page 2018

  29. [29]

    Spatially regularized active diffusion learning for high-dimensional images.Pattern Recognition Letters, 135:213–220, 2020

    James M Murphy. Spatially regularized active diffusion learning for high-dimensional images.Pattern Recognition Letters, 135:213–220, 2020

    work page 2020

  30. [30]

    Active diffusion and vca-assisted image segmentation of hyperspectral images

    Sam L Polk, Kangning Cui, Robert J Plemmons, and James M Murphy. Active diffusion and vca-assisted image segmentation of hyperspectral images. InIGARSS 2022-2022 IEEE International Geoscience and Remote Sensing Symposium, pages 1364–1367. IEEE, 2022. 8

    work page 2022