REVIEW 4 major objections 3 minor 30 references
Large-Scale Sparse Subspace Clustering Using Landmarks
T0 review · 4 major / 3 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read This paper claims that sparse subspace clustering can run in linear time by expressing each point against a small landmark set and reading the spectral embedding from a skinny SVD, without ever forming the n-by-n affinity matrix.
desk verdict A correct and useful SVD trick for landmark-based SSC, but the accuracy claim is only weakly supported by the experiments and an unproven representativity assumption. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the landmark-reduced coefficient matrix $C \in \mathbb{R}^{m \times n}$, whose $j$-th column expresses point $x_j$ sparsely over the landmarks $X_0$. Its absolute value $\tilde{C}$ plays the role of a factorization of the affinity matrix: $W = \tilde{C}^T\tilde{C}$. The key identity is that the normalized affinity's eigen-decomposition follows from the skinny SVD $\tilde{C}D^{-1/2} = U\Sigma P^T$, since $D^{-1/2}WD^{-1/2} = P\Sigma^2 P^T$, so the top $K$ eigenvectors are read off as right singular vectors of the $m \times n$ factor. This couples with the choice of landmark selection (uniform random sampling or K-medoids) to determine how well the reduced affinity preserves subspace membership.
What would settle it
Run FSC on a union of subspaces in which one subspace contributes very few points or has dimension close to the number of landmarks, and compare its top $K$ right singular vectors against full-data SSC: if the under-sampled subspace's points become indistinguishable in the embedding while full-data SSC separates them, the landmark-reduction premise is violated.
Extended reading notes
Core claim
The central claim is that the entire two-step SSC pipeline can be moved into a reduced domain of $m$ landmarks. After selecting landmarks $X_0$, each data point $x_j$ is encoded by solving $\min_{c_j \in \mathbb{R}^m} \|c_j\|_1 + \frac{\lambda}{2}\|x_j - \sum_{i: x_i \in X_0\setminus\{x_j\}} c_{ij} x_i\|^2_2$, producing an $m \times n$ coefficient matrix $C$. Taking $\tilde{C} = |C|$ and $W = \tilde{C}^T\tilde{C}$, the paper shows that the top $K$ eigenvectors of the normalized affinity $D^{-1/2} W D^{-1/2}$ are exactly the top $K$ right singular vectors of $\tilde{C}D^{-1/2} = U\Sigma P^T$. Consequently the spectral embedding is obtained from one $m \times n$ SVD, and the $n \times n$ affinity matrix is never formed. Underlying this is the assertion that the number of landmarks needed scales with the number of subspaces and their intrinsic dimensions, so the reduced-domain computation remains faithful for moderate $m$.
Load-bearing premise
The premise that carries the whole argument is that a small subset of landmarks preserves the subspace-membership structure of the data, so sparse coefficients computed over landmarks separate the same clusters as coefficients computed over the full dataset.
Editorial extensions
If this is right
- For a fixed number of landmarks $m$, both the sparse-coding step and the spectral-embedding step scale linearly in the data size $n$, so the method is viable where full-data SSC is not.
- The $n \times n$ affinity matrix is never constructed, so memory use stays proportional to $m n$ plus the data itself rather than to $n^2$.
- Accuracy can be traded against speed by tuning $m$; the experiments show uniform sampling is already competitive, while K-medoids landmark selection helps most when $m$ is small.
- The method inherits the behavior of the sparse self-expressiveness model: points in the same subspace are encouraged to have nonzero coefficients, and the landmark formulation keeps that signal while shrinking the dictionary.
Reading between the lines
- Editorial inference: the same $m \times n$ SVD trick applies to any clustering method that produces a low-rank affinity factor, for instance low-rank kernel clustering or landmark-based graph cuts, so the paper's spectral step is a transferable subroutine.
- Editorial inference: uniform landmark sampling should be expected to fail when a subspace is underrepresented, so a natural extension is an adaptive landmark selector that monitors reconstruction error per subspace or per cluster.
- Editorial inference: the claim that $m$ depends only on the number and dimensions of subspaces suggests a testable scaling law: fixing accuracy, the minimal $m$ should grow roughly with $\sum_k d_k$ rather than with $n$.
- Editorial inference: because the affinity is $\tilde{C}^T\tilde{C}$, the method resembles low-rank kernel approximations and could be combined with randomized SVD to reduce the cost of the $m \times n$ decomposition when $m$ is large.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes Fast Subspace Clustering (FSC), a landmark-based variant of sparse subspace clustering. Given a dataset X, the method selects a small subset of m landmarks X0, either by uniform sampling or by K-medoids clustering, and then represents each data point as a sparse linear combination of the landmarks by solving the ℓ1-regularized problem in Eq. (5). This yields an m×n coefficient matrix C. The paper's main technical claim is that the spectral embedding can be computed directly from an m×n SVD of the scaled coefficient matrix, without forming the n×n affinity matrix W = |C|^T |C|; Eqs. (6)–(7) show that the top K eigenvectors of the normalized affinity are the top K right singular vectors of |C| D^{-1/2}. Algorithm 1 summarizes the method. Experiments on one synthetic union-of-subspaces configuration and on a subset of MNIST compare FSC against SSC-ADMM, SSC-OMP, and the exemplar-based ESC method, reporting favorable accuracy and runtime scaling.
Significance. If the accuracy claim holds broadly, the method would be a practical contribution: it reduces both the sparse-coding and spectral-clustering costs to linear in n for fixed landmark count m, making SSC feasible on large datasets. The derivation in Eqs. (6)–(7) is correct: D^{-1/2} W D^{-1/2} = (|C| D^{-1/2})^T (|C| D^{-1/2}), so the top eigenvectors are indeed the top right singular vectors. This is a genuine structural insight and the complexity benefit is real for FSC-uniform. However, the 'high clustering accuracy' part of the central claim is not established with the same rigor. The paper supplies no theoretical guarantee that sparse coefficients computed against a small landmark set preserve subspace membership, and the experiments cover only narrow, balanced regimes. The missing hyperparameter reporting and the unanalyzed cost of K-medoids landmark selection further weaken the case. These gaps are fixable, but they are load-bearing for the paper's main contribution.
major comments (4)
- [Section 3, Eq. (5) and the affinity W = |C|^T |C|] The central accuracy claim relies on the unproven premise that sparse coefficients computed against a small landmark set X0 carry the same subspace-membership information as coefficients computed against the full dataset. The self-expressiveness property that underpins SSC has no proven analogue here: the dictionary X0 is not guaranteed to contain points from every subspace, and the ℓ1 objective in Eq. (5) is not constrained to select landmarks from the point's own subspace. The citation to [5] in Section 3 concerns full-dictionary SSC and does not transfer to reduced-dictionary coefficients. If a subspace has no landmark or too few landmarks, points from that subspace may be represented using landmarks from other subspaces, producing cross-subspace edges in W; conversely, points in the same subspace may use disjoint landmark subsets, yielding zero affinity and splitting the cluster. This is load-bearing for the 'high clustering accuracy' claim; at minimum, an explicit analysis or a targeted experiment on imbalanced and under-sampled subspaces is needed.
- [Section 4, Figs. 1 and 2] The experiments are too narrow to support the general accuracy claim. The synthetic setup uses a single balanced configuration (D=16, K=5, d=6, 720 points per subspace), and the MNIST experiment uses a balanced subset of four digits. No experiments address imbalanced subspaces, higher intrinsic dimensions, or varying noise levels, which are precisely the regimes where landmark coverage is most fragile. In addition, the MNIST results in Fig. 2 are reported only as averages, without error bars or significance assessment, so it is unclear whether the advantage over ESC is stable. Broadening the experimental evidence is necessary to justify the claim that FSC maintains high accuracy while scaling linearly.
- [Section 4, experimental setup] The regularization parameter λ in Eq. (5) is never reported, and the hyperparameter selection procedure for FSC is not described. The text states that for the full-data baselines 'the parameters are selected to achieve the best clustering accuracy,' but no equivalent statement or actual values are given for FSC and ESC. This is a reproducibility gap and weakens the fairness of the comparison, since λ directly controls the sparsity of the coefficients on which the affinity is built.
- [Section 3, Algorithm 1] The advertised linear-time scaling is not fully accounted for in the nonuniform variant. The analysis covers the sparse-coding step (linear in n for fixed m) and the SVD of an m×n matrix, but the cost of selecting landmarks by K-medoids clustering is not included or bounded. Standard K-medoids algorithms scale superlinearly in n; if the linear-time claim is meant to apply to FSC-nonuniform, the landmark-selection step needs an explicit complexity analysis, or the paper should state that the linear-time claim applies only to FSC-uniform.
minor comments (3)
- [Eq. (7)] The rank parameter r is introduced but not defined; it should be specified as the rank of |C| D^{-1/2} or as a truncation parameter chosen by the user.
- [Algorithm 1, step 5] The step applies K-means to the rows of the top K right singular vectors, but the paper does not state whether row normalization of these vectors is performed before K-means, which is common in spectral clustering; a brief clarification would help reproducibility.
- [Section 3, Eq. (6)] The computation of the degree matrix D is only described for the diagonal entries; if some data point has all zero coefficients, the corresponding degree is zero and D^{-1/2} is undefined. The paper should mention how such a degenerate case is handled in practice.
Circularity Check
No circularity: the linear-time claim follows from the algorithm's structure and an exact SVD identity; the landmark-accuracy premise is an unproven empirical risk, not a circular reduction.
full rationale
The derivation chain is self-contained. The claimed linear-time property follows directly from the algorithm's structure: solving the fixed-m sparse optimization problem in (5) independently for each of the n points costs O(n) for constant m, and the spectral-clustering step uses the exact identity in Eqs. (6)-(7), namely D^{-1/2} W D^{-1/2} = (\tilde{C} D^{-1/2})^T (\tilde{C} D^{-1/2}), so the top K eigenvectors of the normalized affinity matrix are the top K right singular vectors of \tilde{C}D^{-1/2} without ever forming the n-by-n matrix W. This is standard linear algebra, not an input recycled as an output. The accuracy claims are benchmarked against external algorithms (SSC-ADMM, a modified ADMM solver, SSC-OMP, and ESC) on synthetic data and an MNIST feature set, and no fitted parameter is later relabeled as a prediction. The author's self-citations, for example [18] for efficient solvers and [28] for randomized dimension reduction, are contextual and are not load-bearing for the central derivation. The weakest point, that coefficients computed against a small landmark set preserve subspace membership, is asserted heuristically without proof and is an empirical correctness risk rather than a circular step: the paper does not define the landmark set in terms of the clustering output, nor does it derive the clustering output from the landmark-selection criterion by construction. No equation in the paper reduces to its own input, so the appropriate finding is no significant circularity.
Assumptions & free parameters
free parameters (2)
- regularization parameter λ =
not reported
- number of landmarks m =
varies 100 to 500 in experiments
assumptions (3)
- domain assumption Data points are drawn from a union of K low-dimensional subspaces (Eq. 1) with noise.
- domain assumption A small set of landmarks X0 of size m suffices to reveal the subspace structure, relying on the heuristic that the required sample size is proportional to K and intrinsic dimensions (Section 3, citing [5]).
- domain assumption The top K eigenvectors of the normalized affinity W = \tilde{C}^T \tilde{C} yield the correct clustering assignments, i.e., the standard spectral clustering assumption holds.
Cite this review
Pith. "Pith review of Large-Scale Sparse Subspace Clustering Using Landmarks." pith.science (2026). https://pith.science/paper/4HXQAIUK
@misc{pith2026190800683,
author = {Pith},
title = {Pith review of: Large-Scale Sparse Subspace Clustering Using Landmarks},
year = {2026},
howpublished = {\url{https://pith.science/paper/4HXQAIUK}},
note = {Machine review of arXiv:1908.00683}
}
read the original abstract
Subspace clustering methods based on expressing each data point as a linear combination of all other points in a dataset are popular unsupervised learning techniques. However, existing methods incur high computational complexity on large-scale datasets as they require solving an expensive optimization problem and performing spectral clustering on large affinity matrices. This paper presents an efficient approach to subspace clustering by selecting a small subset of the input data called landmarks. The resulting subspace clustering method in the reduced domain runs in linear time with respect to the size of the original data. Numerical experiments on synthetic and real data demonstrate the effectiveness of our method.
Figures
Reference graph
Works this paper leans on
-
[5]
M. Soltanolkotabi, E. Elhamifar, and E. Cand` es, “Robus t subspace clustering,” The Annals of Statistics , vol. 42, no. 2, pp. 669–699, 2014
work page 2014
-
[1]
Towards K-m eans-friendly spaces: Simul- taneous deep learning and clustering,
B. Yang, X. Fu, N. Sidiropoulos, and M. Hong, “Towards K-m eans-friendly spaces: Simul- taneous deep learning and clustering,” in International Conference on Machine Learning , pp. 3861–3870, 2017
work page 2017
-
[2]
A cluster-then-label semi-supervised learning approach for pathology image classification,
M. Peikari, S. Salama, S. Nofech-Mozes, and A. Martel, “A cluster-then-label semi-supervised learning approach for pathology image classification,” Scientific reports, vol. 8, no. 1, p. 7193, 2018
work page 2018
-
[3]
Subspace clustering,
R. Vidal, “Subspace clustering,” IEEE Signal Processing Magazine , vol. 28, no. 2, pp. 52–68, 2011
2011
-
[4]
A geometric analysis of subspace clustering with outliers,
M. Soltanolkotabi and E. Cand` es, “A geometric analysis of subspace clustering with outliers,” The Annals of Statistics , vol. 40, no. 4, pp. 2195–2238, 2012
work page 2012
-
[6]
Subspace clustering of hi gh-dimensional data: a predictive approach,
B. McWilliams and G. Montana, “Subspace clustering of hi gh-dimensional data: a predictive approach,” Data Mining and Knowledge Discovery , vol. 28, no. 3, pp. 736–772, 2014
work page 2014
-
[7]
Sparse subspace clustering: Algorithm, theory, and applica- tions,
E. Elhamifar and R. Vidal, “Sparse subspace clustering: Algorithm, theory, and applica- tions,” IEEE Transactions on Pattern Analysis and Machine Intelligen ce, vol. 35, no. 11, pp. 2765–2781, 2013
work page 2013
-
[8]
E. Elhamifar and R. Vidal, “Sparse subspace clustering, ” in IEEE Conference on Computer Vision and Pattern Recognition , pp. 2790–2797, 2009
work page 2009
Show all 30 references
-
[9]
A tutorial on spectral clustering,
U. Von Luxburg, “A tutorial on spectral clustering,” Statistics and Computing , vol. 17, no. 4, pp. 395–416, 2007
2007
-
[10]
Fast approximate spect ral clustering,
D. Yan, L. Huang, and M. Jordan, “Fast approximate spect ral clustering,” in International Conference on Knowledge Discovery and Data Mining , pp. 907–916, 2009
2009
-
[11]
Learning dee p representations for graph clustering,
F. Tian, B. Gao, Q. Cui, E. Chen, and T. Liu, “Learning dee p representations for graph clustering,” in AAAI Conference on Artificial Intelligence , pp. 1293–1299, 2014
2014
-
[12]
A randomized app roach to efficient kernel clustering,
F. Pourkamali-Anaraki and S. Becker, “A randomized app roach to efficient kernel clustering,” in IEEE Global Conference on Signal and Information Processing , pp. 207–211, 2016
2016
-
[13]
Spe ctral clustering of large- scale data by directly solving normalized cut,
X. Chen, W. Hong, F. Nie, D. He, M. Yang, and J. Huang, “Spe ctral clustering of large- scale data by directly solving normalized cut,” in International Conference on Knowledge Discovery and Data Mining , pp. 1206–1215, 2018
2018
-
[14]
A simple and fast algorithm for K-med oids clustering,
H. Park and C. Jun, “A simple and fast algorithm for K-med oids clustering,” Expert systems with applications, vol. 36, no. 2, pp. 3336–3341, 2009
2009
-
[15]
Sketched subspace clu stering,
P. Traganitis and G. Giannakis, “Sketched subspace clu stering,” IEEE Transactions on Sig- nal Processing, vol. 66, no. 7, pp. 1663–1675, 2018. 8
2018
-
[16]
Preconditioned data sparsification for big data with applications to PCA and K-means,
F. Pourkamali-Anaraki and S. Becker, “Preconditioned data sparsification for big data with applications to PCA and K-means,” IEEE Transactions on Information Theory , vol. 63, no. 5, pp. 2954–2974, 2017
2017
-
[17]
Distributed optimization and statistical learning via the alternating direction method of multipliers,
S. Boyd, N. Parikh, E. Chu, B. Peleato, and J. Eckstein, “ Distributed optimization and statistical learning via the alternating direction method of multipliers,” Foundations and Trends in Machine learning , vol. 3, no. 1, pp. 1–122, 2011
2011
-
[18]
Efficient solvers for sparse subspace clustering,
F. Pourkamali-Anaraki and S. Becker, “Efficient solvers for sparse subspace clustering,” arXiv preprint arXiv:1804.06291 , 2018
2018 arXiv
-
[19]
Scalable sparse subs pace clustering by orthogonal matching pursuit,
C. You, D. Robinson, and R. Vidal, “Scalable sparse subs pace clustering by orthogonal matching pursuit,” in IEEE Conference on Computer Vision and Pattern Recognition , pp. 3918–3927, 2016
2016
-
[20]
A scalable exem plar-based subspace clustering algorithm for class-imbalanced data,
C. You, C. Li, D. Robinson, and R. Vidal, “A scalable exem plar-based subspace clustering algorithm for class-imbalanced data,” in European Conference on Computer Vision , pp. 68– 85, 2018
2018
-
[21]
Scalable K-means cl ustering via lightweight coresets,
O. Bachem, M. Lucic, and A. Krause, “Scalable K-means cl ustering via lightweight coresets,” in International Conference on Knowledge Discovery and Data Min ing, pp. 1119–1127, 2018
2018
-
[22]
Train ing Gaussian mixture models at scale via coresets,
M. Lucic, M. Faulkner, A. Krause, and D. Feldman, “Train ing Gaussian mixture models at scale via coresets,” Journal of Machine Learning Research , vol. 18, pp. 1–25, 2018
2018
-
[23]
Rando mized clustered Nystr¨ om for large- scale kernel machines,
F. Pourkamali-Anaraki, S. Becker, and M. Wakin, “Rando mized clustered Nystr¨ om for large- scale kernel machines,” in AAAI Conference on Artificial Intelligence , pp. 3960–3967, 2018
2018
-
[24]
Finding structu re with randomness: Probabilis- tic algorithms for constructing approximate matrix decomp ositions,
N. Halko, P. Martinsson, and J. Tropp, “Finding structu re with randomness: Probabilis- tic algorithms for constructing approximate matrix decomp ositions,” SIAM review , vol. 53, no. 2, pp. 217–288, 2011
2011
-
[25]
Sparse modeling for im age and vision processing,
J. Mairal, F. Bach, and J. Ponce, “Sparse modeling for im age and vision processing,” Foun- dations and Trends in Computer Graphics and Vision , vol. 8, no. 2–3, pp. 85–283, 2014
2014
-
[26]
Gradient- based learning applied to docu- ment recognition,
Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient- based learning applied to docu- ment recognition,” Proceedings of the IEEE , vol. 86, no. 11, pp. 2278–2324, 1998
1998
-
[27]
Invariant scattering convolut ion networks,
J. Bruna and S. Mallat, “Invariant scattering convolut ion networks,” IEEE Transactions on Pattern Analysis and Machine Intelligence , vol. 35, no. 8, pp. 1872–1886, 2013
2013
-
[28]
Memory and compu tation efficient PCA via very sparse random projections,
F. Pourkamali-Anaraki and S. Hughes, “Memory and compu tation efficient PCA via very sparse random projections,” in International Conference on Machine Learning , pp. 1341– 1349, 2014
2014
-
[29]
Dimensiona lity-reduced subspace clustering,
R. Heckel, M. Tschannen, and H. B¨ olcskei, “Dimensiona lity-reduced subspace clustering,” Information and Inference: A Journal of the IMA , vol. 6, no. 3, pp. 246–283, 2017
2017
-
[30]
A general framework for understanding compressed sub- space clustering algorithms,
L. Meng, G. Li, J. Yan, and Y. Gu, “A general framework for understanding compressed sub- space clustering algorithms,” IEEE Journal of Selected Topics in Signal Processing , vol. 12, no. 6, pp. 1504–1519, 2018. 9
2018
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.