REVIEW 2 major objections 5 minor 114 references
Group-invariant affinity kernels make spectral embedding recover the quotient geometry of data with known symmetries, with better sample rates.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · grok-4.5
2026-07-13 01:12 UTC pith:PXZ7FCST
load-bearing objection Solid, usable theory: three G-invariant kernels converge pointwise to explicit operators on M/G with a dim(G) rate gain; free-action is the main scope limit, not a hidden flaw. the 2 major comments →
Group Invariant Spectral Embedding
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
Under a free, smooth, isometric action of a compact Lie group G on a compact Riemannian data manifold M, the random-walk graph Laplacian built from any of three G-invariant kernels (minimum, integral, or invariant-features) converges pointwise, for smooth G-invariant test functions, to an explicit second-order differential operator D on the quotient N = M/G, with variance error O_P(n^{-1/2} ε^{-1/2-(d-p)/4}) that reflects the reduced dimension d-p.
What carries the argument
Three G-invariant kernels (K_min = max_g K(x,g·y), K_int = ∫_G K(x,g·y) dη, and K_IF = K(φ(x),φ(y)) for a G-invariant feature map φ) that make the graph Laplacian descend to an operator on the quotient; the main convergence theorem identifies that operator as Δ_N minus a log-δ drift (min/integral) or the pullback of a weighted Laplacian on the image of φ (features).
Load-bearing premise
The group must act freely on the data manifold so that the space of orbits is itself a smooth manifold; if some points are fixed by nontrivial group elements the quotient becomes singular and the stated operators are no longer defined on a manifold.
What would settle it
On a data set whose true quotient geometry is known (e.g., SO(3) point clouds whose only intrinsic motion is a circle), compute the spectral embedding with each invariant kernel and with the ordinary Euclidean kernel at large n; if the invariant embeddings fail to recover a circle while the Euclidean one succeeds, or if the observed sample-complexity scaling does not improve by roughly dim(G), the central claim is false.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes spectral embedding for data with known continuous symmetries by replacing the usual affinity kernel with one of three G-invariant kernels (minimization over G, integration over G, or a G-invariant feature map). Under the hypotheses that a compact Lie group G acts smoothly, freely and isometrically on a compact Riemannian submanifold M without boundary, Theorem 3.7 proves that the associated random-walk graph Laplacians converge pointwise, for smooth G-invariant test functions, to explicit second-order operators on the quotient N = M/G (Laplace–Beltrami plus a log-δ drift for the min and integral kernels; pullback of a weighted Laplacian on the image of the feature map for the invariant-features kernel). The variance term improves from the classical O_P(n^{-1/2} ε^{-1/2-d/4}) to O_P(n^{-1/2} ε^{-1/2-(d-p)/4}), reflecting the drop in effective dimension by p = dim(G). Corollaries treat non-uniform sampling and the constant-orbit-volume case (eigenfunction correspondence). Numerical checks on SO(3) recover the predicted rate slopes; experiments on rotated point clouds, tomographic projections and spinning toys show that the invariant embeddings recover the intrinsic quotient geometry while the Euclidean kernel does not.
Significance. If the pointwise analysis holds, the work supplies a clean, kernel-level route to symmetry-aware unsupervised manifold learning that is complementary to equivariant networks and to data-augmentation arguments in supervised learning. The improved sample-complexity bound (Remark 3.11) is concrete and matches the dimension reduction one expects from quotienting by G. The three kernels cover the main practical constructions (Procrustes/alignment, group averaging, and classical invariants such as Gram matrices or bispectra), and the proofs reuse standard local-smoothing + Bernstein and Riemannian-submersion tools in a transparent way. Public code and an explicit free-action hypothesis (flagged as open for orbifolds) further strengthen the contribution. The main limitation relative to the algorithmic claim is that only pointwise consistency is proved; spectral consistency of eigenvalues/eigenvectors, which Algorithm 1 actually uses, is left open.
major comments (2)
- Theorem 3.7 and Corollaries 3.9–3.15 establish only pointwise convergence of L_RW f. Algorithm 1 and the experimental claims, however, rely on the low-lying eigenvectors of L_RW. Spectral consistency (eigenvalue/eigenvector convergence to those of D on N) is acknowledged as open in the Conclusion but is load-bearing for the method as stated; without it the link from the proved operator limit to the recovered embeddings remains heuristic. A short discussion of what is already known for the classical case (e.g., Calder–Trillos, Cheng–Wu) and which obstacles remain for the invariant kernels would clarify the gap.
- Section 5 chooses distinct hand-tuned bandwidths for different kernels (e.g., ε = 47 vs. 3000 in §5.1; ε = 0.005 vs. 3×10^{-8} in §5.2) with no common selection rule or sensitivity analysis. Because the rate statements balance bias O(ε) against a variance term that depends on ε and on effective dimension, unequal ε choices make the visual comparison of geometry recovery and the claim of better sample efficiency only partially controlled. A single cross-validated or median-heuristic rule applied uniformly, or a short ablation over ε, would make the experimental support for the rate improvement more convincing.
minor comments (5)
- Figure 2 caption and surrounding text: the predicted slopes under ∇f = 0 are stated as −0.75 (Euclidean) and −0.5 (invariant); the fitted Euclidean slope (−0.853) is farther from theory than the invariant ones. A brief remark on the larger sample size needed for the classical rate (already noted in the text) would help the reader interpret the discrepancy.
- Definition 3.6 and Lemma 4.1: the normalization Vol(G) = 1 is used throughout; a one-line reminder that Haar measure is left-invariant and unique up to scale would avoid confusion when readers compare with other conventions (e.g., bi-invariant metrics on SO(3)).
- Example 3.16 / Eq. (31): the factor 1/2 relating the Frobenius and bi-invariant Laplacians on SO(3) is correct but easy to miss; cross-referencing Chirikjian–Kyatkin more explicitly would help.
- Section 3.1 (Computation): the cost discussion for min/integral kernels is useful; a short pointer to the concrete quadrature sizes used in the experiments (m = 200 group samples in §3.5) would make reproducibility easier.
- Typos / notation: “And´ en” and “Shkolnisky” appear with inconsistent accents; “rotoreflection” is fine but “O(d)” vs. “SO(d)” should be checked for consistency in Example 3.3; Appendix D table is helpful and could be referenced earlier.
Circularity Check
No significant circularity: Theorem 3.7 and rates follow from classical local-smoothing/Bernstein arguments, Riemannian submersion geometry, and one published prior lemma; nothing reduces to a fit or self-definition.
full rationale
The load-bearing claim (Thm. 3.7) is a pointwise asymptotic for three explicitly constructed G-invariant kernels. The minimum-kernel case is proved from scratch in §4.1 via the Fubini quotient formula (Lem. 4.1, classical), local Gaussian smoothing on N (Lem. 4.2, App. C, standard Taylor + Ricci expansion), and Bernstein concentration; the first-order coefficient -2 arises directly from the product rule on δ. The invariant-features case reduces in §4.3 to the ordinary non-uniform graph-Laplacian theorem (Thm. 2.2) on the image manifold im ϕ, again with no free parameters. The integral-kernel case invokes Rosen et al. (2024, Thm. 11) plus the classical projection formula of Le (2001) (Lem. 3.13); the cited paper is published, parameter-free under the same free-action hypotheses, and supplies an independent proof rather than an unverified uniqueness claim. Bandwidth ε is a free asymptotic parameter, not fitted to force the claimed operator or rate. Experiments are qualitative geometry-recovery checks and do not enter the derivation. No step equates a “prediction” to its own input by construction, imports a self-uniqueness theorem, or renames a known pattern. The free-action hypothesis is stated explicitly and used throughout; it is not hidden circularity.
Axiom & Free-Parameter Ledger
free parameters (2)
- kernel bandwidth ε (experiments) =
dataset-dependent (e.g. 47, 3000, 0.005, 3e-8)
- number of group samples for min/integral kernels =
m=200 in rate experiment; unspecified elsewhere
axioms (6)
- domain assumption G compact Lie group acts smoothly, freely, and by isometries on compact boundaryless Riemannian submanifold M ⊂ R^D
- domain assumption Orbit-volume density δ is smooth (and positive)
- domain assumption Data are i.i.d. from uniform (or smooth density ρ) measure on M; graph is connected
- standard math Classical pointwise convergence of random-walk graph Laplacians to Laplace–Beltrami / Fokker–Planck operators (Singer 2006; Coifman–Lafon 2006)
- standard math Rosen et al. (2024) Thm. 11: normalized G-GL converges to Δ_M with rate involving (d−p)
- domain assumption For IF kernel, induced map ϕ̄ : N → im ϕ is a diffeomorphism
invented entities (2)
-
minimum kernel K_min
independent evidence
-
orbit-volume density δ and induced δ̄ on N
independent evidence
read the original abstract
Spectral embedding methods are widely used for dimensionality reduction and clustering of high-dimensional datasets with intrinsic low-dimensional structures. Although many datasets of practical interest exhibit invariance under symmetries such as rotations, standard spectral embedding methods do not account for this, treating symmetry-related data points as unrelated. Our approach to this problem is to incorporate the symmetries directly into the affinity kernels used for spectral embedding. We analyze the case of a Riemannian data manifold $M$ with symmetries given by a compact Lie group~$G$ and prove that, under suitable conditions, graph Laplacians constructed from three types of invariant kernels converge pointwise to explicit second-order differential operators on the quotient space $M/G$. Our analysis implies improved convergence rates, as the effective dimension drops according to the dimension of the group. We validate our approach on datasets with $\mathrm{SO}(2)$ or $\mathrm{SO}(3)$ symmetry, and show that $G$-invariant spectral embedding recovers the intrinsic geometry of the data, in contrast to standard spectral embedding, which fails to do so even in the limit of infinite data.
Figures
Reference graph
Works this paper leans on
-
[1]
Journal of Computer and System Sciences , volume =
Towards a theoretical foundation for. Journal of Computer and System Sciences , volume =. 2008 , issn =. doi:10.1016/j.jcss.2007.08.006 , author =
-
[2]
International Journal of Mathematics and Mathematical Sciences , volume =
Le, Huiling , title =. International Journal of Mathematics and Mathematical Sciences , volume =. doi:10.1155/S0161171201003751 , abstract =
-
[3]
Rose, Peter W. and Prli. The. Nucleic Acids Research , year =. doi:10.1093/nar/gkw1000 , pmid =
-
[4]
A variational approach to the consistency of spectral clustering , journal =
Nicol. A variational approach to the consistency of spectral clustering , journal =. 2016 , volume =
2016
-
[5]
2026 , organization=
Fraiman, Michael and Hoyos, Paulina and Bendory, Tamir and Kileel, Joe and Mickelin, Oscar and Sharon, Nir and Singer, Amit , booktitle=. 2026 , organization=
2026
-
[6]
IEEE Transactions on Computational Imaging , year =
Zhao, Zhizhen and Shkolnisky, Yoel and Singer, Amit , title =. IEEE Transactions on Computational Imaging , year =. doi:10.1109/TCI.2016.2514700 , pmid =
-
[7]
Acta Crystallographica Section A , year =
Wolfgang Kabsch , title =. Acta Crystallographica Section A , year =
-
[8]
Maslen , title =
David K. Maslen , title =. Modern Signal Processing , editor =. 2004 , doi =
2004
-
[9]
IEEE Transactions on Pattern Analysis and Machine Intelligence , year =
Shinji Umeyama , title =. IEEE Transactions on Pattern Analysis and Machine Intelligence , year =
-
[10]
Trefethen and J
Lloyd N. Trefethen and J. A. C. Weideman , title =. SIAM Review , year =
-
[11]
Peter H. Sch. A generalized solution of the orthogonal. Psychometrika , year =
-
[12]
Error Estimates for Spectral Convergence of the Graph Laplacian on Random Geometric Graphs Toward the
Nicol. Error Estimates for Spectral Convergence of the Graph Laplacian on Random Geometric Graphs Toward the. Foundations of Computational Mathematics , year =
-
[13]
Lowe , title =
David G. Lowe , title =. International Journal of Computer Vision , year =
-
[14]
2003 , publisher=
Introduction to Smooth Manifolds , author=. 2003 , publisher=
2003
-
[15]
2021 , month =
Nicholas Kayban , title =. 2021 , month =
2021
-
[16]
Marc Alexa , booktitle=. Super-. 2022 , pages=
2022
-
[17]
Eigen-convergence of
Xiuyuan Cheng and Nan Wu , journal=. Eigen-convergence of. 2022 , volume=
2022
-
[18]
1960 , publisher=
The classification of G-spaces , authors=. 1960 , publisher=
1960
-
[20]
Applied and Computational Harmonic Analysis , volume =
The. Applied and Computational Harmonic Analysis , volume =. 2024 , issn =. doi:10.1016/j.acha.2024.101637 , author =
-
[21]
Groups and Geometric Analysis: Integral Geometry, Invariant Differential Operators, and Spherical Functions , author=. doi:10.1090/surv/083 , year=
-
[22]
2023 , doi =
An introduction to optimization on smooth manifolds , author =. 2023 , doi =
2023
-
[23]
, title =
Abraham, Ralph and Marsden, Jerrold E. , title =
-
[24]
On Spectral Clustering: Analysis and an algorithm , url =
Ng, Andrew and Jordan, Michael and Weiss, Yair , booktitle =. On Spectral Clustering: Analysis and an algorithm , url =
-
[25]
Aaditya Rangan and Marina Spivak and Joakim Andén and Alex Barnett , title =. Inverse Problems , abstract =. 2020 , month =. doi:10.1088/1361-6420/ab4e66 , url =
-
[26]
2013 , publisher=
Zhao, Zhizhen and Singer, Amit , journal=. 2013 , publisher=
2013
-
[27]
Rotationally invariant image representation for viewing direction classification in cryo-
Zhao, Zhizhen and Singer, Amit , journal=. Rotationally invariant image representation for viewing direction classification in cryo-. 2014 , publisher=
2014
-
[28]
Neural Information Processing Systems (NeurIPS) , year=
Fixed-rank approximation of a positive-semidefinite matrix from streaming data , author=. Neural Information Processing Systems (NeurIPS) , year=
-
[29]
SIAM Journal on Scientific computing , volume=
An algorithm for the principal component analysis of large data sets , author=. SIAM Journal on Scientific computing , volume=. 2011 , publisher=
2011
-
[30]
Applied and Computational Harmonic Analysis , volume =
From graph to manifold. Applied and Computational Harmonic Analysis , volume =. 2006 , issn =. doi:10.1016/j.acha.2006.03.004 , author =
-
[31]
SIAM Journal on Imaging Sciences , volume =
Landa, Boris and Shkolnisky, Yoel , title =. SIAM Journal on Imaging Sciences , volume =. 2018 , doi =
2018
-
[32]
, title =
Chen, Shuxiao and Dobriban, Edgar and Lee, Jane H. , title =. J. Mach. Learn. Res. , month =. 2020 , issue_date =
2020
-
[33]
Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , year=
Belkin, Mikhail and Niyogi, Partha , journal=. Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , year=
-
[34]
Tenenbaum and Vin de Silva and John C
Joshua B. Tenenbaum and Vin de Silva and John C. Langford , title =. Science , volume =. 2000 , doi =
2000
-
[35]
and Wintgen, P
Sulanke, R. and Wintgen, P. , lccn=. Differentialgeometrie und Faserb. 1972 , publisher=
1972
-
[36]
2023 , publisher=
An Introduction to Optimization on Smooth Manifolds , author=. 2023 , publisher=
2023
-
[37]
Absil and J\'
P.-A. Absil and J\'. Projection-like Retractions on Matrix Manifolds , publisher =. 2012 , journal =
2012
-
[38]
1987 , series =
Transformation Groups , author =. 1987 , series =
1987
-
[39]
Convergence of
Belkin, Mikhail and Niyogi, Partha , booktitle =. Convergence of
-
[40]
Diffusion maps , journal =. 2006 , issn =. doi:10.1016/j.acha.2006.04.006 , author =
-
[41]
Improved spectral convergence rates for graph
Calder, Jeff and Trillos, Nicolas Garcia , journal=. Improved spectral convergence rates for graph. 2022 , publisher=
2022
-
[42]
Methods in Enzymology , volume=
Cryo-electron tomography and subtomogram averaging , author=. Methods in Enzymology , volume=. 2016 , publisher=
2016
-
[43]
Manifold Learning with Arbitrary Norms
Joe Kileel and Amit Moscovich and Nathan Zelesko and Amit Singer. Manifold Learning with Arbitrary Norms. Journal of Fourier Analysis and Applications. 2021. doi:10.1007/s00041-021-09879-2
-
[44]
Proceedings of the Twelth International Conference on Artificial Intelligence and Statistics , pages =
Multi-Manifold Semi-Supervised Learning , author =. Proceedings of the Twelth International Conference on Artificial Intelligence and Statistics , pages =. 2009 , editor =
2009
-
[45]
2017 , editor =
Moscovich, Amit and Jaffe, Ariel and Boaz, Nadler , booktitle =. 2017 , editor =
2017
-
[46]
Journal of the American Statistical Association , volume =
Mingyen Cheng and Hautieng Wu , title =. Journal of the American Statistical Association , volume =. 2013 , publisher =
2013
-
[47]
1995 , publisher=
Group Theory and Physics , author=. 1995 , publisher=
1995
-
[48]
2016 , publisher=
Harmonic Analysis for Engineers and Applied Scientists: Updated and Expanded Edition , author=. 2016 , publisher=
2016
-
[49]
Approximation of functions over manifolds: A Moving Least-Squares approach , journal =
Barak Sober and Yariv Aizenbud and David Levin , keywords =. Approximation of functions over manifolds: A Moving Least-Squares approach , journal =. 2021 , issn =. doi:10.1016/j.cam.2020.113140 , url =
-
[50]
The Journal of Machine Learning Research , volume=
A group-theoretic framework for data augmentation , author=. The Journal of Machine Learning Research , volume=. 2020 , publisher=
2020
-
[51]
Annual Review of Biomedical Data Science , volume=
Computational methods for single-particle electron cryomicroscopy , author=. Annual Review of Biomedical Data Science , volume=. 2020 , publisher=
2020
-
[52]
Acta Crystallographica Section D: Structural Biology , volume=
High-resolution structure determination using high-throughput electron cryo-tomography , author=. Acta Crystallographica Section D: Structural Biology , volume=. 2022 , publisher=
2022
-
[53]
2011 , publisher=
Stochastic Models, Information Theory, and Lie Groups, Volume 2: Analytic Methods and Modern Applications , author=. 2011 , publisher=
2011
-
[54]
2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI) , pages=
Earthmover-based manifold learning for analyzing molecular conformation spaces , author=. 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI) , pages=. 2020 , organization=
2020
-
[55]
The Laplacian on a Riemannian Manifold: An Introduction to Analysis on Manifolds , publisher=
Rosenberg, Steven , year=. The Laplacian on a Riemannian Manifold: An Introduction to Analysis on Manifolds , publisher=
-
[56]
2008 , publisher=
Group Theoretical Methods in Machine Learning , author=. 2008 , publisher=
2008
-
[57]
1946 , edition =
Hermann Weyl , title =. 1946 , edition =
1946
-
[58]
Jean Gallier and Jocelyn Quaintance , title =. 2020 , edition =. doi:10.1007/978-3-030-46040-2 , note =
-
[59]
2004 , edition =
Multiple View Geometry in Computer Vision , author =. 2004 , edition =
2004
-
[60]
Journal of Machine Learning Research , volume =
Visualizing Data using t-SNE , author =. Journal of Machine Learning Research , volume =. 2008 , url =
2008
-
[61]
Reinhart , journal =
Bruce L. Reinhart , journal =. Foliated Manifolds with Bundle-Like Metrics , urldate =. 1959 , doi =
1959
-
[62]
and Weizsäcker, H.v
Smolyanov, O.G. and Weizsäcker, H.v. and Wittich, O. , title =. Potential Analysis , volume =. 2007 , doi =
2007
-
[63]
, title =
Sherbert, Donald R. , title =. Pacific Journal of Mathematics , volume =. 1963 , doi =
1963
-
[64]
Sigurdur Helgason , title =. J. Differential Geometry , volume =. 1972 , pages =
1972
-
[65]
John M. Lee , title =. 2018 , edition =. doi:10.1007/978-3-319-91755-9 , pages =
-
[66]
2017 , edition =
Jost, Jürgen , title =. 2017 , edition =
2017
-
[67]
2009 , series =
Grigor'yan, Alexander , title =. 2009 , series =
2009
-
[68]
2014 , series =
Bakry, Dominique and Gentil, Ivan and Ledoux, Michel , title =. 2014 , series =
2014
-
[69]
1996 , edition =
Risken, Hannes , title =. 1996 , edition =
1996
-
[70]
1996 , isbn =
Sakai, Takashi , title =. 1996 , isbn =
1996
-
[71]
Sasaki, Kyoyu and Dockerill, Susan and Adamiak, Dorota A. and Tickle, Ian J. and Blundell, Tom , year =. X-ray analysis of glucagon and its relationship to receptor binding , volume =. Nature , publisher =. doi:10.1038/257751a0 , number =
-
[72]
Blundell, T.L. and Sasaki, K. and Dockerill, S. and Tickle, I.J. , year =. X-RAY ANALYSIS OF GLUCAGON AND ITS RELATIONSHIP TO RECEPTOR BINDING , url =. doi:10.2210/pdb1gcn/pdb , publisher =
-
[73]
Garrett Wright and Joakim Andén and Vineet Bansal and Junchao Xia and Chris Langfield and Josh Carmichael and Kris Sowattanangkul and Robbie Brook and Yunpeng Shi and Ayelet Heimowitz and Gabi Pragier and Itay Sason and Amit Moscovich and Yoel Shkolnisky and Amit Singer , title =. 2025 , copyright =. doi:10.5281/ZENODO.5657281 , url =
-
[74]
Statistics and Computing , volume =
von Luxburg, Ulrike , title =. Statistics and Computing , volume =. 2007 , doi =
2007
-
[75]
The Annals of Statistics , year =
von Luxburg, Ulrike and Belkin, Mikhail and Bousquet, Olivier , title =. The Annals of Statistics , year =
-
[76]
International Conference on Machine Learning (ICML) , year =
Group Equivariant Convolutional Networks , author =. International Conference on Machine Learning (ICML) , year =
-
[77]
Advances in Neural Information Processing Systems (NeurIPS) , volume =
Deep Sets , author =. Advances in Neural Information Processing Systems (NeurIPS) , volume =. 2017 , url =
2017
-
[78]
and Welling, Max , booktitle =
Cohen, Taco S. and Welling, Max , booktitle =. Steerable. 2017 , url =
2017
-
[79]
International Conference on Learning Representations (ICLR) , year =
Invariant and Equivariant Graph Networks , author =. International Conference on Learning Representations (ICLR) , year =
-
[80]
arXiv preprint arXiv:2104.13478 , year =
Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges , author =. arXiv preprint arXiv:2104.13478 , year =
-
[81]
Communications on Pure and Applied Mathematics , volume =
Singer, Amit and Wu, Hau-Tieng , title =. Communications on Pure and Applied Mathematics , volume =. 2012 , doi =
2012
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.