REVIEW 3 major objections 3 minor 54 references
Accelerated decomposition of bistochastic kernel matrices by low rank approximation
T0 review · 3 major / 3 minor · reviewed 2026-08-04 · deepseek-v4-flash
Pith's one-line read A low-rank approximation of the kernel matrix suffices to compute the leading eigenvectors of its bistochastic normalization in O(N r^2) time, without ever forming the N×N matrix.
desk verdict The dilution construction is a genuine algorithmic contribution, but the abstract promises normalized error bounds that are never derived and the numerics are thinner than advertised. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The pivoted partial Cholesky factorization K ≈ F F^T with F ∈ R^{N×r}, which selects r pivots based on diagonal entries and yields the column Nyström approximation. The workhorse identity is that the low-rank approximate normalized matrix P~ = D~^{-1} F F^T Q~^{-1} F F^T D~^{-1} can be written as G G^T with G = (D~^{-1}U) Σ^2 (Q~^{-1/2}U)^T, where U and Σ come from the EVD of F^T F; computing the reduced QR of each N×r factor and an r×r SVD then gives the leading singular vectors and eigenvalues at O(N r^2) cost. This factorization is what converts the non-rank-revealing product of low-rank matrices into a rank-revealing one.
What would settle it
Take a kernel matrix K for which the exact P and its top eigenvectors can be computed (say N ≈ 2000), run the dilution algorithm at increasing ranks r, and measure the relative trace-norm error ||P - P~||_tr / ||P||_tr. If this error does not decrease as r grows, or if it stays orders of magnitude above the relative error of K~, the central claim about approximating the bistochastic eigen-decomposition fails. The same test can be done on a synthetic kernel with a known low-rank-plus-tail structure to separate approximation error from normalization instability.
Extended reading notes
Core claim
The central claim is that the eigenvalue decomposition of the bistochastic normalization P = D^{-1}KQ^{-1}KD^{-1} of a positive definite kernel matrix K can be approximated without forming P or K in full. Writing K ≈ F F^T by pivoted partial Cholesky with rank r, the paper constructs the approximate normalized matrix P~ = D~^{-1} K~ Q~^{-1} K~ D~^{-1} and proves, via a sequence of QR and r×r SVD steps, that its eigenvectors and eigenvalues are obtained in O(N r^2) time. The resulting P~ is exactly bistochastic and positive semidefinite, its leading eigenvectors approximate those of P, and the cost is linear in N for fixed r. The paper does not claim a proven error bound for P versus P~; the
Load-bearing premise
The paper never bounds how close the approximate bistochastic matrix P~ is to the true P; it only bounds the error of the kernel approximation K~, and the normalization step D,Q is a nonlinear function of K, so the accuracy of the computed eigenvectors and eigenvalues rests on an unproved stability assumption about that nonlinear map.
Editorial extensions
If this is right
- Kernel spectral clustering and diffusion-map-style analyses of datasets too large for O(N^3) eigendecomposition become tractable at O(N r^2) with fixed rank r.
- The N(r+1) kernel evaluations make the method attractive when kernel evaluations are expensive, e.g., in high-dimensional delay embeddings.
- The method produces an exactly bistochastic, positive semidefinite matrix P~ whose top eigenvectors can serve as a basis for reduced modeling, smoothing, or dynamical closure.
- The comparison shows the dilution approach preserves a wider eigenvalue spectrum than naive subsampling, which loses information from rejected samples.
Reading between the lines
- A natural next step is to prove (or disprove) a quantitative stability bound for the normalization map K ↦ P; if such a bound holds, the trace-norm error of K~ would translate directly into eigenfunction error for P.
- The same factorization trick could be applied to other normalizations that are rational functions of K, such as the asymmetric or variable-bandwidth diffusion maps, as long as the denominators can be computed from F.
- The exchanged order of eigenfunctions φ3/φ6 in the numerical results suggests that for nearly degenerate eigenvalues the method is better used to produce a subspace rather than individual eigenfunctions; a subspace error metric would be more meaningful.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes an accelerated algorithm for the approximate eigenvalue decomposition of bistochastically normalized kernel matrices. Given a rank-r randomly pivoted partial Cholesky factor F with K≈FFᵀ, the authors form approximate normalization diagonals D̃ and Q̃ from the row sums of K̃=FFᵀ, and derive a rank-revealing factorization of P̃=D̃^{-1}K̃Q̃^{-1}K̃D̃^{-1}. Algorithm 1 computes the leading r eigenpairs of P̃ in O(Nr²) time and O(N(r+1)) kernel evaluations. A Nyström-based subsampling alternative is also presented, and both methods are tested on spatiotemporal pattern extraction for the Kuramoto-Sivashinsky equation. The algebraic derivation of Algorithm 1 is clean and the cost analysis is plausible, but the paper does not deliver the trace-norm error bounds for normalized matrices promised in the abstract, and the numerical section does not quantify the distance between the true bistochastic matrix P and its approximation P̃.
Significance. If the missing stability estimate were supplied, this would be a useful contribution: matrix-free approximate EVD of bistochastic kernel matrices at O(Nr²) cost is relevant for large-scale kernel methods, and the exact factorization of P̃ in §3.1 is elegant. The paper also provides reproducible code, makes a concrete cost comparison between dilution and subsampling, and uses a nontrivial chaotic testbed. At present, however, the advertised trace-norm guarantees are absent, and the numerical claims rely on an unproved stability assumption about the nonlinear normalization map K↦P.
major comments (3)
- [Abstract; §3.3] The abstract states that the paper derives trace norm error bounds for the approximation of two classes of normalized kernel matrices, and §1.3 promises an accelerated decomposition of the bistochastic normalization. However, no theorem in §3 bounds the distance between P and P̃. The only error estimate used is (1.1), which applies to the unnormalized low-rank factor K̃; §3.3 applies it only to K̃. Because D and Q are nonlinear functions of K and P inverts both diagonals, a small trace error in K does not automatically translate to a small error in P. Section 3.3 itself concedes that D̃ and Q̃ may fail to be positive without giving quantitative conditions. This missing stability estimate is load-bearing, since the computed object is the EVD of P̃, not of P.
- [§4.2] The numerical evaluation never reports the error between P and P̃. The only trace-norm errors quoted (7.15% and 13.25%) are for K̃. Figures 2–4 provide qualitative visual comparisons of eigenfunctions, and the projection errors in Figures 6–7 measure the span of the computed eigenfunctions, not the accuracy of individual eigenpairs of P. To substantiate the claim that Algorithm 1 approximates the EVD of P, the paper should report quantities such as ||P−P̃||_tr/||P||_tr, eigenvalue errors, or eigenspace subspace angles, ideally over multiple random trials; the current experiment appears to be a single realization. The manual adjustment of ϵ to 50 after experimentation should also be disclosed as tuning.
- [§3.2] The Nyström extension formula for bistochastic kernels uses point evaluations p(x, x_j), but p is defined through d and q, which are global integrals over the full measure. The paper does not specify how p(x, x_j) is evaluated for out-of-sample x without forming the full N×N kernel matrix, nor what approximation of d and q is used. As written, the claimed O(Nr³) cost is not justified: a naive evaluation of p through sums over the full dataset would be significantly more expensive. This needs clarification so that the cost comparison between dilution and subsampling is meaningful.
minor comments (3)
- [§4.2] The text says the delay-embedded training dataset has N M = 32,500 product-state samples for N=500 and M=64; this should be 32,000.
- [§3.1, Algorithm 1] The step U ← F V Σ^{-1} assumes Σ is nonsingular. If F^T F is rank deficient, the algorithm should either truncate the zero singular values or justify that K positive definite ensures full column rank of F.
- [§4.2] The statement that the bandwidth value ϵ=15 was refined to ϵ=50 'after experimenting with similar bandwidth values' indicates manual tuning; a brief note that the reported results are with the tuned value would improve transparency.
Circularity Check
No significant circularity: exact algebra maps K~ to an exactly bistochastic P~; the [11] error bound is external input, not output.
full rationale
The derivation chain in Sections 3-3.3 is self-contained. Given K ~= K~ = FF^T (eq. 3.1), the paper defines D~=diag(K~1_N), Q~=diag(K~D~^{-1}1_N), and P~=D~^{-1}K~Q~^{-1}K~D~^{-1}. Equations (3.2)-(3.3) are exact algebraic identities for P~: using F^TF=V Sigma^2 V^T, U=FV Sigma^{-1}, the QR decompositions and the r x r SVD produce an exact EVD of P~. Thus the algorithm's output is the EVD of the matrix it constructs; no fitted parameter is renamed as a prediction. The accuracy input (1.1) is an externally proved bound from Chen-Epperly-Tropp-Webber [11], not a self-citation and not derived from P~. The self-citations ([17], [22], [23], [25], [26], [46]) are contextual/motivational (VSA framework, gaussian kernel choice, Galerkin motivation, variable-bandwidth kernels); none is a load-bearing uniqueness theorem or ansatz. There is, however, an explicit rigor gap that the paper itself flags: Section 3.3 says "If this is not the case, the normalization is not well defined and may lead to unpredictable numerical errors," and the paper never proves a bound on ||P-P~||_tr in terms of ||K-K~||_tr; the numerics report only the K~ trace errors (7.15% and 13.25%), not the normalized-matrix error. This missing stability estimate for the nonlinear normalization map K -> P is a correctness/stability risk, not a circularity, because P~ is constructed from K~ rather than used to define it. Therefore the circularity score is 0.
Assumptions & free parameters
free parameters (2)
- bandwidth epsilon =
50
- approximation rank r =
2048
assumptions (5)
- domain assumption The kernel k is continuous, bounded, positive definite, and nonnegative, so K is symmetric positive semidefinite and P is bistochastic.
- standard math The randomly pivoted partial Cholesky algorithm of Chen et al. satisfies the relative trace-norm error bound (1.1).
- domain assumption Eigenfunctions of the kernel integral operator are invariant under the dynamical symmetry group G when the kernel factors through delay embedding.
- domain assumption Kuramoto–Sivashinsky dynamics with L=22 has an invariant probability measure with compact support, and the Fourier/ETDRK4 discretization accurately samples it.
- standard math The Nyström extension formula for eigenfunctions of a kernel integral operator is valid for the bistochastic kernel p.
Cite this review
Pith. "Pith review of Accelerated decomposition of bistochastic kernel matrices by low rank approximation." pith.science (2026). https://pith.science/paper/DZPDEOLQ
@misc{pith2026251026574,
author = {Pith},
title = {Pith review of: Accelerated decomposition of bistochastic kernel matrices by low rank approximation},
year = {2026},
howpublished = {\url{https://pith.science/paper/DZPDEOLQ}},
note = {Machine review of arXiv:2510.26574}
}
read the original abstract
We develop an accelerated algorithm for the approximate eigenvalue decomposition of symmetrically normalized kernel matrices, focusing on a bistochastic normalization. Our approach constructs a low rank approximation of the original kernel matrix by the pivoted partial Cholesky algorithm, and uses it to compute an approximate decomposition of its normalization without requiring the formation of the full kernel matrix. The cost of the proposed algorithm depends linearly on the size of the employed training dataset and quadratically on the rank of the low rank approximation, offering a significant cost reduction compared to the naive approach. We derive trace norm error bounds for the approximation of two classes of normalized kernel matrices. We apply the proposed algorithm to the kernel based extraction of spatiotemporal patterns from chaotic Kuramoto-Sivashinsky dynamics.
Figures
Figures from the paper (4 more)
Reference graph
Works this paper leans on
-
[11]
Berkooz, P
G. Berkooz, P. Holmes, and J. L. Lumley. The proper orthogonal decomposition in the analysis of turbulent flows.Annu. Rev. Fluid Mech., 25(1):539–575, 1993
1993
-
[1]
Introduction.Data matrices of large size often arise in the application of data driven computational methods to various domains. An example is given by kernel matrices, whose entries are determined by the evaluation of a kernel function on a set of data points. Large kernel matrices arise in kernel based methods such as support vector machines, applied to...
arXiv 2025
-
[2]
Related work.There is a large body of literature on the use of low rank approximation methods to accelerate the implementation of kernel methods and enable their application to large datasets. In this work we are interested in kernel methods that require the computation of the eigenvalue decomposition of a kernel matrix, with kernel spectral clustering be...
-
[3]
We consider again a kernel functionk:X×X→Rsatisfying the assumptions listed in Section 1.1
Bistochastic kernel approximation.In this section we extend the approximation schemes outlined in Section 2 to the case of bistochastic normalized kernel matrices (Section 1.1). We consider again a kernel functionk:X×X→Rsatisfying the assumptions listed in Section 1.1. We use a collection ofN∈Nstate samplesX N ={x n}N−1 n=0 ⊂Xto build the kernel matrixK= ...
-
[4]
importance
Application.We apply the two algorithms for the approximate computation of the EVD of bistochastic normalized kernel matrices to the extraction of patterns from spa- tiotemporal dynamics. As the dynamical model we consider the Kuramoto-Sivashinsky (KS) equation ∂tu=−u∂ su−∂ 2 s u−∂ 4 s u t≥0, s∈S(4.1) with periodic boundary conditions on the one dimension...
-
[5]
Conclusion.We developed an algorithm for the approximate computation of the eigen- value decomposition of bistochastic normalized kernel matrices. The proposed algorithm 18 employs a pivoted partial Cholesky algorithm to construct a low rank approximation of the original kernel matrix and compute the approximate eigenvalue decomposition of its bistochasti...
-
[6]
T. Ando. Schur complements and matrix inequalities: operator-theoretic approach. In F. Zhang, editor,The Schur complement and its applications, pages 137–162. Springer, New York, 2005
2005
-
[7]
Aubry, R
N. Aubry, R. Guyonnet, and R. Lima. Spatiotemporal analysis of complex signals: theory and applications.J. Stat. Phys., 64:683–739, 1991
1991
Show all 54 references
-
[8]
Belabbas and P
M.-A. Belabbas and P. J. Wolfe. Fast low-rank approximation for covariance matrices. InIEEE International Workshop on Computational Advances in Multi-Sensor Adaptive Processing, pages 293–296. IEEE, 2007
2007
-
[9]
Belabbas and P
M.-A. Belabbas and P. J. Wolfe. On landmark selection and sampling in high- dimensional data analysis.Phil. Trans. R. Soc. A, 367(1906):4295–4312, 2009
1906
-
[10]
Belabbas and P
M.-A. Belabbas and P. J. Wolfe. Spectral methods in machine learning and new strate- gies for very large datasets.Proc. Natl. Acad. Sci. USA, 106(2):369–374, 2009
2009
-
[12]
Berry and J
T. Berry and J. Harlim. Variable bandwidth diffusion kernels.Appl. Comput. Harmon. Anal., 40(1):68–96, 2016
2016
-
[13]
Berry and T
T. Berry and T. Sauer. Local kernels and the geometric structure of data.Appl. Comput. Harmon. Anal., 40(3):439–469, 2016
2016
-
[14]
Canuto, Y
C. Canuto, Y. M. Hussaini, A. Quarteroni, and T. A. Zang.Spectral methods. Springer, New York, 2006
2006
-
[15]
M. E. Celebi and K. Aydin, editors.Unsupervised learning algorithms. Springer, Cham, 2016
2016
-
[16]
Y. Chen, E. N. Epperly, J. A. Tropp, and R. J. Webber. Randomly pivoted Cholesky: practical approximation of a kernel matrix with few entry evaluations.Comm. Pure Appl. Math., 78:995–1041, 2024
2024
-
[17]
R. R. Coifman and M. J. Hirn. Bi-stochastic kernels via asymmetric affinity functions. Appl. Comput. Harmon. Anal., 35(1):177–180, 2013
2013
-
[18]
R. R. Coifman and S. Lafon. Diffusion maps.Appl. Comput. Harmon. Anal., 21(1):5– 30, 2006. 19
2006
-
[19]
R. R. Coifman, Y. Shkolnisky, F. J. Sigworth, and A. Singer. Graph laplacian tomog- raphy from unknown random projections.IEEE Trans. Image Process., 17(10):1891– 1899, 2008
2008
-
[20]
S. M. Cox and P. C. Matthews. Exponential time differencing for stiff systems.J. Comput. Phys., 176(2):430–455, 2002
2002
-
[21]
Cvitanovic, R
P. Cvitanovic, R. L. Davidchack, and E. Siminos. On the state space geometry of the Kuramoto–Sivashinsky flow in a periodic domain.SIAM J. Appl. Dyn. Syst., 9(1):1–33, 2010
2010
-
[22]
Das and D
S. Das and D. Giannakis. Delay-coordinate maps and the spectra of Koopman opera- tors.J. Stat. Phys., 175(6):1107–1145, 2019
2019
-
[23]
Drineas, M
P. Drineas, M. W. Mahoney, and N. Cristianini. On the Nystr¨ om method for approx- imating a Gram matrix for improved kernel-based learning.J. Mach. Learn. Res., 6(12):2153–2175, 2005
2005
-
[24]
E. N. Epperly, J. A. Tropp, and R. J. Webber. Embrace rejection: kernel matrix approximation by accelerated randomly pivoted Cholesky.arXiv:2410.03969, 2024
2024 arXiv
-
[25]
Fowlkes, S
C. Fowlkes, S. Belongie, Fan Chung, and J. Malik. Spectral grouping using the Nystr¨ om method.IEEE Trans. Pattern Anal. Machine Intell., 26(2):214–225, 2004
2004
-
[26]
Fowlkes, S
C. Fowlkes, S. Belongie, and J. Malik. Efficient spatiotemporal grouping using the Nystr¨ om method. InIEEE Conference on Computer Vision and Pattern Recognition, volume 1, pages I231–I238. IEEE Comput. Soc., 2001
2001
-
[27]
Freeman, D
D. Freeman, D. Giannakis, B. Mintz, A. Ourmazd, and J. Slawinska. Data assimilation in operator algebras.Proc. Natl. Acad. Sci. USA, 120(8):e2211115120, 2023
2023
-
[28]
D. C. Freeman, D. Giannakis, and J. Slawinska. Quantum mechanics for closure of dynamical systems.SIAM Multiscale Model. Simul., 22(1):283–333, 2024
2024
-
[29]
Garreau, W
D. Garreau, W. Jitkrittum, and M. Kanagawa. Large sample analysis of the median heuristic.arXiv:1707.07269, 2018
2018 arXiv
-
[30]
Giannakis
D. Giannakis. Data-driven spectral decomposition and forecasting of ergodic dynamical systems.Appl. Comput. Harmon. Anal., 47(2):338–396, 2019
2019
-
[31]
Giannakis, A
D. Giannakis, A. Ourmazd, J. Slawinska, and Z. Zhao. Spatiotemporal pattern extrac- tion by spectral analysis of vector-valued observables.J. Nonlinear Sci., 29(5):2385– 2445, 2019
2019
-
[32]
Giraud, J
L. Giraud, J. Langou, and M. Rozloznik. The loss of orthogonality in the Gram-Schmidt orthogonalization process.Comput. Math. Appl., 50(7):1069–1075, 2005
2005
-
[33]
Halko, P.-G
N. Halko, P.-G. Martinsson, and J. A. Tropp. Finding structure with randomness: probabilistic algorithms for constructing approximate matrix decompositions.SIAM Rev., 53(2):217–288, 2011
2011
-
[34]
Hinze and S
M. Hinze and S. Volkwein. Proper orthogonal decomposition surrogate models for nonlinear dynamical systems: error estimates and suboptimal control. In P. Benner, D. C. Sorensen, and V. Mehrmann, editors,Dimension reduction of large-scale systems, pages 261–306. Springer, 2005
2005
-
[35]
R. A. Horn and F. Zhang. Basic properties of the Schur complement. In F. Zhang, editor,The Schur complement and its applications, pages 17–46. Springer, New York, 2005. 20
2005
-
[36]
Kassam and L
A.-K. Kassam and L. N. Trefethen. Fourth-order time-stepping for stiff PDEs.SIAM J. Sci. Comput., 26(4):1214–1233, 2005
2005
-
[37]
I. G. Kevrekidis, B. Nicolaenko, and J. C. Scovel. Back in the saddle again: a computer assisted study of the Kuramoto–Sivashinsky equation.SIAM J. Appl. Math., 50(3):760– 790, 1990
1990
-
[38]
Kumar, M
S. Kumar, M. Mohri, and A. Talwalkar. Sampling techniques for the Nystr¨ om method. Proc. Mach. Learn. Res., 5:304–311, 2009
2009
-
[39]
Kuramoto and T
Y. Kuramoto and T. Tsuzuki. Persistent propagation of concentration waves in dissi- pative media far from thermal equilibrium.Progr. Theor. Phys., 55(2):356–369, 1976
1976
-
[40]
Langone and J
R. Langone and J. A. K. Suykens. Fast kernel spectral clustering.Neurocomputing, 268:27–33, 2017
2017
-
[41]
Martinsson and J
P.-G. Martinsson and J. A. Tropp. Randomized numerical linear algebra: foundations and algorithms.Acta Numerica, 29:403–572, 2020
2020
-
[42]
Nakatsukasa and J
Y. Nakatsukasa and J. A. Tropp. Fast and accurate randomized algorithms for linear systems and eigenvalue problems.SIAM J. Matrix Anal. Appl., 45(2):1183–1214, 2024
2024
-
[43]
A. Ng, M. Jordan, and Y. Weiss. On spectral clustering: analysis and an algorithm. In Advances in Neural Information Processing Systems, volume 14, pages 849–856. MIT Press, 2001
2001
-
[44]
D. T. Papageorgiou and Y. S. Smyrlis. The route to chaos for the Kuramoto-Sivashinsky equation.Theoret. Comput. Fluid Dynamics, 3(1):15–42, 1991
1991
-
[45]
J. C. Robinson.Infinite-dimensional dynamical systems. Cambridge University Press, Cambridge, 2001
2001
-
[46]
Sch¨ olkopf and A
B. Sch¨ olkopf and A. J. Smola.Learning with kernels. MIT Press, Cambridge MA, 2001
2001
-
[47]
G. I. Sivashinsky. Nonlinear analysis of hydrodynamic instability in laminar flames–I. Derivation of basic equations.Acta Astronaut., 4(11-12):1177–1206, 1977
1977
-
[48]
Temam.Infinite-dimensional dynamical systems in mechanics and physics
R. Temam.Infinite-dimensional dynamical systems in mechanics and physics. Springer, New York, 2nd edition, 1997
1997
-
[49]
L. N. Trefethen and D. Bau.Numerical linear algebra. SIAM, Philadelphia, 1997
1997
-
[50]
Udell and A
M. Udell and A. Townsend. Why are big data matrices approximately low rank?SIAM J. Math. Data Sci., 1(1):144–160, 2019
2019
-
[51]
Vales, D
C. Vales, D. C. Freeman, J. Slawinska, and D. Giannakis. Quantum mechanical closure of partial differential equations with symmetries.arXiv:2505.07519, 2025
2025 arXiv
-
[52]
Von Luxburg
U. Von Luxburg. A tutorial on spectral clustering.Stat. Comput., 17(4):395–416, 2007
2007
-
[53]
Williams, M
C. Williams, M. Seeger, and Y. Weiss. Using the Nystr¨ om method to speed up kernel machines. InAdvances in Neural Information Processing Systems, volume 13. MIT Press, 2000
2000
-
[54]
Zelnik-Manor and P
L. Zelnik-Manor and P. Perona. Self-tuning spectral clustering. InAdvances in Neural Information Processing Systems, volume 17, pages 1601–1608. Curran Associates, 2004. 21
2004
Reviewed August 4, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.