{"id":"f324eaf1-25a3-4d38-874b-87b245cad59b","arxiv_id":"2507.14257","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"LDM is a linear approximation of diffusion maps that behaves like a weighted PCA and outperforms PCA on some manifold-structured high-dimensional data.","lead":"This paper introduces the Linearized Diffusion Map (LDM), a linear dimensionality reduction method formed by truncating the diffusion-map kernel to its first-order Taylor expansion. The authors claim LDM captures geometric structure that PCA misses, especially for high-dimensional manifold data, and supports this with experiments on synthetic and real datasets.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"LDM's flagship hypersphere evidence may be vacuous: for unit-norm data the linearized kernel is affine, so with centered data or constant row sums LDM reduces exactly to PCA; the reported high-dimensional advantage is plausibly finite-sample row-sum noise.","rationale":"The reader's weakest assumption targets the Taylor expansion; my concern is more specific and more damaging for the strongest claim. Even granting the expansion, the resulting affine kernel on the paper's own 'ideal testing ground' (uniform hypersphere) reduces to PCA once finite-sample mean fluctuations are removed. Thus the headline experimental evidence does not establish a geometric advantage; it may establish only a finite-sample sensitivity to the direction of the empirical mean, which is not 'explicit manifold structure.' This is a testable alternative explanation: centering the data or increasing N should eliminate the difference. If the test confirms the equivalence, the central claim fails; if the difference persists after centering, the paper would need a mechanistic explanation and error analysis for the truncation. I therefore keep the reader's conditional verdict, but the conditions must include this specific check. Secondary issues, such as the conclusion's unsupported 'complete positivity' claim (the linearized kernel is not PSD in general), reinforce the need for revision but are not the main load-bearing point.","tokens_in":7163,"tokens_out":16476,"duration_ms":200043,"concrete_test":"Re-run the section 4.3 experiment after exactly centering the sphere samples (subtract the empirical mean before building both PCA and LDM), and separately with N large enough at fixed D that the row sums k_i are constant to numerical precision. If the PCA-minus-LDM recall@10 differences in Fig. 3 collapse to zero or become statistically indistinguishable under either modification, the high-dimensional advantage is a finite-sample artifact of the uncentered row-sum normalization. Also report the coefficient of variation of the row sums across gamma; the artifact hypothesis predicts LDM's advantage appears only when this variation is non-negligible.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 2.2 sets eps = 4 * R^2_max. On the section 4.3 hypersphere, every point has R_i^2 = 1, so eps = 4 and the linearized kernel is exactly k_ij = 1 - ||x_i - x_j||^2 / 4 = 1/2 + (1/2) x_i dot x_j, an affine function of the Gram matrix. When the empirical mean is zero, the row sums k_i are constant, so the normalization factors N_i in Eqs. (10)-(11) are constant and the LDM operator (12)-(13) is proportional to 11^T + XX^T. The nonconstant eigenvectors of this matrix are exactly the PCA eigenvectors; hence, in the large-N or exactly-centered limit, LDM and PCA give identical embeddings. The recall advantage in Fig. 3 can then only come from finite-N fluctuations of the empirical mean entering the row sums, not from diffusion geometry. This is precisely the small-gamma = N/D regime the abstract highlights, so the claimed 'explicit manifold structure' advantage is plausibly a sampling artifact. No error bound is supplied for the truncation in section 2.2; at x = ||x_i - x_j||^2 / eps = 1, the true kernel is exp(-1) ~ 0.368 while the linearization is 0, so the 'successful linearization' is not a quantitatively accurate kernel approximation either.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces Linearized Diffusion Map (LDM), a linear dimensionality reduction method obtained by first-order Taylor expansion of the Gaussian RBF kernel with scale ε = 4R²_max, followed by diffusion-map-style row normalization. The authors derive that linearized kPCA reduces to PCA, then define symmetric and asymmetric LDM operators. Experiments compare LDM with PCA on the Swiss roll, MNIST, COIL-20, and random hyperspheres, reporting that LDM outperforms PCA in high-dimensional regimes with explicit manifold structure, while PCA is better on noisy or less structured data. The paper also claims that the linearized kernel is completely positive, making it amenable to NMF.","tokens_in":7496,"tokens_out":7199,"duration_ms":86762,"significance":"If the central claims were correct, LDM would be a computationally efficient linear embedding method that retains some diffusion-geometry benefits, with O(ND) cost and simple interpretability. The derivation in Section 2.3 that linearized kPCA equals PCA is correct, and the linear-operator formulation is a clean way to present both PCA and the proposed variants. However, the paper provides no convergence or approximation theorem relating LDM to diffusion maps, and the flagship experimental evidence on hyperspheres is theoretically fragile. The complete positivity claim is unproved and likely false. As a result, the significance of the contribution is currently not established.","major_comments":[{"comment":"On the unit hypersphere dataset, every point has R_i^2 = 1, so ε = 4R²_max = 4 and the linearized kernel becomes k_ij = 1 - ||x_i - x_j||²/4 = 1/2 + (1/2) x_i·x_j, an affine function of the Gram matrix. When the empirical mean is zero, the row sums k_i are constant, so the normalization factors N_i in Eqs. (10)-(11) are constant and the nonconstant eigenvectors of the LDM operators (12)-(13) are exactly the PCA eigenvectors. Consequently, in the centered or large-N limit, LDM and PCA produce identical embeddings. The recall advantage attributed to LDM in Fig. 3 at small γ = N/D can then only arise from finite-sample fluctuations of the empirical mean entering the row sums, not from diffusion geometry. The authors should derive the spectrum of the LDM operator for this case, re-run the hypersphere experiment with explicitly centered data and with data forced to have zero empirical mean, and report error bars; I expect the claimed advantage to disappear. This is load-bearing for the abstract's claim that LDM outperforms PCA on explicit manifold structures.","section":"Section 4.3"},{"comment":"The paper asserts that ε = 4R²_max gives a 'successful linearization' of the RBF kernel, but no error analysis is provided. At the maximum normalized distance D²_ij/ε = 1, the true kernel value is e^{-1} ≈ 0.368 while the linearization gives 0, so the approximation is not quantitatively accurate at the scale used. More importantly, the paper never proves that the degree-normalized linearized operator approximates the diffusion-map operator or preserves diffusion distances. The LDM operators in Eqs. (12)-(13) are defined by analogy with diffusion maps, but no spectral perturbation bound or consistency result is stated. The geometric interpretation of LDM embeddings is therefore an assertion, not a demonstrated property.","section":"Section 3.2"},{"comment":"The statement that 'the linearized kernel is completely positive' is unproved and, as stated, unjustified. Entrywise nonnegativity of k_ij follows from ε = 4R²_max, but complete positivity requires the existence of a nonnegative matrix V such that K = V V^T, which is a much stronger condition. No such factorization is exhibited, and no argument is given. Since the abstract and conclusion use this claim to motivate NMF applicability, this is a load-bearing assertion that needs either a proof or a clear statement of the precise positivity property that actually holds.","section":"Conclusion"},{"comment":"The experimental evaluation lacks the statistical detail needed to support the comparative claims. Figures 2 and 3 plot recall differences without error bars, standard deviations, or numbers of repeated trials. It is also unclear whether the data were centered before applying PCA and LDM, how ε = 4R²_max was computed for MNIST and COIL-20, how many eigenvectors were retained, and whether the reported differences are stable across random subsamples. Given that the hypersphere experiment is the main quantitative evidence for the paper's central claim, the absence of such details is a serious omission.","section":"Section 4.2"}],"minor_comments":[{"comment":"The notation RiX with mixed-case indices is unusual and is frequently garbled in the text; for example, 'matrix RiX ∈ RN ×D, where indices i = 1, . . . , Nlabel the samples' is missing spaces and the word 'label' is run together with 'N'. The notation should be cleaned up and defined more carefully.","section":"Section 1.0.1"},{"comment":"The sentence 'Therefore, the Bessel correction used in PCA becomes ε = 2(N − 1), i.e. kPCA’s ε absorbs this when N is large' is confusing: the Bessel correction is a factor of 1/(N−1), not a value of ε. Clarify that setting ε = 2(N−1) makes the linearized kPCA matrix equal to the PCA covariance matrix up to the usual normalization.","section":"Section 2.3"},{"comment":"The statement 'this is proportional to PCA and LkPCA with ε = 2' uses the undefined abbreviation LkPCA and an unexplained value of ε. Please define the notation and specify the proportionality constant.","section":"Section 2.4"},{"comment":"There are several typos and formatting issues, e.g., 'This algorithms', 'Johnsonet al.' in the references, and 'V . d.' in the Tenenbaum reference. A careful proofreading pass is needed.","section":"Throughout"}],"recommendation":"reject","confidential_remarks":"The paper's flagship hypersphere result is theoretically fragile: on unit-norm data the linearized kernel is affine in the Gram matrix, so LDM reduces to PCA in the centered limit. The claimed high-dimensional advantage is likely a finite-sample row-sum artifact. Combined with the unproved complete-positivity claim and the absence of any theoretical link to diffusion distances, the paper's central claims are not supportable. A substantially revised manuscript with new theory and controlled experiments could be reconsidered, but the current submission does not meet the bar for publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper's core derivation is correct: linearizing the RBF kernel and centering gives exactly PCA, and the linear-operator formulation is clean and O(ND). The LDM idea of applying a degree normalization to the linearized kernel is a genuine twist, but it is best understood as a weighted PCA variant, not a new geometry. The Swiss roll and real-data figures do show LDM behaves differently from PCA, which is mildly interesting.\n\nThe soft spots are substantial, and the stress-test note lands. On the hypersphere, every point has unit norm, so ε=4 and the linearized kernel is 1/2 + (1/2)x_i·x_j — an affine function of the Gram matrix. If the data are centered, the row sums are constant and LDM reduces exactly to PCA. The reported high-dimensional advantage appears only in the small-N/D regime where the empirical mean fluctuates and the row-sum normalization injects a sample-dependent weighting. That is not diffusion geometry; it is finite-sample noise. The paper gives no error analysis, no error bars, and no eigenvector analysis for the degree-normalized operator, so the central claim is unsupported.\n\nThe NMF claim is also shaky: entrywise nonnegativity is not complete positivity, and the linearized kernel can be indefinite, so 'completely positive' needs proof. The diffusion-distance interpretation is asserted but never shown to survive the truncation; at the edge of the approximation, exp(-1) ≈ 0.37 while the linearization is 0, so the kernel is not even quantitatively accurate.\n\nAll that said, the paper is clearly written and the PCA equivalence in Section 2.3 is a nice pedagogical checkpoint. The method is not a breakthrough, but it is a legitimate question—whether a degree-normalized linearized kernel can ever beat PCA—and a serious referee could push the author to pin down when, if ever, that happens. I would not cite it as a reliable method until the hypersphere claim is redone with centered data and proper baselines (including weighted PCA), but I would send it to review rather than desk-reject, because the flaw is subtle and the fix (honest reporting of when LDM ≠ PCA) is achievable.","headline":"The linearization is sound, but the hypersphere advantage over PCA looks like a centering artifact rather than diffusion geometry.","tokens_in":7968,"tokens_out":2648,"would_cite":false,"duration_ms":34320,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Linearizing the diffusion-map kernel yields a linear dimensionality reduction method, LDM, that runs at PCA speed and preserves manifold geometry better than PCA when the data has explicit curved or spherical structure.","keywords":["dimensionality reduction","diffusion maps","linear embeddings","kernel linearization","PCA comparison","manifold learning","approximate nearest neighbors","non-negative matrix factorization"],"falsifier":"On a noisy high-dimensional hypersphere, compute the exact diffusion-map embedding and compare its pairwise distances with LDM's embedded distances; if LDM distances correlate with PCA distances more than with diffusion distances in the regime where LDM claims superiority, or if the second-order term $\\frac{1}{2}(D_{ij}^2/\\varepsilon)^2$ is non-negligible relative to the retained first-order term for the chosen $\\varepsilon$, the central assumption is refuted.","tokens_in":6961,"feed_emoji":"🌀","tokens_out":9151,"duration_ms":92746,"temperature":0.7,"pith_summary":"This paper introduces the Linearized Diffusion Map (LDM), a linear dimensionality reduction method built by replacing the Gaussian RBF kernel's exponential with its first-order Taylor expansion and then applying the diffusion-map degree normalization inside the resulting linear operator. The paper aims to show that diffusion geometry can survive this linearization, giving an embedding that runs at PCA's $\\mathcal{O}(ND)$ cost while preserving more manifold structure. In experiments on Swiss rolls and high-dimensional hyperspheres, LDM embeddings yield better recall@10 against exact nearest neighbors than PCA does, especially when the ambient dimension is large relative to sample size. On MNIST and COIL-20, PCA retains the advantage, so the paper frames LDM and PCA as complementary rather than competitors. The paper also claims that the LDM kernel's complete positivity makes it directly usable for non-negative matrix factorization.","feed_headline":"Linearized diffusion maps beat PCA on curved high-dimensional data","feed_subtitle":"Same linear-time cost as PCA, yet the embedding preserves explicit manifold geometry better in high dimensions.","key_machinery":"The carrying object is the linearized diffusion kernel operator of equation (12), $L^{\\mathrm{LDM}}_i(v_j) = N_i (1_i - R^2_i/\\varepsilon) N_j v_j - N_i R^2_j (N_j v_j)/\\varepsilon + (2/\\varepsilon) N_i R_{iX} (R^\\dagger_{Xj} (N_j v_j))$, with $N_i$ the inverse square root of the kernel row sums. This is the Gaussian RBF kernel expanded to first order and then symmetrically degree-normalized, so it remains a linear map on vectors even though it encodes the diffusion-map normalization. Repeated application of this operator feeds the Lanczos algorithm, producing a spectral embedding in $\\mathcal{O}(ND)$ time. The asymmetric variant, LDM-A, uses row-only normalization; both variants share the parameter set $\\{N_i, 1_i, R^2_i, R_{iX}, \\varepsilon\\}$.","core_discovery":"The central claim is that a first-order Taylor expansion of the RBF kernel, $\\exp(-D^2_{ij}/\\varepsilon) \\approx 1 - D^2_{ij}/\\varepsilon$ with $\\varepsilon = 4R^2_{\\max}$, can be inserted into the diffusion-map construction without destroying the geometric information that diffusion maps are known to carry. After row-sum degree normalization, the linearized kernel becomes a genuine linear operator, so its top eigenvectors can be computed by repeated application with the Lanczos algorithm, at the same $\\mathcal{O}(ND)$ complexity as PCA. The paper reports that LDM embeddings differ perceptibly from PCA embeddings: on the Swiss roll LDM partially unrolls the sheet whereas PCA produces a spiral, and on uniformly sampled hyperspheres LDM's recall@10 exceeds PCA's when the ratio $\\gamma = N/D$ is small. It also reports that on MNIST and COIL-20, PCA stays superior for approximate-nearest-neighbor retrieval; the stated conclusion is complementarity, not dominance. A further claim is that because the linearized kernel matrix is completely positive (all entries nonnegative), the kernel can be fed directly into non-negative matrix factorization for interpretable latent-factor discovery.","pith_inferences":["A testable extension the paper leaves implicit is to rewrite the LDM operator as a weighted covariance eigenproblem and compare its loadings to PCA's on the same data, which would clarify whether the gain comes from the degree normalization or from the kernel linearization itself.","Because the approximation discards terms beyond first order, the paper's comparison suggests LDM should interpolate between PCA and full diffusion maps; measuring the Taylor remainder on real datasets would show where in that spectrum LDM lands.","The NMF route is stated as a suggestion; a concrete validation would be to apply NMF to the LDM kernel on a parts-based synthetic dataset and measure recovery of the known generative parts."],"forward_implications":["For data lying on an explicit manifold in high ambient dimension—such as a hypersphere with small $\\gamma = N/D$—LDM is the better linear method for approximate-nearest-neighbor retrieval, so practitioners in that regime can replace PCA with LDM at no extra cost.","Because LDM's kernel matrix is completely positive, the kernel can be passed directly to NMF, enabling diffusion-geometry-aware, interpretable factorizations.","On noisy or variance-dominated datasets such as MNIST and COIL-20, PCA remains preferable, so LDM should be treated as an additional embedding in a portfolio rather than a universal replacement.","A natural two-stage pipeline follows: use PCA to quickly narrow candidate neighborhoods, then use LDM to refine those neighborhoods, exploiting the reported complementarity of the two embeddings.","Since both methods are linear and cost $\\mathcal{O}(ND)$, adding LDM to an existing PCA-based ANN pipeline does not change the asymptotic complexity."],"supporting_citations":[{"why":"Defines the diffusion map, diffusion distances, and the symmetric/asymmetric normalizations that LDM linearizes.","marker":"Coifman and Lafon (2006)"},{"why":"Introduces PCA, the baseline whose variance-maximizing embedding LDM is compared against throughout.","marker":"Pearson (1901)"},{"why":"Establishes the Swiss roll as a benchmark and the manifold-unrolling goal LDM partially achieves.","marker":"Tenenbaum et al. (2000)"},{"why":"Reviews linear dimensionality reduction and motivates the gap that a linearized diffusion-map method fills.","marker":"Cunningham and Ghahramani (2015)"},{"why":"Introduces classical MDS, which the paper shows is proportional to PCA and LkPCA after centering.","marker":"Torgerson (1952)"},{"why":"Motivates large-scale approximate nearest-neighbor retrieval, the evaluation task used to compare PCA and LDM.","marker":"Johnsonet al. (2017)"},{"why":"Frames dimensionality reduction as a core tool for visualizing and interpreting high-dimensional data.","marker":"Hinton and Salakhutdinov (2006)"}],"fun_headline_variants":["Linearized diffusion maps outperform PCA on curved high-D data","Linear-time geometric embedding outperforms PCA on manifolds","Diffusion map linearization yields geometry that rivals PCA","LDM provides a linear embedding that excels on high-dimensional manifolds","Linearized diffusion maps offer a geometric edge over PCA in high-D"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the first-order Taylor approximation $\\exp(-d^2/\\varepsilon) \\approx 1 - d^2/\\varepsilon$ with $\\varepsilon = 4R^2_{\\max}$ is accurate enough that the degree-normalized linear operator still reproduces diffusion-map geometry; the paper uses this approximation without proving that diffusion distances survive linearization or quantifying the Taylor remainder.","fun_headline_variants_meta":{"raw":{"variants":["Linearized diffusion maps outperform PCA on curved high-D data","Linear-time geometric embedding outperforms PCA on manifolds","Diffusion map linearization yields geometry that rivals PCA","LDM provides a linear embedding that excels on high-dimensional manifolds","Linearized diffusion maps offer a geometric edge over PCA in high-D"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001546,"raw_usage":{"total_tokens":6181,"prompt_tokens":941,"completion_tokens":5240,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":557,"completion_tokens_details":{"reasoning_tokens":5157}},"tokens_in":557,"tokens_out":5240,"duration_ms":43789,"temperature":1.0,"reasoning_tokens":5157,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T16:14:36.806509+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On a noisy high-dimensional hypersphere, compute the exact diffusion-map embedding and compare its pairwise distances with LDM's embedded distances; if LDM distances correlate with PCA distances more than with diffusion distances in the regime where LDM claims superiority, or if the second-order term $\\frac{1}{2}(D_{ij}^2/\\varepsilon)^2$ is non-negligible relative to the retained first-order term for the chosen $\\varepsilon$, the central assumption is refuted.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the diffusion map, diffusion distances, and the symmetric/asymmetric normalizations that LDM linearizes."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces PCA, the baseline whose variance-maximizing embedding LDM is compared against throughout."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Reviews linear dimensionality reduction and motivates the gap that a linearized diffusion-map method fills."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces classical MDS, which the paper shows is proportional to PCA and LkPCA after centering."}],"review_version":1}