{"id":"39b2ff58-d3ae-4d8e-ac16-4d51beefc664","arxiv_id":"2501.11305","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A post-processing diagonalization step turns SpectralNet's rotationally ambiguous output into the actual eigenvectors, yielding scalable, generalizable spectral embeddings and a generalizable UMAP.","lead":"Sep-SpectralNet is a neural-network method that computes spectral embeddings on large datasets and can embed new data points instantly. It also enables a generalizable version of UMAP, called NUMAP, that preserves global structure better than prior parametric UMAP.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Algorithm 2's minibatch-averaged projected Laplacian has no consistency guarantee; if the estimate is biased, the recovered rotation Q is wrong and the central eigenvector-separation claim collapses.","rationale":"The reader's verdict identified the minibatch averaging heuristic as the weakest assumption. I agree; it is the single most load-bearing issue because the entire method reduces to the correctness of this estimate. If the minibatch average does not converge to the full projected Laplacian, the recovered Q is not the true rotation, and the claimed simultaneous scalability, generalizability, and eigenvector separation is not achieved. The paper provides no error bound or sensitivity analysis, and the algorithmic step is not justified by the theory (Lemma 1 only addresses the full-data case). The proposed test would directly measure whether the estimator is consistent and unbiased. I also note a proof gap in Lemma 1, but the lemma's statement is standard and true, so the substantive risk remains the minibatch estimation. The empirical results on small datasets are encouraging but do not establish convergence. Therefore the verdict remains CONDITIONAL, pending the proposed check.","tokens_in":20934,"tokens_out":7587,"duration_ms":72110,"concrete_test":"On a small dataset (e.g., the moon dataset or a 5k-sample CIFAR10 CLIP subset), train SpectralNet to convergence. Compute the full projected Laplacian M = Y^T L Y using the full dataset graph. Then compute the Algorithm 2 estimate M_T for T = 1, 2, 5, 10, 20, 50 with fresh random minibatches each time, and with two batch sizes (e.g., 256 and 2048). Plot ||M_T - M||_F and the sin^2 eigenvector error (between the recovered Q columns and the true eigenvectors) against T. If the error has a nonzero floor or does not decrease with T, the minibatch average is biased/inconsistent and the method's separation guarantee fails. Also compare M_T's eigenvectors against those of M to isolate the effect of the minibatch approximation.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that diagonalizing the minibatch-averaged projected Laplacian \\tilde{\\Lambda} = (1/T)\\sum_i Y_i^T L_i Y_i (Algorithm 2, line 6) recovers the rotation Q relating SpectralNet's output to the true eigenvectors. This assumes the minibatch average approximates the full-data matrix Y^T L Y, where L is the Laplacian of the full dataset graph. However, each L_i is the Laplacian of the kNN graph constructed on the minibatch itself (App. C.2), not the restriction of the full Laplacian to that minibatch. The induced subgraph of a random minibatch in the full kNN graph differs systematically from the kNN graph of the minibatch alone, so E[Y_i^T L_i Y_i] does not equal a constant times Y^T L Y, and no law of large numbers applies. Without a bias bound, the eigenvectors of \\tilde{\\Lambda} cannot be guaranteed to approximate Q as T grows; the recovered Q could be an arbitrary rotation, invalidating the separation step and every downstream application (Fiedler vector, Diffusion Maps, NUMAP).\n\nA separate but related gap: the proof of Lemma 1 (App. A) asserts \\lambda_i \\leq d_i for the diagonal entries of A^T L A. This is false: for L = diag(0,1) and A = [(v1+v2)/\\sqrt{2}, (v1-v2)/\\sqrt{2}], A^T L A = 0.5 I, whose diagonal entries 0.5 are each greater than \\lambda_1=0 but less than \\lambda_2=1. The proof therefore does not establish A^T L A = \\Lambda; the lemma is true by standard minimax arguments, but the provided justification is invalid, weakening the theoretical grounding of the method.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Sep-SpectralNet, an extension of SpectralNet that adds a post-processing step intended to recover the individual eigenvectors of a graph Laplacian from the rotation-ambiguous output of SpectralNet. The step diagonalizes a minibatch-averaged projected Laplacian, and the authors claim this yields both the eigenvalues and the rotation needed to separate eigenvectors while preserving SpectralNet's scalability and out-of-sample generalization. The paper also introduces NUMAP, a generalizable UMAP variant that uses the separated spectral embedding as an initialization for a second network trained with the UMAP loss, and proposes the Grassmann Score (GS) as a global-structure evaluation metric. Experiments on four real-world datasets report substantially lower sin^2 eigenvector errors for Sep-SpectralNet than for SpectralNet, and NUMAP is reported to improve GS over Parametric UMAP while retaining comparable kNN accuracy.","tokens_in":21320,"tokens_out":7130,"duration_ms":78004,"significance":"If the central claim holds, Sep-SpectralNet would be the first spectral embedding implementation to combine scalability, out-of-sample generalization, and eigenvector separation, and NUMAP would address a known limitation of Parametric UMAP by restoring the spectral initialization that is critical for global structure preservation. The empirical design has real strengths: ground-truth eigenvectors are computed externally on combined train+test graphs, comparisons are run over multiple seeds and datasets, and the improvements in eigenvector separation are large and consistent. The code is publicly available, which supports reproducibility. However, two load-bearing theoretical points need work before the claims can be accepted as stated: the minibatch-averaging step in Algorithm 2 lacks a consistency or bias analysis, and the proof of Lemma 1 in Appendix A is invalid as written. The empirical evidence is suggestive but does not by itself substitute for the missing justification of the core separation step.","major_comments":[{"comment":"The paper states that SpectralNet 'cannot directly be adapted for SE' and attributes this to the rotation and reflection ambiguity of the Rayleigh quotient, supported by Lemma 1. Given that the proof of Lemma 1 is invalid as written and that the practical separation step rests on the unverified minibatch-averaging approximation, the theoretical claim is currently stronger than what the manuscript establishes. Please align the wording of these claims with the actual level of theoretical support, or supply the missing arguments.","section":"Sec. 1 and Sec. 4.1"}],"minor_comments":[{"comment":"The KMNIST v2 entry reads '0.0.044'; this should be '0.044'.","section":"Table 8"},{"comment":"The text says the graph construction is 'similar to the one used by Gomez et al. in UMAP', but the cited reference is McInnes et al. (2018); please correct the attribution or add the intended citation.","section":"App. C.2"},{"comment":"Algorithm 2 computes random-walk Laplacians, while Lemma 1 and the separation derivation are stated for the unnormalized Laplacian; please state explicitly which Laplacian variant the analysis covers, since the eigenvectors differ across variants.","section":"Algorithm 2 and Sec. 4.2"},{"comment":"The sentence 'The eigenvectors matrix of tilde-Lambda is the inverse of the orthogonal matrix Q' would be clearer as 'the transpose', since diagonalizing tilde-Lambda returns Q^T rather than Q^{-1} in the usual convention.","section":"Sec. 4.2"},{"comment":"Eigenvalue approximation results are reported only for Parkinsons Telemonitoring and Appliances Energy Prediction; please report the eigenvalue correlations for CIFAR10 and KMNIST as well, or explain why they are omitted.","section":"Sec. 5.1"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here is my take. The real contribution is Sep-SpectralNet's post-processing step: after SpectralNet has learned the eigenspace up to an orthogonal rotation, diagonalize a small projected Laplacian to recover the rotation and hence the individual eigenvectors. That Rayleigh-Ritz idea is not new—SpecNet2 uses it—but the specific minibatch-averaged estimator and the NUMAP application are new, and the paper shows clearly that the separation works: sin^2 errors drop by an order of magnitude across four datasets, and the method preserves SpectralNet's scalability. The code is public. That is real, reproducible work.\n\nThe soft spots are real but not fatal. Algorithm 2 estimates \\tilde{\\Lambda} by averaging Y_i^T L_i Y_i over minibatches, where L_i is the Laplacian of the kNN graph built on that minibatch. The expectation of this quantity is not obviously a scalar multiple of Y^T L Y; there is no bias bound or consistency argument, and the final eigenvectors of the averaged matrix inherit whatever error that estimate carries. The paper calls this 'simple' without acknowledging the gap. The empirical results are consistent, so I would not call the method invalid, but the theoretical claim is overstated.\n\nThe stress-test note's counterexample to Lemma 1 does not hold up: for L = diag(0,1) and that A, A^T L A = [[.5,-.5],[-.5,.5]], not .5I; its eigenvalues are 0 and 1. The proof is terse and should say the columns of Q are ordered so that D is ascending, but the lemma is standard and fine.\n\nOn NUMAP, the global-structure improvement is significant on CIFAR and Appliances but not on Wine (p=0.118) or Banknote (p=0.056); the text says 'NUMAP better preserves global structure' without that qualification. The new Grassmann Score is a reasonable metric, though using it to evaluate a method designed with it in mind is a mild circularity; the raw visualizations help.\n\nBottom line: this is a useful, honest methods paper with strong empirical evidence for the core separation claim. The theory needs a consistency analysis or at least a frank statement that minibatch averaging is heuristic. If it crossed my desk I would send it to review, asking for those two fixes plus a SpecNet2 comparison. It deserves a serious referee; I would likely cite it.","headline":"A practical post-processing fix for SpectralNet's rotation ambiguity, with strong empirical support; the theory is thinner than the paper lets on.","tokens_in":21851,"tokens_out":4692,"would_cite":true,"duration_ms":47770,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T07","65F15"],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that diagonalizing a minibatch-averaged projected Laplacian recovers the rotation that separates a SpectralNet-trained eigenspace into the true graph-Laplacian eigenvectors, yielding the first spectral-embedding…","keywords":["spectral embedding","eigenvector separation","out-of-sample extension","scalable spectral methods","UMAP","graph Laplacian","Rayleigh quotient","deep learning"],"falsifier":"On a dataset with known ground-truth eigenvectors (for example, a two-moons or two-spheres set), compute the minibatch-averaged projected Laplacian Φ̃ for T = 1, 4, 16, and 64, then compare the recovered eigenvectors to the exact eigenvectors of the full Laplacian. If the $sin^{2}$ Grassmann distance between the recovered and true eigenvectors does not decrease toward zero as T grows, the claim that this average recovers the rotation Q is refuted.","tokens_in":20703,"feed_emoji":"📐","tokens_out":7901,"duration_ms":69501,"temperature":0.7,"pith_summary":"Spectral embedding projects data into the leading eigenspace of a graph Laplacian, but existing implementations must trade off three properties: handling new points, scaling to large datasets, and returning individual eigenvectors rather than a rotated subspace. This paper claims to break that trade-off with Sep-SpectralNet, which keeps SpectralNet's scalable and generalizable neural-network training and adds a cheap post-processing step that separates the learned eigenspace into true eigenvectors. The separation step diagonalizes a small averaged matrix to recover the rotation the network implicitly applied, and the paper shows that this separated embedding can serve as the spectral initialization for a generalizable UMAP variant called NUMAP. If correct, spectral-embedding tools such as Fiedler vectors, diffusion maps, and global-structure-preserving visualization become usable on new data points at large scale.","feed_headline":"A small diagonalization step makes spectral embeddings generalize to new data","feed_subtitle":"It separates eigenvectors, scales to large data, and powers a generalizable UMAP that keeps global structure.","key_machinery":"The central object is the minibatch-averaged projected Laplacian Φ̃ = (1/T) Σ_i Y_i^T L_i Y_i, a small (k+1)×(k+1) matrix. If the trained network output satisfies Y = VQ, then each projected Laplacian equals Q^T Λ Q, so by uniqueness of eigendecomposition the eigenvectors of Φ̃ are the columns of Q^T and its eigenvalues are those of the Laplacian. The supporting identity is Lemma 1, which proves that every minimizer of the Rayleigh quotient under orthogonality is of the form VQ, so the recovered Q fully undoes the network's rotation. This diagonalization post-processing is what turns SpectralNet's eigenspace approximation into a true spectral embedding, and it is also what makes NUMAP's residual-connected UMAP network possible: only separated eigenvectors give a meaningful first-ell-dimensional spectral coordinate for the residual connection to preserve.","core_discovery":"The paper's central claim is that the rotation-and-reflection ambiguity inherent in Rayleigh-quotient minimization—any minimizer of Tr(A^T L A) under orthogonality has the form V Q, where V is the true eigenvector matrix of the Laplacian L—can be resolved in a one-shot post-processing step. For a learned map F_θ whose output lies in the span of the first k+1 eigenfunctions, the unknown rotation Q is encoded in the projected Laplacian Φ = (VQ)^T L (VQ) = Q^T Λ Q; because Q^T is the eigenvector matrix of this small matrix, diagonalizing an estimate of it yields both the Laplacian eigenvalues and the inverse rotation needed to recover the individual eigenvectors from the network output. The paper argues that the estimate can be formed by averaging over a few random minibatches, making the separation step nearly free and preserving SpectralNet's approximately linear scaling. The result is presented as the first spectral-embedding implementation that is simultaneously generalizable, scalable, and eigenvector-separated, and as the basis for NUMAP, a generalizable UMAP that keeps the spectral initialization responsible for UMAP's global-structure preservation.","pith_inferences":["The same diagonalization trick could be applied to any neural network that minimizes a Rayleigh-quotient loss under an orthogonality constraint, not only SpectralNet; the paper does not explore this generalization.","Because the minibatch average is a stochastic estimator of Q^T Λ Q, increasing the number of minibatches T or using importance-sampled batches should improve eigenvector accuracy; the paper does not quantify this trade-off.","Grassmann Score could plausibly be used as a training objective or early-stopping criterion for generalizable embedding methods, though the paper presents it only as an evaluation metric.","If a theoretical convergence bound for the minibatch average is established, this post-processing approach could replace full eigendecomposition in non-neural spectral settings as well, whenever scalability is the bottleneck."],"forward_implications":["Sep-SpectralNet can approximate the Fiedler vector and Fiedler value for out-of-sample points in near-linear time, making algebraic connectivity analysis scalable and generalizable.","Diffusion Maps become generalizable: since Sep-SpectralNet approximates both the Laplacian eigenvectors and eigenvalues, the diffusion coordinates (1-λ_i)^t v_i can be computed for new points by a single feed-forward pass.","NUMAP, a spectral-initialized parametric UMAP, preserves global structure better than Parametric UMAP while maintaining comparable local-structure quality and enabling fast inference; the paper's ablation shows that removing eigenvector separation makes this global preservation inconsistent.","The separation step adds negligible overhead, so the method scales to datasets that are too large for full eigendecomposition while producing test-set embeddings by one forward pass.","The proposed Grassmann Score, comparing the leading Laplacian eigenspaces of the input and embedding graphs, distinguishes global-structure-preserving embeddings that kNN and Silhouette scores cannot tell apart."],"supporting_citations":[{"why":"Provides SpectralNet, the scalable and generalizable neural-network eigenspace learner that Sep-SpectralNet extends.","marker":"Shaham et al., 2018"},{"why":"Supplies the Rayleigh-quotient minimization characterization used in Lemma 1 to prove that all minimizers are of the form VQ.","marker":"Li, 2015"},{"why":"Defines UMAP and the graph-construction scheme that Sep-SpectralNet and NUMAP use for affinity matrices and the Grassmann Score.","marker":"McInnes et al., 2018"},{"why":"Establishes that spectral initialization is what gives UMAP its global-structure preservation, motivating NUMAP's design.","marker":"Kobak & Linderman, 2021"},{"why":"Defines Parametric UMAP, the generalizable baseline that omits spectral initialization and which NUMAP aims to surpass.","marker":"Sainburg et al., 2021"},{"why":"Provides the standard characterization of Laplacian eigenvalues and eigenvectors used to define spectral embedding.","marker":"Von Luxburg, 2007"},{"why":"Defines Diffusion Maps, an application requiring separate eigenvectors and eigenvalues, which Sep-SpectralNet can generalize.","marker":"Coifman & Lafon, 2006b"}],"fun_headline_variants":["One tiny diagonalization makes spectral embeddings generalize","Separating eigenvectors yields generalizable, scalable spectral embeddings","Tiny diagonalization step unlocks generalizable spectral embeddings","Sep-SpectralNet: generalizable spectral embeddings in one step","From spectral embeddings to generalizable UMAP via eigenvector fix"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that averaging a small number of random minibatches approximates the full projected Laplacian well enough to recover the true rotation and eigenvectors, and the paper gives no formal guarantee for how many minibatches are needed.","fun_headline_variants_meta":{"raw":{"variants":["One tiny diagonalization makes spectral embeddings generalize","Separating eigenvectors yields generalizable, scalable spectral embeddings","Tiny diagonalization step unlocks generalizable spectral embeddings","Sep-SpectralNet: generalizable spectral embeddings in one step","From spectral embeddings to generalizable UMAP via eigenvector fix"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000621,"raw_usage":{"total_tokens":2885,"prompt_tokens":960,"completion_tokens":1925,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":576,"completion_tokens_details":{"reasoning_tokens":1847}},"tokens_in":576,"tokens_out":1925,"duration_ms":16282,"temperature":1.0,"reasoning_tokens":1847,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T18:25:03.996367+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On a dataset with known ground-truth eigenvectors (for example, a two-moons or two-spheres set), compute the minibatch-averaged projected Laplacian Φ̃ for T = 1, 4, 16, and 64, then compare the recovered eigenvectors to the exact eigenvectors of the full Laplacian. If the $sin^{2}$ Grassmann distance between the recovered and true eigenvectors does not decrease toward zero as T grows, the claim that this average recovers the rotation Q is refuted.","supporting_citations":[],"review_version":1}