{"id":"ab96798c-e770-492d-b500-2eaa1514b8ec","arxiv_id":"2506.22228","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"NESS uses stability across random initializations of neighbor embeddings to improve and assess smooth single-cell representations.","lead":"Researchers built NESS, a tool that reruns t-SNE or UMAP many times with different random starts to measure how stable each cell's neighbors are, then uses that stability to pick better settings and find cells transitioning between states. If it works broadly, it could make single-cell maps more trustworthy for studying development and disease.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Biological interpretation of NESS local stability hinges on untested continuity assumption linking low density to fast transitions.","rationale":"The reader's weakest assumption was the continuity assumption connecting low-density regions to fast state transitions, and that is indeed the most load-bearing point in the paper. The entire translational value of NESS as a biological transition-state identifier depends on this assumption, because the method itself only quantifies algorithmic stability across random initializations. The empirical stability–density link is about embedding geometry, not biology. The validations cited (MuTrans, scVelo) are themselves model-based and partly derived from the same t-SNE embeddings, so they provide only correlational support that can be confounded by density. A density-matching analysis provides a concrete way to test whether stability has independent biological content. I agree with the reader's conditional verdict: the method is promising as a stability and hyperparameter-selection tool, but the biological interpretation should be presented as conditional on the continuity assumption until directly tested.","tokens_in":36388,"tokens_out":4291,"duration_ms":49293,"concrete_test":"Re-analyze the iPSC and Murine Intestinal datasets: for each cell in the bottom 2% of NESS local stability, match it to a high-stability cell with similar local density (e.g., same decile of mean k-NN distance in PCA space). Compare MuTrans transition entropy between the matched low-stability and high-stability cells. If the entropy difference is no longer significant after density matching, low NESS stability is just a proxy for low density and the biological claim reduces to the untested continuity assumption. If the difference remains significant, stability carries information beyond density, supporting the interpretation.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Although NESS is technically a well-defined stability score, the paper's central biological claim—that NESS local stability identifies transitional vs. stable cell states—requires an extra inferential step. In Results (near Fig 3c,d) the authors state: 'If we assume continuity of the cell state changes, such low-density regions would therefore consist of cells undergoing fast state transitions.' This 'if' is not tested. The empirical association in Fig 3c only shows that unstable cells tend to reside in low-density regions; it says nothing about the rate of state change. Low density can arise from technical dropout, undersampling of a stable but rare population (e.g., quiescent stem cells), or genuine fast transitions. Without a direct test, low NESS stability is indistinguishable from any other low-density indicator. The external validations do not settle this: MuTrans transition entropy is itself model-based and is computed here on t-SNE coordinates (Methods 4.6: reduction_coord='tsne'), so it shares the embedding geometry with NESS; scVelo velocity is known to be sensitive to preprocessing assumptions and, per the paper's own citations (e.g., [26]), can be unreliable. Thus the biological interpretation stands or falls on the continuity assumption.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes NESS, a PCS-guided framework for assessing and improving neighbor embedding (NE) stability in single-cell data. It first provides an empirical, simulation-based, and theoretical evaluation of t-SNE, UMAP, PHATE, and densMAP under varying graph connectivity parameters (GCP), arguing that low GCP causes artificial fragmentation and instability. NESS then uses random initializations to generate multiple embeddings, forms KNN graphs from each, and defines per-cell local stability, global stability, and embedding rareness scores, plus an automated GCP selection workflow. The method is applied to six single-cell datasets spanning hematopoiesis, iPSC differentiation, organoid development, embryoid body formation, neurogenesis, and spermatogenesis. The central claims are that NESS local stability identifies transitional versus stable cell states, that NESS instability scores quantify transcriptional dynamics, and that NESS-assisted embeddings correct artifacts of default NE parameter settings. Validation is performed against MuTrans transition entropy, scVelo RNA velocity, and external cell-state annotations, with comparisons to EMBEDR, DynamicViz, and scDEED.","tokens_in":36699,"tokens_out":3302,"duration_ms":40588,"significance":"If the central claims hold, NESS would be a practically useful and conceptually simple addition to the single-cell visualization toolbox: it is label-free, computationally scalable, implemented in an R package with documented workflows, and it demonstrates on six datasets that default NE hyperparameters can produce artifacts that stability-guided GCP selection mitigates. The paper also contributes a theoretical result linking low graph connectivity to t-SNE distortion, a detailed supplement with proofs, and systematic benchmarking against existing embedding-assessment methods. These are genuine strengths. However, the paper's biological interpretation of local stability as transition-state identity rests on an untested continuity/density assumption, and the main external validations either share the embedding geometry with NESS (MuTrans on t-SNE coordinates) or rely on RNA velocity methods whose reliability the paper itself flags through citation [26]. The theoretical artifact result is conditional on an unverified diameter assumption.","major_comments":[{"comment":"The biological interpretation of NESS local stability as a transition-state indicator depends on the stated assumption that low-density regions contain cells undergoing fast state transitions. This assumption is not tested, and low density can also arise from technical dropout, undersampling of a stable but rare population, or sampling gaps unrelated to dynamics. The association in Fig. 3c only connects low stability to low density, not to transition rate. Because the paper's headline claim is that NESS identifies transitional versus stable cell states, this load-bearing assumption needs direct support, for example via independent perturbation-based transition labels, time-course validation, or a formal sensitivity analysis with synthetic trajectories where cell-state change rates are known.","section":"Results, near Fig. 3c,d; Methods 4.4"},{"comment":"The MuTrans validation is weakened by a shared-geometry concern: MuTrans transition entropy is computed on t-SNE coordinates (Methods 4.6: reduction_coord = 'tsne'), while the NESS local stability scores being validated are also derived from t-SNE embeddings. The observed association in Fig. 4a may therefore reflect common embedding geometry rather than independent evidence of biological transition kinetics. Please report MuTrans transition entropy computed on PCA or diffusion-map coordinates, or provide a quantitative analysis showing that the association is robust to the choice of reduction coordinates.","section":"Methods 4.6 and Results, Fig. 4a"},{"comment":"Theorem 4.1 is explicitly conditional on the assumptions S(phi*) = n^tau for some 0 < tau < 1 and k = O(1). The diameter assumption is used critically in the proof (Supplement B, Proposition B.4 and the final argument) but is not derived or empirically verified for actual t-SNE solutions. Moreover, the theorem is proved for the simplified affinity model in Eq. (4.2) with constant bandwidth and uniform kNN weights, which omits the data-dependent bandwidths of real t-SNE. Without justification that these assumptions are met, the theorem does not establish that actual t-SNE under low perplexity 'inevitably' fragments smooth structures. Please provide empirical support for the diameter scaling, or state the result as a conditional contribution with the scope limitations made explicit.","section":"Methods 4.2, Theorem 4.1 and Supplement B"},{"comment":"The reported improvements in global stability after removing low-stability cells are partly circular, because the GCP is selected using the NESS global stability score itself, and the same score is then used to evaluate the improvement. Figure 3b shows relative improvement in GS, concordance, and neighbor purity, but without a pre-specified holdout procedure or comparison against an independent GCP choice it is unclear how much of the GS improvement is guaranteed by construction. Please report concordance and purity improvements under a fixed, independently chosen GCP, or use a cross-validation scheme that separates GCP selection from evaluation.","section":"Methods 4.3 and Results, Fig. 3b"},{"comment":"The claim that NESS instability score quantifies transcriptional dynamics is supported only by cell-type-level visual comparisons with scVelo total RNA velocity. scVelo is model-based and its reliability is disputed, as the paper itself notes through citation [26]. The paper does not report per-cell correlations between the NESS instability score and RNA velocity, nor does it examine sensitivity of the comparison to scVelo preprocessing choices. Given that this section is one of the paper's two headline biological applications, please provide quantitative per-cell concordance measures and a robustness check, or explicitly soften the claim to 'concordant at the level of cell-type medians under a standard scVelo configuration.'","section":"Results, Fig. 5 and Methods 4.4"}],"minor_comments":[{"comment":"The heading contains a typo: 'sermatogenesis' should be 'spermatogenesis'.","section":"Results, heading near Fig. 5"},{"comment":"The caption contains 'cell tyles'; this should be 'cell types'.","section":"Figure 5 caption"},{"comment":"The sentence 'Thess results demonstrate' contains a typo; it should be 'These results'.","section":"Methods 4.5"},{"comment":"The name 'Murine Instestinal' appears in the paragraph on identifying transitional and stable cell states; it should be 'Murine Intestinal'.","section":"Methods 4.4"},{"comment":"The text lists 'PCH1' among positively correlated genes, but Table S2 lists 'PTCH1'; please make the gene symbol consistent.","section":"Results, iPSC gene section versus Table S2"},{"comment":"The caption reports bottom percentiles 2%, 8%, 15%, and 100%, while Methods 4.4 states p = 2, 10, 20, 100 for the same comparison. Please reconcile these numbers.","section":"Figure 4a caption versus Methods 4.4"},{"comment":"In the pathway enrichment paragraph, 'DA VID' is written with an internal space; it should be 'DAVID'.","section":"Methods 4.4"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is well within scope for a machine-learning/statistical-methods journal and the software release is a strength. The main risk is overclaiming biological interpretation: the continuity/density assumption and the shared-geometry validation with MuTrans are load-bearing, and the theoretical artifact result depends on an unverified diameter assumption. These are fixable with additional analyses or softened claims, which is why I recommend major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The short version: NESS is a genuinely useful method for tuning t-SNE/UMAP parameters and flagging unreliable regions of an embedding, and the paper deserves referee time. The main thing to watch is that the biological reading of the stability score—low stability equals transitional cell state—is an inference step that the paper asserts but does not directly test.\n\nWhat is actually new: the core idea is to run an NE algorithm multiple times from random initializations and compare KNN graphs. The resulting per-cell stability score is clean and intuitive, and the global stability score gives a principled way to pick graph connectivity parameters, which is currently done by default rules of thumb. The theoretical result on t-SNE fragmentation, while proven under a simplified affinity model (constant sigma, uniform k-neighbor weights) and an assumed embedding diameter, is a real contribution—it formalizes the folklore that low perplexity fragments manifolds. The empirical work is thorough: six datasets, external labels, comparisons to EMBEDR, DynamicViz, scDEED, MuTrans, and scVelo, with code and data available. That is solid, reproducible work.\n\nNow the soft spots. The biggest is the continuity assumption. The paper shows that low-stability cells tend to live in low-density regions, and then adds 'If we assume continuity of the cell state changes, such low-density regions would therefore consist of cells undergoing fast state transitions.' That 'if' is untested. Low density could equally come from dropout, undersampling of a rare but stable population, or technical noise. The MuTrans validation does not fully settle this, because MuTrans is run on the same t-SNE coordinates as NESS, so the shared geometry could drive the correlation. The scVelo comparison is suggestive but scVelo is sensitive to preprocessing and not an independent gold standard. The Embryoid Body annotations are the strongest external evidence—transitional states do show lower stability—but that is qualitative and limited to one system. I would like to see a direct test of the assumption, for example using a dataset with time-course experimental labels or an orthogonal trajectory reconstruction, plus a quantitative concordance measure instead of visual comparison.\n\nTwo smaller points: the GCP selection workflow is heuristic (stop when GS > 0.9 or improvement < 5%), and the grid is coarse. The theory also relies on an embedding diameter assumption that may not hold in all settings. These are minor relative to the core contribution.\n\nWho this is for: anyone working on single-cell visualization or trajectory analysis, and methodologists interested in stability-based diagnostics for dimensionality reduction. It is a methods paper with a broad audience. The biological claims are currently a step ahead of the evidence, but the methodology itself is sound and well documented. I would send it to peer review and ask for a revision that either tests the continuity assumption or explicitly reframes the stability score as an uncertainty measure rather than a transition-state detector.","headline":"A useful stability-guided tuning method for neighbor embeddings on single-cell trajectories, with a novel t-SNE fragmentation theorem; the biological transition-state interpretation is plausible but rests on an untested continuity assumption.","tokens_in":37161,"tokens_out":3013,"would_cite":true,"duration_ms":34263,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that the instability of t-SNE and UMAP across random restarts can be converted into a per-cell score that separates truly transitioning cells from stable ones, giving a label-free window into developmental dynamics.","keywords":["single-cell RNA-seq","neighbor embedding","t-SNE","UMAP","stability analysis","cell-state transitions","trajectory inference","transcriptional dynamics"],"falsifier":"Take a time-resolved lineage-tracing dataset with known division and fate-commitment times and ask whether cells in the lowest stability decile are enriched for cells that subsequently divide or change fate; if they are not, the transition-state interpretation of NESS collapses.","tokens_in":36200,"feed_emoji":"🧬","tokens_out":6810,"duration_ms":72189,"temperature":0.7,"pith_summary":"The paper argues that common neighbor embedding algorithms, including t-SNE, UMAP, and PHATE, distort the smooth, continuous structure of single-cell developmental data when their graph connectivity parameter is set too low, and that the resulting fragments and layout variability are artifacts, not biology. It introduces NESS, a procedure that reruns any neighbor embedding many times from random starting points, compares the neighborhood graphs across runs, and assigns each cell a local stability score. The central claim is that this score separates stable cell states from transitional ones along developmental trajectories and provides a proxy for transcriptional dynamics, using only an expression matrix. The paper supports this with correlations to transition entropy from a dynamic model and to independently computed RNA velocity, and it uses the score to resolve neuronal subpopulations in embryoid-body data that earlier analysis could not separate. If correct, NESS makes cell-state dynamics readable from ordinary single-cell RNA-seq data without extra data modalities.","feed_headline":"Stability across random embeddings flags transitional cell states","feed_subtitle":"Low-stability cells mark biological transitions and track RNA-velocity speed in six developmental scRNA-seq datasets.","key_machinery":"The load-bearing object is the NESS local stability score. For a fixed neighbor embedding algorithm and a chosen graph connectivity parameter (perplexity for t-SNE, number of neighbors for UMAP and PHATE), NESS runs the algorithm multiple times under random initialization, constructs a k-nearest-neighbor graph for each embedding, and counts, for every cell, how often each potential neighbor appears across runs. The local stability score is the 75th percentile of those normalized neighbor counts, the global stability score is their mean, and an embedding rareness score flags layouts that deviate strongly from the ensemble. The accompanying theory analyzes t-SNE on a discretized circle and proves that when the graph connectivity parameter is constant and the embedding diameter grows sublinearly, the optimal t-SNE embedding has unbounded bilipschitz distortion, formalizing why low graph connectivity produces gaps and fragmentation.","core_discovery":"The paper's central claim is that the stochasticity of neighbor embedding algorithms, normally treated as a nuisance, carries useful signal. NESS turns random initialization into a structured perturbation: it generates multiple embeddings of the same cells, builds a k-nearest-neighbor graph from each, and measures how often each cell's neighbors are preserved. Cells whose neighborhoods are stable across runs receive high local stability scores; cells whose neighborhoods vary sit in low-stability regions. The paper argues that these low-stability cells are precisely the cells in transitional states, because they occupy low-density regions of the expression manifold where the continuity of cell-state change predicts fast transitions. Across six datasets it reports that low-stability cells align with high transition entropy, that instability scores track total RNA velocity in spermatogenesis and neurogenesis, and that removing unstable cells improves embedding faithfulness; it also claims to resolve neuronal subtypes NS-3 and NS-4 as progenitor-like and intermediate states.","pith_inferences":["Because local stability is linked to local density, NESS scores could also flag sampling gaps and dropout-dominated regions; separating technical sparsity from true biological transition is a natural next test.","The same score could be used for unsupervised gene discovery: genes whose expression tracks local stability across multiple datasets would nominate candidate regulators of cell-state transitions.","The circle theorem suggests the fragmentation mechanism generalizes to any one-dimensional manifold with small graph connectivity; simulations on branching trees would show whether the stability threshold scales with branch geometry.","Since the score is defined purely through embedding stochasticity, the approach extends to any future stochastic embedding algorithm, turning the global stability line chart into a general model-selection device."],"forward_implications":["Analysts can replace default t-SNE, UMAP, or PHATE hyperparameters with the smallest graph connectivity parameter that reaches high NESS global stability, avoiding fragmentation without extra data.","Excluding cells with the lowest local stability scores improves neighborhood concordance and neighbor purity, yielding cleaner trajectory visualizations.","NESS instability scores can serve as a label-free substitute for RNA velocity, so transcriptional dynamics can be inferred from standard scRNA-seq data.","The automated workflow makes stability-guided embedding practical for datasets with tens of thousands of cells, since the cost grows roughly linearly with the graph connectivity parameter.","In datasets with cell-state annotations, low stability is predicted to mark intermediate or progenitor-like states and high stability to mark self-renewing or terminally differentiated populations."],"supporting_citations":[{"why":"Defines t-SNE and its perplexity parameter, which the paper uses as the graph connectivity parameter whose under-specification causes fragmentation.","marker":"[71]"},{"why":"Defines UMAP and the number-of-neighbors parameter, the main large-data embedding the paper stabilizes.","marker":"[54]"},{"why":"Introduces PHATE and the embryoid-body dataset with cell-state annotations used to validate stable versus transitional states.","marker":"[55]"},{"why":"Supplies transition entropy scores, the external dynamic-model measure that low-stability cells are shown to match.","marker":"[89]"},{"why":"Supplies scVelo RNA velocity estimates used to validate NESS instability as a transcriptional-dynamics proxy.","marker":"[9]"},{"why":"Documents that initialization is critical for global structure in t-SNE and UMAP, motivating the random-initialization perturbation strategy.","marker":"[38]"},{"why":"Provides theoretical grounding for t-SNE embeddings on structured data, which the paper extends to prove the low-connectivity fragmentation theorem.","marker":"[15]"},{"why":"Supplies the mouse hematopoiesis dataset whose true lineage hierarchy exposes fragmentation artifacts under default parameters.","marker":"[59]"},{"why":"Supplies the iPSC differentiation dataset with temporal labels used for transition-gene discovery and transition-entropy validation.","marker":"[5]"},{"why":"Supplies the murine intestinal organoid scEU-seq dataset used as a second validation of stability-entropy concordance.","marker":"[6]"}],"fun_headline_variants":["Embedding instability reveals cell transition states","Random seeds expose developmental trajectories","Low-stability cells mark dynamic transitions","Neighbor stability flags transitioning cells","Random embeddings reveal cell dynamics"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The claim that low stability marks biological transition states rests on assuming that cells occupying low-density regions of the expression manifold are undergoing fast state transitions; if that continuity assumption fails, low stability could just be technical noise or sampling gaps.","fun_headline_variants_meta":{"raw":{"variants":["Embedding instability reveals cell transition states","Random seeds expose developmental trajectories","Low-stability cells mark dynamic transitions","Neighbor stability flags transitioning cells","Random embeddings reveal cell dynamics"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000462,"raw_usage":{"total_tokens":2336,"prompt_tokens":999,"completion_tokens":1337,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":615,"completion_tokens_details":{"reasoning_tokens":1281}},"tokens_in":615,"tokens_out":1337,"duration_ms":12739,"temperature":1.0,"reasoning_tokens":1281,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T22:08:02.207440+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a time-resolved lineage-tracing dataset with known division and fate-commitment times and ask whether cells in the lowest stability decile are enriched for cells that subsequently divide or change fate; if they are not, the transition-state interpretation of NESS collapses.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces PHATE and the embryoid-body dataset with cell-state annotations used to validate stable versus transitional states."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies transition entropy scores, the external dynamic-model measure that low-stability cells are shown to match."},{"cited_title":"Kobak and G","cited_arxiv_id":null,"evidence_quote":"Documents that initialization is critical for global structure in t-SNE and UMAP, motivating the random-initialization perturbation strategy."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides theoretical grounding for t-SNE embeddings on structured data, which the paper extends to prove the low-connectivity fragmentation theorem."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the mouse hematopoiesis dataset whose true lineage hierarchy exposes fragmentation artifacts under default parameters."}],"review_version":1}