{"id":"1300cf03-a591-401b-9587-badc56400168","arxiv_id":"2501.18650","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"MOTT uses optimal transport with relaxed marginal constraints to match cell clusters across samples and builds a shared taxonomy for automatic cell-type annotation and sample classification.","lead":"A new pipeline called MOTT aligns cell-type clusters across many single-cell samples using optimal transport with relaxed rules, so cell types missing from some samples are still handled. It then builds a family tree of cell types and can help label new samples automatically.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central accuracy claim rests on idealized inputs: ground-truth clusters and known K, so real-world performance on noisy clusters and unknown taxonomy size remains unvalidated.","rationale":"The reader's weakest_assumption—that the input clusters are faithful proxies for cell types—is exactly the load-bearing concern. The method never pools or re-clusters cells, so any error in upstream clustering propagates directly into the taxonomy. The paper's evaluation artificially removes this error by using ground-truth labels as clusters, and it also supplies the true number of meta-clusters. My reading did not uncover a different concern that is more central to the claim. The technical details of OT-RMC (e.g., the L1 penalty on gap variables) could be examined, but they are not where the claim fails; the empirical evaluation is. I considered whether the term 'simultaneous alignment' overstates the pairwise-then-tile procedure, but the paper explicitly describes that procedure and the SA/RA comparison is meaningful regardless of the naming. I also noted the abstract's 'more than twenty datasets' versus the Results' 11 data sources, which inflates the apparent evidence, but this is secondary to the idealized-input issue. Given that the paper is internally coherent and the method is clearly described, a conditional verdict is appropriate: the central claim should be accepted only after the method is tested on realistically noisy clusters and with unknown K. The proposed concrete test would directly address the gap, so the reader's verdict remains unchanged.","tokens_in":14421,"tokens_out":3580,"duration_ms":40708,"concrete_test":"Run MOTT on three datasets from Table 1 (e.g., Baron, Segerstolpe, and Tasic) using clusters produced by a standard scRNA-seq pipeline (e.g., Seurat or Scanpy with Leiden clustering, at default resolutions) instead of ground-truth labels, and determine the number of meta-clusters by a data-driven rule such as the largest gap in the Ward dendrogram or a stability-based criterion. Compare the resulting ζcell, ζcls, and ARI against the same metrics computed when ground-truth clusters and the true K are used, and also compare against an established annotation tool such as SingleR. If the accuracy drops substantially (e.g., more than 10 percentage points in ζcell) or the chosen K deviates from the true number of cell types, the idealized evaluation is the direct cause and the central claim must be narrowed.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's headline claim—that the taxonomy constructed by MOTT yields highly accurate cell-type annotation—is supported only under conditions that are artificially favorable. In all experiments, the clusters fed to MOTT are the ground-truth cell types themselves (Section 4.1: 'we start with the cell clusters provided for each sample' and 'The 11 datasets we tested include ground truth cell types, which are used to form cell clusters within each sample'). This means MOTT never has to contend with clusters that are split, merged, or misassigned by an upstream clustering algorithm, yet the method's entire premise is to operate on clusters extracted independently from different samples. Second, the number of meta-clusters is set exactly to the number of ground-truth cell types (Section 3.1, Step 3), so the taxonomy evaluation bypasses the model-selection problem that any real application faces. Third, the penalty parameter λ is chosen empirically on the same datasets (Section 4.4: 'By varying λ from 0.005 to 0.15, we find that the optimal value for λ is always around 0.075'), which risks overfitting the evaluation. Finally, no external baseline tools are compared; all baselines are variants of the method itself. Collectively, these choices mean the reported accuracy numbers demonstrate internal consistency among OT/POT and SA/RA variants, but they do not establish that MOTT would deliver accurate annotation when clusters are produced computationally and K is unknown. The central claim is therefore conditional on an idealized evaluation setup that does not match the intended real-world usage scenario.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes MOTT (Multisample OT Taxonomy), a system that constructs a hierarchical taxonomy of cell clusters across multiple single-cell RNA-seq samples. MOTT combines optimal transport with relaxed marginal constraints (OT-RMC) from the authors' prior work to compute pairwise cluster matching weights, tiles these weights into an overall similarity matrix, transforms it to a distance matrix, and applies Ward's hierarchical clustering to form meta-clusters that represent consistent cell-type labels across samples. The system is evaluated on 11 scRNA-seq datasets in two tasks: cell-type annotation accuracy (ARI, cluster-level accuracy, cell-level accuracy) and sample-level classification using cluster proportions as features. The paper reports that MOTT outperforms its method variants (OT, POT, reference alignment) in most settings and achieves competitive sample-level classification. The authors also discuss limitations, including the lack of evaluation of cell-type similarity and the static-sample assumption.","tokens_in":14721,"tokens_out":3353,"duration_ms":34308,"significance":"If the empirical claims were established under realistic conditions, MOTT would be a useful contribution to multi-sample single-cell analysis, particularly because it operates on independently derived clusters without pooling raw data, which is relevant for privacy and large-scale studies. The use of OT-RMC to allow clusters to appear or disappear across samples is well motivated, and the simultaneous alignment across all samples is a sensible design choice. The paper is clearly written and the code/data availability is a strength. However, the empirical support is currently compromised by the evaluation design: input clusters are formed from ground-truth cell types and the number of meta-clusters is set to the known number of cell types, so the reported accuracies do not reflect performance on noisy or unknown upstream clustering. The lack of external baselines and of statistical uncertainty further limits the strength of the claims. The core methodological components are largely from prior work, so the novelty is primarily in the system integration, which is a legitimate but modest contribution.","major_comments":[{"comment":"The evaluation uses ground-truth cell types to form the input clusters within each sample ('The 11 datasets we tested include ground truth cell types, which are used to form cell clusters within each sample') and sets the number of meta-clusters equal to the known number of cell types (Step 3 of §3.1). This makes the reported accuracy numbers conditional on perfect upstream clustering and known taxonomy size, neither of which holds in a realistic application where clusters are produced computationally and the number of cell types is unknown. The abstract's claim that the taxonomy 'can yield highly accurate annotation of cell types' is therefore not established for the intended use case. The authors should evaluate with computationally derived clusters (e.g., from Seurat or other standard tools) and with a data-driven method for choosing the dendrogram cut, or explicitly temper the claim to reflect the idealized setting.","section":"§4.1, §3.1"},{"comment":"The hyperparameter λ is selected by evaluating performance on the same datasets used for the reported results ('By varying λ from 0.005 to 0.15, we find that the optimal value for λ is always around 0.075'). This constitutes tuning on the test data, which can inflate the reported accuracies. The authors should either fix λ based on a validation set or prior reasoning, or report results across a range of λ with appropriate error bars, to demonstrate that the conclusions are not an artifact of overfitting.","section":"§4.4"},{"comment":"All baseline methods are variants of the proposed framework (OT, POT, reference alignment), with no comparison to existing tools for cell-type annotation or cluster matching, such as ClusterMap, scmap, scPred, or other published approaches. Since the paper motivates its contribution by citing limitations of existing methods (e.g., marker-gene dependence, pairwise reference matching), a direct empirical comparison is necessary to support the claimed advantages. The current results show internal consistency among the variants but do not position MOTT relative to the state of the art.","section":"§4.1, §4.4"},{"comment":"The sample-level classification results are obtained with leave-one-out cross-validation on at most 20 samples and are reported without any measure of variability (e.g., confidence intervals) or significance tests. Differences between methods are often small (e.g., OT-RMC-SA vs POT-SA on Bacher, with Acc. 0.6 vs 0.45 but overlapping given the tiny sample size) and appear dataset-specific. The claim that OT-RMC-SA yields 'the most accurate classification across datasets' is not statistically supported as presented.","section":"§4.5, Table 2"}],"minor_comments":[{"comment":"The abstract states 'Experiments on more than twenty datasets' but the paper reports 11 data sources (Table 1), with some datasets having multiple label sets or simulated-sample variants. Please clarify the counting convention or adjust the wording to avoid overstatement.","section":"Abstract, §4.1"},{"comment":"Step 3 notes that the number of groups is set equal to the number of ground-truth cell types, but this is a critical practical assumption. A brief discussion of how the dendrogram would be cut in an unsupervised setting (e.g., using a stability-based criterion or a gap statistic) would help readers assess real-world applicability.","section":"§3.1"},{"comment":"Figure 6 shows sensitivity to λ for three datasets; consider reporting the same analysis for all datasets or providing a summary of the range of λ values that yield near-optimal performance, to strengthen the claim that λ=0.075 is a robust choice.","section":"§4.4"},{"comment":"The related work section would benefit from a brief discussion of recent systematic benchmarks of cell-type annotation methods (beyond ref. [1]) to better contextualize the claimed improvements.","section":"§2, References"},{"comment":"For datasets without sample division (Tasic, He Organ, etc.), the table does not list a number of samples; the reliance on simulated samples is explained later, but a note in the table caption would improve readability.","section":"Table 1, §4.1"},{"comment":"The three panels in Figure 4 are dense and the labels are small; separating the panels or enlarging the figure would make the comparisons easier to parse.","section":"Figure 4"}],"recommendation":"major_revision","confidential_remarks":"The paper's central methodological idea is reasonable, but the evaluation is idealized to the point that the headline claim of 'highly accurate annotation' is not yet credible for real-world use. The authors can address this by adding experiments with computationally derived clusters, a principled approach to choosing the number of meta-clusters, and external baseline comparisons. The λ tuning issue and the lack of uncertainty quantification further weaken the empirical case. These are fixable within the manuscript's scope, hence major revision rather than rejection. Also, the discrepancy between 'more than twenty datasets' in the abstract and the 11 data sources in the paper should be corrected."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: MOTT is a real methods contribution, clearly described, but its headline accuracy claim is supported only under idealized conditions—ground-truth clusters and known K—so the central claim should be read as conditional.\n\nWhat is actually new: the combination. Prior automatic tools either worked pairwise or required marker genes (ClusterMap). Tiling all pairwise OT-RMC matchings into a global similarity matrix and then running Ward hierarchical clustering is a sensible way to build a multi-sample taxonomy, and relaxed marginals are the right tool for cell types that appear in only some samples. The ablations are informative: OT-RMC beats OT, and simultaneous alignment beats reference alignment in almost every dataset. The paper ships code, and the experiments span a reasonable number of real and simulated datasets.\n\nWhere it gets soft. First, the evaluation feeds MOTT the ground-truth cell-type clusters and sets the number of meta-clusters to the known number of cell types. That tests the matching/taxonomy layer in isolation, which is a legitimate ablation, but it does not validate the real use case where clusters come from an unsupervised pipeline and K is unknown. The Wu dataset is a partial exception—clusters there are computationally derived—but K is still fixed to 10 and it is one dataset. Second, no external baseline tool is run; ClusterMap is cited but not compared. The internal OT/POT and SA/RA comparisons show consistency but not superiority in the field. Third, λ is chosen on the same datasets used for evaluation; the reported stability across a range mitigates this, so I call it minor. There are also no error bars or significance tests, which is surprising from a statistics group; simulated-sample splits would give cheap uncertainty estimates.\n\nThe paper deserves a serious referee. The method is new, the writing is clear, and the evaluation gap is fixable: run on computationally derived clusters with unknown K, add one or two external baselines, report variability. I would encourage engagement rather than rejection.","headline":"A genuinely new multi-sample taxonomy method with a solid ablation study, but the accuracy claim rests on an idealized evaluation; worth refereeing with a request for real-world cluster inputs.","tokens_in":15255,"tokens_out":3232,"would_cite":true,"duration_ms":31316,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a taxonomy of cell clusters across multiple single-cell samples can be built automatically by combining optimal transport with relaxed marginal constraints and simultaneous alignment, yielding accurate cell-type…","keywords":["single-cell RNA-seq","cell-type taxonomy","optimal transport","relaxed marginal constraints","cluster alignment","hierarchical clustering","sample classification","meta-cluster"],"falsifier":"Apply MOTT to clusters produced by a standard unsupervised clustering pipeline, such as Louvain on each sample separately, for a dataset with expert-annotated cell types, and compare the meta-cluster labels with the annotations. The central claim would be falsified if the annotation accuracy falls substantially below the cluster-level accuracy reported in the paper for that dataset.","tokens_in":14203,"feed_emoji":"🧬","tokens_out":8923,"duration_ms":79709,"temperature":0.7,"pith_summary":"The paper sets out to show that a taxonomy of cell clusters across many single-cell RNA-seq samples can be constructed automatically, without pooling cells or using an annotated reference. The proposed system, MOTT, combines optimal transport with relaxed marginal constraints (OT-RMC) for pairwise cluster matching with simultaneous alignment of all samples, and then builds a hierarchy by clustering the resulting similarity matrix. The authors report that on more than twenty dataset configurations from eleven scRNA-seq sources, the resulting taxonomy annotates cell types accurately and yields sample-level features that classify phenotypes correctly. The payoff would be consistent cluster labels across samples, which are needed for merging data, finding marker genes, and downstream classification.","feed_headline":"Relaxed optimal transport aligns cell types across samples","feed_subtitle":"Its taxonomy handles missing cell types and beats pairwise alignment across many datasets.","key_machinery":"The load-bearing object is the OT-RMC matching weight matrix $W$. For two samples, each cluster is summarized by a Gaussian $N(\\mu_k^{(i)}, \\Sigma_k^{(i)})$, and the entry $D(C_k^{(i)}, C_l^{(j)})$ of the cost matrix is the standardized squared Wasserstein distance between the two Gaussians. OT-RMC then minimizes $\\langle D, W\\rangle + \\lambda L(g)$ subject to a total-mass constraint and relaxed marginal constraints with gap vectors $g_1, g_2$, penalized by their L1 norm; nonzero gaps let a cluster receive zero induced proportion, so a cell type missing from a sample can be left unmatched. The optimal weight matrices for all sample pairs are row- and column-normalized and averaged, tiled into a global similarity matrix $B$, converted to a distance matrix by $-\\log(\\cdot)$ and division by the maximum, and finally clustered with Ward's linkage to produce the taxonomy tree.","core_discovery":"The central claim is that OT-RMC provides a robust matching weight between clusters of two samples even when cell-type proportions vary strongly and some cell types are absent from one sample, and that assembling all pairwise matching weights into a global similarity matrix before clustering yields a taxonomy that groups clusters of the same cell type across samples. The method represents each cluster by a fitted Gaussian, computes standardized squared Wasserstein distances as matching costs, and solves an OT-RMC problem with gap variables that allow clusters to be left unmatched. The pairwise weight matrices are normalized, tiled into one block matrix, log-transformed into distances, and clustered with Ward's linkage; the number of meta-clusters is set to the known number of cell types, and each meta-cluster is labeled by majority vote. Experiments on more than twenty dataset configurations show high cluster-level and cell-level annotation accuracy, and sample-level classification using taxonomy-derived cell-type proportions matches or exceeds classification based on ground-truth labels in several datasets.","pith_inferences":["A direct follow-up would test MOTT on clusters produced by an unsupervised pipeline rather than ground-truth types; performance would likely drop, but combining MOTT with cluster-quality filters could mitigate the gap.","The OT-RMC plus simultaneous-alignment recipe is domain-agnostic and could be applied to other multi-sample cluster-correspondence problems, such as flow cytometry or spatial transcriptomics.","The hierarchy itself could be validated against known lineage trees or marker-gene overlap to define cell-type similarity, an evaluation the authors explicitly leave open."],"forward_implications":["MOTT can assign consistent cell-type labels to clusters across samples without an annotated reference or marker genes, so it can annotate datasets where no representative training set exists.","Because the relaxed marginal constraints allow unmatched clusters, the method handles cell types that appear in only some samples and strongly varying cluster proportions, conditions under which standard optimal transport fails.","Taxonomy-derived cell-type proportions serve as sample features for phenotype classification, and in the reported tests they match or beat classification based on ground-truth labels.","The taxonomy groups clusters into meta-clusters and also yields a hierarchy of cell-type similarity, although the paper does not validate that similarity against an external standard."],"supporting_citations":[{"why":"Supplies the OT-RMC formulation with gap variables that the method adopts as its core matching technique.","marker":"[25]"},{"why":"Closest existing multi-sample cluster-hierarchy method, built on marker genes and Jaccard distance; serves as the main conceptual baseline.","marker":"[10]"},{"why":"Provides a bipartite-graph-partitioning baseline for cluster matching across samples.","marker":"[3]"},{"why":"Provides the partial optimal transport baseline implementation used for comparison.","marker":"[9]"},{"why":"Gives the regularized optimal transport computational approach underlying the POT baseline.","marker":"[6]"},{"why":"One of the key real-sample datasets with phenotype labels, used for both taxonomy evaluation and sample-level classification.","marker":"[37]"}],"fun_headline_variants":["Relaxed optimal transport aligns cell types despite missing data","OT-RMC builds taxonomy with missing and varying cell types","New optimal transport method handles absent cell types","Align all samples at once with relaxed transport taxonomy","Taxonomy via OT-RMC beats pairwise cluster alignment"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes the input clusters are faithful proxies for cell types; because MOTT never pools cells or re-clusters, any error in the upstream clustering carries directly into the taxonomy, and the experiments use ground-truth cell types as the clusters, leaving behavior on realistic noisy clusters untested.","fun_headline_variants_meta":{"raw":{"variants":["Relaxed optimal transport aligns cell types despite missing data","OT-RMC builds taxonomy with missing and varying cell types","New optimal transport method handles absent cell types","Align all samples at once with relaxed transport taxonomy","Taxonomy via OT-RMC beats pairwise cluster alignment"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000478,"raw_usage":{"total_tokens":2372,"prompt_tokens":954,"completion_tokens":1418,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":570,"completion_tokens_details":{"reasoning_tokens":1343}},"tokens_in":570,"tokens_out":1418,"duration_ms":11364,"temperature":1.0,"reasoning_tokens":1343,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T04:26:16.052252+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Apply MOTT to clusters produced by a standard unsupervised clustering pipeline, such as Louvain on each sample separately, for a dataset with expert-annotated cell types, and compare the meta-cluster labels with the annotations. The central claim would be falsified if the annotation accuracy falls substantially below the cluster-level accuracy reported in the paper for that dataset.","supporting_citations":[{"cited_title":"ClusterMap: compare multiple single cell RNA-Seq datasets across dif- ferent experimental conditions","cited_arxiv_id":null,"evidence_quote":"Closest existing multi-sample cluster-hierarchy method, built on marker genes and Jaccard distance; serves as the main conceptual baseline."},{"cited_title":"flowMatch: Cell population matching and meta-clustering in Flow Cytometry","cited_arxiv_id":null,"evidence_quote":"Provides a bipartite-graph-partitioning baseline for cluster matching across samples."},{"cited_title":"POT: Python Optimal Transport","cited_arxiv_id":null,"evidence_quote":"Provides the partial optimal transport baseline implementation used for comparison."},{"cited_title":"Iterative Bregman projections for regularized transportation problems","cited_arxiv_id":null,"evidence_quote":"Gives the regularized optimal transport computational approach underlying the POT baseline."},{"cited_title":"Single-cell transcriptome profiling of human pancreatic islets in health and type 2 diabetes","cited_arxiv_id":null,"evidence_quote":"One of the key real-sample datasets with phenotype labels, used for both taxonomy evaluation and sample-level classification."}],"review_version":1}