{"id":"95595b46-93bb-40eb-90e2-2df930cd2454","arxiv_id":"2412.03027","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"ScPace, a self-paced SVM that drops high-loss cells before retraining, improves timestamp annotation and supervised pseudotime analysis on simulated and real noisy time-series single-cell RNA-seq data.","lead":"This paper introduces ScPace, a method that automatically detects and corrects noisy time labels in time-series single-cell RNA-seq data by repeatedly training support vector machines on the cells the model is most confident about. It could make developmental trajectory analyses, such as how heart cells regenerate after injury, more reliable when experimental collection-time labels are imperfect.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"ScPace's core noise-detection step is not validated against known mislabels; high-loss cells may be boundary or biologically heterogeneous cells, so the timestamp-calibration claim rests on an untested equivalence.","rationale":"The reader's weakest_assumption correctly identifies that the hinge-loss threshold cannot cleanly separate noisy labels from genuine biological heterogeneity, and the GSE90047 case study itself concedes this. That is the most load-bearing concern because the central claim is that ScPace outperforms prior methods at timestamp calibration; if the method merely removes hard cells, then both the cross-validation gains and the pseudotime enhancements lose their specific interpretation. The concrete test I propose would settle this by measuring detection quality against known mislabels and by benchmarking random deletion. The PCA leakage and circular pseudotime evaluation are real but secondary; the detection question is more fundamental. Since the paper's current evidence is suggestive but not conclusive on this point, the reader's CONDITIONAL verdict remains appropriate.","tokens_in":21144,"tokens_out":2186,"duration_ms":24032,"concrete_test":"On Sim1 (or Sim2) with 40% swap mislabeling, after ScPace training, compare the set {i: v_i=0} against the ground-truth mislabeled cells. Report precision, recall, and F1 of detection, plus the fraction of v=0 cells that were originally correct labels. If precision or recall is low, the method is not detecting noise. Then run the deletion-based pseudotime analysis with the same number of randomly removed cells as a negative control; if random deletion yields comparable Spearman/Kendall gains, the enhancement claim is not specific to ScPace's selection.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The latent variable update in Eq. (16) assigns v_i=0 whenever the multi-class hinge loss from Eq. (19) exceeds lambda. This operationalizes 'noisy' as 'hard for the current one-vs-one SVM.' The paper never shows that high-loss cells coincide with artificially injected mislabels except through the aggregate correction score (Eq. 25), which measures reclassification accuracy, not detection precision/recall. The GSE90047 case study (Section 5.1) explicitly finds overlapping cholangiocyte states at E14.5/E15.5 flagged as noisy and recommends reclassification rather than deletion because the apparent noise 'may reflect biological variability.' That concession admits the threshold cannot separate label errors from genuine cells on a developmental continuum. If the deleted set is enriched for boundary cells rather than true mislabels, the pseudotime gains in Tables 4 and 5 can arise from removing hard-to-order cells, not from timestamp correction. The central comparative advantage claimed over ScReclassify (latent variable indicator versus probability sampling) is therefore not established at the level of detecting true mislabels.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes ScPace, a timestamp calibration method for time-series scRNA-seq data. The method augments a one-vs-one support vector machine with a latent-variable indicator v_i (Eq. 16) that, after each training iteration, flags cells whose multi-class hinge loss (Eq. 19) exceeds a threshold lambda; flagged cells are either deleted or reclassified. The SVM is re-weighted and the regularization parameter C is increased over iterations (Algorithm 2). The authors evaluate ScPace in stratified cross-validation experiments with artificially injected swap and random mislabeling on three simulated datasets (Splatter linear and bifurcation paths) and four real time-series datasets, reporting accuracy and a correction score (Eq. 25) against ScReclassify, CleanLab, and several baseline classifiers. They also apply the calibrated timestamps to Psupertime supervised pseudotime analysis and report that ScPace enhances pseudotime performance more than the alternatives. The paper claims that ScPace is the first method designed specifically for timestamp calibration and that its latent-variable selection mechanism improves over probability-sampling approaches.","tokens_in":21422,"tokens_out":4061,"duration_ms":42189,"significance":"If the stated results hold, ScPace would be a practically useful contribution: timestamp noise is a real concern in time-series scRNA-seq, and the paper provides a concrete algorithmic proposal, a public implementation, and a fairly broad experimental setup that includes simulated and real data, two noise models, several mislabeling rates, and sensitivity analyses. The simulation-based cross-validation against artificially mislabeled ground truth is a sensible evaluation strategy, and the authors should be credited for making the code available and for reporting computational timings. However, the central comparative claims are currently not established because of three load-bearing methodological issues: potential information leakage in the dimensionality-reduction step, a circular metric for pseudotime enhancement, and the absence of direct validation that the latent-variable flag actually identifies mislabeled cells rather than hard or boundary cells. These issues affect the interpretation of Figures 4-7, Tables 4-5, and the core claim of superiority over ScReclassify and CleanLab.","major_comments":[{"comment":"The manuscript does not state whether PCA/KernelPCA is fitted on the training folds only in the cross-validation experiments. Algorithm 2 explicitly says 'Conduct PCA or KernelPCA on Original Datasets', and Section 3.1.1 says dimensional reduction is conducted on 'both training and testing sets'. If PCA or KernelPCA is fitted on the full dataset before the k-fold split, information from the holdout test set leaks into the training procedure, which would inflate the reported accuracy and correction scores in Figures 4-7. The authors correctly specify for ScReclassify that dimensionality reduction on the test set should use parameters determined from the training set, but no such statement is given for ScPace. This needs to be fixed by fitting the reduction inside each training fold, and the paper should report whether the conclusions change.","section":"Section 3.1.1, Algorithm 2, Section 4.2.1"},{"comment":"The pseudotime 'enhancement' results are measured as the Spearman and Kendall correlation between the Psupertime pseudotime values and the calibrated timestamps — that is, the timestamps produced by the very calibration method being evaluated. On the original real datasets, this measures self-consistency rather than agreement with any independent ground truth. On simulated data, the paper should instead correlate the pseudotime with the known true simulation time, and on real data it should compare against the original uncalibrated timestamps or an external biological ordering. As written, the larger correlations in Tables 4 and 5 may partly reflect the calibration procedure making the labels more self-consistent, not making them more correct, so the enhancement claim is not supported.","section":"Section 4.3.1, Tables 4 and 5"},{"comment":"The paper never directly validates that cells flagged as noisy (v_i = 0) coincide with the artificially injected mislabels. The correction score in Eq. (25) measures the fraction of mislabeled cells whose final reclassified label matches the original true label; it does not report detection precision or recall of the v-flag. The GSE90047 case study in Section 5.1 explicitly finds that overlapping developmental states between E14.5 and E15.5 are flagged as noisy and that the apparent noise 'may reflect biological variability', which the authors themselves use to argue against deletion for that dataset. This concession shows that the threshold in Eqs. (16) and (21) does not separate label errors from genuine boundary cells. Consequently, the deletion-based pseudotime improvements in Tables 4 and 5 could arise from removing hard-to-order cells rather than from correcting timestamp labels. The authors should report precision/recall of the v-flag against known mislabels in the simulations and examine the stability of deletion-based gains when the threshold is varied.","section":"Section 3.1.3, Eq. (16), Eq. (25), Section 5.1"}],"minor_comments":[{"comment":"The instruction 'Conduct PCA or KernelPCA on Original Datasets' conflicts with the need for leakage-free cross-validation; please clarify the intended usage for both the calibration setting and the cross-validation setting.","section":"Algorithm 2"},{"comment":"In Eq. (5), the symbol m is used both for the original feature dimension (Eq. 1) and for the number of classes in the class-penalty vector; please use distinct notation.","section":"Eq. (5)"},{"comment":"Please clarify whether artificial mislabeling is applied only to the training folds while the test folds retain their original clean labels, and how the correction score in Eq. (25) is aggregated across folds.","section":"Section 4.2.1"},{"comment":"The caption of Figure 16 repeats the caption of Figure 1; it should describe the cardiomyocyte prediction result shown in the figure.","section":"Figure 16"},{"comment":"In Section 6, '125Gi' should be '125 GiB'.","section":"Section 6"},{"comment":"The claim that ScPace is 'the first method specifically designed' for timestamp calibration is strong and should be supported by a more systematic comparison with existing label-noise and timestamp-calibration approaches.","section":"Abstract and Introduction"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a relevant problem and the core algorithmic idea is reasonable, but the reported evidence is not yet reliable. The PCA-leakage question and the circular pseudotime evaluation are the most urgent: both can be fixed within the manuscript's scope by rerunning experiments with fold-internal preprocessing and by evaluating pseudotime against independent ground truth. The third major point, about the lack of direct validation of the v-flag, is also fixable using the existing simulation framework. I would not recommend rejection, but the authors should be asked to present the revised results prominently; if the leakage or circularity changes the main conclusions, the claims will need to be substantially weakened."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper gives us ScPace: a self-paced SVM with a hard latent-variable selection rule for timestamp calibration in time-series scRNA-seq. The application is real, and the simulation evidence is decent — at 40% swap mislabeling, ScPace clearly beats the baselines, and the accuracy numbers are consistent across simulated and real datasets. The code is out, the sensitivity analysis is a plus, and the writing, while occasionally sloppy, is understandable. If the method works, it fills a genuine niche, since timestamp noise is a known pain point and ScReclassify was not built for this exact purpose.\n\nThe soft spots are real, though. The stress-test note is right: the paper never shows that the cells flagged as noisy are actually the mislabeled ones. Equation (25) reports an aggregate correction score, but not detection precision or recall. You cannot tell from the paper whether ScPace is pulling out true errors or just hard-to-classify boundary cells. The GSE90047 case study is the tell: Section 5.1 explicitly says the detected overlapping cholangiocytes may reflect biological variability, not technical label error, and recommends reclassification instead of deletion. That is a direct admission that the threshold cannot separate \"wrong\" from \"hard.\"\n\nThe PCA issue is also legitimate. Algorithm 2 says to conduct PCA on the original dataset, and the paper never states that the reduction is fit on the training fold only. If PCA is fit on the full data before cross-validation, that leaks test information and inflates the reported gains. The pseudotime enhancement evaluation is circular: the reference labels are ScPace's own reclassified or deleted labels, so the correlation partly measures self-consistency, not agreement with an independent truth.\n\nNone of this is fatal to the idea. The cross-validation against artificially mislabeled ground truth is independent and supports the method at the level of final accuracy. The weighted-SVM derivation is standard. But the core mechanism needs direct validation. A revised paper with detection precision/recall on simulated mislabels, a clean statement about PCA fitting, and a pseudotime evaluation using an independent reference (e.g., known cell ordering or a method that does not feed on ScPace's output) would largely address my concerns.\n\nI would send this to a serious referee, but expect heavy revision. The reader's conditional verdict is right, and the stress-test concern is the main thing to fix.","headline":"ScPace is a plausible tool for a real problem, but the paper's central claim—that its loss threshold identifies mislabeled timestamps—is under-supported, and the pseudotime evaluation is circular.","tokens_in":21877,"tokens_out":3326,"would_cite":false,"duration_ms":34102,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"ScPace introduces a latent-variable classifier to detect and correct noisy collection-time labels in time-series single-cell RNA-seq, improving timestamp annotation and supervised pseudotime analysis.","keywords":["Timestamp calibration","Time-series single-cell RNA-seq","Self-paced learning","Label noise","Latent variable indicator","Support vector machine","Supervised pseudotime analysis","ScPace"],"falsifier":"Run ScPace on a simulated four-timestamp dataset with 40% swap mislabeling and compare its flagged cells against the known mislabeled set; if the flagged cells do not substantially overlap the true mislabeled cells (low recall) or if reclassifying them does not beat a baseline SVM trained on clean labels, the central claim would be falsified.","tokens_in":20972,"feed_emoji":"⏱️","tokens_out":8409,"duration_ms":76463,"temperature":0.7,"pith_summary":"The paper seeks to establish that noisy timestamps—cells assigned to the wrong collection time—are a correctable bottleneck in time-series single-cell RNA-seq analysis, and that the most effective correction comes from a latent-variable indicator that lets a classifier flag its own suspicious training samples instead of sampling candidates probabilistically. It introduces ScPace, an SVM-based calibration method that iteratively trains only on cells it currently considers clean, computes a multiclass hinge loss for every cell, and then deletes or reclassifies cells whose loss exceeds a threshold. On simulated and real datasets with artificially injected swap and random mislabeling up to 40%, ScPace reports the highest accuracy among the compared classifiers, including 97.33% on the linear simulated dataset Sim1 under swap mislabeling, and it improves the correlation of supervised pseudotime analysis on real datasets. If the claim holds, timestamp calibration becomes a practical preprocessing step that makes downstream trajectory inference more reliable without requiring users to discard whole time points.","feed_headline":"ScPace filters noisy single-cell timestamps with 97% accuracy","feed_subtitle":"A self-paced classifier flags mislabeled time points and sharpens supervised pseudotime correlations on real datasets.","key_machinery":"The load-bearing object is a per-cell latent variable v_i ∈ {0,1} (Eq. 16) that is optimized together with an SVM. After each training round, ScPace computes the Crammer-Singer multiclass hinge loss (Eq. 19) for every cell from one-vs-one RBF SVM decision values converted to one-vs-rest scores by Algorithm 1 and Eqs. (17)-(18); cells with loss above the threshold λ are assigned v_i=0 and excluded from the next round, while cells below threshold are kept. The regularization parameter C grows by a rate p each iteration to avoid underfitting after sample removal, and class-specific penalties handle imbalanced timestamps. This mechanism replaces probability-based sample selection with deterministic margin-based selection, which is the step that carries the argument.","core_discovery":"The central claim is that a binary latent variable v_i, updated by thresholding a multiclass hinge loss, separates correctly timestamped cells from mislabeled cells in time-series scRNA-seq data, and that acting on those flags improves both timestamp automatic annotation and supervised pseudotime analysis. The paper argues that prior calibration approaches select candidate noisy samples by probability sampling, which can pick unimportant cells, whereas ScPace selects deterministically by loss magnitude after training a weighted one-vs-one SVM. In cross-validation with swap and random mislabeling, ScPace stays above roughly 90% accuracy even at a 40% mislabeling rate and outperforms the existing semi-supervised calibration method and confident-learning baselines on most simulated and real datasets. The authors also report that reclassifying or deleting the flagged cells before running supervised pseudotime analysis raises the resulting trajectory correlations, and they position ScPace as the first method specifically designed for timestamp calibration.","pith_inferences":["The same latent-variable thresholding scheme could be transferred to other ordinal label-noise problems, such as batch-effect labels or progression stages, whenever a base classifier produces per-sample margins.","Because the paper's sensitivity analysis identifies the regularization parameter C as the most performance-sensitive hyperparameter, an adaptive C schedule may matter more than threshold tuning in real deployments.","A diagnostic that separates technical from biological high-loss cells—for example, stability of v=0 flags across bootstrap resamples—could decide automatically between deletion and reclassification, which the paper currently leaves as a user choice.","The reported failure above 50% mislabeling suggests a natural stress test: at 60% noise the clean subset may no longer represent the true distribution, and combining ScPace with robust ordinal regression could push the usable noise range higher."],"forward_implications":["Time-series scRNA-seq pipelines can treat ScPace as a preprocessing step that cleans timestamps before timestamp annotation or supervised pseudotime, improving trajectory correlations on noisy data.","Because ScPace maintains accuracy above about 90% even at 40% swap mislabeling on simulated data, downstream analyses can tolerate higher label noise than previously assumed.","The deletion mode suits technical noise, while the reclassification mode is recommended when high-loss cells reflect overlapping biological states, as the GSE90047 case shows.","ScPace can run with or without PCA or kernel PCA, so it remains applicable to datasets where dimension reduction hides informative signal."],"supporting_citations":[{"why":"Supplies the prior semi-supervised calibration method that ScPace improves on; its probability-sampling selection is the limitation the latent variable replaces.","marker":"[10]"},{"why":"Introduces self-paced learning, the easy-to-hard training paradigm on which ScPace's iterative clean-subset training is built.","marker":"[12]"},{"why":"Psupertime is the supervised pseudotime method whose accuracy and trajectory correlations measure the benefit of calibration.","marker":"[30]"},{"why":"Provides the multiclass hinge loss used to score each cell's loss and drive the latent-variable update.","marker":"[34]"},{"why":"Splatter generates the simulated linear and bifurcation datasets Sim1-Sim3 used for benchmark comparisons.","marker":"[35]"},{"why":"Provides the confident-learning baseline that ScPace is compared against in cross-validation and pseudotime enhancement.","marker":"[40]"},{"why":"Adasampling is the semi-supervised selection technique underlying ScReclassify, providing the contrast for ScPace's deterministic selection.","marker":"[31]"}],"fun_headline_variants":["ScPace auto-calibrates noisy timestamps in single-cell time series","Self-paced classifier flags mislabeled cells, sharpens pseudotime","ScPace: cleaning timestamps to reveal true cell dynamics","Boosting single-cell trajectory analysis with ScPace timestamp calibration"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing assumption is that a single loss threshold separates genuinely mislabeled cells from cells that are correctly labeled but biologically unusual, so that removing or relabeling high-loss cells removes noise rather than real developmental states.","fun_headline_variants_meta":{"raw":{"variants":["ScPace auto-calibrates noisy timestamps in single-cell time series","Self-paced classifier flags mislabeled cells, sharpens pseudotime","ScPace: cleaning timestamps to reveal true cell dynamics","Boosting single-cell trajectory analysis with ScPace timestamp calibration"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000348,"raw_usage":{"total_tokens":1922,"prompt_tokens":981,"completion_tokens":941,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":597,"completion_tokens_details":{"reasoning_tokens":866}},"tokens_in":597,"tokens_out":941,"duration_ms":8877,"temperature":1.0,"reasoning_tokens":866,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T22:50:19.237630+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run ScPace on a simulated four-timestamp dataset with 40% swap mislabeling and compare its flagged cells against the known mislabeled set; if the flagged cells do not substantially overlap the true mislabeled cells (low recall) or if reclassifying them does not beat a baseline SVM trained on clean labels, the central claim would be falsified.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the prior semi-supervised calibration method that ScPace improves on; its probability-sampling selection is the limitation the latent variable replaces."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces self-paced learning, the easy-to-hard training paradigm on which ScPace's iterative clean-subset training is built."},{"cited_title":"Macnair, R","cited_arxiv_id":null,"evidence_quote":"Psupertime is the supervised pseudotime method whose accuracy and trajectory correlations measure the benefit of calibration."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the multiclass hinge loss used to score each cell's loss and drive the latent-variable update."},{"cited_title":"Zappia, B","cited_arxiv_id":null,"evidence_quote":"Splatter generates the simulated linear and bifurcation datasets Sim1-Sim3 used for benchmark comparisons."}],"review_version":1}