{"id":"69bcaac6-7efe-4618-b356-1c8c3ce060cc","arxiv_id":"2501.18714","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":8,"one_line_summary":"A deep-learning-enhanced SarcGraph detects sarcomeres in immature and mature hiPSC-CMs and uses per-sarcomere features to predict expert organization scores and cluster cells without labels.","lead":"The authors upgraded SarcGraph, a tool that finds sarcomeres in heart muscle cell images, with a deep-learning filter and a smarter network-scoring step, then tested it on the public Allen Institute dataset. It reports fewer false positives in immature cells, longer detected myofibrils, and score predictions matching expert ratings at Pearson 0.77 to 0.79 on held-out data.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Central false-positive reduction claim is not validated against ground truth; observed sarcomere count drops could reflect threshold-induced false negatives.","rationale":"The reader's weakest assumption is the Otsu recall ceiling noted in Section 4.3; I agree that is a real limitation. However, I see a more immediate load-bearing gap: the central claim of reduced false positives is never directly measured. The reported evidence, Fig 4A-i, is a decrease in detected counts. Without a precision/recall benchmark, the modified pipeline could simply be more conservative rather than more accurate. This is especially plausible because the pipeline introduces several aggressive filtering steps: the classifier threshold of 0.3, the rule-based graph pruning, and the removal of single-sarcomere myofibrils. The supervised correlation with expert scores does not disambiguate, since low expert score is strongly associated with low detected count regardless of whether those counts are true. The missing ground-truth comparison is addressable and would settle the claim. I do not see this as fatal; the code is public, the held-out test correlations are useful, and the claim could be true. But the paper should remain conditional on such validation, which is why I do not change the reader's verdict.","tokens_in":26260,"tokens_out":3703,"duration_ms":36268,"concrete_test":"Select a stratified random sample of ~100 cells from the Allen dataset across expert scores 1-5 and from Train, Test FISH, and Test Live. Have an expert annotate all visible z-discs and sarcomere chains in the alpha-actinin-2 channel. Run the original and modified SarcGraph pipelines on these cells and compute per-cell precision, recall, and F1 for z-discs and for sarcomeres, using a matching tolerance such as a predicted z-disc within 3 pixels of an annotated z-disc. Then sweep the z-disc probability threshold from 0.1 to 0.7 and check whether the low-score count drop is due to removing false positives (precision up, recall flat) rather than removing true z-discs (recall down).","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's headline improvement is that the modified pipeline 'significantly reduced false positive sarcomere detections' (abstract, Section 4.1). The evidence is aggregate: mean sarcomere counts in low-score cells fall from 192 to 15 (Fig 4A-i), plus visual examples. This is not sufficient. A reduction in detected counts is equally consistent with a loss of true detections. The classifier uses a hand-set z-disc probability threshold of 0.3 (Section 3.3.1), the graph pruning is rule-based, and Section 3.4.2 removes all single-sarcomere myofibrils; any of these can suppress true z-discs and sarcomeres, especially in immature cells where true structures are punctate. The paper reports no precision, recall, or F1 comparison against manually annotated z-disc or sarcomere ground truth, nor any sensitivity analysis for the 0.3 threshold. The subsequent supervised and unsupervised results do not repair this: correlations of 0.77 and 0.79 with expert scores can be high even if detection is dominated by false negatives, because sparse low-score cells are easy to identify by low counts. The paper itself acknowledges in Section 4.3 that Otsu thresholding 'occasionally fails to detect z-discs that are visible to expert observers,' and since the DL classifier operates only on Otsu contours, this recall ceiling is inherited by all downstream features. Therefore the load-bearing premise that the new detections are more accurate, not merely fewer, is unverified.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript extends the SarcGraph framework for quantifying sarcomere organization in hiPSC-CM images by adding a deep learning-based z-disc classifier, a z-disc location correction step, an ensemble graph scoring method with probabilistic averaging, and a myofibril extension post-processing step. The modified pipeline is applied to roughly 7,000 expert-scored cells from the Allen Institute dataset, from which nine structural features per cell are extracted and used to train an SVR model that predicts organization scores with Pearson correlations of 0.85 on Train, 0.77 on Test FISH, and 0.79 on Test Live. The authors additionally apply ExKMC explainable clustering to assign low/medium/high organization labels. The central claim is that the modified pipeline significantly reduces false positive sarcomere detections in immature cells and detects longer myofibrils in mature samples.","tokens_in":26585,"tokens_out":4733,"duration_ms":39079,"significance":"If the central detection-accuracy claim were properly validated, this would be a valuable contribution to cardiac image analysis: it addresses an acknowledged limitation of the original SarcGraph, works on single images, extracts per-sarcomere features, and is open-source. The held-out test correlations and the explainable clustering results are encouraging evidence that the extracted features carry predictive signal. However, the current evidence does not establish that the pipeline detects more true structures rather than fewer structures overall; the missing ground-truth detection metrics are essential to assessing the practical value of the method.","major_comments":[{"comment":"The claim that the modified pipeline 'significantly reduced false positive sarcomere detections' is not validated against ground truth. The evidence is an aggregate drop in sarcomere counts in low-score cells (Figure 4A-i) and visual examples (Figure 4B). A reduction in detected counts is equally consistent with a loss of true detections, especially because the z-disc threshold of 0.3 (Section 3.3.1), the rule-based pruning, and the removal of single-sarcomere myofibrils (Section 3.4.2) can all suppress true positives. The paper reports no precision, recall, or F1 comparison against manual annotations of z-discs or sarcomeres, and no sensitivity analysis for the 0.3 threshold. Without such metrics, the headline improvement is unverified.","section":"4.1, Abstract"},{"comment":"The deep learning z-disc classifier is a central component, but its classification performance is never reported: there is no precision, recall, AUC, or confusion matrix on the manually labeled validation set. The 0.3 probability threshold and the other hand-set parameters (7-pixel distance, 22.5 degree angle, 60% intensity ratio) are introduced without justification or sensitivity analysis. Since all downstream features inherit the classifier's errors, the reader cannot determine whether the reported correlations reflect accurate detection or merely threshold-induced sparsity.","section":"3.2, 3.3.1"},{"comment":"The paper itself notes in Section 4.3 that 'the Otsu thresholding method occasionally fails to detect z-discs that are visible to expert observers due to variations in image brightness.' Because the deep learning classifier operates only on contours found by Otsu thresholding, this is a recall ceiling that affects every downstream feature. The authors should quantify how frequently Otsu misses visible z-discs, e.g., by comparing Otsu contours against manual annotations on a subsample, and discuss how the missed detections affect the SVR and clustering results. The current claim that the pipeline is more accurate is not supported while this ceiling is uncharacterized.","section":"4.3, Conclusion"},{"comment":"The statistical evidence for the supervised model is incomplete: the SVR hyperparameters (C=10, gamma=0.1) were selected on the Train set, so the Train correlation of 0.85 is not a predictive assessment, and the held-out test correlations are reported without confidence intervals, error bars, or significance tests. In addition, the comparison between SVR and decision-tree clustering in Table 1 treats the latter as a scoring method, but the cluster labels are evaluated against the same expert scores used to choose k and the feature thresholds; the claim that the unsupervised approach 'avoids the bias of expert-assigned organization scores' is therefore only partially true.","section":"4.3, Table 1"}],"minor_comments":[{"comment":"The sentence 'These modification significantly reduced...' contains a grammatical error; 'modification' should be plural.","section":"Abstract"},{"comment":"The phrase 'processing contour’s cropped region' is awkward; consider 'the cropped region centered on the contour being processed.'","section":"3.2.1"},{"comment":"The indicator function notation '1_{lmin < li < lmax}' is not previously defined; please define it explicitly.","section":"Eq. (1)"},{"comment":"There is a typo in the text: 'false positive sarcomere dection' should be 'detection.'","section":"4.1"},{"comment":"The transformation of continuous SVR scores into three categories uses thresholds at 2.33 and 3.67 without stating the rationale; please justify these boundaries.","section":"4.4, Figure 7"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is a preprint with open-source code, which is commendable. The main gap is the absence of detection-level ground-truth evaluation; without it, the central claim is unverifiable. The authors should be encouraged to add precision/recall analyses and sensitivity studies. The paper may be suitable for a specialized bioimage analysis journal or a methods journal after revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know two things before reading this one: the pipeline is a genuine improvement over the original SarcGraph for immature cells, and the main advertised benefit—fewer false-positive sarcomeres—is not actually validated against ground truth.\n\nWhat is actually new: they integrate a deep-learning z-disc classifier (SimCLR-trained EfficientNetV2 plus a DINOv2-MLP, averaged) into SarcGraph, replace the original edge scoring with a probabilistic ensemble of four scores, and add a myofibril-extension post-process. They then apply the full pipeline to the Allen Institute hiPSC-CM dataset and, for the first time, extract per-sarcomere features from it. The held-out results—Pearson correlations of 0.77 on Test FISH and 0.79 on Test Live for predicting expert scores—are credible, especially given the measured inter-rater noise (ICC 0.68, Pearson 0.65). The unsupervised explainable clustering with a two-feature decision tree is a nice touch and generalizes reasonably well.\n\nThe soft spots are concentrated in the detection-evaluation section. The paper claims the modification \"significantly reduced false positive sarcomere detections,\" but the evidence is an aggregate drop in mean sarcomere count from 192 to 15 in low-score cells plus visuals. That drop is equally consistent with losing true detections, and in immature cells with punctate z-bodies, over-pruning is a real risk. The z-disc classifier threshold (0.3), the myofibril extension thresholds (7 px, 22.5°, etc.), and the pruning rules are all hand-set without sensitivity analysis. The reader's stress-test is right: without precision/recall or F1 against manually annotated z-discs/sarcomeres, the load-bearing claim is unverified. The authors do acknowledge in Section 4.3 that Otsu thresholding occasionally misses z-discs visible to experts, and since the DL classifier only sees Otsu contours, that recall ceiling propagates downstream.\n\nAlso note the Train Pearson of 0.85 reflects tuning on the same set; the test correlations are the honest evidence, and they are moderate but meaningful. The paper would be substantially stronger with detection-level precision/recall on a few hundred annotated cells, error bars or confidence intervals on the feature statistics, and a threshold sensitivity analysis.\n\nThis is an addressable set of issues rather than a fatal flaw. The code and data are open, the work is reproducible, and the paper is honest about several limitations. I would send it to peer review, but I would push for the detection validation before acceptance.\n\nFor the reading group: it is worth a look if anyone works on cardiac image analysis or detection pipelines with weak ground truth, though the missing precision/recall will frustrate the stats-minded readers.","headline":"A real engineering improvement to SarcGraph with credible held-out score predictions, but the headline claim of fewer false positives is never directly measured.","tokens_in":27126,"tokens_out":1718,"would_cite":true,"duration_ms":17837,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Adding a deep-learning z-disc classifier and ensemble graph scoring to SarcGraph makes sarcomere detection reliable in immature hiPSC-CMs, recovers longer myofibrils, and yields features that predict expert organization scores and support…","keywords":["hiPSC-CM","SarcGraph","sarcomere detection","z-disc classification","deep learning","structural organization","explainable clustering","expert scoring bias"],"falsifier":"Apply the modified pipeline to a held-out set of immature hiPSC-CM images with expert-visible z-discs, and compare its z-disc and myofibril output against a version in which a human or a learned segmenter guarantees every expert-visible z-disc becomes a contour. If the guaranteed-contour version finds many more true myofibrils or much higher feature correlations with expert scores, then the claim that the deep-learning classifier fixes detection on top of Otsu is the part that fails.","tokens_in":26037,"feed_emoji":"🔬","tokens_out":8983,"duration_ms":75206,"temperature":0.7,"pith_summary":"The authors set out to make SarcGraph, a graph-based tool that detects individual sarcomeres in heart-cell images, work on immature and mature hiPSC-CMs alike. They find that the original pipeline over-reports z-discs in immature cells, where puncta and z-bodies are common, and under-detects long myofibril chains because of its local edge-scoring rule. Their fix is a deep-learning z-disc classifier that filters contour candidates plus a graph-scoring ensemble and a myofibril-extension step. On roughly 7,000 expert-scored cells from a public hiPSC-CM dataset, the modified pipeline cuts mean sarcomere counts in low-score cells from about 192 to 15, raises mean myofibril length in every score group, and produces per-sarcomere features that predict expert organization scores with Pearson correlations of 0.77 on held-out FISH images and 0.79 on held-out live images. These results give cardiac researchers a scalable, object-level measure of structural organization that can also flag bias in manual scoring.","feed_headline":"Deep-learning filter sharpens sarcomere detection in immature heart cells","feed_subtitle":"Per-sarcomere features predict expert organization scores and expose rater bias across thousands of hiPSC-CM images.","key_machinery":"The load-bearing object is the modified SarcGraph pipeline itself: a two-phase graph-based detector in which detected z-discs are nodes and potential sarcomeres are edges. The new machinery has two parts. A deep-learning z-disc classifier, an ensemble of a contrastively pretrained convolutional model and a vision-transformer feature extractor with a small classifier head, assigns each contour a probability of being a true z-disc; the 0.3 threshold replaces the original length-only filter. An ensemble graph-scoring scheme, combining the original edge score, pruning validity, average node z-disc probability, and global myofibril alignment through probabilistic ensemble averaging, plus a rule-based myofibril-extension post-process, replaces the single local edge score. The classifier supplies the probability inputs used both for filtering and for edge scoring, so the deep-learning step is the hinge of the whole pipeline.","core_discovery":"The central claim is that two targeted changes make SarcGraph reliable on immature as well as mature cells. First, every contour produced by Laplacian-of-Gaussian filtering and Otsu thresholding is scored by an ensemble of two deep-learning classifiers, and contours with predicted z-disc probability below 0.3 are discarded; the surviving contours are also re-centered at local intensity peaks. Second, the sarcomere graph is scored by four complementary methods, the original local edge score, pruning validity, node z-disc probabilities, and a global myofibril-alignment score, fused by probabilistic ensemble averaging, then post-processed by a myofibril-extension step that bridges breaks at crossings or missed z-discs. The authors report that in low-organization cells mean detected sarcomeres drop from 192 to 15 per cell, while mean myofibril length rises in every expert-score group, for example from 1.95 to 4.75 sarcomeres in low-score cells. The extracted per-sarcomere features predict expert organization scores with Pearson correlations of 0.85 on the training set, 0.77 on held-out FISH cells, and 0.79 on held-out live cells.","pith_inferences":["The same Otsu-first architecture means the next bottleneck is contour recall; replacing Otsu with a learned segmentation, such as the kind of U-Net used by prior tools, could let the classifier recover dim z-discs that are currently missed.","Because the two-feature decision tree labels cells without expert scores, it could serve as a weak label for training future models on the full roughly 31,000-cell dataset, not just the expert-scored subset.","The roughly 10 minutes per cell image makes full video-scale contraction analysis impractical; the ensemble scoring could be distilled into a single lightweight score or run on downsampled graphs to regain the speed of the original SarcGraph.","The same per-sarcomere features could be aligned with the transcript measurements in the public dataset to ask which genes correlate with the structural labels, something the paper's focus on image-derived scores leaves open."],"forward_implications":["Sarcomere count becomes a usable organization feature: low-score cells now yield near-zero detections instead of hundreds of false positives, so the feature separates low from high organization without expert input.","Myofibril length becomes a usable maturity readout: mean detected myofibril length rises in every score group, reaching maxima over 12 to 17 sarcomeres, so short-chain fragmentation is no longer conflated with immaturity.","Per-cell feature vectors from individual sarcomeres predict expert organization scores on unseen data, with Pearson r of 0.77 on test FISH and 0.79 on test live, making image-level scoring replaceable by an automated pipeline.","A two-feature decision tree using z-disc classification ratio and sarcomere density reproduces expert-based low, medium, and high categories with correlations comparable to or better than the supervised SVR on test data, giving a label-free, explainable alternative.","Feature-space analysis exposes systematic expert bias: medium- and high-score cells shift between training and test sets, and the model under-predicts test scores, so the pipeline can be used to audit manual scoring."],"supporting_citations":[{"why":"This is the original SarcGraph method that the paper modifies; it supplies the graph framework, z-disc detection, and the baseline for all comparisons.","marker":"[69]"},{"why":"Documents the original SarcGraph edge-scoring formula and pruning strategy that the ensemble method extends.","marker":"[41]"},{"why":"Contributed the public hiPSC-CM image dataset, expert organization scores, and train/test splits used for all supervised and unsupervised analyses.","marker":"[20]"},{"why":"Otsu thresholding is the contour-detection step whose failures motivate the deep-learning classifier and whose retention is the pipeline's fragile assumption.","marker":"[50]"},{"why":"SimCLR provides the contrastive representation learning used to train the first z-disc classifier on labeled and unlabeled contour crops.","marker":"[9]"},{"why":"DINOv2 supplies the pretrained feature extractor used by the second z-disc classifier, which requires no fine-tuning.","marker":"[49]"},{"why":"Probabilistic ensemble averaging is the fusion rule that combines the four graph-scoring methods into a single sarcomere probability.","marker":"[10]"},{"why":"ExKMC's explainable clustering method supplies the decision-tree algorithm used to group cells into low, medium, and high organization.","marker":"[18]"},{"why":"SarcApp is the prior deep-learning binarization approach that the authors contrast with their contour-level classifier.","marker":"[47]"},{"why":"CellProfiler produced the cell segmentations in the dataset, which define the cell-area feature used in normalization and clustering.","marker":"[7]"}],"fun_headline_variants":["AI boosts SarcGraph to spot sarcomeres in immature heart cells","Deep learning upgrades SarcGraph for immature cell analysis","SarcGraph gets deep learning, reveals per-sarcomere data and rater bias","Deep learning sharpens SarcGraph for stem cell heart muscle analysis"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"Everything downstream depends on Otsu thresholding and contour detection turning a real z-disc into a contour in the first place; the deep-learning classifier can only filter or re-center contours that already exist, and the paper notes that Otsu occasionally misses z-discs visible to expert observers because of image-brightness variation.","fun_headline_variants_meta":{"raw":{"variants":["AI boosts SarcGraph to spot sarcomeres in immature heart cells","Deep learning upgrades SarcGraph for immature cell analysis","SarcGraph gets deep learning, reveals per-sarcomere data and rater bias","Deep learning sharpens SarcGraph for stem cell heart muscle analysis"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000616,"raw_usage":{"total_tokens":2946,"prompt_tokens":1114,"completion_tokens":1832,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":730,"completion_tokens_details":{"reasoning_tokens":1754}},"tokens_in":730,"tokens_out":1832,"duration_ms":13159,"temperature":1.0,"reasoning_tokens":1754,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T22:45:27.234070+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Apply the modified pipeline to a held-out set of immature hiPSC-CM images with expert-visible z-discs, and compare its z-disc and myofibril output against a version in which a human or a learned segmenter guarantees every expert-visible z-disc becomes a contour. If the guaranteed-contour version finds many more true myofibrils or much higher feature correlations with expert scores, then the claim that the deep-learning classifier fixes detection on top of Otsu is the part that fails.","supporting_citations":[{"cited_title":"Sarc-graph: Automated segmentation, tracking, and analysis of sarcomeres in hipsc-derived cardiomyocytes","cited_arxiv_id":null,"evidence_quote":"This is the original SarcGraph method that the paper modifies; it supplies the graph framework, z-disc detection, and the baseline for all comparisons."},{"cited_title":"Sarcgraph: A python package for analyzing the contractile behavior of pluripotent stem cell-derived cardiomyocytes","cited_arxiv_id":null,"evidence_quote":"Documents the original SarcGraph edge-scoring formula and pruning strategy that the ensemble method extends."},{"cited_title":"Cell states beyond transcriptomics: Integrating structural organization and gene expression in hipsc-derived cardiomyocytes","cited_arxiv_id":null,"evidence_quote":"Contributed the public hiPSC-CM image dataset, expert organization scores, and train/test splits used for all supervised and unsupervised analyses."},{"cited_title":"A simple framework for contrastive learning of visual representations","cited_arxiv_id":null,"evidence_quote":"SimCLR provides the contrastive representation learning used to train the first z-disc classifier on labeled and unlabeled contour crops."},{"cited_title":"Multimodal object detection via probabilistic ensembling","cited_arxiv_id":null,"evidence_quote":"Probabilistic ensemble averaging is the fusion rule that combines the four graph-scoring methods into a single sarcomere probability."},{"cited_title":"ExKMC: Expanding Explainable $k$-Means Clustering","cited_arxiv_id":"2006.02399","evidence_quote":"ExKMC's explainable clustering method supplies the decision-tree algorithm used to group cells into low, medium, and high organization."},{"cited_title":"Independent regulation of z-lines and m-lines during sarcomere assembly in cardiac myocytes revealed by the automatic image analysis software sarcapp","cited_arxiv_id":null,"evidence_quote":"SarcApp is the prior deep-learning binarization approach that the authors contrast with their contour-level classifier."},{"cited_title":"Cellprofiler: image analysis software for identifying and quantifying cell phenotypes","cited_arxiv_id":null,"evidence_quote":"CellProfiler produced the cell segmentations in the dataset, which define the cell-area feature used in normalization and clustering."}],"review_version":1}