{"id":"037e1e45-20f4-4e40-b6c8-327e7d2a8f81","arxiv_id":"2507.00903","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A deep learning model segments myocardial T1/T2 maps with accuracy comparable to human experts, and a random forest combining multiple statistical features improves automated disease detection.","lead":"The paper tests whether a deep learning model can outline the heart muscle on T1/T2 MRI maps as well as human experts, and whether combining several statistical features with machine learning improves detection of heart disease. In a single-center study of 144 patients, the automated segmentation matched expert agreement and a random forest classifier reached an F1-score of 92.7%.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Disease-detection claim is confounded by circular labeling: the reference standard includes T1/T2-visible oedema/fibrosis while the classifier is trained on T1/T2 features, so the reported F1=92.7% may reflect inclusion criteria rather than independent diagnostic accuracy.","rationale":"The reader's weakest assumption is exactly the load-bearing concern I identify: the reference labels are not independent of the input features. The paper's own inclusion criteria for 'normal' and 'myocarditis' incorporate T1/T2 mapping and LGE findings, and the classifier uses T1/T2 map statistics. Therefore the reported F1 of 92.7% may largely reflect reconstruction of the label definition rather than true diagnostic performance. This concern is concrete, located in the Methods/Data section, and is not a matter of outside consensus; it is an internal consistency problem between the stated label definitions and the features used. The segmentation claim, by contrast, is better supported: DICE and MAPE comparisons to two observers, Pearson correlations, and Bland-Altman analyses provide convergent evidence that automated contours are close to manual ones, and the model-observer agreement exceeding inter-observer agreement is a meaningful result. However, the segmentation result does not rescue the disease-detection claim, because the classification evaluation inherits the label-dependence problem. The paper also has secondary weaknesses—small test set (n=29), potential selection of the best feature combination based on test performance, and a pooled disease group that obscures subgroup differences—but the label circularity is the most fundamental. A re-analysis with an independent reference standard is feasible using the existing dataset and would settle whether the ML improvement over single cutoffs is real. I therefore agree with the reader's conditional verdict: the manuscript should not be fully accepted without addressing the reference-standard issue, but the segmentation contribution and the methodological framing have sufficient merit to warrant revision rather than rejection.","tokens_in":11516,"tokens_out":5790,"duration_ms":73126,"concrete_test":"Recompute the random forest classification from Table 5 (bottom) using an independently defined reference standard: (i) define myocarditis by clinical criteria alone, without requiring oedema or LGE; (ii) define normal as consecutive patients without known cardiac disease, not selected for normal T1/T2 maps; and (iii) keep sarcoidosis/systemic-disease patients only when the extra-cardiac diagnosis is confirmed independently of CMR. Report F1, precision, and recall on the same 29-subject test split. If the F1 drops materially below 92.7%, the original claim is inflated by label circularity; if it remains high, the concern is refuted. In addition, report per-subgroup ROC/AUC for each T1/T2 feature to show where the discriminative signal originates.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central disease-detection claim depends on the disease label being informative independently of the T1/T2 features used by the classifier. That independence fails in the paper's own definitions. In the Methods/Data section, the 'normal cardiac MRI' group is defined to have 'no fibrosis or oedema according to T1/T2 mapping or LGE imaging,' and the 'myocarditis' group requires '≥1 cardiac MRI abnormality (myocardial oedema or LGE with myocarditis pattern).' Thus the binary label 'diseased' versus 'normal' is constructed, at least in part, from the same T1/T2 mapping abnormalities that the random forest uses as inputs. A classifier can achieve high F1 by learning to reproduce the oedema/fibrosis criterion embedded in the label, rather than by detecting disease independent of CMR. The sarcoidosis and systemic-disease subgroups were included regardless of myocardial involvement and are therefore potentially independent, but they are pooled with myocarditis in the binary task, so their contribution cannot be separated. As a result, the headline F1=92.7% is best interpreted as an upper bound on detecting CMR-defined abnormality, not as evidence of automated clinical disease detection. Because the label definition affects every reported classifier comparison, this is the most load-bearing weakness in the paper.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript evaluates a DenseUNet for automatic segmentation of the left-ventricular blood pool and myocardium on native/post-contrast T1 and T2 maps, pretrained on public T1-weighted data and fine-tuned on 144 subjects. On a 261-map test set annotated by two observers, the model's average myocardial Dice score (85.4%) exceeds inter-observer agreement (81.6%), with mean absolute percentage error of 1.75% in T1/T2 values. From the automatic masks, the authors extract average, lower-quartile, median, and upper-quartile T1/T2 features and compare Youden-optimal single-feature cutoffs with five machine-learning classifiers for a binary normal-versus-diseased task; a random forest using all features achieves F1=92.7% and recall=100%, reported as significantly better than the cutoff approach. The authors conclude that deep learning facilitates segmentation and that combining multiple statistical features with machine learning improves disease detection. The segmentation claim is well supported by the evaluation; the disease-detection claim is weakened by a reference-standard definition that overlaps with the classifier inputs and by the very small test set.","tokens_in":11808,"tokens_out":5944,"duration_ms":68849,"significance":"If the segmentation result is taken on its own, the paper makes a credible and useful contribution: model-observer Dice exceeding inter-observer Dice across all map types, with Bland-Altman and Pearson correlation analyses showing that automatic contours have minimal impact on feature quantification, is a meaningful benchmark for automated T1/T2 mapping pipelines. The speed of the pipeline (0.4 s per patient) is also practically relevant. The machine-learning classification comparison, however, does not currently provide valid evidence for the headline claim of improved disease detection, because the 'diseased' label is partly defined by the same T1/T2 mapping abnormalities that the classifier uses as features. The feature-combination idea (quartile statistics plus ML) is worth testing, but the present evaluation conflates label construction with diagnostic accuracy. With an independent label definition and appropriate uncertainty quantification on a larger test set, the classification component would be significant; as it stands, the contribution is mainly the segmentation evaluation.","major_comments":[{"comment":"The binary label 'diseased' is not independent of the classifier features. The 'normal cardiac MRI' group is defined as having 'no fibrosis or oedema according to T1/T2 mapping or LGE imaging,' and the 'myocarditis' group requires at least one cardiac MRI abnormality, including myocardial oedema, which is directly assessable from T1/T2 mapping. Because the random forest in Table 5 is trained on native T1/T2 features, the reported F1=92.7% and recall=100% may largely reflect the classifier reproducing the oedema/fibrosis criterion embedded in the label rather than detecting disease independently of CMR. The sarcoidosis and systemic-disease subgroups are labeled without requiring myocardial involvement and are therefore potentially independent, but they are pooled with myocarditis, so their contribution cannot be separated. Please re-run the classification with labels that do not incorporate T1/T2 mapping findings, or, if that is not possible, report subgroup analyses for sarcoidosis and systemic disease separately and explicitly reframe the claim as detection of CMR-defined abnormality rather than disease detection.","section":"Data and Disease Detection (pp. 7-10)"},{"comment":"The classification evaluation rests on only 29 test patients (10 normal, 19 diseased from Table 1), yet F1, precision, and recall are reported as point estimates without confidence intervals, and the Wilcoxon comparisons between classification approaches are not described in terms of the observations entering the test. With 29 patients, one or two reclassified cases materially change F1; moreover, the 'best' feature combination for each classifier was selected on a 15-patient validation subset, so the reported test performance is conditional on that selection. Please report per-patient predictions, bootstrap or exact binomial confidence intervals, and a precise description of the statistical test (e.g., McNemar on paired predictions or a permutation test over patients), and clarify how the p-values in Table 5 were computed.","section":"Table 5 and Statistical Analysis (pp. 12-16)"},{"comment":"The claim that machine learning improves disease detection over single cutoffs is based on comparing the best-performing random forest (selected after trying five classifiers and several feature combinations) with the best threshold method (T1 UQ), rather than a pre-specified comparison. Because multiple configurations were examined, the reported p-values do not account for selection or multiple comparisons. Please report the full set of configurations tried, or use a nested cross-validation or a pre-registered decision rule, so that the 'ML improves over cutoff' claim is not an artifact of best-of-many selection.","section":"Feature selection and comparison of methods (pp. 11-16)"}],"minor_comments":[{"comment":"The MAPE formula as printed lacks the absolute value; it should be |G - M| / G x 100, otherwise negative errors are possible and the metric is not a true percentage error.","section":"Eq. 3, Statistical Analysis"},{"comment":"The text says 'flip angle 35° for T1 and 70° for T1'; the second should presumably be '70° for T2'.","section":"Data acquisition, p. 7"},{"comment":"In the sentence listing T2 best performers, 'T1 A (AUC = 62.6%)' appears to be a typo for 'T2 A (AUC = 62.6%)'.","section":"Results, p. 15"},{"comment":"The reported 'average DICE of 85.4% ± 1.6%' is not directly shown in Table 3; please clarify that it is the mean of the Model-vs-Observer1 and Model-vs-Observer2 myocardial Dice values, and state the corresponding standard deviation over the same units.","section":"Table 3, Mapping Segmentation"},{"comment":"Please specify how many test patients have both native T1 and T2 maps, since the random forest using 'all features' requires T2 features for every patient; if any values are missing, describe the imputation or exclusion rule.","section":"Feature Analysis and Disease Detection, p. 11"}],"recommendation":"major_revision","confidential_remarks":"The circularity concern is substantive and load-bearing for the disease-detection claim; the segmentation portion is solid and could stand on its own. I recommend major revision rather than rejection because the issue is addressable through reanalysis with independent labels (or explicit subgroup reporting) and by reframing the claim, and because the segmentation evaluation is a real contribution. The small test set and selection procedure also need to be reported honestly, but those are fixable within the manuscript's scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the segmentation evaluation is the real content of this paper and it is good; the disease-detection numbers should not be taken at face value because the label definition and the classifier inputs overlap. The F1=92.7% is likely measuring how well the random forest reproduces the CMR-based inclusion criteria, not independent diagnostic ability.\n\nWhat the paper does well: the DenseU-Net segmentation is evaluated against two independent observers on a held-out set, with model-observer DICE (85.4%) exceeding inter-observer DICE (81.6%). That's a clean, reproducible result, and the MAPE and Bland-Altman plots support it. The authors also check that features extracted from automatic vs manual masks correlate strongly (PCC > 0.9), which is the right kind of sensitivity analysis. The segmentation claim is independent of the classification issue.\n\nThe problem is the classification. The normal group is defined by 'no fibrosis or oedema according to T1/T2 mapping or LGE', and the myocarditis group requires '≥1 cardiac MRI abnormality (myocardial oedema or LGE)'. Since the classifier uses T1/T2 features, the binary label is partly constructed from the same signal. So the AUCs and the F1 gain over cutoff thresholds are inflated. You can't separate how much is detecting disease from how much is detecting the inclusion criterion. The sarcoidosis and systemic-disease groups were included regardless of myocardial involvement, so they could have been an independent test, but they're pooled into the same 'diseased' class, so their signal is lost.\n\nAdditional soft spots: the test set is 29 patients; the best feature combination per classifier is selected on the test set and then reported as the result; the Wilcoxon comparisons are underpowered and multiple tests are unadjusted. These are all fixable in revision, but together they lower confidence in the reported F1.\n\nI'd send this to peer review. The segmentation work is worth publishing, and the classification section can be salvaged if the authors either restrict the claim to 'detecting CMR-defined abnormality' or re-analyze with labels defined without T1/T2 mapping. I wouldn't cite the disease-detection number, but I might cite the segmentation result.","headline":"Solid segmentation evaluation, but the disease-detection claim is circular because the reference standard is partly defined by the same T1/T2 features the classifier uses.","tokens_in":12333,"tokens_out":2561,"would_cite":true,"duration_ms":30121,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Automated myocardial T1/T2 map segmentation beats expert inter-observer agreement, and combining four statistical features per map in a random forest lifts disease detection to F1 = 92.7% from 66.7% for single cutoffs.","keywords":["myocardium","deep learning","T1 mapping","T2 mapping","cardiac MRI segmentation","random forest","disease detection","parametric mapping"],"falsifier":"Re-classify the same 29 test subjects after re-defining the disease label without any cardiac MRI information, for example biopsy-confirmed myocarditis and biopsy-confirmed sarcoidosis only, then retrain the random forest on the same eight T1/T2 features and compare the test-set F1. If it drops near the 66.7% single-cutoff level, the reported gain reflects label-feature overlap rather than diagnostic ability. A second complementary check is a multi-center evaluation, since T1/T2 values shift across scanners and protocols and the paper itself identifies per-center recalibration as necessary.","tokens_in":11362,"feed_emoji":"🫀","tokens_out":10519,"duration_ms":113668,"temperature":0.7,"pith_summary":"Manual contouring of myocardial T1 and T2 maps is time-consuming and observer-dependent, and clinical decisions usually turn on a single average relaxation time compared with a fixed cutoff. This paper tests whether a deep learning model can replace the manual contouring and whether a machine-learning classifier can replace the single cutoffs. On 261 held-out T1/T2 maps, the segmentation model outlines the myocardium with an average Dice score of 85.4%, above the 81.6% agreement between two expert observers, while changing the extracted T1/T2 values by only about 1.75% on average. Feeding the lower quartile, median, upper quartile, and average of native T1 and T2 myocardial values into a random forest raises disease detection to F1 = 92.7% with 100% recall on the test set, significantly better than the best single-feature cutoff (F1 = 66.7%, $p<0.001$). If the results hold, automated analysis of T1/T2 maps could remove one source of variability from cardiac MRI and replace one-number thresholds with a more informative feature combination.","feed_headline":"Automated heart-map pipeline beats expert agreement and single cutoffs","feed_subtitle":"A deep learning model outlines heart muscle better than two experts agree, and combined features lift disease detection to 92.7% F1.","key_machinery":"The central object is the pair of myocardial masks produced by the DenseUnet segmentation model on native T1 and T2 maps, from which four statistics are computed per slice: the average, lower quartile, median, and upper quartile of myocardial pixel relaxation times, then averaged over slices per patient. DenseUnet is a U-Net with dense convolutional blocks, pre-trained on 10,065 T1-weighted images and fine-tuned on 877 T1/T2 maps, and the whole pipeline runs in about 0.4 seconds per patient. These eight per-patient features are the entire input to the disease classifiers. The mechanism carrying the argument is the contrast between using each feature alone through a Youden-J-optimized cutoff and using all features together in a standard machine-learning classifier: that comparison isolates feature combination, rather than segmentation quality or algorithm choice, as the source of the F1 gain.","core_discovery":"The paper claims that a fully automated segment-and-classify pipeline is feasible and diagnostically useful for inflammatory and infiltrative myocardial disease. Its first claim is segmentation quality: a DenseUnet pre-trained on T1-weighted images and fine-tuned on T1/T2 maps yields a myocardial Dice score of 85.4% overall, exceeding the 81.6% Dice between the two independent expert observers on the same test set, and the model's extracted T1/T2 features correlate with manual features above 0.9. The second claim is that disease detection improves when multiple statistical features are combined: on the held-out test subset, every single-feature threshold performs poorly (best F1 66.7%), while a random forest trained on all eight native T1/T2 features (average, lower quartile, median, upper quartile for each map) reaches F1 = 92.7%, precision 86.4%, and recall 100%, a significant improvement over the cutoff baseline ($p<0.001$). The paper also claims classifier choice matters little, since logistic regression, SVM, Perceptron, and random forest all exceed F1 = 85%, so the gain comes from combining features rather than from the particular algorithm.","pith_inferences":["If the quartile results transfer, disease may appear first in the tail of the myocardial relaxation-time distribution rather than its center: the T1 upper-quartile feature had the highest single-feature AUC (75.2%), so histogram or voxel-level features are a natural extension beyond four summary statistics.","The 100% recall figure comes from a 29-subject test set and a label definition that already includes cardiac MRI abnormalities; in a larger, biopsy-defined cohort the precision-recall balance would likely shift, and the paper's own single-center, single-scanner limitation should be read as a bound on generalizability.","Because T2 features underperformed despite better T2 segmentation, improving T2 map signal-to-noise, through acquisition or post-processing, may do more for disease detection than further improving contour accuracy.","The same segment-then-classify recipe could be applied to post-contrast T1 maps or extracellular-volume maps, which the paper leaves as future work; whether quartile features help there is untested."],"forward_implications":["A myocardium contour from the model overlaps a second expert's contour about as well as two experts overlap each other, so manual delineation of T1/T2 maps can be replaced or cross-checked automatically.","Using all four statistics from native T1 and T2 maps in a random forest yields 100% recall on the 29-subject test set, meaning no known diseased subject was missed, with precision 86.4%.","Single-feature cutoffs are the weak link: the best T1 quartile cutoff reaches only F1 = 66.7%, while the same features combined in a logistic regression already reach F1 = 90.0%.","The pipeline runs in about 0.4 seconds per patient, so adding it to the reading workflow costs essentially no time."],"supporting_citations":[{"why":"Consensus criteria defining the myocarditis group as clinical suspicion plus oedema or late-gadolinium-enhancement abnormality; this sets the label the classifier is trained and tested to predict.","marker":"[21]"},{"why":"Defines the sarcoidosis subgroup included as diseased and reports abnormal T1/T2 mapping values in these patients independent of LGE, the rationale for counting sarcoidosis as a disease class.","marker":"[16]"},{"why":"Supplies the 10,065 T1-weighted images with public annotations used to pre-train the segmentation network before fine-tuning on T1/T2 maps.","marker":"[22]"},{"why":"Provides the dense-connectivity and U-Net architecture of which the DenseUnet segmentation model is built.","marker":"[23, 24]"},{"why":"Describes maximizing Youden's J on ROC curves, the method used to set the single-feature cutoffs that serve as the baseline classification approach.","marker":"[25, 26, 27]"},{"why":"Gives the nonparametric test used to compare feature AUCs and to judge which features separate diseased from healthy subjects.","marker":"[28]"},{"why":"Reports an end-to-end cine-MRI segmentation-plus-classification pipeline whose roughly 92-93% accuracy is the closest comparison for the F1 = 92.7% result.","marker":"[31]"},{"why":"Represent the conventional average-value-plus-single-cutoff approach to T1/T2 disease classification, the clinical baseline the machine-learning approach is compared against.","marker":"[14, 15, 16, 17]"}],"fun_headline_variants":["AI heart-map segmentation beats expert agreement","Combining MRI features boosts heart disease detection to 93% F1","Deep learning outdoes manual heart-mapping and single cutoffs","Automated cardiac maps: better than experts, better than cutoffs"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the reference \"diseased\" labels are independent of the input features: myocarditis was defined by clinical criteria plus at least one cardiac MRI abnormality (oedema or late gadolinium enhancement), and sarcoidosis was included on the strength of abnormal T1/T2 mapping values, so a classifier fed T1/T2 maps may partly be predicting its own label.","fun_headline_variants_meta":{"raw":{"variants":["AI heart-map segmentation beats expert agreement","Combining MRI features boosts heart disease detection to 93% F1","Deep learning outdoes manual heart-mapping and single cutoffs","Automated cardiac maps: better than experts, better than cutoffs"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000256,"raw_usage":{"total_tokens":1661,"prompt_tokens":1118,"completion_tokens":543,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":734,"completion_tokens_details":{"reasoning_tokens":474}},"tokens_in":734,"tokens_out":543,"duration_ms":6592,"temperature":1.0,"reasoning_tokens":474,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T21:04:25.725339+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-classify the same 29 test subjects after re-defining the disease label without any cardiac MRI information, for example biopsy-confirmed myocarditis and biopsy-confirmed sarcoidosis only, then retrain the random forest on the same eight T1/T2 features and compare the test-set F1. If it drops near the 66.7% single-cutoff level, the reported gain reflects label-feature overlap rather than diagnostic ability. A second complementary check is a multi-center evaluation, since T1/T2 values shift across scanners and protocols and the paper itself identifies per-center recalibration as necessary.","supporting_citations":[{"cited_title":"Eur Heart J DOI: 10.1093/eurheartj/eht210","cited_arxiv_id":null,"evidence_quote":"Consensus criteria defining the myocarditis group as clinical suspicion plus oedema or late-gadolinium-enhancement abnormality; this sets the label the classifier is trained and tested to predict."},{"cited_title":"Circ Cardiovasc Imaging DOI: 10.1161/CIRCIMAGING.116.005022","cited_arxiv_id":null,"evidence_quote":"Defines the sarcoidosis subgroup included as diseased and reports abnormal T1/T2 mapping values in these patients independent of LGE, the rationale for counting sarcoidosis as a disease class."},{"cited_title":"Available via https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/N1R1Q4","cited_arxiv_id":null,"evidence_quote":"Supplies the 10,065 T1-weighted images with public annotations used to pre-train the segmentation network before fine-tuning on T1/T2 maps."},{"cited_title":"Biometrics 837- 845","cited_arxiv_id":null,"evidence_quote":"Gives the nonparametric test used to compare feature AUCs and to judge which features separate diseased from healthy subjects."},{"cited_title":"Statistical Atlases and Computational Models of the Heart (Quebec City) p 120-129","cited_arxiv_id":null,"evidence_quote":"Reports an end-to-end cine-MRI segmentation-plus-classification pipeline whose roughly 92-93% accuracy is the closest comparison for the F1 = 92.7% result."}],"review_version":1}