Pith. sign in

REVIEW 3 major objections 4 minor 17 references

Left Ventricle Quantification Using Direct Regression with Segmentation Regularization and Ensembles of Pretrained 2D and 3D CNNs

T0 review · 3 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash

Pith's one-line read The paper claims that direct regression with pretrained CNNs, segmentation regularization, and ensemble search can quantify left-ventricle indices from a small cardiac MRI dataset, reaching 111 ± 76 mm² area error on cross-validation…

desk verdict Honest challenge report with solid transfer-learning gains, but the ensemble-search validation leaks selection bias and the CV-to-test drop undercuts the robustness claim. read the letter →

arxiv 1908.04181 v1 pith:UJ5KHGEZ submitted 2019-08-12 eess.IV cs.CV

classification eess.IVcs.CV
keywords leftventriclequantificationcardiacMRIdirectregressiontransferlearningsegmentationregularizationensembleselection3DCNNspatio-temporal
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that direct regression with pretrained convolutional networks can quantify the left ventricle from cardiac MRI even when the dataset is very small (56 patients). The method combines ImageNet-pretrained 2D and 3D CNNs, a segmentation-based regularizer that adds a segmentation loss through a decoder branch during training, and an exhaustive search that selects the best ensemble of models for each output task. On the LVQuan19 training set, 5-fold cross-validation gives mean absolute errors of 111 ± 76 mm² for area, 1.84 ± 0.9 mm for dimensions, and 1.22 ± 0.6 mm for regional wall thickness, with 6.7% cardiac-phase classification error. The paper also reports that the same ensemble on the challenge test set gives substantially larger errors (371 mm² area, 3.02 mm dimension, 2.53 mm RWT, 11.5% phase), which it attributes to the test set being more challenging and different from the training distribution.

What carries the argument

Three devices carry the argument. Pretrained initialization transfers ImageNet image-classification features: every 2D backbone starts from ImageNet weights, and the 3D variants are built by copying each 2D kernel across the temporal depth and multiplying by 1/dc, which makes parameter-heavy 3D spatio-temporal CNNs trainable on 56 patients. Segmentation-based regularization attaches a decoder before global average pooling; a cross-entropy loss on the predicted myocardium and cavity maps backpropagates through the whole network, so the shared representation is pulled toward anatomy while the regression heads read out areas, dimensions, wall thicknesses, and phase. Ensemble search exhaustively evaluates all subsets of the top-20 configurations and selects, per task, the subset whose averaged predictions minimize error on the cross-validation folds, with multi-crop evaluation covering each 300×300 image.

What would settle it

Run the selected ensemble on a held-out test set of similar cardiac MR images and compare errors: if they stay close to the cross-validation range (area ≈ 111 mm², dimension ≈ 1.84 mm, RWT ≈ 1.22 mm, phase ≈ 6.7%), the claim generalizes; if they reproduce the reported test-set errors (371 mm², 3.02 mm, 2.53 mm, 11.5%), the cross-validation-based claim fails to transfer.

Watch

Extended reading notes

Core claim

Under the LVQuan19 challenge conditions, the paper's central claim is that the best way to exploit a small, high-resolution cardiac MRI set is to start from ImageNet-pretrained 2D classifiers, extend selected ones to 3D by copying 2D kernels into 3D kernels and rescaling by 1/dc, force the shared features to also predict myocardium and cavity segmentations through a decoder branch, and then average the task-specific subset of models selected by exhaustive ensemble search. With that recipe, the per-slice predictions are within 111 ± 76 mm² for cavity and myocardium area, 1.84 ± 0.9 mm for the three cavity dimensions, and 1.22 ± 0.6 mm for the six regional wall thicknesses, and the cardiac phase is wrong in 6.7% of slices under cross-validation. The authors are careful to state that the segmentation mask is used only as a regularizer, never as an explicit input at inference, and that the ensemble-selection step was checked on an extra held-out split to guard against overfitting the cross-validation folds. They also report the test-set results explicitly as being substantially worse, concluding that generalizable LV quantification remains open.

Load-bearing premise

The reported accuracy rests on the assumption that the 56-patient cross-validation folds are representative of the challenge test distribution, an assumption the paper's own test-set numbers contradict.

Editorial extensions

If this is right

  • Pretrained weights provide the largest single gain: area MAE drops from 199 mm² with random initialization to 139 mm² with ImageNet initialization on DenseNet121 2D.
  • Adding segmentation regularization improves area and dimension regression significantly over the same architecture without it, even though the predicted segmentation is not used at inference.
  • 3D spatio-temporal CNNs initialized by copying 2D kernels outperform their 2D counterparts on area and dimension regression despite the small dataset.
  • Selecting a task-specific ensemble subset by exhaustive search beats averaging all models, and the gain persists on a separate evaluation split.
  • The sizeable drop from cross-validation to test-set performance shows the challenge test set differs from the training distribution, so the practical gain on unseen data is smaller than the cross-validated numbers suggest.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same 2D-to-3D weight-copying initialization could be applied to other spatio-temporal medical imaging tasks with scarce annotated volumes, such as cine CMR motion analysis or 4D imaging; the paper demonstrates it only for 20-frame cardiac slice sequences.
  • Because the segmentation mask is used only as a loss, the method cannot correct segmentation errors at inference; adding a contour-guidance or segmentation-readout path might narrow the training-to-test gap.
  • The ensemble search selects subsets per task, and the paper does not report the total training cost of the roughly twenty configurations; that cost could be a practical barrier for replication.
  • The large test-set gap suggests that test-time adaptation or intensity-domain matching to the training distribution may recover part of the lost performance; this is testable with the same models on the challenge test split.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The manuscript proposes a method for automatic left ventricle (LV) quantification from cardiac MRI in the setting of the small LVQuan19 challenge dataset. The method combines pretrained 2D CNNs and 3D spatio-temporal CNNs initialized from ImageNet weights, an architecture-independent segmentation decoder used as a regularizer, and a search scheme that selects an optimal ensemble of trained configurations for each regression/classification task. The authors report patient-level 5-fold cross-validation results on the 56-patient training set, with mean absolute errors of 111 ± 76 mm² for area, 1.84 ± 0.9 mm for dimensions, 1.22 ± 0.6 mm for regional wall thickness, and a 6.7% cardiac phase classification error for the selected ensemble. They also report LVQuan19 test-set results, which are substantially worse, and compare against a multi-task learning baseline.

Significance. If the cross-validation results are taken at face value, the paper makes several useful empirical contributions: it demonstrates that ImageNet transfer learning helps substantially in a very small cardiac MRI dataset, that segmentation-based regularization can improve direct regression, and that 3D spatio-temporal variants initialized from 2D weights are competitive with 2D models. The ensemble-selection idea is relevant to challenge settings, and the authors make an explicit effort to evaluate selection overfitting with an additional split. The experimental design is mostly careful, including patient-level 5-fold CV, Wilcoxon signed-rank tests, and a broad comparison of architectures. However, the central claim that the ensemble search robustly improves over simple averaging is weakened by a selection-leakage flaw in the evaluation protocol, and the statistical evidence for the improvement is incomplete. The test-set degradation is honestly acknowledged but leaves the external validity of the method open.

major comments (3)
  1. [§2.2 (Ensembling) and Table 1 (starred rows)] The additional test split does not remove the selection leakage from the ensemble evaluation. The pool S of top-20 configurations is ranked using the concatenated out-of-fold predictions ŷ_i = ∪_j ŷ_ij, which include the second (evaluation) portion of each fold. Consequently, a configuration that performs well on the evaluation portion by chance is more likely to enter S, and then can be selected by the subset search run on the first portion. Therefore the 'Ensemble Optimal*' row in Table 1 is not an unbiased estimate of the ensemble-selection strategy, and the Wilcoxon test comparing 'Ensemble Optimal*' with 'Ensemble Average*' does not establish that the search scheme generalizes. Please redo this evaluation by ranking the pool using only the first-portion predictions from each fold, or by performing a fully nested cross-validation in which both pool ranking and subset selection are done inside the training folds only.
  2. [§3 (Results)] The claim that the optimal ensemble is significantly better than averaging is load-bearing for the paper's main contribution, but no p-values, test statistics, or effect sizes are reported for the comparisons between 'Ensemble Average' and 'Ensemble Optimal' or for the starred split evaluations. The multiple comparisons across the four tasks also warrant correction or at least explicit discussion. Please provide the full test results for both the original CV and the split evaluation, or remove the significance claims.
  3. [Table 1 and Discussion] The test-set results (371 mm² for area, 3.02 mm for dimensions, 2.53 mm for RWT, 11.5% phase error) are two to three times worse than the cross-validation estimates, which directly undercuts the abstract's emphasis on improving robustness through the proposed approach. The discussion acknowledges the gap, but it does not provide an analysis of whether the degradation is due to the ensemble search, the transfer-learning strategy, or a fundamental distribution shift. Please either temper the abstract's robustness claim or add a per-patient and per-fold analysis of the test-set errors that would allow the reader to attribute the gap.
minor comments (4)
  1. [§3] There are several typos: 'performce' should be 'performance', 'averging' should be 'averaging', and 'statically significant' should be 'statistically significant'. In §4, 'statisticially' should also be corrected.
  2. [§2.2 (Ensembling)] The number of configurations in the full set S and the relationship between the top-20 pool and the full configuration space are not stated. Please report these numbers, as they affect the computational cost and the interpretation of the ensemble-search bias.
  3. [Table 1] For the starred rows, the number of patients in the first and second portions of the additional split is not given. Please state the split sizes so the reader can judge the stability of the ensemble-selection results.
  4. [All] The manuscript does not state whether code or trained models will be released. Given the small dataset and the complexity of the ensemble search, providing code or detailed prediction files would substantially improve reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the reported results are empirical cross-validated measurements, and the acknowledged model-selection caveat is a statistical bias, not a circular derivation.

full rationale

The paper makes no first-principles derivation of LV indices; all numbers are supervised-learning errors on held-out folds with standard MSE and cross-entropy losses. No target quantity is used to define a fitted parameter: the segmentation decoder is an auxiliary loss against ground-truth masks, and the ensemble subset is selected by minimizing error on CV predictions, making the non-starred 'Ensemble Optimal' row an optimization value rather than an unbiased estimate, but this is not a derivation that reduces to its own inputs. The paper explicitly flags this limitation: 'Our proposed strategy potentially leads to implicit overfitting of the subset choice to the CV sets. Therefore, results reported for the CV sets might overestimate the performance gain of our ensembling strategy.' The extra split it introduces is not fully clean because 'S only includes our top 20 configurations, ranked by individual CV performance' and that ranking uses the full out-of-fold predictions, so the second evaluation portion can influence pool membership; however, this is a selection-leakage or correctness issue, not circularity in which a claimed prediction is equivalent by construction to its input. The self-citations [2,9] support transfer learning and multi-crop evaluation but are peripheral and not load-bearing, and no uniqueness theorem or derivation is imported from same-author prior work. Thus no circular step is exhibited.

Assumptions & free parameters 6 free parameters · 3 assumptions · 0 invented entities

The central claims depend on several hand-chosen hyperparameters and on the transferability of ImageNet features and segmentation masks to small cardiac MRI data. No new physical or conceptual entities are introduced; the free parameters listed are standard training choices that are tuned on the evaluation folds, with limited external validation.

free parameters (6)
  • segmentation loss weight λS = 0.1
    Chosen by hand in Section 2.2 (Training); balances the auxiliary cross-entropy loss against the regression loss.
  • phase classification loss weight λP = 0.05
    Hand-selected in Section 2.2 (Training).
  • learning rate = 1e-4
    Adam optimizer learning rate; no tuning procedure is reported.
  • batch size = 8
    Small batch chosen to induce more variation during training, as stated in Section 2.2 (Data Augmentation).
  • 3D slice count NS = 3, 5, 7, 10
    Four options considered for 3D CNNs; the final models use one of these values, but no separate validation set is described for this choice.
  • augmentation ranges = rotation [0,360], scaling [0.8,1.2], crop 300x300 to 224x224
    Data augmentation and crop parameters in Sections 2.1 and 2.2, chosen without a reported sensitivity analysis.
assumptions (3)
  • domain assumption ImageNet-pretrained weights transfer useful features to cardiac MRI regression.
    The entire approach relies on this; Section 2.2 (Model Input Strategy) and the Discussion claim it is the main driver of improvement.
  • domain assumption Ground-truth myocardium and cavity segmentation masks provide an appropriate auxiliary supervision signal for LV index regression.
    Segmentation regularization uses LVQuan19 training masks; Section 2.2 (Segmentation Regularization).
  • standard math The Wilcoxon signed-rank test on paired median absolute errors is a valid significance test for these comparisons.
    Used in Section 3 to claim significance; p-values are not reported and multiple comparisons are not corrected.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Left Ventricle Quantification Using Direct Regression with Segmentation Regularization and Ensembles of Pretrained 2D and 3D CNNs." pith.science (2026). https://pith.science/paper/UJ5KHGEZ

@misc{pith2026190804181,
  author       = {Pith},
  title        = {Pith review of: Left Ventricle Quantification Using Direct Regression with Segmentation Regularization and Ensembles of Pretrained 2D and 3D CNNs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UJ5KHGEZ}},
  note         = {Machine review of arXiv:1908.04181}
}
read the original abstract

Cardiac left ventricle (LV) quantification provides a tool for diagnosing cardiac diseases. Automatic calculation of all relevant LV indices from cardiac MR images is an intricate task due to large variations among patients and deformation during the cardiac cycle. Typical methods are based on segmentation of the myocardium or direct regression from MR images. To consider cardiac motion and deformation, recurrent neural networks and spatio-temporal convolutional neural networks (CNNs) have been proposed. We study an approach combining state-of-the-art models and emphasizing transfer learning to account for the small dataset provided for the LVQuan19 challenge. We compare 2D spatial and 3D spatio-temporal CNNs for LV indices regression and cardiac phase classification. To incorporate segmentation information, we propose an architecture-independent segmentation-based regularization. To improve the robustness further, we employ a search scheme that identifies the optimal ensemble from a set of architecture variants. Evaluating on the LVQuan19 Challenge training dataset with 5-fold cross-validation, we achieve mean absolute errors of 111 +- 76mm^2, 1.84 +- 0.9mm and 1.22 +- 0.6mm for area, dimension and regional wall thickness regression, respectively. The error rate for cardiac phase classification is 6.7%.

Figures

Figures reproduced from arXiv: 1908.04181 by the authors.

Figure 1
Figure 1. Overview of our approach. We use both 2D CNNs with 2D slices (top) and 3D CNNs with temporally stacked slices (bottom). Both are initialized with pretrained weights from ImageNet. The initial and Conv2D/Conv3D blocks have a different struc￾ture based on the respective architecture. 2D CNN Approaches. The key idea of our approach is to use pretrained architectures for indices regression. Thus, we consider a pool of p… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

17 extracted references · 17 canonical work pages

  1. [1]

    (2016) A combined deep-learning and deformable-model approach to fully automatic segmentation of the left ventricle in cardiac MRI

    Avendi, M., Kheradvar, A., Jafarkhani, H. (2016) A combined deep-learning and deformable-model approach to fully automatic segmentation of the left ventricle in cardiac MRI. Med Image Anal 30, 108–119

  2. [2]

    (2019) Skin Lesion Classification Using CNNs with Patch- Based Attention and Diagnosis-Guided Loss Weighting

    Gessert, N., Sentker, T., Madesta, F., Schmitz, R., Kniep, H., Baltruschat, I., Werner, R., Schlaefer, A. (2019) Skin Lesion Classification Using CNNs with Patch- Based Attention and Diagnosis-Guided Loss Weighting. IEEE Trans Biomed Eng

  3. [3]

    (2016) Deep residual learning for image recog- nition

    He, K., Zhang, X., Ren, S., Sun, J. (2016) Deep residual learning for image recog- nition. In: CVPR, pp. 770–778

  4. [4]

    (2018) Squeeze-and-excitation networks

    Hu, J., Shen, L., Sun, G. (2018) Squeeze-and-excitation networks. In: CVPR, pp. 7132–7141

  5. [5]

    (2017) Densely con- nected convolutional networks

    Huang, G., Liu, Z., Van Der Maaten, L., Weinberger, K.Q. (2017) Densely con- nected convolutional networks. In: CVPR, pp. 4700–4708

  6. [6]

    (2018) Full Quantification of Left Ventri- cle Using Deep Multitask Network with Combination of 2D and 3D Convolution on 2D+ t Cine MRI

    Jang, Y., Kim, S., Shim, H., Chang, H.J. (2018) Full Quantification of Left Ventri- cle Using Deep Multitask Network with Combination of 2D and 3D Convolution on 2D+ t Cine MRI. In: International Workshop on Statistical Atlases and Com- putational Models of the Heart, pp. 476–483. Springer

  7. [7]

    (2009) The role of cardiovascular magnetic resonance imaging in heart failure

    Karamitsos, T.D., Francis, J.M., Myerson, S., Selvanayagam, J.B., Neubauer, S. (2009) The role of cardiovascular magnetic resonance imaging in heart failure. J Am Coll Cardiol 54(15), 1407–1424 Left Ventricle Quantification with Pretrained CNNs 9

  8. [8]

    (2018) Left Ventricle Full Quantification using Deep Layer Aggre- gation based Multitask Relationship Learning

    Li, J., Hu, Z. (2018) Left Ventricle Full Quantification using Deep Layer Aggre- gation based Multitask Relationship Learning. In: International Workshop on Statistical Atlases and Computational Models of the Heart, pp. 381–388. Springer

Show all 17 references
  1. [9]

    Gessert and M

    N. Gessert and M. Lutz and M. Heyder and S. Latus and D. M. Leistner and Y. S. Abdelwahed and A. Schlaefer (2019) Automatic Plaque Detection in IVOCT Pullbacks Using Convolutional Neural Networks. IEEE Transactions on Medical Imaging 38(2), 426–434

  2. [10]

    (2017) Combining deep learning and level set for the automated segmentation of the left ventricle of the heart from cardiac cine magnetic resonance

    Ngo, T.A., Lu, Z., Carneiro, G. (2017) Combining deep learning and level set for the automated segmentation of the left ventricle of the heart from cardiac cine magnetic resonance. Med Image Anal 35, 159–171

  3. [11]

    (2016) Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning

    Shin, H.C., Roth, H.R., Gao, M., Lu, L., Xu, Z., Nogues, I., Yao, J., Mollura, D., Summers, R.M. (2016) Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning. IEEE Trans Med Imag 35(5), 1285–1298

  4. [12]

    (2015) Quantification of LV function and mass by cardiovascular magnetic reso- nance: multi-center variability and consensus contours

    Suinesiaputra, A., Bluemke, D.A., Cowan, B.R., Friedrich, M.G., Kramer, C.M., Kwong, R., Plein, S., Schulz-Menger, J., Westenberg, J.J., Young, A.A., et al. (2015) Quantification of LV function and mass by cardiovascular magnetic reso- nance: multi-center variability and consen...

  5. [13]

    (2019) Quantification of Full Left Ventricular metrics via Deep Regression Learning with Contour-guidance

    Wang, W., Wanga, Y., Wu, Y., Lin, T., Li, S., Chen, B. (2019) Quantification of Full Left Ventricular metrics via Deep Regression Learning with Contour-guidance. IEEE Access

  6. [14]

    (2017) Aggregated residual trans- formations for deep neural networks

    Xie, S., Girshick, R., Doll´ ar, P., Tu, Z., He, K. (2017) Aggregated residual trans- formations for deep neural networks. In: CVPR, pp. 1492–1500

  7. [15]

    (2018) Calculation of Anatomical and Functional Metrics Using Deep Learning in Cardiac MRI: Comparison Between Direct and Segmentation-Based Estimation

    Xu, H., Schneider, J.E., Grau, V. (2018) Calculation of Anatomical and Functional Metrics Using Deep Learning in Cardiac MRI: Comparison Between Direct and Segmentation-Based Estimation. In: International Workshop on Statistical Atlases and Computational Models of the Heart, p...

  8. [16]

    (2018) Full left ventricle quan- tification via deep multitask relationships learning

    Xue, W., Brahm, G., Pandey, S., Leung, S., Li, S. (2018) Full left ventricle quan- tification via deep multitask relationships learning. Med Image Anal 43, 54–65

  9. [17]

    (2017) Full quantification of left ventricle via deep multitask learning network respecting intra- and inter-task relatedness

    Xue, W., Lum, A., Mercado, A., Landis, M., Warrington, J., Li, S. (2017) Full quantification of left ventricle via deep multitask learning network respecting intra- and inter-task relatedness. In: MICCAI, pp. 276–284. Springer

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.