Pith. sign in

REVIEW 3 major objections 5 minor 11 references

Multimodal HIE Lesion Segmentation in Neonates: A Comparative Study of Loss Functions

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read On the BONBID-HIE neonatal dataset, a fixed 3D U-Net trained with compound losses—Dice-Focal-HausdorffDT or Tversky-HausdorffDT—beats standalone Dice, Tversky, HausdorffDT, and Dice-Focal losses, with Tversky-HausdorffDT best on Dice and…

desk verdict A cleanly run small loss-function comparison whose headline ranking rests on a 4-case validation set with weights tuned on the same set—worth a serious referee, but the specific best-loss claim needs test-set or cross-validated support. read the letter →

arxiv 2502.09148 v1 pith:X6BDWJWZ submitted 2025-02-13 cs.CV

Annayah Usman , Abdul Haseeb , Tahir Syed This is my paper · ORCID
classification cs.CV
keywords HIElesionsegmentationneonatalMRIcompoundlossfunctionsTversky-HausdorffDTDice-Focal-Hausdorff3DU-NetBONBID-HIEboundary-aware
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks which loss function is best for segmenting hypoxic-ischemic encephalopathy (HIE) lesions in neonatal MRI, a difficult task because lesions are diffuse, multifocal, and often occupy under 1% of the brain. The authors fix one 3D U-Net and identical preprocessing and augmentation, then compare six losses on the BONBID-HIE validation set. Their central claim is that compound losses combining a region-based term with a boundary-aware log-Hausdorff distance term outperform standalone losses. In particular, Tversky-HausdorffDT gives the best Dice and Normalized Surface Dice, and Dice-Focal-HausdorffDT gives the lowest Mean Surface Distance. If correct, the result suggests task-specific compound losses are a cheap, high-impact choice for small medical segmentation datasets.

What carries the argument

The central objects are the two proposed compound losses, linear combinations of a region-based term and a boundary term: $$L_{DF-HDT}=\$\alpha$\,DFL+\$\beta$\,\log(HDT L), \qquad L_{Tv-HDT}=\$\alpha$\,TverskyLoss+\$\beta$\,\log(HDT L),$$ with $\alpha=0.9$, $\beta=0.1$. The $\log$ dampens outlier distances so the boundary term emphasizes overall surface alignment rather than rare extreme errors. These losses are optimized by a fixed 3D U-Net with three encoder-decoder blocks taking a two-channel input of ADC and ZADC maps, and the comparison is read off Dice, MSD, and NSD scores on the four-case validation split.

What would settle it

Run the exact same six-loss comparison on the full 44-case hidden test set of BONBID-HIE (or a k-fold cross-validation over the 85 training cases) and check whether Tversky-HausdorffDT and Dice-Focal-HausdorffDT still beat standalone Dice on Dice and NSD; if their advantage disappears or the ordering flips, the paper's central claim fails.

Watch

Extended reading notes

Core claim

The discovery is that on HIE lesion segmentation the choice of loss function changes both the numeric score and the shape of predicted masks, and neither region overlap alone nor boundary distance alone is sufficient: Dice over- or under-segments small lesions, Tversky over-segments large lesions, HausdorffDT misses small lesions entirely, and Dice-Focal improves over Dice. The two proposed compounds—Dice-Focal-HausdorffDT and Tversky-HausdorffDT, linear combinations with weights 0.9 and 0.1 of a region loss and log(HausdorffDT)—rank highest. Tversky-HausdorffDT reaches Dice 0.50 and NSD 0.5325, Dice-Focal-HausdorffDT reaches MSD 1.4225, versus baseline Dice at 0.38 with MSD 15.07. The paper treats this as evidence that combining region-based and boundary-aware losses yields more accurate HIE lesion segmentation even with limited training data.

Load-bearing premise

The whole ranking rests on four validation patients, and the same four were used to tune the compound-loss weights; if those four are not representative of the hidden test set, the reported best loss may not generalize.

Editorial extensions

If this is right

  • A 3D U-Net trained with Tversky-HausdorffDT reaches Dice 0.50 and Normalized Surface Dice 0.5325, a direct gain over the Dice-loss baseline of 0.38 and 0.385.
  • Dice-Focal-HausdorffDT gives the lowest Mean Surface Distance (1.4225 versus 15.07 for the Dice baseline), so the boundary term visibly tightens surface alignment.
  • Standalone Tversky and HausdorffDT losses fall below the baseline, so the paper's recipe is to combine rather than to choose one kind of loss.
  • Compound losses need more epochs (59 to 72 versus 32 for Dice) but converge far sooner than the 400 to 1500 epochs reported by other BONBID-HIE entries.
  • The best Dice score still only matches the existing roughly 0.5 benchmark, meaning loss optimization alone does not solve the hardest tiny-lesion cases.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The top-two compound losses differ by only 0.0075 in Dice on a four-case validation split, so an editor would not treat their ordering as stable until the hidden 44-case test set is scored.
  • Because the $\log(HDT L)$ term also acts as a smoothness regularizer, a fair ablation would replace it with an unrelated distance-aware penalty to isolate whether the gain is boundary-specific.
  • The same pair of compound losses could transfer to other diffuse, low-volume lesion segmentation tasks, but that transfer is untested.
  • A k-fold cross-validation on the 85 training cases would be a cheap way to check whether compound losses still dominate before relying on the hidden test set.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The manuscript presents a controlled comparison of six loss functions for segmenting hypoxic-ischemic encephalopathy (HIE) lesions in neonatal multimodal MRI using the BONBID-HIE dataset. A fixed 3D U-Net with shared preprocessing and augmentation is trained with Dice, Dice-Focal, Tversky, Hausdorff Distance, and two proposed compound losses (Dice-Focal-HausdorffDT and Tversky-HausdorffDT). The authors report that compound losses outperform standalone losses and that Tversky-HausdorffDT achieves the highest Dice (0.5000) and NSD (0.5325), while Dice-Focal-HausdorffDT achieves the lowest MSD (1.4225) on the validation split. The paper concludes that task-specific compound losses improve HIE lesion segmentation even with limited data.

Significance. The controlled experimental design is a genuine strength: a single architecture, identical preprocessing and augmentation, and clear loss definitions allow the loss-function comparison to be isolated from architecture effects. The authors provide both quantitative and qualitative comparisons, and they explicitly acknowledge several limitations, including the validation-only evaluation and resampling distortion. If the reported ranking were confirmed on an independent test set, the finding that simple linear combinations of region-based and boundary-aware losses improve HIE segmentation would be a useful practical contribution and would complement existing loss-function surveys. However, the current evidence base is too narrow to support the central claim as stated.

major comments (3)
  1. [Sec. 4.1, Table 1] The central claim that Tversky-HausdorffDT is the best loss rests entirely on the official 4-case validation split. No per-case scores, confidence intervals, or paired significance tests are reported. The margins separating the top methods are very small: Dice 0.5000 vs 0.4925, NSD 0.5325 vs 0.5300, and MSD 1.6250 vs 1.4225. With n=4, these differences are within plausible random variation, so the observed ordering cannot be distinguished from selection noise. The authors should evaluate on the held-out test set or perform patient-level cross-validation and report error bars or significance tests before claiming that one loss is optimal.
  2. [Sec. 2, Eq. (5)-(6)] The compound-loss weights alpha=0.9 and beta=0.1 were chosen by 'empirically testing combinations' on the same validation set that is later used as the evaluation set (Sec. 4.1). This selection-on-the-evaluation-set procedure creates an optimism bias: the reported superiority of the compound losses may reflect the weight search rather than a stable property of the loss formulations. The paper should either use an inner validation split for weight selection, describe the full grid of tested combinations and the corresponding scores, or explicitly frame the results as validation-set selection outcomes.
  3. [Sec. 2, Eq. (4)-(6)] The definition of Hausdorff Distance Loss as HDTL = 1/(1+HausdorffDistance) is not a differentiable loss as written; the true Hausdorff distance is a max-min quantity and backpropagating through it is not standard. The paper states that all losses are implemented through MONAI, but MONAI's HausdorffDTLoss is a differentiable distance-transform-based approximation that is not equivalent to Eq. (4). The manuscript needs to state the exact MONAI formulation used and reconcile it with Eq. (4); otherwise the reader cannot tell which objective was actually optimized.
minor comments (5)
  1. [Sec. 2, Eq. (2)] The text says 'equal balance is given to both Dice and Focal loss' but does not specify the default value of alpha in Eq. (2); if alpha=0.5 is intended, say so explicitly.
  2. [Sec. 3.3, Eq. (8)] The denominator in the NSD formula should presumably be |delta(q)|+|delta(p)|, not |delta(q)+delta(p)|; please correct the typo.
  3. [Header, pp. 2-8] The running header contains the placeholder 'MatAE here goes short title A PREPRINT', which is a template artifact that should be removed.
  4. [References] The reference to the BONBID-HIE challenge is truncated ('...and Lesion S Grand Challenge') and should be completed.
  5. [References] Some bibliographic entries use incomplete author lists, e.g., 'Karimi and et al.' and 'Sadegh and et al.'; please provide full citations.

Circularity Check

1 steps flagged · score 6.0 of 10

Best-loss ranking is a selection result, not a prediction: compound-loss weights were tuned on the same 4-case validation split that produces the headline Table 1 scores.

  1. fitted input called prediction [Section 2 (Eqs. 5-6); Section 3.4 (Table 1); Section 4.1 (Limitations)]
    "We've kept α as 0.9 and β as 0.1 after empirically testing combinations from {(α, β) : α ∈ [0.5, 0.9], β∈ [0.1, 0.5]}. / Only the validation set is used as a proxy for evaluation, as the test set is held out by the challenge organizers and has not been publicly released yet, following the conclusion of the challenge."

    The α and β weights defining both proposed compound losses were selected by empirical testing on the validation split. The paper's central result, Table 1, is then computed on that same validation split and used to conclude that Tversky-HausdorffDT is best. Since the evaluation data were already used to choose the compound-loss hyperparameters (and early stopping), the reported ranking is a fitted selection rather than an independent prediction. The margins separating the top two compound losses (Dice 0.5000 vs 0.4925; NSD 0.5325 vs 0.5300; MSD 1.6250 vs 1.4225) are small relative to a 4-case sample, and the paper's own limitation states that only the validation set was used for evaluation.

full rationale

The paper is otherwise self-contained: loss functions are standard definitions, the comparison protocol uses a fixed architecture and preprocessing, and there is no load-bearing self-citation or uniqueness-theorem import. The circularity is concentrated in the evaluation design. The authors fit the compound-loss coefficients (α=0.9, β=0.1) by testing candidate combinations on the validation split, then present the validation-split metrics in Table 1 as the evidence that compound losses outperform standalone losses and that Tversky-HausdorffDT is the best loss. With n=4 validation cases, the top-method margins are within plausible noise (Dice Δ=0.0075, NSD Δ=0.0025), and no per-case scores, confidence intervals, or paired tests are given. The abstract's predictive language ('Tversky-HausdorffDT Loss achieves the highest Dice and Normalized Surface Dice scores') is therefore a description of the same data used for model selection, not an independent test. This matches the fitted-input-called-prediction pattern and warrants a score of 6: the central ranking partially reduces to the tuning procedure, but the loss-function formulations themselves and the controlled comparison setup retain independent content.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The central comparison depends on six hand-set or validation-tuned weights: two for Tversky loss and four for the two compound losses. The compound weights are the most serious because they were selected on the same validation set used to report final scores, so part of the reported ranking is a fitted artifact. No new physical or conceptual entities are introduced.

free parameters (6)
  • Tversky loss false-positive weight alpha = 0.3
    Chosen by hand to emphasize false negatives (alpha < beta); enters Eq. 3 and affects all Tversky-based results.
  • Tversky loss false-negative weight beta = 0.7
    Chosen by hand to emphasize false negatives; enters Eq. 3.
  • Dice-Focal-HausdorffDT weighting alpha = 0.9
    Selected 'after empirically testing combinations' on the validation set; enters Eq. 5.
  • Dice-Focal-HausdorffDT weighting beta = 0.1
    Selected together with alpha on the validation set; enters Eq. 5.
  • Tversky-HausdorffDT weighting alpha = 0.9
    Selected on the validation set; enters Eq. 6.
  • Tversky-HausdorffDT weighting beta = 0.1
    Selected on the validation set; enters Eq. 6.
assumptions (4)
  • domain assumption Fixed 3D U-Net architecture and identical preprocessing isolate the effect of loss functions on segmentation quality.
    Sec 3.2.1 states the model and preprocessing are fixed across experiments; this assumes no interaction between architecture and loss choice.
  • domain assumption The 4-case official validation split is a representative proxy for the hidden 44-case test set.
    Sec 4.1 states 'Only the validation set is used as a proxy for evaluation'; with n=4 this assumption is fragile.
  • domain assumption MONAI's implementations of the named loss functions and metrics are correct and behave as described.
    All losses and metrics are 'implemented through MONAI' (Sec 2); the unexplained Inf MSD for HausdorffDT in Table 1 suggests this assumption may be violated.
  • domain assumption Resampling masks with nearest-neighbor interpolation preserves lesion geometry enough for metric comparison.
    Sec 3.2.2 notes resampled label masks could introduce distortion; the authors acknowledge in Sec 4.1 that reverse-resampling predictions would be more robust.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Multimodal HIE Lesion Segmentation in Neonates: A Comparative Study of Loss Functions." pith.science (2026). https://pith.science/paper/X6BDWJWZ

@misc{pith2026250209148,
  author       = {Pith},
  title        = {Pith review of: Multimodal HIE Lesion Segmentation in Neonates: A Comparative Study of Loss Functions},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/X6BDWJWZ}},
  note         = {Machine review of arXiv:2502.09148}
}
read the original abstract

Segmentation of Hypoxic-Ischemic Encephalopathy (HIE) lesions in neonatal MRI is a crucial but challenging task due to diffuse multifocal lesions with varying volumes and the limited availability of annotated HIE lesion datasets. Using the BONBID-HIE dataset, we implemented a 3D U-Net with optimized preprocessing, augmentation, and training strategies to overcome data constraints. The goal of this study is to identify the optimal loss function specifically for the HIE lesion segmentation task. To this end, we evaluated various loss functions, including Dice, Dice-Focal, Tversky, Hausdorff Distance (HausdorffDT) Loss, and two proposed compound losses -- Dice-Focal-HausdorffDT and Tversky-HausdorffDT -- to enhance segmentation performance. The results show that different loss functions predict distinct segmentation masks, with compound losses outperforming standalone losses. Tversky-HausdorffDT Loss achieves the highest Dice and Normalized Surface Dice scores, while Dice-Focal-HausdorffDT Loss minimizes Mean Surface Distance. This work underscores the significance of task-specific loss function optimization, demonstrating that combining region-based and boundary-aware losses leads to more accurate HIE lesion segmentation, even with limited training data.

Figures

Figures reproduced from arXiv: 2502.09148 by the authors.

Figure 1
Figure 1. Overview of the Experimental Setup. The setup processes two input modalities: 3D ADC and ZADC maps, resampling them with trilinear interpolation and normalizing their intensities separately. The concatenated 2-channel 3D input (2, 192, 192, 32) then undergoes various data augmentations. A U-Net with three encoder-decoder blocks is trained using different loss functions, and segmentation performance is evaluated usin… view at source ↗
Figure 2
Figure 2. Visual evaluation of predicted segmentation masks versus ground truth on Axial ADC maps across all loss [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

11 extracted references · 6 canonical work pages

  1. [4]

    Rethinking dice loss for medical image segmentation

    Rui Zhao et al. Rethinking dice loss for medical image segmentation. In 2020 IEEE International Conference on Data Mining (ICDM), pages 851–860. IEEE,

    work page 2020

  2. [10]

    URL http://arxiv.org/abs/1809.04430. TorchIO. Torchio, n.d. URL https://torchio.readthedocs.io/. 9 MatAE here goes short title A PREPRINT PyTorch. Pytorch, n.d. URL https://pytorch.org/. T. Eelbode, J. Bertels, M. Berman, D. Vandermeulen, F. Maes, R. Bisschops, and M. B. Blaschko. Optimization for medical image segmentation: theory and practice when evalu...

    arXiv

  3. [2008]

    Intrapartum-related neonatal encephalopathy incidence and impairment at regional and global levels for 2010 with trends from

    Anne CC Lee, Naoko Kozuki, Hannah Blencowe, Theo V os, Adil Bahalim, Gary L Darmstadt, Susan Niermeyer, Matthew Ellis, Nicola J Robertson, Simon Cousens, et al. Intrapartum-related neonatal encephalopathy incidence and impairment at regional and global levels for 2010 with trends from

    work page 2010

  4. [2011]

    doi:10.1053/j.nainr.2011.07.004. Seetha Shankaran, Abbot R Laptook, Richard A Ehrenkranz, Jon E Tyson, Scott A McDonald, Edward F Donovan, Avroy A Fanaroff, W Kenneth Poole, Linda L Wright, Rosemary D Higgins, et al. Whole-body hypothermia for neonates with hypoxic–ischemic encephalopathy. New England Journal of Medicine , 353(15):1574–1584,

    doi:10.1053/j.nainr.2011.07.004 2011

  5. [2015]

    Kevin Murphy, Niek E Van Der Aa, Simona Negro, Floris Groenendaal, Linda S De Vries, Max A Viergever, Geraldine B Boylan, Manon JNL Benders, and Ivana Išgum

    URL https://doi.org/10.48550/arxiv.1505.04597. Kevin Murphy, Niek E Van Der Aa, Simona Negro, Floris Groenendaal, Linda S De Vries, Max A Viergever, Geraldine B Boylan, Manon JNL Benders, and Ivana Išgum. Automatic quantification of ischemic injury on diffusion-weighted mri of neonatal hypoxic ischemic encephalopathy. NeuroImage: Clinical, 14:222–232,

    arXiv doi:10.48550/arxiv.1505.04597

  6. [2017]

    URL https://arxiv.org/abs/1706.05721. D. Karimi and et al. Reducing the hausdorff distance in medical image segmentation with convolutional neural networks. IEEE Transactions on Medical Imaging, 39(2):499–513,

    arXiv

  7. [2018]

    doi:10.1002/mp.13300. MONAI. Monai - home, n.d. URL https://monai.io/. O. Ronneberger, P. Fischer, and T. Brox. U-net: Convolutional networks for biomedical image segmentation. arXiv (Cornell University),

    work page doi:10.1002/mp.13300

  8. [2020]

    URL https://doi

    doi:10.1109/tmi.2020.3002417. URL https://doi. org/10.1109/tmi.2020.3002417. J. Zhu, A. Hamdi, Y . Qi, Y . Jin, and J. Wu. Medical sam 2: Segment medical images as video via segment anything model

    work page arXiv 2020

Show all 11 references
  1. [2021]

    A survey of loss functions for semantic segmentation

    Shruti Jadon. A survey of loss functions for semantic segmentation. In 2020 IEEE Conference on Computational Intelligence in Bioinformatics and Computational Biology (CIBCB) , pages 1–7. IEEE,

    2020

  2. [2023]

    Ernest M Graham, Kristin A Ruis, Adam L Hartman, Frances J Northington, and Harold E Fox

    URL https://doi.org/10.1101/ 2023.06.30.546841. Ernest M Graham, Kristin A Ruis, Adam L Hartman, Frances J Northington, and Harold E Fox. A systematic review of the role of intrapartum hypoxia-ischemia in the causation of neonatal encephalopathy. American Journal of Obstetrics...

    2023

  3. [2024]

    Cornell University

    URL https://doi.org/10.48550/arxiv.2408.00874. Cornell University. 10

    doi arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.