REVIEW 4 major objections 5 minor 23 references
UnDA transfers structural knowledge across unpaired imaging modalities by aligning uncertainty-weighted class tokens, improving target segmentation without paired scans.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-01 07:05 UTC pith:K6ACARWF
load-bearing objection A sensible training recipe, but the baseline comparison and internal inconsistencies undercut the boundary-precision claim. the 4 major comments →
UnDA: Unpaired Domain Alignment for Cross-Modal Knowledge Transfer in Medical Imaging
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The paper claims that structural knowledge can be transferred across unpaired modalities by aligning compact class tokens instead of raw features. Its central discovery is that weighting the optimal-transport plan by source prediction confidence suppresses noisy supervision, while a per-class prototype contrastive objective stabilizes alignment across unpaired batches. Across BraTS 2023 and MM-WHS, the full framework consistently improves Dice and HD95 over the target-only baseline; on MM-WHS it reaches 82.71% mean Dice and 13.94 mm HD95, trailing the best compared method by 3.28 Dice points on average but posting the best mean HD95. The alignment module is removed at inference, so the repor
What carries the argument
The Alignment Module pools bottleneck features into K class tokens via attention, giving each token a semantic-class meaning; a frozen anchor model provides reference tokens and updates class prototypes by exponential moving average. UCT-OT solves an entropy-regularized optimal transport problem (Sinkhorn) whose source marginal is weighted by exp(−α·entropy), so confident source tokens carry more mass. ProtoNCE pulls target tokens toward the prototype memory, enforcing global discriminability. A two-layer residual MLP adapter maps target tokens into the shared token space. All alignment components are discarded after training.
Load-bearing premise
The framework assumes labeled target-domain images are available for the supervised segmentation loss; if target labels are missing, the claimed transfer gains have not been demonstrated.
What would settle it
An ablation that removes the target segmentation loss L_seg from UnDA's Phase 2 (keeping only UCT-OT and ProtoNCE) and compares results on BraTS or MM-WHS against the same target model trained with those labels would reveal whether the improvements come from cross-modal alignment or simply from reusing target labels; if Dice and HD95 return to the target-only baseline, the paper's central transfer claim is unsupported.
If this is right
- If the framework works as claimed, CT-only segmentation models can be trained to near-MRI accuracy using only MRI labels at training time, with zero extra computation at deployment.
- The uncertainty-weighting principle could reduce noise from unreliable predictions in any distillation or pseudo-labeling pipeline, not just cross-modal medical imaging.
- The reported boundary-distance gains on MM-WHS imply fewer large localized segmentation errors, which matters more clinically than small uniform Dice improvements.
- Because the alignment module is backbone-agnostic, it can be attached to existing encoders without redesigning the architecture, making the gains portable to new segmentation tasks.
- Gains on small, high-variability cardiac structures suggest the method specifically helps where domain shift is most severe.
Where Pith is reading between the lines
- Because UnDA uses labeled target images during training, its comparison against unsupervised UDA baselines is not apples-to-apples; a fair test would either grant those baselines the same target labels or remove UnDA's target segmentation loss entirely.
- The uncertainty-weighting recipe—scaling source contributions by prediction confidence—is general and could be applied to feature-alignment losses in other modality gaps (e.g., histology-to-radiology) or even to cross-modal tasks outside medicine.
- A natural extension would be to withholds target labels from Phase 2 and measure how much UCT-OT and ProtoNCE alone transfer; the paper does not report this, so the relative contribution of label reuse versus alignment is unquantified.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes UnDA, a two-stage framework for unpaired cross-modal medical image segmentation. In Phase 1, an anchor model is trained on a labeled source modality with a class-token pooling module. In Phase 2, the anchor is frozen and the target model is trained with segmentation supervision on labeled target images plus two auxiliary losses: Uncertainty-Weighted Optimal Transport (UCT-OT) and a prototypical contrastive loss (ProtoNCE). The alignment module is removed at inference. Experiments are reported on BraTS 2023 (T2-FLAIR to T1-native) and MM-WHS (MRI to CT), with comparisons against several unsupervised UDA baselines on MM-WHS and an ablation study. The central claim is that UnDA consistently improves target-modality accuracy and boundary precision without paired training data.
Significance. If the claims are substantiated, the contribution is valuable: it provides a backbone-agnostic way to distill structural knowledge across unpaired modalities, with zero inferencetime overhead, and reports large improvements over target-only baselines (e.g., MM-WHS mean Dice 71.59 to 82.71 and mean HD95 21.41 to 13.94). The idea of uncertainty-weighting in optimal transport alignment is a reasonable and potentially useful extension over plain OT-based domain alignment. The use of class tokens and prototype memory is also well-motivated. However, the current manuscript has several load-bearing empirical problems that must be resolved before the claims can be accepted.
major comments (4)
- [§2.1, §2.3, Table 3] The target phase optimizes Lseg = Dice + BCE (Eq. 10) on labeled target images D_G defined in §2.1, so UnDA uses target ground-truth labels. Table 3 compares against SE_ASA, DAR-UNet, SIFA v2, MAPSeg, and UMMKD, which are unsupervised UDA methods by design. The paper never states whether these baselines were given target labels. If the official implementations were run without target labels while UnDA uses them, the comparison is confounded: the reported mean-Dice advantage over SIFA v2 (+11.38) and MAPSeg (+21.36) could come from target supervision rather than from UCT-OT or ProtoNCE. The authors must specify the labeling protocol for each baseline and ideally run a controlled comparison where baselines receive the same target labels, or re-frame the method as target-supervised adaptation rather than unsupervised UDA.
- [§3.3 (Table 2) vs §4 (Table 4)] The 'UnDA (Ours)' row is inconsistent between the main results and the ablation. Table 2 reports LA Dice 89.58 and LV HD95 9.11, while Table 4, which uses the same column order (LV Myo, LA, LV, RA, RV, AA, PA), reports LA Dice 85.53 and LV HD95 11.32 for the same final model. These are not rounding differences. This inconsistency makes the component-wise conclusions in Section 4 (e.g., the gains of UCT-OT and ProtoNCE) unreliable as printed. All numbers should be verified so that identical configurations yield identical metrics across tables.
- [§4, Table 4] In the ablation, the no-alignment baseline 'MR→CT' has mean HD95 13.58 mm, which is better than the full UnDA model's mean HD95 13.94 mm. Adding ProtoNCE (16.91), plain OT (18.63), or UCT-OT (17.12) individually also worsens mean HD95 relative to no alignment. This directly contradicts the abstract's claim of 'improved boundary precision' and undermines the statement that uncertainty-weighted OT improves boundary accuracy. The authors need to explain this pattern, correct the numbers if there is an error, or soften the HD95 claim.
- [§1/§3.3 vs Table 1] Several quantitative claims in the text do not match Table 1. The abstract says Tumor Core HD95 is reduced from 29.1 mm to 12.4 mm, but Table 1 reports 29.12 mm to 10.10 mm. In §3.3, the text says Dice improves by +2.3 (WT), +4.1 (TC), and +4.0 (ET), but the table values imply +5.0, +8.8, and +9.3, respectively. The stated HD95 reductions (7.21 mm WT, 16.69 mm TC, 11.42 mm ET) also differ from those computed from the table (10.70 mm WT, 19.02 mm TC, 6.64 mm ET). These inconsistencies must be resolved before the experimental claims can be trusted.
minor comments (5)
- [Tables 1–4] No error bars, standard deviations, or number of runs are reported for any metric. Given the inconsistencies already present, the authors should provide variance estimates and ideally statistical significance tests for the central comparisons.
- [§3.4] The description of how baselines were modified to the common backbone is too terse. For each baseline, specify which modules were kept, which were replaced, and how hyperparameters (e.g., loss weights, training epochs) were chosen, so that the comparison is reproducible.
- [Table 3] UMMKD achieves higher mean Dice (85.99 vs 82.71) and better HD95 on several individual structures. The text frames UnDA as 'best mean HD95' and 'more balanced,' which is defensible, but the claim in the abstract of 'consistent improvements' should be qualified given that a prior method outperforms UnDA on mean Dice.
- [§2.2, Eq. (8)] The notation is slightly unclear: wi is defined with both i and k indices in 'a_i = wi / sum_k wk', which should be written as a_i = exp(-αu_i) / Σ_k exp(-αu_k). Please fix the subscript ambiguity.
- [Throughout] There are several typographical issues, including 'T able' in table headings, and the references section formatting. A careful proofread is advised.
Circularity Check
No circular derivation found; the main concern is a target-label supervision confound in baseline comparisons, which is an experimental issue rather than circularity.
full rationale
UnDA's training objective is defined independently. Phase 2 optimizes L_target = L_seg + lambda_OT L_UCT-OT + lambda_P L_ProtoNCE (Eq. 12), where L_seg = Dice + BCE (Eq. 10) is a standard supervised segmentation loss on the labeled target data D_G, and L_UCT-OT (Eq. 13) is computed from model features via the transport plan in Eq. 9. ProtoNCE uses prototype memories updated from frozen anchor tokens (Eq. 6). None of the reported Dice/HD95 numbers are used as inputs to these losses; the uncertainty weights w_i = exp(-alpha u_i) and the loss weights lambda_OT = 0.2, lambda_P = 0.5 are hyperparameters, not fitted to the evaluation metric. The anchor model is a frozen teacher, not a renamed target predictor. There are no author self-citations, and no uniqueness result or ansatz is imported from the authors' prior work. The strongest legitimate concern is experimental: Phase 2 uses labeled target images, while SIFA v2 and MAPSeg are unsupervised UDA methods by design; if the modified baseline implementations did not receive target labels, the Table 3 comparison is confounded by additional supervision. This is a fairness/validity concern, not a circular step. Separately, Table 4's UnDA row reports LA Dice 85.53 and HD95 11.32, while Table 2 reports LA Dice 89.58 and HD95 9.11 for the identical model; this internal inconsistency undermines the ablation's component-wise conclusions but again is not circularity. Overall, the paper's central empirical claims do not reduce by construction to their own inputs.
Axiom & Free-Parameter Ledger
free parameters (5)
- lambda_OT =
0.2
- lambda_P =
0.5
- alpha (uncertainty temperature) =
not reported
- epsilon (Sinkhorn entropy regularizer) =
not reported
- beta (EMA momentum for prototype memory) =
not reported
axioms (4)
- standard math Entropy-regularized OT solved by Sinkhorn iterations converges and yields a valid transport plan
- domain assumption Target-domain dense labels are available for Lseg in Phase 2
- domain assumption Class tokens learned on anchor features correspond to the same semantic classes in target after linear projection
- ad hoc to paper Exponential weighting wi = exp(-alpha ui) monotonically down-weights uncertain tokens
Cite this review
Pith. "Pith review of UnDA: Unpaired Domain Alignment for Cross-Modal Knowledge Transfer in Medical Imaging." pith.science (2026). https://pith.science/paper/K6ACARWF
@misc{pith2026260721546,
author = {Pith},
title = {Pith review of: UnDA: Unpaired Domain Alignment for Cross-Modal Knowledge Transfer in Medical Imaging},
year = {2026},
howpublished = {\url{https://pith.science/paper/K6ACARWF}},
note = {Machine review of arXiv:2607.21546}
}
read the original abstract
Multimodal based approaches often outperform single modality approaches in downstream tasks as the different modalities provide complementary information, yet acquiring paired clinical data remains a significant challenge in real world scenarios. While cross-modal knowledge distillation addresses this, existing methods often struggle with large modality gaps and the propagation of noise from uncertain source-domain predictions. To overcome these challenges, we propose UnDA, an anchor-guided framework for unpaired cross-modal distillation. Our approach introduces a backbone-agnostic Alignment Module that extracts semantically structured class tokens via an attention based pooling mechanism. To ensure robust knowledge transfer, we propose Uncertainty-Weighted Optimal Transport (UCT-OT), which dynamically weights feature-level alignment based on prediction confidence, effectively suppressing noisy supervision. Furthermore, a per-class ProtoNCE objective maintains stable prototype memories to enforce global discriminability across unpaired batches. Evaluations on representative segmentation tasks under strictly unpaired settings show consistent improvements in accuracy and boundary precision in the target modality, demonstrating that meaningful structural knowledge can be transferred across heterogeneous data sources without paired datasets.
Figures
Reference graph
Works this paper leans on
-
[1]
IEEE Trans
Dou, Q., Liu, Q., Heng, P.A., Glocker, B.: Unpaired multi-modal segmentation via knowledge distillation. IEEE Trans. Med. Imaging39(7), 2415–2425 (2020)
2020
-
[2]
IEEE Transactions on Medical Imaging41(5), 1057–1068 (2021)
Jiang, J., Rimner, A., Deasy, J.O., Veeraraghavan, H.: Unpaired Cross-Modality Educed Distillation (CMEDL) for Medical Image Segmentation. IEEE Transactions on Medical Imaging41(5), 1057–1068 (2021)
2021
-
[3]
International Conference on Medical Image Computing and Computer-Assisted In- tervention (MICCAI)14227, 113–123 (2023)
Guan, Q., Xie, Y., Yang, B., Zhang, J., Liao, Z., Wu, Q., Xia, Y.: Unpaired Cross- Modal Interaction Learning for COVID-19 Segmentation on Limited CT Images. International Conference on Medical Image Computing and Computer-Assisted In- tervention (MICCAI)14227, 113–123 (2023)
2023
-
[4]
Inter- national Conference on Medical Image Computing and Computer-Assisted Inter- vention (MICCAI)12905, 307–316 (2021)
Wang, Q., Che, H., Ding, W., Xiang, L., Li, G., Li, Z., Cui, S.: Colorectal Polyp Classification from White-light Colonoscopy Images via Domain Alignment. Inter- national Conference on Medical Image Computing and Computer-Assisted Inter- vention (MICCAI)12905, 307–316 (2021)
2021
-
[5]
arXiv preprint arXiv:2407.17620 (2024)
Alberb, M., Elbatel, M., Elgebaly, A., Montoya-del-Angel, R., Li, X., Martí, R.: Co- MoTo: Unpaired Cross-Modal Lesion Distillation Improves Breast Lesion Detection in Tomosynthesis. arXiv preprint arXiv:2407.17620 (2024)
Pith/arXiv arXiv 2024
-
[6]
In: Advances in Neural Information Processing Systems (NeurIPS), vol
Ryoo,M.,Piergiovanni,A.J.,Arnab,A.,Dehghani,M.,Angelova,A.:TokenLearner: Adaptive Space-Time Tokenization for Videos. In: Advances in Neural Information Processing Systems (NeurIPS), vol. 34, pp. 12786–12797 (2021)
2021
-
[7]
In: International Conference on Learn- ing Representations (ICLR) (2021)
Dosovitskiy, A., Beyer, L., Kolesnikov, A., et al.: An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. In: International Conference on Learn- ing Representations (ICLR) (2021)
2021
-
[8]
In: Advances in Neural Information Processing Systems (NeurIPS), vol
Cuturi, M.: Sinkhorn Distances: Lightspeed Computation of Optimal Transport. In: Advances in Neural Information Processing Systems (NeurIPS), vol. 26, pp. 2292– 2300 (2013)
2013
-
[9]
arXiv preprint arXiv:2107.02314 (2021)
Baid, U., Ghodasara, S., Mohan, S., et al.: The RSNA-ASNR-MICCAI BraTS 2021 Benchmark on Brain Tumor Segmentation and Radiogenomic Classification. arXiv preprint arXiv:2107.02314 (2021)
Pith/arXiv arXiv 2021
-
[10]
IEEE Transactions on Medical Imaging34(10), 1993–2024 (2015)
Menze, B.H., Jakab, A., Bauer, S., et al.: The Multimodal Brain Tumor Image Seg- mentation Benchmark (BRATS). IEEE Transactions on Medical Imaging34(10), 1993–2024 (2015)
1993
-
[11]
Sci- entific Data4(1) (2017)
Bakas, S., Akbari, H., Sotiras, A., et al.: Advancing The Cancer Genome Atlas glioma MRI collections with expert segmentation labels and radiomic features. Sci- entific Data4(1) (2017)
2017
-
[12]
The Cancer Imaging Archive (2017).https: //doi.org/10.7937/K9/TCIA.2017.KLXWJJ1Q
Bakas, S., Akbari, H., Sotiras, A., et al.: Segmentation Labels for the Pre-operative Scans of the TCGA-GBM collection. The Cancer Imaging Archive (2017).https: //doi.org/10.7937/K9/TCIA.2017.KLXWJJ1Q
-
[13]
Bakas, S., Akbari, H., Sotiras, A., et al.: Segmentation Labels for the Pre-operative Scans of the TCGA-LGG collection. The Cancer Imaging Archive (2017).https: //doi.org/10.7937/K9/TCIA.2017.GJQ7R0EF UnDA: Unpaired Domain Alignment for cross modal tasks 11
-
[14]
IEEE Transactions on Pattern Analysis and Machine Intelli- gence41(12), 2933–2946 (2019)
Zhuang, X.: Multivariate Mixture Model for Myocardial Segmentation Combining Multi-Source Images. IEEE Transactions on Pattern Analysis and Machine Intelli- gence41(12), 2933–2946 (2019)
2019
-
[15]
Medical Image Analysis31, 77–87 (2016)
Zhuang, X., Shen, J.: Multi-Scale Patch and Multi-Modality Atlases for Whole Heart Segmentation of MRI. Medical Image Analysis31, 77–87 (2016)
2016
-
[16]
GitHub repository (2023).https://github
Wang, L.: 3DUNet-BraTS-PyTorch. GitHub repository (2023).https://github. com/LightersWang/3DUNet-BraTS-PyTorch
2023
-
[17]
In: International Conference on Learning Representations (ICLR) (2021).https://openreview.net/forum?id=KmykpuSrjcq
Li, J., Zhou, P., Xiong, C., Hoi, S.C.H.: Prototypical Contrastive Learning of Unsu- pervised Representations. In: International Conference on Learning Representations (ICLR) (2021).https://openreview.net/forum?id=KmykpuSrjcq
2021
-
[18]
In: European Conference on Computer Vision (ECCV), pp
Damodaran, B.B., Kellenberger, B., Flamary, R., Tuia, D., Courty, N.: DeepJDOT: Deep Joint Distribution Optimal Transport for Unsupervised Domain Adaptation. In: European Conference on Computer Vision (ECCV), pp. 467–483 (2018)
2018
-
[19]
IEEE Journal of Biomedical and Health Informatics26(10), 4976– 4986 (2022)
Yao, K., Su, Z., Huang, K., Yang, X., Sun, J., Hussain, A., Coenen, F.: A Novel 3D Unsupervised Domain Adaptation Framework for Cross-Modality Medical Image Segmentation. IEEE Journal of Biomedical and Health Informatics26(10), 4976– 4986 (2022)
2022
-
[20]
Proceedings of the AAAI Conference on Artificial Intelligence37(1), 623–631 (2023)
Feng, W., Ju, L., Wang, L., Song, K., Zhao, X., Ge, Z.: Unsupervised Domain Adaptation for Medical Image Segmentation by Selective Entropy Constraints and Adaptive Semantic Alignment. Proceedings of the AAAI Conference on Artificial Intelligence37(1), 623–631 (2023)
2023
-
[21]
In: Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp
Zhang, X., Wu, Y., Angelini, E., Li, A., Guo, J., Rasmussen, J.M., O’Connor, T.G., Wadhwa, P.D., Jackowski, A.P., Li, H., Posner, J., Laine, A.F., Wang, Y.: MAPSeg: Unified Unsupervised Domain Adaptation for Heterogeneous Medical Image Seg- mentation Based on 3D Masked Autoencoding and Pseudo-Labeling. In: Proceed- ings of the IEEE/CVF Conference on Compu...
2024
-
[22]
arXiv preprint arXiv:2002.02255 (2020)
Chen, C., Dou, Q., Chen, H., Qin, J., Heng, P.A.: Unsupervised Bidirectional Cross-Modality Adaptation via Deeply Synergistic Image and Feature Alignment for Medical Image Segmentation. arXiv preprint arXiv:2002.02255 (2020)
Pith/arXiv arXiv 2002
-
[23]
In: Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence (AAAI), pp
Chen, C., Dou, Q., Chen, H., Qin, J., Heng, P.A.: Synergistic Image and Fea- ture Adaptation: Towards Cross-Modality Domain Adaptation for Medical Image Segmentation. In: Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence (AAAI), pp. 865–872 (2019)
2019
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.