Pith. sign in

REVIEW 4 major objections 5 minor 23 references

UnDA transfers structural knowledge across unpaired imaging modalities by aligning uncertainty-weighted class tokens, improving target segmentation without paired scans.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

T0 review · deepseek-v4-flash

2026-08-01 07:05 UTC pith:K6ACARWF

load-bearing objection A sensible training recipe, but the baseline comparison and internal inconsistencies undercut the boundary-precision claim. the 4 major comments →

arxiv 2607.21546 v1 pith:K6ACARWF submitted 2026-07-23 cs.CV

UnDA: Unpaired Domain Alignment for Cross-Modal Knowledge Transfer in Medical Imaging

classification cs.CV
keywords cross-modal knowledge distillationunpaired domain adaptationmedical image segmentationoptimal transportuncertainty weightingprototype contrastive learningclass tokensMRI-to-CT transfer
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper aims to show that a segmentation model for one imaging modality can be improved using labeled images from a different modality, even when the two scan sets are unpaired. Its framework, UnDA, freezes a source-trained 'anchor' model, extracts compact class tokens from both encoders, and aligns target tokens to anchor tokens with an optimal-transport loss whose transport masses are reweighted by source prediction confidence, plus a per-class prototype contrastive objective. The alignment module is removed at inference, so the deployed target model adds zero computational overhead. Across brain-tumor (T2-FLAIR to T1-Native) and cardiac (MRI to CT) benchmarks, UnDA reports consistent gains in Dice and boundary distance (HD95) over the same model trained only on target data, and the best average boundary metric among the compared methods.

Core claim

The paper claims that structural knowledge can be transferred across unpaired modalities by aligning compact class tokens instead of raw features. Its central discovery is that weighting the optimal-transport plan by source prediction confidence suppresses noisy supervision, while a per-class prototype contrastive objective stabilizes alignment across unpaired batches. Across BraTS 2023 and MM-WHS, the full framework consistently improves Dice and HD95 over the target-only baseline; on MM-WHS it reaches 82.71% mean Dice and 13.94 mm HD95, trailing the best compared method by 3.28 Dice points on average but posting the best mean HD95. The alignment module is removed at inference, so the repor

What carries the argument

The Alignment Module pools bottleneck features into K class tokens via attention, giving each token a semantic-class meaning; a frozen anchor model provides reference tokens and updates class prototypes by exponential moving average. UCT-OT solves an entropy-regularized optimal transport problem (Sinkhorn) whose source marginal is weighted by exp(−α·entropy), so confident source tokens carry more mass. ProtoNCE pulls target tokens toward the prototype memory, enforcing global discriminability. A two-layer residual MLP adapter maps target tokens into the shared token space. All alignment components are discarded after training.

Load-bearing premise

The framework assumes labeled target-domain images are available for the supervised segmentation loss; if target labels are missing, the claimed transfer gains have not been demonstrated.

What would settle it

An ablation that removes the target segmentation loss L_seg from UnDA's Phase 2 (keeping only UCT-OT and ProtoNCE) and compares results on BraTS or MM-WHS against the same target model trained with those labels would reveal whether the improvements come from cross-modal alignment or simply from reusing target labels; if Dice and HD95 return to the target-only baseline, the paper's central transfer claim is unsupported.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • If the framework works as claimed, CT-only segmentation models can be trained to near-MRI accuracy using only MRI labels at training time, with zero extra computation at deployment.
  • The uncertainty-weighting principle could reduce noise from unreliable predictions in any distillation or pseudo-labeling pipeline, not just cross-modal medical imaging.
  • The reported boundary-distance gains on MM-WHS imply fewer large localized segmentation errors, which matters more clinically than small uniform Dice improvements.
  • Because the alignment module is backbone-agnostic, it can be attached to existing encoders without redesigning the architecture, making the gains portable to new segmentation tasks.
  • Gains on small, high-variability cardiac structures suggest the method specifically helps where domain shift is most severe.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Because UnDA uses labeled target images during training, its comparison against unsupervised UDA baselines is not apples-to-apples; a fair test would either grant those baselines the same target labels or remove UnDA's target segmentation loss entirely.
  • The uncertainty-weighting recipe—scaling source contributions by prediction confidence—is general and could be applied to feature-alignment losses in other modality gaps (e.g., histology-to-radiology) or even to cross-modal tasks outside medicine.
  • A natural extension would be to withholds target labels from Phase 2 and measure how much UCT-OT and ProtoNCE alone transfer; the paper does not report this, so the relative contribution of label reuse versus alignment is unquantified.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes UnDA, a two-stage framework for unpaired cross-modal medical image segmentation. In Phase 1, an anchor model is trained on a labeled source modality with a class-token pooling module. In Phase 2, the anchor is frozen and the target model is trained with segmentation supervision on labeled target images plus two auxiliary losses: Uncertainty-Weighted Optimal Transport (UCT-OT) and a prototypical contrastive loss (ProtoNCE). The alignment module is removed at inference. Experiments are reported on BraTS 2023 (T2-FLAIR to T1-native) and MM-WHS (MRI to CT), with comparisons against several unsupervised UDA baselines on MM-WHS and an ablation study. The central claim is that UnDA consistently improves target-modality accuracy and boundary precision without paired training data.

Significance. If the claims are substantiated, the contribution is valuable: it provides a backbone-agnostic way to distill structural knowledge across unpaired modalities, with zero inferencetime overhead, and reports large improvements over target-only baselines (e.g., MM-WHS mean Dice 71.59 to 82.71 and mean HD95 21.41 to 13.94). The idea of uncertainty-weighting in optimal transport alignment is a reasonable and potentially useful extension over plain OT-based domain alignment. The use of class tokens and prototype memory is also well-motivated. However, the current manuscript has several load-bearing empirical problems that must be resolved before the claims can be accepted.

major comments (4)
  1. [§2.1, §2.3, Table 3] The target phase optimizes Lseg = Dice + BCE (Eq. 10) on labeled target images D_G defined in §2.1, so UnDA uses target ground-truth labels. Table 3 compares against SE_ASA, DAR-UNet, SIFA v2, MAPSeg, and UMMKD, which are unsupervised UDA methods by design. The paper never states whether these baselines were given target labels. If the official implementations were run without target labels while UnDA uses them, the comparison is confounded: the reported mean-Dice advantage over SIFA v2 (+11.38) and MAPSeg (+21.36) could come from target supervision rather than from UCT-OT or ProtoNCE. The authors must specify the labeling protocol for each baseline and ideally run a controlled comparison where baselines receive the same target labels, or re-frame the method as target-supervised adaptation rather than unsupervised UDA.
  2. [§3.3 (Table 2) vs §4 (Table 4)] The 'UnDA (Ours)' row is inconsistent between the main results and the ablation. Table 2 reports LA Dice 89.58 and LV HD95 9.11, while Table 4, which uses the same column order (LV Myo, LA, LV, RA, RV, AA, PA), reports LA Dice 85.53 and LV HD95 11.32 for the same final model. These are not rounding differences. This inconsistency makes the component-wise conclusions in Section 4 (e.g., the gains of UCT-OT and ProtoNCE) unreliable as printed. All numbers should be verified so that identical configurations yield identical metrics across tables.
  3. [§4, Table 4] In the ablation, the no-alignment baseline 'MR→CT' has mean HD95 13.58 mm, which is better than the full UnDA model's mean HD95 13.94 mm. Adding ProtoNCE (16.91), plain OT (18.63), or UCT-OT (17.12) individually also worsens mean HD95 relative to no alignment. This directly contradicts the abstract's claim of 'improved boundary precision' and undermines the statement that uncertainty-weighted OT improves boundary accuracy. The authors need to explain this pattern, correct the numbers if there is an error, or soften the HD95 claim.
  4. [§1/§3.3 vs Table 1] Several quantitative claims in the text do not match Table 1. The abstract says Tumor Core HD95 is reduced from 29.1 mm to 12.4 mm, but Table 1 reports 29.12 mm to 10.10 mm. In §3.3, the text says Dice improves by +2.3 (WT), +4.1 (TC), and +4.0 (ET), but the table values imply +5.0, +8.8, and +9.3, respectively. The stated HD95 reductions (7.21 mm WT, 16.69 mm TC, 11.42 mm ET) also differ from those computed from the table (10.70 mm WT, 19.02 mm TC, 6.64 mm ET). These inconsistencies must be resolved before the experimental claims can be trusted.
minor comments (5)
  1. [Tables 1–4] No error bars, standard deviations, or number of runs are reported for any metric. Given the inconsistencies already present, the authors should provide variance estimates and ideally statistical significance tests for the central comparisons.
  2. [§3.4] The description of how baselines were modified to the common backbone is too terse. For each baseline, specify which modules were kept, which were replaced, and how hyperparameters (e.g., loss weights, training epochs) were chosen, so that the comparison is reproducible.
  3. [Table 3] UMMKD achieves higher mean Dice (85.99 vs 82.71) and better HD95 on several individual structures. The text frames UnDA as 'best mean HD95' and 'more balanced,' which is defensible, but the claim in the abstract of 'consistent improvements' should be qualified given that a prior method outperforms UnDA on mean Dice.
  4. [§2.2, Eq. (8)] The notation is slightly unclear: wi is defined with both i and k indices in 'a_i = wi / sum_k wk', which should be written as a_i = exp(-αu_i) / Σ_k exp(-αu_k). Please fix the subscript ambiguity.
  5. [Throughout] There are several typographical issues, including 'T able' in table headings, and the references section formatting. A careful proofread is advised.

Circularity Check

0 steps flagged

No circular derivation found; the main concern is a target-label supervision confound in baseline comparisons, which is an experimental issue rather than circularity.

full rationale

UnDA's training objective is defined independently. Phase 2 optimizes L_target = L_seg + lambda_OT L_UCT-OT + lambda_P L_ProtoNCE (Eq. 12), where L_seg = Dice + BCE (Eq. 10) is a standard supervised segmentation loss on the labeled target data D_G, and L_UCT-OT (Eq. 13) is computed from model features via the transport plan in Eq. 9. ProtoNCE uses prototype memories updated from frozen anchor tokens (Eq. 6). None of the reported Dice/HD95 numbers are used as inputs to these losses; the uncertainty weights w_i = exp(-alpha u_i) and the loss weights lambda_OT = 0.2, lambda_P = 0.5 are hyperparameters, not fitted to the evaluation metric. The anchor model is a frozen teacher, not a renamed target predictor. There are no author self-citations, and no uniqueness result or ansatz is imported from the authors' prior work. The strongest legitimate concern is experimental: Phase 2 uses labeled target images, while SIFA v2 and MAPSeg are unsupervised UDA methods by design; if the modified baseline implementations did not receive target labels, the Table 3 comparison is confounded by additional supervision. This is a fairness/validity concern, not a circular step. Separately, Table 4's UnDA row reports LA Dice 85.53 and HD95 11.32, while Table 2 reports LA Dice 89.58 and HD95 9.11 for the identical model; this internal inconsistency undermines the ablation's component-wise conclusions but again is not circularity. Overall, the paper's central empirical claims do not reduce by construction to their own inputs.

Axiom & Free-Parameter Ledger

5 free parameters · 4 axioms · 0 invented entities

The central claims rest on a handful of hyperparameters (lambda_OT, lambda_P, alpha, epsilon, beta), a critical domain assumption that target labels are available, and an ad hoc confidence-weighting form. No new physical entities are introduced. The prototype memory and class tokens are algorithmic constructs, not independently evidenced entities.

free parameters (5)
  • lambda_OT = 0.2
    Loss weight for UCT-OT in Eq. (12); no search procedure or validation curve reported.
  • lambda_P = 0.5
    Loss weight for ProtoNCE; default chosen, not justified.
  • alpha (uncertainty temperature) = not reported
    Controls confidence weighting in Eq. (8); value never given, so the core uncertainty mechanism is under-specified.
  • epsilon (Sinkhorn entropy regularizer) = not reported
    Affects the transport plan in Eq. (9) but is not specified in the paper.
  • beta (EMA momentum for prototype memory) = not reported
    Controls prototype update rate in Eq. (6); value not given.
axioms (4)
  • standard math Entropy-regularized OT solved by Sinkhorn iterations converges and yields a valid transport plan
    Invoked in Eq. (9) via [8] without discussing convergence guarantees or epsilon schedule.
  • domain assumption Target-domain dense labels are available for Lseg in Phase 2
    Problem setup §2.1 defines labeled target data D_G; Eq. (10) uses Dice+BCE on target. This makes it supervised target training, not unsupervised domain adaptation.
  • domain assumption Class tokens learned on anchor features correspond to the same semantic classes in target after linear projection
    Eqs. (4)-(5) pool bottleneck features into K class tokens; alignment assumes these are semantically comparable across modalities.
  • ad hoc to paper Exponential weighting wi = exp(-alpha ui) monotonically down-weights uncertain tokens
    Eq. (8) introduces alpha without derivation or ablation of the functional form.

pith-pipeline@v1.3.0-alltime-deepseek · 7973 in / 13238 out tokens · 123981 ms · 2026-08-01T07:05:51.259198+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of UnDA: Unpaired Domain Alignment for Cross-Modal Knowledge Transfer in Medical Imaging." pith.science (2026). https://pith.science/paper/K6ACARWF

@misc{pith2026260721546,
  author       = {Pith},
  title        = {Pith review of: UnDA: Unpaired Domain Alignment for Cross-Modal Knowledge Transfer in Medical Imaging},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/K6ACARWF}},
  note         = {Machine review of arXiv:2607.21546}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Multimodal based approaches often outperform single modality approaches in downstream tasks as the different modalities provide complementary information, yet acquiring paired clinical data remains a significant challenge in real world scenarios. While cross-modal knowledge distillation addresses this, existing methods often struggle with large modality gaps and the propagation of noise from uncertain source-domain predictions. To overcome these challenges, we propose UnDA, an anchor-guided framework for unpaired cross-modal distillation. Our approach introduces a backbone-agnostic Alignment Module that extracts semantically structured class tokens via an attention based pooling mechanism. To ensure robust knowledge transfer, we propose Uncertainty-Weighted Optimal Transport (UCT-OT), which dynamically weights feature-level alignment based on prediction confidence, effectively suppressing noisy supervision. Furthermore, a per-class ProtoNCE objective maintains stable prototype memories to enforce global discriminability across unpaired batches. Evaluations on representative segmentation tasks under strictly unpaired settings show consistent improvements in accuracy and boundary precision in the target modality, demonstrating that meaningful structural knowledge can be transferred across heterogeneous data sources without paired datasets.

Figures

Figures reproduced from arXiv: 2607.21546 by Abu Raihan Mostofa Kamal, Ahsan Bulbul, Md Azam Hossain, Rafsan Jany, Shadab Tanjeed Ahmad, Tahsinul Islam.

Figure 1
Figure 1. Figure 1: Overview of UnDA: Phase 1 trains the anchor model on source-domain im￾ages; a presence head supervises the class token pool via LBCE-pres, producing class￾discriminative token representations. Phase 2 freezes the anchor to generate per-class uncertainty maps and update prototype memory Pc; the target encoder produces class tokens refined by an MLP residual adapter and aligned via prototypical contrastive l… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

23 extracted references · 3 linked inside Pith

  1. [1]

    IEEE Trans

    Dou, Q., Liu, Q., Heng, P.A., Glocker, B.: Unpaired multi-modal segmentation via knowledge distillation. IEEE Trans. Med. Imaging39(7), 2415–2425 (2020)

  2. [2]

    IEEE Transactions on Medical Imaging41(5), 1057–1068 (2021)

    Jiang, J., Rimner, A., Deasy, J.O., Veeraraghavan, H.: Unpaired Cross-Modality Educed Distillation (CMEDL) for Medical Image Segmentation. IEEE Transactions on Medical Imaging41(5), 1057–1068 (2021)

  3. [3]

    International Conference on Medical Image Computing and Computer-Assisted In- tervention (MICCAI)14227, 113–123 (2023)

    Guan, Q., Xie, Y., Yang, B., Zhang, J., Liao, Z., Wu, Q., Xia, Y.: Unpaired Cross- Modal Interaction Learning for COVID-19 Segmentation on Limited CT Images. International Conference on Medical Image Computing and Computer-Assisted In- tervention (MICCAI)14227, 113–123 (2023)

  4. [4]

    Inter- national Conference on Medical Image Computing and Computer-Assisted Inter- vention (MICCAI)12905, 307–316 (2021)

    Wang, Q., Che, H., Ding, W., Xiang, L., Li, G., Li, Z., Cui, S.: Colorectal Polyp Classification from White-light Colonoscopy Images via Domain Alignment. Inter- national Conference on Medical Image Computing and Computer-Assisted Inter- vention (MICCAI)12905, 307–316 (2021)

  5. [5]

    arXiv preprint arXiv:2407.17620 (2024)

    Alberb, M., Elbatel, M., Elgebaly, A., Montoya-del-Angel, R., Li, X., Martí, R.: Co- MoTo: Unpaired Cross-Modal Lesion Distillation Improves Breast Lesion Detection in Tomosynthesis. arXiv preprint arXiv:2407.17620 (2024)

  6. [6]

    In: Advances in Neural Information Processing Systems (NeurIPS), vol

    Ryoo,M.,Piergiovanni,A.J.,Arnab,A.,Dehghani,M.,Angelova,A.:TokenLearner: Adaptive Space-Time Tokenization for Videos. In: Advances in Neural Information Processing Systems (NeurIPS), vol. 34, pp. 12786–12797 (2021)

  7. [7]

    In: International Conference on Learn- ing Representations (ICLR) (2021)

    Dosovitskiy, A., Beyer, L., Kolesnikov, A., et al.: An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. In: International Conference on Learn- ing Representations (ICLR) (2021)

  8. [8]

    In: Advances in Neural Information Processing Systems (NeurIPS), vol

    Cuturi, M.: Sinkhorn Distances: Lightspeed Computation of Optimal Transport. In: Advances in Neural Information Processing Systems (NeurIPS), vol. 26, pp. 2292– 2300 (2013)

  9. [9]

    arXiv preprint arXiv:2107.02314 (2021)

    Baid, U., Ghodasara, S., Mohan, S., et al.: The RSNA-ASNR-MICCAI BraTS 2021 Benchmark on Brain Tumor Segmentation and Radiogenomic Classification. arXiv preprint arXiv:2107.02314 (2021)

  10. [10]

    IEEE Transactions on Medical Imaging34(10), 1993–2024 (2015)

    Menze, B.H., Jakab, A., Bauer, S., et al.: The Multimodal Brain Tumor Image Seg- mentation Benchmark (BRATS). IEEE Transactions on Medical Imaging34(10), 1993–2024 (2015)

  11. [11]

    Sci- entific Data4(1) (2017)

    Bakas, S., Akbari, H., Sotiras, A., et al.: Advancing The Cancer Genome Atlas glioma MRI collections with expert segmentation labels and radiomic features. Sci- entific Data4(1) (2017)

  12. [12]

    The Cancer Imaging Archive (2017).https: //doi.org/10.7937/K9/TCIA.2017.KLXWJJ1Q

    Bakas, S., Akbari, H., Sotiras, A., et al.: Segmentation Labels for the Pre-operative Scans of the TCGA-GBM collection. The Cancer Imaging Archive (2017).https: //doi.org/10.7937/K9/TCIA.2017.KLXWJJ1Q

  13. [13]

    The Cancer Imaging Archive (2017).https: //doi.org/10.7937/K9/TCIA.2017.GJQ7R0EF UnDA: Unpaired Domain Alignment for cross modal tasks 11

    Bakas, S., Akbari, H., Sotiras, A., et al.: Segmentation Labels for the Pre-operative Scans of the TCGA-LGG collection. The Cancer Imaging Archive (2017).https: //doi.org/10.7937/K9/TCIA.2017.GJQ7R0EF UnDA: Unpaired Domain Alignment for cross modal tasks 11

  14. [14]

    IEEE Transactions on Pattern Analysis and Machine Intelli- gence41(12), 2933–2946 (2019)

    Zhuang, X.: Multivariate Mixture Model for Myocardial Segmentation Combining Multi-Source Images. IEEE Transactions on Pattern Analysis and Machine Intelli- gence41(12), 2933–2946 (2019)

  15. [15]

    Medical Image Analysis31, 77–87 (2016)

    Zhuang, X., Shen, J.: Multi-Scale Patch and Multi-Modality Atlases for Whole Heart Segmentation of MRI. Medical Image Analysis31, 77–87 (2016)

  16. [16]

    GitHub repository (2023).https://github

    Wang, L.: 3DUNet-BraTS-PyTorch. GitHub repository (2023).https://github. com/LightersWang/3DUNet-BraTS-PyTorch

  17. [17]

    In: International Conference on Learning Representations (ICLR) (2021).https://openreview.net/forum?id=KmykpuSrjcq

    Li, J., Zhou, P., Xiong, C., Hoi, S.C.H.: Prototypical Contrastive Learning of Unsu- pervised Representations. In: International Conference on Learning Representations (ICLR) (2021).https://openreview.net/forum?id=KmykpuSrjcq

  18. [18]

    In: European Conference on Computer Vision (ECCV), pp

    Damodaran, B.B., Kellenberger, B., Flamary, R., Tuia, D., Courty, N.: DeepJDOT: Deep Joint Distribution Optimal Transport for Unsupervised Domain Adaptation. In: European Conference on Computer Vision (ECCV), pp. 467–483 (2018)

  19. [19]

    IEEE Journal of Biomedical and Health Informatics26(10), 4976– 4986 (2022)

    Yao, K., Su, Z., Huang, K., Yang, X., Sun, J., Hussain, A., Coenen, F.: A Novel 3D Unsupervised Domain Adaptation Framework for Cross-Modality Medical Image Segmentation. IEEE Journal of Biomedical and Health Informatics26(10), 4976– 4986 (2022)

  20. [20]

    Proceedings of the AAAI Conference on Artificial Intelligence37(1), 623–631 (2023)

    Feng, W., Ju, L., Wang, L., Song, K., Zhao, X., Ge, Z.: Unsupervised Domain Adaptation for Medical Image Segmentation by Selective Entropy Constraints and Adaptive Semantic Alignment. Proceedings of the AAAI Conference on Artificial Intelligence37(1), 623–631 (2023)

  21. [21]

    In: Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp

    Zhang, X., Wu, Y., Angelini, E., Li, A., Guo, J., Rasmussen, J.M., O’Connor, T.G., Wadhwa, P.D., Jackowski, A.P., Li, H., Posner, J., Laine, A.F., Wang, Y.: MAPSeg: Unified Unsupervised Domain Adaptation for Heterogeneous Medical Image Seg- mentation Based on 3D Masked Autoencoding and Pseudo-Labeling. In: Proceed- ings of the IEEE/CVF Conference on Compu...

  22. [22]

    arXiv preprint arXiv:2002.02255 (2020)

    Chen, C., Dou, Q., Chen, H., Qin, J., Heng, P.A.: Unsupervised Bidirectional Cross-Modality Adaptation via Deeply Synergistic Image and Feature Alignment for Medical Image Segmentation. arXiv preprint arXiv:2002.02255 (2020)

  23. [23]

    In: Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence (AAAI), pp

    Chen, C., Dou, Q., Chen, H., Qin, J., Heng, P.A.: Synergistic Image and Fea- ture Adaptation: Towards Cross-Modality Domain Adaptation for Medical Image Segmentation. In: Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence (AAAI), pp. 865–872 (2019)