Pith. sign in

REVIEW 4 major objections 6 minor 53 references

MulModSeg: Enhancing Unpaired Multi-Modal Medical Image Segmentation with Modality-Conditioned Text Embedding and Alternating Training

T0 review · 4 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read MulModSeg adds a frozen text encoder and alternating training so one segmentation model handles CT and MR scans without paired data.

desk verdict Modest but useful extension of text-driven segmentation to unpaired CT/MR; the modality-conditioning claim is weaker than the Dice gains suggest. read the letter →

arxiv 2411.15576 v1 pith:KZOAZP6E submitted 2024-11-23 eess.IV cs.CV

Chengyin Li , Hui Zhu , Rafi Ibn Sultan , Hassan Bagher Ebadian , Prashant Khanduri , Chetty Indrin , Kundan Thind , Dongxiao Zhu This is my paper · ORCID
classification eess.IVcs.CV
keywords medicalimagesegmentationunpairedmulti-modalCTandMRimagingmodality-conditionedtextembeddingalternatingtrainingfrozenCLIPencoderDicescore
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes a simple add-on, MulModSeg, that lets a single encoder-decoder segmentation network handle both CT and MR images without paired scans. Its two ingredients are a frozen text encoder that turns prompts such as 'A CT imaging of a spleen' into conditioning vectors, and an alternating training loop that feeds CT and MR batches in turn. The authors report mean Dice of 87.14 on CT and 85.33 on MR for abdominal organs with a UNet backbone, up from 82.50 and 81.91 for a vision-only model, and similar or larger gains with a SwinUNETR backbone and on cardiac substructures. If correct, this offers a way to add modality awareness to existing architectures without new branches, paired data, or large computational overhead.

What carries the argument

The load-bearing mechanism is a text-driven segmentation head. A frozen CLIP text encoder (a contrastive vision-language model pretrained on natural images) converts a modality-and-organ prompt into a vector; this vector is concatenated with a global image feature from the encoder and passed through an MLP that emits the weights and biases for three 1x1x1 convolutions applied to the decoder's final feature map. The second mechanism is alternating training (ALT), which cycles CT and MR batches so a single model is updated by both modalities in each iteration. Together they let one backbone produce class- and modality-aware masks without modifying the encoder-decoder structure or requiring paired data.

What would settle it

Run the AMOS experiment with a class-only prompt that omits the modality word, keeping ALT training otherwise identical; if the Dice scores match the V3-CLIP results (87.14 CT, 85.33 MR), the modality word is not carrying the gain. The paper's own confused-prompt result (drops of 0.77 on CT and 0.23 on MR) makes this the decisive check.

Watch

Extended reading notes

Core claim

MulModSeg's central claim is that modality awareness can be injected into an encoder-decoder segmentation network at almost no architectural cost. For each organ class, a prompt of the form 'A {CT/MR} imaging of a [CLS]' is passed through the frozen CLIP text encoder; the resulting vector is concatenated with the pooled encoder features, and a small MLP turns the pair into the weights of three 1x1x1 convolutions that produce the class mask. Training alternates CT and MR batches so the same weights see both modalities in every iteration. On AMOS abdominal organs with a UNet backbone the method reports mean Dice 87.14 for CT and 85.33 for MR, versus 82.50 and 81.91 for a vision-only baseline, and on MMWHS cardiac substructures it reports 92.72 (UNet) and 93.31 (SwinUNETR) mean Dice on the CT target, above the compared methods.

Load-bearing premise

The key assumption is that a text encoder trained on photographs of everyday objects can tell CT from MR through words alone well enough to improve medical segmentation; the paper's own prompt-swap test suggests this signal is small.

Editorial extensions

If this is right

  • One model replaces two modality-specific segmenters for CT and MR, reducing annotated-data and storage costs.
  • The gain persists when CT outnumbers MR by 3 to 1, so the method suits imbalanced clinical datasets.
  • Because the text embeddings are frozen and precomputed, the added parameters and inference cost are negligible.
  • Changing the prompt at inference time is enough to point the model at a different modality, making deployment to a new scanner type a text change rather than a retrain.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The confused-prompt experiment suggests the modality word itself contributes little: a test with class-only prompts could determine whether the real driver is generic text conditioning rather than true CT-vs-MR awareness.
  • If generic text conditioning is the active ingredient, the same MLP-generated convolution weights could be reused for other conditioning axes, such as scanner manufacturer, contrast phase, or organ-at-risk.
  • Because the text encoder is frozen, extending MulModSeg to new modalities only requires new prompts, but the value of that extension depends on whether the text embedding space separates those modality names as strongly as it separates CT from MR.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes MulModSeg, a training strategy for unpaired multi-modal medical image segmentation (CT and MR). The method augments standard encoder-decoder backbones (3D UNet and SwinUNETR) with a text branch that uses a frozen CLIP text encoder to produce modality-conditioned class embeddings, and an alternating training (ALT) procedure that alternately feeds CT and MR batches. Experiments on AMOS (abdominal multi-organ) and MMWHS (cardiac substructures) report Dice improvements over vision-only baselines and several prior methods, with the best results obtained using a domain-specific prompt template (V3-CLIP). The main claims are that modality-conditioned text embeddings add modality awareness with negligible architectural modification, and that ALT enables effective learning from unpaired modalities.

Significance. If substantiated, the method would offer a lightweight way to make existing segmentation architectures multi-modal without paired data or major structural changes, which is practically appealing. Strengths include the release of code, validation across two backbones and two datasets, and a clear ablation structure that isolates text embedding and ALT contributions. However, the central novelty—modality conditioning via CLIP text embeddings—rests on evidence that is currently weak: the confused-prompt experiment shows only tiny performance drops, and the evaluation lacks statistical grounding. The paper also selects its prompt template on the test set, which inflates the headline numbers. The claimed consistent superiority over prior methods is therefore not yet convincingly established.

major comments (4)
  1. [Table 1 and Section 4.2] The prompt template V3-CLIP is selected after evaluating V1, V2, and V3 on the AMOS test set, since Table 1 reports test-set Dice scores for each template. This makes the headline values (87.14 CT, 85.33 MR) optimistic and contradicts the statement in Section 4.1 that hyperparameters are obtained through cross-validation over the training set. The template should be chosen on a validation split or through nested cross-validation, and the reported test scores should be from a model whose design did not use the test set for selection.
  2. [Tables 2, 3, 4, 5 and Section 4.1] No error bars or significance tests are reported for any of the experimental results, and the model is evaluated using the last training epoch rather than a validation-based selection. Given the small test sets (e.g., only 5 volumes per modality in the MMWHS split) and the small observed differences in several comparisons (e.g., Table 2 SwinUNETR CT: 85.02 ALT vs 85.06 without ALT; Table 7 drops of 0.77 and 0.23 Dice points), the claim that MulModSeg 'consistently outperforms' baselines is not statistically supported. The authors should report mean and standard deviation over multiple seeds and perform appropriate significance tests, especially for the claimed improvements that are smaller than one Dice point.
  3. [Table 7 and Section 4.2] The confused-prompt experiment in Table 7 provides weak evidence for the central modality-conditioning mechanism: applying the wrong modality prompt lowers Dice by only 0.77 on CT and 0.23 on MR. If the text embedding carried substantial modality-specific information, one would expect a clearly larger penalty when every test sample receives the wrong modality conditioning. This result suggests the model may be relying mainly on class information and the visual appearance of the image, with the modality prompt contributing little. To support the claimed modality awareness, the authors should provide a direct analysis of the text embeddings (e.g., cosine distances between 'CT imaging of X' and 'MR imaging of X' versus distances between different classes), and include an ablation with a shared modality-neutral prompt to quantify the contribution of the modality token specifically.
  4. [Table 4] The comparison with state-of-the-art methods is not sufficiently controlled. The caption states that 'All the techniques have the same UNet [34] and SwinUNETR [12] baseline,' but reference [34] is the X-shape method rather than UNet, and it is unclear whether the listed prior methods (e.g., Li et al. [19], Bastico et al. [1]) were reimplemented with the same backbone, training epochs, loss, and data split. If these methods were taken from their original papers with different backbones or protocols, the reported differences do not isolate the benefit of MulModSeg. Please clarify the exact setup for each baseline or rerun all baselines under identical conditions.
minor comments (6)
  1. [References] The citation [7] used for 'frozen CLIP' is not the CLIP paper (Conneau and Lample is a cross-lingual language model), and the actual CLIP reference (Radford et al., 2021) is missing. Please correct the citation.
  2. [Section 3.2] The generation of convolution parameters from the MLP is underspecified: it is not stated how many hidden layers the MLP has, how the output vector is split into weights and biases for the three Conv-2 layers, or whether the MLP is shared across classes. Please provide exact dimensional details for reproducibility.
  3. [Section 4.1] The balanced data splitting says 'each with 54 scans' for AMOS, but AMOS contains 162 CT and 54 MR scans; specifying that only 54 of the 162 CT scans were used in the balanced setting would avoid ambiguity.
  4. [Table 8] The inference time overhead is not 'comparable' as stated: with text embedding, UNet inference time increases from 2.402s to 3.272s (36%) and SwinUNETR from 3.387s to 4.197s (24%). If this overhead is acceptable, the text should acknowledge the actual increase rather than describing it as negligible.
  5. [Algorithm 1] The alternating training loop runs for max(len(CT_loader), len(MR_loader)) iterations and cycles the shorter loader, which means samples from the larger dataset may be seen multiple times per epoch while the smaller dataset is fully cycled. The potential imbalance effect on training should be discussed or the algorithm should be described more precisely.
  6. [Table 6] The label 'MR' in the rows 'training with MR data' is ambiguous because the table header only shows CT and MR average Dice; please clarify which data was used for training in each row (e.g., 'trained on CT only' vs 'trained on MR only').

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the text embeddings come from a frozen external CLIP encoder, the parameter generation follows cited external designs, and the reported gains are tested against external benchmarks; prompt selection and the weak confused-prompt ablation are correctness concerns, not circular reasoning.

full rationale

The manuscript contains no circular argument in the sense used by this pass. The text embeddings are generated by a frozen external CLIP encoder using fixed templates and are not fitted to the segmentation labels or to the reported test Dice values; the controller MLP and conditional convolution parameterization follow explicitly cited external designs [22, 32]. ALT is a deterministic data-loading schedule and is not defined in terms of the predicted masks or the evaluation metric. The self-citations that appear ([17, 18]) are limited to related-work background and are not load-bearing for the central claim. Prompt-template selection does introduce a possible evaluation bias: Table 1 is used to choose V3-CLIP as the default, and the manuscript later reports V3-CLIP as the best configuration, so the headline numbers should be read as model-selected rather than as a single pre-registered configuration. However, this is a statistical-selection concern, not a derivation that reduces to its own inputs, and the method is still benchmarked against external state-of-the-art results. The weak confused-prompt ablation in Table 7 is a genuine threat to the interpretation that the text branch encodes modality rather than class information, but it is an empirical weakness of the mechanism, not circularity: the claim remains externally testable. One reference-quality issue is that CLIP is cited via reference [7], which appears to describe a different language-model pretraining work, but this is a citation error and not evidence of circularity. No equation, fitted parameter, or self-citation chain makes the reported predictions equivalent to the paper's inputs by construction.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The central claim rests on two assumptions: that CLIP text embeddings convey modality information useful for segmentation, and that unpaired CT/MR data can be trained jointly via ALT. The first is weakly supported by Table 7; the second is the problem setting itself. No invented entities are introduced.

free parameters (3)
  • Prompt template selection = A {CT/MR} imaging of a [CLS]
    V3-CLIP was chosen as the default because it achieved the highest Dice on the AMOS test set (Table 1), which is test-set selection rather than validation-based selection.
  • Output channels of FPre_head = 8
    Design choice stated in Section 3.2; affects the capacity of the text-driven head but is not optimized in the paper.
  • CT:MR data ratio = 1:1 for main experiments
    Balanced splitting is used for the main results; other ratios are tested in Table 5, so the choice is a controlled experimental variable.
assumptions (3)
  • domain assumption Frozen CLIP text encoder provides meaningful embeddings of medical modality-organ phrases.
    Invoked in Section 3.2 and tested in Table 1; the small drop in Table 7 from wrong modality prompts suggests the modality component is weak.
  • domain assumption Unpaired CT and MR images from different patients can be jointly trained with ALT without performance degradation from domain shift.
    This is the core problem setting; the paper does not compare to image translation or registration methods beyond citing them.
  • standard math Sum of Dice loss and cross-entropy loss with the specified preprocessing is appropriate for multi-organ segmentation.
    Standard choices in medical segmentation; not central to the method's novelty.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MulModSeg: Enhancing Unpaired Multi-Modal Medical Image Segmentation with Modality-Conditioned Text Embedding and Alternating Training." pith.science (2026). https://pith.science/paper/KZOAZP6E

@misc{pith2026241115576,
  author       = {Pith},
  title        = {Pith review of: MulModSeg: Enhancing Unpaired Multi-Modal Medical Image Segmentation with Modality-Conditioned Text Embedding and Alternating Training},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KZOAZP6E}},
  note         = {Machine review of arXiv:2411.15576}
}
read the original abstract

In the diverse field of medical imaging, automatic segmentation has numerous applications and must handle a wide variety of input domains, such as different types of Computed Tomography (CT) scans and Magnetic Resonance (MR) images. This heterogeneity challenges automatic segmentation algorithms to maintain consistent performance across different modalities due to the requirement for spatially aligned and paired images. Typically, segmentation models are trained using a single modality, which limits their ability to generalize to other types of input data without employing transfer learning techniques. Additionally, leveraging complementary information from different modalities to enhance segmentation precision often necessitates substantial modifications to popular encoder-decoder designs, such as introducing multiple branched encoding or decoding paths for each modality. In this work, we propose a simple Multi-Modal Segmentation (MulModSeg) strategy to enhance medical image segmentation across multiple modalities, specifically CT and MR. It incorporates two key designs: a modality-conditioned text embedding framework via a frozen text encoder that adds modality awareness to existing segmentation frameworks without significant structural modifications or computational overhead, and an alternating training procedure that facilitates the integration of essential features from unpaired CT and MR inputs. Through extensive experiments with both Fully Convolutional Network and Transformer-based backbones, MulModSeg consistently outperforms previous methods in segmenting abdominal multi-organ and cardiac substructures for both CT and MR modalities. The code is available in this {\href{https://github.com/ChengyinLee/MulModSeg_2024}{link}}.

Figures

Figures reproduced from arXiv: 2411.15576 by the authors.

Figure 1
Figure 1. Schematic representation of the MulModSeg strategy for multi-modal medical image segmentation with (A) modality-conditioned [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Visual comparison of segmentation results on AMOS [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

53 extracted references · 43 canonical work pages

  1. [22]

    Clip-driven universal model for organ segmentation and tumor detection

    Jie Liu, Yixiao Zhang, Jie-Neng Chen, Junfei Xiao, Yongyi Lu, Bennett A Landman, Yixuan Yuan, Alan Yuille, Yucheng Tang, and Zongwei Zhou. Clip-driven universal model for organ segmentation and tumor detection. In Proceedings of the IEEE/CVF International Conference on Computer Vi- sion, pages 21152–21164, 2023. 2, 3

    work page 2023

  2. [34]

    Multi-modal learning from unpaired images: Application to multi-organ segmentation in ct and mri

    Vanya V Valindria, Nick Pawlowski, Martin Rajchl, Ioannis Lavdas, Eric O Aboagye, Andrea G Rockall, Daniel Rueck- ert, and Ben Glocker. Multi-modal learning from unpaired images: Application to multi-organ segmentation in ct and mri. In 2018 IEEE winter conference on applications of com- puter vision (WACV), pages 547–556. IEEE, 2018. 2, 7

    work page 2018

  3. [12]

    Swin unetr: Swin transformers for semantic segmentation of brain tumors in mri images

    Ali Hatamizadeh, Vishwesh Nath, Yucheng Tang, Dong Yang, Holger R Roth, and Daguang Xu. Swin unetr: Swin transformers for semantic segmentation of brain tumors in mri images. In International MICCAI Brainlesion Workshop, pages 272–284. Springer, 2021. 1, 2, 3, 7

    work page 2021

  4. [19]

    Towards cross-modality medical image segmentation with online mutual knowledge distillation

    Kang Li, Lequan Yu, Shujun Wang, and Pheng-Ann Heng. Towards cross-modality medical image segmentation with online mutual knowledge distillation. In Proceedings of the AAAI conference on artificial intelligence , pages 775–783,

    work page

  5. [1]

    A simple and robust frame- work for cross-modality medical image segmentation ap- plied to vision transformers

    Matteo Bastico, David Ryckelynck, Laurent Cort ´e, Yannick Tillier, and Etienne Decenci`ere. A simple and robust frame- work for cross-modality medical image segmentation ap- plied to vision transformers. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 4128– 4138, 2023. 5, 7

    work page 2023

  6. [2]

    Monai: An open-source framework for deep learning in healthcare

    M Jorge Cardoso, Wenqi Li, Richard Brown, Nic Ma, Eric Kerfoot, Yiheng Wang, Benjamin Murrey, Andriy Myro- nenko, Can Zhao, Dong Yang, et al. Monai: An open-source framework for deep learning in healthcare. arXiv preprint arXiv:2211.02701, 2022. 5

    arXiv 2022

  7. [3]

    Adversarial image synthesis for unpaired multi-modal cardiac data

    Agisilaos Chartsias, Thomas Joyce, Rohan Dharmakumar, and Sotirios A Tsaftaris. Adversarial image synthesis for unpaired multi-modal cardiac data. In Simulation and Syn- thesis in Medical Imaging: Second International Workshop, SASHIMI 2017, Held in Conjunction with MICCAI 2017, Qu´ ebec City, QC, Canada, September 10, 2017, Proceedings 2, pages 3–13. Spr...

    work page 2017

  8. [4]

    Transunet: Transformers make strong encoders for medi- cal image segmentation

    Jieneng Chen, Yongyi Lu, Qihang Yu, Xiangde Luo, Ehsan Adeli, Yan Wang, Le Lu, Alan L Yuille, and Yuyin Zhou. Transunet: Transformers make strong encoders for medi- cal image segmentation. arXiv preprint arXiv:2102.04306,

    arXiv

Show all 53 references
  1. [5]

    Generative text-guided 3d vision- language pretraining for unified medical image segmenta- tion

    Yinda Chen, Che Liu, Wei Huang, Sibo Cheng, Rossella Ar- cucci, and Zhiwei Xiong. Generative text-guided 3d vision- language pretraining for unified medical image segmenta- tion. arXiv preprint arXiv:2306.04811, 2023. 2

    2023 arXiv

  2. [6]

    CrDoCo: Pixel-level Domain Transfer with Cross-Domain Consistency, Jan

    Yun-Chun Chen, Yen-Yu Lin, Ming-Hsuan Yang, and Jia- Bin Huang. CrDoCo: Pixel-level Domain Transfer with Cross-Domain Consistency, Jan. 2020. arXiv:2001.03182 [cs]. 2

    2020 arXiv

  3. [7]

    Cross-lingual lan- guage model pretraining

    Alexis Conneau and Guillaume Lample. Cross-lingual lan- guage model pretraining. Advances in neural information processing systems, 32, 2019. 2

    2019

  4. [8]

    ResViT: Residual vision transformers for multi-modal medical im- age synthesis

    Onat Dalmaz, Mahmut Yurt, and Tolga C ¸ ukur. ResViT: Residual vision transformers for multi-modal medical im- age synthesis. IEEE Transactions on Medical Imaging , 41(10):2598–2614, Oct. 2022. arXiv:2106.16031 [cs, eess]. 2

    2022 arXiv

  5. [9]

    Hyperdense- net: a hyper-densely connected cnn for multi-modal im- age segmentation

    Jose Dolz, Karthik Gopinath, Jing Yuan, Herve Lombaert, Christian Desrosiers, and Ismail Ben Ayed. Hyperdense- net: a hyper-densely connected cnn for multi-modal im- age segmentation. IEEE Transactions on medical imaging , 38(5):1116–1126, 2018. 1

    2018

  6. [10]

    Unpaired multi-modal segmentation via knowledge distilla- tion

    Qi Dou, Quande Liu, Pheng Ann Heng, and Ben Glocker. Unpaired multi-modal segmentation via knowledge distilla- tion. IEEE transactions on medical imaging , 39(7):2415– 2425, 2020. 1, 2

    2020

  7. [11]

    Multi- modal multi-stream unet model for liver segmentation

    Hagar Louye Elghazy and Mohamed Waleed Fakhr. Multi- modal multi-stream unet model for liver segmentation. In 2021 IEEE World AI IoT Congress (AIIoT) , pages 0028–

    2021

  8. [13]

    Unetr: Transformers for 3d med- ical image segmentation

    Ali Hatamizadeh, Yucheng Tang, Vishwesh Nath, Dong Yang, Andriy Myronenko, Bennett Landman, Holger R Roth, and Daguang Xu. Unetr: Transformers for 3d med- ical image segmentation. In Proceedings of the IEEE /CVF winter conference on applications of computer vision, pages 574–5...

    2022

  9. [14]

    Amos: A large-scale abdominal multi-organ benchmark for versatile medical image segmen- tation

    Yuanfeng Ji, Haotian Bai, Chongjian Ge, Jie Yang, Ye Zhu, Ruimao Zhang, Zhen Li, Lingyan Zhanng, Wanling Ma, Xiang Wan, et al. Amos: A large-scale abdominal multi-organ benchmark for versatile medical image segmen- tation. Advances in Neural Information Processing Systems, 35:...

    2022

  10. [15]

    Unpaired cross-modality educed distillation (cmedl) for medical image segmentation.IEEE Transactions on medical imaging, 41(5):1057–1068, 2021

    Jue Jiang, Andreas Rimner, Joseph O Deasy, and Harini Veeraraghavan. Unpaired cross-modality educed distillation (cmedl) for medical image segmentation.IEEE Transactions on medical imaging, 41(5):1057–1068, 2021. 2

    2021

  11. [16]

    Biobert: a pre-trained biomedical language representation model for biomedical text mining

    Jinhyuk Lee, Wonjin Yoon, Sungdong Kim, Donghyeon Kim, Sunkyu Kim, Chan Ho So, and Jaewoo Kang. Biobert: a pre-trained biomedical language representation model for biomedical text mining. Bioinformatics, 36(4):1234–1240,

  12. [17]

    Autoprosam: Au- tomated prompting sam for 3d multi-organ segmentation

    Chengyin Li, Prashant Khanduri, Yao Qiang, Rafi Ibn Sul- tan, Indrin Chetty, and Dongxiao Zhu. Autoprosam: Au- tomated prompting sam for 3d multi-organ segmentation. arXiv preprint arXiv:2308.14936, 2023. 1

    2023 arXiv

  13. [18]

    Focalunetr: A focal transformer for boundary-aware prostate segmentation using ct images

    Chengyin Li, Yao Qiang, Rafi Ibn Sultan, Hassan Bagher- Ebadian, Prashant Khanduri, Indrin J Chetty, and Dongxiao Zhu. Focalunetr: A focal transformer for boundary-aware prostate segmentation using ct images. In International Conference on Medical Image Computing and Computer-...

    2023

  14. [20]

    M- flag: Medical vision-language pre-training with frozen lan- guage models and latent space geometry optimization

    Che Liu, Sibo Cheng, Chen Chen, Mengyun Qiao, Weitong Zhang, Anand Shah, Wenjia Bai, and Rossella Arcucci. M- flag: Medical vision-language pre-training with frozen lan- guage models and latent space geometry optimization. In International Conference on Medical Image Computing...

  15. [21]

    A modality-collaborative convolution and transformer hybrid network for unpaired multi-modal medical image segmen- tation with limited annotations

    Hong Liu, Yuzhou Zhuang, Enmin Song, Xiangyang Xu, Guangzhi Ma, Coskun Cetinkaya, and Chih-Cheng Hung. A modality-collaborative convolution and transformer hybrid network for unpaired multi-modal medical image segmen- tation with limited annotations. Medical Physics, 2023. 2

    2023

  16. [23]

    Decoupled weight decay regularization

    I Loshchilov. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017. 5 9

    2017 arXiv

  17. [24]

    Deep neural networks for medical image segmentation

    Priyanka Malhotra, Sheifali Gupta, Deepika Koundal, Atef Zaguia, Wegayehu Enbeyle, et al. Deep neural networks for medical image segmentation. Journal of Healthcare Engi- neering, 2022, 2022. 1

    2022

  18. [25]

    Mirror u-net: Marry- ing multimodal fission with multi-task learning for seman- tic segmentation in medical imaging

    Zdravko Marinov, Simon Reiß, David Kersting, Jens Kleesiek, and Rainer Stiefelhagen. Mirror u-net: Marry- ing multimodal fission with multi-task learning for seman- tic segmentation in medical imaging. In Proceedings of the IEEE/CVF International Conference on Computer Vision ...

    2023

  19. [26]

    Direct comparison of mri and x-ray ct technologies for 3d imaging of root systems in soil: potential and challenges for root trait quantification

    Ralf Metzner, Anja Eggert, Dagmar van Dusschoten, Daniel Pflugfelder, Stefan Gerth, Ulrich Schurr, Norman Uhlmann, and Siegfried Jahnke. Direct comparison of mri and x-ray ct technologies for 3d imaging of root systems in soil: potential and challenges for root trait quantific...

    2015

  20. [27]

    V-net: Fully convolutional neural networks for volumetric medical image segmentation

    Fausto Milletari, Nassir Navab, and Seyed-Ahmad Ahmadi. V-net: Fully convolutional neural networks for volumetric medical image segmentation. In 2016 fourth international conference on 3D vision (3DV), pages 565–571. Ieee, 2016. 2

    2016

  21. [28]

    Few-shot 3d multi-modal medical image segmentation using generative adversarial learning

    Arnab Kumar Mondal, Jose Dolz, and Christian Desrosiers. Few-shot 3d multi-modal medical image segmentation using generative adversarial learning. arXiv preprint arXiv:1810.12241, 2018. 2

    2018 arXiv

  22. [29]

    U- net: Convolutional networks for biomedical image segmen- tation

    Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U- net: Convolutional networks for biomedical image segmen- tation. In Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Pa...

    2015

  23. [30]

    Batch Normalization Embeddings for Deep Domain Generaliza- tion, May 2021

    Mattia Segu, Alessio Tonioni, and Federico Tombari. Batch Normalization Embeddings for Deep Domain Generaliza- tion, May 2021. arXiv:2011.12672 [cs]. 2

    2021 arXiv

  24. [31]

    Self-Supervised Pre-Training of Swin Trans- formers for 3D Medical Image Analysis, Mar

    Yucheng Tang, Dong Yang, Wenqi Li, Holger Roth, Ben- nett Landman, Daguang Xu, Vishwesh Nath, and Ali Hatamizadeh. Self-Supervised Pre-Training of Swin Trans- formers for 3D Medical Image Analysis, Mar. 2022. arXiv:2111.14791 [cs]. 5

    2022 arXiv

  25. [32]

    Conditional con- volutions for instance segmentation

    Zhi Tian, Chunhua Shen, and Hao Chen. Conditional con- volutions for instance segmentation. In Computer Vision– ECCV 2020: 16th European Conference, Glasgow, UK, Au- gust 23–28, 2020, Proceedings, Part I 16 , pages 282–298. Springer, 2020. 3

    2020

  26. [33]

    Tganet: Text-guided attention for improved polyp seg- mentation

    Nikhil Kumar Tomar, Debesh Jha, Ulas Bagci, and Sharib Ali. Tganet: Text-guided attention for improved polyp seg- mentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 151–

  27. [35]

    Valindria, Nick Pawlowski, Martin Rajchl, Ioannis Lavdas, Eric O

    Vanya V . Valindria, Nick Pawlowski, Martin Rajchl, Ioannis Lavdas, Eric O. Aboagye, Andrea G. Rockall, Daniel Rueck- ert, and Ben Glocker. Multi-modal Learning from Unpaired Images: Application to Multi-organ Segmentation in CT and MRI. In 2018 IEEE Winter Conference on Appli...

    2018

  28. [36]

    Medclip: Contrastive learning from unpaired medical images and text

    Zifeng Wang, Zhenbang Wu, Dinesh Agarwal, and Jimeng Sun. Medclip: Contrastive learning from unpaired medical images and text. arXiv preprint arXiv:2210.10163, 2022. 5

    2022 arXiv

  29. [37]

    Toward unpaired multi-modal medical image seg- mentation via learning structured semantic consistency

    Jie Yang, Ye Zhu, Chaoqun Wang, Zhen Li, and Ruimao Zhang. Toward unpaired multi-modal medical image seg- mentation via learning structured semantic consistency. In Medical Imaging with Deep Learning, 2023. 2

    2023

  30. [38]

    Dillman, Nehal A

    Huixian Zhang, Hailong Li, Jonathan R. Dillman, Nehal A. Parikh, and Lili He. Multi-Contrast MRI Image Synthesis Using Switchable Cycle-Consistent Generative Adversarial Networks. Diagnostics, 12(4):816, Apr. 2022. Number: 4 Publisher: Multidisciplinary Digital Publishing Institute. 2

    2022

  31. [39]

    Dodnet: Learning to segment multi-organ and tumors from multiple partially labeled datasets

    Jianpeng Zhang, Yutong Xie, Yong Xia, and Chunhua Shen. Dodnet: Learning to segment multi-organ and tumors from multiple partially labeled datasets. In Proceedings of the IEEE/CVF conference on computer vision and pattern recog- nition, pages 1195–1204, 2021. 5

    2021

  32. [40]

    Modality-aware mutual learning for multi-modal medical image segmenta- tion

    Yao Zhang, Jiawei Yang, Jiang Tian, Zhongchao Shi, Cheng Zhong, Yang Zhang, and Zhiqiang He. Modality-aware mutual learning for multi-modal medical image segmenta- tion. In Medical Image Computing and Computer Assisted Intervention–MICCAI 2021: 24th International Conference, S...

    2021

  33. [41]

    Cross-Task Feedback Fusion GAN for Joint MR-CT Synthesis and Segmentation of Target and Organs-At-Risk

    Yiwen Zhang, Liming Zhong, Hai Shu, Zhenhui Dai, Kaiyi Zheng, Zefeiyun Chen, Qianjin Feng, Xuetao Wang, and Wei Yang. Cross-Task Feedback Fusion GAN for Joint MR-CT Synthesis and Segmentation of Target and Organs-At-Risk. IEEE Transactions on Artificial Intelligence , pages 1–12,

  34. [42]

    Translating and segmenting multimodal medical volumes with cycle-and shape-consistency generative adversarial network

    Zizhao Zhang, Lin Yang, and Yefeng Zheng. Translating and segmenting multimodal medical volumes with cycle-and shape-consistency generative adversarial network. In Pro- ceedings of the IEEE conference on computer vision and pat- tern Recognition, pages 9242–9251, 2018. 7

    2018

  35. [43]

    Translating and Segmenting Multimodal Medical V olumes with Cycle- and Shape-Consistency Generative Adversarial Network, Mar

    Zizhao Zhang, Lin Yang, and Yefeng Zheng. Translating and Segmenting Multimodal Medical V olumes with Cycle- and Shape-Consistency Generative Adversarial Network, Mar

  36. [44]

    Cross-modality medical image detection and segmentation by transfer learning of shapel priors

    Yefeng Zheng. Cross-modality medical image detection and segmentation by transfer learning of shapel priors. In 2015 IEEE 12th International Symposium on Biomedical Imaging (ISBI), pages 424–427, Apr. 2015. ISSN: 1945-8452. 2

    2015

  37. [45]

    Ariadne’s thread: Using text prompts to improve segmentation of infected areas from chest x-ray images

    Yi Zhong, Mengqiu Xu, Kongming Liang, Kaixin Chen, and Ming Wu. Ariadne’s thread: Using text prompts to improve segmentation of infected areas from chest x-ray images. In International Conference on Medical Image Computing and Computer-Assisted Intervention , pages 724–733. Springer,

  38. [46]

    A review of deep learning in medical imaging: Imaging traits, 10 technology trends, case studies with progress highlights, and future promises

    S Kevin Zhou, Hayit Greenspan, Christos Davatzikos, James S Duncan, Bram Van Ginneken, Anant Madabhushi, Jerry L Prince, Daniel Rueckert, and Ronald M Summers. A review of deep learning in medical imaging: Imaging traits, 10 technology trends, case studies with progress highli...

  39. [47]

    Latent correlation representation learning for brain tumor segmentation with missing mri modalities

    Tongxue Zhou, St ´ephane Canu, Pierre Vera, and Su Ruan. Latent correlation representation learning for brain tumor segmentation with missing mri modalities. IEEE Transac- tions on Image Processing, 30:4263–4274, 2021. 1

    2021

  40. [48]

    A review: Deep learning for medical image segmentation using multi- modality fusion

    Tongxue Zhou, Su Ruan, and St ´ephane Canu. A review: Deep learning for medical image segmentation using multi- modality fusion. Array, 3:100004, 2019. 1

    2019

  41. [49]

    Generalizable cross-modality medical image segmentation via style augmentation and dual normalization

    Ziqi Zhou, Lei Qi, Xin Yang, Dong Ni, and Yinghuan Shi. Generalizable cross-modality medical image segmentation via style augmentation and dual normalization. In Proceed- ings of the IEEE /CVF conference on computer vision and pattern recognition, pages 20856–20865, 2022. 1

    2022

  42. [50]

    Generalizable Cross-modality Medical Image Segmen- tation via Style Augmentation and Dual Normalization

    Ziqi Zhou, Lei Qi, Xin Yang, Dong Ni, and Yinghuan Shi. Generalizable Cross-modality Medical Image Segmen- tation via Style Augmentation and Dual Normalization. In 2022 IEEE/CVF Conference on Computer Vision and Pat- tern Recognition (CVPR) , pages 20824–20833, New Or- leans, ...

    2022

  43. [51]

    Evaluation of algorithms for multi-modality whole heart segmentation: an open-access grand challenge

    Xiahai Zhuang, Lei Li, Christian Payer, Darko ˇStern, Mar- tin Urschler, Mattias P Heinrich, Julien Oster, Chunliang Wang, ¨Orjan Smedby, Cheng Bian, et al. Evaluation of algorithms for multi-modality whole heart segmentation: an open-access grand challenge. Medical image anal...

    2019

  44. [2019]

    arXiv:1802.09655 [cs]. 2

    arXiv

  45. [2022]

    Conference Name: IEEE Transactions on Artificial Intelligence. 2

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.