Pith. sign in

REVIEW 2 major objections 6 minor 50 references

LoG turns clinical text location cues into multi-scale localization maps that guide medical image segmentation, beating prior text-guided methods on three datasets.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 03:28 UTC pith:RNEGG27M

load-bearing objection A well-designed localization-infused fusion method whose internal ablations are consistent, but the 'consistently outperforms' claim outruns the mixed-protocol comparison in Table I. the 2 major comments →

arxiv 2607.16327 v1 pith:RNEGG27M submitted 2026-07-15 cs.CV

Localization-Infused Vision-Language Semantic Fusion for Text-Guided Medical Image Segmentation

Songyue Han , Mingye Zou , Shuchang Ye , Lei Bi , Mingyuan Meng This is my paper
classification cs.CV
keywords medical image segmentationtext-guided segmentationvision-language fusionlocalizationmulti-scale predictionclinical reportsDice scoredeep learning
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that text-guided medical image segmentation improves when the model is forced to explicitly predict where the target is, at multiple scales, rather than just blending a holistic text embedding into visual features. It proposes LoG, which jointly predicts multi-scale localization maps from image and report, then injects them into segmentation at three levels: feature fusion, attention gating, and loss supervision. On chest X-ray, CT, and colonoscopy datasets with paired clinical reports, LoG reports Dice scores of 91.59%, 80.71%, and 94.59%, consistently ahead of existing methods. The ablation evidence pins the gain to the localization branch: removing it costs 3.44% average Dice. The authors note the method assumes paired structured reports and was tested only on 2D images.

Core claim

LoG's central discovery is that target localization should be an explicit intermediate representation for vision-language fusion in medical segmentation. By performing patch-wise localization prediction at four image scales with bidirectional cross-attention between visual and textual features, the model extracts what the authors call localization-oriented semantics, embedded both as implicit feature-space tokens and explicit localization maps. These semantics are then used in three complementary ways: Localization-Guided Feature Fusion cross-attends visual tokens to localized semantic tokens; Localization-Gated Attention Fusion multiplies visual features by a spatial gating mask derived fro

What carries the argument

The central object is a multi-scale localization-oriented semantic extraction module that converts an image-report pair into feature-space semantic tokens and explicit patch-wise localization maps at four scales. The maps are produced by a lightweight head on top of token-level cross-attention between a visual encoder and a medical text encoder. The maps are then recycled into the segmentation pipeline in three fusion mechanisms: LGFF (cross-attention-based feature infusion), LGAF (spatial gating with a sigmoid mask built from all four localization maps), and LCLF (a localization-consistency loss term). This design turns location language into spatial supervision.

Load-bearing premise

The head-to-head comparison numbers come from a mix of officially reported results and re-implementations, so the claimed 'consistently outperforms' rests on the assumption that differences in training protocol do not account for the margin.

What would settle it

Run every baseline with the same training code, image resolution, augmentation, and epoch count as LoG; if LoG still leads on all three datasets, the architectural contribution is confirmed.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Text-guided segmentation systems should treat explicit localization prediction as a core component: removing the localization branch causes the largest single performance drop (3.44% average DSC).
  • Multi-level fusion is additive: each of the three levels contributes a small but consistent gain (about 2.17%, 0.95%, and 0.69% average DSC, respectively).
  • The localization branch remains useful even without text input: the no-text variant still beats many text-guided baselines, suggesting spatial self-guidance is part of the gain.
  • Truncating reports to 4 tokens drops average DSC by 2.35%, while truncating to 12 tokens slightly improves it, implying most location cues sit in a small central span of the report.
  • Localization patch size is not a critical hyperparameter; performance is stable across the tested granularities.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Because the localization branch helps even with no text at all, the same localization-infused design could be applied to image-only segmentation pipelines as a form of self-supervision.
  • The localization-consistency loss should transfer to 3D volumetric segmentation, where anatomical location language is more informative, if patch-wise labels are replaced with voxel-grid labels.
  • The authors' stated limitation to structured reports suggests a natural stress test: feeding the model free-text clinical notes with paraphrased or contradictory location phrases and measuring whether localization maps and Dice scores degrade.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 6 minor

Summary. The paper proposes LoG, a text-guided medical image segmentation framework that augments a ConvNeXt-Tiny/BiomedBERT encoder pair with multi-scale patch-level localization prediction. Localization predictions are used in three fusion mechanisms: localization-guided feature fusion (LGFF) via cross-attention, localization-gated attention fusion (LGAF) via spatial gating, and localization-constrained loss fusion (LCLF) via a consistency loss. On QaTa-COV19, MosMedData+, and Kvasir-SEG the authors report DSC of 91.59%, 80.71%, and 94.59%, respectively, outperforming compared image-only, pretraining-based, and text-guided methods. Ablations show monotonic gains from adding each component and identify localization prediction as the largest contributor.

Significance. If the empirical claims hold, LoG is a useful architecture-level contribution: it converts holistic textual embeddings into explicit multi-scale localization cues and integrates them at feature, attention, and loss levels. The ablation logic is clean, and the authors correctly supervise localization against patch labels derived from ground-truth masks (Section III-E), preventing the consistency loss from trivial collapse. The main weakness is that the headline comparative claim rests on a mixed-protocol comparison without statistical validation; the architecture could still be sound even if the benchmark comparison is contaminated. Code and weights are not yet available, so independent verification is currently limited.

major comments (2)
  1. [Section IV-C, Table I] The headline claim is not established by the reported evidence. Table I mixes official results from other papers with re-implementations, so the compared numbers are not produced under a common protocol (Section IV-B: 224x224, batch 32, 100 epochs). Official numbers can differ in image size, schedule, split, and post-processing. Margins are small on two datasets: 0.16 DSC over PCSA-Seg on QaTa-COV19 and 1.41 DSC over FMISeg on MosMedData+. Moreover, no-text LoG (Table VII, 85.99 avg) already exceeds several text-guided baselines, so part of the apparent advantage may reflect backbone capacity rather than the proposed cross-modal mechanism. Please re-evaluate all comparison methods under one protocol, identify which entries are official vs re-implemented, and report multi-seed mean±std with significance tests, or soften the claim.
  2. [Tables II-V] Component ablations are internally consistent but are single Dice values without variance. Claimed gains are very small: 0.29% for updating F_T (Table III), 0.41% for multi-scale vs single-scale LGAF (Table IV), and 0.14-0.25% for removing L_con at individual scales (Table V). These differences are within typical run-to-run noise. The claim that each fusion level contributes monotonically needs support from repeated runs. Please provide mean±std over at least three seeds (or paired significance tests) for the main ablations, or state that differences are not statistically significant.
minor comments (6)
  1. [Section IV-B, Eq. (5)] The loss weights lambda_loc (0.5/0.4/0.2 per dataset) and lambda_con are chosen empirically on the validation set, but no sensitivity analysis is reported for these choices. A small sensitivity table would help assess robustness.
  2. [Section III-E, Eq. (4)] The consistency loss denominator can be zero if the predicted segmentation is empty. Add a small epsilon (or handle the empty-prediction case) to avoid numerical instability.
  3. [Fig. 5] The DSC labels below the qualitative panels appear misaligned (six numeric values are shown for what appears to be seven method columns). Please clarify which value corresponds to each method, and fix the 'Qata-COV19' typo.
  4. [Section II-A / Experimental setup] Please specify the exact fusion operation used in the Table II baseline and whether the same visual encoder/decoder and training budget are used. This will make the baseline comparison more transparent.
  5. [Section IV-B] The paper does not report parameter counts, FLOPs, or inference time for LoG and the compared methods. Given the added cross-attention and localization heads, these numbers would help readers assess the cost of the improvement.
  6. [Code Availability] The statement that code will be released only upon acceptance limits immediate reproducibility. Consider providing a public repository with the main scripts at submission.

Circularity Check

0 steps flagged

No significant circularity; the localization supervision is independently grounded in ground-truth masks, and the sole self-citation in a design role ([43]) is non-load-bearing.

full rationale

The central claim is an architecture-level improvement validated by ablations (Tables II-V), not a quantity defined by its inputs. Multi-scale localization maps are supervised independently from the ground-truth mask: "the corresponding downsampled ground-truth segmentation mask Y^s is divided into patch regions of size P_s, and a patch is labeled as positive if the foreground ratio inside the patch exceeds a predefined threshold τ" (Sec. III-E), and the consistency loss is explicitly prevented from trivial collapse: "the localization prediction L̂s is independently supervised by L_loc against the patch-wise ground-truth localization labels, which penalizes degenerate all-positive localization maps" (Sec. III-E). The final segmentation is optimized by Dice/focal/CE losses directly against the mask (Eq. 5), so the reported DSC is not constructed from the localization branch. The one self-citation in a design role is [43] — "we follow [43] and introduce two hybrid CNN-MLP stages" (Sec. III-A) — but those high-resolution features "are directly propagated via skip connections to the segmentation decoder, without participating in cross-modal vision-language semantic modeling and integration", so they are not load-bearing for the localization-infused fusion claim; [28] and [29] appear only in related-work discussion. The Table I comparison mixes official numbers and re-implementations (Sec. IV-C), which is a reproducibility/protocol concern, not circularity. Score reflects one minor, non-load-bearing self-citation.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 0 invented entities

The method introduces no new physical or mathematical entities. It relies on several empirical design choices (patch sizes, threshold, loss weights), a domain assumption that the text is informative, and the assumption that auxiliary localization supervision from the segmentation mask helps segmentation.

free parameters (4)
  • Localization patch sizes P1-P4 = 6x6, 5x5, 4x4, 3x3 (default)
    Sensitivity analysis in Table VI; the second-row setting is selected as the default for a balance of performance and compute, i.e., a hand-chosen configuration.
  • Patch positive threshold tau = 5%
    Section IV-B: set to 5% for generating patch-wise localization labels; no sensitivity analysis is reported for this threshold.
  • Loss weight lambda_loc = 0.5 (QaTa-COV19), 0.4 (MosMedData+), 0.2 (Kvasir-SEG)
    Section IV-B: empirically determined on the validation sets per dataset.
  • Loss weights lambda_seg, lambda_con = 0.1 each
    Section IV-B: fixed manually, not tuned.
axioms (4)
  • domain assumption Paired textual reports in QaTa-COV19, MosMedData+, and Kvasir-SEG are semantically aligned with image content and contain usable location cues.
    Section IV-A describes the reports as clinician-formulated descriptions; if the text is noisy or decoupled from the image, the text-guidance component of LoG would be weaker than claimed.
  • domain assumption Patch-wise localization labels generated from downsampled ground-truth masks with threshold tau provide a beneficial intermediate objective for segmentation.
    Section III-E derives Ls from the GT mask; Table III shows removing localization prediction costs 3.44% average DSC, so the framework depends on this auxiliary task transferring to segmentation.
  • domain assumption Pretrained ConvNeXt-Tiny and BiomedBERT encoders provide feature representations suitable for the cross-attention and gating modules.
    The architecture assumes these off-the-shelf encoders are sufficient; no experiments vary the encoders.
  • standard math Standard deep-learning optimization assumptions (Adam, piecewise-constant LR, 100 epochs) reach a stable optimum.
    Empirical training assumption, not proven; standard for this literature.

pith-pipeline@v1.3.0-alltime-deepseek · 19362 in / 10594 out tokens · 102115 ms · 2026-08-02T03:28:19.024234+00:00 · methodology

0 comments
read the original abstract

Medical image segmentation is essential for modern computer-aided medicine. Recently, text-guided segmentation has shown promise by incorporating clinician-formulated textual reports as semantic guidance for image segmentation. These reports describe target appearance, location, and neighboring anatomy, providing explicit guidance for localization and delineation. Existing text-guided segmentation methods typically extract textual semantics implicitly through a pretrained text encoder and then integrate vision-language semantics via straightforward image-text feature fusion. However, these methods do not explicitly capture target-oriented information embedded in textual reports, particularly target location, and do not explore multi-level information fusion strategies beyond basic feature-level fusion, limiting the extraction and integration of critical textual semantics. In this study, we propose LoG, a localization-infused vision-language fusion framework for text-guided medical image segmentation. By jointly performing multi-scale target localization tasks, LoG explicitly captures target-oriented vision-language semantics and enables three-level localization-infused semantic fusion: (i) localization-guided feature fusion that directly infuses location-relevant semantics into visual features, (ii) localization-gated attention fusion that redirects multi-scale localization predictions to reinforce critical regions, and (iii) localization-constrained loss fusion that supervises segmentation based on spatial consistency with target localization. Extensive experiments on three benchmark datasets, involving three medical imaging modalities with paired textual reports, demonstrate that LoG achieves Dice scores of 91.59%, 80.71%, and 94.59% on QaTa-COV19, MosMedData+, and Kvasir-SEG, respectively, consistently outperforming state-of-the-art medical image segmentation methods.

Figures

Figures reproduced from arXiv: 2607.16327 by Lei Bi, Mingye Zou, Mingyuan Meng, Shuchang Ye, Songyue Han.

Figure 1
Figure 1. Figure 1: Illustration of existing text-guided medical image segmentation methods and the proposed LoG. (a) Existing methods typically encode the textual [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overall workflow of the proposed LoG framework. LoG consists of a visual encoder, a textual encoder, and a localization-oriented vision–language [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Internal structure of localization-oriented semantic extraction and Localization-Guided Feature Fusion (LGFF) at scale [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Internal structure of Localization-Gated Attention Fusion (LGAF) at scale [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Qualitative comparison with representative medical image segmentation methods on Kvasir-SEG (rows 1–2), MosMedData+ (rows 3–4), and QaTa [PITH_FULL_IMAGE:figures/full_fig_p011_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Visualization of the input image, ground truth (GT), localization map, LGAF gating mask, and segmentation prediction from left to right. Green and [PITH_FULL_IMAGE:figures/full_fig_p012_6.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

50 extracted references · 7 linked inside Pith

  1. [1]

    Deep learning tech- niques for medical image segmentation: Achievements and challenges,

    M. H. Hesamian, W. Jia, X. He, and P. Kennedy, “Deep learning tech- niques for medical image segmentation: Achievements and challenges,” Journal of Digital Imaging, vol. 32, no. 4, pp. 582–596, aug 2019

    2019

  2. [2]

    Deep learning for medical image-based cancer diagnosis,

    X. Jiang, Z. Hu, S. Wang, and Y . Zhang, “Deep learning for medical image-based cancer diagnosis,”Cancers, vol. 15, no. 14, p. 3608, 2023

    2023

  3. [3]

    U-Net: Convolutional net- works for biomedical image segmentation,

    O. Ronneberger, P. Fischer, and T. Brox, “U-Net: Convolutional net- works for biomedical image segmentation,” inMedical Image Comput- ing and Computer-Assisted Intervention – MICCAI 2015, ser. Lecture Notes in Computer Science, N. Navab, J. Hornegger, W. M. Wells, and A. F. Frangi, Eds., vol. 9351. Cham, Switzerland: Springer, 2015, pp. 234–241

    2015

  4. [4]

    UNet++: A nested U-Net architecture for medical image segmentation,

    Z. Zhou, M. M. R. Siddiquee, N. Tajbakhsh, and J. Liang, “UNet++: A nested U-Net architecture for medical image segmentation,” inDeep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support, ser. Lecture Notes in Computer Science, D. Stoyanov, Z. Taylor, G. Carneiro, and T. Syeda-Mahmood, Eds., vol. 11045. Cham, Switzerlan...

    2018

  5. [5]

    Attention U-Net: Learning where to look for the pancreas,

    O. Oktay, J. Schlemper, L. L. Folgoc, M. Lee, M. Heinrich, K. Misawa, K. Mori, S. McDonagh, N. Y . Hammerla, B. Kainz, B. Glocker, and D. Rueckert, “Attention U-Net: Learning where to look for the pancreas,”arXiv preprint arXiv:1804.03999, 2018. [Online]. Available: https://arxiv.org/abs/1804.03999

    Pith/arXiv arXiv 2018

  6. [6]

    Swin-Unet: Unet-like pure transformer for medical image segmenta- tion,

    H. Cao, Y . Wang, J. Chen, D. Jiang, X. Zhang, Q. Tian, and M. Wang, “Swin-Unet: Unet-like pure transformer for medical image segmenta- tion,” inComputer Vision – ECCV 2022 Workshops, ser. Lecture Notes in Computer Science, vol. 13803. Cham, Switzerland: Springer, 2023, pp. 205–218

    2022

  7. [7]

    nnU-Net: A self-configuring method for deep learning-based biomedical image segmentation,

    F. Isensee, P. F. Jaeger, S. A. A. Kohl, J. Petersen, and K. H. Maier- Hein, “nnU-Net: A self-configuring method for deep learning-based biomedical image segmentation,”Nature Methods, vol. 18, no. 2, pp. 203–211, 2021

    2021

  8. [8]

    Vision-language models for vision tasks: A survey,

    J. Zhang, J. Huang, S. Jin, and S. Lu, “Vision-language models for vision tasks: A survey,”IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 46, no. 8, pp. 5625–5644, 2024

    2024

  9. [9]

    Learning transferable visual models from natural language supervision,

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, G. Krueger, and I. Sutskever, “Learning transferable visual models from natural language supervision,” inProceedings of the 38th International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, M. Meila and T. Zhang, Ed...

    2021

  10. [10]

    GLoRIA: A multimodal global-local representation learning framework for label- efficient medical image recognition,

    S.-C. Huang, L. Shen, M. P. Lungren, and S. Yeung, “GLoRIA: A multimodal global-local representation learning framework for label- efficient medical image recognition,” inProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2021, pp. 3942– 3951

    2021

  11. [12]

    LViT: Language meets vision transformer in medical image segmentation,

    Z. Li, Y . Li, Q. Li, P. Wang, D. Guo, L. Lu, D. Jin, Y . Zhang, and Q. Hong, “LViT: Language meets vision transformer in medical image segmentation,”IEEE Transactions on Medical Imaging, vol. 43, no. 1, pp. 96–107, 2024

    2024

  12. [13]

    Ariadne’s thread: Using text prompts to improve segmentation of infected areas from chest x-ray images,

    Y . Zhong, M. Xu, K. Liang, K. Chen, and M. Wu, “Ariadne’s thread: Using text prompts to improve segmentation of infected areas from chest x-ray images,” inMedical Image Computing and Computer Assisted Intervention – MICCAI 2023, ser. Lecture Notes in Computer Science, vol. 14223. Cham, Switzerland: Springer, 2023, pp. 724–733

    2023

  13. [14]

    Text-guided cross-position attention for segmentation: Case of medical image,

    G.-E. Lee, S. H. Kim, J. Cho, S. T. Choi, and S.-I. Choi, “Text-guided cross-position attention for segmentation: Case of medical image,” inMedical Image Computing and Computer Assisted Intervention – MICCAI 2023, ser. Lecture Notes in Computer Science, vol. 14224. Cham, Switzerland: Springer, 2023, pp. 537–546

    2023

  14. [15]

    Harnessing text insights with visual alignment for medical image segmentation,

    Q. Zeng, H. Luo, Z. Lu, Y . Xie, Z. Wang, Y . Zhang, and Y . Xia, “Harnessing text insights with visual alignment for medical image segmentation,”IEEE Transactions on Medical Imaging, vol. 45, no. 2, pp. 477–489, 2026

    2026

  15. [16]

    ViTexNet: Vision-text guided dynamic convolution network for medical image segmentation,

    R. Bhardwaj, U. Y . Tambe, and D. R. Neog, “ViTexNet: Vision-text guided dynamic convolution network for medical image segmentation,” inMedical Image Computing and Computer Assisted Intervention – MICCAI 2025, ser. Lecture Notes in Computer Science, vol. 15966. Cham, Switzerland: Springer, 2025, pp. 690–699

    2025

  16. [17]

    Progressive cross-scale semantic alignment for language-guided medical image segmentation,

    H. Xue, Y . Dai, Q. Li, Y . Yao, and Y . Teng, “Progressive cross-scale semantic alignment for language-guided medical image segmentation,” Knowledge-Based Systems, vol. 340, p. 115745, 2026

    2026

  17. [18]

    Making the most of text semantics to improve biomedical vision–language processing,

    B. Boecking, N. Usuyama, S. Bannur, D. C. Castro, A. Schwaighofer, S. Hyland, M. T. Wetscherek, T. Naumann, A. Nori, J. Alvarez-Valle, H. Poon, and O. Oktay, “Making the most of text semantics to improve biomedical vision–language processing,” inComputer Vision – ECCV 2022, ser. Lecture Notes in Computer Science, vol. 13696. Cham, Switzerland: Springer, 2...

    2022

  18. [19]

    Multimodal spatial attention module for targeting multimodal PET–CT lung tumor segmen- tation,

    X. Fu, L. Bi, A. Kumar, M. Fulham, and J. Kim, “Multimodal spatial attention module for targeting multimodal PET–CT lung tumor segmen- tation,”IEEE Journal of Biomedical and Health Informatics, vol. 25, pp. 3507–3516, 2021

    2021

  19. [20]

    Task-driven image fusion with learnable fusion loss,

    H. Bai, J. Zhang, Z. Zhao, Y . Wu, L. Deng, Y . Cui, T. Feng, and S. Xu, “Task-driven image fusion with learnable fusion loss,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025, pp. 7457–7468

    2025

  20. [21]

    OSegNet: Operational segmentation network for COVID-19 detection using chest X-ray images,

    A. Degerli, S. Kiranyaz, M. E. H. Chowdhury, and M. Gabbouj, “OSegNet: Operational segmentation network for COVID-19 detection using chest X-ray images,” in2022 IEEE International Conference on Image Processing (ICIP), 2022, pp. 2306–2310

    2022

  21. [23]

    Kvasir-SEG: A segmented polyp dataset,

    D. Jha, P. H. Smedsrud, M. A. Riegler, P. Halvorsen, T. de Lange, D. Johansen, and H. D. Johansen, “Kvasir-SEG: A segmented polyp dataset,” inMultiMedia Modeling, ser. Lecture Notes in Computer Science, vol. 11962. Cham, Switzerland: Springer, 2020, pp. 451–462

    2020

  22. [24]

    MAdapter: A better interaction between image and language for medical image segmentation,

    X. Zhang, B. Ni, Y . Yang, and L. Zhang, “MAdapter: A better interaction between image and language for medical image segmentation,” inMed- ical Image Computing and Computer Assisted Intervention – MICCAI 2024, ser. Lecture Notes in Computer Science, vol. 15009. Cham, Switzerland: Springer, 2024, pp. 425–434

    2024

  23. [25]

    Frequency- domain multi-modal fusion for language-guided medical image seg- mentation,

    B. Yu, J. Yang, Z. Du, Y . Huang, C. Li, and L. Wang, “Frequency- domain multi-modal fusion for language-guided medical image seg- mentation,” inMedical Image Computing and Computer Assisted In- tervention – MICCAI 2025, ser. Lecture Notes in Computer Science, vol. 15968. Cham, Switzerland: Springer, 2025, pp. 278–288

    2025

  24. [26]

    EviVLM: When evidential learning meets vision–language model for medical image segmentation,

    Q. Pan, Z. Li, G. Yang, Q. Yang, and B. Ji, “EviVLM: When evidential learning meets vision–language model for medical image segmentation,” IEEE Transactions on Medical Imaging, vol. 45, pp. 1369–1382, 2026

    2026

  25. [27]

    Cross- modal conditioned reconstruction for language-guided medical image segmentation,

    X. Huang, H. Li, M. Cao, L. Chen, C. You, and D. An, “Cross- modal conditioned reconstruction for language-guided medical image segmentation,”IEEE Transactions on Medical Imaging, vol. 44, pp. 1821–1835, 2025

    2025

  26. [28]

    SGSeg: Enabling text- free inference in language-guided segmentation of chest X-rays via self-guidance,

    S. Ye, M. Meng, M. Li, D. Feng, and J. Kim, “SGSeg: Enabling text- free inference in language-guided segmentation of chest X-rays via self-guidance,” inMedical Image Computing and Computer Assisted Intervention – MICCAI 2024, ser. Lecture Notes in Computer Science, vol. 15008. Cham, Switzerland: Springer, 2024, pp. 242–252

    2024

  27. [29]

    Alleviating textual reliance in medical language-guided segmentation via prototype-driven semantic approximation,

    S. Ye, U. Naseem, M. Meng, and J. Kim, “Alleviating textual reliance in medical language-guided segmentation via prototype-driven semantic approximation,” inProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2025, pp. 22 316–22 326. [Online]. Available: https://arxiv.org/abs/2507.11055

    Pith/arXiv arXiv 2025

  28. [30]

    Cross-aware early fusion with stage- divided vision and language transformer encoders for referring image segmentation,

    Y . Cho, H. Yu, and S.-J. Kang, “Cross-aware early fusion with stage- divided vision and language transformer encoders for referring image segmentation,”IEEE Transactions on Multimedia, vol. 26, pp. 5823– 5833, 2024

    2024

  29. [31]

    CM- MaskSD: Cross-modality masked self-distillation for referring image segmentation,

    W. Wang, X. He, Y . Zhang, L. Guo, J. Shen, J. Li, and J. Liu, “CM- MaskSD: Cross-modality masked self-distillation for referring image segmentation,”IEEE Transactions on Multimedia, vol. 26, pp. 6906– 6916, 2024

    2024

  30. [32]

    Bring adaptive binding prototypes to generalized referring expression segmentation,

    W. Li, Z. Zhao, H. Bai, and F. Su, “Bring adaptive binding prototypes to generalized referring expression segmentation,”IEEE Transactions on Multimedia, vol. 27, pp. 6059–6069, 2025

    2025

  31. [33]

    Deep learning-based clas- sification of healthy aging controls, mild cognitive impairment and alzheimer’s disease using fusion of MRI–PET imaging,

    V . P. S. Rallabandi and K. Seetharaman, “Deep learning-based clas- sification of healthy aging controls, mild cognitive impairment and alzheimer’s disease using fusion of MRI–PET imaging,”Biomedical Signal Processing and Control, vol. 80, p. 104312, 2023

    2023

  32. [34]

    A survey on ensemble learning under the era of deep learning,

    Y . Yang, H. Lv, and N. Chen, “A survey on ensemble learning under the era of deep learning,”Artificial Intelligence Review, vol. 56, pp. 5545–5589, 2023

    2023

  33. [35]

    Advancing multi-modal beam prediction with cross-modal feature enhancement and dynamic 14 fusion mechanism,

    Q. Zhu, Y . Wang, W. Li, H. Huang, and G. Gui, “Advancing multi-modal beam prediction with cross-modal feature enhancement and dynamic 14 fusion mechanism,”IEEE Transactions on Communications, 2025, early Access

    2025

  34. [36]

    AMFuse: Add-multiply-based cross-modal fusion network for multi-spectral semantic segmentation,

    H. Liu, F. Chen, Z. Zeng, and X. Tan, “AMFuse: Add-multiply-based cross-modal fusion network for multi-spectral semantic segmentation,” Remote Sensing, vol. 14, no. 14, p. 3368, 2022

    2022

  35. [37]

    ICAFusion: Iterative cross-attention guided feature fusion for multispectral object detection,

    J. Shen, Y . Chen, Y . Liu, X. Zuo, H. Fan, and W. Yang, “ICAFusion: Iterative cross-attention guided feature fusion for multispectral object detection,”Pattern Recognition, vol. 145, p. 109913, 2024

    2024

  36. [38]

    Pre-gating and contextual attention gate: A new fusion method for multi-modal data tasks,

    D. Zhang, R. Nayak, and M. A. Bashar, “Pre-gating and contextual attention gate: A new fusion method for multi-modal data tasks,”Neural Networks, vol. 179, p. 106553, 2024

    2024

  37. [39]

    MsgFusion: Medical semantic guided two-branch network for multi- modal brain image fusion,

    J. Wen, F. Qin, J. Du, M. Fang, X. Wei, C. L. P. Chen, and P. Li, “MsgFusion: Medical semantic guided two-branch network for multi- modal brain image fusion,”IEEE Transactions on Multimedia, vol. 26, pp. 944–957, 2024

    2024

  38. [40]

    MCFNet: A multimodal collaborative fusion network for fine-grained semantic classification,

    Y . Qiao, X. Zhong, X. Gu, and Z. Yu, “MCFNet: A multimodal collaborative fusion network for fine-grained semantic classification,” arXiv preprint arXiv:2505.23365, 2025. [Online]. Available: https: //arxiv.org/abs/2505.23365

    Pith/arXiv arXiv 2025

  39. [41]

    A simple framework for contrastive learning of visual representations,

    T. Chen, S. Kornblith, M. Norouzi, and G. Hinton, “A simple framework for contrastive learning of visual representations,” inProceedings of the 37th International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, vol. 119. PMLR, 2020, pp. 1597–1607. [Online]. Available: https://proceedings.mlr.press/v119/chen20j.html

    2020

  40. [42]

    A ConvNet for the 2020s,

    Z. Liu, H. Mao, C.-Y . Wu, C. Feichtenhofer, T. Darrell, and S. Xie, “A ConvNet for the 2020s,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022, pp. 11 976– 11 986

    2022

  41. [44]

    MAXIM: Multi-axis MLP for image processing,

    Z. Tu, H. Talebi, H. Zhang, F. Yang, P. Milanfar, A. Bovik, and Y . Li, “MAXIM: Multi-axis MLP for image processing,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022, pp. 5769–5780

    2022

  42. [45]

    Domain-specific language model pretraining for biomedical natural language processing,

    Y . Gu, R. Tinn, H. Cheng, M. Lucas, N. Usuyama, X. Liu, T. Naumann, J. Gao, and H. Poon, “Domain-specific language model pretraining for biomedical natural language processing,”ACM Transactions on Computing for Healthcare, vol. 3, no. 1, pp. 1–23, 2022

    2022

  43. [46]

    Available: https://arxiv.org/abs/2311.16707

    [Online]. Available: https://arxiv.org/abs/2311.16707

    Pith/arXiv arXiv

  44. [47]

    Focal loss for dense object detection,

    T.-Y . Lin, P. Goyal, R. Girshick, K. He, and P. Doll ´ar, “Focal loss for dense object detection,” inProceedings of the IEEE International Conference on Computer Vision (ICCV), 2017, pp. 2980–2988

    2017

  45. [48]

    Exploring transfer learning in medical image segmentation using vision–language models,

    K. Poudel, M. Dhakal, P. Bhandari, R. Adhikari, S. Thapaliya, and B. Khanal, “Exploring transfer learning in medical image segmentation using vision–language models,”arXiv preprint arXiv:2308.07706, 2023. [Online]. Available: https://arxiv.org/abs/2308.07706

    Pith/arXiv arXiv 2023

  46. [49]

    V-Net: Fully convolutional neural networks for volumetric medical image segmentation,

    F. Milletari, N. Navab, and S.-A. Ahmadi, “V-Net: Fully convolutional neural networks for volumetric medical image segmentation,” in2016 Fourth International Conference on 3D Vision (3DV), 2016, pp. 565– 571

    2016

  47. [52]

    Segment anything in medical images,

    J. Ma, Y . He, F. Li, L. Han, C. You, and B. Wang, “Segment anything in medical images,”Nature Communications, vol. 15, p. 654, 2024. Songyue Hanreceived the B.S. degree from Shi- jiazhuang Army Command College, Shijiazhuang, China, in 2015, and the M.S. degree from the Army Engineering University of PLA, Nanjing, China, in

    2024

  48. [2020]

    Available: https://arxiv.org/abs/2005.06465

    [Online]. Available: https://arxiv.org/abs/2005.06465

    Pith/arXiv arXiv 2005

  49. [2022]

    degree and is a jointly trained Ph.D

    He is currently pursuing the Ph.D. degree and is a jointly trained Ph.D. candidate with Air Force Engineering University and Beijing Zhong- guancun Academy. His research interests include large language models, multimodal medical image segmentation, and the interpretability of diffusion models. Mingye Zoureceived the B.E. degree from Harbin Institute of T...

    2022

  50. [2023]

    Available: https://arxiv.org/abs/2303.00915

    [Online]. Available: https://arxiv.org/abs/2303.00915

    Pith/arXiv arXiv