TextSLIP: Text Self-Supervised CLIP for Medical Report Generation

Haoyu Jiang; Ziping Cong

REVIEW 4 major objections 5 minor 19 references

TextSLIP claims that adding a self-supervised text-contrastive loss to CLIP gives the visual encoder finer-grained linguistic supervision, improving medical report generation beyond the CLIP baseline alone.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 06:09 UTC pith:UWXKTU6F

load-bearing objection A sensible, cheap text-side self-supervision idea for medical CLIP with a well-designed controlled ablation, but the paper's own tables contradict its 'consistent improvements' headline, and the GPT-4-generated references make the reported numbers hard to interpret. the 4 major comments →

arxiv 2607.21970 v1 pith:UWXKTU6F submitted 2026-07-24 cs.CV cs.AI

TextSLIP: Text Self-Supervised CLIP for Medical Report Generation

Haoyu Jiang , Ziping Cong This is my paper

classification cs.CV cs.AI

keywords medical report generationcontrastive learningself-supervised learningCLIPESimCSEvision-language pretrainingbrain MRIradiology report generation

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that standard CLIP's image-text contrastive training leaves the textual embedding space poorly structured, which limits how precisely the visual encoder can learn medical semantics. TextSLIP adds an ESimCSE-style self-supervised text contrastive objective — augmenting reports by repeating words and contrasting against a momentum-encoded queue — so that text embeddings become more discriminative and the visual encoder receives finer-grained linguistic supervision. Pretrained on 7 million brain MRI image-text pairs and fine-tuned inside the R2Gen report generator, TextSLIP reports consistent BLEU/ROUGE-L improvements over CLIP baselines, including roughly 3% higher BLEU-4 in several settings. The authors present this as an initial validation; broader medical domains and clinically grounded metrics remain explicitly acknowledged future work.

Core claim

The central claim is that intra-modal text contrastive learning, on top of cross-modal CLIP alignment, improves medical report generation by making the textual embedding space more discriminative. Concretely, TextSLIP uses word-repetition augmentation to create positive text pairs, a momentum text encoder with a FIFO queue for negatives, and minimizes L = L_CLIP + lambda * L_ESimCSE. In controlled ablations where the only difference from the CLIP baseline is the text self-supervision term, TextSLIP outperforms CLIP-pretrained and ResNet-101 visual encoders across T1n, T1c, T2w, and T2f MRI modalities.

What carries the argument

The load-bearing object is the ESimCSE text-contrastive branch: random word duplication produces positive views of the same report, while the momentum text encoder and FIFO queue supply a large bank of negatives for InfoNCE. This objective is added to the standard CLIP image-text InfoNCE loss with a weighting lambda. The paper's argument is that this term imposes structure on the text embedding space, and that structure is what lets the visual encoder learn finer-grained alignments between images and report phrases.

Load-bearing premise

The evaluation uses reports that were automatically generated and translated/structured by a model-based pipeline as ground truth, so the BLEU and ROUGE scores measure agreement with synthetic references rather than with real clinician-written reports; the paper itself notes that current evaluation relies on lexical metrics and a structured processing pipeline.

What would settle it

Run the same pretraining and fine-tuning comparison on a public radiology dataset whose reports were written by clinicians, and compute BLEU/ROUGE against the real reports. If TextSLIP no longer beats the same-data CLIP baseline, the claim that text self-supervision improves report generation would be falsified; alternatively, ablating the word-repetition augmentation to random token dropout and seeing the gains disappear would test whether the specific ESimCSE mechanism matters.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

If text self-supervision is the cause, pretraining CLIP-style models for medical report generation should include a text-side contrastive loss even when image-text data is abundant.
The gains transfer to the R2Gen report generator as an off-the-shelf visual encoder, implying the learned visual representations are more useful for downstream generation, not just retrieval.
Because TextSLIP uses only the already-available text of image-text pairs, it adds no annotation cost over CLIP pretraining.
The framework is domain-agnostic (MRI is just the testbed), so the same objective can be applied to other medical modalities such as chest X-ray or pathology.
Higher-order BLEU improvements suggest longer, more detailed report phrases are generated, not just single-word matches.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The reported numbers rest on synthetic reference reports produced by an automated model-based pipeline; until the same comparison is run against authentic radiologist-written reports, the BLEU/ROUGE gains should be read as improvements on a proxy, not proof of clinical report quality.
A cheap falsifying experiment: replace the word-repetition augmentation with a semantics-preserving synonym/entity-preserving augmentation; if the gains vanish, the mechanism is about self-supervision generally, not about ESimCSE specifically.
The method could be stress-tested on a modality where authentic reports exist (e.g., chest X-ray datasets) to see whether the 3% BLEU-4 gain survives the shift from synthetic to real ground truth.
Text-side discriminability may help other vision-language tasks beyond generation, such as zero-shot classification or cross-modal retrieval, which CLIP-style encoders are typically used for; the paper does not test this.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

A sensible, cheap text-side self-supervision idea for medical CLIP with a well-designed controlled ablation, but the paper's own tables contradict its 'consistent improvements' headline, and the GPT-4-generated references make the reported numbers hard to interpret.

read the letter

Quick take: the core idea is simple and plausible — add ESimCSE-style word-repetition contrast on the text side to CLIP, so the text encoder gets a structured embedding space that then guides the visual encoder. That is a natural mirror of SLIP, and applying it to medical VLP on 7M brain MRI pairs appears to be new in the cited literature. The controlled ablation, where the only difference is the pretraining objective, is exactly the right way to isolate the effect, and the code is public. Those are real strengths.

The soft spots are serious, and the stress-test note is right about the most load-bearing one. The paper claims TextSLIP 'consistently surpasses both baselines across all modalities' (Section 3.3), but Table 2 shows TextSLIP ROUGE-L on T1n at 0.5527, below the unpretrained ResNet-101's 0.5578, and Table 1 shows the same T1n ROUGE-L below UniMed-CLIP's 0.5543, plus T2w BL-4 below UniMed-CLIP. That is not noise; the point estimates are right there. The abstract and conclusion repeat the 'consistent improvements' phrasing. That needs to be corrected to 'improvements on most settings' or the exceptions need to be addressed.

The reader's synthetic-reference concern is also valid. The BRATS2023-GLI reports are translated and structured by AutoRG Brain with GPT-4 prompts, and BLEU/ROUGE are computed against those machine-generated references. If the references do not resemble real clinical reports, the absolute scores mean less. The paper acknowledges this in its limitations, which is honest, but it still leans on the numbers.

Other issues are smaller but worth fixing: the loss weight lambda is never reported, there are no error bars or significance tests, and the 7M-pair curation gets one sentence. The direction of the effect is mostly positive, so these are fixable problems, not a dead end.

Who gets value? People working on medical VLP and report generation, as a baseline idea and a cautionary example of claim-reporting mismatch. I would send it to peer review with a request for major revision — correct the consistency claim, report lambda and variance, and ideally add a reference-quality check — rather than desk reject it.

Referee Report

4 major / 5 minor

Summary. The paper proposes TextSLIP, a CLIP-style medical vision-language pretraining framework that augments the standard image-text contrastive objective with an ESimCSE-style intra-modal text contrastive loss. Text augmentation is random word repetition; negatives are drawn from a momentum text encoder through a FIFO queue. The model is pretrained on roughly 7 million brain MRI image-text pairs from MedTrinity-25M and evaluated by fine-tuning the visual encoder in the R2Gen report-generation architecture on BRATS2023-GLI, using BLEU and ROUGE-L. The central claim is that text-side self-supervision provides finer-grained linguistic supervision to the visual encoder, yielding consistent improvements over CLIP trained on the same data and over existing CLIP-style encoders.

Significance. If the central empirical claim were supported, the paper would offer a simple, generally applicable addition to medical CLIP pretraining. The controlled ablation is the right way to isolate the proposed objective, the pretraining scale is nontrivial, and the authors state that the code is public. However, the evidence as presented does not establish the claimed 'consistent improvements': several published point estimates contradict the descriptor, no uncertainty quantification or significance testing is provided, and the evaluation references are machine-generated reports. The conceptual direction is promising, but the current manuscript needs substantial empirical and presentational repair.

major comments (4)

[§3.3, Tables 1–2] The central claim that 'TextSLIP consistently surpasses both baselines across all modalities' is contradicted by the paper's own tables. In Table 1, T1n ROUGE-L for TextSLIP is 0.5527 versus UniMed-CLIP's 0.5543; in Table 2, T1n ROUGE-L for TextSLIP is 0.5527 versus ResNet-101's 0.5578; and in Table 1, T2w BL-4 for TextSLIP is 0.3103 versus UniMed-CLIP's 0.3121. No error bars, multiple seeds, or significance tests are reported, so even the favorable differences cannot be distinguished from noise. The abstract and Section 3.3 must be revised to reflect the actual pattern, or new experiments must provide statistical evidence for consistency.
[§3.1] The reference reports used for evaluation are 'translated and structured into English using AutoRG Brain with GPT-4 prompts.' BLEU and ROUGE-L therefore measure lexical overlap with machine-generated text, not clinical correctness or fidelity to real radiology practice. Since every number in Tables 1 and 2 depends on this reference set, the reported improvements do not establish that TextSLIP generates better medical reports. The authors should validate the synthetic references (e.g., expert review or comparison with original reports) and/or add clinically meaningful metrics such as CheXpert-style labels, RadGraph, or expert evaluation.
[§2.2, Eq. (8); §3.2] The loss weight λ in Eq. (8) is never reported, nor is the temperature τ used in Eqs. (2)–(7). Because λ controls the contribution of the proposed text self-supervision, a downstream-tuned λ could compromise the ablation's ability to isolate the objective. The authors must report the value of λ and describe how it was selected (validation versus test), together with τ and queue size. In addition, the absence of multiple seeds or confidence intervals for any reported metric makes small differences, such as 0.0002, uninterpretable.
[§3.1–3.2] No overlap or deduplication analysis is described between the 7M MedTrinity-25M pretraining set and the BRATS2023-GLI fine-tuning/test set. MedTrinity aggregates over 30 public sources; if any BRATS studies or derived slices appear in the pretraining data, the downstream comparison is invalid. The authors should report an overlap analysis and exclude any overlapping studies, or explicitly justify that no leakage is possible.

minor comments (5)

[§4] The conclusion refers to 'qualitative analyses,' but no qualitative results or figures are presented in the manuscript. Either include them or remove the reference.
[Abstract] There is a stray space before the period in 'improving reporting consistency and clinical workflows .' Please fix.
[Table 2] The table header contains 'T able' instead of 'Table.' Please correct the typo.
[Fig. 1] Figure 1 is difficult to parse: the matrix/queue notation, the relation between pretrain and finetune stages, and the 'FIFO Memory Queue' are not clearly labeled. Enlarging and annotating the figure would improve reproducibility and readability.
[§3.2] Implementation details report the momentum coefficient (α=0.99) but omit the FIFO queue size and the temperature τ. These are needed to reproduce the method.

Circularity Check

0 steps flagged

No significant circularity: the central claim is an empirical comparison, not a derivation from the target metrics.

full rationale

The paper's central claim is that adding an ESimCSE-style intra-modal text contrastive loss to CLIP improves downstream medical report generation metrics. This is an empirical, measured result, not a formal derivation. The text self-supervision objective (Eq. 7) uses fixed word-repetition augmentation and a momentum-encoder FIFO queue; no fitted parameter in that objective encodes BLEU or ROUGE-L, and the downstream fine-tuning in R2Gen is a separate stage. The overall loss (Eq. 8) contains an unreported weighting hyperparameter λ, but an unreported or even validation-tuned hyperparameter is a standard practice and does not make the prediction reducible to the input by construction. The controlled ablation compares TextSLIP against a CLIP baseline pretrained on the same 7M brain MRI pairs and an unpretrained ResNet-101, so the comparison isolates the pretraining objective rather than relying on a self-citation or definitional identity. There are no self-citations by the present authors in the reference list. The manuscript's claim of 'consistent improvements' is contradicted by its own tables (e.g., Table 2 T1n ROUGE-L: TextSLIP 0.5527 vs ResNet-101 0.5578; Table 1 T2w BL-4: TextSLIP 0.3103 vs UniMed-CLIP 0.3121), but that is an internal-validity/correctness problem, not circularity. Likewise, the use of GPT-4-structured synthetic reports as references is a benchmark-validity concern, not a circular derivation. No load-bearing step reduces to its own input by construction.

Axiom & Free-Parameter Ledger

3 free parameters · 4 axioms · 0 invented entities

The paper introduces no new physical or conceptual entities; it combines existing components (CLIP, ESimCSE, R2Gen). The free parameters are standard hyperparameters, with λ unreported. The key implicit assumptions are the validity of synthetic references and the transferability of the 7M-pair corpus to BRATS.

free parameters (3)

λ (ESimCSE loss weight) = not reported
Balances L_CLIP and L_ESimCSE in Eq. 8; value is not given in the paper, and it directly controls how much text self-supervision contributes. If tuned on the downstream validation set, it could overstate gains.
τ (InfoNCE temperature) = not reported
Temperature in Eqs. 2, 3, and 7; not specified. It affects the sharpness of all contrastive losses and is typically a tuned hyperparameter.
α (momentum coefficient) = 0.99
Exponential moving average for the momentum text encoder (Eq. 6); reported in the paper, standard value.

axioms (4)

domain assumption The 7M brain MRI subset of MedTrinity-25M is a suitable pretraining corpus for downstream BRATS report generation.
The transfer between the pretraining corpus and the fine-tuning dataset is assumed in §3.1 and evaluated in §3.3, but no analysis shows the distributions match.
ad hoc to paper Reports generated by AutoRG Brain with GPT-4 are valid reference texts for measuring report generation quality.
§3.1 uses these AI-generated reports as ground truth for BLEU/ROUGE. If the synthetic reports are noisy or unrepresentative, all evaluation numbers are called into question.
domain assumption Word duplication (ESimCSE) is an effective text augmentation for medical reports and yields more discriminative embeddings.
§2.2 adopts ESimCSE without demonstrating that random word repetition preserves medical semantics (e.g., negation, anatomy, uncertainty). This is the core mechanism of the proposed method.
domain assumption 2D slices along three axes, with concatenated embeddings, preserve the 3D lesion context needed for report generation.
§3.2 describes extracting 2D slices centered on lesion masks and concatenating embeddings from three axes; no ablation shows this representation is sufficient for the report decoder.

pith-pipeline@v1.3.0-alltime-deepseek · 7007 in / 9023 out tokens · 84796 ms · 2026-08-01T06:09:19.594185+00:00 · methodology

0 comments

read the original abstract

Automating radiology report generation is important for improving reporting consistency and clinical workflows . While Contrastive Language--Image Pretraining (CLIP) has advanced medical vision language modeling, existing CLIP-style approaches may still provide insufficient fine-grained semantic supervision for complex report generation. Standard CLIP primarily optimizes cross-modal alignment, without explicitly structuring the textual embedding space that guides visual representation learning. To address this limitation, we propose TextSLIP, a general medical vision-language pretraining framework that augments CLIP with intra-modal text contrastive learning. By improving textual embedding discriminability through self-supervised augmented text pairs, TextSLIP is designed to provide finer-grained linguistic supervision to the visual encoder. As an initial validation, we pretrain TextSLIP on a curated dataset of 7 million brain MRI image-text pairs and fine-tune the pretrained visual encoder within a report generation architecture. In controlled comparisons with CLIP-style baselines, TextSLIP shows consistent improvements on report generation metrics. Ablation studies further suggest that text-side self-supervision contributes to the observed gains. These results indicate that text-level contrastive learning is a promising direction for improving medical visual-textual alignment, while broader validation across additional medical domains remains an important next step.

Figures

Figures reproduced from arXiv: 2607.21970 by Haoyu Jiang, Ziping Cong.

**Figure 1.** Figure 1: Overview of TextSLIP 2.1 Contrastive Learning Foundations TextSLIP builds upon two core contrastive learning paradigms: cross-modal image-text alignment and intra-modal text contrast. The following subsections detail these foundational components. Image-Text Contrastive Alignment Contrastive Language–Image Pretraining [12] learns visual representations by aligning images with corresponding captions. The … view at source ↗

discussion (0)

Reference graph

Works this paper leans on

19 extracted references · 6 linked inside Pith

[1]

ArXiv pp

Adewole, M., Rudie, J.D., Gbdamosi, A., Toyobo, O., Raymond, C., Zhang, D., Omidiji, O., Akinola, R., Suwaid, M.A., Emegoakor, A., et al.: The brain tumor segmentation (brats) challenge 2023: Glioma segmentation in sub-saharan africa patient population (brats-africa). ArXiv pp. arXiv–2305 (2023)

2023
[2]

In: Proceed- ings of the 28th international conference on computational linguistics

Chakraborty, S., Bisong, E., Bhatt, S., Wagner, T., Elliott, R., Mosconi, F.: Biomedbert: A pre-trained biomedical language model for qa and ir. In: Proceed- ings of the 28th international conference on computational linguistics. pp. 669–679 (2020)

2020
[3]

Chen, Z., Shen, Y., Song, Y., Wan, X.: Cross-modal memory networks for radiology report generation. In: Proceedings of the 59th annual meeting of the association for computational linguistics and the 11th international joint conference on natural language processing (volume 1: long papers). pp. 5904–5914 (2021)

2021
[4]

In: Proceedings of the 2020 conference on empirical methods in natural language processing (EMNLP)

Chen, Z., Song, Y., Chang, T.H., Wan, X.: Generating radiology reports via memory-driven transformer. In: Proceedings of the 2020 conference on empirical methods in natural language processing (EMNLP). pp. 1439–1449 (2020)

2020
[5]

arXiv preprint arXiv:2412.10372 (2024)

Khattak, M.U., Kunhimon, S., Naseer, M., Khan, S., Khan, F.S.: Unimed-clip: Towards a unified image-text pretraining paradigm for diverse medical imaging modalities. arXiv preprint arXiv:2412.10372 (2024)

Pith/arXiv arXiv 2024
[6]

arXiv preprint arXiv:2407.16684 (2024)

Lei, J., Zhang, X., Wu, C., Dai, L., Zhang, Y., Zhang, Y., Wang, Y., Xie, W., Li, Y.: Autorg-brain: Grounded report generation for brain mri. arXiv preprint arXiv:2407.16684 (2024)

Pith/arXiv arXiv 2024
[7]

In:Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

Li, M., Lin, B., Chen, Z., Lin, H., Liang, X., Chang, X.: Dynamic graph en- hanced contrastive learningfor chestx-ray report generation. In:Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 3334–3343 (2023)

2023
[8]

In: Text sum- marization branches out

Lin, C.Y.: Rouge: A package for automatic evaluation of summaries. In: Text sum- marization branches out. pp. 74–81 (2004)

2004
[9]

In: European conference on computer vision

Mu, N., Kirillov, A., Wagner, D., Xie, S.: Slip: Self-supervision meets language- image pre-training. In: European conference on computer vision. pp. 529–544. Springer (2022)

2022
[10]

arXiv preprint arXiv:1807.03748 (2018)

Oord, A.v.d., Li, Y., Vinyals, O.: Representation learning with contrastive predic- tive coding. arXiv preprint arXiv:1807.03748 (2018)

Pith/arXiv arXiv 2018
[11]

In: Proceedings of the 40th annual meeting of the Association for Computational Linguistics

Papineni, K., Roukos, S., Ward, T., Zhu, W.J.: Bleu: a method for automatic evaluation of machine translation. In: Proceedings of the 40th annual meeting of the Association for Computational Linguistics. pp. 311–318 (2002) TextSLIP: Text Self-Supervised CLIP for Medical Report Generation 9

2002
[12]

In: International conference on machine learning

Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al.: Learning transferable visual models from natural language supervision. In: International conference on machine learning. pp. 8748–8763. PmLR (2021)

2021
[13]

In: Proceedings of the computer vision and pattern recognition con- ference

Wang, X., Wang, F., Li, Y., Ma, Q., Wang, S., Jiang, B., Tang, J.: Cxpmrg-bench: Pre-training and benchmarking for x-ray medical report generation on chexpert plus dataset. In: Proceedings of the computer vision and pattern recognition con- ference. pp. 5123–5133 (2025)

2025
[14]

In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

Wang, Z., Liu, L., Wang, L., Zhou, L.: Metransformer: Radiology report genera- tion by transformer with multiple learnable expert tokens. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 11558– 11567 (2023)

2023
[15]

In: Proceedings of the 29th International Conference on Computational Linguistics

Wu, X., Gao, C., Zang, L., Han, J., Wang, Z., Hu, S.: Esimcse: Enhanced sample building method for contrastive learning of unsupervised sentence embedding. In: Proceedings of the 29th International Conference on Computational Linguistics. pp. 3898–3907 (2022)

2022
[16]

arXiv preprint arXiv:2408.02900 (2024)

Xie, Y., Zhou, C., Gao, L., Wu, J., Li, X., Zhou, H.Y., Liu, S., Xing, L., Zou, J., Xie, C., et al.: Medtrinity-25m: A large-scale multimodal dataset with multigranular annotations for medicine. arXiv preprint arXiv:2408.02900 (2024)

Pith/arXiv arXiv 2024
[17]

arXiv preprint arXiv:2309.16671 (2023)

Xu, H., Xie, S., Tan, X.E., Huang, P.Y., Howes, R., Sharma, V., Li, S.W., Ghosh, G., Zettlemoyer, L., Feichtenhofer, C.: Demystifying clip data. arXiv preprint arXiv:2309.16671 (2023)

Pith/arXiv arXiv 2023
[18]

In: Proceedings of the AAAI conference on artificial intelligence

Yan, B., Pei, M.: Clinical-bert: Vision-language pre-training for radiograph diag- nosis and reports generation. In: Proceedings of the AAAI conference on artificial intelligence. vol. 36, pp. 2982–2990 (2022)

2022
[19]

arXiv preprint arXiv:2303.00915 (2023)

Zhang, S., Xu, Y., Usuyama, N., Xu, H., Bagga, J., Tinn, R., Preston, S., Rao, R., Wei, M., Valluri, N., et al.: Biomedclip: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairs. arXiv preprint arXiv:2303.00915 (2023)

Pith/arXiv arXiv 2023

[1] [1]

ArXiv pp

Adewole, M., Rudie, J.D., Gbdamosi, A., Toyobo, O., Raymond, C., Zhang, D., Omidiji, O., Akinola, R., Suwaid, M.A., Emegoakor, A., et al.: The brain tumor segmentation (brats) challenge 2023: Glioma segmentation in sub-saharan africa patient population (brats-africa). ArXiv pp. arXiv–2305 (2023)

2023

[2] [2]

In: Proceed- ings of the 28th international conference on computational linguistics

Chakraborty, S., Bisong, E., Bhatt, S., Wagner, T., Elliott, R., Mosconi, F.: Biomedbert: A pre-trained biomedical language model for qa and ir. In: Proceed- ings of the 28th international conference on computational linguistics. pp. 669–679 (2020)

2020

[3] [3]

Chen, Z., Shen, Y., Song, Y., Wan, X.: Cross-modal memory networks for radiology report generation. In: Proceedings of the 59th annual meeting of the association for computational linguistics and the 11th international joint conference on natural language processing (volume 1: long papers). pp. 5904–5914 (2021)

2021

[4] [4]

In: Proceedings of the 2020 conference on empirical methods in natural language processing (EMNLP)

Chen, Z., Song, Y., Chang, T.H., Wan, X.: Generating radiology reports via memory-driven transformer. In: Proceedings of the 2020 conference on empirical methods in natural language processing (EMNLP). pp. 1439–1449 (2020)

2020

[5] [5]

arXiv preprint arXiv:2412.10372 (2024)

Khattak, M.U., Kunhimon, S., Naseer, M., Khan, S., Khan, F.S.: Unimed-clip: Towards a unified image-text pretraining paradigm for diverse medical imaging modalities. arXiv preprint arXiv:2412.10372 (2024)

Pith/arXiv arXiv 2024

[6] [6]

arXiv preprint arXiv:2407.16684 (2024)

Lei, J., Zhang, X., Wu, C., Dai, L., Zhang, Y., Zhang, Y., Wang, Y., Xie, W., Li, Y.: Autorg-brain: Grounded report generation for brain mri. arXiv preprint arXiv:2407.16684 (2024)

Pith/arXiv arXiv 2024

[7] [7]

In:Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

Li, M., Lin, B., Chen, Z., Lin, H., Liang, X., Chang, X.: Dynamic graph en- hanced contrastive learningfor chestx-ray report generation. In:Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 3334–3343 (2023)

2023

[8] [8]

In: Text sum- marization branches out

Lin, C.Y.: Rouge: A package for automatic evaluation of summaries. In: Text sum- marization branches out. pp. 74–81 (2004)

2004

[9] [9]

In: European conference on computer vision

Mu, N., Kirillov, A., Wagner, D., Xie, S.: Slip: Self-supervision meets language- image pre-training. In: European conference on computer vision. pp. 529–544. Springer (2022)

2022

[10] [10]

arXiv preprint arXiv:1807.03748 (2018)

Oord, A.v.d., Li, Y., Vinyals, O.: Representation learning with contrastive predic- tive coding. arXiv preprint arXiv:1807.03748 (2018)

Pith/arXiv arXiv 2018

[11] [11]

In: Proceedings of the 40th annual meeting of the Association for Computational Linguistics

Papineni, K., Roukos, S., Ward, T., Zhu, W.J.: Bleu: a method for automatic evaluation of machine translation. In: Proceedings of the 40th annual meeting of the Association for Computational Linguistics. pp. 311–318 (2002) TextSLIP: Text Self-Supervised CLIP for Medical Report Generation 9

2002

[12] [12]

In: International conference on machine learning

Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al.: Learning transferable visual models from natural language supervision. In: International conference on machine learning. pp. 8748–8763. PmLR (2021)

2021

[13] [13]

In: Proceedings of the computer vision and pattern recognition con- ference

Wang, X., Wang, F., Li, Y., Ma, Q., Wang, S., Jiang, B., Tang, J.: Cxpmrg-bench: Pre-training and benchmarking for x-ray medical report generation on chexpert plus dataset. In: Proceedings of the computer vision and pattern recognition con- ference. pp. 5123–5133 (2025)

2025

[14] [14]

In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

Wang, Z., Liu, L., Wang, L., Zhou, L.: Metransformer: Radiology report genera- tion by transformer with multiple learnable expert tokens. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 11558– 11567 (2023)

2023

[15] [15]

In: Proceedings of the 29th International Conference on Computational Linguistics

Wu, X., Gao, C., Zang, L., Han, J., Wang, Z., Hu, S.: Esimcse: Enhanced sample building method for contrastive learning of unsupervised sentence embedding. In: Proceedings of the 29th International Conference on Computational Linguistics. pp. 3898–3907 (2022)

2022

[16] [16]

arXiv preprint arXiv:2408.02900 (2024)

Xie, Y., Zhou, C., Gao, L., Wu, J., Li, X., Zhou, H.Y., Liu, S., Xing, L., Zou, J., Xie, C., et al.: Medtrinity-25m: A large-scale multimodal dataset with multigranular annotations for medicine. arXiv preprint arXiv:2408.02900 (2024)

Pith/arXiv arXiv 2024

[17] [17]

arXiv preprint arXiv:2309.16671 (2023)

Xu, H., Xie, S., Tan, X.E., Huang, P.Y., Howes, R., Sharma, V., Li, S.W., Ghosh, G., Zettlemoyer, L., Feichtenhofer, C.: Demystifying clip data. arXiv preprint arXiv:2309.16671 (2023)

Pith/arXiv arXiv 2023

[18] [18]

In: Proceedings of the AAAI conference on artificial intelligence

Yan, B., Pei, M.: Clinical-bert: Vision-language pre-training for radiograph diag- nosis and reports generation. In: Proceedings of the AAAI conference on artificial intelligence. vol. 36, pp. 2982–2990 (2022)

2022

[19] [19]

arXiv preprint arXiv:2303.00915 (2023)

Zhang, S., Xu, Y., Usuyama, N., Xu, H., Bagga, J., Tinn, R., Preston, S., Rao, R., Wei, M., Valluri, N., et al.: Biomedclip: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairs. arXiv preprint arXiv:2303.00915 (2023)

Pith/arXiv arXiv 2023