Pith. sign in

REVIEW 3 major objections 7 minor 34 references

Anatomy-Grounded Weakly Supervised Prompt Tuning for Chest X-ray Latent Diffusion Models

T0 review · 3 major / 7 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that fine-tuning only the anatomy token embeddings of a pre-trained chest X-ray latent diffusion model, guided by a weak Gaussian supervision signal derived from radiology reports and a small anatomy-box dataset, yields…

desk verdict A useful prompt-tuning recipe for medical phrase grounding, with an undisclosed train/test overlap that should be fixed before the SOTA claim is trusted. read the letter →

arxiv 2506.10633 v1 pith:QIPYA7CJ submitted 2025-06-12 cs.CV

Konstantinos Vilouras , Ilias Stogiannidis , Junyu Yan , Alison Q. O'Neil , Sotirios A. Tsaftaris This is my paper
classification cs.CV
keywords phrasegroundinglatentdiffusionmodelschestX-rayweaklysupervisedlearningprompttuningcross-attentionout-of-distributiongeneralization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that a pre-trained chest X-ray latent diffusion model can be repurposed for phrase grounding without retraining its image or text encoders. The authors argue that the model's cross-attention maps leak diffusely over the image, and that neither free-text radiology prompts nor a simplified '{location} {pathology}' format fix this. Their remedy is to fine-tune only the anatomy token embeddings in the prompt, using a weak supervision signal: a per-location 2D Gaussian built from a small set of anatomy bounding boxes, combined with a diversity loss that separates token-level attention. On the MS-CXR phrase-grounding benchmark the fine-tuned model sets a new state of the art, and it also outperforms contrastive baselines on the out-of-distribution VinDr-CXR test set. The claim matters because it suggests frozen diffusion models can be converted into localization tools with minimal data and parameters, using only unstructured radiology reports as supervision.

What carries the argument

The load-bearing object is the stack of cross-attention activation maps $A \in \mathbb{R}^{T\times L\times D\times S}$ extracted from 30 diffusion timesteps and 4 U-Net layers (one bottleneck, three decoder) of the frozen LDM. The maps are averaged over the token dimension, reshaped into spatial feature maps, and compared against a fixed per-location 2D Gaussian lookup table (LUT) whose means and variances come from Chest Imagenome bounding boxes. The supervision target is a linear mix of the stopped-gradient attention map and the Gaussian, $\mathrm{trg} = \mathbf{1}_{T>\epsilon}\cdot\mathrm{sg}(A_{\mathrm{sp}}) + \alpha T$, and the objective is $\mathcal{L} = \mathcal{L}_{\mathrm{div}} + \mathcal{L}_{\mathrm{loc}}$, where $\mathcal{L}_{\mathrm{loc}}$ is a cosine localization loss against this target and $\mathcal{L}_{\mathrm{div}}$ penalizes pairwise cosine overlap between $\ell_2$-normalized token attention maps. The only trainable parameters are 46 token embeddings (locations, pathologies, and special tokens), each 1024-dimensional; the VAE, CLIP text encoder, and denoising U-Net stay frozen.

What would settle it

Compare per-location phrase-grounding gains (fine-tuned minus frozen LDM) between anatomical terms that have direct Chest Imagenome box statistics and the mapped terms ('lingular', 'base', 'pleural'); if the mapped terms show no improvement, or the fine-tuned attention simply follows the fixed Gaussian even when the ground-truth box is elsewhere, then the method's apparent success is driven by the prior rather than by prompt tuning.

Watch

Extended reading notes

Core claim

The central discovery is that a frozen text-to-image latent diffusion model, whose cross-attention activations normally bleed across the whole chest X-ray, can be steered to the correct anatomy by updating only a small codebook of location-token embeddings. Fine-tuning uses a curated set of 6,480 image-report pairs: RadGraph-XL extracts anatomical entities, a lookup table converts each entity into a fixed 2D Gaussian target derived from Chest Imagenome bounding-box statistics, and the training objective pushes the model's averaged cross-attention maps toward a dynamically mixed target while penalizing overlap between token-level attention maps. After tuning, the cross-attention heatmaps themselves serve as phrase-grounding predictions, achieving higher CNR and mIoU than BioViL, BioViL-T and the frozen LDM on MS-CXR-loc, and similar gains on VinDr-CXR. The authors interpret this as evidence that token-level prompt tuning, rather than architectural change or full fine-tuning, is sufficient to align clinical language with image regions.

Load-bearing premise

The load-bearing premise is that a fixed Gaussian per anatomical term, averaged from Chest Imagenome bounding boxes (with 'lingular' mapped to 'left mid lung zone' and 'base' to 'lower lung zone'), is a correct-enough spatial prior for that term across every X-ray in the fine-tuning and evaluation sets, so that pushing attention toward the Gaussian teaches the right location rather than a systematically wrong one.

Editorial extensions

If this is right

  • A diffusion model's cross-attention can be used directly as a phrase-grounding heatmap: the fine-tuned LDM beats contrastive baselines (BioViL, BioViL-T) without any task-specific detection head.
  • Because only a (46, 1024) embedding codebook is updated, fine-tuning is cheap (about 3 hours on a single RTX 3090) and needs only about 6,500 curated image-report pairs.
  • The localization gains transfer across hospitals: VinDr-CXR, collected with different scanners and demographics, also shows higher CNR/mIoU than the baselines.
  • Standardizing free-text reports to '{location} {pathology}' prompts preserves average grounding performance, so the reported gains come from the alignment tuning rather than from prompt simplification.
  • Adding more anatomical locations or pathologies only requires adding an entry to the Gaussian lookup table and a corresponding token embedding, so the framework extends without architectural changes.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The recipe is not tied to chest X-rays: any frozen text-to-image diffusion model with exposed token embeddings could in principle be steered with the same two losses, provided a spatial prior (even a crude one) can be written down for each concept of interest.
  • Because the supervision signal is the Gaussian prior itself, per-location performance should track the quality of the LUT; locations that had to be mapped from a neighbouring zone ('lingular', 'base', 'pleural') are a natural pressure test for whether the gains are real alignment or prior-fitting.
  • The ablation in which removing the diversity loss slightly improves average mIoU, driven by pneumothorax, suggests a pathology-aware weighting of $\mathcal{L}_{\mathrm{div}}$ (perhaps down-weighting it for pathologies intrinsically tied to one anatomy) could improve results further; the paper leaves this open.
  • If the approach scales to larger report corpora, the same coarse supervision could generate training signal for many more fine-grained anatomical terms than the 27 used here, effectively bootstrapping a localization model from text alone plus a small anatomy atlas.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. This paper proposes a parameter-efficient fine-tuning method to improve the alignment between text prompts and anatomical regions in a pre-trained chest X-ray latent diffusion model (LDM). The authors first show that the model's cross-attention activations are diffuse when prompted with either free-text report sentences or structured '{location} {pathology}' prompts (Figure 1). They then curate a fine-tuning set of 6,480 MIMIC-CXR samples by extracting anatomy entities with RadGraph-XL, standardizing prompts, and mapping each of 27 anatomical locations to a fixed 2D Gaussian derived from Chest Imagenome bounding-box statistics. Only the anatomy/pathology token embeddings (a codebook of 46 x 1024 entries) are updated, while the VAE, CLIP text encoder, and U-Net remain frozen; the objective combines a token diversity loss with a localization loss whose target is a masked, detached copy of the model's own activations plus the Gaussian prior (Eqs. 2-4). The fine-tuned model is evaluated on a filtered subset of MS-CXR (MS-CXR-loc, 1,043 samples) and on VinDr-CXR with synthetic prompts, reporting average CNR of 1.37 vs 0.93 and mIoU of 29.5 vs 22.0 for the fine-tuned vs frozen LDM on MS-CXR-loc, with bootstrap confidence intervals. The paper claims state-of-the-art phrase grounding on both benchmarks and provides ablations, an oracle upper bound, a prompt-format analysis, and a limitations section.

Significance. If the claims hold, the paper offers a practical way to repurpose a generative model for weakly supervised phrase grounding with very few trainable parameters, and the evaluation design is more thorough than is typical: it includes an out-of-distribution dataset, a prompt-format analysis (Appendix D), an oracle upper bound, a MAIRA-2 comparison, and an honest limitations section. The empirical results are also consistent: across most classes the fine-tuned LDM's bootstrap intervals do not overlap those of the frozen LDM (e.g., Pneumothorax CNR from -0.07 to 1.38 on MS-CXR-loc, Table 1), and the gains transfer to VinDr-CXR (Table 3). The data-curation pipeline combining RadGraph-XL entity extraction with Chest Imagenome-derived Gaussian priors is a useful methodological contribution, and the commitment to release code is welcome.

major comments (3)
  1. [Section 3.2, Section 4.1, Table 1] The central state-of-the-art claim requires an explicit audit of the overlap between the fine-tuning set and the evaluation set. The 6,480-sample fine-tuning set is drawn from the ~70k samples used to train the LDM (Section 3.2, Appendix A), and MS-CXR-loc is also a subset of MIMIC-CXR (Section 4.1), yet the paper only reports the overlap with Chest Imagenome (28 images) and never the overlap with MS-CXR-loc. Because the objective in Eq. 3 updates the shared anatomy token embeddings from every fine-tuning sample, any MS-CXR-loc image present in the fine-tuning set would partially optimize the embeddings for the exact evaluation images and could inflate the results in Table 1. Please report the image-level and subject-level overlap between the fine-tuning set and MS-CXR-loc (and VinDr-CXR for completeness), and, if the overlap is non-zero, re-run the evaluation on the non-overlapping subset. This is a standard leakage check and it is load-bearing for the abstract's SOTA claim.
  2. [Section 3.3, Eq. (3), Table 2] The supervision target in Eq. (3) is dominated by the model's own detached activations, so the external anatomical prior plays a smaller role than the paper's framing suggests. With the stated threshold epsilon = 1e-5, the binary mask 1_{T>epsilon} is nearly all-ones for broad locations such as left/right lung, giving trg approximately equal to sg(A_sp) + 0.1*T; the localization loss then mostly sharpens the model's existing attention, with the Gaussian acting as a weak additive bias. This is consistent with Table 2, where the alpha = 0 ablation (no Gaussian term at all) already accounts for most of the improvement over the frozen LDM (CNR 1.30 vs 1.37, mIoU 28.5 vs 29.5). The authors should (i) report the average fraction of active mask entries per location, and (ii) add an ablation with the target set to the pure Gaussian (trg = T) to isolate the contribution of the anatomical prior. As written, the claim that the method steers attention 'towards the anatomical area' overstates the role of the external, anatomy-grounded signal.
  3. [Abstract, Section 4.1, Table 1] The abstract's claim of a 'new state-of-the-art on a standard benchmark dataset (MS-CXR)' is stronger than what the experiments support. The evaluation is performed on MS-CXR-loc, a filtered subset of MS-CXR that discards pairs for which RadGraph-XL predicts no anatomy entity, under standardized '{location} {pathology}' prompts rather than the original MS-CXR sentences. Moreover, on the average CNR metric the proposed model ties with BioViL-T on MS-CXR-loc (both 1.37), so the statement in Section 4.6 that the method outperforms contrastive baselines 'by a significant margin' should be qualified. Please reword the claims to name MS-CXR-loc and the prompt transformation, and ideally report results on the unfiltered MS-CXR set with original prompts so that comparison with the published BioViL/BioViL-T numbers is direct; the prompt-format analysis in Appendix D mitigates but does not eliminate this comparability issue.
minor comments (7)
  1. [Table 4] The Pneumothorax confidence interval on VinDr-CXR reads '[5,61, 23.3]', which appears to be a typo for '[5.61, 23.3]', and the Lung Opacity row reports a point estimate of 32.2 equal to the lower CI bound '[32.2, 41.4]', which also appears erroneous.
  2. [Figure 5, Section 3.2] The per-pathology counts in Figure 5 sum to 7,298 (including the Edema and Cardiomegaly counts given in the caption), but the text states that the fine-tuning set contains 6,480 samples with each sample assigned to exactly one pathology; these numbers should be reconciled.
  3. [Section 3.3] The codebook size is given as 46 entries for 27 locations, 8 pathologies, and 4 special tokens, which accounts for only 39 entries; please clarify what the remaining entries are (e.g., sub-token splits such as 'bibasilar' or 'costophrenic').
  4. [Eq. (3), Section 3.3] Please state the normalization of the Gaussian T (e.g., peak value 1 versus unit integral) and specify that the mask 1_{T>epsilon} is applied elementwise; with epsilon = 1e-5 the mask is close to all-ones for large anatomical structures, which is relevant for interpreting the target.
  5. [Appendix B] The synthetic VinDr-CXR prompts are generated by matching ground-truth boxes to the same lookup table that provides the supervision signals, so the OOD evaluation only exercises the 27 LUT locations; a sentence acknowledging this would clarify the scope of the OOD claim.
  6. [Section 4.5, Table 1] The results correspond to a single run; for classes with small gaps (e.g., Cardiomegaly CNR 1.13 for the fine-tuned LDM versus 1.16 for the frozen LDM on MS-CXR-loc), it would be useful to report run-to-run variability or at least acknowledge explicitly that the fine-tuned model does not improve every class.
  7. [Table 3] The absence of an Edema row in Table 3 is only explained in the Appendix C caption; please note this in the main text or in the Table 3 caption as well.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the Gaussian supervision and evaluation benchmarks are external, and the one self-citation plus the self-referential loss term are not load-bearing.

full rationale

The claimed derivation chain is not circular. The spatial supervision signal is an external 2D Gaussian lookup table computed from Chest Imagenome bounding-box annotations: 'those parameters are calculated from the gold standard subset of the Chest Imagenome (Wu et al., 2021) dataset that contains 1,000 images with bounding box annotations per anatomical location' (Section 3.2), and the SOTA/robustness claims are measured against external benchmarks, MS-CXR-loc and VinDr-CXR, with ground-truth boxes. The fine-tuning objective in Equation 3 does mix the model's own detached activations sg(A_sp) into the target, so part of the learning signal is self-referential; however, the Gaussian mask and the alpha*T regularizer are the externally grounded components, and the paper's own ablation (Table 2: alpha=0 gives 1.30/28.5, alpha=1 gives 1.18/24.6, alpha=0.1 gives 1.37/29.5) shows the external term is not decorative. The only self-citation, (Vilouras et al., 2024), supplies the heatmap-extraction protocol used identically for the frozen and fine-tuned LDM baselines; it is a controlled evaluation tool, not an unverified uniqueness theorem or a premise that forces the result. A remaining validity concern is that the fine-tuning set and MS-CXR-loc are both drawn from MIMIC-CXR, and the paper does not report their overlap; this is a potential train/test leakage risk, but it is not a circularity that can be established from the paper's own equations because the paper does not state that any evaluation image was used for fine-tuning.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the assumption that cross-attention maps are a valid localization signal, that RadGraph-XL reliably extracts anatomy mentions, and that the fixed Gaussian lookup table from Chest Imagenome is a faithful spatial prior. No new physical entities are introduced. The free parameters include per-location Gaussian statistics fitted to Chest Imagenome and the α/ε hyperparameters in the loss, plus the hand-selected timestep and layer ranges.

free parameters (4)
  • Per-location 2D Gaussian parameters (µx, µy, σx, σy) for 27 anatomical locations = Values derived from Chest Imagenome bounding boxes (not enumerated in text)
    Fitted to Chest Imagenome gold-standard boxes; define the weak supervision target for each location.
  • α (regularizer weight in Eq. 3) = 0.1
    Chosen by ablation (Table 2); balances the Gaussian regularizer against the masked self-activation term.
  • ε (mask threshold in Eq. 3) = 1e-5
    Hand-set threshold to define the binary mask over the Gaussian support.
  • Diffusion timestep range and selected cross-attention layers = [30,60], layers: bottleneck + first 3 decoder
    Hand-selected extraction range; affects what the supervision loss sees.
assumptions (4)
  • domain assumption Cross-attention activations of the pre-trained LDM are a meaningful proxy for phrase grounding localization.
    The method and the heatmap extraction rely on cross-attention as the localization signal (Section 3.3, equations 1-4); if this proxy is invalid, the supervision is moot.
  • domain assumption The fixed LUT Gaussians derived from Chest Imagenome are valid spatial priors for each location across all images.
    Section 3.2 and Appendix A; e.g., 'lingular' is mapped to 'left mid lung zone', and 'base' to 'lower lung zone'.
  • domain assumption RadGraph-XL's ANAT-DP predictions are reliable for extracting location mentions.
    Section 3.2 and Limitations; the authors note false negatives (e.g., 'right pneumothorax') but no false positives.
  • domain assumption The standardized '{location} {pathology}' prompt format preserves enough information for grounding.
    Section 4.6 and Appendix D; they show average metrics are stable across formats, but some classes fluctuate.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Anatomy-Grounded Weakly Supervised Prompt Tuning for Chest X-ray Latent Diffusion Models." pith.science (2026). https://pith.science/paper/QIPYA7CJ

@misc{pith2026250610633,
  author       = {Pith},
  title        = {Pith review of: Anatomy-Grounded Weakly Supervised Prompt Tuning for Chest X-ray Latent Diffusion Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QIPYA7CJ}},
  note         = {Machine review of arXiv:2506.10633}
}
read the original abstract

Latent Diffusion Models have shown remarkable results in text-guided image synthesis in recent years. In the domain of natural (RGB) images, recent works have shown that such models can be adapted to various vision-language downstream tasks with little to no supervision involved. On the contrary, text-to-image Latent Diffusion Models remain relatively underexplored in the field of medical imaging, primarily due to limited data availability (e.g., due to privacy concerns). In this work, focusing on the chest X-ray modality, we first demonstrate that a standard text-conditioned Latent Diffusion Model has not learned to align clinically relevant information in free-text radiology reports with the corresponding areas of the given scan. Then, to alleviate this issue, we propose a fine-tuning framework to improve multi-modal alignment in a pre-trained model such that it can be efficiently repurposed for downstream tasks such as phrase grounding. Our method sets a new state-of-the-art on a standard benchmark dataset (MS-CXR), while also exhibiting robust performance on out-of-distribution data (VinDr-CXR). Our code will be made publicly available.

Figures

Figures reproduced from arXiv: 2506.10633 by the authors.

Figure 1
Figure 1. Cross-attention leakage in a pre-trained LDM. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of our proposed data curation process. ( [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Mapping from anatomical locations to 2D Gaussians based on Chest Imagenome (Wu et al., 2021) gold standard annotations. For clarity, we overlay the Gaussians on top of a randomly selected CXR from the same dataset. Note that we use these targets during our proposed fine-tuning method. including location terms, pathology terms and spe￾cial tokens (<BoS>, <EoS>, <pad>, and the <and> token which is used for binding), a… view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Impact of prompt format on phrase grounding [PITH_FULL_IMAGE:figures/full_fig_p012_4.png]
Figure 5
Figure 5. Figure 5: Statistics of the fine-tuning set. For each pathology label, we present the number of samples per anatomical [PITH_FULL_IMAGE:figures/full_fig_p013_5.png]
Figure 6
Figure 6. Figure 6: Un-normalized cross-attention visualizations for randomly selected samples from the MS-CXR-loc dataset. [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

34 extracted references · 22 canonical work pages

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Shruthi Bannur, Kenza Bouzid, Daniel C Castro, Anton Schwaighofer, Anja Thieme, Sam Bond-Taylor, Maximilian Ilse, Fernando P \'e rez-Garc \' a, Valentina Salvatelli, Harshita Sharma, and 1 others. 2024. Maira-2: Grounded radiology report generation. arXiv preprint arXiv:2406.04449

    arXiv 2024

  4. [4]

    Shruthi Bannur, Stephanie Hyland, Qianchu Liu, Fernando Perez-Garcia, Maximilian Ilse, Daniel C Castro, Benedikt Boecking, Harshita Sharma, Kenza Bouzid, Anja Thieme, and 1 others. 2023. Learning to exploit temporal structure for biomedical vision-language processing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pa...

    work page 2023

  5. [5]

    Benedikt Boecking, Naoto Usuyama, Shruthi Bannur, Daniel C Castro, Anton Schwaighofer, Stephanie Hyland, Maria Wetscherek, Tristan Naumann, Aditya Nori, Javier Alvarez-Valle, and 1 others. 2022. Making the most of text semantics to improve biomedical vision--language processing. In European conference on computer vision, pages 1--21. Springer

    work page 2022

  6. [6]

    Pierre Chambon, Christian Bluethgen, Curtis P Langlotz, and Akshay Chaudhari. 2022. Adapting pretrained vision-language foundational models to medical imaging domains. arXiv preprint arXiv:2210.04133

    arXiv 2022

  7. [7]

    Ming Dai, Lingfeng Yang, Yihao Xu, Zhenhua Feng, and Wankou Yang. 2024. Simvg: A simple framework for visual grounding with decoupled multi-modal fusion. Advances in neural information processing systems, 37:121670--121698

    work page 2024

  8. [8]

    Bram De Wilde, Anindo Saha, Maarten de Rooij, Henkjan Huisman, and Geert Litjens. 2023. Medical diffusion on a budget: textual inversion for medical image generation. arXiv preprint arXiv:2303.13430

    arXiv 2023

Show all 34 references
  1. [9]

    Jean-Benoit Delbrouck, Pierre Chambon, Christian Bluethgen, Emily Tsai, Omar Almusa, and Curtis Langlotz. 2022. https://doi.org/10.18653/v1/2022.findings-emnlp.319 Improving the factual correctness of radiology report generation with semantic rewards . In Findings of the Assoc...

    2022 doi

  2. [10]

    Jean-Benoit Delbrouck, Pierre Chambon, Zhihong Chen, Maya Varma, Andrew Johnston, Louis Blankemeier, Dave Van Veen, Tan Bui, Steven Truong, and Curtis Langlotz. 2024. https://doi.org/10.18653/v1/2024.findings-acl.765 R ad G raph- XL : A large-scale expert-annotated dataset for...

    2024 doi

  3. [11]

    Mischa Dombrowski, Hadrien Reynaud, Johanna P M \"u ller, Matthew Baugh, and Bernhard Kainz. 2024. Trade-offs in fine-tuned diffusion models between accuracy and interpretability. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 21037--21045

    2024

  4. [12]

    Gian Mario Favero, Parham Saremi, Emily Kaczmarek, Brennan Nichyporuk, and Tal Arbel. 2025. Conditional diffusion models are medical image classifiers that provide explainability and uncertainty for free. arXiv preprint arXiv:2502.03687

    2025 arXiv

  5. [13]

    Yu Gu, Jianwei Yang, Naoto Usuyama, Chunyuan Li, Sheng Zhang, Matthew P Lungren, Jianfeng Gao, and Hoifung Poon. 2023. Biomedjourney: Counterfactual biomedical image generation by instruction-learning from multimodal patient journeys. arXiv preprint arXiv:2310.10765

    2023 arXiv

  6. [14]

    Eric Hedlin, Gopal Sharma, Shweta Mahajan, Xingzhe He, Hossam Isack, Abhishek Kar, Helge Rhodin, Andrea Tagliasacchi, and Kwang Moo Yi. 2024. Unsupervised keypoints from pretrained diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog...

    2024

  7. [15]

    Shih-Cheng Huang, Liyue Shen, Matthew P Lungren, and Serena Yeung. 2021. Gloria: A multimodal global-local representation learning framework for label-efficient medical image recognition. In Proceedings of the IEEE/CVF international conference on computer vision, pages 3942--3951

    2021

  8. [16]

    Saahil Jain, Ashwin Agrawal, Adriel Saporta, Steven QH Truong, Du Nguyen Duong, Tan Bui, Pierre Chambon, Yuhao Zhang, Matthew P Lungren, Andrew Y Ng, and 1 others. 2021. Radgraph: Extracting clinical entities and relations from radiology reports. arXiv preprint arXiv:2106.14463

    2021 arXiv

  9. [17]

    Alistair Johnson, Matt Lungren, Yifan Peng, Zhiyong Lu, Roger Mark, Seth Berkowitz, and Steven Horng. 2019. Mimic-cxr-jpg-chest radiographs with structured labels. PhysioNet, 101:215--220

    2019

  10. [18]

    Amirhossein Kazerouni, Ehsan Khodapanah Aghdam, Moein Heidari, Reza Azad, Mohsen Fayyaz, Ilker Hacihaliloglu, and Dorit Merhof. 2023. Diffusion models in medical imaging: A comprehensive survey. Medical image analysis, 88:102846

    2023

  11. [19]

    Ira Ktena, Olivia Wiles, Isabela Albuquerque, Sylvestre-Alvise Rebuffi, Ryutaro Tanno, Abhijit Guha Roy, Shekoofeh Azizi, Danielle Belgrave, Pushmeet Kohli, Taylan Cemgil, and 1 others. 2024. Generative models improve fairness of medical classifiers under distribution shifts. ...

    2024

  12. [20]

    Grace Luo, Lisa Dunlap, Dong Huk Park, Aleksander Holynski, and Trevor Darrell. 2023. Diffusion hyperfeatures: Searching through time and space for semantic correspondence. Advances in Neural Information Processing Systems, 36:47500--47510

    2023

  13. [21]

    Chong Ma, Hanqi Jiang, Wenting Chen, Yiwei Li, Zihao Wu, Xiaowei Yu, Zhengliang Liu, Lei Guo, Dajiang Zhu, Tuo Zhang, and 1 others. 2024. Eye-gaze guided multi-modal alignment for medical representation learning. Advances in Neural Information Processing Systems, 37:6126--6153

    2024

  14. [22]

    Jered McInerney, Geoffrey Young, Jan-Willem van de Meent, and Byron Wallace. 2022. https://doi.org/10.18653/v1/2022.emnlp-main.238 That`s the wrong lung! evaluating and improving the interpretability of unsupervised multimodal encoders for medical data . In Proceedings of the ...

    2022 doi

  15. [23]

    Ron Mokady, Amir Hertz, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. 2023. Null-text inversion for editing real images using guided diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 6038--6047

    2023

  16. [24]

    Ha Q Nguyen, Khanh Lam, Linh T Le, Hieu H Pham, Dat Q Tran, Dung B Nguyen, Dung D Le, Chi M Pham, Hang TT Tong, Diep H Dinh, and 1 others. 2022. Vindr-cxr: An open dataset of chest x-rays with radiologist’s annotations. Scientific Data, 9(1):429

    2022

  17. [25]

    Fernando P \'e rez-Garc \' a, Sam Bond-Taylor, Pedro P Sanchez, Boris van Breugel, Daniel C Castro, Harshita Sharma, Valentina Salvatelli, Maria TA Wetscherek, Hannah Richardson, Matthew P Lungren, and 1 others. 2024. Radedit: stress-testing biomedical vision models via diffus...

    2024

  18. [26]

    Walter HL Pinaya, Petru-Daniel Tudosiu, Jessica Dafflon, Pedro F Da Costa, Virginia Fernandez, Parashkev Nachev, Sebastien Ourselin, and M Jorge Cardoso. 2022. Brain imaging generation with latent diffusion models. In MICCAI Workshop on Deep Generative Models, pages 117--126. Springer

    2022

  19. [27]

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj \"o rn Ommer. 2022. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684--10695

    2022

  20. [28]

    Maya Varma, Jean-Benoit Delbrouck, Sarah Hooper, Akshay Chaudhari, and Curtis Langlotz. 2023. Villa: Fine-grained vision-language representation learning from real-world data. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 22225--22235

    2023

  21. [29]

    Konstantinos Vilouras, Pedro Sanchez, Alison Q O'Neil, and Sotirios A Tsaftaris. 2024. Zero-shot medical phrase grounding with off-the-shelf diffusion models. IEEE Journal of Biomedical and Health Informatics

    2024

  22. [30]

    Tobias Weber, Michael Ingrisch, Bernd Bischl, and David R \"u gamer. 2023. Cascaded latent diffusion models for high-resolution chest x-ray synthesis. In Pacific-Asia conference on knowledge discovery and data mining, pages 180--191. Springer

    2023

  23. [31]

    Chaoyi Wu, Xiaoman Zhang, Ya Zhang, Yanfeng Wang, and Weidi Xie. 2023. Medklip: Medical knowledge enhanced language-image pre-training for x-ray diagnosis. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 21372--21383

    2023

  24. [32]

    Joy Wu, Nkechinyere Agu, Ismini Lourentzou, Arjun Sharma, Joseph Paguio, Jasper Seth Yao, Edward Christopher Dee, William Mitchell, Satyananda Kashyap, Andrea Giovannini, and 1 others. 2021. Chest imagenome dataset. Physio Net

    2021

  25. [33]

    Feiyang Yu, Mark Endo, Rayan Krishnan, Ian Pan, Andy Tsai, Eduardo Pontes Reis, Eduardo Kaiser Ururahy Nunes Fonseca, Henrique Min Ho Lee, Zahra Shakeri Hossein Abad, Andrew Y Ng, and 1 others. 2023. Evaluating progress in automatic chest x-ray radiology report generation. Pat...

    2023

  26. [34]

    Ke Yu, Shantanu Ghosh, Zhexiong Liu, Christopher Deible, and Kayhan Batmanghelich. 2022. Anatomy-guided weakly-supervised abnormality localization in chest x-rays. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 658--668. Springer

    2022

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.