RECORD
Chest-Diffusion: A Light-Weight Text-to-Image Model for Report-to-CXR Generation
Computer Vision and Pattern Recognition · Submitted 30 Jun 2024
abstract compiled · 2026-09-23 · no language-model review
The authors' main claim Experiments demonstrate that our Chest-Diffusion achieves the lowest FID score 24.456, under the computation budget of 118.918 GFLOPs, which is nearly one-third of the computational complexity of SD.
The compiled record
The paper's own sentences, arranged. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. Numbers in the tables are the authors', checked by code. No sentence in this record was written by a model.
Open the compiled record
The abstract, sentence by sentence
Text-to-image generation has important implications for generation of diverse and controllable images. Several attempts have been made to adapt Stable Diffusion (SD) to the medical domain. However, the large distribution difference between medical reports and natural texts, as well as high computational complexity in common stable diffusion limit the authenticity and feasibility of the generated medical images. To solve above problems, we propose a novel light-weight transformer-based diffusion model learning framework, Chest-Diffusion, for report-to-CXR generation. Chest-Diffusion employs a domain-specific text encoder to obtain accurate and expressive text features to guide image generation, improving the authenticity of the generated images. Meanwhile, we introduce a light-weight transformer architecture as the denoising model, reducing the computational complexity of the diffusion model. Experiments demonstrate that our Chest-Diffusion achieves the lowest FID score 24.456, under the computation budget of 118.918 GFLOPs, which is nearly one-third of the computational complexity of SD.
background contribution method result interpretation main claim
| # | sentence | role | the authors' own work | states the main conclusion | strength |
|---|---|---|---|---|---|
| s0 | Text-to-image generation has important implications for generation of diverse and controllable images. | background 1.00 | 0.04 | 0.02 | asserted 1.00 |
| s1 | Several attempts have been made to adapt Stable Diffusion (SD) to the medical domain. | background 1.00 | 0.04 | 0.02 | asserted 1.00 |
| s2 | However, the large distribution difference between medical reports and natural texts, as well as high computational complexity in common stable diffusion limit the authenticity and feasibility of the generated medical images. | gap 0.55 | 0.08 | 0.04 | asserted 1.00 |
| s3 | To solve above problems, we propose a novel light-weight transformer-based diffusion model learning framework, Chest-Diffusion, for report-to-CXR generation. | contribution 0.98 | 0.82 | 0.17 | asserted 0.99 |
| s4 | Chest-Diffusion employs a domain-specific text encoder to obtain accurate and expressive text features to guide image generation, improving the authenticity of the generated images. | method 0.94 | 0.85 | 0.08 | asserted 1.00 |
| s5 | Meanwhile, we introduce a light-weight transformer architecture as the denoising model, reducing the computational complexity of the diffusion model. | method 0.89 | 0.91 | 0.07 | asserted 1.00 |
| s6 | Experiments demonstrate that our Chest-Diffusion achieves the lowest FID score 24.456, under the computation budget of 118.918 GFLOPs, which is nearly one-third of the computational complexity of SD. | result 1.00 | 0.96 | 0.66 | asserted 1.00 |
What was examined, and what was not
- arXiv listing abstract v1
- full text
- figures and tables
- references and citing sentences
- relations to other papers
- P abstract
- S sections
- T body sentences
- C citations
- R cross-paper relations
The paper
Original abstract
Text-to-image generation has important implications for generation of diverse and controllable images. Several attempts have been made to adapt Stable Diffusion (SD) to the medical domain. However, the large distribution difference between medical reports and natural texts, as well as high computational complexity in common stable diffusion limit the authenticity and feasibility of the generated medical images. To solve above problems, we propose a novel light-weight transformer-based diffusion model learning framework, Chest-Diffusion, for report-to-CXR generation. Chest-Diffusion employs a domain-specific text encoder to obtain accurate and expressive text features to guide image generation, improving the authenticity of the generated images. Meanwhile, we introduce a light-weight transformer architecture as the denoising model, reducing the computational complexity of the diffusion model. Experiments demonstrate that our Chest-Diffusion achieves the lowest FID score 24.456, under the computation budget of 118.918 GFLOPs, which is nearly one-third of the computational complexity of SD.
Human review
How the literature uses it
1 arXiv paper cite this in 2 sentences
Papers citing it · 1 sentence without a confident label
Cites it as background 1 paper
-
Recent advancements have extended these capabilities to the domain of synthetic text-to-medical image generation (T2MedI) [ref] .
Fair Text to Medical Image Diffusion Model with Subgroup Distribution Aligned Tuning · Introduction · 0.83
What it cites · 2 sentences without a confident label
Uses the method or tool of 1 paper
-
To obtain the informative features of medical reports to facilitate the generation of realistic and diverse CXRs from reports, we employ the domain-specific CLIP model, BiomedCLIP [ref] , as our text encoder.
BiomedCLIP: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairs · Chest-Diffusion · 0.99
Uses data or values from 1 paper
-
To develop Chest Diffusion, we employed the publicly available dataset, MIMIC-CXR [ref] , provided by the Beth Israel Deaconess Medical Center.
MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographs · Dataset and evaluation · 1.00
Compares with 1 paper
-
We also compare it with the current SOTA autoregressive report-to-CXR model LLM-CXR [ref] .
LLM-CXR: Instruction-Finetuned LLM for CXR Image Understanding and Generation · Comparison with the State-of-the-Art Method · 1.00
Cites as background 3 papers
-
An emerging and prominent method is the diffusion model [ref] - [ref] , which uses a Markov chain process to denoise images by modeling them with Gaussian distributions [ref] .
GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models · Introduction · 0.91
-
RoentGen [ref] thoroughly explores the finetuning strategies of SD, which keeps the autoencoder frozen and finetunes the denoising model as well as the text encoder of the pre-trained SD.
RoentGen: Vision-Language Foundation Model for Chest X-ray Generation · Introduction · 0.92
-
Chambon et al. [ref] investigates whether SD could be adapted to the medical domain in a few-shot manner.
Adapting Pretrained Vision-Language Foundational Models to Medical Imaging Domains · Introduction · 0.90
Discussion
Know something about this paper? Leave a comment, or highlight any passage above to comment on it.