Pith. sign in

REVIEW 4 major objections 6 minor 1 cited by

Ultrasound Image Generation using Latent Diffusion Models

T0 review · 4 major / 6 minor · reviewed 2026-08-08 · deepseek-v4-flash

Pith's one-line read This paper claims that fine-tuning Stable Diffusion on 780 breast ultrasound images produces synthetic images that look realistic to experts and improve an external classifier from 81% to 87% AUC.

desk verdict Plausible feasibility demo of Stable Diffusion fine-tuning for breast ultrasound, but the one quantitative claim (81% to 87% AUC) is under-specified and should not be treated as evidence of utility yet. read the letter →

arxiv 2502.08580 v1 pith:KKSNBMMI submitted 2025-02-12 cs.CV

classification cs.CV
keywords latentdiffusionmodelsultrasoundimagegenerationbreastStablefine-tuningControlNetsyntheticdataaugmentationBUSIdataset
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that a large pre-trained latent diffusion model (Stable Diffusion v1.5) can be adapted to a small, publicly available breast ultrasound dataset (BUSI, 780 images) and produce synthetic ultrasound images that are realistic and category-controlled. The authors report that the generated images looked realistic to three experienced ultrasound scientists and a radiologist, and that adding some generated images to the training set improved a ResNet-50 classifier's area under the ROC curve from 81% to 87% on an external breast imaging dataset (B25). They also show that conditioning with ControlNet on segmentation masks lets a user generate images that respect a specified lesion shape. If the claims hold, a large text-to-image model can serve as a practical generative prior for medical ultrasound, producing training data for classification and segmentation without physics-based simulation.

What carries the argument

The central object is Stable Diffusion, a latent diffusion model: a variational autoencoder compresses 512x512 images to 64x64 latents, a U-Net learns to denoise these latents, and CLIP text embeddings guide the denoising. The paper fine-tunes this pre-trained model on BUSI, a small dataset of 780 images, which is feasible in a few hours on a single A100 GPU. For spatial control, the paper adds ControlNet, a trainable copy of the network whose outputs are added to the frozen decoder through zero-convolution layers, trained with the segmentation masks from BUSI as the conditioning input. This combination lets the pre-trained generative prior be redirected toward ultrasound while retaining the ability to follow a text prompt and, with ControlNet, a segmentation mask.

What would settle it

Re-running the augmentation experiment with a fully documented protocol, fixed number and selection of generated images, the ResNet-50 recipe from [24], and five random seeds, then reporting per-seed AUC with confidence intervals for the difference between augmented and non-augmented classifiers on B25 would settle the claim; if the difference is not significant, the utility claim as currently stated would be refuted.

Watch

Extended reading notes

Core claim

On its own terms, the paper claims that Stable Diffusion v1.5, fine-tuned on the BUSI dataset, can generate high-quality breast ultrasound images from simple text prompts such as "Ultrasound image of a normal/benign/malignant breast", with textures (skin, fat, glandular tissue, muscle, ribs) close to real ultrasound and with benign lesions appearing as round black cysts while malignant lesions are irregular. A quantitative evaluation with a ResNet-50 trained as in [24] found that augmenting the training data with some generated images raised AUC from 81% to 87% when tested on B25, a different breast imaging dataset; the authors interpret this as evidence that the generated samples are useful for downstream classification. Conditioning the fine-tuned generator through ControlNet with segmentation masks produced multiple plausible images that match the input mask, giving the user control over the lesion shape and location.

Load-bearing premise

The entire utility claim rests on the assumption that the reported 81% to 87% AUC improvement is real and not an artifact of an incomplete or biased evaluation; if that protocol was flawed, the central claim that generated images improve classification collapses.

Editorial extensions

If this is right

  • A foundation text-to-image model can be steered into a medical imaging domain with a few hundred images and modest compute, without physics-based ultrasound simulation.
  • Synthetic breast ultrasound images can be used to enlarge training sets, and the reported external-dataset AUC gain (81% to 87%) suggests they can improve generalization rather than merely memorizing the training distribution.
  • Segmentation-mask conditioning makes it possible to generate many plausible ultrasound images for a given lesion shape, which could support studies of shape-dependent classifier behavior and segmentation model training.
  • Releasing the code gives other groups a reproducible baseline for ultrasound generation with latent diffusion models, lowering the barrier to applying the same recipe to other organs or pathologies.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct extension the paper does not run is a diversity check: computing pairwise similarity or a perceptual distance among generated images, and between generated and BUSI images, would show whether the classifier gain comes from genuine variation or from near-duplicates of common BUSI patterns.
  • Repeating the 81% to 87% AUC comparison with a fully specified protocol and confidence intervals would convert the reported gain from a single number into an established effect size.
  • Because the conditioning is mask-based, the same pipeline could synthesize lesion shapes drawn from another distribution, such as physics-based simulations or clinical sketches, blending generative realism with controlled anatomy; the paper mentions this direction only as future work.
  • The same fine-tuning recipe likely transfers to other ultrasound domains with small public datasets, such as cardiac or lung ultrasound, but this transfer is not tested in the paper.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes fine-tuning Stable Diffusion v1.5 on the BUSI breast ultrasound dataset to generate realistic ultrasound images from text prompts, and extends this with ControlNet to condition generation on segmentation masks. The authors report that the generated images appeared realistic to three experienced ultrasound scientists and a radiologist, and they provide two quantitative results: (1) augmenting a training set with some generated images improved ResNet-50 AUC on a separate B25 dataset from 81% to 87%, and (2) a classifier trained on images generated by a model fine-tuned on 20% of BUSI achieved 94% AUC on the generator's training set. The paper argues this demonstrates the feasibility of adapting large latent diffusion models to small medical imaging datasets with modest computational cost.

Significance. If the quantitative claims are substantiated, this work would offer a practical and timely contribution: a pipeline for generating synthetic ultrasound images to augment small datasets, with user control through segmentation masks. The approach of fine-tuning a large pre-trained latent diffusion model rather than training from scratch is sensible, and the reported single-GPU fine-tuning cost is attractive. The qualitative examples are plausible and suggest the method captures key ultrasound texture and lesion morphology. However, the central utility claim currently rests on an under-specified evaluation and a self-referential accuracy figure, so the significance is conditional on additional evidence.

major comments (4)
  1. [Section 3.1.2] The 81% to 87% AUC improvement on B25 is the central quantitative claim, but the evaluation protocol is under-specified. The paper does not report the size and class balance of B25, the preprocessing applied, the number of generated images added, the criteria behind 'some of the generated images,' the number of training seeds, or confidence intervals/statistical tests. Without these details, the reader cannot assess whether the 6-point AUC gain is statistically meaningful or reproducible. Please provide a complete experimental protocol or explicitly label this as a preliminary observation.
  2. [Section 3.1.2] The improvement from 81% to 87% AUC is not attributed to the synthetic images specifically because no matched-size control is reported. Adding an equivalent number of real BUSI images (or even repeated real images) to the training set might produce a similar gain. Please include a control experiment with real-image augmentation of the same size, and report per-seed variance to distinguish the effect of synthetic images from the effect of simply increasing training data size.
  3. [Section 3.1.2] The 94% AUC result is self-referential: the classifier is tested on the exact subset of BUSI used to fine-tune the generator. High accuracy on this set mainly demonstrates that the generator has memorized or closely reproduced its training distribution, not that the generated images generalize to unseen data. Please remove this result or re-frame it as an internal consistency check, and evaluate on a held-out subset or an external dataset if generalization is intended.
  4. [Section 3.1.1] The realism claim rests on informal expert opinion, but no evaluation methodology is described. Please report how many images were assessed, whether the experts were blinded to the source (generated vs. real), what instructions they received, and whether there was any inter-rater agreement metric. A forced-choice or Turing-style test against real BUSI images would substantially strengthen this claim.
minor comments (6)
  1. [Section 2.1] The term 'U-NET' should be 'U-Net' for consistency with the literature, and 'VAE' is used without expanding the abbreviation at first use.
  2. [Section 3.1.2] The B25 dataset is not defined; please add a citation and describe how it was obtained, its size, and its preprocessing.
  3. [Section 3.1.2] The number of generated images used for augmentation is not stated; please specify the total and per-class counts, as well as the selection procedure.
  4. [Section 3.1.3] The sentence ending with 'The Figure' at the end of Section 3.1.3 appears incomplete; please proofread and complete the reference to Figure 4.
  5. [Section 2.1] The paper mentions fine-tuning 'in a few hours using a single A100 GPU' but does not report exact training time, batch size, learning rate, or number of steps; adding these would aid reproducibility.
  6. [Abstract] The code release URL (http://code.sonography.ai/) is mentioned but no repository link or license is provided; please include an active and permanent link.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; the B25 benchmark is independent and the 94% self-check is not used as external evidence.

full rationale

The central quantitative claim—augmenting training with Stable Diffusion-generated breast ultrasound images improves ResNet-50 AUC on B25 from 81% to 87%—rests on an external dataset (B25) that is not used to fine-tune or fit the generator, so the evaluation is not circular by construction. The 94% AUC result is explicitly described as predicting the generative model's own training set; it is a self-referential sanity check and is not used as evidence of external generalization, so it does not form part of a circular derivation chain. The only self-citation, reference [24], is used to specify the ResNet-50 training setup, not to justify the validity of the generated images or the B25 improvement, and is therefore not load-bearing. The under-specified B25 protocol (missing test-set size, preprocessing, number of generated images, and error bars) is a correctness and reproducibility limitation, not a circularity. No equation or parameter in the paper is defined in terms of the quantity it supposedly predicts.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The paper has no explicit mathematical derivation and introduces no new entities. The central claims rest on empirical evaluation, so the ledger records the hand-picked data split and the domain assumptions about transferability, dataset representativeness, expert assessment, and external evaluation.

free parameters (1)
  • BUSI training subset for the single-classifier experiment = 20% of BUSI
    Section 3.1.2 reports 94% AUC when a ResNet-50 is trained on generated images from a Stable Diffusion model fine-tuned on only 20% of BUSI; the choice of 20% is arbitrary, not justified, and directly affects the reported 94% AUC.
assumptions (4)
  • domain assumption Stable Diffusion v1.5 and ControlNet pre-trained weights are appropriate foundation models for ultrasound image generation
    The method builds on the premise that features learned on natural images transfer to ultrasound; transferability is not validated in the paper beyond qualitative inspection.
  • domain assumption BUSI dataset is representative of breast ultrasound images
    Fine-tuning on 780 BUSI images is assumed to yield a model whose outputs generalize to other datasets, e.g., B25.
  • domain assumption Visual assessment by three experienced US scientists and a US radiologist is a reliable measure of realism
    No structured scoring protocol, inter-rater agreement, or blinded comparison is reported; realism is asserted from informal expert opinion.
  • domain assumption The B25 dataset and the ResNet-50 training setup from reference [24] are correctly applied in the external evaluation
    The quantitative improvement claim relies on an external test whose preprocessing and split details are not reported in this paper.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Ultrasound Image Generation using Latent Diffusion Models." pith.science (2026). https://pith.science/paper/KKSNBMMI

@misc{pith2026250208580,
  author       = {Pith},
  title        = {Pith review of: Ultrasound Image Generation using Latent Diffusion Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KKSNBMMI}},
  note         = {Machine review of arXiv:2502.08580}
}
read the original abstract

Diffusion models for image generation have been a subject of increasing interest due to their ability to generate diverse, high-quality images. Image generation has immense potential in medical imaging because open-source medical images are difficult to obtain compared to natural images, especially for rare conditions. The generated images can be used later to train classification and segmentation models. In this paper, we propose simulating realistic ultrasound (US) images by successive fine-tuning of large diffusion models on different publicly available databases. To do so, we fine-tuned Stable Diffusion, a state-of-the-art latent diffusion model, on BUSI (Breast US Images) an ultrasound breast image dataset. We successfully generated high-quality US images of the breast using simple prompts that specify the organ and pathology, which appeared realistic to three experienced US scientists and a US radiologist. Additionally, we provided user control by conditioning the model with segmentations through ControlNet. We will release the source code at http://code.sonography.ai/ to allow fast US image generation to the scientific community.

Figures

Figures reproduced from arXiv: 2502.08580 by the authors.

Figure 1
Figure 1. Current results of two models for image generation: (first row) Stable Diffusion 1.5 (second row) ChatGPT4o. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Demonstration of the main principles of Stable Diffusion and ControlNet. The ControlNet is a copy of the [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Some results of our fine-tuning on BUSI images. Each row represents a category, (a) normal, (b) benign, (c) [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Samples conditioned by segmentation masks, generated with ControlNet on BUSI. Each row represents a [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Breast Ultrasound Tumor Generation via Mask Generator and Text-Guided Network:A Clinically Controllable Framework with Downstream Evaluation

    eess.IV 2025-07 conditional novelty 5.0 of 10

    A text-and-mask guided diffusion framework generates realistic synthetic breast ultrasound tumors, and augmenting real training data with these images improves downstream classification and segmentation across six pub...

Reference graph

Works this paper leans on

30 extracted references · 27 canonical work pages · cited by 1 Pith paper

  1. [1]

    High-resolution image synthesis with latent diffusion models,

    Rombach, R., Blattmann, A., Lorenz, D., Esser, P., and Ommer, B., “High-resolution image synthesis with latent diffusion models,” in [ Proceedings of the IEEE/CVF conference on computer vision and pattern recognition], 10684–10695 (2022)

  2. [2]

    Dataset of breast ultrasound images,

    Al-Dhabyani, W., Gomaa, M., Khaled, H., and Fahmy, A., “Dataset of breast ultrasound images,” Data in brief 28, 104863 (2020)

  3. [3]

    Adding conditional control to text-to-image diffusion models,

    Zhang, L., Rao, A., and Agrawala, M., “Adding conditional control to text-to-image diffusion models,” in [Proceedings of the IEEE/CVF International Conference on Computer Vision], 3836–3847 (2023)

  4. [4]

    Synthetically enhanced: unveiling synthetic data’s potential in medical imaging research,

    Khosravi, B., Li, F., Dapamede, T., Rouzrokh, P., Gamble, C. U., Trivedi, H. M., Wyles, C. C., Sellergren, A. B., Purkayastha, S., Erickson, B. J., et al., “Synthetically enhanced: unveiling synthetic data’s potential in medical imaging research,” EBioMedicine 104 (2024)

  5. [5]

    Augmenting medical image classifiers with synthetic data from latent diffusion models,

    Sagers, L. W., Diao, J. A., Melas-Kyriazi, L., Groh, M., Rajpurkar, P., Adamson, A. S., Rotemberg, V., Daneshjou, R., and Manrai, A. K., “Augmenting medical image classifiers with synthetic data from latent diffusion models,” arXiv preprint arXiv:2308.12453(2023)

  6. [6]

    Make the most of must, an open-source matlab ultrasound toolbox,

    Garcia, D., “Make the most of must, an open-source matlab ultrasound toolbox,” in [ 2021 IEEE interna- tional ultrasonics symposium (IUS)], 1–4, IEEE (2021)

  7. [7]

    Field: A program for simulating ultrasound systems,

    Jensen, J. A., “Field: A program for simulating ultrasound systems,” Medical & Biological Engineering & Computing 34(sup. 1), 351–353 (1997)

  8. [8]

    k-wave: Matlab toolbox for the simulation and reconstruction of photoacoustic wave fields,

    Treeby, B. E. and Cox, B. T., “k-wave: Matlab toolbox for the simulation and reconstruction of photoacoustic wave fields,” Journal of biomedical optics15(2), 021314–021314 (2010)

Show all 30 references
  1. [9]

    Ultraray: Full-path ray tracing for enhancing realism in ultrasound simulation,

    Duelmer, F., Azampour, M. F., and Navab, N., “Ultraray: Full-path ray tracing for enhancing realism in ultrasound simulation,” arXiv preprint arXiv:2501.05828(2025)

  2. [10]

    Ultrasound segmentation using u-net: learning from simulated data and testing on real data,

    Behboodi, B. and Rivaz, H., “Ultrasound segmentation using u-net: learning from simulated data and testing on real data,” in [ 2019 41st annual international conference of the IEEE engineering in medicine and biology society (EMBC)], 6628–6631, IEEE (2019)

  3. [11]

    Diffusion as sound propagation: Physics- inspired model for ultrasound image generation,

    Dom ´ ınguez, M., Velikova, Y., Navab, N., and Azampour, M. F., “Diffusion as sound propagation: Physics- inspired model for ultrasound image generation,” in [International Conference on Medical Image Computing and Computer-Assisted Intervention], 613–623, Springer (2024)

  4. [12]

    Echo from noise: synthetic ultra- sound image generation using diffusion models for real image segmentation,

    Stojanovski, D., Hermida, U., Lamata, P., Beqiri, A., and Gomez, A., “Echo from noise: synthetic ultra- sound image generation using diffusion models for real image segmentation,” in [ International Workshop on Advances in Simplifying Medical Ultrasound], 34–43, Springer (2023)

  5. [13]

    Diffusion model-based data augmentation for lung ultrasound classification with limited data,

    Zhang, X., Gangopadhyay, A., Chang, H.-M., and Soni, R., “Diffusion model-based data augmentation for lung ultrasound classification with limited data,” in [ ML4H@NeurIPS ], (2023)

  6. [14]

    Ultrasound image synthesis using generative ai for lung ultrasound detection,

    Chou, Y.-C., Li, G. Y., Chen, L., Zahiri, M., Balaraju, N., Patil, S., Hicks, B., Schnittke, N., Kessler, D. O., Shupp, J., et al., “Ultrasound image synthesis using generative ai for lung ultrasound detection,” arXiv preprint arXiv:2501.06356 (2025)

  7. [15]

    Echonet-synthetic: Privacy-preserving video generation for safe medical data sharing,

    Reynaud, H., Meng, Q., Dombrowski, M., Ghosh, A., Day, T., Gomez, A., Leeson, P., and Kainz, B., “Echonet-synthetic: Privacy-preserving video generation for safe medical data sharing,” in [ International Conference on Medical Image Computing and Computer-Assisted Intervention]...

  8. [16]

    Ultrasound image enhancement with the variance of diffusion models,

    Zhang, Y., Huneau, C., Idier, J., and Mateus, D., “Ultrasound image enhancement with the variance of diffusion models,” in [ 2024 IEEE Ultrasonics, Ferroelectrics, and Frequency Control Joint Symposium (UFFC-JS)], 1–5, IEEE (2024)

  9. [17]

    Denoising plane wave ultrasound images using diffusion probabilistic models,

    Asgariandehkordi, H., Goudarzi, S., Sharifzadeh, M., Basarab, A., and Rivaz, H., “Denoising plane wave ultrasound images using diffusion probabilistic models,” IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control(2024)

  10. [18]

    Dehazing ultrasound using diffusion models,

    Stevens, T. S., Meral, F. C., Yu, J., Apostolakis, I. Z., Robert, J.-L., and Van Sloun, R. J., “Dehazing ultrasound using diffusion models,” IEEE Transactions on Medical Imaging(2024)

  11. [19]

    Deep ultrasound denoising using diffusion probabilistic models,

    Asgariandehkordi, H., Goudarzi, S., Basarab, A., and Rivaz, H., “Deep ultrasound denoising using diffusion probabilistic models,” in [ 2023 IEEE International Ultrasonics Symposium (IUS)], 1–4, IEEE (2023)

  12. [20]

    Denoising diffusion probabilistic models,

    Ho, J., Jain, A., and Abbeel, P., “Denoising diffusion probabilistic models,” Advances in neural information processing systems33, 6840–6851 (2020)

  13. [21]

    Diffusion models beat gans on image synthesis,

    Dhariwal, P. and Nichol, A., “Diffusion models beat gans on image synthesis,” Advances in neural informa- tion processing systems34, 8780–8794 (2021)

  14. [22]

    Learning transferable visual models from natural language supervision,

    Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al., “Learning transferable visual models from natural language supervision,” in [ International conference on machine learning], 8748–8763, PMLR (2021)

  15. [23]

    U-net: Convolutional networks for biomedical image segmen- tation,

    Ronneberger, O., Fischer, P., and Brox, T., “U-net: Convolutional networks for biomedical image segmen- tation,” in [Medical image computing and computer-assisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part III...

  16. [24]

    Vision mamba for classification of breast ultrasound images,

    Nasiri-Sarvi, A., Hosseini, M. S., and Rivaz, H., “Vision mamba for classification of breast ultrasound images,” MICCAI Deep Breath Workshop(2024)

  17. [25]

    Automated breast ultrasound lesions detection using convolutional neural networks,

    Yap, M. H., Pons, G., Marti, J., Ganau, S., Sentis, M., Zwiggelaar, R., Davison, A. K., and Marti, R., “Automated breast ultrasound lesions detection using convolutional neural networks,” IEEE journal of biomedical and health informatics22(4), 1218–1226 (2017)

  18. [26]

    Visual prompt tuning,

    Jia, M., Tang, L., Chen, B.-C., Cardie, C., Belongie, S., Hariharan, B., and Lim, S.-N., “Visual prompt tuning,” in [ European Conference on Computer Vision], 709–727, Springer (2022)

  19. [27]

    Segmentation of arm ultrasound images in breast cancer-related lymphedema: A database and deep learning algorithm,

    Goudarzi, S., Whyte, J., Boily, M., Towers, A., Kilgour, R. D., and Rivaz, H., “Segmentation of arm ultrasound images in breast cancer-related lymphedema: A database and deep learning algorithm,” IEEE Transactions on Biomedical Engineering70(9), 2552–2563 (2023)

  20. [28]

    Registration of pre-and postresection ultrasound volumes with noncorresponding regions in neurosurgery,

    Zhou, H. and Rivaz, H., “Registration of pre-and postresection ultrasound volumes with noncorresponding regions in neurosurgery,” IEEE journal of biomedical and health informatics20(5), 1240–1249 (2016)

  21. [29]

    Cinepro: Robust training of foundation models for cancer detection in prostate ultrasound cineloops,

    Harmanani, M., Jamzad, A., To, M. N. N., Wilson, P. F., Guo, Z., Fooladgar, F., Sojoudi, S., Gilany, M., Chang, S., Black, P., et al., “Cinepro: Robust training of foundation models for cancer detection in prostate ultrasound cineloops,” arXiv preprint arXiv:2501.12331(2025)

  22. [30]

    Active inference and deep generative modeling for cognitive ultrasound,

    Van Sloun, R. J., “Active inference and deep generative modeling for cognitive ultrasound,” IEEE Trans- actions on Ultrasonics, Ferroelectrics, and Frequency Control(2024)

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.