{"id":"2d500bb4-be75-48e7-b4f5-2cb4a1f3ad70","arxiv_id":"2506.23641","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"VAP-Diffusion generates medical images conditioned on MLLM-written visual attribute prompts stored in a class-specific bank and stabilized by prototype matching.","lead":"This paper uses an AI vision-language model to write detailed descriptions of medical images, then uses those descriptions to guide an image generator so it can create more varied synthetic medical scans and photos. Tests on skin, chest, and colon images show bigger diversity and stronger classifier training gains.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"VAPS grounding is never validated; the ablation only removes text conditioning, so the diversity/classification gains could come from arbitrary text diversity rather than accurate attribute prompts.","rationale":"The reader's weakest assumption identifies the same core risk: the paper never validates that VAPS descriptions are grounded in the actual images. My concern sharpens this into a concrete experimental gap: the only ablation removes text conditioning entirely, so it cannot isolate grounding. Since test-time prompts are sampled from a training-image prompt bank, even generic or hallucinated descriptions could add useful conditioning variability, which would explain the observed diversity and downstream classification gains without supporting the claimed mechanism of accurate visual attribute guidance. This is load-bearing because the paper's novelty and framing depend on VAPS producing faithful descriptions. The proposed control (image-blind MLLM prompting) directly tests whether image grounding matters while preserving text-conditional structure. I also note the internal inconsistency in Table 1, where StyleGAN achieves better FID on three datasets and better Precision on two, contradicting the conclusion's 'best generation results' phrasing. Both issues justify a conditional verdict, but neither alone proves the method is invalid; the control experiment would settle the grounding question empirically. Therefore I recommend keeping the reader's CONDITIONAL verdict unchanged.","tokens_in":8376,"tokens_out":6208,"duration_ms":74807,"concrete_test":"Run a matched control for Table 2 (and Table 3): generate descriptions with the same MLLM and VAPS prompts but replace the input image with a fixed neutral or blank gray image at Step 1 and Step 3, keeping text length, style, CSPB, and PCM identical. If IS, Recall, FID, and downstream mAUC stay within noise of the VAP-Diffusion row, the reported gains are not attributable to image-grounded attribute prompts. As a secondary check, compute BiomedCLIP cosine similarity between each stored description and its source image; if it is not significantly higher than similarity to other images of the same class, the descriptions are not image-specific.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 2.2 claims VAPS mitigates hallucination, but the paper reports no measure of description accuracy against ground truth, human ratings, or image-text alignment. The only ablation (Table 2, 'w/o VAPS') removes all text conditioning, restoring the UViT baseline (FID 52.760, Recall 0.275 on ISIC2018). This design cannot distinguish the contribution of image-grounded attribute text from the contribution of any per-sample text condition. At test time, prompts are randomly sampled from the class-specific prompt bank built from training images, so a prompt is no longer tied to the image it was generated from; if the MLLM descriptions are imprecise or class-generic, the method reduces to randomized text-conditioned diffusion. Because the paper's novelty and the 'grounded visual attribute prompt' claim rest on VAPS, the reported improvements in IS, Recall, and downstream mAUC are not evidence for correct attribute guidance without a control that keeps text conditioning but removes image grounding. Separately, Table 1 itself does not support 'best generation results' since StyleGAN has better FID on ISIC2018/2019 and ChestXray14 and better Precision on ISIC2019/Colonoscopy.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes VAP-Diffusion, a diffusion-based medical image generator that conditions on visual attribute descriptions produced by a pre-trained multimodal large language model (MLLM). The method has three components: VAPS, a chain-of-thought prompting strategy meant to elicit grounded descriptions; CSPB, a per-class bank of stored descriptions used as test-time conditions; and PCM, a prototype-conditioning mechanism that regularizes test-time text embeddings toward training prototypes. The approach is evaluated on four medical image datasets (ISIC2018, ISIC2019, ChestXray14, and a colonoscopy database) with FID, IS, Precision, Recall, and downstream classification mAUC/F1 metrics. The paper reports consistent improvements in diversity (IS, Recall) and in downstream classification performance over StyleGAN and several diffusion baselines, and ablation experiments indicate that removing VAPS or PCM degrades performance.","tokens_in":8616,"tokens_out":4132,"duration_ms":43312,"significance":"If the central claim holds, the paper offers a practical way to inject fine-grained attribute information into medical image synthesis without manual annotation, with a credible downstream data-augmentation benefit. Strengths include the breadth of evaluation across three modalities, the ablations showing that VAPS and PCM each contribute, and the reproducible code repository link. The main limitation is that the paper's realism claims are only partially supported by its own metrics: VAP-Diffusion is worse than StyleGAN in FID on three of four datasets and worse in Precision on two datasets. In addition, the claim that VAPS yields hallucination-free, image-grounded descriptions is never directly validated. The paper is a useful contribution to medical image generation if these issues are addressed, but at present the evidence is not fully commensurate with the conclusions.","major_comments":[{"comment":"The conclusion that VAP-Diffusion 'achieves the best generation results' is not supported by Table 1. StyleGAN achieves better FID on ISIC2018 (16.968 vs 19.790), ISIC2019 (23.725 vs 25.232), and ChestXray14 (36.221 vs 40.487), and better Precision on ISIC2019 (0.457 vs 0.416) and Colonoscopy (0.464 vs 0.405). The paper claims 'more realistic and diverse' generation, but on the realism metrics VAP-Diffusion is not consistently superior. Please revise the claim to describe a fidelity-diversity trade-off, or provide additional evidence (e.g., tuned StyleGAN baselines at the same resolution and training budget) to justify the realism claim.","section":"§3.1, Table 1; §4"},{"comment":"The central claim that VAPS produces 'accurate and informative descriptions' and mitigates hallucination is never tested. The paper reports no measure of description accuracy against ground-truth annotations, human ratings, or image-text alignment. The only ablation (w/o VAPS) removes text conditioning entirely, reverting to the UViT baseline, so it cannot distinguish the contribution of image-grounded attribute prompts from the contribution of any per-sample text condition. A control that keeps text conditioning but breaks image grounding (e.g., randomly reassigning descriptions across images, or using generic class-level templates) is necessary before attributing the gains to correct visual attribute guidance.","section":"§2.2, §3.2, Table 2"},{"comment":"The evaluation protocol for 'unseen input texts' is underspecified. The prompt bank is built from training images, and Table 2 states that VAP-Diffusion is evaluated with unseen texts, but the paper never states whether the test prompt bank is disjoint from the training prompt bank, how many prompts are sampled per class, or how the class distribution of the unseen prompts is controlled. Without a clear train/test split over descriptions and a description-level leakage check, the robustness claim for PCM against out-of-distribution text inputs is not established.","section":"§2.3, §2.4, Table 2"},{"comment":"Several load-bearing implementation details are missing: the specific pre-trained MLLM used in VAPS is never named, the value of the balancing term α in Eq. (3) is not given, training hyperparameters (resolution, batch size, diffusion steps, number of training images) are omitted, and the protocols for computing FID/IS/Precision/Recall and for training the downstream classifiers are not described. These details are necessary to reproduce the reported numbers and to assess the fairness of the comparisons across methods.","section":"§3 overall"}],"minor_comments":[{"comment":"The notation 't2 = MLLM(Question2)' suggests the second prompting step does not receive the input image X0; if Question2 is meant to be used together with the image, the equation should be 't2 = MLLM(X0, Question2)' for consistency with Question1 and Question3.","section":"§2.2"},{"comment":"Table 2 reports FID=26.767 for VAP-Diffusion on ISIC2018, while Table 1 reports 19.790 for the same dataset. The difference is presumably due to the unseen-text evaluation setting, but this should be stated directly in the table caption to avoid confusion.","section":"Table 2"},{"comment":"Reference [16] is cited both for hallucination and for chain-of-thought prompting, but [16] is the chain-of-thought paper; please add a dedicated hallucination reference. Also, references [10] and [29] are the same paper (Rombach et al., LDM) and should be consolidated.","section":"References"},{"comment":"The red arrows indicating repetitive content in StyleGAN images are difficult to discern at the current figure resolution; higher-resolution crops would make the qualitative comparison more convincing.","section":"Fig. 2"}],"recommendation":"major_revision","confidential_remarks":"The paper is likely a MICCAI-style submission and the empirical direction is plausible, but the central 'grounded visual attribute prompt' claim is currently under-supported. The missing control experiments and unstated MLLM identity are fixable within the manuscript's scope, so I do not recommend rejection. The most important revision is to address the mismatch between the conclusion and Table 1, and to add a text-conditioning control to the ablation study."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things you should know first. The downstream classification results are the real contribution: across dermoscopy, chest X-ray, and colonoscopy, augmenting with VAP-Diffusion synthetic images beats StyleGAN-augmented training, especially at 1% and 10% real data, with gains up to ~12 points on some metrics. Second, the paper's headline claim that it achieves \"the best generation results\" does not survive its own Table 1: StyleGAN has better FID on three of four datasets and better Precision on three of four. The method wins big on Recall and IS, so the honest claim is diversity and downstream utility, not overall realism.\n\nWhat's new: the specific combination of chain-of-thought visual attribute prompts, a class-specific prompt bank, and a prototype condition mechanism is new, and the framework is sensible and reproducible in principle. The ablations show both VAPS and PCM matter, and the prototype-based robustness idea is neat.\n\nNow the soft spots, in proportion. The main one is that VAPS's \"grounded\" claim is never actually tested. The paper motivates VAPS as reducing hallucination, but it never measures description accuracy against human ratings, ground truth, or any image-text alignment metric. The only ablation (w/o VAPS) removes text conditioning entirely, so it cannot separate accurate attribute guidance from any per-sample text condition. The stress-test is right: a control that keeps text conditioning but uses arbitrary or shuffled descriptions would be needed to support the grounding story. Until then, the gains in diversity and downstream classification could simply come from varying text conditions, which is still useful but not what the abstract claims.\n\nOther soft spots are more minor. The MLLM is never named, which matters because different MLLMs have very different hallucination rates. The exact VAPS prompts for colorectal and chest X-ray are not included, only the dermatology example. There are no error bars or significance tests anywhere. The github URL exists but no code or commit is visible. And the \"unseen descriptions\" evaluation samples from the same training distribution, so it is not truly out-of-distribution.\n\nWho this is for: a medical imaging audience using generative models for data augmentation or data sharing. The downstream evidence is strong enough to take seriously. With a grounding control, error bars, and a named MLLM, this could be a solid MICCAI-style paper. As is, it deserves peer review, but reviewers should push for those fixes and the authors should temper the \"best generation results\" claim.","headline":"Useful diversity and downstream gains, but the grounding claim for the MLLM prompts is unvalidated and the realism claim conflicts with the paper's own FID/precision numbers.","tokens_in":9118,"tokens_out":4094,"would_cite":true,"duration_ms":39781,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that conditioning a diffusion model on MLLM-generated visual attribute descriptions—shape, color, size, and texture—makes medical image generation more realistic and diverse, and that synthetic images from it improve…","keywords":["Medical image generation","Diffusion models","Multimodal large language models","Visual attribute prompts","Chain-of-thought prompting","Data augmentation","Dermoscopic imaging","Chest X-ray"],"falsifier":"Take a held-out set of medical images, run VAPS to obtain descriptions, and compare those descriptions with expert-annotated attributes or human ratings for the same images; if the descriptions frequently disagree with the images, the grounding claim fails. A complementary check is to swap the prompt bank with random or mismatched attribute text and keep the pipeline otherwise identical: if FID, recall, and downstream gains do not deteriorate, the reported improvement is an effect of text conditioning in general, not of correct visual attribute guidance.","tokens_in":8197,"feed_emoji":"🩺","tokens_out":13562,"duration_ms":131459,"temperature":0.7,"pith_summary":"VAP-Diffusion is a medical image generator that conditions a diffusion model on fine-grained visual attribute descriptions produced by a pre-trained multimodal large language model (MLLM), rather than on class labels alone. The paper argues that this added text guidance splits the hard problem of modeling an entire class distribution into many narrower attribute subproblems, making generated skin lesion, chest X-ray, and colonoscopy images more realistic and more diverse. Because descriptions are unavailable at test time, the method stores per-class descriptions in a prompt bank and uses a prototype mechanism to keep arbitrary test descriptions close to training conditions. The authors report that VAP-Diffusion beats existing generators on FID, Inception Score, precision, and recall, and that its synthetic images improve downstream classification, especially with only 1% or 10% of real data available.","feed_headline":"MLLM text prompts improve medical image generation and classifiers","feed_subtitle":"Chain-of-thought visual descriptions guide diffusion models, boosting realism and scarce-data classification.","key_machinery":"The core mechanism is Visual Attribute Prompt Strategy (VAPS), a three-question chain-of-thought protocol that first asks the MLLM for an uninterpreted description of the image, then for the possible manifestations of shape, color, size, and texture in that imaging modality, and finally for a concise image-grounded summary. The Class-Specific Prompt Bank (CSPB) stores these descriptions per category and randomly retrieves one at inference, so no description is needed at test time. The Prototype Condition Mechanism (PCM) builds a trainable prototype vector per class, forces it to reconstruct the noisy encoder features through cross-attention, and uses a zero-initialized linear layer to inject the text prior gradually, which keeps test-time embeddings close to training prototypes. These modules sit on top of a latent diffusion backbone with UViT as the denoising network and BiomedCLIP as the text encoder.","core_discovery":"The central claim is that MLLM-generated textual descriptions of visual attributes—shape, color, size, and texture—provide a richer and more usable conditioning signal for medical image diffusion than class labels alone. VAP-Diffusion obtains these descriptions through a three-stage chain-of-thought prompt strategy, stores them per class, and trains a UViT latent diffusion model conditioned on their BiomedCLIP embeddings. At inference, a description is sampled from the class-specific bank, and a prototype condition mechanism keeps the text embedding near the training prototypes of that class so unseen or freely written descriptions remain valid. The paper reports consistently better realism and diversity than StyleGAN, LDM, DiT, and other baselines on four datasets from three modalities, and shows that augmenting classifiers with its synthetic images yields mAUC and F1 improvements, up to 11.9 percent, when real data are scarce.","pith_inferences":["A natural test the paper does not run is per-attribute controllability: generate images conditioned on one specific attribute value, such as an irregular border or light color, and check with an attribute classifier or human reader whether that property appears more often than chance.","The reported downstream gains are largest for scarce-data settings, which suggests a focused extension: use the prompt bank as a balanced sampler to explicitly augment minority classes in imbalanced skin lesion and chest X-ray datasets.","Because the paper never measures description accuracy against expert labels, an immediate research step is to compare VAPS outputs with existing attribute annotations in dermoscopic datasets; that would separate genuine attribute grounding from generic text-conditioning benefits.","The prototype mechanism could be tested as an interpretability tool: the learned per-class prototypes encode what the model thinks are typical attribute combinations, so inspecting them may reveal which textual attributes the MLLM and the generator rely on most."],"forward_implications":["If the central claim holds, medical image generators can obtain fine-grained visual conditions automatically from MLLMs, without manual attribute annotation.","Classifiers trained with VAP-Diffusion synthetic data should improve most in low-data regimes; the paper demonstrates gains at 1% and 10% real-data ratios on ISIC2018, ChestXray14, and colonoscopy data.","Because descriptions are stored in a per-class bank and test embeddings are pulled toward training prototypes, deployment does not require re-running the MLLM or having a real description at hand.","The same three-module recipe—attribute prompting, a prompt bank, and prototype regularization—should transfer to other medical modalities that can be described by shape, color, size, and texture, once question templates are adapted."],"supporting_citations":[{"why":"Supplies the UViT denoising backbone that VAP-Diffusion builds on and must beat in generation comparisons.","marker":"[31]"},{"why":"BiomedCLIP is the text encoder that turns VAPS descriptions into the conditioning embeddings used by the generator.","marker":"[17]"},{"why":"Chain-of-thought prompting is the template for VAPS, the method used to elicit grounded descriptions from the MLLM.","marker":"[16]"},{"why":"StyleGAN is the main GAN baseline and the source of synthetic data in the downstream classification comparison.","marker":"[27]"},{"why":"Latent diffusion supplies the pre-trained encoder/decoder that maps images to latent space and serves as the LDM baseline.","marker":"[29]"},{"why":"ISIC2018 is the primary dermoscopic dataset used for generation and downstream experiments.","marker":"[18]"},{"why":"ISIC2019 is the second dermoscopic dataset used to support the four-dataset generation claim.","marker":"[19]"},{"why":"ChestXray14 is the chest X-ray dataset used for generation and downstream evaluation.","marker":"[20]"},{"why":"Public colonoscopy datasets that make up the colonoscopy database used for generation and downstream evaluation.","marker":"[21–23]"}],"fun_headline_variants":["MLLM descriptions boost medical image diffusion realism","Chain-of-thought MLLM prompts enrich medical image synthesis","Visual attribute prompts from MLLMs improve diffusion models","Better medical images via MLLM-crafted attribute descriptions","MLLM-generated text guides diffusion for diverse medical images"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing assumption is that the MLLM descriptions are genuinely grounded in the images, so the text condition carries correct attribute information rather than plausible but unrelated language; the paper motivates VAPS as reducing hallucination but never measures description accuracy against ground truth, human ratings, or a baseline.","fun_headline_variants_meta":{"raw":{"variants":["MLLM descriptions boost medical image diffusion realism","Chain-of-thought MLLM prompts enrich medical image synthesis","Visual attribute prompts from MLLMs improve diffusion models","Better medical images via MLLM-crafted attribute descriptions","MLLM-generated text guides diffusion for diverse medical images"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000254,"raw_usage":{"total_tokens":1564,"prompt_tokens":936,"completion_tokens":628,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":552,"completion_tokens_details":{"reasoning_tokens":549}},"tokens_in":552,"tokens_out":628,"duration_ms":6661,"temperature":1.0,"reasoning_tokens":549,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T21:35:25.752121+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a held-out set of medical images, run VAPS to obtain descriptions, and compare those descriptions with expert-annotated attributes or human ratings for the same images; if the descriptions frequently disagree with the images, the grounding claim fails. A complementary check is to swap the prompt bank with random or mismatched attribute text and keep the pipeline otherwise identical: if FID, recall, and downstream gains do not deteriorate, the reported improvement is an effect of text conditioning in general, not of correct visual attribute guidance.","supporting_citations":[{"cited_title":"In: CVPR, pp","cited_arxiv_id":null,"evidence_quote":"Supplies the UViT denoising backbone that VAP-Diffusion builds on and must beat in generation comparisons."},{"cited_title":"In: NeurIPS, vol 35, pp","cited_arxiv_id":null,"evidence_quote":"Chain-of-thought prompting is the template for VAPS, the method used to elicit grounded descriptions from the MLLM."},{"cited_title":"In: NeurIPS, vol 34, pp","cited_arxiv_id":null,"evidence_quote":"StyleGAN is the main GAN baseline and the source of synthetic data in the downstream classification comparison."},{"cited_title":"In: CVPR, pp","cited_arxiv_id":null,"evidence_quote":"Latent diffusion supplies the pre-trained encoder/decoder that maps images to latent space and serves as the LDM baseline."},{"cited_title":"In: CVPR, pp","cited_arxiv_id":null,"evidence_quote":"ChestXray14 is the chest X-ray dataset used for generation and downstream evaluation."}],"review_version":1}