{"id":"9e25828b-24be-4ce6-9b48-f58f91708c94","arxiv_id":"2411.17535","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"Prototype-guided diffusion improves expert-rated medical plausibility of synthetic images without requiring new human feedback during training.","lead":"This paper presents a diffusion model for medical images that uses learned class prototypes to guide image generation, aiming to make synthetic images more medically plausible. The authors report that this prototype guidance improves expert-rated plausibility and downstream classification accuracy on bone marrow and skin lesion datasets compared to standard classifier-free guidance.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Expert plausibility ratings rest on a single unblinded rater per dataset; the headline improvement could reflect rater bias rather than a true gain.","rationale":"The reader's weakest assumption is precisely the load-bearing vulnerability: the only direct evidence of improved medical plausibility is a single unblinded expert per dataset. The downstream classifier improvements are a useful supporting signal, but they do not measure plausibility directly and could reflect spurious image statistics rather than clinical validity. The method itself is coherent: pretrained CLIP embeddings are used to learn class prototypes, these prototypes initialize and freeze the diffusion model's class embeddings, and the comparison against a classifier-free guidance baseline with randomly initialized embeddings is a reasonable experimental design. However, because the outcome variable is a subjective expert judgment collected without blinding or replication, the central quantitative claim cannot be separated from rater expectation. The paper also omits code, statistical tests, and an explanation for dropping two bone-marrow classes, but these are secondary to the missing validity of the evaluation. The conditional verdict is appropriate; no additional downgrade is needed unless the proposed blinded multi-rater check fails.","tokens_in":9896,"tokens_out":5241,"duration_ms":52675,"concrete_test":"Re-run the Section 5.3 evaluation with at least two board-certified experts per dataset, blinded to model identity and shown images in randomized order, using the same listed criteria. Compute per-rater plausibility rates, Cohen's kappa between raters, and a confidence interval for the baseline-versus-proposed difference. If the gain does not replicate under blinding, or if inter-rater agreement is low, the central claim is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that prototype guidance substantially improves medical plausibility is supported only by the expert evaluations in Section 5.3 and Tables 3 and 4. Each dataset was rated by exactly one expert, with no blinding to which model generated each image, no randomization of presentation order, no inter-rater reliability measure, and no statistical analysis of the percentage differences. The reported gains (about 11 and 14 percentage points) are exactly the kind of effect that an unblinded rater with a stake in the outcome could produce or exaggerate, especially when the criteria are qualitative and the images are 64x64 patches. Without replicated blinded ratings, the headline improvement is not empirically established.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes IMPROVE, a prototype-guided diffusion framework for medical image generation. The authors learn per-class prototypes from CLIP embeddings of the training images, then freeze these prototypes and use them in place of randomly initialized class embeddings to condition a UNet-based diffusion model, with the prototype and time embeddings concatenated and fed into the network. Experiments are performed on the Bone Marrow and HAM10000 datasets at 64x64 resolution, with 100 images per class used for training. A single pathologist (Bone Marrow) and a single dermatologist (HAM10000) rate generated images for medical plausibility, and downstream ResNext-50 classifiers are trained on the synthetic images. The paper reports plausibility increases from 48.8% to 60% for HAM10000 and from 48.07% to 62.57% for Bone Marrow, and downstream F1 improvements of about 11 and 4 percentage points over a classifier-free guidance baseline.","tokens_in":10030,"tokens_out":5328,"duration_ms":48172,"significance":"If the reported gains are real and reproducible, the method would be a meaningful contribution: it removes the need for human feedback during training, which is a major bottleneck in medical image generation, and it shows a downstream benefit in classification. The idea of using learned prototypes as a substitute for human guidance is interesting and builds naturally on existing prototype-learning and diffusion literature. The paper also provides per-class plausibility breakdowns and an independent downstream evaluation, which are both strengths. However, the current evaluation is not sufficient to establish the central claim: the plausibility measurements rely on single, unblinded raters with no statistical analysis, and two Bone Marrow classes are omitted from the plausibility table without justification. The method description is also underspecified regarding the baseline and guidance mechanism. The potential significance is real, but the evidence as presented is suggestive rather than conclusive.","major_comments":[{"comment":"The central claim of improved plausibility rests entirely on ratings from a single expert per dataset (one pathologist for Bone Marrow, one dermatologist for HAM10000), with no blinding, no randomization of presentation order, no inter-rater reliability, and no statistical testing. The reported increases of 11.2 and 14.5 percentage points are well within the range that an unblinded rater's expectation could produce, especially when the images are 64x64 patches rated against qualitative criteria. To support the headline claim, the authors should provide ratings from multiple blinded experts, report inter-rater agreement, and compute confidence intervals or a significance test for the differences. Without this, the evidence is anecdotal.","section":"Section 5.3, Tables 3 and 4"},{"comment":"Table 4 reports plausibility for only 14 of the 16 Bone Marrow classes listed in Table 1; the classes 'ART (Artefact)' and 'NIF (Not identifiable)' are omitted. The paper does not state whether the average plausibility (48.07% to 62.57%) is computed over the 14 listed classes or over all 16, and no justification is given for the omission. The downstream classifier in Table 5 is similarly trained on 14 cell types. This omission could affect the claimed average improvement and the conclusion that the improvement is consistent across two domains. Please clarify the exact classes used in each analysis and report results for all classes or explain why these two are excluded.","section":"Table 4 and Section 6"},{"comment":"The description of the baseline and the proposed guidance mechanism is underspecified. The paper refers to the baseline as 'Classifier Free Guidance', but standard classifier-free guidance requires randomly dropping the conditioning information during training and using a guidance scale at sampling. The text does not state whether this dropout was applied, nor does it give the guidance scale for either model. The proposed method is described only as concatenating frozen prototype embeddings with time embeddings and feeding them to UNet layers; no equation or algorithmic pseudocode defines how the prototype conditioning modifies the noise prediction. Because the claimed advantage over the baseline could in principle be due to differences in conditioning handling rather than the prototypes themselves, the comparison needs a precise specification of both models and identical training and sampling protocols.","section":"Section 4.2 and Section 5.3"},{"comment":"The downstream classification results are reported as single-point precision, recall, and F1 values without variance or significance testing. For example, Table 5 reports F1 of 74.58% for the proposed method versus 63.40% for the baseline, but no confidence intervals are given; given that only one synthetic training set of each type was generated and one classifier trained, the observed gap could be within run-to-run variability. Please report results over multiple synthetic datasets and training seeds, with mean plus/minus standard deviation and a significance test, to support the claim that the improvements are consistent.","section":"Tables 5 and 6"}],"minor_comments":[{"comment":"In the sentence describing the downstream classifier, 'seven cell types for HAM1000 dataset' should read 'HAM10000 dataset'.","section":"Section 6"},{"comment":"The header 'Morphological Cell Type' is inaccurate for the dermatology dataset; rename it to 'Lesion Type' or similar.","section":"Table 3"},{"comment":"The annotation platform is written 'Labelstudio'; the standard name is 'Label Studio'.","section":"Section 5.3"},{"comment":"Reference [31] for HAM10000 lists only the dataset name as author; the full citation (Tschandl, Rosendahl, and Kittler, Scientific Data, 2018) should be provided. Reference [17] is also missing author names.","section":"References"},{"comment":"The architecture name 'UNet' is written as 'UNET' inconsistently; please standardize.","section":"Section 4.2"},{"comment":"The abstract says the method improves plausibility 'without the need for any human feedback,' but the evaluation uses human experts; consider clarifying that human feedback is not used during training or generation, only for evaluation.","section":"Abstract"}],"recommendation":"major_revision","confidential_remarks":"The single-rater, unblinded evaluation is the main risk to the paper's central claim. The missing two Bone Marrow classes and the underspecified baseline/guidance protocol also need to be addressed. If the authors can provide multi-rater blinded evaluation with statistics and clarify the method, the paper could be suitable for publication; as it stands, the evidence is suggestive but not convincing."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is an incremental but reasonable tweak to ProtoDiffusion—freeze CLIP-learned prototypes as class embeddings—and the downstream classification gains look real. But the headline 'substantially enhances medical plausibility' is supported only by one unblinded rater per dataset, no statistics, and two unexplained missing classes. The paper needs a serious referee, but the current evidence does not establish the plausibility claim.\n\nWhat's actually new: the architectural change is small (initialize class embeddings with CLIP prototypes, freeze them), but it's a clean idea and the empirical evaluation is new. The downstream classifier experiment (Tables 5, 6) shows F1 gains of ~11 points on Bone Marrow and ~4 on HAM10000 over a classifier-free guidance baseline, tested on held-out real images. That is a proper head-to-head comparison and is the strongest evidence in the paper. The two-domain consistency (bone marrow cytology and dermoscopy) helps.\n\nThe soft spots are the expert evaluations in Section 5.3 and Tables 3-4. Each dataset was rated by a single expert, with no blinding, no randomization, no inter-rater reliability, and no statistical testing. The percentages are given without confidence intervals; many per-class differences are within sampling noise. The stress-test note is right: the 11-14 point aggregate gap could be inflated by an unblinded rater. This is not a manufactured concern; it is the main load-bearing evidence for the paper's title claim. The paper also omits two of the sixteen Bone Marrow classes (ART, NIF) from the plausibility table without explanation, and never compares to the RLHF approach that motivates the 'no human feedback' framing. No code or model weights are released, so the downstream results can't be independently checked yet.\n\nI'm not bothered by the prototype learning being on the same 100 images per class used for training—that's standard, not circular. The CLIP prototype initialization is a reasonable way to inject domain knowledge without human feedback.\n\nWho this is for: researchers working on synthetic medical data generation and evaluation. It's a decent workshop-level contribution but the central claim is under-supported. I'd send it to peer review because the method and the downstream experiment deserve scrutiny, but I'd expect major revisions: multiple blinded raters, inter-rater agreement, confidence intervals or significance tests, an explanation for the missing classes, and ideally a comparison to the RLHF baseline.","headline":"A sensible but incremental prototype-guided diffusion tweak; the downstream F1 gains look real, but the headline plausibility claim rests on a single unblinded rater per dataset, no statistics, and two unexplained missing classes.","tokens_in":10546,"tokens_out":3521,"would_cite":false,"duration_ms":74627,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that seeding a diffusion model's class embeddings with learned prototype vectors raises the share of expert-approved synthetic medical images from about 48% to 60–63% on two datasets, without any human feedback during…","keywords":["diffusion models","medical image generation","prototype learning","biological plausibility","classifier-free guidance","synthetic data augmentation","bone marrow cytology","dermatoscopy"],"falsifier":"Have two or more independent experts, blinded to which model produced each image, rate the same generated sets using the paper's stated criteria; if inter-rater agreement is low, or if the prototype-guided advantage over classifier-free guidance disappears under blinded scoring, the central claim is unsupported.","tokens_in":9729,"feed_emoji":"🩺","tokens_out":7399,"duration_ms":63618,"temperature":0.7,"pith_summary":"The paper's central claim is that medical plausibility of synthetic images, not pixel fidelity, is the bottleneck in medical image generation, and that this bottleneck can be addressed without the costly human-feedback loop used in prior RLHF-based work. The proposed fix is to condition a diffusion model on learned per-class prototype vectors—compact class summaries distilled from image embeddings—rather than on randomly initialized class embeddings. On bone marrow cytology and dermatoscopic skin-lesion datasets, expert raters judged roughly 63% and 60% of the prototype-guided images plausible, versus about 48% for classifier-free guidance. Downstream classifiers trained on the synthetic images improved by about 11 and 4 F1 points, narrowing the gap to classifiers trained on real images. If the evaluation holds, this gives a cheaper route to medically plausible synthetic data for augmenting rare and long-tailed medical datasets.","feed_headline":"Prototype-guided diffusion lifts expert-approved medical images to 63%","feed_subtitle":"Learned per-class prototypes replace human raters in guiding diffusion; plausibility climbs from ~48% to 60–63% on two clinical datasets.","key_machinery":"The central object is the per-class prototype vector: a learned representative embedding for each class, computed from CLIP image embeddings using distance-based cross-entropy and a prototype loss that rewards intra-class compactness and inter-class separability. This vector is concatenated with the sinusoidal time embedding and injected into the UNet layers of the diffusion model, with its weights frozen during training. It carries the argument because it replaces the randomly initialized class embedding of classifier-free guidance with a semantically meaningful target, steering generation toward class-typical morphology without requiring a reward model or human ratings.","core_discovery":"The paper's discovery is that the conditioning signal in a diffusion model can carry medical semantics: replacing random class embeddings with frozen prototype vectors learned from CLIP embeddings of 100 images per class makes the denoising process land in biologically plausible regions of image space. With identical architecture and hyperparameters, the only intervention is the prototype-initialized, frozen class embedding concatenated with the time embedding and fed into the UNet's layers. The paper reports that the fraction of expert-approved images rises from 48.8% to 60% on HAM10000 and from 48.07% to 62.57% on Bone Marrow, with gains in most individual classes and particularly large gains for hard cell types like hairy cells and basophils. No expert feedback is used during training; the experts appear only in the final evaluation. The same synthetic datasets also train better downstream classifiers, giving F1 scores of 74.58% and 75.08% versus 63.40% and 71.10% for the classifier-free baseline, although still below the 79.66% and 85.91% achieved with real images.","pith_inferences":["A direct comparison against the RLHF-aligned generator of prior work, at matched annotation budgets, would test whether prototype guidance captures most of the plausibility benefit without the expert-in-the-loop cost; the paper only compares against classifier-free guidance.","The same recipe—learn compact per-class prototypes and freeze them as diffusion conditioning—could be applied to other structured generation domains, such as pathology patches, retinal imaging, or even non-medical domains where class-typical structure matters.","Because each dataset was scored by one unblinded expert, a plausible expectation is that the effect size will shrink under multi-rater blinded protocols; that expectation is directly testable.","The dependency on CLIP embeddings means the method's transfer to modalities where CLIP features are weak may require a domain-specific feature extractor; this is an open question, not a result of the paper."],"forward_implications":["Synthetic datasets produced this way can augment rare and long-tailed medical classes with a higher fraction of expert-acceptable images than classifier-free diffusion, at no extra annotation cost during training.","The method transfers across at least two modalities (hematology and dermatology), suggesting the prototype-initialization mechanism is not tied to one type of tissue or stain.","Training a downstream classifier on prototype-guided synthetic images yields F1 gains of about 11 points (Bone Marrow) and 4 points (HAM10000) over the classifier-free baseline, so the plausibility improvement is accompanied by usable class signal.","Classifiers trained on real images still outperform those trained on synthetic images, so the method's promise is augmentation rather than full replacement of real data.","Because prototypes are learned from a small sample (100 images per class) and then frozen, the pipeline is computationally light enough for small medical datasets."],"supporting_citations":[{"why":"Supplies the Protodiffusion architecture whose class embeddings this paper replaces with learned, frozen prototypes.","marker":"[3]"},{"why":"Introduces human expert feedback as the gold standard for medical plausibility and motivates the RLHF baseline that this work aims to avoid.","marker":"[30]"},{"why":"Defines classifier-free guidance, the conditioning framework and baseline against which the prototype-guided results are measured.","marker":"[15]"},{"why":"Provides the convolutional prototype learning losses (distance-based cross-entropy and prototype loss) used to learn the prototype vectors.","marker":"[35]"},{"why":"Supplies the expert-annotated Bone Marrow dataset used for training and evaluation.","marker":"[17]"},{"why":"Supplies the HAM10000 dermatoscopic dataset used for training and evaluation.","marker":"[31]"},{"why":"Provides the DDIM sampling procedure used to generate 100 images per class over 50 iterations.","marker":"[29]"}],"fun_headline_variants":["No-feedback diffusion lifts medical image approval from 48% to 63%","CLIP prototypes guide diffusion to 63% expert-approved without human input","Prototype-guided diffusion: no RLHF needed for medical plausibility boost","From 48% to 63% medical plausibility: diffusion with frozen prototypes"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the single expert's plausibility judgments are a valid and reproducible gold standard: one pathologist and one dermatologist rated unblinded image sets, with no inter-rater reliability or statistical testing reported.","fun_headline_variants_meta":{"raw":{"variants":["No-feedback diffusion lifts medical image approval from 48% to 63%","CLIP prototypes guide diffusion to 63% expert-approved without human input","Prototype-guided diffusion: no RLHF needed for medical plausibility boost","From 48% to 63% medical plausibility: diffusion with frozen prototypes"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001029,"raw_usage":{"total_tokens":4377,"prompt_tokens":1030,"completion_tokens":3347,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":646,"completion_tokens_details":{"reasoning_tokens":3264}},"tokens_in":646,"tokens_out":3347,"duration_ms":22420,"temperature":1.0,"reasoning_tokens":3264,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T11:59:15.123628+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Have two or more independent experts, blinded to which model produced each image, rate the same generated sets using the paper's stated criteria; if inter-rater agreement is low, or if the prototype-guided advantage over classifier-free guidance disappears under blinded scoring, the central claim is unsupported.","supporting_citations":[{"cited_title":"Protodiffusion: classifier-free diffusion guid- ance with prototype learning","cited_arxiv_id":null,"evidence_quote":"Supplies the Protodiffusion architecture whose class embeddings this paper replaces with learned, frozen prototypes."},{"cited_title":"Aligning synthetic medical images with clinical knowledge using human feedback","cited_arxiv_id":null,"evidence_quote":"Introduces human expert feedback as the gold standard for medical plausibility and motivates the RLHF baseline that this work aims to avoid."},{"cited_title":"An expert-annotated dataset of bone marrow cytology in hematologic malignan- cies, 2021","cited_arxiv_id":null,"evidence_quote":"Supplies the expert-annotated Bone Marrow dataset used for training and evaluation."},{"cited_title":"The HAM10000 dataset, a large collec- tion of multi-source dermatoscopic images of common pig- mented skin lesions, 2018","cited_arxiv_id":null,"evidence_quote":"Supplies the HAM10000 dermatoscopic dataset used for training and evaluation."}],"review_version":1}