{"id":"0cafb4ed-7633-4ece-91e9-fe6627b229d0","arxiv_id":"2507.05063","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"Adding 5,000 CytoDiff-generated synthetic white blood cell images per class is reported to improve ResNet-50 accuracy from 27% to 78% and CLIP accuracy from 62% to 77% on the Munich AML dataset.","lead":"CytoDiff fine-tunes a diffusion model with LoRA to synthesize white blood cell images from a few real examples. Adding these synthetic images to a small imbalanced dataset is reported to raise classifier accuracy substantially.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Mixed test sets include synthetic images from the same generator used for training, so the reported gains are not measured on real patient images; this is explicitly acknowledged in §3.4.3.","rationale":"I read the paper in good faith. The method is a plausible application of stable diffusion 2.1 + LoRA, inspired by DataDream, and the code is public. The authors also include honest limitation statements, notably the domain-bias paragraph in §3.4.3. My stress-test does not question the sincerity or internal consistency of the implementation; it questions whether the experimental measurement supports the strongest claim. The strongest claim is an accuracy improvement from adding synthetic images. For that claim to hold, the classifier must be evaluated on images that are comparable before and after augmentation. The paper's mixed-test protocol breaks this comparability: the augmented model is evaluated on a test set that includes synthetic images generated by the very same pipeline used to create its training data. Because a model trained on synthetic images can exploit low-level generator artifacts, high accuracy on such a test set is expected even if the model has no ability to recognize real cytomorphology. The synthetic-only result of 0.82 accuracy is consistent with this concern, since that test set is also synthetic. The concrete test is to re-evaluate the already-trained models on real-only test images, ideally with patient-level splitting to avoid patient leakage. If the improvement persists, the concern is resolved. If not, the paper should be revised to present the real-only evaluation as the primary result, with the mixed-test results clearly labeled as an internal consistency check. My recommendation therefore leaves the reader's conditional verdict unchanged: the paper is promising but needs the real-image evaluation before the central claim can be accepted.","tokens_in":9942,"tokens_out":4787,"duration_ms":57120,"concrete_test":"Re-run the Table 1 experiments with the test set composed exclusively of real images, using patient-level grouping so no patient appears in both training and test (with bootstrapping or a dedicated held-out cohort for minority classes). Compare the real+synthetic ResNet-50 and CLIP accuracies against the real-only baseline on the same real-only test set; if the gain over 27%/62% is not reproduced, the reported benefit is an artifact of domain overlap.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that adding synthetic images improves classification of real blood-cell images. The evaluation in §3.4.3 does not test this: test sets for the mixed experiments are constructed by combining real and synthetic images, and the authors state that 'this setup introduces a domain bias that may favor models trained entirely on synthetic data, as the test distribution partially overlaps with the training domain.' Because the synthetic test images are produced by the same CytoDiff model (same LoRA weights, prompts, and few-shot real examples) used to generate synthetic training images, a classifier can achieve high accuracy by recognizing generator-specific artifacts—background, blur, staining shifts, or repeated patterns—rather than morphological features of real cells. Thus the headline numbers '27% to 78%' (ResNet-50) and '62% to 77%' (CLIP) compare a real-only test set with a mixed test set, so the improvement is not measured on real patient images. The synthetic-only experiment (§3.4.2) reaching 82% accuracy is likewise evaluated on synthetic test data and does not establish generalization to real cells. The authors disclose this limitation, which is commendable, but the central claim remains unsupported without an evaluation on real-only test images.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript introduces CytoDiff, a Stable Diffusion 2.1 model fine-tuned with LoRA on a few real images per white blood cell class, and uses it to synthesize images for the Munich AML Morphology Dataset. The authors then train ResNet-50 and CLIP classifiers on real-only, synthetic-only, and mixed real-plus-synthetic data, reporting accuracy gains from 27% to 78% for ResNet-50 and from 62% to 77% for CLIP when 5,000 synthetic images per class are added. Additional experiments vary the number of few-shot real images used for generation and the number of synthetic images added, with FID used to assess image quality.","tokens_in":10325,"tokens_out":3975,"duration_ms":45938,"significance":"If the reported gains held on real clinical images, the approach would be a valuable contribution to rare-class cytomorphology classification and to privacy-preserving data sharing in biomedical imaging. The paper is clearly written, makes the code publicly available, and includes useful ablations on few-shot guidance and synthetic dataset size. The authors also deserve credit for explicitly acknowledging the domain-bias limitation of their mixed test sets. However, the primary evaluation as presented does not currently support the central claim that synthetic data improves classification of real patient cells, so the significance of the result depends on a fix to the evaluation protocol.","major_comments":[{"comment":"The headline improvement from 27% to 78% (ResNet-50) and from 62% to 77% (CLIP) compares a real-only test set with a mixed real-plus-synthetic test set. The synthetic test images are generated by the same CytoDiff model, with the same LoRA weights and prompts, that produced the synthetic training images. As the paper itself notes, \"this setup introduces a domain bias that may favor models trained entirely on synthetic data, as the test distribution partially overlaps with the training domain.\" Therefore the accuracy gain cannot be attributed to improved classification of real patient cells. The central claim of the paper remains unsupported without an evaluation on a real-only test set or another protocol that eliminates overlap between synthetic training and test distributions.","section":"Section 3.4.3, Table 1"},{"comment":"The synthetic-only experiment, in which ResNet-50 reaches 82% accuracy, is evaluated on synthetic test images. This demonstrates that the generated classes are separable by a classifier, but it does not establish that a classifier trained on synthetic data generalizes to real morphological variation. The synthetic-only result should be framed as an image-quality and separability check, not as evidence for the diagnostic-improvement claim.","section":"Section 3.4.2, Table S1"},{"comment":"To disentangle the effect of the synthetic training data from the change in test distribution, the authors should report the real-only classifier evaluated on the same mixed test set used for the real-plus-synthetic classifier. Without this control, the comparison is confounded by both training data and test distribution, so the reported increment cannot be attributed specifically to the addition of synthetic training images. The authors' justification that real-only test sets would contain too few samples per class motivates careful handling of small classes, but it does not remove the need for a real-only test evaluation, at least on classes with sufficient samples or with appropriate confidence intervals.","section":"Section 3.4.3"}],"minor_comments":[{"comment":"Reference [20] lists \"Christian Metak,\" but the correct name of the dataset author is Christian Matek; please correct this typo.","section":"References"},{"comment":"The sentence \"The experimental setup and results are presented in 3.5.1\" is imprecise, since the numerical mixed-data results appear in Table 1 and Figure 5 rather than only in Section 3.5.1; please also cross-reference Section 3.5.2.","section":"Section 3.4.3"},{"comment":"The text states that \"in the final strategy employed, the loss function is computed in a manner that treated both image types equally,\" but it does not report the exact value of lambda_1 or the precise balancing used in the final experiments; please provide the exact loss weighting.","section":"Section 2.2"},{"comment":"Figure 5 would be more informative if it included the real-only baseline at zero synthetic images per class for both classifiers, since the current curves begin at 100 synthetic images per class and the improvement relative to the baseline is therefore not visible.","section":"Figure 5"},{"comment":"The text describes a 5-fold cross-validation while also stating that each fold uses a 60/20/20 split; please clarify whether the reported metrics are fold-averaged and whether the 60/20/20 percentages are the proportions within each fold.","section":"Section 3.1"}],"recommendation":"major_revision","confidential_remarks":"The authors' honest disclosure of the domain-bias limitation is commendable, but it places the burden on them to provide a real-only evaluation. The current headline numbers in the abstract are therefore potentially misleading, and the core claim cannot be verified from the reported experiments. I would not recommend rejection solely because the approach is unconventional; the issue is internal evaluation validity. With a real-only test evaluation, or at least a carefully controlled test protocol that avoids synthetic train/test overlap, the paper could become a solid contribution."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper is a straightforward extension of DataDream to white blood cell images: fine-tune Stable Diffusion 2.1 with LoRA on attention layers, hand-craft class prompts, generate 5,000 synthetic images per class, and show classification gains. The method is not new, but the application is real and the code is public. The authors also deserve credit for showing the generated images and for explicitly acknowledging the evaluation limitation in §3.4.3.\n\nThe problem is that the main claim—synthetic data improves classification of real patient images—is not actually tested. The real-only baseline is evaluated on real test images. The mixed real+synthetic experiments use test sets that mix real and synthetic images, where the synthetic test images come from the same generator used to produce training data. The authors state this \"introduces a domain bias that may favor models trained entirely on synthetic data.\" So the 27%→78% and 62%→77% comparisons confound the data source with the test distribution. A classifier can do well on the mixed test by recognizing generator artifacts rather than cell morphology. The synthetic-only experiment (§3.4.2) is evaluated on synthetic test data, so it also doesn't establish generalization to real cells.\n\nThis is the load-bearing flaw, not a minor quibble. The paper's own text concedes it. What's missing is an evaluation on real test images, even if some minority classes have only a handful of test samples. The authors say that would be statistically unreliable, which is fair, but then the honest conclusion should be \"we don't yet know whether synthetic data helps on real images.\" The current wording overstates what is shown.\n\nSmaller issues: the manual selection of training images and prompt design are not fully reproducible, though the code helps. The FID comparison to DataDream is apples-to-oranges since the domains differ. The novelty is incremental, but that's not a sin.\n\nWho is this for? Researchers working on synthetic data for medical imaging, especially cytomorphology, who want a starting point and a cautionary example. It deserves a serious referee because the empirical question is relevant and the flaw is fixable with a clean real-only test evaluation. I would send it to review, but I would expect major revision.\n\nReading group: maybe—good example of how easy it is to let test-set construction drift in synthetic-data experiments.","headline":"Honest but flawed: the main accuracy gains compare a real-only test with a mixed real+synthetic test, so the central claim is untested.","tokens_in":10686,"tokens_out":1560,"would_cite":false,"duration_ms":18593,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"CytoDiff generates synthetic white blood cell images from a few real samples and lifts ResNet-50 accuracy from 27% to 78% and CLIP accuracy from 62% to 77% on a severely imbalanced AML morphology dataset.","keywords":["synthetic data","diffusion models","LoRA fine-tuning","white blood cell classification","class imbalance","medical imaging","few-shot generation","acute myeloid leukemia"],"falsifier":"Train ResNet-50 and CLIP on the real-plus-synthetic training set, but evaluate only on held-out real images from patients excluded from training (or from an external site); if the accuracy gain over the real-only baseline (27% for ResNet-50, 62% for CLIP) narrows or disappears, the central claim of synthetic-data-driven improvement is not supported.","tokens_in":9751,"feed_emoji":"🩸","tokens_out":7023,"duration_ms":64681,"temperature":0.7,"pith_summary":"The paper claims that a diffusion model fine-tuned on a handful of real white blood cell images can generate synthetic training data that rescues classifiers on a severely imbalanced leukemia morphology dataset. Adding 5,000 synthetic images per class to the real training set raises ResNet-50 accuracy from 27% to 78% and CLIP accuracy from 62% to 77%. The authors argue this makes synthetic data a practical tool for medical diagnostics, where rare cell types are few and patient privacy limits data sharing. The takeaway is that generative augmentation, not just hand-designed transformations, can mitigate class imbalance.","feed_headline":"Synthetic blood-cell images raise accuracy from 27% to 78%","feed_subtitle":"Adding 5,000 AI-generated cell images per class rescues rare-class performance on a leukemia dataset.","key_machinery":"The central machinery is a text-to-image stable diffusion 2.1 model whose attention layers, in both the text encoder and the diffusion U-Net, are adapted with low-rank adaptation (LoRA) weights, a parameter-efficient fine-tuning that changes only small matrices in the cross-attention and self-attention pathways. A few carefully selected real images per class (1, 4, 8, or 16) and detailed per-class text prompts steer generation toward biologically plausible cells; the resulting synthetic images are then added to the real training data, with a combined cross-entropy loss that weights real and synthetic contributions. The same prompts used for generation double as class descriptions for CLIP classification, linking the generation and evaluation pipelines.","core_discovery":"On the Munich AML Morphology Dataset, with 15 white blood cell classes and only 16 or fewer real images for several rare classes, CytoDiff generates 5,000 synthetic images per class using stable diffusion 2.1 fine-tuned with LoRA weights on attention layers, guided by few-shot real images and per-class text prompts describing nuclear shape, granules, staining, and magnification. With these synthetic images added to the real training set, ResNet-50's test accuracy rises from 27% to 78% and macro F1 from 0.07 to 0.80, while CLIP rises from 62% to 77% accuracy and 0.23 to 0.78 macro F1. Training on synthetic images alone reaches 82% accuracy and macro F1 for ResNet-50, and accuracy keeps climbing with the number of synthetic images up to the largest tested count, 5,000 per class. The authors interpret this as evidence that synthetic generation can balance the data distribution and encode the morphological distinctions needed for classification.","pith_inferences":["A purely real hold-out test set, independent of the generator, is the decisive next experiment; the paper's own Section 3.4.3 notes its mixed test set shares a domain with the synthetic training data, so part of the reported gain could be distribution match rather than learned biology.","If the accuracy gains persist on real-only evaluation, the method likely transfers to other rare-cell classification tasks, such as malaria or other leukemias, where class imbalance is equally severe.","The FID scores (roughly 50-90) are high compared with natural-image generation, yet downstream classification improves substantially, suggesting image-quality metrics like FID may be poor proxies for how useful synthetic medical images are for training.","A testable extension: vary the number of real seed images (1, 4, 8, 16) and measure classifier accuracy to quantify how much real data the generator needs before synthetic augmentation pays off."],"forward_implications":["Classifiers trained on real-plus-synthetic data reach near-perfect AUC (0.99) and high F1 on rare classes, suggesting synthetic augmentation could make minority cell types distinguishable in practice.","Accuracy improves monotonically with synthetic image count up to 5,000 per class, indicating the method has not saturated and may benefit from even larger synthetic sets.","Because generation needs only 1 to 16 real images per class, labs with tiny collections of rare cells could still build usable training sets without collecting thousands of patients.","Synthetic images can be shared without exposing patient data, offering a route to collaborative diagnostics under privacy constraints.","ResNet-50 surpasses CLIP when enough synthetic data is added, implying that a simple CNN plus balanced data can capture the fine morphological features that CLIP extracts from text prompts."],"supporting_citations":[{"why":"Provides the few-shot guided dataset generation approach that CytoDiff adapts, and the FID comparison baseline.","marker":"[8]"},{"why":"Supplies the stable diffusion 2.1 base model for text-to-image synthesis.","marker":"[14]"},{"why":"Motivates LoRA conditioning on attention layers for efficient diffusion fine-tuning.","marker":"[4]"},{"why":"The Munich AML Morphology Dataset, the real-world imbalanced data source for training and evaluation.","marker":"[9]"},{"why":"Defines the ResNet-50 architecture used as one classifier in all experiments.","marker":"[7]"},{"why":"Defines the CLIP dual-encoder model used as the second classifier and prompt-based evaluation.","marker":"[12]"},{"why":"Establishes the premise that synthetic medical images can support privacy-preserving training.","marker":"[5]"},{"why":"Provides background on FID reliability for evaluating generative medical image models.","marker":"[22]"}],"fun_headline_variants":["AI-generated blood cells boost classifier from 27% to 78%","LoRA-tuned diffusion model lifts leukemia diagnosis accuracy","Synthetic cells rescue rare-class performance, accuracy triples","CytoDiff synthesizes blood cells, ResNet accuracy jumps to 78%","Few-shot AI images turn 27% accuracy into 78% on AML data"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported improvement is measured on test sets that mix real images with synthetic images from the same generator used to create the training data, so the synthetic test images may be easier for the classifier to recognize than real unseen patient cells would be.","fun_headline_variants_meta":{"raw":{"variants":["AI-generated blood cells boost classifier from 27% to 78%","LoRA-tuned diffusion model lifts leukemia diagnosis accuracy","Synthetic cells rescue rare-class performance, accuracy triples","CytoDiff synthesizes blood cells, ResNet accuracy jumps to 78%","Few-shot AI images turn 27% accuracy into 78% on AML data"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000167,"raw_usage":{"total_tokens":1264,"prompt_tokens":962,"completion_tokens":302,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":578,"completion_tokens_details":{"reasoning_tokens":209}},"tokens_in":578,"tokens_out":302,"duration_ms":3710,"temperature":1.0,"reasoning_tokens":209,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T19:32:06.585844+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train ResNet-50 and CLIP on the real-plus-synthetic training set, but evaluate only on held-out real images from patients excluded from training (or from an external site); if the accuracy gain over the real-only baseline (27% for ResNet-50, 62% for CLIP) narrows or disappears, the central claim of synthetic-data-driven improvement is not supported.","supporting_citations":[{"cited_title":"Datadream: Few-shot guided dataset generation","cited_arxiv_id":null,"evidence_quote":"Provides the few-shot guided dataset generation approach that CytoDiff adapts, and the FID comparison baseline."},{"cited_title":"Simple Drop-in LoRA Conditioning on Attention Layers Will Improve Your Diffusion Model","cited_arxiv_id":"2405.03958","evidence_quote":"Motivates LoRA conditioning on attention layers for efficient diffusion fine-tuning."},{"cited_title":"A single-cell morphological dataset of leuko- cytes from aml patients and non-malignant controls","cited_arxiv_id":null,"evidence_quote":"The Munich AML Morphology Dataset, the real-world imbalanced data source for training and evaluation."},{"cited_title":"Deep residual learning for image recognition","cited_arxiv_id":null,"evidence_quote":"Defines the ResNet-50 architecture used as one classifier in all experiments."},{"cited_title":"Learning transferable visual models from natural language supervision","cited_arxiv_id":null,"evidence_quote":"Defines the CLIP dual-encoder model used as the second classifier and prompt-based evaluation."},{"cited_title":"Synthetic medical images for robust, privacy-preserving training of ar- tificial intelligence: application to retinopathy of prematurity diagnosis","cited_arxiv_id":null,"evidence_quote":"Establishes the premise that synthetic medical images can support privacy-preserving training."}],"review_version":1}