{"id":"512941f1-eda1-481f-a35e-59af04453f0d","arxiv_id":"2608.05471","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A 512x512 EDM2 diffusion model trained on open fetal ultrasound data produces synthetic images with lower FID and improved downstream plane-classification accuracy compared with an earlier 128x128 approach.","lead":"This paper trains a 512x512 diffusion model (EDM2) on public fetal ultrasound images to create synthetic images across six anatomical classes. If the approach holds up, it could give AI researchers a scalable source of training data for fetal ultrasound without relying on scarce or privacy-restricted clinical sets.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Memorization check missing: generated images may be near-duplicates of real FETAL PLANES DB images, which would trivially lower FID and could leak test labels into the downstream classifier.","rationale":"The reader's weakest assumption is the same: the conclusion that synthetic pretraining surpasses real-data-only training rests on the untested premise that generated images are not close copies of real training images. The authors themselves acknowledge the memorization risk in Section 2 but provide no duplication check, making this the most load-bearing gap. If the generated set contains near-duplicates of real training images, the FID advantage is trivial and the downstream gain could be leakage, especially if the generator saw the classifier test split. The data-volume confound noted by the reader is also real: the 93.36% vs 92.32% comparison is against an external real-only baseline and is not matched in training budget or repeated with confidence intervals. That concern is subordinate, because the primary comparison with Tian et al. also uses synthetic pretraining, and adding more data does not by itself invalidate the method as a practical augmentation. The single decisive check is a nearest-neighbor/duplication analysis; if it passes, the remaining matched-baseline issue would still justify a CONDITIONAL verdict until the authors add matched real-only baselines and confidence intervals. The released code and model weights make such a check feasible, which is a genuine strength, but the missing analysis prevents the stronger claim from being accepted as stated.","tokens_in":5580,"tokens_out":8041,"duration_ms":75988,"concrete_test":"For each of the 30,000 generated images, compute the distance (LPIPS and SSIM, plus pixel-level L2) to its nearest neighbor in the full FETAL PLANES DB, and compare the generated-to-real nearest-neighbor distribution with the real-to-real nearest-neighbor distribution. If a non-trivial fraction of generated images are closer to a real image than typical real-real pairs, or exceed a preset duplication threshold (e.g., SSIM > 0.95), memorization is confirmed. Also check whether any generated image's nearest neighbor falls in the classifier test split; if so, re-run the fine-tuning experiment after removing all generated images that match test images. Report the threshold, the fraction flagged, and the accuracy after exclusion.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim in Section 3, that fine-tuning on synthetic data reaches 93.36% ensemble accuracy and surpasses the 92.32% real-data-only baseline, depends on the generated images being novel samples rather than memorized copies of the real training data. The authors explicitly flag this risk in Section 2: 'there is a risk of the diffusion model memorizing training data.' Yet they report no nearest-neighbor, retrieval, or overlap analysis. This matters for both parts of the claim: FID is measured against the real FETAL PLANES DB distribution, so near-duplicates would lower FID artificially; and the downstream classifiers are pretrained on synthetic images and then fine-tuned and evaluated on FETAL PLANES DB. If the EDM2 model was trained on the full dataset, including images that later appear in the classifier test split, generated near-duplicates could leak labels and produce the reported accuracy gain without any genuine synthetic-data benefit. The paper therefore lacks the minimal evidence needed to distinguish a real generative contribution from dataset replication.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes an EDM2-based diffusion framework that generates 512x512 fetal ultrasound images across six anatomical classes, trained on FETAL PLANES DB together with three auxiliary public datasets. The authors report FID scores on 5000 generated images per class, comparing favorably with Tian et al., and evaluate downstream fetal plane classification with ResNet50, DenseNet169, MedMamba, and a soft-voting ensemble. They claim that fine-tuning on synthetic data followed by real data reaches 93.36% ensemble accuracy, surpassing 92.32% for real-data-only training. A single experienced clinician rated 100 generated images at a mean realism of 2.67/5, with real images rated higher (3.12 vs 2.07). Code, models, data, and the survey are publicly released.","tokens_in":5673,"tokens_out":3157,"duration_ms":31061,"significance":"If the central claim holds, the paper would offer a practically useful open-source recipe for high-resolution synthetic fetal ultrasound that improves downstream classification, addressing data scarcity and privacy constraints. The reproducibility artifacts are a genuine strength: code, pretrained models, data, and the evaluation survey are all released, and the comparison with Tian et al. is direct. The paper is also honest in reporting that clinician-assessed realism is lower for synthetic images and in acknowledging the memorization risk in Section 2. However, the headline accuracy gain is about one percentage point and is reported without confidence intervals, and the missing memorization check and volume-matched baseline leave the central claim under-supported.","major_comments":[{"comment":"The headline comparison of 93.36% versus 92.32% ensemble accuracy is reported without confidence intervals, standard deviations across training seeds, or a significance test. Since the difference is roughly one percentage point and the numbers come from ensembles, the claim that synthetic pretraining 'surpasses' real-data-only training is not yet established. Please report repeated-run variation or bootstrap intervals for the test set.","section":"§3, Table 2"},{"comment":"The authors explicitly write that 'there is a risk of the diffusion model memorizing training data,' but the paper provides no nearest-neighbor, retrieval, or overlap analysis between generated images and the real FETAL PLANES DB images. Because FID is computed against the same dataset and the downstream classifier is evaluated on FETAL PLANES DB, near-duplicates would lower FID and could leak test labels. The paper also does not state whether the EDM2 generator was trained on the entire FETAL PLANES DB, including images that later appear in the classifier test split. A memorization or duplication check is load-bearing for both the image-quality and classification claims.","section":"§2"},{"comment":"There is no control for total training-data volume: the synthetic-pretraining protocol uses 30,000 generated images before real fine-tuning, while the real-only baseline apparently sees only the real training set. The observed improvement could therefore be due to seeing more images rather than to the synthetic nature of those images. Please add a real-data-only pretraining baseline matched in number of images and compute budget, or otherwise demonstrate that the gain is specific to synthetic pretraining.","section":"§3, fine-tuning protocol"},{"comment":"The FID scores are reported as point estimates without confidence intervals or multiple sampling seeds. FID is a sample-based statistic, and while the overall gap of 176.85 versus 104.25 looks substantial, the class-level numbers would be more convincing with bootstrap intervals or repeated-generation runs. Please provide uncertainty estimates for the FID comparison as well as for the classification accuracies.","section":"§3, Table 1"}],"minor_comments":[{"comment":"The dataset name is given as 'FPU23' in the text but as 'FPUS23' in reference [19]; please align the name with the cited source.","section":"§2"},{"comment":"The table caption contains a formatting artifact ('T able 1'), and the text has small typos such as 'from' in the first sentence of Section 2; a careful proofread is needed.","section":"Table 1"},{"comment":"The survey description says 100 images are sampled with a deterministic random seed, but it does not state whether the sample is stratified by class or how many real versus synthetic images are included; please specify the sampling procedure and the class balance.","section":"Appendix B"},{"comment":"Figure 2 is referenced but the appendix includes only its caption; the validation-loss plot itself should be included in the manuscript file.","section":"Appendix A"}],"recommendation":"major_revision","confidential_remarks":"This is a short empirical paper with strong reproducibility artifacts. The main risk is the small accuracy gap combined with the absence of a memorization check and a volume-matched baseline; these are fixable with additional experiments and should be required before publication. The paper fits the scope of the venue but needs the requested verification to make the central claim credible."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe paper delivers what it promises: a 512×512 EDM2-based fetal ultrasound generator trained on open data, with better FID than the previous 128×128 diffusion approach (104.25 vs 176.85 overall) and improved downstream classification. The open code, weights, and survey infrastructure are real assets. The comparison to Tian et al. follows their training setup and classifiers, so the numbers are directly comparable.\n\nThe central claim, that synthetic pretraining \"surpasses real-data-only training\" (93.36% vs 92.32%), is where I'd push back. Two problems. First, no memorization or nearest-neighbor check. The authors themselves note in Section 2 the risk of the model memorizing training data, but they never test for it. Since FID is measured against the same FETAL PLANES DB distribution and the classifier is fine-tuned and evaluated on that dataset, near-duplicate generations would inflate both metrics. Second, the winning protocol uses 30,000 synthetic images before fine-tuning, while the real-only baseline gets only the real data. So the gain could simply be more training data, not better synthetic data. No confidence intervals are reported for the accuracy gap either.\n\nThe clinical evaluation is a minor soft spot: one clinician on 100 images, with real images rated higher (3.12 vs 2.07). The survey appendix is reproducible, but the real/synthetic composition of the 100 images isn't clearly stated. This part is more of a sanity check than evidence of clinical utility.\n\nNone of this sinks the paper. The FID improvement is large and plausible, and the reproducibility commitments are better than most. But the headline claim needs the missing checks to stand. A simple retraining experiment holding total data volume constant, plus a nearest-neighbor analysis of generated vs. real training images, would settle it.\n\nWho should read this: anyone working in synthetic medical imaging or data augmentation for ultrasound. It's a good example of applying SOTA diffusion models to a small-data clinical domain, warts and all. I'd send it to peer review; the methodology is sound enough to warrant referee time, and the concerns are addressable in revision.","headline":"A credible engineering advance in fetal ultrasound synthesis whose headline accuracy claim needs a memorization check and a data-volume control before it holds.","tokens_in":6288,"tokens_out":2862,"would_cite":true,"duration_ms":25927,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"An EDM2 model trained on open fetal ultrasound datasets beats the prior 128×128 method on FID and pushes ensemble accuracy past real-data-only training.","keywords":["fetal ultrasound","synthetic medical imaging","diffusion models","EDM2","image quality","Fréchet Inception Distance","downstream classification","open datasets"],"falsifier":"A nearest-neighbor or perceptual-similarity search from each generated image back to the real FETAL PLANES training set would settle the memorization question; if a substantial share of synthetic images are near-duplicates, the 93.36% fine-tuning accuracy cannot be credited to novel synthetic data.","tokens_in":5293,"feed_emoji":"🩺","tokens_out":16187,"duration_ms":126211,"temperature":0.7,"pith_summary":"The paper claims that a modern diffusion architecture, EDM2, trained on a weighted mix of open fetal ultrasound datasets, can generate 512×512 synthetic images across six anatomical classes that are closer to real images and more useful for training classifiers than the previous lower-resolution generator. On a standard image-fidelity metric (Fréchet Inception Distance), with both sets downsized to 128×128 for a fair comparison, the method reports 104.25 overall against 176.85 for the prior approach. After pretraining on synthetic images and fine-tuning on real ones, an ensemble of three classifiers reaches 93.36% accuracy, above the 92.32% obtained with real data alone. A clinician's realism rating is still lower for synthetic images (2.07 out of 5) than for real ones (3.12), so the paper positions synthetic images as a training resource rather than a replacement for real scans.","feed_headline":"Synthetic fetal scans hit 93.36% accuracy, beating real-data training","feed_subtitle":"EDM2 cut FID from 176.85 to 104.25 and pushed six-class ensemble accuracy past real-data training.","key_machinery":"The central object is the EDM2 diffusion model, a denoising-diffusion architecture designed around improved training dynamics, here run in Small and XL sizes and combined with autoguidance, in which a small model guides a larger one to sharpen outputs. Training mixes the six-class FETAL PLANES database with three additional open ultrasound datasets, weighting FETAL PLANES at 2.0 and the others at 1.0 in a mean-squared-error loss, allowing roughly twice as many training steps and a lower validation loss. The generation pipeline applies center cropping, random horizontal flipping, and resizing to 512×512, and the downstream test reuses three classifiers with a soft-voting ensemble so the comparison with the prior method is direct.","core_discovery":"The central claim is that scaling fetal ultrasound synthesis to 512×512 with the EDM2 diffusion architecture, and training on a weighted mixture of public datasets, produces synthetic images that improve on the prior 128×128 diffusion method on both image fidelity and downstream task performance. The paper reports a lower FID in every one of the six anatomical classes, an overall FID of 104.25 versus 176.85, and a soft-voting ensemble accuracy of 93.36% after synthetic pretraining followed by real-data fine-tuning, which it says surpasses the 92.32% obtained on real-world data alone. These results are offered as evidence that high-resolution synthetic fetal ultrasound can serve as a pretraining resource when clinical datasets are scarce, while the clinician study is reported as evidence that the generated images still contain visible artefacts such as smoothing and speckle irregularities.","pith_inferences":["A plausible alternative explanation the paper does not rule out is that part of the fine-tuning gain comes from having roughly 30,000 extra pretraining images rather than from the synthetic nature of those images; pretraining on an equal number of real images would isolate that effect.","If a nearest-neighbor check shows the generated images are not memorized copies, the result would support training generative models on pooled open ultrasound data as a general low-resource strategy.","The low realism score points to a concrete next step: adding physics-based ultrasound speckle and tissue-artefact simulation on top of the diffusion output, which a repeated clinician survey could test.","The weighting scheme that favours FETAL PLANES is a simple lever; a testable extension is to sweep these weights and see whether gains on the primary classes trade off against diversity on the auxiliary ones."],"forward_implications":["If the claim holds, high-resolution synthetic fetal ultrasound can be produced from open datasets without new clinical data, lowering a common barrier to classifier development.","The reported FID improvement across all six classes suggests the method narrows the distribution gap between synthetic and real ultrasound more uniformly than the earlier 128×128 generator.","Synthetic pretraining followed by real fine-tuning becomes a concrete training recipe, with the paper's ensemble accuracy of 93.36% above the 92.32% real-data-only baseline.","Because the clinician study still rates synthetic images lower in realism, the immediate practical role of these images is data augmentation, not diagnostic replacement."],"supporting_citations":[{"why":"Supplies the six-class fetal plane database that anchors training, evaluation, and the downstream classification task.","marker":"[3]"},{"why":"Defines the prior lower-resolution synthesis baseline and the classifier fine-tuning protocol this work compares against.","marker":"[22]"},{"why":"Provides the EDM2 diffusion architecture on which the generative model is built.","marker":"[15]"},{"why":"Supplies the autoguidance technique used to improve generated-image quality.","marker":"[14]"},{"why":"Adds the largest auxiliary ultrasound phantom dataset to the multi-dataset training mix.","marker":"[19]"},{"why":"Defines the Fréchet Inception Distance metric used to measure image fidelity.","marker":"[9]"},{"why":"One of the three classifiers whose ensemble accuracy is the downstream evaluation.","marker":"[8]"},{"why":"One of the three classifiers whose ensemble accuracy is the downstream evaluation.","marker":"[11]"},{"why":"One of the three classifiers whose ensemble accuracy is the downstream evaluation.","marker":"[25]"}],"fun_headline_variants":["EDM2 synthetic fetal ultrasound beats real data at 93.36% accuracy","Synthetic fetal ultrasound: 512px diffusion lifts accuracy to 93.36%","Diffusion model generates 512px fetal ultrasound with 93.36% accuracy","High-res synthetic fetal scans rival real data: 93.36% ensemble accuracy"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The headline result depends on the untested assumption that the generated images are new synthetic samples rather than near-copies of the real training images, a risk the paper itself identifies in Section 2 but does not check.","fun_headline_variants_meta":{"raw":{"variants":["EDM2 synthetic fetal ultrasound beats real data at 93.36% accuracy","Synthetic fetal ultrasound: 512px diffusion lifts accuracy to 93.36%","Diffusion model generates 512px fetal ultrasound with 93.36% accuracy","High-res synthetic fetal scans rival real data: 93.36% ensemble accuracy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000937,"raw_usage":{"total_tokens":3981,"prompt_tokens":892,"completion_tokens":3089,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":508,"completion_tokens_details":{"reasoning_tokens":3001}},"tokens_in":508,"tokens_out":3089,"duration_ms":19085,"temperature":1.0,"reasoning_tokens":3001,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-08T12:27:14.145986+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A nearest-neighbor or perceptual-similarity search from each generated image back to the real FETAL PLANES training set would settle the memorization question; if a substantial share of synthetic images are near-duplicates, the 93.36% fine-tuning accuracy cannot be credited to novel synthetic data.","supporting_citations":[{"cited_title":"IET Image Processing19(1), e70151 (2025)","cited_arxiv_id":null,"evidence_quote":"Defines the prior lower-resolution synthesis baseline and the classifier fine-tuning protocol this work compares against."},{"cited_title":"IEEE Access11, 58308–58317 (2023)","cited_arxiv_id":null,"evidence_quote":"Adds the largest auxiliary ultrasound phantom dataset to the multi-dataset training mix."},{"cited_title":"Advances in neural information processing systems30(2017) EDM2-Based Fetal Ultrasound Generation 5","cited_arxiv_id":null,"evidence_quote":"Defines the Fréchet Inception Distance metric used to measure image fidelity."}],"review_version":1}