{"id":"25110f86-fdc3-4711-9676-464c0e383047","arxiv_id":"2411.18926","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Synthetic images from diffusion models, used as pre-training data, substantially improve polyp localization when real data is scarce, with the benefit shrinking as more real images are added.","lead":"Researchers trained diffusion models to generate synthetic colonoscopy images with polyp location annotations, and used them to pre-train a YOLO-based polyp detector before fine-tuning on small sets of real images. The study asks how much real data is needed when synthetic pre-training is available, reporting large gains at 10 to 100 real images.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The claimed benefit of diffusion-generated pre-training is confounded by the absence of an equivalent real-data pre-training control: the generator itself is trained on tens of thousands of labeled real frames, so the experiment does not test whether synthetic data, rather than extra exposure to…","rationale":"I read the paper's strongest claim as a causal one: that pre-training on diffusion-generated colonoscopy images, rather than merely training on more data, improves low-data polyp localization. The reported A-vs-B comparison does support the weaker statement that adding a synthetic pre-training stage before fine-tuning helps relative to no such stage. But it does not isolate the synthetic nature of that pre-training, because modality B also receives far more total training and, indirectly, access to the same real source distribution used to fit the generator. The reader's weakest assumption was about external validity and overlaps between training and deployment data. That is a legitimate concern, but the more immediate logical gap is internal: the experiment never compares synthetic pre-training with an equivalent real-data pre-training control. Since the generative models are trained on tens of thousands of real labeled frames, the low-data setting is not as low as the narrative suggests. This concern does not require external facts about dataset overlap to matter; it follows from the paper's own experimental design. I do not think it overturns the verdict category, because the basic empirical finding that modality B helps is plausible and the paper is already CONDITIONAL, but the condition should include running the real-data pre-training control before attributing the gain to diffusion-based augmentation.","tokens_in":9587,"tokens_out":10247,"duration_ms":99064,"concrete_test":"Run a control: modality C, pre-train YOLOv9 for 13 epochs on 80,000 real labeled colonoscopy frames drawn from the same source pool used to train the diffusion models (e.g., LDPolyp, SUN, PolypGEN, and BKAI), then fine-tune on the identical n-image subsets with the same three seeds and evaluate on the same four benchmarks. If modality C meets or exceeds modality B at n=50 and n=100, the diffusion-specific contribution is unsupported and the headline should be reframed as 'more pre-training data helps'; if B clearly outperforms C, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 6 compares modality A (train on n real images only) with modality B (13 epochs on 80,000 synthetic images, then fine-tune on the same n real images). Both start from the same ImageNet-pretrained YOLOv9 weights. The reported 2-4x mAP gain at n=50/100 therefore conflates two variables: the use of synthetic content and the addition of a large, domain-related pre-training stage. Section 5 shows the five generative models were trained on 17,360 deduplicated LDPolyp frames plus SUN, PolypGEN, and BKAI, on the order of 36,000 labeled real frames. If those real frames had instead been used for the 13-epoch pre-training in a modality C control, the comparison would isolate whether the diffusion model is necessary. The paper's own conclusions frame the result as a property of synthetic augmentation, but this control is missing. This is load-bearing because the practical claim that generated data can substitute for scarce real annotations requires showing that the synthetic data is not merely a proxy for 'more real data, plus a longer training schedule.' The low-data framing is also weakened: acquiring enough labeled frames to train the generator is itself a substantial data-collection effort.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies whether synthetic colonoscopy images generated by conditional latent diffusion models can serve as a pre-training corpus for a YOLOv9-based polyp localizer when only small amounts of real annotated data are available. Five generative training variants are built from four public datasets, and 80,000 synthetic images are generated for each variant. The authors compare modality A (training only on 10--1000 real images from LDPolyp, SUN, and PolypGEN) with modality B (pre-training on 80,000 synthetic images, then fine-tuning on the same real subsets), evaluating mean average precision on Kvasir-SEG, ETIS-Larib, POLAR, and KUMC. They report that modality B gives large gains in the low-data regime, with about 2--4x mAP at 50 and 100 real images, and that gains diminish as the real dataset grows.","tokens_in":9826,"tokens_out":6263,"duration_ms":59228,"significance":"If the effect is real, the paper provides a practical recipe for improving polyp localization when annotations are scarce, and it surveys several diffusion training strategies in one comparison. The study has notable strengths: it evaluates on four external benchmarks, uses three seeds, considers five generative variants, and reports FID, IS, and precision/recall. However, the reported numbers are not sufficient to establish the strength of the effect: numeric mAP values and error bars are missing, and, more importantly, the experimental design lacks a control that isolates the contribution of synthetic content from the contribution of a large domain-related pre-training stage. The overlap between the generative training data and the real fine-tuning subsets further limits what can be concluded about low-data deployment.","major_comments":[{"comment":"The modality A versus modality B comparison conflates synthetic content with pre-training scale. Modality B adds 13 epochs over 80,000 synthetic images before fine-tuning, while modality A starts from ImageNet weights; this is not only a test of synthetic data but also of additional domain-specific exposure. Because the generator was trained on roughly 36,000 real labeled frames from the same datasets (Sections 4--5), the improvement could in principle be reproduced by pre-training on those real frames. I ask for a modality C control: pre-train YOLOv9 on real frames from LDPolyp/SUN/PolypGEN (with a matching number of gradient updates and image resolution), then fine-tune on the same small subsets and evaluate on the same benchmarks. If modality C matches modality B, the paper's conclusion that the diffusion model is the active ingredient would not be supported; if it is worse, the conclusion would be strengthened.","section":"§6, Figure 2"},{"comment":"No numeric table or error bars are provided. Figure 2 plots averaged curves, and the quantitative claims are stated only qualitatively: '2 to 4 times' at 50/100 images, '1.1 to 1.5 times' at 250+ images, and 'plateaus' on Kvasir-SEG and ETIS-Larib. Please report mean and standard deviation over the three seeds for each condition (dataset, subset size, modality, synthetic variant), and indicate whether the differences are statistically significant. Without these numbers, the reader cannot judge the size or reliability of the reported gains or reproduce them from the paper.","section":"§6, Figure 2"},{"comment":"The fine-tuning subsets are sampled from the same public datasets used to train the generative models. As a result, the experiments measure the benefit of synthetic pre-training when the target domain is already represented in the generator's training data; near-duplicates of the real fine-tuning images may even appear in the generated set (Section 5). The paper should either use fine-tuning data from acquisition sources not seen by the generator, or provide a memorization/duplication analysis and explicitly discuss how the result would transfer to a site with a different acquisition protocol. This is especially relevant because the title's 'low data regime' framing understates the fact that training the generator itself required tens of thousands of labeled frames.","section":"§4 vs §6"}],"minor_comments":[{"comment":"There is a typo, 'numbwer', and the DDPM loss equation is incomplete as typeset, with a missing closing bracket and an unspecified expectation; please correct the formula.","section":"Section 3"},{"comment":"The five modality B variants are distinguished only by color, which is difficult to read in grayscale or for color-blind readers; please use distinct line styles or marker shapes and add a dedicated legend entry for modality A.","section":"Figure 2"},{"comment":"The formula for the number of real-data epochs, 10000/(bs·n), is under-explained; please state the resulting epoch counts for each n and justify the schedule, for example by stating that it keeps the number of gradient updates roughly constant.","section":"Section 6"},{"comment":"A summary table of the five generative variants (input resolutions, training data, sampling steps, number of generated images) would make the experiments easier to follow; currently the descriptions appear only as bullets in the text.","section":"Section 5"},{"comment":"The deduplication threshold for LDPolyp is not reported; please provide the threshold value or at least the resulting neighborhood-graph statistics so that the preprocessing is reproducible.","section":"Section 4"},{"comment":"The sentence 'Denoising Diffusion models is a promising Deep Learning technique' contains a subject-verb agreement error; please revise.","section":"Abstract"}],"recommendation":"major_revision","confidential_remarks":"The main technical issue is the missing real-data pre-training control; I would like to see that experiment before publication. I also note that the paper does not release code or the generated synthetic datasets, which limits reproducibility, but the lack of the control is the primary reason for my recommendation."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper is a solid empirical exercise in an established line, but the headline number—synthetic pretraining giving 2–4× mAP at 50–100 real images—is not backed by the experiment as run. The missing control is the elephant in the room: modality B gets 13 epochs of pretraining on 80k synthetic images, while modality A only trains on the small real subset. The generator was trained on ~36k labeled real frames from the same source datasets. Had those real frames been used for the same 13-epoch pretraining (or any real-data pretraining from the same domain), the comparison would isolate whether synthetic data is doing anything beyond \"more domain data plus a longer training schedule.\" The paper does not provide that control, so the causal claim about synthetic data is unestablished.\n\nThat said, the paper is not without merit. It systematically compares five LDM training strategies (VAE upscaling, finetuning, alternate batch, alternate epoch, mixed) and finds that all five yield nearly identical downstream performance, despite large differences in FID/IS/precision/recall. That is a useful negative result: generative model quality metrics are not predictive of downstream utility, and the simplest training method suffices. The scaling study across 10–1000 real images on four evaluation datasets is also a sensible way to map the benefit curve.\n\nThe execution has soft spots beyond the missing control. The paper reports only plots, with no numeric mAP tables or error bars, even though it ran three seeds. The claim of \"small variation between initializations\" is not shown. The overlap between the generative training data and the transfer-learning subsets (LDPolyp, SUN, PolypGEN) means the evaluation is not testing adaptation to a genuinely new acquisition protocol. And the low-data framing is slightly misleading: to use this pipeline, you need a large labeled dataset to train the generator in the first place. The paper does acknowledge that the gain diminishes as real data grows, but it does not discuss the cost of obtaining the generator's training data.\n\nThe core observation—pretraining on off-distribution synthetic data helps a lot when real data is scarce—is plausible and likely reproducible in spirit, but the exact magnitudes need to be re-evaluated with a proper control. This deserves peer review, but the authors should be asked to add a modality C pretrained on real frames, report numeric results with variances, and discuss the data cost of training the generator.\n\nRecommendation: send it to serious peer review with strong revision requests. It is worth engaging with, but not as-is.","headline":"Useful empirical comparison of diffusion-based polyp generation, but the central synthetic-data claim lacks a real-data pretraining control.","tokens_in":10382,"tokens_out":6175,"would_cite":false,"duration_ms":50690,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Pre-training a YOLOv9 polyp detector on 80,000 diffusion-generated colonoscopy frames, then fine-tuning it on as few as 50–100 real frames, roughly doubles to quadruples localization mAP on four external benchmarks.","keywords":["diffusion models","polyp localization","low-data regime","transfer learning","medical imaging","data augmentation","latent diffusion models","object detection"],"falsifier":"Take a colonoscopy dataset collected at a site not represented among LDPolyp, SUN, PolypGEN, or BKAI; train a detector with and without the 80,000-image synthetic pre-training, fine-tuning both on the same small real subset from that new site. If the pre-training gain drops to near zero on out-of-distribution target data, the claim that synthetic pre-training broadly reduces real-data needs would be falsified.","tokens_in":9393,"feed_emoji":"🩺","tokens_out":8384,"duration_ms":65359,"temperature":0.7,"pith_summary":"The paper asks how little real annotated data a polyp-localization detector can get away with if it is first pre-trained on synthetic colonoscopy images. It claims that a YOLOv9 model pre-trained on 80,000 frames generated by a mask-conditioned latent diffusion model, then fine-tuned on only 50–100 real labeled frames, achieves roughly two to four times the mean average precision of the same model trained only on real data. The benefit is largest in the very-low-data regime and shrinks as real data grows, but it persists up to 1000 real images on several benchmarks. If this holds, generative pre-training is a practical way to reduce the annotation burden in medical imaging.","feed_headline":"Pre-training on synthetic colon images quadruples low-data mAP","feed_subtitle":"Tiny real datasets (50–100 images) plus diffusion-generated pre-training beat real-only training by 2–4x in mAP.","key_machinery":"The engine of the argument is a conditional Latent Diffusion Model (LDM): the fixed SDXL VAE maps colonoscopy frames into a smaller latent space, and a binary polyp mask derived from the bounding box is appended as an extra latent channel, so the denoising U-Net learns to generate image and mask together. The paper trains this generator with five recipes that mix the heterogeneous source datasets (LDPolyp, SUN, PolypGEN, BKAI) at different resolutions, samples 80,000 synthetic image–mask pairs per recipe, and uses them as pre-training data for a YOLOv9 object detector that is then fine-tuned on small real subsets.","core_discovery":"On the paper's terms, the central discovery is that synthetic pre-training works: conditioning a latent diffusion model on binary polyp masks lets it generate realistic colonoscopy frames with their localization annotations, and those synthetic frames are an effective pre-training corpus for YOLOv9 in the low-data regime. Across four external polyp-detection benchmarks, the modality that first trains on 80,000 generated images and then fine-tunes on a small random subset of real data (10–1000 images) consistently outperforms training on the real subset alone, with the largest relative gains at 50 and 100 real images (about 2–4x mAP). The paper also reports that the five different diffusion training strategies it compares produce a narrow band of downstream performance, and that generative quality scores such as FID or Inception Score do not predict which synthetic dataset helps most.","pith_inferences":["A natural test of the mechanism is to fine-tune the trained generative model to a target hospital's colonoscopy videos (e.g., via LoRA) and check whether the real-data threshold drops further, as the authors suggest as future work.","The same mask-conditioned pre-training recipe could plausibly transfer to other scarce-annotation localization tasks in endoscopy (e.g., Barrett's esophagus or capsule endoscopy), provided the synthetic images are conditioned on the appropriate structure.","The results leave open whether the gain comes from the diversity of the four public datasets or from the diffusion model's ability to interpolate between them; ablating per-source diversity would separate these.","A direct comparison against classical augmentation (flips, rotations, CutMix) on the same low-data subsets would quantify how much of the gain is specifically due to generative data rather than to more training samples."],"forward_implications":["With as few as 25–50 real images, a detector pre-trained on generated frames reaches mAP comparable to training on far larger real collections, suggesting a direct trade-off between data collection and synthetic pre-training.","The pre-training advantage persists but narrows as real data grows, so the decision to collect more real data can be weighed against generating synthetic data at near-zero marginal annotation cost.","Since all five generative training strategies behave similarly, the simplest recipe (VAE upscaling to 640×640) is sufficient for producing useful pre-training data.","The weak link between image-quality metrics and downstream detector accuracy implies that FID and Inception Score are not reliable guides for selecting a synthetic dataset for localization pre-training."],"supporting_citations":[{"why":"Supplies the latent diffusion architecture (LDM) that generates the synthetic colonoscopy images.","marker":"[21]"},{"why":"Defines the DDPM denoising objective used to train the diffusion models.","marker":"[7]"},{"why":"The YOLOv9 object detector whose low-data polyp localization performance is measured.","marker":"[25]"},{"why":"LDPolypVideo, the largest source dataset for training the generative models and for the real-data subsets.","marker":"[15]"},{"why":"SUN colonoscopy video database, another source for generative training and real-data subsets.","marker":"[17]"},{"why":"PolypGEN multicenter dataset, a source of heterogeneous images for generative training and real subsets.","marker":"[1]"},{"why":"Kvasir-SEG, one of the four external benchmarks on which mAP is evaluated.","marker":"[11]"},{"why":"ETIS-Larib, one of the four external polyp-detection benchmarks.","marker":"[23]"},{"why":"POLAR, one of the four external benchmarks used for zero-shot evaluation.","marker":"[8]"},{"why":"KUMC, the fourth external benchmark used to assess localization performance.","marker":"[13]"}],"fun_headline_variants":["Synthetic pre-training lifts polyp mAP 2–4x on tiny datasets","Diffusion-generated colon frames beat real-only low-data training","Fake colon images, real mAP gains for polyp localization","For polyp detection, synthetic pre-training wins with 50 real images"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The real training subsets and the generative model's training data are drawn from the same public colonoscopy datasets, so the measured benefit of synthetic pre-training depends on the target domain resembling those datasets; a deployment site with different equipment, resolution, or polyp appearance could see the advantage shrink or disappear.","fun_headline_variants_meta":{"raw":{"variants":["Synthetic pre-training lifts polyp mAP 2–4x on tiny datasets","Diffusion-generated colon frames beat real-only low-data training","Fake colon images, real mAP gains for polyp localization","For polyp detection, synthetic pre-training wins with 50 real images"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000275,"raw_usage":{"total_tokens":1641,"prompt_tokens":940,"completion_tokens":701,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":556,"completion_tokens_details":{"reasoning_tokens":626}},"tokens_in":556,"tokens_out":701,"duration_ms":6937,"temperature":1.0,"reasoning_tokens":626,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T10:44:07.093064+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a colonoscopy dataset collected at a site not represented among LDPolyp, SUN, PolypGEN, or BKAI; train a detector with and without the 80,000-image synthetic pre-training, fine-tuning both on the same small real subset from that new site. If the pre-training gain drops to near zero on out-of-distribution target data, the claim that synthetic pre-training broadly reduces real-data needs would be falsified.","supporting_citations":[{"cited_title":"High-resolution image synthesis with latent diffusion models","cited_arxiv_id":null,"evidence_quote":"Supplies the latent diffusion architecture (LDM) that generates the synthetic colonoscopy images."},{"cited_title":"Yolov9: Learning what you want to learn using programmable gradient information, 2024","cited_arxiv_id":null,"evidence_quote":"The YOLOv9 object detector whose low-data polyp localization performance is measured."},{"cited_title":"LDPolypVideo benchmark: a large-scale colonoscopy video dataset of diverse polyps","cited_arxiv_id":null,"evidence_quote":"LDPolypVideo, the largest source dataset for training the generative models and for the real-data subsets."},{"cited_title":"Development of a computer-aided detection system for colonoscopy and a publicly accessible large colonoscopy video database (with video)","cited_arxiv_id":null,"evidence_quote":"SUN colonoscopy video database, another source for generative training and real-data subsets."},{"cited_title":"A multi-centre polyp detection and segmentation dataset for generalisability assessment","cited_arxiv_id":null,"evidence_quote":"PolypGEN multicenter dataset, a source of heterogeneous images for generative training and real subsets."},{"cited_title":"Kvasir-seg: A segmented polyp dataset","cited_arxiv_id":null,"evidence_quote":"Kvasir-SEG, one of the four external benchmarks on which mAP is evaluated."},{"cited_title":"To- ward embedded detection of polyps in wce images for early diagnosis of colorectal cancer","cited_arxiv_id":null,"evidence_quote":"ETIS-Larib, one of the four external polyp-detection benchmarks."},{"cited_title":"Computer-aided diagnosis for optical diagnosis of diminutive colorectal polyps including sessile serrated lesions: a real-time comparison with screening endoscopists","cited_arxiv_id":null,"evidence_quote":"POLAR, one of the four external benchmarks used for zero-shot evaluation."},{"cited_title":"Colonoscopy polyp detection and classification: Dataset creation and comparative evaluations","cited_arxiv_id":null,"evidence_quote":"KUMC, the fourth external benchmark used to assess localization performance."}],"review_version":1}