{"id":"9b4242f1-4657-4b80-8aef-a5f5fe482146","arxiv_id":"2411.11016","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"TSG classifies real versus synthetic images by feeding the image at a fixed noise timestep through a frozen diffusion U-Net and classifying its predicted noise map.","lead":"A new detector for AI-made images uses the noise predictor inside a diffusion model as a feature extractor, then classifies those features with a standard network. It claims high accuracy and roughly 10 times faster detection than a leading reconstruction-based method on the GenImage benchmark.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Universal claim depends on the ImageNet-ImageNet overlap: real test images are in-distribution for the frozen U-Net, so cross-generator accuracy may not transfer to non-ImageNet real photos.","rationale":"The reader's weakest-assumption judgment is right on target. The method is concrete and reproducible, and the GenImage experiments are a credible demonstration of cross-generator detection within a benchmark. However, the paper's title and abstract promise a universal detector, and that promise is currently supported only by experiments where the real half of the data is drawn from the same distribution as the feature extractor's training set. This is not an accusation of circularity; it is a well-defined external-validity gap: the U-Net never learned a general 'real image' concept, only ImageNet's density. There is no evidence that features which separate ImageNet originals from generated images will separate COCO photographs from generated images. An alternative concern is that t=0 is outside the U-Net's trained timestep range, making the method's theoretical framing questionable; but since the empirical performance is high, that issue would not change the practical verdict. The distribution overlap, by contrast, determines whether the headline accuracy means what it claims. I therefore recommend keeping the CONDITIONAL verdict, with the explicit condition that the authors validate on at least one non-ImageNet real-image source.","tokens_in":9935,"tokens_out":7524,"duration_ms":148427,"concrete_test":"Use the released code (github.com/NuayHL/TimeStepGenerating) with the frozen ADM U-Net and the classifier trained on, say, the ADM (t=0) subset. Extract TSG features for 10,000 real images from COCO-val (or LSUN-val) and for another 10,000 real images from GenImage's validation real set. Run the classifier and measure the real-image false positive rate (real labeled fake) on each set. If the COCO false positive rate is substantially higher than the GenImage rate (e.g., >10 percentage points), the detector's real class is ImageNet-specific, and the universal claim fails. Repeat with a second generator (e.g., SD V1.5) to ensure the effect is on the real class, not the fake class.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is a universal detector, but the empirical setting creates a shortcut risk. The feature extractor is the class-unconditional ADM U-Net trained on ImageNet (Section 4.1). GenImage real images are also ImageNet images. Hence, in every experiment the real class is exactly the extractor's training distribution, while synthetic images are out-of-distribution. The classifier may learn to separate 'familiar ImageNet input' from 'unfamiliar OOD input' rather than a general real-vs-fake boundary. This directly contradicts the paper's stated independence from specific datasets and leaves 'universal' unsupported: no experiment uses real photographs outside ImageNet (e.g., COCO, LSUN, or camera photos). The Grad-CAM and JPEG robustness studies all remain inside ImageNet-based subsets. Consequently, the 94.9% average accuracy is a within-distribution result for the real class, and its external validity is an untested assumption.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Time Step Generating (TSG), a synthetic image detector that uses a pretrained diffusion model U-Net as a fixed feature extractor. For an input image I and a chosen time step t, it computes the noise prediction F = ε_θ(I, t) and feeds F into a ResNet-50 binary classifier. The authors evaluate TSG on five GenImage subsets (BigGAN, VQDM, SD V1.5, ADM, Wukong), report an average cross-generator accuracy of 94.9% at t=0, and measure a roughly 10x speedup over DIRE. They also include JPEG-robustness experiments on three additional subsets and a mixed-training experiment achieving 100.0% accuracy on all test sets.","tokens_in":10144,"tokens_out":3795,"duration_ms":40942,"significance":"The core idea is simple and attractive: instead of running full diffusion inversion and reconstruction, one forward pass through a frozen diffusion U-Net at a small time step yields features that separate real from synthetic images. If the universality claim were supported, this would be a practical and fast detector with strong cross-generator generalization. The paper is also commendable for reporting code availability and for transparently comparing t=0 and t=50. However, the current evidence does not establish universality: the real test images are drawn from ImageNet, which is also the training distribution of the frozen feature extractor, and no experiment uses real photographs from a different distribution. The absence of error bars and the suspicious exactly-100.0% mixed-training result further limit the strength of the empirical claims.","major_comments":[{"comment":"The feature extractor is the class-unconditional ImageNet diffusion model (ADM) released with [7], and the real images in all GenImage subsets are also ImageNet images. Consequently, in every experiment the real class is exactly the extractor's training distribution, while all synthetic images are out-of-distribution. The classifier may therefore learn to separate 'familiar ImageNet input' from 'unfamiliar OOD input' rather than a general real-vs-synthetic boundary. The abstract's claim that TSG 'does not rely on ... specific datasets' is directly contradicted by this setup, and the claimed universality is untested. Please add experiments with real photographs from a different distribution (e.g., COCO, LSUN, or a camera-captured dataset), or substantially temper the universality claim.","section":"Sections 4.1 and 4.3"},{"comment":"All accuracies reported in Table 1 and Table 4 come from a single train/validation split with no error bars or multiple seeds. More importantly, Table 4 reports exactly 100.0% accuracy on all test sets, including the Un-bias sets (Glide, SD V1.4, Midjourney), for a classifier trained on a mix of ADM, SD V1.5, and BigGAN. This perfect transfer is difficult to reconcile with the much lower cross-generator accuracies in Figure 4 (e.g., classifiers trained on BigGAN often perform poorly on diffusion subsets). The authors should explain the protocol, report mean and standard deviation over at least three seeds, and investigate whether a trivial cue (e.g., an averaging artifact in the feature extractor or a leak in the data pipeline) is responsible for the 100.0% numbers.","section":"Tables 1 and 4, Sections 4.5 and 4.8"},{"comment":"The 'unbiased' datasets are constructed by filtering GenImage subsets (Glide, SD V1.4, Midjourney) by a JPEG 'compression rate greater than 96'. However, the real images in these subsets are still ImageNet images, so the robustness experiment only demonstrates invariance to JPEG compression within the same distribution that the feature extractor was trained on. The terminology 'unbiased' is therefore misleading, and the phrase 'compression rate greater than 96' is ambiguous (it could mean quality factor > 96 or file-size ratio). Please clarify the exact JPEG quality parameter and, if the universal-claim revision is pursued, test JPEG robustness on non-ImageNet real images as well.","section":"Section 4.6 and Table 3"},{"comment":"The theoretical justification for why t=0 should be optimal is not fully developed. The score-based argument in Figure 2 posits that real images lie at high probability density and generated images at lower density, but this is not quantified and does not explain the empirical advantage of t=0 over t=50. The paper only compares two values of t and then fixes t=0; a systematic sweep over t (e.g., t=0, 10, 50, 100, 200) would strengthen the claim that the time step is a controllable and meaningful parameter rather than a selected hyperparameter.","section":"Section 3.2 and Figure 2"}],"minor_comments":[{"comment":"There are typos: 'vary challenging' should be 'very challenging', and in the Introduction 'TGS simplifies the approach further' should be 'TSG'.","section":"Abstract and Section 1"},{"comment":"The table formatting is broken; the column headers are unclear. Please present the DIRE and TSG rows with clear labels for the number of sampling steps, batch size, and measured time.","section":"Table 2"},{"comment":"The phrase 'compression rate greater than 96' should be specified precisely, e.g., 'JPEG quality factor > 96' or 'compression ratio > 96%', since it is currently ambiguous.","section":"Section 4.6"},{"comment":"The discussion of Figure 4 would benefit from explicit numbers for the t=50 average accuracy, since Table 1 reports only the aggregated values for t=0 and t=50.","section":"Section 4.4"},{"comment":"The conclusion states the method is '19% better than LaRE2', but the table reports 94.9 vs. 75.6, which is 19.3 percentage points. Please use 'percentage points' to avoid confusion.","section":"Conclusion"}],"recommendation":"major_revision","confidential_remarks":"The main blocker is the ImageNet-to-ImageNet overlap between the feature extractor's training distribution and the real-image test distribution, which undermines the universality claim as currently stated. The 100.0% mixed-training result also raises a red flag that should be investigated before publication. If the authors can add cross-distribution experiments on real photos and resolve the 100.0% anomaly, the paper could be a solid contribution to the deepfake-detection literature."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know about TSG. It is the cleanest version yet of the 'use a diffusion network as a feature extractor' idea: feed the image at a fixed timestep t=0, take the U-Net's predicted noise map as a feature image, and let a ResNet-50 do the classification. On GenImage it gets about 94.9% average cross-generator accuracy, which is about 19 points above LaRE2, and it's roughly 10x faster than DIRE. It is also genuinely new: DIRE uses reconstruction error, LaRE2 uses latent reconstruction error, FakeInversion uses inversion feature maps, and none of them directly classifies the noise prediction at a single timestep.\n\nThe soft spot is the one you'd expect from the title. The paper calls the detector 'universal,' but every real image in every experiment comes from GenImage, whose real images are ImageNet validation photos. The feature extractor is the class-unconditional ADM U-Net released by Dhariwal and Nichol, trained on ImageNet. So the real class is exactly the extractor's training distribution, and the fake class is out-of-distribution for it. The classifier may well be learning 'familiar ImageNet-like' vs 'everything else' rather than a general real-vs-fake boundary. The paper gives an elegant score-based explanation for why noise prediction should differ, but that argument only holds when real images sit near the training manifold. A COCO photograph or a phone snapshot would be out-of-distribution for the U-Net, and the method might flag it as fake. There is no experiment with any real photo outside ImageNet, so the headline claim is an assumption, not a result.\n\nOther concerns are smaller but still real. All accuracies come from a single split with no error bars. The mixed-training experiment in Table 4 reports exactly 100.0% on all test sets, which is either a broken experiment or a miracle, and the paper doesn't reconcile it with the weaker cross-generator results in Section 4.3. The baseline list is thin: GramNet, DIRE, LaRE2, with no FakeInversion or CLIP-based detector. The JPEG robustness study is confined to curated subsets and doesn't test the distribution gap.\n\nWhere does this leave us? The core idea is sound in principle and the engineering is sensible, but the evidence does not support the universal claim. The paper deserves peer review, because the method is novel, simple, and potentially useful, and the experiments can be fixed. I would send it with a request for a non-ImageNet real set, error bars, and a plausible explanation for the 100% result. If you want a sharp example of a benchmark shortcut that undermines a generalization claim, this is a good one for reading group.","headline":"TSG is a fast, clever deepfake detector that works by classifying predicted noise from a frozen diffusion U-Net; the catch is that its 'universal' label rests on an ImageNet-to-ImageNet shortcut no one tests.","tokens_in":10642,"tokens_out":4747,"would_cite":false,"duration_ms":43311,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A fixed, pretrained diffusion U-Net, evaluated on an image at a small timestep, produces noise-prediction features that let a ResNet-50 classify it as real or synthetic with 94.9% average cross-generator accuracy on GenImage.","keywords":["deepfake detection","synthetic image detection","diffusion model","noise prediction","feature extraction","time step","GenImage","ResNet-50"],"falsifier":"Take a set of real photographs that are clearly outside ImageNet's domain (e.g., medical scans or satellite imagery), run them through TSG with a classifier trained on ImageNet real images, and measure accuracy; if it falls to near chance while generated images from the same new domain are still detected, the universality claim fails. Alternatively, rerun the GenImage cross-validation after replacing the real half of the test set with non-ImageNet photographs and check whether the 94.9% average drops.","tokens_in":9752,"feed_emoji":"🔍","tokens_out":4997,"duration_ms":48209,"temperature":0.7,"pith_summary":"The paper proposes Time Step Generating (TSG), a synthetic-image detector that feeds an image through a frozen U-Net from a pretrained diffusion model at a fixed timestep near zero and uses the network's noise prediction as a feature map for a ResNet-50 classifier. The authors claim this single forward pass exposes fine-grained differences between real and generated images without needing the slow inversion-and-reconstruction pipeline of prior methods. On five subsets of the GenImage benchmark the method averages 94.9% accuracy across generators, about 19 points above LaRE2, and runs roughly ten times faster than DIRE. If correct, TSG offers a fast, generator-agnostic way to detect diffusion-generated images.","feed_headline":"One U-Net pass spots AI images at 94.9%","feed_subtitle":"A frozen diffusion model's noise prediction, not slow reconstruction, powers a fast detector that beats LaRE2 by 19 points.","key_machinery":"The central object is the TSG feature $F = \\epsilon_\\theta(I, t)$: the noise prediction of a frozen, class-unconditional ImageNet diffusion U-Net evaluated on the query image $I$ at a fixed timestep $t$ (typically 0). This is a single forward pass, in contrast to reconstruction-based detectors that invert and denoise. The timestep acts as a detail-preservation dial: at $t=0$ the predicted noise retains the finest high-frequency details, which the classifier uses; at $t=50$ some detail is lost and accuracy drops slightly. The paper interprets the mechanism through score-based diffusion: real images sit at high probability density, so the estimated score at their location differs from the score of generated images.","core_discovery":"On the paper's own terms, the central discovery is that the noise estimated by a pretrained diffusion U-Net at a small timestep t encodes a reliable real-versus-synthetic signal. Because real images already lie at high-density regions of the data distribution while generated images sit nearby but off the true manifold, the score (equivalently the predicted noise) at the same timestep differs systematically. TSG defines the feature as $F = \\epsilon_\\theta(I, t)$ and feeds $F$ into ResNet-50 for binary classification; no inversion, no denoising loop, and no task-specific generation model is required. The authors report 94.9% average accuracy across five generators and robustness to JPEG compression.","pith_inferences":["The paper's 'universal' claim is tested only against real images from ImageNet, the same distribution the extractor was trained on; an editorially suggested extension is to measure TSG on real photographs from a different domain (e.g., camera-phone or medical imagery), where the score separation may shrink.","Because TSG exposes the U-Net's internal estimate of noise, the same machinery could be turned toward attribution—matching the noise pattern to the generator's fingerprint—rather than just binary real/fake.","The timestep sweep suggests a calibration strategy for other detectors: search over small t to maximize high-frequency detail, and use larger t to check robustness of a given classifier.","If the score-based interpretation is right, TSG should work with any sufficiently expressive diffusion backbone, not just ADM, which could be tested by swapping in Stable Diffusion or DiT U-Nets."],"forward_implications":["A single fixed extractor can serve multiple generators: classifiers trained on one diffusion model's output transfer to other diffusion models and to BigGAN, so a detector could be deployed without per-generator retraining.","Detection becomes cheap enough for screening: with about a tenth of DIRE's compute, one forward pass through a U-Net plus a ResNet-50 gives a label, making large-scale moderation more practical.","JPEG compression does not break the feature: cross-validation on compressed subsets of Glide, SD V1.4, and Midjourney stays high, so the method is applicable to images as shared online.","Training one classifier on a mix of GAN and diffusion outputs yields near-perfect accuracy on both families and on unseen compressed sets, suggesting a route to a single universal detector.","The timestep t is a tunable knob: t=0 is best for accuracy, while larger t trades detail for slightly lower but still strong performance."],"supporting_citations":[{"why":"Supplies the pretrained class-unconditional ImageNet diffusion model whose U-Net is used as the frozen feature extractor.","marker":"[7]"},{"why":"Defines the DDPM forward and reverse processes and the noise-prediction objective that TSG's feature formulation builds on.","marker":"[1]"},{"why":"Stable Diffusion V1.5 is one of the five GenImage subsets used in training and cross-validation.","marker":"[9]"},{"why":"DIRE is the reconstruction-based baseline that TSG is compared against in accuracy and speed.","marker":"[18]"},{"why":"LaRE2 is the latent reconstruction baseline that TSG outperforms by roughly 19 points on average.","marker":"[19]"},{"why":"GenImage provides the large-scale benchmark, subset splits, and the real-image test set for all reported accuracies.","marker":"[20]"},{"why":"ResNet-50 is the classifier that consumes the TSG feature maps and produces the real/fake decision.","marker":"[35]"},{"why":"BigGAN supplies the GAN-generated subset used to test cross-generator generalization.","marker":"[36]"},{"why":"VQDM supplies one of the diffusion-generated subsets in the cross-validation.","marker":"[37]"},{"why":"Wukong supplies another diffusion-generated subset in the cross-validation.","marker":"[38]"}],"fun_headline_variants":["Diffusion U-Net noise signals real vs synthetic at 94.9%","One noise prediction from diffusion U-Net detects AI images at 94.9%","Universal AI image detector: single diffusion pass, 94.9% accuracy","AI fakes caught by diffusion noise at 94.9% without reconstruction","Fast universal detector: one U-Net noise step spots AI images at 94.9%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method is called universal but the frozen feature extractor was trained on ImageNet and every real image used in the experiments also comes from ImageNet, so the reported accuracy may rely on that distributional overlap and might drop on real photos from other sources.","fun_headline_variants_meta":{"raw":{"variants":["Diffusion U-Net noise signals real vs synthetic at 94.9%","One noise prediction from diffusion U-Net detects AI images at 94.9%","Universal AI image detector: single diffusion pass, 94.9% accuracy","AI fakes caught by diffusion noise at 94.9% without reconstruction","Fast universal detector: one U-Net noise step spots AI images at 94.9%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000953,"raw_usage":{"total_tokens":4053,"prompt_tokens":926,"completion_tokens":3127,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":542,"completion_tokens_details":{"reasoning_tokens":3019}},"tokens_in":542,"tokens_out":3127,"duration_ms":20308,"temperature":1.0,"reasoning_tokens":3019,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T19:00:48.898561+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a set of real photographs that are clearly outside ImageNet's domain (e.g., medical scans or satellite imagery), run them through TSG with a classifier trained on ImageNet real images, and measure accuracy; if it falls to near chance while generated images from the same new domain are still detected, the universality claim fails. Alternatively, rerun the GenImage cross-validation after replacing the real half of the test set with non-ImageNet photographs and check whether the 94.9% average drops.","supporting_citations":[{"cited_title":"Diffusion models beat gans on image synthesis","cited_arxiv_id":null,"evidence_quote":"Supplies the pretrained class-unconditional ImageNet diffusion model whose U-Net is used as the frozen feature extractor."},{"cited_title":"High-resolution image synthesis with latent diffusion models","cited_arxiv_id":null,"evidence_quote":"Stable Diffusion V1.5 is one of the five GenImage subsets used in training and cross-validation."},{"cited_title":"Dire for diffusion-generated image detection","cited_arxiv_id":null,"evidence_quote":"DIRE is the reconstruction-based baseline that TSG is compared against in accuracy and speed."},{"cited_title":"Lareˆ2: Latent reconstruction error based method for diffusion-generated image detection","cited_arxiv_id":null,"evidence_quote":"LaRE2 is the latent reconstruction baseline that TSG outperforms by roughly 19 points on average."},{"cited_title":"Genimage: A million-scale benchmark for de- tecting ai-generated image","cited_arxiv_id":null,"evidence_quote":"GenImage provides the large-scale benchmark, subset splits, and the real-image test set for all reported accuracies."},{"cited_title":"Deep residual learning for image recognition","cited_arxiv_id":null,"evidence_quote":"ResNet-50 is the classifier that consumes the TSG feature maps and produces the real/fake decision."},{"cited_title":"Large scale GAN training for high fidelity natural image synthe- sis","cited_arxiv_id":null,"evidence_quote":"BigGAN supplies the GAN-generated subset used to test cross-generator generalization."},{"cited_title":"Vec- tor quantized diffusion model for text-to-image synthesis","cited_arxiv_id":null,"evidence_quote":"VQDM supplies one of the diffusion-generated subsets in the cross-validation."}],"review_version":1}