{"id":"49455f25-e0fc-4f72-9c35-64417311302d","arxiv_id":"2412.07539","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":2.0,"correctness_risk":"high","formal_verification":"none","parameter_count":2,"one_line_summary":"Diffusion-based reconstruction models are evaluated for image anomaly detection and are claimed to outperform classical methods, but the paper lacks the experimental detail needed to support the claim.","lead":"This paper benchmarks diffusion-based image reconstruction models (DDPM, DiT, DTE) against three classical anomaly detectors on small image datasets and a Mini-ImageNet subset. It claims diffusion models are more scalable and robust, but the reported table and missing experimental details do not fully support that claim.","discovery_kind":"incremental","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table 1 itself contradicts the claim that diffusion methods outperform conventional methods on all datasets, and the sole high-resolution result is an undocumented Mini-ImageNet AUC; the central claim is unsupported.","rationale":"The reader's REJECT verdict is well supported. The paper's central claim has two components: universal superiority of diffusion-based methods over conventional baselines, and scalability to high-resolution datasets. The first is directly contradicted by Table 1: for CIFAR10_7, OCSVM (67.06) beats every diffusion method (DDPM 66.82, DiT 66.30, DTE 52.21), and for MNIST-C_fog, OCSVM (91.09) beats DiT (90.78) and DTE (51.15) even though DDPM is highest. The caption's 'diffusion-based methods outperform conventional methods on all datasets' therefore cannot be true as stated. The second component depends on a single Mini-ImageNet AUC of 0.635, with no protocol details: the subset composition is unspecified, the model size and diffusion timesteps are absent, the anomaly scoring pipeline is not described, and the figure is not legible in the supplied text. The appendix references many additional figures that are not present, which further impairs reproducibility. I agree with the reader's concern about the Mini-ImageNet test, but I see the internal counterexample in Table 1 as the more load-bearing problem because it falsifies the headline claim even if every reported number is accurate. In good faith, the paper does correctly review standard diffusion preliminaries, and some diffusion variants do achieve top scores on several compact datasets, so the direction is not implausible. However, evidence quality is the issue: no code, hyperparameters, error bars, or prior-diffusion-anomaly-detector baselines are provided. The existing REJECT verdict remains appropriate, so my recommendation is UNCHANGED.","tokens_in":9941,"tokens_out":4443,"duration_ms":40376,"concrete_test":"Treat Table 1 as data and compute, for each row, max{IForest, OCSVM, COPOD} versus max{DDPM, DiT, DTE}; if CIFAR10_7 (67.06 versus 66.82) or any other row has conventional > diffusion, the caption's 'all datasets' claim is false. To test scalability, request the exact Mini-ImageNet class composition, DiT/DDPM architecture sizes, inference timesteps, and anomaly scoring code from the authors and rerun Figure 1; if the 0.635 AUC cannot be reproduced under a documented protocol, the scalability conclusion fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"To support the abstract and Table 1 caption, the reported experiments would need to show diffusion-based methods winning or matching conventional methods on every dataset. They do not. Row CIFAR10_7 gives OCSVM 67.06 versus DDPM 66.82, DiT 66.30, and DTE 52.21; row MNIST-C_fog gives OCSVM 91.09 with DiT 90.78 and DTE 51.15 below it, only DDPM 92.60 above. Thus the strongest claim is internally contradicted by the paper's own numbers. The high-resolution scalability claim rests entirely on Figure 1 and one AUC value (Ours = 0.635) for 'Mini-ImageNet', with no subset composition, model size, number of diffusion timesteps, anomaly-score definition, or calibration procedure; the paper says categorical subsets were used but not which classes, and the appendix lists many figures that are not present in the text. No code, hyperparameters, or error bars are provided. The evidence supports at most that some diffusion variants win on some compact datasets, not universal superiority, scalability, and robustness.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper benchmarks reconstruction-based diffusion anomaly detectors (DDPM, DiT, and DTE) against classical baselines (Isolation Forest, OCSVM, COPOD) on several compact image datasets from ADBench (CIFAR-10, MNIST-C, SVHN, FashionMNIST, MVTec-AD) and one high-resolution Mini-ImageNet subset, reporting AUC-ROC scores. The authors claim that diffusion-based methods exhibit superior adaptability, scalability, and robustness, with the Table 1 caption stating that diffusion methods outperform conventional methods on all datasets.","tokens_in":10130,"tokens_out":4800,"duration_ms":44649,"significance":"If the empirical claims were supported, the paper would offer practical evidence that diffusion reconstruction models are a reliable off-the-shelf choice for image anomaly detection, including high-resolution industrial and natural image settings. The paper does provide a multi-dataset comparison and uses a standard metric, which are useful starting points. However, the central claims are internally contradicted by the paper's own table, no uncertainty quantification is provided, and the sole high-resolution result is undocumented. The absence of code, hyperparameters, and a reproducible protocol means the benchmark cannot currently be verified or reused.","major_comments":[{"comment":"The caption 'Diffusion-based methods outperform conventional methods on all datasets' is contradicted by the table itself. On CIFAR10_7, OCSVM (67.06) beats DDPM (66.82), DiT (66.30), and DTE (52.21); on MNIST-C_fog, OCSVM (91.09) beats DiT (90.78). Additionally, DTE falls below chance (50%) on many rows, including MVTec-AD_capsule (44.15), MVTec-AD_zipper (49.08), and MVTec-AD_bottle (50.23). The Abstract's claim of 'superior adaptability, scalability, and robustness' is therefore not supported by the reported numbers.","section":"Section 4.4, Table 1"},{"comment":"No error bars, standard deviations, confidence intervals, or significance tests are reported for any AUC value. Since several comparisons are within one point (e.g., CIFAR10_4: DDPM 77.42 vs OCSVM 76.93; MVTec-AD_leather: DDPM 99.38 vs OCSVM 99.39), the differences may be pure noise. The claim of robustness across noisy or adversarial datasets is not statistically established.","section":"Section 4.4, Table 1"},{"comment":"The high-resolution scalability claim rests entirely on a single reported value, 'Ours = 0.635' for Mini-ImageNet. The paper does not specify which architecture (DDPM, DiT, or DTE) is used, the subset composition, model size, number of diffusion timesteps, the definition of reconstruction error, or how the anomaly score is calibrated. Without these details, this central scalability result is not verifiable.","section":"Section 4.3, Figure 1"},{"comment":"COPOD returns exactly 50.00% on 18 of the 28 compact datasets, which strongly suggests either a degenerate anomaly score or a faulty evaluation setup. The paper offers no explanation for this behavior, and this casts doubt on the fairness and validity of the baseline comparison.","section":"Section 4.4, Table 1"},{"comment":"The experimental protocol is not reproducible: no training hyperparameters (learning rate, batch size, epochs, optimizer), no architecture details (U-Net depth, DiT patch size), no noise schedule, and no code are provided. Furthermore, the manuscript lists many auxiliary figures (Ablation1.png, Experiment1.png through Experiment15.png, training_protocol.png, vae_intuition_diagram.png) that are never cited or discussed in the body, so any supporting evidence they might contain is not accessible to the reader.","section":"Sections 3.5 and 4.4"}],"minor_comments":[{"comment":"There is a typo in the first sentence: 'f or' should be 'for'.","section":"Abstract"},{"comment":"The term DTE is never defined, and reference [5] is cited for both DiT and DTE, but [5] is the DDIM paper by Song et al., not a source for Diffusion Transformers or Denoising Transformer Embeddings.","section":"Section 2.4 and References"},{"comment":"The citation for 'Dickstein et al.' is misattributed: the title 'Deep generative image models using a Laplacian pyramid of adversarial networks' is the Denton et al. paper, whereas the relevant Sohl-Dickstein et al. work is 'Deep Unsupervised Learning using Nonequilibrium Thermodynamics'.","section":"Section 3.1, Reference [28]"},{"comment":"FashionMNIST appears in Table 1 (e.g., FashionMNIST_6, FashionMNIST_3, FashionMNIST_9) but is not described in the dataset enumeration in Section 4.1.","section":"Section 4.1"},{"comment":"The paper states that 'We have performed experiments on two categories of datasets' and calls MVTec-AD a 'compact' dataset, but MVTec-AD images are not small-scale (they are typically hundreds of pixels); this categorization is inconsistent with the later claim that Mini-ImageNet is the only high-resolution dataset.","section":"Section 4.2"}],"recommendation":"reject","confidential_remarks":"The manuscript's central empirical claim is contradicted by its own Table 1, the high-resolution result is undocumented, and the lack of error bars and protocol details prevents verification. The reference list also contains multiple mismatches. As it stands, the paper does not meet the bar for publication, and the issues are systemic rather than local."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a fairly routine empirical comparison of DDPM, DiT, and DTE against IForest, OCSVM, and COPOD on ADBench-style datasets plus a Mini-ImageNet subset. The idea is not new—reconstruction-based diffusion anomaly detection is already in their own citations—and the paper adds no architecture, theory, or dataset. What it does do is run a reasonably wide set of datasets, including several MVTec-AD categories, and reports AUC scores for each method. That breadth is useful for a reader who wants a quick sanity check on how these families compare.\n\nThe problems are hard to miss. The abstract and the Table 1 caption claim diffusion methods outperform conventional methods on all datasets. The table itself disagrees: on CIFAR10_7, OCSVM gets 67.06 versus DDPM 66.82 and DiT 66.30; on MNIST-C_fog OCSVM beats DiT and DTE; DTE falls below random on several MVTec categories (e.g., capsule 44.15). So the central claim is contradicted by the paper's own numbers. The high-resolution scalability claim rests entirely on Figure 1 and one AUC value for \"Ours\" on Mini-ImageNet, with no subset composition, no model size, no timesteps, no anomaly-score definition, and no calibration. That is not reproducible.\n\nThere are also no error bars or significance tests anywhere, despite the fact that differences of fractions of an AUC point are treated as meaningful. The reference list contains several misattributions: [4] and [5] are both given the same DDIM title, [28] is not Sohl-Dickstein's paper, and [31] is not the ViT paper. The appendix lists many figures that never appear in the text.\n\nThe paper is not a waste of time as a rough guide to what these methods do on standard benchmarks, but as written it does not support its conclusions. It needs a major overhaul: correct the overclaims, provide protocol details for the high-resolution experiment, add error bars, fix the references, and ideally compare against prior diffusion-based AD methods (AnoDDPM, DiAD) rather than only classical baselines. I would not send it to peer review in its current form; a desk reject with a note to resubmit after substantial revision seems right.","headline":"A routine diffusion-anomaly-detection benchmark whose own table contradicts its claim of universal superiority, and whose high-resolution result is an undocumented single number.","tokens_in":10688,"tokens_out":2471,"would_cite":false,"duration_ms":24296,"reading_group":"no","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Diffusion-based generative models can serve as strong anomaly detectors by scoring reconstruction error, and the paper claims they beat classical methods like Isolation Forest, One-Class SVM, and COPOD on all tested image datasets.","keywords":["anomaly detection","diffusion models","DDPM","Diffusion Transformer","reconstruction error","unsupervised learning","AUC-ROC","scalability"],"falsifier":"Re-run the Mini-ImageNet experiment with the subset composition, model size, number of inference timesteps, and anomaly-score calibration documented, and check whether a DiT-based detector still exceeds OCSVM and IForest in AUC-ROC; also check every row of Table 1 to see whether at least one diffusion variant beats every classical method, since any counterexample row (for instance CIFAR10_7) would refute the 'all datasets' claim as stated.","tokens_in":9724,"feed_emoji":"🔍","tokens_out":9215,"duration_ms":76415,"temperature":0.7,"pith_summary":"This paper argues that diffusion-based generative models—models that learn to add noise to data and then reverse the process—can act as strong unsupervised anomaly detectors by scoring how well a test image can be reconstructed after training only on normal images. The authors claim that this reconstruction-error signal outperforms classical detectors such as Isolation Forest, One-Class SVM, and COPOD on all the datasets they test, including small image benchmarks and a high-resolution ImageNet subset. If the claim holds, practitioners could adopt diffusion models as an off-the-shelf choice for industrial defect detection and other settings where anomalies are rare and normal data is plentiful.","feed_headline":"Diffusion models outrank classical anomaly detectors","feed_subtitle":"Reconstruction error from DDPMs and Diffusion Transformers flags defects and outliers on compact and high-res images.","key_machinery":"The central object is the reconstruction error of a trained diffusion model. The forward process corrupts an input $x_0$ by $x_t = \\sqrt{\\bar{\\alpha}_t}x_0 + \\sqrt{1-\\bar{\\alpha}_t}\\epsilon_t$, and a U-Net (for DDPM) or a Vision Transformer (for DiT) learns to predict the added noise $\\epsilon_t$ under the simple objective $\\mathcal{L}_{\\text{simple}}(\\theta) = \\|\\epsilon_\\theta(x_t) - \\epsilon_t\\|_2^2$. At test time, the model denoises a noisy version of the input, and the magnitude of the difference between the original and the reconstruction serves as the anomaly score: normal inputs reconstruct well, anomalies do not.","core_discovery":"The core discovery is that the gap between a normal image and its diffusion reconstruction is consistently smaller than the gap for an anomalous image, and that this separation is strong enough to rank anomalies more accurately than classical one-class classifiers. The paper reports AUC-ROC numbers for DDPM, Diffusion Transformer (DiT), and Denoising Transformer Embedding (DTE) variants across compact image sets and a Mini-ImageNet subset, and interprets the results as evidence that the iterative denoising trajectory learns a rich model of the normal distribution. In the high-resolution scalability test, the DiT-based detector reaches an AUC of 0.635, ahead of One-Class SVM (0.594) and Isolation Forest (0.584), which the authors read as demonstrating that diffusion models scale to high-dimensional data without the performance plateau seen in traditional methods.","pith_inferences":["A re-run of the Mini-ImageNet experiment with a fully documented protocol—subset composition, model size, inference timesteps, and anomaly-score calibration—would settle whether the scalability conclusion generalizes beyond the single reported AUC of 0.635.","The 'outperform on all datasets' claim depends on how 'outperform' is defined; checking each row of Table 1 to see whether at least one diffusion variant beats every classical method would clarify whether the claim holds literally.","The per-dataset spread among DDPM, DiT, and DTE suggests that the choice of diffusion architecture matters as much as the choice between diffusion and classical methods; selecting the best variant per dataset could change the overall comparison.","A natural extension of the reconstruction-error mechanism is to measure how the anomaly score responds to controlled perturbations of normal inputs, which would probe how sharply the learned normal manifold is separated."],"forward_implications":["Practitioners can use diffusion-based reconstruction error as an unsupervised anomaly score without needing anomaly labels, only a training set of normal images.","The reported gains over Isolation Forest, One-Class SVM, and COPOD suggest diffusion models are a viable drop-in replacement for classical detectors on compact image benchmarks.","The Mini-ImageNet result implies that scaling to 224×224 images does not erase the advantage of diffusion methods, pointing to DiT-style architectures for real-world high-resolution deployment.","Because reconstruction error is tied to the learned normal distribution, the same training recipe applies across datasets with only the normal class available, supporting domain transfer to industrial and medical imaging."],"supporting_citations":[{"why":"Defines the forward and reverse diffusion process the paper uses as its reconstruction machinery.","marker":"[29]"},{"why":"Supplies the simple denoising objective and covariance optimization used to train the noise-prediction network.","marker":"[34]"},{"why":"Motivates the Diffusion Transformer (DiT) architecture for high-resolution image modeling, the basis of the scalability claim.","marker":"[32]"},{"why":"Shows classifier-guided denoising can separate normal from anomalous samples, grounding the reconstruction-error approach.","marker":"[23]"},{"why":"Trains a denoising network for normal-image reconstruction, the direct template for this paper's DDPM anomaly scoring.","marker":"[25]"},{"why":"Combines DDPM-based reconstructions across timesteps to compute anomaly scores, informing the scoring procedure.","marker":"[26]"},{"why":"Introduces a diffusion-based framework for multi-class anomaly detection that motivates reconstruction error as a discriminator.","marker":"[35]"},{"why":"Studies why diffusion reconstruction errors separate anomalies and informs the interpretation of the results.","marker":"[36]"}],"fun_headline_variants":["Diffusion reconstruction error ranks anomalies better","Diffusion gap flags anomalies better than classical detectors","Reconstruction error: diffusion's edge over one-class methods","Diffusion models see anomaly gap classical detectors miss","Diffusion reconstruction separates normal from anomalous data"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The central claim collapses if the single Mini-ImageNet scalability experiment is not a fair and fully specified test, because the paper gives no details of the subset, model size, inference timesteps, or how the anomaly score was calibrated.","fun_headline_variants_meta":{"raw":{"variants":["Diffusion reconstruction error ranks anomalies better","Diffusion gap flags anomalies better than classical detectors","Reconstruction error: diffusion's edge over one-class methods","Diffusion models see anomaly gap classical detectors miss","Diffusion reconstruction separates normal from anomalous data"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000444,"raw_usage":{"total_tokens":2190,"prompt_tokens":835,"completion_tokens":1355,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":451,"completion_tokens_details":{"reasoning_tokens":1285}},"tokens_in":451,"tokens_out":1355,"duration_ms":10641,"temperature":1.0,"reasoning_tokens":1285,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T18:44:15.731335+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the Mini-ImageNet experiment with the subset composition, model size, number of inference timesteps, and anomaly-score calibration documented, and check whether a DiT-based detector still exceeds OCSVM and IForest in AUC-ROC; also check every row of Table 1 to see whether at least one diffusion variant beats every classical method, since any counterexample row (for instance CIFAR10_7) would refute the 'all datasets' claim as stated.","supporting_citations":[{"cited_title":"Denoising di ffusion probabilistic models","cited_arxiv_id":null,"evidence_quote":"Defines the forward and reverse diffusion process the paper uses as its reconstruction machinery."},{"cited_title":"Improved deno ising diffusion probabilistic mod- els","cited_arxiv_id":null,"evidence_quote":"Supplies the simple denoising objective and covariance optimization used to train the noise-prediction network."},{"cited_title":"Enhancement of the flow of vibrated grains through narrow apertures by addition of small particles","cited_arxiv_id":"2103.05420","evidence_quote":"Motivates the Diffusion Transformer (DiT) architecture for high-resolution image modeling, the basis of the scalability claim."},{"cited_title":"Exact solutions for time-dependent complex symmetric potential well","cited_arxiv_id":"2206.04593","evidence_quote":"Shows classifier-guided denoising can separate normal from anomalous samples, grounding the reconstruction-error approach."},{"cited_title":"Anoddp m: Anomaly detection with denois- ing diffusion probabilistic models","cited_arxiv_id":null,"evidence_quote":"Trains a denoising network for normal-image reconstruction, the direct template for this paper's DDPM anomaly scoring."},{"cited_title":"Tim e-series anomaly detection using ddpm-based reconstruction","cited_arxiv_id":null,"evidence_quote":"Combines DDPM-based reconstructions across timesteps to compute anomaly scores, informing the scoring procedure."},{"cited_title":"Diad: A diffusion-based fr amework for multi-class anomaly detection, 2023","cited_arxiv_id":null,"evidence_quote":"Introduces a diffusion-based framework for multi-class anomaly detection that motivates reconstruction error as a discriminator."}],"review_version":1}