{"id":"e5ebcd1c-53c9-4acc-a88a-d1a410bc6260","arxiv_id":"2412.09026","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"A patch-level diffusion predictor with appearance and temporal-difference motion conditions achieves 98.6, 91.3, 79.2, and 63.4 frame-level AUC on Ped2, Avenue, ShanghaiTech, and UBnormal.","lead":"This paper trains a patch-based diffusion model to predict the next video frame from six observed frames, using the first frame as an appearance guide and frame differences as a motion guide, and scores anomalies by the mismatch between prediction and reality. It reports top frame-level AUC on four surveillance datasets, ahead of the prior diffusion baseline FPDM by roughly one point.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The detector conditions on raw test frames that may contain the anomaly; if the denoiser copies anomalous content through the motion concatenation or appearance skip connections, Eq. 16 collapses and the reported AUCs are not evidence of detection.","rationale":"The reader's weakest_assumption targets exactly the load-bearing mechanism: at test time, both conditions are raw frames that may contain the anomaly, and the paper provides no evidence that the memory bank or normal-only training prevents the denoiser from copying that anomalous content into the predicted frame. The concern is not merely a disagreement with the community's consensus; it is an internal architectural risk. The motion condition is directly concatenated with the noisy target, and the appearance encoder's hierarchical features are injected into the U-Net decoder via ControlNet-style skip connections, so the memory bank does not filter the raw appearance and motion information. If the model can reconstruct an anomalous object from these conditions, the MSE in Eq. 16 becomes low for anomalous frames, and the high AUC numbers would reflect background/static-scene differences rather than an inability to generate anomalies. This is especially severe on Ped2 where alpha = 0, so the score relies entirely on the future-frame term. The paper does not include an experiment that isolates this copying behavior, e.g., by comparing anomaly scores in the presence and absence of the anomalous object in the conditions. The proposed concrete test would settle the issue: if anomalous-region prediction error is low, then the conditions leak the anomaly and the central superiority claim is not established as stated; if anomalous-region error is high, the concern is resolved and the conditional acceptance can be maintained. Other issues identified by the reader (no code, no error bars, per-dataset alpha tuning) are real but secondary; the copying risk is the one that directly threatens the causal story behind the reported numbers. The verdict should remain conditional pending this test.","tokens_in":12358,"tokens_out":4245,"duration_ms":48312,"concrete_test":"Run MA-PDM with the published settings on Ped2 (alpha = 0) for test clips where the abnormal object is present in all six observed frames and in the target frame. Compute the per-pixel MSE between the predicted frame and the true frame separately inside the ground-truth anomalous region and inside a matched normal region. If the anomalous-region MSE is comparable to or lower than the normal-region MSE, the model is successfully predicting the anomaly from the conditions, so the score in Eq. 16 cannot separate anomalous from normal frames and the reported 98.6 AUC is not attributable to failure to generate anomalies. A synthetic control would be to paste a salient abnormal object into F1 of a normal test clip (leaving F^{N+1} unchanged) and check whether the predicted frame contains the pasted object.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that MA-PDM outperforms prior methods on four VAD benchmarks (Table 1). The anomaly score (Eq. 16) is an MSE between the DDIM-predicted frame and the true frame. At inference, the two conditions driving the denoiser are the raw initial frame Fa = F1 and the raw temporal difference Fm (Eq. 3), both taken from the test clip. If those frames contain the anomaly, the denoiser is conditioned on the very signal it is supposed to fail to predict. Nothing in the architecture guarantees the anomaly is not copied: the motion condition is concatenated channel-wise with the noisy target before the U-Net (Sec. 3.4), and the appearance encoder's hierarchical features hA_e are added directly to the decoder via a ControlNet-style skip connection, bypassing the patch memory bank (Eqs. 8-10). The memory bank only constrains the semantic embedding S, not the injected appearance features. Consequently, for an anomaly present in the observed frames and persisting into the target, the model could generate a target frame containing the anomaly, making MSE low and the anomaly score low. The risk is particularly acute on Ped2, where alpha = 0 (Sec. 4.1) so the score uses only the future-frame term. The paper neither states nor tests the assumption that conditioning on anomalous test frames will not help generate the anomaly. This is the load-bearing premise for the reported superiority.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes MA-PDM, a patch-based diffusion model for video anomaly detection within a future-frame-prediction framework. An appearance condition is taken from the first frame of a six-frame clip, a motion condition is defined as the temporal difference between consecutive frames, and a DDIM-based noise estimator is conditioned on both. A location-indexed patch memory bank constrains semantic embeddings from the appearance encoder, while hierarchical appearance features are injected into the decoder via ControlNet-style skip connections. At inference, sliding-window patches are denoised, merged, and the anomaly score is the MSE between the predicted and true future frame plus an alpha-weighted MSE between reconstructed and true appearance. Experiments on Ped2, Avenue, ShanghaiTech, and UBnormal report frame-level AUCs of 98.6, 91.3, 79.2, and 63.4, respectively, with ablations over modules, patch sizes, memory variants, and motion representations.","tokens_in":12705,"tokens_out":4000,"duration_ms":42811,"significance":"If the reported results are reproducible, MA-PDM is a competitive contribution: it outperforms the diffusion-based FPDM on Avenue, ShanghaiTech, and UBnormal, and is competitive with AED-MAE without synthetic anomalies. The patch-level formulation is well motivated for small anomalous objects, and the ablations provide evidence that each proposed component contributes to performance. The paper does not, however, provide code, error bars, or a validation protocol for the per-dataset hyperparameters, so the small reported margins cannot currently be assessed for robustness. The central assumption that conditioning on raw test frames does not leak anomalous content into the generated frame is also untested; this is the main risk to the validity of the anomaly score.","major_comments":[{"comment":"The conditions used at inference are raw test frames: the appearance condition Fa = F1 and the motion condition Fm = F[:-1] - F[1:] are taken from the observed part of the test clip, which may itself contain the anomaly. The motion condition is concatenated channel-wise with the noisy target, and the hierarchical appearance features hA_e are injected directly into the decoder without passing through the patch memory bank. If the denoiser can copy anomalous content from these conditions, the MSE in Eq. (16) collapses and the reported AUCs no longer reflect detection of the anomaly. The paper neither states nor tests the assumption that conditioning on anomalous test frames will not help generate the anomalous target. This is load-bearing for the central claim, and the authors should add an explicit test, for example by comparing detection on anomalies that first appear only in the target frame versus anomalies already visible in the conditioning frames, or by ablating the conditions on anomalous frames.","section":"Sec. 3.2, Eq. (4); Sec. 3.4; Sec. 3.5, Eq. (16)"},{"comment":"The anomaly-score weight alpha is set per dataset to (0, 0.2, 0.3, 1) and the training epochs are set per dataset to (1000, 300, 30, 40), but no validation procedure is described. Since alpha=0 on Ped2 means the appearance reconstruction term is completely discarded, and since the reported improvements over prior methods are small on several datasets, the headline numbers are a function of parameters selected on the test distributions. The authors should report a validation protocol, a sensitivity analysis over alpha and epochs, and ideally multiple runs with standard deviations. Without this, the claim that the method 'outperforms all other methods assessed' is not yet supported.","section":"Sec. 4.1 and Table 1"},{"comment":"The reported margins are sometimes very small (e.g., 79.2 vs. 79.1 on ShanghaiTech and a tie at 91.3 on Avenue), yet no error bars, number of runs, or statistical significance tests are provided. Frame-level AUC differences of this size can easily be within run-to-run noise, especially without a fixed random seed or code release. The authors should provide standard deviations over multiple training runs and, if possible, per-dataset significance tests, before claiming state-of-the-art status.","section":"Table 1 and Sec. 4.2"}],"minor_comments":[{"comment":"The patch-location example is numerically inconsistent: for coordinates (58,58), p = (58/64 - 1/2)*4 + (58/64 - 1/2) yields approximately 2.03, not 5. Please correct the formula or the example.","section":"Sec. 3.4, Eq. (8)"},{"comment":"The sliding-window stride is described as r=32 in Sec. 3.5 but as 'a stride of 64' in Sec. 4.1. Please clarify which stride is used for the reported results.","section":"Sec. 3.5 and Sec. 4.1"},{"comment":"The text says 'our technique achieves the highest results on four datasets with a patch size of 64,' but Table 3 reports results for only Ped2, Avenue, and ShanghaiTech; no UBnormal results are shown for the patch-size ablation. Also, the text states that patch size 32 is more efficient for Ped2, but the table shows a tie at 98.6 between sizes 32 and 64 under random cropping.","section":"Table 3 and Sec. 4.3"},{"comment":"The training details say 'The batch size in all three datasets is 16' although four datasets are used; please correct this.","section":"Sec. 4.1"},{"comment":"There are several typos and unfinished textual references that should be cleaned up, including 'parampters' in Sec. 3.4 and the repeated sentence 'Detail framework is shown in Supplementary Material' with no supplementary material available in the arXiv version.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The paper is a reasonable fit for a computer-vision anomaly-detection venue. The main issues are evaluation hygiene and an untested conditioning assumption rather than an obvious fatal flaw. I would be willing to reconsider after the authors provide a validation protocol, error bars, and an explicit test of whether anomalous content can be copied from the conditioning frames."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper is a workmanlike combination of known pieces: patch-based diffusion (from weather restoration), future-frame prediction for VAD, and a memory bank, plus appearance and motion conditioning. The specific fusion and the location-indexed patch memory are new, and the ablation study is thorough. The reported AUCs are competitive, and the method is clearly described.\n\nBut the evaluation has a few loose ends that matter. The anomaly score weight alpha is chosen per dataset (0, 0.2, 0.3, 1) with no validation protocol, and the training epochs vary a lot across datasets. No error bars or code are provided, so it's hard to know how solid the small margins are. The patch location formula in Sec. 3.4 looks wrong as printed (for coordinates (58,58) it gives 0, not 5), which suggests careless editing.\n\nThe bigger issue is the one the stress-test raises: at inference, the appearance condition is the raw first frame and the motion is the temporal difference between observed frames, so the conditions may contain the anomaly. The memory bank only filters the semantic embedding; the hierarchical appearance features are injected directly via ControlNet-style skip connections. So there is a real pathway for the denoiser to copy anomalous content from the conditions into the predicted frame, which would collapse the anomaly score. The paper doesn't state or test why this doesn't happen, or how often it does. The reported high AUCs suggest the model is not simply copying, but without experiments that ablate conditioning on anomalous vs. normal inputs, this remains an untested load-bearing assumption.\n\nI would not desk-reject this. The architecture is plausible, the ablations are informative, and the concerns are addressable. A serious referee should ask for code, a validation protocol for alpha, error bars or repeated runs, and an analysis of the anomaly-copying failure mode (e.g., by comparing scores on anomalies present in the conditioning frames vs. those appearing later). If those experiments check out, this is a solid contribution. If not, the current claims are oversold.\n\nFor a reading group, this could be useful to discuss benchmarking practices in VAD rather than the method itself.\n\nRecommendation: send to peer review, but flag the evaluation weaknesses and the copying concern prominently.","headline":"A plausible patch-diffusion recipe for VAD with strong numbers but a few evaluation loose ends and an untested vulnerability to anomaly copying through the appearance skip connections.","tokens_in":13238,"tokens_out":3773,"would_cite":false,"duration_ms":39346,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that a patch-based conditional diffusion model, guided by appearance and motion, reaches state-of-the-art frame-level AUC on four video anomaly detection benchmarks.","keywords":["video anomaly detection","diffusion model","frame prediction","patch-based processing","appearance and motion conditions","memory bank","temporal difference","DDIM"],"falsifier":"Run the trained MA-PDM on a test clip where the anomaly is already visible in the first observed frame and in the temporal-difference condition, and check whether the MSE anomaly score in Eq. 16 stays high or collapses; if the denoiser copies anomalous content from the conditioning frames, the detection margin would vanish exactly on such clips.","tokens_in":12157,"feed_emoji":"🎥","tokens_out":10807,"duration_ms":93097,"temperature":0.7,"pith_summary":"The paper tries to establish that a diffusion model operating on image patches, rather than whole frames or latent features, can detect video anomalies by predicting the next frame from the previous six. It argues that anomalies are often small local deviations in both what things look like and how they move, so the denoiser should be guided by an appearance condition (the first observed frame) and a motion condition (temporal differences between consecutive frames). Training only on normal videos, the model learns to generate normal continuations; at test time, anomalous content is flagged by a high mean-squared error between the predicted frame and the actual one. If correct, the method would beat existing prediction- and reconstruction-based detectors on Ped2, Avenue, ShanghaiTech, and UBnormal, reaching frame-level AUCs of 98.6, 91.3, 79.2, and 63.4 respectively.","feed_headline":"Patch diffusion model beats prior detectors on four anomaly sets","feed_subtitle":"Conditioning the denoiser on appearance and motion flags small abnormal objects in surveillance videos.","key_machinery":"The central object is the patch-based conditional denoising network MA-PDM, which treats future-frame prediction as a conditional generation problem. Its load-bearing parts are: (1) random patch cropping during training and sliding-window patch merging with an overlap normalization matrix $R$ at inference, so diffusion runs on $64\\times64$ patches instead of full $256\\times256$ frames; (2) the appearance condition $F_a=F_1$ and the motion condition $F_m=F[:,-1]-F[:,1:]$, a temporal difference that is concatenated with the noisy frame as input to the noise-estimation U-Net; (3) a patch memory bank $M\\in\\mathbb{R}^{P\\times N\\times D}$ addressed by cross-attention to store normal semantics per patch location, with the filtered semantics embedded through adaptive layer normalization; and (4) the DDIM reverse process that assembles the predicted patch noises into a full-image noise map. This machinery carries the argument by making the denoiser able to recover only normal local patterns, so abnormal objects produce a high reconstruction error.","core_discovery":"The paper claims that the right unit for video anomaly detection is a local patch in pixel space, not a whole-frame or latent feature, and that a conditional diffusion model trained to denoise patches can be made into a top-performing frame-prediction detector by feeding it two complementary conditions: appearance, taken from the first of six observed frames, and motion, taken as the temporal difference between those frames. A learnable patch memory bank stores normal semantic patterns per patch location, and the filtered semantics are injected into every residual block through adaptive layer normalization, while the appearance encoder's hierarchical features are fused into the U-Net decoder through residual conditioning. At inference, Gaussian noise is denoised with DDIM over sliding-window patches, overlapping predictions are merged by an overlap normalization matrix, and the anomaly score is the MSE between the reconstructed future frame (plus the reconstructed first frame) and the ground truth. On four benchmarks the paper reports frame-level AUCs of 98.6 on Ped2, 91.3 on Avenue, 79.2 on ShanghaiTech, and 63.4 on UBnormal, and states that the approach outperforms all methods assessed.","pith_inferences":["A direct testable extension would be to use the same appearance and motion conditions with classifier-free guidance; the paper does not explore guidance strength, which could trade normal-frame fidelity against anomaly sensitivity.","Because the anomaly score is the pixel-level MSE to the true future frame, the method implicitly assumes that every anomaly changes pixel content; slow or subtle anomalies that leave most pixels unchanged could evade detection even if the memory bank represents them.","The sliding-window patch predictions could be reused as a coarse anomaly heatmap, but the paper does not evaluate pixel-level localization, so any localization benefit from patch processing remains an unmeasured byproduct.","The ablation treats random cropping as data augmentation; if its main effect is dataset enlargement, the relative gain over whole-image diffusion may shrink on datasets with more varied scenes or non-static cameras."],"forward_implications":["Replacing whole-image diffusion with $64\\times64$ patch processing improves anomaly detection across datasets, and the improvement is larger when both the appearance and motion conditions are added on top.","Temporal difference is both faster and more effective than optical flow as a motion condition, with the paper reporting higher AUC and roughly double the frames per second.","Storing normal patterns per patch location in a memory bank outperforms a global memory bank, which supports the claim that local normality is the right granularity for surveillance anomalies.","The full MA-PDM model improves over the previous diffusion-based detector FPDM on Avenue, ShanghaiTech, and UBnormal, and matches or exceeds a masked-autoencoder baseline without using synthetic anomalies.","The method produces a working detector on UBnormal, a fully synthetic virtual-scene dataset, which the paper treats as a one-class anomaly detection test."],"supporting_citations":[{"why":"FPDM, the two-stage latent-space diffusion detector that MA-PDM identifies as the main diffusion baseline and claims to improve on Avenue, Shanghai, and UBnormal.","marker":"Yan et al. 2023"},{"why":"AED-MAE, the masked-autoencoder detector whose reported scores set the comparison that MA-PDM claims to match or exceed without synthetic anomalies.","marker":"Ristea et al. 2024"},{"why":"MNAD, the memory-augmented normality baseline that the MA-PDM ablation reproduces as Case 2 and uses to measure the gain from the patch diffusion and motion condition.","marker":"Stephen and Menon 2020"},{"why":"Source of the patch-partition and overlap-normalization merging strategy used to train on patches and assemble full-frame noise maps at inference.","marker":"Özdenizci and Legenstein 2023"},{"why":"DDIM, the accelerated reverse-sampling procedure used at inference with five denoising steps.","marker":"Nichol and Dhariwal 2021"},{"why":"Residual conditioning, the mechanism for fusing appearance-encoder features into the noise-estimation U-Net decoder.","marker":"Zhang and Agrawala 2023"},{"why":"DDPM, the underlying denoising diffusion formulation and noise-estimation loss that the patch model adapts.","marker":"Ho, Jain, and Abbeel 2020"},{"why":"Motivates patch-level processing for anomaly detection by showing that local patches localize small defects in industrial images.","marker":"Roth et al. 2022"}],"fun_headline_variants":["Patch diffusion with motion and appearance beats anomaly detectors","Patch diffusion with dual conditions spots small abnormal objects","Appearance and motion guide patch denoiser for anomaly detection","Patch denoiser with motion and appearance flags tiny anomalies"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that a model trained only on normal videos, conditioned on raw test frames that may themselves contain the anomaly, will still fail to generate the anomalous content, so the reconstruction error stays high.","fun_headline_variants_meta":{"raw":{"variants":["Patch diffusion with motion and appearance beats anomaly detectors","Patch diffusion with dual conditions spots small abnormal objects","Appearance and motion guide patch denoiser for anomaly detection","Patch denoiser with motion and appearance flags tiny anomalies"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000618,"raw_usage":{"total_tokens":2872,"prompt_tokens":950,"completion_tokens":1922,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":566,"completion_tokens_details":{"reasoning_tokens":1858}},"tokens_in":566,"tokens_out":1922,"duration_ms":14248,"temperature":1.0,"reasoning_tokens":1858,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T17:22:29.410162+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the trained MA-PDM on a test clip where the anomaly is already visible in the first observed frame and in the temporal-difference condition, and check whether the MSE anomaly score in Eq. 16 stays high or collapses; if the denoiser copies anomalous content from the conditioning frames, the detection margin would vanish exactly on such clips.","supporting_citations":[{"cited_title":"T.; Popescu, M.; Khan, F","cited_arxiv_id":null,"evidence_quote":"AED-MAE, the masked-autoencoder detector whose reported scores set the comparison that MA-PDM claims to match or exceed without synthetic anomalies."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"MNAD, the memory-augmented normality baseline that the MA-PDM ablation reproduces as Case 2 and uses to measure the gain from the patch diffusion and motion condition."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Motivates patch-level processing for anomaly detection by showing that local patches localize small defects in industrial images."}],"review_version":1}