{"id":"51d54d0c-54ae-4d01-b845-3da3c7fd5056","arxiv_id":"2608.05069","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"VQ-VAD turns pose sequences into a discrete motion codebook and detects anomalies by reconstruction error, showing competitive in-domain and cross-dataset results on four benchmarks.","lead":"This paper trains a VQ-GAN-style model on normal human pose sequences, converting motion into a discrete set of tokens and flagging anomalies when a sequence cannot be reconstructed from those tokens. The method reports competitive results across four surveillance benchmarks, including transfer from one dataset to another without retraining.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Central claim lacks a continuous-latent control: quantization is not isolated as the cause of VQ-VAD's in-domain or cross-domain performance.","rationale":"The reader's verdict is CONDITIONAL, based primarily on unspecified pose preprocessing and missing error bars. Those are legitimate reproducibility concerns. However, I identify a more directly load-bearing gap: the central claim is about discrete representations, yet the design never isolates the quantizer's contribution. This is not a disagreement with the reader's concerns; it is a complementary, arguably more fundamental issue. A matched continuous-latent ablation is standard practice when proposing a novel representation mechanism, and its absence means the observed performance cannot be attributed to the paper's core idea. The concern is concrete and testable with a single experiment. It does not force a change from CONDITIONAL to REJECT, because the reported numbers may still be correct and the method may still work; it strengthens the conditionality: the central claim should be accepted only once this control is provided. Therefore the verdict remains CONDITIONAL, which is unchanged from the reader's assessment.","tokens_in":8730,"tokens_out":3718,"duration_ms":36449,"concrete_test":"Retrain VQ-VAD with the quantization branch removed: use the same encoder, decoder, discriminator, reconstruction loss, and adversarial loss, but replace the nearest-neighbor codebook with a continuous latent bottleneck of the same dimension (e.g., 12 tokens of 1024-dim, no quantization loss). Train this continuous VQ-VAD on the same CMU Panoptic splits and evaluate on HR-SHT under the identical cross-domain protocol. If the continuous baseline reaches within 1.5 AUC-ROC points of 76.69%, the discrete codebook is not the demonstrated driver of transfer and the central claim needs qualification. If the continuous baseline is substantially worse (e.g., >3 points), the quantization-specific benefit is supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that learning discrete motion representations via a VQ-GAN codebook yields competitive and transferable anomaly detection. For this claim to hold, the discrete codebook must be a decisive ingredient in the observed results. However, every experiment in Section 4.2 compares VQ-VAD to separately trained methods with different architectures, losses, and preprocessing. Nowhere is VQ-VAD compared to a matched continuous-latent autoencoder with the same encoder, decoder, discriminator, and reconstruction loss but without vector quantization. Consequently, the reported 81.83% in-domain AUC on HR-SHT and 76.69% cross-domain AUC from CMU Panoptic could plausibly be driven by the temporal convolutional encoder-decoder, the adversarial loss, or the simple reconstruction-error scoring, rather than by quantization itself. The paper's novel contribution is explicitly the discrete vocabulary (Section 1 and 3), so this missing ablation is a direct threat to the central claim's internal validity. Without it, the experiments are consistent with the hypothesis that any sufficiently expressive pose-sequence autoencoder would perform similarly, which would undermine the claimed advantage of discretization for cross-domain generalization. This is a load-bearing concern because the paper's stated novelty and the interpretation of its cross-domain results depend on the unique role of the codebook.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes VQ-VAD, a pose-based video anomaly detection framework that adapts VQ-GAN to temporal keypoint sequences. An encoder maps a keypoint sequence to a continuous latent, a learned codebook quantizes it into discrete motion tokens, and a decoder reconstructs the sequence; the reconstruction error serves as the frame-level anomaly score. The model is trained only on normal motion. Experiments cover three protocols: in-domain, cross-domain (CMU Panoptic to SHT/HR-SHT/NWPUC/HuVAD), and cross-dataset (SHT to NWPUC/HuVAD). The paper reports competitive AUC-ROC values, e.g., 81.83% in-domain on HR-SHT, 76.69% cross-domain on HR-SHT, and 59.66% cross-dataset on HuVAD.","tokens_in":8975,"tokens_out":3097,"duration_ms":30343,"significance":"If the claims hold, the paper introduces a plausible and novel direction for pose-based VAD: learning a discrete motion vocabulary that transfers across domains. The training and inference pipeline is coherent, the use of no anomaly labels is a strength, and the evaluation across three protocols is broader than typical in this area. The promise of public code is also a plus. However, the central attribution of the results to vector quantization is not supported by a matched continuous-latent control, and the single-run AUC values with no variance estimates weaken the quantitative claims. These issues are addressable but currently limit confidence in the paper's main contribution.","major_comments":[{"comment":"The central claim that discrete motion representations are beneficial is not isolated by any control condition. Every comparison in Table 3 is against a separately trained method with a different architecture, loss, and preprocessing. To support the claim that vector quantization is the decisive ingredient, the authors should compare VQ-VAD against a matched continuous-latent autoencoder using the same encoder, decoder, adversarial loss, and reconstruction-error scoring, with only the quantization step removed or replaced by a continuous bottleneck. Without this ablation, the reported in-domain and cross-domain results could be attributed to the temporal convolutional encoder-decoder, the adversarial loss, or the scoring mechanism rather than to the learned codebook.","section":"Section 4.2, Table 3"},{"comment":"The pose extraction and preprocessing pipeline is critically underspecified. The paper does not state which pose detector is used for SHT, HR-SHT, NWPUC, or HuVAD, how keypoints are normalized across datasets with different camera viewpoints and skeleton scales, or how 2D (CMU Panoptic) and 2D/3D keypoints from other benchmarks are aligned into the common representation X ∈ R^{F×C}. Since all anomaly scores are computed in this input space, pose-estimation noise or inconsistent coordinate conventions could dominate the reconstruction error and drive the reported AUC differences. The authors should specify the detector, normalization, and any keypoint-confidence filtering, and ideally include an analysis of pose-estimation noise on the anomaly scores.","section":"Section 3.1"},{"comment":"All AUC-ROC results are reported as single runs with no standard deviations, confidence intervals, or significance tests. This is especially important for the headline cross-dataset and cross-domain comparisons: the HuVAD in-domain gap (VQ-VAD 65.33 vs. STG-NF 57.57) and the cross-dataset gap (59.66 vs. 54.32) could be within run-to-run variance, given the typical instability of adversarial training. The authors should report mean and standard deviation over multiple seeds (at least three) and, for the key comparisons, a paired significance test or an alternative such as multiple-run bootstrap intervals.","section":"Section 4.2, Tables 3 and 4"},{"comment":"The model configuration F4-K1024 is selected solely on reconstruction metrics from the CMU Panoptic validation set (Table 1), but no ablation studies the effect of codebook size K or temporal compression factor on anomaly detection performance. The mapping from reconstruction quality (MPJPE, L1, TS) to anomaly detection AUC is not established. To strengthen the paper, the authors should report at least a small ablation varying K and compression in the in-domain setting on one benchmark (e.g., SHT), showing whether the configuration chosen for reconstruction is also optimal for anomaly discrimination.","section":"Section 4.1.3 and 4.2"}],"minor_comments":[{"comment":"The formula labeled L1 reconstruction error appears to square the absolute difference (the notation 'ห...หଶ' with subscript 2), which is inconsistent with the text describing an average of absolute differences and with the name L1. Please correct the formula to use the absolute value without squaring, or rename the metric.","section":"Section 3.1, Eq. (3)"},{"comment":"The first row is labeled '𝐹4−𝐾2556' which appears to be a typo for F4-K256; also, the ordering of rows is not strictly by increasing codebook size or compression, which makes the table harder to read.","section":"Table 1"},{"comment":"Table 2 lists SHT, NWPUC, and HuVAD but omits HR-SHT, although HR-SHT is a primary benchmark in Table 3. Please add the HR-SHT statistics or explain that they are part of the SHT row.","section":"Table 2"},{"comment":"References [24] and [25] appear unrelated to video anomaly detection (radiation therapy planning and influencer impact on affective polarization). Please verify whether these citations are necessary and, if not, remove them to avoid distracting the reader.","section":"References"},{"comment":"The cross-dataset experiment compares only VQ-VAD and STG-NF. Since the paper claims 'stronger robustness of VQ-VAD's discrete motion representations under domain shift,' including at least one additional recent pose-based baseline with published cross-dataset numbers would make the comparison more convincing.","section":"Section 4.2.3 and Table 4"}],"recommendation":"major_revision","confidential_remarks":"The manuscript has a high concentration of self-citations (e.g., [1], [6], [7], [19], [20], [26]), and two references ([24], [25]) seem unrelated to the topic. For a journal submission, I would expect the authors to broaden the related-work coverage and justify any repeated self-citations. The central missing piece, though, is the continuous-latent control ablation; without it, the paper's novelty claim is not empirically supported. The single-run evaluation is also a consistency concern across the field's standards. These are fixable within the scope of a revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: VQ-VAD is a sensible adaptation of VQ-GAN to temporal keypoints for anomaly detection, and the three-protocol evaluation is a real contribution. But the paper does not include the one experiment that would make its central claim stick: a matched continuous-latent autoencoder with the same encoder, decoder, and adversarial loss, minus the codebook. Without that, the observed AUCs on SHT/HR-SHT and the cross-domain numbers could come from the temporal conv architecture, the adversarial loss, or the reconstruction-error scoring. So the discrete vocabulary as the causal ingredient is unproven.\n\nWhat's new: this is the first application of vector quantization to pose sequences for VAD as far as the cited literature goes. The framing of motion as a vocabulary is reasonable, and the cross-domain and cross-dataset protocols are useful additions. Reconstruction metrics on CMU Panoptic (Table 1) show expected trends with compression and codebook size, and the authors picked F4-K1024 based on those results, which is defensible.\n\nSoft spots, in rough order:\n\n1. Missing continuous-latent control. You need the same model with a continuous bottleneck. The stress-test flags this correctly. It is not fatal to the paper's usefulness, but it is fatal to the claim that discretization is what helps.\n\n2. No error bars or significance tests. All AUCs are single runs, and the differences they emphasize (e.g., 59.66 vs 54.32 on HuVAD) are presented without variance, so we can't tell if they are real.\n\n3. Pose preprocessing is a black box. The paper says keypoints are flattened, but never says which detector, how 2D/3D keypoints are aligned, or normalization. Since pose estimation noise can dominate reconstruction error, this matters.\n\n4. Baseline comparisons are a bit uneven: in-domain VQ-VAD is behind STG-NF on the main benchmarks, and the advantage on HuVAD is used to claim robustness. That is fine, but the headline 'strong in-domain accuracy' overstates a middle-of-the-pack result.\n\nMinor: hyperparameters chosen on validation is normal, not a flaw. The self-citations are mostly to the group's own prior work, but they are relevant and not excessive.\n\nBottom line: this is a solid submission for a workshop or a conference, provided the authors add the continuous-latent ablation and report error bars. The idea is worth publishing as an exploratory study. It deserves peer review, but it needs heavy revision before the central claim is believable.","headline":"Useful first step toward discrete motion vocabularies for pose VAD, but the central claim about quantization is not yet isolated from a matched continuous baseline.","tokens_in":9491,"tokens_out":2244,"would_cite":false,"duration_ms":21696,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Adapting vector-quantized GANs to keypoint sequences yields a discrete motion vocabulary whose reconstruction error flags human behavior anomalies and transfers across datasets without retraining.","keywords":["video anomaly detection","pose-based anomaly detection","vector quantization","motion codebook","keypoint sequences","reconstruction error","cross-domain generalization","human motion representation"],"falsifier":"Take the same test videos, extract keypoints with two independent pose estimators, and run the frozen VQ-VAD model on both sets; if the AUC-ROC differs by more than a few points across estimators, or if adding controlled Gaussian noise to joint coordinates of normal sequences produces reconstruction errors as large as those of true anomalies, the anomaly signal is dominated by pose-estimation artifacts rather than behavioral content.","tokens_in":8565,"feed_emoji":"🎥","tokens_out":8602,"duration_ms":70750,"temperature":0.7,"pith_summary":"VQ-VAD claims that human motion can be captured by a discrete vocabulary of learned motion tokens, and that anomalies are visible as failures to reconstruct through that vocabulary. The paper adapts a vector-quantized GAN to temporal keypoint sequences, training exclusively on normal motion and scoring each new sequence by reconstruction error. This yields in-domain accuracy of 81.83% AUC-ROC on HR-SHT, cross-domain accuracy of 76.69% on HR-SHT when trained only on the Panoptic dataset, and competitive cross-dataset robustness. The intended payoff is a pose-based anomaly detector that transfers across cameras and datasets without retraining, avoiding the appearance and privacy problems of pixel-based surveillance analysis.","feed_headline":"1024-token motion codebook flags anomalies at 81.8% AUC","feed_subtitle":"Pose-based VAD learns a discrete motion vocabulary that transfers across datasets, reaching 76.7% on HR-SHT.","key_machinery":"The load-bearing object is the vector-quantized autoencoder with adversarial training. A 1D temporal-convolutional encoder maps each keypoint sequence to a continuous latent, a finite codebook of 1024 learned motion embeddings replaces each latent by its nearest neighbor, and a decoder reconstructs the sequence from those discrete tokens. The training loss combines reconstruction MSE, a commitment loss that ties encoder outputs to codebook entries, and an adversarial loss from a discriminator that encourages temporally realistic motion. At inference, the squared reconstruction error over the quantized sequence acts as the anomaly score, and the codebook itself is the 'motion vocabulary' of normal behavior.","core_discovery":"On its own terms, the paper establishes a new application of discrete representation learning to pose-based video anomaly detection. VQ-VAD encodes 48-frame keypoint windows into continuous latents, maps each latent to its nearest neighbor in a learned 1024-entry motion codebook, and reconstructs the keypoint sequence from the discrete tokens. Training on normal sequences only, the model uses mean-squared reconstruction error as the anomaly score, taking the maximum error over individuals in a frame. The central result is that this tokenized representation generalizes: training on the Panoptic lab dataset reaches 75.09% AUC-ROC on SHT and 76.69% on HR-SHT without any target-domain adaptation, and in-domain training reaches 81.83% on HR-SHT. The paper argues this is because the codebook learns transferable motion primitives rather than scene-specific appearance, and shows the largest gains over a normalizing-flow baseline on the harder HuVAD dataset under domain shift (59.66% vs 54.32%).","pith_inferences":["A natural extension beyond the paper is to use the discrete token sequence itself, not just its reconstruction error, for anomaly localization: the specific codebook entries that fail to explain a motion window could indicate which body region deviates.","Because the paper does not specify the pose estimator or keypoint normalization, a controlled comparison of two pose detectors on identical video would reveal whether reconstruction errors reflect behavior or input noise; such an experiment would sharpen the method's claims of domain transfer.","The motion vocabulary might serve as a shared representation for other human-centric tasks, such as activity recognition or motion retrieval, since the same tokens are learned without anomaly labels.","An adaptive-scoring variant could maintain per-scene token statistics online and flag tokens that become rare, potentially improving sensitivity on datasets like HuVAD where the paper's cross-domain scores remain near 57-60%."],"forward_implications":["If the claim holds, pose-based anomaly detectors can be trained on one motion-capture domain and deployed on surveillance cameras without per-scene retraining, since the discrete tokens encode motion rather than appearance.","Reconstruction-error thresholding with standard criteria like equal error rate is sufficient for deployment, giving a practical anomaly alarm from a single score per frame.","Larger codebooks (up to 1024 tokens) and lower temporal compression (F4) improve reconstruction fidelity, so system designers can trade codebook size against compute for target accuracy.","Frame-level scoring as the maximum over individuals means one deviant person flags the frame, making the method conservative toward missed anomalies while raising the risk of false alarms in dense crowds.","The discrete representation supports interpretable analysis, since each reconstruction failure can be attributed to the nearest codebook entries the sequence fails to match."],"supporting_citations":[{"why":"Supplies the VQ-GAN architecture and training losses (reconstruction, commitment, adversarial) that VQ-VAD adapts to keypoint sequences.","marker":"[8]"},{"why":"Cited as the VQ-GAN inspiration; provides the time-agnostic VQ-GAN variant used for the temporal compression and codebook design.","marker":"[9]"},{"why":"The normalizing-flow pose anomaly detector that is the primary comparison baseline, especially for cross-dataset generalization (Table 4).","marker":"[11]"},{"why":"The Panoptic multi-view motion-capture dataset used to train the motion codebook and to evaluate representation quality.","marker":"[14]"},{"why":"Supplies the SHT and HR-SHT anomaly benchmarks used for in-domain and cross-domain evaluation.","marker":"[15]"},{"why":"The HuVAD benchmark, the hardest target where VQ-VAD reports its largest gains over the baseline.","marker":"[7]"},{"why":"The NWPUC benchmark used for cross-dataset generalization testing.","marker":"[3]"}],"fun_headline_variants":["Motion codebook learns discrete primitives for transferable VAD","Vector-quantized keypoint sequences spot anomalies via reconstruction","Pose-based VAD gains from tokenizing motion into a codebook","Discrete motion representation transfers across anomaly benchmarks","Lab-trained codebook transfers to HR-SHT at 76.7% AUC"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The input keypoint sequences must be clean, consistently normalized, and directly comparable across the source and target datasets, even though the paper does not specify the pose estimator, keypoint normalization, or how 2D and 3D keypoints are aligned, so measured reconstruction errors could partly reflect pose-estimation noise rather than abnormal behavior.","fun_headline_variants_meta":{"raw":{"variants":["Motion codebook learns discrete primitives for transferable VAD","Vector-quantized keypoint sequences spot anomalies via reconstruction","Pose-based VAD gains from tokenizing motion into a codebook","Discrete motion representation transfers across anomaly benchmarks","Lab-trained codebook transfers to HR-SHT at 76.7% AUC"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000845,"raw_usage":{"total_tokens":3732,"prompt_tokens":1050,"completion_tokens":2682,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":666,"completion_tokens_details":{"reasoning_tokens":2595}},"tokens_in":666,"tokens_out":2682,"duration_ms":21176,"temperature":1.0,"reasoning_tokens":2595,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T10:01:57.364638+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the same test videos, extract keypoints with two independent pose estimators, and run the frozen VQ-VAD model on both sets; if the AUC-ROC differs by more than a few points across estimators, or if adding controlled Gaussian noise to joint coordinates of normal sequences produces reconstruction errors as large as those of true anomalies, the anomaly signal is dominated by pose-estimation artifacts rather than behavioral content.","supporting_citations":[{"cited_title":"Taming transformers for high-resolution image synthesis","cited_arxiv_id":null,"evidence_quote":"Supplies the VQ-GAN architecture and training losses (reconstruction, commitment, adversarial) that VQ-VAD adapts to keypoint sequences."},{"cited_title":"Long video generation with time-agnostic vqgan and time sensitive transformer","cited_arxiv_id":null,"evidence_quote":"Cited as the VQ-GAN inspiration; provides the time-agnostic VQ-GAN variant used for the temporal compression and codebook design."},{"cited_title":"Normalizing flows for human pose anomaly detection","cited_arxiv_id":null,"evidence_quote":"The normalizing-flow pose anomaly detector that is the primary comparison baseline, especially for cross-dataset generalization (Table 4)."},{"cited_title":"Future frame prediction for anomaly detection–a new baseline","cited_arxiv_id":null,"evidence_quote":"Supplies the SHT and HR-SHT anomaly benchmarks used for in-domain and cross-domain evaluation."},{"cited_title":"Towards adaptive human-centric video anomaly detection: A comprehensive framework and a new benchmark","cited_arxiv_id":null,"evidence_quote":"The HuVAD benchmark, the hardest target where VQ-VAD reports its largest gains over the baseline."},{"cited_title":"A new comprehensive benchmark for semi-supervised video anomaly detection and anticipation","cited_arxiv_id":null,"evidence_quote":"The NWPUC benchmark used for cross-dataset generalization testing."}],"review_version":1}