{"id":"5ac6f16c-90d5-4be9-ac9c-7d46fc7d4121","arxiv_id":"2506.18368","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"SeeKer models skeleton sequences as autoregressive Gaussian densities over individual keypoints and uses weighted negative log-likelihood as the anomaly score, achieving state-of-the-art AUROC on UBnormal and MSAD-HR.","lead":"A new skeleton-based anomaly detector predicts where each body keypoint should be next and flags frames where observed keypoints are unlikely, and it beats previous methods on two benchmark datasets while matching the best on a third. The method is simple, interpretable, and efficient, which may make it a strong default baseline for safety monitoring applications.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Appendix B causal mask definitions contradict Eq. (2): as printed, the output mask conditions each keypoint on future keypoints, so the autoregressive claim rests on the released code correcting this.","rationale":"The reader's weakest_assumption correctly identifies the single most load-bearing issue. The paper's central claim is not merely 'AUROC is higher' but 'high AUROC is achieved by a causal, autoregressive, keypoint-level density model.' Appendix B is the only specification of the mask implementation, and Eqs. (20)-(21) as written implement a non-causal model that conditions each keypoint on future keypoints, invalidating Eq. (2) and the causal narrative. This is a resolvable concern because the code is public; the likely outcome is a typo in Eq. (21), but until verified, the correctness of the central contribution remains conditional. I do not see a more fundamental objection: the evaluation protocol is standard, the strongest baselines are reproduced or use shared skeleton extraction, and the ablations support the main design choices. The missing exact hyperparameters and aggregate variance are secondary reproducibility issues that conditional acceptance should request, not grounds for rejection. Therefore the reader's CONDITIONAL verdict stands unchanged.","tokens_in":16747,"tokens_out":11360,"duration_ms":118219,"concrete_test":"Run a perturbation probe on the released model (github.com/adelic99/seeker): take a fixed input window, then modify the coordinates of keypoint block n+1 while holding all other inputs constant, and measure whether the predicted mean or variance for keypoint block n changes. Under the intended factorization Eq. (2), the prediction for block n must remain identical; under Eq. (21), it will change. This directly distinguishes the two masks without relying on code-reading.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that SeeKer implements the autoregressive factorization in Eq. (2) with causal keypoint conditioning. The printed masks in Appendix B are formally inconsistent with that claim. Eq. (20) defines the hidden-layer mask as lower-block-triangular (M_l_{i,j}=1 if floor(j/D) <= floor(i/D)), which is standard MADE-style self-or-past connectivity. Eq. (21) defines the output-layer mask as upper-diagonal (M_o_{i,j}=1 if floor(j/D) > floor(i/D)), so the predicted distribution for keypoint block i would be computed from hidden units in blocks j>i, i.e., from future keypoints. This is the opposite of the stated goal ('without direct influence from its own current and all succeeding representation') and does not implement p(X_{t,n}|X_{t,<n},X_Delta). In a masked fully-connected network, the output mask is what determines the autoregressive property; if the code follows Eq. (21), the method is not the causal density estimator described, and its strong empirical performance may even benefit from future-keypoint conditioning. The interpretability claim (identifying which keypoint triggered the anomaly) and the novelty of 'causal prediction' would also collapse. The concern is resolvable because the code is public, but it must be checked.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes SeeKer, a skeleton-based video anomaly detection method that factorizes the joint density of a skeleton sequence autoregressively at the keypoint level. Each keypoint's location is modeled by a multivariate Gaussian whose mean and covariance are predicted by a masked fully connected network from the preceding keypoints in the current frame and from keypoints in past frames. The anomaly score is a confidence-weighted sum of per-keypoint negative log-likelihoods, and frame-level scores are obtained by max-pooling over skeletons. Experiments on UBnormal, ShanghaiTech, and MSAD-HR report AUROC/AP improvements over several baselines, including state-of-the-art skeleton methods, and ablations analyze covariance choice, prediction granularity, confidence weighting, smoothing, keypoint ordering, and architecture. Code is publicly released.","tokens_in":16973,"tokens_out":6309,"duration_ms":64900,"significance":"If the described method is implemented as claimed, this is a valuable empirical contribution: a conceptually simple density-estimation baseline that outperforms more complex graph-based and diffusion-based skeleton anomaly detectors on UBnormal and MSAD-HR, while remaining competitive on ShanghaiTech. The paper also contributes a compositional, interpretable anomaly score and a principled way to incorporate keypoint-detector confidence. The ablations are thorough and the code release supports reproducibility. The main caveat is the formal inconsistency in the printed mask definitions, which, if not resolved, would invalidate the claimed autoregressive formulation.","major_comments":[{"comment":"The blockwise masks as printed contradict the autoregressive factorization in Eq. (2) and the stated causal-prediction claim. The hidden-layer mask M_l in Eq. (20) is lower-block-triangular (M_l_{i,j}=1 if floor(j/D) <= floor(i/D)), but the output-layer mask M_o in Eq. (21) is upper-block-triangular (M_o_{i,j}=1 if floor(j/D) > floor(i/D)). Consequently, the predicted distribution for keypoint block i is computed from hidden units in blocks j>i, i.e., from future keypoints, which is the opposite of the paper's claim that each keypoint is predicted 'without direct influence from its own current and all succeeding representation.' If the released code follows Eq. (21), the method is not the causal keypoint density estimator described, and the strong empirical results could be attributed in part to future-keypoint conditioning. The interpretability claim and the novelty of causal keypoint prediction would also collapse. This must be resolved by checking the released code and correcting the text (or, if the code indeed uses the printed masks, by reframing the method).","section":"Appendix B, Eqs. (20)-(21)"},{"comment":"The exact architecture configuration used for the main results in Tables 1-3 is not reported. The implementation details mention training epochs, optimizer, learning rate, and batch size, but not the number of hidden layers, the expansion factor, or the sequence length T used for the reported numbers. Appendix C.1 (Table 9) provides only a range of values and validation AUROCs, not the selected configuration. This prevents reproduction of the headline results from the paper text alone. Please state the exact hyperparameters used for each dataset and clarify whether the same configuration was used for all tables.","section":"Section 4 and Appendix C.1"}],"minor_comments":[{"comment":"The phrase 'upper diagonal' in the text describing Eq. (21) is used to mean excluding 'its own current and all succeeding representation,' but an upper-diagonal mask actually includes succeeding elements and excludes preceding ones. Please rephrase to avoid confusion.","section":"Appendix B, Eq. (21)"},{"comment":"The notation in Eq. (15) is inconsistent with Eq. (3): the covariance is written as Σ = σI, but the quadratic term uses diag(1/σ) and the log-determinant term is ln σx + ln σy, which mixes standard-deviation and variance conventions. Please clarify the parameterization.","section":"Appendix A, Eq. (15)"},{"comment":"There are several typos, including 'ilustrated' (Figure 3 caption), 'siding-window' (Section 3.1), 'preceeding' (multiple places), 'have have' (Section 5.3), 'ShangahaiTech' (Appendix C.2), and 'autoreregressive' (Appendix B). Please proofread.","section":"Throughout"},{"comment":"In Eq. (9), the notation s(Xt|XΔ) uses Δ implicitly defined earlier, but the frame-level score is over skeletons Xt in frame It; please define how Δ is chosen for each skeleton in a frame with multiple people.","section":"Section 3.2, Eq. (9)"},{"comment":"The claim that the authors are 'the first to introduce the measurement of AP in this context' seems overstated given that AP is a standard detection metric; please rephrase to be more specific about what exactly is new about this evaluation.","section":"Section 4, Metrics"}],"recommendation":"major_revision","confidential_remarks":"The paper reports strong empirical results and releases code, but the mask definitions in Appendix B are formally inconsistent with the central autoregressive claim. I recommend verifying the released code and correcting the text or the method before publication. The novelty is modest, but the paper is well-structured and the empirical contribution is meaningful if the causal formulation is confirmed."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: SeeKer is a genuinely simple and effective skeleton-based anomaly detector that outperforms the previous SOTA on UBnormal and MSAD-HR by a large margin. The core idea is a MADE-style autoregressive density estimator over keypoints, with confidence-weighted per-keypoint log-likelihoods as the anomaly score. The empirical story is solid: good ablations, public code, consistent results. The main caveat is the masking definitions in Appendix B: as printed, the output mask is reversed, so the stated causal factorization does not follow from the equations. This is almost certainly a typo, but it needs to be checked against the released code, because if the code follows the printed mask, the 'causal' claim and the interpretability story collapse. That would turn the paper from a well-motivated method into a puzzling one whose performance might be aided by future keypoint conditioning.\n\nWhat's new: applying autoregressive factorization at the keypoint level for skeleton sequences, with confidence weighting from the pose detector. The components are standard, but the combination is new and the empirical gains are real relative to the baselines they reproduce. The ablations are thorough: learned covariance matters a lot (10+ pp), keypoint-level granularity beats skeleton-level, confidence weighting adds a consistent boost, and the method is robust to keypoint ordering and smoothing. The authors also honestly report that smoothing helps them less than it helps STG-NF.\n\nSoft spots: (1) The Appendix B mask contradiction is the one load-bearing issue. The stress-test is right: Eq. (20) and Eq. (21) as printed implement the opposite of the intended autoregressive order. Fixing the appendix is trivial, but the authors should confirm the released code uses the correct mask. (2) The exact hyperparameters for the main results (sequence length, hidden layers, expansion factor) are not stated in the implementation details; they are only visible through Table 9's validation grid. Minor, but easy to fix. (3) The paper doesn't say which baseline numbers were reproduced versus taken from previous papers, except for MSAD-HR and STG-NF's AP. Standard practice, but worth clarifying.\n\nWho it's for: researchers in video anomaly detection who want a strong, interpretable, cheap skeleton baseline. It also serves as a useful example of a simple method beating more complex ones. The paper deserves a serious referee: the empirical work is solid, the code is available, and the only serious issue is a resolvable masking inconsistency. I'd recommend accepting it for review with a request to fix the appendix and release the exact configuration.","headline":"A simple masked autoregressive keypoint density estimator that beats the previous SOTA on UBnormal and MSAD-HR; the results are credible, but the causal mask definitions in Appendix B are reversed as printed and need to be checked against the code.","tokens_in":17526,"tokens_out":3000,"would_cite":true,"duration_ms":28652,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"SeeKer claims that a skeleton is anomalous when its keypoint locations are improbable under an autoregressive Gaussian density, surpassing all prior methods on UBnormal and MSAD-HR and staying competitive on ShanghaiTech.","keywords":["video anomaly detection","skeleton-based anomaly detection","autoregressive density estimation","keypoint-level factorization","Gaussian conditional likelihood","masked fully connected network","keypoint detection confidence","pose abnormality detection"],"falsifier":"Inspect the released mask matrices of the masked fully connected model and verify that the prediction for keypoint n does not depend on any keypoint with index greater than n in the current skeleton; alternatively, train the model with the appendix's printed output mask and show that performance collapses or that the model is conditioning on future joints, which would contradict the paper's account of its mechanism.","tokens_in":16514,"feed_emoji":"🦴","tokens_out":11371,"duration_ms":93320,"temperature":0.7,"pith_summary":"SeeKer claims that the density of a skeleton sequence can be factorized autoregressively at the level of individual keypoints: the location of each joint is modeled as a multivariate Gaussian conditioned on the preceding joints of the same skeleton and on all joints of the past few skeletons. A masked fully connected network predicts the mean and covariance for every keypoint in one forward pass, and training maximizes the resulting sequence log-likelihood. At test time a skeleton is anomalous when its keypoint locations are improbable under these conditionals, and the anomaly score is the sum of per-keypoint log-likelihoods weighted by the keypoint detector's confidence. This estimator reaches AUROC 77.9 versus 71.8 for the previous best skeleton method on UBnormal and 61.1 versus 55.7 on MSAD-HR, while coming in second among skeleton methods on ShanghaiTech. The broader point is that compositional causality in skeleton sequences, rather than elaborate graph or generative architectures, carries much of the signal for human-related anomaly detection.","feed_headline":"Simple keypoint-density model beats prior skeleton anomaly detectors","feed_subtitle":"Autoregressive joint-location densities beat graph, diffusion, and score-matching baselines on UBnormal and MSAD-HR.","key_machinery":"The carrying object is the keypoint-level autoregressive factorization, $p_\\theta(X_{t,n}\\mid X_{t,<n}, X_\\Delta)=\\mathcal{N}(\\mu_\\theta,\\Sigma_\\theta)$, implemented by a causally masked fully connected network. The mask is a blockwise triangular pattern that lets the prediction for keypoint $n$ attend only to keypoints $1,\\dots,n-1$ of the current skeleton plus all keypoints of the past $\\Delta$ skeletons, while all per-keypoint distributions are produced in a single forward pass. The anomaly score aggregates these conditionals as a confidence-weighted sum of log-likelihoods, so the final decision decomposes into per-keypoint contributions that can be inspected individually.","core_discovery":"On its own terms, the paper establishes that the joint density of a skeleton sequence can be written as a product over time and keypoints of Gaussian conditional densities, $\\ln p_\\theta(X)=\\sum_{t}\\sum_{n}\\ln\\mathcal{N}(X_{t,n}\\mid\\mu_\\theta,\\Sigma_\\theta)$, where $\\mu_\\theta$ and $\\Sigma_\\theta$ are regressed from the preceding keypoints of the current skeleton and all keypoints of the past $\\Delta$ skeletons. Optimizing this objective is equivalent to minimizing a Mahalanobis-distance loss with a $\\ln\\det\\Sigma_\\theta$ regularizer. The inference score is $s(X_t\\mid X_\\Delta)=-\\sum_n c_{t,n}\\ln p_{\\theta_{MLE}}(X_{t,n}\\mid X_{t,<n}, X_\\Delta)$, with $c_{t,n}$ the detector confidence, and a thresholded version of this score classifies frames. The paper reports AUROC 77.9 (vs. 71.8 for the previous best skeleton method) on UBnormal, 61.1 (vs. 55.7) on MSAD-HR, and 85.5 (vs. 85.9) on ShanghaiTech, with the best AP and RBDC among skeleton methods on ShanghaiTech. Ablations show that learned covariance, keypoint-level granularity, and confidence weighting each contribute substantially to the final performance.","pith_inferences":["Editorial inference: because the model operates on raw 2D keypoint coordinates with a small MLP, the same density estimator could be dropped into other pose-based tasks such as motion forecasting, action segmentation, or human-robot safety wherever normal motion forms a well-defined distribution.","Editorial inference: the predicted covariance is a generative object, so SeeKer could be used to sample plausible future poses rather than only to score observed ones; a testable extension is to compare sampled keypoints against subsequent frames as a forecasting error signal.","Editorial inference: the confidence-weighting idea generalizes beyond skeletons: any anomaly score built from likelihoods of detected components could be reweighted by detector confidence, for example object-detection boxes or face keypoints.","Editorial inference: the success of a simple masked fully connected MLP over transformers suggests that for low-dimensional, low-variance sequences, expressive but unconstrained architectures add little, and this may transfer to other compact bodily signals such as IMU or mocap data."],"forward_implications":["Keypoint-level autoregressive density estimation is a strong baseline: a simple masked MLP outperforms graph-based normalizing flows, diffusion forecasters, and score-matching energy models on UBnormal and MSAD-HR.","Confidence-weighted scoring gives a principled way to fold keypoint-detector uncertainty into the decision; removing the confidence weights lowers AUROC on both UBnormal and ShanghaiTech.","Because the anomaly score is a sum over keypoints, the method exposes which joint or joints triggered the alarm, enabling per-keypoint localization of abnormal motion.","Modeling at keypoint granularity is consistently better than modeling entire skeletons at once, and the result is essentially invariant to the ordering of keypoints.","SeeKer detects anomalous frame boundaries without post-hoc Gaussian smoothing, whereas the strongest previous skeleton baseline needs smoothing to reach its reported numbers."],"supporting_citations":[{"why":"Supplies the masked fully connected architecture whose blockwise triangular masks enforce the causal keypoint ordering.","marker":"[21]"},{"why":"The strongest skeleton-based baseline that SeeKer is compared against and surpasses on UBnormal and MSAD-HR.","marker":"[23]"},{"why":"The UBnormal benchmark with precise labels on which SeeKer reports its largest gains.","marker":"[3]"},{"why":"The ShanghaiTech benchmark where SeeKer is competitive and improves AP and RBDC among skeleton methods.","marker":"[34]"},{"why":"The MSAD-HR benchmark where SeeKer is the first reported method and beats reproduced baselines.","marker":"[62]"},{"why":"The off-the-shelf keypoint detector used by the pipeline, which provides the skeletons and per-keypoint confidence scores.","marker":"[15]"},{"why":"The diffusion-based skeleton baseline reproduced on MSAD-HR and compared against.","marker":"[16]"},{"why":"The multimodal energy-based baseline that SeeKer outperforms using only skeleton input.","marker":"[42]"}],"fun_headline_variants":["Overlooked sequential keypoint density beats skeleton anomaly detectors","Simple Gaussian keypoint model tops skeleton anomaly benchmarks","Keypoint-level density estimation: new state-of-the-art for skeleton anomalies","Autoregressive keypoint density: simple baseline wins on UBnormal and MSAD-HR","Density estimator for skeletons outdoes graph, diffusion, and score-matching"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The scheme works only if the pattern that controls which keypoints influence the prediction for each joint actually conditions each joint on earlier joints, as the paper's main equation requires; the printed pattern in the appendix appears to do the opposite, so the claim rests on the released code matching the intended ordering rather than the appendix text.","fun_headline_variants_meta":{"raw":{"variants":["Overlooked sequential keypoint density beats skeleton anomaly detectors","Simple Gaussian keypoint model tops skeleton anomaly benchmarks","Keypoint-level density estimation: new state-of-the-art for skeleton anomalies","Autoregressive keypoint density: simple baseline wins on UBnormal and MSAD-HR","Density estimator for skeletons outdoes graph, diffusion, and score-matching"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000739,"raw_usage":{"total_tokens":3337,"prompt_tokens":1016,"completion_tokens":2321,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":632,"completion_tokens_details":{"reasoning_tokens":2228}},"tokens_in":632,"tokens_out":2321,"duration_ms":19586,"temperature":1.0,"reasoning_tokens":2228,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T18:50:57.440920+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Inspect the released mask matrices of the masked fully connected model and verify that the prediction for keypoint n does not depend on any keypoint with index greater than n in the current skeleton; alternatively, train the model with the appendix's printed output mask and show that performance collapses or that the model is conditioning on future joints, which would contradict the paper's account of its mechanism.","supporting_citations":[{"cited_title":"MADE: masked autoencoder for distribution es- timation","cited_arxiv_id":null,"evidence_quote":"Supplies the masked fully connected architecture whose blockwise triangular masks enforce the causal keypoint ordering."},{"cited_title":"Normalizing flows for hu- man pose anomaly detection","cited_arxiv_id":null,"evidence_quote":"The strongest skeleton-based baseline that SeeKer is compared against and surpasses on UBnormal and MSAD-HR."},{"cited_title":"Ubnor- mal: New benchmark for supervised open-set video anomaly detection","cited_arxiv_id":null,"evidence_quote":"The UBnormal benchmark with precise labels on which SeeKer reports its largest gains."},{"cited_title":"Fu- ture frame prediction for anomaly detection - A new base- line","cited_arxiv_id":null,"evidence_quote":"The ShanghaiTech benchmark where SeeKer is competitive and improves AP and RBDC among skeleton methods."},{"cited_title":"Advancing video anomaly detection: A concise re- view and a new dataset","cited_arxiv_id":null,"evidence_quote":"The MSAD-HR benchmark where SeeKer is the first reported method and beats reproduced baselines."},{"cited_title":"Alpha- pose: Whole-body regional multi-person pose estimation and tracking in real-time","cited_arxiv_id":null,"evidence_quote":"The off-the-shelf keypoint detector used by the pipeline, which provides the skeletons and per-keypoint confidence scores."},{"cited_title":"Multimodal motion conditioned diffusion model for skeleton-based video anomaly detection","cited_arxiv_id":null,"evidence_quote":"The diffusion-based skeleton baseline reproduced on MSAD-HR and compared against."},{"cited_title":"MULDE: multiscale log- density estimation via denoising score matching for video anomaly detection","cited_arxiv_id":null,"evidence_quote":"The multimodal energy-based baseline that SeeKer outperforms using only skeleton input."}],"review_version":1}