{"id":"c3c7b4e1-2090-4dd5-a481-b031430828d4","arxiv_id":"2412.03452","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"An anomaly-score-guided pipeline finds and labels rare animal behaviors with a small labeling budget, outperforming random sampling in the reported experiments.","lead":"The paper proposes a pipeline that uses an anomaly detection model to find rare animal behaviors in large unlabeled video or sensor datasets, so researchers only need to review the most unusual samples. Instead of sifting through hours of footage, users review a shortlist of outliers and then train a classifier that can scan the whole dataset.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The reported 70% improvement over random sampling is confounded: the proposed method trains on twice as many samples for the same labeling budget, so the benefit may come from pseudo-labeled normals or dataset size rather than anomaly-guided review.","rationale":"The reader's verdict identified the anomaly-score meaningfulness assumption as the weakest point, and that is indeed a real limitation (e.g., FishLarvae1, where fine-grained eye-coordinate differences were missed and AuPRC dropped to 0.27). However, the more load-bearing issue for the paper's central comparative claim is the confounded baseline: the method trains on twice as many samples under the same labeling effort, so the observed improvement cannot be attributed to the anomaly-guided sampling mechanism. The reader mentioned this confound in the rationale but did not elevate it to the weakest assumption. I agree with the reader's overall CONDITIONAL verdict because the code is public and the concern is empirically testable; a clean control experiment would settle it. The paper has genuine strengths: clear problem framing, multi-dataset evaluation, public code and data, and a parameter-light pipeline. The 70% vs 64.82% discrepancy in the abstract is a minor but worth-noting consistency issue, not the main concern. The recommendation is to keep CONDITIONAL, with the condition being that the authors run the equal-training-size / pseudo-labeling control before the comparative claim is accepted as evidence for anomaly-guided review.","tokens_in":18084,"tokens_out":4110,"duration_ms":43791,"concrete_test":"Run a control on the synthetic and three biological datasets: for each N_reviewed, train the ST-GCN on N_reviewed random samples (human-reviewed) plus N_reviewed pseudo-labeled samples drawn from the same center of the anomaly-score distribution used by the proposed method, so that both arms have 2N training samples and identical pseudo-labeling; only the reviewed pool differs (random vs anomaly tail). If the proposed method's AuPRC advantage over this control is not significantly greater than zero (e.g., via paired bootstrap across seeds), then the reported improvement is attributable to training-set size / pseudo-labeling rather than anomaly-guided review.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"Section 3.2 explicitly states that for a labeling budget of N_reviewed, the proposed method trains on N_reviewed reviewed samples from the anomaly tail plus N_reviewed pseudo-labeled normal samples from the distribution center, yielding a training set of size 2*N_reviewed, while random sampling trains on only N_reviewed samples. All reported comparisons (Table 1, Figures 5-6, the 64.82% mean improvement, and the abstract's 70%) therefore compare classifiers trained on datasets of different sizes and with different labeling protocols. The improvement could be driven entirely by the extra pseudo-labeled normal samples or by the doubled training-set size, rather than by the anomaly-score-guided selection of which samples to review. The paper does not include any control that equalizes training-set size or pseudo-labeling, so the central claim that anomaly-guided sampling is responsible for the advantage is not established. This is a load-bearing concern because the method's novelty rests on directing human review toward anomalous samples; if random review plus the same pseudo-labeling achieves comparable performance, the contribution of the anomaly-detection component is unsupported.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a pipeline for efficiently discovering rare animal behaviors in large unlabeled pose/acceleration datasets. The pipeline trains an unsupervised graph-based anomaly detector (STG-NF) on the full unlabeled data, uses the anomaly scores to direct human review toward the high-score tail, pseudo-labels samples near the score mean as normal, and trains an ST-GCN binary classifier on the resulting dataset. The authors evaluate the method on a synthetic dataset and on three real biological datasets (FishLarvae1, PoseR, Meerkat) across induced rarity levels, reporting that it consistently outperforms random sampling by an average of roughly 64-70% in AuPRC. The paper includes publicly available code, synthetic data, and raw experimental outputs, and it provides a statistical analysis of the results. The central comparison, however, is confounded because the proposed method trains on twice as many samples as the random-sampling baseline for the same human-review budget.","tokens_in":18263,"tokens_out":3705,"duration_ms":37702,"significance":"If the reported improvement over random sampling were established, the pipeline would be a practically valuable tool for ethologists and ecologists who need to train rare-behavior classifiers without existing labeled examples. The paper's strengths include its focus on a realistic and underserved problem, the use of multiple real-world datasets, the inclusion of a controlled synthetic experiment, and the release of code and data. The methodological confound described below, however, prevents the current results from supporting the paper's central claim. The contribution is therefore not yet established, though the general approach is plausible and worth pursuing.","major_comments":[{"comment":"The comparison to random sampling is confounded. The paper explicitly states in Section 3.2 that for a labeling budget of N_reviewed, the proposed method trains on N_reviewed reviewed tail samples plus N_reviewed pseudo-labeled center samples, yielding a training set of size 2*N_reviewed, while random sampling trains on only N_reviewed samples. All reported improvements (the abstract's 70%, Section 5.1's 64.82% +/- 2.36, Table 1, and Figures 5-6) therefore compare classifiers trained on datasets of different sizes and with different labeling protocols. The advantage could be driven entirely by the extra pseudo-labeled normal samples or by the doubled training-set size, rather than by the anomaly-score-guided selection of which samples to review. To support the central claim, please add a control experiment in which random sampling is augmented with the same pseudo-labeling procedure (e.g., N_reviewed randomly sampled and reviewed samples plus N_reviewed pseudo-labeled samples from the anomaly-score center, or from the full dataset), trained under identical conditions. Without such a control, the contribution of the anomaly-detection component is not established.","section":"Section 3.2"},{"comment":"The result on FishLarvae1 shows a small and likely non-significant improvement (AuPRC 0.27 +/- 0.067 vs. 0.21 +/- 0.022 for random sampling, Table 1). The paper attributes this to the anomaly detector failing on fine-grained eye-coordinate differences (Section 6), which is an honest limitation. However, this weakens the blanket claim that the method 'consistently outperformed traditional random sampling.' Please report per-dataset effect sizes with confidence intervals or formal significance tests, and temper the abstract and conclusion accordingly, noting that the advantage is dataset-dependent and may be minimal when the anomaly detector does not align with the behavior of interest.","section":"Section 5.2"},{"comment":"The slope comparison in Section 5.1 (random method slope = 0.17 +/- 0.05; proposed method slope = -0.04 +/- 0.06) is also subject to the same confound. Random sampling's stronger dependence on rarity could be a consequence of its smaller effective training set rather than of the sampling strategy itself. Recompute these slopes in the proposed control setup (with equal training-set size and pseudo-labeling for both methods) before drawing conclusions about rarity robustness.","section":"Section 5.1"}],"minor_comments":[{"comment":"The abstract states an 'average improvement of 70%' while Section 5.1 reports '64.82% +/- 2.36'; please reconcile these numbers.","section":"Abstract"},{"comment":"The sliding window size f=8 is mentioned for segmentation, but the FishLarvae1 dataset contains variable-length clips; please clarify how variable-length sequences are handled during training and inference.","section":"Section 3.1"},{"comment":"The paper states that STG-NF models were 'trained for 4 epochs as we found it was enough for models to converge,' but also that no validation set was used (Section S1.2). Please describe the convergence check and how hyperparameters were selected without a validation set.","section":"Section 4.3"},{"comment":"The term 'behaviorSD' is used without definition in the caption; please define it or refer to the text where it is introduced.","section":"Figure 3"},{"comment":"The Meerkat dataset is split randomly rather than by individual, which may leak individual identity across train and test; this limitation is acknowledged in the supplementary but should also be stated in the main text.","section":"Supplementary S1.2.3"}],"recommendation":"major_revision","confidential_remarks":"The paper is within scope for a computer vision or computational behavior venue, and the released code and data are commendable. The main issue is the confounded comparison, which is fixable with additional control experiments; if the authors provide those, the paper could become a solid contribution. I do not see a circularity problem in using STG-NF, as it is an independently published baseline, but the authors should be aware that two co-authors are also authors of that model and may want to explicitly state their reuse of it as a fixed component."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's the short version: this is a genuinely useful applied paper, and the core idea is worth taking seriously. But the headline claim—that anomaly-guided sampling beats random sampling—is not actually established by the experiments as reported, because the proposed method trains on twice as many samples for the same labeling effort.\n\nWhat's new: it's the first test of STG-NF (a graph normalizing flow anomaly detector) on animal pose and accelerometry data, and the authors adapt it sensibly, including a three-vertex plane graph for tri-axial acceleration. The evaluation spans a synthetic control plus three real behavioral datasets from two modalities, with public code, synthetic data, and raw experimental outputs on Zenodo. The statistical modeling of AuPRC as a function of rarity, labeling effort, and similarity is more careful than most work in this space.\n\nThe soft spot is the one the stress-test flagged, and it's real. Section 3.2 states plainly that for a labeling budget of N_reviewed, the proposed pipeline trains on N_reviewed reviewed tail samples plus N_reviewed pseudo-labeled center samples, i.e., 2*N_reviewed total, while random sampling only gets N_reviewed. Every comparison in Table 1 and Figures 5–6 inherits this difference. There is no control where random sampling also receives the same number of pseudo-labeled normals, or where both methods are given the same total training-set size. So the reported 64.82% mean improvement on synthetic data, and the abstract's \"70%,\" could be driven by the extra pseudo-labeled data rather than by the anomaly score's guidance about which samples to review. That is a load-bearing issue for the paper's main claim.\n\nThat said, the idea is still plausible, and the paper is honest about its limitations. The FishLarvae1 result, where the anomaly detector missed the subtle eye-coordinate differences and the pipeline's AuPRC dropped to 0.27, is a fair failure case that tells you exactly where the method depends on the anomaly score being meaningful. The abstract's 70% versus the text's 64.82% is a minor numerical inconsistency, easily fixed.\n\nWho gets value from this: behavioral biologists facing large unlabeled pose/acceleration datasets, and people working on active learning or anomaly-guided data selection. I'd send it to review, but with a request for additional controls that equalize training-set size and pseudo-labeling. If the advantage survives those controls, the paper will be solid.","headline":"Useful applied pipeline for rare-behavior labeling, but the central random-sampling comparison is confounded by a doubled training-set size.","tokens_in":18840,"tokens_out":2549,"would_cite":true,"duration_ms":24117,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Rare animal behaviors can be found by anomaly scores, not random search","keywords":["rare behavior detection","animal behavior","anomaly detection","normalizing flows","graph convolutional networks","pose estimation","accelerometry","labeling budget"],"falsifier":"On a labeled dataset with known rare behaviors, compute the precision of the highest-scoring anomaly tail for the true rare class; if that precision equals the base rate of the rare class, then the anomaly-score ranking carries no information and the pipeline cannot beat random labeling. The FishLarvae1 eye-coordinate case is a concrete place to run this check.","tokens_in":17840,"feed_emoji":"🐾","tokens_out":9395,"duration_ms":87856,"temperature":0.7,"pith_summary":"The paper addresses a bottleneck in behavioral ecology and neurobiology: animal recordings are long, and the behaviors a researcher cares about—strikes, C-bends, running—are rare. It claims that an unsupervised graph-based anomaly detector trained on unlabeled pose or acceleration clips can rank the clips by how unusual they are, and that this ranking can replace random search with a small, targeted labeling budget. Concretely, the pipeline reviews the high-anomaly tail and pseudo-labels the middle of the score distribution as normal, then trains a graph classifier; the paper reports an average improvement of 70% in performance over random sampling, with gains that grow as the behavior becomes rarer, down to 0.02% of the data. If correct, this gives biologists a way to build rare-behavior training sets from recordings they already have, with no pre-existing examples of the behavior.","feed_headline":"Anomaly scores beat random search for rare animal behaviors","feed_subtitle":"Unsupervised graph ranking finds behaviors as rare as 0.02% and trains classifiers with half the labeling effort.","key_machinery":"The machinery is the anomaly-score split. The STG-NF model (spatio-temporal graph normalizing flow) maps graph sequences of animal keypoints—or of planar accelerations—to a latent Gaussian via invertible transformations, and its negative log-likelihood serves as the anomaly score for each clip. The pipeline draws normal samples from the center of the anomaly-score distribution, pseudo-labels them without review, and draws abnormal samples from the high-score tail for human review; this two-pool split is what turns an unsupervised ranker into a training-set builder. The downstream classifier is a shallow ST-GCN (spatio-temporal graph convolutional network), and the paper uses a sliding window of $f=8$ frames to score long pose sequences.","core_discovery":"The paper's central claim is that anomaly scores from a spatio-temporal graph normalizing flow carry enough information about animal motion to organize human annotation. For an unlabeled dataset of clipped pose or acceleration sequences, the model assigns each clip a likelihood-based anomaly score; samples near the mean of the score distribution are pseudo-labeled normal, while the high-scoring tail is sent to a human reviewer who keeps the true rare behaviors. A shallow ST-GCN classifier trained on this mix is claimed to outperform random sampling at the same labeling effort, with the largest advantage under high rarity and with performance that stays nearly constant as rarity increases. The paper demonstrates this on a synthetic kinematic dataset and on three published biological datasets, including one accelerometry dataset, and reports that the pipeline still halves annotation effort when the behavior is not rare.","pith_inferences":["A testable extension is to insert a nuisance-anomaly filter: the PoseR cleaning step shows the anomaly detector is especially sensitive to skeleton-estimation flicker, so separating tracking artifacts from behavioral anomalies could sharpen the tail for true rare behaviors.","The FishLarvae1 result suggests an explicit benchmark where the rare behavior is encoded in a small subset of keypoints (the eye coordinates); a synthetic version of that setting would test whether local keypoint attention is needed for the anomaly score to remain a useful rarity signal.","Because pseudo-labeling the score mean rules out hard positives, an adaptive loop that re-ranks the dataset with the first trained classifier and proactively reviews near-boundary samples could extend the pipeline's usefulness when the behavior is less rare.","The synthetic dataset's simple kinematic rule (frequency/amplitude swap) makes it a reusable controlled testbed for comparing any future rare-behavior sampling scheme."],"forward_implications":["At a fixed labeling budget, the pipeline yields a training set of $N_{\\mathrm{reviewed}}$ human-reviewed anomalies plus $N_{\\mathrm{pseudo}}$ pseudo-labeled normals, roughly twice the size of what random sampling yields for the same review effort.","For rare behaviors at or below 1% of the data, classifier performance stays nearly flat as rarity increases, while random sampling loses roughly 0.17 AuPRC for every order-of-magnitude increase in rarity.","Starting from unlabeled pose or acceleration data, a researcher needs no pre-existing rare examples and no assumption about the number or type of rare behavior classes.","The same graph representation transfers from pose keypoints to tri-axial accelerometry by constructing planar two-channel acceleration graphs, so the pipeline is not tied to one recording modality.","Even when the target behavior is common enough that random sampling performs as well, the anomaly-guided approach still halves the annotation effort."],"supporting_citations":[{"why":"Supplies the STG-NF spatio-temporal graph normalizing flow used to assign anomaly scores to unlabeled clips.","marker":"[11]"},{"why":"Supplies the ST-GCN classifier architecture and the spatio-temporal graph blocks that both the detector and downstream classifier use.","marker":"[30]"},{"why":"Provides the FishLarvae1 larval zebrafish pose dataset used as one of the three biological testbeds.","marker":"[13]"},{"why":"Provides the PoseR larval zebrafish pose dataset with behavior labels and the original train/test splits.","marker":"[19]"},{"why":"Provides the Meerkat tri-axial accelerometry dataset that demonstrates transfer of the pipeline to a non-pose modality.","marker":"[6]"},{"why":"Defines the prior rare-category learning setting with a small initial sample set and a single rare category, which this paper relaxes.","marker":"[18]"}],"fun_headline_variants":["Rare behavior hunt: anomaly scores beat random picks","Anomaly scoring finds rare behaviors with 70% better performance","Rare behavior needle found with half the annotation effort","Spot behaviors as rare as 0.02% using anomaly-based sampling","Sift less, find more: anomaly scores for rare animal moves"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that 'unusual' as measured by the anomaly detector lines up with 'rare and worth finding,' and that clips near the average score can be trusted as normal without review; the FishLarvae1 case, where the detector missed a fine-grained difference and pipeline AuPRC dropped to 0.27, shows where that premise gives way.","fun_headline_variants_meta":{"raw":{"variants":["Rare behavior hunt: anomaly scores beat random picks","Anomaly scoring finds rare behaviors with 70% better performance","Rare behavior needle found with half the annotation effort","Spot behaviors as rare as 0.02% using anomaly-based sampling","Sift less, find more: anomaly scores for rare animal moves"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000581,"raw_usage":{"total_tokens":2760,"prompt_tokens":994,"completion_tokens":1766,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":610,"completion_tokens_details":{"reasoning_tokens":1680}},"tokens_in":610,"tokens_out":1766,"duration_ms":14356,"temperature":1.0,"reasoning_tokens":1680,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T22:22:57.605753+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On a labeled dataset with known rare behaviors, compute the precision of the highest-scoring anomaly tail for the true rare class; if that precision equals the base rate of the rare class, then the anomaly-score ranking carries no information and the pipeline cannot beat random labeling. The FishLarvae1 eye-coordinate case is a concrete place to run this check.","supporting_citations":[{"cited_title":"Deep autoencoder-based behavioral pattern recognition outperforms standard statistical methods in high-dimensional zebrafish studies","cited_arxiv_id":null,"evidence_quote":"Supplies the STG-NF spatio-temporal graph normalizing flow used to assign anomaly scores to unlabeled clips."},{"cited_title":"Daniel Salzman, Dora Angelaki, Andr\\' e s Bendesky, The International Brain Laboratory The International Brain Laboratory, John P Cunningham, and Liam Paninski","cited_arxiv_id":null,"evidence_quote":"Supplies the ST-GCN classifier architecture and the spatio-temporal graph blocks that both the detector and downstream classifier use."},{"cited_title":"Surface defect saliency of magnetic tile","cited_arxiv_id":null,"evidence_quote":"Provides the FishLarvae1 larval zebrafish pose dataset used as one of the three biological testbeds."},{"cited_title":"Mark, Deva Ramanan, and Kayvon Fatahalian","cited_arxiv_id":null,"evidence_quote":"Provides the PoseR larval zebrafish pose dataset with behavior labels and the original train/test splits."},{"cited_title":"On model evaluation under non-constant class imbalance","cited_arxiv_id":null,"evidence_quote":"Provides the Meerkat tri-axial accelerometry dataset that demonstrates transfer of the pipeline to a non-pose modality."},{"cited_title":"Unsupervised quantification of naturalistic animal behaviors for gaining insight into the brain","cited_arxiv_id":null,"evidence_quote":"Defines the prior rare-category learning setting with a small initial sample set and a single rare category, which this paper relaxes."}],"review_version":1}