Towards a Rigorous Evaluation of Time-series Anomaly Detection
Reviewed by Pith T0 review T1 audit T2 compute T3 formal T4 kernel pith:URODOY7Jrecord.jsonopen to challenge →
read the original abstract
In recent years, proposed studies on time-series anomaly detection (TAD) report high F1 scores on benchmark TAD datasets, giving the impression of clear improvements in TAD. However, most studies apply a peculiar evaluation protocol called point adjustment (PA) before scoring. In this paper, we theoretically and experimentally reveal that the PA protocol has a great possibility of overestimating the detection performance; that is, even a random anomaly score can easily turn into a state-of-the-art TAD method. Therefore, the comparison of TAD methods after applying the PA protocol can lead to misguided rankings. Furthermore, we question the potential of existing TAD methods by showing that an untrained model obtains comparable detection performance to the existing methods even when PA is forbidden. Based on our findings, we propose a new baseline and an evaluation protocol. We expect that our study will help a rigorous evaluation of TAD and lead to further improvement in future researches.
This paper has not been read by Pith yet.
Forward citations
Cited by 5 Pith papers
-
HEPA: A Self-Supervised Horizon-Conditioned Event Predictive Architecture for Time Series
HEPA pretrains via horizon-conditioned JEPA on unlabeled data then fine-tunes only the predictor for event survival CDFs, outperforming PatchTST, iTransformer, MAE and Chronos-2 on at least 10 of 14 benchmarks with fi...
-
Modeling Normal Is All You Need: Joint Latent Clustering for Anomaly Detection in Multimodal Cyber-Physical Systems
A VaDE-based latent clustering detector that drops reconstruction wins a fair, difficulty-stratified anomaly detection protocol on three CPS datasets, with margins tracking dataset multimodality.
-
HEPA: A Self-Supervised Horizon-Conditioned Event Predictive Architecture for Time Series
HEPA combines JEPA self-supervised pretraining with horizon-conditioned fine-tuning to predict rare events in multivariate time series as a monotonic survival distribution, outperforming PatchTST, iTransformer, MAE, a...
-
HEPA: A Self-Supervised Horizon-Conditioned Event Predictive Architecture for Time Series
HEPA combines self-supervised JEPA pretraining on time series representations with horizon-conditioned finetuning to predict rare events via survival CDFs, outperforming PatchTST, iTransformer, MAE, and Chronos-2 on a...
-
Detecting the Undetectable: Enhancing Unsupervised time series Anomaly Detection via Active Learning
Active learning with masked reconstruction and minimax training raises AUC by 12.39% across 28 test cases on four multivariate datasets and seven unsupervised backbones.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.