REVIEW 6 cited by
An Evaluation of Change Point Detection Algorithms
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
An Evaluation of Change Point Detection Algorithms
read the original abstract
Change point detection is an important part of time series analysis, as the presence of a change point indicates an abrupt and significant change in the data generating process. While many algorithms for change point detection have been proposed, comparatively little attention has been paid to evaluating their performance on real-world time series. Algorithms are typically evaluated on simulated data and a small number of commonly-used series with unreliable ground truth. Clearly this does not provide sufficient insight into the comparative performance of these algorithms. Therefore, instead of developing yet another change point detection method, we consider it vastly more important to properly evaluate existing algorithms on real-world data. To achieve this, we present a data set specifically designed for the evaluation of change point detection algorithms that consists of 37 time series from various application domains. Each series was annotated by five human annotators to provide ground truth on the presence and location of change points. We analyze the consistency of the human annotators, and describe evaluation metrics that can be used to measure algorithm performance in the presence of multiple ground truth annotations. Next, we present a benchmark study where 14 algorithms are evaluated on each of the time series in the data set. Our aim is that this data set will serve as a proving ground in the development of novel change point detection algorithms.
Forward citations
Cited by 6 Pith papers
-
Generalized Stochastic Approximation of the Log-Likelihood Ratio for Robust Sequential Change-Point Detection
A new approximation method for the log-likelihood ratio allows robust sequential change-point detection in non-Gaussian processes using moments up to order 3s.
-
CHASM: Online Changepoint Detection in Temporal and Cross-Variable Dependence
CHASM detects changes in temporal and cross-variable dependence in multivariate time series by monitoring the truncated eigenvalue sequence of a recursively estimated DMD operator, using optimal assignment and augment...
-
Accurate Evaluation of Quickest Changepoint Detectors via Non-parametric Survival Analysis
Kaplan-Meier-based non-parametric estimators for ARL and ADD in quickest changepoint detection are derived with bias bounds and shown to be asymptotically unbiased for finite sequences without extrapolation.
-
Exploring Statistical Change Point Detection Techniques for Performance Anomaly Detection at Mozilla
Ensemble voting strategies for change point detection improve F1-score by 11% over Mozilla's T-test method on a new ground-truth dataset of 174 performance time series annotated by practitioners.
-
Anomaly Detection and Root Cause Analysis for Microservice Systems
Thesis proposes BARO for metrics, EventADL for events, TORAI for multimodal RCA without call graphs, and RCAEval benchmark with systematic evaluation of causal methods.
-
Generalized Stochastic Approximation of the Log-Likelihood Ratio for Robust Sequential Change-Point Detection
A unified approximation framework for the log-likelihood ratio on polynomial/logarithmic/fractional-power bases using moments up to order 3s adapts CUSUM/GRSh/SRP procedures to non-Gaussian change-point detection and ...
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.