Pith. sign in

REVIEW 3 major objections 6 minor 2 cited by

We Need to Rethink Benchmarking in Anomaly Detection

T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A two-line quantile rule performs comparably to deep anomaly detection methods on the ADBench benchmark, and the paper argues this shows aggregate benchmarking hides the real differences that scenario-specific evaluation would reveal.

desk verdict A thoughtful, well-argued position paper on scenario-specific benchmarking, but its load-bearing claim that shared structural properties make within-scenario performance transferable is asserted, not validated. read the letter →

arxiv 2507.15584 v2 pith:DAZZRDN5 submitted 2025-07-21 cs.LG

Philipp Röchner , Simon Klüttermann , Kevin Kammler , Franz Rothlauf , Emmanuel Müller , Daniel Schlör This is my paper
classification cs.LG
keywords anomalydetectionbenchmarkingscenario-specificevaluationmetricstaxonomyunsupervisedmodelselectionpositionpaper
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This position paper argues that the apparent stagnation in anomaly detection research is largely an artifact of how algorithms are benchmarked. The authors demonstrate that a trivial algorithm, which scores each point by the fraction of its features falling outside the 5th–95th percentile range, ranks comparably to deep learning methods on the ADBench collection despite failing on simple cases such as an anomaly inside an annulus of normal points. They conclude that aggregating results across heterogeneous datasets hides real performance differences and gives practitioners no reliable way to choose an algorithm. Their proposal is to define scenarios as groups of applications sharing structural properties such as anomaly type, data modality, and processing time, and to benchmark separately within each scenario.

What carries the argument

The machinery is the notion of a scenario, defined formally through structural properties $P_{\mathrm{str}}$ and contextual properties $P_{\mathrm{ctx}}$. Structural properties---anomaly type, anomaly ratio, data modality, dimensionality, supervision, processing time, evaluation priority, and operational constraints---directly constrain algorithm design; contextual properties describe the deployment setting and are used only to help experts identify the structural properties. A scenario is the set of applications sharing the same structural properties, and the paper argues this grouping supplies the inductive basis for transferring model-selection evidence across datasets, constrains admissible preprocessing and ensembles, and fixes which metrics and evaluation protocols fit. The companion instrument is the Quantiles algorithm, a deliberately trivial baseline that isolates the failure of aggregated evaluation.

What would settle it

A decisive check would be to take a scenario defined by the paper, for example real-time sequential anomaly detection, select several datasets that satisfy those structural properties, and compare algorithm rankings within the scenario against rankings across randomly chosen datasets. If within-scenario ranks are no more consistent across datasets than across-scenario ranks, the central premise that shared structural properties make performance transferable would fail. A second falsifier would be to find an application whose structural properties match a scenario but where the best algorithm is systematically different from the scenario's best, showing the taxonomy underspecifies requirements.

Watch

Extended reading notes

Core claim

The central claim is that current aggregated anomaly detection benchmarks are inadequate, and the field should move to scenario-specific benchmarking. The paper introduces a distinction between contextual properties and structural properties; a scenario groups applications that share structural properties regardless of application domain. It argues that within such a scenario, performance on one dataset provides evidence for performance on another, because shared structural properties determine the algorithmic requirements. It supports the diagnosis with a critical-difference experiment in which the two-line Quantiles algorithm ranks second among DTE, DeepSVDD, GoAD, and OCSVM on ADBench and is not statistically different from them, even though it cannot detect an anomaly inside an annulus of normal points. On this basis the paper claims that specialized algorithms will show real advantages when evaluated per scenario, and that scenario-specific choices for preprocessing, model selection, ensembles, metrics, and evaluation protocols will make progress measurable and guidance reliable.

Load-bearing premise

The load-bearing premise is that grouping applications by structural properties such as anomaly type and processing time makes performance on one dataset in a scenario predict performance on others; if real applications within a scenario still differ in ways those properties miss, the promised reliable guidance does not follow.

Editorial extensions

If this is right

  • Overall leaderboards that average ranks across heterogeneous datasets should be supplemented or replaced by per-scenario leaderboards, because only within-scenario comparisons reveal which algorithm fits the application.
  • A new algorithm should not need to beat baselines everywhere; a clear win inside one scenario is the transferable result.
  • Benchmark dataset curation should record standardized structural-property metadata so datasets can be grouped, split, or merged into scenarios.
  • Preprocessing, hyperparameter selection, and ensemble construction become scenario-constrained design problems, making individual pipeline components reusable.
  • Practitioners can choose algorithms by looking up results for their scenario, such as real-time sequential detection, instead of extrapolating from a broad aggregate.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct test the paper does not run: train a model-selection rule on datasets in one scenario and test it on held-out datasets in the same versus different scenarios; if within-scenario transfer is no better, the taxonomy needs refinement.
  • The scenario logic also implies that when an application's structural properties change over time, the effective scenario changes, so drift in anomaly definitions is not just noise but a change in the problem instance.
  • Scenario-specific metrics like diversity, severity, and fairness could reshuffle current rankings, because ROC-AUC and PR-AUC reward detectors that repeatedly find easy anomalies rather than broad coverage.
  • A quick empirical check of the paper's diagnosis is to select ADBench datasets matching one scenario and see whether algorithm rankings stabilize; this would test whether aggregation, not method weakness, causes the apparent tie.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. This position paper argues that current aggregated benchmarking practice in anomaly detection is fundamentally misleading. The authors support this by introducing a trivial feature-wise baseline called Quantiles, which scores observations by the fraction of features outside the 5th–95th percentile range of the training data, and claim it performs competitively with deep learning methods on the ADBench benchmark. From this observation, they argue that progress in the field appears stagnant only because evaluation aggregates over heterogeneous application contexts. The paper proposes a new evaluation paradigm based on 'scenarios,' defined as groups of applications sharing the same structural properties Pstr (e.g., anomaly type, data modality, processing time), and argues that benchmarking within such scenarios would enable meaningful model selection, preprocessing choices, and metric design. The paper discusses taxonomies, benchmark design, evaluation objectives and protocols, and includes a 'Call to Action' for the community, as well as a section of alternative views with responses.

Significance. If the scenario-based benchmarking proposal is correct, it would redirect how anomaly detection algorithms are compared, moving the field away from global leaderboards toward application-aware evaluations that could give practitioners actionable guidance. The paper's strengths are its clear articulation of a plausible weakness in current evaluation (supported by a concrete, albeit under-specified, demonstration), its careful separation of contextual and structural properties, and its constructive suggestions for preprocessing, model selection, ensembles, and metrics. The authors also honestly acknowledge the risk of fragmentation and discuss alternative viewpoints. However, the central inductive premise—that shared structural properties make algorithm performance transferable within a scenario—is not validated empirically, and the motivating experiment in Figure 1 lacks reproducibility details. The paper is a genuine position statement, not a derivation, and its value depends on whether the community finds the proposed taxonomy and the transferability assumption plausible enough to act upon.

major comments (3)
  1. [Section 1 (Figure 1)] The motivating experiment in Figure 1 is under-specified, and the entire argument rests on it. Please provide a complete protocol: the list of ADBench datasets used, the train/test split and contamination settings, the hyperparameters chosen for each of the five methods, the number of random seeds, and the statistical test used to draw the critical difference diagram. The Quantiles algorithm also needs a precise description (e.g., how ties are handled, whether features are independently standardized, and what is done when a feature has zero variance). As written, the result is not reproducible. In addition, the annulus failure is asserted but never measured; if this is an illustrative example, say so explicitly, and if it is an empirical claim, include the measurement. The claim that Quantiles is 'trivial' (2 lines of code) is a rhetorical device and should be separated from the empirical comparison.
  2. [Section 3.2 (and Section 2.2)] The proposal's practical value depends on the premise that shared structural properties Pstr make algorithm performance transferable within a scenario. The paper states this premise in Section 3.2, but provides no empirical evidence that ranking consistency is higher within the scenarios of Table 1 than across them. The ADBench anomaly-type breakdown cited in Section 2.1.2 varies only a single property and does not validate multi-property scenarios. This is a load-bearing assumption for the call to action in Section 5: 'providing practitioners with reliable guidance' requires that within-scenario transfer holds. Please either add a pilot study using existing benchmarks (e.g., ADBench or Macrodata [38]) with scenario labels based on Table 1, or explicitly frame the premise as a hypothesis to be tested and soften the corresponding claims in Sections 3.2 and 5.
  3. [Section 2.2 (Table 1)] The list of structural properties in Table 1 is presented as the starting point, which is appropriate for a position paper, but the paper also claims that these properties 'determine the algorithmic requirements for all datasets in that scenario.' That strong claim is not supported by the cited work on taxonomy development [39] or by the examples in Section 2.3. At minimum, the paper should distinguish between properties that are known to affect algorithm choice (e.g., anomaly type) and those that are hypothesized to do so (e.g., processing time), and it should acknowledge that the set Pstr itself is part of the proposal to be validated rather than a settled list.
minor comments (6)
  1. [Section 1 (Figure 1)] The number of lines of code in Figure 1 is an unusual metric and is not defined; if it is meant to emphasize simplicity, define it or drop it.
  2. [Section 2.3 (Example 2.1)] The statement that failure patterns are 'often known' and therefore 'best suited to semi-supervised methods' is a strong generalization; consider softening or citing evidence.
  3. [Section 3.1] The discussion of normalization is useful but should acknowledge that z-score standardization is also a standard part of many deep anomaly detection pipelines and may not always be harmful; the recommendation should be nuanced.
  4. [Section 4.1] The claim that ROC-AUC is 'preferable when missing anomalies is the primary risk' conflates rank-based metrics with threshold-based operating points; ROC-AUC does not specify a decision threshold, so the sentence needs rewriting.
  5. [Section 6.3] The response to the 'many objectives' concern is reasonable, but the sentence 'the community only needs to reach an agreement within each scenario' underestimates the coordination cost; suggest acknowledging that even within scenarios, metric choice can be contentious.
  6. [References] Some references are incomplete (e.g., [37] lacks page numbers for the BDTA proceedings), and the citation [38] appears to be a future-dated preprint; please verify bibliographic details.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the paper is a position paper; its Quantiles demonstration is an external-benchmark measurement and the scenario framework is an explicitly inductive proposal, not a derivation from its own definitions.

full rationale

The paper contains no hidden fitted-parameter prediction or definitional equivalence. The Quantiles result in Section 1 and Figure 1 is an empirical ranking on the external ADBench benchmark using standard hyperparameters; it is used only to motivate the argument, and it does not construct or entail the paper's scenario proposal. The scenario framework in Section 2.2 is explicitly a proposal with iterative refinement, and the transferability claim in Section 3.2 is stated as an inductive basis while conceding that 'similar performance cannot be guaranteed'; therefore, it is not a self-definitional reduction. The self-citations ([3], [38], [66], [79]) appear only as example contexts, a critique of an existing benchmark, or supporting empirical illustrations, and none is load-bearing for the central claim that anomaly detection should be benchmarked within scenarios. The paper also explicitly acknowledges open challenges and alternative views in Sections 4.2 and 6, rather than suppressing them. Any weakness in the transfer premise is an empirical validation gap, not a circularity. No specific reduction of a result to its inputs by construction can be exhibited.

Assumptions & free parameters 1 free parameters · 4 assumptions · 1 invented entities

The central claim rests on a small number of domain assumptions about how anomalies are defined and how current benchmarks represent the field. The proposed scenario framework introduces the scenario construct as a new abstract entity without empirical validation. The Quantiles demonstration introduces hand-chosen quantile thresholds as a free parameter.

free parameters (1)
  • Quantiles thresholds (5th and 95th percentiles per feature) = 0.05 and 0.95 quantiles, chosen by hand
    The demonstration baseline scores points by the fraction of features outside the 5%-95% training range. The thresholds are arbitrary choices and are not justified or varied; results could depend on them.
assumptions (4)
  • domain assumption Anomalies are rare and different from normal data (Hawkins definition)
    Invoked in the introduction to frame the problem; standard in the field, but it is a modeling choice that the benchmarking critique relies on.
  • domain assumption Current aggregated benchmarks, exemplified by ADBench, fairly represent the field's evaluation practice
    The paper's critique generalizes from ADBench and similar benchmarks; if ADBench is not representative, the motivating evidence weakens. Location: Section 1, Figure 1.
  • ad hoc to paper Shared structural properties P_str imply transferability of algorithm performance within a scenario
    Assumed in Section 3.2 without empirical validation: 'performance on one dataset within a scenario provides evidence for performance on another'. This is the load-bearing assumption of the proposed framework.
  • domain assumption The no-free-lunch perspective applies to anomaly detection
    Used in Section 2 to argue no one-size-fits-all algorithm; reasonable but not demonstrated for the specific claims about scenarios.
invented entities (1)
  • Scenario (grouping of applications sharing structural properties P_str)
    purpose: Unit of benchmarking and algorithm selection, replacing global aggregation
    The paper postulates that scenario-based evaluation improves model selection and transfers, but provides no empirical validation or falsifiable prediction; it is a proposed construct, not yet a demonstrated one.

how reviews work

0 comments
Cite this review

Pith. "Pith review of We Need to Rethink Benchmarking in Anomaly Detection." pith.science (2026). https://pith.science/paper/DAZZRDN5

@misc{pith2026250715584,
  author       = {Pith},
  title        = {Pith review of: We Need to Rethink Benchmarking in Anomaly Detection},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DAZZRDN5}},
  note         = {Machine review of arXiv:2507.15584}
}
read the original abstract

Despite the continuous proposal of new anomaly detection algorithms and extensive benchmarking efforts, progress seems to stagnate, with only minor performance differences between established baselines and new algorithms. In this position paper, we argue that this stagnation is due to limitations in how we evaluate anomaly detection algorithms. In current benchmarks, a trivial algorithm that only checks for extreme values in individual features performs competitively with state-of-the-art deep learning methods, despite failing on simple cases such as anomalies within an annulus of normal points. Moreover, existing benchmarks do not adequately reflect the diversity of anomaly detection applications, making it difficult for practitioners to reliably select algorithms for their applications. Consequently, we need to rethink benchmarking in anomaly detection. In our opinion, anomaly detection should be studied using scenarios that group applications sharing relevant characteristics, defined through a common taxonomy. Benchmarking within scenarios enables scenario-specific choices for preprocessing, metrics, and model selection, clarifying which advances transfer across similar applications and providing practitioners with reliable guidance for their specific contexts.

Figures

Figures reproduced from arXiv: 2507.15584 by the authors.

Figure 1
Figure 1. Critical difference diagram. The number in brackets is the number of lines of code in each [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. The proposed scenario-specific benchmarking. Each scenario describes relevant character [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. MacrOData: New Benchmarks of Thousands of Datasets for Tabular Outlier Detection

    cs.LG 2026-02 accept novelty 8.0 of 10

    MacrOData supplies three large, curated benchmark suites totaling 2,446 datasets for tabular outlier detection, complete with standardized splits, metadata, and a public leaderboard.

  2. Evaluating Tabular Representation Learning for Network Intrusion Detection

    cs.LG 2026-05 unverdicted novelty 5.0 of 10

    Tabular representation learning for network intrusion detection exhibits strong dataset-model dependency, with supervised methods outperforming unsupervised anomaly detection and limited but possible cross-dataset gen...

Reference graph

Works this paper leans on

82 extracted references · 61 canonical work pages · cited by 2 Pith papers

  1. [38]

    MacrOData: New Benchmarks of Thousands of Datasets for Tabular Outlier Detection

    Xueying Ding, Simon Klüttermann, Haomin Wen, Yilong Chen, and Leman Akoglu. Macrodata: New benchmarks of thousands of datasets for tabular outlier detection, 2026. URL https: //arxiv.org/abs/2602.09329

    work page Pith review arXiv 2026

  2. [39]

    Nickerson, Upkar Varshney, and Jan Muntermann

    Robert C. Nickerson, Upkar Varshney, and Jan Muntermann. A method for taxonomy develop- ment and its application in information systems.Eur. J. Inf. Syst., 22(3):336–359, 2013. doi: 10.1057/EJIS.2012.26. URLhttps://doi.org/10.1057/ejis.2012.26

    doi:10.1057/ejis.2012.26 2013

  3. [1]

    Meta-survey on outlier and anomaly detection.Neurocomputing, 555:126634, 2023

    Madalina Olteanu, Fabrice Rossi, and Florian Yger. Meta-survey on outlier and anomaly detection.Neurocomputing, 555:126634, 2023. ISSN 0925-2312. doi: https://doi.org/10.1016/j. neucom.2023.126634. URL https://www.sciencedirect.com/science/article/pii/ S0925231223007579

    arXiv 2023

  4. [2]

    A comparative study of anomaly detection schemes in network intrusion detection

    Aleksandar Lazarevic, Levent Ertöz, Vipin Kumar, Aysel Ozgur, and Jaideep Srivastava. A comparative study of anomaly detection schemes in network intrusion detection. InSDM, pages 25–36. SIAM, 2003

    2003

  5. [3]

    Unsupervised anomaly detection of implausible elec- tronic health records: a real-world evaluation in cancer registries.BMC Medical Research Methodology, 23(1):125, 2023

    Philipp Röchner and Franz Rothlauf. Unsupervised anomaly detection of implausible elec- tronic health records: a real-world evaluation in cancer registries.BMC Medical Research Methodology, 23(1):125, 2023

    2023

  6. [4]

    Andrew Gadsden, and John Yawney

    Waleed Hilal, S. Andrew Gadsden, and John Yawney. Financial fraud: A review of anomaly detection techniques and recent advances.Expert Syst. Appl., 193:116429, 2022

    work page 2022

  7. [5]

    D. M. Hawkins.Identification of Outliers. Monographs on Applied Probability and Statistics. Springer, 1980

    work page 1980

  8. [6]

    Guilherme Oliveira Campos, Arthur Zimek, Jörg Sander, Ricardo J. G. B. Campello, Barbora Micenková, Erich Schubert, Ira Assent, and Michael E. Houle. On the evaluation of unsupervised outlier detection: measures, datasets, and an empirical study.Data Min. Knowl. Discov., 30(4): 891–927, 2016

    work page 2016

Show all 82 references
  1. [7]

    Adbench: Anomaly detection benchmark

    Songqiao Han, Xiyang Hu, Hailiang Huang, Mingqi Jiang, and Yue Zhao. Adbench: Anomaly detection benchmark. InNeural Information Processing Systems (NeurIPS), 2022

    2022

  2. [8]

    Learning multiple layers of features from tiny images

    Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. Master’s thesis, University of Tront, 2009

    2009

  3. [9]

    Imagenet: A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. InCVPR, pages 248–255. IEEE Computer Society, 2009

    2009

  4. [10]

    Efficient algorithms for mining outliers from large data sets

    Sridhar Ramaswamy, Rajeev Rastogi, and Kyuseok Shim. Efficient algorithms for mining outliers from large data sets. InSIGMOD Conference, pages 427–438. ACM, 2000

    2000

  5. [11]

    An anomaly detection approach based on isolation forest algorithm for streaming data using sliding window

    Zhiguo Ding and Minrui Fei. An anomaly detection approach based on isolation forest algorithm for streaming data using sliding window. InICONS, pages 12–17. International Federation of Automatic Control, 2013

    2013

  6. [12]

    Unsupervised anomaly detection algorithms on real-world data: How many do we need?J

    Roel Bouman, Zaharah Bukhsh, and Tom Heskes. Unsupervised anomaly detection algorithms on real-world data: How many do we need?J. Mach. Learn. Res., 25:105:1–105:34, 2024

    2024

  7. [13]

    On diffusion modeling for anomaly detection

    Victor Livernoche, Vineet Jain, Yashar Hezaveh, and Siamak Ravanbakhsh. On diffusion modeling for anomaly detection. InICLR. OpenReview.net, 2024

    2024

  8. [14]

    Vandermeulen, Alexander Binder, Emmanuel Müller, and Marius Kloft

    Lukas Ruff, Nico Görnitz, Lucas Deecke, Shoaib Ahmed Siddiqui, Robert A. Vandermeulen, Alexander Binder, Emmanuel Müller, and Marius Kloft. Deep one-class classification. InICML, volume 80 ofProceedings of Machine Learning Research, pages 4390–4399. PMLR, 2018

    2018

  9. [15]

    Deep autoencoding gaussian mixture model for unsupervised anomaly detection

    Bo Zong, Qi Song, Martin Renqiang Min, Wei Cheng, Cristian Lumezanu, Dae ki Cho, and Haifeng Chen. Deep autoencoding gaussian mixture model for unsupervised anomaly detection. InInternational Conference on Learning Representations, 2018. URL https: //api.semanticscholar.org/Co...

    2018

  10. [16]

    Sahand Hariri, Matias Carrasco Kind, and Robert J. Brunner. Extended isolation forest.CoRR, abs/1811.02141, 2018

    2018 arXiv

  11. [17]

    Classification-based anomaly detection for general data

    Liron Bergman and Yedid Hoshen. Classification-based anomaly detection for general data. InICLR. OpenReview.net, 2020. URL http://dblp.uni-trier.de/db/conf/iclr/ iclr2020.html#BergmanH20

    2020

  12. [18]

    One-class classifier networks for target recognition applications

    Mr Moya. One-class classifier networks for target recognition applications. InProc. World Congress on Neural Networks, 1993, pages 797–801, 1993

    1993

  13. [19]

    Anomaly detection: A survey.ACM Comput

    Varun Chandola, Arindam Banerjee, and Vipin Kumar. Anomaly detection: A survey.ACM Comput. Surv., 41(3):15:1–15:58, July 2009. ISSN 0360-0300. doi: 10.1145/1541880.1541882. URLhttps://doi.org/10.1145/1541880.1541882

    2009

  14. [20]

    Aggarwal.Outlier Analysis

    Charu C. Aggarwal.Outlier Analysis. Springer, 2013. ISBN 978-1-4614-6396-2. URL http://dx.doi.org/10.1007/978-1-4614-6396-2

    2013 doi

  15. [21]

    On the nature and types of anomalies: a review of deviations in data.Interna- tional Journal of Data Science and Analytics, 12(4):297–331, October 2021

    Ralph Foorthuis. On the nature and types of anomalies: a review of deviations in data.Interna- tional Journal of Data Science and Analytics, 12(4):297–331, October 2021. ISSN 2364-415X, 2364-4168. doi: 10.1007/s41060-021-00265-1. URL https://link.springer.com/10. 1007/s41060-0...

    2021 doi

  16. [22]

    No free lunch but a cheaper supper: A general framework for streaming anomaly detection.Expert Systems with Applications, 155:113453, October 2020

    Ece Calikus, Sławomir Nowaczyk, Anita Sant’Anna, and Onur Dikmen. No free lunch but a cheaper supper: A general framework for streaming anomaly detection.Expert Systems with Applications, 155:113453, October 2020. ISSN 0957-4174. doi: 10.1016/j.eswa.2020.113453. URLhttps://www...

    2020

  17. [23]

    E. H. Simpson. The Interpretation of Interaction in Contingency Tables.Journal of the Royal Statistical Society. Series B (Methodological), 13(2):238–241, 1951. ISSN 0035-9246. URL https://www.jstor.org/stable/2984065. Publisher: [Royal Statistical Society, Oxford University Press]

    1951

  18. [24]

    Anomaly Detection in Computer Networks: A State-of-the-Art Review.J

    Sherenaz Al-Haj Baddar, Alessio Merlo, and Mauro Migliardi. Anomaly Detection in Computer Networks: A State-of-the-Art Review.J. Wirel. Mob. Networks Ubiquitous Comput. Dependable Appl., 5(4):29–64, 2014

    2014

  19. [25]

    Ruchi Makani and B. V . R. Reddy. Taxonomy of Machine Leaning Based Anomaly Detec- tion and its suitability.Procedia Computer Science, 132:1842–1849, January 2018. ISSN 1877-0509. doi: 10.1016/j.procs.2018.05.133. URL https://www.sciencedirect.com/ science/article/pii/S1877050...

    2018 doi

  20. [26]

    A taxonomy and platform for anomaly detection

    Gheorghe Sebestyen, Anca Hangan, Zoltan Czako, and Gyorgy Kovacs. A taxonomy and platform for anomaly detection. In2018 IEEE International Conference on Automation, Quality and Testing, Robotics (AQTR), pages 1–6, May 2018. doi: 10.1109/AQTR.2018.8402710. URL https://ieeexplor...

    2018

  21. [27]

    Gilberto Fernandes, Joel J. P. C. Rodrigues, Luiz Fernando Carvalho, Jalal F. Al-Muhtadi, and Mario Lemes Proença. A comprehensive survey on network anomaly detection.Telecommunica- tion Systems, 70(3):447–489, March 2019. ISSN 1572-9451. doi: 10.1007/s11235-018-0475-8. URLhtt...

    2019 doi

  22. [28]

    Deep Learning for Anomaly Detection: A Survey, January 2019

    Raghavendra Chalapathy and Sanjay Chawla. Deep Learning for Anomaly Detection: A Survey, January 2019. URLhttp://arxiv.org/abs/1901.03407. arXiv:1901.03407 [cs]

    2019 arXiv

  23. [29]

    A Taxonomy of Anomalies in Log Data, November 2021

    Thorsten Wittkopp, Philipp Wiesner, Dominik Scheinert, and Odej Kao. A Taxonomy of Anomalies in Log Data, November 2021. URL http://arxiv.org/abs/2111.13462. arXiv:2111.13462 [cs]

    2021 arXiv

  24. [30]

    Deep Learning for Anomaly Detection: A Review.ACM Computing Surveys, 54(2):1–38, March 2022

    Guansong Pang, Chunhua Shen, Longbing Cao, and Anton van den Hengel. Deep Learning for Anomaly Detection: A Review.ACM Computing Surveys, 54(2):1–38, March 2022. ISSN 0360-0300, 1557-7341. doi: 10.1145/3439950. URL http://arxiv.org/abs/2007.02500. arXiv:2007.02500 [cs]. 11

    2022 arXiv

  25. [31]

    Taxonomy of Anomaly Detection Techniques in Crowd Scenes.Sensors, 22(16):6080, January 2022

    Amnah Aldayri and Waleed Albattah. Taxonomy of Anomaly Detection Techniques in Crowd Scenes.Sensors, 22(16):6080, January 2022. ISSN 1424-8220. doi: 10.3390/s22166080. URL https://www.mdpi.com/1424-8220/22/16/6080. Number: 16 Publisher: Multidis- ciplinary Digital Publishing Institute

    2022 doi

  26. [32]

    Navigating the metric maze: a taxonomy of evaluation metrics for anomaly detection in time series.Data Mining and Knowledge Discovery, 38 (3):1027–1068, May 2024

    Sondre Sørbø and Massimiliano Ruocco. Navigating the metric maze: a taxonomy of evaluation metrics for anomaly detection in time series.Data Mining and Knowledge Discovery, 38 (3):1027–1068, May 2024. ISSN 1573-756X. doi: 10.1007/s10618-023-00988-8. URL https://doi.org/10.1007...

    2024 doi

  27. [33]

    Estevez-Tapiador, Pedro Garcia-Teodoro, and Jesus E

    Juan M. Estevez-Tapiador, Pedro Garcia-Teodoro, and Jesus E. Diaz-Verdejo. Anomaly de- tection methods in wired networks: a survey and taxonomy.Computer Communications, 27 (16):1569–1584, October 2004. ISSN 0140-3664. doi: 10.1016/j.comcom.2004.07.002. URL https://www.scienced...

    2004 doi

  28. [34]

    Taxonomy of Anomaly Based Intrusion Detection System: A Review.International Journal of Scientific and Research Publications, 2 (12), 2012

    Manasi Gyanchandani, J L Rana, and R N Yadav. Taxonomy of Anomaly Based Intrusion Detection System: A Review.International Journal of Scientific and Research Publications, 2 (12), 2012

    2012

  29. [35]

    The elephant in the room: Towards a reliable time-series anomaly detection benchmark.Advances in Neural Information Processing Systems, 37:108231– 108261, 2024

    Qinghua Liu and John Paparrizos. The elephant in the room: Towards a reliable time-series anomaly detection benchmark.Advances in Neural Information Processing Systems, 37:108231– 108261, 2024

    2024

  30. [36]

    Bmad: Benchmarks for medical anomaly detection

    Jinan Bao, Hanshi Sun, Hanqiu Deng, Yinsheng He, Zhaoxiang Zhang, and Xingyu Li. Bmad: Benchmarks for medical anomaly detection. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4042–4053, 2024

    2024

  31. [37]

    Netflow datasets for machine learning-based network intrusion detection systems

    Mohanad Sarhan, Siamak Layeghy, Nour Moustafa, and Marius Portmann. Netflow datasets for machine learning-based network intrusion detection systems. InBig data technologies and applications: 10th EAI international conference, BDTA 2020, and 13th EAI international conference on...

    2020

  32. [40]

    Marius Zollner

    Daniel Bogdoll, Maximilian Nitsche, and J. Marius Zollner. Anomaly detection in autonomous driving: A survey. In2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), page 4487–4498. IEEE, June 2022. doi: 10.1109/cvprw56347.2022. 00495. URLhttp:...

    2022

  33. [41]

    A survey of anomaly detection methods for power grids.International Journal of Information Security, 22(6):1799–1832, Dec 2023

    Srinidhi Madabhushi and Rinku Dewri. A survey of anomaly detection methods for power grids.International Journal of Information Security, 22(6):1799–1832, Dec 2023. ISSN 1615-5270. doi: 10.1007/s10207-023-00720-z. URL https://doi.org/10.1007/ s10207-023-00720-z

    2023 doi

  34. [42]

    Lochner and B.A

    M. Lochner and B.A. Bassett. Astronomaly: Personalised active anomaly detection in as- tronomical data.Astronomy and Computing, 36:100481, 2021. ISSN 2213-1337. doi: https://doi.org/10.1016/j.ascom.2021.100481. URL https://www.sciencedirect.com/ science/article/pii/S2213133721000354

    2021

  35. [43]

    Financial fraud: a review of anomaly detection techniques and recent advances.Expert systems With applications, 193, 2022

    Waleed Hilal, S Andrew Gadsden, and John Yawney. Financial fraud: a review of anomaly detection techniques and recent advances.Expert systems With applications, 193, 2022

    2022

  36. [44]

    Sunil Kumar Singh and Pradeep Kumar Roy. Detecting malicious dns over https traffic us- ing machine learning.2020 International Conference on Innovation and Intelligence for Informatics, Computing and Technologies (3ICT), pages 1–6, 2020. URL https://api. semanticscholar.org/C...

    2020

  37. [45]

    Anomaly detection for automated data quality monitoring in the cms detector, 2025

    Andrew Brinkerhoff, Chosila Sutantawibul, Robert White, Caio Daumann, Chad Freer, Indara Suarez, Samuel May, Vivan Nguyen, Jonathan Guiang, Bennett Marsh, Darin Acosta, Alex Aubuchon, Emanuela Barberis, Aaron Bundock, Evan Collins, Preston Epps, Johannes Erdmann, Henning Flaec...

    2025

  38. [46]

    Idrisov, Divine Okeke, Abdullatif Albaseer, Mohamed Abdallah, and Federico M

    Ildar N. Idrisov, Divine Okeke, Abdullatif Albaseer, Mohamed Abdallah, and Federico M. Ibanez. Leveraging digital twin and machine learning techniques for anomaly detection in power electronics dominated grid, 2025. URLhttps://arxiv.org/abs/2501.13474

    2025 arXiv

  39. [47]

    Anomaly detection in double-entry bookkeeping data by federated learning system with non- model sharing approach, 2025

    Sota Mashiko, Yuji Kawamata, Tomoru Nakayama, Tetsuya Sakurai, and Yukihiko Okada. Anomaly detection in double-entry bookkeeping data by federated learning system with non- model sharing approach, 2025. URLhttps://arxiv.org/abs/2501.12723

    2025 arXiv

  40. [48]

    ADGym: Design Choices for Deep Anomaly Detection, October

    Minqi Jiang, Chaochuan Hou, Ao Zheng, Songqiao Han, Hailiang Huang, Qingsong Wen, Xiyang Hu, and Yue Zhao. ADGym: Design Choices for Deep Anomaly Detection, October

  41. [49]

    Muñoz, Rob J

    Sevvandi Kandanaarachchi, Mario A. Muñoz, Rob J. Hyndman, and Kate Smith-Miles. On normalization and algorithm selection for unsupervised outlier detection.Data Min. Knowl. Discov., 34(2):309–354, 2020

    2020

  42. [50]

    Breunig, Hans-Peter Kriegel, Raymond T

    Markus M. Breunig, Hans-Peter Kriegel, Raymond T. Ng, and Jörg Sander. LOF: identifying density-based local outliers. In Weidong Chen, Jeffrey F. Naughton, and Philip A. Bernstein, editors,Proceedings of the 2000 ACM SIGMOD International Conference on Management of Data, May 1...

  43. [51]

    Zheng Li, Yue Zhao, Xiyang Hu, Nicola Botta, Cezar Ionescu, and George H. Chen. ECOD: unsupervised outlier detection using empirical cumulative distribution functions.IEEE Trans. Knowl. Data Eng., 35(12):12181–12193, 2023

    2023

  44. [52]

    Remarks on Some Nonparametric Estimates of a Density Function.The Annals of Mathematical Statistics, 27(3):832 – 837, 1956

    Murray Rosenblatt. Remarks on Some Nonparametric Estimates of a Density Function.The Annals of Mathematical Statistics, 27(3):832 – 837, 1956. doi: 10.1214/aoms/1177728190

    1956

  45. [53]

    Anomaly detection using autoencoders with nonlinear dimensionality reduction

    Mayu Sakurada and Takehisa Yairi. Anomaly detection using autoencoders with nonlinear dimensionality reduction. InProceedings of the MLSDA 2014 2nd Workshop on Machine Learning for Sensory Data Analysis, MLSDA’14, page 4–11, 2014. ISBN 9781450331593. doi: 10.1145/2689746.2689747

    2014

  46. [54]

    Copod: Copula-based outlier detection

    Zheng Li, Yue Zhao, Nicola Botta, Cezar Ionescu, and Xiyang Hu. Copod: Copula-based outlier detection. In2020 IEEE International Conference on Data Mining (ICDM), 2020. doi: 10.1109/ICDM50108.2020.00135

    2020

  47. [55]

    Variational inference with normalizing flows

    Danilo Jimenez Rezende and Shakir Mohamed. Variational inference with normalizing flows. InProceedings of the 32nd International Conference on International Conference on Machine Learning - Volume 37, ICML’15, page 1530–1538. JMLR.org, 2015

    2015

  48. [56]

    A multivariate gaussian mixture model for anomaly detection in transient current signature of control element drive mechanism.Nuclear Engineer- ing and Design, 402:112098, 2023

    Adebena Oluwasegun and Jae-Cheon Jung. A multivariate gaussian mixture model for anomaly detection in transient current signature of control element drive mechanism.Nuclear Engineer- ing and Design, 402:112098, 2023. ISSN 0029-5493

    2023

  49. [57]

    A survey on gans for anomaly detection, 2021

    Federico Di Mattia, Paolo Galeone, Michele De Simoni, and Emanuele Ghelfi. A survey on gans for anomaly detection, 2021

    2021

  50. [58]

    Hyperparameter sensitivity in deep outlier detection: Analysis and a scalable hyper-ensemble solution.Advances in Neural Information Processing Systems, 35:9603–9616, 2022

    Xueying Ding, Lingxiao Zhao, and Leman Akoglu. Hyperparameter sensitivity in deep outlier detection: Analysis and a scalable hyper-ensemble solution.Advances in Neural Information Processing Systems, 35:9603–9616, 2022. 13

    2022

  51. [59]

    Automatic unsupervised outlier model selection

    Yue Zhao, Ryan Rossi, and Leman Akoglu. Automatic unsupervised outlier model selection. In M. Ranzato, A. Beygelzimer, Y . Dauphin, P.S. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems, volume 34, pages 4489–4502. Curran Associates, In...

    2021

  52. [60]

    Rossi, and Leman Akoglu

    Yue Zhao, Ryan A. Rossi, and Leman Akoglu. Automating outlier detection via meta-learning,

  53. [61]

    Hypex: Hyperparameter optimiza- tion in time series anomaly detection

    Sebastian Schmidl, Phillip Wenig, and Thorsten Papenbrock. Hypex: Hyperparameter optimiza- tion in time series anomaly detection. InDatenbanksysteme für Business, Technologie und Web,

  54. [62]

    Flaml: A fast and lightweight automl library

    Chi Wang, Qingyun Wu, Markus Weimer, and Erkang (Eric) Zhu. Flaml: A fast and lightweight automl library. InFourth Conference on Machine Learning and Systems (MLSys 2021), April 2021. URL https://www.microsoft.com/en-us/research/publication/ flaml-a-fast-and-lightweight-automl...

    2021

  55. [63]

    Internal evaluation of unsupervised outlier detection.ACM Transactions on Knowledge Discovery from Data (TKDD), 14(4):1–42, 2020

    Henrique O Marques, Ricardo JGB Campello, Jörg Sander, and Arthur Zimek. Internal evaluation of unsupervised outlier detection.ACM Transactions on Knowledge Discovery from Data (TKDD), 14(4):1–42, 2020

    2020

  56. [64]

    URLhttps://api.semanticscholar.org/CorpusID:257104910

  57. [65]

    A study on anomaly detection ensembles.Journal of Applied Logic, 21, 12 2016

    Alvin Chiang, Esther David, Yuh-Jye Lee, Guy Leshem, and Yi-Ren Yeh. A study on anomaly detection ensembles.Journal of Applied Logic, 21, 12 2016. doi: 10.1016/j.jal.2016.12.002

    2016 doi

  58. [66]

    Evaluating and comparing heterogeneous ensemble methods for unsupervised anomaly detection

    Simon Klüttermann and Emmanuel Müller. Evaluating and comparing heterogeneous ensemble methods for unsupervised anomaly detection. InIJCNN, 2023

    2023

  59. [67]

    Ma, Yue Zhao, Xiaorong Zhang, and Leman Akoglu

    Martin Q. Ma, Yue Zhao, Xiaorong Zhang, and Leman Akoglu. The need for unsupervised outlier model selection: A review and evaluation of internal evaluation strategies.SIGKDD Explor. Newsl., 25(1):19–35, July 2023. ISSN 1931-0145. doi: 10.1145/3606274.3606277. URLhttps://doi.or...

    2023

  60. [68]

    Matthew B. A. McDermott, Haoran Zhang, Lasse Hyldig Hansen, Giovanni Angelotti, and Jack Gallifant. A closer look at auroc and auprc under class imbalance, 2025. URL https: //arxiv.org/abs/2401.06091

    2025 arXiv

  61. [69]

    Diversity in recommender systems - A survey.Knowl

    Matevz Kunaver and Tomaz Pozrl. Diversity in recommender systems - A survey.Knowl. Based Syst., 123:154–162, 2017

    2017

  62. [70]

    Springer, 04 2017

    Charu Aggarwal and Saket Sathe.Outlier Ensembles. Springer, 04 2017. ISBN 978-3-319- 54764-0. doi: 10.1007/978-3-319-54765-7

    2017 doi

  63. [71]

    Y . Dai, M. Jayaratne, and B. Jayatilleke. Question-aware outlier answer detection for fairer ai scoring of interviews. In38th Annual Conference of the Society for Industrial & Organizational Psychology (SIOP), apr 2023

    2023

  64. [72]

    Guoxiang Zhong, Fagui Liu, Jun Jiang, and C. L. Philip Chen. Causeformer: Interpretable anomaly detection with stepwise attention for cloud service.IEEE Transactions on Network and Service Management, 21(1):637–652, 2024. doi: 10.1109/TNSM.2023.3299846

    2024

  65. [73]

    Subha and J

    S. Subha and J. G. R. Sathiaseelan. Effective anomaly detection approach to classify noisy data using robust noise detection and removal technique in iot healthcare data.SN Computer Science, 4(5):522, Jul 2023. ISSN 2661-8907. doi: 10.1007/s42979-023-01890-2. URL https://doi.o...

    2023 doi

  66. [74]

    On bootstrapping machine learning performance predictors via analytical models.CoRR, abs/1410.5102, 2014

    Diego Didona and Paolo Romano. On bootstrapping machine learning performance predictors via analytical models.CoRR, abs/1410.5102, 2014

    2014 arXiv

  67. [75]

    A bootstrap algorithm for fast supervised learning, 2023

    Michael A Kouritzin, Stephen Styles, and Beatrice-Helen Vritsiou. A bootstrap algorithm for fast supervised learning, 2023. URLhttps://arxiv.org/abs/2305.03099. 14

    2023 arXiv

  68. [76]

    Isolation-based anomaly detection.ACM Trans

    Fei Tony Liu, Kai Ming Ting, and Zhi-Hua Zhou. Isolation-based anomaly detection.ACM Trans. Knowl. Discov. Data, 6(1):3:1–3:39, 2012

    2012

  69. [77]

    Bootstrap methods: Another look at the jackknife.The Annals of Statistics, 7(1):1–26, 1979

    B Efron. Bootstrap methods: Another look at the jackknife.The Annals of Statistics, 7(1):1–26, 1979

    1979

  70. [78]

    Anomaly detection techniques for unsupervised machine learning, 2022

    Albin Iivari. Anomaly detection techniques for unsupervised machine learning, 2022. URL https://www.diva-portal.org/smash/get/diva2:1674371/FULLTEXT01.pdf. 15 credits

    2022

  71. [79]

    Area under the precision-recall curve: point estimates and confidence intervals

    Kendrick Boyd, Kevin H Eng, and C David Page. Area under the precision-recall curve: point estimates and confidence intervals. InMachine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2013, Prague, Czech Republic, September 23-27, 2013, Proceedin...

    2013

  72. [82]

    About test-time training for outlier detection, 2024

    Simon Klüttermann and Emmanuel Müller. About test-time training for outlier detection, 2024. URLhttps://arxiv.org/abs/2404.03495. 15

    2024 arXiv

  73. [2021]

    URLhttps://arxiv.org/abs/2009.10606

    2009 arXiv

  74. [2023]

    arXiv:2309.15376 [cs]

    URLhttp://arxiv.org/abs/2309.15376. arXiv:2309.15376 [cs]

    arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.