Pith. sign in

REVIEW 1 major objections 7 minor 1 cited by

Are Anomaly Scores Telling the Whole Story? A Benchmark for Multilevel Anomaly Detection

T0 review · 1 major / 7 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Anomaly scores that reflect severity: multimodal large language models outperform conventional detectors in a new six-dataset benchmark.

desk verdict Useful benchmark idea and thorough experiments, but the headline MLLM advantage is confounded by the prompts giving away the severity rubric. read the letter →

arxiv 2411.14515 v1 pith:3SXFRUD6 submitted 2024-11-21 cs.LG cs.AIcs.CV

Tri Cao , Minh-Huy Trinh , Ailin Deng , Quoc-Nam Nguyen , Khoa Duong , Ngai-Man Cheung , Bryan Hooi This is my paper · ORCID
classification cs.LGcs.AIcs.CV
keywords multilevelanomalydetectionseverityalignmentMAD-BenchmultimodallargelanguagemodelsscoresC-indexKendallTau-bbenchmark
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that standard anomaly detection scores measure deviation from normality, not the practical severity of an anomaly, and that this gap undermines real-world use. To make the gap measurable, the authors introduce Multilevel Anomaly Detection (MAD), a setting where the anomaly score should rank severity levels, and build MAD-Bench, a benchmark spanning one-class novelty detection, industrial inspection, and medical imaging. The central finding is that MLLM-based baselines, prompted with a few normal reference images, assign severity-aligned scores more consistently than conventional unsupervised anomaly detectors on all datasets except VisA-MAD. If the benchmark's severity labels are accepted, the paper establishes that prior knowledge and test-time reference comparison matter more for severity alignment than the inductive biases of reconstruction- or distance-based detectors.

What carries the argument

The load-bearing object is the MAD benchmark itself: six datasets (MultiDogs-MAD, MVTec-MAD, VisA-MAD, DRD-MAD, Covid19-MAD, SkinLesion-MAD) with severity levels either taken from existing medical labels or manually assigned from class labels. The evaluation protocol uses two rank-based metrics: the C-index, a generalization of AUROC that measures whether higher-severity samples receive higher scores, and Kendall's Tau-b, a stricter measure that also penalizes score ties within the same severity level. The MLLM baselines work by a few-shot prompting scheme: three normal images plus one inference image are fed to the model with a domain-specific severity rubric, asking for a 0-100 score, so that the model's world knowledge and reasoning substitute for training on anomalous data.

What would settle it

Re-annotate the severity levels of MVTec-MAD, VisA-MAD, MultiDogs-MAD, and SkinLesion-MAD independently by domain experts (e.g., plant engineers for MVTec, dermatologists for SkinLesion) and rerun the benchmark. If the relative ordering of MLLM versus conventional methods changes materially, the paper's central finding would collapse to an artifact of the authors' annotations.

Watch

Extended reading notes

Core claim

The paper's central claim, stated as Finding 1 in Section 5.2, is that MLLM-based models 'demonstrated consistently better performance than conventional methods across all datasets, except the VisA-MAD dataset,' when evaluated on severity alignment via C-index and Kendall's Tau-b. On MAD-Bench, the best MLLM baselines achieve an average C-index around 81%, versus roughly 74% for the best conventional methods, with the gap largest on medical datasets (DRD-MAD and Covid19-MAD), where conventional methods fall near 65%. The paper also reports three secondary findings: binary and multilevel detection metrics are strongly correlated overall, yet some models, especially MLLMs, rank better on multilevel than binary evaluation; conventional models are biased toward the spatial area of the anomalous region, while MLLMs show less of this bias; and all models degrade under input corruption, particularly when fine-grained features are required.

Load-bearing premise

The benchmark's ground truth is the authors' manual severity assignments for four of the six datasets; if those assignments do not match the severity that practitioners actually care about, then every model ranking and finding measures the authors' rubric rather than practical severity.

Editorial extensions

If this is right

  • If severity alignment is the goal, conventional AD models cannot be evaluated by binary AUROC alone; MAD-Bench provides a protocol that any new detector can be measured against.
  • MLLM-based scoring, especially with few-shot normal references, becomes a strong baseline for any anomaly-detection application where severity ordering matters, such as medical triage or industrial quality control.
  • The documented area bias of conventional methods implies that detectors relying on spatial features will systematically under-score small but dangerous defects, which should be a known limitation in deployment.
  • The strong correlation between binary and multilevel metrics means that a model that fails at binary detection is unlikely to excel at severity ranking, simplifying model selection in practice.
  • The finding that all models lose performance under input corruption, particularly noise, suggests that robustness to image degradation must be tested before deploying severity-aware detectors in real environments.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's comparison is implicitly a test of whether semantic understanding can be separated from low-level feature deviation; if the finding holds, it suggests that severity is a semantic property that conventional detectors cannot capture without supervision.
  • A natural extension is a hybrid pipeline where a fast conventional detector localizes anomalies and an MLLM assigns the severity score; the paper itself flags this direction in Section 6, but does not test it.
  • Because the MLLM baselines were evaluated with temperature 0 and a fixed set of normal reference images, one could test how sensitive the rankings are to the choice of reference images, which the paper does not report.
  • The manual severity assignments for MVTec-MAD, VisA-MAD, MultiDogs-MAD, and SkinLesion-MAD depend on the authors' interpretation of economic or clinical impact; re-annotating the same data with domain experts could change the relative standing of the methods.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

1 major / 7 minor

Summary. The paper proposes Multilevel Anomaly Detection (MAD), a setting in which an anomaly score should reflect the practical severity of an anomaly rather than only binary normality. It introduces MAD-Bench, a benchmark built from six datasets (MultiDogs-MAD, MVTec-MAD, VisA-MAD, DRD-MAD, Covid19-MAD, and SkinLesion-MAD) with multiple severity levels, and evaluates ten conventional unsupervised anomaly detectors alongside four MLLM-based baselines that receive a few normal reference images plus a textual prompt. Performance is measured by AUROC, C-index, and Kendall's Tau-b. The central finding is that MLLM-based models generally achieve better severity alignment than conventional models, which the authors attribute to prior knowledge and contextual reasoning. Six additional findings examine the correlation between binary and multilevel performance, bias toward anomalous area, detection per severity level, robustness when light anomalies are included as normal, and sensitivity to input corruption. The code and datasets are promised for public release.

Significance. The benchmark is a useful and timely resource: it defines a concrete evaluation protocol for severity-aware anomaly scores, covers multiple application domains, and includes both open-source conventional baselines and proprietary MLLM baselines with documented prompts. The emphasis on severity alignment addresses a real gap in anomaly detection evaluation. However, the headline claim that MLLMs are consistently better at severity alignment is undermined by an information asymmetry between the compared model families: the MLLM prompts in Appendix B contain the same severity taxonomy and score brackets that define the ground-truth labels in Section 4.1, while conventional models receive no semantic severity information. The manual severity labels for four datasets are also not externally validated. If these concerns are addressed, the benchmark could serve as a valuable community resource; in its current form, the central comparative finding should be regarded as not yet established.

major comments (1)
  1. [Tables 3–6; Appendix F] All reported results are point estimates without error bars, confidence intervals, or significance tests. The averaged C-index values in Table 3 show small gaps between top-ranked models (e.g., MMAD-4o at 81.24 vs. MMAD-Sonnet at 79.77, and RRD at 76.14 vs. PNI at 74.37); without per-subset variance it is impossible to tell whether these differences, and the ranks derived from them, are meaningful. The paper's headline Finding 1 depends on such differences. I request that the authors report per-subset standard deviations or bootstrap intervals, and perform a paired significance test (e.g., Wilcoxon signed-rank across dataset subsets) for the MLLM-versus-conventional comparison.
minor comments (7)
  1. [References] Reference [1] lists the author as 'Ahdasdwdasd,' which appears to be a placeholder or corrupted name; please verify the citation for the normal-skin dataset.
  2. [Table 10] In the header of Table 10, the column 'Fryum' appears twice; the second occurrence is presumably a different subset and should be renamed or removed.
  3. [Section 4.1] The sentence 'For the remaining datasets, which contain class labels for each sample. (e.g., defect types, disease types, or class names), we manually assign severity levels based on these class labels.' contains a misplaced period after 'sample'; the sentence should be joined.
  4. [Throughout] The dataset name is spelled inconsistently: 'Visa-MAD' in Section 5.2, 'ViSA-MAD' in Figure 3, and 'VisA-MAD' elsewhere. Please standardize to 'VisA-MAD'.
  5. [Appendix B] The prompts contain minor typos, including 'anwser' and 'even you are not sure about the anwser' in the MultiDogs prompt; please proofread the appendix text.
  6. [Table 13] The header 'Leve 0' in Table 13 should be 'Level 0'.
  7. [Table 3] The table caption states that the best MLLM and best conventional baselines are highlighted in red and blue, but the visible reproduction contains no color; please ensure the colors are shown in the final version or use symbols.

Circularity Check

1 steps flagged · score 6.0 of 10

MLLM prompts supply the same severity rubric used to construct the ground-truth labels, making the central Finding 1 comparison partially circular.

  1. self definitional [Section 4.1.2 vs Appendix B.2 (industrial prompt); Finding 1 in Section 5.2]
    "For the remaining datasets, which contain class labels for each sample, we manually assign severity levels based on these class labels. ... Level 1 (minor defects that are easily repairable), Level 2 (moderate defects with some economic impact), and Level 3 (severe defects with high economic impact). ... If the inference image shows minor defects that are unlikely to impact functionality and may be easily repairable, assign a score between 1 and 30."

    The C-index and Kendall's Tau-b reward scores ordered exactly by the authors' severity levels. Those levels are created by the authors' manual assignment from class labels, and the MLLM prompt supplies the same level descriptions and the corresponding score intervals. Thus the MLLM's severity-aligned scores are instructions followed, not an independently discovered property: the benchmark's ground-truth ordering is written into the prompt. Conventional baselines (Skip-GAN, PatchCore, RRD, etc.) never receive this rubric and output only deviation scores, so the comparison in Finding 1 is asymmetric by construction. The zero-shot results in Table 6 remove normal reference images but keep the score-range instructions, so they do not control for rubric injection.

full rationale

The paper does not fit parameters to the target metric, so the usual fitted-input circularity is absent. However, the central benchmark design embeds the manually created severity rubric into the MLLM prompts. The ground-truth labels in Section 4.1 are defined by the same level descriptions and score ranges that the prompts in Appendix B instruct the MLLMs to use. Therefore, the high C-index of MLLMs in Finding 1 is partly a direct consequence of handing them the answer key, while conventional baselines receive no such rubric. This makes the comparison between MLLMs and conventional methods asymmetric and makes the headline result partially circular. The remaining content—the perceptual mapping from images to severity levels—is not circular, and the paper does present detailed per-subset results that allow external scrutiny. Score 6: the central Finding 1 reduces in part to rubric injection, but not purely to a fit or to a self-citation chain.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The paper is an empirical benchmark rather than a parameter-free derivation. Its central claims rest on hand-assigned severity labels, on prompt rubrics that encode the same labels, and on standard rank-correlation metrics. No new physical entities or model components are introduced beyond the MMAD prompting wrappers, which are not independent postulated entities.

free parameters (6)
  • Manual severity labels for MVTec-MAD = 4 levels across 14 object and texture categories
    Assigned by the authors based on perceived economic impact of each defect class; no external cost data or inter-rater reliability is reported.
  • Manual severity labels for VisA-MAD = 4 levels across 9 subsets
    Assigned by the authors; the mapping from defect classes to levels is not fully specified in the main text.
  • Manual severity labels for MultiDogs-MAD = 5 levels: normal breed, other dog breeds, cats, birds, flowers
    The class-similarity ordering is chosen by the authors; another ordering could change model rankings.
  • Manual severity labels for SkinLesion-MAD = 3 anomaly levels: benign, precancerous, cancerous
    The grouping of lesion types into three levels is a design choice informed by doctor consensus, but the specific mapping is not externally benchmarked.
  • MLLM prompt score brackets = e.g., 1-30, 31-70, 71-100 for industrial inspection
    Hand-designed ranges in Appendix B directly mirror the severity levels used to label the test sets, making the MLLM scoring protocol partly self-referential.
  • Exclusion threshold for ambiguous classes = unspecified
    Section 4.1 says ambiguous classes or samples that do not reflect assigned class names are excluded, but no quantitative criterion is given.
assumptions (5)
  • domain assumption Severity is an ordinal property of anomalies that can be consistently labeled from class-level semantics.
    Section 4.1 relies on this for manual severity assignment in MVTec-MAD, VisA-MAD, MultiDogs-MAD, and SkinLesion-MAD.
  • domain assumption The economic impact of a defect can be inferred from its defect class in MVTec and VisA.
    Used to assign industrial severity levels; no cost data or domain-expert validation is provided.
  • standard math C-index and Kendall's Tau-b are valid measures of severity alignment.
    Section 3.2 adopts these standard rank-correlation metrics; their formulas are conventional and no derivation error is apparent.
  • ad hoc to paper MLLM 0-100 anomaly scores are directly comparable with anomaly scores from conventional models.
    Section 5.2 ranks both model families on the same metrics, but they receive different information: MLLMs get prompts and reference images while conventional models only get normal training data.
  • domain assumption The training set contains only L0 normal samples.
    Section 3.1 defines Dtrain as a subset of L0; RQ5 later expands the normal class only at test time, which requires this asymmetry.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Are Anomaly Scores Telling the Whole Story? A Benchmark for Multilevel Anomaly Detection." pith.science (2026). https://pith.science/paper/3SXFRUD6

@misc{pith2026241114515,
  author       = {Pith},
  title        = {Pith review of: Are Anomaly Scores Telling the Whole Story? A Benchmark for Multilevel Anomaly Detection},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3SXFRUD6}},
  note         = {Machine review of arXiv:2411.14515}
}
read the original abstract

Anomaly detection (AD) is a machine learning task that identifies anomalies by learning patterns from normal training data. In many real-world scenarios, anomalies vary in severity, from minor anomalies with little risk to severe abnormalities requiring immediate attention. However, existing models primarily operate in a binary setting, and the anomaly scores they produce are usually based on the deviation of data points from normal data, which may not accurately reflect practical severity. In this paper, we address this gap by making three key contributions. First, we propose a novel setting, Multilevel AD (MAD), in which the anomaly score represents the severity of anomalies in real-world applications, and we highlight its diverse applications across various domains. Second, we introduce a novel benchmark, MAD-Bench, that evaluates models not only on their ability to detect anomalies, but also on how effectively their anomaly scores reflect severity. This benchmark incorporates multiple types of baselines and real-world applications involving severity. Finally, we conduct a comprehensive performance analysis on MAD-Bench. We evaluate models on their ability to assign severity-aligned scores, investigate the correspondence between their performance on binary and multilevel detection, and study their robustness. This analysis offers key insights into improving AD models for practical severity alignment. The code framework and datasets used for the benchmark will be made publicly available.

Figures

Figures reproduced from arXiv: 2411.14515 by the authors.

Figure 1
Figure 1. (a) Binary Anomaly Detection classifies data as either in-distribution (ID) or out-of-distribution (OOD), without accounting [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. MLLM-based baselines enable few-shot multilevel AD [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Binary AD performance across severity levels. An upward trend is observed in most models (RQ4). [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Performance changes under expansions of the normal class definition. “ [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. CovHuSeg: An Enhanced Approach for Kidney Pathology Segmentation

    eess.IV 2024-11 conditional novelty 3.0 of 10

    Convex hull post-processing raises kidney glomeruli segmentation Dice scores by 0.005 to 0.033 across four models and four data splits.

Reference graph

Works this paper leans on

73 extracted references · 62 canonical work pages · cited by 1 Pith paper

  1. [1]

    Our-normal-skin, 2023

    Ahdasdwdasd. Our-normal-skin, 2023. 4

    work page 2023

  2. [2]

    Skip-ganomaly: Skip connected and adversarially trained encoder-decoder anomaly detection

    Samet Akc ¸ay, Amir Atapour-Abarghouei, and Toby P Breckon. Skip-ganomaly: Skip connected and adversarially trained encoder-decoder anomaly detection. In2019 Interna- tional Joint Conference on Neural Networks (IJCNN), pages 1–8. IEEE, 2019. 4, 5, 6, 12, 13, 14, 15, 16, 17, 18

    work page 2019

  3. [3]

    Anomalib: A deep learning library for anomaly detection

    Samet Akcay, Dick Ameln, Ashwin Vaidya, Barath Laksh- manan, Nilesh Ahuja, and Utku Genc. Anomalib: A deep learning library for anomaly detection. In 2022 IEEE In- ternational Conference on Image Processing (ICIP) , pages 1706–1710. IEEE, 2022. 2

    work page 2022

  4. [4]

    Pni: Indus- trial anomaly detection using position and neighborhood in- formation

    Jaehyeok Bae, Jae-Han Lee, and Seyun Kim. Pni: Indus- trial anomaly detection using position and neighborhood in- formation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 6373–6383, 2023. 2, 4, 5, 6, 12, 13, 14, 15, 16, 17, 18

    work page 2023

  5. [5]

    Mvtec ad–a comprehensive real-world dataset for unsupervised anomaly detection

    Paul Bergmann, MiFchael Fauser, David Sattlegger, and Carsten Steger. Mvtec ad–a comprehensive real-world dataset for unsupervised anomaly detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9592–9600, 2019. 1, 4

    work page 2019

  6. [6]

    Uninformed students: Student-teacher anomaly detection with discriminative latent embeddings

    Paul Bergmann, Michael Fauser, David Sattlegger, and Carsten Steger. Uninformed students: Student-teacher anomaly detection with discriminative latent embeddings. In Proceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition, pages 4183–4192, 2020. 2

    2020

  7. [7]

    Rethinking Autoencoders for Medical Anomaly Detection from A Theoretical Perspective

    Yu Cai, Hao Chen, and Kwang-Ting Cheng. Rethinking au- toencoders for medical anomaly detection from a theoretical perspective. arXiv preprint arXiv:2403.09303, 2024. 4, 5, 6, 12, 13, 14, 15, 16, 17, 18

    work page Pith review arXiv 2024

  8. [8]

    Anomaly de- tection under distribution shift

    Tri Cao, Jiawen Zhu, and Guansong Pang. Anomaly de- tection under distribution shift. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 6511–6523, 2023. 2, 3, 8

    work page 2023

Show all 73 references
  1. [9]

    Deep one-class classification via interpolated gaussian descriptor

    Yuanhong Chen, Yu Tian, Guansong Pang, and Gustavo Carneiro. Deep one-class classification via interpolated gaussian descriptor. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 383–392, 2022. 1, 2, 3, 4, 5, 6, 12, 13, 14, 15, 16, 17, 18

    2022

  2. [10]

    Skin lesion analysis toward melanoma detection 2018: A challenge hosted by the interna- tional skin imaging collaboration (isic)

    Noel Codella, Veronica Rotemberg, Philipp Tschandl, M Emre Celebi, Stephen Dusza, David Gutman, Brian Helba, Aadi Kalloo, Konstantinos Liopyris, Michael Marchetti, et al. Skin lesion analysis toward melanoma detection 2018: A challenge hosted by the interna- tional skin imagin...

    2018 arXiv

  3. [11]

    Dataset for covid-19 segmen- tation and severity scoring

    VV Danilov, Alex Proutski, Alexander Kirpich, DE Lit- manovich, and Yuriy Gankin. Dataset for covid-19 segmen- tation and severity scoring. Mendeley Data, 10, 2022. 4

    2022

  4. [12]

    Anomaly detection via reverse distillation from one-class embedding

    Hanqiu Deng and Xingyu Li. Anomaly detection via reverse distillation from one-class embedding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9737–9746, 2022. 1, 2, 4, 5, 6, 12, 13, 14, 15, 16, 17, 18

    2022

  5. [13]

    Anovl: Adapting vision-language models for unified zero- shot anomaly localization

    Hanqiu Deng, Zhaoxiang Zhang, Jinan Bao, and Xingyu Li. Anovl: Adapting vision-language models for unified zero- shot anomaly localization. arXiv preprint arXiv:2308.15939,

    arXiv

  6. [14]

    Di- abetic retinopathy detection

    Emma Dugas, Jared, Jorge, and Will Cukierski. Di- abetic retinopathy detection. https : / / kaggle . com / competitions / diabetic - retinopathy - detection ,

  7. [15]

    Memorizing normality to detect anomaly: Memory-augmented deep autoencoder for unsupervised anomaly detection

    Dong Gong, Lingqiao Liu, Vuong Le, Budhaditya Saha, Moussa Reda Mansour, Svetha Venkatesh, and Anton van den Hengel. Memorizing normality to detect anomaly: Memory-augmented deep autoencoder for unsupervised anomaly detection. In Proceedings of the IEEE/CVF Inter- national Con...

  8. [16]

    Drocc: Deep robust one- class classification

    Sachin Goyal, Aditi Raghunathan, Moksh Jain, Harsha Vard- han Simhadri, and Prateek Jain. Drocc: Deep robust one- class classification. In International conference on machine learning, pages 3711–3721. PMLR, 2020. 1, 2

    2020

  9. [17]

    Remembering normality: Memory- guided knowledge distillation for unsupervised anomaly de- tection

    Zhihao Gu, Liang Liu, Xu Chen, Ran Yi, Jiangning Zhang, Yabiao Wang, Chengjie Wang, Annan Shu, Guannan Jiang, and Lizhuang Ma. Remembering normality: Memory- guided knowledge distillation for unsupervised anomaly de- tection. In Proceedings of the IEEE/CVF International Con- f...

    2023

  10. [18]

    Cflow-ad: Real-time unsupervised anomaly detection with localization via conditional normalizing flows

    Denis Gudovskiy, Shun Ishizaka, and Kazuki Kozuka. Cflow-ad: Real-time unsupervised anomaly detection with localization via conditional normalizing flows. In Proceed- ings of the IEEE/CVF winter conference on applications of computer vision, pages 98–107, 2022. 2, 4, 5, 6, 12,...

    2022

  11. [19]

    Adbench: Anomaly detection benchmark

    Songqiao Han, Xiyang Hu, Hailiang Huang, Minqi Jiang, and Yue Zhao. Adbench: Anomaly detection benchmark. Advances in Neural Information Processing Systems , 35: 32142–32159, 2022. 2

    2022

  12. [20]

    The meaning and use of the area under a receiver operating characteristic (roc) curve

    James A Hanley and Barbara J McNeil. The meaning and use of the area under a receiver operating characteristic (roc) curve. Radiology, 143(1):29–36, 1982. 3, 1

    1982

  13. [21]

    A diffusion-based framework for multi-class anomaly detection

    Haoyang He, Jiangning Zhang, Hongxu Chen, Xuhai Chen, Zhishan Li, Xu Chen, Yabiao Wang, Chengjie Wang, and Lei Xie. A diffusion-based framework for multi-class anomaly detection. In Proceedings of the AAAI Conference on Artifi- cial Intelligence, pages 8472–8480, 2024. 3

    2024

  14. [22]

    Benchmarking neu- ral network robustness to common corruptions and perturba- tions

    Dan Hendrycks and Thomas Dietterich. Benchmarking neu- ral network robustness to common corruptions and perturba- tions. Proceedings of the International Conference on Learn- ing Representations, 2019. 8

    2019

  15. [23]

    Divide-and-assemble: Learning 9 block-wise memory for unsupervised anomaly detection

    Jinlei Hou, Yingying Zhang, Qiaoyong Zhong, Di Xie, Shil- iang Pu, and Hong Zhou. Divide-and-assemble: Learning 9 block-wise memory for unsupervised anomaly detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 8791–8800, 2021. 2

    2021

  16. [24]

    Mmad: The first-ever comprehensive benchmark for mul- timodal large language models in industrial anomaly detec- tion

    Xi Jiang, Jian Li, Hanqiu Deng, Yong Liu, Bin-Bin Gao, Yifeng Zhou, Jialin Li, Chengjie Wang, and Feng Zheng. Mmad: The first-ever comprehensive benchmark for mul- timodal large language models in industrial anomaly detec- tion. 2024. 2, 5

    2024

  17. [25]

    A new measure of rank correlation

    Maurice G Kendall. A new measure of rank correlation. Biometrika, 30(1-2):81–93, 1938. 3, 1

    1938

  18. [26]

    Novel dataset for fine-grained image categorization

    Aditya Khosla, Nityananda Jayadevaprakash, Bangpeng Yao, and Li Fei-Fei. Novel dataset for fine-grained image categorization. In First Workshop on Fine-Grained Visual Categorization, IEEE Conference on Computer Vision and Pattern Recognition, Colorado Springs, CO, 2011. 4

    2011

  19. [27]

    Pyramid- flow: High-resolution defect contrastive localization using pyramid normalizing flow

    Jiarui Lei, Xiaobo Hu, Yue Wang, and Dong Liu. Pyramid- flow: High-resolution defect contrastive localization using pyramid normalizing flow. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 14143–14152, 2023. 2

    2023

  20. [28]

    Cutpaste: Self-supervised learning for anomaly de- tection and localization

    Chun-Liang Li, Kihyuk Sohn, Jinsung Yoon, and Tomas Pfister. Cutpaste: Self-supervised learning for anomaly de- tection and localization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 9664–9674, 2021. 2

    2021

  21. [29]

    Promptad: Learn- ing prompts with only normal samples for few-shot anomaly detection

    Xiaofan Li, Zhizhong Zhang, Xin Tan, Chengwei Chen, Yanyun Qu, Yuan Xie, and Lizhuang Ma. Promptad: Learn- ing prompts with only normal samples for few-shot anomaly detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 16838– 1...

    2024

  22. [30]

    Omni-frequency channel- selection representations for unsupervised anomaly detec- tion

    Yufei Liang, Jiangning Zhang, Shiwei Zhao, Runze Wu, Yong Liu, and Shuwen Pan. Omni-frequency channel- selection representations for unsupervised anomaly detec- tion. arXiv preprint arXiv:2203.00259, 2022. 2

    2022 arXiv

  23. [31]

    Omni-frequency channel- selection representations for unsupervised anomaly detec- tion

    Yufei Liang, Jiangning Zhang, Shiwei Zhao, Runze Wu, Yong Liu, and Shuwen Pan. Omni-frequency channel- selection representations for unsupervised anomaly detec- tion. IEEE Transactions on Image Processing, 2023. 4, 5, 6, 12, 13, 14, 15, 16, 17, 18

    2023

  24. [32]

    Simplenet: A simple network for image anomaly detection and localization

    Zhikang Liu, Yiming Zhou, Yuansheng Xu, and Zilei Wang. Simplenet: A simple network for image anomaly detection and localization. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition , pages 20402–20411, 2023. 2

    2023

  25. [33]

    Hierarchical vector quantized transformer for multi-class unsupervised anomaly detection

    Ruiying Lu, YuJie Wu, Long Tian, Dongsheng Wang, Bo Chen, Xiyang Liu, and Ruimin Hu. Hierarchical vector quantized transformer for multi-class unsupervised anomaly detection. Advances in Neural Information Processing Sys- tems, 36:8487–8500, 2023. 3

    2023

  26. [34]

    Fake it until you make it: Towards accurate near-distribution novelty detection

    Hossein Mirzaei, Mohammadreza Salehi, Sajjad Shahabi, Efstratios Gavves, Cees GM Snoek, Mohammad Sabokrou, and Mohammad Hossein Rohban. Fake it until you make it: Towards accurate near-distribution novelty detection. In The Eleventh International Conference on Learning Represe...

    2022

  27. [35]

    Automated flower classification over a large number of classes

    Maria-Elena Nilsback and Andrew Zisserman. Automated flower classification over a large number of classes. In In- dian Conference on Computer Vision, Graphics and Image Processing, 2008. 4

    2008

  28. [36]

    Learning memory-guided normality for anomaly detection

    Hyunjong Park, Jongyoun Noh, and Bumsub Ham. Learning memory-guided normality for anomaly detection. In Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 14372–14381, 2020. 1, 2

    2020

  29. [37]

    Omkar M Parkhi, Andrea Vedaldi, Andrew Zisserman, and C. V . Jawahar. Cats and dogs. In 2012 IEEE Conference on Computer Vision and Pattern Recognition , pages 3498– 3505, 2012. 4

    2012

  30. [38]

    G2d: Generate to detect anomaly

    Masoud Pourreza, Bahram Mohammadi, Mostafa Khaki, Samir Bouindour, Hichem Snoussi, and Mohammad Sabokrou. G2d: Generate to detect anomaly. In Proceed- ings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 2003–2012, 2021. 1, 2

    2003

  31. [39]

    Towards to- tal recall in industrial anomaly detection

    Karsten Roth, Latha Pemula, Joaquin Zepeda, Bernhard Sch¨olkopf, Thomas Brox, and Peter Gehler. Towards to- tal recall in industrial anomaly detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 14318–14328, 2022. 2, 4, 5, 6, 12...

    2022

  32. [40]

    Deep one-class classifica- tion

    Lukas Ruff, Robert Vandermeulen, Nico Goernitz, Lucas Deecke, Shoaib Ahmed Siddiqui, Alexander Binder, Em- manuel M¨uller, and Marius Kloft. Deep one-class classifica- tion. In International conference on machine learning, pages 4393–4402. PMLR, 2018. 1, 2

    2018

  33. [41]

    Deep end-to-end one-class classifier

    Mohammad Sabokrou, Mahmood Fathy, Guoying Zhao, and Ehsan Adeli. Deep end-to-end one-class classifier. IEEE transactions on neural networks and learning systems , 32 (2):675–684, 2020. 1, 2

    2020

  34. [42]

    Multiresolution knowledge distillation for anomaly detection

    Mohammadreza Salehi, Niousha Sadjadi, Soroosh Baselizadeh, Mohammad H Rohban, and Hamid R Ra- biee. Multiresolution knowledge distillation for anomaly detection. In Proceedings of the IEEE/CVF confer- ence on computer vision and pattern recognition , pages 14902–14912, 2021. 1, 2

    2021

  35. [43]

    f-anogan: Fast unsupervised anomaly detection with generative adversarial networks

    Thomas Schlegl, Philipp Seeb ¨ock, Sebastian M Waldstein, Georg Langs, and Ursula Schmidt-Erfurth. f-anogan: Fast unsupervised anomaly detection with generative adversarial networks. Medical image analysis, 54:30–44, 2019. 1

    2019

  36. [44]

    Natural synthetic anomalies for self-supervised anomaly detection and localization

    Hannah M Schl ¨uter, Jeremy Tan, Benjamin Hou, and Bern- hard Kainz. Natural synthetic anomalies for self-supervised anomaly detection and localization. In European Conference on Computer Vision, pages 474–489. Springer, 2022. 2

    2022

  37. [45]

    Estimating the sup- port of a high-dimensional distribution

    Bernhard Sch ¨olkopf, John C Platt, John Shawe-Taylor, Alex J Smola, and Robert C Williamson. Estimating the sup- port of a high-dimensional distribution. Neural computation, 13(7):1443–1471, 2001. 2

    2001

  38. [46]

    Anomaly detection using score-based per- turbation resilience

    Woosang Shin, Jonghyeon Lee, Taehan Lee, Sangmoon Lee, and Jong Pil Yun. Anomaly detection using score-based per- turbation resilience. In Proceedings of the IEEE/CVF In- ternational Conference on Computer Vision , pages 23372– 23382, 2023. 2, 4, 5, 6, 12, 13, 14, 15, 16, 17, 18

    2023

  39. [47]

    Anomaly detection in medical imaging with deep perceptual autoencoders

    Nina Shvetsova, Bart Bakker, Irina Fedulova, Heinrich Schulz, and Dmitry V Dylov. Anomaly detection in medical imaging with deep perceptual autoencoders. IEEE Access, 9: 118571–118583, 2021. 1 10

    2021

  40. [48]

    Support vector data description

    David MJ Tax and Robert PW Duin. Support vector data description. Machine learning, 54:45–66, 2004. 2

    2004

  41. [49]

    Revisiting reverse distillation for anomaly detection

    Tran Dinh Tien, Anh Tuan Nguyen, Nguyen Hoang Tran, Ta Duc Huy, Soan Duong, Chanh D Tr Nguyen, and Steven QH Truong. Revisiting reverse distillation for anomaly detection. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition , pages 24511–2452...

    2023

  42. [50]

    The ham10000 dataset, a large collection of multi-source der- matoscopic images of common pigmented skin lesions

    Philipp Tschandl, Cliff Rosendahl, and Harald Kittler. The ham10000 dataset, a large collection of multi-source der- matoscopic images of common pigmented skin lesions. Sci- entific data, 5(1):1–9, 2018. 4

    2018

  43. [51]

    On the c-statistics for eval- uating overall adequacy of risk prediction procedures with censored survival data

    Hajime Uno, Tianxi Cai, Michael J Pencina, Ralph B D’Agostino, and Lee-Jen Wei. On the c-statistics for eval- uating overall adequacy of risk prediction procedures with censored survival data. Statistics in medicine, 30(10):1105– 1117, 2011. 3, 1

    2011

  44. [52]

    Real-iad: A real-world multi-view dataset for benchmarking versatile industrial anomaly detec- tion

    Chengjie Wang, Wenbing Zhu, Bin-Bin Gao, Zhenye Gan, Jiangning Zhang, Zhihao Gu, Shuguang Qian, Mingang Chen, and Lizhuang Ma. Real-iad: A real-world multi-view dataset for benchmarking versatile industrial anomaly detec- tion. In Proceedings of the IEEE/CVF Conference on Com-...

  45. [53]

    Student-teacher feature pyramid matching for anomaly de- tection

    Guodong Wang, Shumin Han, Errui Ding, and Di Huang. Student-teacher feature pyramid matching for anomaly de- tection. arXiv preprint arXiv:2103.04257, 2021. 1, 2

    2021 arXiv

  46. [54]

    Caltech-ucsd birds 200

    Peter Welinder, Steve Branson, Takeshi Mita, Catherine Wah, Florian Schroff, Serge Belongie, and Pietro Perona. Caltech-ucsd birds 200. 2010. 4

    2010

  47. [55]

    A deep one-class neural network for anomalous event detection in complex scenes

    Peng Wu, Jing Liu, and Fang Shen. A deep one-class neural network for anomalous event detection in complex scenes. IEEE transactions on neural networks and learning systems, 31(7):2609–2622, 2019. 1, 2

    2019

  48. [56]

    Im-iad: Indus- trial image anomaly detection benchmark in manufacturing

    Guoyang Xie, Jinbao Wang, Jiaqi Liu, Jiayi Lyu, Yong Liu, Chengjie Wang, Feng Zheng, and Yaochu Jin. Im-iad: Indus- trial image anomaly detection benchmark in manufacturing. IEEE Transactions on Cybernetics, 2024. 2

    2024

  49. [57]

    Learning semantic context from nor- mal samples for unsupervised anomaly detection

    Xudong Yan, Huaidong Zhang, Xuemiao Xu, Xiaowei Hu, and Pheng-Ann Heng. Learning semantic context from nor- mal samples for unsupervised anomaly detection. In Pro- ceedings of the AAAI Conference on Artificial Intelligence , pages 3110–3118, 2021. 1, 2

    2021

  50. [58]

    Patch svdd: Patch-level svdd for anomaly detection and segmentation

    Jihun Yi and Sungroh Yoon. Patch svdd: Patch-level svdd for anomaly detection and segmentation. In Proceedings of the Asian Conference on Computer Vision, 2020. 1, 2

    2020

  51. [59]

    A unified model for multi-class anomaly detection

    Zhiyuan You, Lei Cui, Yujun Shen, Kai Yang, Xin Lu, Yu Zheng, and Xinyi Le. A unified model for multi-class anomaly detection. Advances in Neural Information Pro- cessing Systems, 35:4571–4584, 2022. 3

    2022

  52. [60]

    Old is gold: Redefining the adversarially learned one-class classifier training paradigm

    Muhammad Zaigham Zaheer, Jin-ha Lee, Marcella Astrid, and Seung-Ik Lee. Old is gold: Redefining the adversarially learned one-class classifier training paradigm. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14183–14193, 2020. 1, 2

    2020

  53. [61]

    Spearman rank correlation

    Jerrold H Zar. Spearman rank correlation. Encyclopedia of biostatistics, 7, 2005. 6

    2005

  54. [62]

    Draem- a discriminatively trained reconstruction embedding for sur- face anomaly detection

    Vitjan Zavrtanik, Matej Kristan, and Danijel Skoˇcaj. Draem- a discriminatively trained reconstruction embedding for sur- face anomaly detection. In Proceedings of the IEEE/CVF international conference on computer vision , pages 8330– 8339, 2021. 2

    2021

  55. [63]

    Recon- struction by inpainting for visual anomaly detection

    Vitjan Zavrtanik, Matej Kristan, and Danijel Sko ˇcaj. Recon- struction by inpainting for visual anomaly detection. Pattern Recognition, 112:107706, 2021. 1

    2021

  56. [64]

    Dsr– a dual subspace re-projection network for surface anomaly detection

    Vitjan Zavrtanik, Matej Kristan, and Danijel Sko ˇcaj. Dsr– a dual subspace re-projection network for surface anomaly detection. In European conference on computer vision, pages 539–554. Springer, 2022. 2

    2022

  57. [65]

    Ader: A comprehensive benchmark for multi-class visual anomaly detection

    Jiangning Zhang, Haoyang He, Zhenye Gan, Qingdong He, Yuxuan Cai, Zhucun Xue, Yabiao Wang, Chengjie Wang, Lei Xie, and Yong Liu. Ader: A comprehensive benchmark for multi-class visual anomaly detection. arXiv preprint arXiv:2406.03262, 2024. 2, 3

    2024 arXiv

  58. [66]

    Destseg: Segmentation guided denoising student-teacher for anomaly detection

    Xuan Zhang, Shiyu Li, Xi Li, Ping Huang, Jiulong Shan, and Ting Chen. Destseg: Segmentation guided denoising student-teacher for anomaly detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3914–3923, 2023. 2

    2023

  59. [67]

    Realnet: A feature selection network with realistic synthetic anomaly for anomaly detection

    Ximiao Zhang, Min Xu, and Xiuzhuang Zhou. Realnet: A feature selection network with realistic synthetic anomaly for anomaly detection. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 16699–16708, 2024. 2

    2024

  60. [68]

    Toward generalist anomaly detection via in-context residual learning with few-shot sam- ple prompts

    Jiawen Zhu and Guansong Pang. Toward generalist anomaly detection via in-context residual learning with few-shot sam- ple prompts. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17826– 17836, 2024. 2, 5

    2024

  61. [69]

    Spot-the-difference self-supervised pre- training for anomaly detection and segmentation

    Yang Zou, Jongheon Jeong, Latha Pemula, Dongqing Zhang, and Onkar Dabeer. Spot-the-difference self-supervised pre- training for anomaly detection and segmentation. In Eu- ropean Conference on Computer Vision , pages 392–408. Springer, 2022. 1, 4 11 A. Evaluation Metrics A.1. A...

    2022

  62. [70]

    The severity level of sample xi is greater than that of sample xj, and the anomaly score of xi is also greater than that of xj or if

  63. [71]

    The pair is discordant if it is in the reverse ordering for severity levels and anomaly score, or the values are arranged in opposite directions

    The severity level of sample xi is less than that of sample xj, and the anomaly score of xi is also less than that of xj. The pair is discordant if it is in the reverse ordering for severity levels and anomaly score, or the values are arranged in opposite directions. That is, if:

  64. [72]

    The severity level of sample xi is greater than that of sample xj, but the anomaly score of xi is less than that of xj or if

  65. [73]

    The two observations are tied if the severity level of sample xi is equal that of xj and/or the anomaly score of sample xi is equal that of xj

    The severity level of sample xi is less than that of sample xj, but the anomaly score of xi is greater than that of xj. The two observations are tied if the severity level of sample xi is equal that of xj and/or the anomaly score of sample xi is equal that of xj. Kendall’s Tau...

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.