Pith. sign in

REVIEW 8 cited by

MMAD: A Comprehensive Benchmark for Multimodal Large Language Models in Industrial Anomaly Detection

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2410.09453 v3 pith:CFCO5KAG submitted 2024-10-12 cs.AI cs.CV

Xi Jiang , Jian Li , Hanqiu Deng , Yong Liu , Bin-Bin Gao , Yifeng Zhou , Jialin Li , Chengjie Wang
show 1 more author
Feng Zheng
This is my paper
classification cs.AIcs.CV
keywords industrialmllmsmodelsmmadanomalydetectionlanguagebenchmark
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

In the field of industrial inspection, Multimodal Large Language Models (MLLMs) have a high potential to renew the paradigms in practical applications due to their robust language capabilities and generalization abilities. However, despite their impressive problem-solving skills in many domains, MLLMs' ability in industrial anomaly detection has not been systematically studied. To bridge this gap, we present MMAD, the first-ever full-spectrum MLLMs benchmark in industrial Anomaly Detection. We defined seven key subtasks of MLLMs in industrial inspection and designed a novel pipeline to generate the MMAD dataset with 39,672 questions for 8,366 industrial images. With MMAD, we have conducted a comprehensive, quantitative evaluation of various state-of-the-art MLLMs. The commercial models performed the best, with the average accuracy of GPT-4o models reaching 74.9%. However, this result falls far short of industrial requirements. Our analysis reveals that current MLLMs still have significant room for improvement in answering questions related to industrial anomalies and defects. We further explore two training-free performance enhancement strategies to help models improve in industrial scenarios, highlighting their promising potential for future research.

Discussion (0). Sign in to comment.

Forward citations

Cited by 8 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. O-VAD: Industrial Video Anomaly Detection through Object-Centric Tracking and Reasoning

    cs.CV 2026-07 conditional novelty 6.0 of 10

    An object-centric, training-free agentic pipeline that tracks object state changes and reasons over them with a vision-language model achieves strong video-level AUROC on Phys-AD, LiquidAD, and IPAD, while producing i...

  2. Global Logic and Local Search: Dual-Stream Multimodal In-Context Learning for Verifiable Industrial Anomaly Detection

    cs.CV 2026-07 conditional novelty 6.0 of 10

    A training-free dual-stream multimodal framework (PVLA + SAM 3 global logic + MCTS local search) improves verifiable industrial anomaly QA without defective training samples.

  3. AD-FM: Multimodal LLMs for Anomaly Detection via Multi-Stage Reasoning and Fine-Grained Reward Optimization

    cs.CV 2025-08 conditional novelty 6.0 of 10

    AD-FM combines multi-stage reasoning with localization-aware rewards to fine-tune MLLMs for anomaly detection, improving average accuracy by about 22 percentage points over the base model.

  4. EMIT: Enhancing MLLMs for Industrial Anomaly Detection via Difficulty-Aware GRPO

    cs.CV 2025-07 conditional novelty 6.0 of 10

    A difficulty-aware GRPO training scheme with response resampling, advantage reweighting, GPT-generated text samples, and heatmap-guided contrastive embeddings improves InternVL3-8B by 7.77 percentage points on the MMA...

  5. SAGE: A Visual Language Model for Anomaly Detection via Fact Enhancement and Entropy-aware Alignment

    cs.CL 2025-07 conditional novelty 6.0 of 10

    SAGE improves zero-shot and one-shot industrial anomaly reasoning by fusing VLM-generated facts with reference-image comparison and entropy-aware preference optimization.

  6. PB-IAD: Utilizing multimodal foundation models for semantic industrial anomaly detection in dynamic manufacturing environments

    cs.CV 2025-08 conditional novelty 5.0 of 10

    With carefully layered prompts and one or three reference samples, GPT-4.1 detects anomalies in cable images and crimp-force features at F1 levels that PatchCore and Isolation Forest reach only after training on dozen...

  7. Foundation Models and Transformers for Anomaly Detection: A Survey

    cs.LG 2025-07 reject novelty 4.0 of 10

    A taxonomy and literature review of Transformer-based visual anomaly detection, compromised by fabricated citations with dummy arXiv IDs.

  8. A Comprehensive Survey for Real-World Industrial Defect Detection: Challenges, Approaches, and Prospects

    cs.CV 2025-07 conditional novelty 3.0 of 10

    A broad survey of industrial defect detection that structures the field by closed-set vs open-set and 2D vs 3D methods, with an emphasis on the rise of open-set anomaly detection.

Pith tools