REVIEW 56 references
Treating an image and its reconstruction as two modalities and matching them locally finds subtle industrial defects more precisely than pixel differencing.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · grok-4.5
2026-07-30 18:54 UTC pith:UTNHNCZU
load-bearing objection Competent multi-class UAD systems paper: bidirectional windowed matching on input/reconstruction plus wavelet fusion works on the tables, but headline numbers disagree with Table I and everything rides on the frozen reconstructor.
XMatchAD: A Cross-Modal Matching Perspective on Reconstruction-based Anomaly Detection
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
Reconstruction-based unsupervised anomaly detection can be reformulated as pseudo cross-modal matching: once a diffusion or lightweight reconstructor supplies an anomaly-free counterpart, local bidirectional windowed cross-attention between DINOv2 features of the two images, followed by adaptive high-frequency fusion, produces anomaly maps that are more sensitive to subtle defects and have sharper boundaries than cosine-similarity or global differencing baselines, yielding superior multi-class detection and localization on MVTec-AD, VisA and MPDD.
What carries the argument
Attention-guided cross-modal matching (window-based multi-head self- and cross-attention, W-MSA/W-MCA) plus adaptive frequency-aware fusion: the former locally aligns and mutually refines the two modality streams; the latter couples high-frequency wavelet sub-bands across scales to sharpen anomaly boundaries.
Load-bearing premise
The frozen reconstructor must produce sufficiently clean, anomaly-free images so that residual mismatches after feature encoding are true defects rather than reconstruction artifacts.
What would settle it
Replace the reconstructor with one that systematically leaves residual defects or invents structure on known anomalous images; if image- and pixel-AUROC then collapse toward the plain cosine-similarity baseline, the central matching claim fails.
If this is right
- Factory inspection systems that already run a reconstructor can gain sharper localization by swapping the final differencing step for local cross-modal matching.
- High-frequency wavelet guidance becomes a reusable recipe for boundary-sensitive anomaly heads on other reconstruction pipelines.
- Multi-class unified models can reduce false negatives on small or low-contrast defects without class-specific training.
- Inference cost is dominated by reconstruction speed; faster reconstructors immediately shorten the end-to-end latency of the full pipeline.
Where Pith is reading between the lines
- The same local-matching-plus-frequency-fusion block could be attached to embedding-only teachers that never reconstruct, testing whether the reconstructor is essential or merely convenient.
- If window size is made adaptive to estimated defect scale, the method might close remaining gaps on extremely small or elongated anomalies.
- Bidirectional matching scores themselves could serve as an unsupervised signal for active sampling of hard normal variants during training.
Editorial analysis
A structured set of objections, weighed in public.
Circularity Check
No significant circularity: standard reconstruction-based UAD architecture evaluated on external benchmarks; metrics are not forced by construction.
full rationale
XMatchAD reframes reconstruction-based unsupervised anomaly detection as pseudo cross-modal matching between an input and a frozen reconstructor’s output (GLAD or Dinomaly), then trains windowed self/cross-attention and frequency-aware fusion with BCE against synthetic anomaly masks and reports I-/P-AUROC/AP/F1 on held-out MVTec-AD, VisA, and MPDD splits against external SOTA. The headline numbers are empirical test-set outcomes, not algebraic restatements of fitted constants. Loss head weights (α, β, γ) and architectural choices (W-MSA/W-MCA, high-frequency DWT coupling) are design hyperparameters ablated on the same external splits; they do not define the reported AUROC by construction. Self-citations (e.g., CostFilter-AD) appear only as related work and are not load-bearing uniqueness or uniqueness-theorem premises. The reconstructor quality assumption is a modeling premise, not a circular derivation. Derivation chain is self-contained engineering plus external evaluation; score 0.
Axiom & Free-Parameter Ledger
free parameters (5)
- loss weights α, β, γ =
0.8, 0.1, 0.1
- top-K pixels K for image score =
250
- architecture scales L, C′, H′, W′ =
L=3, C′=48, H′=W′=28
- learning rate =
4e-3
- GLAD denoising steps =
25
axioms (5)
- domain assumption Models trained only on normal data, with synthetic anomaly supervision for the segmentation heads, generalize to real industrial defects under multi-class evaluation.
- domain assumption A pretrained DINOv2 encoder provides sufficiently discriminative frozen features for both input and reconstruction branches.
- domain assumption Anomalies are spatially localized and continuous, so windowed attention is an appropriate inductive bias versus global attention.
- ad hoc to paper High-frequency wavelet subbands carry the boundary/texture cues needed for sharp anomaly delineation; low-frequency content is less useful or harmful.
- standard math Softmax attention and residual MLP updates are valid differentiable matching operators replacing cosine similarity (Eq. 1 → Eqs. 4–6).
invented entities (2)
-
Pseudo cross-modal matching formulation (input vs reconstruction as modalities)
no independent evidence
-
Frequency-aware Cross-modal Fusion (FCF) with product-coupled high-frequency gates
no independent evidence
read the original abstract
The remarkable success of reconstruction-based methods in Unsupervised Anomaly Detection (UAD) lies in their ability to identify and localize anomalies by modeling discrepancies between input images and their reconstructed counterparts. However, these approaches often struggle to capture subtle anomalies and tend to produce blurred anomaly boundaries, which significantly limits their effectiveness, particularly in complex multi-class scenarios. To address these issues, we present XMatchAD, a novel UAD framework that reinterprets the task from a pseudo cross-modal matching perspective. Specifically, the input and reconstructed images are treated as two complementary modalities and their matching relationships are precisely exploited for anomaly detection. First, a pre-trained feature extractor is employed to encode discriminative representations. Second, an attention-guided cross-modal matching mechanism is introduced to match local inter-modal anomaly-related patterns while mutually refining the features. This enhances the sensitivity to anomalies with diverse shapes and subtle deviations and significantly improves the precision of anomaly detection and localization. Third, we design an adaptive frequency-aware fusion module that further delineates sharp anomaly boundaries through the coupling of high-frequency components from cross-modal multi-scale representations. Comprehensive evaluations on MVTec-AD, VisA, and MPDD benchmarks demonstrate that our method consistently achieves superior performance, outperforming state-of-the-art methods in multi-class anomaly detection and localization. The code will be released at https://github.com/Mingxiu-Cai/XMatchAD.
Figures
Reference graph
Works this paper leans on
-
[1]
Aadc-net: A multimodal deep learning framework for automatic anomaly detection in real-time surveillance,
D. Tri Phan, V . Hoang Minh Doan, J. Choi, B. Lee, and J. Oh, “Aadc-net: A multimodal deep learning framework for automatic anomaly detection in real-time surveillance,”IEEE Transactions on Instrumentation and Measurement, vol. 74, pp. 1–13, 2025
2025
-
[3]
Anomaly detection for medical images using heterogeneous auto-encoder,
S. Lu, W. Zhang, H. Zhao, H. Liu, N. Wang, and H. Li, “Anomaly detection for medical images using heterogeneous auto-encoder,”IEEE Transactions on Image Processing, vol. 33, pp. 2770–2782, 2024
2024
-
[4]
Enhancing unsupervised anomaly detection with score-guided network,
Z. Huang, B. Zhang, G. Hu, L. Li, Y . Xu, and Y . Jin, “Enhancing unsupervised anomaly detection with score-guided network,”IEEE Transactions on Neural Networks and Learning Systems, vol. 35, no. 10, pp. 14 754–14 769, 2023
2023
-
[5]
Exploiting structural consistency of chest anatomy for unsupervised anomaly detection in radiography images,
T. Xiang, Y . Zhang, Y . Lu, A. Yuille, C. Zhang, W. Cai, and Z. Zhou, “Exploiting structural consistency of chest anatomy for unsupervised anomaly detection in radiography images,”IEEE Transactions on Pat- tern Analysis and Machine Intelligence, vol. 46, no. 9, pp. 6070–6081, 2024
2024
-
[6]
Unistad: An unified triple-tower student–teacher model for multi-class anomaly detection and localization,
H. Liu and J. Sun, “Unistad: An unified triple-tower student–teacher model for multi-class anomaly detection and localization,”IEEE Trans- actions on Circuits and Systems for Video Technology, vol. 35, no. 4, pp. 3196–3208, 2025
2025
-
[7]
Boosting global- local feature matching via anomaly synthesis for multi-class point cloud anomaly detection,
Y . Cheng, Y . Cao, D. Wang, W. Shen, and W. Li, “Boosting global- local feature matching via anomaly synthesis for multi-class point cloud anomaly detection,”IEEE Transactions on Automation Science and Engineering, vol. 22, pp. 12 560–12 571, 2025
2025
-
[8]
Costfilter- ad: Enhancing anomaly detection through matching cost filtering,
Z. Zhang, M. Cai, H. Wang, G. Wu, T. Chai, and X. Zhu, “Costfilter- ad: Enhancing anomaly detection through matching cost filtering,”arXiv preprint arXiv:2505.01476, 2025
arXiv 2025
-
[9]
Revisiting reverse distillation for anomaly detection,
T. D. Tien, A. T. Nguyen, N. H. Tran, T. D. Huy, S. Duong, C. D. T. Nguyen, and S. Q. Truong, “Revisiting reverse distillation for anomaly detection,” inCVPR, 2023, pp. 24 511–24 520
2023
-
[10]
A unified anomaly synthesis strategy with gradient ascent for industrial anomaly detection and localization,
Q. Chen, H. Luo, C. Lv, and Z. Zhang, “A unified anomaly synthesis strategy with gradient ascent for industrial anomaly detection and localization,” inECCV, 2024, pp. 37–54
2024
-
[11]
Unsupervised anomaly segmentation via deep feature reconstruction,
Y . Shi, J. Yang, and Z. Qi, “Unsupervised anomaly segmentation via deep feature reconstruction,”Neurocomputing, vol. 424, pp. 9–22, 2021
2021
-
[12]
Towards visually explaining variational autoencoders,
W. Liu, R. Li, M. Zheng, S. Karanam, Z. Wu, B. Bhanu, R. J. Radke, and O. Camps, “Towards visually explaining variational autoencoders,” inCVPR, 2020, pp. 8642–8651
2020
-
[13]
Divide- and-assemble: Learning block-wise memory for unsupervised anomaly detection,
J. Hou, Y . Zhang, Q. Zhong, D. Xie, S. Pu, and H. Zhou, “Divide- and-assemble: Learning block-wise memory for unsupervised anomaly detection,” inICCV, 2021, pp. 8791–8800
2021
-
[14]
Memorizing normality to detect anomaly: Memory- augmented deep autoencoder for unsupervised anomaly detection,
D. Gong, L. Liu, V . Le, B. Saha, M. R. Mansour, S. Venkatesh, and A. v. d. Hengel, “Memorizing normality to detect anomaly: Memory- augmented deep autoencoder for unsupervised anomaly detection,” in ICCV, 2019, pp. 1705–1714
2019
-
[15]
Omni-frequency channel-selection representations for unsupervised anomaly detection,
Y . Liang, J. Zhang, S. Zhao, R. Wu, Y . Liu, and S. Pan, “Omni-frequency channel-selection representations for unsupervised anomaly detection,” IEEE Transactions on Image Processing, vol. 32, pp. 4327–4340, 2023
2023
-
[16]
Reconstruction by inpainting for visual anomaly detection,
V . Zavrtanik, M. Kristan, and D. Sko ˇcaj, “Reconstruction by inpainting for visual anomaly detection,”Pattern Recognition, vol. 112, p. 107706, 2021
2021
-
[17]
Unsupervised surface anomaly detection with diffusion probabilistic model,
X. Zhang, N. Li, J. Li, T. Dai, Y . Jiang, and S.-T. Xia, “Unsupervised surface anomaly detection with diffusion probabilistic model,” inICCV, 2023, pp. 6782–6791
2023
-
[18]
Glad: Towards better reconstruction with global and local adaptive diffusion models for unsupervised anomaly detection,
H. Yao, M. Liu, Z. Yin, Z. Yan, X. Hong, and W. Zuo, “Glad: Towards better reconstruction with global and local adaptive diffusion models for unsupervised anomaly detection,” inECCV, 2024, pp. 1–17
2024
-
[19]
A diffusion-based framework for multi-class anomaly detection,
H. He, J. Zhang, H. Chen, X. Chen, Z. Li, X. Chen, Y . Wang, C. Wang, and L. Xie, “A diffusion-based framework for multi-class anomaly detection,” inAAAI, vol. 38, no. 8, 2024, pp. 8472–8480
2024
-
[20]
Adtr: Anomaly detection transformer with feature reconstruction,
Z. You, K. Yang, W. Luo, L. Cui, Y . Zheng, and X. Le, “Adtr: Anomaly detection transformer with feature reconstruction,” inICONIP, 2022, pp. 298–310
2022
-
[21]
Draem-a discriminatively trained reconstruction embedding for surface anomaly detection,
V . Zavrtanik, M. Kristan, and D. Sko ˇcaj, “Draem-a discriminatively trained reconstruction embedding for surface anomaly detection,” in ICCV, 2021, pp. 8330–8339
2021
-
[22]
Diffusionad: Norm-guided one-step denoising diffusion for anomaly detection,
H. Zhang, Z. Wang, D. Zeng, Z. Wu, and Y .-G. Jiang, “Diffusionad: Norm-guided one-step denoising diffusion for anomaly detection,”IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 47, no. 8, pp. 7140–7152, 2025
2025
-
[23]
Dsr–a dual subspace re- projection network for surface anomaly detection,
V . Zavrtanik, M. Kristan, and D. Sko ˇcaj, “Dsr–a dual subspace re- projection network for surface anomaly detection,” inECCV, 2022, pp. 539–554
2022
-
[24]
Unsu- pervised anomaly detection via masked diffusion posterior sampling,
D. Wu, S. Fan, X. Zhou, L. Yu, Y . Deng, J. Zou, and B. Lin, “Unsu- pervised anomaly detection via masked diffusion posterior sampling,” arXiv preprint arXiv:2404.17900, 2024
Pith/arXiv arXiv 2024
-
[25]
Anomaly detection with conditioned denoising diffusion models,
A. Mousakhan, T. Brox, and J. Tayyub, “Anomaly detection with conditioned denoising diffusion models,” inDAGM German Conference on Pattern Recognition, 2024, pp. 181–195
2024
-
[26]
A unified model for multi-class anomaly detection,
Z. You, L. Cui, Y . Shen, K. Yang, X. Lu, Y . Zheng, and X. Le, “A unified model for multi-class anomaly detection,”NeurIPS, vol. 35, pp. 4571–4584, 2022
2022
-
[27]
Destseg: Segmentation guided denoising student-teacher for anomaly detection,
X. Zhang, S. Li, X. Li, P. Huang, J. Shan, and T. Chen, “Destseg: Segmentation guided denoising student-teacher for anomaly detection,” inCVPR, 2023, pp. 3914–3923
2023
-
[28]
Just noticeable learning for unsupervised anomaly localization and detection,
Y . Zhao, “Just noticeable learning for unsupervised anomaly localization and detection,” inICME, 2022, pp. 01–06
2022
-
[30]
Learning semantic context from normal samples for unsupervised anomaly detection,
X. Yan, H. Zhang, X. Xu, X. Hu, and P.-A. Heng, “Learning semantic context from normal samples for unsupervised anomaly detection,” in AAAI, vol. 35, no. 4, 2021, pp. 3110–3118
2021
-
[31]
A survey on multimodal large language models for autonomous driving,
C. Cui, Y . Ma, X. Cao, W. Ye, Y . Zhou, K. Liang, J. Chen, J. Lu, Z. Yang, K.-D. Liaoet al., “A survey on multimodal large language models for autonomous driving,” inWACV, 2024, pp. 958–979
2024
-
[32]
Spatio-contextual deep network-based multimodal pedestrian detection for autonomous driving,
K. Dasgupta, A. Das, S. Das, U. Bhattacharya, and S. Yogamani, “Spatio-contextual deep network-based multimodal pedestrian detection for autonomous driving,”IEEE transactions on intelligent transportation systems, vol. 23, no. 9, pp. 15 940–15 950, 2022
2022
-
[33]
Unified domain adaptive semantic segmentation,
Z. Zhang, G. Wu, J. Zhang, X. Zhu, D. Tao, and T. Chai, “Unified domain adaptive semantic segmentation,”IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 47, no. 8, pp. 6731–6748, 2025
2025
-
[34]
Multi-granularity con- trastive cross-modal collaborative generation for end-to-end long-term video question answering,
T. Yu, K. Fu, J. Zhang, Q. Huang, and J. Yu, “Multi-granularity con- trastive cross-modal collaborative generation for end-to-end long-term video question answering,”IEEE Transactions on Image Processing, vol. 33, pp. 3115–3129, 2024
2024
-
[35]
History aware multimodal transformer for vision-and-language navigation,
S. Chen, P.-L. Guhur, C. Schmid, and I. Laptev, “History aware multimodal transformer for vision-and-language navigation,”NeurIPS, vol. 34, pp. 5834–5847, 2021
2021
-
[36]
Cross-modal map learning for vision and language navigation,
G. Georgakis, K. Schmeckpeper, K. Wanchoo, S. Dan, E. Miltsakaki, D. Roth, and K. Daniilidis, “Cross-modal map learning for vision and language navigation,” inCVPR, 2022, pp. 15 460–15 470. JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021 13
2022
-
[37]
An overview of deep learning methods for multimodal medical data mining,
F. Behrad and M. S. Abadeh, “An overview of deep learning methods for multimodal medical data mining,”Expert Systems with Applications, vol. 200, p. 117006, 2022
2022
-
[38]
Plug-and-play regula- tors for image-text matching,
H. Diao, Y . Zhang, W. Liu, X. Ruan, and H. Lu, “Plug-and-play regula- tors for image-text matching,”IEEE Transactions on Image Processing, vol. 32, pp. 2322–2334, 2023
2023
-
[39]
Rfnet: Region-aware fusion network for incomplete multi-modal brain tumor segmentation,
Y . Ding, X. Yu, and Y . Yang, “Rfnet: Region-aware fusion network for incomplete multi-modal brain tumor segmentation,” inICCV, 2021, pp. 3975–3984
2021
-
[40]
Equivariant multi-modality image fusion,
Z. Zhao, H. Bai, J. Zhang, Y . Zhang, K. Zhang, S. Xu, D. Chen, R. Timofte, and L. Van Gool, “Equivariant multi-modality image fusion,” inCVPR, 2024, pp. 25 912–25 921
2024
-
[41]
U-net: Convolutional networks for biomedical image segmentation,
O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation,” inMICCAI, 2015, pp. 234–241
2015
-
[42]
Restormer: Efficient transformer for high-resolution image restoration,
S. W. Zamir, A. Arora, S. Khan, M. Hayat, F. S. Khan, and M.-H. Yang, “Restormer: Efficient transformer for high-resolution image restoration,” inCVPR, 2022, pp. 5728–5739
2022
-
[43]
Gafusion: Adaptive fusing lidar and camera with multiple guidance for 3d object detection,
X. Li, B. Fan, J. Tian, and H. Fan, “Gafusion: Adaptive fusing lidar and camera with multiple guidance for 3d object detection,” inCVPR, 2024, pp. 21 209–21 218
2024
-
[44]
Event-assisted low-light video object segmentation,
H. Li, J. Wang, J. Yuan, Y . Li, W. Weng, Y . Peng, Y . Zhang, Z. Xiong, and X. Sun, “Event-assisted low-light video object segmentation,” in CVPR, 2024, pp. 3250–3259
2024
-
[45]
Cross-modal implicit relation reasoning and aligning for text-to-image person retrieval,
D. Jiang and M. Ye, “Cross-modal implicit relation reasoning and aligning for text-to-image person retrieval,” inCVPR, 2023, pp. 2787– 2797
2023
-
[46]
See more and know more: Zero-shot point cloud segmentation via multi-modal visual data,
Y . Lu, Q. Jiang, R. Chen, Y . Hou, X. Zhu, and Y . Ma, “See more and know more: Zero-shot point cloud segmentation via multi-modal visual data,” inICCV, 2023, pp. 21 674–21 684
2023
-
[47]
Align before fuse: Vision and language representation learning with momentum distillation,
J. Li, R. Selvaraju, A. Gotmare, S. Joty, C. Xiong, and S. C. H. Hoi, “Align before fuse: Vision and language representation learning with momentum distillation,”NeurIPS, vol. 34, pp. 9694–9705, 2021
2021
-
[48]
Emerging properties in self-supervised vision transformers,
M. Caron, H. Touvron, I. Misra, H. J ´egou, J. Mairal, P. Bojanowski, and A. Joulin, “Emerging properties in self-supervised vision transformers,” inICCV, 2021, pp. 9650–9660
2021
-
[49]
Swin transformer: Hierarchical vision transformer using shifted windows,
Z. Liu, Y . Lin, Y . Cao, H. Hu, Y . Wei, Z. Zhang, S. Lin, and B. Guo, “Swin transformer: Hierarchical vision transformer using shifted windows,” inICCV, 2021, pp. 10 012–10 022
2021
-
[50]
Attention is all you need,
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,”NeurIPS, vol. 30, 2017
2017
-
[51]
Wavelet based image fusion techniques—an introduction, review and comparison,
K. Amolins, Y . Zhang, and P. Dare, “Wavelet based image fusion techniques—an introduction, review and comparison,”ISPRS Journal of photogrammetry and Remote Sensing, vol. 62, no. 4, pp. 249–263, 2007
2007
-
[52]
Xnet: Wavelet- based low and high frequency fusion networks for fully-and semi- supervised semantic segmentation of biomedical images,
Y . Zhou, J. Huang, C. Wang, L. Song, and G. Yang, “Xnet: Wavelet- based low and high frequency fusion networks for fully-and semi- supervised semantic segmentation of biomedical images,” inICCV, 2023, pp. 21 085–21 096
2023
-
[53]
Simplenet: A simple network for image anomaly detection and localization,
Z. Liu, Y . Zhou, Y . Xu, and Z. Wang, “Simplenet: A simple network for image anomaly detection and localization,” inCVPR, 2023, pp. 20 402– 20 411
2023
-
[54]
Exploring plain vit features for multi-class unsupervised visual anomaly detection,
J. Zhang, X. Chen, Y . Wang, C. Wang, Y . Liu, X. Li, M.-H. Yang, and D. Tao, “Exploring plain vit features for multi-class unsupervised visual anomaly detection,”Computer Vision and Image Understanding, vol. 253, p. 104308, 2025
2025
-
[55]
Anomalydino: Boosting patch-based few-shot anomaly detection with dinov2,
S. Damm, M. Laszkiewicz, J. Lederer, and A. Fischer, “Anomalydino: Boosting patch-based few-shot anomaly detection with dinov2,” in WACV. IEEE, 2025, pp. 1319–1329
2025
-
[56]
Dinomaly: The less is more philosophy in multi-class unsupervised anomaly detec- tion,
J. Guo, S. Lu, W. Zhang, F. Chen, H. Li, and H. Liao, “Dinomaly: The less is more philosophy in multi-class unsupervised anomaly detec- tion,” inProceedings of the Computer Vision and Pattern Recognition Conference, 2025, pp. 20 405–20 415
2025
-
[57]
On estimation of a probability density function and mode,
E. Parzen, “On estimation of a probability density function and mode,” The annals of mathematical statistics, vol. 33, no. 3, pp. 1065–1076, 1962
1962
-
[58]
Grad-cam: Visual explanations from deep networks via gradient-based localization,
R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra, “Grad-cam: Visual explanations from deep networks via gradient-based localization,” inICCV, 2017, pp. 618–626. Mingxiu Caireceived the MS degree from the School of Computer Science, Nanjing Audit Uni- versity, China, in 2024. She is currently working toward a Ph.D. degree in the S...
2017
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.