REVIEW 4 major objections 6 minor 15 references
Filter or Compensate: Towards Invariant Representation from Distribution Shift for Anomaly Detection
T0 review · 4 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read The paper claims that teacher–student misalignment on distribution-specific information is the main cause of anomaly-detection failure under distribution shift, and that compensating for this information while filtering abnormal content…
desk verdict Incremental but real extension of GNL: consistent OOD gains, overclaimed ID results, and a missing test-time ablation. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The machinery is the DiSCo–DiIFi pair. DiSCo is a stack of Dynamic Convolution, InstanceNorm, and LeakyReLU blocks inserted after each student block, with a residual shortcut; it is trained by $L_{\mathrm{Co}}$ to reconstruct distribution-specific information that aligns the student with the teacher. DiIFi uses the final DiSCo output $C_1(f^{D_1})$ as the estimate of distribution-specific content, transforms it to earlier spatial resolutions with convolutional blocks, and supervises the earlier DiSCo outputs with an MSE loss ($L_{\mathrm{mse}}$). This forces early blocks to filter abnormal and distribution-specific information, so the anomaly score is driven by genuine anomalies rather than by the distribution shift.
What would settle it
Train a linear classifier on the feature maps $C_1(f^{D_1})$ produced by FiCo to predict either the object class or the corruption type; if class prediction accuracy is far above chance, $C_1$ encodes semantic content, contradicting the assumption that it holds only distribution-specific information, and the DiIFi supervision via Eqs. (7)–(8) is mis-specified.
Extended reading notes
Core claim
The paper's central discovery is that teacher–student discrepancy in RD-based anomaly detection is dominated by distribution-specific information rather than true anomalies, so existing methods mistake style changes for defects. By adding a Distribution-Specific Compensation module after each student block, trained to align compensated features with the teacher, and a Distribution-Invariant Filter module that propagates the final block's distribution-specific estimate back to earlier blocks, FiCo restores alignment on OOD data while keeping genuine anomalies as the main source of discrepancy. This decomposition yields consistent improvements on corrupted test sets and, because the compensation also helps on clean data, better in-distribution accuracy than prior RD-based methods.
Load-bearing premise
The paper assumes that each student feature map splits cleanly into a distribution-invariant part and a distribution-specific part, and that the final DiSCo output $C_1(f^{D_1})$ captures only the latter, so it can be used to supervise earlier blocks; if $C_1$ also encodes semantic or anomaly-relevant content, the compensation–filter mechanism is mis-specified and the gains may not transfer.
Editorial extensions
If this is right
- On MVTec, PACS, and CIFAR-10 with brightness, contrast, defocus blur, and gaussian-noise corruptions, FiCo reports higher sample-level AUROC than all compared methods.
- FiCo improves in-distribution performance relative to RD-based methods, indicating the compensation helps even without distribution shift.
- Anomaly maps and score distributions show that FiCo concentrates activation on true anomalous regions while suppressing distribution-specific regions.
- The added modules are trainable end-to-end and add modest training and inference time compared with GNL.
Reading between the lines
- If the assumed decomposition holds, the same compensation-plus-filter pattern could transfer to other distillation-based AD frameworks and to other distribution-shift types (e.g., unseen categories, medical domain drift) beyond the four corruptions tested.
- A strong test of the mechanism would be to vary shift severity; if FiCo's gains scale with corruption intensity, that would confirm misalignment, not augmentation, is the active factor.
- The reliance on $C_1(f^{D_1})$ as a pure distribution-specific prior could be relaxed by learning the prior from multiple augmentations, potentially removing the paper's weakest assumption while retaining the gains.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes FiCo, a reverse-distillation (RD) method for anomaly detection under distribution shift. FiCo adds a Distribution-Specific Compensation (DiSCo) module after each student block, forming a residual shortcut f^Dk_F = Ck(f^Dk) + f^Dk, and a Distribution-Invariant Filter (DiIFi) module that aligns earlier-block DiSCo residuals to the final-block DiSCo output. The training objective combines cosine alignment losses (LCo), an MSE loss (Lmse) for DiIFi, and a consistency loss (Lnor). Experiments on MVTec, PACS, and CIFAR-10 with OOD scenarios report higher average sample-level AUROC than GNL and other baselines, and the authors claim state-of-the-art performance as well as improved ID performance over RD-based methods.
Significance. If the reported OOD gains are robust, FiCo is a practical, low-overhead extension of the GNL/RD framework, and the released code is a useful asset. The paper is empirical rather than theoretical, but the benchmark coverage (three datasets, multiple corruption/domain shifts) is reasonable. However, the advertised claim of improved ID performance over RD-based methods is not consistently true across the three benchmarks, and the paper does not directly demonstrate that DiSCo compensates only distribution-specific information. These issues affect the strength of the central claims but are addressable with additional analysis and corrected statements.
major comments (4)
- [Abstract and Table 3] The abstract and the third contribution state that FiCo 'even achieves better results on the ID scenario compared with RD-based methods.' This is contradicted by Table 3 on CIFAR-10: FiCo's ID AUROC is 80.5, while RD achieves 84.6 and GNL achieves 82.3. The claim is also not true against GNL on CIFAR-10. The authors should either restrict the claim to MVTec and PACS or provide a qualified statement that accounts for the CIFAR-10 result.
- [Approach, Eq. 6, and Inference] LCo in Eq. 6 supervises each DiSCo module Ck to make f^Dk_F align with the teacher feature f^Ek on normal augmented images. At inference, DiIFi is discarded while DiSCo remains active. Because f^Dk_F = Ck(f^Dk) + f^Dk, Ck is unconstrained on anomalous inputs and could partially compensate anomaly-induced discrepancies, shrinking the very teacher-student gap that RD-based detection relies on. The paper needs a test-time ablation in which DiSCo is disabled or replaced by the identity mapping, to show that the OOD gains come from compensation of distribution-specific information rather than from a general reduction of anomaly scores. The qualitative anomaly maps in Fig. 4 are not sufficient evidence that anomaly sensitivity is preserved.
- [Approach, Eq. 8] The DiIFi module aligns Ck(f^Dk_n) to Ik(C1(f^D1_n)), but C1 is itself trained only through LCo. This is a circular specification: nothing in the training objective forces C1 to isolate distribution-specific content rather than semantic or anomaly-relevant information. If C1 encodes more than distribution-specific variation, then Eq. 8 does not implement 'filtering all abnormal information' as claimed. A direct evaluation of what C1 captures (e.g., probing its output on OOD shifts vs. anomalous patterns) would be needed to support the mechanism.
- [Appendix, Algorithm 1] Algorithm 1 optimizes over E, phi, D, C, and I parameters, but the Preliminaries state that the teacher network E is frozen. This is a direct inconsistency that affects reproducibility. Also, the text around Eq. 11 says that LRD and Llowf are replaced by LFi and LCo, yet LFi in Eq. 10 still contains Llowf; the relationship between the baseline losses and the proposed losses should be stated precisely.
minor comments (6)
- [Contributions] There is a typo in the first contribution: 'anomlous patterns' should be 'anomalous patterns'.
- [Analysis, Filter and Compensation Process] The phrase 'resti-tute the distribution-specific noise' should read 'restitute' or 'restore'.
- [Table 5] The last column is labeled 'Results' without specifying that the numbers are sample-level AUROC percentages; this should be stated for clarity.
- [Hyper-parameter Sensitivity] The sensitivity analysis in Fig. 3 is reported only on MVTec and without error bars or multiple runs; a brief note on the stability across datasets and runs would improve the claim that 'any combination of hyper-parameter values can surpass GNL.'
- [Implementation Details] The number of DiSCo blocks M is fixed to 4 but is not included in the hyper-parameter sensitivity study; an ablation on M would be informative.
- [Algorithm 1] The pseudo-code contains 'Computer the overall loss' instead of 'Compute the overall loss', and the number of augmentations N is not shown in the loop; specifying N would make the algorithm more reproducible.
Circularity Check
No significant circularity: reported AUROC gains are held-out measurements, and no prediction reduces to a fitted input or self-citation.
full rationale
The paper's central claim is empirical: FiCo achieves higher sample-level AUROC than prior methods on MVTec, PACS, and CIFAR-10 under OOD corruptions. These numbers are measured on test sets that the task description explicitly excludes from training ("Note that no data from test dataset is available during the training process"), so the headline results are not constructed from the method's own equations. The DiSCo and DiIFi modules are trained with explicit losses (Eqs. 6, 8, 9, 10) and evaluated with the standard RD anomaly-scoring procedure plus EFDM test-time augmentation; no parameter is fitted directly to the reported AUROC, and no claimed improvement follows by definition from the loss objective. The assumption that C1 captures distribution-specific rather than anomaly-relevant information is an architectural or soundness premise, not a circular derivation: it is falsifiable by the held-out AUROC, and the ablation in Table 4 shows incremental gains from each component. The only self-citations (Chen et al. 2023, 2024) appear in related-work context for person re-identification and domain generalization and are not load-bearing for FiCo's architecture or evaluation. The appendix's acknowledgment of remaining ID/OOD discrepancy further confirms that the paper reports measured limitations rather than deriving its own success. No specific reduction of a prediction to its inputs, fitted parameter, or self-citation chain can be exhibited, so no circular step is identified.
Assumptions & free parameters
free parameters (4)
- alpha (balance weight for LCo) =
0.05
- beta (balance weight for Lmse) =
0.02 for MVTec and CIFAR-10, 0.1 for PACS
- gamma (balance weight for Lnor) =
1
- M (number of DyConv blocks in DiSCo) =
4
assumptions (5)
- ad hoc to paper Student feature maps decompose into distribution-invariant and distribution-specific components.
- domain assumption A student trained only on normal samples will not reconstruct anomalous patterns, so teacher-student discrepancy indicates anomalies.
- domain assumption Pretrained WideResNet50 features are suitable normal-pattern references for anomaly scoring.
- domain assumption Augmented views of a normal image simulate the distribution shifts seen at test time.
- domain assumption Test-time augmentation via EFDM is a valid way to align OOD test samples with the training distribution.
invented entities (1)
-
Distribution-specific information component
Cite this review
Pith. "Pith review of Filter or Compensate: Towards Invariant Representation from Distribution Shift for Anomaly Detection." pith.science (2026). https://pith.science/paper/YSVCCOWX
@misc{pith2026241210115,
author = {Pith},
title = {Pith review of: Filter or Compensate: Towards Invariant Representation from Distribution Shift for Anomaly Detection},
year = {2026},
howpublished = {\url{https://pith.science/paper/YSVCCOWX}},
note = {Machine review of arXiv:2412.10115}
}
read the original abstract
Recent Anomaly Detection (AD) methods have achieved great success with In-Distribution (ID) data. However, real-world data often exhibits distribution shift, causing huge performance decay on traditional AD methods. From this perspective, few previous work has explored AD with distribution shift, and the distribution-invariant normality learning has been proposed based on the Reverse Distillation (RD) framework. However, we observe the misalignment issue between the teacher and the student network that causes detection failure, thereby propose FiCo, Filter or Compensate, to address the distribution shift issue in AD. FiCo firstly compensates the distribution-specific information to reduce the misalignment between the teacher and student network via the Distribution-Specific Compensation (DiSCo) module, and secondly filters all abnormal information to capture distribution-invariant normality with the Distribution-Invariant Filter (DiIFi) module. Extensive experiments on three different AD benchmarks demonstrate the effectiveness of FiCo, which outperforms all existing state-of-the-art (SOTA) methods, and even achieves better results on the ID scenario compared with RD-based methods. Our code is available at https://github.com/znchen666/FiCo.
Figures
Figures from the paper (2 more)
Reference graph
Works this paper leans on
-
[1]
The results are shown in Table 6
Results on MVTec. The results are shown in Table 6. It can be observed that performance on most OOD scenarios has improved with a relatively large margin
-
[2]
The results are shown in Table 7
Results on PACS. The results are shown in Table 7. There still exists discrepancy between ID and OOD scenar- ios that future research should focus on different types of domain shifts for distribution-invariant learning
-
[3]
In Proceedings of the IEEE/CVF conference on com- puter vision and pattern recognition, 11030–11039
Dynamic convolution: Attention over convolution ker- nels. In Proceedings of the IEEE/CVF conference on com- puter vision and pattern recognition, 11030–11039. Chen, Z.; Wang, W.; Zhao, Z.; Su, F.; Men, A.; and Dong, Y . 2023. Cluster-instance normalization: A statis- tical relation-aware normalization for generalizable person re-identification. IEEE Tran...
arXiv 2023
-
[5]
In International Con- ference on Pattern Recognition, 475–489
Padim: a patch distribution modeling framework for anomaly detection and localization. In International Con- ference on Pattern Recognition, 475–489. Springer. Deng, H.; and Li, X. 2022. Anomaly detection via reverse distillation from one-class embedding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 9737–9746. Fore...
work page 2022
-
[6]
arXiv preprint arXiv:2010.01412
Sharpness-aware minimization for efficiently improv- ing generalization. arXiv preprint arXiv:2010.01412. Goodfellow, I.; Pouget-Abadie, J.; Mirza, M.; Xu, B.; Warde-Farley, D.; Ozair, S.; Courville, A.; and Bengio, Y
arXiv 2010
-
[7]
Communications of the ACM, 63(11): 139–144
Generative adversarial networks. Communications of the ACM, 63(11): 139–144. Gu, Z.; Liu, L.; Chen, X.; Yi, R.; Zhang, J.; Wang, Y .; Wang, C.; Shu, A.; Jiang, G.; and Ma, L. 2023. Remem- bering Normality: Memory-guided Knowledge Distillation for Unsupervised Anomaly Detection. In Proceedings of the IEEE/CVF International Conference on Computer Vi- sion, ...
arXiv 2023
-
[11]
In European Con- ference on Computer Vision, 535–552
Style-hallucinated dual consistency learning for do- main generalized semantic segmentation. In European Con- ference on Computer Vision, 535–552. Springer. Zhou, K.; Yang, Y .; Qiao, Y .; and Xiang, T. 2021. Domain generalization with mixstyle. arXiv preprint arXiv:2104.02008. Zhu, J.; and Pang, G. 2024. Toward generalist anomaly de- tection via in-conte...
arXiv 2021
-
[15]
The one-class novelty detection results on CIFAR-10 are shown in Table 8
Results on CIFAR-10. The one-class novelty detection results on CIFAR-10 are shown in Table 8. Gaussian noise and defocus blur are still two difficult domains that require further exploration. Table 6: Results on MVTec for FiCo. Category ID Br Co Bl No carpet 99.24 98.60 98.23 99.04 98.48 leather 100 100 99.93 100 100 grid 99.50 98.91 98.16 98.75 98.08 ti...
Show all 15 references
-
[450]
Springer. Tien, T. D.; Nguyen, A. T.; Tran, N. H.; Huy, T. D.; Duong, S.; Nguyen, C. D. T.; and Truong, S. Q. 2023. Revisiting reverse distillation for anomaly detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 24511–24520. Ulyanov...
2023 arXiv
-
[2017]
2009) to ex- pound the results presented in the main paper
and CIFAR-10 (Krizhevsky, Hinton et al. 2009) to ex- pound the results presented in the main paper
2009
-
[2019]
In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 9592–9600
MVTec AD–A comprehensive real-world dataset for unsupervised anomaly detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 9592–9600. Bergmann, P.; Fauser, M.; Sattlegger, D.; and Steger, C
-
[2020]
In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 4183–4192
Uninformed students: Student-teacher anomaly detec- tion with discriminative latent embeddings. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 4183–4192. Cao, T.; Zhu, J.; and Pang, G. 2023. Anomaly detection under distribution shift. In ...
2023
-
[2021]
In In- ternational Conference on Machine Learning , 2546–2558
Transfer-based semantic anomaly detection. In In- ternational Conference on Machine Learning , 2546–2558. PMLR. Defard, T.; Setkov, A.; Loesch, A.; and Audigier, R
-
[2022]
In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 650–656
Anoddpm: Anomaly detection with denoising diffu- sion probabilistic models using simplex noise. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 650–656. Xu, B.; Wang, N.; Chen, T.; and Li, M. 2015. Empirical evaluation of rectified activ...
2015 arXiv
-
[2023]
In Proceedings of the IEEE/CVF International Conference on Computer Vision, 6782–6791
Unsupervised surface anomaly detection with diffu- sion probabilistic model. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 6782–6791. Zhang, X.; Xu, M.; and Zhou, X. 2024. RealNet: A feature selection network with realistic synthetic anomaly for a...
2024
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.