Pith. sign in

REVIEW 4 major objections 6 minor 15 references

Filter or Compensate: Towards Invariant Representation from Distribution Shift for Anomaly Detection

T0 review · 4 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Pith's one-line read The paper claims that teacher–student misalignment on distribution-specific information is the main cause of anomaly-detection failure under distribution shift, and that compensating for this information while filtering abnormal content…

desk verdict Incremental but real extension of GNL: consistent OOD gains, overclaimed ID results, and a missing test-time ablation. read the letter →

arxiv 2412.10115 v1 pith:YSVCCOWX submitted 2024-12-13 cs.CV

classification cs.CV
keywords anomalydetectiondistributionshiftout-of-distributiongeneralizationreversedistillationknowledgeinvariantrepresentationfeaturecompensationtest-timeaugmentation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that recent anomaly detectors collapse under distribution shift because, in the reverse-distillation framework, the teacher and student networks disagree on distribution-specific information (such as brightness or noise) and that disagreement is scored as an anomaly. FiCo addresses this by compensating for distribution-specific information with a DiSCo module, which reconstructs and adds back what the student lost, and by filtering abnormal information with a DiIFi module, which forces earlier student blocks to discard distribution-specific content. The reported result is that FiCo raises sample-level AUROC on the MVTec, PACS, and CIFAR-10 benchmarks under common corruptions, beating existing state-of-the-art methods, and also improves in-distribution performance relative to RD-based methods. The central claim is that the misalignment itself, not the distribution shift per se, is the correct target for intervention.

What carries the argument

The machinery is the DiSCo–DiIFi pair. DiSCo is a stack of Dynamic Convolution, InstanceNorm, and LeakyReLU blocks inserted after each student block, with a residual shortcut; it is trained by $L_{\mathrm{Co}}$ to reconstruct distribution-specific information that aligns the student with the teacher. DiIFi uses the final DiSCo output $C_1(f^{D_1})$ as the estimate of distribution-specific content, transforms it to earlier spatial resolutions with convolutional blocks, and supervises the earlier DiSCo outputs with an MSE loss ($L_{\mathrm{mse}}$). This forces early blocks to filter abnormal and distribution-specific information, so the anomaly score is driven by genuine anomalies rather than by the distribution shift.

What would settle it

Train a linear classifier on the feature maps $C_1(f^{D_1})$ produced by FiCo to predict either the object class or the corruption type; if class prediction accuracy is far above chance, $C_1$ encodes semantic content, contradicting the assumption that it holds only distribution-specific information, and the DiIFi supervision via Eqs. (7)–(8) is mis-specified.

Watch

Extended reading notes

Core claim

The paper's central discovery is that teacher–student discrepancy in RD-based anomaly detection is dominated by distribution-specific information rather than true anomalies, so existing methods mistake style changes for defects. By adding a Distribution-Specific Compensation module after each student block, trained to align compensated features with the teacher, and a Distribution-Invariant Filter module that propagates the final block's distribution-specific estimate back to earlier blocks, FiCo restores alignment on OOD data while keeping genuine anomalies as the main source of discrepancy. This decomposition yields consistent improvements on corrupted test sets and, because the compensation also helps on clean data, better in-distribution accuracy than prior RD-based methods.

Load-bearing premise

The paper assumes that each student feature map splits cleanly into a distribution-invariant part and a distribution-specific part, and that the final DiSCo output $C_1(f^{D_1})$ captures only the latter, so it can be used to supervise earlier blocks; if $C_1$ also encodes semantic or anomaly-relevant content, the compensation–filter mechanism is mis-specified and the gains may not transfer.

Editorial extensions

If this is right

  • On MVTec, PACS, and CIFAR-10 with brightness, contrast, defocus blur, and gaussian-noise corruptions, FiCo reports higher sample-level AUROC than all compared methods.
  • FiCo improves in-distribution performance relative to RD-based methods, indicating the compensation helps even without distribution shift.
  • Anomaly maps and score distributions show that FiCo concentrates activation on true anomalous regions while suppressing distribution-specific regions.
  • The added modules are trainable end-to-end and add modest training and inference time compared with GNL.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the assumed decomposition holds, the same compensation-plus-filter pattern could transfer to other distillation-based AD frameworks and to other distribution-shift types (e.g., unseen categories, medical domain drift) beyond the four corruptions tested.
  • A strong test of the mechanism would be to vary shift severity; if FiCo's gains scale with corruption intensity, that would confirm misalignment, not augmentation, is the active factor.
  • The reliance on $C_1(f^{D_1})$ as a pure distribution-specific prior could be relaxed by learning the prior from multiple augmentations, potentially removing the paper's weakest assumption while retaining the gains.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes FiCo, a reverse-distillation (RD) method for anomaly detection under distribution shift. FiCo adds a Distribution-Specific Compensation (DiSCo) module after each student block, forming a residual shortcut f^Dk_F = Ck(f^Dk) + f^Dk, and a Distribution-Invariant Filter (DiIFi) module that aligns earlier-block DiSCo residuals to the final-block DiSCo output. The training objective combines cosine alignment losses (LCo), an MSE loss (Lmse) for DiIFi, and a consistency loss (Lnor). Experiments on MVTec, PACS, and CIFAR-10 with OOD scenarios report higher average sample-level AUROC than GNL and other baselines, and the authors claim state-of-the-art performance as well as improved ID performance over RD-based methods.

Significance. If the reported OOD gains are robust, FiCo is a practical, low-overhead extension of the GNL/RD framework, and the released code is a useful asset. The paper is empirical rather than theoretical, but the benchmark coverage (three datasets, multiple corruption/domain shifts) is reasonable. However, the advertised claim of improved ID performance over RD-based methods is not consistently true across the three benchmarks, and the paper does not directly demonstrate that DiSCo compensates only distribution-specific information. These issues affect the strength of the central claims but are addressable with additional analysis and corrected statements.

major comments (4)
  1. [Abstract and Table 3] The abstract and the third contribution state that FiCo 'even achieves better results on the ID scenario compared with RD-based methods.' This is contradicted by Table 3 on CIFAR-10: FiCo's ID AUROC is 80.5, while RD achieves 84.6 and GNL achieves 82.3. The claim is also not true against GNL on CIFAR-10. The authors should either restrict the claim to MVTec and PACS or provide a qualified statement that accounts for the CIFAR-10 result.
  2. [Approach, Eq. 6, and Inference] LCo in Eq. 6 supervises each DiSCo module Ck to make f^Dk_F align with the teacher feature f^Ek on normal augmented images. At inference, DiIFi is discarded while DiSCo remains active. Because f^Dk_F = Ck(f^Dk) + f^Dk, Ck is unconstrained on anomalous inputs and could partially compensate anomaly-induced discrepancies, shrinking the very teacher-student gap that RD-based detection relies on. The paper needs a test-time ablation in which DiSCo is disabled or replaced by the identity mapping, to show that the OOD gains come from compensation of distribution-specific information rather than from a general reduction of anomaly scores. The qualitative anomaly maps in Fig. 4 are not sufficient evidence that anomaly sensitivity is preserved.
  3. [Approach, Eq. 8] The DiIFi module aligns Ck(f^Dk_n) to Ik(C1(f^D1_n)), but C1 is itself trained only through LCo. This is a circular specification: nothing in the training objective forces C1 to isolate distribution-specific content rather than semantic or anomaly-relevant information. If C1 encodes more than distribution-specific variation, then Eq. 8 does not implement 'filtering all abnormal information' as claimed. A direct evaluation of what C1 captures (e.g., probing its output on OOD shifts vs. anomalous patterns) would be needed to support the mechanism.
  4. [Appendix, Algorithm 1] Algorithm 1 optimizes over E, phi, D, C, and I parameters, but the Preliminaries state that the teacher network E is frozen. This is a direct inconsistency that affects reproducibility. Also, the text around Eq. 11 says that LRD and Llowf are replaced by LFi and LCo, yet LFi in Eq. 10 still contains Llowf; the relationship between the baseline losses and the proposed losses should be stated precisely.
minor comments (6)
  1. [Contributions] There is a typo in the first contribution: 'anomlous patterns' should be 'anomalous patterns'.
  2. [Analysis, Filter and Compensation Process] The phrase 'resti-tute the distribution-specific noise' should read 'restitute' or 'restore'.
  3. [Table 5] The last column is labeled 'Results' without specifying that the numbers are sample-level AUROC percentages; this should be stated for clarity.
  4. [Hyper-parameter Sensitivity] The sensitivity analysis in Fig. 3 is reported only on MVTec and without error bars or multiple runs; a brief note on the stability across datasets and runs would improve the claim that 'any combination of hyper-parameter values can surpass GNL.'
  5. [Implementation Details] The number of DiSCo blocks M is fixed to 4 but is not included in the hyper-parameter sensitivity study; an ablation on M would be informative.
  6. [Algorithm 1] The pseudo-code contains 'Computer the overall loss' instead of 'Compute the overall loss', and the number of augmentations N is not shown in the loop; specifying N would make the algorithm more reproducible.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: reported AUROC gains are held-out measurements, and no prediction reduces to a fitted input or self-citation.

full rationale

The paper's central claim is empirical: FiCo achieves higher sample-level AUROC than prior methods on MVTec, PACS, and CIFAR-10 under OOD corruptions. These numbers are measured on test sets that the task description explicitly excludes from training ("Note that no data from test dataset is available during the training process"), so the headline results are not constructed from the method's own equations. The DiSCo and DiIFi modules are trained with explicit losses (Eqs. 6, 8, 9, 10) and evaluated with the standard RD anomaly-scoring procedure plus EFDM test-time augmentation; no parameter is fitted directly to the reported AUROC, and no claimed improvement follows by definition from the loss objective. The assumption that C1 captures distribution-specific rather than anomaly-relevant information is an architectural or soundness premise, not a circular derivation: it is falsifiable by the held-out AUROC, and the ablation in Table 4 shows incremental gains from each component. The only self-citations (Chen et al. 2023, 2024) appear in related-work context for person re-identification and domain generalization and are not load-bearing for FiCo's architecture or evaluation. The appendix's acknowledgment of remaining ID/OOD discrepancy further confirms that the paper reports measured limitations rather than deriving its own success. No specific reduction of a prediction to its inputs, fitted parameter, or self-citation chain can be exhibited, so no circular step is identified.

Assumptions & free parameters 4 free parameters · 5 assumptions · 1 invented entities

The central claim relies on an assumed decomposition of student features into invariant and distribution-specific parts, plus several inherited assumptions from reverse distillation and GNL. Four hyperparameters are selected per benchmark. The only invented entity is the conceptual latent variable 'distribution-specific information', which has no independent evidence.

free parameters (4)
  • alpha (balance weight for LCo) = 0.05
    Selected on MVTec and CIFAR-10; sensitivity shown in Fig. 3a.
  • beta (balance weight for Lmse) = 0.02 for MVTec and CIFAR-10, 0.1 for PACS
    Selected per benchmark; sensitivity shown in Fig. 3b.
  • gamma (balance weight for Lnor) = 1
    Selected on MVTec and CIFAR-10; sensitivity shown in Fig. 3c.
  • M (number of DyConv blocks in DiSCo) = 4
    Set to 4 for all datasets with no sensitivity study.
assumptions (5)
  • ad hoc to paper Student feature maps decompose into distribution-invariant and distribution-specific components.
    Stated as an assumption in the Approach section; no independent evidence or measurement supports the decomposition.
  • domain assumption A student trained only on normal samples will not reconstruct anomalous patterns, so teacher-student discrepancy indicates anomalies.
    Inherited from reverse distillation literature and used throughout FiCo's design.
  • domain assumption Pretrained WideResNet50 features are suitable normal-pattern references for anomaly scoring.
    Backbone choice is inherited from GNL and RD; no analysis of feature reliability is provided.
  • domain assumption Augmented views of a normal image simulate the distribution shifts seen at test time.
    Inherited from GNL; the consistency losses in Eqs. 2, 3, and 9 rely on this.
  • domain assumption Test-time augmentation via EFDM is a valid way to align OOD test samples with the training distribution.
    Inherited from GNL and kept fixed for fair comparison.
invented entities (1)
  • Distribution-specific information component
    purpose: Postulated latent part of student features that DiSCo compensates and DiIFi filters
    No external measurement exists; the existence and separability of this component is assumed, not demonstrated.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Filter or Compensate: Towards Invariant Representation from Distribution Shift for Anomaly Detection." pith.science (2026). https://pith.science/paper/YSVCCOWX

@misc{pith2026241210115,
  author       = {Pith},
  title        = {Pith review of: Filter or Compensate: Towards Invariant Representation from Distribution Shift for Anomaly Detection},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YSVCCOWX}},
  note         = {Machine review of arXiv:2412.10115}
}
read the original abstract

Recent Anomaly Detection (AD) methods have achieved great success with In-Distribution (ID) data. However, real-world data often exhibits distribution shift, causing huge performance decay on traditional AD methods. From this perspective, few previous work has explored AD with distribution shift, and the distribution-invariant normality learning has been proposed based on the Reverse Distillation (RD) framework. However, we observe the misalignment issue between the teacher and the student network that causes detection failure, thereby propose FiCo, Filter or Compensate, to address the distribution shift issue in AD. FiCo firstly compensates the distribution-specific information to reduce the misalignment between the teacher and student network via the Distribution-Specific Compensation (DiSCo) module, and secondly filters all abnormal information to capture distribution-invariant normality with the Distribution-Invariant Filter (DiIFi) module. Extensive experiments on three different AD benchmarks demonstrate the effectiveness of FiCo, which outperforms all existing state-of-the-art (SOTA) methods, and even achieves better results on the ID scenario compared with RD-based methods. Our code is available at https://github.com/znchen666/FiCo.

Figures

Figures reproduced from arXiv: 2412.10115 by the authors.

Figure 1
Figure 1. Anomaly map from different scenarios of SOTA [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The overall architecture of our method FiCo, in [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Experimental results on hyper-parameters on the [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Anomaly map of f Dk and f Dk F on the MVTec benchmark. Each row represents a different scenario, in￾cluding ID, defocus blur and gaussian noise. For each sce￾nario, two examples are shown with the original image, the groundtruth label, anomaly map from f Dk and from f …
Figure 5
Figure 5. Figure 5: Anomaly scores of FiCo and GNL (Cao, Zhu, and [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

15 extracted references · 9 canonical work pages

  1. [1]

    The results are shown in Table 6

    Results on MVTec. The results are shown in Table 6. It can be observed that performance on most OOD scenarios has improved with a relatively large margin

  2. [2]

    The results are shown in Table 7

    Results on PACS. The results are shown in Table 7. There still exists discrepancy between ID and OOD scenar- ios that future research should focus on different types of domain shifts for distribution-invariant learning

  3. [3]

    In Proceedings of the IEEE/CVF conference on com- puter vision and pattern recognition, 11030–11039

    Dynamic convolution: Attention over convolution ker- nels. In Proceedings of the IEEE/CVF conference on com- puter vision and pattern recognition, 11030–11039. Chen, Z.; Wang, W.; Zhao, Z.; Su, F.; Men, A.; and Dong, Y . 2023. Cluster-instance normalization: A statis- tical relation-aware normalization for generalizable person re-identification. IEEE Tran...

  4. [5]

    In International Con- ference on Pattern Recognition, 475–489

    Padim: a patch distribution modeling framework for anomaly detection and localization. In International Con- ference on Pattern Recognition, 475–489. Springer. Deng, H.; and Li, X. 2022. Anomaly detection via reverse distillation from one-class embedding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 9737–9746. Fore...

  5. [6]

    arXiv preprint arXiv:2010.01412

    Sharpness-aware minimization for efficiently improv- ing generalization. arXiv preprint arXiv:2010.01412. Goodfellow, I.; Pouget-Abadie, J.; Mirza, M.; Xu, B.; Warde-Farley, D.; Ozair, S.; Courville, A.; and Bengio, Y

  6. [7]

    Communications of the ACM, 63(11): 139–144

    Generative adversarial networks. Communications of the ACM, 63(11): 139–144. Gu, Z.; Liu, L.; Chen, X.; Yi, R.; Zhang, J.; Wang, Y .; Wang, C.; Shu, A.; Jiang, G.; and Ma, L. 2023. Remem- bering Normality: Memory-guided Knowledge Distillation for Unsupervised Anomaly Detection. In Proceedings of the IEEE/CVF International Conference on Computer Vi- sion, ...

  7. [11]

    In European Con- ference on Computer Vision, 535–552

    Style-hallucinated dual consistency learning for do- main generalized semantic segmentation. In European Con- ference on Computer Vision, 535–552. Springer. Zhou, K.; Yang, Y .; Qiao, Y .; and Xiang, T. 2021. Domain generalization with mixstyle. arXiv preprint arXiv:2104.02008. Zhu, J.; and Pang, G. 2024. Toward generalist anomaly de- tection via in-conte...

  8. [15]

    The one-class novelty detection results on CIFAR-10 are shown in Table 8

    Results on CIFAR-10. The one-class novelty detection results on CIFAR-10 are shown in Table 8. Gaussian noise and defocus blur are still two difficult domains that require further exploration. Table 6: Results on MVTec for FiCo. Category ID Br Co Bl No carpet 99.24 98.60 98.23 99.04 98.48 leather 100 100 99.93 100 100 grid 99.50 98.91 98.16 98.75 98.08 ti...

Show all 15 references
  1. [450]

    Springer. Tien, T. D.; Nguyen, A. T.; Tran, N. H.; Huy, T. D.; Duong, S.; Nguyen, C. D. T.; and Truong, S. Q. 2023. Revisiting reverse distillation for anomaly detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 24511–24520. Ulyanov...

  2. [2017]

    2009) to ex- pound the results presented in the main paper

    and CIFAR-10 (Krizhevsky, Hinton et al. 2009) to ex- pound the results presented in the main paper

  3. [2019]

    In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 9592–9600

    MVTec AD–A comprehensive real-world dataset for unsupervised anomaly detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 9592–9600. Bergmann, P.; Fauser, M.; Sattlegger, D.; and Steger, C

  4. [2020]

    In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 4183–4192

    Uninformed students: Student-teacher anomaly detec- tion with discriminative latent embeddings. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 4183–4192. Cao, T.; Zhu, J.; and Pang, G. 2023. Anomaly detection under distribution shift. In ...

  5. [2021]

    In In- ternational Conference on Machine Learning , 2546–2558

    Transfer-based semantic anomaly detection. In In- ternational Conference on Machine Learning , 2546–2558. PMLR. Defard, T.; Setkov, A.; Loesch, A.; and Audigier, R

  6. [2022]

    In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 650–656

    Anoddpm: Anomaly detection with denoising diffu- sion probabilistic models using simplex noise. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 650–656. Xu, B.; Wang, N.; Chen, T.; and Li, M. 2015. Empirical evaluation of rectified activ...

  7. [2023]

    In Proceedings of the IEEE/CVF International Conference on Computer Vision, 6782–6791

    Unsupervised surface anomaly detection with diffu- sion probabilistic model. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 6782–6791. Zhang, X.; Xu, M.; and Zhou, X. 2024. RealNet: A feature selection network with realistic synthetic anomaly for a...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.