{"id":"130f3bb9-b28e-4ca9-8cef-4fa7f1c4cdeb","arxiv_id":"2412.10115","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"FiCo adds a compensation module and a filter module to reverse distillation, improving image anomaly detection when test images undergo brightness, blur, noise, or style shifts.","lead":"This paper introduces FiCo, a method for image anomaly detection that stays accurate when test images differ from training images in brightness, blur, noise, or style. It adds two small modules to the reverse distillation framework, one compensating for the changed appearance and one filtering out irrelevant variation, and reports higher average scores on three benchmarks.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The compensation loss (Eq. 6) can reward DiSCo for reconstructing teacher features on anomalous inputs, and since DiIFi is test-time discarded, nothing prevents anomaly masking; a test-time DiSCo ablation is needed.","rationale":"The reader's weakest assumption identifies the feature decomposition as the soft spot; I agree and sharpen it to a concrete failure mode. The load-bearing issue is not merely that C1 may encode semantics, but that the training objective actively encourages it. Eq. 6 pushes compensated features toward teacher features; without an explicit constraint forcing the residual to be orthogonal to anomaly content, the easiest way to reduce LCo on normal data may also reduce the teacher-student gap on anomalies. The DiIFi module, which is supposed to enforce the decomposition, takes C1 as its teaching signal (Eq. 8), so it cannot validate C1's independence from anomaly content. And because DiIFi is absent at inference, any bad inductive bias in C1 is uncorrected. This is directly testable by a test-time ablation.\n\nOther concerns (CIFAR-10 ID contradiction, no error bars) are real but secondary. The CIFAR-10 ID claim is a wording/consistency issue; lack of error bars affects confidence but not the mechanism. The proposed test would settle whether the reported OOD improvements are due to the claimed filter-compensate process or to a trivial 'always match teacher' behavior.\n\nCredit: The paper reports consistent OOD improvements across three benchmarks and stepwise ablations showing each component helps, which is positive evidence. However, the ablations do not separate training-time from test-time effects, so they cannot rule out the anomaly-masking scenario.\n\nVerdict remains CONDITIONAL; the condition should include this test or equivalent analysis before the central claim is accepted.","tokens_in":15471,"tokens_out":8923,"duration_ms":94551,"concrete_test":"On MVTec or PACS OOD settings, compute sample-level AUROC for three inference variants: (1) full FiCo with DiSCo active; (2) DiSCo ablated (set all Ck outputs to zero, scoring with f^Dk only); (3) DiSCo active only on normal OOD images and zeroed on anomalous images. If (1) is not clearly better than (2), or if (3) shows DiSCo reduces anomaly scores on anomalous images more than on normal OOD images, then compensation is masking anomalies and the central mechanism claim fails. Report per-class and average AUROC with error bars over multiple seeds.","verdict_should_be":"UNCHANGED","load_bearing_attack":"FiCo's central mechanism rests on DiSCo compensating only distribution-specific information, but the only supervision for DiSCo is LCo (Eq. 6), which minimizes teacher-student cosine distance on normal augmented images. Because f^Dk_F = Ck(f^Dk)+f^Dk, the residual Ck is free to learn any mapping that reconstructs teacher features. On anomalous test images, Ck's behavior is unconstrained; it can partially compensate anomaly-induced discrepancies, shrinking the very gap that RD-based AD uses to detect anomalies. The DiIFi module is supposed to prevent this by aligning earlier blocks to C1(f^D1) (Eq. 8), but C1 is itself trained with the same LCo, making the filter's target circular. Moreover, DiIFi is discarded at inference, leaving DiSCo unopposed. The paper's qualitative Fig. 4 is not sufficient evidence that compensation preserves anomaly sensitivity. If Ck reconstructs teacher features for anomalous inputs, the reported OOD gains could reflect uniformly lower anomaly scores rather than better normal-vs-anomaly separation.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes FiCo, a reverse-distillation (RD) method for anomaly detection under distribution shift. FiCo adds a Distribution-Specific Compensation (DiSCo) module after each student block, forming a residual shortcut f^Dk_F = Ck(f^Dk) + f^Dk, and a Distribution-Invariant Filter (DiIFi) module that aligns earlier-block DiSCo residuals to the final-block DiSCo output. The training objective combines cosine alignment losses (LCo), an MSE loss (Lmse) for DiIFi, and a consistency loss (Lnor). Experiments on MVTec, PACS, and CIFAR-10 with OOD scenarios report higher average sample-level AUROC than GNL and other baselines, and the authors claim state-of-the-art performance as well as improved ID performance over RD-based methods.","tokens_in":15742,"tokens_out":3807,"duration_ms":40427,"significance":"If the reported OOD gains are robust, FiCo is a practical, low-overhead extension of the GNL/RD framework, and the released code is a useful asset. The paper is empirical rather than theoretical, but the benchmark coverage (three datasets, multiple corruption/domain shifts) is reasonable. However, the advertised claim of improved ID performance over RD-based methods is not consistently true across the three benchmarks, and the paper does not directly demonstrate that DiSCo compensates only distribution-specific information. These issues affect the strength of the central claims but are addressable with additional analysis and corrected statements.","major_comments":[{"comment":"The abstract and the third contribution state that FiCo 'even achieves better results on the ID scenario compared with RD-based methods.' This is contradicted by Table 3 on CIFAR-10: FiCo's ID AUROC is 80.5, while RD achieves 84.6 and GNL achieves 82.3. The claim is also not true against GNL on CIFAR-10. The authors should either restrict the claim to MVTec and PACS or provide a qualified statement that accounts for the CIFAR-10 result.","section":"Abstract and Table 3"},{"comment":"LCo in Eq. 6 supervises each DiSCo module Ck to make f^Dk_F align with the teacher feature f^Ek on normal augmented images. At inference, DiIFi is discarded while DiSCo remains active. Because f^Dk_F = Ck(f^Dk) + f^Dk, Ck is unconstrained on anomalous inputs and could partially compensate anomaly-induced discrepancies, shrinking the very teacher-student gap that RD-based detection relies on. The paper needs a test-time ablation in which DiSCo is disabled or replaced by the identity mapping, to show that the OOD gains come from compensation of distribution-specific information rather than from a general reduction of anomaly scores. The qualitative anomaly maps in Fig. 4 are not sufficient evidence that anomaly sensitivity is preserved.","section":"Approach, Eq. 6, and Inference"},{"comment":"The DiIFi module aligns Ck(f^Dk_n) to Ik(C1(f^D1_n)), but C1 is itself trained only through LCo. This is a circular specification: nothing in the training objective forces C1 to isolate distribution-specific content rather than semantic or anomaly-relevant information. If C1 encodes more than distribution-specific variation, then Eq. 8 does not implement 'filtering all abnormal information' as claimed. A direct evaluation of what C1 captures (e.g., probing its output on OOD shifts vs. anomalous patterns) would be needed to support the mechanism.","section":"Approach, Eq. 8"},{"comment":"Algorithm 1 optimizes over E, phi, D, C, and I parameters, but the Preliminaries state that the teacher network E is frozen. This is a direct inconsistency that affects reproducibility. Also, the text around Eq. 11 says that LRD and Llowf are replaced by LFi and LCo, yet LFi in Eq. 10 still contains Llowf; the relationship between the baseline losses and the proposed losses should be stated precisely.","section":"Appendix, Algorithm 1"}],"minor_comments":[{"comment":"There is a typo in the first contribution: 'anomlous patterns' should be 'anomalous patterns'.","section":"Contributions"},{"comment":"The phrase 'resti-tute the distribution-specific noise' should read 'restitute' or 'restore'.","section":"Analysis, Filter and Compensation Process"},{"comment":"The last column is labeled 'Results' without specifying that the numbers are sample-level AUROC percentages; this should be stated for clarity.","section":"Table 5"},{"comment":"The sensitivity analysis in Fig. 3 is reported only on MVTec and without error bars or multiple runs; a brief note on the stability across datasets and runs would improve the claim that 'any combination of hyper-parameter values can surpass GNL.'","section":"Hyper-parameter Sensitivity"},{"comment":"The number of DiSCo blocks M is fixed to 4 but is not included in the hyper-parameter sensitivity study; an ablation on M would be informative.","section":"Implementation Details"},{"comment":"The pseudo-code contains 'Computer the overall loss' instead of 'Compute the overall loss', and the number of augmentations N is not shown in the loop; specifying N would make the algorithm more reproducible.","section":"Algorithm 1"}],"recommendation":"major_revision","confidential_remarks":"The core empirical direction is sound, and the code release is valuable. The main concern is that the paper overstates the ID-performance claim and does not provide a test-time DiSCo ablation to rule out anomaly masking. These issues are fixable within the manuscript's scope, so I recommend major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Read it. The short version: this is an incremental but genuine extension of GNL for anomaly detection under distribution shift, and the OOD improvements are consistent across three benchmarks. The abstract overclaims ID gains, and there is a missing test-time ablation the authors should be asked to add.\n\nWhat's actually new: the DiSCo and DiIFi modules with the LCo, Lmse, and Lnor losses are not in GNL or RD. The paper identifies a real misalignment between teacher and student under OOD and adds a compensation pathway plus a filter that supervises earlier blocks. The held-out AUROC on MVTec, PACS, and CIFAR-10 shows consistent 1–3 point gains over GNL, and the component ablation is clean: each piece adds a bit (0.45, 1.59, 0.90 points). The hyperparameter sensitivity analysis is also reassuringly flat.\n\nSoft spots, in proportion:\n\nFirst, the abstract's claim that FiCo also beats RD-based methods on ID is contradicted by CIFAR-10, where RD gets 84.6, GNL 82.3, and FiCo 80.5. That is true on MVTec and PACS but not on CIFAR-10. A conditional accept should require fixing that sentence.\n\nSecond, the compensation mechanism is under-specified at test time. DiSCo is trained only to make compensated student features match teacher features on normal augmented images. At inference, DiIFi is discarded, so DiSCo is unopposed. Nothing in the training objective explicitly prevents C from compensating anomaly-induced discrepancies, which would shrink the very gap RD-based detection relies on. The paper's Fig. 4 shows a few anomaly maps, but that is not evidence. I would want a quantitative ablation: inference with DiSCo removed, with it active, and ideally with DiIFi kept too. If OOD AUROC degrades when DiSCo is removed, the story holds; if it improves, the mechanism is wrong. This is a test the authors can run without much effort.\n\nThird, minor: no error bars or significance tests, and the gains are small enough that single-seed runs could be noisy. The consistency across three benchmarks helps, but variance would be nice. The student-feature decomposition into invariant and specific parts is asserted rather than verified; that is acceptable if the empirical result holds, but it weakens the explanatory story.\n\nFourth, the code repository is promised but I cannot verify it from the text. At review time I'd want the link to actually work.\n\nOverall, the paper deserves peer review. The method is new relative to the cited literature, the experiments are standard, and the results are plausible. The issues are fixable with an ablation and a corrected abstract. I'd lean conditional accept, not reject.","headline":"Incremental but real extension of GNL: consistent OOD gains, overclaimed ID results, and a missing test-time ablation.","tokens_in":16267,"tokens_out":3866,"would_cite":true,"duration_ms":38789,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that teacher–student misalignment on distribution-specific information is the main cause of anomaly-detection failure under distribution shift, and that compensating for this information while filtering abnormal content…","keywords":["anomaly detection","distribution shift","out-of-distribution generalization","reverse distillation","knowledge distillation","invariant representation","feature compensation","test-time augmentation"],"falsifier":"Train a linear classifier on the feature maps $C_1(f^{D_1})$ produced by FiCo to predict either the object class or the corruption type; if class prediction accuracy is far above chance, $C_1$ encodes semantic content, contradicting the assumption that it holds only distribution-specific information, and the DiIFi supervision via Eqs. (7)–(8) is mis-specified.","tokens_in":15286,"feed_emoji":"🔍","tokens_out":12394,"duration_ms":102625,"temperature":0.7,"pith_summary":"The paper argues that recent anomaly detectors collapse under distribution shift because, in the reverse-distillation framework, the teacher and student networks disagree on distribution-specific information (such as brightness or noise) and that disagreement is scored as an anomaly. FiCo addresses this by compensating for distribution-specific information with a DiSCo module, which reconstructs and adds back what the student lost, and by filtering abnormal information with a DiIFi module, which forces earlier student blocks to discard distribution-specific content. The reported result is that FiCo raises sample-level AUROC on the MVTec, PACS, and CIFAR-10 benchmarks under common corruptions, beating existing state-of-the-art methods, and also improves in-distribution performance relative to RD-based methods. The central claim is that the misalignment itself, not the distribution shift per se, is the correct target for intervention.","feed_headline":"Compensate, don't just filter, to beat shifted anomaly detection","feed_subtitle":"Compensating for distribution-specific features and filtering abnormal ones boosts AUROC on MVTec, PACS, and CIFAR-10.","key_machinery":"The machinery is the DiSCo–DiIFi pair. DiSCo is a stack of Dynamic Convolution, InstanceNorm, and LeakyReLU blocks inserted after each student block, with a residual shortcut; it is trained by $L_{\\mathrm{Co}}$ to reconstruct distribution-specific information that aligns the student with the teacher. DiIFi uses the final DiSCo output $C_1(f^{D_1})$ as the estimate of distribution-specific content, transforms it to earlier spatial resolutions with convolutional blocks, and supervises the earlier DiSCo outputs with an MSE loss ($L_{\\mathrm{mse}}$). This forces early blocks to filter abnormal and distribution-specific information, so the anomaly score is driven by genuine anomalies rather than by the distribution shift.","core_discovery":"The paper's central discovery is that teacher–student discrepancy in RD-based anomaly detection is dominated by distribution-specific information rather than true anomalies, so existing methods mistake style changes for defects. By adding a Distribution-Specific Compensation module after each student block, trained to align compensated features with the teacher, and a Distribution-Invariant Filter module that propagates the final block's distribution-specific estimate back to earlier blocks, FiCo restores alignment on OOD data while keeping genuine anomalies as the main source of discrepancy. This decomposition yields consistent improvements on corrupted test sets and, because the compensation also helps on clean data, better in-distribution accuracy than prior RD-based methods.","pith_inferences":["If the assumed decomposition holds, the same compensation-plus-filter pattern could transfer to other distillation-based AD frameworks and to other distribution-shift types (e.g., unseen categories, medical domain drift) beyond the four corruptions tested.","A strong test of the mechanism would be to vary shift severity; if FiCo's gains scale with corruption intensity, that would confirm misalignment, not augmentation, is the active factor.","The reliance on $C_1(f^{D_1})$ as a pure distribution-specific prior could be relaxed by learning the prior from multiple augmentations, potentially removing the paper's weakest assumption while retaining the gains."],"forward_implications":["On MVTec, PACS, and CIFAR-10 with brightness, contrast, defocus blur, and gaussian-noise corruptions, FiCo reports higher sample-level AUROC than all compared methods.","FiCo improves in-distribution performance relative to RD-based methods, indicating the compensation helps even without distribution shift.","Anomaly maps and score distributions show that FiCo concentrates activation on true anomalous regions while suppressing distribution-specific regions.","The added modules are trainable end-to-end and add modest training and inference time compared with GNL."],"supporting_citations":[{"why":"The GNL baseline that FiCo extends; supplies the distribution-invariant normality learning framework and the evaluation protocol.","marker":"(Cao, Zhu, and Pang 2023)"},{"why":"Reverse Distillation, the teacher–student framework in which the misalignment issue arises.","marker":"(Deng and Li 2022)"},{"why":"EFDM test-time augmentation used during inference, retained from GNL's protocol.","marker":"(Zhang et al. 2022)"},{"why":"Dynamic Convolution, the core building block of the DiSCo module.","marker":"(Chen et al. 2020)"},{"why":"MVTec AD, one of the three benchmarks on which FiCo's OOD performance is evaluated.","marker":"(Bergmann et al. 2019)"},{"why":"PACS, the domain-generalization benchmark used to test FiCo on real-world domain shifts.","marker":"(Li et al. 2017)"},{"why":"CIFAR-10, the benchmark for one-class novelty detection under corruptions.","marker":"(Krizhevsky, Hinton et al. 2009)"}],"fun_headline_variants":["FiCo: compensate then filter for anomaly detection under shift","Don't just filter, compensate: FiCo's fix for shifted AD","FiCo: compensate distribution-specific info, filter the rest","Beating shift: FiCo compensates then filters anomalies","Two-step fix: compensate distribution shift, filter anomalies"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper assumes that each student feature map splits cleanly into a distribution-invariant part and a distribution-specific part, and that the final DiSCo output $C_1(f^{D_1})$ captures only the latter, so it can be used to supervise earlier blocks; if $C_1$ also encodes semantic or anomaly-relevant content, the compensation–filter mechanism is mis-specified and the gains may not transfer.","fun_headline_variants_meta":{"raw":{"variants":["FiCo: compensate then filter for anomaly detection under shift","Don't just filter, compensate: FiCo's fix for shifted AD","FiCo: compensate distribution-specific info, filter the rest","Beating shift: FiCo compensates then filters anomalies","Two-step fix: compensate distribution shift, filter anomalies"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000542,"raw_usage":{"total_tokens":2571,"prompt_tokens":891,"completion_tokens":1680,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":507,"completion_tokens_details":{"reasoning_tokens":1597}},"tokens_in":507,"tokens_out":1680,"duration_ms":12114,"temperature":1.0,"reasoning_tokens":1597,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T16:19:16.034093+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train a linear classifier on the feature maps $C_1(f^{D_1})$ produced by FiCo to predict either the object class or the corruption type; if class prediction accuracy is far above chance, $C_1$ encodes semantic content, contradicting the assumption that it holds only distribution-specific information, and the DiIFi supervision via Eqs. (7)–(8) is mis-specified.","supporting_citations":[{"cited_title":"2009) to ex- pound the results presented in the main paper","cited_arxiv_id":null,"evidence_quote":"CIFAR-10, the benchmark for one-class novelty detection under corruptions."}],"review_version":1}