Pith. sign in

REVIEW 5 major objections 6 minor 38 references

Feature Bank Enhancement for Distance-based Out-of-Distribution Detection

T0 review · 5 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Feature Bank Enhancement (FBE) is a post-hoc method that clips extreme training features to percentile boundaries and reports state-of-the-art OOD detection on ImageNet-1k and CIFAR-10.

desk verdict A simple, plausible feature-bank clipping trick that likely helps, but the paper hides its lambda tuning and its theory section is mathematically broken. read the letter →

arxiv 2507.14178 v1 pith:IROXMRDN submitted 2025-07-11 cs.LG cs.AI

Yuhang Liu , Yuefei Wu , Bin Shi , Bo Dong This is my paper
classification cs.LGcs.AI
keywords out-of-distributiondetectionfeaturebankdistance-basedmethodsextremefeaturespercentileclippingpost-hocOODnear-OODImageNetbenchmark
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Feature Bank Enhancement (FBE) is a post-hoc method that repairs distance-based out-of-distribution detection by editing the stored training features before scoring. The paper argues that a small fraction of training features sit far from the main cluster and break the assumption that ID samples are always closer to training data than OOD samples; FBE identifies these extreme features per dimension and clips them to the λ-percentile boundary of the feature bank. On ImageNet-1k the method reports state-of-the-art average FPR95 of 25.29 and AUROC of 93.73, and on CIFAR-10 an FPR95 of 8.72 with AUROC of 98.27, improving near-OOD detection while preserving far-OOD performance. The appeal is that it works on any pretrained model without retraining or architecture changes, so more reliable OOD warnings could be added to already-deployed systems.

What carries the argument

The central object is the per-dimension λ-percentile boundary of the feature bank. For each feature dimension j, FBE computes the mean μ_j and the λ-th percentile d*_j of the absolute deviations |z_ij − μ_j|; any stored feature coordinate outside [μ_j − d*_j, μ_j + d*_j] is clipped to the nearer boundary. This produces a refined feature bank Z*_n that replaces the original bank when computing distance-based scores, shrinking the influence of extreme training features on the distance geometry that decides ID versus OOD.

What would settle it

Construct a synthetic feature bank in which extreme features are outliers only along a correlated direction (e.g., a diagonal in 2D), so no single coordinate is extreme, and measure whether FBE's per-dimension clipping improves or degrades a distance-based OOD score on that data. If per-dimension clipping fails to repair the synthetic outliers or reduces ID/OOD separation, the marginal-deviation premise is false.

Watch

Extended reading notes

Core claim

The paper establishes that constraining extreme training features to percentile-derived separation boundaries in the feature bank improves the OOD detection capability of distance-based score functions. Extreme training features—those far from the feature-bank mean in individual dimensions—give near-OOD samples artificially high similarity, because distances from those extremes to near-OOD features can be smaller than to ID features. FBE computes per-dimension thresholds d*_j as the λ-th percentile of absolute deviations from the mean, then rectifies every stored feature so that each coordinate lies within [μ_j − d*_j, μ_j + d*_j]. The refined feature bank makes ID samples receive higher scores and near-OOD samples lower scores; combined with NNGuide the method reports the best average FPR95 and AUROC on both ImageNet-1k and CIFAR-10 among all compared score functions, and it further improves ReAct-based truncation.

Load-bearing premise

The load-bearing premise is that harmful extreme training features are exactly those that deviate far from the mean in individual dimensions, and that clipping each dimension on its own keeps the distance geometry that separates ID from OOD intact.

Editorial extensions

If this is right

  • Any distance-based OOD score function (KNN, ViM, NNGuide, SSD) can be plugged into FBE and typically improves, with KNN showing the largest gain in FPR95 on ImageNet-1k.
  • FBE is compatible with network truncation; ReAct + FBE reports the best average FPR95 (17.89) and AUROC (95.83) among truncation baselines on four OOD datasets.
  • On the near-OOD iNaturalist subset, FBE improves FPR95 from 12.19 (NNGuide) to 8.72, a 3.47-point gain over the second-best baseline, while far-OOD Textures performance is roughly maintained.
  • The improvement comes at negligible runtime cost: the FBE operation adds about 3 to 6 seconds to the total processing time over five OOD datasets.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because FBE clips each dimension independently, it implicitly assumes harmful extremes are marginal outliers; if extreme training features are harmful mainly through joint correlations, a direction-aware clipping (e.g., along principal components or using Mahalanobis distance) might preserve more useful geometry, which the paper does not test.
  • The method's sensitivity to λ suggests a natural extension: instead of a global percentile per dimension, λ could be tuned per class or per feature channel, or chosen using a validation OOD set, potentially improving the reported numbers further.
  • The same percentile-clipping idea could transfer to other feature-bank settings, such as retrieval or few-shot learning, wherever stored reference features are used for nearest-neighbour scoring.
  • The theoretical argument assumes OOD features follow an epsilon-skew-normal distribution with larger variance than ID; testing that distributional assumption on real feature banks would show whether the Monte Carlo result transfers beyond the assumed family.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper proposes Feature Bank Enhancement (FBE), a post-hoc transformation that clips per-dimension deviations from the feature-bank mean to percentile-based boundaries (Eq. 7), and combines the modified feature bank with distance-based OOD score functions. The authors report state-of-the-art average FPR95/AUROC on ImageNet-1k and CIFAR-10 benchmarks (Tables 1-2), compatibility with ReAct (Table 3), runtime overhead (Table 4), and a Monte Carlo analysis in Section 6.1 intended to show that FBE increases the probability that an ID sample receives a higher score than an OOD sample.

Significance. If the results hold, FBE provides a simple, training-free enhancement for distance-based OOD detectors, with modest computational cost and broad applicability to KNN, ViM, NNGuide, and SSD, as well as to network-truncation methods. The empirical study covers standard large-scale and small-scale benchmarks and includes a useful efficiency analysis. However, the theoretical justification is invalid as written and the main empirical results omit the chosen hyperparameter and error bars, so the state-of-the-art claim is currently supported only by a single configuration that is not fully disclosed.

major comments (5)
  1. [Section 6.1, Eq. (11)] Both branches of the epsilon-skew-normal density are written with the same denominator σ_out(1+ε), so the skewness parameter ε cancels and the distribution reduces to a Gaussian; the claimed modeling of positively skewed OOD features is not realized. This invalidates the theoretical setup used to derive Formula 10.
  2. [Section 6.1 vs Section 4 / Algorithm 1] Section 6.1 misapplies the FBE definition from Section 4: Eq. (6) defines λ as a percentile in [0,100] and d* as the λ-th percentile of absolute deviations, while Algorithm 1 clips at µ_j ± d*_j. In Section 6.1, however, λ is set to 1.96 and interpreted as a z-score threshold, and the rectified training feature is modeled as clipping N(µ,σ_in²) at µ±λ. Since λ=1.96 as a percentile would clip almost all the mass rather than 5%, and since the simulation uses a single training feature rather than the feature bank with empirically estimated d*, Figure 4 does not simulate Algorithm 1 and does not establish Formula 10 for FBE.
  3. [Figure 4 caption] The caption states that 'The positive value correspond to Pr{d*in<d*out} < Pr{din<dout}', which is the opposite of the inequality in Formula 10. The text says the whole surface is above z=0 and concludes Formula 10 is valid; the caption therefore contradicts the paper's own claim and indicates a substantive error in the statement of the result.
  4. [Tables 1-2 / Section 6.2] The value of λ used for each benchmark is never reported, and Section 6.2 only shows qualitative sensitivity without a selection protocol. Since the reported state-of-the-art averages depend on this hyperparameter, the omission prevents reproducibility and leaves open the possibility that λ was chosen using OOD performance. Please report the chosen λ per benchmark and describe the selection rule (e.g., using a validation split of ID data only).
  5. [Tables 1-3] The results are reported without error bars or multiple-seed runs. Some FBE gains are small (e.g., Table 2 average FPR95 8.72 vs 9.63 for NNGuide), so without variance estimates the improvement claims are not fully supported. Please provide mean±std over at least three runs for the main tables.
minor comments (6)
  1. [Abstract] 'therapy making' should be 'thereby making'.
  2. [Section 4] The sentence following Eq. (5), beginning with 'di By performing this operation', is grammatically broken and should be rewritten.
  3. [Section 6.1] The sentence 'Substituting the expressions of din, dout, d*in, d*out into Formula 10, we can obtain the specific calculation formulas' is vague; no such formulas are presented.
  4. [Section 5.4] 'We re-ported' should be 'We reported'.
  5. [Conclusion] 'supplementary experiments are to' should be 'supplementary experiments are conducted to'.
  6. [General] The paper does not mention whether code will be released; providing code would improve reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the proposed FBE operation and its empirical evaluation do not reduce to the fitted parameters, and no self-citation chain is load-bearing.

full rationale

I found no circular step in the derivation chain. FBE (Algorithm 1, Eqs. 5-7) is defined independently of the reported metrics: extreme training features are clipped to per-dimension percentile boundaries d* computed from the feature bank, and the improved score function in Eq. (8) is just the base distance score evaluated on the modified bank. The empirical results in Tables 1-3 are compared against external baselines on standard OOD benchmarks, so the improvements are not logically contained in the definition of the method. The theoretical Section 6.1 contains serious internal inconsistencies: lambda is set to 1.96 and interpreted as a z-score threshold in the Monte Carlo simulation, although Eq. (6) defines lambda as a percentile in [0,100], and Eq. (11) gives the same denominator on both branches so the skewness parameter epsilon has no effect. These are correctness and validity flaws in the supporting theory, not circularity, because the simulation does not assume the inequality it claims to demonstrate. The only external authority invoked, ReAct [26], is cited as a modeling framework rather than as a self-citation, and no uniqueness theorem or prior result by these authors is used to force the choice of FBE. The paper's own limitation note about SSD (Section 6.4) and the unreported lambda values are reproducibility concerns, not evidence that a prediction reduces by construction to a fit.

Assumptions & free parameters 1 free parameters · 3 assumptions · 0 invented entities

The central claim depends on one fitted hyperparameter (λ) and on distributional assumptions in the theoretical analysis that are not validated against real features. No new entities are introduced.

free parameters (1)
  • λ (retention rate percentile) = not reported; tuned per benchmark
    Controls the strictness of clipping in Eq. (6); Figure 5 shows performance peaks for intermediate λ, and no default or selection procedure is given in Section 5 or 6.2.
assumptions (3)
  • domain assumption ID features follow a Gaussian distribution N(μ, σ_in^2) and OOD features follow an epsilon-skew-normal distribution with shared mean μ and σ_out > σ_in.
    Used in Section 6.1 to justify the theoretical improvement; not empirically validated on the actual feature banks used in the experiments.
  • domain assumption Per-dimension marginal percentiles around the mean define a meaningful boundary between typical and extreme training features.
    Central to Eq. (5)-(7); assumes each dimension can be treated independently, ignoring correlations between feature dimensions.
  • domain assumption The distance-based OOD assumption holds: OOD features are relatively distant from training features, and extreme training features specifically distort this relationship for near-OOD samples.
    Motivates the whole method; asserted in the introduction and illustrated in Figure 2, but not rigorously established as a causal mechanism.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Feature Bank Enhancement for Distance-based Out-of-Distribution Detection." pith.science (2026). https://pith.science/paper/IROXMRDN

@misc{pith2026250714178,
  author       = {Pith},
  title        = {Pith review of: Feature Bank Enhancement for Distance-based Out-of-Distribution Detection},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IROXMRDN}},
  note         = {Machine review of arXiv:2507.14178}
}
read the original abstract

Out-of-distribution (OOD) detection is critical to ensuring the reliability of deep learning applications and has attracted significant attention in recent years. A rich body of literature has emerged to develop efficient score functions that assign high scores to in-distribution (ID) samples and low scores to OOD samples, thereby helping distinguish OOD samples. Among these methods, distance-based score functions are widely used because of their efficiency and ease of use. However, deep learning often leads to a biased distribution of data features, and extreme features are inevitable. These extreme features make the distance-based methods tend to assign too low scores to ID samples. This limits the OOD detection capabilities of such methods. To address this issue, we propose a simple yet effective method, Feature Bank Enhancement (FBE), that uses statistical characteristics from dataset to identify and constrain extreme features to the separation boundaries, therapy making the distance between samples inside and outside the distribution farther. We conducted experiments on large-scale ImageNet-1k and CIFAR-10 respectively, and the results show that our method achieves state-of-the-art performance on both benchmark. Additionally, theoretical analysis and supplementary experiments are conducted to provide more insights into our method.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

38 extracted references · 26 canonical work pages

  1. [1]

    Cimpoi, S

    M. Cimpoi, S. Maji, I. Kokkinos, S. Mohamed, and A. Vedaldi. De- scribing textures in the wild. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3606–3613, 2014

    work page 2014

  2. [2]

    J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee, 2009

    2009

  3. [3]

    Djurisic, N

    A. Djurisic, N. Bozanic, A. Ashok, and R. Liu. Extremely simple activation shaping for out-of-distribution detection. arXiv preprint arXiv:2209.09858, 2022

    arXiv 2022

  4. [4]

    Drummond and R

    N. Drummond and R. Shearer. The open world assumption. In eSI Workshop: The Closed World of Databases meets the Open World of the Semantic Web, volume 15, page 1, 2006

    work page 2006

  5. [5]

    B. He, L. Noci, D. Paliotta, I. Schlag, and T. Hofmann. Understand- ing and minimising outlier features in neural network training. arXiv preprint arXiv:2405.19279, 2024

    work page Pith review arXiv 2024

  6. [6]

    K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016

    2016

  7. [7]

    R. He, Y . Yuan, Z. Han, F. Wang, W. Su, Y . Yin, T. Liu, and Y . Gong. Exploring channel-aware typical features for out-of-distribution detec- tion. In Proceedings of the AAAI conference on artificial intelligence , volume 38, pages 12402–12410, 2024

    work page 2024

  8. [8]

    Hendrycks and K

    D. Hendrycks and K. Gimpel. A baseline for detecting misclassified and out-of-distribution examples in neural networks. arXiv preprint arXiv:1610.02136, 2016

    arXiv 2016

Show all 38 references
  1. [9]

    Hendrycks, S

    D. Hendrycks, S. Basart, M. Mazeika, A. Zou, J. Kwon, M. Mostajabi, J. Steinhardt, and D. Song. Scaling out-of-distribution detection for real-world settings. arXiv preprint arXiv:1911.11132, 2019

    1911 arXiv

  2. [10]

    Y .-C. Hsu, Y . Shen, H. Jin, and Z. Kira. Generalized odin: Detect- ing out-of-distribution image without learning from out-of-distribution data. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10951–10960, 2020

    2020

  3. [11]

    Huang, A

    R. Huang, A. Geng, and Y . Li. On the importance of gradients for de- tecting distributional shifts in the wild. Advances in Neural Information Processing Systems, 34:677–689, 2021

    2021

  4. [12]

    Huang, D

    X. Huang, D. Kroening, W. Ruan, J. Sharp, Y . Sun, E. Thamo, M. Wu, and X. Yi. A survey of safety and trustworthiness of deep neural net- works: Verification, testing, adversarial attack and defence, and inter- pretability. Computer Science Review, 37:100270, 2020

    2020

  5. [13]

    Krizhevsky, G

    A. Krizhevsky, G. Hinton, et al. Learning multiple layers of features from tiny images. 2009

    2009

  6. [14]

    K. Lee, K. Lee, H. Lee, and J. Shin. A simple unified framework for detecting out-of-distribution samples and adversarial attacks. Advances in neural information processing systems, 31, 2018

    2018

  7. [15]

    Liang, Y

    S. Liang, Y . Li, and R. Srikant. Enhancing the reliability of out- of-distribution image detection in neural networks. arXiv preprint arXiv:1706.02690, 2017

    2017 arXiv

  8. [16]

    Litjens, T

    G. Litjens, T. Kooi, B. E. Bejnordi, A. A. A. Setio, F. Ciompi, M. Ghafoorian, J. A. Van Der Laak, B. Van Ginneken, and C. I. Sánchez. A survey on deep learning in medical image analysis. Medical image analysis, 42:60–88, 2017

    2017

  9. [17]

    W. Liu, X. Wang, J. Owens, and Y . Li. Energy-based out-of-distribution detection. Advances in neural information processing systems , 33: 21464–21475, 2020

    2020

  10. [18]

    Metropolis and S

    N. Metropolis and S. Ulam. The monte carlo method. Journal of the American statistical association, 44(247):335–341, 1949

    1949

  11. [19]

    G. S. Mudholkar and A. D. Hutson. The epsilon–skew–normal distri- bution for analyzing near-normal data. Journal of statistical planning and inference, 83(2):291–309, 2000

    2000

  12. [20]

    Netzer, T

    Y . Netzer, T. Wang, A. Coates, A. Bissacco, B. Wu, A. Y . Ng, et al. Reading digits in natural images with unsupervised feature learning. In NIPS workshop on deep learning and unsupervised feature learning , volume 2011, page 4. Granada, 2011

    2011

  13. [21]

    Nguyen, J

    A. Nguyen, J. Yosinski, and J. Clune. Deep neural networks are eas- ily fooled: High confidence predictions for unrecognizable images. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 427–436, 2015

    2015

  14. [22]

    J. Park, Y . G. Jung, and A. B. J. Teoh. Nearest neighbor guidance for out-of-distribution detection. In Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision, pages 1686–1695, 2023

    2023

  15. [23]

    Sehwag, M

    V . Sehwag, M. Chiang, and P. Mittal. Ssd: A unified framework for self- supervised outlier detection. arXiv preprint arXiv:2103.12051, 2021

    2021 arXiv

  16. [24]

    Y . Song, N. Sebe, and W. Wang. Rankfeat: Rank-1 feature removal for out-of-distribution detection. Advances in Neural Information Process- ing Systems, 35:17885–17898, 2022

    2022

  17. [25]

    Sun and Y

    Y . Sun and Y . Li. Dice: Leveraging sparsification for out-of-distribution detection. In European Conference on Computer Vision , pages 691–

  18. [26]

    Y . Sun, C. Guo, and Y . Li. React: Out-of-distribution detection with rec- tified activations. Advances in Neural Information Processing Systems, 34:144–157, 2021

    2021

  19. [27]

    Y . Sun, Y . Ming, X. Zhu, and Y . Li. Out-of-distribution detection with deep nearest neighbors. InInternational Conference on Machine Learn- ing, pages 20827–20840. PMLR, 2022

    2022

  20. [28]

    Van der Maaten and G

    L. Van der Maaten and G. Hinton. Visualizing data using t-sne. Journal of machine learning research, 9(11), 2008

    2008

  21. [29]

    Van Horn, O

    G. Van Horn, O. Mac Aodha, Y . Song, Y . Cui, C. Sun, A. Shepard, H. Adam, P. Perona, and S. Belongie. The inaturalist species classifi- cation and detection dataset. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 8769–8778, 2018

    2018

  22. [30]

    H. Wang, Z. Li, L. Feng, and W. Zhang. Vim: Out-of-distribution with virtual-logit matching. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4921–4930, 2022

    2022

  23. [31]

    J. Xiao, J. Hays, K. A. Ehinger, A. Oliva, and A. Torralba. Sun database: Large-scale scene recognition from abbey to zoo. In 2010 IEEE com- puter society conference on computer vision and pattern recognition , pages 3485–3492. IEEE, 2010

    2010

  24. [32]

    P. Xu, K. A. Ehinger, Y . Zhang, A. Finkelstein, S. R. Kulkarni, and J. Xiao. Turkergaze: Crowdsourcing saliency with webcam based eye tracking. arXiv preprint arXiv:1504.06755, 2015

    2015 arXiv

  25. [33]

    J. Yang, K. Zhou, Y . Li, and Z. Liu. Generalized out-of-distribution detection: A survey. arXiv preprint arXiv:2110.11334, 2021

    2021 arXiv

  26. [34]

    J. Yang, P. Wang, D. Zou, Z. Zhou, K. Ding, W. Peng, H. Wang, G. Chen, B. Li, Y . Sun, et al. Openood: Benchmarking generalized out- of-distribution detection. Advances in Neural Information Processing Systems, 35:32598–32611, 2022

    2022

  27. [35]

    F. Yu, A. Seff, Y . Zhang, S. Song, T. Funkhouser, and J. Xiao. Lsun: Construction of a large-scale image dataset using deep learning with humans in the loop. arXiv preprint arXiv:1506.03365, 2015

    2015 arXiv

  28. [36]

    Zhang and X

    Z. Zhang and X. Xiang. Decoupling maxlogit for out-of-distribution detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3388–3397, 2023

    2023

  29. [37]

    B. Zhou, A. Lapedriza, A. Khosla, A. Oliva, and A. Torralba. Places: A 10 million image database for scene recognition. IEEE transactions on pattern analysis and machine intelligence, 40(6):1452–1464, 2017

    2017

  30. [38]

    Y . Zhu, Y . Chen, C. Xie, X. Li, R. Zhang, H. Xue, X. Tian, Y . Chen, et al. Boosting out-of-distribution detection with typical features. Advances in Neural Information Processing Systems, 35:20758–20769, 2022

    2022

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.