Pith. sign in

REVIEW 2 major objections 6 minor 93 references

Kernel PCA for Out-of-Distribution Detection: Non-Linear Kernel Selection and Approximation

T0 review · 2 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Kernel PCA with a Cosine-Gaussian kernel detects out-of-distribution inputs by how poorly they reconstruct in a learned non-linear feature subspace, at inference cost independent of training-set size.

desk verdict A useful, well-engineered extension of the authors' NeurIPS KPCA-OOD work, but the 'new SOTA' claim is not yet supported by a clean evaluation protocol. read the letter →

arxiv 2505.15284 v2 pith:7YUEFTFX submitted 2025-05-21 cs.LG cs.CV

Kun Fang , Qinghua Tao , Mingzhen He , Kexin Lv , Runze Yang , Haibo Hu , Xiaolin Huang , Jie Yang
show 1 more author
Longbing Cao
This is my paper · ORCID
classification cs.LGcs.CV
keywords out-of-distributiondetectionkernelPCACosine-GaussianrandomFourierfeaturesNyströmapproximationreconstructionerrorfeaturesubspaceenergy-basedsampling
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that out-of-distribution (OoD) detection can be recast as a non-linear subspace problem: in-distribution (InD) features from a trained network are assumed to concentrate in a low-dimensional non-linear subspace, and OoD features are assumed to fall outside it. Using Kernel PCA, the paper constructs that subspace from InD features and scores new samples by reconstruction error. To make the subspace discriminative, it derives a Cosine-Gaussian kernel: cosine normalization removes the harmful norm imbalance between InD and OoD features, and the Gaussian term preserves the $\ell^2$ distance structure that separates them. To make KPCA feasible at ImageNet scale, it approximates the kernel with random Fourier features and a data-dependent Nyström mapping whose landmarks are low-confidence InD samples. If correct, this gives a detection method that matches or beats existing feature, logit, and gradient baselines while keeping inference cost independent of training-set size.

What carries the argument

The load-bearing object is the Cosine-Gaussian kernel $k(z_1,z_2)=k_{\mathrm{gau}}(\phi_{\cos}(z_1),\phi_{\cos}(z_2))$, where $\phi_{\cos}(z)=z/\|z\|_2$ is the exact cosine map and $k_{\mathrm{gau}}$ is the Gaussian kernel $e^{-\gamma\|z_1-z_2\|_2^2}$. This composite kernel is the mechanism: it first projects features onto the unit sphere, neutralizing the norm gap that breaks PCA's centering step, then measures $\ell^2$ proximity in that normalized space, capturing the distance relation that separates InD and OoD. The detection score is the KPCA reconstruction error in the mapped space; the paper approximates the infinite-dimensional Gaussian part by explicit mappings $\Phi=\phi_{\mathrm{rff}}\circ\phi_{\cos}$ or $\Phi=\phi_{\mathrm{nys}}\circ\phi_{\cos}$, with the Nyström landmarks chosen by lowest energy $E(x;f)=T\log\sum_i e^{f_i(x)/T}$, so the principal components anchor near the InD-OoD boundary.

What would settle it

Take a trained ImageNet-1K model, build the KPCAnys subspace from InD features, and generate OoD test samples by adding perturbations to InD features that stay within the learned non-linear principal subspace, for example by moving along the top eigenvectors in the mapped space. If the reconstruction-error AUROC on such in-manifold OoD samples stays near 50%, the core premise that OoD features fall outside the InD subspace is falsified; if it remains high, the method's sensitivity to distribution shift is confirmed.

Watch

Extended reading notes

Core claim

The central claim is that a non-linear feature subspace learned by KPCA, with a composite Cosine-Gaussian kernel, yields reconstruction errors that separate InD from OoD features more sharply than linear PCA or distance-based scores. The paper identifies two non-linear patterns behind the disparity: imbalanced feature norms, which cosine normalization corrects before centering; and a beneficial $\ell^2$-distance relation in the normalized space, which the Gaussian kernel preserves. Approximating the Gaussian part with random Fourier features or, better, a Nyström mapping built from low-energy InD support landmarks keeps the computation cheap and even sharpens the subspace around the InD-OoD boundary. On ImageNet-1K with ResNet50 and ViT, the method reports state-of-the-art FPR and AUROC, including when combined with feature rectification.

Load-bearing premise

The method assumes in-distribution features intrinsically lie in a low-dimensional non-linear subspace that out-of-distribution features do not share; if OoD features also sit close to that learned subspace, reconstruction errors will not separate the two distributions.

Editorial extensions

If this is right

  • OoD detection can be performed without storing or scanning the full training set: inference costs $O(M_r)$ or $O(M_n)$ with $M_r=4096$ and $M_n=2048$, versus $O(N_{tr})$ for nearest-neighbour methods, and memory drops from roughly 20 GiB to under 100 MiB on ImageNet-1K.
  • The same two-pattern kernel recipe, normalize norms then preserve $\ell^2$ distance, is shown to beat alternative kernels, since Laplacian, polynomial, and plain Gaussian kernels all hurt detection without the cosine step.
  • Data-dependent kernel approximation can outperform data-independent approximation for detection: energy-selected landmarks beat uniform and high-energy landmarks, and 1024 low-energy landmarks outperform 4096 random Fourier features.
  • KPCA reconstruction errors compose with feature rectification, so the subspace score can be stacked with methods like ReAct to improve detection further.
  • An ensemble of the random-Fourier and Nyström variants reaches the best reported AUROC and FPR on the ViT-B/16 benchmark among the compared methods.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The energy-based landmark selection is effectively a cheap margin-aware subspace learner; one could test whether other uncertainty proxies, such as predictive entropy or gradient norms, sharpen the InD-OoD boundary even further.
  • Because the kernel is built from generic feature geometry rather than image-specific structure, the same Cosine-Gaussian KPCA recipe may transfer to anomaly detection or open-set recognition on non-image modalities where norm imbalance and $\ell^2$ distance patterns appear.
  • A stress test the paper does not run is to draw OoD samples from the same coarse categories as InD or from a second fine-tuned model; those are the cases where the premise that OoD features lie outside the InD subspace is least favorable.
  • The method can be read as a kernelized version of the nearest-neighbour baseline, with the gains coming from replacing hard nearest-neighbour distances by subspace projection; this interpretation predicts that kernel bandwidth and subspace dimension interact the way the sensitivity analysis shows.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 6 minor

Summary. The manuscript proposes an out-of-distribution (OoD) detection method based on Kernel PCA with a composite Cosine-Gaussian kernel. The kernel is motivated by two empirically observed patterns in deep features: an adverse imbalance of feature norms that is mitigated by cosine normalization, and a beneficial l2-distance relation that is captured by a Gaussian kernel on the normalized features. To scale KPCA to large training sets such as ImageNet-1K, the authors approximate the Gaussian part with random Fourier features or a Nyström map, and for the Nyström approximation they select low-energy InD training samples as landmarks. The detection score is the KPCA reconstruction error in the approximate feature space. Experiments on ImageNet-1K with ResNet50, MobileNetV2, and ViT-B/16 report improved detection accuracy and lower inference cost compared with KNN and other baselines.

Significance. If the reported results hold, the paper makes a practical contribution: an OoD detector with O(M_r) or O(M_n) inference cost, a data-dependent landmark selection scheme, and a clear algorithmic pipeline with publicly released code. The complexity analysis and the kernel ablations are useful empirical additions. However, the central SOTA claim is not yet supported by the evaluation protocol as reported, because model selection appears to use the same OoD test sets on which performance is measured and because no variance over runs is reported. In addition, Proposition 1, which underpins the comparison between exact and approximate KPCA, appears incorrect as stated. These issues are locally fixable, but they require re-running or re-reporting experiments under a cleaner protocol before the main claim can be taken at face value.

major comments (2)
  1. [§5.1, §5.6, Tables 1–3] The evaluation protocol does not separate model selection from evaluation. Section 5.1 describes no validation split, and Section 5.6 reports sensitivity of the Gaussian width gamma, the explained-variance ratio (q), M_r, and M_n averaged over the same four OoD test sets (iNaturalist, SUN, Places, Textures) used in Tables 1–3. Since Section 7 concedes that 'careful parameter tuning' is required, and the sensitivity curves in Figures 8–10 are steep (e.g., KPCArff FPR varies by roughly 30 points across the tested gamma range in Figure 9), selecting a favorable configuration on the evaluation OoD sets could manufacture the reported margins. The final hyperparameter values for gamma and the explained-variance ratio are not stated for Tables 1–3, and all tables report a single run even though KPCArff depends on random omega,u draws. Please report the exact model-selection protocol, add a validation split that does not use OoD labels, and provide means and standard deviations over multiple runs.
  2. [§4.4, Proposition 1, Eq. (10), Appendix A] Proposition 1 as stated is not a correct formula for the exact KPCA reconstruction error. In centered KPCA, the projection coordinate onto principal component j is proportional to (1/sqrt(lambda_j)) U_j^T k_c(z), where k_c is the centered kernel vector, and the residual squared error after keeping q components is ||Phi_c(z)||^2 minus the sum of the retained projection energies, i.e., sum_{j=q+1}^N (U_j^T k_c(z))^2 / lambda_j (up to the centering terms). Eq. (10) omits the eigenvalue normalization, the kernel centering, and the norm of the centered test point. The appendix proof only re-derives the explicit-feature-space identity in Eq. (13) and does not bridge to the kernel-space statement in Proposition 1. Because Figure 6 is computed with this formula, the claimed approximation performance of RFF and Nyström relative to exact KPCA is not established by the current derivation.
minor comments (6)
  1. [Table 2] Several entries appear to have missing separators, such as '53.8287.25' and '36.9791.31', which should be corrected to separate FPR and AUROC values.
  2. [Appendix B.1] 'KPCnys' is a typo for 'KPCAnys' in the first paragraph of Appendix B.1.
  3. [§5.2.3] The phrase 'the learned subspace well seizes the InD-OoD disparities' is awkward; 'captures' or 'characterizes' would be clearer.
  4. [§5.6, Figures 8–10] For each sensitivity curve, the values of the fixed hyperparameters are not reported; please state them so that the reader can reproduce the figures.
  5. [Figure 6] The comparison between exact and approximate KPCA is shown without error bars or repeated-seed details, so the claim of a 'marginal gap' is not quantitatively supported.
  6. [Algorithm 1] The notation '\tildeΛ- 1 2' in line 10 is hard to read; please typeset it as \tilde{\Lambda}^{-1/2}.

Circularity Check

1 steps flagged · score 6.0 of 10

Reported SOTA is selected on the same OoD test sets; the kernel derivation itself is not circular.

  1. fitted input called prediction [Section 5.1 (Setups), Section 5.6 (Sensitivity Analysis), Section 7 (Conclusion), Tables 1-4]
    "OoD datasets include iNaturalist, SUN, Places and Textures, which have been widely adopted in existing methods. ... A comprehensive sensitivity analysis is executed to illustrate the effects of hyper-parameters in KPCArff and KPCAnys on the OoD detection performance. ... All experiments are conducted on the ImageNet-1K benchmark and the PyTorch-released ResNet50 checkpoint."

    The paper's only support for "new SOTA detection results" is the ImageNet-1K tables. The same four OoD sets (iNaturalist, SUN, Places, Textures) are used both for those tables and for the sensitivity curves of gamma, explained-variance ratio (q), M_r, and M_n in Figures 8-10. No validation split or hyperparameter-selection procedure is described; Section 7 states only that the method "require[s] careful parameter tuning." If the reported configurations were chosen after inspecting FPR/AUROC on these OoD sets, then the headline results are selected on the evaluation data: the "prediction" is fitted, not independent. The steep curves in Figures 9-10 (FPR varying by tens of points over the tested gamma range) show that the selection can force the reported margin.

full rationale

The kernel derivation (Sections 3.1-3.2) is an inductive design from feature-norm imbalance and l2-distance relations; it is not an equation that reduces to the target metric. The approximation methods are standard RFF and Nyström, and the energy-based landmark selection uses logits, not reconstruction errors. Proposition 1 is cited from the authors' conference version [34] and proved in Appendix A, but it is used only for the auxiliary approximation analysis of Figure 6, not for the main SOTA claim; self-citation is therefore not load-bearing. The one significant circularity is the evaluation protocol: hyperparameters are analyzed on the same four OoD test sets used to build the comparison tables, with no reported validation split or selection rule. Under that protocol the "new SOTA" numbers are consistent with having been fit to the benchmark, which makes the headline empirical prediction partially circular. Score 6 rather than higher because the underlying mechanism has independent content and the circularity is in the selection of hyperparameters, not in the derivation of the kernel or the reconstruction error.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The method introduces no new physical or formal entities. The free parameters are standard hyperparameters, but several are tuned on the evaluation benchmarks and some values are not reported. The main burden is the domain assumption that InD features form a compact non-linear subspace and that low-energy landmarks are boundary-like support vectors.

free parameters (5)
  • Gaussian kernel width gamma = not reported in main text
    Selected by hand; sensitivity analysis in Figure 9 shows performance varies strongly with gamma, especially for KPCAnys.
  • Number of principal components q = selected via explained variance ratio
    q is chosen as the minimum number of components exceeding an explained variance threshold; the threshold is not specified and its effect is shown in Figure 8.
  • Number of random features Mr = 4096 in Table 4
    The number of RFFs controls approximation quality; sensitivity analysis in Figure 10 shows performance improves with Mr.
  • Number of Nyström landmarks Mn = 2048 in Table 4
    The number of landmarks controls low-rank approximation quality; sensitivity analysis in Figure 10 shows performance improves with Mn.
  • Temperature T in energy function = not reported
    Used in Eq. 8 to define the energy E(x; f); the value of T is not specified in the paper, adding an unstated tuning choice.
assumptions (5)
  • standard math Bochner's theorem for random Fourier features
    Used in Section 4.1 to sample random features from the Fourier transform of the Gaussian kernel.
  • domain assumption InD features reside in a low-dimensional non-linear subspace
    Stated in Section 1 and Section 3; if OoD features also fit the InD subspace, the reconstruction-error detector fails.
  • domain assumption Feature norm imbalance between InD and OoD is systematic
    Motivates the cosine kernel in Section 3.1; shown only on selected datasets, not derived.
  • domain assumption Low-energy InD samples approximate support vectors near the InD-OoD boundary
    Proposed in Section 4.2 to justify energy-based Nyström landmark selection; the claim is qualitative and not formally proven.
  • standard math Kernel trick and positive-definiteness of the Gaussian kernel
    Underpins KPCA and the approximation mappings throughout Sections 3 and 4.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Kernel PCA for Out-of-Distribution Detection: Non-Linear Kernel Selection and Approximation." pith.science (2026). https://pith.science/paper/7YUEFTFX

@misc{pith2026250515284,
  author       = {Pith},
  title        = {Pith review of: Kernel PCA for Out-of-Distribution Detection: Non-Linear Kernel Selection and Approximation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7YUEFTFX}},
  note         = {Machine review of arXiv:2505.15284}
}
read the original abstract

Out-of-Distribution (OoD) detection is vital for the reliability of deep neural networks, the key of which lies in effectively characterizing the disparities between OoD and In-Distribution (InD) data. In this work, such disparities are exploited through a fresh perspective of non-linear feature subspace. That is, a discriminative non-linear subspace is learned from InD features to capture representative patterns of InD, while informative patterns of OoD features cannot be well captured in such a subspace due to their different distribution. Grounded on this perspective, we exploit the deviations of InD and OoD features in such a non-linear subspace for effective OoD detection. To be specific, we leverage the framework of Kernel Principal Component Analysis (KPCA) to attain the discriminative non-linear subspace and deploy the reconstruction error on such subspace to distinguish InD and OoD data. Two challenges emerge: (i) the learning of an effective non-linear subspace, i.e., the selection of kernel function in KPCA, and (ii) the computation of the kernel matrix with large-scale InD data. For the former, we reveal two vital non-linear patterns that closely relate to the InD-OoD disparity, leading to the establishment of a Cosine-Gaussian kernel for constructing the subspace. For the latter, we introduce two techniques to approximate the Cosine-Gaussian kernel with significantly cheap computations. In particular, our approximation is further tailored by incorporating the InD data confidence, which is demonstrated to promote the learning of discriminative subspaces for OoD data. Our study presents new insights into the non-linear feature subspace for OoD detection and contributes practical explorations on the associated kernel design and efficient computations, yielding a KPCA detection method with distinctively improved efficacy and efficiency.

Figures

Figures reproduced from arXiv: 2505.15284 by the authors.

Figure 1
Figure 1. The t-SNE [6] visualization on the original features z (left) and the features Φ(z) in subspace (right). Our KPCA detection method alleviates the linear inseparability between InD and OoD features in the original z-space via the mapping Φ with substantially improved OoD detection performance, illustrated by the distinguishable reconstruction errors. computationally intensive backward propagation processes. More focu… view at source ↗
Figure 2
Figure 2. The framework of our KPCA detection method. A Cosine-Gaussian kernel is devised to model the [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Illustrations on the impact of the cosine kernel [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Low dimensional embeddings via MDS [39] on InD and OoD data in the z-space and ϕcos(z)-space, respectively. InD: CIFAR10 [36]. OoD: LSUN [37] and places365 [38]. 6 [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: An illustration on the comparisons between the naive uniform sampling and the proposed low-energy sampling, [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: Approximation performance of the reconstruction errors on InD (left) and OoD (right) data between exact [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]
Figure 7
Figure 7. Figure 7: Ablation studies on the sampling scheme for Nyström in [PITH_FULL_IMAGE:figures/full_fig_p015_7.png]
Figure 8
Figure 8. Figure 8: A sensitivity analysis on the explained variance ratio of [PITH_FULL_IMAGE:figures/full_fig_p016_8.png]
Figure 9
Figure 9. Figure 9: A sensitivity analysis on the Gaussian kernel width [PITH_FULL_IMAGE:figures/full_fig_p016_9.png]
Figure 10
Figure 10. Figure 10: A sensitivity analysis on the number of RFFs [PITH_FULL_IMAGE:figures/full_fig_p016_10.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

93 extracted references · 60 canonical work pages

  1. [1]

    Denoising diffusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020

    2020

  2. [2]

    Training language models to follow instructions with human feedback

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35:27730–27744, 2022

    2022

  3. [3]

    Trustworthy ai: A computational perspective

    Haochen Liu, Yiqi Wang, Wenqi Fan, Xiaorui Liu, Yaxin Li, Shaili Jain, Yunhao Liu, Anil Jain, and Jiliang Tang. Trustworthy ai: A computational perspective. ACM Transactions on Intelligent Systems and Technology, 14(1):1–59, 2022

    2022

  4. [4]

    Identifying and mitigating the security risks of generative ai

    Clark Barrett, Brad Boyd, Elie Bursztein, Nicholas Carlini, Brad Chen, Jihye Choi, Amrita Roy Chowdhury, Mihai Christodorescu, Anupam Datta, Soheil Feizi, et al. Identifying and mitigating the security risks of generative ai. Foundations and Trends® in Privacy and Security, 6(1):1–52, 2023

    2023

  5. [5]

    Generalized out-of-distribution detection: A survey

    Jingkang Yang, Kaiyang Zhou, Yixuan Li, and Ziwei Liu. Generalized out-of-distribution detection: A survey. International Journal of Computer Vision, 132(12):5635–5662, 2024

    2024

  6. [6]

    Visualizing data using t-sne

    Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research, 9(11), 2008

    2008

  7. [7]

    Energy-based out-of-distribution detection

    Weitang Liu, Xiaoyun Wang, John Owens, and Yixuan Li. Energy-based out-of-distribution detection. Advances in neural information processing systems, 33:21464–21475, 2020

    2020

  8. [8]

    Wood: Wasserstein-based out-of- distribution detection

    Yinan Wang, Wenbo Sun, Jionghua Jin, Zhenyu Kong, and Xiaowei Yue. Wood: Wasserstein-based out-of- distribution detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(2):944–956, 2023

    2023

Show all 93 references
  1. [9]

    On the importance of gradients for detecting distributional shifts in the wild

    Rui Huang, Andrew Geng, and Yixuan Li. On the importance of gradients for detecting distributional shifts in the wild. Advances in Neural Information Processing Systems, 34:677–689, 2021

    2021

  2. [10]

    Low-dimensional gradient helps out-of- distribution detection

    Yingwen Wu, Tao Li, Xinwen Cheng, Jie Yang, and Xiaolin Huang. Low-dimensional gradient helps out-of- distribution detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024

    2024

  3. [11]

    React: Out-of-distribution detection with rectified activations

    Yiyou Sun, Chuan Guo, and Yixuan Li. React: Out-of-distribution detection with rectified activations. Advances in Neural Information Processing Systems, 34:144–157, 2021

    2021

  4. [12]

    Out-of-distribution detection with deep nearest neighbors

    Yiyou Sun, Yifei Ming, Xiaojin Zhu, and Yixuan Li. Out-of-distribution detection with deep nearest neighbors. In International Conference on Machine Learning, pages 20827–20840. PMLR, 2022

    2022

  5. [13]

    Revisit pca-based technique for out-of-distribution detection

    Xiaoyuan Guan, Zhouwu Liu, Wei-Shi Zheng, Yuren Zhou, and Ruixuan Wang. Revisit pca-based technique for out-of-distribution detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 19431–19439, 2023

    2023

  6. [14]

    Boosting out-of-distribution detection with typical features

    Yao Zhu, YueFeng Chen, Chuanlong Xie, Xiaodan Li, Rong Zhang, Hui Xue, Xiang Tian, Yaowu Chen, et al. Boosting out-of-distribution detection with typical features. Advances in Neural Information Processing Systems, 35:20758–20769, 2022

    2022

  7. [15]

    Vra: variational rectified activation for out-of-distribution detection

    Mingyu Xu, Zheng Lian, Bin Liu, and Jianhua Tao. Vra: variational rectified activation for out-of-distribution detection. Advances in Neural Information Processing Systems, 36:28941–28959, 2023

    2023

  8. [16]

    Extremely simple activation shaping for out-of-distribution detection

    Andrija Djurisic, Nebojsa Bozanic, Arjun Ashok, and Rosanne Liu. Extremely simple activation shaping for out-of-distribution detection. In The Eleventh International Conference on Learning Representations, 2023

    2023

  9. [17]

    Scaling for training time and post-hoc out-of-distribution detection enhancement

    Kai Xu, Rongyu Chen, Gianni Franchi, and Angela Yao. Scaling for training time and post-hoc out-of-distribution detection enhancement. In The Twelfth International Conference on Learning Representations, 2024. 18

    2024

  10. [18]

    Discriminability-driven channel selection for out-of-distribution detection

    Yue Yuan, Rundong He, Yicong Dong, Zhongyi Han, and Yilong Yin. Discriminability-driven channel selection for out-of-distribution detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 26171–26180, 2024

    2024

  11. [19]

    Line: Out-of-distribution detection by leveraging important neurons

    Yong Hyun Ahn, Gyeong-Moon Park, and Seong Tae Kim. Line: Out-of-distribution detection by leveraging important neurons. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 19852–19862. IEEE, 2023

    2023

  12. [20]

    Rankfeat: Rank-1 feature removal for out-of-distribution detection

    Yue Song, Nicu Sebe, and Wei Wang. Rankfeat: Rank-1 feature removal for out-of-distribution detection. Advances in Neural Information Processing Systems, 35:17885–17898, 2022

    2022

  13. [21]

    A simple unified framework for detecting out-of- distribution samples and adversarial attacks

    Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. A simple unified framework for detecting out-of- distribution samples and adversarial attacks. Advances in neural information processing systems, 31, 2018

    2018

  14. [22]

    Nearest neighbor guidance for out-of-distribution detection

    Jaewoo Park, Yoon Gyo Jung, and Andrew Beng Jin Teoh. Nearest neighbor guidance for out-of-distribution detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 1686–1695, 2023

    2023

  15. [23]

    Out-of- distribution detection based on in-distribution data patterns memorization with modern hopfield energy

    Jinsong Zhang, Qiang Fu, Xu Chen, Lun Du, Zelin Li, Gang Wang, Shi Han, Dongmei Zhang, et al. Out-of- distribution detection based on in-distribution data patterns memorization with modern hopfield energy. In The Eleventh International Conference on Learning Representations, 2022

    2022

  16. [24]

    Characterizing submanifold region for out-of-distribution detection

    Xuhui Li, Zhen Fang, Yonggang Zhang, Ning Ma, Jiajun Bu, Bo Han, and Haishuai Wang. Characterizing submanifold region for out-of-distribution detection. IEEE Transactions on Knowledge and Data Engineering, 2024

    2024

  17. [25]

    Out-of-distribution detection with subspace techniques and probabilistic modeling of features

    Ibrahima Ndiour, Nilesh Ahuja, and Omesh Tickoo. Out-of-distribution detection with subspace techniques and probabilistic modeling of features. arXiv preprint arXiv:2012.04250, 2020

    2012 arXiv

  18. [26]

    Out-of-distribution detection using union of 1-dimensional subspaces

    Alireza Zaeemzadeh, Niccolo Bisagno, Zeno Sambugaro, Nicola Conci, Nazanin Rahnavard, and Mubarak Shah. Out-of-distribution detection using union of 1-dimensional subspaces. In Proceedings of the IEEE/CVF conference on Computer Vision and Pattern Recognition, pages 9452–9461, 2021

    2021

  19. [27]

    Kernel principal component analysis

    Bernhard Schölkopf, Alexander Smola, and Klaus-Robert Müller. Kernel principal component analysis. In International conference on artificial neural networks, pages 583–588. Springer, 1997

    1997

  20. [28]

    Nonlinear component analysis as a kernel eigenvalue problem

    Bernhard Schölkopf, Alexander Smola, and Klaus-Robert Müller. Nonlinear component analysis as a kernel eigenvalue problem. Neural computation, 10(5):1299–1319, 1998

    1998

  21. [29]

    Imagenet: A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In IEEE Conference on Computer Vision and Pattern Recognition, pages 248–255, 2009

    2009

  22. [30]

    Karl Pearson. Liii. on lines and planes of closest fit to systems of points in space. The London, Edinburgh, and Dublin philosophical magazine and journal of science, 2(11):559–572, 1901

    1901

  23. [31]

    Principal component analysis

    Hervé Abdi and Lynne J Williams. Principal component analysis. Wiley interdisciplinary reviews: computational statistics, 2(4):433–459, 2010

    2010

  24. [32]

    Random features for large-scale kernel machines.Advances in neural information processing systems, 20, 2007

    Ali Rahimi and Benjamin Recht. Random features for large-scale kernel machines.Advances in neural information processing systems, 20, 2007

    2007

  25. [33]

    Using the nyström method to speed up kernel machines

    Christopher Williams and Matthias Seeger. Using the nyström method to speed up kernel machines. In Advances in Neural Information Processing Systems, volume 13. MIT Press, 2000

    2000

  26. [34]

    Kernel pca for out-of- distribution detection

    Kun Fang, Qinghua Tao, Kexin Lv, Mingzhen He, Xiaolin Huang, and JIE YANG. Kernel pca for out-of- distribution detection. In Advances in Neural Information Processing Systems, volume 37, pages 134317–134344, 2024

    2024

  27. [35]

    Fourier analysis on groups, volume 121967

    Walter Rudin. Fourier analysis on groups, volume 121967. Wiley Online Library, 1962

    1962

  28. [36]

    Learning multiple layers of features from tiny images

    A Krizhevsky. Learning multiple layers of features from tiny images. Master’s thesis, University of Toronto, 2009

    2009

  29. [37]

    Lsun: Construction of a large-scale image dataset using deep learning with humans in the loop

    Fisher Yu, Ari Seff, Yinda Zhang, Shuran Song, Thomas Funkhouser, and Jianxiong Xiao. Lsun: Construction of a large-scale image dataset using deep learning with humans in the loop. arXiv preprint arXiv:1506.03365, 2015

    2015 arXiv

  30. [38]

    Places: A 10 million image database for scene recognition

    Bolei Zhou, Agata Lapedriza, Aditya Khosla, Aude Oliva, and Antonio Torralba. Places: A 10 million image database for scene recognition. IEEE transactions on pattern analysis and machine intelligence, 40(6):1452–1464, 2017

    2017

  31. [39]

    Modern multidimensional scaling: Theory and applications

    Ingwer Borg and Patrick JF Groenen. Modern multidimensional scaling: Theory and applications. Springer Science & Business Media, 2007. 19

    2007

  32. [40]

    Petros Drineas and Michael W. Mahoney. On the nyström method for approximating a gram matrix for improved kernel-based learning. Journal of Machine Learning Research, 6(72):2153–2175, 2005

    2005

  33. [41]

    Sampling methods for the nyström method

    Sanjiv Kumar, Mehryar Mohri, and Ameet Talwalkar. Sampling methods for the nyström method. The Journal of Machine Learning Research, 13(1):981–1006, 2012

    2012

  34. [42]

    Support-vector networks

    Corinna Cortes and Vladimir Vapnik. Support-vector networks. Machine learning, 20:273–297, 1995

    1995

  35. [43]

    Randomized nonlinear component analysis

    David Lopez-Paz, Suvrit Sra, Alex Smola, Zoubin Ghahramani, and Bernhard Schölkopf. Randomized nonlinear component analysis. In International conference on machine learning, pages 1359–1367. PMLR, 2014

    2014

  36. [44]

    Streaming kernel pca withO(√n) random features

    Enayat Ullah, Poorya Mianjy, Teodor Vanislavov Marinov, and Raman Arora. Streaming kernel pca withO(√n) random features. 31, 2018

    2018

  37. [45]

    Gain with no pain: Efficiency of kernel-pca by nyström sampling

    Nicholas Sterge, Bharath Sriperumbudur, Lorenzo Rosasco, and Alessandro Rudi. Gain with no pain: Efficiency of kernel-pca by nyström sampling. In International Conference on Artificial Intelligence and Statistics, pages 3642–3652. PMLR, 2020

    2020

  38. [46]

    Kernel pca with the nyström method

    Fredrik Hallgren. Kernel pca with the nyström method. arXiv preprint arXiv:2109.05578, 2021

    2021 arXiv

  39. [47]

    Approximate kernel pca: Computational versus statistical trade-off

    Bharath K Sriperumbudur and Nicholas Sterge. Approximate kernel pca: Computational versus statistical trade-off. The Annals of Statistics, 50(5):2713–2736, 2022

    2022

  40. [48]

    Statistical optimality and computational efficiency of nystrom kernel pca

    Nicholas Sterge and Bharath K Sriperumbudur. Statistical optimality and computational efficiency of nystrom kernel pca. Journal of Machine Learning Research, 23(337):1–32, 2022

    2022

  41. [49]

    The pre-image problem in kernel methods

    JT-Y Kwok and IW-H Tsang. The pre-image problem in kernel methods. IEEE transactions on neural networks, 15(6):1517–1525, 2004

    2004

  42. [50]

    Rankfeat&rankweight: Rank-1 feature/weight removal for out-of- distribution detection

    Yue Song, Wei Wang, and Nicu Sebe. Rankfeat&rankweight: Rank-1 feature/weight removal for out-of- distribution detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024

    2024

  43. [51]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In IEEE Conference on Computer Vision and Pattern Recognition, pages 770–778, 2016

    2016

  44. [52]

    Mobilenetv2: Inverted residuals and linear bottlenecks

    Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chieh Chen. Mobilenetv2: Inverted residuals and linear bottlenecks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4510–4520, 2018

    2018

  45. [53]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at...

    2021

  46. [54]

    The inaturalist species classification and detection dataset

    Grant Van Horn, Oisin Mac Aodha, Yang Song, Yin Cui, Chen Sun, Alex Shepard, Hartwig Adam, Pietro Perona, and Serge Belongie. The inaturalist species classification and detection dataset. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 8...

    2018

  47. [55]

    Sun database: Large-scale scene recognition from abbey to zoo

    Jianxiong Xiao, James Hays, Krista A Ehinger, Aude Oliva, and Antonio Torralba. Sun database: Large-scale scene recognition from abbey to zoo. In 2010 IEEE computer society conference on computer vision and pattern recognition, pages 3485–3492. IEEE, 2010

    2010

  48. [56]

    Describing textures in the wild

    Mircea Cimpoi, Subhransu Maji, Iasonas Kokkinos, Sammy Mohamed, and Andrea Vedaldi. Describing textures in the wild. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3606–3613, 2014

    2014

  49. [57]

    Supervised contrastive learning

    Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan. Supervised contrastive learning. Advances in neural information processing systems, 33:18661–18673, 2020

    2020

  50. [58]

    A baseline for detecting misclassified and out-of-distribution examples in neural networks

    Dan Hendrycks and Kevin Gimpel. A baseline for detecting misclassified and out-of-distribution examples in neural networks. In International Conference on Learning Representations, 2016

    2016

  51. [59]

    Scaling out-of-distribution detection for real-world settings

    Dan Hendrycks, Steven Basart, Mantas Mazeika, Andy Zou, Joseph Kwon, Mohammadreza Mostajabi, Jacob Steinhardt, and Dawn Song. Scaling out-of-distribution detection for real-world settings. In International Conference on Machine Learning, pages 8759–8773. PMLR, 2022

    2022

  52. [60]

    Enhancing the reliability of out-of-distribution image detection in neural networks

    Shiyu Liang, Yixuan Li, and R Srikant. Enhancing the reliability of out-of-distribution image detection in neural networks. In International Conference on Learning Representations, 2018

    2018

  53. [61]

    Vim: Out-of-distribution with virtual-logit matching

    Haoqi Wang, Zhizhong Li, Litong Feng, and Wayne Zhang. Vim: Out-of-distribution with virtual-logit matching. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4921–4930, 2022. 20

    2022

  54. [62]

    Decoupling maxlogit for out-of-distribution detection

    Zihan Zhang and Xiang Xiang. Decoupling maxlogit for out-of-distribution detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3388–3397, 2023

    2023

  55. [63]

    Ssd: A unified framework for self-supervised outlier detection

    Vikash Sehwag, Mung Chiang, and Prateek Mittal. Ssd: A unified framework for self-supervised outlier detection. In International Conference on Learning Representations, 2020

    2020

  56. [64]

    Pytorch: An imperative style, high-performance deep learning library

    Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32, 2019

    2019

  57. [65]

    Dice: Leveraging sparsification for out-of-distribution detection

    Yiyou Sun and Yixuan Li. Dice: Leveraging sparsification for out-of-distribution detection. In European Conference on Computer Vision, pages 691–708. Springer, 2022

    2022

  58. [66]

    Gradorth: A simple yet efficient out-of-distribution detection with orthogonal projection of gradients

    Sima Behpour, Thang Long Doan, Xin Li, Wenbin He, Liang Gou, and Liu Ren. Gradorth: A simple yet efficient out-of-distribution detection with orthogonal projection of gradients. Advances in Neural Information Processing Systems, 36:38206–38230, 2023

    2023

  59. [67]

    Billion-scale similarity search with gpus

    Jeff Johnson, Matthijs Douze, and Hervé Jégou. Billion-scale similarity search with gpus. IEEE Transactions on Big Data, 7(3):535–547, 2019

    2019

  60. [68]

    Fast and scalable polynomial kernels via explicit feature maps

    Ninh Pham and Rasmus Pagh. Fast and scalable polynomial kernels via explicit feature maps. In Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 239–247, 2013

    2013

  61. [69]

    Kernel pca for novelty detection

    Heiko Hoffmann. Kernel pca for novelty detection. Pattern recognition, 40(3):863–874, 2007

    2007

  62. [70]

    Generalized odin: Detecting out-of-distribution image without learning from out-of-distribution data

    Yen-Chang Hsu, Yilin Shen, Hongxia Jin, and Zsolt Kira. Generalized odin: Detecting out-of-distribution image without learning from out-of-distribution data. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10951–10960, 2020

    2020

  63. [71]

    Gradient-regularized out-of- distribution detection

    Sina Sharifi, Taha Entesari, Bardia Safaei, Vishal M Patel, and Mahyar Fazlyab. Gradient-regularized out-of- distribution detection. In European Conference on Computer Vision, pages 459–478. Springer, 2024

    2024

  64. [72]

    Block selection method for using feature norm in out-of-distribution detection

    Yeonguk Yu, Sungho Shin, Seongju Lee, Changhyun Jun, and Kyoobin Lee. Block selection method for using feature norm in out-of-distribution detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15701–15711, 2023

    2023

  65. [73]

    Wdiscood: Out-of-distribution detection via whitened linear discriminant analysis

    Yiye Chen, Yunzhi Lin, Ruinian Xu, and Patricio A Vela. Wdiscood: Out-of-distribution detection via whitened linear discriminant analysis. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 5298–5307, 2023

    2023

  66. [74]

    Revisiting deep ensemble for out-of-distribution detection: A loss landscape perspective

    Kun Fang, Qinghua Tao, Xiaolin Huang, and Jie Yang. Revisiting deep ensemble for out-of-distribution detection: A loss landscape perspective. International Journal of Computer Vision, 132(12):6107–6126, 2024

    2024

  67. [75]

    Unsupervised out-of-distribution detection with diffusion inpainting

    Zhenzhen Liu, Jin Peng Zhou, Yufan Wang, and Kilian Q Weinberger. Unsupervised out-of-distribution detection with diffusion inpainting. In International Conference on Machine Learning, pages 22528–22538. PMLR, 2023

    2023

  68. [76]

    Csi: Novelty detection via contrastive learning on distributionally shifted instances

    Jihoon Tack, Sangwoo Mo, Jongheon Jeong, and Jinwoo Shin. Csi: Novelty detection via contrastive learning on distributionally shifted instances. Advances in neural information processing systems, 33:11839–11852, 2020

    2020

  69. [77]

    Is out-of-distribution detection learnable? Advances in Neural Information Processing Systems, 35:37199–37213, 2022

    Zhen Fang, Yixuan Li, Jie Lu, Jiahua Dong, Bo Han, and Feng Liu. Is out-of-distribution detection learnable? Advances in Neural Information Processing Systems, 35:37199–37213, 2022

    2022

  70. [78]

    Provable guarantees for understanding out-of-distribution detection

    Peyman Morteza and Yixuan Li. Provable guarantees for understanding out-of-distribution detection. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 7831–7840, 2022

    2022

  71. [79]

    Out-of-distribution detection by regaining lost clues

    Zhilin Zhao, Longbing Cao, and S Yu Philip. Out-of-distribution detection by regaining lost clues. Artificial Intelligence, 339:104275, 2025

    2025

  72. [80]

    Supervision adaptation balancing in-distribution generalization and out-of-distribution detection

    Zhilin Zhao, Longbing Cao, and Kun-Yu Lin. Supervision adaptation balancing in-distribution generalization and out-of-distribution detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(12):15743– 15758, 2023

    2023

  73. [81]

    Random features for kernel approximation: A survey on algorithms, theory, and beyond

    Fanghui Liu, Xiaolin Huang, Yudong Chen, and Johan AK Suykens. Random features for kernel approximation: A survey on algorithms, theory, and beyond. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(10):7128–7148, 2021

    2021

  74. [82]

    End-to-end kernel learning via generative random fourier features

    Kun Fang, Fanghui Liu, Xiaolin Huang, and Jie Yang. End-to-end kernel learning via generative random fourier features. Pattern Recognition, 134:109057, 2023

    2023

  75. [83]

    Nyström method vs random fourier features: A theoretical and empirical comparison

    Tianbao Yang, Yu-Feng Li, Mehrdad Mahdavi, Rong Jin, and Zhi-Hua Zhou. Nyström method vs random fourier features: A theoretical and empirical comparison. Advances in neural information processing systems, 25, 2012. 21

    2012

  76. [84]

    Reconciling modern machine-learning practice and the classical bias–variance trade-off

    Mikhail Belkin, Daniel Hsu, Siyuan Ma, and Soumik Mandal. Reconciling modern machine-learning practice and the classical bias–variance trade-off. Proceedings of the National Academy of Sciences, 116(32):15849–15854, 2019

    2019

  77. [85]

    Convolutional spectral kernel learning with generalization guarantees

    Jian Li, Yong Liu, and Weiping Wang. Convolutional spectral kernel learning with generalization guarantees. Artificial Intelligence, 313:103803, 2022

    2022

  78. [86]

    Learning in feature spaces via coupled covariances: Asymmetric kernel svd and nyström method

    Qinghua Tao, Francesco Tonin, Alex Lambert, Yingyi Chen, Panagiotis Patrinos, and Johan Suykens. Learning in feature spaces via coupled covariances: Asymmetric kernel svd and nyström method. In International Conference on Machine Learning, pages 47929–47947. PMLR, 2024

    2024

  79. [87]

    Nyströmformer: A nyström-based algorithm for approximating self-attention

    Yunyang Xiong, Zhanpeng Zeng, Rudrasis Chakraborty, Mingxing Tan, Glenn Fung, Yin Li, and Vikas Singh. Nyströmformer: A nyström-based algorithm for approximating self-attention. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pages 14138–14148, 2021

    2021

  80. [88]

    Nyström method for accurate and scalable implicit differentiation

    Ryuichiro Hataya and Makoto Yamada. Nyström method for accurate and scalable implicit differentiation. In International Conference on Artificial Intelligence and Statistics, pages 4643–4654. PMLR, 2023

    2023

  81. [89]

    Deep kernel learning

    Andrew Gordon Wilson, Zhiting Hu, Ruslan Salakhutdinov, and Eric P Xing. Deep kernel learning. In Artificial intelligence and statistics, pages 370–378. PMLR, 2016

    2016

  82. [90]

    Reading digits in natural images with unsupervised feature learning

    Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. Reading digits in natural images with unsupervised feature learning. In Proceedings of the NIPS Workshop on Deep Learning and Unsupervised Feature Learning, 2011

    2011

  83. [91]

    Turkergaze: Crowdsourcing saliency with webcam based eye tracking

    Pingmei Xu, Krista A Ehinger, Yinda Zhang, Adam Finkelstein, Sanjeev R Kulkarni, and Jianxiong Xiao. Turkergaze: Crowdsourcing saliency with webcam based eye tracking. arXiv preprint arXiv:1504.06755, 2015. 22 Appendix A Proof The proof of Proposition 1 is presented. Proof. Re...

    2015 arXiv

  84. [92]

    In this sense, the reconstruction error is interpreted as the norm of features projected into the residual subspace, i.e., theU p-space. Therefore, to obtain KPCA reconstruction errors via the kernel matrix K, we can employ the kernel trick to project ˆz into the residual spac...

  85. [93]

    B.1 Experiments on MobileNet Table 6 supplements detection results on the ImageNet-1K benchmark and MobileNetV2

    on the ImageNet-1K [29] benchmark, and results of ResNet18 [51] on the CIFAR10 [36] benchmark. B.1 Experiments on MobileNet Table 6 supplements detection results on the ImageNet-1K benchmark and MobileNetV2. We use the pre-trained checkpoint of MobileNetV2 released by PyTorch ...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.