Pith. sign in

REVIEW 2 major objections 6 minor 68 references

Two activation subspaces beat OOD detection baselines

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

ActSub decomposes activations via SVD of the classifier head into decisive and insignificant subspaces, using cosine similarity on the insignificant part for far-OOD and shaped energy on the decisive part for near-OOD, achieving SOTA on standard benchmarks.

T0 review reviewed 2026-08-05 challenge →

load-bearing objection Solid, reproducible OOD detector built on a clean subspace decomposition; the main weaknesses are an asserted causal mechanism and missing error bars, but the empirical package deserves a serious referee. the 2 major comments →

arxiv 2508.21695 v1 pith:2HLSBLSY submitted 2025-08-29 cs.LG cs.CV

Activation Subspaces for Out-of-Distribution Detection

Bar{\i}\c{s} Z\"ong\"ur , Robin Hesse , Stefan Roth This is my paper
classification cs.LG cs.CV
keywords out-of-distribution detectionactivation subspacessingular value decompositionactivation shapingcosine similarityenergy scorenullspacedeep neural networks
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that a trained classifier's activations contain two kinds of directions: those that drive its decisions (decisive) and those that barely affect them (insignificant). It shows that insignificant directions, because the classification loss hardly touches them, act like features of a random network and separate far out-of-distribution inputs from in-distribution inputs better than raw activations. At the same time, pruning those insignificant directions before applying activation shaping improves detection of near out-of-distribution inputs, which are confusable with in-distribution ones. Combining a cosine-similarity score on the insignificant subspace with an energy score on the shaped decisive subspace yields state-of-the-art OOD detection on standard benchmarks. If correct, the paper establishes that the part of the representation the classifier 'does not care about' carries a clean signal for novelty.

Core claim

The central discovery is that the SVD of the classification head's weight matrix W = UΣVᵀ defines two orthogonal subspaces: the first k right singular vectors span a decisive subspace that dominates the logits, and the remaining directions span an insignificant subspace that is nearly invisible to the loss. The paper claims that activations projected onto the insignificant subspace are 'untainted' by the cross-entropy objective, so in-distribution and out-of-distribution samples become more separable there than in the raw activation space; this shows up particularly for far-OOD data. Conversely, the decisive subspace is where activation shaping methods should operate, because insignificant d

What carries the argument

The SVD of the linear classification head's weight matrix (W = UΣVᵀ), whose right singular vectors are split into a decisive basis (top k) and an insignificant basis (the rest); activations are projected onto the two bases, with k chosen so the decisive and insignificant components have equal expected norm. The decisive projection is fed through an activation shaping function (e.g., SCALE) and scored by free energy, while the insignificant projection is scored by average cosine similarity to training activations; the two scores are combined multiplicatively with an exponent λ.

Load-bearing premise

The method assumes that the directions the classifier barely uses are untouched by training, so they behave like random, task-agnostic features; if cross-entropy training implicitly shapes those directions too, the gain of the insignificant subspace would vanish.

What would settle it

Train the same architecture on the same data with a loss that acts on the insignificant subspace directly—for instance, adding a small penalty that encourages the weight matrix's right singular vectors to spread uniformly, or using a supervised contrastive loss that shapes all activation directions—and measure the Far-OOD AUC of the insignificant-subspace score. If the advantage of ActSub's →S over raw-activation scores disappears or shrinks substantially, the 'untainted directions' posit is wrong; if the advantage persists even when those directions are explicitly regularized, the subspace ge

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Far-OOD detection improves because OOD samples deviate more strongly in directions the classifier ignored than in the directions it aligned to during training.
  • Near-OOD detection improves because activation shaping on the decisive component avoids interference from insignificant directions that change channel rankings without changing the model output.
  • The method is post-hoc: it needs only the trained weights and a random subset of training activations, and works across backbones (ResNet-50, MobileNetV2, DenseNet-101, ViT-B/16) and OOD benchmarks.
  • The subspace split is parameter-light: k is set automatically by balancing the norms of the two components, and the remaining λ is tuned on a small validation range.
  • Since the two scores target different shift regimes, the product form makes the method robust to datasets where one component alone fails (e.g., Textures).

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • This suggests that the insignificant subspace could be reused as a general-purpose random-feature embedding for other novelty-detection tasks, such as detecting out-of-distribution samples in semantic segmentation, whenever the head is linear.
  • A testable extension: if the Far-OOD advantage comes from training neglect, then training with a loss that shapes all right singular directions (e.g., contrastive or weight-decayed in the V basis) should weaken ActSub's insignificant-subspace score; if it does not, the geometry itself is the driver.
  • The decomposition is not limited to classification backbones: any feature extractor with a linear probe on top can be split this way, so the method could transfer to self-supervised representations, where the classifier is trained on top of frozen features.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 6 minor

Summary. The paper proposes ActSub, a post-hoc out-of-distribution (OOD) detection method that decomposes a network's penultimate-layer activations into decisive and insignificant components via SVD of the final linear classification head. The insignificant component is scored by average cosine similarity to training activations (Far-OOD), the decisive component is scored by energy on activation-shaped logits (Near-OOD), and the two scores are combined multiplicatively as ↔S = →S^λ · ←S. Experiments on ImageNet-1k (ResNet-50, MobileNetV2), CIFAR10/100 (DenseNet-101), and OpenOOD report improved AUC/FPR over prior baselines. The paper claims the insignificant subspace is 'untainted' by the classification objective, which is presented as the causal explanation for Far-OOD performance.

Significance. If the empirical results hold, ActSub is a simple and effective post-hoc detector that combines distance- and shaping-based cues. The paper's strengths are its extensive evaluation across backbones and benchmarks, ablations comparing SVD to PCA/null-space bases, sensitivity studies for data volume and the shaping parameter, and public code. The central theoretical claim—that the insignificant subspace behaves like random neural network features because training 'does not touch' those directions—is not established; Propositions 3.1 and 3.2 are asserted without proof or supporting training-dynamics evidence. The empirical method may still be valid, but the causal framing in the abstract and Section 3 currently overreaches.

major comments (2)
  1. [Sec. 3, Propositions 3.1 and 3.2; Eq. (6)] The 'untainted' premise is load-bearing for the central claim in the abstract and Section 3, but it is not supported. The nullspace argument applies to a fixed weight matrix; during training W itself changes, so directions in the final small-singular-value subspace are not guaranteed to have been unaffected by the loss. Moreover, the operative subspace is selected by the norm-balance criterion of Eq. (6), not by a singular-value threshold, so it can include directions with non-negligible impact on logits. Thus the statement that these directions 'only marginally affect the output' is not guaranteed for the subspace actually used. Please either (i) provide empirical evidence that the insignificant subspace's activation statistics are indistinguishable from those of a randomly initialized network, or (ii) reframe Propositions 3.1 and 3.2 as hypotheses and present the subspace decomposition
  2. [Sec. 4 and Supplementary Material (λ values, p tuning, Tab. 1 baseline selection)] The reported 'state-of-the-art' results rely on per-backbone hyperparameter tuning. The supplement states λ = 0.5, 1, 2 for MobileNetV2, ViT, and ResNet-50, and the shaping parameter p of SCALE is tuned on validation splits (Fig. 8). While disclosed, the main tables do not report the exact λ and p used for each result, and the sensitivity analysis is provided only for OpenOOD, not for the primary ImageNet-1k or CIFAR results. Additionally, baseline numbers in Tab. 1 are 'top results from the respective paper' rather than a unified implementation. To strengthen the SotA claim, please report the full hyperparameter settings for every experiment and, if possible, run baselines under a common codebase. The current presentation makes it difficult to judge whether the gains are due to the method itself or to more favorable hyperparameter selection.
minor comments (6)
  1. [Sec. 3.2, Eq. (7)] The number of nearest neighbors N is never specified. Eq. (7) uses N, and the text mentions a 10% random subset, but the actual N value used in experiments is missing. Please report it.
  2. [Sec. 3.1, Eq. (4)] The notation ←V^T and →V^T is ambiguous: these are n×n matrices with zero rows, not orthonormal bases. Clarify the dimensions and the projection operation to avoid confusion.
  3. [Sec. 3.1, Fig. 4] The text states that the smallest singular vector 'almost perfectly aligns' with the softmax-invariant direction 'for many backbone models,' but Fig. 4 only shows ResNet-50. Add analogous plots for MobileNetV2 and ViT or soften the claim.
  4. [Supplementary Table 8] There is a citation mismatch: 'SHE [37]' in the table appears to refer to [62], while [37] is GEN. Please correct the reference labels.
  5. [Sec. 3, Proposition labels] Items labeled 'Proposition 3.1' and 'Proposition 3.2' are not proven. In a technical paper, these should be called 'Observations' or 'Hypotheses' unless a proof is provided.
  6. [Tab. 1 footnote] Selecting 'top results from the respective paper' per baseline can introduce bias. Please provide a table listing the exact configuration (e.g., which variant of SCALE or fDBD, what hyperparameters) for each baseline entry.

Circularity Check

0 steps flagged

No significant circularity: the method is evaluated on external benchmarks, and the subspace scores are not defined in terms of the target OOD labels.

full rationale

The paper's derivation chain is self-contained. The SVD decomposition of the classifier head (Eq. 3) is a fixed linear-algebraic operation on the trained weights; the decisive/insignificant components (Eqs. 4–5) are defined by that decomposition, not by OOD labels. The score →S (Eq. 7) compares cosine similarities to ID training activations, and ←S (Eq. 9) is energy after activation shaping; neither is constructed from OOD test data. The combination ↔S (Eq. 10) is a simple product of the two scores. The hyperparameters k (Eq. 6) and λ are selected on ID training activations and a disclosed validation split, respectively, which is standard practice and is not a fit to the reported test benchmarks. The propositions in Sec. 3 are motivational posits rather than derivations, but they are not used to define the scores, so the unsupported causal explanation (the 'untainted' subspace story) does not make the empirical evaluation circular. The paper cites external work (e.g., null space analysis [8] and random neural networks [10]) without author overlap, and none of the load-bearing claims reduce to self-citation. Therefore the central SOTA claim is an empirical result against external benchmarks, not a consequence of the definitions.

Axiom & Free-Parameter Ledger

5 free parameters · 6 axioms · 0 invented entities

The method introduces no new physical or latent entities, only orthogonal projections of existing activations. The main dependencies are empirical assumptions about training dynamics (untainted insignificant subspace, SVD alignment with softmax invariance) and several fitted hyperparameters (k, λ, shaping settings).

free parameters (5)
  • k (subspace split index) = Not reported numerically; chosen to balance norms via Eq. (6) on training activations
    The number of decisive directions. Selected by minimizing the mean absolute norm difference between decisive and insignificant components on ID training activations.
  • lambda (score exponent) = 0.5 (MobileNetV2), 1 (ViT), 2 (ResNet-50); 0.5 for ViT with GEN
    Weights the insignificant-component score against the decisive-component score. Tuned via hyperparameter search on a validation OOD set (NINCO for ImageNet, MNIST for CIFAR, OpenOOD validation for OpenOOD).
  • Activation shaping hyperparameters (e.g., pruning percentage p of SCALE) = Tuned in supplement; shown in sensitivity plots
    The shaping function Φ is a baseline method (SCALE or GEN) with its own hyperparameters. The supplement tunes these using ImageNet-R as additional ID data, which could leak OOD-shift information.
  • N (number of top similar training samples for cosine similarity) = Unspecified in main text
    The number of training insignificant components averaged in Eq. (7) when computing →S. Not stated; presumably a small constant.
  • Random subset fraction for cosine similarity references = 10%
    A random 10% of the training set is used to build the reference set for →S. The exact subset is not seeded.
axioms (6)
  • standard math The weight matrix W of a classifier admits an SVD W = UΣV^T whose right singular vectors span subspaces of decreasing importance to the logits.
    Invoked in Eq. (3) and used throughout Sec. 3.1.
  • standard math For a linear classifier with n > c, the null space of W is non-trivial, providing exact output-invariant directions.
    Used in Sec. 3 to justify the existence of insignificant directions.
  • domain assumption Cross-entropy training enforces activation alignment with class directions, leaving insignificant directions mostly unchanged from random initialization ('untainted').
    Stated as a posit in Sec. 3: 'DNNs tend to enforce activations to align with known classification directions due to the cross-entropy loss' and 'the model retains its behavior from the random initialization of weights' for insignificant directions.
  • domain assumption The softmax-invariant direction p = W†1 aligns with the right singular vector corresponding to the smallest non-negative singular value.
    Empirically observed and used to justify the SVD basis as capturing the softmax-invariant direction; see Fig. 4 and Sec. 3.1.
  • domain assumption Directions that only marginally affect the classifier output are useful for OOD detection (Proposition 3.1).
    This is the load-bearing premise of the →S score, not proved, only argued intuitively and validated empirically.
  • domain assumption Insignificant directions interfere with activation shaping methods (Proposition 3.2).
    Motivates applying shaping only to the decisive subspace; asserted without formal derivation.

reviewed 2026-08-05 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Activation Subspaces for Out-of-Distribution Detection." pith.science (2026). https://pith.science/paper/2HLSBLSY

@misc{pith2026250821695,
  author       = {Pith},
  title        = {Pith review of: Activation Subspaces for Out-of-Distribution Detection},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2HLSBLSY}},
  note         = {Machine review of arXiv:2508.21695}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

To ensure the reliability of deep models in real-world applications, out-of-distribution (OOD) detection methods aim to distinguish samples close to the training distribution (in-distribution, ID) from those farther away (OOD). In this work, we propose a novel OOD detection method that utilizes singular value decomposition of the weight matrix of the classification head to decompose the model's activations into decisive and insignificant components, which contribute maximally, respectively minimally, to the final classifier output. We find that the subspace of insignificant components more effectively distinguishes ID from OOD data than raw activations in regimes of large distribution shifts (Far-OOD). This occurs because the classification objective leaves the insignificant subspace largely unaffected, yielding features that are ''untainted'' by the target classification task. Conversely, in regimes of smaller distribution shifts (Near-OOD), we find that activation shaping methods profit from only considering the decisive subspace, as the insignificant component can cause interference in the activation space. By combining two findings into a single approach, termed ActSub, we achieve state-of-the-art results in various standard OOD benchmarks.

Figures

Figures reproduced from arXiv: 2508.21695 by Bar{\i}\c{s} Z\"ong\"ur, Robin Hesse, Stefan Roth.

Figure 1
Figure 1. Figure 1: False positive rate (FPR) for OOD detection when OOD inputs are close to (Near-OOD) and far from (Far-OOD) the ID domain. Our approach outperforms established baselines, espe￾cially for more challenging Near-OOD examples. functions can be based on model inherent information, such as logits [20, 36, 64], distances [15, 31, 35, 52], or (shaped) activations [11, 51, 57, 58]. In this work, we show that DNN act… view at source ↗
Figure 2
Figure 2. Figure 2: Softmax output (softmax1 ) for the class associated with Logit1 , assuming a setting with two classes. The black lines repre￾sent contour lines for the output values 0.1, 0.5, and 0.9. Note how they are parallel, illustrating that there is a direction in the logit space that does not affect the softmax output. Insignificant directions for OOD detection. We let F : R m 7→ R n denote a DNN that maps an input… view at source ↗
Figure 3
Figure 3. Figure 3: Flow diagram of our proposed ActSub algorithm. Briefly, an input x is fed through a model F to obtain the activation a, which is decomposed into the insignificant component →a and the decisive component ←a using SVD on the weight matrix W. Our final score ↔ S is a combination of → S (obtained by comparing →a to insignificant activations from the training set →a (i) ) and ← S (obtained by applying activatio… view at source ↗
Figure 4
Figure 4. Figure 4: Projection of the softmax-invariant direction p from Eq. (2) onto the right singular vectors of the weight matrix (SVD) or the eigenvectors of the activation covariance (PCA). The x-axis represents the respective directions in descending order of their singular values, respectively, eigenvalues. The y-axis shows the magnitude of the projection onto each of the corresponding di￾rections. The softmax-invaria… view at source ↗
Figure 6
Figure 6. Figure 6: Classification and OOD detection accuracy (%, ↑) for different ratios of the norms of decisive and insignificant compo￾nents with ResNet (orange) and MobileNet (blue). The AUC is calculated by the average of datasets in OpenOOD [63], and ID is ImageNet-1k [9, 44]. sive component. With this observation, we emphasize the complementary effect of our two score functions. Choice of hyperparameter k. Ideally, th… view at source ↗
Figure 8
Figure 8. Figure 8: Sensitivity of pruning percentage p of SCALE. We report the AUC for our method ActSub ( ↔ S) for different percentages of pruning (p) for Near-OOD and Far-OOD from the OpenOOD [63] benchmark. We use a ResNet-50 [16] model trained on ImageNet￾1k [9, 44]. curacy has been shown to decrease after 85 % [57]. We believe this is because SCALE needs to keep more chan￾nels to capture discriminative information when… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

68 extracted references · 64 canonical work pages

  1. [1]

    LINe: Out-of-distribution detection by leveraging important neurons

    Yong Hyun Ahn, Gyeong-Moon Park, and Seong Tae Kim. LINe: Out-of-distribution detection by leveraging important neurons. In CVPR, pages 19852–19862, 2023. 2, 6, 7, iv

    work page 2023

  2. [2]

    GradOrth: A simple yet efficient out- of-distribution detection with orthogonal projection of gra- dients

    Sima Behpour, Thang Long Doan, Xin Li, Wenbin He, Liang Gou, and Liu Ren. GradOrth: A simple yet efficient out- of-distribution detection with orthogonal projection of gra- dients. In NeurIPS, 2023. 2

    work page 2023

  3. [3]

    In or out? Fixing ImageNet out-of-distribution detection evalu- ation

    Julian Bitterwolf, Maximilian M ¨uller, and Matthias Hein. In or out? Fixing ImageNet out-of-distribution detection evalu- ation. In ICML, pages 2471–2506, 2023. 7, i, ii

    work page 2023

  4. [4]

    A random CNN sees objects: One inductive bias of CNN and its applications

    Yun-Hao Cao and Jianxin Wu. A random CNN sees objects: One inductive bias of CNN and its applications. In AAAI, pages 194–202, 2022. 1

    work page 2022

  5. [5]

    Entropy maximization and meta classification for out-of- distribution detection in semantic segmentation

    Robin Chan, Matthias Rottmann, and Hanno Gottschalk. Entropy maximization and meta classification for out-of- distribution detection in semantic segmentation. In ICCV, pages 5108–5117, 2021. 2

    work page 2021

  6. [6]

    GAIA: Delving into gradient- based attribution abnormality for out-of-distribution detec- tion

    Jinggang Chen, Junjie Li, Xiaoyang Qu, Jianzong Wang, Jiguang Wan, and Jing Xiao. GAIA: Delving into gradient- based attribution abnormality for out-of-distribution detec- tion. In NeurIPS, 2023. 2

    work page 2023

  7. [7]

    Describing textures in the wild

    Mircea Cimpoi, Subhransu Maji, Iasonas Kokkinos, Sammy Mohamed, and Andrea Vedaldi. Describing textures in the wild. In CVPR, pages 3606–3613, 2014. 5, 7, i, iv

    work page 2014

  8. [8]

    Matthew Cook, Alina Zare, and Paul D. Gader. Outlier detection through null space analysis of neural networks. arXiv:2007.01263 [cs.LG], 2020. 1, 2, 3

    Pith/arXiv arXiv 2007

  9. [9]

    ImageNet: A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. ImageNet: A large-scale hierarchical image database. In CVPR, pages 248–255, 2009. 3, 5, i, ii, iii

    work page 2009

  10. [10]

    The separation capacity of random neural networks

    Sjoerd Dirksen, Martin Genzel, Laurent Jacques, and Alexander Stollenwerk. The separation capacity of random neural networks. Int. J. Comput. Vision, 23(309):1–47, 2022. 1, 3

    work page 2022

  11. [11]

    Extremely simple activation shaping for out- of-distribution detection

    Andrija Djurisic, Nebojsa Bozanic, Arjun Ashok, and Rosanne Liu. Extremely simple activation shaping for out- of-distribution detection. In ICLR, 2023. 1, 2, 3, 6, 7, i, iv

    work page 2023

  12. [12]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. In ICLR, 2021. i

    work page 2021

  13. [13]

    Kernel PCA for out-of-distribution detection

    Kun Fang, Qinghua Tao, Kexin Lv, Mingzhen He, Xiaolin Huang, and Jie Yang. Kernel PCA for out-of-distribution detection. In NeurIPS, 2024. 2, 5, 6, 7

    work page 2024

  14. [14]

    WeiPer: OOD detection using weight perturbations of class projections

    Maximilian Granz, Manuel Heurich, and Tim Landgraf. WeiPer: OOD detection using weight perturbations of class projections. In NeurIPS, 2024. 2, 7, iii, iv

    work page 2024

  15. [15]

    Revisit PCA-based technique for out-of- distribution detection

    Xiaoyuan Guan, Zhouwu Liu, Wei-Shi Zheng, Yuren Zhou, and Ruixuan Wang. Revisit PCA-based technique for out-of- distribution detection. In ICCV, pages 19374–19382, 2023. 1, 2, 5, 6, 7

    work page 2023

  16. [16]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, pages 770–778, 2016. 3, 5, i, ii

    work page 2016

  17. [17]

    A baseline for detect- ing misclassified and out-of-distribution examples in neural networks

    Dan Hendrycks and Kevin Gimpel. A baseline for detect- ing misclassified and out-of-distribution examples in neural networks. In ICLR, 2017. 1, 2, 5, 6, 7, iv

    work page 2017

  18. [18]

    Dietterich

    Dan Hendrycks, Mantas Mazeika, and Thomas G. Dietterich. Deep anomaly detection with outlier exposure. In ICLR,

    work page

  19. [19]

    The many faces of robustness: A criti- cal analysis of out-of-distribution generalization

    Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kada- vath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, Dawn Song, Jacob Steinhardt, and Justin Gilmer. The many faces of robustness: A criti- cal analysis of out-of-distribution generalization. In ICCV, pages 8320–8329, 2021. ii

    work page 2021

  20. [20]

    Scaling out-of-distribution detection for real-world settings

    Dan Hendrycks, Steven Basart, Mantas Mazeika, Andy Zou, Joseph Kwon, Mohammadreza Mostajabi, Jacob Steinhardt, and Dawn Song. Scaling out-of-distribution detection for real-world settings. In ICML, pages 8759–8773, 2022. 1, 2

    work page 2022

  21. [21]

    Energy-based Hopfield boosting for out-of-distribution detection

    Claus Hofmann, Simon Schmid, Bernhard Lehner, Daniel Klotz, and Sepp Hochreiter. Energy-based Hopfield boosting for out-of-distribution detection. In NeurIPS, 2024. 2

    work page 2024

  22. [22]

    Belongie

    Grant Van Horn, Oisin Mac Aodha, Yang Song, Yin Cui, Chen Sun, Alexander Shepard, Hartwig Adam, Pietro Per- ona, and Serge J. Belongie. The iNaturalist species classi- fication and detection dataset. In CVPR, pages 8769–8778,

    work page

  23. [23]

    Weinberger

    Gao Huang, Zhuang Liu, Laurens van der Maaten, and Kil- ian Q. Weinberger. Densely connected convolutional net- works. In CVPR, pages 770–778, 2017. 7

    work page 2017

  24. [24]

    MOS: Towards scaling out-of- distribution detection for large semantic space

    Rui Huang and Yixuan Li. MOS: Towards scaling out-of- distribution detection for large semantic space. In CVPR, pages 8710–8719, 2021. 2, 5

    work page 2021

  25. [25]

    On the impor- tance of gradients for detecting distributional shifts in the wild

    Rui Huang, Andrew Geng, and Yixuan Li. On the impor- tance of gradients for detecting distributional shifts in the wild. In NeurIPS, pages 677–689, 2021. 2

    work page 2021

  26. [26]

    Batch Normalization: Accelerating deep network training by reducing internal co- variate shift

    Sergey Ioffe and Christian Szegedy. Batch Normalization: Accelerating deep network training by reducing internal co- variate shift. In ICML, pages 448–456, 2015. 2

    work page 2015

  27. [27]

    Directional convergence and alignment in deep learning

    Ziwei Ji and Matus Telgarsky. Directional convergence and alignment in deep learning. In NeurIPS, 2020. 3

    work page 2020

  28. [28]

    Donald E. Knuth. Two notes on notation. The American Mathematical Monthly, 99(5):403–422, 1992. 2

    work page 1992

  29. [29]

    Learning multiple layers of features from tiny images

    Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical report, University of Toronto, 2009. 7, ii, iii, iv

    work page 2009

  30. [30]

    MNIST handwritten digit database, 2010

    Yann LeCun, Corinna Cortes, and Chris Burges. MNIST handwritten digit database, 2010. ii

    work page 2010

  31. [31]

    A simple unified framework for detecting out-of-distribution 9 samples and adversarial attacks

    Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. A simple unified framework for detecting out-of-distribution 9 samples and adversarial attacks. In NeurIPS, pages 7167– 7177, 2018. 1, 2, 6

    work page 2018

  32. [32]

    Se- TAR: Out-of-distribution detection with selective low-rank approximation

    Yixia Li, Boya Xiong, Guanhua Chen, and Yun Chen. Se- TAR: Out-of-distribution detection with selective low-rank approximation. In NeurIPS, 2024. 2

    work page 2024

  33. [33]

    Shiyu Liang, Yixuan Li, and R. Srikant. Enhancing the re- liability of out-of-distribution image detection in neural net- works. In ICLR, 2018. 1, 6, 7

    work page 2018

  34. [34]

    MOOD: Multi- level out-of-distribution detection

    Ziqian Lin, Sreya Dutta Roy, and Yixuan Li. MOOD: Multi- level out-of-distribution detection. In CVPR, pages 15313– 15323, 2021. 2

    work page 2021

  35. [35]

    Fast decision boundary based out- of-distribution detector

    Litian Liu and Yao Qin. Fast decision boundary based out- of-distribution detector. In ICML, 2024. 1, 2, 6

    work page 2024

  36. [36]

    Owens, and Yixuan Li

    Weitang Liu, Xiaoyun Wang, John D. Owens, and Yixuan Li. Energy-based out-of-distribution detection. In NeurIPS,

    work page

  37. [37]

    GEN: Pushing the limits of softmax-based out-of-distribution de- tection

    Xixi Liu, Yaroslava Lochman, and Christopher Zach. GEN: Pushing the limits of softmax-based out-of-distribution de- tection. In CVPR, pages 23946–23955, 2023. 1, 2, 7, i, ii, iii, iv

    work page 2023

  38. [38]

    Learning with mixture of prototypes for out-of-distribution detection

    Haodong Lu, Dong Gong, Shuo Wang, Jason Xue, Lina Yao, and Kristen Moore. Learning with mixture of prototypes for out-of-distribution detection. In ICLR, 2024. 2

    work page 2024

  39. [39]

    How to exploit hyperspherical embeddings for out-of-distribution detection? In ICLR, 2023

    Yifei Ming, Yiyou Sun, Ousmane Dia, and Yixuan Li. How to exploit hyperspherical embeddings for out-of-distribution detection? In ICLR, 2023. 2

    work page 2023

  40. [40]

    Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bis- sacco, Bo Wu, and Andrew Y . Ng. Reading digits in natural images with unsupervised feature learning. In NIPS*2011. 7, iv

    work page 2011

  41. [41]

    Nearest neighbor guidance for out-of-distribution detection

    Jaewoo Park, Yoon Gyo Jung, and Andrew Beng Jin Teoh. Nearest neighbor guidance for out-of-distribution detection. In ICCV, pages 1686–1695, 2023. 2, 4, 6, 7

    work page 2023

  42. [42]

    Yang, Zachary DeVito, Mar- tin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala

    Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zem- ing Lin, Natalia Gimelshein, Luca Antiga, Alban Desmai- son, Andreas K ¨opf, Edward Z. Yang, Zachary DeVito, Mar- tin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Py- Torch: An imperative styl...

    work page 2019

  43. [43]

    Liu, Abhijit Guha Roy, Shreyas Padhy, and Balaji Lakshminarayanan

    Jie Ren, Stanislav Fort, Jeremiah Z. Liu, Abhijit Guha Roy, Shreyas Padhy, and Balaji Lakshminarayanan. A simple fix to Mahalanobis distance for improving near-OOD detection. arXiv:2106.09022 [cs.LG], 2021. iii

    Pith/arXiv arXiv 2021

  44. [44]

    Bernstein, Alexander C

    Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, San- jeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael S. Bernstein, Alexander C. Berg, and Li Fei-Fei. ImageNet large scale visual recognition chal- lenge. Int. J. Comput. Vision, 115(3):211–252, 2015. 3, 5, i, ii, iii

    work page 2015

  45. [45]

    Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chieh Chen

    Mark Sandler, Andrew G. Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chieh Chen. MobileNetV2: Inverted residuals and linear bottlenecks. InCVPR, pages 4510–4520,

    work page

  46. [46]

    SSD: A unified framework for self-supervised outlier detection

    Vikash Sehwag, Mung Chiang, and Prateek Mittal. SSD: A unified framework for self-supervised outlier detection. In ICLR, 2021. 2

    work page 2021

  47. [47]

    L. S. Shapley. 17. A Value for n-Person Games, pages 307–

    work page

  48. [48]

    RankFeat: Rank-1 fea- ture removal for out-of-distribution detection

    Yue Song, Nicu Sebe, and Wei Wang. RankFeat: Rank-1 fea- ture removal for out-of-distribution detection. In NeurIPS,

    work page

  49. [49]

    G. Strang. Linear Algebra and Its Applications . Thomson, Brooks/Cole, 2006. 3

    work page 2006

  50. [50]

    DICE: Leveraging sparsification for out-of-distribution detection

    Yiyou Sun and Yixuan Li. DICE: Leveraging sparsification for out-of-distribution detection. In ECCV, pages 691–708,

    work page

  51. [51]

    ReAct: Out-of- distribution detection with rectified activations

    Yiyou Sun, Chuan Guo, and Yixuan Li. ReAct: Out-of- distribution detection with rectified activations. In NeurIPS,

    work page

  52. [52]

    Out-of- distribution detection with deep nearest neighbors

    Yiyou Sun, Yifei Ming, Xiaojin Zhu, and Yixuan Li. Out-of- distribution detection with deep nearest neighbors. In ICML, pages 20827–20840, 2022. 1, 2, 6, 7, iii, iv

    work page 2022

  53. [53]

    Lempitsky

    Dmitry Ulyanov, Andrea Vedaldi, and Victor S. Lempitsky. Deep image prior. In CVPR, pages 9446–9454, 2018. 1

    work page 2018

  54. [54]

    Open-set recognition: A good closed-set classifier is all you need

    Sagar Vaze, Kai Han, Andrea Vedaldi, and Andrew Zisser- man. Open-set recognition: A good closed-set classifier is all you need. In ICLR, 2022. 7

    work page 2022

  55. [55]

    ViM: Out-of-distribution with virtual-logit matching

    Haoqi Wang, Zhizhong Li, Litong Feng, and Wayne Zhang. ViM: Out-of-distribution with virtual-logit matching. In CVPR, pages 4911–4920, 2022. 2, 3, 5, 6, 7, iii

    work page 2022

  56. [56]

    Ehinger, Aude Oliva, and Antonio Torralba

    Jianxiong Xiao, James Hays, Krista A. Ehinger, Aude Oliva, and Antonio Torralba. SUN database: Large-scale scene recognition from abbey to zoo. In CVPR, pages 3485–3492,

    work page

  57. [57]

    Scaling for training time and post-hoc out-of-distribution de- tection enhancement

    Kai Xu, Rongyu Chen, Gianni Franchi, and Angela Yao. Scaling for training time and post-hoc out-of-distribution de- tection enhancement. In ICLR, 2024. 1, 2, 3, 6, 7, i, ii, iii, iv

    work page 2024

  58. [58]

    VRA: Variational rectified activation for out-of-distribution detec- tion

    Mingyu Xu, Zheng Lian, Bin Liu, and Jianhua Tao. VRA: Variational rectified activation for out-of-distribution detec- tion. In NeurIPS, 2023. 1, 2, 3, 6

    work page 2023

  59. [59]

    Ehinger, Yinda Zhang, Adam Finkel- stein, Sanjeev R

    Pingmei Xu, Krista A. Ehinger, Yinda Zhang, Adam Finkel- stein, Sanjeev R. Kulkarni, and Jianxiong Xiao. TurkerGaze: Crowdsourcing saliency with webcam based eye tracking. arXiv:1504.06755 [cs.CV], 2015. 7, iv

    Pith/arXiv arXiv 2015

  60. [60]

    Generalized out-of-distribution detection: A survey

    Jingkang Yang, Kaiyang Zhou, Yixuan Li, and Ziwei Liu. Generalized out-of-distribution detection: A survey. Int. J. Comput. Vision, 132(12):5635–5662, 2024. 1

    work page 2024

  61. [61]

    Discriminability-driven channel selection for out-of-distribution detection

    Yue Yuan, Rundong He, Yicong Dong, Zhongyi Han, and Yilong Yin. Discriminability-driven channel selection for out-of-distribution detection. In CVPR, pages 26171–26180,

    work page

  62. [62]

    Out- of-distribution detection based on in-distribution data pat- terns memorization with modern Hopfield energy

    Jinsong Zhang, Qiang Fu, Xu Chen, Lun Du, Zelin Li, Gang Wang, Xiaoguang Liu, Shi Han, and Dongmei Zhang. Out- of-distribution detection based on in-distribution data pat- terns memorization with modern Hopfield energy. In ICLR,

    work page

  63. [63]

    OpenOOD v1.5: Enhanced benchmark for out-of-distribution detection

    Jingyang Zhang, Jingkang Yang, Pengyun Wang, Haoqi Wang, Yueqian Lin, Haoran Zhang, Yiyou Sun, Xuefeng Du, Kaiyang Zhou, Wayne Zhang, Yixuan Li, Ziwei Liu, Yiran Chen, and Hai Li. OpenOOD v1.5: Enhanced benchmark for out-of-distribution detection. arXiv:2306.09301 [cs.LG],

    Pith/arXiv arXiv

  64. [64]

    Decoupling MaxLogit for out-of-distribution detection

    Zihan Zhang and Xiang Xiang. Decoupling MaxLogit for out-of-distribution detection. In CVPR, pages 3388–3397,

    work page

  65. [65]

    Places: A 10 million image database for scene recognition

    Bolei Zhou, `Agata Lapedriza, Aditya Khosla, Aude Oliva, and Antonio Torralba. Places: A 10 million image database for scene recognition. IEEE T. Pattern Anal. Mach. Intell. , 40(6):1452–1464, 2018. 5, 7, iv

    work page 2018

  66. [66]

    Dependence on the data volume

    Yao Zhu, Yuefeng Chen, Chuanlong Xie, Xiaodan Li, Rong Zhang, Hui Xue, Xiang Tian, Bolun Zheng, and Yaowu Chen. Boosting out-of-distribution detection with typical features. In NeurIPS, 2022. 2, 6, 7 11 Activation Subspaces for Out-of-Distribution Detection Supplementary Material 20 30 40 50 2 3 Score of the decisive component ( ← S) Score of the insignif...

    work page 2022

  67. [318]

    Princeton University Press, 1953. 2

    work page 1953

  68. [2021]

    1, 2, 3, 5, 6, 7, i, iv

    work page

This paper was first reviewed by deepseek-v4-flash on August 5, 2026.