REVIEW 3 major objections 5 minor 33 references
Mining In-distribution Attributes in Outliers for Out-of-distribution Detection
T0 review · 3 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read Out-of-distribution images carry in-distribution features, and a training framework that mines them improves detection.
desk verdict Solid empirical paper with a novel capped-softmax loss for OOD training; the theoretical story is more heuristic than proof. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the extended multi-view data model (MVDM): every input is a bag of patches whose feature vectors have coefficients, with ID inputs carrying main features (total coefficient at least 1) and OOD inputs carrying only minor ID features (total coefficient in $[\Omega(1),0.4]$). This model justifies MaxLogit as the OOD score and powers the multi-view learning objective that caps soft labels at $\epsilon$; the cap makes the gradient vanish for pure-noise categories and moderate responses to minor ID features. Theorems 1 and 2 are the bridge that converts the model's feature-coefficient ordering into finite-sample false-negative bounds for the MaxLogit detector.
What would settle it
Construct an auxiliary OOD set from ID images whose features are preserved at near-main scale (for example, CIFAR images with small crops or mild corruptions), train MVOL and OE+MaxLogit on it, and compare FPR95: if MVOL's advantage over OE disappears or reverses as the 0.4-coefficient ceiling is violated, the central assumption is refuted.
Extended reading notes
Core claim
On the paper's own terms, the discovery is a structured view of ID/OOD correlation: define ID data as mixtures of main features (coefficient sums in $[1,O(1)]$) and minor features, and define OOD data as minor ID features with coefficient sums in $[\Omega(1),0.4]$ plus noise. Under this extended multi-view data model, a well-calibrated network's MaxLogit response to an outlier is bounded below its response to ID data, making MaxLogit an interpretable OOD score; Theorems 1 and 2 turn that ordering into false-negative bounds of $\frac{1}{2}(1-\mu+o(1))$ for a calibrated single model and $o(1)$ for an ensemble-distilled model. The learning objective $L_{\mathrm{MVOL}}$ sets soft target $p^{(t)}_{j,i}=\min(\mathrm{logit}_i(F^{(t)},X^{\mathrm{out}}_j),\epsilon)$, so categories carrying minor ID features keep moderate logit targets while pure-noise categories get smaller gradient weight, addressing what the paper identifies as outlier exposure's flaw of overemphasizing noise and underemphasizing minor features. Empirically the paper reports lower FPR95 than OE, energy-regularized learning, and OE+MaxLogit baselines, and stable performance when auxiliary wild data contains up to 50% ID noise.
Load-bearing premise
The theoretical claims rest on the assumption that every outlier sample is made of minor in-distribution features with total coefficient between a small constant and 0.4 plus noise; if real outliers contain strong, main-scale ID features, the ordering of scores that makes MaxLogit separate ID from OOD can break.
Editorial extensions
If this is right
- If outliers really are minor ID features plus noise, then the correct OOD score is the largest ID-feature response (MaxLogit), not a measure of total softmax uncertainty.
- Training with the $\epsilon$-capped soft targets should detect OOD inputs better than uniform-target outlier exposure while degrading ID accuracy less.
- Under ensemble distillation, where a student learns all features, the theorems predict near-zero false-negative rates, so MVOL should be paired with distillation or self-distillation for maximal separation.
- In wild-dataset settings with ID noise, the capped targets automatically reduce the gradient on the true ID class, which explains why MVOL's FPR95 degrades more slowly as noise level $\alpha$ rises.
- MVOL can be applied through fine-tuning as well as from-scratch training, preserving more ID accuracy than outlier-exposure fine-tuning.
Reading between the lines
- A natural next test is a diagnostic: measure the distribution of total feature coefficients of the auxiliary OOD set; if a substantial fraction of outliers exceed the $0.4$ ceiling, MVOL's theoretical advantage should shrink.
- The same $\epsilon$-capped soft-target trick could generalize to open-set recognition or misclassification detection; the paper's own MisD experiments hint at this.
- One could generate synthetic auxiliary outliers by recombining minor ID features under the model, giving a principled alternative to random auxiliary data collection.
- The single threshold $\epsilon$ is the main free parameter; a per-category or adaptive threshold derived from the ID logit distribution might remove the need for tuning, and the sensitivity table suggests a mild optimum.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes MVOL, an outlier-exposure method for OOD detection. It extends the multi-view data model of Allen-Zhu and Li by positing that OOD samples consist mainly of minor in-distribution features plus noise (Definition 3). It uses MaxLogit as the OOD score and a modified auxiliary loss that caps soft-label targets at epsilon (Eqs. 5-6) to avoid uniformly suppressing ID attributes in outliers. The authors provide theoretical bounds on the false negative rate for 'calibrated' single and ensemble-distillation models (Theorems 1-2) and report experiments on CIFAR-10/CIFAR-100 with six OOD test sets, plus wild-dataset simulations with ID noise. The main empirical findings are FPR95 improvements over OE and OE+MaxLogit, most notably on CIFAR-100 in the ensemble-distillation setting, and robustness when auxiliary data contains ID images.
Significance. If the empirical findings hold, MVOL is a useful addition to the OOD toolkit: it is simple, reproducible (code released), and gives consistent FPR95 gains, especially under ensemble distillation and with contaminated auxiliary data. The paper also evaluates misclassification detection and sensitivity of the epsilon hyperparameter, which is good practice. However, the theoretical claims as stated do not yet validate the method: the theorems hold only for an ideal equal-weight classifier satisfying Assumptions 1/2 and an unproven Induction Hypothesis 3, and no link is established between the MVOL update and that calibrated regime. The empirical contribution is thus stronger than the theoretical one; the paper's abstract overstates the theoretical insight.
major comments (3)
- [Appendix B, Induction Hypothesis 3 and Theorems 1-2] The proofs of Theorems 1 and 2 are conditional on Assumption 1/2 (an 'ideal classifier' with equal total learned weight on every feature) and on Induction Hypothesis 3, which is asserted without proof. The manuscript does not show that the MVOL objective in Eqs. (5)-(7) drives a randomly initialized network into or near this calibrated regime; the only support is the heuristic gradient discussion in Appendix E. Consequently the FNR bounds characterize a hypothetical classifier, not the model produced by MVOL. Because the abstract and introduction claim 'theoretical insights on the effectiveness of MVOL', this missing link is a load-bearing gap.
- [Method, Eq. (7) and Appendix E] For any auxiliary outlier whose softmax probabilities all lie below epsilon (epsilon=0.02 on CIFAR-100), the target p_j,i in Eq. (6) equals logit_i for every i, so sum_n p_j,n = 1 and the auxiliary-loss gradient in Eq. (7) is identically zero. The objective then updates only high-confidence outliers. No convergence or calibration analysis is provided to explain why acting only on such samples yields the global behavior assumed by Induction Hypothesis 3 and Assumptions 1/2. This directly affects the central claim that MVOL 'effectively utilizes' auxiliary OOD data.
- [Definition 3, Assumption 1 and Proposition 2] The separation result is largely a restatement of the assumed data model: Definition 3 fixes OOD coefficient sums in [Omega(1), 0.4], ID main-feature sums at least 1, and Assumption 1 imposes equal learned weight per feature, so z(X_out) < z(X_in) makes MaxLogit separate by construction. The independent content of the 'new insight on MaxLogit' therefore rests entirely on the realism of Definition 3, which is not empirically validated beyond the qualitative observation in Figure 1(b). The theoretical claims should be reframed as a conditional analysis rather than a demonstrated property of MVOL-trained models.
minor comments (5)
- [Preliminaries, first paragraph] The text reads 'tow-layer convolutional network'; this should be 'two-layer convolutional network'.
- [Method, Eq. (7)] Equation (7) contains an unmatched parenthesis in 'L(F(t)' and should be 'L(F(t))'.
- [Appendix E, Training Details] The stated momentum value of 0.09 is likely a typo for 0.9, the standard Nesterov momentum value; please verify and correct.
- [Appendix E, Table 3] For CIFAR-100 the chosen epsilon=0.020 does not give the best FPR95 in the sensitivity table (epsilon=0.015 yields 33.70 vs 33.90); please justify the choice or note that the difference is within run-to-run variance.
- [Results, Table 1 discussion] The claim that MVOL is 'overall stronger' should be qualified: in the ensemble-distillation CIFAR-10 row MVOL's AUROC (98.61) is marginally below OE+MaxLogit (98.62), even though FPR95 improves; this is acceptable but the statement should be precise.
Circularity Check
MaxLogit justification is a corollary of Definition 3 and Assumption 1; MVOL's empirical claims remain independent.
-
self definitional
[Method, Extended Multi-view Data Model (Definition 3); New Insight on MaxLogit as an OOD Score; Appendix B, Main Analysis, Proposition 2, Assumption 1, Theorems 1-2]
"Suppose Induction Hypothesis 3 holds, based on the definition of Dout, Din s and Din m, we know that Ω(1) ≤ z(X out) ≤ 0.8, 1 ≤ z(X in s ) ≤ O(1) and 2 ≤ z(X in m ) ≤ O(1). Therefore, we can get z(X out) < z(X in s ) and z(X out) < z(X in m ). When Assumption 1 is satisfied, FI(X out)(X out) = ... Combined with Proposition 2, we can get FI(X out)(X out) < FI(X ins )(X in s ), ... which supports the ID/OOD discriminability of MaxLogit."
The claimed theoretical basis for MaxLogit is encoded in the data definitions before any logit is analyzed. Definition 3 caps OOD feature-coefficient sums at 0.4 while Definition 1 gives ID main-feature sums at least 1 (and 2 for multi-view), so the inequality z(X_out) < z(X_in) is true by construction. Assumption 1 merely translates those coefficient inequalities into equal per-feature logit weights, and the proof text explicitly says the separation follows 'based on the definition of Dout, Din s and Din m'. The FNR bounds in Theorems 1 and 2 are arithmetic combinations of the same chosen thresholds with the mixing proportion µ.
full rationale
The paper contains no load-bearing self-citations: the MVDM framework is imported from Allen-Zhu and Li 2023, an external reference, and none of the present authors' prior work is cited for the central mechanism. The main circularity is theoretical: the MaxLogit separation result is not derived from an independently established property of trained networks, but is built into Definition 3 (OOD has minor-feature coefficient sums in [Ω(1),0.4]) and the idealized equal-weight classifier of Assumptions 1-2. Moreover, the proofs also rely on an unproven Induction Hypothesis 3, so the chain from definitions to logit separation is explicitly assumed rather than established. This lowers the value of the theoretical 'insight' but does not by itself invalidate the paper's empirical contribution: the MVOL objective in Eq. (7), the sensitivity analysis for ε, and the Tables 1-2 comparisons are independent of the circular theorem and provide genuine evidence for the central claim that MVOL handles auxiliary and wild datasets well. Score 6 reflects that a central theoretical 'prediction' reduces to its own definitions, while the empirical core of the paper remains non-circular.
Assumptions & free parameters
free parameters (2)
- epsilon (logit cap threshold) =
0.1 (CIFAR-10), 0.02 (CIFAR-100)
- minor feature coefficient upper bound in Dout =
0.4
assumptions (3)
- ad hoc to paper Definition 3: OOD samples consist only of minor ID features with coefficient sums in [Omega(1), 0.4] plus noise.
- ad hoc to paper Assumption 1 and Assumption 2: an ideal classifier gives equal total response to all learned ID features up to o(1).
- domain assumption Allen-Zhu and Li (2023) MVDM feature-learning dynamics, including their induction hypotheses, remain valid in the OOD training setting.
Cite this review
Pith. "Pith review of Mining In-distribution Attributes in Outliers for Out-of-distribution Detection." pith.science (2026). https://pith.science/paper/G2HP6KCK
@misc{pith2026241211466,
author = {Pith},
title = {Pith review of: Mining In-distribution Attributes in Outliers for Out-of-distribution Detection},
year = {2026},
howpublished = {\url{https://pith.science/paper/G2HP6KCK}},
note = {Machine review of arXiv:2412.11466}
}
read the original abstract
Out-of-distribution (OOD) detection is indispensable for deploying reliable machine learning systems in real-world scenarios. Recent works, using auxiliary outliers in training, have shown good potential. However, they seldom concern the intrinsic correlations between in-distribution (ID) and OOD data. In this work, we discover an obvious correlation that OOD data usually possesses significant ID attributes. These attributes should be factored into the training process, rather than blindly suppressed as in previous approaches. Based on this insight, we propose a structured multi-view-based out-of-distribution detection learning (MVOL) framework, which facilitates rational handling of the intrinsic in-distribution attributes in outliers. We provide theoretical insights on the effectiveness of MVOL for OOD detection. Extensive experiments demonstrate the superiority of our framework to others. MVOL effectively utilizes both auxiliary OOD datasets and even wild datasets with noisy in-distribution data. Code is available at https://github.com/UESTC-nnLab/MVOL.
Figures
Figures from the paper (3 more)
Reference graph
Works this paper leans on
-
[1]
Allen-Zhu, Z.; and Li, Y. 2023. Towards Understanding Ensemble, Knowledge Distillation and Self-Distillation in Deep Learning. In The Eleventh International Conference on Learning Representations
work page 2023
-
[2]
Bai, H.; Canal, G.; Du, X.; Kwon, J.; Nowak, R. D.; and Li, Y. 2023. Feed two birds with one scone: Exploiting wild data for both out-of-distribution generalization and detection. In International Conference on Machine Learning, 1454--1471
work page 2023
-
[3]
Chen, J.; Li, Y.; Wu, X.; Liang, Y.; and Jha, S. 2021. ATOM : Robustifying Out-of-Distribution Detection Using Outlier Mining. In Proceedings of European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML-PKDD), 430--445
work page 2021
-
[4]
Cimpoi, M.; Maji, S.; Kokkinos, I.; Mohamed, S.; and Vedaldi, A. 2014. Describing textures in the wild. In Proceedings of the IEEE conference on computer vision and pattern recognition, 3606--3613
2014
-
[5]
Djurisic, A.; Bozanic, N.; Ashok, A.; and Liu, R. 2023. Extremely Simple Activation Shaping for Out-of-Distribution Detection. In The Eleventh International Conference on Learning Representations
work page 2023
-
[6]
Du, X.; Sun, Y.; Zhu, J.; and Li, Y. 2023. Dream the Impossible: Outlier Imagination with Diffusion Models. In Advances in Neural Information Processing Systems, volume 36, 60878--60901. Curran Associates, Inc
work page 2023
-
[7]
Du, X.; Wang, Z.; Cai, M.; and Li, Y. 2022. VOS: Learning What You Don't Know by Virtual Outlier Synthesis. In International Conference on Learning Representations
work page 2022
-
[8]
Duchi, J.; Hazan, E.; and Singer, Y. 2011. Adaptive subgradient methods for online learning and stochastic optimization. Journal of machine learning research, 12(7)
2011
Show all 33 references
-
[9]
Hendrycks, D.; Basart, S.; Mazeika, M.; Zou, A.; Kwon, J.; Mostajabi, M.; Steinhardt, J.; and Song, D. 2022. Scaling Out-of-Distribution Detection for Real-World Settings. In Proceedings of the 39th International Conference on Machine Learning, volume 162, 8759--8773
2022
-
[10]
Hendrycks, D.; and Gimpel, K. 2017. A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks. In International Conference on Learning Representations
2017
-
[11]
Hendrycks, D.; Mazeika, M.; and Dietterich, T. 2019. Deep Anomaly Detection with Outlier Exposure. In International Conference on Learning Representations
2019
-
[12]
Hinton, G.; Vinyals, O.; and Dean, J. 2015. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531
2015 arXiv
-
[13]
B.; Nowak, R.; and Li, Y
Katz-Samuels, J.; Nakhleh, J. B.; Nowak, R.; and Li, Y. 2022. Training OOD Detectors in their Natural Habitats. In Proceedings of the 39th International Conference on Machine Learning, volume 162, 10848--10865
2022
-
[14]
Lee, K.; Lee, H.; Lee, K.; and Shin, J. 2018 a . Training Confidence-calibrated Classifiers for Detecting Out-of-Distribution Samples. In International Conference on Learning Representations
2018
-
[15]
Lee, K.; Lee, K.; Lee, H.; and Shin, J. 2018 b . A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks. In Advances in Neural Information Processing Systems, volume 31
2018
-
[16]
Liang, S.; Li, Y.; and Srikant, R. 2018. Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks. In International Conference on Learning Representations
2018
-
[17]
Liu, W.; Wang, X.; Owens, J.; and Li, Y. 2020. Energy-based out-of-distribution detection. Advances in neural information processing systems, 33: 21464--21475
2020
-
[18]
Ming, Y.; Fan, Y.; and Li, Y. 2022. POEM : Out-of-Distribution Detection with Posterior Sampling. In Proceedings of International Conference on Machine Learning (ICML), 15650--15665
2022
-
[19]
Y.; et al
Netzer, Y.; Wang, T.; Coates, A.; Bissacco, A.; Wu, B.; Ng, A. Y.; et al. 2011. Reading digits in natural images with unsupervised feature learning. In NIPS workshop on deep learning and unsupervised feature learning, volume 2011, 7
2011
-
[20]
Nguyen, A.; Yosinski, J.; and Clune, J. 2015. Deep Neural Networks Are Easily Fooled: High Confidence Predictions for Unrecognizable Images. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
2015
-
[21]
Tao, L.; Du, X.; Zhu, J.; and Li, Y. 2023. Non-parametric Outlier Synthesis. In The Eleventh International Conference on Learning Representations
2023
-
[22]
Vaze, S.; Han, K.; Vedaldi, A.; and Zisserman, A. 2022. Open-Set Recognition: A Good Closed-Set Classifier is All You Need. In International Conference on Learning Representations
2022
-
[23]
Wei, H.; Xie, R.; Cheng, H.; Feng, L.; An, B.; and Li, Y. 2022. Mitigating Neural Network Overconfidence with Logit Normalization. In Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, 23631--23644
2022
-
[24]
A.; Zhang, Y.; Finkelstein, A.; Kulkarni, S
Xu, P.; Ehinger, K. A.; Zhang, Y.; Finkelstein, A.; Kulkarni, S. R.; and Xiao, J. 2015. Turkergaze: Crowdsourcing saliency with webcam based eye tracking. arXiv preprint arXiv:1504.06755
2015 arXiv
-
[25]
Yang, J.; Wang, H.; Feng, L.; Yan, X.; Zheng, H.; Zhang, W.; and Liu, Z. 2021. Semantically Coherent Out-of-Distribution Detection. In 2021 IEEE/CVF International Conference on Computer Vision, ICCV 2021, Montreal, QC, Canada, October 10-17, 2021 , 8281--8289
2021
-
[26]
Yu, F.; Seff, A.; Zhang, Y.; Song, S.; Funkhouser, T.; and Xiao, J. 2015. Lsun: Construction of a large-scale image dataset using deep learning with humans in the loop. arXiv preprint arXiv:1506.03365
2015 arXiv
-
[27]
Zagoruyko, S.; and Komodakis, N. 2016. Wide Residual Networks. In British Machine Vision Conference 2016. British Machine Vision Association
2016
-
[28]
Zheng, H.; Wang, Q.; Fang, Z.; Xia, X.; Liu, F.; Liu, T.; and Han, B. 2023. Out-of-distribution detection learning with unreliable out-of-distribution sources. In Advances in Neural Information Processing Systems, volume 36, 72110--72123
2023
-
[29]
Zhou, B.; Lapedriza, A.; Khosla, A.; Oliva, A.; and Torralba, A. 2017. Places: A 10 million image database for scene recognition. IEEE transactions on pattern analysis and machine intelligence, 40(6): 1452--1464
2017
-
[30]
Zhu, F.; Cheng, Z.; Zhang, X.-Y.; and Liu, C.-L. 2023 a . Openmix: Exploring outlier samples for misclassification detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 12074--12083
2023
-
[31]
Zhu, J.; Geng, Y.; Yao, J.; Liu, T.; Niu, G.; Sugiyama, M.; and Han, B. 2023 b . Diversified Outlier Exposure for Out-of-Distribution Detection via Informative Extrapolation. In Advances in Neural Information Processing Systems, volume 36, 22702--22734
2023
-
[32]
, " * write output.state after.block = add.period write newline
ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all...
-
[33]
write newline
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.