Entropic Out-of-Distribution Detection

Adriano L. I. Oliveira; Cleber Zanchettin; David Mac\^edo; Teresa Ludermir; Tsang Ing Ren

REVIEW 4 major objections 5 minor 46 references

Entropic Out-of-Distribution Detection

T0 review · 4 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that replacing the SoftMax loss with an isotropic distance-based IsoMax loss, scored by output entropy, detects out-of-distribution inputs with no accuracy drop, no extra data, and no hyperparameter tuning.

desk verdict Useful new OOD baseline, but the 'no tuning' claim rests on an entropic scale that was selected with OOD data in view. read the letter →

arxiv 1908.05569 v13 pith:BWV2P2T5 submitted 2019-08-15 cs.LG cs.CVstat.ML

David Macêdo , Tsang Ing Ren , Cleber Zanchettin , Adriano L. I. Oliveira , Teresa Ludermir This is my paper · ORCID

classification cs.LGcs.CVstat.ML

keywords out-of-distributiondetectionIsoMaxlossentropicscoreentropymaximizationtrickmaximumprincipleSoftanisotropyseamlessOODdistance-basedclassification

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that poor out-of-distribution (OOD) detection in deep classifiers is not a limitation of the models themselves but a side effect of the SoftMax loss, which produces anisotropic features and overconfident low-entropy outputs. It proposes replacing SoftMax with an IsoMax loss that depends only on distances to learned class prototypes, plus an entropic score that measures the negative entropy of the output probabilities. Under a self-imposed 'seamless' constraint—no accuracy drop, no outlier data, no hyperparameter tuning, and inference as fast as a normal forward pass—the paper reports large OOD detection gains on CIFAR10, CIFAR100, and SVHN with both DenseNet and ResNet. If the claim holds, it means a reliability improvement is available to any existing classifier at essentially zero cost. The key move is the entropy maximization trick: training with logits multiplied by a constant entropic scale that is removed before inference, so outputs stay high-entropy and therefore easier to separate from unknown inputs.

What carries the argument

The central object is the IsoMax loss, an isotropic, distance-only cross-entropy loss. With $\|\cdot\|$ the Euclidean norm, $f_\theta(x)$ the embedding, and $p_\varphi^j$ the learnable prototype of class $j$, the training loss is $L_I(\hat y^{(k)}|x) = -\log \frac{\exp(-E_s \|f_\theta(x)-p_\varphi^k\|)}{\sum_j \exp(-E_s \|f_\theta(x)-p_\varphi^j\|)}$, where the entropic scale $E_s=10$ is applied during training and removed at inference. Removal turns the output distribution into the high-entropy form $\frac{\exp(-\|f_\theta(x)-p_\varphi^i\|)}{\sum_j \exp(-\|f_\theta(x)-p_\varphi^j\|)}$, and the negative entropy of that distribution is the entropic score used for detection. The mechanism aligns feature geometry with classification geometry and keeps posterior entropy near its maximum $\log N$, so unknown inputs land far from the prototypes in a predictable way.

What would settle it

Train identical architectures on a held-out dataset with SoftMax and IsoMax at $E_s=10$ and compare TNR@TPR95 on an OOD set the paper did not use; the paper predicts a consistent large gain, so a vanishing gain or an in-distribution mean entropy far below $\log N$ would falsify the transfer claim.

Watch

Extended reading notes

Core claim

The central claim is that IsoMax+ES—training with the IsoMax loss and scoring with the negative entropy of the output distribution—achieves state-of-the-art out-of-distribution detection under a set of constraints no current alternative meets at once: classification accuracy equal to SoftMax-trained models, no outlier or background data, no hyperparameter validation, and single-pass fast inference. In the paper's experiments on CIFAR10, CIFAR100, and SVHN with DenseNet and ResNet, replacing SoftMax with IsoMax consistently raises TNR@TPR95 and AUROC across every OOD set, with the largest gains coming from high-entropy training rather than from isotropy alone. The paper takes this as evidence that SoftMax loss anisotropy and overconfidence are the root causes of weak OOD detection, and that a distance-based loss producing near-maximum-entropy posteriors is the corrective.

Load-bearing premise

The fixed entropic scale $E_s=10$, chosen after observing OOD detection results on SVHN, is assumed to transfer to all datasets and models without further validation; if a different $E_s$ is needed on a new dataset or architecture, the promised absence of hyperparameter validation fails.

Editorial extensions

If this is right

A SoftMax-trained classifier can be turned into an OOD-aware one by retraining with IsoMax and scoring with the entropic score, with no architectural change and no extra inference cost.
The seamless task becomes a practical evaluation mode: OOD methods can be compared without accuracy-drop compensation, outlier data, or tuning-dependent results.
Non-seamless techniques such as input preprocessing, outlier exposure, and energy scores can be composed with IsoMax as a stronger starting point instead of SoftMax.
The observed correlation between higher posterior entropy and better OOD detection suggests that any training change pushing output entropy up without hurting accuracy is a candidate OOD improvement.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Editorial inference: the fixed $E_s=10$ may need to grow with class count $N$, since the paper's saturation argument is demonstrated on small datasets; testing $E_s$ values on a 1000-class task would show whether the no-tuning property survives scale-up.
Editorial inference: because the loss is purely distance-based, the same prototype geometry could anchor open-set recognition or few-shot classifiers, tasks the paper does not study.
Editorial inference: the entropy maximization trick is tied to removing the scale at inference, and the paper reports difficulties with SoftMax and cosine logits; identifying which isotropic loss families tolerate the trick is an open follow-up.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Dear [colleague],

The headline: this is a genuinely useful baseline for OOD detection, but the 'seamless, no tuning' claim isn't fully earned because the entropic scale was chosen after looking at OOD results. The core idea is neat: replace the affine softmax logits with distances to trainable prototypes, and train with a large logit scale that is removed at inference, leaving a near-maximum-entropy distribution. As far as I know, that specific combination is new. The experiments are broad: two architectures, three training sets, three OOD sources, three metrics, plus a clean ablation separating isotropy from the entropy trick. The authors also honestly label the non-seamless comparison as unfair, and they release code. That part is all solid.

The main soft spot is exactly the one the stress-test note flags. Section V-A says they picked Es=10 because higher entropic scales improve OOD detection in Fig 2e; Section V-B then says validation of Es is 'not even possible' because OOD access is forbidden. That's inconsistent. The mitigating point is that Fig 3 shows performance is flat for Es >= 10 across their dataset/model combinations, so the chosen value isn't a tight fit. But those are the same OOD sources used in the main tables, not held-out data. If someone applies IsoMax to a new architecture or domain, we don't yet know Es=10 will work without checking. So the 'no hyperparameter validation' property is plausible, not demonstrated.

Second, there are no error bars or multiple seeds anywhere. Their margin over Mahalanobis is often under 2%, so the state-of-the-art claim would be more convincing with variance estimates. That said, for the purpose of showing a cheap improvement over the softmax baseline, the lack of error bars is a minor issue.

Third, the conclusion leans on a causal claim that 'low OOD detection performance ... is due to the SoftMax loss drawbacks.' The ablation supports a correlation, not causation; and 'does not present any drawbacks compared to the SoftMax loss' is stronger than what is shown. The maximum-entropy explanation is a plausible story, not a derivation.

Who is this for? Practitioners working on OOD detection. It's a free improvement in a common setting and deserves a serious referee. My recommendation: engage with it, but ask for a held-out check where Es=10 is fixed on one dataset and then applied without modification to new datasets or architectures, plus at least a few seeds to show the differences are stable.

Referee Report

4 major / 5 minor

Summary. The paper proposes replacing the SoftMax loss with the IsoMax loss, an isotropic distance-based loss with class prototypes, combined with an 'entropic scale' that is used during training and removed at inference, and an 'entropic score' (negative output entropy) for out-of-distribution detection. The authors claim this yields a seamless OOD detection method: no classification accuracy drop, no outlier data collection, no hyperparameter validation, and fast inference. They report experiments on CIFAR10, CIFAR100, and SVHN with DenseNet and ResNet architectures, showing improved OOD detection over SoftMax baselines in Table II and competitive or better numbers against non-seamless methods in Table III.

Significance. If the seamless claim holds, the IsoMax loss would be a valuable drop-in replacement giving free reliability improvements. The paper's strengths include a clear and practically motivated problem definition, a simple and reproducible method with released source code, comprehensive tables covering multiple datasets, architectures, OOD sources, and metrics, and an explicit separation between seamless and non-seamless evaluation. The reliance on a single entropic scale value selected with access to OOD data is, however, a load-bearing issue for the 'no hyperparameter validation' property, and the absence of error bars makes several numerical comparisons difficult to interpret.

major comments (4)

[V-A and V-B] The selection of Es=10 is performed using OOD detection results: Section V-A states that Fig. 2e, obtained with a DenseNet trained on SVHN and evaluated on the same OOD sets used later in Table II (CIFAR10, TinyImageNet, LSUN), shows that higher entropic scales improve OOD detection, and the authors 'decided to use Es=10.' Section V-B then claims that validation of Es is 'not even possible because access to the OOD or outlier samples is not allowed in seamless OOD detection.' These statements are internally inconsistent. The seamless property, which is a central claim of the paper, requires that Es be set without OOD data; the current evidence only shows that Es=10 works on the datasets and OOD sources used for its selection.
[V-A, Fig. 3] The stability analysis in Fig. 3, which reports that OOD performance is approximately stable for Es>=10 regardless of dataset and model, is performed on the same datasets and OOD sources that motivated the choice of Es=10. This is not a truly held-out test of the transfer of Es=10 to new datasets, architectures, or OOD sources. If Es must be tuned per dataset or architecture, the 'no hyperparameter validation' claim fails; if it transfers, the paper provides no evidence beyond the already-seen configurations. The authors should either provide a genuinely held-out validation (e.g., a dataset or model not used in any earlier figure or table) or explicitly soften the seamless claim.
[Tables II and III] All reported metrics appear to come from single training runs, with no error bars, standard deviations, or multiple seeds. This matters concretely: in Table III, the differences between IsoMax+ES and Mahalanobis are less than 2 percentage points in most rows (e.g., DenseNet CIFAR10/SVHN AUROC 96.6 vs 97.6; ResNet CIFAR100/SVHN AUROC 85.3 vs 84.4), and in several rows IsoMax+ES is lower. Without variance estimates or significance testing, the claims that IsoMax+ES 'significantly improves' baseline performance and achieves state-of-the-art seamless results are not fully supported. The authors should report means and standard deviations over at least a few seeds, or otherwise justify why single runs suffice.
[V-D, Table III footnote] The footnote to Table III acknowledges that validating Es using adversarial examples cannot produce significant gains and therefore the authors 'prefer to keep Es=10.' This is an admission that Es is treated as a hyperparameter that could in principle be validated. It strengthens the concern in the first major comment: the paper's own description of non-seamless methods repeatedly criticizes hyperparameter validation, yet the proposed method's single free parameter is selected using the test OOD sources. The distinction between seamless and non-seamless evaluation should be redrawn to make clear what is actually validated and on what data.

minor comments (5)

[Fig. 2 caption] The phrase 'regardless of the out-of-distribution (out-dist)' is informal; consider spelling out 'out-distribution' and clarifying the caption's sentence structure.
[Table III caption] The entry 'NA' for ACET is not explained in the caption or the text. Please state that ACET was not evaluated on DenseNet, or provide the reason.
[Section IV] The training details list dropout, weight decay, learning rate schedule, but do not specify whether standard data augmentation (e.g., random crops and flips for CIFAR) is used. This information is important for reproducibility.
[Table IV] The caption and text refer to 'presumed computational cost and energy consumption rates,' but only inference delays are measured. The energy-consumption claim is inference and not direct measurement; please either measure energy or explicitly label it as an assumption.
[Section III-C] The choice to initialize all prototypes to the zero vector is stated as an observation without analysis. Adding a sentence on why zero initialization avoids the 'oscillations in performance' seen with Xavier or Kaiming initializations would help readers understand the stability behavior.

Circularity Check

1 steps flagged · score 4.0 of 10

Entropic scale Es=10 is selected using OOD detection results, so the 'no hyperparameter validation' seamless claim is not established; otherwise the empirical comparison is self-contained.

fitted input called prediction [Section V-A, Fig. 2e, Fig. 3 caption; Section V-B]
"Therefore, we decided to use Es= 10 for IsoMax loss (see also Fig. 3). ... Es validation cannot significantly improve the OOD detection performance. In fact, this is not even possible because access to the OOD or outlier samples is not allowed in seamless OOD detection."

The entropic scale is not derived from the maximum-entropy principle; it is chosen after observing OOD detection performance. Section V-A states that the authors trained DenseNets on SVHN with 'distinct entropic scale values' and used 'the entropic score and the TNR@TPR95' to evaluate OOD detection (Fig. 2), then 'decided to use Es=10' because higher Es improved OOD detection on the same OOD sets (CIFAR10, TinyImageNet, LSUN) that later appear in Table II.

full rationale

The paper's core contribution is empirical and mostly self-contained: IsoMax loss is defined by replacing affine logits with distances to prototypes (Eq. 2), the entropic score is the negative entropy of the output distribution, and the comparison against SoftMax baselines is a fair, reproducible experiment. There is no self-citation chain, no uniqueness theorem imported from the authors, and no ansatz smuggled through prior work. The only significant circularity is the handling of Es: it is selected using OOD detection performance on a subset of the data (DenseNet/SVHN against CIFAR10, TinyImageNet, and LSUN), and then the method is advertised as requiring 'no hyperparameter validation.' This does not make the measured improvement of IsoMax+ES over SoftMax+ES vanish, but it does weaken the 'seamless' claim because one hyperparameter was tuned toward the target metric. The stability analysis in Fig. 3 is on overlapping datasets and OOD sources, not on truly held-out combinations. Overall score 4: the central empirical finding has independent content, but one load-bearing parameter is fitted to the target task while being presented as hyperparameter-free.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The central claim depends on a handful of domain assumptions: the maximum entropy principle as a design goal, the causal role of SoftMax anisotropy, the choice of Euclidean distance, and the empirical initialization trick. The only fitted numeric free parameter is the entropic scale, whose selection is entangled with OOD test data.

free parameters (1)

Entropic scale Es = 10
Chosen in Section V-A from an ablation on SVHN with DenseNet showing that Es at least 10 yields near-maximum entropy and stable OOD performance; this selection used OOD evaluation data, then is presented as a fixed constant requiring no validation.

assumptions (4)

domain assumption Output probability distributions should obey the maximum entropy principle, i.e., be as high-entropy as possible given constraints.
Motivates the entropy maximization trick in Sections I and III-B; no formal derivation that high entropy improves OOD detection.
domain assumption SoftMax loss anisotropy and overconfidence are the main causes of poor OOD detection.
Core causal claim in Section I, supported by literature pointers and the paper's own experiments, not by a proof.
domain assumption Nonsquared Euclidean distance is the appropriate metric for the IsoMax loss.
Section III-E states the covariance matrix of the Mahalanobis distance is not differentiable, so nonsquared Euclidean is called optimal; this rules out alternatives by fiat.
ad hoc to paper Zero initialization of class prototypes with weight decay stabilizes training.
Section III-C; observed empirically to avoid oscillations, no theoretical justification.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Entropic Out-of-Distribution Detection." pith.science (2026). https://pith.science/paper/BWV2P2T5

@misc{pith2026190805569,
  author       = {Pith},
  title        = {Pith review of: Entropic Out-of-Distribution Detection},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BWV2P2T5}},
  note         = {Machine review of arXiv:1908.05569}
}

read the original abstract

Out-of-distribution (OOD) detection approaches usually present special requirements (e.g., hyperparameter validation, collection of outlier data) and produce side effects (e.g., classification accuracy drop, slower energy-inefficient inferences). We argue that these issues are a consequence of the SoftMax loss anisotropy and disagreement with the maximum entropy principle. Thus, we propose the IsoMax loss and the entropic score. The seamless drop-in replacement of the SoftMax loss by IsoMax loss requires neither additional data collection nor hyperparameter validation. The trained models do not exhibit classification accuracy drop and produce fast energy-efficient inferences. Moreover, our experiments show that training neural networks with IsoMax loss significantly improves their OOD detection performance. The IsoMax loss exhibits state-of-the-art performance under the mentioned conditions (fast energy-efficient inference, no classification accuracy drop, no collection of outlier data, and no hyperparameter validation), which we call the seamless OOD detection task. In future work, current OOD detection methods may replace the SoftMax loss with the IsoMax loss to improve their performance on the commonly studied non-seamless OOD detection problem.

Figures

Figures reproduced from arXiv: 1908.05569 by the authors.

**Figure 2.** (a) SoftMax loss minimizes both the cross-entropy and the mean entropy of the posterior probabilities. (b) IsoMax loss produces low mean entropy [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

**Figure 3.** AUROC represents the mean AUROC considering all out-of-distribution data. The classification accuracy and the mean OOD detection performance are approximately stable for Es= 10 or higher regardless of the dataset and model. Es validation cannot significantly improve the OOD detection performance. In fact, this is not even possible because access to the OOD or outlier samples is not allowed in seamless OOD detection.… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

46 extracted references · 41 canonical work pages

[1]

A discriminative feature learning approach for deep face recognition,

Y . Wen, K. Zhang, Z. Li, and Y . Qiao, “A discriminative feature learning approach for deep face recognition,” European Conference on Computer Vision, 2016

work page 2016
[2]

A simple uniﬁed framework for detecting out-of-distribution samples and adversarial attacks,

K. Lee, K. Lee, H. Lee, and J. Shin, “A simple uniﬁed framework for detecting out-of-distribution samples and adversarial attacks,” Neural Information Processing Systems , 2018

work page 2018
[3]

Distance- based image classiﬁcation: Generalizing to new classes at near-zero cost,

T. Mensink, J. J. Verbeek, F. Perronnin, and G. Csurka, “Distance- based image classiﬁcation: Generalizing to new classes at near-zero cost,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 35, no. 11, pp. 2624–2637, 2013

work page 2013
[4]

Towards open set recognition,

W. J. Scheirer, A. Rocha, A. Sapkota, and T. E. Boult, “Towards open set recognition,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 35, no. 7, pp. 1757–1772, 2013

work page 2013
[5]

Probability models for open set recognition,

W. J. Scheirer, L. P. Jain, and T. E. Boult, “Probability models for open set recognition,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 36, no. 11, pp. 2317–2324, 2014

work page 2014
[6]

Towards open world recognition,

A. Bendale and T. Boult, “Towards open world recognition,” IEEE International Conference on Computer Vision and Pattern Recognition , 2015

work page 2015
[7]

The extreme value machine,

E. Rudd, L. P. Jain, W. J. Scheirer, and T. Boult, “The extreme value machine,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 40, no. 3, pp. 762–768, 2018

work page 2018
[8]

Learning conﬁdence for out-of-distribution detection in neural networks,

T. DeVries and G. W. Taylor, “Learning conﬁdence for out-of-distribution detection in neural networks,” CoRR, vol. abs/1802.04865, 2018

arXiv 2018

Show all 46 references

[9]

Generalized ODIN: Detect- ing out-of-distribution image without learning from out-of-distribution data,

Y .-C. Hsu, Y . Shen, H. Jin, and Z. Kira, “Generalized ODIN: Detect- ing out-of-distribution image without learning from out-of-distribution data,” IEEE International Conference on Computer Vision and Pattern Recognition, 2020

2020
[10]

Why ReLU networks yield high-conﬁdence predictions far away from the training data and how to mitigate the problem,

M. Hein, M. Andriushchenko, and J. Bitterwolf, “Why ReLU networks yield high-conﬁdence predictions far away from the training data and how to mitigate the problem,” IEEE International Conference on Computer Vision and Pattern Recognition , 2018

2018
[11]

Enhancing the reliability of out-of- distribution image detection in neural networks,

S. Liang, Y . Li, and R. Srikant, “Enhancing the reliability of out-of- distribution image detection in neural networks,”International Conference on Learning Representations , 2018

2018
[12]

Reducing network agnosto- phobia,

A. R. Dhamija, M. Günther, and T. Boult, “Reducing network agnosto- phobia,” Neural Information Processing Systems , 2018

2018
[13]

Deep anomaly detection with outlier exposure,

D. Hendrycks, M. Mazeika, and T. Dietterich, “Deep anomaly detection with outlier exposure,” International Conference on Learning Represen- tations, 2019

2019
[14]

Outlier exposure with conﬁdence control for out-of-distribution detection,

A.-A. Papadopoulos, M. R. Rajati, N. Shaikh, and J. Wang, “Outlier exposure with conﬁdence control for out-of-distribution detection,” CoRR, vol. abs/1906.03509, 2019

1906 arXiv
[15]

Energy-based out-of- distribution detection,

W. Liu, X. Wang, J. D. Owens, and Y . Li, “Energy-based out-of- distribution detection,” CoRR, vol. abs/2010.03759, 2020

2010 arXiv
[16]

Hyperparameter-free out-of-distribution detection using cosine similarity,

E. Techapanurak, M. Suganuma, and T. Okatani, “Hyperparameter-free out-of-distribution detection using cosine similarity,” Proceedings of the Asian Conference on Computer Vision (ACCV) , November 2020

2020
[17]

What uncertainties do we need in bayesian deep learning for computer vision?,

A. Kendall and Y . Gal, “What uncertainties do we need in bayesian deep learning for computer vision?,” Neural Information Processing Systems , 2017

2017
[18]

Leveraging uncertainty information from deep neural networks for disease detection,

C. Leibig, V . Allken, M. S. Ayhan, P. Berens, and S. Wahl, “Leveraging uncertainty information from deep neural networks for disease detection,” Scientiﬁc Reports, vol. 7, 2017

2017
[19]

Conﬁdence estimation in deep neural networks via density modelling,

A. Subramanya, S. Srinivas, and R. V . Babu, “Conﬁdence estimation in deep neural networks via density modelling,” CoRR, vol. abs/1707.07013, 2017

2017 arXiv
[20]

Predictive uncertainty estimation via prior networks,

A. Malinin and M. Gales, “Predictive uncertainty estimation via prior networks,” Neural Information Processing Systems , 2018

2018
[21]

Accurate uncertainties for deep learning using calibrated regression,

V . Kuleshov, N. Fenner, and S. Ermon, “Accurate uncertainties for deep learning using calibrated regression,” International Conference on Machine Learning, 2018

2018
[22]

Large-margin softmax loss for convolutional neural networks.,

W. Liu, Y . Wen, Z. Yu, and M. Yang, “Large-margin softmax loss for convolutional neural networks.,” International Conference on Machine Learning, 2016

2016
[23]

On calibration of modern neural networks,

C. Guo, G. Pleiss, Y . Sun, and K. Q. Weinberger, “On calibration of modern neural networks,” International Conference on Machine Learning, 2017

2017
[24]

Information theory and statistical mechanics,

E. T. Jaynes, “Information theory and statistical mechanics,” Physical Review, vol. 106, pp. 620–630, 1957

1957
[25]

Information theory and statistical mechanics. II,

E. T. Jaynes, “Information theory and statistical mechanics. II,” Physical Review, vol. 108, pp. 171–190, 1957

1957
[26]

Elements of information theory,

T. M. Cover and J. A. Thomas, “Elements of information theory,” Wiley Series in Telecommunications and Signal Processing , 2006

2006
[27]

A less biased evaluation of out-of-distribution sample detectors,

A. Shafaei, M. Schmidt, and J. J. Little, “A less biased evaluation of out-of-distribution sample detectors,” British Machine Vision Conference, 2019

2019
[28]

Understanding the difﬁculty of training deep feedforward neural networks,

X. Glorot and Y . Bengio, “Understanding the difﬁculty of training deep feedforward neural networks,” International Conference on Artiﬁcial Intelligence and Statistics , 2010

2010
[29]

Delving deep into rectiﬁers: Surpassing human-level performance on imagenet classiﬁcation,

K. He, X. Zhang, S. Ren, and J. Sun, “Delving deep into rectiﬁers: Surpassing human-level performance on imagenet classiﬁcation,” Inter- national Conference on Computer Vision , 2016

2016
[30]

Likelihood ratios for out-of- distribution detection,

J. Ren, P. J. Liu, E. Fertig, J. Snoek, R. Poplin, M. A. DePristo, J. V . Dillon, and B. Lakshminarayanan, “Likelihood ratios for out-of- distribution detection,” Neural Information Processing Systems , 2019

2019
[31]

Additive margin softmax for face veriﬁcation,

F. Wang, J. Cheng, W. Liu, and H. Liu, “Additive margin softmax for face veriﬁcation,” IEEE Signal Processing Letters, vol. 25, no. 7, pp. 926–930, 2018

2018
[32]

ArcFace: Additive angular margin loss for deep face recognition,

J. Deng, J. Guo, N. Xue, and S. Zafeiriou, “ArcFace: Additive angular margin loss for deep face recognition,” IEEE International Conference on Computer Vision and Pattern Recognition , 2019

2019
[33]

ImageNet: A large-scale hierarchical image database,

J. Deng, W. Dong, R. Socher, L. Li, K. Li, and F. Li, “ImageNet: A large-scale hierarchical image database,” IEEE International Conference on Computer Vision and Pattern Recognition , 2009

2009
[34]

LSUN: construction of a large-scale image dataset using deep learning with humans in the loop,

F. Yu, Y . Zhang, S. Song, A. Seff, and J. Xiao, “LSUN: construction of a large-scale image dataset using deep learning with humans in the loop,” CoRR, vol. abs/1506.03365, 2015

2015 arXiv
[35]

A baseline for detecting misclassiﬁed and out-of-distribution examples in neural networks,

D. Hendrycks and K. Gimpel, “A baseline for detecting misclassiﬁed and out-of-distribution examples in neural networks,” International Conference on Learning Representations , 2017

2017
[36]

Densely connected convolutional networks,

G. Huang, Z. Liu, L. v. d. Maaten, and K. Q. Weinberger, “Densely connected convolutional networks,” IEEE International Conference on Computer Vision and Pattern Recognition , 2017

2017
[37]

Identity mappings in deep residual networks,

K. He, X. Zhang, S. Ren, and J. Sun, “Identity mappings in deep residual networks,” European Conference on Computer Vision , 2016

2016
[38]

Learning multiple layers of features from tiny images,

A. Krizhevsky, “Learning multiple layers of features from tiny images,” Science Department, University of Toronto , 2009

2009
[39]

Reading digits in natural images with unsupervised feature learning,

Y . Netzer and T. Wang, “Reading digits in natural images with unsupervised feature learning,” Neural Information Processing Systems , 2011

2011
[40]

Robustness may be at odds with accuracy,

D. Tsipras, S. Santurkar, L. Engstrom, A. Turner, and A. Madry, “Robustness may be at odds with accuracy,” 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019, 2019

2019
[41]

On evaluating adversarial robustness,

N. Carlini, A. Athalye, N. Papernot, W. Brendel, J. Rauber, D. Tsipras, I. J. Goodfellow, A. Madry, and A. Kurakin, “On evaluating adversarial robustness,” CoRR, vol. abs/1902.06705, 2019

1902 arXiv
[42]

Green AI,

R. Schwartz, J. Dodge, N. A. Smith, and O. Etzioni, “Green AI,” CoRR, vol. abs/1907.10597, 2019

1907 arXiv
[43]

Adversarial training for free!,

A. Shafahi, M. Najibi, A. Ghiasi, Z. Xu, J. P. Dickerson, C. Studer, L. S. Davis, G. Taylor, and T. Goldstein, “Adversarial training for free!,” Neural Information Processing Systems , 2019

2019
[44]

Detecting out-of-distribution examples with gram matrices,

C. S. Sastry and S. Oore, “Detecting out-of-distribution examples with gram matrices,” International Conference on Machine Learning , vol. 119, pp. 8491–8501, 2020

2020
[45]

On mixup training: Improved calibration and predictive uncertainty for deep neural networks,

S. Thulasidasan, G. Chennupati, J. A. Bilmes, T. Bhattacharya, and S. Michalak, “On mixup training: Improved calibration and predictive uncertainty for deep neural networks,” Neural Information Processing Systems, 2019

2019
[46]

Cutmix: Regularization strategy to train strong classiﬁers with localizable features,

S. Yun, D. Han, S. J. Oh, S. Chun, J. Choe, and Y . Yoo, “Cutmix: Regularization strategy to train strong classiﬁers with localizable features,” International Conference on Computer Vision , 2019

2019

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [1]

A discriminative feature learning approach for deep face recognition,

Y . Wen, K. Zhang, Z. Li, and Y . Qiao, “A discriminative feature learning approach for deep face recognition,” European Conference on Computer Vision, 2016

work page 2016

[2] [2]

A simple uniﬁed framework for detecting out-of-distribution samples and adversarial attacks,

K. Lee, K. Lee, H. Lee, and J. Shin, “A simple uniﬁed framework for detecting out-of-distribution samples and adversarial attacks,” Neural Information Processing Systems , 2018

work page 2018

[3] [3]

Distance- based image classiﬁcation: Generalizing to new classes at near-zero cost,

T. Mensink, J. J. Verbeek, F. Perronnin, and G. Csurka, “Distance- based image classiﬁcation: Generalizing to new classes at near-zero cost,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 35, no. 11, pp. 2624–2637, 2013

work page 2013

[4] [4]

Towards open set recognition,

W. J. Scheirer, A. Rocha, A. Sapkota, and T. E. Boult, “Towards open set recognition,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 35, no. 7, pp. 1757–1772, 2013

work page 2013

[5] [5]

Probability models for open set recognition,

W. J. Scheirer, L. P. Jain, and T. E. Boult, “Probability models for open set recognition,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 36, no. 11, pp. 2317–2324, 2014

work page 2014

[6] [6]

Towards open world recognition,

A. Bendale and T. Boult, “Towards open world recognition,” IEEE International Conference on Computer Vision and Pattern Recognition , 2015

work page 2015

[7] [7]

The extreme value machine,

E. Rudd, L. P. Jain, W. J. Scheirer, and T. Boult, “The extreme value machine,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 40, no. 3, pp. 762–768, 2018

work page 2018

[8] [8]

Learning conﬁdence for out-of-distribution detection in neural networks,

T. DeVries and G. W. Taylor, “Learning conﬁdence for out-of-distribution detection in neural networks,” CoRR, vol. abs/1802.04865, 2018

arXiv 2018