Pith. sign in

REVIEW 4 major objections 7 minor 43 references

Explorations of the Softmax Space: Knowing When the Neural Network Doesn't Know

T0 review · 4 major / 7 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that the mean softmax vector of a network's correct predictions can serve as a class centroid, and that Euclidean distance from a new prediction to that centroid is a reliable, training-free confidence score for deciding…

desk verdict The abstention claim is evaluated on the training data used to derive centroids and thresholds, so the central result is an in-sample fit; the idea is plausible but needs proper held-out evaluation and baselines. read the letter →

arxiv 2502.00456 v2 pith:M3OQYQ72 submitted 2025-02-01 cs.LG cs.CV

Daniel Sikar , Artur d'Avila Garcez , Tillman Weyde This is my paper · ORCID
classification cs.LGcs.CV
keywords uncertaintyquantificationsoftmaxprobabilityspacedistributionshiftout-of-distributiondetectionconfidenceestimationabstentionK-meansclusteringnot-knownanswers
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a network's own softmax outputs contain enough information to say when the network should not answer. The proposal is simple: for each class, average the softmax vectors of all training examples the network got right, and treat that average as the class centroid. Then measure the Euclidean distance from any new prediction's softmax vector to the centroid of the predicted class. If that distance exceeds a per-class threshold, chosen as the smallest distance seen for a wrong prediction on the training set, the system answers 'not known' instead of committing to a label. If this works, any trained softmax classifier gains an abstention option with no retraining and no extra architecture.

What carries the argument

The central object is the class centroid, computed by Algorithm 1 as the arithmetic mean of all correct-prediction softmax vectors for a given class. The companion object is the threshold, computed by Algorithm 2 as the minimum distance from any incorrect training prediction to the centroid of the class it was assigned to. The method works by treating the K-dimensional softmax output as a point in the probability simplex, where all centroids are valid probability distributions and pairwise distances are bounded by $\sqrt{2}$; Euclidean distance to the predicted-class centroid then functions as the confidence score, and the threshold creates a 'not known' region. The paper also uses shell-density computations in these high-dimensional spaces to characterize how in-distribution points concentrate near centroids while out-of-distribution points populate outer shells.

What would settle it

Train a classifier where one class contains two visually distinct subpopulations, compute the per-class mean centroid and the minimum-incorrect-distance threshold, and measure whether correct predictions from either subpopulation fall beyond the threshold more often than incorrect predictions do; if a substantial share of correct near-mode examples are rejected while incorrect examples are accepted, the central proxy fails.

Watch

Extended reading notes

Core claim

The paper's central claim is that the mean of the softmax probability vectors for all correct predictions of a given class forms a usable centroid for that class in softmax space, and that Euclidean distance from a new prediction's softmax vector to that centroid tracks the network's confidence. Defining a class threshold as the smallest distance from an incorrect training prediction to the class centroid gives a conservative rule: predictions beyond the threshold are rejected and the network returns 'not known'. Across MNIST with a CNN and CIFAR-10 with a Vision Transformer, the authors find that correct predictions cluster tightly near their centroids, centroids are nearly maximally separated in the probability simplex (pairwise distances near $\sqrt{2}$), and out-of-distribution inputs—English handwritten characters and CIFAR-10 images fed to the MNIST model—fall farther from centroids, producing high exclusion rates at small thresholds. The authors conclude that this distance metric is a consistent, lightweight proxy for confidence and a practical basis for deferring low-confidence decisions to human operators.

Load-bearing premise

The load-bearing premise is that each class's softmax outputs cluster tightly and unimodally around their mean, so the arithmetic mean of correct predictions is a representative centroid and Euclidean distance to it is a trustworthy confidence score; the paper does not provide distributional evidence for this across classes.

Editorial extensions

If this is right

  • Any softmax classifier can gain an abstention option by computing per-class centroids and thresholds from training-set predictions, with no retraining or architectural changes.
  • Decreasing the distance threshold trades retention against accuracy: on the reported settings, MNIST accuracy rises from about 98% at threshold 0.8 toward 99% at 0.05 while an additional 8% of correct predictions are rejected, and CIFAR-10/ViT shows even tighter clusters.
  • Inputs from a genuinely different distribution are more likely to land beyond threshold: at threshold 0.05, MNISTified CIFAR-10 rejects 97% of examples while in-distribution CIFAR-10 rejects 1.2%.
  • The threshold can be set from safety requirements: picking the minimum incorrect-prediction distance gives a conservative rule, while larger or smaller thresholds let an operator choose how much retention to sacrifice.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural stress test the paper does not run is multimodal classes: if one class is produced by two visually distinct subpopulations, the arithmetic-mean centroid could sit between the modes and the distance rule could reject confident correct predictions from either mode; a testable extension is to fit one centroid per mode and compare abstention quality.
  • Because the threshold is calibrated on the training set's wrong predictions, it inherits the network's blind spots: if training errors are few or unrepresentative, the threshold can be too tight or too loose, and per-class thresholds tuned on a held-out validation set would likely be a safer deployment recipe.
  • The near-$\sqrt{2}$ centroid separation suggests the method's usefulness depends on the network being well trained and reasonably calibrated; for poorly calibrated networks, post-hoc recalibration before computing centroids could change results.
  • The method's 'not known' output is not a calibrated probability, so in safety-critical pipelines it would likely serve as a cheap first-stage filter rather than a complete uncertainty quantification.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper proposes a confidence/abstention mechanism for neural network classifiers with softmax outputs. For each class, a centroid is computed as the mean softmax vector of correct training predictions (Algorithm 1), and a per-class threshold is defined as the minimum distance of any incorrect training prediction to that centroid (Algorithm 2). At deployment, predictions whose distance to the predicted-class centroid exceeds the threshold are meant to be labelled 'unknown' and deferred. The authors evaluate the approach on MNIST with a CNN and on CIFAR-10 with a ViT, varying distance thresholds and reporting accuracy, retention, and exclusion rates for in-distribution and 'MNISTified' out-of-distribution datasets. The central claim is that Euclidean distance to a mean softmax centroid is a suitable confidence proxy and can determine when automated predictions should be deferred to human operators.

Significance. If the central claim were established on held-out data, the method would be attractive: it is computationally lightweight, requires no retraining or network modification, and the reported trends are qualitatively consistent across two architectures. The paper also makes a useful exploratory contribution by relating softmax geometry to model accuracy and by testing out-of-distribution data through the MNISTified experiments. However, the significance is currently limited by the absence of held-out evaluation, the circular definition of the 'conservative' threshold, the lack of baselines, and the absence of error bars. The reported accuracy changes are small, and the main experimental figure is explicitly based on training data, so the paper does not yet substantiate its deployment-oriented claims.

major comments (4)
  1. [§4, Figure 3 and Table 1] The evaluation supporting the main accuracy–retention claim is performed on the training data: Figure 3's caption states that the training dataset results are shown, and Table 1's totals (60,000 for MNIST and 50,000 for CIFAR-10) are the training-set sizes. Because the centroids (Algorithm 1) and thresholds are derived from the same correct/incorrect predictions used to compute accuracy and retention, the monotone improvement in accuracy as the threshold decreases is an in-sample statement and may not transfer to deployment. Please report the same analyses on the held-out test sets (10,000 examples each) and include error bars or confidence intervals from multiple runs or bootstrap resamples.
  2. [§1, §3, Algorithm 2] The proposed 'conservative' threshold is defined as the minimum distance of incorrect training predictions to the class centroid; 100% accuracy on the training set under this threshold holds by construction, and the paper provides no evidence that this threshold yields any particular accuracy on unseen data. The experiments do not actually use Algorithm 2: Figure 3 and Table 1 sweep thresholds from 0.8 to 0.05, so the threshold rule that the paper proposes is never evaluated. Please evaluate the Algorithm 2 threshold on the held-out test set and report the resulting accuracy and retention; if the threshold is intended only as a heuristic, state this explicitly and justify the range used.
  3. [§4, Figure 3] No baselines or statistical error bars are reported. The claim that Euclidean distance to a mean softmax centroid is a suitable confidence proxy needs comparison with standard alternatives—such as maximum softmax probability, predictive entropy, or temperature-scaled softmax—to show that the geometric distance adds information beyond what is already available from the softmax output. The reported accuracy changes are small (MNIST 98.0% to 99.0%, CIFAR-10 99.4% to 99.7%), and without intervals it is unclear whether the effect is reliable or an artifact of the specific threshold grid.
  4. [§3, Cluster Density] The paper's own cluster-density discussion states that in 10 dimensions volume concentrates in outer shells and that this creates challenges for clustering and nearest-neighbor calculations, which is in direct tension with the use of a fixed Euclidean distance threshold to a mean centroid as a confidence score. Please provide distributional evidence—for example, per-class histograms or quantile plots of distances for correct versus incorrect predictions—showing that the distance separates the groups in the softmax space, and discuss whether a class-conditional or Mahalanobis distance would be more appropriate than absolute Euclidean distance.
minor comments (7)
  1. [§4] The term 'MNISTify' is used without definition; please define the transformation applied to English Handwritten Characters and CIFAR-10 images.
  2. [Abstract and §1] The notation for the deferred answer is inconsistent ('not known' in the abstract, 'unknown' elsewhere); please choose one term and use it consistently.
  3. [§3, Eq. (1)] In Equation (1), the scaling factor k appears both as the shell index and in r_k = k·r, which is confusing; please use distinct symbols for the shell index and the radius scaling factor.
  4. [§3, Algorithm 2] Algorithm 2 does not specify how the distance to the predicted class centroid is computed before the minimum is taken; please state the exact distance definition and whether it is the distance to the predicted-class centroid or to the true-class centroid.
  5. [Figure 3] The axis labels and legend in Figure 3 are very small and the y-axis range (98–100%) makes the accuracy differences visually misleading; please enlarge the figure and add grid lines to improve readability.
  6. [Table 1] Table 1 has formatting artifacts in the 'At/Above' columns (e.g., '0 .0' and '100 .0'); please fix the spacing and align the numbers.
  7. [References] Reference [10] appears to be a duplicate of reference [9]; please check the reference list and remove redundant entries.

Circularity Check

3 steps flagged · score 6.0 of 10

The conservative 100%-accuracy threshold is defined from training errors and evaluated on the same training set; the accuracy-retention trend is in-sample by construction.

  1. self definitional [Abstract; Section 3 (Algorithm 2); Section 5 (Conclusion)]
    "Defining a distance threshold for a class as the smallest distance from an incorrect prediction to the given class centroid offers a simple approach to adding not known answers to any network classification falling outside of the threshold. ... We adopt the most conservative possible threshold value at which model predictions are expected to be 100% accurate."

    The threshold is computed as the minimum distance of incorrect training predictions to the class centroid. For any training example with an incorrect prediction, its distance is by definition greater than or equal to that minimum, so rejecting every prediction 'falling outside of the threshold' removes all training errors by construction. The claim that this threshold yields 100% accurate predictions is therefore a tautology about the training set used to define the threshold, not a predictive statement about unseen data. The paper offers no held-out evaluation at this threshold.

  2. fitted input called prediction [Section 4, 'Varying Thresholds' paragraph; Figure 3 caption; Table 1]
    "Figure 3 depict test set accuracy, retention ... versus threshold (distance to predicted class centroid) for CIFAR/ViT (top) and MNIST/CNN (bottom). ... where the training dataset results are shown. ... Total examples: CIFAR-10 (50,000), MNIST (60,000)."

    The text calls Figure 3 'test set accuracy', but its own caption states that training dataset results are shown, and the totals in Table 1 are exactly the MNIST and CIFAR-10 training sets (60,000 and 50,000), not the 10,000-example test sets described in Section 4. The centroids and thresholds used in the method are computed from those same training predictions, so the reported accuracy-versus-threshold and retention curves are in-sample summaries of the fitting data rather than predictions of deployment behavior. The claimed consistency across datasets and architectures is thus not evidence of generalization.

1 more flagged steps
  1. self definitional [Section 4, 'Clustering' paragraph]
    "We observe that calculating the average softmax outputs for all the correct classifications provides a good initialization for the K-means algorithm centroids. Clustering converges quickly and assigns all but two of the approximately 59,000 correctly classified examples (in the case of MNIST/CNN) to the correct class centroids."

    The centroids used in the 'unsupervised' clustering are defined by Algorithm 1 as the per-class means of the very same correct training predictions. A mean is the point that minimizes the sum of squared Euclidean distances to the points in the set, so the observation that correct training points lie near 'their' centroid and are assigned to it with high fidelity is a mathematical consequence of the construction, not an independent confirmation that the mean softmax vector is a trustworthy confidence proxy. The validation is performed on the same data used to define the centroids.

full rationale

The paper's central derivation is partially circular. The headline conservative threshold is defined as the minimum distance of incorrect training predictions to the class centroid, so applying that threshold to the same training set guarantees 100% accuracy among retained predictions by construction; this is not a prediction about new data. The accuracy-retention evidence in Figure 3 and Table 1 is computed on the MNIST and CIFAR-10 training sets used to build the centroids and thresholds, despite the surrounding text referring to test-set results, so the reported trend is an in-sample artifact. The high-fidelity clustering observation is also unsurprising because the centroids are means of the correct predictions being measured. There are no load-bearing self-citations, and the out-of-distribution exclusion analysis (English characters and MNISTified CIFAR-10) does provide some independent empirical content not determined by the threshold definition. However, the paper's strongest claim — that the proposed distance metric offers an efficient way to determine when predictions are acceptable — rests in significant part on fitted inputs being presented as predictions, so a moderate circularity score is warranted.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the empirical assumption that softmax outputs of correct predictions form compact, unimodal clusters, and that Euclidean distance to the class-conditional mean is a meaningful confidence score. The thresholds are either hand-swept or fitted to training errors. No invented entities are introduced.

free parameters (3)
  • Global distance thresholds = 0.05, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8 (manual sweep)
    Used to define acceptance regions in Figure 3 and Table 1; chosen by hand to explore trade-offs rather than derived or predicted.
  • Conservative per-class thresholds = not reported in the paper; defined as the minimum distance of incorrect predictions for each class
    Computed from training data by Algorithm 2; this guarantees all training errors fall outside the accepted region, so the 100% training accuracy claim is definitional.
  • Class centroid vectors = 10x10 matrix of mean softmax outputs per class (computed from training set)
    Empirical means over correct predictions; all distances and thresholds depend on these data-fitted vectors.
assumptions (4)
  • domain assumption Correctly classified training examples form coherent, roughly unimodal clusters in softmax space
    Algorithm 1 assigns a single mean centroid per class; a multimodal class would make the mean unrepresentative and distances to it unreliable.
  • domain assumption Euclidean distance in softmax space is a valid measure of prediction confidence
    The method uses raw Euclidean distance to centroids as the confidence proxy; no alternative metric or calibration is tested, and the paper notes curse-of-dimensionality effects in Section 3.
  • domain assumption Training-set misclassifications are representative of test-time low-confidence predictions
    The conservative threshold is derived from incorrect predictions in the training set, so it assumes the distance distribution of training errors matches deployment errors.
  • standard math Standard geometry of the probability simplex, including Euclidean distance and the sqrt(2) maximum separation
    Used in the geometric discussion in Section 3; this is standard mathematics and not controversial.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Explorations of the Softmax Space: Knowing When the Neural Network Doesn't Know." pith.science (2026). https://pith.science/paper/M3OQYQ72

@misc{pith2026250200456,
  author       = {Pith},
  title        = {Pith review of: Explorations of the Softmax Space: Knowing When the Neural Network Doesn't Know},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/M3OQYQ72}},
  note         = {Machine review of arXiv:2502.00456}
}
read the original abstract

Ensuring the reliability of automated decision-making based on neural networks will be crucial as Artificial Intelligence systems are deployed more widely in critical situations. This paper proposes a new approach for measuring confidence in the predictions of any neural network that relies on the predictions of a softmax layer. We identify that a high-accuracy trained network may have certain outputs for which there should be low confidence. In such cases, decisions should be deferred and it is more appropriate for the network to provide a \textit{not known} answer to a corresponding classification task. Our approach clusters the vectors in the softmax layer to measure distances between cluster centroids and network outputs. We show that a cluster with centroid calculated simply as the mean softmax output for all correct predictions can serve as a suitable proxy in the evaluation of confidence. Defining a distance threshold for a class as the smallest distance from an incorrect prediction to the given class centroid offers a simple approach to adding \textit{not known} answers to any network classification falling outside of the threshold. We evaluate the approach on the MNIST and CIFAR-10 datasets using a Convolutional Neural Network and a Vision Transformer, respectively. The results show that our approach is consistent across datasets and network models, and indicate that the proposed distance metric can offer an efficient way of determining when automated predictions are acceptable and when they should be deferred to human operators.

Figures

Figures reproduced from arXiv: 2502.00456 by the authors.

Figure 1
Figure 1. Left to right, MNIST Training Data Image ID 8688 digit 6, the network softmax output and the distances to class centroids, correctly classified as 6 and incorrectly clustered as 5. Notice that the y axis is not on logarithmic scale in this case. 3 ( [PITH_FULL_IMAGE:figures/full_fig_p007_1.png] view at source ↗
Figure 2
Figure 2. Left to right, MNIST Training Data Image ID 35537 digit 6, the network softmax output and the distances to class centroids, correctly classified and correctly clustered as 6. The next example presents a contrasting case to the previous edge case [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Retention, accuracy and correct-incorrect ratio vs Threshold ViT trained on CIFAR-10 and CNN trained on MNIST where the training dataset results are shown. The red plot represents the ratio of correct to incorrect predictions across all classes at given thresholds e.g. for CNN/MNIST at threshold 0.8 the ratio is 64:1, at threshold 0.05 the ratio is 632:1. The green plot is accuracy at every threshold e.g. at thresho… view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: English Handwritten Alphabetic Characters nearest distance and example, and averages [PITH_FULL_IMAGE:figures/full_fig_p010_4.png]
Figure 5
Figure 5. Figure 5: Percentage of examples at or above thresholds for distance to predicted class centroid across five datasets, showing higher exclusion rates for English Alphabetical Characters and MNISTified CIFAR￾10. Total examples: CIFAR-10 (50,000), MNIST (60,000), Eng. Digits (550)…
Figure 6
Figure 6. Figure 6: Density of points in 10-dimensional spherical shells, visualized as concentric rings. Color intensity represents log10(ρk), with brighter colors indicating higher density. English Alphabetical Characters and MNISTified CIFAR-10 show higher density in outer shells, refl…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

43 extracted references · 31 canonical work pages

  1. [1]

    arXiv preprint arXiv:1606.06565 (2016)

    Amodei, D., Olah, C., Steinhardt, J., Christiano, P., Schulman, J., Mané, D.: Concrete problems in ai safety. arXiv preprint arXiv:1606.06565 (2016)

    arXiv 2016

  2. [2]

    In: Proceedings of the eighteenth annual ACM-SIAM symposium on Discrete algorithms

    Arthur, D., Vassilvitskii, S.: k-means++: The advantages of careful seeding. In: Proceedings of the eighteenth annual ACM-SIAM symposium on Discrete algorithms. pp. 1027–1035 (2007)

    work page 2007

  3. [3]

    In: Proceedings of the International Conference on Computer Vision Theory and Applications, Lisbon, Portugal (February 2009)

    de Campos, T.E., Babu, B.R., Varma, M.: Character recognition in natural images. In: Proceedings of the International Conference on Computer Vision Theory and Applications, Lisbon, Portugal (February 2009)

    work page 2009

  4. [4]

    ACM computing surveys (CSUR) 41(3), 1–58 (2009)

    Chandola, V., Banerjee, A., Kumar, V.: Anomaly detection: A survey. ACM computing surveys (CSUR) 41(3), 1–58 (2009)

    work page 2009

  5. [5]

    Springer, New York, NY, 3rd edn

    Conway, J.H., Sloane, N.J.A.: Sphere Packings, Lattices and Groups. Springer, New York, NY, 3rd edn. (1998)

    work page 1998

  6. [6]

    In: 2009 IEEE conference on computer vision and pattern recognition

    Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE conference on computer vision and pattern recognition. pp. 248–255. Ieee (2009)

    2009

  7. [7]

    arXiv preprint arXiv:2010.11929 (2020)

    Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al.: An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020)

    arXiv 2020

  8. [8]

    Wiley, New York, NY, 2nd edn

    Duda, R.O., Hart, P.E., Stork, D.G.: Pattern Classification. Wiley, New York, NY, 2nd edn. (2000)

    work page 2000

Show all 43 references
  1. [9]

    Ester, M., Kriegel, H.P., Sander, J., Xu, X.: A density-based algorithm for discovering clusters in large spatial databases with noise. In: Kdd. vol. 96, pp. 226–231 (1996)

    1996

  2. [10]

    Proceedings of the 2nd International Conference on Knowledge Discovery and Data Mining (KDD’96) pp

    Ester, M., Kriegel, H.P., Sander, J., Xu, X.: A density-based algorithm for discovering clusters in large spatial databases with noise. Proceedings of the 2nd International Conference on Knowledge Discovery and Data Mining (KDD’96) pp. 226–231 (1996)

    1996

  3. [11]

    arXiv preprint arXiv:1805.01039 (2018)

    Feng, G., He, J., Polson, N.G.: Deep learning-based quantitative trading strategies for stock markets. arXiv preprint arXiv:1805.01039 (2018)

    2018 arXiv

  4. [12]

    International Conference on Machine Learning pp

    Gal, Y., Ghahramani, Z.: Dropout as a bayesian approximation: Representing model uncertainty in deep learning. International Conference on Machine Learning pp. 1050–1059 (2016)

    2016

  5. [13]

    ACM Computing Surveys (CSUR)46(4), 1–37 (2014)

    Gama, J., Žliobait˙ e, I., Bifet, A., Pechenizkiy, M., Bouchachia, A.: A survey on concept drift adaptation. ACM Computing Surveys (CSUR)46(4), 1–37 (2014)

    2014

  6. [14]

    The Journal of Machine Learning Research 17(1), 2096–2030 (2016)

    Ganin, Y., Ustinova, E., Ajakan, H., Germain, P., Larochelle, H., Laviolette, F., Marchand, M., Lem- pitsky, V.: Domain-adversarial training of neural networks. The Journal of Machine Learning Research 17(1), 2096–2030 (2016)

    2016

  7. [15]

    In: Interna- tional Conference on Learning Representations (2014)

    Goodfellow, I.J., Shlens, J., Szegedy, C.: Explaining and harnessing adversarial examples. In: Interna- tional Conference on Learning Representations (2014)

    2014

  8. [16]

    The Journal of Machine Learning Research13(1), 723–773 (2012)

    Gretton, A., Borgwardt, K.M., Rasch, M.J., Schölkopf, B., Smola, A.: A kernel two-sample test. The Journal of Machine Learning Research13(1), 723–773 (2012)

    2012

  9. [17]

    In: Interna- tional Conference on Machine Learning

    Guo, C., Pleiss, G., Sun, Y., Weinberger, K.Q.: On calibration of modern neural networks. In: Interna- tional Conference on Machine Learning. pp. 1321–1330. PMLR (2017)

    2017

  10. [18]

    In: International Conference on Learning Representations (2018) Explorations of the Softmax Space: Knowing When the Neural Network Doesn’t Know 15

    Guo, C., Rana, M., Cisse, M., Van Der Maaten, L.: Countering adversarial images using input trans- formations. In: International Conference on Learning Representations (2018) Explorations of the Softmax Space: Knowing When the Neural Network Doesn’t Know 15

    2018

  11. [19]

    Springer, New York, NY, 2nd edn

    Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer, New York, NY, 2nd edn. (2009)

    2009

  12. [20]

    In: Proceedings of the IEEE/CVF international conference on computer vision

    Hendrycks, D., Basart, S., Mu, N., Kadavath, S., Wang, F., Dorundo, E., Desai, R., Zhu, T., Parajuli, S., Guo, M., et al.: The many faces of robustness: A critical analysis of out-of-distribution generalization. In: Proceedings of the IEEE/CVF international conference on compu...

    2021

  13. [21]

    In: International Conference on Learning Representations (2017)

    Hendrycks, D., Gimpel, K.: A baseline for detecting misclassified and out-of-distribution examples in neural networks. In: International Conference on Learning Representations (2017)

    2017

  14. [22]

    Psychometrika32(3), 241–254 (1967)

    Johnson, S.C.: Hierarchical clustering schemes. Psychometrika32(3), 241–254 (1967)

    1967

  15. [23]

    Kendall, A., Gal, Y.: What uncertainties do we need in bayesian deep learning for computer vision? Advances in Neural Information Processing Systems30 (2017)

    2017

  16. [24]

    The Annals of Mathematical Statistics 22(1), 79–86 (1951)

    Kullback, S., Leibler, R.A.: On information and sufficiency. The Annals of Mathematical Statistics 22(1), 79–86 (1951)

    1951

  17. [25]

    Scientific Reports7(1), 1–14 (2017)

    Leibig, C., Allken, V., Ayhan, M.S., Berens, P., Wahl, S.: Leveraging uncertainty information from deep neural networks for disease detection. Scientific Reports7(1), 1–14 (2017)

    2017

  18. [26]

    In: International Conference on Learning Representations (2018)

    Liang, S., Li, Y., Srikant, R.: Enhancing the reliability of out-of-distribution image detection in neural networks. In: International Conference on Learning Representations (2018)

    2018

  19. [27]

    IEEE transactions on information theory28(2), 129–137 (1982)

    Lloyd, S.: Least squares quantization in pcm. IEEE transactions on information theory28(2), 129–137 (1982)

    1982

  20. [28]

    In: International Conference on Learning Representations (2017)

    Madry, A., Makelov, A., Schmidt, L., Tsipras, D., Vladu, A.: Towards deep learning models resistant to adversarial attacks. In: International Conference on Learning Representations (2017)

    2017

  21. [29]

    arXiv preprint arXiv:1811.06817 (2018)

    Michelmore, R., Kwiatkowska, M., Gal, Y.: Evaluating uncertainty quantification in end-to-end au- tonomous driving control. arXiv preprint arXiv:1811.06817 (2018)

    2018 arXiv

  22. [30]

    arXiv preprint arXiv:1109.2378 (2011)

    Müllner, D.: Modern hierarchical, agglomerative clustering algorithms. arXiv preprint arXiv:1109.2378 (2011)

    2011 arXiv

  23. [31]

    IEEE Transactions on Knowledge and Data Engi- neering 22(10), 1345–1359 (2009)

    Pan, S.J., Yang, Q.: A survey on transfer learning. IEEE Transactions on Knowledge and Data Engi- neering 22(10), 1345–1359 (2009)

    2009

  24. [32]

    IEEE Signal Processing Magazine32(3), 53–69 (2015)

    Patel, V.M., Gopalan, R., Li, R., Chellappa, R.: Visual domain adaptation: A survey of recent advances. IEEE Signal Processing Magazine32(3), 53–69 (2015)

    2015

  25. [33]

    The MIT Press (2009)

    Quiñonero-Candela, J., Sugiyama, M., Schwaighofer, A., Lawrence, N.D.: Dataset shift in machine learning. The MIT Press (2009)

    2009

  26. [34]

    ACM Transactions on Database Systems (TODS)42(3), 1–21 (2017)

    Schubert, E., Sander, J., Ester, M., Kriegel, H.P., Xu, X.: Dbscan revisited, revisited: why and how you should (still) use dbscan. ACM Transactions on Database Systems (TODS)42(3), 1–21 (2017)

    2017

  27. [35]

    IEEE Transactions on pattern analysis and machine intelligence 22(8), 888–905 (2000)

    Shi, J., Malik, J.: Normalized cuts and image segmentation. IEEE Transactions on pattern analysis and machine intelligence 22(8), 888–905 (2000)

    2000

  28. [36]

    Journal of Statistical Planning and Inference90(2), 227–244 (2000)

    Shimodaira, H.: Improving predictive inference under covariate shift by weighting the log-likelihood function. Journal of Statistical Planning and Inference90(2), 227–244 (2000)

    2000

  29. [37]

    The Journal of Machine Learning Research8, 985–1005 (2007)

    Sugiyama, M., Krauledat, M., Müller, K.R.: Covariate shift adaptation by importance weighted cross validation. The Journal of Machine Learning Research8, 985–1005 (2007)

    2007

  30. [38]

    In: International Conference on Learning Representations (2013)

    Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., Fergus, R.: Intriguing properties of neural networks. In: International Conference on Learning Representations (2013)

    2013

  31. [39]

    Neurocomputing312, 135–153 (2018)

    Wang, M., Deng, W.: Deep visual domain adaptation: A survey. Neurocomputing312, 135–153 (2018)

    2018

  32. [40]

    Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., Davison, J., Shleifer, S., von Platen, P., Ma, C., Jernite, Y., Plu, J., Xu, C., Scao, T.L., Gugger, S., Drame, M., Lhoest, Q., Rush, A.M.: Huggingface’s transfo...

    2020

  33. [41]

    Wu, B., Xu, C., Dai, X., Wan, A., Zhang, P., Yan, Z., Tomizuka, M., Gonzalez, J., Keutzer, K., Vajda, P.: Visual transformers: Token-based image representation and processing for computer vision (2020)

    2020

  34. [42]

    Xu,D.,Tian,Y.:Acomprehensivesurveyofclusteringalgorithms.AnnalsofDataScience 2(2),165–193 (2015)

    2015

  35. [43]

    In: Proceedings of the eighth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

    Zadrozny, B., Elkan, C.: Transforming classifier scores into accurate multiclass probability estimates. In: Proceedings of the eighth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. pp. 694–699 (2002)

    2002

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.