Pith. sign in

REVIEW 4 major objections 5 minor 52 references

TRUST: Test-time Resource Utilization for Superior Trustworthiness

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read TRUST ranks predictions by the distance of each test image to its predicted class's feature-space center, and accuracy rises as low-scoring samples are removed.

desk verdict A promising test-time uncertainty idea whose headline claim is only demonstrated on easy class subsets for two of four datasets, and whose mode-projection interpretation is asserted, not proven. read the letter →

arxiv 2506.06048 v1 pith:NXYIHVBR submitted 2025-06-06 cs.LG cs.AIcs.CV

Haripriya Harikumar , Santu Rana This is my paper
classification cs.LGcs.AIcs.CV
keywords uncertaintyestimationtest-timeoptimizationselectiveclassificationepistemicout-of-distributiondetectionconfidencescoringfeature-spacegeometryneuralcollapse
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a reliability score computed at test time—by slightly perturbing each test image until it lands on the nearest dense center ('mode') of its predicted class in the model's feature space, then measuring the cosine distance to that mode—ranks predictions by true accuracy. The authors argue that classifier weights carry noise that corrupts standard confidence estimates, but that this noise can be bypassed with extra computation per sample. On CIFAR-10, TinyImageNet, CAMELYON-17, and ImageNet, they report that accuracy rises monotonically as lower-scoring samples are removed, and that TRUST improves on dropout, ViM, SIRC, LogitNorm, and density-aware baselines under risk-based metrics. They also report that the gap between training and test TRUST distributions tracks accuracy drops under corruption and detects out-of-distribution data. If true, the score gives a way to decide which predictions to trust without retraining the model.

What carries the argument

The score's engine is the test-time optimization in Eq. 1: $\arg\min_{\Delta x} L(x_{\mathrm{test}}+\Delta x, y_{\mathrm{test}})+\lambda\|\Delta x\|_1$, run with an elevated softmax temperature $T$, whose converged input $x_{\mathrm{test}}^{\mathrm{mode}}=x_{\mathrm{test}}+\Delta x$ is taken to be the nearest mode of the predicted class's micro-cluster. The TRUST score is then the cosine distance between the final-layer features $f^l_\theta(x_{\mathrm{test}})$ and $f^l_\theta(x_{\mathrm{test}}^{\mathrm{mode}})$. The surrounding argument uses hypersphere concentration (Theorem 1), near-orthogonality of high-dimensional points (Theorem 2 and Corollary 1), and a noise-robustness bound (Theorem 3) showing that cosine-distance noise is damped by a factor $\sin^2(\omega)$ relative to direct score noise.

What would settle it

Take a trained model and a held-out test set; compute TRUST scores, then independently estimate each predicted class's true feature-space mode (for example by density estimation over training features or by averaging near neighbors). If the cosine distance from test samples to their TRUST-optimized points does not rank-match the distance to the independently estimated nearest mode for a substantial fraction of samples, the score is not measuring what the derivation assumes.

Watch

Extended reading notes

Core claim

TRUST (Test-time Resource Utilization for Superior Trustworthiness) is a sample-specific reliability score defined as the cosine similarity between the final-layer feature vector of a test image and the feature vector of that image after a constrained test-time optimization. The optimization minimizes cross-entropy plus an L1 penalty on the input perturbation, with a high softmax temperature, to move the image to its predicted class's nearest feature-space mode while keeping the change sparse. The paper argues that under high-dimensional concentration of measure, training points form well-separated micro-clusters around class modes, so the angular distance to the nearest mode is a proxy for epistemic uncertainty; it proves that noisy estimation of this cosine distance causes no more sorting errors than the same noise applied directly to an uncertainty score. Empirically, TRUST defines a monotonic subset-selection function: on every tested dataset and architecture, accuracy increases as lower-scored samples are removed, and it outperforms the compared baselines on AURC and AUSE while also separating in-distribution from out-of-distribution data and exposing train-test misalignment.

Load-bearing premise

The whole score rests on the assumption that the test-time optimization moves each test image to the nearest dense center of its predicted class in the model's feature space, rather than to some other point.

Editorial extensions

If this is right

  • On all four datasets tested, retaining only the top 20% of TRUST-ranked samples lifts accuracy well above full-set accuracy, so TRUST can be used for selective classification.
  • Because TRUST improves LogitNorm-trained models further, it complements training-time confidence methods rather than replacing them.
  • The train-test TRUST distribution gap (measured by MMD) correlates with accuracy drop, giving a signal that a model or test distribution has shifted.
  • TRUST scores separate in-distribution from out-of-distribution samples without needing OOD data or retraining.
  • TRUST's monotonicity holds across CNN and ViT architectures and across feature layers, with the final layer giving the best stratification.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the paper observes early convergence within a few hundred epochs, the per-sample computational cost may be reducible well below the reported several seconds per image, making batch or interactive deployment more practical than the stated limitation suggests.
  • The score is a pure geometric distance on learned features, so the same construction may transfer to other modalities with learned feature spaces, even though the paper only tests image classifiers.
  • The monotonicity claim suggests an internal consistency check: if a model's TRUST-sorted accuracy is flat or decreasing, that may indicate its feature space lacks the assumed micro-cluster geometry.
  • The paper's own observation that TRUST's effectiveness diminishes with smaller architectures points to a testable boundary: the method should be expected to help only once a model's feature space is high-dimensional enough for the concentration arguments to apply.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces TRUST, a test-time reliability score computed as the cosine distance between a test sample's penultimate-layer feature and the feature of an optimized counterfactual point obtained by minimizing cross-entropy plus an L1 input perturbation. The authors argue that this optimization projects the test sample to the nearest mode of its predicted class's micro-cluster, mitigating the effect of noisy classifier weights on uncertainty estimates. They report that the score produces monotonic accuracy stratification, improves AURC/AUSE over baselines on CIFAR-10, CAMELYON-17, TinyImageNet, and ImageNet, detects OOD inputs, and exposes differences between CNN and ViT feature geometries. Theoretical results are provided on high-dimensional concentration, a noise-robustness bound for cosine scoring, and ablations on temperature, regularization, and convergence.

Significance. If the central geometric assumption is valid, the proposal is a novel and practically interesting alternative to sampling- or retraining-based uncertainty estimators: it adds computation at test time rather than modifying training, and it is orthogonal to training losses such as LogitNorm. The paper is honest about its main limitation (per-sample cost in seconds; effectiveness diminishes for small architectures). A valuable positive is that the evaluation is post-hoc with fixed hyperparameters, so the reported gains are not fitted to the benchmark metrics. However, the current evidence does not yet support the population-level monotonicity claim because two of the four datasets are evaluated on accuracy-filtered class subsets, and the theoretical link between Eq. (1) and a feature-space mode is asserted rather than proved. The paper would be strengthened by full-population evaluation and by a synthetic or low-dimensional verification of the mode-projection property.

major comments (4)
  1. [4.1.1, Table 1] The TinyImageNet and ImageNet rows of Table 1 are computed on restricted class subsets (classes with ≥60% accuracy; a random subset of 100 classes with 80–90% accuracy), yet Section 5.1 and the abstract present these as evidence that TRUST defines a monotonic subset-selection function over the test population and outperforms baselines on four benchmark datasets. This filter removes hard, low-confidence samples—exactly the cases a reliability score must rank—so the observed monotonicity and favorable AURC/AUSE on those rows may not extend to the full test population. Moreover, unless the same class filter was applied to all baselines (which is not stated), the reported numbers are not comparable to the baselines' full-test-set results.
  2. [3.1, Eq. (1)] The paper's central mechanistic claim is that minimizing Eq. (1) 'projects x_test to its nearest mode' (Section 3.1), so that the cosine distance between x_test and the optimized point equals epistemic uncertainty. No proof is given that the stationary point of Eq. (1) is a mode of the predicted class's feature distribution, nor is it shown that the high-temperature softmax preserves micro-cluster membership. Since the TRUST score's meaning depends entirely on this equivalence, the claim is unsupported as it stands. A concrete fix would be to validate the mode-projection property on a synthetic mixture of Gaussians or on a low-dimensional learned subspace, and to compare the optimized features with the actual density maxima of the training features.
  3. [3.2.1, Theorem 3] The theoretical noise-robustness result is not connected to the actual algorithm. Theorem 3 assumes a Gaussian perturbation on the angle ω and shows that cosine scoring has lower effective variance than direct noise, but the paper does not bound the estimation error of the mode obtained from Eq. (1), nor does it show that this error follows the assumed noise model. The proof also relies on a first-order Taylor approximation and a variance comparison; without an additional argument, a variance ordering does not by itself establish the claimed ordering of sorting-error probabilities. Consequently, the theoretical section does not yet substantiate the paper's claim that test-time optimization provides superior robustness over direct scoring.
  4. [5.1, Table 1] The claim of superior performance over state-of-the-art baselines ('state-of-the-art performance in identifying reliable predictions' in Section 1) is only partially supported by the experiments. On CAMELYON-17 only Dropout and CrossEntro+TRUST are reported; on ImageNet no baseline is reported; and on TinyImageNet only Dropout is reported. Without comparing against ViM, SIRC, or LogitNorm (or at least strong softmax/MSP baselines) on the full test sets of those datasets, the cross-dataset superiority claim is not established.
minor comments (5)
  1. [1] The manuscript states 'Code is available at LINK'; the placeholder 'LINK' is not a functional repository URL, which harms reproducibility and should be replaced with a real link.
  2. [Supplementary, Theorem 5 proof] In Step 3 of the proof, solving Φ(t√d) ≈ 2/n² yields t√d ≈ -√(2 ln(n²/2)) ≈ -2√(ln n) for large n, not -√(2 ln n) as stated. The final formula is therefore off by a factor of √2; since the theorem is used only as a qualitative motivation for micro-cluster separation, the main conclusions are unaffected, but the proof should be corrected.
  3. [Abstract, 5.1] The phrase 'defines a monotonic subset-selection function' suggests a proven property, but the paper provides only empirical evidence at a few selected percentile thresholds (20%, 40%, 60%, 80%, 100%). Consider rephrasing to 'exhibits monotonic accuracy stratification at the evaluated thresholds'.
  4. [3.2.1, Theorem 3] The notation in Theorem 3 is confusing: δθ is introduced but δω is used in the statement and proof, and the sentence 'For an equivalent score functions' is grammatically incorrect. Clarifying the noise model would improve readability.
  5. [4.1.1] The sentence 'For TinyImageNet, we selected classes with ≥60% accuracy (overall: 65.19%)' should specify whether 65.19% is the model's full-test accuracy or the subset's accuracy, to avoid ambiguity when interpreting Table 1.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: TRUST is an independent test-time ranking score; its empirical claims are evaluated against external accuracy and OOD labels.

full rationale

The derivation chain starts from a geometric assumption (high-dimensional features form micro-clusters), defines an optimization problem (Eq. 1) whose solution is called the nearest mode, and defines the TRUST score as the cosine distance to that solution. No parameter is fitted to the reported AURC/AUSE or accuracy values; T = 5.0, lambda = 0.001, and 10k epochs are fixed and ablated. The mode identification is asserted rather than proven in Section 3.1, but this is a correctness/assumption gap, not circularity: the score's definition does not presuppose the accuracy ranking. Evaluation of monotonic accuracy uses ground-truth labels, AURC/AUSE use risk-coverage curves, and OOD detection uses SVHN labels, all external to the score construction. The MMD-versus-accuracy-drop analysis (Fig. 5b) is a descriptive correlation, not a fitted prediction. The TinyImageNet/ImageNet class-subset protocol is a support/validity concern rather than a circular one. There are no load-bearing self-citations; prior work is cited as background. The ablation-based hyperparameter choice on CIFAR-10 is a possible leakage issue, but it does not make any reported quantity equal to its input by construction.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The central machinery rests on a hypothetical micro-cluster geometry that is not established for the tested models, and on an unproven equivalence between the optimization solution and the geometric mode. Hyperparameters are fixed by hand with only weak ablations. Subset selection on two datasets weakens the empirical generalization.

free parameters (6)
  • Softmax temperature T = 5.0
    Chosen by hand; ablations show minor impact within 5 to 100 but lower T performs slightly better. Location: Section 4.1.1 and Ablation 7.2.
  • L1 regularization weight lambda = 0.001
    Chosen by hand via ablation; variations affect convergence loss and slightly affect accuracy. Location: Section 4.1.1, Ablation 7.2.
  • Number of optimization epochs = 10,000
    Fixed for all experiments; convergence occurs within a few hundred epochs, so 10k is a conservative cap. Location: Section 4.1.1, Ablation 7.1.
  • Feature layer for TRUST score = final layer
    Experiments indicate final layer most reliable; no principled selection criterion. Location: Section 3.1.
  • TinyImageNet class subset = classes with >=60% accuracy
    Post-hoc selection of easier classes for analysis; inflates reported accuracies. Location: Section 4.1.1.
  • ImageNet class subset = 100 classes with 80-90% accuracy
    Randomly chosen moderate-accuracy classes; results not representative of full ImageNet. Location: Section 4.1.1.
assumptions (4)
  • domain assumption Micro-cluster topology
    Assumes final-layer features lie on a hypersphere with well-separated micro-clusters per class; Section 3. The concentration theorem only shows points concentrate on a sphere, not that they cluster.
  • ad hoc to paper Eq.1 solution is the nearest mode
    Assumes that minimizing CE loss with L1 sparsity in input space moves a test sample to the mode of its predicted class's micro-cluster in feature space; no proof given. Section 3.1.
  • domain assumption Angular distance to mode is proportional to epistemic uncertainty
    Assumes cosine distance to the nearest micro-cluster mode is a valid epistemic uncertainty proxy. Section 3.
  • domain assumption Optimization reduces weight-noise effect
    Assumes additional test-time computation mitigates noisy weight effects sufficiently for Eq.1's solution to be informative. Section 3.2.1, Remark 1.

how reviews work

0 comments
Cite this review

Pith. "Pith review of TRUST: Test-time Resource Utilization for Superior Trustworthiness." pith.science (2026). https://pith.science/paper/NXYIHVBR

@misc{pith2026250606048,
  author       = {Pith},
  title        = {Pith review of: TRUST: Test-time Resource Utilization for Superior Trustworthiness},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NXYIHVBR}},
  note         = {Machine review of arXiv:2506.06048}
}
read the original abstract

Standard uncertainty estimation techniques, such as dropout, often struggle to clearly distinguish reliable predictions from unreliable ones. We attribute this limitation to noisy classifier weights, which, while not impairing overall class-level predictions, render finer-level statistics less informative. To address this, we propose a novel test-time optimization method that accounts for the impact of such noise to produce more reliable confidence estimates. This score defines a monotonic subset-selection function, where population accuracy consistently increases as samples with lower scores are removed, and it demonstrates superior performance in standard risk-based metrics such as AUSE and AURC. Additionally, our method effectively identifies discrepancies between training and test distributions, reliably differentiates in-distribution from out-of-distribution samples, and elucidates key differences between CNN and ViT classifiers across various vision datasets.

Figures

Figures reproduced from arXiv: 2506.06048 by the authors.

Figure 1
Figure 1. Geometrical Intuition of Our Approach: Traditional methods [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Comparison of TRUST score-based accuracies on CIFAR-10: (a) class-wise, (b) layer-wise, and (c) across [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Comparison of model-wise TRUST score rankings and distributions on the CIFAR-10 test dataset: (a) [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (15 more)
Figure 4
Figure 4. Figure 4: TRUST score distributions for CIFAR-10. (a) Training vs. test samples for automobile and cat classes [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: (a) Accuracy vs. TRUST-stratified CIFAR-10 test samples under noise corruptions: Uni_p%, Bright_b, [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]
Figure 6
Figure 6. Figure 6: TRUST score-based mode and tail samples from CIFAR-10: (a) ‘Automobile’ and ‘Cat’ class examples [PITH_FULL_IMAGE:figures/full_fig_p011_6.png]
Figure 7
Figure 7. Figure 7: Convergence of the loss on the CIFAR-10 dataset for varying softmax temperature ( [PITH_FULL_IMAGE:figures/full_fig_p020_7.png]
Figure 8
Figure 8. Figure 8: Effect of varying softmax temperature (T) and regularization parameter (λ) on TRUST score-based accuracies for CIFAR-10. hyperparameters: softmax temperature T = 5.0 and regularization parameter λ = 0.001. These plots show accuracy trends across the first 100 training …
Figure 9
Figure 9. Figure 9: TRUST score-based accuracy comparison on CIFAR-10 with [PITH_FULL_IMAGE:figures/full_fig_p022_9.png]
Figure 10
Figure 10. Figure 10: TRUST score scatter plots for CIFAR-10 test samples on the PreactResNet18 model, colored by class [PITH_FULL_IMAGE:figures/full_fig_p023_10.png]
Figure 11
Figure 11. Figure 11: Mode samples with highest TRUST scores for each class (Class 0 to Class 9) in CIFAR-10. [PITH_FULL_IMAGE:figures/full_fig_p023_11.png]
Figure 12
Figure 12. Figure 12: Mode samples with lowest TRUST scores for each class (Class 0 to Class 9) in CIFAR-10. [PITH_FULL_IMAGE:figures/full_fig_p024_12.png]
Figure 13
Figure 13. Figure 13: Accuracy drop vs MMD for original test and uniform noise from 1% to % 9% for CIFAR-10 PreactResNet18 25 [PITH_FULL_IMAGE:figures/full_fig_p025_13.png]
Figure 14
Figure 14. Figure 14: Accuracy drop vs MMD Gaussian noise, Gaussian blur, different brightness and SVHN dataset for [PITH_FULL_IMAGE:figures/full_fig_p026_14.png]
Figure 15
Figure 15. Figure 15: Class-wise (Class 0 to Class 5) TRUST score distribution of CIFAR-10 train (in cyan), test (in green) 27 [PITH_FULL_IMAGE:figures/full_fig_p027_15.png]
Figure 16
Figure 16. Figure 16: Class-wise (Class 6 to Class 9) TRUST score distribution of CIFAR-10 train (in cyan), test (in green) [PITH_FULL_IMAGE:figures/full_fig_p028_16.png]
Figure 17
Figure 17. Figure 17: AUSE plots of CIFAR-10 test dataset on CrossEntro+TRUST, LogitNorm, and LogitNorm+TRUST for [PITH_FULL_IMAGE:figures/full_fig_p028_17.png]
Figure 18
Figure 18. Figure 18: Class-wise AUSE plots of CIFAR-10 test dataset for CrossEntro+TRUST, LogitNorm, and Logit [PITH_FULL_IMAGE:figures/full_fig_p029_18.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

52 extracted references · 45 canonical work pages

  1. [1]

    An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale.arXiv preprint arXiv: 2010.11929, 2020

    Dosovitskiy Alexey. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale.arXiv preprint arXiv: 2010.11929, 2020

    arXiv 2010

  2. [2]

    AI in Health: Keeping the Human in the Loop, 2023

    Suzanne Bakken. AI in Health: Keeping the Human in the Loop, 2023

    work page 2023

  3. [3]

    Camelyon17 dataset

    Peter Bandi. Camelyon17 dataset

    work page

  4. [4]

    Emergent and Predictable Memorization in Large Language Models.Advances in Neural Information Processing Systems, 2024

    Stella Biderman, Usvsn Prashanth, Lintang Sutawika, Hailey Schoelkopf, Quentin Anthony, Shivanshu Purohit, and Edward Raff. Emergent and Predictable Memorization in Large Language Models.Advances in Neural Information Processing Systems, 2024

    work page 2024

  5. [5]

    Quantifying Memorization across Neural Language Models.arXiv preprint arXiv:2202.07646, 2022

    Nicholas Carlini, Daphne Ippolito, Matthew Jagielski, Katherine Lee, Florian Tramer, and Chiyuan Zhang. Quantifying Memorization across Neural Language Models.arXiv preprint arXiv:2202.07646, 2022

    arXiv 2022

  6. [6]

    Tackling Prediction Uncertainty in Machine Learning for Healthcare.Nature Biomedical Engineering, 2023

    Michelle Chua, Doyun Kim, Jongmun Choi, Nahyoung G Lee, Vikram Deshpande, Joseph Schwab, Michael H Lev, Ramon G Gonzalez, Michael S Gee, and Synho Do. Tackling Prediction Uncertainty in Machine Learning for Healthcare.Nature Biomedical Engineering, 2023

    work page 2023

  7. [7]

    Uncertainty Estimation by Fisher Information-Based Evidential Deep Learning

    Danruo Deng, Guangyong Chen, Yang Yu, Furui Liu, and Pheng-Ann Heng. Uncertainty Estimation by Fisher Information-Based Evidential Deep Learning. InInternational Conference on Machine Learning, 2023. 11

    work page 2023

  8. [8]

    Imagenet: A Large-Scale Hierarchical Image Database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A Large-Scale Hierarchical Image Database. InProceedings of the IEEE/CVF International Conference on Computer Vision, 2009

    work page 2009

Show all 52 references
  1. [9]

    ViM: Out-of-Distribution With Virtual-Logit Matching

    Wang et al. ViM: Out-of-Distribution With Virtual-Logit Matching. InProceedings of the IEEE/CVF International Conference on Computer Vision, 2022

    2022

  2. [10]

    Augmenting Softmax Information for Selective Classification with Out-of-Distribution Data

    Xia et al. Augmenting Softmax Information for Selective Classification with Out-of-Distribution Data. InAsian Conference on Computer Vision, 2022

    2022

  3. [11]

    Uncertainty Estimation by Density Aware Evidential Deep Learning.International Conference on Machine Learning, 2024

    Yoon et al. Uncertainty Estimation by Density Aware Evidential Deep Learning.International Conference on Machine Learning, 2024

    2024

  4. [12]

    Rethinking Confidence Calibration for Failure Prediction

    Zhu et al. Rethinking Confidence Calibration for Failure Prediction. InEuropean Conference on Computer Vision, 2022

    2022

  5. [13]

    OpenMix: Exploring Outlier Samples for Misclassification Detection

    Zhu et al. OpenMix: Exploring Outlier Samples for Misclassification Detection. InProceedings of the IEEE/CVF International Conference on Computer Vision, 2023

    2023

  6. [14]

    RCL: Reliable Continual Learning for Unified Failure Detection

    Zhu et al. RCL: Reliable Continual Learning for Unified Failure Detection. InProceedings of the IEEE/CVF International Conference on Computer Vision, 2024

    2024

  7. [15]

    Skin Cancer Detection utilizing Deep Learning: Classification of Skin Lesion Images using a Vision Transformer.arXiv preprint arXiv:2407.18554, 2024

    Carolin Flosdorf, Justin Engelker, Igor Keller, and Nicolas Mohr. Skin Cancer Detection utilizing Deep Learning: Classification of Skin Lesion Images using a Vision Transformer.arXiv preprint arXiv:2407.18554, 2024

    2024 arXiv

  8. [16]

    Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning

    Yarin Gal and Zoubin Ghahramani. Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning. InInternational Conference on Machine Learning, 2016

    2016

  9. [17]

    A Survey of Uncertainty in Deep Neural Networks.Artificial Intelligence Review, 2023

    Jakob Gawlikowski, Cedrique Rovile Njieutcheu Tassi, Mohsin Ali, Jongseok Lee, Matthias Humt, Jianxiang Feng, Anna Kruspe, Rudolph Triebel, Peter Jung, Ribana Roscher, et al. A Survey of Uncertainty in Deep Neural Networks.Artificial Intelligence Review, 2023

    2023

  10. [18]

    Deep Learning: Applications, Architectures, Models, Tools, and Frameworks: A Comprehensive Survey.CAAI Transactions on Intelligence Technology, 2023

    Mehdi Gheisari, Fereshteh Ebrahimzadeh, Mohamadtaghi Rahimi, Mahdieh Moazzamigodarzi, Yang Liu, Pijush Kanti Dutta Pramanik, Mohammad Ali Heravi, Abolfazl Mehbodniya, Mustafa Ghaderzadeh, Moham- mad Reza Feylizadeh, et al. Deep Learning: Applications, Architectures, Models, To...

    2023

  11. [19]

    Springer Science & Business Media, 2011

    Robert M Gray.Entropy and information theory. Springer Science & Business Media, 2011

    2011

  12. [20]

    Deep Learning for Finance: Deep Portfolios.Applied Stochastic Models in Business and Industry, 2017

    James B Heaton, Nick G Polson, and Jan Hendrik Witte. Deep Learning for Finance: Deep Portfolios.Applied Stochastic Models in Business and Industry, 2017

    2017

  13. [21]

    Generalized ODIN: Detecting Out-Of-Distribution Image without Learning from Out-Of-Distribution Data

    Yen-Chang Hsu, Yilin Shen, Hongxia Jin, and Zsolt Kira. Generalized ODIN: Detecting Out-Of-Distribution Image without Learning from Out-Of-Distribution Data. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020

    2020

  14. [22]

    Look Before You Leap: An Exploratory Study of Uncertainty Measurement for Large Language Models.arXiv preprint arXiv:2307.10236, 2023

    Yuheng Huang, Jiayang Song, Zhijie Wang, Shengming Zhao, Huaming Chen, Felix Juefei-Xu, and Lei Ma. Look Before You Leap: An Exploratory Study of Uncertainty Measurement for Large Language Models.arXiv preprint arXiv:2307.10236, 2023

    2023 arXiv

  15. [23]

    Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methods.Machine Learning, 2021

    Eyke Hüllermeier and Willem Waegeman. Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methods.Machine Learning, 2021. 12

    2021

  16. [24]

    To trust or not to trust a classifier.Advances in neural information processing systems, 31, 2018

    Heinrich Jiang, Been Kim, Melody Guan, and Maya Gupta. To trust or not to trust a classifier.Advances in neural information processing systems, 31, 2018

    2018

  17. [25]

    What Uncertainties do we need in Bayesian Deep Learning for Computer Vision? Advances in Neural Information Processing Systems, 2017

    Alex Kendall and Yarin Gal. What Uncertainties do we need in Bayesian Deep Learning for Computer Vision? Advances in Neural Information Processing Systems, 2017

    2017

  18. [26]

    Adam: A Method for Stochastic Optimization

    Diederik P Kingma and Jimmy Ba. Adam: A Method for Stochastic Optimization. InInternational Conference on Learning Representations, 2015

    2015

  19. [27]

    Variational Dropout and the Local Reparameterization Trick.Advances in Neural Information Processing Systems, 2015

    Durk P Kingma, Tim Salimans, and Max Welling. Variational Dropout and the Local Reparameterization Trick.Advances in Neural Information Processing Systems, 2015

    2015

  20. [28]

    Bayesian Neural Networks.Biological Cybernetics, 1989

    Igor Kononenko. Bayesian Neural Networks.Biological Cybernetics, 1989

    1989

  21. [29]

    The cifar-10 dataset.online: http://www

    Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton. The cifar-10 dataset.online: http://www. cs. toronto. edu/kriz/cifar. html, 2014

    2014

  22. [30]

    Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles.Advances in Neural Information Processing Systems, 2017

    Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles.Advances in Neural Information Processing Systems, 2017

    2017

  23. [31]

    Enhancing the Reliability of Out-Of-Distribution Image Detection in Neural Networks.arXiv preprint arXiv:1706.02690, 2017

    Shiyu Liang, Yixuan Li, and Rayadurgam Srikant. Enhancing the Reliability of Out-Of-Distribution Image Detection in Neural Networks.arXiv preprint arXiv:1706.02690, 2017

    2017 arXiv

  24. [32]

    Uncertainty Quantification Metrics for Deep regression.Pattern Recognition Letters, 186, 2024

    Simon Kristoffersson Lind, Ziliang Xiong, Per-Erik Forssén, and Volker Krüger. Uncertainty Quantification Metrics for Deep regression.Pattern Recognition Letters, 186, 2024

    2024

  25. [33]

    Human-in-the-Loop Machine Learning: A State of the Art.Artificial Intelligence Review, 2023

    Eduardo Mosqueira-Rey, Elena Hernández-Pereira, David Alonso-Ríos, José Bobes-Bascarán, and Ángel Fernández-Leal. Human-in-the-Loop Machine Learning: A State of the Art.Artificial Intelligence Review, 2023

    2023

  26. [34]

    Deep Deterministic Uncertainty: A New Simple Baseline

    Jishnu Mukhoti, Andreas Kirsch, Joost van Amersfoort, Philip HS Torr, and Yarin Gal. Deep Deterministic Uncertainty: A New Simple Baseline. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023

    2023

  27. [35]

    Reading Digits in Natural Images with Unsupervised Feature Learning

    Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Baolin Wu, Andrew Y Ng, et al. Reading Digits in Natural Images with Unsupervised Feature Learning. InNeurIPS workshop on deep learning and unsupervised feature learning, 2011

    2011

  28. [36]

    Can You Trust Your Model’s Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift.Advances in neural information processing systems, 2019

    Yaniv Ovadia, Emily Fertig, Jie Ren, Zachary Nado, David Sculley, Sebastian Nowozin, Joshua Dillon, Balaji Lakshminarayanan, and Jasper Snoek. Can You Trust Your Model’s Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift.Advances in neural information processin...

    2019

  29. [37]

    Towards understanding neural collapse: The effects of batch normalization and weight decay.arXiv preprint arXiv:2309.04644, 2023

    Leyan Pan and Xinyuan Cao. Towards understanding neural collapse: The effects of batch normalization and weight decay.arXiv preprint arXiv:2309.04644, 2023

    2023 arXiv

  30. [38]

    Understanding Softmax Confidence and Uncertainty.arXiv preprint arXiv:2106.04972, 2021

    Tim Pearce, Alexandra Brintrup, and Jun Zhu. Understanding Softmax Confidence and Uncertainty.arXiv preprint arXiv:2106.04972, 2021

    2021 arXiv

  31. [39]

    Improving the Reliability for Confidence Estimation

    Haoxuan Qu, Yanchao Li, Lin Geng Foo, Jason Kuen, Jiuxiang Gu, and Jun Liu. Improving the Reliability for Confidence Estimation. InEuropean Conference on Computer Vision, 2022

    2022

  32. [40]

    Berg, and Li Fei-Fei

    Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. ImageNet Large Scale Visual Recognition Challenge.International Journal of Computer Vision, 2015. 13

    2015

  33. [41]

    A Less Biased Evaluation of Out-Of-Distribution Sample Detectors.arXiv preprint arXiv:1809.04729, 2018

    Alireza Shafaei, Mark Schmidt, and James J Little. A Less Biased Evaluation of Out-Of-Distribution Sample Detectors.arXiv preprint arXiv:1809.04729, 2018

    2018 arXiv

  34. [42]

    A Mathematical Theory of Communication.The Bell System Technical Journal, 1948

    Claude Elwood Shannon. A Mathematical Theory of Communication.The Bell System Technical Journal, 1948

    1948

  35. [43]

    A Human-centric Perspective on Model Monitoring

    Murtuza N Shergadwala, Himabindu Lakkaraju, and Krishnaram Kenthapadi. A Human-centric Perspective on Model Monitoring. InAAAI Conference on Human Computation and Crowdsourcing, 2022

    2022

  36. [44]

    Epistemic Uncertainty Quantification Tutorial

    Laura P Swiler, Thomas L Paez, and Randall L Mayes. Epistemic Uncertainty Quantification Tutorial. In Proceedings of International Modal Analysis Conference, 2009

    2009

  37. [45]

    Can You Rely on Your Model Evaluation? Improving Model Evaluation with Synthetic Test Data.Advances in Neural Information Processing Systems, 2024

    Boris van Breugel, Nabeel Seedat, Fergus Imrie, and Mihaela van der Schaar. Can You Rely on Your Model Evaluation? Improving Model Evaluation with Synthetic Test Data.Advances in Neural Information Processing Systems, 2024

    2024

  38. [46]

    Mitigating neural network overconfidence with logit normalization

    Hongxin Wei, Renchunzi Xie, Hao Cheng, Lei Feng, Bo An, and Yixuan Li. Mitigating neural network overconfidence with logit normalization. InInternational Conference on Machine Learning, 2022

    2022

  39. [47]

    A Novel Characterization of the Population Area Under the Risk Coverage Curve (AURC) and Rates of Finite Sample Estimators.arXiv preprint arXiv:2410.15361, 2024

    Han Zhou, Jordy Van Landeghem, Teodora Popordanoska, and Matthew B Blaschko. A Novel Characterization of the Population Area Under the Risk Coverage Curve (AURC) and Rates of Finite Sample Estimators.arXiv preprint arXiv:2410.15361, 2024

    arXiv

  40. [48]

    S Kevin Zhou, Hayit Greenspan, and Dinggang Shen.Deep learning for medical image analysis. 2023

    2023

  41. [49]

    Unsupervised Self-driving Attention Prediction via Uncertainty Mining and Knowledge Embedding

    Pengfei Zhu, Mengshi Qi, Xia Li, Weijian Li, and Huadong Ma. Unsupervised Self-driving Attention Prediction via Uncertainty Mining and Knowledge Embedding. InProceedings of the IEEE/CVF International Conference on Computer Vision, 2023. 14 Supplementary Material Theorem detail...

    2023

  42. [50]

    Norm Concentration:As the dimensionality d→ ∞, the Euclidean norm||x|| concentrates around √ dσ , meaning that for any smallϵ >0,P r ||x|| − √ dσ < ϵ √ dσ →1

  43. [51]

    Specifically, the probability that a randomly chosen point lies within a thin shell of radius √ dσ±ϵapproaches 1 asd→ ∞

    Surface Concentration:Consequently, as d grows large, the pointsx become increasingly concentrated near the surface of a hypersphere with radius √ dσ centered at the origin. Specifically, the probability that a randomly chosen point lies within a thin shell of radius √ dσ±ϵapp...

  44. [52]

    This implies that the random vectorx is concentrated within a thin shell of radius√ dσ±ϵ √ dσ

    Surface Concentration From the norm concentration result, we know that the Euclidean norm∥x∥ is highly likely to lie in the interval [ √ dσ−ϵ √ dσ, √ dσ + ϵ √ dσ]. This implies that the random vectorx is concentrated within a thin shell of radius√ dσ±ϵ √ dσ. Moreformally, let ...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.