Pith. sign in

REVIEW 5 major objections 6 minor 53 references

This paper proves that every optimal abstention decision under coverage, risk, or regret constraints is a threshold on a convex combination of the true aleatoric and epistemic uncertainties, unifying selective classification with the episte

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 00:58 UTC pith:SFFZUBFA

load-bearing objection Clean unification theorem; empirical rank-inversion claim is real but conditional on the annotator proxy—deserves a serious referee. the 5 major comments →

arxiv 2607.14817 v1 pith:SFFZUBFA submitted 2026-07-16 cs.LG cs.AI

Evaluating Epistemic Uncertainty: Beyond OOD Detection and Active Learning

Jakub Paplh\'am , Willem Waegeman , Eyke H\"ullermeier , Vojt\v{e}ch Franc This is my paper
classification cs.LG cs.AI
keywords epistemic uncertaintyaleatoric uncertaintyselective classificationregretreject optionuncertainty disentanglementOOD detectionactive learning
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper is trying to establish that epistemic uncertainty only becomes meaningful when paired with the action it informs, and that the right action-based measure is reducible error, or regret. Its central theoretical result is a unification: every optimal abstention policy under budgets on coverage, risk, or regret is a threshold on one convex combination of true aleatoric and epistemic uncertainty, so selective classification and the epistemic reject-option are two endpoints of a single family. The paper then shows that popular surrogate benchmarks, OOD detection and active learning, have Bayes-optimal scores that reject different regions of the input space, and that rankings from those tasks can invert the rankings obtained by measuring actual regret on datasets with dense human labels. It also argues that rank correlation between learned aleatoric and epistemic components does not predict whether the components are operationally useful, and proposes the distance from a method's achievable risk-regret-coverage surface to the oracle surface, the Pareto-gap, as the diagnostic. A reader should care because evaluation practices shape which uncertainty methods get adopted, and this paper gives a decision-theoretic basis for choosing the evaluation.

Core claim

Theorem 1 is the paper's central claim: for any feasible instance of the coverage-maximization, risk-minimization, or regret-minimization problems, an optimal selector exists of the form C*(x)=1{(1−λ)A*ℓ(x)+λE*ℓ(x)≤τ} with λ∈[1/2,1]. Applying the standard optimality conditions to the three constrained problems yields this shared form, with λ=1/2 recovering selective classification and λ=1 recovering the epistemic reject-option. On the empirical side, the paper discovers that the score that optimally rejects for OOD detection (input likelihood ratio) and the scores that optimally acquire data (mutual information or expected error reduction) are not the same as the score that optimally isolate

What carries the argument

The central object is the unified selector C*(x)=1{(1−λ)A*ℓ(x)+λE*ℓ(x)≤τ}, λ∈[1/2,1], derived by applying KKT optimality conditions to three constrained optimization problems. It converts the choice of operating point in coverage, risk, and regret into a choice of λ and τ, and it generates the Pareto-optimal surface S* of achievable (coverage, risk, regret) triples. The companion diagnostic is the Pareto-gap, the distance between S* and a method's achievable surface built from linear combinations of its learned components. The benchmark machinery that makes exact frequentist regret computable is the availability of dense human annotation distributions, which supply an oracle Bayes predictor.

Load-bearing premise

The evaluation treats the dense human annotation distribution on each input as the true data-generating p*(y|x); if real deployment labels come from a different process, such as a deterministic labeler or another annotator population, the reported regrets measure agreement with those annotators rather than with the actual task.

What would settle it

Take a dataset with both dense human annotations and a known deterministic ground truth, such as synthetic data with a controlled generative function and noise, compute AuReC for several methods against each target, and compare rankings. If the ranking that looks best under human labels is not best under the deterministic truth, or if a random component pair achieves a small Pareto-gap, the framework's empirical foundation would be undermined.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • Any optimal rejection strategy under coverage, risk, or regret budgets is captured by the thresholded convex combination; no other scoring function needs to be considered.
  • Proxy evaluations through OOD detection or active learning can invert method rankings, so they should not be treated as surrogates for epistemic regret.
  • High rank correlation between learned components is not proof of failed disentanglement, since the ground-truth components themselves can be strongly correlated.
  • The Pareto-gap offers a way to compare decompositions by whether linear combinations of their components can approach the oracle risk-regret-coverage surface.
  • Exact frequentist regret evaluation requires dense annotation distributions; without them, true reducible error cannot be computed for real data.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A natural extension is to connect the Pareto-gap to active learning acquisition decisions: the paper shows EER and regret are globally different objectives, but does not test whether minimizing the Pareto-gap improves acquisition efficiency; that remains a testable hypothesis.
  • If this line is right, benchmark suites for uncertainty quantification should include dense-label tasks as a first-class evaluation, since single-label datasets cannot validate epistemic estimates; this would change how UQ methods are selected in practice.
  • The theorem's λ∈[1/2,1] constraint applies to ground-truth components, while the empirical evaluation permits unrestricted weights for learned components; this implicitly assumes miscalibration can be repaired linearly, which is worth testing on poorly calibrated models.
  • The one-dimensional sandbox predicts that similar rejection-region divergence will appear in higher dimensions, but the magnitude likely depends on how strongly marginal density, noise, and function complexity overlap; that is a concrete, checkable prediction.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper argues that standard proxy evaluations of epistemic uncertainty (OOD detection, active learning) are misaligned with the decision-theoretic notion of epistemic uncertainty as reducible error (regret). It builds on the epistemic reject-option framework of Franc and Paplham, defines aleatoric and epistemic components via a Bayesian excess-risk decomposition, and proves (Theorem 1, Eq. 10) that the optimal selector for any of three coverage/risk/regret constrained problems is a threshold on a convex combination of the true aleatoric and epistemic uncertainties, with the mixing weight restricted to [1/2, 1]. The paper then proposes a Pareto-gap diagnostic for assessing the joint operational utility of learned aleatoric/epistemic decompositions, arguing that Spearman rank correlation is not a reliable disentanglement measure. Empirically, the authors benchmark MC-Dropout, Evidential Networks, Deep Ensembles, and DDU on CIFAR-10H, DCIC, and APPA-REAL using dense human annotations as p*(y|x), and compare regret-based rankings with OOD and active-learning rankings adopted from a concurrent anonymous submission. They report rank inversions, e.g., Evidential Networks being best on AuReC but worst on OOD/AL proxies.

Significance. If the main theorem is correct, it is a clean and useful unification: selective classification (total-uncertainty thresholding) and epistemic reject-option (epistemic-uncertainty thresholding) become endpoints of a single family parameterized by lambda, and the Pareto-gap provides a principled way to evaluate uncertainty decompositions by operational utility rather than by correlation. The closed-form sandbox in Section 4.2 is a nice illustration that the Bayes-optimal scorers for different tasks can reject disjoint regions. The paper also makes a fair, if not fully established, point that high rank correlation between learned components need not imply poor functional behavior. However, the empirical case is substantially weakened by (i) treating dense human annotations as exact p*(y|x) without uncertainty quantification or robustness analysis, (ii) relying on an anonymous concurrent submission for all proxy-task numbers, (iii) comparing regret results on image datasets with proxy results on different image datasets and tabular data, and (iv) omitting error bars from the central tables. These issues are load-bearing because the paper's headline contribution is not just the theorem

major comments (5)
  1. [Section 6.1, Tables 1-3] The paper repeatedly calls the regret values 'exact' and treats dense human annotations as the true p*(y|x). But human annotation distributions are finite-sample estimates and may not equal the deployment distribution (different annotator population, deterministic labeling, etc.). The limitations paragraph concedes this, but the central empirical claims (e.g., 'Evidential networks outperform all other baselines' and 'DDU performs worst in pure regret') are computed against this proxy. Since the whole benchmark is built on this identification, the paper needs at least a robustness check (e.g., bootstrap over annotators, or an explicit statement and discussion of how rankings depend on the annotation distribution). As it stands, the reported AuReC and Pareto-gap values are conditional on the annotator proxy, not on the deployment task.
  2. [Section 6.3, Tables 4-5] The proxy-task results (OOD detection, active learning) are adopted from an anonymous concurrent submission [Anonymous, 2026] and are evaluated on different datasets than the regret benchmarks: OOD on OpenOOD CIFAR-10/ImageNet, active learning on OpenML tabular data, whereas regret is evaluated on CIFAR-10H, DCIC, and APPA-REAL. The headline rank inversions (e.g., Evidential being best on CIFAR-10H AuReC but worst on OpenML NAUC) therefore conflate task misalignment with dataset/domain shift. To support the claim that 'decision-theoretic rankings can disagree with proxy-task rankings,' the comparison should be controlled on the same data distribution, or at minimum the authors should show that the inversions persist when proxy tasks and regret are measured on the same inputs. This is a load-bearing issue for the empirical contribution.
  3. [Tables 1-3] All three central tables report values 'averaged across 5 seeds' but no standard deviations, confidence intervals, or significance tests. Point estimates such as CIFAR-10H CE AuReC 0.0673 (Evidential) vs. 0.0752 (Ensemble) could easily be within seed noise. The paper makes strong ranking claims ('Evidential networks outperform all other baselines,' 'DDU performs the worst'), but without variance information the reader cannot assess whether these differences are meaningful. Standard deviations and pairwise significance tests (or at least a scatter of per-seed values) are necessary for the central empirical claim of rank inversion.
  4. [Section 5.2, Eq. (11); Section 6.2] The Pareto-gap is computed by optimizing the linear-combination weights (w1, w2) and threshold tau directly on the test set. While the paper explicitly discloses this as an oracle evaluation, it is still a fitted quantity: the gap measures the capacity of the estimated components under test-set overfitting, not their performance under a deployment protocol. The paper does not specify the search procedure, the number of surface points used in the IGD+ calculation, or the sensitivity of the ranking to the optimization. Without this, the Pareto-gap results in Tables 1-3 could reflect optimization artifacts. A bootstrap or nested protocol would strengthen the diagnostic.
  5. [Section 6.4] The claim that Spearman rank correlation is 'largely blind' to joint operational utility is supported by a single illustrative example: Evidential Networks on CIFAR-10H with CE loss, where rs(A-hat,E-hat)=0.999 yet P-gap is the best. The paper does not provide a systematic analysis across the 8 DCIC datasets and APPA-REAL (Tables 2-3 contain many relevant rows). For instance, DDU has negative rs on many DCIC datasets, which would traditionally indicate good disentanglement, yet its P-gap is often worse than other methods. A scatter plot or correlation between rs and P-gap across all datasets/losses would be needed to substantiate the general claim that rank correlation does not predict operational utility.
minor comments (6)
  1. [Eq. (7)] The symbol rho is used both for the coverage functional rho(C) and for the risk budget rho in (P_rho). This is confusing; use, e.g., b_R and b_Re for the budgets.
  2. [Definition 1, Eq. (2)] The notation h(x,theta) = argmin over y-hat is ambiguous for losses like cross-entropy where the prediction is a distribution rather than a point. Clarify the prediction space, especially since the experiments treat APPA-REAL as a 101-class distribution.
  3. [Appendix B] The Dinkelbach argument is sketched too briefly. In particular, the claim that the coefficients c1 and c2 remain 'identically the same' after fractional linearization needs a more formal derivation; the dependence of q* on the optimal selector C* should be stated explicitly.
  4. [Figure 1] The bottom-right panel with the five rejection regions is hard to read in monochrome; consider adding distinct hatching or direct callouts to the regions A, B, C.
  5. [Conclusion] The sentence 'the used metrics are strictly benchmarking tools and cannot single-label data' is incomplete and should be revised (e.g., 'cannot be applied to single-label data').
  6. [References] The paper relies on 'probly [Anonymous, 2026]' and 'Anonymous concurrent submission' for a substantial portion of the empirical results. If the review process allows, the authors should make the relevant framework/results available or at least provide an appendix with the exact implementation and raw numbers, otherwise reproducibility is compromised.

Circularity Check

0 steps flagged

No circular derivation; the main theorem is a KKT consequence of the definitions, and the oracle/fitted evaluation measures are explicitly disclosed as such.

full rationale

Theorem 1 is derived from the definitions T*=A*+E*, Risk, Regret, and Coverage via Lagrange/KKT analysis, not from any fitted constant or proxy performance. The proof in the appendix algebraically converts the Lagrangian into a threshold on (1-λ)A*+λE*, so the result is a genuine mathematical consequence rather than an input renamed as output. The paper does cite the authors' own earlier framework (Franc and Paplhám 2025) for the AuReC metric and the epistemic reject-option, but the present derivation does not depend on that citation for its proof; the endpoint λ=1 recovers prior work rather than importing it as evidence. The Pareto-gap is computed by fitting linear weights and thresholds on the test set, but the paper explicitly labels this an 'oracle evaluation' for measuring capacity, not a deployment prediction, so it is not a hidden fitted-input-called-prediction. The use of dense human annotations as p*(y|x) is a strong identification assumption and is honestly flagged as a limitation in Section 7, but treating a proxy distribution as ground truth is not circularity under the rules used here. Overall, no load-bearing step reduces to its own inputs; the only mild concern is the heavy dependence on the authors' prior framework, which is not itself circular.

Axiom & Free-Parameter Ledger

1 free parameters · 5 axioms · 0 invented entities

No invented physical entities. The Pareto-gap and achievable surface are mathematical evaluation quantities, not entities with independent falsifiable predictions. The main free parameters are the oracle weights in the Pareto-gap diagnostic; the theorem itself has no fitted constants.

free parameters (1)
  • Pareto-gap oracle weights (w1, w2) and threshold τ = not reported
    Section 6.2: 'we search for the optimal linear combination weights (w1,w2) and thresholds τ directly on the test set.' These are test-set-optimized values used to compute every Pareto-gap in Tables 1–3; they inflate apparent method utility and are not deployment parameters.
axioms (5)
  • standard math KKT conditions apply to the selector optimization problems and Lagrange multipliers exist
    Used in Appendix A proof of Theorem 1; requires constraint qualification, which is not discussed.
  • standard math The score (1−λ)A*+λE* has continuous distribution under x~p* for every λ∈[1/2,1]
    Technical assumption in Theorem 1; fails for discrete inputs but the authors claim LP tie-handling covers it.
  • domain assumption Epistemic uncertainty is correctly defined as expected regret / reducible error under the deployment loss ℓ
    Adopted from Franc & Paplhám (2025) and Hofman et al.; the entire evaluation framework treats this as the operational goal.
  • domain assumption Dense human annotation distributions approximate the true p*(y|x) closely enough that regret computed against them is 'exact'
    Section 6.1 says dense annotations allow 'the exact Bayes-optimal predictor and the true regret'; this is the load-bearing empirical premise.
  • standard math Linearization of normalized metrics via Dinkelbach preserves the optimal selector and the λ bound
    Appendix B proof relies on Dinkelbach (1967).

pith-pipeline@v1.3.0-alltime-deepseek · 18450 in / 16526 out tokens · 133540 ms · 2026-08-02T00:58:35.683663+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of Evaluating Epistemic Uncertainty: Beyond OOD Detection and Active Learning." pith.science (2026). https://pith.science/paper/SFFZUBFA

@misc{pith2026260714817,
  author       = {Pith},
  title        = {Pith review of: Evaluating Epistemic Uncertainty: Beyond OOD Detection and Active Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SFFZUBFA}},
  note         = {Machine review of arXiv:2607.14817}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Current evaluation of epistemic uncertainty relies on tasks such as out-ofdistribution detection and active learning. However, the Bayes-optimal decision strategies for these tasks do not coincide with the scores commonly used to quantify epistemic uncertainty. Building on the epistemic reject-option framework, we evaluate epistemic uncertainty using its ability to identify regret, the reducible error. Formulating selective prediction as a constrained optimization over coverage, expected risk, and regret, we prove the optimal selector is a thresholded convex combination of the ground-truth aleatoric and epistemic uncertainties. This theoretical unification exposes a weakness in recent uncertainty disentanglement literature: we demonstrate that standard correlation metrics between learned components do not necessarily predict their actual operational utility. We instead propose to evaluate the achievable risk, regret, coverage surface of the decomposition as a diagnostic for joint disentanglement and utility. Benchmarking standard methods on datasets with dense human annotations reveals that decision-theoretic rankings can disagree substantially with proxy-task rankings, including pairwise rank inversions between methods that are top-ranked on one criterion and bottom-ranked on other.

Figures

Figures reproduced from arXiv: 2607.14817 by Eyke H\"ullermeier, Jakub Paplh\'am, Vojt\v{e}ch Franc, Willem Waegeman.

Figure 1
Figure 1. Figure 1: Sandbox: Rejection regions of five distinct Bayes-optimal scores at 80% coverage. Left: A closed-form Bayesian regression setup featuring a true oscillating function (f ∗ ), heteroscedas￾tic noise (±σ(x)), and discrete training samples. We annotate three regions of the x-axis: (A) a fast-oscillating target with low noise, (B) an artificially sparse data well, and (C) a region of high aleatoric noise. Top R… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

53 extracted references · 1 canonical work pages

  1. [1]

    arXiv preprint arXiv:2511.04855 , year=

    Epistemic Reject Option Prediction , author=. arXiv preprint arXiv:2511.04855 , year=

    Pith/arXiv arXiv

  2. [2]

    Proceedings of The 27th International Conference on Artificial Intelligence and Statistics , pages =

    Integrating Uncertainty Awareness into Conformalized Quantile Regression , author =. Proceedings of The 27th International Conference on Artificial Intelligence and Statistics , pages =. 2024 , editor =

    2024

  3. [3]

    Decomposition of Uncertainty in

    Depeweg, Stefan and Hern. Decomposition of Uncertainty in. International Conference on Machine Learning , pages=

  4. [4]

    Proceedings of the Thirty-Ninth Conference on Uncertainty in Artificial Intelligence (UAI) , volume=

    Quantifying Aleatoric and Epistemic Uncertainty in Machine Learning: Are Conditional Entropy and Mutual Information Appropriate Measures? , author=. Proceedings of the Thirty-Ninth Conference on Uncertainty in Artificial Intelligence (UAI) , volume=

  5. [5]

    Machine Learning , volume=

    Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methods , author=. Machine Learning , volume=

  6. [6]

    Advances in Neural Information Processing Systems , year=

    Selective Classification for Deep Neural Networks , author=. Advances in Neural Information Processing Systems , year=

  7. [7]

    International Conference on Learning Representations , year=

    Bias-Reduced Uncertainty Estimation for Deep Neural Classifiers , author=. International Conference on Learning Representations , year=

  8. [8]

    IEEE Transactions on Information Theory , volume=

    On Optimum Recognition Error and Reject Tradeoff , author=. IEEE Transactions on Information Theory , volume=

  9. [9]

    Computer Vision – ECCV 2024: 18th European Conference, Milan, Italy, September 29 – October 4, 2024, Proceedings, Part LXXXIV , pages =

    Franc, Vojtech and Paplham, Jakub and Prusa, Daniel , title =. Computer Vision – ECCV 2024: 18th European Conference, Milan, Italy, September 29 – October 4, 2024, Proceedings, Part LXXXIV , pages =. 2024 , isbn =. doi:10.1007/978-3-031-72907-2_25 , abstract =

    work page doi:10.1007/978-3-031-72907-2_25 2024

  10. [10]

    Augmenting Softmax Information for Selective Classification with Out-of-Distribution Data

    Xia, Guoxuan and Bouganis, Christos-Savvas. Augmenting Softmax Information for Selective Classification with Out-of-Distribution Data. Computer Vision -- ACCV 2022. 2023

    2022

  11. [11]

    On Nonlinear Fractional Programming , urldate =

    Werner Dinkelbach , journal =. On Nonlinear Fractional Programming , urldate =

  12. [12]

    Schmarje, Lars and Grossmann, Vasco and Zelenka, Claudius and Dippel, Sabine and Kiko, Rainer and Oszust, Mariusz and Pastell, Matti and Stracke, Jenny and Valros, Anna and Volkmann, Nina and Koch, Reinahrd , journal =

  13. [13]

    and Gal, Yarin , booktitle=

    Mukhoti, Jishnu and Kirsch, Andreas and van Amersfoort, Joost and Torr, Philip H.S. and Gal, Yarin , booktitle=. Deep Deterministic Uncertainty: A New Simple Baseline , year=

  14. [14]

    Proceedings of the 31st International Conference on Neural Information Processing Systems , pages =

    Lakshminarayanan, Balaji and Pritzel, Alexander and Blundell, Charles , title =. Proceedings of the 31st International Conference on Neural Information Processing Systems , pages =. 2017 , isbn =

    2017

  15. [15]

    Proceedings of The 33rd International Conference on Machine Learning , pages =

    Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning , author =. Proceedings of The 33rd International Conference on Machine Learning , pages =. 2016 , editor =

    2016

  16. [16]

    Proceedings of the 32nd International Conference on Neural Information Processing Systems , pages =

    Sensoy, Murat and Kaplan, Lance and Kandemir, Melih , title =. Proceedings of the 32nd International Conference on Neural Information Processing Systems , pages =. 2018 , publisher =

    2018

  17. [17]

    2023 , url=

    Salem Lahlou and Moksh Jain and Hadi Nekoei and Victor I Butoi and Paul Bertin and Jarrid Rector-Brooks and Maksym Korablyov and Yoshua Bengio , journal=. 2023 , url=

    2023

  18. [18]

    Journal of Machine Learning Research , volume=

    Optimal Strategies for Reject Option Classifiers , author=. Journal of Machine Learning Research , volume=

  19. [19]

    2026 , eprint=

    Position: Epistemic uncertainty estimation methods are fundamentally incomplete , author=. 2026 , eprint=

    2026

  20. [20]

    2026 , eprint=

    Measuring Orthogonality as the Blind-Spot of Uncertainty Disentanglement , author=. 2026 , eprint=

    2026

  21. [21]

    The Thirteenth International Conference on Learning Representations , year=

    From Risk to Uncertainty: Generating Predictive Uncertainty Measures via Bayesian Estimation , author=. The Thirteenth International Conference on Learning Representations , year=

  22. [22]

    Traub, Jeremias and Bungert, Till J. and L\". Overcoming Common Flaws in the Evaluation of Selective Classification Systems , url =. Advances in Neural Information Processing Systems , doi =

  23. [23]

    International Conference on Algorithmic Learning Theory , year=

    Learning with Rejection , author=. International Conference on Algorithmic Learning Theory , year=

  24. [24]

    arXiv preprint arXiv:2404.12215 , year=

    Quantifying Aleatoric and Epistemic Uncertainty with Proper Scoring Rules , author=. arXiv preprint arXiv:2404.12215 , year=

    Pith/arXiv arXiv

  25. [25]

    arXiv preprint arXiv:2402.10727 , year=

    Predictive Uncertainty Quantification via Risk Decompositions for Strictly Proper Scoring Rules , author=. arXiv preprint arXiv:2402.10727 , year=

    Pith/arXiv arXiv

  26. [26]

    International Conference on Artificial Intelligence and Statistics , year=

    Uncertainty Estimates of Predictions via a General Bias-Variance Decomposition , author=. International Conference on Artificial Intelligence and Statistics , year=

  27. [27]

    International Conference on Machine Learning , year=

    On Second-Order Scoring Rules for Epistemic Uncertainty Quantification , author=. International Conference on Machine Learning , year=

  28. [28]

    AAAI Conference on Artificial Intelligence , year=

    Uncertainty Quantification for Machine Learning: One Size Does Not Fit All , author=. AAAI Conference on Artificial Intelligence , year=

  29. [29]

    Advances in Neural Information Processing Systems Datasets and Benchmarks Track , year=

    Benchmarking Uncertainty Disentanglement: Specialized Uncertainties for Specialized Tasks , author=. Advances in Neural Information Processing Systems Datasets and Benchmarks Track , year=

  30. [30]

    International Conference on Learning Representations , year=

    What Can We Learn from the Selective Prediction and Uncertainty Estimation Performance of 523 ImageNet Classifiers? , author=. International Conference on Learning Representations , year=

  31. [31]

    arXiv preprint arXiv:1112.5745 , year=

    Bayesian Active Learning for Classification and Preference Learning , author=. arXiv preprint arXiv:1112.5745 , year=

    Pith/arXiv arXiv

  32. [32]

    International Conference on Machine Learning , year=

    Toward Optimal Active Learning Through Sampling Estimation of Error Reduction , author=. International Conference on Machine Learning , year=

  33. [33]

    arXiv preprint arXiv:2501.18268 , year=

    Reducing Aleatoric and Epistemic Uncertainty through Multi-modal Data Acquisition , author=. arXiv preprint arXiv:2501.18268 , year=

    arXiv

  34. [34]

    arXiv preprint arXiv:2408.12175 , year=

    How Disentangled are Your Classification Uncertainties? , author=. arXiv preprint arXiv:2408.12175 , year=

    arXiv

  35. [35]

    IEEE/CVF International Conference on Computer Vision , year=

    Human Uncertainty Makes Classification More Robust , author=. IEEE/CVF International Conference on Computer Vision , year=

  36. [36]

    Lucas Beyer and Olivier J. H. Are we done with ImageNet? , journal =. 2020 , url =. 2006.07159 , timestamp =

    Pith/arXiv arXiv 2020

  37. [37]

    , author=

    Apparent and real age estimation in still images with deep residual regressors on APPA-REAL database. , author=. 12th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), 2017 , year=

    2017

  38. [38]

    What Uncertainties Do We Need in

    Kendall, Alex and Gal, Yarin , booktitle=. What Uncertainties Do We Need in

  39. [39]

    Hendrickx, Kilian and Perini, Lorenzo and Van der Plas, Dries and Meert, Wannes and Davis, Jesse , title =. Mach. Learn. , month = mar, pages =. 2024 , issue_date =. doi:10.1007/s10994-024-06534-x , abstract =

    doi:10.1007/s10994-024-06534-x 2024

  40. [40]

    Fundamentals of Nonparametric Bayesian Inference , author=

  41. [41]

    Karush, William , biburl =

  42. [42]

    and Tucker, Albert W

    Kuhn, Harold W. and Tucker, Albert W. , title =. Proceedings of the Second Berkeley Symposium on Mathematical Statistics and Probability , year =

  43. [43]

    Modified Distance Calculation in Generational Distance and Inverted Generational Distance

    Ishibuchi, Hisao and Masuda, Hiroyuki and Tanigaki, Yuki and Nojima, Yusuke. Modified Distance Calculation in Generational Distance and Inverted Generational Distance. Evolutionary Multi-Criterion Optimization. 2015

    2015

  44. [44]

    The 40th Conference on Uncertainty in Artificial Intelligence , year=

    How to Fix a Broken Confidence Estimator: Evaluating Post-hoc Methods for Selective Classification with Deep Neural Networks , author=. The 40th Conference on Uncertainty in Artificial Intelligence , year=

  45. [45]

    arXiv preprint arXiv:2306.09301 , year=

    OpenOOD v1.5: Enhanced Benchmark for Out-of-Distribution Detection , author=. arXiv preprint arXiv:2306.09301 , year=

    Pith/arXiv arXiv

  46. [46]

    7th International Conference on Automatic Face and Gesture Recognition (FGR06) , year=

    MORPH: a longitudinal image database of normal adult age-progression , author=. 7th International Conference on Automatic Face and Gesture Recognition (FGR06) , year=

  47. [47]

    2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) , year=

    AgeDB: The First Manually Collected, In-the-Wild Age Database , author=. 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) , year=

    2017

  48. [48]

    Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) , year =

    Photo Dating by Facial Age Aggregation , author =. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) , year =

  49. [49]

    Image Vis

    Investigating Bias in Deep Face Analysis: The KANFace Dataset and Empirical Study , author=. Image Vis. Comput. , year=

  50. [50]

    2023 , Eprint =

    Maksim Kuprashevich and Irina Tolstykh , Title =. 2023 , Eprint =

    2023

  51. [51]

    , title =

    Anonymous, A. , title =

  52. [52]

    Patterns , volume =

    Bernd Bischl and Giuseppe Casalicchio and Taniya Das and Matthias Feurer and Sebastian Fischer and Pieter Gijsbers and Subhaditya Mukherjee and Andreas C Müller and László Németh and Luis Oala and Lennart Purucker and Sahithya Ravi and Jan N van Rijn and Prabhant Singh and Joaquin Vanschoren and Jos van der Velde and Marcel Wever , title =. Patterns , vol...

    arXiv 2025

  53. [53]

    Yucen Lily Li and Daohan Lu and Polina Kirichenko and Shikai Qiu and Tim G. J. Rudner and C. Bayan Bruss and Andrew Gordon Wilson , booktitle=. Position: Supervised Classifiers Answer the Wrong Questions for