REVIEW 4 cited by
Benchmarking Uncertainty Disentanglement: Specialized Uncertainties for Specialized Tasks
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Uncertainty quantification, once a singular task, has evolved into a spectrum of tasks, including abstained prediction, out-of-distribution detection, and aleatoric uncertainty quantification. The latest goal is disentanglement: the construction of multiple estimators that are each tailored to one and only one source of uncertainty. This paper presents the first benchmark of uncertainty disentanglement. We reimplement and evaluate a comprehensive range of uncertainty estimators, from Bayesian over evidential to deterministic ones, across a diverse range of uncertainty tasks on ImageNet. We find that, despite recent theoretical endeavors, no existing approach provides pairs of disentangled uncertainty estimators in practice. We further find that specialized uncertainty tasks are harder than predictive uncertainty tasks, where we observe saturating performance. Our results provide both practical advice for which uncertainty estimators to use for which specific task, and reveal opportunities for future research toward task-centric and disentangled uncertainties. All our reimplementations and Weights & Biases logs are available at https://github.com/bmucsanyi/untangle.
Forward citations
Cited by 4 Pith papers
-
Controllable Diversity in Normalization-Based Implicit Ensembles via Softmax-Temperature Modulation
Softmax-temperature regularisation of sigmoid-bounded normalisation scales controls member diversity in implicit ensembles, matching deep ensembles cheaply across CNNs and transformers.
-
Learning Credal Ensembles via Distributionally Robust Optimization
An ensemble trained with distributionally robust optimization at several reweighting intensities yields credal predictions whose uncertainty better separates in-distribution from out-of-distribution samples.
-
Aleatoric and Epistemic Uncertainty Measures for Ordinal Classification through Binary Reduction
Order-consistent binary reduction, summing entropy or variance uncertainties over all ordered splits, provides competitive aleatoric and epistemic uncertainty measures for ordinal classification.
-
Generative AI for Autonomous Driving: A Review
A review of generative models (VAEs, GANs, diffusion, transformers, LLMs) applied to map generation, scenario generation, trajectory prediction, and motion planning for autonomous driving.
Discussion (0). Continue with ORCID to comment.