CUI uses conformal prediction sets, with a hand-tuned coverage compensation, to measure uncertainty and reweight adaptation in continual test-time adaptation, improving error rates on three corruption benchmarks.
Test-time Recalibration of Conformal Predictors Under Distribution Shift Based on Unlabeled Examples
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
Modern image classifiers are very accurate, but the predictions come without uncertainty estimates. Conformal predictors provide uncertainty estimates by computing a set of classes containing the correct class with a user-specified probability based on the classifier's probability estimates. To provide such sets, conformal predictors often estimate a cutoff threshold for the probability estimates based on a calibration set. Conformal predictors guarantee reliability only when the calibration set is from the same distribution as the test set. Therefore, conformal predictors need to be recalibrated for new distributions. However, in practice, labeled data from new distributions is rarely available, making calibration infeasible. In this work, we consider the problem of predicting the cutoff threshold for a new distribution based on unlabeled examples. While it is impossible in general to guarantee reliability when calibrating based on unlabeled examples, we propose a method that provides excellent uncertainty estimates under natural distribution shifts, and provably works for a specific model of a distribution shift.
citation-role summary
baseline 1
citation-polarity summary
fields
cs.LG 1years
2025 1verdicts
CONDITIONAL 1roles
baseline 1polarities
baseline 1representative citing papers
citing papers explorer
-
Conformal Uncertainty Indicator for Continual Test-Time Adaptation
CUI uses conformal prediction sets, with a hand-tuned coverage compensation, to measure uncertainty and reweight adaptation in continual test-time adaptation, improving error rates on three corruption benchmarks.