REVIEW 2 cited by
Calibration Error for Decision Making
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
abstract
Calibration allows predictions to be reliably interpreted as probabilities by decision makers. We propose a decision-theoretic calibration error, the Calibration Decision Loss (CDL), defined as the maximum improvement in decision payoff obtained by calibrating the predictions, where the maximum is over all payoff-bounded decision tasks. Vanishing CDL guarantees the payoff loss from miscalibration vanishes simultaneously for all downstream decision tasks. We show separations between CDL and existing calibration error metrics, including the most well-studied metric Expected Calibration Error (ECE). Our main technical contribution is a new efficient algorithm for online calibration that achieves near-optimal $O(\frac{\log T}{\sqrt{T}})$ expected CDL, bypassing the $\Omega(T^{-0.472})$ lower bound for ECE by Qiao and Valiant (2021).
Forward citations
Cited by 2 Pith papers
-
Full Swap Regret and Discretized Calibration
New algorithms minimize swap regret against arbitrary functions on convex action sets, yielding O~(T^{1/3}) ℓ2-calibration error and O~(max(T^{1/3}, sqrt(ǫT))) discretized-calibration error.
-
Persuasive Prediction via Decision Calibration
A data-driven sender can learn a near-optimal decision-calibrated predictor without knowing the prior, but the proof as written has a critical Lagrangian error and the Bayesian benchmark is restricted by construction.
Discussion (0). Continue with ORCID to comment.