REVIEW 2 cited by
When are Bayesian model probabilities overconfident?
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
When are Bayesian model probabilities overconfident?
read the original abstract
Bayesian model comparison is often based on the posterior distribution over the set of compared models. This distribution is often observed to concentrate on a single model even when other measures of model fit or forecasting ability indicate no strong preference. Furthermore, a moderate change in the data sample can easily shift the posterior model probabilities to concentrate on another model. We document overconfidence in two high-profile applications in economics and neuroscience. To shed more light on the sources of overconfidence we derive the sampling variance of the Bayes factor in univariate and multivariate linear regression. The results show that overconfidence is likely to happen when i) the compared models give very different approximations of the data-generating process, ii) the models are very flexible with large degrees of freedom that are not shared between the models, and iii) the models underestimate the true variability in the data.
Forward citations
Cited by 2 Pith papers
-
Robust Bayesian Predictive Model Selection using Bregman Divergence
A Bregman-divergence generalization of ELPD enables robust predictive model selection by tuning sensitivity to tail mismatch via a parameter β.
-
Confidence Calibration in Large Language Models
LLMs show average overconfidence moderated by a strong hard-easy effect, quantified across tasks and via the new LifeEval actuarial-probability benchmark.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.