Predictive Bayesian inference posteriors concentrate onto a forward-model-dependent quantity and produce miscalibrated credible sets unless the predictive model contains the true data-generating process.
On subjective uncertainty quantification and calibration in natural language generation
5 Pith papers cite this work. Polarity classification is still indexing.
5
Pith papers citing it
citation-role summary
background 1
citation-polarity summary
verdicts
UNVERDICTED 5roles
background 1polarities
background 1representative citing papers
Introduces a Bayesian framework viewing LLM prompts as textual parameters and proposes MHLP, a novel MCMC algorithm using LLM proposals, to perform inference and improve accuracy plus uncertainty quantification on benchmarks.
Empirical study finds verbalized per-token confidence methods in LLMs for MT perform similarly to internal signals on error detection and calibration but show little correlation.
Decision-theoretic uncertainty quantification formalizes evaluation of generative domain adaptation trustworthiness for PPG-based atrial fibrillation classification by linking uncertainty to downstream task utility.
Supervised fine-tuning degrades the correlation between confidence scores and output quality in language models, driven by factors like training distribution similarity rather than true quality.
citing papers explorer
-
Concentration and Calibration in Predictive Bayesian Inference
Predictive Bayesian inference posteriors concentrate onto a forward-model-dependent quantity and produce miscalibrated credible sets unless the predictive model contains the true data-generating process.
-
Textual Bayes: Quantifying Prompt Uncertainty in LLM-Based Systems
Introduces a Bayesian framework viewing LLM prompts as textual parameters and proposes MHLP, a novel MCMC algorithm using LLM proposals, to perform inference and improve accuracy plus uncertainty quantification on benchmarks.
-
Speaking in Self-Assessing Tongues: On the Verbalized Confidence of LLMs in Machine Translation
Empirical study finds verbalized per-token confidence methods in LLMs for MT perform similarly to internal signals on error detection and calibration but show little correlation.
-
Trustworthy deep domain adaptation for wearable photoplethysmography signal analysis with decision-theoretic uncertainty quantification
Decision-theoretic uncertainty quantification formalizes evaluation of generative domain adaptation trustworthiness for PPG-based atrial fibrillation classification by linking uncertainty to downstream task utility.
-
Confident in a Confidence Score: Investigating the Sensitivity of Confidence Scores to Supervised Fine-Tuning
Supervised fine-tuning degrades the correlation between confidence scores and output quality in language models, driven by factors like training distribution similarity rather than true quality.