API Is Enough: Conformal Prediction for Large Language Models Without Logit-Access
read the original abstract
This study aims to address the pervasive challenge of quantifying uncertainty in large language models (LLMs) without logit-access. Conformal Prediction (CP), known for its model-agnostic and distribution-free features, is a desired approach for various LLMs and data distributions. However, existing CP methods for LLMs typically assume access to the logits, which are unavailable for some API-only LLMs. In addition, logits are known to be miscalibrated, potentially leading to degraded CP performance. To tackle these challenges, we introduce a novel CP method that (1) is tailored for API-only LLMs without logit-access; (2) minimizes the size of prediction sets; and (3) ensures a statistical guarantee of the user-defined coverage. The core idea of this approach is to formulate nonconformity measures using both coarse-grained (i.e., sample frequency) and fine-grained uncertainty notions (e.g., semantic similarity). Experimental results on both close-ended and open-ended Question Answering tasks show our approach can mostly outperform the logit-based CP baselines.
This paper has not been read by Pith yet.
Forward citations
Cited by 8 Pith papers
-
Remember with Confidence: Uncertainty Quantification for Spatio-temporal Memory with Probabilistic Guarantees
Introduces object-level semantic uncertainty for VLM memory, the UQ-DAAAM refinement system, and probabilistic guarantees that selected high-quality views reduce uncertainty more effectively.
-
ST-BCP: Tightening Coverage Bound for Backward Conformal Prediction via Non-Conformity Score Transformation
ST-BCP tightens the coverage bound in Backward Conformal Prediction by applying a computable data-dependent transformation to nonconformity scores, reducing the average gap from 4.20% to 1.12% on benchmarks while prov...
-
Conformal Path Reasoning: Trustworthy Knowledge Graph Question Answering via Path-Level Calibration
CPR uses query-level conformal calibration over path scores and a PUCT-trained RCVNet to achieve valid coverage guarantees and smaller prediction sets in KGQA, reporting 45% higher empirical coverage and 52% smaller s...
-
Conformal Path Reasoning: Trustworthy Knowledge Graph Question Answering via Path-Level Calibration
CPR improves empirical coverage rate by 34% and reduces average prediction set size by 40% in KGQA benchmarks via query-level path calibration and RCVNet for discriminative nonconformity scores.
-
From Debate to Decision: Conformal Social Choice for Safe Multi-Agent Deliberation
Conformal Social Choice aggregates verbalized probabilities from LLM debates via linear opinion pooling and uses split conformal prediction to generate prediction sets that guarantee inclusion of the correct answer wi...
-
Strategic Decision Support for AI Agents
The paper introduces an optimization framework for AI agents to strategically seek support, proving a threshold policy on support value and providing an online algorithm to control missed-support error without distrib...
-
Position: Uncertainty Quantification in LLMs is Just Unsupervised Clustering
Mainstream UQ for LLMs reduces to unsupervised clustering of internal generation consistency and therefore cannot detect confident hallucinations or provide reliable safety signals.
-
Differentiable Conformal Training for LLM Reasoning Factuality
DCF relaxes non-differentiable conformal factuality for LLM reasoning chains into a trainable form, yielding up to 141% higher retention of true claims on benchmarks while preserving reliability guarantees.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.