A sharp Sauer inequality for multiclass and list prediction is established in terms of the DS dimension, tight for every alphabet size k, list size ℓ, and dimension value.
super hub Canonical reference
A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantification
Canonical reference. 86% of citing Pith papers cite this work as background.
110
Pith papers citing it
Background
86% of classified citations
abstract
Black-box machine learning models are now routinely used in high-risk settings, like medical diagnostics, which demand uncertainty quantification to avoid consequential model failures. Conformal prediction is a user-friendly paradigm for creating statistically rigorous uncertainty sets/intervals for the predictions of such models. Critically, the sets are valid in a distribution-free sense: they possess explicit, non-asymptotic guarantees even without distributional assumptions or model assumptions. One can use conformal prediction with any pre-trained model, such as a neural network, to produce sets that are guaranteed to contain the ground truth with a user-specified probability, such as 90%. It is easy-to-understand, easy-to-use, and general, applying naturally to problems arising in the fields of computer vision, natural language processing, deep reinforcement learning, and so on. This hands-on introduction is aimed to provide the reader a working understanding of conformal prediction and related distribution-free uncertainty quantification techniques with one self-contained document. We lead the reader through practical theory for and examples of conformal prediction and describe its extensions to complex machine learning tasks involving structured outputs, distribution shift, time-series, outliers, models that abstain, and more. Throughout, there are many explanatory illustrations, examples, and code samples in Python. With each code sample comes a Jupyter notebook implementing the method on a real-data example; the notebooks can be accessed and easily run using our codebase.
hub tools
citation-role summary
background 6
method 1
citation-polarity summary
claims ledger
- abstract Black-box machine learning models are now routinely used in high-risk settings, like medical diagnostics, which demand uncertainty quantification to avoid consequential model failures. Conformal prediction is a user-friendly paradigm for creating statistically rigorous uncertainty sets/intervals for the predictions of such models. Critically, the sets are valid in a distribution-free sense: they possess explicit, non-asymptotic guarantees even without distributional assumptions or model assumptions. One can use conformal prediction with any pre-trained model, such as a neural network, to produ
authors
co-cited works
representative citing papers
MiCP is the first conformal prediction method for multi-turn LLM pipelines that allocates per-turn error budgets to enable adaptive stopping with an overall coverage guarantee, shown to reduce turns and cost on RAG and ReAct benchmarks.
Generative world models used as closed-loop test oracles require a five-level admissibility ladder (L0-L4) because visual fidelity does not predict action-robustness.
A Certify-then-Rectify wrapper around HNSW uses conformal risk control plus EVT-estimated graph stretch to deliver average-case HNSW speed with high-probability exact k-NN recovery.
FALCON is a novel conformal prediction technique that learns locally calibrated confidence intervals for neural network surrogates modeling LHC scattering amplitudes.
SOCP uses self-organizing maps for unsupervised group discovery to enable local calibration in conformal prediction, reducing regional coverage gaps on benchmarks with small set-size increases while preserving validity guarantees.
Introduces PowerPhase benchmark for massive-variate power-system forecasting and PowerForge model that achieves best average rank on safety-fidelity metrics across all tested grids.
Introduces object-level semantic uncertainty for VLM memory, the UQ-DAAAM refinement system, and probabilistic guarantees that selected high-quality views reduce uncertainty more effectively.
An adaptive two-phase semantic filter using clustering then a hybrid proxy trained on LLM confidence achieves 1.6-2.0x speedup over prior methods at 90% accuracy on 10K document corpora.
The paper defines Cherry-pick Override (CCO) as unauthorized directional commitment by LLM judges under mixed evidence and quantifies its prevalence (>84% on AVeriTeC conflicting subset) while testing intervention ladders and a two-channel reference probe.
Proposes a scale-calibrated median-of-means estimator for robust aggregation of distributed PCA estimates on the product of Euclidean space and Grassmann manifold.
The paper derives that calibration-conditional coverage follows a Beta(k, n+1-k) law under continuous i.i.d. exchangeability and quantifies non-i.i.d. departures via Wasserstein distances on transported beta laws, yielding explicit bounds in scale-shift, clustered, and mixing regimes.
ACI-style updates with boundary actions achieve adversarial coverage validity and sublinear cost regret in conformal selection under bandit feedback.
GRAPHLCP improves localized conformal prediction on graphs by using feature-aware densification and Personalized PageRank kernels to incorporate topology for better coverage and efficiency.
TRACE creates valid conformal prediction sets for complex generative models by scoring outputs via averaged denoising or velocity errors along stochastic transport paths instead of likelihoods.
Trimming helps conformal prediction under contamination precisely when the anomaly score separates retention probabilities without biasing clean scores, otherwise the retained mixture coefficient prevents substantial decontamination.
PUICL is a transformer pretrained on synthetic PU data from structural causal models that solves positive-unlabeled classification via in-context learning without gradient updates or fitting.
SCALE uses Spectral Graph Conditional Exchangeability (SGCE) and graph wavelets to achieve valid coverage and improved efficiency in conformal prediction for non-exchangeable graph time series by conformalizing high-frequency residuals conditioned on low-frequency embeddings.
An intrinsic effective sample size for manifold MCMC is defined via kernel discrepancy as the number of independent draws yielding equivalent expected squared discrepancy to the target.
PLACE delivers a closed-form certified classification method for point clouds and graphs based on persistent homology with explicit excess-risk bounds, selection rules, and training-time certificates.
The profile maximum likelihood estimator for the location in anisotropic hyperbolic wrapped normal models is strongly consistent, asymptotically normal, and attains the Hájek-Le Cam minimax lower bound under squared geodesic loss.
A GNN predicts Gaussians over QAOA parameters to create graph-conditioned trust regions that reduce circuit evaluations for MaxCut from 85-343 down to 45 while keeping approximation ratios within 3 points of heuristics.
A model-agnostic adaptive conformal anomaly detection approach uses weighted quantile bounds learned from past foundation model predictions to deliver interpretable p-value scores with stable calibration under shifts for time series monitoring.
Random team assignments in a professional firm reveal that indirect ties strongly increase new direct tie formation, while effects of degree and local density are smaller and less robust.
citing papers explorer
-
An Optimal Sauer Lemma Over $k$-ary Alphabets
A sharp Sauer inequality for multiclass and list prediction is established in terms of the DS dimension, tight for every alphabet size k, list size ℓ, and dimension value.
-
Adaptive Stopping for Multi-Turn LLM Reasoning
MiCP is the first conformal prediction method for multi-turn LLM pipelines that allocates per-turn error budgets to enable adaptive stopping with an overall coverage guarantee, shown to reduce turns and cost on RAG and ReAct benchmarks.
-
Validate the Dream Before You Trust Its Verdict: Admissibility for World-Model Simulators
Generative world models used as closed-loop test oracles require a five-level admissibility ladder (L0-L4) because visual fidelity does not predict action-robustness.
-
HNSW with Accuracy Guarantees Using Graph Spanners
A Certify-then-Rectify wrapper around HNSW uses conformal risk control plus EVT-estimated graph stretch to deliver average-case HNSW speed with high-probability exact k-NN recovery.
-
Local Conformal Predictions for Calibrated Surrogates
FALCON is a novel conformal prediction technique that learns locally calibrated confidence intervals for neural network surrogates modeling LHC scattering amplitudes.
-
Self-Organized Conformal Prediction: Reducing Regional Coverage Gaps with Unsupervised Group Discovery
SOCP uses self-organizing maps for unsupervised group discovery to enable local calibration in conformal prediction, reducing regional coverage gaps on benchmarks with small set-size increases while preserving validity guarantees.
-
Navigating the Safety-Fidelity Trade-off: Massive-Variate Time Series Forecasting for Power Systems via Probabilistic Scenarios
Introduces PowerPhase benchmark for massive-variate power-system forecasting and PowerForge model that achieves best average rank on safety-fidelity metrics across all tested grids.
-
Remember with Confidence: Uncertainty Quantification for Spatio-temporal Memory with Probabilistic Guarantees
Introduces object-level semantic uncertainty for VLM memory, the UQ-DAAAM refinement system, and probabilistic guarantees that selected high-quality views reduce uncertainty more effectively.
-
Fast LLM-Based Semantic Filtering: From a Unified Framework to an Adaptive Two-Phase Method
An adaptive two-phase semantic filter using clustering then a hybrid proxy trained on LLM confidence achieves 1.6-2.0x speedup over prior methods at 90% accuracy on 10K document corpora.
-
Cherry-pick Override: Unsafe Directional Commitment in LLM Judges under Mixed Evidence
The paper defines Cherry-pick Override (CCO) as unauthorized directional commitment by LLM judges under mixed evidence and quantifies its prevalence (>84% on AVeriTeC conflicting subset) while testing intervention ladders and a two-channel reference probe.
-
Scale-Calibrated Median-of-Means for Robust Distributed Principal Component Analysis
Proposes a scale-calibrated median-of-means estimator for robust aggregation of distributed PCA estimates on the product of Euclidean space and Grassmann manifold.
-
Conformal Prediction via Transported Beta Laws
The paper derives that calibration-conditional coverage follows a Beta(k, n+1-k) law under continuous i.i.d. exchangeability and quantifies non-i.i.d. departures via Wasserstein distances on transported beta laws, yielding explicit bounds in scale-shift, clustered, and mixing regimes.
-
Efficient Online Conformal Selection with Limited Feedback
ACI-style updates with boundary actions achieve adversarial coverage validity and sublinear cost regret in conformal selection under bandit feedback.
-
GRAPHLCP: Structure-Aware Localized Conformal Prediction on Graphs
GRAPHLCP improves localized conformal prediction on graphs by using feature-aware densification and Personalized PageRank kernels to incorporate topology for better coverage and efficiency.
-
TRACE: Transport Alignment Conformal Prediction via Diffusion and Flow Matching Models
TRACE creates valid conformal prediction sets for complex generative models by scoring outputs via averaged denoising or velocity errors along stochastic transport paths instead of likelihoods.
-
When Does Trimming Help Conformal Prediction? A Retained-Law Diagnostic under Calibration Contamination
Trimming helps conformal prediction under contamination precisely when the anomaly score separates retention probabilities without biasing clean scores, otherwise the retained mixture coefficient prevents substantial decontamination.
-
In-Context Positive-Unlabeled Learning
PUICL is a transformer pretrained on synthetic PU data from structural causal models that solves positive-unlabeled classification via in-context learning without gradient updates or fitting.
-
Delving into Non-Exchangeability for Conformal Prediction in Graph-Structured Multivariate Time Series
SCALE uses Spectral Graph Conditional Exchangeability (SGCE) and graph wavelets to achieve valid coverage and improved efficiency in conformal prediction for non-exchangeable graph time series by conformalizing high-frequency residuals conditioned on low-frequency embeddings.
-
Intrinsic effective sample size for manifold-valued Markov chain Monte Carlo via kernel discrepancy
An intrinsic effective sample size for manifold MCMC is defined via kernel discrepancy as the number of independent draws yielding equivalent expected squared discrepancy to the target.
-
A Closed-Form Persistence-Landmark Pipeline for Certified Point-Cloud and Graph Classification
PLACE delivers a closed-form certified classification method for point clouds and graphs based on persistent homology with explicit excess-risk bounds, selection rules, and training-time certificates.
-
Profile Likelihood Inference for Anisotropic Hyperbolic Wrapped Normal Models on Hyperbolic Space
The profile maximum likelihood estimator for the location in anisotropic hyperbolic wrapped normal models is strongly consistent, asymptotically normal, and attains the Hájek-Le Cam minimax lower bound under squared geodesic loss.
-
Query-Efficient Quantum Approximate Optimization via Graph-Conditioned Trust Regions
A GNN predicts Gaussians over QAOA parameters to create graph-conditioned trust regions that reduce circuit evaluations for MaxCut from 85-343 down to 45 while keeping approximation ratios within 3 points of heuristics.
-
Adaptive Conformal Anomaly Detection with Time Series Foundation Models for Signal Monitoring
A model-agnostic adaptive conformal anomaly detection approach uses weighted quantile bounds learned from past foundation model predictions to deliver interpretable p-value scores with stable calibration under shifts for time series monitoring.
-
Causal inference for social network formation
Random team assignments in a professional firm reveal that indirect ties strongly increase new direct tie formation, while effects of degree and local density are smaller and less robust.
-
Answer Only as Precisely as Justified: Calibrated Claim-Level Specificity Control for Agentic Systems
Compositional selective specificity (CSS) decomposes generated answers into claims and emits each at the most specific level supported by evidence, raising overcommitment-aware utility from 0.846 to 0.913 on LongFact while retaining 0.938 specificity.
-
Diagnosing LLM Judge Reliability: Conformal Prediction Sets and Transitivity Violations
LLM judges display per-document transitivity violations in 33-67% of cases despite low aggregate rates, while conformal prediction set widths serve as reliable indicators of document-level difficulty with cross-judge agreement.
-
Conformal Margin Risk Minimization: An Envelope Framework for Robust Learning under Label Noise
CMRM adds a conformal quantile regularization on prediction margins to any loss, improving noisy-label classification accuracy up to 3.39% across methods and benchmarks while preserving performance at zero noise.
-
Conformal Risk Control under Non-Monotone Losses: Theory and Finite-Sample Guarantees
Conformal risk control for bounded non-monotone losses over a grid of size m achieves excess risk of order sqrt(log m / n) with n calibration samples, which is minimax optimal.
-
Post-Selection Distributional Model Evaluation
PS-DME is a new framework that controls post-selection false coverage rate for distributional KPI estimates via e-values and is provably more sample-efficient than data splitting under explicit conditions.
-
From Plausibility to Verifiability: Risk-Controlled Generative OCR with Vision-Language Models
A model-agnostic Geometric Risk Controller reduces extreme errors in VLM-based OCR by requiring cross-view consensus before accepting outputs.
-
Safe Planning in Interactive Environments via Iterative Policy Updates and Adversarially Robust Conformal Prediction
The work develops an iterative safe planner that adjusts conformal prediction bounds across policy updates via sensitivity analysis to maintain distribution-free safety guarantees despite interaction-induced distribution shifts.
-
Eigenvalue Calibration for Semantic Embeddings of Large Language Models
Temperature scaling of density-matrix eigenvalues from LLM semantic embeddings optimizes proper-score calibration and corrects systematic overconfidence so entropy equals risk.
-
OmniPilot: An Uncertainty-Aware LLM Inference Advisor for Heterogeneous GPU Clusters
OmniPilot combines conformal quantile regression with OOD detection to rank LLM serving configurations on mixed GPUs, reporting 6.2% MAPE throughput prediction and 95% top-1 accuracy on 460 benchmark runs while abstaining on unsupported cases.
-
Uncertainty Quantification via Invariant-Measure Conformal Prediction
Proposes imCP framework that uses independent samples from the invariant measure of a Markov process for conformal calibration in one-step and multi-step predictions of learned dynamical systems.
-
Privacy-Preserving Decentralized Cooperative Localization with Range-Only Measurements: A Convex Optimization Based Approach
Develops a privacy-preserving decentralized cooperative localization method using SDP-based maximum volume ellipsoids and dual variable exchange for range-only measurements.
-
Randomized neural operator for parametric PDEs with fast training and conformal uncertainty quantification
PCA-RaNN recasts latent neural operator learning as PCA-reduced random-feature linear regression, achieving 1-3 orders faster training than standard methods on PDE benchmarks while adding conformal uncertainty quantification.
-
HJ-SafeDMP: Hamilton-Jacobi Reachability-Guided Dynamic Movement Primitives for Provably Safe Robot Motion
HJ-SafeDMP learns a control barrier value function offline from demonstrations via finite-difference HJ recursion and uses it as a closed-form safety filter on DMP outputs, with conformal prediction for coverage guarantees.
-
Uncertainty Quantification for Computer-Use Agents: A Benchmark across Vision-Language Models and GUI Grounding Datasets
Argus benchmark shows UQ method rankings for GUI grounding agents are stable within models across datasets but degrade across model classes and to closed-source vendors.
-
Reliable Conformal Prediction for Ordinal Classification Using the Ranked Probability Score
RPS-based conformal prediction for ordinal classification yields median-centered contiguous sets with a favorable width-miscoverage tradeoff compared to prior methods.
-
Neural Conjugate Aggregation: Identifiable Unsupervised Multi-Sensor Regression under Heterogeneous Sensor Bias
NCAM is a hierarchical Bayesian model using neural networks and conjugate Gaussian inference to learn sensor-specific biases for unsupervised multi-source regression, with added conformal prediction for coverage guarantees.
-
Signed Evidence Flow: Conflict-Aware and Stability-Calibrated Data Analysis
SEF quantifies support versus opposition behind a prediction, and cross-fitted tests show conflict adds error-ranking information on some data sets while a permutation gate decides when that direction supports review.
-
PRecover 1.0: Process Rate Recovery with Machine Learning
Machine learning models recover most warm-rain and ice microphysical process rates from standard ICON model outputs for accumulation intervals of 10 minutes or less using a two-step classification-regression approach with calibrated uncertainty.
-
Show, Don't Ask: Generative Visual Disambiguation for Composed Image Retrieval with Turn-Valid Coverage
CLARA achieves turn-valid conformal coverage in ambiguous composed image retrieval by replacing text clarification with user selection among snapped real-image prototypes and reweighting calibration accordingly.
-
An Energy-Driven Framework for Privacy-Aware Synthetic Data Generation
An energy-based constrained sampling method generates privacy-aware synthetic mixed-type tabular data while aiming to preserve predictive utility and limit memorization.
-
Gaming-Resistant Insurance Contracts for Autonomous AI Agents: Strategy-Proof Toll Mechanism Design
The paper characterizes a five-attack space for AI-agent insurance and proves joint incentive compatibility by adding common-control aggregation, interface escalation fees, and model-identity menus to a base runtime, plus a two-parameter premium family.
-
Conformal Bayes under Label Shift: Post-Hoc Calibration vs. In-Training Adaptation
Compares post-hoc vs. in-training strategies for conformal Bayes under label shift, finding regime-dependent efficiency gains with up to 43% narrower sets in high-dimensional cases.
-
SPACR: Single-Pass Adaptive Training of Uncertainty-Aware Conformal Regressors
SPACR is a single-pass training method for conformal regressors that jointly optimizes validity and efficiency to yield valid intervals at multiple confidence levels from one model.
-
Operator learning for the 2D incompressible Navier-Stokes equations: a conformal prediction approach in the data-scarce regime
A perturbation-based conformal prediction wrapper on Fourier Neural Operators yields narrower uncertainty bands than prior methods for 2D incompressible Navier-Stokes while preserving coverage in data-scarce regimes.
-
Conformal Prediction for Neural Operators: Distribution-Free Uncertainty Quantification in Physics Simulation
First application of split conformal prediction to neural operators, providing distribution-free intervals with 89.1% empirical coverage on heat conduction benchmarks and an adaptive normalized variant using MC Dropout.
-
Co-GLANCE: Uncertainty-Aware Active Perception for Heterogeneous Robot Teaming
Co-GLANCE distills vision-language models into an end-to-end onboard model for occlusion segmentation and robot allocation, using conformal prediction plus selective abstention to trigger active perception and achieve 25-36% higher accuracy with 350x lower latency than cloud baselines.