Using the same data for tuning and calibration in conformal prediction introduces only small coverage bias for simple tuners, and this 'tuning bias' scales up with parameter count and down with calibration set size.
Weighted Aggregation of Conformity Scores for Classification
1 Pith paper cite this work. Polarity classification is still indexing.
abstract
Conformal prediction is a powerful framework for constructing prediction sets with valid coverage guarantees in multi-class classification. However, existing methods often rely on a single score function, which can limit their efficiency and informativeness. We propose a novel approach that combines multiple score functions to improve the performance of conformal predictors by identifying optimal weights that minimize prediction set size. Our theoretical analysis establishes a connection between the weighted score functions and subgraph classes of functions studied in Vapnik-Chervonenkis theory, providing a rigorous mathematical basis for understanding the effectiveness of the proposed method. Experiments demonstrate that our approach consistently outperforms single-score conformal predictors while maintaining valid coverage, offering a principled and data-driven way to enhance the efficiency and practicality of conformal prediction in classification tasks.
fields
cs.LG 1years
2025 1verdicts
REJECT 1representative citing papers
citing papers explorer
-
Parametric Scaling Law of Tuning Bias in Conformal Prediction
Using the same data for tuning and calibration in conformal prediction introduces only small coverage bias for simple tuners, and this 'tuning bias' scales up with parameter count and down with calibration set size.