Pith. sign in

REVIEW 9 cited by

Learning Optimal Conformal Classifiers

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2110.09192 v3 pith:OHZK3TEC submitted 2021-10-18 cs.LG cs.CVstat.MEstat.ML

David Stutz , Krishnamurthy (Dj) Dvijotham , Ali Taylan Cemgil , Arnaud Doucet This is my paper · ORCID
classification cs.LGcs.CVstat.MEstat.ML
keywords trainingconfidenceconformalsetsconftrclassesclassifiersduring
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

Modern deep learning based classifiers show very high accuracy on test data but this does not provide sufficient guarantees for safe deployment, especially in high-stake AI applications such as medical diagnosis. Usually, predictions are obtained without a reliable uncertainty estimate or a formal guarantee. Conformal prediction (CP) addresses these issues by using the classifier's predictions, e.g., its probability estimates, to predict confidence sets containing the true class with a user-specified probability. However, using CP as a separate processing step after training prevents the underlying model from adapting to the prediction of confidence sets. Thus, this paper explores strategies to differentiate through CP during training with the goal of training model with the conformal wrapper end-to-end. In our approach, conformal training (ConfTr), we specifically "simulate" conformalization on mini-batches during training. Compared to standard training, ConfTr reduces the average confidence set size (inefficiency) of state-of-the-art CP methods applied after training. Moreover, it allows to "shape" the confidence sets predicted at test time, which is difficult for standard CP. On experiments with several datasets, we show ConfTr can influence how inefficiency is distributed across classes, or guide the composition of confidence sets in terms of the included classes, while retaining the guarantees offered by CP.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 9 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Improving Backward Conformal Prediction via Non-Conformity Score Transformation

    stat.ML 2026-02 reject novelty 7.0 of 10

    ST-BCP tightens the coverage bound in Backward Conformal Prediction by applying a computable data-dependent transformation to nonconformity scores, reducing the average gap from 4.20% to 1.12% on benchmarks while prov...

  2. Optimized Conformal Selection: Powerful Selective Inference After Conformity Score Optimization

    stat.ME 2024-11 accept novelty 7.0 of 10

    OptCS allows conformity scores to be optimized on calibration and test data while preserving finite-sample false discovery rate control in conformal selection.

  3. Conformal Changepoint Localization and Root Cause Analysis with Corrupted Observations

    cs.LG 2026-07 conditional novelty 6.0 of 10

    Weighted conformal changepoint localization (W-CONCH) and root-cause analysis (W-CROC) downweight likely-corrupted observations via classifier uncertainty, preserving coverage and shrinking confidence sets.

  4. Direct Prediction Set Minimization via Bilevel Conformal Classifier Training

    cs.LG 2025-06 conditional novelty 6.0 of 10

    DPSM reformulates conformal training as a bilevel problem with quantile regression in the lower level and claims an O(1/sqrt n) learning bound, cutting prediction set size by about 20% in experiments.

  5. Test-time augmentation improves efficiency in conformal prediction

    cs.LG 2025-05 conditional novelty 6.0 of 10

    Applying learned test-time augmentation before conformal scoring reduces prediction set sizes by 10-14% with no loss of nominal coverage.

  6. Training-Aware Risk Control for Intensity Modulated Radiation Therapies Quality Assurance with Conformal Prediction

    cs.LG 2025-01 conditional novelty 6.0 of 10

    A training-aware conformal risk control method triages IMRT plans for measurement QA, achieving 100% sensitivity and 76% specificity on pooled hospital data while reducing measurements by 75%.

  7. Classification uncertainty for transient gravitational-wave noise artefacts with optimised conformal prediction

    gr-qc 2024-12 conditional novelty 6.0 of 10

    For Gravity Spy glitch classification, the optimal conformal prediction nonconformity measure depends on the chosen metric: baseline wins for F1 and average set size, maxscore2 wins for singleton count.

  8. Dynamic Estimation Loss Control in Variational Quantum Sensing via Online Conformal Inference

    quant-ph 2025-05 conditional novelty 5.0 of 10

    A dynamic variational quantum sensing method using online conformal inference controls the long-term estimation loss at a user-specified level while updating circuit and estimator parameters.

  9. Enhancing Adversarial Robustness with Conformal Prediction: A Framework for Guaranteed Model Reliability

    cs.LG 2025-06 reject novelty 4.0 of 10

    An adversarial attack and defense that respectively enlarge and shrink conformal prediction sets, with experiments on CIFAR-10, CIFAR-100 and mini-ImageNet.

Pith tools