Pith. sign in

REVIEW 6 major objections 3 minor 52 references

WQLCP: Weighted Adaptive Conformal Prediction for Robust Uncertainty Quantification Under Distribution Shifts

T0 review · 6 major / 3 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that weighting conformal calibration by VAE reconstruction-loss ratios maintains target coverage under distribution shifts while shrinking prediction sets, on ImageNet variant benchmarks.

desk verdict The reconstruction-loss scaling idea is worth testing, but this draft does not actually define the algorithm, so the empirical claims are not yet supported. read the letter →

arxiv 2505.19587 v1 pith:DGUP6HB2 submitted 2025-05-26 cs.LG cs.CV

Shadi Alijani , Homayoun Najjaran This is my paper
classification cs.LGcs.CV
keywords conformalpredictiondistributionshiftuncertaintyquantificationvariationalautoencoderweightedsetsImageNetcoverageguarantee
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes two conformal prediction methods that use a variational autoencoder's reconstruction loss as an uncertainty signal under distribution shift. The first, RLSCP, scales test scores by the reconstruction-loss quantile of the whole test set. The second, WQLCP, additionally reweights calibration samples by the ratio of their reconstruction losses to test reconstruction losses, so the conformal quantile adapts to how far the test data have drifted. The authors claim that WQLCP maintains the target coverage rate on ImageNetV2, ImageNetR, and ImageNetA while producing smaller prediction sets than weighted and self-supervised baselines. The paper also reports remaining under-coverage on the most adversarial subset and inflated sets on fine-grained classes.

What carries the argument

The central object is the reconstruction-loss ratio weight $w(x_j) \propto L_{\text{cal}}(x_j)/(L_{\text{test}}(x_i)+\epsilon)$ used in the weighted quantile estimator, together with the reconstruction-loss quantile $\mathrm{RL}_{\text{test}} = q_{1-\alpha}(L_{\text{test}})$ that scales test scores. The VAE reconstruction loss acts as a shift-severity meter: higher loss on test inputs signals greater drift, which raises the scaling factor and up-weights calibration samples that resemble the test distribution. The weighted quantile adjusts the threshold toward the test distribution, and the scaling enlarges sets when the whole test set is shifted.

What would settle it

Take an in-distribution test set and artificially inflate every test image's reconstruction loss by adding fixed noise that does not change the true label; if WQLCP's weighted quantile is driven by the loss ratio rather than a true density ratio, coverage should drop below $1-\alpha$, whereas a method with a genuine likelihood-ratio weight would keep coverage.

Watch

Extended reading notes

Core claim

On the paper's own terms, WQLCP establishes that a VAE's reconstruction loss, aggregated over a shifted test set, can serve both as a score-scaling factor and as a calibration-weighting signal that recovers near-nominal coverage under covariate shift. The method computes a weighted quantile of calibration conformal scores, with weights proportional to the ratio of each calibration sample's reconstruction loss to the test reconstruction loss, then scales test scores by the reconstruction-loss quantile. On ImageNetV2, ImageNetR, and ImageNetA, the paper reports that WQLCP with a ViT-VAE backbone matches or exceeds the coverage of weighted conformal prediction and self-supervised conformal prediction while reducing average prediction set size, for example 0.9403 coverage with set size 6.7003 on ImageNetV2 versus 0.8603 and 17.0003 for SSCP. The paper claims that this procedure preserves the finite-sample statistical guarantees of conformal prediction while adapting to shifts.

Load-bearing premise

The coverage guarantee rests on the assumption that the ratio of reconstruction losses between calibration and test inputs behaves like a true likelihood-ratio weight, and the paper does not show that this ratio equals any density ratio.

Editorial extensions

If this is right

  • If WQLCP holds, practitioners can maintain target coverage on drifted image benchmarks without explicitly estimating density ratios between calibration and test distributions.
  • Combining score scaling with weighted quantiles yields a coverage-versus-set-size trade-off that the paper reports dominates fixed-threshold methods on the tested benchmarks.
  • Transformer-based VAEs give smaller coverage drops across shift severities, suggesting that backbone choice matters for reconstruction-loss-based conformal prediction.
  • The claimed finite-sample guarantee implies that calibration can be performed once on a fixed calibration set and reused across different shifted test sets while preserving coverage.
  • The reported failure modes indicate that the method still under-covers the most adversarially shifted samples and over-expands sets on fine-grained classes.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The weight formula as written uses a single test sample's loss in every calibration weight, so the method's behavior depends on which test sample is chosen; a robust implementation would average over test losses or use the test-set quantile.
  • If reconstruction-loss ratios are not true likelihood ratios, the finite-sample coverage guarantee is not automatically inherited from weighted conformal prediction theory; an empirical sanity check is to run WQLCP on in-distribution data with artificially inflated test losses and measure coverage.
  • The global scaling step means WQLCP adapts to dataset-level shifts rather than per-input difficulty; per-sample reweighting could further reduce set sizes on heterogeneous test sets.
  • A testable extension is to replace the VAE loss with any self-supervised or generative density estimate and compare whether coverage tracking under shift improves or degrades.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

6 major / 3 minor

Summary. This paper proposes two conformal prediction methods for distribution-shifted image classification: RLSCP, which scales conformal scores by a quantile of VAE reconstruction losses over the test set, and WQLCP, which additionally reweights calibration samples by ratios of calibration-to-test reconstruction losses before computing a weighted quantile threshold. The authors claim that WQLCP preserves finite-sample CP guarantees while maintaining coverage and reducing prediction set sizes on ImageNetV2, ImageNetR, and ImageNetA. The manuscript reports experiments across ResNet, ViT, and DeiT backbones, an ablation of the VAE KL weight beta, and a failure-mode analysis.

Significance. The problem addressed is important: conformal prediction under distribution shift is an active area, and using reconstruction losses as an uncertainty signal is a plausible heuristic. The paper also contains useful empirical components, including multiple backbone comparisons and a failure-mode analysis. However, the central methodological contribution is not well-defined: the weight formula in Eq. (7) is incomplete, the weighted quantile computed in Algorithm 1 is never used, no coverage guarantee is proved, and the reported numbers contain apparent typographical errors. Because these issues affect the core claim, the manuscript in its current form does not establish a valid conformal prediction procedure.

major comments (6)
  1. [3.2.1, Eq. (7), Algorithm 1 Step 2] Equation (7) defines w(xj) proportional to Lcal(xj)/(Ltest(xi)+epsilon), but in Algorithm 1 Step 2 this quantity is computed for each calibration sample xj and references an undefined test index xi; no aggregation over the test set is specified. As written, the weights are not a function of a single calibration sample and the test distribution, so the weighted quantile in Eq. (8) has no well-defined input. This must be corrected and the likelihood-ratio interpretation justified before any coverage claim can be evaluated.
  2. [Algorithm 1, Steps 3 and 5] The weighted quantile qhat computed in Step 3 is never used: Step 5 constructs prediction sets as {y : sscaled(xi,y) >= tau_test} with an undefined tau_test, and the only scaling applied is RLtest. Consequently, WQLCP as implemented is not a weighted conformal method, and the paper's claimed improvement over RLSCP cannot be attributed to the weighted quantile mechanism.
  3. [Section 1 and Section 3.2] The paper claims that WQLCP preserves the finite-sample statistical guarantees of CP, but no proof or formal argument is given. The weights in Eq. (7) depend on the test reconstruction losses, so the calibration scores are not exchangeable with the test scores under any stated distributional assumption; no condition such as a known or estimable likelihood ratio dP_test/dP_cal is established. In the absence of a coverage theorem, the central finite-sample guarantee is unsupported.
  4. [Section 3.1.2 and Section 4.3] Both RLSCP and WQLCP use the full test set (via RLtest and Ltest) to determine the scaling and weights, and beta in Eq. (3)/Fig. 4 is selected on ImageNetA and then evaluated on ImageNetA. This means the reported coverage and set sizes are conditional on the test data used to fit the procedure; they do not describe performance on a fresh test sample, which is what a conformal coverage guarantee requires. The authors should either state that the evaluation is transductive or provide a separate non-tuned test set.
  5. [Algorithm 1, Steps 1-3] Algorithm 1 is not self-contained: the inputs are only Dtest and Dcal, but Step 3 uses conformal scores sj that are never defined and require labels or a base model; the base model and score function are not listed among the inputs. This omission makes the procedure impossible to reproduce and obscures what is being calibrated.
  6. [Table 1] Table 1 contains entries that are not credible as printed, e.g., APS/DeiT-S on ImageNet reports '0.9039 / 90.531' and APS/DeiT-B reports '0.8987 / 11.602'; several set sizes are inconsistent with the corresponding coverages and with other rows. These apparent typographical errors undermine the quantitative comparison and should be corrected with exact numbers and error bars.
minor comments (3)
  1. [Section 2.1] The text uses 'WQLC' instead of 'WQLCP' in the sentence introducing the proposed method.
  2. [Table 1 header] The header 'Average coverage / Average setsize' contains a typo ('setsize' should be 'set size'), and the table would benefit from a note clarifying whether the reported numbers are means over runs or single-run values.
  3. [Section 4.6] The failure-mode percentages '12.1%' and '4.8%' are presented without confidence intervals or sample counts; adding these would make the analysis more informative.

Circularity Check

1 steps flagged · score 6.0 of 10

WQLCP's headline ImageNetA coverage is selected by grid search on that benchmark; no load-bearing self-citation or definitional circularity found elsewhere.

  1. fitted input called prediction [Section 4.5 (Ablation Studies), Figure 4 caption; Table 1 (WQLCP ViT-VAE ImageNetA entry)]
    "Coverage vs β on ImageNetA, showing optimal β = 1.2 achieves 0.7402 coverage."

    The hyperparameter β is selected by grid search to maximize coverage on ImageNetA, and the selected value is then used to produce the WQLCP ViT-VAE result reported for ImageNetA in Table 1 (0.7402 coverage, 9.501 set size). The number 0.7402 is not an independent prediction: it is the optimum of the tuning objective on the same benchmark. The paper's headline claim that WQLCP 'maintains coverage' on ImageNetA therefore reduces, for that dataset, to reporting the best value found by the search, i.e., the reported performance is fitted rather than predicted.

full rationale

The only concrete circularity I can exhibit is the ImageNetA headline number: Section 4.5 states that β = 1.2 was selected via grid search, and Figure 4 confirms the selection was made by coverage on ImageNetA, with the same 0.7402 coverage value that Table 1 reports for WQLCP(ViT-VAE) on ImageNetA. That reported coverage is therefore the selected optimum of the tuning procedure on the same benchmark, not an out-of-sample prediction; the claim of 'maintaining coverage' on ImageNetA is partly forced by construction. I do not find a circular derivation in the mathematical chain itself: the reconstruction-loss scaling is an explicit ansatz rather than derived from the target coverage, and the citations to Tibshirani et al. and Barber et al. are external results, not author self-citations carrying the argument. Separately, the paper has serious non-circularity flaws: Eq. (7) is undefined because Ltest(x_i) is not specified inside the calibration loop, Algorithm 1 computes qhat but never uses it in Step 5 where τtest is undefined, and no likelihood-ratio justification is supplied for the weights; these undermine the 'finite-sample statistical guarantees' claim but are not equivalences by definition. Score 6 reflects the partial circularity/selection in the central ImageNetA result; other benchmark comparisons still carry some independent empirical content.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The method rests on a few empirical assumptions about reconstruction loss and on the validity of weighted quantiles, but the weight formula is not well-defined and no proof is provided.

free parameters (2)
  • beta (VAE KL weight) = 1.2
    Selected via grid search on ImageNetA (Section 4.5, Fig 4) to optimize coverage on the target test set. The reported ImageNetA coverage depends on this fitted value.
  • epsilon (numerical stability) = not specified
    Introduced in Eq. 7 to avoid division by zero, but its value is never given, and its effect on the weights is not discussed.
assumptions (3)
  • domain assumption The VAE reconstruction loss is a valid uncertainty metric that increases with distribution shift severity.
    Section 3.1.2 and Figure 2. The paper asserts a correlation but does not prove it; the method's success relies on this.
  • ad hoc to paper The weighted quantile in Eq. 8 provides a valid threshold for conformal prediction under distribution shift.
    The paper claims finite-sample guarantees (Section 1) but provides no theorem. Weighted CP validity requires exact likelihood ratios (Tibshirani et al., 2019), not heuristically estimated weights.
  • ad hoc to paper Using test-set reconstruction losses (including the label-free test set) does not invalidate the CP coverage statement.
    The method computes L_test on the test set and uses it for scaling and weights, but the paper does not discuss the transductive nature of this choice or its effect on the guarantee.

how reviews work

0 comments
Cite this review

Pith. "Pith review of WQLCP: Weighted Adaptive Conformal Prediction for Robust Uncertainty Quantification Under Distribution Shifts." pith.science (2026). https://pith.science/paper/DGUP6HB2

@misc{pith2026250519587,
  author       = {Pith},
  title        = {Pith review of: WQLCP: Weighted Adaptive Conformal Prediction for Robust Uncertainty Quantification Under Distribution Shifts},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DGUP6HB2}},
  note         = {Machine review of arXiv:2505.19587}
}
read the original abstract

Conformal prediction (CP) provides a framework for constructing prediction sets with guaranteed coverage, assuming exchangeable data. However, real-world scenarios often involve distribution shifts that violate exchangeability, leading to unreliable coverage and inflated prediction sets. To address this challenge, we first introduce Reconstruction Loss-Scaled Conformal Prediction (RLSCP), which utilizes reconstruction losses derived from a Variational Autoencoder (VAE) as an uncertainty metric to scale score functions. While RLSCP demonstrates performance improvements, mainly resulting in better coverage, it quantifies quantiles based on a fixed calibration dataset without considering the discrepancies between test and train datasets in an unexchangeable setting. In the next step, we propose Weighted Quantile Loss-scaled Conformal Prediction (WQLCP), which refines RLSCP by incorporating a weighted notion of exchangeability, adjusting the calibration quantile threshold based on weights with respect to the ratio of calibration and test loss values. This approach improves the CP-generated prediction set outputs in the presence of distribution shifts. Experiments on large-scale datasets, including ImageNet variants, demonstrate that WQLCP outperforms existing baselines by consistently maintaining coverage while reducing prediction set sizes, providing a robust solution for CP under distribution shifts.

Figures

Figures reproduced from arXiv: 2505.19587 by the authors.

Figure 1
Figure 1. (a) CP coverage violations under distribution shifts. [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Comparison of reconstruction loss and softmax mean [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Overview of the WQLCP framework. The model leverages VAEs to compute reconstruction losses for uncertainty estimation, [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: β-VAE ablation study: (Left) Reconstruction MSE loss vs KL divergence trade-off. (Right) Coverage vs β on ImageNetA, showing optimal β = 1.2 achieves 0.7402 coverage. gates this degradation, achieving 74.0% coverage on Ima￾geNetA at a highly efficient set size of 9.50 …

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

52 extracted references · 37 canonical work pages

  1. [22]

    Auto-encoding variational bayes

    Diederik P Kingma. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013

    arXiv 2013

  2. [1]

    Vision transformers in domain adaptation and domain generalization: a study of robustness

    Shadi Alijani, Jamil Fayyad, and Homayoun Najjaran. Vision transformers in domain adaptation and domain generalization: a study of robustness. Neural Computing and Applications, 36(29):17979–18007, 2024

    work page 2024

  3. [2]

    Concrete problems in ai safety

    Dario Amodei, Chris Olah, Jacob Steinhardt, Paul Christiano, John Schulman, and Dan Man ´e. Concrete problems in ai safety. arXiv preprint arXiv:1606.06565, 2016

    arXiv 2016

  4. [3]

    Uncertainty sets for image classifiers us- ing conformal prediction

    Anastasios Angelopoulos, Stephen Bates, Jitendra Malik, and Michael I Jordan. Uncertainty sets for image classifiers us- ing conformal prediction. arXiv preprint arXiv:2009.14193, 2020

    arXiv 2009

  5. [4]

    Conformal prediction: A gentle introduction

    Anastasios N Angelopoulos, Stephen Bates, et al. Conformal prediction: A gentle introduction. Foundations and Trends® in Machine Learning, 16(4):494–591, 2023

    work page 2023

  6. [5]

    Balancing reconstruction error and kullback-leibler divergence in variational autoen- coders

    Andrea Asperti and Matteo Trentin. Balancing reconstruction error and kullback-leibler divergence in variational autoen- coders. Ieee Access, 8:199440–199448, 2020

    work page 2020

  7. [6]

    Conformal prediction beyond ex- changeability

    Rina Foygel Barber, Emmanuel J Candes, Aaditya Ramdas, and Ryan J Tibshirani. Conformal prediction beyond ex- changeability. The Annals of Statistics, 51(2):816–845, 2023

    work page 2023

  8. [7]

    The need for uncertainty quantification in machine-assisted medical decision making

    Edmon Begoli, Tanmoy Bhattacharya, and Dimitri Kusnezov. The need for uncertainty quantification in machine-assisted medical decision making. Nature Machine Intelligence, 1(1): 20–23, 2019

    work page 2019

Show all 52 references
  1. [8]

    Principal uncertainty quantification with spatial correlation for image restoration problems

    Omer Belhasin, Yaniv Romano, Daniel Freedman, Ehud Rivlin, and Michael Elad. Principal uncertainty quantification with spatial correlation for image restoration problems. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023

    2023

  2. [9]

    Improved online conformal prediction via strongly adaptive online learning

    Aadyot Bhatnagar, Huan Wang, Caiming Xiong, and Yu Bai. Improved online conformal prediction via strongly adaptive online learning. In International Conference on Machine Learning, pages 2337–2363. PMLR, 2023

    2023

  3. [10]

    Language models are few-shot learners

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Sub- biah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901, 2020

    1901

  4. [11]

    Kandinsky conformal prediction: Efficient calibration of image segmentation algorithms

    Joren Brunekreef, Eric Marcus, Ray Sheombarsing, Jan-Jakob Sonke, and Jonas Teuwen. Kandinsky conformal prediction: Efficient calibration of image segmentation algorithms. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 4135–4143, 2024

    2024

  5. [12]

    Robust validation: Confident predictions even when distributions shift

    Maxime Cauchois, Suyash Gupta, Alnur Ali, and John C Duchi. Robust validation: Confident predictions even when distributions shift. Journal of the American Statistical Associ- ation, pages 1–66, 2024

    2024

  6. [13]

    Imagenet: A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee, 2009

    2009

  7. [14]

    An image is worth 16x16 words: Trans- formers for image recognition at scale

    Alexey Dosovitskiy. An image is worth 16x16 words: Trans- formers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020

    2010 arXiv

  8. [15]

    Em- pirical validation of conformal prediction for trustworthy skin lesions classification

    Jamil Fayyad, Shadi Alijani, and Homayoun Najjaran. Em- pirical validation of conformal prediction for trustworthy skin lesions classification. Computer Methods and Programs in Biomedicine, page 108231, 2024

    2024

  9. [16]

    Are foundation models for computer vision good conformal predictors? arXiv preprint arXiv:2412.06082, 2024

    Leo Fillioux, Julio Silva-Rodr ´ıguez, Ismail Ben Ayed, Paul-Henry Courn `ede, Maria Vakalopoulou, Stergios Christodoulidis, and Jose Dolz. Are foundation models for computer vision good conformal predictors? arXiv preprint arXiv:2412.06082, 2024

    2024

  10. [17]

    Closure: Fast quantification of pose uncertainty sets

    Yihuai Gao, Yukai Tang, Han Qi, and Heng Yang. Closure: Fast quantification of pose uncertainty sets. arXiv preprint arXiv:2403.09990, 2024

    2024 arXiv

  11. [18]

    Conformal inference for online prediction with arbitrary distribution shifts.Journal of Machine Learning Research, 25(162):1–36, 2024

    Isaac Gibbs and Emmanuel J Cand`es. Conformal inference for online prediction with arbitrary distribution shifts.Journal of Machine Learning Research, 25(162):1–36, 2024

    2024

  12. [19]

    The many faces of robust- ness: A critical analysis of out-of-distribution generalization

    Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kada- vath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, et al. The many faces of robust- ness: A critical analysis of out-of-distribution generalization. In Proceedings of the IEEE/CVF internation...

    2021

  13. [20]

    Natural adversarial examples

    Dan Hendrycks, Kevin Zhao, Steven Basart, Jacob Steinhardt, and Dawn Song. Natural adversarial examples. In Proceed- ings of the IEEE/CVF conference on computer vision and pattern recognition, pages 15262–15271, 2021

    2021

  14. [21]

    Empirically validat- ing conformal prediction on modern vision architectures un- der distribution shift and long-tailed data

    Kevin Kasa and Graham W Taylor. Empirically validat- ing conformal prediction on modern vision architectures un- der distribution shift and long-tailed data. arXiv preprint arXiv:2307.01088, 2023

    2023 arXiv

  15. [23]

    Confor- mal prediction with large language models for multi-choice question answering

    Bhawesh Kumar, Charlie Lu, Gauri Gupta, Anil Palepu, David Bellamy, Ramesh Raskar, and Andrew Beam. Confor- mal prediction with large language models for multi-choice question answering. arXiv preprint arXiv:2305.18404, 2023

    2023 arXiv

  16. [24]

    Conformal deci- sion theory: Safe autonomous decisions from imperfect pre- dictions

    Jordan Lekeufack, Anastasios N Angelopoulos, Andrea Ba- jcsy, Michael I Jordan, and Jitendra Malik. Conformal deci- sion theory: Safe autonomous decisions from imperfect pre- dictions. In 2024 IEEE International Conference on Robotics and Automation (ICRA), pages 11668–11675. ...

    2024

  17. [25]

    Safe planning in dynamic environments using conformal prediction

    Lars Lindemann, Matthew Cleaveland, Gihyun Shim, and George J Pappas. Safe planning in dynamic environments using conformal prediction. IEEE Robotics and Automation Letters, 2023

    2023

  18. [26]

    Bayesian networks for uncertainty-aware diagnostics in chest x-rays.IEEE Trans- actions on Medical Imaging, 42(1):45–57, 2023

    Xin Liu, Zhen Wang, and Xiaoyan Tang. Bayesian networks for uncertainty-aware diagnostics in chest x-rays.IEEE Trans- actions on Medical Imaging, 42(1):45–57, 2023

    2023

  19. [27]

    Improving trustworthiness of ai disease severity rating in medical imaging with ordinal conformal prediction sets

    Charles Lu, Anastasios N Angelopoulos, and Stuart Pomer- antz. Improving trustworthiness of ai disease severity rating in medical imaging with ordinal conformal prediction sets. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 54...

    2022

  20. [28]

    Confor- mal semantic image segmentation: Post-hoc quantification of predictive uncertainty

    Luca Mossina, Joseba Dalmau, and L ´eo And ´eol. Confor- mal semantic image segmentation: Post-hoc quantification of predictive uncertainty. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 3574–3584, 2024

    2024

  21. [29]

    Exploring uncertainty measures in deep networks for trustworthy medical image segmentation

    Tanuja Nair, A Sinha, M Abedin, O Akinola, M Al-Shabi, et al. Exploring uncertainty measures in deep networks for trustworthy medical image segmentation. Journal of Medical Imaging, 7(5):053501, 2020

    2020

  22. [30]

    Can you trust your model’s uncertainty? evaluating predictive uncertainty under dataset shift

    Yaniv Ovadia, Emily Fertig, Jie Ren, Zachary Nado, David Sculley, Sebastian Nowozin, Joshua Dillon, Balaji Lakshmi- narayanan, and Jasper Snoek. Can you trust your model’s uncertainty? evaluating predictive uncertainty under dataset shift. Advances in neural information proces...

    2019

  23. [31]

    Conformal language modeling

    Victor Quach, Adam Fisch, Tal Schuster, Adam Yala, Jae Ho Sohn, Tommi S Jaakkola, and Regina Barzilay. Conformal language modeling. arXiv preprint arXiv:2306.10193, 2023

    2023 arXiv

  24. [32]

    Do imagenet classifiers generalize to im- agenet? In International conference on machine learning , pages 5389–5400

    Benjamin Recht, Rebecca Roelofs, Ludwig Schmidt, and Vaishaal Shankar. Do imagenet classifiers generalize to im- agenet? In International conference on machine learning , pages 5389–5400. PMLR, 2019

    2019

  25. [33]

    Robots that ask for help: Uncer- tainty alignment for large language model planners

    Allen Z Ren, Anushri Dixit, Alexandra Bodrova, Sumeet Singh, Stephen Tu, Noah Brown, Peng Xu, Leila Takayama, Fei Xia, Jake Varley, et al. Robots that ask for help: Uncer- tainty alignment for large language model planners. arXiv preprint arXiv:2307.01928, 2023

    2023 arXiv

  26. [34]

    Classi- fication with valid and adaptive coverage.Advances in Neural Information Processing Systems, 33:3581–3591, 2020

    Yaniv Romano, Matteo Sesia, and Emmanuel Candes. Classi- fication with valid and adaptive coverage.Advances in Neural Information Processing Systems, 33:3581–3591, 2020

    2020

  27. [35]

    Uncertain: Modern topics in uncertainty estima- tion

    Aaron Roth. Uncertain: Modern topics in uncertainty estima- tion. Unpublished Lecture Notes, page 2, 2022

    2022

  28. [36]

    Least ambiguous set-valued classifiers with bounded error levels

    Mauricio Sadinle, Jing Lei, and Larry Wasserman. Least ambiguous set-valued classifiers with bounded error levels. Journal of the American Statistical Association , 114(525): 223–234, 2019

    2019

  29. [37]

    Improving adaptive conformal prediction using self-supervised learning

    Nabeel Seedat, Alan Jeffares, Fergus Imrie, and Mihaela van der Schaar. Improving adaptive conformal prediction using self-supervised learning. In International Conference on Artificial Intelligence and Statistics, pages 10160–10177. PMLR, 2023

    2023

  30. [38]

    A tutorial on conformal prediction

    Glenn Shafer and Vladimir V ovk. A tutorial on conformal prediction. Journal of Machine Learning Research , 9(3), 2008

    2008

  31. [39]

    Mastering the game of go without human knowledge

    David Silver, Julian Schrittwieser, Karen Simonyan, Ioannis Antonoglou, Aja Huang, Arthur Guez, Thomas Hubert, Lucas Baker, Matthew Lai, Adrian Bolton, et al. Mastering the game of go without human knowledge. nature, 550(7676):354–359, 2017

    2017

  32. [40]

    Advanced topics in statistical learning: Con- formal prediction, 2023

    Ryan Tibshirani. Advanced topics in statistical learning: Con- formal prediction, 2023

    2023

  33. [41]

    Conformal prediction under covariate shift

    Ryan J Tibshirani, Rina Foygel Barber, Emmanuel Candes, and Aaditya Ramdas. Conformal prediction under covariate shift. Advances in neural information processing systems, 32, 2019

    2019

  34. [42]

    Deit iii: Revenge of the vit

    Hugo Touvron, Matthieu Cord, and Herv ´e J´egou. Deit iii: Revenge of the vit. In European conference on computer vision, pages 516–533. Springer, 2022

    2022

  35. [43]

    Conformal prediction in clinical medical sciences

    Janette Vazquez and Julio C Facelli. Conformal prediction in clinical medical sciences. Journal of Healthcare Informatics Research, 6(3):241–252, 2022

    2022

  36. [44]

    Improving decision-making in open-world agents with conformal predic- tion and monty hall

    Harit Vishwakarma, Alan Mishler, Thomas Cook, Niccolo Dalmasso, Natraj Raman, and Sumitra Ganesh. Improving decision-making in open-world agents with conformal predic- tion and monty hall. In NeurIPS 2024 Workshop on Open- World Agents

    2024

  37. [45]

    Algorithmic learning in a random world

    Vladimir V ovk, Alexander Gammerman, and Glenn Shafer. Algorithmic learning in a random world. Springer, 2005

    2005

  38. [46]

    Safe task planning for language-instructed multi-robot systems using conformal prediction

    Jun Wang, Guocheng He, and Yiannis Kantaros. Safe task planning for language-instructed multi-robot systems using conformal prediction. arXiv preprint arXiv:2402.15368 , 2024

    2024 arXiv

  39. [47]

    Uncertainty-guided domain adaptation for robust medical image segmentation

    Yifan Wang, Jia Chen, and Laura Smith. Uncertainty-guided domain adaptation for robust medical image segmentation. In IEEE International Symposium on Biomedical Imaging (ISBI), pages 245–255. IEEE, 2023

    2023

  40. [48]

    Object pose estimation with statistical guarantees: Conformal keypoint detection and geometric uncertainty propagation

    Heng Yang and Marco Pavone. Object pose estimation with statistical guarantees: Conformal keypoint detection and geometric uncertainty propagation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8947–8958, 2023

    2023

  41. [49]

    Rr-cp: Reliable-region-based conformal prediction for trust- worthy medical image classification

    Yizhe Zhang, Shuo Wang, Yejia Zhang, and Danny Z Chen. Rr-cp: Reliable-region-based conformal prediction for trust- worthy medical image classification. In International Work- shop on Uncertainty for Safe Utilization of Machine Learning in Medical Imaging, pages 12–21. Springer, 2023

    2023

  42. [50]

    Discounted adaptive online prediction

    Zhiyu Zhang, David Bombara, and Heng Yang. Discounted adaptive online prediction. arXiv preprint arXiv:2402.02720, 2024

    2024 arXiv

  43. [51]

    High- resolution uncertainty quantification for medical image seg- mentation

    Fangyu Zhou, Shih-Cheng Li, Xiaowei Yu, et al. High- resolution uncertainty quantification for medical image seg- mentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), pages 134–145. Springer, 2022

    2022

  44. [52]

    Con- formal prediction: A data perspective

    Xiaofan Zhou, Baiting Chen, Yu Gui, and Lu Cheng. Con- formal prediction: A data perspective. arXiv preprint arXiv:2410.06494, 2024

    2024 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.