REVIEW 3 major objections 2 minor 1 cited by
Multiply Robust Conformal Risk Control with Coarsened Data
T0 review · 3 major / 2 minor · reviewed 2026-08-05 · deepseek-v4-flash
Pith's one-line read This paper claims that conformal prediction can still give distribution-free valid prediction intervals when training data are coarsened—missing outcomes, missing covariates, or censored time-to-event outcomes—by deriving the efficient infl
desk verdict Promising but unverifiable from the abstract alone; the 'distribution-free' claim needs qualification about the conformal step's exchangeability. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The efficient influence function (EIF) of the target quantile under a coarsened-data semiparametric model, combined with a conformal risk-control wrapper. The EIF is the bias-correcting score that accounts for missingness and censoring; the conformal wrapper converts the corrected scores into finite-sample risk control guarantees.
What would settle it
Simulate coarsened training data where the probability of observing the outcome depends on the unobserved outcome itself, so missingness is not coarsening at random, and check the empirical coverage of the proposed intervals across many replications; if coverage falls materially below the nominal level, the central claim is refuted.
Extended reading notes
Core claim
The paper's central claim is that one can obtain distribution-free prediction regions with guaranteed risk control even when the training sample is coarsened, meaning some outcomes are missing, some covariates are missing, or time-to-event outcomes are censored. The construction derives the efficient influence function of the quantile being predicted under a semiparametric model for the coarsened data, then feeds this correction into a new conformal risk control procedure. The result is prediction intervals that keep their nominal coverage without distributional assumptions on the outcome, while the semiparametric formulation allows the nuisance functions to be estimated by flexible machine
Load-bearing premise
The coarsening mechanism is correctly specified as a semiparametric model under which the efficient influence function is well-defined; if missingness is informative or the model is misspecified, the adjustment can be biased and the guarantee can fail.
Editorial extensions
If this is right
- Prediction intervals can be built directly from training sets with partially observed outcomes and covariates while keeping the stated coverage guarantee.
- Time-to-event training data with censored outcomes no longer force the user to discard or impute; the censoring is folded into the adjustment.
- Because nuisance functions can be learned by flexible machine learning, the method scales to high-dimensional or complex data without parametric outcome models.
- Under covariate shift, the same construction yields stronger coverage than standard conformal intervals.
- In monotone missingness settings, the framework produces multiply robust prediction sets that remain valid if some of several candidate models are misspecified.
Reading between the lines
- If the construction is as general as it appears, the same EIF-plus-risk-control recipe should extend to other target functionals such as conditional quantiles or expected loss, not just marginal quantiles.
- A practical litmus test would be a survival-analysis benchmark with heavy administrative censoring: intervals should stay calibrated even when censoring rates vary across covariate groups.
- The multiply robust property suggests an ensemble interpretation: one could deliberately fit several candidate coarsening models and let the conformal step choose which correction to trust, potentially improving robustness beyond the paper's stated scenarios.
- The framework's dependence on the EIF implies that its coverage guarantee is only as good as the semiparametric model; users who cannot justify coarsening at random should expect the method to degrade in a measurable, quantifiable way.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a general framework for conformal prediction with coarsened training data, in which the outcome or some covariates may be missing or censored. The central idea is to derive the efficient influence function (EIF) of the quantile of the outcome under a semiparametric coarsening model and to combine the resulting adjusted scores with a conformal risk control procedure. The abstract claims distribution-free valid prediction regions, improved coverage under covariate shift, multiply robust prediction sets in monotone missingness scenarios, and supporting simulation studies. However, the submitted full text is largely an unreadable encoded placeholder: no equations, theorem statements, proofs, or numeric results are legible, so the claims cannot currently be audited.
Significance. If the results are correct, the paper addresses a practically important gap in conformal prediction: handling missing or censored training data without giving up finite-sample distribution-free guarantees. Deriving EIFs for the target quantile is a principled way to incorporate flexible nuisance estimators, and the claimed multiply robust and covariate-shift extensions would be valuable additions to the conformal inference toolbox. The paper deserves credit for identifying a real limitation of standard conformal methods and for proposing a semiparametrically grounded solution. That said, the significance can only be assessed after the manuscript is made readable and after the key theoretical claim about the conformal step is stated with precise assumptions.
major comments (3)
- [Full text (all sections)] The full text is not readable: mathematical expressions, equations, and proofs appear as mis-encoded placeholders, and no theorem statements or simulation tables are legible. The abstract alone is insufficient to establish the correctness of the proposed construction. A complete, properly rendered manuscript with numbered assumptions, theorem statements, and proofs is required before the central claim can be evaluated.
- [Abstract, conformal risk control step] The abstract states that prediction regions are 'distribution-free valid' but does not state whether the guarantee is exact finite-sample, asymptotic after cross-fitting, or first-order asymptotic. The EIF is a first-order correction for the target quantile; it does not imply that individual pseudo-scores are exchangeable with the test score, which is what exact conformal risk control requires. If the pseudo-scores only match the target quantile in expectation, the tail behavior controlling miscoverage may not be preserved. The authors should either prove a (weighted) exchangeability property for their construction or explicitly state the weaker guarantee they obtain.
- [Abstract, semiparametric model assumptions] The abstract says the EIF is derived 'under a given semiparametric model for the coarsened data' but does not list the identifying conditions: coarsening at random, positivity of the missingness propensity, correct specification of the coarsening model, and rate conditions on the nuisance estimators. If the coarsening model is misspecified or the propensity can be arbitrarily close to zero, the EIF-based adjustment is biased and the 'distribution-free' property with respect to the outcome may no longer hold. These assumptions and regularity conditions must be stated precisely.
minor comments (2)
- [Full text] The document contains numerous uninterpretable character sequences, making it impossible to identify equation numbers, section numbers, or even the bibliography. A clean, correctly encoded PDF is a prerequisite for any meaningful review.
- [Abstract] The phrase 'multiply robust' is used without a definition in the abstract; it should be clarified whether this means robustness to misspecification of one of several nuisance models, and if so, which ones.
Circularity Check
No circularity identifiable; no equation-level reduction demonstrated.
full rationale
The available manuscript text is heavily corrupted by extraction artifacts, so the only clear derivation-chain statement is the abstract: the paper derives the efficient influence function (EIF) of the target quantile under a semiparametric coarsening model and then combines this with a conformal risk control procedure. On its face, this is a two-stage construction: the EIF is derived from the model, and the conformal threshold is selected to control a risk. There is no visible equation or passage showing that the conformal threshold is defined as the EIF-based quantile estimate, that a fitted parameter is renamed as a prediction, or that a central premise is imported from a self-citation. The skeptical concern that EIF-adjusted pseudo-scores may not be exchangeable with the test score is a validity/correctness objection, not a circularity objection, and the instructions explicitly require exhibiting a specific reduction (Eq. X = Eq. Y by construction) before flagging circularity. Because no such reduction can be located in the readable portions of the paper, the appropriate finding is no significant circularity.
Assumptions & free parameters
assumptions (3)
- domain assumption The coarsening mechanism follows a correctly specified semiparametric model, plausibly coarsening at random.
- domain assumption Standard conformal exchangeability and risk-control assumptions hold, with any covariate shift handled by the paper's stated conditions.
- standard math Regularity conditions for pathwise differentiability and for nuisance function estimation rates hold.
Cite this review
Pith. "Pith review of Multiply Robust Conformal Risk Control with Coarsened Data." pith.science (2026). https://pith.science/paper/OUKM67AI
@misc{pith2026250815489,
author = {Pith},
title = {Pith review of: Multiply Robust Conformal Risk Control with Coarsened Data},
year = {2026},
howpublished = {\url{https://pith.science/paper/OUKM67AI}},
note = {Machine review of arXiv:2508.15489}
}
read the original abstract
Conformal Prediction (CP) has recently received a tremendous amount of interest, leading to a wide range of new theoretical and methodological results for predictive inference with formal theoretical guarantees. However, the vast majority of CP methods assume that all units in the training data have fully observed data on both the outcome and covariates of primary interest, an assumption that rarely holds in practice. In reality, training data are often missing the outcome, a subset of covariates, or both on some units. In addition, time-to-event outcomes in the training set may be censored due to dropout or administrative end-of-follow-up. Accurately accounting for such coarsened data in the training sample while fulfilling the primary objective of well-calibrated conformal predictive inference, requires robustness and efficiency considerations. In this paper, we consider the general problem of obtaining distribution-free valid prediction regions for an outcome given coarsened training data. Leveraging modern semiparametric theory, we achieve our goal by deriving the efficient influence function of the quantile of the outcome we aim to predict, under a given semiparametric model for the coarsened data, carefully combined with a novel conformal risk control procedure. Our principled use of semiparametric theory has the key advantage of facilitating flexible machine learning methods such as random forests to learn the underlying nuisance functions of the semiparametric model. A straightforward application of the proposed general framework produces prediction intervals with stronger coverage properties under covariate shift, as well as the construction of multiply robust prediction sets in monotone missingness scenarios. We further illustrate the performance of our methods through various simulation studies.
Forward citations
Cited by 1 Pith paper
-
A Joint Finite-Sample Certificate for Adaptive Selective Conformal Risk Control
A joint finite-sample certificate for adaptive selective conformal risk control that treats selected risk as a ratio and couples empirical-Bernstein, Clopper-Pearson, and closeness bounds.
Reference graph
Works this paper leans on
-
[1]
Anastasios N Angelopoulos, Stephen Bates, Adam Fisch, Lihua Lei, and Tal Schuster. Conformal risk control. arXiv preprint arXiv:2208.02814, 2022
arXiv 2022
-
[2]
Near-optimal rate of consistency for linear models with missing values
Alexis Ayme, Claire Boyer, Aymeric Dieuleveut, and Erwan Scornet. Near-optimal rate of consistency for linear models with missing values. In International Conference on Machine Learning, pages 1211--1243. PMLR, 2022
work page 2022
-
[3]
Predictive inference with the jackknife+
Rina Foygel Barber, Emmanuel J Candes, Aaditya Ramdas, and Ryan J Tibshirani. Predictive inference with the jackknife+. The Annals of Statistics, 49 0 (1): 0 486--507, 2021
work page 2021
-
[4]
Conformal prediction beyond exchangeability
Rina Foygel Barber, Emmanuel J Candes, Aaditya Ramdas, and Ryan J Tibshirani. Conformal prediction beyond exchangeability. The Annals of Statistics, 51 0 (2): 0 816--845, 2023
2023
-
[5]
Predictive inference with weak supervision
Maxime Cauchois, Suyash Gupta, Alnur Ali, and John C Duchi. Predictive inference with weak supervision. Journal of Machine Learning Research, 25 0 (118): 0 1--45, 2024 a
work page 2024
-
[6]
Robust validation: Confident predictions even when distributions shift
Maxime Cauchois, Suyash Gupta, Alnur Ali, and John C Duchi. Robust validation: Confident predictions even when distributions shift. Journal of the American Statistical Association, 119 0 (548): 0 3033--3044, 2024 b
work page 2024
-
[7]
Distributional conformal prediction
Victor Chernozhukov, Kaspar W \"u thrich, and Yinchu Zhu. Distributional conformal prediction. Proceedings of the National Academy of Sciences, 118 0 (48): 0 e2107794118, 2021
work page 2021
-
[8]
Methotrexate and mortality in patients with rheumatoid arthritis: a prospective study
Hyon K Choi, Miguel A Hern \'a n, John D Seeger, James M Robins, and Frederick Wolfe. Methotrexate and mortality in patients with rheumatoid arthritis: a prospective study. The Lancet, 359 0 (9313): 0 1173--1177, 2002
work page 2002
Show all 50 references
-
[9]
Relative expected instantaneous loss bounds
J \"u rgen Forster and Manfred K Warmuth. Relative expected instantaneous loss bounds. Journal of Computer and System Sciences, 64 0 (1): 0 76--102, 2002
2002
-
[10]
The limits of distribution-free conditional predictive inference
Rina Foygel Barber, Emmanuel J Candes, Aaditya Ramdas, and Ryan J Tibshirani. The limits of distribution-free conditional predictive inference. Information and Inference: A Journal of the IMA, 10 0 (2): 0 455--482, 2021
2021
-
[11]
Conformal prediction with conditional guarantees
Isaac Gibbs, John J Cherian, and Emmanuel J Cand \`e s. Conformal prediction with conditional guarantees. arXiv preprint arXiv:2305.12616, 2023
2023 arXiv
-
[12]
Nearest neighbor based conformal prediction
Laszlo Gy \"o rfi and Harro Walk. Nearest neighbor based conformal prediction. In Annales de l'ISUP, volume 63, pages 173--190, 2019
2019
-
[13]
Introduction to the special section on missing data
Julie Josse and Jerome P Reiter. Introduction to the special section on missing data. 2018
2018
-
[14]
On the consistency of supervised learning with missing values
Julie Josse, Jacob M Chen, Nicolas Prost, Ga \"e l Varoquaux, and Erwan Scornet. On the consistency of supervised learning with missing values. Statistical Papers, 65 0 (9): 0 5447--5479, 2024
2024
-
[15]
Batch multivalid conformal prediction
Christopher Jung, Georgy Noarov, Ramya Ramalingam, and Aaron Roth. Batch multivalid conformal prediction. arXiv preprint arXiv:2209.15145, 2022
2022 arXiv
-
[16]
Predictive inference is free with the jackknife+-after-bootstrap
Byol Kim, Chen Xu, and Rina Barber. Predictive inference is free with the jackknife+-after-bootstrap. Advances in Neural Information Processing Systems, 33: 0 4138--4149, 2020
2020
-
[17]
Exchangeability, conformal prediction, and rank tests
Arun Kumar Kuchibhotla. Exchangeability, conformal prediction, and rank tests. arXiv preprint arXiv:2005.06095, 2020
2005 arXiv
-
[18]
Unified methods for censored longitudinal data and causality
Mark J Laan and James M Robins. Unified methods for censored longitudinal data and causality. Springer, 2003
2003
-
[19]
Neumiss networks: differentiable programming for supervised learning with missing values
Marine Le Morvan, Julie Josse, Thomas Moreau, Erwan Scornet, and Ga \"e l Varoquaux. Neumiss networks: differentiable programming for supervised learning with missing values. Advances in Neural Information Processing Systems, 33: 0 5980--5990, 2020 a
2020
-
[20]
Linear predictor on linearly-generated data with missing values: non consistency and solutions
Marine Le Morvan, Nicolas Prost, Julie Josse, Erwan Scornet, and Ga \"e l Varoquaux. Linear predictor on linearly-generated data with missing values: non consistency and solutions. In International Conference on Artificial Intelligence and Statistics, pages 3165--3174. PMLR, 2020 b
2020
-
[21]
What’sa good imputation to predict with missing values? Advances in Neural Information Processing Systems, 34: 0 11530--11540, 2021
Marine Le Morvan, Julie Josse, Erwan Scornet, and Ga \"e l Varoquaux. What’sa good imputation to predict with missing values? Advances in Neural Information Processing Systems, 34: 0 11530--11540, 2021
2021
-
[22]
Distribution-free prediction bands for non-parametric regression
Jing Lei and Larry Wasserman. Distribution-free prediction bands for non-parametric regression. Journal of the Royal Statistical Society Series B: Statistical Methodology, 76 0 (1): 0 71--96, 2014
2014
-
[23]
Distribution-free prediction sets
Jing Lei, James Robins, and Larry Wasserman. Distribution-free prediction sets. Journal of the American Statistical Association, 108 0 (501): 0 278--287, 2013
2013
-
[24]
Distribution-free predictive inference for regression
Jing Lei, Max G’Sell, Alessandro Rinaldo, Ryan J Tibshirani, and Larry Wasserman. Distribution-free predictive inference for regression. Journal of the American Statistical Association, 113 0 (523): 0 1094--1111, 2018
2018
-
[25]
Conformal inference of counterfactuals and individual treatment effects
Lihua Lei and Emmanuel J Cand \'e s. Conformal inference of counterfactuals and individual treatment effects. Journal of the Royal Statistical Society Series B: Statistical Methodology, 83 0 (5): 0 911--938, 2021
2021
-
[26]
Statistical analysis with missing data
Roderick JA Little and Donald B Rubin. Statistical analysis with missing data. John Wiley & Sons, 2019
2019
-
[27]
Inductive confidence machines for regression
Harris Papadopoulos, Kostas Proedrou, Volodya Vovk, and Alex Gammerman. Inductive confidence machines for regression. In Machine learning: ECML 2002: 13th European conference on machine learning Helsinki, Finland, August 19--23, 2002 proceedings 13, pages 345--356. Springer, 2002
2002
-
[28]
Super learner in prediction
Eric C Polley and Mark J Van der Laan. Super learner in prediction. 2010
2010
-
[29]
Prediction sets adaptive to unknown covariate shift
Hongxiang Qiu, Edgar Dobriban, and Eric Tchetgen Tchetgen. Prediction sets adaptive to unknown covariate shift. Journal of the Royal Statistical Society Series B: Statistical Methodology, 85 0 (5): 0 1680--1705, 2023
2023
-
[30]
Non-response models for the analysis of non-monotone ignorable missing data
James M Robins and Richard D Gill. Non-response models for the analysis of non-monotone ignorable missing data. Statistics in medicine, 16 0 (1): 0 39--56, 1997
1997
-
[31]
Estimation of regression coefficients when some regressors are not always observed
James M Robins, Andrea Rotnitzky, and Lue Ping Zhao. Estimation of regression coefficients when some regressors are not always observed. Journal of the American statistical Association, 89 0 (427): 0 846--866, 1994
1994
-
[32]
Conformalized quantile regression
Yaniv Romano, Evan Patterson, and Emmanuel Candes. Conformalized quantile regression. Advances in neural information processing systems, 32, 2019
2019
-
[33]
Classification with valid and adaptive coverage
Yaniv Romano, Matteo Sesia, and Emmanuel Candes. Classification with valid and adaptive coverage. Advances in neural information processing systems, 33: 0 3581--3591, 2020
2020
-
[34]
Characterization of parameters with a mixed bias property
Andrea Rotnitzky, Ezequiel Smucler, and James M Robins. Characterization of parameters with a mixed bias property. Biometrika, 108 0 (1): 0 231--238, 2021
2021
-
[35]
Conformal prediction using conditional histograms
Matteo Sesia and Yaniv Romano. Conformal prediction using conditional histograms. Advances in Neural Information Processing Systems, 34: 0 6304--6315, 2021
2021
-
[36]
Conformal time-series forecasting
Kamile Stankeviciute, Ahmed M Alaa, and Mihaela van der Schaar. Conformal time-series forecasting. Advances in neural information processing systems, 34: 0 6216--6228, 2021
2021
-
[37]
On inverse probability weighting for nonmonotone missing at random data
BaoLuo Sun and Eric J Tchetgen Tchetgen. On inverse probability weighting for nonmonotone missing at random data. Journal of the American Statistical Association, 113 0 (521): 0 369--379, 2018
2018
-
[38]
Inverse-probability-weighted estimation for monotone and nonmonotone missing data
BaoLuo Sun, Neil J Perkins, Stephen R Cole, Ofer Harel, Emily M Mitchell, Enrique F Schisterman, and Eric J Tchetgen Tchetgen. Inverse-probability-weighted estimation for monotone and nonmonotone missing data. American journal of epidemiology, 187 0 (3): 0 585--591, 2018
2018
-
[39]
Statistical methods for robust inference in causal and missing data models
Eric Joel Tchetgen Tchetgen. Statistical methods for robust inference in causal and missing data models. Harvard University, 2006
2006
-
[40]
Conformal prediction under covariate shift
Ryan J Tibshirani, Rina Foygel Barber, Emmanuel Candes, and Aaditya Ramdas. Conformal prediction under covariate shift. Advances in neural information processing systems, 32, 2019
2019
-
[41]
Semiparametric theory and missing data, volume 4
Anastasios A Tsiatis. Semiparametric theory and missing data, volume 4. Springer, 2006
2006
-
[42]
The missing indicator method: From low to high dimensions
Mike Van Ness, Tomas M Bosschieter, Roberto Halpin-Gregorio, and Madeleine Udell. The missing indicator method: From low to high dimensions. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 5004--5015, 2023
2023
-
[43]
Conditional validity of inductive conformal predictors
Vladimir Vovk. Conditional validity of inductive conformal predictors. In Asian conference on machine learning, pages 475--490. PMLR, 2012
2012
-
[44]
Mondrian confidence machine
Vladimir Vovk, David Lindsay, Ilia Nouretdinov, and Alex Gammerman. Mondrian confidence machine. Technical Report, 2003
2003
-
[45]
Algorithmic learning in a random world, volume 29
Vladimir Vovk, Alexander Gammerman, and Glenn Shafer. Algorithmic learning in a random world, volume 29. Springer, 2005
2005
-
[46]
Finite-sample efficient conformal prediction
Yachong Yang and Arun Kumar Kuchibhotla. Finite-sample efficient conformal prediction. arXiv preprint arXiv:2104.13871, 5, 2021
2021 arXiv
-
[47]
Forster-warmuth counterfactual regression: A unified learning approach
Yachong Yang, Arun Kumar Kuchibhotla, and Eric Tchetgen Tchetgen. Forster-warmuth counterfactual regression: A unified learning approach. arXiv preprint arXiv:2307.16798, 2023
2023 arXiv
-
[48]
Doubly robust calibration of prediction sets under covariate shift
Yachong Yang, Arun Kumar Kuchibhotla, and Eric Tchetgen Tchetgen. Doubly robust calibration of prediction sets under covariate shift. Journal of the Royal Statistical Society Series B: Statistical Methodology, page qkae009, 2024
2024
-
[49]
Conformal prediction with missing values
Margaux Zaffran, Aymeric Dieuleveut, Julie Josse, and Yaniv Romano. Conformal prediction with missing values. In International Conference on Machine Learning, pages 40578--40604. PMLR, 2023
2023
-
[50]
Predictive uncertainty quantification with missing covariates
Margaux Zaffran, Julie Josse, Yaniv Romano, and Aymeric Dieuleveut. Predictive uncertainty quantification with missing covariates. arXiv preprint arXiv:2405.15641, 2024
2024 arXiv
Reviewed August 5, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.