REVIEW 4 major objections 4 minor 1 cited by
Prediction-Powered E-Values
T0 review · 4 major / 4 minor · reviewed 2026-08-08 · deepseek-v4-flash
Pith's one-line read Prediction-powered e-values make any e-value-based inference procedure valid when expensive labels are scarce, preserving anytime-validity and post-hoc validity.
desk verdict A useful and mostly sound extension of prediction-powered inference to e-values, but the abstract overreaches on missingness and the e-process proof has a genuine gap. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the prediction-powered e-value component $e_i^{\mathrm{ppi}} = e_i(\mu_i(X_i)) + (e_i(Y_i)-e_i(\mu_i(X_i)))\xi_i/\pi_i(X_i)$, a debiasing identity adapted to e-value products. Conditional on the past, this component has expectation exactly $E[e_i(Y_i)]$, so multiplying the components preserves the e-value property; the lower bound $\pi_i(X_i) \ge 1-a_i/b_i$ keeps the component nonnegative even when the label is missing. Because the correction is applied multiplicatively through the product, it also carries the test-supermartingale and e-process structure from the base e-values to the prediction-powered process. The same component is used as a plug-in replacement inside larger algorithms, which is what makes the method modular.
What would settle it
Simulate a stream in which label availability depends on an unobserved confounder correlated with the outcome (for instance, clinicians order confirmatory tests more often for high-risk patients), then run the prediction-powered e-value under the null; if the expected e-value exceeds 1 or the rejection rate exceeds the nominal level at a fixed sample size, the missing-at-random assumption is carrying the result.
Extended reading notes
Core claim
On the paper's own terms, the central discovery is Theorem 2.1: for a stream $(X_i,Y_i)$ with labels $Y_i$ collected according to independent Bernoulli indicators $\xi_i \sim \mathrm{Bern}(\pi_i(X_i))$, and base e-value components $e_i$ bounded in $[a_i,b_i]$ with $\pi_i \ge 1 - a_i/b_i$, the process $E_n^{\mathrm{ppi}} = \prod_{i=1}^n \left[e_i(\mu_i(X_i)) + (e_i(Y_i)-e_i(\mu_i(X_i)))\frac{\xi_i}{\pi_i(X_i)}\right]$ is a valid e-value for the same null. The reason is that each component has the same conditional expectation as $e_i(Y_i)$, so backward induction transfers validity, and nonnegativity is forced by the bound on $\pi_i$. The same transfer holds when the base e-values form a test supermartingale or an e-process, and therefore any algorithm whose validity depends only on receiving valid e-values—Assumption 2.5—remains valid when its inputs are replaced by prediction-powered e-values. Power is quantified by a growth-rate bound: the expected log of the prediction-powered e-value is at least the base growth rate minus an average Wasserstein distance between predictions and true labels.
Load-bearing premise
The load-bearing premise is that whether a label is collected is conditionally independent of the label itself given the cheap features and the past, and that the collection probability $\pi_i(X_i)$ is known and never falls below $1-a_i/b_i$; if labels are missing for reasons correlated with their values, the debiasing equality fails and the e-value is no longer valid.
Editorial extensions
If this is right
- Any e-value-based hypothesis test can be run with a stream of cheap features, a predictive model, and a small random subset of true labels, and remains valid under the null.
- If the base e-values form a test supermartingale or an e-process, the prediction-powered process does too, so anytime-valid confidence sequences and tests at arbitrary stopping times inherit validity.
- The predictive model and the label-collection policy can be updated online as labels arrive, without sacrificing validity.
- Inference tasks beyond Z-estimation, including change-point detection and constraint-based causal discovery, become prediction-powered; the case studies show large cost reductions at comparable or better power than labelled-only baselines.
- Post-hoc validity is preserved, so the significance level can be chosen after seeing the e-value.
Reading between the lines
- Beyond the paper's claims: because the debiasing identity is componentwise, prediction-powered e-values can likely be composed with any e-value combination rule (product, weighted average, U-statistic) without a new proof, opening multiple-testing and post-selection settings.
- The Wasserstein gap in the power bound suggests an actionable diagnostic that the paper does not pursue: if model predictions drift far from collected labels, the collection probability can be raised selectively; an adaptive rule targeting the empirical Wasserstein distance is a natural next step.
- The method's validity rests on known propensities; in observational missing-data settings these must be estimated, and the paper's guarantees do not automatically cover estimated $\pi_i$, so a sensitivity analysis would be needed.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces prediction-powered e-values. For a stream of cheap features X_i and expensive labels Y_i, with labeling indicators ξ_i ∼ Bern(π_i(X_i)), and for any base e-value component e_i(Y_i), it defines eppi_i = e_i(μ_i(X_i)) + (e_i(Y_i) − e_i(μ_i(X_i))) ξ_i/π_i(X_i) and Eppi_n = ∏_{i=1}^n eppi_i. Theorem 2.1 states that Eppi_n is a valid e-value and that the prediction-powered process preserves test-supermartingale and e-process properties. Proposition 2.3 translates this into confidence intervals/sequences, and Proposition 2.6 extends the construction to arbitrary algorithms whose validity depends only on the input e-values. Theorem 2.2 gives an expected-log-growth bound in terms of a Wasserstein distance between model outputs and labels. Four experiments illustrate the method on diabetes prevalence estimation, online risk monitoring, change-point detection, and causal discovery.
Significance. If the results hold as stated, the paper substantially broadens prediction-powered inference from Z-estimation settings to the full e-value toolbox, including anytime-valid testing, post-hoc validity, confidence sequences, and modular integration into larger algorithms such as change-point and causal-discovery procedures. The core validity argument is a clean expectation/tower-property computation, and the paper is explicit that the predictive model and the labeling policy may be updated online. The breadth of the case studies is a genuine strength. However, the advertised generality is exactly as strong as the coarsening assumption, and the power analysis currently rests on a non-rigorous Wasserstein argument, so the manuscript needs revision before the broader claims can be accepted.
major comments (4)
- [Section 2.1, Theorem 2.1] The validity claim is conditional on an active-labeling model, but the paper does not state this strongly enough. The debiasing identity E[eppi_i | F_i] = E[e_i(Y_i) | F_i] requires P(ξ_i = 1 | F_i) = π_i(X_i) together with conditional independence of ξ_i and Y_i given the features and the past. When labels are passively missing not at random, this fails and E[Eppi_n] can exceed 1 under H0. For example, if Y is uniform on [-1,1], e_i(y) = 1 + 0.1y, π_i = 0.5, but P(ξ_i = 1 | Y_i) = 0.9 for Y_i > 0 and 0.1 otherwise, then E[eppi_i] = 1.04. The abstract's phrase "data can be missing" and the diabetes case study, which simulates missingness, should not be read as covering general passive missingness; the known-propensity active-sampling condition should appear as a highlighted assumption.
- [Theorem 2.2 and Appendix A, Lemma A.3/Theorem A.4] The Wasserstein term W(μ_i(X_i) ∥ Y_i) is not defined. μ_i(X_i) and Y_i are random variables, while Lemma A.3 applies to distributions; in the proof the authors condition on Y_i and apply Lemma A.3 to the conditional law of μ_i(X_i), so the distance should be written as W(L(μ_i(X_i) | Y_i, ξ_i, π_i(X_i), F_i), δ_{Y_i}) or with equivalent notation. In addition, the constant c in the proof is c_i = u L_i max{(1 − π_i)/π_i, 1}, which depends on i; the theorem either needs a uniform bound or the c_i must appear inside the sum. Most importantly, the proof asserts that the log is Lipschitz because its argument is "bounded away from zero" by ε_i > 0, but ε_i is only claimed to be positive for each i, not uniformly, and the lower bound also depends on b_i. No uniform Lipschitz constant is established, so the growth-rate guarantee is not proven as stated.
- [Appendix A, Lemma A.2 and Theorem A.1(ii)] The proof conditions on an arbitrary event A and uses expressions such as E[· | A, F_k], but A is not assumed to be measurable in the filtration, so the tower-property steps are not justified. In the application to the e-process claim, A = {τ = n} need not belong to the sigma-field at time k, and may not belong to the pre-decision sigma-field at time n if τ is a stopping time with respect to the full data stream. Since e-process preservation is used in Proposition 2.6 and supports the anytime-validity claims, this needs a proper treatment, for example by applying optional stopping to the stopped process rather than conditioning on A.
- [Proposition 2.4 and Proposition A.8] The proof applies Theorem 2.2 and pulls the constant c outside the integral over θ. If the bounds a_i, b_i, the Lipschitz constants L_i, and hence the constant c depend on the null θ, this step requires a uniform-in-θ version of Theorem 2.2. The statement should either assume such uniformity explicitly or let c(θ) remain inside the integral.
minor comments (4)
- [Section 2.1 / Appendix A] The filtration F_i is never defined precisely; the authors should state whether it contains X_i but not ξ_i, Y_i, or both. A precise definition would remove much of the ambiguity in Lemma A.2 and the main validity proof.
- [Figures 1–4] The figures report single runs without error bars or repeated-seed summaries, so comparative statements such as "consistently note its much improved performance" are not quantitatively supported.
- [Section 3.2] There is a typo in the imputation-baseline description: "prediction-powreed inference" should be "prediction-powered inference."
- [Proposition 2.6] The statement of Proposition 2.6 is essentially a direct consequence of Assumption 2.5; labeling it as a corollary would better indicate the strength of the result.
Circularity Check
No circularity: the prediction-powered e-value validity proof is a direct conditional-expectation and martingale argument whose only inputs are the stated missingness mechanism and the base e-value validity.
full rationale
The central claim, Theorem 2.1, is that Eppi_n = product_i eppi_i is a valid e-value. The proof in Appendix A (Lemma A.2) shows by backward induction that E[prod_i eppi_i | F_k] equals E[prod_i ei(Y_i) | F_k], using only the definition eppi_i = ei(mu_i(X_i)) + (ei(Y_i) - ei(mu_i(X_i))) xi_i / pi_i(X_i), the conditional Bernoulli property xi_i ~ Bern(pi_i(X_i)) independent of Y_i given the past, and the boundedness condition pi_i(X_i) >= 1 - a_i/b_i that guarantees nonnegativity. This is a genuine derivation, not a restatement of the input: the base e-values' validity E[En] <= 1 is imported as a hypothesis, and the new process's validity is obtained from the matching-conditional-expectation lemma. No parameter is fitted and then reported as a prediction; mu_i and pi_i may be updated predictably, but validity holds for any such predictable choice because the xi_i / pi_i(X_i) importance-weighting exactly cancels the model-dependent term in conditional expectation. The power bound, Theorem 2.2, is an external benchmark rather than a forced consequence: it lower-bounds the expected log-growth of Eppi_n by that of the base e-value minus an explicit penalty involving the Wasserstein distance between mu_i(X_i) and Yi, and it is proved separately via Lemma A.3. Proposition 2.6 is a compositional statement whose validity hinges on Assumption 2.5 together with Theorem 2.1; it is a corollary of the already-proved validity result, not a circular input to it. The paper's citations of prediction-powered inference and active statistical inference (Angelopoulos et al. 2023a; Zrnic and Candes 2024) motivate the estimator but are not needed for Theorem 2.1, whose proof is self-contained. The missing-not-at-random concern raised in the reader's take is a boundary condition on the data-acquisition model, not an internal circularity: if xi_i depends on Yi, the cancellation E[(ei(Yi) - ei(mu_i(Xi))) xi_i / pi_i(Xi) | F_i] = E[ei(Yi) - ei(mu_i(Xi)) | F_i] fails, but the paper states the conditional-Bernoulli assumption explicitly in Section 2.1. Therefore no circular step is present.
Assumptions & free parameters
free parameters (4)
- clipping constant c =
varies per experiment (e.g., solves Eq. (2) for π_inf = 1%)
- Taylor expansion point a in Appendix B.2 =
unspecified
- labeling budget π_inf =
1% (Section 3.1), 0.5% (Section 3.2), 10% (Section 3.4)
- batch size B =
100 (Section 3.4)
assumptions (5)
- domain assumption ξ_i independent of Y_i given past and X_i, and E[ξ_i|past,X_i] = π_i(X_i)
- domain assumption e_i components are predictable and bounded in [a_i,b_i] with a_i > 0
- domain assumption the algorithm A's validity depends only on input e-values being valid (Assumption 2.5)
- ad hoc to paper e_i are L_i-Lipschitz and π_i bounded away from the lower bound by ϵ_i (Theorem 2.2)
- domain assumption In the causal discovery experiment, Fisher's z-test is valid (normality and asymptoticity) so calibrated e-values are approximate
Cite this review
Pith. "Pith review of Prediction-Powered E-Values." pith.science (2026). https://pith.science/paper/AP6OKNEK
@misc{pith2026250204294,
author = {Pith},
title = {Pith review of: Prediction-Powered E-Values},
year = {2026},
howpublished = {\url{https://pith.science/paper/AP6OKNEK}},
note = {Machine review of arXiv:2502.04294}
}
read the original abstract
Quality statistical inference requires a sufficient amount of data, which can be missing or hard to obtain. To this end, prediction-powered inference has risen as a promising methodology, but existing approaches are largely limited to Z-estimation problems such as inference of means and quantiles. In this paper, we apply ideas of prediction-powered inference to e-values. By doing so, we inherit all the usual benefits of e-values -- such as anytime-validity, post-hoc validity and versatile sequential inference -- as well as greatly expand the set of inferences achievable in a prediction-powered manner. In particular, we show that every inference procedure that can be framed in terms of e-values has a prediction-powered counterpart, given by our method. We showcase the effectiveness of our framework across a wide range of inference tasks, from simple hypothesis testing and confidence intervals to more involved procedures for change-point detection and causal discovery, which were out of reach of previous techniques. Our approach is modular and easily integrable into existing algorithms, making it a compelling choice for practical applications.
Figures
Figures from the paper (1 more)
Forward citations
Cited by 1 Pith paper
-
Improving Backward Conformal Prediction via Non-Conformity Score Transformation
ST-BCP tightens the coverage bound in Backward Conformal Prediction by applying a computable data-dependent transformation to nonconformity scores, reducing the average gap from 4.20% to 1.12% on benchmarks while prov...
Reference graph
Works this paper leans on
-
[3]
URL https://api.semanticscholar. org/CorpusID:268041436. Gr¨unwald, P. Beyond neyman-pearson: E-values en- able hypothesis testing with a data-driven alpha. Pro- ceedings of the National Academy of Sciences of the United States of America , 121 39:e2302098121,
-
[5]
URL https://api.semanticscholar. org/CorpusID:252531771. Gu, Y . and Xia, D. Local prediction-powered infer- ence. ArXiv, abs/2409.18321, 2024. URL https: //api.semanticscholar.org/CorpusID: 272968866. Howard, S. R., Ramdas, A., McAuliffe, J. D., and Sekhon, J. S. Time-uniform, nonparametric, nonasymp- totic confidence sequences. The Annals of Statistics ,
arXiv 2024
-
[8]
URL https://api.semanticscholar. org/CorpusID:272989144. Peters, J., Janzing, D., and Sch ¨olkopf, B. Elements of causal inference: Foundations and learning algorithms
-
[11]
URL https://api.semanticscholar. org/CorpusID:273707651. Ramdas, A., Gr ¨unwald, P. D., V ovk, V ., and Shafer, G. Game-theoretic statistics and safe anytime-valid inference. ArXiv, abs/2210.01948, 2022. URL https: //api.semanticscholar.org/CorpusID: 252715629. Shaer, S., Maman, G., and Romano, Y . Model-x sequential testing for conditional independence v...
arXiv 2022
-
[12]
URL https://api.semanticscholar. org/CorpusID:258426776. Spirtes, P., Meek, C., and Richardson, T. S. Causal infer- ence in the presence of latent variables and selection bias. In Conference on Uncertainty in Artificial Intelligence,
-
[16]
URL https://api.semanticscholar. org/CorpusID:252992535. Xu, Z., Wang, R., and Ramdas, A. Post-selection inference for e-value based confidence intervals. Electronic Journal of Statistics , 2022. URL https: //api.semanticscholar.org/CorpusID: 247619119. Zheng, Y ., Huang, B., Chen, W., Ramsey, J., Gong, M., Cai, R., Shimizu, S., Spirtes, P., and Zhang, K....
work page 2022
-
[1939]
URL https://api.semanticscholar. org/CorpusID:123425777. V ovk, V . and Wang, R. E-values: Calibration, combi- nation, and applications. Political Methods: Quan- titative Methods eJournal , 2019. URL https: //api.semanticscholar.org/CorpusID: 221834569. Wang, R. and Ramdas, A. False discovery rate control with e-values. Journal of the Royal Statistical So...
work page 2019
-
[1956]
URL https://api.semanticscholar. org/CorpusID:16143351. Koning, N. W. Post-hoc α hypothesis testing and the post-hoc p-value. 2023. URL https: //api.semanticscholar.org/CorpusID: 266191165. Little, R. J. A. and Rubin, D. B. Statistical analysis with missing data, third edition. Wiley Series in Prob- ability and Statistics , 2019. URL https://api. semantic...
work page 2023
Show all 17 references
-
[1995]
org/CorpusID:11987717
URL https://api.semanticscholar. org/CorpusID:11987717. Ville, J.-L. ´Etude critique de la notion de collectif
-
[1998]
Boyeau, P., Angelopoulos, A
DOI: https://doi.org/10.24432/C50K5N. Boyeau, P., Angelopoulos, A. N., Yosef, N., Malik, J., and Jordan, M. I. Autoeval done right: Using syn- thetic data for model evaluation. ArXiv, abs/2403.07008,
-
[2017]
org/CorpusID:86533208
URL https://api.semanticscholar. org/CorpusID:86533208. 9 Prediction-Powered E-Values Podkopaev, A. and Ramdas, A. Tracking the risk of a deployed model and detecting harm- ful distribution shifts. ArXiv, abs/2110.06177,
-
[2018]
org/CorpusID:219767477
URL https://api.semanticscholar. org/CorpusID:219767477. Kelly, J. L. A new interpretation of informa- tion rate. IRE Trans. Inf. Theory , 2:185–189,
-
[2020]
org/CorpusID:221516157
URL https://api.semanticscholar. org/CorpusID:221516157. Waudby-Smith, I. and Ramdas, A. Estimating means of bounded random variables by betting. Jour- nal of the Royal Statistical Society Series B: Statistical Methodology , 2020. URL https: //api.semanticscholar.org/CorpusID:...
2020 arXiv
-
[2021]
org/CorpusID:238634210
URL https://api.semanticscholar. org/CorpusID:238634210. Podkopaev, A. and Ramdas, A. Sequential predictive two- sample and independence testing. ArXiv, abs/2305.00143, 2023a. URL https://api.semanticscholar. org/CorpusID:258426601. Podkopaev, A. and Ramdas, A. Sequential pred...
-
[2022]
org/CorpusID:248496494
URL https://api.semanticscholar. org/CorpusID:248496494. Gr¨unwald, P., Henzi, A., and Lardy, T. Anytime-valid tests of conditional independence under model-x. Journal of the American Statistical Association, 119:1554 – 1565,
-
[2023]
nY i=k eppi i | A, Fk # = E
URL https://api.semanticscholar. org/CorpusID:263134612. Zrnic, T. and Candes, E. J. Active statistical infer- ence. ArXiv, abs/2403.03208, 2024. URL https: //api.semanticscholar.org/CorpusID: 268248530. 10 Prediction-Powered E-Values A. Proofs Throughout, we denote by Fi the ...
2024 arXiv
-
[2024]
org/CorpusID:268363495
URL https://api.semanticscholar. org/CorpusID:268363495. Burr, T. L. Causation, prediction, and search. Techno- metrics, 45:272 – 273, 2003. URL https://api. semanticscholar.org/CorpusID:10562706. CDC. Cdc – 2014 brfss survey data and documenta- tion, 2015. URL https://www.cdc...
2003 arXiv
Reviewed August 8, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.