Pith. sign in

REVIEW 4 major objections 4 minor 1 cited by

Prediction-Powered E-Values

T0 review · 4 major / 4 minor · reviewed 2026-08-08 · deepseek-v4-flash

Pith's one-line read Prediction-powered e-values make any e-value-based inference procedure valid when expensive labels are scarce, preserving anytime-validity and post-hoc validity.

desk verdict A useful and mostly sound extension of prediction-powered inference to e-values, but the abstract overreaches on missingness and the e-process proof has a genuine gap. read the letter →

arxiv 2502.04294 v2 pith:AP6OKNEK submitted 2025-02-06 stat.ML cs.LGstat.ME

classification stat.MLcs.LGstat.ME MSC 62L1062F0362L12
keywords prediction-poweredinferencee-valuesanytime-validtestsupermartingalee-processsequentialconfidencesequencesmissinglabels
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper establishes that any inference procedure built on e-values—nonnegative statistics with expectation at most 1 under the null, so that a large value is evidence against it—has a prediction-powered counterpart that remains valid when expensive labels are scarce. The construction replaces each base e-value component $e_i(Y_i)$ with a debiased version that combines a model prediction $e_i(\mu_i(X_i))$ with a correction using the few labels actually collected, weighted by their sampling probabilities. Under a missing-at-random assumption on label collection, the product of these components is still an e-value; if the base e-values were a test supermartingale or an e-process, the prediction-powered process inherits that property. This expands prediction-powered inference far beyond the means-and-quantiles settings of earlier work, to anytime-valid testing, confidence sequences, change-point detection, and causal discovery, while allowing the predictive model and collection policy to be updated online.

What carries the argument

The load-bearing object is the prediction-powered e-value component $e_i^{\mathrm{ppi}} = e_i(\mu_i(X_i)) + (e_i(Y_i)-e_i(\mu_i(X_i)))\xi_i/\pi_i(X_i)$, a debiasing identity adapted to e-value products. Conditional on the past, this component has expectation exactly $E[e_i(Y_i)]$, so multiplying the components preserves the e-value property; the lower bound $\pi_i(X_i) \ge 1-a_i/b_i$ keeps the component nonnegative even when the label is missing. Because the correction is applied multiplicatively through the product, it also carries the test-supermartingale and e-process structure from the base e-values to the prediction-powered process. The same component is used as a plug-in replacement inside larger algorithms, which is what makes the method modular.

What would settle it

Simulate a stream in which label availability depends on an unobserved confounder correlated with the outcome (for instance, clinicians order confirmatory tests more often for high-risk patients), then run the prediction-powered e-value under the null; if the expected e-value exceeds 1 or the rejection rate exceeds the nominal level at a fixed sample size, the missing-at-random assumption is carrying the result.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is Theorem 2.1: for a stream $(X_i,Y_i)$ with labels $Y_i$ collected according to independent Bernoulli indicators $\xi_i \sim \mathrm{Bern}(\pi_i(X_i))$, and base e-value components $e_i$ bounded in $[a_i,b_i]$ with $\pi_i \ge 1 - a_i/b_i$, the process $E_n^{\mathrm{ppi}} = \prod_{i=1}^n \left[e_i(\mu_i(X_i)) + (e_i(Y_i)-e_i(\mu_i(X_i)))\frac{\xi_i}{\pi_i(X_i)}\right]$ is a valid e-value for the same null. The reason is that each component has the same conditional expectation as $e_i(Y_i)$, so backward induction transfers validity, and nonnegativity is forced by the bound on $\pi_i$. The same transfer holds when the base e-values form a test supermartingale or an e-process, and therefore any algorithm whose validity depends only on receiving valid e-values—Assumption 2.5—remains valid when its inputs are replaced by prediction-powered e-values. Power is quantified by a growth-rate bound: the expected log of the prediction-powered e-value is at least the base growth rate minus an average Wasserstein distance between predictions and true labels.

Load-bearing premise

The load-bearing premise is that whether a label is collected is conditionally independent of the label itself given the cheap features and the past, and that the collection probability $\pi_i(X_i)$ is known and never falls below $1-a_i/b_i$; if labels are missing for reasons correlated with their values, the debiasing equality fails and the e-value is no longer valid.

Editorial extensions

If this is right

  • Any e-value-based hypothesis test can be run with a stream of cheap features, a predictive model, and a small random subset of true labels, and remains valid under the null.
  • If the base e-values form a test supermartingale or an e-process, the prediction-powered process does too, so anytime-valid confidence sequences and tests at arbitrary stopping times inherit validity.
  • The predictive model and the label-collection policy can be updated online as labels arrive, without sacrificing validity.
  • Inference tasks beyond Z-estimation, including change-point detection and constraint-based causal discovery, become prediction-powered; the case studies show large cost reductions at comparable or better power than labelled-only baselines.
  • Post-hoc validity is preserved, so the significance level can be chosen after seeing the e-value.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper's claims: because the debiasing identity is componentwise, prediction-powered e-values can likely be composed with any e-value combination rule (product, weighted average, U-statistic) without a new proof, opening multiple-testing and post-selection settings.
  • The Wasserstein gap in the power bound suggests an actionable diagnostic that the paper does not pursue: if model predictions drift far from collected labels, the collection probability can be raised selectively; an adaptive rule targeting the empirical Wasserstein distance is a natural next step.
  • The method's validity rests on known propensities; in observational missing-data settings these must be estimated, and the paper's guarantees do not automatically cover estimated $\pi_i$, so a sensitivity analysis would be needed.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper introduces prediction-powered e-values. For a stream of cheap features X_i and expensive labels Y_i, with labeling indicators ξ_i ∼ Bern(π_i(X_i)), and for any base e-value component e_i(Y_i), it defines eppi_i = e_i(μ_i(X_i)) + (e_i(Y_i) − e_i(μ_i(X_i))) ξ_i/π_i(X_i) and Eppi_n = ∏_{i=1}^n eppi_i. Theorem 2.1 states that Eppi_n is a valid e-value and that the prediction-powered process preserves test-supermartingale and e-process properties. Proposition 2.3 translates this into confidence intervals/sequences, and Proposition 2.6 extends the construction to arbitrary algorithms whose validity depends only on the input e-values. Theorem 2.2 gives an expected-log-growth bound in terms of a Wasserstein distance between model outputs and labels. Four experiments illustrate the method on diabetes prevalence estimation, online risk monitoring, change-point detection, and causal discovery.

Significance. If the results hold as stated, the paper substantially broadens prediction-powered inference from Z-estimation settings to the full e-value toolbox, including anytime-valid testing, post-hoc validity, confidence sequences, and modular integration into larger algorithms such as change-point and causal-discovery procedures. The core validity argument is a clean expectation/tower-property computation, and the paper is explicit that the predictive model and the labeling policy may be updated online. The breadth of the case studies is a genuine strength. However, the advertised generality is exactly as strong as the coarsening assumption, and the power analysis currently rests on a non-rigorous Wasserstein argument, so the manuscript needs revision before the broader claims can be accepted.

major comments (4)
  1. [Section 2.1, Theorem 2.1] The validity claim is conditional on an active-labeling model, but the paper does not state this strongly enough. The debiasing identity E[eppi_i | F_i] = E[e_i(Y_i) | F_i] requires P(ξ_i = 1 | F_i) = π_i(X_i) together with conditional independence of ξ_i and Y_i given the features and the past. When labels are passively missing not at random, this fails and E[Eppi_n] can exceed 1 under H0. For example, if Y is uniform on [-1,1], e_i(y) = 1 + 0.1y, π_i = 0.5, but P(ξ_i = 1 | Y_i) = 0.9 for Y_i > 0 and 0.1 otherwise, then E[eppi_i] = 1.04. The abstract's phrase "data can be missing" and the diabetes case study, which simulates missingness, should not be read as covering general passive missingness; the known-propensity active-sampling condition should appear as a highlighted assumption.
  2. [Theorem 2.2 and Appendix A, Lemma A.3/Theorem A.4] The Wasserstein term W(μ_i(X_i) ∥ Y_i) is not defined. μ_i(X_i) and Y_i are random variables, while Lemma A.3 applies to distributions; in the proof the authors condition on Y_i and apply Lemma A.3 to the conditional law of μ_i(X_i), so the distance should be written as W(L(μ_i(X_i) | Y_i, ξ_i, π_i(X_i), F_i), δ_{Y_i}) or with equivalent notation. In addition, the constant c in the proof is c_i = u L_i max{(1 − π_i)/π_i, 1}, which depends on i; the theorem either needs a uniform bound or the c_i must appear inside the sum. Most importantly, the proof asserts that the log is Lipschitz because its argument is "bounded away from zero" by ε_i > 0, but ε_i is only claimed to be positive for each i, not uniformly, and the lower bound also depends on b_i. No uniform Lipschitz constant is established, so the growth-rate guarantee is not proven as stated.
  3. [Appendix A, Lemma A.2 and Theorem A.1(ii)] The proof conditions on an arbitrary event A and uses expressions such as E[· | A, F_k], but A is not assumed to be measurable in the filtration, so the tower-property steps are not justified. In the application to the e-process claim, A = {τ = n} need not belong to the sigma-field at time k, and may not belong to the pre-decision sigma-field at time n if τ is a stopping time with respect to the full data stream. Since e-process preservation is used in Proposition 2.6 and supports the anytime-validity claims, this needs a proper treatment, for example by applying optional stopping to the stopped process rather than conditioning on A.
  4. [Proposition 2.4 and Proposition A.8] The proof applies Theorem 2.2 and pulls the constant c outside the integral over θ. If the bounds a_i, b_i, the Lipschitz constants L_i, and hence the constant c depend on the null θ, this step requires a uniform-in-θ version of Theorem 2.2. The statement should either assume such uniformity explicitly or let c(θ) remain inside the integral.
minor comments (4)
  1. [Section 2.1 / Appendix A] The filtration F_i is never defined precisely; the authors should state whether it contains X_i but not ξ_i, Y_i, or both. A precise definition would remove much of the ambiguity in Lemma A.2 and the main validity proof.
  2. [Figures 1–4] The figures report single runs without error bars or repeated-seed summaries, so comparative statements such as "consistently note its much improved performance" are not quantitatively supported.
  3. [Section 3.2] There is a typo in the imputation-baseline description: "prediction-powreed inference" should be "prediction-powered inference."
  4. [Proposition 2.6] The statement of Proposition 2.6 is essentially a direct consequence of Assumption 2.5; labeling it as a corollary would better indicate the strength of the result.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the prediction-powered e-value validity proof is a direct conditional-expectation and martingale argument whose only inputs are the stated missingness mechanism and the base e-value validity.

full rationale

The central claim, Theorem 2.1, is that Eppi_n = product_i eppi_i is a valid e-value. The proof in Appendix A (Lemma A.2) shows by backward induction that E[prod_i eppi_i | F_k] equals E[prod_i ei(Y_i) | F_k], using only the definition eppi_i = ei(mu_i(X_i)) + (ei(Y_i) - ei(mu_i(X_i))) xi_i / pi_i(X_i), the conditional Bernoulli property xi_i ~ Bern(pi_i(X_i)) independent of Y_i given the past, and the boundedness condition pi_i(X_i) >= 1 - a_i/b_i that guarantees nonnegativity. This is a genuine derivation, not a restatement of the input: the base e-values' validity E[En] <= 1 is imported as a hypothesis, and the new process's validity is obtained from the matching-conditional-expectation lemma. No parameter is fitted and then reported as a prediction; mu_i and pi_i may be updated predictably, but validity holds for any such predictable choice because the xi_i / pi_i(X_i) importance-weighting exactly cancels the model-dependent term in conditional expectation. The power bound, Theorem 2.2, is an external benchmark rather than a forced consequence: it lower-bounds the expected log-growth of Eppi_n by that of the base e-value minus an explicit penalty involving the Wasserstein distance between mu_i(X_i) and Yi, and it is proved separately via Lemma A.3. Proposition 2.6 is a compositional statement whose validity hinges on Assumption 2.5 together with Theorem 2.1; it is a corollary of the already-proved validity result, not a circular input to it. The paper's citations of prediction-powered inference and active statistical inference (Angelopoulos et al. 2023a; Zrnic and Candes 2024) motivate the estimator but are not needed for Theorem 2.1, whose proof is self-contained. The missing-not-at-random concern raised in the reader's take is a boundary condition on the data-acquisition model, not an internal circularity: if xi_i depends on Yi, the cancellation E[(ei(Yi) - ei(mu_i(Xi))) xi_i / pi_i(Xi) | F_i] = E[ei(Yi) - ei(mu_i(Xi)) | F_i] fails, but the paper states the conditional-Bernoulli assumption explicitly in Section 2.1. Therefore no circular step is present.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central validity theorem rests on standard martingale expectations plus the domain assumption that labels are collected at random given features. The power theorem adds Lipschitz and margin assumptions. The experiments introduce user-chosen budgets and clipping constants that are not part of the main theory. No new entities are postulated.

free parameters (4)
  • clipping constant c = varies per experiment (e.g., solves Eq. (2) for π_inf = 1%)
    Chosen per experiment to enforce bounded e-value components while meeting a labeling budget; not fitted to data but affects power in the case studies.
  • Taylor expansion point a in Appendix B.2 = unspecified
    Used to derive approximately optimal collection probabilities π_i; the approximation quality depends on this point.
  • labeling budget π_inf = 1% (Section 3.1), 0.5% (Section 3.2), 10% (Section 3.4)
    User-set budget for expensive label collection; experiments vary it.
  • batch size B = 100 (Section 3.4)
    Chosen so asymptotic p-values are approximately valid; a free parameter of the causal discovery experiment.
assumptions (5)
  • domain assumption ξ_i independent of Y_i given past and X_i, and E[ξ_i|past,X_i] = π_i(X_i)
    Core debiasing identity in Section 2.1 and Lemma A.2; fails under missing-not-at-random.
  • domain assumption e_i components are predictable and bounded in [a_i,b_i] with a_i > 0
    Needed to keep e^{ppi}_i nonnegative and to allow clipping; stated in Section 2.1.
  • domain assumption the algorithm A's validity depends only on input e-values being valid (Assumption 2.5)
    Used for Proposition 2.6; modules that also rely on other properties of e-values may not transfer.
  • ad hoc to paper e_i are L_i-Lipschitz and π_i bounded away from the lower bound by ϵ_i (Theorem 2.2)
    Needed for the power bound via Wasserstein distance; not needed for validity.
  • domain assumption In the causal discovery experiment, Fisher's z-test is valid (normality and asymptoticity) so calibrated e-values are approximate
    Section 3.4 relies on asymptotic p-values to construct e-values.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Prediction-Powered E-Values." pith.science (2026). https://pith.science/paper/AP6OKNEK

@misc{pith2026250204294,
  author       = {Pith},
  title        = {Pith review of: Prediction-Powered E-Values},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AP6OKNEK}},
  note         = {Machine review of arXiv:2502.04294}
}
read the original abstract

Quality statistical inference requires a sufficient amount of data, which can be missing or hard to obtain. To this end, prediction-powered inference has risen as a promising methodology, but existing approaches are largely limited to Z-estimation problems such as inference of means and quantiles. In this paper, we apply ideas of prediction-powered inference to e-values. By doing so, we inherit all the usual benefits of e-values -- such as anytime-validity, post-hoc validity and versatile sequential inference -- as well as greatly expand the set of inferences achievable in a prediction-powered manner. In particular, we show that every inference procedure that can be framed in terms of e-values has a prediction-powered counterpart, given by our method. We showcase the effectiveness of our framework across a wide range of inference tasks, from simple hypothesis testing and confidence intervals to more involved procedures for change-point detection and causal discovery, which were out of reach of previous techniques. Our approach is modular and easily integrable into existing algorithms, making it a compelling choice for practical applications.

Figures

Figures reproduced from arXiv: 2502.04294 by the authors.

Figure 1
Figure 1. Prediction-powered confidence sequences. The plot shows the p-landscape (i.e., parameter on the x-axis, reciprocal of the e-value on the y-axis) for the confidence sequence generated by our method (green), along with those for inference using only la￾belled samples (purple) and by using an imputation approach. The 95% confidence intervals for each p-landscape (i.e., region where the p-landscape is above 0.05) is sha… view at source ↗
Figure 2
Figure 2. Prediction-powered anytime-valid hypothesis testing. The plot shows the e-values over time for testing two null hy￾potheses – one on the bottom, which should be rejected, and one on top, which should not be rejected. Our prediction-powered e-values provide the strongest valid signal for rejection (E ≥ 20 for a significance level of 95%, marked by the dashed lines), as the imputation approach rejects before the null … view at source ↗
Figure 3
Figure 3. Prediction-powered change-point detection via e-values. The plot shows the exponential moving average of a time series (in blue), with the few collected labels denoted by the scattered Xs. Our prediction-powered methods detect the change-point accurately, while the base method that only considers the labelled data points does not detect any change-point. and update the non-prediction-powered e-value in Equa￾tion (3)… view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Prediction-powered causal discovery with e-values. We compare our prediction-powered causal discovery method with one that uses only labelled data. The lighter nodes correspond to the costly variables, while the darker nodes correspond to cheaper readily-available ones…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Improving Backward Conformal Prediction via Non-Conformity Score Transformation

    stat.ML 2026-02 reject novelty 7.0 of 10

    ST-BCP tightens the coverage bound in Backward Conformal Prediction by applying a computable data-dependent transformation to nonconformity scores, reducing the average gap from 4.20% to 1.12% on benchmarks while prov...

Reference graph

Works this paper leans on

17 extracted references · 11 canonical work pages · cited by 1 Pith paper

  1. [3]

    org/CorpusID:268041436

    URL https://api.semanticscholar. org/CorpusID:268041436. Gr¨unwald, P. Beyond neyman-pearson: E-values en- able hypothesis testing with a data-driven alpha. Pro- ceedings of the National Academy of Sciences of the United States of America , 121 39:e2302098121,

  2. [5]

    org/CorpusID:252531771

    URL https://api.semanticscholar. org/CorpusID:252531771. Gu, Y . and Xia, D. Local prediction-powered infer- ence. ArXiv, abs/2409.18321, 2024. URL https: //api.semanticscholar.org/CorpusID: 272968866. Howard, S. R., Ramdas, A., McAuliffe, J. D., and Sekhon, J. S. Time-uniform, nonparametric, nonasymp- totic confidence sequences. The Annals of Statistics ,

  3. [8]

    org/CorpusID:272989144

    URL https://api.semanticscholar. org/CorpusID:272989144. Peters, J., Janzing, D., and Sch ¨olkopf, B. Elements of causal inference: Foundations and learning algorithms

  4. [11]

    org/CorpusID:273707651

    URL https://api.semanticscholar. org/CorpusID:273707651. Ramdas, A., Gr ¨unwald, P. D., V ovk, V ., and Shafer, G. Game-theoretic statistics and safe anytime-valid inference. ArXiv, abs/2210.01948, 2022. URL https: //api.semanticscholar.org/CorpusID: 252715629. Shaer, S., Maman, G., and Romano, Y . Model-x sequential testing for conditional independence v...

  5. [12]

    org/CorpusID:258426776

    URL https://api.semanticscholar. org/CorpusID:258426776. Spirtes, P., Meek, C., and Richardson, T. S. Causal infer- ence in the presence of latent variables and selection bias. In Conference on Uncertainty in Artificial Intelligence,

  6. [16]

    org/CorpusID:252992535

    URL https://api.semanticscholar. org/CorpusID:252992535. Xu, Z., Wang, R., and Ramdas, A. Post-selection inference for e-value based confidence intervals. Electronic Journal of Statistics , 2022. URL https: //api.semanticscholar.org/CorpusID: 247619119. Zheng, Y ., Huang, B., Chen, W., Ramsey, J., Gong, M., Cai, R., Shimizu, S., Spirtes, P., and Zhang, K....

  7. [1939]

    org/CorpusID:123425777

    URL https://api.semanticscholar. org/CorpusID:123425777. V ovk, V . and Wang, R. E-values: Calibration, combi- nation, and applications. Political Methods: Quan- titative Methods eJournal , 2019. URL https: //api.semanticscholar.org/CorpusID: 221834569. Wang, R. and Ramdas, A. False discovery rate control with e-values. Journal of the Royal Statistical So...

  8. [1956]

    org/CorpusID:16143351

    URL https://api.semanticscholar. org/CorpusID:16143351. Koning, N. W. Post-hoc α hypothesis testing and the post-hoc p-value. 2023. URL https: //api.semanticscholar.org/CorpusID: 266191165. Little, R. J. A. and Rubin, D. B. Statistical analysis with missing data, third edition. Wiley Series in Prob- ability and Statistics , 2019. URL https://api. semantic...

Show all 17 references
  1. [1995]

    org/CorpusID:11987717

    URL https://api.semanticscholar. org/CorpusID:11987717. Ville, J.-L. ´Etude critique de la notion de collectif

  2. [1998]

    Boyeau, P., Angelopoulos, A

    DOI: https://doi.org/10.24432/C50K5N. Boyeau, P., Angelopoulos, A. N., Yosef, N., Malik, J., and Jordan, M. I. Autoeval done right: Using syn- thetic data for model evaluation. ArXiv, abs/2403.07008,

  3. [2017]

    org/CorpusID:86533208

    URL https://api.semanticscholar. org/CorpusID:86533208. 9 Prediction-Powered E-Values Podkopaev, A. and Ramdas, A. Tracking the risk of a deployed model and detecting harm- ful distribution shifts. ArXiv, abs/2110.06177,

  4. [2018]

    org/CorpusID:219767477

    URL https://api.semanticscholar. org/CorpusID:219767477. Kelly, J. L. A new interpretation of informa- tion rate. IRE Trans. Inf. Theory , 2:185–189,

  5. [2020]

    org/CorpusID:221516157

    URL https://api.semanticscholar. org/CorpusID:221516157. Waudby-Smith, I. and Ramdas, A. Estimating means of bounded random variables by betting. Jour- nal of the Royal Statistical Society Series B: Statistical Methodology , 2020. URL https: //api.semanticscholar.org/CorpusID:...

  6. [2021]

    org/CorpusID:238634210

    URL https://api.semanticscholar. org/CorpusID:238634210. Podkopaev, A. and Ramdas, A. Sequential predictive two- sample and independence testing. ArXiv, abs/2305.00143, 2023a. URL https://api.semanticscholar. org/CorpusID:258426601. Podkopaev, A. and Ramdas, A. Sequential pred...

  7. [2022]

    org/CorpusID:248496494

    URL https://api.semanticscholar. org/CorpusID:248496494. Gr¨unwald, P., Henzi, A., and Lardy, T. Anytime-valid tests of conditional independence under model-x. Journal of the American Statistical Association, 119:1554 – 1565,

  8. [2023]

    nY i=k eppi i | A, Fk # = E

    URL https://api.semanticscholar. org/CorpusID:263134612. Zrnic, T. and Candes, E. J. Active statistical infer- ence. ArXiv, abs/2403.03208, 2024. URL https: //api.semanticscholar.org/CorpusID: 268248530. 10 Prediction-Powered E-Values A. Proofs Throughout, we denote by Fi the ...

  9. [2024]

    org/CorpusID:268363495

    URL https://api.semanticscholar. org/CorpusID:268363495. Burr, T. L. Causation, prediction, and search. Techno- metrics, 45:272 – 273, 2003. URL https://api. semanticscholar.org/CorpusID:10562706. CDC. Cdc – 2014 brfss survey data and documenta- tion, 2015. URL https://www.cdc...

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.