Pith. sign in

REVIEW 4 major objections 4 minor 41 references

PAC Off-Policy Prediction of Contextual Bandits

T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper proves that logged data from one policy can be turned into a prediction interval for another policy's reward that is probably approximately correct: with confidence $1-\delta$, its conditional miscoverage is at most $\epsilon$.

desk verdict A credible PAC extension for off-policy prediction, with a load-bearing bounded-ratio assumption the abstract glosses over and one explicitly unproven bound; worth refereeing after fixes. read the letter →

arxiv 2507.16236 v1 pith:3KIHCNWQ submitted 2025-07-22 stat.ML cs.LG

Yilong Wan , Yuqiang Li , Xianyi Wu This is my paper · ORCID
classification stat.MLcs.LG
keywords ContextualBanditsOff-PolicyEvaluationProbablyApproximatelyCorrectInferenceConformalPredictionReinforcementLearningIntervalsRejectionSampling
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that off-policy evaluation in contextual bandits can be made training-conditional rather than merely marginal: from data logged under a behavior policy, one can build a prediction interval for the reward of a different target policy such that the interval's miscoverage is at most $\epsilon$ with confidence at least $1-\delta$. The construction, PACOPP, corrects the distribution shift by rejection sampling and then applies a PAC-valid conformal threshold, avoiding any need to estimate reward densities. The authors prove finite-sample PAC validity, show that the achieved confidence approaches $1-\delta$ and coverage approaches $\epsilon$ at a $1/\sqrt{n}$ rate, and prove asymptotic equivalence to the oracle quantile interval. They also analyze the case of an unknown behavior policy, where the coverage guarantee degrades by the estimation error of the policy ratio. A sympathetic reader would care because safety-critical deployment needs guarantees that hold for the data set actually collected, not only averaged over repeated data sets.

What carries the argument

The load-bearing objects are three. The first is the policy ratio $w(s,a)=\pi_e(a|s)/\pi_b(a|s)$ with finite supremum $B$: rejection sampling keeps $(S_i,A_i,R_i)$ with probability $w(S_i,A_i)/B$, and Proposition 1 shows the accepted $(S_i,R_i)$ pairs are i.i.d. from the target marginal $P^{\pi_e}$, the distribution of the test reward. The second is the family of candidate intervals $\hat C_\tau(S)=[\hat q_{\epsilon_{lo}}(S)-\tau,\hat q_{\epsilon_{up}}(S)+\tau]$ from conformal quantile regression, scored by non-conformity values $\tau_i=\max\{\hat q_{\epsilon_{lo}}(S_i)-R_i,\,R_i-\hat q_{\epsilon_{up}}(S_i)\}$. The third is the PAC threshold $k(M,\epsilon,\delta)=\max\{k:F_{\mathrm{Bin}(M,\epsilon)}(k)\le\delta\}$, which lets the algorithm pick the smallest $\tau=\tau_{(M-k)}$ such that no more than $k$ of the $M$ calibration points fall outside; the binomial tail is what converts one data set into a $1-\delta$-confident, $\epsilon$-coverage statement. The efficiency theorems then use the Berry-Esseen inequality to control the normal approximation of that binomial tail, and total-variation distance to bound the extra coverage loss when the behavior policy must be estimated.

What would settle it

Take a two-action bandit with $\pi_b$ choosing action 1 with probability 1 and $\pi_e$ choosing action 2 with probability 1: then $B=\infty$, Proposition 1 fails, and Algorithm 1 has no accepted sample from the target policy, so the claimed $(\epsilon,\delta)$ guarantee cannot hold for any fixed sample size. A quantitative check would simulate the known-$\pi_b$ setting with a large but finite $B$ and compare the empirical probability that $L^{\pi_e}(\hat C_{\tilde\tau})\le\epsilon$ with the upper bound $1-\delta+C/\sqrt{n}$ from Theorem 2; exceeding the bound by more than the constant would refute the stated rate.

Watch

Extended reading notes

Core claim

The central claim is that a probably approximately correct prediction interval for the reward of a target policy can be constructed from offline data logged under a different behavior policy, without estimating the reward distribution. The procedure PACOPP first rejects each logged tuple with probability $1 - w(S_i,A_i)/B$, where $w(s,a)=\pi_e(a|s)/\pi_b(a|s)$ and $B=\sup w$; the surviving context-reward pairs are then i.i.d. from the target marginal $P^{\pi_e}$ (Proposition 1). On a calibration split of this accepted sample, it fits conditional quantile estimators $\hat q_{\epsilon_{lo}}$ and $\hat q_{\epsilon_{up}}$, forms the nested intervals $\hat C_\tau(S)=[\hat q_{\epsilon_{lo}}(S)-\tau,\hat q_{\epsilon_{up}}(S)+\tau]$, and chooses $\tilde\tau=\tau_{(M-k(M,\epsilon,\delta))}$ with $k(M,\epsilon,\delta)=\max\{k:F_{\mathrm{Bin}(M,\epsilon)}(k)\le\delta\}$. Theorem 1 proves that this output is an $(\epsilon,\delta)$-PAC prediction interval; Theorems 2 and 3 show that the confidence in coverage approaches $1-\delta$ and miscoverage concentrates around $\epsilon$ at a $1/\sqrt{n}$ rate; Theorem 5 proves asymptotic equality with the oracle interval $[q_{\epsilon_{lo}}(S),q_{\epsilon_{up}}(S)]$ under the target policy; Theorems 6 and 7 extend the guarantee to an estimated behavior policy, with coverage loss bounded by the estimation error of the weight function.

Load-bearing premise

The whole construction depends on a finite density-ratio bound $B=\sup_{s,a}\pi_e(a|s)/\pi_b(a|s)$: if the target policy can choose actions that the behavior policy essentially never picks, rejection sampling cannot produce an i.i.d. sample from the target distribution and the PAC guarantee has nothing to hold onto.

Editorial extensions

If this is right

  • For any prescribed $\epsilon$ and $\delta$, Algorithm 1 returns a prediction interval whose data-conditional miscoverage is at most $\epsilon$ with probability at least $1-\delta$, with no assumptions on the reward distribution or action-space geometry.
  • Theorems 2 and 3 give matching upper and lower bounds: the probability that coverage misses its nominal level decays at a $1/\sqrt{n}$ rate, so the interval is not just valid but efficient.
  • Theorem 5 shows that, when the conditional quantile estimators are consistent, the interval converges in Lebesgue measure to the oracle interval $[q_{\epsilon_{lo}}(S),q_{\epsilon_{up}}(S)]$ under the target policy.
  • When the behavior policy is unknown, Theorem 6 bounds the coverage loss by the $\ell^1$ estimation error $\Delta_w$ of the weights, and Theorem 7 gives an explicit PAC bound under maximum-likelihood estimation over a finite policy class.
  • The method applies to continuous action spaces and requires no estimate of the reward density, unlike weighted-conformal baselines.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The finite ratio condition $B<\infty$ is also a practical design rule: a target policy that ever assigns positive mass where the behavior policy is zero lies outside the method, so safe deployment needs either full-support logging or a clipped target.
  • Because the calibration threshold $k(M,\epsilon,\delta)$ uses only the binomial tail, the same PAC construction should extend to other offline inference tasks, such as quantile regression, risk measures, or treatment effects, wherever a rejection-sampling step can produce an i.i.d. target sample.
  • A note for readers: the second inequality in Theorem 6 (equation (14)) is asserted with its proof deferred to Appendix A.6, which omits the proof, so the unknown-policy efficiency statement rests on an assertion rather than a demonstrated argument.
  • Rejection sampling discards roughly $1-1/B$ of the logged data; a natural testable extension is a full-data importance-weighted version that reuses rejected samples, whose bias-variance trade-off the current analysis does not cover.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes PACOPP, an algorithm for constructing probably approximately correct (PAC) prediction intervals for the reward of a target policy in offline contextual bandits, where data are collected under a possibly different behavior policy. The method applies rejection sampling to the offline data with weights equal to the target-to-behavior policy ratio, then uses a split-conformal construction whose threshold is a binomial quantile determined by the prescribed (ε, δ). Theorems 1--3 give finite-sample PAC validity and confidence/coverage-rate bounds, Theorem 5 claims asymptotic equivalence with the oracle quantile interval, and Theorems 6--7 extend the construction to an estimated behavior policy.

Significance. If the guarantees hold as stated, the paper makes a useful contribution to off-policy uncertainty quantification: it provides finite-sample training-conditional validity, rather than only marginal coverage, and the threshold is a binomial quantile rather than a data-fit constant, so the main PAC statement is not circular. The self-contained proof of Theorem 1 is a genuine strength, as is the explicit treatment of the unknown-behavior-policy case and the released simulation code. The principal limitation is that the whole construction is conditional on B = sup πe/πb < ∞, which is a substantive restriction on the policy pair; this limits the scope of the abstract's headline claims but does not, by itself, invalidate the main theorem.

major comments (4)
  1. [Abstract; §3.1] The abstract's claim that the method works 'without relying on any distributional or space assumptions' is stronger than what is actually proven. Algorithm 1 and Proposition 1 require B = sup_{(s,a)} πe(a|s)/πb(a|s) < ∞ (Section 3.1), and Algorithm 1's step 7 computes this supremum explicitly. The paper is transparent about this condition in Section 3.1, but the headline claims in the abstract and introduction should be qualified to state that the PAC guarantee requires the target policy to be dominated by the behavior policy in the likelihood-ratio sense.
  2. [Theorem 6; Appendix A.6, Eq. (14)] The second assertion of Theorem 6 is presented as a theorem, with the text saying 'a proof deferred to A.6', but Appendix A.6 ends with 'follows similarly and is omitted here.' Since this bound is part of the claimed counterpart to Theorems 1--3, it is not acceptable to leave it unproved in a formal theorem. The authors should either supply the full proof or demote the statement to a conjecture/remark with the missing argument sketched.
  3. [Appendix A.3, proof of Theorem 2, Eq. (18)] Equation (18) writes F_{m,ε}(√m y_m), but y_m is already standardized by √m, so the argument appears to be a typo for F_{m,ε}(y_m). In addition, the final bound in the proof contains a term C1 B/√γ that is not of order 1/√n, so the constant in Theorem 2 as stated does not match the proof's decomposition. Please correct the typo and reconcile the constants so that the stated C/√n rate is what the proof actually delivers.
  4. [Corollary 4] The second inequality, P[ε < LP ≤ ε + Δn] > δ − C/(√n Δn), does not follow from Theorems 2 and 3 as written. Theorem 2 only bounds P[LP ≤ ε] from above, which gives a lower bound on P[LP > ε], and no upper bound on P[LP > ε + Δn] is established in the displayed theorems or in the surrounding text. The corollary therefore needs a proof or a precise reference to a bound that supplies the missing tail control.
minor comments (4)
  1. [Algorithm 1, function Pac-Cp] The pseudo-code does not define the (|D| + 1)-th smallest value when k = −1; the text defines τ_{M+1} = ∞, and the function should explicitly return ∞ in that case.
  2. [Section 4.3 and Figure 2 caption] The caption lists δ = 0.15 while the text reports PAC-0.25 for the same configuration; the reported simulation settings should be made consistent.
  3. [Throughout] There are several typos and infelicities, including 'aotonomous' (Introduction), 'dut to' (Section 3), 'dismatch' (Section 3.1), and 'In a mathematical view of point' (Introduction); the manuscript should be proofread.
  4. [Section 4.1] The statement that COPP-RS achieves marginal coverage 'under the conditions in Theorem 1' should specify the exact weighted-quantile construction and the conditions on the rejection sampling step, since the threshold is the 1−ε empirical quantile with the (M+1)-point distribution.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; the PAC guarantee is a self-contained binomial calibration argument.

full rationale

The derivation chain is self-contained and does not reduce any claimed prediction to a fitted input or self-citation. Proposition 1, the rejection-sampling step, is proved in Appendix A.1 (and cited from Owen as a standard tool); its conclusion that accepted samples are i.i.d. from the target marginal distribution is derived from the acceptance probability, not assumed as the PAC result. Theorem 1 is then proved directly: the threshold k(M, epsilon, delta) is the delta-quantile of Bin(M, epsilon), a quantity fixed by the user's (epsilon, delta) rather than fitted to data, and the proof shows P[LP_pi_e(C_tau_tilde) > epsilon] <= delta using monotonicity of the binomial CDF. No empirical parameter is renamed as a prediction. Theorems 2 and 3 use concentration inequalities, and Theorem 5 uses explicit consistency and regularity assumptions, all independent of the PAC conclusion being derived. The unknown-policy part (Theorem 6) applies the same internal Theorem 1 conditionally, which is legitimate chaining rather than circularity. Two genuine limitations are present but are not circular: the proof of inequality (14) is stated as omitted in Appendix A.6, and the assumption B < infinity is load-bearing for the rejection-sampling step and is not reflected in the abstract's sweeping phrasing. These are completeness and scope concerns, not cases where an output equals an input by construction.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

The central claim rests on a bounded likelihood ratio, i.i.d. data, consistency of quantile estimators for the asymptotic result, and a finite policy class for the MLE guarantee. No new physical or mathematical entities are introduced, and no constants are fitted to data.

assumptions (5)
  • domain assumption Observational data D are i.i.d. from PS, πb, PR, and the test point is independent of D.
    Section 2 problem formulation; the entire OPE setup. If data are dependent or the test point shares information with D, the exchangeability arguments in Proposition 1 and Theorem 1 fail.
  • domain assumption The density ratio πe/πb is bounded above by a finite constant B.
    Section 3.1: 'The paper proceeds silently under the assumption that B < ∞'. Rejection sampling acceptance probability (1/B)w is otherwise not a valid probability, so the algorithm cannot be run as stated and Proposition 1 fails.
  • domain assumption Quantile estimators are consistent and the score T has density bounded away from zero near 0 (Assumptions 1 and 2 in Appendix A.5).
    Needed for Theorem 5's asymptotic equivalence to the oracle interval. These are standard but not free; they fail for discrete rewards or badly estimated quantiles.
  • domain assumption For unknown behavior policy, πb belongs to a finite policy class Π and the MLE is used (Theorem 7).
    Appendix A.7; enables the explicit Δw bound. For general nonparametric estimators no such guarantee is given.
  • standard math No ties among calibration scores almost surely (Theorems 2, 3, and 6).
    Used to replace strict inequalities and order statistics; holds for continuous rewards and non-atomic score distributions, but may fail with estimated quantiles and discrete rewards.

how reviews work

0 comments
Cite this review

Pith. "Pith review of PAC Off-Policy Prediction of Contextual Bandits." pith.science (2026). https://pith.science/paper/3KIHCNWQ

@misc{pith2026250716236,
  author       = {Pith},
  title        = {Pith review of: PAC Off-Policy Prediction of Contextual Bandits},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3KIHCNWQ}},
  note         = {Machine review of arXiv:2507.16236}
}
read the original abstract

This paper investigates off-policy evaluation in contextual bandits, aiming to quantify the performance of a target policy using data collected under a different and potentially unknown behavior policy. Recently, methods based on conformal prediction have been developed to construct reliable prediction intervals that guarantee marginal coverage in finite samples, making them particularly suited for safety-critical applications. To further achieve coverage conditional on a given offline data set, we propose a novel algorithm that constructs probably approximately correct prediction intervals. Our method builds upon a PAC-valid conformal prediction framework, and we strengthen its theoretical guarantees by establishing PAC-type bounds on coverage. We analyze both finite-sample and asymptotic properties of the proposed method, and compare its empirical performance with existing methods in simulations.

Figures

Figures reproduced from arXiv: 2507.16236 by the authors.

Figure 1
Figure 1. Empirical P[ϵ − ∆ϵ < LP πe (Cˆ τ˜) ≤ ϵ]. Annotation: The simulation was conducted under ϵ = 0.2 and δ = 0.1. The left is for varying sample sizes n with fixed ∆ϵ = 0.05 and the right for varying ∆ϵ with fixed n = 2000. For each n, 10,000 test points were generated to evaluate LP πe (Cˆ τ˜), and 10,000 simulation runs were conducted to estimate the empirical probability. Corollary 4 shows that Cˆ τ˜ achieves exact 1 … view at source ↗
Figure 2
Figure 2. Empirical coverages and average lengths of prediction intervals based on COPP, [PITH_FULL_IMAGE:figures/full_fig_p013_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

41 extracted references · 25 canonical work pages

  1. [1]

    Flambe: Structural complexity and representation learning of low rank mdps

    Alekh Agarwal, Sham Kakade, Akshay Krishnamurthy, and Wen Sun. Flambe: Structural complexity and representation learning of low rank mdps. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 20095--20107. Curran Associates, Inc., 2020. URL https://proceedings.neuri...

    work page 2020

  2. [2]

    Angelopoulos, Rina Foygel Barber, and Stephen Bates

    Anastasios N. Angelopoulos, Rina Foygel Barber, and Stephen Bates. Theoretical foundations of conformal prediction, 2025. URL https://arxiv.org/abs/2411.11824

    arXiv 2025

  3. [3]

    Conformal Prediction for Reliable Machine Learning: Theory, Adaptations and Applications

    Vineeth Balasubramanian, Shen-Shyang Ho, and Vladimir Vovk. Conformal Prediction for Reliable Machine Learning: Theory, Adaptations and Applications. Newnes, 2014

    work page 2014

  4. [4]

    Yash Chandak, Scott Niekum, Bruno da Silva, Erik Learned-Miller, Emma Brunskill, and Philip S. Thomas. Universal off-policy evaluation. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems, volume 34, pages 27475--27490. Curran Associates, Inc., 2021. URL https://proceedi...

    2021

  5. [5]

    Coindice: Off-policy confidence interval estimation

    Bo Dai, Ofir Nachum, Yinlam Chow, Lihong Li, Csaba Szepesvari, and Dale Schuurmans. Coindice: Off-policy confidence interval estimation. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 9398--9411. Curran Associates, Inc., 2020. URL https://proceedings.neurips.c...

    work page 2020

  6. [6]

    Doubly robust policy evaluation and learning

    Miroslav Dud\' k, John Langford, and Lihong Li. Doubly robust policy evaluation and learning. In Proceedings of the 28th International Conference on International Conference on Machine Learning, ICML'11, page 1097–1104, Madison, WI, USA, 2011. Omnipress. ISBN 9781450306195

    2011

  7. [7]

    Probability: Theory and Examples

    Rick Durrett. Probability: Theory and Examples. Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge University Press, 5 edition, 2019

    work page 2019

  8. [8]

    Accountable off-policy evaluation with kernel B ellman statistics

    Yihao Feng, Tongzheng Ren, Ziyang Tang, and Qiang Liu. Accountable off-policy evaluation with kernel B ellman statistics. In Hal Daumé III and Aarti Singh, editors, Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pages 3102--3111. PMLR, 13--18 Jul 2020. URL https://proceedings.m...

    work page 2020

Show all 41 references
  1. [9]

    Conformal off-policy evaluation in markov decision processes

    Daniele Foffano, Alessio Russo, and Alexandre Proutiere. Conformal off-policy evaluation in markov decision processes. In 2023 62nd IEEE Conference on Decision and Control (CDC), pages 3087--3094, 2023. doi:10.1109/CDC49753.2023.10383469

    2023

  2. [10]

    The limits of distribution-free conditional predictive inference

    Rina Foygel Barber, Emmanuel J Candès, Aaditya Ramdas, and Ryan J Tibshirani. The limits of distribution-free conditional predictive inference. Information and Inference: A Journal of the IMA, 10 0 (2): 0 455--482, 08 2020. ISSN 2049-8772. doi:10.1093/imaiai/iaaa017. URL https...

    2020 doi

  3. [11]

    Bootstrapping with models: Confidence intervals for off-policy evaluation

    Josiah Hanna, Peter Stone, and Scott Niekum. Bootstrapping with models: Confidence intervals for off-policy evaluation. Proceedings of the AAAI Conference on Artificial Intelligence, 31 0 (1), Feb. 2017. doi:10.1609/aaai.v31i1.11123. URL https://ojs.aaai.org/index.php/AAAI/art...

    2017 doi

  4. [12]

    Off-policy risk assessment in contextual bandits

    Audrey Huang, Liu Leqi, Zachary Lipton, and Kamyar Azizzadenesheli. Off-policy risk assessment in contextual bandits. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems, volume 34, pages 237...

    2021

  5. [13]

    Doubly robust off-policy value evaluation for reinforcement learning

    Nan Jiang and Lihong Li. Doubly robust off-policy value evaluation for reinforcement learning. In Maria Florina Balcan and Kilian Q. Weinberger, editors, Proceedings of The 33rd International Conference on Machine Learning, volume 48 of Proceedings of Machine Learning Research...

    2016

  6. [14]

    Double reinforcement learning for efficient off-policy evaluation in markov decision processes

    Nathan Kallus and Masatoshi Uehara. Double reinforcement learning for efficient off-policy evaluation in markov decision processes. Journal of Machine Learning Research, 21 0 (167): 0 1--63, 2020. URL http://jmlr.org/papers/v21/19-827.html

    2020

  7. [15]

    Being optimistic to be conservative: Quickly learning a cvar policy

    Ramtin Keramati, Christoph Dann, Alex Tamkin, and Emma Brunskill. Being optimistic to be conservative: Quickly learning a cvar policy. Proceedings of the AAAI Conference on Artificial Intelligence, 34 0 (04): 0 4436--4443, Apr. 2020. doi:10.1609/aaai.v34i04.5870. URL https://o...

    2020 doi

  8. [16]

    Regression quantiles

    Roger Koenker and Gilbert Bassett Jr. Regression quantiles. Econometrica, 46 0 (1): 0 33--50, 1978. URL https://onlinelibrary.wiley.com/doi/abs/0012-9682(197801)46:1&lt;33:RQ&gt;2.0.CO;2-J

    1978

  9. [17]

    Batch policy learning under constraints

    Hoang Le, Cameron Voloshin, and Yisong Yue. Batch policy learning under constraints. In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 3703...

    2019

  10. [18]

    Tibshirani, and Larry Wasserman and

    Jing Lei, Max G’Sell, Alessandro Rinaldo, Ryan J. Tibshirani, and Larry Wasserman and. Distribution-free predictive inference for regression. Journal of the American Statistical Association, 113 0 (523): 0 1094--1111, 2018. doi:10.1080/01621459.2017.1307116. URL https://doi.or...

    2018

  11. [19]

    Lihua Lei and Emmanuel J. Candès. Conformal inference of counterfactuals and individual treatment effects. Journal of the Royal Statistical Society Series B: Statistical Methodology, 83 0 (5): 0 911--938, 10 2021. ISSN 1369-7412. doi:10.1111/rssb.12445. URL https://doi.org/10....

    2021 doi

  12. [20]

    Breaking the curse of horizon: Infinite-horizon off-policy estimation

    Qiang Liu, Lihong Li, Ziyang Tang, and Dengyong Zhou. Breaking the curse of horizon: Infinite-horizon off-policy estimation. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume...

    2018

  13. [21]

    Quantile regression forests

    Nicolai Meinshausen. Quantile regression forests. Journal of Machine Learning Research, 7 0 (35): 0 983--999, 2006. URL http://jmlr.org/papers/v7/meinshausen06a.html

    2006

  14. [22]

    Art B. Owen. Monte Carlo Theory, Methods and Examples. 2013

    2013

  15. [23]

    Pac confidence sets for deep neural networks via calibrated prediction

    Sangdon Park, Osbert Bastani, Nikolai Matni, and Insup Lee. Pac confidence sets for deep neural networks via calibrated prediction. In International Conference on Learning Representations, 2020. URL https://openreview.net/forum?id=BJxVI04YvB

    2020

  16. [24]

    PAC prediction sets under covariate shift

    Sangdon Park, Edgar Dobriban, Insup Lee, and Osbert Bastani. PAC prediction sets under covariate shift. In International Conference on Learning Representations, 2022. URL https://openreview.net/forum?id=DhP9L8vIyLc

    2022

  17. [25]

    Eligibility traces for off-policy policy evaluation

    Doina Precup, Richard S Sutton, and Satinder Singh. Eligibility traces for off-policy policy evaluation. In ICML, volume 2000, pages 759--766. Citeseer, 2000

    2000

  18. [26]

    Conformalized quantile regression

    Yaniv Romano, Evan Patterson, and Emmanuel Candes. Conformalized quantile regression. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d Alch\' e -Buc, E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019....

    2019

  19. [27]

    Importance resampling for off-policy prediction

    Matthew Schlegel, Wesley Chung, Daniel Graves, Jian Qian, and Martha White. Importance resampling for off-policy prediction. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d Alch\' e -Buc, E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems, vo...

    2019

  20. [28]

    Matteo Sesia and Emmanuel J. Candès. A comparison of some conformal quantile regression methods. Stat, 9 0 (1): 0 e261, 2020. doi:https://doi.org/10.1002/sta4.261. URL https://onlinelibrary.wiley.com/doi/abs/10.1002/sta4.261. e261 sta4.261

    2020 doi

  21. [29]

    A tutorial on conformal prediction

    Glenn Shafer and Vladimir Vovk. A tutorial on conformal prediction. Journal of Machine Learning Research, 9 0 (12): 0 371--421, 2008. URL http://jmlr.org/papers/v9/shafer08a.html

    2008

  22. [30]

    Statistical inference of the value function for reinforcement learning in infinite-horizon settings

    Chengchun Shi, Sheng Zhang, Wenbin Lu, and Rui Song. Statistical inference of the value function for reinforcement learning in infinite-horizon settings. Journal of the Royal Statistical Society Series B: Statistical Methodology, 84 0 (3): 0 765--793, 12 2021. ISSN 1369-7412. ...

    2021 doi

  23. [31]

    Sutton and Andrew G

    Richard S. Sutton and Andrew G. Barto. Reinforcement Learning: An Introduction. MIT Press, 2nd edition, 2018

    2018

  24. [32]

    Conformal off-policy prediction in contextual bandits

    Muhammad Faaiz Taufiq, Jean-Francois Ton, Rob Cornish, Yee Whye Teh, and Arnaud Doucet. Conformal off-policy prediction in contextual bandits. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems, volu...

    2022

  25. [33]

    James W. Taylor. A quantile regression neural network approach to estimating the conditional density of multiperiod returns. Journal of Forecasting, 19 0 (4): 0 299--311, 2000. doi:https://doi.org/10.1002/1099-131X(200007)19:4<299::AID-FOR775>3.0.CO;2-V. URL https://onlinelibr...

    doi

  26. [34]

    Data-efficient off-policy policy evaluation for reinforcement learning

    Philip Thomas and Emma Brunskill. Data-efficient off-policy policy evaluation for reinforcement learning. In Maria Florina Balcan and Kilian Q. Weinberger, editors, Proceedings of The 33rd International Conference on Machine Learning, volume 48 of Proceedings of Machine Learni...

    2016

  27. [35]

    High-confidence off-policy evaluation

    Philip Thomas, Georgios Theocharous, and Mohammad Ghavamzadeh. High-confidence off-policy evaluation. Proceedings of the AAAI Conference on Artificial Intelligence, 29 0 (1), Feb. 2015. doi:10.1609/aaai.v29i1.9541. URL https://ojs.aaai.org/index.php/AAAI/article/view/9541

    2015 doi

  28. [36]

    Conformal prediction under covariate shift

    Ryan J Tibshirani, Rina Foygel Barber, Emmanuel Candes, and Aaditya Ramdas. Conformal prediction under covariate shift. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d Alch\' e -Buc, E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume ...

    2019

  29. [37]

    A review of off-policy evaluation in reinforcement learning, 2022

    Masatoshi Uehara, Chengchun Shi, and Nathan Kallus. A review of off-policy evaluation in reinforcement learning, 2022. URL https://arxiv.org/abs/2212.06355

    2022 arXiv

  30. [38]

    Various techniques used in connection with random digits

    John von Neumann. Various techniques used in connection with random digits. In A. S. Householder, G. E. Forsythe, and H. H. Germond, editors, Monte Carlo Method, volume 12 of National Bureau of Standards Applied Mathematics Series, chapter 13, pages 36--38. US Government Print...

    1951

  31. [39]

    Conditional validity of inductive conformal predictors

    Vladimir Vovk. Conditional validity of inductive conformal predictors. Machine Learning, 92 0 (2): 0 349--376, Sep 2013. ISSN 1573-0565. doi:10.1007/s10994-013-5355-6. URL https://doi.org/10.1007/s10994-013-5355-6

    2013 doi

  32. [40]

    Algorithmic Learning in a Random World

    Vladimir Vovk, Alexander Gammerman, and Glenn Shafer. Algorithmic Learning in a Random World. Springer, 2005

    2005

  33. [41]

    Conformal off-policy prediction

    Yingying Zhang, Chengchun Shi, and Shikai Luo. Conformal off-policy prediction. In Francisco Ruiz, Jennifer Dy, and Jan-Willem van de Meent, editors, Proceedings of The 26th International Conference on Artificial Intelligence and Statistics, volume 206 of Proceedings of Machin...

    2023

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.