Pith. sign in

REVIEW 2 major objections 5 minor 21 references

Just Trial Once: Ongoing Causal Validation of Machine Learning Models

T0 review · 2 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Given data from a single prior randomized trial, the causal impact of deploying any new ML model—including models never trialed—can be tightly bounded under three stated assumptions, and the bounds cannot be improved without further…

desk verdict A clean partial-identification result for evaluating untrialed ML models from prior RCT data, with a load-bearing monotonicity assumption that its falsification test does not actually secure. read the letter →

arxiv 2502.09467 v2 pith:RDME5J3K submitted 2025-02-13 stat.ME

Jacob M. Chen , Michael Oberst This is my paper · ORCID
classification stat.ME MSC 62D20
keywords causalinferencepolicyvalueboundsrandomizedcontrolledtrialmodelupdatesdecisionsupportoff-policyevaluationperformancemonotonicitytight
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks whether a single randomized trial of one or a few ML decision-support models can also tell you the causal effect of deploying a different, updated model that was never in the trial. Its answer is yes, up to bounds: under three assumptions—outcomes are monotone in model performance, there is a neutral output whose effect does not depend on performance, and outcomes are bounded—the policy value of any new model is sandwiched between a lower bound using the worst agreeing trialed model and an upper bound using the best agreeing trialed model, with $Y_{\min}$ and $Y_{\max}$ where no trialed model agrees. The bounds are provably tight, so no additional assumption-free tightening is possible, and simple weighted-mean estimators give asymptotically valid confidence intervals. The practical point is that expensive repeated trials may be unnecessary for validating model updates, and that trial designs with multiple models are worth running because they make future updates evaluable.

What carries the argument

The load-bearing object is the family of agreement sets built from the trial's models: for each $x$, $\Pi^e(x)$ collects trialed policies whose output matches $\pi_e(x)$, and $\Pi^e_{\le}(x)$, $\Pi^e_{\ge}(x)$ restrict to those with performance at most or at least that of $\pi_e$; the extremal sets $\widetilde{\Pi}^e_{\le}(x)$ and $\widetilde{\Pi}^e_{\ge}(x)$ pick the next-worst and next-best performing agreeing models. The bounds replace the unobserved outcome under $\pi_e$ with observed outcomes under these extremal agreeing models, relying on Performance Monotonicity (Assumption 3.1) to order outcomes by model performance, Neutral Actions (Assumption 3.2) to reuse control-arm data, and Bounded Outcomes (Assumption 3.3) to cap the residual uncertainty where no trialed model agrees.

What would settle it

Run the paper's Proposition 3.1 test on actual trial data: identify a subpopulation where two trialed models with different measured performance produce the same output, and compare mean outcomes; if the higher-performing model's mean outcome is significantly lower, Assumption 3.1 is false. The analogous Proposition 3.2 test, comparing outcomes where both models output the neutral action, would falsify Assumption 3.2 if the means differ.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is Theorem 3.1: under the structural model of Assumption 2.1 and Assumptions 3.1 to 3.3, the expected outcome under a new policy $\pi_e$ satisfies $L(\pi_e) \le \mathbb{E}[Y(A=\pi_e, M=f_M(\pi_e))] \le U(\pi_e)$, where for each covariate value $x$ the lower bound uses outcome data from trialed models that agree with $\pi_e(x)$ and have next-worst performance (or $Y_{\min}$ if none exists), and the upper bound uses next-best performance (or $Y_{\max}$). Theorem 3.2 shows these bounds are tight in that two structural causal models matching the same observed data can realize $L$ and $U$ respectively, so no sharper bounds follow from the assumptions alone. Proposition 3.4 turns the bounds into inverse-probability-weighted sample means with normal confidence intervals, and Propositions 3.1 and 3.2 give observable checks that can falsify the two behavioral assumptions from the trial data itself.

Load-bearing premise

If a better-performing model ever makes clinicians less attentive to its outputs, so that for a fixed output a higher-performance model leads to worse outcomes than a lower-performance one, the monotonicity assumption fails and the direction of the bounds can flip.

Editorial extensions

If this is right

  • A practitioner with one completed multi-arm cluster RCT can report an interval for the causal effect of any proposed model update without running a new trial, as long as the three assumptions hold.
  • Trial designs that randomize several models with genuinely different outputs and a spread of performance levels produce narrower bounds for future updates, because more $x$-values have agreeing next-worst and next-best trialed models.
  • Exact point identification occurs only in the special case where the agreement and performance structure makes the bound gap zero, such as a trialed model with equal performance on every case, or only neutral outputs; otherwise the honest answer is an interval.
  • Choosing updates by raw accuracy can be wrong: the simulation shows a less accurate model can have a higher lower bound on patient outcomes than a more accurate rival, so outcome-based bounds, not performance metrics, should drive deployment decisions.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the same bounds were applied to observational deployment data rather than RCT data, the independence of policy assignment and covariates would fail, so one would need propensity-style adjustment and the falsification tests would lose their clean interpretation; the RCT requirement is doing real work.
  • The bounds are pessimistic by design on never-agreeing regions, and a natural testable extension is to replace $Y_{\min}$ and $Y_{\max}$ with domain-specific constraints such as “alerts are never harmful when correct,” which would tighten the intervals without contradicting the tightness result.
  • Falsification via Proposition 3.1 only inspects agreement regions, so performance monotonicity remains untested precisely on the covariate regions where no trialed model matches $\pi_e$; trial designers who want to validate future models should therefore maximize overlap between candidate models' action regions.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. This paper proposes a method for evaluating the causal impact of deploying a new machine-learning model πe using only data from a prior randomized controlled trial that trialed other models. The authors formalize a structural causal model in which model outputs are deterministic functions of covariates, model performance M is a scalar property of the deployed model, and outcomes depend on both the output and M. Under assumptions of performance monotonicity (Assumption 3.1), existence of a neutral action whose outcome is invariant to M (Assumption 3.2), and bounded outcomes (Assumption 3.3), they derive upper and lower bounds on the policy value (Theorem 3.1), prove these bounds are tight (Theorem 3.2), provide an estimator with asymptotically valid confidence intervals (Proposition 3.4), and propose falsification tests for the first two assumptions (Propositions 3.1 and 3.2). A simulation study illustrates that the most accurate model need not have the largest causal impact.

Significance. If the results are correct, this is a useful contribution to the emerging literature on evaluating ML decision-support systems after deployment. The formal bounds are derived cleanly, and the tightness construction in Theorem 3.2 is a notable strength: it shows precisely where unobserved (x,a,m) combinations force the use of Ymin and Ymax. The paper also ships reproducible code for the simulation and gives concrete trial-design recommendations. The main caveat is that the practical applicability hinges on Assumption 3.1 (performance monotonicity), and the proposed falsification tests provide only weak, aggregate necessary conditions for it; this limits the strength of the claim that the assumptions 'can be checked' from the RCT data.

major comments (2)
  1. [Section 1, Related literature] In the Related literature paragraph, the paper states that 'we allow for the fact that actions taken for one patient can influence outcomes for other patients.' This is not supported by the model. In Assumption 2.1 the outcome is generated by Y=f_Y(A,X,M,ϵ_Y), and the DAG in Fig. 2 contains no edges between outcomes or actions of different patients; D enters only through Π. All results in Section 3 therefore assume no interference (SUTVA). The sentence should be removed or the model and proofs should be extended to include interference; as written, the scope claim contradicts the formal setup.
  2. [Assumption 3.1 / Proposition 3.1] Assumption 3.1 is the central ordering assumption used in Lemma D.2 (inequalities (22)–(23)), and therefore in every step of Theorem 3.1. The falsification test in Proposition 3.1 only compares the aggregate means E[Y|X∈X_agree,Π=π2] and E[Y|X∈X_agree,Π=π1]. This is a necessary condition: if the test rejects, Assumption 3.1 is false. However, a violation of monotonicity on one subset of X_agree can be offset by opposite differences on other subsets, so the test can pass even when the pointwise inequality required by Lemma D.2 fails. Moreover, in the regions described by equations (10)–(11) of Proposition 3.3, there is no agreeing model on one side of the performance ordering, so no falsification data exist at all, yet those regions enter the bounds through Ymin or Ymax. The authors should either develop stronger (e.g., conditional-on-subset) tests or revise the claim in the introduction that 'both of our main assumptions can be checked using RCT data'; as stated, the testable implications do not secure the key assumption on the regions where the bounds are most fragile.
minor comments (5)
  1. [Appendix D, proof of Theorem 3.2] In the proof after Eq. (58), the statement that f_L(x,a,m_e)=F^{-1}_{Y|x,a,hprev(m_e)} for all x with \tildeΠ^e_≤(X) non-empty is incomplete: when m_e belongs to the observed performance set M_a(x), the fourth line of Eq. (56) applies instead of the hprev case. The final equality (62) still holds, but the proof should split into the cases m_e∈M_a(x) and m_e∉M_a(x).
  2. [Section 1, Related literature] There is a typo: 'related tooff-policy policy evaluation' should read 'related to off-policy policy evaluation.'
  3. [Theorem 3.1 and Proposition 3.3] The notation '1{π_e ≠ a0}' uses the policy π_e as shorthand for π_e(X); although this is acknowledged in footnote 3, the displayed formulas would be clearer if the covariate argument were written explicitly as 1{π_e(X) ≠ a0}.
  4. [Proposition 3.4] The piecewise definitions of ψ_L and ψ_U mix indicator notation in the first and third cases with bare Ymin/Ymax in the second and fourth cases; making all cases uniform would improve readability.
  5. [Section 5, simulation study] The claim that the lower bound for patient outcomes under πe0 exceeds outcomes under all trialed models is visible in Fig. 5, but the text would benefit from reporting the estimated bound values and confidence intervals explicitly.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the bounds follow from explicit assumptions, and sharpness is established by constructing competing SCMs that match the observed distribution.

full rationale

The paper's central claim, Theorem 3.1, is a direct logical consequence of the stated SCM (Assumption 2.1) and the assumptions of performance monotonicity, neutral actions, and bounded outcomes (Assumptions 3.1-3.3). The lower and upper bounds are constructed by replacing unobserved conditional means either with those of next-worst or next-best agreeing trialed models (via Lemma D.2) or with Ymin/Ymax (via Lemma D.3); no quantity is fitted to the target estimand, and the target estimand does not appear in the definitions of the bound formulas. Theorem 3.2 establishes tightness by constructing two structural causal models that agree with the observed distribution and attain the bounds through unobserved counterfactual values; this is the standard sharpness construction and does not import the conclusion into the assumptions. The falsification tests in Propositions 3.1 and 3.2 check necessary implications of the assumptions; they are acknowledged as partial checks, and their incompleteness is a correctness or validation limitation, not a circular dependency. The simulation is generated to satisfy the assumptions and therefore does not independently validate them, but the paper does not claim that the simulation validates the assumptions. There are no load-bearing self-citations: the technical citations are to standard causal-inference references (Pearl, Richardson and Robins, Malinsky et al., van der Vaart) and are used for definitions and asymptotic results, not to justify the paper's own conclusions. Overall, the derivation chain is self-contained with respect to its stated assumptions.

Assumptions & free parameters 1 free parameters · 6 assumptions · 0 invented entities

The central theorem rests on four domain assumptions (DAG, monotonicity, neutral action, boundedness) plus standard causal and mathematical background. No numerical parameters are fitted to data; the only hand-chosen numbers are simulation coefficients used for illustration. The monotonicity assumption is the most fragile and is only partially falsifiable.

free parameters (1)
  • Simulation outcome coefficients = e.g., 0.56, 0.48, 0.40 in Table 1 / Appendix C
    Hand-chosen parameters in Section 5 and Appendix C to create a demonstration where the most accurate model is not the best for patient outcomes. They are not used in the theorems and do not affect the central claim.
assumptions (6)
  • domain assumption Assumption 2.1: Data generating process with SCM Y=f_Y(A,X,M,ε_Y), Π=π_D, A=Π(X), M=f_M(Π), with ε_D, ε_X, ε_Y mutually independent
    This DAG (Fig. 2) rules out unmeasured confounders and asserts that model impact is fully mediated by output A and scalar performance M. Used throughout, e.g., Eq. (34) and consistency/ignorability.
  • domain assumption Assumption 3.1: Performance monotonicity, for mi<mj, Y(A=a,M=mi) <= Y(A=a,M=mj) for all a
    Load-bearing for using lower-performing agreeing models as lower bounds and higher-performing ones as upper bounds. The paper itself notes violations are possible when trust in low-risk outputs erodes.
  • domain assumption Assumption 3.2: Existence of a neutral action a0 whose potential outcome does not depend on M
    Needed to leverage control-arm data and to identify outcomes when πe outputs a0. Proposition 3.2 provides a falsification test but only a necessary condition.
  • domain assumption Assumption 3.3: Bounded outcomes, Ymin <= Y <= Ymax
    Provides fallback bounds when no trialed model agrees with πe (Eqs. (5) and (7)); without it the bounds would be infinite.
  • standard math Consistency and conditional ignorability: Y(a,m) ⟂ Π | X, and observed Y equals the potential outcome under observed (A,M)
    Follows from Assumption 2.1; used in proofs of Propositions 3.1, 3.2, and Theorem 3.1 (Appendix D).
  • standard math Central limit theorem for i.i.d. sample means
    Used in Proposition 3.4 to construct asymptotically valid confidence intervals.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Just Trial Once: Ongoing Causal Validation of Machine Learning Models." pith.science (2026). https://pith.science/paper/RDME5J3K

@misc{pith2026250209467,
  author       = {Pith},
  title        = {Pith review of: Just Trial Once: Ongoing Causal Validation of Machine Learning Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RDME5J3K}},
  note         = {Machine review of arXiv:2502.09467}
}
read the original abstract

Machine learning (ML) models are increasingly used as decision-support tools in high-risk domains. Evaluating the causal impact of deploying such models can be done with a randomized controlled trial (RCT) that randomizes users to ML vs. control groups and assesses the effect on relevant outcomes. However, ML models are inevitably updated over time, and we often lack evidence for the causal impact of these updates. While the causal effect could be repeatedly validated with ongoing RCTs, such experiments are expensive and time-consuming to run. In this work, we present an alternative solution: using only data from a prior RCT, we give conditions under which the causal impact of a new ML model can be precisely bounded or estimated, even if it was not included in the RCT. Our assumptions incorporate two realistic constraints: ML predictions are often deterministic, and their impacts depend on user trust in the model. Based on our analysis, we give recommendations for trial designs that maximize our ability to assess future versions of an ML model. Our hope is that our trial design recommendations will save practitioners time and resources while allowing for quicker deployments of updates to ML models.

Figures

Figures reproduced from arXiv: 2502.09467 by the authors.

Figure 1
Figure 1. (a) The goal of this paper is to predict the causal impact of deploying a new model πe, given data from a cluster randomized trial that randomizes sets of users (e.g., hospitals) to one of a set of trial models that does not include πe. (b) The first challenge: Relevant outcomes (e.g., of patients) are not only influenced by model outputs, but also by how users actually respond to the outputs of model-based decision… view at source ↗
Figure 2
Figure 2. The directed acyclic graph (DAG) G depicting the causal relationships in our problem setup (Assumption 2.1). We draw circles around the nodes Π, A, and M to represent that these variables are deterministic given their parents (the nodes that have a direct edge to them). decision-making in criminal justice settings [Imai et al., 2023, Ben-Michael et al., 2024, 2025], but our goal differs: Rather than evaluating the i… view at source ↗
Figure 3
Figure 3. Illustration from Example 1, demonstrating the challenge of our task. We can use data from both the control arm, that does not raise alerts, and the treatment arm, that raises alerts for patients with risk scores greater than T ∗ , to infer patient outcomes when πl does not raise alerts for patients with risk scores less than T l . Next, we can use data from π1 to infer patient outcomes when πl raises alerts for pat… view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Bar graph giving a visual representation of the [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Bar graph showing accuracy at predicting disease [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: SWIG showing G(a, m). In this section, we give a brief overview of additional topics in causal inference that we use in our proofs and provide references for further reading on these topics. First, independencies implied in the full data distribution P(V) can be read o…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

21 extracted references · 15 canonical work pages

  1. [1]

    Prospective, multi-site study of patient outcomes after implementation of the TREWS machine learning-based early warning system for sepsis

    Roy Adams, Katharine E Henry, Anirudh Sridharan, Hossein Soleimani, Andong Zhan, Nishi Rawat, Lauren Johnson, David N Hager, Sara E Cosgrove, Andrew Markowski, Eili Y Klein, Edward S Chen, Mustapha O Saheed, Maureen Henley, Sheila Miranda, Katrina Houston, Robert C Linton, Anushree R Ahluwalia, Albert W Wu, and Suchi Saria. Prospective, multi-site study o...

    work page 2022

  2. [2]

    Does AI help humans make better decisions? A statistical evaluation framework for experimental and observational studies

    Eli Ben-Michael, D James Greiner, Melody Huang, Kosuke Imai, Zhichao Jiang, and Sooahn Shin. Does AI help humans make better decisions? A statistical evaluation framework for experimental and observational studies. arXiv, 2403: 0 v3, 2024

    work page 2024

  3. [3]

    Safe policy learning through extrapolation: Application to pre-trial risk assessment

    Eli Ben-Michael, D James Greiner, Kosuke Imai, and Zhichao Jiang. Safe policy learning through extrapolation: Application to pre-trial risk assessment. Journal of the American Statistical Association, pages 1--23, 2025

    work page 2025

  4. [4]

    Shashikumar, Atul Malhotra, Robert L

    Aaron Boussina, Supreeth P. Shashikumar, Atul Malhotra, Robert L. Owens, Robert El-Kareh, Christopher A. Longhurst, Kimberly Quintero, Allison Donahue, Theodore C. Chan, Shamim Nemati, and Gabriel Wardi. Impact of a deep learning sepsis prediction model on quality of care and survival. npj Digital Medicine, 7, 1 2024. doi:10.1038/s41746-023-00986-6. URL h...

    doi:10.1038/s41746-023-00986-6 2024

  5. [5]

    Artificial intelligence and machine learning ( AI / ML )-enabled medical devices

    FDA. Artificial intelligence and machine learning ( AI / ML )-enabled medical devices. U.S . Food and Drug Administration , 2024. URL https://www.fda.gov/medical-devices/software-medical-device-samd/artificial-intelligence-and-machine-learning-aiml-enabled-medical-devices. Accessed January 14th, 2024

    work page 2024

  6. [6]

    Gohil, Edward Septimus, Ken Kleinman, Neha Varma, Taliser R

    Shruti K. Gohil, Edward Septimus, Ken Kleinman, Neha Varma, Taliser R. Avery, Lauren Heim, Risa Rahm, William S. Cooper, Mandelin Cooper, Laura E. McLean, Naoise G. Nickolay, Robert A. Weinstein, L. Hayley Burgess, Micaela H. Coady, Edward Rosen, Selsebil Sljivo, Kenneth E. Sands, Julia Moody, Justin Vigeant, Syma Rashid, Rebecca F. Gilbert, Kim N. Smith,...

    work page arXiv 2018

  7. [7]

    Gohil, Edward Septimus, Ken Kleinman, Neha Varma, Taliser R

    Shruti K. Gohil, Edward Septimus, Ken Kleinman, Neha Varma, Taliser R. Avery, Lauren Heim, Risa Rahm, William S. Cooper, Mandelin Cooper, Laura E. McLean, Naoise G. Nickolay, Robert A. Weinstein, L. Hayley Burgess, Micaela H. Coady, Edward Rosen, Selsebil Sljivo, Kenneth E. Sands, Julia Moody, Justin Vigeant, Syma Rashid, Rebecca F. Gilbert, Kim N. Smith,...

    work page arXiv 2007

  8. [8]

    Randomised controlled trials evaluating artificial intelligence in clinical practice: A scoping review

    Ryan Han, Juli \'a n N Acosta, Zahra Shakeri, John P A Ioannidis, Eric J Topol, and Pranav Rajpurkar. Randomised controlled trials evaluating artificial intelligence in clinical practice: A scoping review. The Lancet Digital Health, 6: 0 e367--e373, 5 2024. doi:10.1016/s2589-7500(24)00047-5. URL http://dx.doi.org/10.1016/s2589-7500(24)00047-5

    doi:10.1016/s2589-7500(24)00047-5 2024

Show all 21 references
  1. [9]

    Experimental evaluation of algorithm-assisted human decision-making: A pplication to pretrial public safety assessment

    Kosuke Imai, Zhichao Jiang, D James Greiner, Ryan Halen, and Sooahn Shin. Experimental evaluation of algorithm-assisted human decision-making: A pplication to pretrial public safety assessment. Journal of the Royal Statistical Society Series A: Statistics in Society, 186: 0 16...

    2023 doi

  2. [10]

    AI as an intervention: improving clinical outcomes relies on a causal approach to AI development and validation

    Shalmali Joshi, I \ n igo Urteaga, Wouter AC van Amsterdam, George Hripcsak, Pierre Elias, Benjamin Recht, No \'e mie Elhadad, James Fackler, Mark P Sendak, Jenna Wiens, et al. AI as an intervention: improving clinical outcomes relies on a causal approach to AI development and...

    2025

  3. [11]

    A potential outcomes calculus for identifying conditional path-specific effects

    Daniel Malinsky, Ilya Shpitser, and Thomas Richardson. A potential outcomes calculus for identifying conditional path-specific effects. In The 22nd International Conference on Artificial Intelligence and Statistics, pages 3080--3088. PMLR, 2019

    2019

  4. [12]

    We need more randomized clinical trials of AI , 2024

    David Ouyang and Joseph Hogan. We need more randomized clinical trials of AI , 2024

    2024

  5. [13]

    Causality

    Judea Pearl. Causality. Cambridge university press, 2009

    2009

  6. [14]

    Probabilistic reasoning in intelligent systems: networks of plausible inference

    Judea Pearl. Probabilistic reasoning in intelligent systems: networks of plausible inference. Elsevier, 2014

    2014

  7. [15]

    Transportability of causal and statistical relations: A formal approach

    Judea Pearl and Elias Bareinboim. Transportability of causal and statistical relations: A formal approach. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 25, pages 247--254, 2011

    2011

  8. [16]

    Single world intervention graphs ( SWIG s): A unification of the counterfactual and graphical approaches to causality

    Thomas S Richardson and James M Robins. Single world intervention graphs ( SWIG s): A unification of the counterfactual and graphical approaches to causality. Center for the Statistics and the Social Sciences, University of Washington Series. Working Paper, 128 0 (30): 0 2013, 2013

    2013

  9. [17]

    Real-world integration of a sepsis deep learning technology into routine clinical care: Implementation study

    Mark P Sendak, William Ratliff, Dina Sarro, Elizabeth Alderton, Joseph Futoma, Michael Gao, Marshall Nichols, Mike Revoir, Faraz Yashar, Corinne Miller, Kelly Kester, Sahil Sandhu, Kristin Corey, Nathan Brajer, Christelle Tan, Anthony Lin, Tres Brown, Susan Engelbosch, Kevin A...

    2020

  10. [18]

    The use of propensity scores to assess the generalizability of results from randomized trials

    Elizabeth A Stuart, Stephen R Cole, Catherine P Bradshaw, and Philip J Leaf. The use of propensity scores to assess the generalizability of results from randomized trials. Journal of the Royal Statistical Society Series A: Statistics in Society, 174 0 (2): 0 369--386, 2011

    2011

  11. [19]

    A review of off-policy evaluation in reinforcement learning

    Masatoshi Uehara, Chengchun Shi, and Nathan Kallus. A review of off-policy evaluation in reinforcement learning. arXiv preprint arXiv:2212.06355, 2022

    2022 arXiv

  12. [20]

    Akerman, Thomas H

    Ross Upton, Ashley P. Akerman, Thomas H. Marwick, Casey L. Johnson, Hania Piotrowska, Mamta Bajre, Maria Breen, Helen Dawes, Hakim-Moulay Dehbi, Tine Descamps, Victoria Harris, Will Hawkes, Samuel Krasner, Emily Sanderson, Natalie Savage, Ben Thompson, Victoria Williamson, Wil...

    2024

  13. [21]

    A. W. van der Vaart. Asymptotic Statistics. Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge University Press, 1998

    1998

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.