Learning Treatment Allocations with Risk Control Under Partial Identifiability

Dave Zachariah; Sofia Ek

REVIEW 3 major objections 5 minor 51 references

Learning Treatment Allocations with Risk Control Under Partial Identifiability

T0 review · 3 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A policy-learning method certifies, with finite samples, that treatment risk stays below a chosen tolerance even when the risk is only partially identified.

desk verdict Promising combination of sensitivity weights and conformal risk control, but Theorem 4.3's proof has a load-bearing in-sample/out-of-sample gap and the RCT setting has an identification error. read the letter →

arxiv 2505.08378 v1 pith:QLW5U3Z2 submitted 2025-05-13 stat.ML cs.LG

Sofia Ek , Dave Zachariah This is my paper · ORCID

classification stat.MLcs.LG

keywords treatmentallocationriskcontrolpartialidentifiabilityfinite-sampleguaranteesmiscalibrationboundspolicylearningunmeasuredconfoundingrandomizedtrials

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes a way to learn treatment-allocation policies that keep the "treatment risk"—the proportion of treated patients who suffer a non-beneficial outcome rather than benefit—below a chosen tolerance, with a finite-sample probabilistic guarantee. The setting is deliberately hard: the risk need not be point-identifiable, because observational data may have unmeasured confounders and randomized trials may recruit a population different from the target. The method assumes only a user-specified bound on how miscalibrated the nominal treatment-assignment or trial-selection odds can be, and then certifies the policy against all degrees of miscalibration up to that bound. If the paper is right, a clinician can be told that, with confidence at least $1-\alpha$, no more than a fraction $\tau$ of treated patients will experience the non-beneficial outcome, even though the exact risk is unknown.

What carries the argument

The load-bearing object is the miscalibration weight $W^\Gamma$. It is built from the nominal assignment odds (observational case) or selection odds (trial case) together with the assumed miscalibration factor $\Gamma$, and Lemma 4.1 shows that multiplying the observed loss $L$ by $W^\Gamma$ gives an upper bound on the unidentifiable population and treatment risks. The method then learns a policy from one split of the data by minimizing an empirical estimate of the population-risk bound subject to a nominal treatment-risk constraint, and uses a second split to compute a finite-sample upper confidence bound on the treatment risk (for instance via the Bentkus bound), choosing the tightest tolerance $t_n$ that still satisfies $\tau>T^\alpha_n(t')$ for all smaller tolerances. The proof of Theorem 4.3 runs the confidence-bound event and the equality assumption for $\pi(X;\tau)$ together to show that violation of the target risk has probability at most $\alpha$.

What would settle it

Simulate a randomized trial in which an unobserved variable shifts both trial selection and outcome, so the trial covariate distribution differs from the target population. Run the method with $\Gamma=1$, evaluate the true treatment risk $T(\pi)$ on the target distribution over many datasets, and check whether fewer than $1-\alpha$ of the runs satisfy $T(\pi)\le\tau$; if not, the RCT certification in Theorem 4.3 fails.

Watch

Extended reading notes

Core claim

The central claim is Theorem 4.3: if the nominal policy $\pi(X;\tau)$ obtained from the empirical constrained problem (12) satisfies its treatment-risk constraint with equality, then the policy $\pi(X;t_n)$ selected by the empirical tolerance (14) satisfies $P(T(\pi)\le \tau \mid S=s)\ge 1-\alpha$ for any degree of miscalibration up to a prespecified $\Gamma$. Here $T(\pi)=P_\pi(L=1\mid A=1,S=0)$ is the treatment risk, and $S=s$ is the sampling condition (observational data for $s=0$ or randomized trial data for $s=1$). The argument converts an unidentifiable risk into an upper bound via importance weights $W^\Gamma$, then controls that upper bound in finite samples. The experiments illustrate the resulting trade-off between lowering population risk and keeping treatment risk under $\tau$, on simulated data and on the STAR and IST trials.

Load-bearing premise

The central load-bearing premise is that the nominal odds are miscalibrated by at most the chosen $\Gamma$ and that, in the trial case, the trial covariate distribution matches the target population; if either fails, the certification is unsupported.

Editorial extensions

If this is right

With finite sample sizes, the learned policy is certified to keep treatment risk below $\tau$ with probability at least $1-\alpha$, without assuming point identification.
The user trades off population benefit against non-maleficence: smaller $\tau$ treats fewer patients and raises population risk, as shown in the synthetic and STAR results.
The guarantee holds simultaneously for all miscalibration levels up to $\Gamma$, so a defensible $\Gamma$ (for example, benchmarked by omitting covariates) makes the policy robust to unmeasured confounding or selection.
The same procedure handles observational data, randomized trial data, and the mixed case where an observational study is conducted on a study population that differs from the target.
Applying the method to STAR and IST data yields simple fast-and-frugal decision-tree policies whose estimated treatment risk stays below $\tau$ across random splits.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Beyond the paper: for randomized trial data, the certification inherits the assumption that the trial covariate distribution matches the target population, because the policy's treatment-probability denominator $p_\pi(A=1\mid S=0)$ is identified from trial data only when $p(x\mid S=1)=p(x\mid S=0)$; if selection into the trial shifts the covariate mix, the claimed RCT guarantee is not supported by
Beyond the paper: the finite-sample guarantee is conditional on $\Gamma$ actually bounding the unknown odds; the paper benchmarks $\Gamma$ by omitting covariates, but if the true unmeasured factor has a larger effect than those benchmarks, the guarantee has no force.
Beyond the paper: the same split-sample risk-control logic could be applied subgroup-wise; the paper notes that aggregate control may be insufficient for fairness, so stratifying by sensitive covariates is a natural next step.
Beyond the paper: the Bentkus bound is tight for binary losses, but for non-binary or high-variance losses other confidence bounds could tighten the method, as Remark 4.4 hints.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Read Ek & Zachariah's paper on certifiable treatment allocations under partial identifiability. The short version: the problem is real and the combination of miscalibration-bounded weights with split-conformal risk control is a natural and promising idea, but as written the central certification theorem doesn't go through, and the RCT case has an identification error.

What's new: they extend conformal risk control to policy learning with partially identified treatment risk, using Tan-style miscalibration bounds to get valid upper bounds on the risk, then select a policy threshold t_n on a separate split. The experiments with fast-and-frugal trees on synthetic and STAR/IST data are clear and honest. The appendix on benchmarking Gamma is practical and useful.

The soft spots are serious, not minor. First, Theorem 4.3's proof. In (18) they infer from the in-sample constraint bEm[V(t)] <= t that t >= (1/m) sum E[V_i(t)] = E[V(t)] = T(t). That equality is at best an unconditional expectation over the training split; for the realized D_m, the empirical constraint gives no bound on the conditional risk T(t) of the learned policy. The chain t_n >= T(t_n) > tau is therefore unjustified, and the event whose probability (8) controls is not the event they bound. The assumption that pi(X;tau) is constraint-active doesn't repair this, because the equality is about the empirical constraint, not the conditional risk. The theorem proves a weaker statement about risk averaged over D_m.

Second, Remark 4.2. For RCT data, p_pi(A=1|S=0) = integral 1(pi(x)=1) p(x|S=0) dx, and that is not identified from trial data under their own selection model (4) where U affects S. They seem to slip in p(x|S=s) with the target population, but for s=1 that's the trial covariate distribution. The denominator of the treatment-risk bound is therefore not identifiable from the trial, and the certified guarantee for the RCT case is unsupported. They also need p(S=1)/p(S=0) in the weights, which is another unknown.

Otherwise the observational part is mostly coherent if you're willing to condition on a fixed D_m in (13); the split-conformal ingredient is standard. But these two issues are load-bearing. The paper deserves a serious referee because the approach is worth fixing; I'd send it out, but any referee should insist on a corrected proof and a clear treatment of the RCT target-population quantities.

For my own work, I wouldn't cite the guarantee as a theorem until the proof is fixed.

Recommendation: engage with it as promising work in progress; ask for major revision.

Referee Report

3 major / 5 minor

Summary. The paper proposes a method for learning treatment-allocation policies that minimize population risk subject to a constraint on the treatment risk, defined as the probability of a non-beneficial outcome among treated patients. The method is designed for settings in which the treatment risk is not point-identifiable, either because of unmeasured confounding in observational data or because of unmeasured selection into a randomized trial. The authors model the degree of miscalibration of the propensity or selection odds by a parameter Gamma, derive upper bounds on the population and treatment risks (Lemma 4.1), and then use sample splitting: the first half of the data, D_m, is used to learn a family of policies indexed by a nominal tolerance t; the second half, D_n, is used to construct an upper confidence bound on the treatment risk of each policy and to select an empirical tolerance t_n (Algorithm 1, Theorem 4.3). The paper also reports simulation experiments and applications to the STAR and International Stroke Trial datasets.

Significance. If the central finite-sample guarantee were established, the paper would make a useful contribution: it extends distribution-free risk-control ideas from prediction sets to policy learning under partial identifiability, while keeping the policy class interpretable. The miscalibration model via odds-ratio bounds is clearly stated and the empirical evaluation is extensive, including both confounding and selection-bias settings. The authors also correctly identify that the treatment risk is not point-identifiable in either data regime and that a certification statement must account for this. However, the main theoretical result, Theorem 4.3, has a proof gap that is load-bearing, and the randomized-trial extension has an identifiability problem. These issues affect the paper's central claim, so the contribution is currently not established.

major comments (3)

[Theorem 4.3, Eq. (18)] The proof of Theorem 4.3 relies on the inequality t ≥ T(t), derived in Eq. (18) by taking expectations of the empirical constraint in (12). This step is valid only if T(t) is defined as the marginal expectation E[V(t)] averaged over the training split D_m. But the guarantee (8) is about the treatment risk of the realized policy conditional on the data, i.e., E[V(t)|D_m]. For the conditional risk, the in-sample constraint bE_m[V(t)] ≤ t gives no bound; conditional on D_m, the sample mean can be below t while the population mean is above t. Consequently the chain 'T(t_n) > τ implies t_n ≥ T(t_n) > τ' is invalid. If T(t) is instead read as the marginal expectation, then (13) and the final event control the risk averaged over D_m, not the risk of the returned policy, which is a strictly weaker statement than (8). The equality assumption on π(X;τ) does not repair this, because it is an equality of the empirical constraint, not of the conditional risk.
[Remark 4.2 and Lemma 4.1 (RCT case)] For randomized trial data, the denominator pπ(A=1|S=0) in Lemma 4.1 and in Eq. (11) is not identifiable from trial data under the paper's own model (4), because the trial covariate distribution p(x|S=1) need not equal the target distribution p(x|S=0). Remark 4.2 writes pπ(A=1|S=0) = ∫1(π(x)=1)p(x|S=s)dx; when s=1, the right-hand side is the treatment probability under the trial covariate distribution, not the target-population quantity. The weight W^Γ in (11) also contains p(S=1)/p(S=0), which is not specified or bounded. As a result, the upper bound (9) for the RCT case is not computable from the available data, and the STAR and IST experiments in Section 5.2 and Appendices A.4–A.5 do not have a supported finite-sample guarantee unless an additional assumption equating target and trial covariate distributions is introduced.
[Theorem 4.3, equality assumption] Theorem 4.3 is conditional on the nominal policy π(X;τ) achieving the constraint in (12) with equality. Since (12) is an inequality constraint, equality is a data-dependent event whose probability is neither controlled nor guaranteed by Algorithm 1. If the constraint is inactive, the proof's step T(τ)=τ fails, and the theorem gives no certificate. The authors should either justify that equality can be enforced by construction (e.g., by choosing τ on a grid where the constraint binds) or provide a proof that does not require this condition.

minor comments (5)

[Section 2] There are typos in the text describing the DAGs: 'were' should be 'where' and 'unbserved' should be 'unobserved'.
[Algorithm 1] Step 2 loops over t in the continuum (0,1); the practical implementation uses a finite grid, but this discretization and its effect on the guarantee should be stated explicitly.
[Eq. (14)] The definition of t_n as an arg min subject to a constraint involving all t' ≤ t presumes a well-defined feasible set; since T_n^α(t) need not be monotone, the authors should discuss existence, uniqueness, and computation of t_n.
[Appendix B, Theorem B.1] The proof concludes that E[V_{n+1}(t_n)|E=1] ≤ τ and says this holds with probability at least 1−α; the probabilistic statement should be made precise, since E[V_{n+1}|E=1] is a conditional expectation over the data, not a random variable with an obvious coverage interpretation.
[Remark 4.4] The Bentkus bound is described for binary losses, but the paper does not discuss whether the bound in (13) needs to hold simultaneously over t for the proof of Theorem 4.3; a clarifying sentence would help.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the finite-sample certification is an external conformal-style argument; the flagged issues are identifiability and conditioning gaps, not circular reductions.

full rationale

The paper's derivation chain is self-contained against external benchmarks. Lemma 4.1 derives upper bounds on R(pi) and T(pi) from the miscalibration model (6)-(7) via importance weighting; these are inequalities with W^Gamma constructed from Gamma, not assumed conclusions. The finite-sample certificate in Theorem 4.3 follows the argument of Bates et al. (2021), which the paper credits, using a user-specified tau and alpha and a confidence bound (15) that is external to the authors. The empirical tolerance tn is chosen conservatively so that tau exceeds the upper bound for all smaller tolerances; no fitted parameter is later relabeled as a prediction. The only self-citation (Ek and Zachariah 2024, Appendix A.2) is used to benchmark Gamma in an illustrative sensitivity analysis and is paired with Huang et al. (2021); it is not load-bearing for the theorem. Two non-circular concerns should be weighed separately. Remark 4.2 asserts identifiability of p_pi(A=1|S=0) from trial data by writing it as an integral of p(x|S=s); under the paper's own model (4), p(x|S=1) need not equal p(x|S=0), so the RCT guarantee is unsupported as stated. Also, Eq. (18) applies an unconditional expectation to an in-sample constraint and concludes t >= T(t); for the realized data-dependent policy T(t) is conditional on D_m, so the step conflates marginal and conditional risk. Both are correctness risks, not circular reductions of the claim to its inputs.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The method introduces no new causal entities. The free parameters are the miscalibration bound Gamma, the Bentkus bound Vmax, and for RCT data the selection ratio. The key axioms are the causal model, the Gamma-bounded odds deviations, the active-constraint assumption in Theorem 4.3, and the unsupported identification claim for the RCT treatment-probability denominator.

free parameters (3)

Gamma, degree of miscalibration = Gamma=2 in confounding and selection experiments; benchmarked 1.5 to 1.7 for STAR propensity; Gamma=1 for STAR and IST…
The certified guarantee is valid only if the true odds miscalibration is at most Gamma. The paper benchmarks Gamma by omitting covariates but ultimately treats it as a user-specified input. The authors' own Figure 4a shows that setting Gamma too low destroys risk control.
Vmax, upper limit on the weighted loss V(t) = Not specified in the text
The Bentkus confidence bound in Eq. (15) requires an upper limit Vmax on V(t). The paper says V(t) is expected to be near 0 or clustered near Vmax but does not state how Vmax is chosen in the experiments.
p(S=1)/p(S=0), selection ratio in RCT weights = Not specified
The RCT weight in Eq. (11) includes the overall selection ratio p(S=1)/p(S=0). If this ratio is estimated from finite data, the pointwise upper bound (7) and the finite-sample guarantee require accounting for estimation error, which the paper does not address.

assumptions (5)

domain assumption Causal factorization in Eqs. (3) and (4) with unobserved U affecting treatment or selection and outcome.
The partial-identification argument is built on these directed acyclic graph models. If the causal structure is different, the importance weights and bounds change.
domain assumption Odds-ratio miscalibration bounds in Eqs. (6) and (7) hold for all X,U with a known Gamma.
Lemma 4.1 uses these bounds to construct W^Gamma. If the true odds deviate by more than Gamma, the upper bounds in Eq. (9) are not valid and the certificate fails.
ad hoc to paper The nominal policy pi(X;tau) from Eq. (12) achieves its empirical constraint with equality.
Theorem 4.3 requires T(tau)=tau, which follows only when the inequality constraint in (12) is binding. Algorithm 1 does not enforce or check equality, so this is an unguaranteed condition.
ad hoc to paper For RCT data, p_pi(A=1|S=0) is computable as the integral of 1(pi(x)=1) over p(x|S=s).
Remark 4.2 asserts this identification. For trial data S=s=1, this equates the target covariate distribution p(x|S=0) with the trial distribution p(x|S=1), which is not implied by Eq. (4) and is generally false when U affects selection.
standard math Data are i.i.d. and the random split into D_m and D_n is independent.
The conformal-style guarantees in Theorem 4.3 and Appendix B rely on exchangeability or i.i.d. sampling and independent sample splitting.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Learning Treatment Allocations with Risk Control Under Partial Identifiability." pith.science (2026). https://pith.science/paper/QLW5U3Z2

@misc{pith2026250508378,
  author       = {Pith},
  title        = {Pith review of: Learning Treatment Allocations with Risk Control Under Partial Identifiability},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QLW5U3Z2}},
  note         = {Machine review of arXiv:2505.08378}
}

read the original abstract

Learning beneficial treatment allocations for a patient population is an important problem in precision medicine. Many treatments come with adverse side effects that are not commensurable with their potential benefits. Patients who do not receive benefits after such treatments are thereby subjected to unnecessary harm. This is a `treatment risk' that we aim to control when learning beneficial allocations. The constrained learning problem is challenged by the fact that the treatment risk is not in general identifiable using either randomized trial or observational data. We propose a certifiable learning method that controls the treatment risk with finite samples in the partially identified setting. The method is illustrated using both simulated and real data.

Figures

Figures reproduced from arXiv: 2505.08378 by the authors.

**Figure 1.** Tolerance τ versus (a) treatment risk and (b) population risk of learned allocation policies π. The proposed learning method is set to limit the treatment risk to be no greater than τ with a probability of at least 90%. The shaded regions (10-90th percentiles) represent the resulting risks of policies learned from 1000 different datasets. Thus τ trades off two types of risks (a) and (b). The details of the example a… view at source ↗

**Figure 2.** Structural causal models, specified by acyclic directed graphs, that describe the data generat [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

**Figure 3.** Treatment allocation policies π(X;tn) learned from a synthetic dataset under different risk tolerance levels τ , assuming Γ = 1 in Equation (12). 0% 5% 10% 15% 20% 25% 30% 35% 40% 0% 10% 20% 30% 40% τ Pπ(L = 1|A = 1, S = 0) Γ = 1 Γ = 2 (a) 0% 5% 10% 15% 20% 25% 30% 35% 40% 0% 20% 40% 60% 80% 100% τ Pπ(L = 1|S = 0) (b) [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

Figures from the paper (10 more)

**Figure 4.** Figure 4: The treatment risk tolerance τ versus treatment risk, T(π), and the population risk, R(π), under π. The learned policies are certified to control the risk with probability of at least 1−α = 90%, up to a specified degree of miscalibration (Γ = 1, 2). The shaded blue reg…

**Figure 5.** Figure 5: Treatment and population risks of policies learned from STAR dataset. The shaded regions [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]

**Figure 6.** Figure 6: Examples of learned treatment policies π(X;tn) using the STAR dataset, with student covariates X in grey, for different risk tolerances τ . The final dataset consists of 4218 students. A 50 percent of the samples is randomly split for policy construction in (12), where…

**Figure 7.** Figure 7: Benchmarking Γ. (a) Reliability diagram of the observed odds against the average predicted nominal odds. (b) Benchmarking Γ using omitted covariates. 0% 5% 10% 15% 20% 25% 30% 35% 40% 0% 10% 20% 30% 40% τ Pπ(L = 1|A = 1, S = 0) Γ = 1 Γ = 2 (a) 0% 5% 10% 15% 20% 25% 30%…

**Figure 8.** Figure 8: The treatment risk under policy π, and the population risk under the same policy for different values of the treatment risk tolerance τ and different confounding assumptions (Γ = 1, 2). (a) Treatment risk falls below any given τ with a probability of at least 90%. (b) …

**Figure 9.** Figure 9: The treatment risk under policy π, and the population risk under the same policy for different values of the treatment risk tolerance τ and different confounding assumptions (Γ = 1, 2). (a) For Γ = 1, the coverage guarantee that the treatment risk should be below τ wit…

**Figure 10.** Figure 10: The estimated probability of treatment risk, [PITH_FULL_IMAGE:figures/full_fig_p016_10.png]

**Figure 11.** Figure 11: The expected treatment risk, E[T(π)], and the expected population risk, E[R(π)], for different values of the treatment risk tolerance τ and different confounding assumptions (Γ = 1, 2) using known weights. (a) Expected treatment risk falls below any given τ for both Γ…

**Figure 12.** Figure 12: The expected treatment risk, E[T(π)], and the expected population risk, E[R(π)], for different values of the treatment risk tolerance τ and different confounding assumptions (Γ = 1, 2). The weights are confounded. (a) For Γ = 1, the policy is invalid across a range of…

**Figure 13.** Figure 13: The estimated probability of the loss for treated individuals, [PITH_FULL_IMAGE:figures/full_fig_p018_13.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

51 extracted references · 27 canonical work pages

[1]

Achilles, Helen Pate Bain, Fred Bellott, Jayne Boyd-Zaharias, Jeremy Finn, John Folger, John Johnston, and Elizabeth Word

C.M. Achilles, Helen Pate Bain, Fred Bellott, Jayne Boyd-Zaharias, Jeremy Finn, John Folger, John Johnston, and Elizabeth Word. Tennessee's Student Teacher Achievement Ratio (STAR) project , 2008. URL https://doi.org/10.7910/DVN/SIWH9F

work page doi:10.7910/dvn/siwh9f 2008
[2]

Externally valid policy choice

Christopher Adjaho and Timothy Christensen. Externally valid policy choice. arXiv preprint arXiv:2205.05561v3, 1, 2023

arXiv 2023
[3]

Conformal risk control

Anastasios N Angelopoulos, Stephen Bates, Adam Fisch, Lihua Lei, and Tal Schuster. Conformal risk control. arXiv preprint arXiv:2208.02814, 2022

arXiv 2022
[4]

Recursive partitioning for heterogeneous causal effects

Susan Athey and Guido Imbens. Recursive partitioning for heterogeneous causal effects. Proceedings of the National Academy of Sciences, 113 0 (27): 0 7353--7360, 2016

2016
[5]

Policy learning with observational data

Susan Athey and Stefan Wager. Policy learning with observational data. Econometrica, 89 0 (1): 0 133--161, 2021

2021
[6]

Distribution-free, risk-controlling prediction sets

Stephen Bates, Anastasios Angelopoulos, Lihua Lei, Jitendra Malik, and Michael Jordan. Distribution-free, risk-controlling prediction sets. Journal of the ACM (JACM), 68 0 (6): 0 1--34, 2021

work page 2021
[7]

Safe policy learning through extrapolation: Application to pre-trial risk assessment

Eli Ben-Michael, D James Greiner, Kosuke Imai, and Zhichao Jiang. Safe policy learning through extrapolation: Application to pre-trial risk assessment. Journal of the American Statistical Association, pages 1--23, 2025

2025
[8]

Intelligible models for healthcare: Predicting pneumonia risk and hospital 30-day readmission

Rich Caruana, Yin Lou, Johannes Gehrke, Paul Koch, Marc Sturm, and Noemie Elhadad. Intelligible models for healthcare: Predicting pneumonia risk and hospital 30-day readmission. In Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining, pages 1721--1730, 2015

2015

Show all 51 references

[9]

Optimal discrete decisions when payoffs are partially identified

Timothy Christensen, Hyungsik Roger Moon, and Frank Schorfheide. Optimal discrete decisions when payoffs are partially identified. arXiv preprint arXiv:2204.11748v2, 2023

2023
[10]

Individualized decision-making under partial identification: Three perspectives, two optimality results, and one paradox

Yifan Cui. Individualized decision-making under partial identification: Three perspectives, two optimality results, and one paradox. Harvard Data Science Review, 3 0 (3), 2021

2021
[11]

Risk controlled decision trees and random forests for precision medicine

Kevin Doubleday, Jin Zhou, Hua Zhou, and Haoda Fu. Risk controlled decision trees and random forests for precision medicine. Statistics in medicine, 41 0 (4): 0 719--735, 2022

2022
[12]

Doubly robust policy evaluation and learning

Miroslav Dud \' k, John Langford, and Lihong Li. Doubly robust policy evaluation and learning. In Proceedings of the 28th International Conference on International Conference on Machine Learning, pages 1097--1104, 2011

2011
[13]

Externally valid policy evaluation from randomized trials using additional observational data

Sofia Ek and Dave Zachariah. Externally valid policy evaluation from randomized trials using additional observational data. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. URL https://openreview.net/forum?id=2pgc5xDJ1b

2024
[14]

Simple heuristics that make us smart

Gerd Gigerenzer, Peter M Todd, the ABC Research Group, et al. Simple heuristics that make us smart. Oxford University Press, 2000

2000
[15]

The international stroke trial (ist): a randomised trial of aspirin, subcutaneous heparin, both, or neither among 19 435 patients with acute ischaemic stroke

International Stroke Trial Collaborative Group et al. The international stroke trial (ist): a randomised trial of aspirin, subcutaneous heparin, both, or neither among 19 435 patients with acute ischaemic stroke. The Lancet, 349 0 (9065): 0 1569--1581, 1997

1997
[16]

Harrell Jr, Karel GM Moons, Thomas PA Debray, and Johannes B Reitsma

Jeroen Hoogland, Joanna IntHout, Michail Belias, Maroeska M Rovers, Richard D Riley, Frank E. Harrell Jr, Karel GM Moons, Thomas PA Debray, and Johannes B Reitsma. A tutorial on individualized treatment effect prediction from randomized trials with a binary endpoint. Statistic...

2021
[17]

Off-policy risk assessment in contextual bandits

Audrey Huang, Liu Leqi, Zachary Lipton, and Kamyar Azizzadenesheli. Off-policy risk assessment in contextual bandits. Advances in Neural Information Processing Systems, 34: 0 23714--23726, 2021

2021
[18]

Sensitivity analysis for the generalization of experimental results

Melody Y Huang. Sensitivity analysis for the generalization of experimental results. Journal of the Royal Statistical Society Series A: Statistics in Society, page qnae012, 03 2024

2024
[19]

From temporary help jobs to permanent employment: what can we learn from matching estimators and their sensitivity? Journal of applied econometrics, 23 0 (3): 0 305--327, 2008

Andrea Ichino, Fabrizia Mealli, and Tommaso Nannicini. From temporary help jobs to permanent employment: what can we learn from matching estimators and their sensitivity? Journal of applied econometrics, 23 0 (3): 0 305--327, 2008

2008
[20]

Causal inference in statistics, social, and biomedical sciences

Guido W Imbens and Donald B Rubin. Causal inference in statistics, social, and biomedical sciences. Cambridge university press, 2015

2015
[21]

Balanced policy evaluation and learning

Nathan Kallus. Balanced policy evaluation and learning. Advances in neural information processing systems, 31, 2018

2018
[22]

What's the harm? sharp bounds on the fraction negatively affected by treatment

Nathan Kallus. What's the harm? sharp bounds on the fraction negatively affected by treatment. Advances in Neural Information Processing Systems, 35: 0 15996--16009, 2022

2022
[23]

Minimax-optimal policy learning under unobserved confounding

Nathan Kallus and Angela Zhou. Minimax-optimal policy learning under unobserved confounding. Management Science, 67 0 (5): 0 2870--2890, 2021

2021
[24]

Removing hidden confounding by experimental grounding

Nathan Kallus, Aahlad Manas Puli, and Uri Shalit. Removing hidden confounding by experimental grounding. Advances in neural information processing systems, 31, 2018

2018
[25]

Classification in the wild: The science and art of transparent decision making

Konstantinos V Katsikopoulos, Ozgur Simsek, Marcus Buckmann, and Gerd Gigerenzer. Classification in the wild: The science and art of transparent decision making. MIT Press, 2021

2021
[26]

Who should be treated? empirical welfare maximization methods for treatment choice

Toru Kitagawa and Aleksey Tetenov. Who should be treated? empirical welfare maximization methods for treatment choice. Econometrica, 86 0 (2): 0 591--616, 2018

2018
[27]

Experimental estimates of education production functions

Alan B Krueger. Experimental estimates of education production functions. The quarterly journal of economics, 114 0 (2): 0 497--532, 1999

1999
[28]

Distribution-free predictive inference for regression

Jing Lei, Max G’Sell, Alessandro Rinaldo, Ryan J Tibshirani, and Larry Wasserman. Distribution-free predictive inference for regression. Journal of the American Statistical Association, 113 0 (523): 0 1094--1111, 2018

2018
[29]

Trustworthy policy learning under the counterfactual no-harm criterion

Haoxuan Li, Chunyuan Zheng, Yixiao Cao, Zhi Geng, Yue Liu, and Peng Wu. Trustworthy policy learning under the counterfactual no-harm criterion. In International Conference on Machine Learning, pages 20575--20598. PMLR, 2023

2023
[30]

Identification problems in the social sciences and everyday life

Charles F Manski. Identification problems in the social sciences and everyday life. Southern Economic Journal, 70 0 (1): 0 11--21, 2003

2003
[31]

Statistical treatment rules for heterogeneous populations

Charles F Manski. Statistical treatment rules for heterogeneous populations. Econometrica, 72 0 (4): 0 1221--1246, 2004

2004
[32]

Identification for prediction and decision

Charles F Manski. Identification for prediction and decision. Harvard University Press, 2007

2007
[33]

Quasi-oracle estimation of heterogeneous treatment effects

Xinkun Nie and Stefan Wager. Quasi-oracle estimation of heterogeneous treatment effects. Biometrika, 108 0 (2): 0 299--319, 2021

2021
[34]

Elements of causal inference: foundations and learning algorithms

Jonas Peters, Dominik Janzing, and Bernhard Sch \"o lkopf. Elements of causal inference: foundations and learning algorithms. The MIT Press, 2017

2017
[35]

Performance guarantees for individualized treatment rules

Min Qian and Susan A Murphy. Performance guarantees for individualized treatment rules. Annals of statistics, 39 0 (2): 0 1180, 2011

2011
[36]

Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead

Cynthia Rudin. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature machine intelligence, 1 0 (5): 0 206--215, 2019

2019
[37]

The international stroke trial database

Peter AG Sandercock, Maciej Niewada, Anna Cz onkowska, and International Stroke Trial Collaborative Group. The international stroke trial database. Trials, 12 0 (1): 0 101, 2011

2011
[38]

Perspective on ‘harm’in personalized medicine

Aaron L Sarvet and Mats J Stensrud. Perspective on ‘harm’in personalized medicine. American Journal of Epidemiology, page kwad162, 2023

2023
[39]

Origin and uses of primum non nocere—above all, do no harm! The Journal of Clinical Pharmacology, 45 0 (4): 0 371--377, 2005

Cedric M Smith. Origin and uses of primum non nocere—above all, do no harm! The Journal of Clinical Pharmacology, 45 0 (4): 0 371--377, 2005

2005
[40]

Counterfactual risk minimization: Learning from logged bandit feedback

Adith Swaminathan and Thorsten Joachims. Counterfactual risk minimization: Learning from logged bandit feedback. In International Conference on Machine Learning, pages 814--823. PMLR, 2015

2015
[41]

A distributional approach for causal inference using propensity scores

Zhiqiang Tan. A distributional approach for causal inference using propensity scores. Journal of the American Statistical Association, 101 0 (476): 0 1619--1637, 2006

2006
[42]

Decision curve analysis: a novel method for evaluating prediction models

Andrew J Vickers and Elena B Elkin. Decision curve analysis: a novel method for evaluating prediction models. Medical Decision Making, 26 0 (6): 0 565--574, 2006

2006
[43]

Algorithmic learning in a random world

Vladimir Vovk, Alexander Gammerman, and Glenn Shafer. Algorithmic learning in a random world. Springer Science & Business Media, 2005

2005
[44]

Estimation and inference of heterogeneous treatment effects using random forests

Stefan Wager and Susan Athey. Estimation and inference of heterogeneous treatment effects using random forests. Journal of the American Statistical Association, 113 0 (523): 0 1228--1242, 2018

2018
[45]

Learning optimal personalized treatment rules in consideration of benefit and risk: with an application to treating type 2 diabetes patients with insulin therapies

Yuanjia Wang, Haoda Fu, and Donglin Zeng. Learning optimal personalized treatment rules in consideration of benefit and risk: with an application to treating type 2 diabetes patients with insulin therapies. Journal of the American Statistical Association, 113 0 (521): 0 1--13, 2018

2018
[46]

All of statistics: a concise course in statistical inference

Larry Wasserman. All of statistics: a concise course in statistical inference. Springer Science & Business Media, 2013

2013
[47]

Epidemiology by Design: A Causal Approach to the Health Sciences

Daniel Westreich. Epidemiology by Design: A Causal Approach to the Health Sciences. Oxford University Press, Incorporated, 2019. ISBN 9780190665760

2019
[48]

Optimal decision rules under partial identification

Kohei Yata. Optimal decision rules under partial identification. arXiv preprint arXiv:2111.04926v4, 2025

2025 arXiv
[49]

A robust method for estimating optimal treatment regimes

Baqun Zhang, Anastasios A Tsiatis, Eric B Laber, and Marie Davidian. A robust method for estimating optimal treatment regimes. Biometrics, 68 0 (4): 0 1010--1018, 2012

2012
[50]

Using decision lists to construct interpretable and parsimonious treatment regimes

Yichi Zhang, Eric B Laber, Anastasios Tsiatis, and Marie Davidian. Using decision lists to construct interpretable and parsimonious treatment regimes. Biometrics, 71 0 (4): 0 895--904, 2015

2015
[51]

Estimating individualized treatment rules using outcome weighted learning

Yingqi Zhao, Donglin Zeng, A John Rush, and Michael R Kosorok. Estimating individualized treatment rules using outcome weighted learning. Journal of the American Statistical Association, 107 0 (499): 0 1106--1118, 2012

2012

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.

[1] [1]

Achilles, Helen Pate Bain, Fred Bellott, Jayne Boyd-Zaharias, Jeremy Finn, John Folger, John Johnston, and Elizabeth Word

C.M. Achilles, Helen Pate Bain, Fred Bellott, Jayne Boyd-Zaharias, Jeremy Finn, John Folger, John Johnston, and Elizabeth Word. Tennessee's Student Teacher Achievement Ratio (STAR) project , 2008. URL https://doi.org/10.7910/DVN/SIWH9F

work page doi:10.7910/dvn/siwh9f 2008

[2] [2]

Externally valid policy choice

Christopher Adjaho and Timothy Christensen. Externally valid policy choice. arXiv preprint arXiv:2205.05561v3, 1, 2023

arXiv 2023

[3] [3]

Conformal risk control

Anastasios N Angelopoulos, Stephen Bates, Adam Fisch, Lihua Lei, and Tal Schuster. Conformal risk control. arXiv preprint arXiv:2208.02814, 2022

arXiv 2022

[4] [4]

Recursive partitioning for heterogeneous causal effects

Susan Athey and Guido Imbens. Recursive partitioning for heterogeneous causal effects. Proceedings of the National Academy of Sciences, 113 0 (27): 0 7353--7360, 2016

2016

[5] [5]

Policy learning with observational data

Susan Athey and Stefan Wager. Policy learning with observational data. Econometrica, 89 0 (1): 0 133--161, 2021

2021

[6] [6]

Distribution-free, risk-controlling prediction sets

Stephen Bates, Anastasios Angelopoulos, Lihua Lei, Jitendra Malik, and Michael Jordan. Distribution-free, risk-controlling prediction sets. Journal of the ACM (JACM), 68 0 (6): 0 1--34, 2021

work page 2021

[7] [7]

Safe policy learning through extrapolation: Application to pre-trial risk assessment

Eli Ben-Michael, D James Greiner, Kosuke Imai, and Zhichao Jiang. Safe policy learning through extrapolation: Application to pre-trial risk assessment. Journal of the American Statistical Association, pages 1--23, 2025

2025

[8] [8]

Intelligible models for healthcare: Predicting pneumonia risk and hospital 30-day readmission

Rich Caruana, Yin Lou, Johannes Gehrke, Paul Koch, Marc Sturm, and Noemie Elhadad. Intelligible models for healthcare: Predicting pneumonia risk and hospital 30-day readmission. In Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining, pages 1721--1730, 2015

2015