Pith. sign in

REVIEW 1 major objections 5 minor 12 references

When Can You Trust Offline Evaluation of Equal-Cost Top-k Allocation? A Controlled, Reproducible Benchmark and Practitioner's Guide

T0 review · 1 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that trust in offline evaluation of deterministic top-k allocation is governed by logger-target action alignment and is rankable in advance from logged data, whenever the propensity model is credible.

desk verdict A careful, self-scoped benchmark; the RQ2 alignment result and RQ3 algebra are solid, but the guide's RQ3 wording overreaches the known-propensity setting. read the letter →

arxiv 2608.12489 v1 pith:JD5X3XNH submitted 2026-08-12 cs.LG stat.MEstat.ML

Binshuang Li This is my paper · ORCID
classification cs.LGstat.MEstat.ML
keywords off-policyevaluationtop-kallocationweakoverlapeffectivesamplesizedoublyrobustestimationoptimizer'scursepropensityofflinebenchmark
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Off-policy evaluation is meant to tell an organization what a budget-constrained targeting rule would earn before deployment, but a deterministic top-k rule removes all averaging over actions, so the importance weights become either zero or the reciprocal of the logger's probability of the rule's chosen action. This paper establishes that the decisive risk factor is the alignment between the logger and the rule's actions, not the sharpness of the logging policy: sharpening a score-aligned logger barely moves overlap, while an action-level disagreement collapses effective sample size and multiplies weighting error. It then shows the risk is rankable ex ante from logged data across logging environments, with effective sample size as a valid between-environment screen that is weak within a single log and whose cut points do not transfer, and that a credible propensity model is a prerequisite because propensity-estimation error is the largest measured degradation and can invert the screen. For the optimizer's curse, when the rule is learned on the evaluation data, cross-fitting only the outcome nuisance makes the optimistic bias worse, while honest policy-level splitting removes the reuse by targeting the learning procedure's value rather than the full-sample policy's value. Across the benchmark's stresses, doubly robust estimation is the most stable estimator, making it the defensible default when at least one nuisance model is credible.

What carries the argument

The load-bearing object is the deterministic-target importance weight $w_i = \pi_e(a_i|x_i)/\pi_b(a_i|x_i)$, which takes only the values $0$ when the logger's action differs from the rule's, or $1/\pi_b(a_i|x_i)$ when it matches, so every weak-overlap mechanism runs through the logger's probability of the rule's chosen action. The paper's overlap diagnostic, the normalized Kish effective-sample-size fraction, factors exactly into an action-coverage term times a dispersion term on matched units, which is why it ranks risk across logging environments but not candidates within one log. The optimizer's-curse argument is carried by an exact finite-sample identity, $\widehat{V}_{\mathrm{in}} - \widehat{V}_{\mathrm{honest}} = \frac{1}{n}\sum_i (1 - w_i \mathbf{1}[a_i=\pi_e(x_i)])(\widehat{\mu}_{\mathrm{in}} - \widehat{\mu}_{\mathrm{honest}})(x_i,\pi_e(x_i))$, whose expectation under known propensities is $-\mathrm{Cov}(w\mathbf{1}[a=\pi_e], \widehat{\mu}_{\mathrm{in}})$; whether that covariance is positive and material is the empirical content the eight known-effect regimes test. A third named mechanism is the sharpening limit of Proposition 1, which gives the population ESS fraction limits for score-aligned and action-aligned loggers, approximately $\varepsilon/(1-c)$ versus $1-\varepsilon$.

What would settle it

Run the RQ3 comparison on a real deployment log with estimated, not known, propensities and compute the sample covariance between the weight-times-action-match indicator and the in-sample outcome nuisance on the evaluation fold; if that covariance is zero or negative, nuisance-only cross-fitting will not worsen the reuse bias, and the paper's central RQ3 guidance fails for that regime.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central claim is that weak-overlap failure in offline evaluation of equal-cost top-k allocation is a property of logger–target action alignment, not of logging sharpness: for a deterministic target the weight is $\pi_e(a|x)/\pi_b(a|x)\in\{0, 1/\pi_b(a|x)\}$, so the quantity that governs support is $\pi_b(a_{\pi_e}(x)|x)$, the logger's probability of the exact action the rule selects. A sharpening-limit argument shows a score-aligned logger sits on a finite plateau whose population effective-sample-size fraction approaches $\varepsilon/(1-c)$, while an action-aligned logger sharpens toward full support; the benchmark confirms the reversal by re-centring the logger at the budget cutoff. The paper also claims that the optimizer's curse is not cured by nuisance-only cross-fitting: an exact finite-sample identity expresses the difference between in-sample and honest DR evaluation as a covariance between the weighted action-match indicator and the in-sample outcome nuisance, and in all eight known-effect regimes that covariance is positive, so frozen-policy cross-fitting is more optimistic than plain DR. Honest policy-level splitting reduces the reuse bias by 58–92% across those regimes, at the price of changing the estimand to the learning procedure's value. Finally, replacing the exact propensity with an out-of-fold estimate is the largest degradation measured, and a poor propensity model can invert the ESS diagnostic from useful to misleading; doubly robust estimation is essentially immune to that stress.

Load-bearing premise

The load-bearing premise is that the in-sample outcome model is elevated precisely on the observations the estimator weights most heavily, and that this pattern holds for real deployment logs with estimated propensities; if that pattern is absent, the guidance that nuisance-only cross-fitting makes the optimizer's curse worse does not transfer.

Editorial extensions

If this is right

  • A practitioner can compute the ESS fraction and support deficiency from logged actions and propensities and use them to rank logging situations before deployment, treating a low ESS as a reason to seek a better-aligned log or a model-based estimator rather than as a pass/fail certificate.
  • The same diagnostic should not be used to choose among candidate policies within the single log a practitioner already holds; within one log its median correlation with realized error is near zero.
  • When the targeting rule was learned from the same data used to evaluate it, nuisance-only cross-fitting should be avoided; honest policy-level splitting is the way to remove reuse bias, accepting that the estimand becomes the learning procedure's value.
  • Doubly robust estimation is the safer choice when outcome-model adequacy or overlap is uncertain, because it remains stable where direct modeling degrades and where fully weight-based IPS fails.
  • Policy selection requires a single shared log for all candidates; scoring each candidate on its own aligned log compares policy–logger pairs, not policies, and IPS tends to over-select the easiest-to-evaluate candidate.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Going beyond the paper: the covariance identity suggests a practical diagnostic: compute the sample covariance between the weight-action-match indicator and the in-sample outcome nuisance on the evaluation fold; a non-positive value marks a regime where nuisance-only cross-fitting is not the paper's danger.
  • Going beyond the paper: because the sharpening limit shows score-aligned loggers sit on a plateau, evaluation designers should treat logger–target action alignment rather than temperature as the primary overlap axis; the paper notes its own first design swept temperature on a score-aligned logger.
  • Going beyond the paper: the propensity-model inversion suggests a sanity check: if a flexible propensity model makes the ESS screen report the supposedly safest cells as the most dangerous, the model is likely near-constant and the screen is measuring coverage rather than dispersion.
  • Going beyond the paper: because mechanisms transfer but calibrated cut points do not, a deployment team could calibrate its own ESS threshold on a small randomized or labeled pilot before relying on the screen.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

1 major / 5 minor

Summary. The paper asks when off-policy evaluation (OPE) of deterministic equal-cost top-k allocation policies can be trusted, and benchmarks six estimators (DM, IPS, SNIPS, DR, Switch-DR, mIPS) on five datasets plus two six-DGP hardening suites, keeping exact-value references (synthetic, IHDP, ACIC-style) separate from noisy Horvitz-Thompson references on randomized trials (Hillstrom, Lenta, Jobs). Three headline claims are made. First, weak overlap is governed by logger-target action alignment rather than logging sharpness alone: Proposition 1 gives an exact sharpening limit showing a score-aligned logger's flat overlap is a finite-range plateau, and the empirical reversal under a cutoff-centred logger confirms it; effective sample size ranks overlap risk across logging environments, but not within a single fixed log, and its cut points do not transfer. Second, under policy-evaluation reuse (the optimizer's curse), cross-fitting only the outcome nuisance leaves the reuse bias in place and, across the eight known-effect regimes studied, makes it worse, while honest policy-level splitting cuts bias by 58-92% at the price of targeting the learning procedure's estimand. Third, propensity-estimation error is the largest degradation measured, can invert the ESS diagnostic (AUC 0.05 under a marginal propensity model), and leaves DR essentially unchanged. A non-simulated paired reference (Twins) replicates all three mechanisms but not the calibrated cut points.

Significance. If the claims hold, the paper supplies a decision procedure practitioners can apply before trusting an OPE estimate of a top-k rule: assess logger-target alignment and propensity-model credibility, use ESS only for across-environment ranking, and answer policy-evaluation reuse with policy-level honesty rather than nuisance cross-fitting. The negative results are as valuable as the positive ones: the within-log failure of ESS, the non-transferability of cut points, and the inversion of the screen under a poor propensity model are concrete scoping results. Methodological strengths are substantial and partly machine-checked: a released benchmark with documented make targets and per-row git commit hashes; numerical agreement of four estimators with Open Bandit Pipeline to floating-point precision; a derived-and-then-tested sharpening limit (Proposition 1); leave-one-dataset-out and out-of-fold nuisance reruns; measured coverage of the paper's own conditional intervals (DM 0.89 vs nominal 0.95, reported); an external reference read off recorded paired outcomes (Twins) with an explicit account of what fails to transfer; and Appendix A tabulating every claim against its evidence.

major comments (1)
  1. [§5.4; abstract ¶2; §6 item 3; conclusion] The headline statements of the RQ3 result (abstract: 'cross-fitting the nuisance alone leaves the reuse bias in place and makes it worse'; conclusion: 'cross-fitting the nuisance alone makes the optimism worse'; Practitioner's Guide item 3: 'Nuisance-only cross-fitting worsened it') are stated without the known-propensity qualifier that the paper's own derivation imposes. In Section 5.4 the identity is V_in - V_honest = (1/n) sum_i (1 - w_i * 1[a_i=pi_e(x_i)])(mu_in - mu_honest)(x_i, pi_e(x_i)), and the text correctly notes that its expectation equals -Cov(w * 1[a=pi_e], mu_in) only 'once cross-fitting makes mu_honest independent of the evaluation draw and the propensities are known, so E[w * 1[a=pi_e]|x] = 1 — as they are here by construction.' The body is properly scoped ('We do not establish when the covariance is material'; Appendix A: 'under the benchmark's known propensities'), but the practitioner-facing deliverables are not. This is load-bearing because (i) every RQ3 test, including the Twins replication, uses propensities known by construction since all logging is synthesized; (ii) with estimated propensities E[w_hat * 1[a=pi_e]|x] = pi_b(pi_e(x)|x)/pi_b_hat(pi_e(x)|x) != 1, so an additional first-order propensity-error term enters whose sign is untested; and (iii) Section 5.3 shows propensity error is the dominant degradation in realizable settings, so the unqualified 'worse' is transferred to exactly the regime the paper itself flags as hardest. The actionable core of item 3 — under reuse, use honest policy-level splitting; nuisance-only cross-fitting does not remove the reuse bias — survives the sign question, so the necessary fix is local: qualify 'makes it worse' in the abstract, conclusion, and guide as holding under the benchmark's known-propensity, studied-learner regimes, and ideally add an RQ3 rerun with out-of-fold estimated propensities using the machinery already built in Section 5.3/Table 9.
minor comments (5)
  1. [§5.4, Appendix A] The negated indicator notation '⊮[a_i=pi_e(x_i)]' is nonstandard and can be misread as a disjointness or not-an-element symbol; please write 1 - 1{a_i = pi_e(x_i)} or an explicit complement notation.
  2. [Table 4] The column header 'nuis. x-fit % removed' is awkward for rows in which the reported change is negative; a header such as '% change in |bias|' with a note that negative values are degradations would prevent misreading.
  3. [Figure 1] The line 'RQ2 — the effect of logging sharpness depends on logger–target action alignment' immediately above Figure 1 appears to be a duplicate floating title, since the caption below it states the same point; it should be removed or merged into the caption.
  4. [§1, Contribution 1] The phrase 'the failure gradient sharpens to 0.0%/1.7%/26.7%' is hard to parse because the verb 'sharpens' collides with the paper's logging-sharpness terminology; consider 'the failure gradient becomes 0.0%/1.7%/26.7%' or 'steepens to'.
  5. [Abstract] The sentence 'Effective sample size ... ranks this risk across logging environments — guidance for designing logs and choosing estimator families, since it is weak at ranking candidate policies within a single fixed log' would be clearer if the within-log limitation were its own sentence rather than a 'since' clause, which momentarily reads as if the limitation were the reason for the guidance.

Circularity Check

0 steps flagged · score 2.0 of 10

No load-bearing circularity; central claims are derived and tested, with one contextual self-citation.

full rationale

The derivation chain is self-contained. Proposition 1 is derived from the definition of the importance weight and Kish ESS, then tested on the benchmark grid and a cutoff-centred rerun; the paper explicitly calls the sharpening behavior a prediction and reports the predicted reversal. RQ3's reuse-bias identity is exact algebra: V_in minus V_honest equals a sum whose expectation is minus Cov(w times the action-match indicator, mu_hat_in) only under known propensities, and the paper states that "whether the covariance is positive and material ... is empirical" and that the eight regimes are the test; the abstract's unqualified phrasing is a scope limitation, not a circular reduction. The ESS screen is not introduced as a new result: the paper cites Austin (2011) and states "Our contribution is validating ESS as an ex-ante risk ranking, not the diagnostic itself"; its thresholds are calibrated on the benchmark and applied to held-out DGP families and the Twins paired reference, and the paper reports that cut points do not transfer. The honest-splitting bias reduction is explicitly labeled "close to definitional" and is framed as a change of estimand, with the practitioner-facing quantity measured against the full-sample target separately. The only self-citation (Li 2026, Related Work) motivates a neighboring selection question and carries no uniqueness or derivation weight in this paper. External anchors, including Open Bandit Pipeline numerical agreement, ACIC-style real-covariate suites, and the Twins recorded paired outcomes, provide independent checks. I therefore find no significant circularity; the minor self-citation is contextual and not load-bearing.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central claims rest on the benchmark construction (synthetic logging, known and floored propensities, five primary datasets) and on standard causal inference assumptions for the datasets used; the paper states these limits in Section 4.1. No new physical or conceptual entities are postulated; the estimators and diagnostics are existing methods, and the empirical quantities are measurements rather than invented constructs.

free parameters (4)
  • Propensity floor epsilon = 0.02 primary; 0.0002 in floor sweep
    Hand-set design constant that bounds importance weights by 50, makes weight clipping a no-op, and defines the bounded-weight regime in which every failure is measured. Section 4 and Appendix B state it as a design property with consequences.
  • Fragility screen cut points = ESS fraction 0.36; support deficiency 0.02
    Calibrated on the 540-cell alignment sweep to flag cells with median absolute relative bias above 10 percent. Section 5.2 reports that these cut points do not transfer and must be recalibrated on the local sample size.
  • Support-deficiency threshold kappa = 0.05; sensitivity checked at 0.01 and 0.10
    Nested constant defining which target-selected actions are barely covered by the logger. Appendix B notes that kappa equal to 0.01 is degenerate under the 0.02 propensity floor.
  • Error target delta for failure classification = 10 percent primary; 2 percent on the Hillstrom-covariate suite
    Defines when a cell counts as an IPS failure. The paper reports the full delta grid to avoid post hoc target selection, and the 2 percent threshold is matched to that suite's error scale rather than chosen for AUC.
assumptions (5)
  • domain assumption The logging propensity is exactly known and equals the true action-assignment probability in the logged data.
    Used throughout Section 5 for exact-propensity IPS and DR comparisons and in the Section 5.4 identity E[w times action-match indicator | x] equals 1. True by construction of the synthesized and rejection-sampled logging; Section 4.1 flags observational logs as outside scope.
  • domain assumption The cohort-conditional estimand V(pi_e)_n is identified from the logged sample with the allocation fixed.
    Section 3 defines the estimand and Section 4 holds allocation and cutoff fixed, which justifies the bootstrap and the use of exact references on synthetic and semi-synthetic data.
  • domain assumption The Horvitz-Thompson reference on randomized trials is unbiased for the evaluation-split value.
    Section 4 and Appendix P use an unbiased but noisy HT estimate on the randomized evaluation split. The paper shows the reference shares noise with IPS and that RCT-reference comparisons cannot resolve estimator ordering.
  • domain assumption Twins paired outcomes are exchangeable proxies for potential outcomes.
    Appendix N.3 reads the reference value off recorded paired outcomes, but heavier-lighter twin pairs are not true potential-outcome pairs. The paper states that co-twin exchangeability is plausible but not proven.
  • standard math Standard limiting results for ESS/n hold under the floored positive propensities.
    Proposition 1 in Section 4 and Appendix B derive the sharpening limit; the floor epsilon greater than zero ensures finite second moments so sample sums can be replaced by population expectations.

how reviews work

0 comments
Cite this review

Pith. "Pith review of When Can You Trust Offline Evaluation of Equal-Cost Top-k Allocation? A Controlled, Reproducible Benchmark and Practitioner's Guide." pith.science (2026). https://pith.science/paper/JD5X3XNH

@misc{pith2026260812489,
  author       = {Pith},
  title        = {Pith review of: When Can You Trust Offline Evaluation of Equal-Cost Top-k Allocation? A Controlled, Reproducible Benchmark and Practitioner's Guide},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JD5X3XNH}},
  note         = {Machine review of arXiv:2608.12489}
}
read the original abstract

Organizations decide whom to treat under a budget and want to know what a targeting rule would have earned before deploying it. Off-policy evaluation promises this from logged data, but the deployable rule is a deterministic top-k policy: it removes all averaging over actions, so weak overlap hits the estimate directly. We benchmark six estimators across five datasets and two known-effect sweeps, and validate the mechanisms against a non-simulated paired reference. First, weak overlap is governed by logger-target action alignment, not by logging sharpness alone: what governs support is the logger's probability of the target's actions. Sharpening a logger built from the target's own score barely moves overlap over the tested range; action-level disagreement collapses it. Effective sample size ranks this risk across logging environments, but is weak at ranking candidates within the single log a practitioner holds, and its cut point does not transfer. Second, the optimizer's curse is not fixed by cross-fitting the outcome nuisance. When the rule is fit on the data used to evaluate it, cross-fitting the nuisance alone leaves the reuse bias in place and makes it worse. Honest policy-level splitting avoids the reuse by targeting the learning procedure's value -- a change of estimand, not a de-biasing of the full-sample policy. Third, propensity-estimation error is the largest degradation we measure: an out-of-fold estimate hurts IPS more than any other stress we apply, leaves doubly-robust estimation almost unchanged, and can invert the overlap diagnostic itself. Logging is synthesized and propensities floored at 0.02, so every failure occurs with bounded weights; the floor also reduces the two tuned hybrids to their untuned parents, leaving four practically distinct estimators, and all exact-value surfaces are synthetic or semi-synthetic. We release the benchmark; public data only.

Figures

Figures reproduced from arXiv: 2608.12489 by the authors.

Figure 1
Figure 1. RQ2 — overlap risk is a property of logger–target [PITH_FULL_IMAGE:figures/full_fig_p010_1.png] view at source ↗
Figure 2
Figure 2. Schematic: the two-dimensional decision map of Sections 5.1–5.3, drawn. Positions are qualitative, [PITH_FULL_IMAGE:figures/full_fig_p012_2.png] view at source ↗
Figure 3
Figure 3. RQ1 — median relative RMSE by estimator, per dataset (lower is better). [PITH_FULL_IMAGE:figures/full_fig_p026_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: RQ4 — selection quality under three logging designs, on both slates (top: three candidates; bottom: [PITH_FULL_IMAGE:figures/full_fig_p027_4.png]
Figure 5
Figure 5. Figure 5: RQ2 fragility map — IPS |relative bias| against two logged-data diagnostics. Lower effective sample size (left) and higher support deficiency (right) both track larger error, quantified by the Spearman correlations in the text. 27 [PITH_FULL_IMAGE:figures/full_fig_p02…
Figure 6
Figure 6. Figure 6: RQ3 — honest policy-level splitting, not nuisance cross-fitting, addresses optimization bias. Mean [PITH_FULL_IMAGE:figures/full_fig_p028_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

12 extracted references · 10 canonical work pages

  1. [2]

    Median relative RMSE, same convention as Table 2; best per column in bold. Four decimals because several columns separate only in the fourth.IPS,SNIPSand mIPSuse no outcome model and are numerically identical to their in-sample counterparts.DMis best or tied-best on three of the five datasets here (Hillstromwithin0.0001ofmIPS,IHDP,synthetic) against four ...

    work page arXiv 2018

  2. [4]

    Clipping does not help

    means no unit can fall below it — the same floor that gives weight clipping no purchase (Section 5.1). The floor also makes 20 support deficiency a floor-dependentquantity (mass in[0.02,κ )) where ESS fraction is floor-robust, which is one reason we expect the latter to be the half of the screen that transfers. Does weight clipping rescue IPS?Truncating i...

    work page 2020

  3. [5]

    Naoki Nishimura, Ken Kobayashi, and Kazuhide Nakata

    arXiv:2207.00632. Naoki Nishimura, Ken Kobayashi, and Kazuhide Nakata. Balancing immediate revenue and future off-policy evaluation in coupon allocation.arXiv preprint arXiv:2407.11039,

    arXiv

  4. [11]

    poor model

    The RQ2 mechanism replicates in the form Section 5.2 states it: sharpening a self-aligned logger leaves overlap flat (median ESS fraction0.604, 0.556,0 .527at τ = 0.5, 2, 5), while the misaligned and candidate-independent loggers collapse under the same sharpening (0.388and0 .173at τ=0.5). The screen’s association is comparable to the primary sweep: Spear...

    work page 2024

  5. [12]

    shared ref

    As throughout RQ4, the full-budget pointk=1is excluded. The same paired per-cell contrast fornormalized regret(DM−IPS ; negative favoursDM), which is what Section 5.5 reads as the regret gap: 3-candidate slate 7-policy slate dataset per-cand. aligned indep. per-cand. aligned indep. Hillstrom−0.047−0.085−0.030 +0.092 +0.037 +0.083 IHDP−0.213−0.297−0.330−0....

    work page arXiv

  6. [25]

    median relative RMSE

    The cohort-conditional estimand of Section 3 is writtenV (πe)n, distinct from the populationV(π e). Compute and software.All experiments run on a single 10-core Apple M-series CPU; the full reproduction chain (every make target in Appendix B.1) takes roughly17hours end to end, the longest single target being the propensity sweep at about3.3hours. Python3 ...

    work page 2022

  7. [2011]

    Off-Policy Evaluation with Policy-Dependent Optimization Response

    Wenshuo Guo, Michael I Jordan, and Angela Zhou. Off-policy evaluation with policy-dependent optimization response.arXiv preprint arXiv:2202.12958,

    work page Pith review arXiv

  8. [2017]

    Break- ing determinism: Stochastic modeling for reliable off-policy evaluation in ad auctions.arXiv preprint arXiv:2512.03354,

    Hongseon Yeom, Jaeyoul Shin, Soojin Min, Jeongmin Yoon, Seunghak Yu, and Dongyeop Kang. Break- ing determinism: Stochastic modeling for reliable off-policy evaluation in ad auctions.arXiv preprint arXiv:2512.03354,

    work page arXiv

Show all 12 references
  1. [2019]

    Logging policy design for off-policy evaluation.arXiv preprint arXiv:2605.15108,

    Connor Douglas, Joel Persson, and Foster Provost. Logging policy design for off-policy evaluation.arXiv preprint arXiv:2605.15108,

    arXiv

  2. [2022]

    Atlantic causal inference conference (acic) data analysis challenge 2017.arXiv preprint arXiv:1905.09515,

    P Richard Hahn, Vincent Dorie, and Jared S Murray. Atlantic causal inference conference (acic) data analysis challenge 2017.arXiv preprint arXiv:1905.09515,

    2017 arXiv

  3. [2024]

    B. Li. UpliftBench: Revealing outcome-regime and objective mismatch in uplift evaluation.arXiv preprint arXiv:2608.00915,

    arXiv

  4. [2026]

    Philip Thomas and Emma Brunskill

    arXiv:2603.21485. Philip Thomas and Emma Brunskill. Data-efficient off-policy policy evaluation for reinforcement learning. In International Conference on Machine Learning (ICML),

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.