Pith. sign in

REVIEW 5 minor 1 cited by

UpliftBench: Revealing Outcome-Regime and Objective Mismatch in Uplift Evaluation

T0 review · 0 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read On the IHDP benchmark, the Qini ranking shows no detectable alignment with effect accuracy while AUUC and uplift-at-k stay informative.

desk verdict F1 is a solid, carefully-controlled empirical finding on Qini's silent failure on continuous outcomes; F2's empirical half is fragile but honestly disclosed, and the artifact deserves a serious referee. read the letter →

arxiv 2608.00915 v1 pith:DQIHEPXJ submitted 2026-08-02 cs.LG

Binshuang Li This is my paper · ORCID
classification cs.LG
keywords upliftmodelingheterogeneoustreatmenteffectscausalmachinelearningevaluationmetricsbenchmarkdesignmodelselectionpolicyQinicoefficient
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper builds UpliftBench, an outer-test-isolated benchmark of 12 uplift estimators across 25 instances from seven dataset families, to test whether published disagreements about the best uplift model come from the models or from the metric. It establishes that the disagreement is largely about metrics: on the continuous IHDP benchmark, the Qini coefficient's model ranking shows no detectable correlation with effect accuracy (mean rank correlation +0.07, 95% CI [-0.03, +0.16] across all 100 realizations), while AUUC and uplift-at-k stay informative. On the binary Jobs study, ranking metrics fail to select models for sign-threshold policies because they discard the score level, and calibrating the decision threshold removes 81% of the Qini-selection regret. Both findings are bounded: F1 appears on one of three continuous families, and F2 vanishes under a budgeted-value objective.

What carries the argument

The argument is carried by four objects. (1) The unnormalized Qini coefficient, defined by the count-corrected cumulative gain g(k)=sum y_i t_i - (sum y_i (1-t_i)) T_k/C_k integrated against a chord baseline; Lemma 5.1 proves Qini is the treated-count-weighted prefix-mean AUUC, g(k)=T_k u(k), and Lemma 5.2 separates the score into a depth-weighted contrast plus an interleaving term, localizing the IHDP divergence to treated-count weighting. (2) The rank-invariance boundary (Proposition 5.3): any metric that depends on the model only through the induced ranking is invariant to strictly increasing transformations, including shifts, whereas the sign-threshold policy pi(x)=1[tau(x)>=0] is not, so ranking metrics cannot identify the threshold-optimal model from rank information alone. (3) The multi-regime benchmark design: seven dataset families with different outcome types and reference objectives, scored under outer-test isolation with cluster bootstraps over benchmark realizations as the inferential unit. (4) A hand-checkable 12-unit counterexample where a single large control outcome makes Qini prefer a near-random model while AUUC and effect accuracy prefer the better model.

What would settle it

Estimate the actual between-split dependence of the Jobs policy-risk estimates directly, for example by a random-effects variance decomposition, and re-run the cross-repeat selection rotation; if the estimated dependence exceeds the threshold at which the paper's interval still excludes zero, the selection-regret claim includes zero and its empirical half collapses to the structural argument. For F1, a controlled sweep of the paper's own R-composite predictor across new continuous-outcome families with known effects would settle whether any ex-ante quantity can predict when Qini decouples from effect accuracy.

Watch

Extended reading notes

Core claim

The central claim is that the choice of evaluation metric, not the estimators themselves, explains much of the disagreement among published uplift benchmarks. Two distinct failures are identified. On IHDP (continuous outcomes, known effects), the unnormalized Qini's model ranking does not track effect accuracy: across all 100 realizations its mean rank correlation with effect accuracy is +0.07 [-0.03, +0.16], and as pairwise concordance it is a coin flip (0.52 [0.49, 0.56]), while AUUC shows a paired advantage of +0.49 [+0.40, +0.59]. Structurally, Qini is treated-count-weighted AUUC (Lemma 5.1), so the divergence localizes to treated-count weighting. On Jobs (binary, experimentally identified policy objective), every ranking metric correlates negatively with policy value, and metric selection is indistinguishable from random at 14-15% regret, while direct policy-risk selection beats random; this follows a rank-invariance boundary (Proposition 5.3), and calibrated thresholds remove 81% of the Qini-selection regret. The paper frames both as bounded, benchmark-discovered failure signatures, not universal laws.

Load-bearing premise

The Jobs selection-regret claim (F2) assumes the ten released re-splits are approximately exchangeable clusters: the 95% regret interval excludes zero only for between-split dependence rho <= 0.12, while per-model risk vectors across splits already correlate at +0.41, so if the true split dependence is larger, the empirical half of F2 collapses to the structural rank-invariance argument.

Editorial extensions

If this is right

  • On continuous outcomes, practitioners should not use the unnormalized Qini as the sole selection criterion: on the standard IHDP benchmark its ranking is statistically indistinguishable from random relative to effect accuracy.
  • For sign-threshold deployment rules, a ranking metric must be paired with a threshold calibrated on held-out selection data; doing so cut the Qini-selection regret on Jobs by 81%.
  • When the deployment rule is rank-based (budgeted top-k targeting), ranking metrics transmit selection signal and the F2 gap disappears, so the metric must be chosen to match the intended decision rule.
  • Benchmark winner claims are protocol-scoped: standings should be reported per regime and per objective rather than as a single leaderboard.
  • Library defaults matter: causalml computes Qini on continuous outcomes with no warning, and its shipped score behaves like the analyzed object, so the failure is silent for downstream users.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension implied by the paper's Lemma-based decomposition: any cumulative-sum functional that weights by treated count, not just Qini, should show the same decoupling on IHDP, while mean-based and depth-weighted functionals should stay aligned; rescoring stored predictions under alternative weight functions would settle whether treated-count weighting is the active ingredient.
  • If the composite predictor R (effect-to-heterogeneity ratio inflated by treatment imbalance) survives a controlled sweep, F1 could become a diagnosable condition: a practitioner could compute it ex ante from observed data and know when to distrust Qini.
  • The broader lesson extends beyond uplift modeling: whenever the scoring metric and the deployment objective are invariant to different transformations of the predicted scores, benchmark rankings can decouple from decision quality.
  • A direct stress test of F2's scope would replace the ten re-splits with genuinely independent randomized experiments; if ranking-metric selection still fails to beat random selection there, the regret figure becomes a general property of rank-only selection for threshold policies.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

0 major / 5 minor

Summary. The paper introduces UpliftBench, an outer-test-isolated benchmark of 12 uplift estimators across seven dataset families, scored by six objectives including ranking metrics, effect accuracy, and policy objectives. It reports two main findings. F1: on the semi-synthetic continuous IHDP benchmark, the unnormalized Qini ranking shows no detectable alignment with true effect accuracy (mean rank correlation +0.07, 95% CI [-0.03, +0.16] over all 100 IHDP realizations), while AUUC and uplift-at-k remain informative; the paired AUUC-over-Qini gap is +0.49 [+0.40, +0.59] and survives estimator exclusions, base-learner swaps, and tuning-objective changes. F2: on the Jobs benchmark, ranking metrics are structurally insufficient for sign-threshold policies because they discard score level (Proposition 5.3), and a within-sample split-rotation analysis suggests direct policy-risk selection has lower benchmark regret than Qini/AUUC/uplift-at-k selection, with threshold calibration removing 81% of Qini-selection regret. Both findings are explicitly bounded: F1 does not replicate on ACIC or Revenue-Synthetic, and F2 vanishes under a budgeted-value objective. The paper releases versioned loaders, protocol code, result artifacts, and a leaderboard.

Significance. The paper is a careful, well-scoped empirical study of a question that matters for benchmark design: metric choice, not just estimator quality, can drive published conclusions. F1's core statistic is strong: the paired AUUC-over-Qini gap is positive across all 100 IHDP realizations with confidence intervals excluding zero, and the result is tested against a wide set of confounders, including a Qini-tuned candidate panel, estimator exclusions, and implementation variants. The reference objectives are external ground-truth quantities, so the comparison is not circular. F2's structural proposition is correct, and the threshold-calibration remedy is practically actionable. The empirical half of F2 is appropriately labeled in the body as a descriptive within-sample case study, and the paper honestly discloses the sensitivity of its confidence intervals to between-split dependence. The reproducibility package appears unusually complete, with versioned data loaders, fixed protocols, git-hash-stamped result schemas, and documented regeneration targets.

minor comments (5)
  1. [Abstract and Contribution (4)] The abstract states as an empirical result that direct policy-risk selection 'yields lower benchmark regret than random model selection while Qini, AUUC, and uplift-at-k do not', but the body's own design-effect sensitivity shows the regret interval excludes zero only for between-split dependence rho-bar <= 0.12, with raw cross-split risk correlations around +0.41. I recommend using the same descriptive-case-study register in the abstract and contribution list as in Section 5.2, so the headline matches the disclosed evidential strength.
  2. [Appendix E.1] The family-level ordering of the composite R (IHDP 3.52, ACIC 1.07, Revenue-Synthetic 0.39) is described as a 'one-in-six chance event'; because this comparison was selected after the gradient was observed, the one-in-six framing overstates its evidential value. The text later notes the absence of pre-registration credit, but the earlier sentence should be rephrased to avoid a post-hoc probability claim.
  3. [Appendix I] The statement that the 10K and 100K Criteo orderings 'did not agree' is based on a single pair of non-nested stratified subsamples, and the paper correctly interprets this as a resolution limitation. The wording could nonetheless more sharply distinguish 'limited resolution' from 'evidence of misranking' in the opening sentences of the resolution paragraph.
  4. [Figure 3 and Table 8] The labels 'Student-t3' and 'Log-n.' render awkwardly and should be typeset with proper mathematical notation, e.g. Student-t_3 and log-normal, to match journal conventions.
  5. [Section 5.1 and Table 6] The sentence 'F1 is an artifact of neither metric's selection' is supported by the tuning-off and AUUC-tuning arms, but the reader must wait until Appendix E.2 for the actual numbers. Consider moving the one-sentence summary of Table 5 into the main text, since the Qini-tuned panel is a natural initial concern.

Circularity Check

0 steps flagged · score 0.0 of 10

No meaningful circularity: the findings are empirical comparisons against external reference objectives, and the one conditioning choice (Qini-tuned panel) is controlled by explicit reruns.

full rationale

UpliftBench's two findings are empirical benchmark comparisons against external reference objectives: F1 uses √PEHE on IHDP, where true effects are known from the simulation, and F2 uses RCT-estimated policy risk on Jobs. Neither reference objective is defined in terms of the metrics under test, and no metric value is fitted and then renamed as a prediction. F1's comparison of Qini, AUUC, and uplift-at-k is computed on identical out-of-fold predictions, so the AUUC-over-Qini gap is a measured difference between functionals, not a construction artifact. Lemma 5.1 is an exact identity relating Qini to a treated-count-weighted AUUC; it is used to localize the discrepancy after the fact, not to derive the finding, and the paper explicitly treats the mechanism as open. F2's structural claim (Prop. 5.3) is a standard rank-invariance non-identifiability result with an elementary proof; the empirical half separates selection and evaluation across repeated-CV seeds and is honestly disclosed as within-sample, with a design-effect sensitivity bounding how much the cluster bootstrap can claim. The one place the protocol conditions the candidate panel on a scrutinized metric — inner tuning by Qini — is disclosed in Section 7 and controlled by rerunning with tuning off and with AUUC as the inner objective; F1 persists in both arms, so the finding is not induced by the tuning choice. There are no load-bearing self-citations, no imported uniqueness theorem, and no ansatz smuggled in via citation; the paper even declines to claim the F2 observation as wholly new. Limitations are stated explicitly rather than hidden. Overall, the derivation chain is self-contained against external benchmarks and no reported statistic reduces by construction to a fitted input or a redefinition.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The benchmark conclusions rest on standard semi-synthetic benchmark assumptions (known potential outcomes in IHDP, an unbiased IPW policy-risk estimate on the randomized Jobs subset) and on the correctness of the outer-test-isolated protocol. The F2 inference additionally assumes approximate exchangeability of the ten Jobs re-splits, which the paper tests with a design-effect sensitivity. No free parameters are fitted to produce the findings; protocol constants (fold counts, seeds, tuning budgets, k=0.3) are fixed and disclosed.

assumptions (4)
  • domain assumption IHDP semi-synthetic potential outcomes provide valid ground truth for computing √PEHE as effect accuracy.
    F1's reference objective is −√PEHE on IHDP; if the simulated potential outcomes were invalid, the metric comparison would have no meaningful ground truth.
  • domain assumption On Jobs, the RCT-estimated IPW policy risk following Shalit et al. is an unbiased estimator of sign-threshold policy value.
    F2 uses policy risk as the deployment objective; the paper adopts the established benchmark definition with the experimental treated fraction.
  • domain assumption The cluster bootstrap treats the ten Jobs re-splits as exchangeable clusters; the CI excludes zero only for between-split dependence ρ̄ ≤ 0.12.
    The F2 selection-regret interval is sensitive to this assumption, which the paper quantifies explicitly.
  • domain assumption The outer-test-isolated protocol is correctly implemented so no test-fold outcomes influence preprocessing, propensity estimation, or tuning; the one disclosed deviation involves covariates/treatment only inside the tuning loop.
    If test-fold information leaked, the reported metric comparisons could be biased; the paper discloses the partial deviation and its scope.

how reviews work

0 comments
Cite this review

Pith. "Pith review of UpliftBench: Revealing Outcome-Regime and Objective Mismatch in Uplift Evaluation." pith.science (2026). https://pith.science/paper/DQIHEPXJ

@misc{pith2026260800915,
  author       = {Pith},
  title        = {Pith review of: UpliftBench: Revealing Outcome-Regime and Objective Mismatch in Uplift Evaluation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DQIHEPXJ}},
  note         = {Machine review of arXiv:2608.00915}
}
abstract

Uplift modeling (conditional-average-treatment-effect estimation) drives personalized targeting, yet published uplift benchmarks frequently disagree on which estimator performs best; we show the disagreement is substantially about metrics, not models. UpliftBench evaluates 12 uplift estimators under an outer-test-isolated, multi-objective protocol across seven dataset families; its two findings are identified where a reference objective exists -- F1 on the standard continuous benchmark (IHDP), F2 in a within-sample case study on Jobs. On that benchmark, Qini shows no detectable alignment with effect accuracy -- across all 100 IHDP realizations its mean rank correlation with effect accuracy is +0.07, 95% CI [-0.03, +0.16] -- while AUUC is consistently more aligned (paired prefix-mean-AUUC-over-Qini gap +0.49 [+0.40, +0.59]; the shipped cumulative-gain AUUC aligns better still, +0.73). On Jobs, ranking metrics are structurally insufficient for a sign-threshold policy because they discard the score level; empirically, within the released split-rotation analysis direct policy-risk selection yields lower benchmark regret than random model selection while Qini, AUUC, and uplift-at-$k$ do not (14-15% regret). Calibrating the decision threshold removes 81% of the Qini-selection regret. Both findings are bounded, not universal: F1 is not detected on either validation family (the ACIC and Revenue-Synthetic gaps are both indistinguishable from zero), and F2 vanishes under a budgeted-value objective where rank suffices. UpliftBench releases versioned loaders, fixed protocols, result artifacts, and a reproducible living leaderboard; the public repository accompanies the paper.

Figures

Figures reproduced from arXiv: 2608.00915 by the authors.

Figure 1
Figure 1. Two findings of metric disagreement. Mean [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. The findings are robust to the base learner. Switching LightGBM [PITH_FULL_IMAGE:figures/full_fig_p010_2.png] view at source ↗
Figure 3
Figure 3. Controlled probes of the F1 failure signature (controlled DGP; 8 fixed models of graded quality, latent CATE fixed, [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Qini (z-scored within dataset) vs. calibration ECE, by regime, shown [PITH_FULL_IMAGE:figures/full_fig_p015_4.png]
Figure 5
Figure 5. Figure 5: Policy value vs. targeting budget 𝑘 (marketing RCTs). The budget-optimal model changes with 𝑘 — e.g. on Hillstrom the best model shifts from ClassTrans at 𝑘=0.1 to SoloModel at 𝑘=0.9 — so no single ranking fixes the deployment choice. SD(𝜏), used to identify, not to re…
Figure 6
Figure 6. Figure 6: Within IHDP, outcome kurtosis does not predict [PITH_FULL_IMAGE:figures/full_fig_p017_6.png]
Figure 7
Figure 7. Figure 7: Per-regime leaderboard, each panel scored by its appropriate metric (color = within-dataset rank, green best); “—” [PITH_FULL_IMAGE:figures/full_fig_p020_7.png]
Figure 8
Figure 8. Figure 8: Supporting analyses for Section 5.3–5.2. [PITH_FULL_IMAGE:figures/full_fig_p020_8.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. When Can You Trust Offline Evaluation of Equal-Cost Top-k Allocation? A Controlled, Reproducible Benchmark and Practitioner's Guide

    cs.LG 2026-08 accept novelty 5.5 of 10

    Offline evaluation of deterministic top-k allocation is trustworthy only under logger-target action alignment and credible propensities; nuisance-only cross-fitting worsens the optimizer's curse, and propensity-estima...

Reference graph

Works this paper leans on

42 extracted references · 35 canonical work pages · cited by 1 Pith paper

  1. [1]

    Keith Battocchi, Eleanor Dillon, Maggie Hei, Greg Lewis, Paul Oka, Miruna Oprescu, and Vasilis Syrgkanis. 2019. EconML: A Python Package for ML- Based Heterogeneous Treatment Effects Estimation. https://github.com/py- why/EconML

    work page 2019

  2. [2]

    Björn Bokelmann and Stefan Lessmann. 2024. Improving uplift model evaluation on randomized controlled trial data.European Journal of Operational Research 313, 2 (2024), 691–707

    work page 2024

  3. [3]

    Huigang Chen, Totte Harinen, Jeong-Yoon Lee, Mike Yung, and Zhenyu Zhao

  4. [4]

    Alicia Curth, David Svensson, James Weatherall, and Mihaela van der Schaar

    work page

  5. [5]

    Janez Demšar. 2006. Statistical comparisons of classifiers over multiple data sets. Journal of Machine Learning Research7 (2006), 1–30

    work page 2006

  6. [6]

    Floris Devriendt, Jeroen Berrevoets, and Wouter Verbeke. 2021. Why you should stop predicting customer churn and start using uplift models.Information Sciences 548 (2021), 497–515

    work page 2021

  7. [7]

    Floris Devriendt, Darie Moldovan, and Wouter Verbeke. 2018. A literature survey and experimental evaluation of the state-of-the-art in uplift modeling: A stepping stone toward the development of prescriptive analytics.Big Data6, 1 (2018), 13–41

    work page 2018

  8. [8]

    Eustache Diemert, Artem Betlei, Christophe Broisin, and Massih-Reza Amini

    work page

Show all 42 references
  1. [9]

    Vincent Dorie, Jennifer Hill, Uri Shalit, Marc Scott, and Dan Cervone. 2019. Automated versus do-it-yourself methods for causal inference: Lessons learned from a data analysis competition.Statist. Sci.34, 1 (2019), 43–68

    2019

  2. [10]

    Leo Guelman, Montserrat Guillén, and Ana M Pérez-Marín. 2015. Uplift random forests.Cybernetics & Systems46, 3-4 (2015), 230–248

    2015

  3. [11]

    Pierre Gutierrez and Jean-Yves Gérardy. 2017. Causal inference and uplift mod- elling: A review of the literature.ICML Workshop on Predictive Causality67 (2017), 1–13

    2017

  4. [12]

    Bowei He, Yunpeng Weng, Xing Tang, Ziqiang Cui, Zexu Sun, Liang Chen, Xiuqiang He, and Chen Ma. 2024. Rankability-enhanced revenue uplift model- ing framework for online marketing. InProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD)

    2024

  5. [13]

    Jennifer L Hill. 2011. Bayesian nonparametric modeling for causal inference. Journal of Computational and Graphical Statistics20, 1 (2011), 217–240

    2011

  6. [14]

    Kevin Hillstrom. 2008. Mine that data! Kevin Hillstrom’s email marketing chal- lenge. https://blog.minethatdata.com/2008/03/minethatdata-e-mail-analytics- and-data.html

    2008

  7. [15]

    Maciej Jaskowski and Szymon Jaroszewicz. 2012. Uplift modeling for clinical trial data.ICML Workshop on Clinical Data Analysis(2012)

    2012

  8. [16]

    Kevin Kane, Victor SY Lo, and Jianying Zheng. 2014. Mining for the truly re- sponsive customers and prospects using true-lift modeling.Journal of Marketing Analytics2, 4 (2014), 218–238. UpliftBench: Revealing Outcome-Regime and Objective Mismatch in Uplift Evaluation

    2014

  9. [17]

    Sayash Kapoor and Arvind Narayanan. 2023. Leakage and the reproducibility crisis in machine-learning-based science.Patterns4, 9 (2023)

    2023

  10. [18]

    Guolin Ke, Qi Meng, Thomas Finley, Taifeng Wang, Wei Chen, Weidong Ma, Qiwei Ye, and Tie-Yan Liu. 2017. LightGBM: A highly efficient gradient boosting decision tree.Advances in Neural Information Processing Systems30 (2017)

    2017

  11. [19]

    Edward H Kennedy. 2020. Optimal doubly robust estimation of heterogeneous causal effects.arXiv preprint arXiv:2004.14497(2020)

    2020 arXiv

  12. [20]

    Volodymyr Kuleshov, Nathan Fenner, and Stefano Ermon. 2018. Accurate uncer- tainties for deep learning using calibrated regression.International Conference on Machine Learning(2018), 2796–2804

    2018

  13. [21]

    Sören R Künzel, Jasjeet S Sekhon, Peter J Bickel, and Bin Yu. 2019. Metalearners for estimating heterogeneous treatment effects using machine learning.Proceed- ings of the National Academy of Sciences116, 10 (2019), 4156–4165

    2019

  14. [22]

    Robert J LaLonde. 1986. Evaluating the econometric evaluations of training programs with experimental data.The American Economic Review76, 4 (1986), 604–620

    1986

  15. [23]

    Yan Leng and Drew Dimmery. 2024. Calibration of Heterogeneous Treatment Effects in Randomized Experiments.Information Systems Research(2024)

    2024

  16. [24]

    Lenta. 2022. Lenta Uplift Modelling Dataset. https://www.uplift-modeling.com/ en/latest/api/datasets/fetch_lenta.html

    2022

  17. [25]

    Divyat Mahajan, Ioannis Mitliagkas, Brady Neal, and Vasilis Syrgkanis. 2024. Empirical analysis of model selection for heterogeneous causal effect estimation. InInternational Conference on Learning Representations (ICLR)

    2024

  18. [26]

    Nikita Maksimov, Anvar Kurmukov, and Elena Shevchenko. 2020. scikit-uplift: uplift modeling in scikit-learn style in Python. https://github.com/maks-sh/scikit- uplift

    2020

  19. [27]

    MegaFon. 2021. MegaFon Uplift Competition Dataset. https://www.uplift- modeling.com/en/latest/api/datasets/fetch_megafon.html

    2021

  20. [28]

    Xinkun Nie and Stefan Wager. 2021. Quasi-oracle estimation of heterogeneous treatment effects.Biometrika108, 2 (2021), 299–319

    2021

  21. [29]

    Nicholas J Radcliffe and Patrick D Surry. 2007. Using control groups to target on predicted lift: Building and assessing uplift models.Direct Marketing Analytics Journal1 (2007), 14–21

    2007

  22. [30]

    Piotr Rzepakowski and Szymon Jaroszewicz. 2012. Decision trees for uplift modeling with single and multiple treatments.Knowledge and Information Systems32, 2 (2012), 303–327

    2012

  23. [31]

    Uri Shalit, Fredrik D Johansson, and David Sontag. 2017. Estimating individual treatment effect: generalization bounds and algorithms.International Conference on Machine Learning(2017), 3076–3085

    2017

  24. [32]

    Lars van der Laan, Ernesto Ulloa-Rivero, Marco Carone, and Alex Luedtke. 2023. Causal Isotonic Calibration for Heterogeneous Treatment Effects. InProceedings of the 40th International Conference on Machine Learning (ICML) (PMLR, Vol. 202)

    2023

  25. [33]

    Brecht Verbeken, Marie-Anne Guerry, Wouter Verbeke, and Sam Verboven. 2025. Uplift model evaluation with ordinal dominance graphs.Journal of Machine Learning Research26 (2025)

    2025

  26. [34]

    Stefan Wager and Susan Athey. 2018. Estimation and inference of heterogeneous treatment effects using random forests.J. Amer. Statist. Assoc.113, 523 (2018), 1228–1242

    2018

  27. [35]

    X5 Group. 2021. X5 Retail Group Uplift Dataset. https://www.uplift-modeling. com/en/latest/api/datasets/fetch_x5.html

    2021

  28. [36]

    Steve Yadlowsky, Scott Fleming, Nigam Shah, Emma Brunskill, and Stefan Wager

  29. [37]

    Yuxuan Yang, Dugang Liu, and Yiyan Huang. 2026. Evaluating uplift modeling under structural biases: Insights into metric stability and model robustness. In Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD)

    2026

  30. [38]

    simple learners lead

    Minqin Zhu, Zexu Sun, Ruoxuan Xiong, Anpeng Wu, Baohong Li, Caizhi Tang, Jun Zhou, Fei Wu, and Kun Kuang. 2025. Rethinking causal ranking: A balanced perspective on uplift model evaluation. InProceedings of the 42nd International Conference on Machine Learning (ICML). A Deferr...

    2025

  31. [2018]

    A large scale benchmark for uplift modeling.KDD Workshop on Causal Discovery, Prediction and Decision(2018)

    2018

  32. [2020]

    CausalML: Python package for causal machine learning.arXiv preprint arXiv:2002.11631(2020)

    2020 arXiv

  33. [2021]

    InAdvances in Neural Information Processing Systems (NeurIPS) Datasets and Benchmarks Track

    Really Doing Great at Estimating CATE? A Critical Look at ML Benchmark- ing Practices in Treatment Effect Estimation. InAdvances in Neural Information Processing Systems (NeurIPS) Datasets and Benchmarks Track

  34. [2025]

    Evaluating Treatment Prioritization Rules via Rank-Weighted Average Treatment Effects.J. Amer. Statist. Assoc.120, 549 (2025), 38–51

    2025

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.