Pith. sign in

REVIEW 3 major objections 4 minor 50 references

Uplift Modeling for Multiple Treatments with Cost Optimization

T0 review · 3 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper extends X- and R-learners to multiple treatments and adds cost-aware net-value optimization, improving net value in synthetic and real experiments.

desk verdict Useful multi-arm cost-aware uplift extension with a correct X-Learner, but the net-value R-Learner in Eq. 16 is not a valid R-Learner and the multi-arm R-Learner is under-specified; the paper is worth a conditional referee pass. read the letter →

arxiv 1908.05372 v3 pith:OF66GC2Y submitted 2019-08-14 stat.ML cs.LGstat.AP

Zhenyu Zhao , Totte Harinen This is my paper · ORCID
classification stat.MLcs.LGstat.AP
keywords upliftmodelingconditionalaveragetreatmenteffectmeta-learnersmulti-treatmentcost-sensitiveoptimizationnetvalueA/Btestingtargetingpolicy
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that two meta-learner algorithms for estimating individual treatment effects, the X-Learner and the R-Learner, can be extended from single-treatment experiments to multi-treatment experiments, and that the extended versions outperform existing multi-treatment uplift methods on both synthetic and real data. It further claims that when treatments carry different costs, modifying these learners to optimize net value—conversion value minus impression and triggered costs—produces targeting policies with higher average net value than standard uplift models or assigning everyone the same treatment. This matters because many industry experiments run multiple arms with heterogeneous costs, and existing uplift tooling has largely ignored both complications. If the extensions work as claimed, practitioners can reuse standard regression and classification models to personalize treatment choice while explicitly trading off cost and conversion.

What carries the argument

The load-bearing mechanism is the pseudo-effect construction of the X-Learner and the residual orthogonalization of the R-Learner. In the X-Learner, treatment-arm and control observations are used to make counterfactual predictions, and the differences become pseudo-outcomes that a second-stage regressor learns; the final CATE is a propensity-weighted blend of these regressions. In the R-Learner, the outcome is centered by a mean-outcome estimator and the treatment indicator by its propensity, so the CATE is fit on the residualized signal. The paper's net-value modification plugs $(v-s_t)Y - c_t$ into these constructions, making a net-value CATE the target of optimization.

What would settle it

Generate synthetic data with known net-value CATEs per arm and compare NV-R-Learner estimates to them, varying the gap between per-arm triggered costs and the global average cost; if estimation bias grows with that gap, the objective in Equation (16) is not identifying the net-value CATE.

Watch

Extended reading notes

Core claim

The central discovery is a cost-aware, multi-treatment extension of meta-learner uplift modeling. For the X-Learner, the paper estimates response functions for each treatment arm, forms pseudo-effects comparing each arm against control, and combines the resulting CATE estimates with arm-specific propensity weights. For the R-Learner, it plugs per-arm propensity and outcome estimators into the residual minimization objective. To incorporate costs, both learners replace the raw outcome with net value $(v-s_t)Y - c_t$, where $v$ is the conversion value, $s_t$ the triggered cost, and $c_t$ the impression cost of arm $t$, so the estimand becomes a net-value CATE rather than a conversion CATE. In synthetic and real promotion experiments, the net-value variants yield larger average net value per user than standard meta-learners and than any fixed treatment or control.

Load-bearing premise

The paper assumes without proof that the modified R-Learner objective in Equation (16) correctly estimates the net-value conditional average treatment effect.

Editorial extensions

If this is right

  • Multi-arm uplift can be built by wrapping existing regression and classification libraries, since the extended meta-learners require only standard base models.
  • Targeting policies can now decide per user between no treatment, treatment A, or treatment B, selecting the arm with the highest predicted net-value uplift.
  • Costs enter as two separate components—fixed impression cost and conversion-triggered cost—so promotions with high redemption costs are naturally penalized in the optimization.
  • In the reported real-data experiment, the net-value models improve average net value even when they do not improve conversion rate, reflecting the explicit cost trade-off.
  • The extended learners are computationally faster than the published decision-tree and Contextual Treatment Selection baselines in the paper's experiments.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The net-value R-Learner objective in Equation (16) is justified only heuristically: it subtracts a global-average cost term rather than a regression of the net-value outcome, so a formal identification proof or a corrected estimator is the natural next step.
  • The same cost-adjusted pseudo-effect construction should transfer to newer meta-learners, such as doubly robust learners, and to continuous outcomes where the conversion value becomes a quantity rather than a constant.
  • In multi-arm settings without a control, the paper's pairwise-majority-vote recommendation ignores that arms may have different propensity weights; a direct multi-arm CATE formulation would likely be more reliable.
  • A production takeaway is that storing treatment tags, features, and outcome labels together lets the net-value framework be applied to new experiments without changing the online serving layer.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper extends two meta-learner uplift models, the X-Learner and the R-Learner, to the multi-treatment setting and additionally proposes "net value" variants that incorporate per-treatment impression and triggered costs. The proposed models are evaluated on synthetic and real-world experimental data, with the claimed contributions being multi-treatment support, cost-aware net-value optimization, an empirical comparison against existing multi-treatment methods, and a description of a production implementation at Uber.

Significance. If the technical claims are correct, the paper would provide a useful practical extension of uplift modeling to a common industry setting: experiments with several treatment arms and heterogeneous costs. The empirical study is fairly broad (synthetic two-arm, multi-arm with and without control, net-value scenarios, and a large real dataset), and the description of a production platform is valuable for practitioners. The X-Learner extension is straightforward and appears sound. The main technical risk is the proposed net-value R-Learner objective, which is not derived and appears not to identify the claimed net-value CATE; this concern directly affects the validity of the reported NV-R-Learner results.

major comments (3)
  1. [§VI.B.2, Eq. (16)] The net-value R-Learner objective in Eq. (16) is not derived and, as written, does not identify the net-value CATE defined in Eqs. (13)–(14). For a multi-arm trial, the net-value outcome is Z_i = (v - s_{A_i})Y_i - c_{A_i}, and a valid R-Learner would center Z_i by its conditional mean m_Z(x) = Σ_k e_{t_k}(x)[(v - s_{t_k})μ_{t_k}(x) - c_{t_k}], producing the residual (Z_i - m_Z^{(-i)}(X_i)) - (W_i - e^{(-i)}(X_i))τ(X_i). Eq. (16) instead centers using (v - \bar{s})\hat{m}^{(-i)}(X_i) - \bar{c}, where \hat{m} is a regression estimate of E[Y|X] and \bar{s}, \bar{c} are global sample averages of costs. The algebraic difference between m_Z(x) and (v - \bar{s})m(x) - \bar{c} is -Σ_k e_{t_k}(x)(\bar{s} - s_{t_k})μ_{t_k}(x) - [\bar{c} - Σ_k e_{t_k}(x)c_{t_k}], which is generally nonzero when costs differ across arms and conditional outcome means are heterogeneous. Consequently, the Eq. (16) objective does not estimate the net-value CATE, and the reported NV-R-Learner improvements in Figs. 5 and 6 cannot be attributed to estimating the claimed target unless this identification issue is resolved.
  2. [§VI.A.2] The multi-treatment R-Learner is never formally defined. The text states that propensity scores \hat{e}_{t_j}(x) and mean outcomes \hat{m}_{t_j}(x) are estimated and plugged into Eq. (16), but Eq. (16) contains a single scalar W_i and a single scalar e, which are meaningful only in a two-arm comparison. No definition is given for W_i or for e(x) in a design with more than two arms, and it is unclear whether each treatment is compared against control in a one-versus-rest fashion, in paired comparisons, or via some other encoding. Without this definition, the proposed multi-treatment R-Learner in Section VI.A.2 cannot be implemented or evaluated, and the empirical results labeled R-Learner and NV-R-Learner are not backed by a well-specified algorithm.
  3. [§V.A and §VII.A.3] The paper advertises support for “multiple treatments without a control,” but the proposed X-Learner and R-Learner extensions in Section VI are all formulated relative to a control group t_0. In the no-control experiment, the authors fall back on pairwise comparisons with majority voting, which is not described in the method section and is not a consequence of the proposed extensions. This makes the empirical comparison in Figure 4 difficult to interpret as an evaluation of the proposed methods, and it leaves the claimed no-control capability unsupported.
minor comments (4)
  1. [§VI.A.2 and §VI.B.2] The text in Section VI.A.2 refers to “Equation 16” before Eq. (16) is introduced in Section VI.B.2; the standard multi-treatment R-Learner objective is missing, and the equation numbering should be adjusted so that the reference points to an existing equation.
  2. [Eq. (12)] The superscript notation \hat{e}^{(-1)}_{t_j} is never defined; presumably it means leave-one-out without observation i, which should be written as \hat{e}^{(-i)}_{t_j} to avoid confusion with an exponent.
  3. [Eq. (15)] The notation x_{t_j}^i and Y_{t_j}^i is introduced but not clearly defined; in particular, it is not obvious that the superscript indicates the group from which the observation originates, and the pseudo-effect construction would benefit from explicit indexing of the i-th observation.
  4. [Throughout] There are typographical and stylistic errors, such as “benefit” in the abstract, “overperform” in Section VII.A.4, and inconsistent naming of treatment arms (e.g., “Treatment 3” vs. t_3); these should be corrected in a revision.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the meta-learner extensions are constructed from prespecified costs and outcomes and evaluated on held-out data.

full rationale

The paper's claimed extensions are self-contained. Section VI.A defines the multi-treatment X-Learner by substituting group-specific response functions and propensity scores into the standard X-Learner construction (Eqs. 8-12), and the net-value version uses the same pseudo-outcome logic with the prespecified value v and costs c_tj, s_tj (Eq. 15). Section VI.B.2 states the net-value R-Learner objective (Eq. 16); even if that objective is misspecified because it centers with global average costs rather than the conditional net-value mean and does not define W_i/e(x) for more than two arms, misspecification is a correctness risk, not a circular reduction: the output is not statistically forced to be the input. All evaluations are on held-out synthetic data and a held-out real experiment sample, with costs fixed by the experimenter rather than fitted to the outcome. The only sense in which net-value models outperform standard learners is that the latter intentionally ignore costs; that is an expected property of the problem setup, not an equivalence between derivation and input. Citations to Künzel et al. and Nie and Wager are to established external meta-learner frameworks and are not used to import a uniqueness theorem. No load-bearing self-citation or renamed fit was found.

Assumptions & free parameters 2 free parameters · 5 assumptions · 0 invented entities

The central claim depends on standard causal assumptions, known costs and conversion value, the correctness of the R-Learner extension, and the validity of the evaluation metrics. The R-Learner net-value objective is an ad hoc assumption not derived from the standard R-Learner loss.

free parameters (2)
  • Conversion value v = 1 (assumed constant)
    Used to define net value in Eqs. 13-15. Not estimated from data and treated as known prior, but the optimization results and the net value CATE target depend on this choice.
  • Treatment costs (impression costs c_tj and triggered costs s_tj) = Set per scenario, e.g., s_t1=0.01*c_m and c_t3=0.1*c_m in synthetic trials
    Assumed known constants in both synthetic and real experiments. If misspecified, the estimated net value CATE changes and the reported policy advantages may not hold.
assumptions (5)
  • domain assumption Neyman-Rubin potential outcomes framework with randomized or unconfounded treatment assignment
    Used throughout Section III to define CATE and to justify estimating uplift from experiment data.
  • domain assumption Conversion value v and treatment costs are known constants, independent of X
    Eqs. 13-16 use v, c_tj, and s_tj as fixed inputs; no uncertainty or heterogeneity in costs or value is modeled.
  • ad hoc to paper The objective in Eq. 16 identifies the net value CATE
    The R-Learner net-value loss is asserted without proof; the residual uses (v - \bar s)\hat m and (c_i - \bar c) rather than a net-value conditional mean, so this is an unverified modeling assumption.
  • domain assumption Synthetic data generation (Madelon-style features, additive uplift update Y = min(Y + Y', 1)) approximates real uplift structure
    Section VII.A. The relative algorithm ranking on synthetic data may not transfer to real settings.
  • domain assumption AUUC and matched-assignment net value differences are unbiased policy value metrics
    Used for evaluation in Section VII. This requires known propensities or equal randomization and no selection from conditioning on model recommendations.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Uplift Modeling for Multiple Treatments with Cost Optimization." pith.science (2026). https://pith.science/paper/OF66GC2Y

@misc{pith2026190805372,
  author       = {Pith},
  title        = {Pith review of: Uplift Modeling for Multiple Treatments with Cost Optimization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OF66GC2Y}},
  note         = {Machine review of arXiv:1908.05372}
}
read the original abstract

Uplift modeling is an emerging machine learning approach for estimating the treatment effect at an individual or subgroup level. It can be used for optimizing the performance of interventions such as marketing campaigns and product designs. Uplift modeling can be used to estimate which users are likely to benefit from a treatment and then prioritize delivering or promoting the preferred experience to those users. An important but so far neglected use case for uplift modeling is an experiment with multiple treatment groups that have different costs, such as for example when different communication channels and promotion types are tested simultaneously. In this paper, we extend standard uplift models to support multiple treatment groups with different costs. We evaluate the performance of the proposed models using both synthetic and real data. We also describe a production implementation of the approach.

Figures

Figures reproduced from arXiv: 1908.05372 by the authors.

Figure 1
Figure 1. An illustration of the relationship between the response variable and different types of features: an informative classification feature influencing the [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. The uplift curves for various algorithms under the two-arm setting. [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 4
Figure 4. The Results of the No-control (Three-Arm) Optimisation Task. The [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figures from the paper (3 more)
Figure 5
Figure 5. Figure 5: Net Value Optimization with Varying Cost. In the net value optimization task, the net value meta-learner models show better results than the standard [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Average Net Value Per User with 95% Confidence Interval in Real Data Example. Both net value models NV-Rlearner and NV-Xlearner perform significantly better than other models and original experiment groups. are 0.53% in Control, 1.39% in Treatment 1 and 2.18% in Treatm…
Figure 7
Figure 7. Figure 7: Flowchart for Uplift Modeling Implementation for Online and Offline [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

50 extracted references · 47 canonical work pages

  1. [1]

    Uplift random forests,

    L. Guelman, M. Guill ´en, and A. M. P ´erez-Mar´ın, “Uplift random forests,” Cybern. Syst., vol. 46, no. 3-4, pp. 230–248, May 2015

    work page 2015

  2. [2]

    Causal inference and uplift modeling a review of the literature,

    P. Gutierrez and J.-Y . Gerardy, “Causal inference and uplift modeling a review of the literature,” JMLR: Workshop and Conference Proceedings 67, 2016

    work page 2016

  3. [3]

    Meta-learners for estimating heterogeneous treatment effects using machine learning,

    S. R. K ¨unzel, J. S. Sekhon, P. J. Bickel, and B. Yu, “Meta-learners for estimating heterogeneous treatment effects using machine learning,” Jun. 2017

    work page 2017

  4. [4]

    Decision trees for uplift modeling with single and multiple treatments,

    P. Rzepakowski and S. Jaroszewicz, “Decision trees for uplift modeling with single and multiple treatments,” Knowl. Inf. Syst. , vol. 32, no. 2, pp. 303–327, Aug. 2012

    work page 2012

  5. [5]

    Ensemble methods for uplift modeling,

    M. Sołtys, S. Jaroszewicz, and P. Rzepakowski, “Ensemble methods for uplift modeling,” Data Min. Knowl. Discov. , vol. 29, no. 6, pp. 1531– 1559, Nov. 2015

    work page 2015

  6. [6]

    Estimation and inference of heterogeneous treatment effects using random forests,

    S. Wager and S. Athey, “Estimation and inference of heterogeneous treatment effects using random forests,” Oct. 2015

    work page 2015

  7. [7]

    Support vector machines for uplift modeling,

    L. Zaniewicz and S. Jaroszewicz, “Support vector machines for uplift modeling,” in 2013 IEEE 13th International Conference on Data Mining Workshops, Dec. 2013, pp. 131–138

    work page 2013

  8. [8]

    Uplift modeling with multiple treatments and general response types,

    Y . Zhao, X. Fang, and D. Simchi-Levi, “Uplift modeling with multiple treatments and general response types,” May 2017

    work page 2017

Show all 50 references
  1. [9]

    Online controlled experiments at large scale,

    R. Kohavi, A. Deng, B. Frasca, T. Walker, Y . Xu, and N. Pohlmann, “Online controlled experiments at large scale,” in Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining . ACM, 2013, pp. 1168–1176

    2013

  2. [10]

    Con- trolled experiments on the web: survey and practical guide,

    R. Kohavi, R. Longbotham, D. Sommerfield, and R. M. Henne, “Con- trolled experiments on the web: survey and practical guide,”Data mining and knowledge discovery , vol. 18, no. 1, pp. 140–181, 2009

    2009

  3. [11]

    Inform product change through exper- imentation with data-driven behavioral segmentation,

    Z. Zhao, Y . He, and M. Chen, “Inform product change through exper- imentation with data-driven behavioral segmentation,” in Data Science and Advanced Analytics (DSAA), 2017 IEEE International Conference on. IEEE, 2017, pp. 69–78

    2017

  4. [12]

    Quasi-Oracle estimation of heterogeneous treat- ment effects,

    X. Nie and S. Wager, “Quasi-Oracle estimation of heterogeneous treat- ment effects,” Dec. 2017

    2017

  5. [13]

    Hive: a warehousing solution over a map- reduce framework,

    A. Thusoo, J. S. Sarma, N. Jain, Z. Shao, P. Chakka, S. Anthony, H. Liu, P. Wyckoff, and R. Murthy, “Hive: a warehousing solution over a map- reduce framework,” Proceedings of the VLDB Endowment, vol. 2, no. 2, pp. 1626–1629, 2009

    2009

  6. [14]

    Incremental value modeling,

    B. Hansotia and B. Rukstales, “Incremental value modeling,” Research Council Journal, 2001

    2001

  7. [15]

    Using control groups to target on predicted lift: Building and assessing uplift models,

    N. J. Radcliffe, “Using control groups to target on predicted lift: Building and assessing uplift models,” Direct Market J Direct Market Assoc Anal Council, vol. 1, pp. 14–21, 2007

    2007

  8. [16]

    Recursive partitioning for heterogeneous causal effects,

    S. Athey and G. Imbens, “Recursive partitioning for heterogeneous causal effects,” Apr. 2015

    2015

  9. [17]

    Subgroup analysis via recursive partitioning,

    X. Su, “Subgroup analysis via recursive partitioning,” J. Mach. Learn. Res., vol. 10, pp. 141–158, 2009

    2009

  10. [18]

    Modeling heterogeneous treatment effects in survey experiments with bayesian adaptive regression trees,

    D. P. Green and H. L. Kern, “Modeling heterogeneous treatment effects in survey experiments with bayesian adaptive regression trees,” Public Opin. Q., vol. 76, no. 3, pp. 491–511, 2012

    2012

  11. [19]

    Bayesian regression tree models for causal inference: regularization, confounding, and het- erogeneous effects,

    P. Richard Hahn, J. S. Murray, and C. Carvalho, “Bayesian regression tree models for causal inference: regularization, confounding, and het- erogeneous effects,” Jun. 2017

    2017

  12. [20]

    Generic machine learning inference on heterogenous treatment effects in random- ized experiments,

    V . Chernozhukov, M. Demirer, E. Duflo, and I. Fernandez-Val, “Generic machine learning inference on heterogenous treatment effects in random- ized experiments,” Dec. 2017

    2017

  13. [21]

    Some methods for heterogeneous treatment effect estimation in high dimensions,

    S. Powers, J. Qian, K. Jung, A. Schuler, N. H. Shah, T. Hastie, and R. Tibshirani, “Some methods for heterogeneous treatment effect estimation in high dimensions,” Stat. Med. , vol. 37, no. 11, pp. 1767– 1787, May 2018

    2018

  14. [22]

    Estimating individual treatment effect in observational data using random forest methods,

    M. Lu, S. Sadiq, D. J. Feaster, and H. Ishwaran, “Estimating individual treatment effect in observational data using random forest methods,” J. Comput. Graph. Stat. , vol. 27, no. 1, pp. 209–219, Feb. 2018

    2018

  15. [23]

    Estimating individual treatment effect: generalization bounds and algorithms,

    U. Shalit, F. D. Johansson, and D. Sontag, “Estimating individual treatment effect: generalization bounds and algorithms,” Jun. 2016

    2016

  16. [24]

    Transfer learning for estimating causal effects using neural networks,

    S. R. K ¨unzel, B. C. Stadie, N. Vemuri, V . Ramakrishnan, J. S. Sekhon, and P. Abbeel, “Transfer learning for estimating causal effects using neural networks,” Aug. 2018

    2018

  17. [25]

    Estimating heterogeneous treatment effects using neural networks with the Y- Learner,

    B. C. Stadie, S. R. K ¨unzel, N. Vemuri, and J. S. Sekhon, “Estimating heterogeneous treatment effects using neural networks with the Y- Learner,” Sep. 2018

    2018

  18. [26]

    Uci machine learning repository,

    A. Asuncion and D. Newman, “Uci machine learning repository,” 2007

    2007

  19. [27]

    Package ‘uplift’,

    L. Guelman, “Package ‘uplift’,” CRAN, 2014

    2014

  20. [28]

    Multi-armed bandit algorithms and empiri- cal evaluation,

    J. Vermorel and M. Mohri, “Multi-armed bandit algorithms and empiri- cal evaluation,” in European conference on machine learning. Springer, 2005, pp. 437–448

    2005

  21. [29]

    Simulation studies of multi-armed bandits with covariates (invited paper),

    N. G. Pavlidis, D. K. Tasoulis, and D. J. Hand, “Simulation studies of multi-armed bandits with covariates (invited paper),” in Tenth Interna- tional Conference on Computer Modeling and Simulation (uksim 2008) , Apr. 2008, pp. 493–498

    2008

  22. [30]

    Improved algorithms for linear stochastic bandits,

    Y . Abbasi-yadkori, D. P ´al, and C. Szepesv ´ari, “Improved algorithms for linear stochastic bandits,” in Advances in Neural Information Processing Systems 24, J. Shawe-Taylor, R. S. Zemel, P. L. Bartlett, F. Pereira, and K. Q. Weinberger, Eds. Curran Associates, Inc., 2011, ...

    2011

  23. [31]

    A contextual-bandit approach to personalized news article recommendation,

    L. Li, W. Chu, J. Langford, and R. E. Schapire, “A contextual-bandit approach to personalized news article recommendation,” in Proceedings of the 19th International Conference on World Wide Web , ser. WWW ’10. New York, NY , USA: ACM, 2010, pp. 661–670

  24. [32]

    The epoch-greedy algorithm for contextual multi-armed bandits,

    J. Langford and T. Zhang, “The epoch-greedy algorithm for contextual multi-armed bandits,” Proceedings of the 20th International Conference on, 2007

    2007

  25. [33]

    Contextual multi-armed bandits,

    T. Lu, D. P ´al, and M. P´al, “Contextual multi-armed bandits,” inProceed- ings of the Thirteenth international conference on Artificial Intelligence and Statistics, 2010, pp. 485–492

    2010

  26. [34]

    Online controlled experiments and a/b testing,

    R. Kohavi and R. Longbotham, “Online controlled experiments and a/b testing,” Encyclopedia of machine learning and data mining , pp. 922– 929, 2017

    2017

  27. [35]

    Unbiased offline evaluation of contextual-bandit-based news article recommendation algorithms,

    L. Li, W. Chu, J. Langford, and X. Wang, “Unbiased offline evaluation of contextual-bandit-based news article recommendation algorithms,” Mar. 2010

    2010

  28. [36]

    Doubly robust policy evaluation and optimization,

    M. Dud ´ık, D. Erhan, J. Langford, and L. Li, “Doubly robust policy evaluation and optimization,” Mar. 2015

    2015

  29. [37]

    Counterfactual estimation and optimization of click metrics in search engines: A case study,

    L. Li, S. Chen, J. Kleban, and A. Gupta, “Counterfactual estimation and optimization of click metrics in search engines: A case study,” in Proceedings of the 24th International Conference on World Wide Web . ACM, May 2015, pp. 929–934

    2015

  30. [38]

    Reinforcement learning for uplift modeling,

    C. Li, X. Yan, X. Deng, Y . Qi, W. Chu, L. Song, J. Qiao, J. He, and J. Xiong, “Reinforcement learning for uplift modeling,” Nov. 2018

    2018

  31. [39]

    Con- textual multi-armed bandits for causal marketing,

    N. Sawant, C. B. Namballa, N. Sadagopan, and H. Nassif, “Con- textual multi-armed bandits for causal marketing,” arXiv preprint arXiv:1810.01859, 2018

    2018 arXiv

  32. [40]

    Estimating causal effects of treatments in randomized and nonrandomized studies,

    D. B. Rubin, “Estimating causal effects of treatments in randomized and nonrandomized studies,” J. Educ. Psychol., vol. 66, no. 5, pp. 688–701, 1974

    1974

  33. [41]

    Sur les applications de la th ´eorie des probabilit ´es aux experiences agricoles: Essai des principes,

    J. Neyman, “Sur les applications de la th ´eorie des probabilit ´es aux experiences agricoles: Essai des principes,” Roczniki Nauk Rolniczych , vol. 10, pp. 1–51, 1923

    1923

  34. [42]

    Causal inference using potential outcomes,

    D. B. Rubin, “Causal inference using potential outcomes,” J. Am. Stat. Assoc., vol. 100, no. 469, pp. 322–331, Mar. 2005

    2005

  35. [43]

    Statistics and causal inference,

    P. W. Holland, “Statistics and causal inference,” J. Am. Stat. Assoc. , vol. 81, no. 396, pp. 945–960, 1986

    1986

  36. [44]

    On the role of the propensity score in efficient semiparametric estimation of average treatment effects,

    J. Hahn, “On the role of the propensity score in efficient semiparametric estimation of average treatment effects,” Econometrica, vol. 66, no. 2, pp. 315–331, 1998

    1998

  37. [45]

    Matching as an econo- metric evaluation estimator: Evidence from evaluating a job training programme,

    J. J. Heckman, H. Ichimura, and P. E. Todd, “Matching as an econo- metric evaluation estimator: Evidence from evaluating a job training programme,” Rev. Econ. Stud., vol. 64, no. 4, pp. 605–654, 1997

    1997

  38. [46]

    Estimating conditional average treatment effects,

    J. Abrevaya, Y .-C. Hsu, and R. P. Lieli, “Estimating conditional average treatment effects,” J. Bus. Econ. Stat. , vol. 33, no. 4, pp. 485–505, Oct. 2015

    2015

  39. [47]

    Generalized random forests,

    S. Athey, J. Tibshirani, and S. Wager, “Generalized random forests,” Oct. 2016

    2016

  40. [48]

    Random forests for uplift modeling: An insurance customer retention case,

    L. Guelman, M. Guill ´en, and A. M. P ´erez-Mar´ın, “Random forests for uplift modeling: An insurance customer retention case,” in Modeling and Simulation in Engineering, Economics and Management . Springer Berlin Heidelberg, 2012, pp. 123–133

    2012

  41. [49]

    Scikit-learn: Machine learning in Python,

    F. Pedregosa, G. Varoquaux, A. Gramfort, V . Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V . Dubourg, J. Vander- plas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duch- esnay, “Scikit-learn: Machine learning in Python,” Journal of Machine ...

    2011

  42. [50]

    Result analysis of the nips 2003 feature selection challenge,

    I. Guyon, S. Gunn, A. Ben-Hur, and G. Dror, “Result analysis of the nips 2003 feature selection challenge,” in Advances in neural information processing systems, 2005, pp. 545–552

    2003

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.