Pith. sign in

REVIEW 4 major objections 5 minor 98 references

A Two-armed Bandit Framework for A/B Testing

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A two-armed bandit test statistic improves A/B testing power while controlling type-I error.

desk verdict Useful incremental extension of the two-armed bandit test to ATE testing with doubly robust pseudo-outcomes; the proof has a real but fixable gap, and the paper deserves refereeing. read the letter →

arxiv 2507.18118 v1 pith:OQBX77Y3 submitted 2025-07-24 stat.ML cs.LGstat.AP

Jinjuan Wang , Qianglin Wen , Yu Zhang , Xiaodong Yan , Chengchun Shi This is my paper
classification stat.MLcs.LGstat.AP MSC 62D2062F0362F4062G1062L05
keywords A/Btestingtwo-armedbanditdoublyrobustestimationpermutationtestcausalinferencereinforcementlearningridesharingtype-Ierrorcontrol
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to improve A/B testing in the small-sample, small-signal regime typical of online experiments. It proposes building a test statistic from a two-armed bandit process on doubly robust pseudo-outcomes, and calibrating it with a permutation procedure. The central claim is that this statistic separates null from alternative by the shape of its distribution, not its mean, so it can detect small positive treatment effects with higher power than standard z-tests while still controlling type-I error. The construction extends to dynamic settings with carryover effects through double reinforcement learning pseudo-outcomes. Asymptotic theorems, simulations, and ride-sharing data are used to back the claim.

What carries the argument

The central object is the two-armed bandit policy that starts uniformly and, at each step, feeds the next unit's pseudo-outcome in with a positive sign if the current cumulative sum is positive and a negative sign if it is not. This sign-switching creates a self-normalized sum whose limiting distribution is the bandit distribution, with a density that is standard normal under the null, more peaked under negative effects, and bimodal with mass away from zero under positive effects. The doubly robust pseudo-outcome supplies the per-unit scores in i.i.d. settings, while a double reinforcement learning pseudo-outcome does the same in dynamic settings, and the permutation step aggregates multiple orderings via Cauchy or quantile-based p-value combination.

What would settle it

Simulate n=1000 null datasets with confounded assignment and cross-fitted nuisance estimators whose products of squared errors just satisfy the paper's rate condition; if the empirical rejection rate of P-TAB at α=0.05 exceeds the nominal level, or the empirical distribution of |T_n| deviates from the claimed bandit null, the type-I error claim is refuted.

Watch

Extended reading notes

Core claim

The paper's central claim is that the null and alternative hypotheses for an average treatment effect can be distinguished by the shape of a sequentially adaptive test statistic. Under the null, the two-armed bandit statistic is asymptotically standard normal; under a positive treatment effect it becomes bimodal with mass pushed away from zero, so the absolute value of the statistic carries the signal. The P-TAB procedure formalizes this: cross-fitted doubly robust pseudo-outcomes estimate each unit's contribution to the treatment effect, the zero-crossing bandit policy builds the statistic, and permutation with p-value combination removes ordering sensitivity. Theorems 1 and 2 state that the p-values control type-I error at the nominal level and that power tends to one against any fixed positive effect, in both i.i.d. and Markov decision process settings.

Load-bearing premise

The test's validity depends on treating the estimated pseudo-outcomes as if they were i.i.d. observations whose mean and variance converge to the truth fast enough, and the proof verifies only those two moments rather than the full distribution of the sequentially weighted sums.

Editorial extensions

If this is right

  • The P-TAB p-value is asymptotically valid: under the null the rejection rate is at most the nominal level, and under any fixed positive effect it tends to one.
  • Because validity depends only on the product of nuisance-error rates, the procedure tolerates slow nonparametric or machine-learning estimators for the outcome regression and propensity score, provided cross-fitting is used.
  • In the ride-sharing order-dispatch evaluations, P-TAB and TAB show higher power than the DRL-based z-test for small policy improvements, with P-TAB ahead of TAB in most settings.
  • On the subsidy-policy datasets, P-TAB yields the smallest p-values among P-TAB, TAB, and DML, and rejects the null in the two A/B datasets where TAB alone fails on one.
  • The permutation aggregation converts the ordering-dependent bandit statistic into an ordering-robust test, which is why P-TAB outperforms plain TAB in finite samples.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not claim this, but the same bandit-statistic construction should transfer to other per-unit efficient scores, such as those for quantile treatment effects or instrument-variable settings, since only the pseudo-outcome construction would change.
  • A testable extension the paper leaves open is to tune the switching threshold instead of fixing it at zero; this could buy further power but would require a new null distribution.
  • The paper's logic also implies the permutation gain is largest when n is small; at very large n, P-TAB and plain TAB should coincide, which a reader can check in the reported tables.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes P-TAB, a two-armed bandit framework for A/B testing in both i.i.d. and dynamic (MDP) settings. The procedure first constructs doubly robust pseudo-outcomes, then applies a two-armed bandit dynamic policy to build a test statistic, and finally aggregates permutation-based p-values to reduce ordering sensitivity. The main theoretical claims are type-I error control and consistency against fixed alternatives, stated as Theorem 1 for the i.i.d. setting and Theorem 2 for the dynamic setting. The empirical sections report extensive simulations and applications to five ride-sharing datasets, showing higher power for P-TAB than DML/DRL-based z-tests. The central idea is to exploit the shape difference of the bandit distribution under null and alternative hypotheses rather than only the mean shift.

Significance. If the theoretical claims are fully established, the paper would make a useful contribution by connecting the strategic central limit theorem and two-armed bandit tests to doubly robust causal inference, with a practical permutation wrapper that mitigates ordering dependence. The empirical work is extensive: multiple data-generating processes, confounded and unconfounded settings, heavy-tailed errors, high-dimensional covariates, dynamic switchback designs, and real ride-sharing data. The pseudo-outcome construction is standard and the comparison against DML, DRL, KTE, and xKTE is informative. However, the load-bearing proof of the SCLT for estimated, cross-fitted pseudo-outcomes is incomplete, and the theorem statement does not match the algorithm's use of cross-fitting or the Cauchy p-value combination. The paper also ships no code, which limits reproducibility of the numerical comparisons. The power advantage over DML/DRL is demonstrated empirically but not derived theoretically, which is acceptable if framed as a finite-sample finding.

major comments (4)
  1. [Supplementary A1.3, Steps I-II] The proof of Theorem 1 establishes only marginal moment convergence of the estimated pseudo-outcomes, namely \tilde\mu = \mu + o_p(n^{-1/2}) and \tilde\sigma^2 = \sigma^2 + o_p(n^{-1/2}), and then plugs \tilde\kappa_n and \tilde\sigma_0 into the bandit density (4). This does not establish the required distributional limit for the policy-dependent, self-normalized statistic T_n(\bar\pi_n^*) = \sum_i (1-2\theta_i)\tilde\mu_i/(\sqrt{n}\tilde\sigma). In particular, no verification is given that the SCLT hypotheses—i.i.d. or martingale-increment structure, adaptedness of \theta_i to the filtration generated by the pseudo-outcomes, and control of the full-sample normalizer \tilde\sigma—hold for the cross-fitted \tilde\mu_i, nor is an argument provided that integrates the conditional law over the nuisance estimators. Since the type-I error bound in Theorem 1(i) rests on the bandit distribution being the limiting law of |T_n|, this is a load-bearing gap.
  2. [Section 2.3 and Supplementary A1.3, Step III] The validity proof in Step III is written only for the quantile-based combination Q(\gamma), but Algorithm 1 and the numerical experiments use the Cauchy combination in equation (5). No theorem or lemma shows that the final p-value computed as 0.5 - \arctan(\tilde{T}_n)/\pi controls the type-I error under the dependence structure of the permuted p-values p_b. The citation to Liu and Xie (2020) supplies an approximation, not a formal level guarantee, and the theorem statement should either be restricted to a combination method with a proven level property or supplemented with the missing result for Cauchy combination.
  3. [Theorem 1, Algorithm 1, and Assumption A1] Theorem 1 is stated under Assumptions 1-3 and Assumption A2, but the proof in Supplementary A1.3 explicitly conditions on an external dataset D (Assumption A1) for the nuisance estimates, whereas Algorithm 1 uses K-fold cross-fitting. In the cross-fitted version, each pseudo-outcome \tilde\mu_i depends on nuisance estimators trained on the other folds, creating dependence across the entire sample. The proof does not address this dependence or justify that the moment bounds in Step I extend to the cross-fitted setting. The same mismatch appears in Theorem 2, where Assumption A4 posits an external dataset but Algorithm 2 uses cross-fitting. The theorem statements should be aligned with the algorithm actually implemented.
  4. [Supplementary A1.4, variance bound for Theorem 2] The variance bound for the DRL pseudo-outcome is not fully established as written. The derivation of Var(\tilde\mu_a) - \bar\sigma^{2,a} bounds the difference by a constant times \sqrt{E[(A_1+A_2+A_3)^2]} plus a term involving E(\tilde\mu_a)-\bar\mu_a, but the final equality '= o(n^{-1/2})' is not justified for all terms shown: the cross terms involving A_2 and A_3 require additional orthogonality or product-structure conditions beyond Assumptions A5-A6, and the conditional-on-external-data treatment is again informal. This matters because Theorem 2's type-I error and consistency claims rely on the same plug-in step as Theorem 1.
minor comments (5)
  1. [Algorithm 1 and Algorithm 2] The notation '\mu' is used both for the ATE and for the sample mean of the pseudo-outcomes in the variance estimator \hat\sigma^2; use \bar\mu or another symbol to avoid ambiguity.
  2. [Section 2.3] The oracle construction permutes the pairs of potential outcomes, while the practical algorithm permutes the univariate pseudo-outcomes \tilde\mu_i. The paper should state explicitly that the permutation is applied to the pseudo-outcomes as surrogates and clarify the sense in which this matches the oracle permutation.
  3. [Supplementary A2.1] The symbols p_a and \sigma_0 are used for the treatment probability and residual standard deviation in the simulations, while \sigma_0 also denotes \sqrt{1+\mu^2/\sigma^2} in the bandit density (4). This overloading is confusing and should be resolved.
  4. [Supplementary A1.4] The proof refers to 'Equation (6)' for the definition of \tilde\mu_a; the numbering should be checked and the displayed variance decomposition should be cleaned up before publication.
  5. [General] No code or data repository is provided for the simulations or the real-data analysis; making the code available would materially improve reproducibility, given that the reported power gains are empirically demonstrated rather than derived.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: the test's ingredients are independent of its conclusions; the SCLT import is a published theorem, and the proof gap is a technical validity issue, not a circular reduction.

full rationale

The paper's derivation chain is not circular. Step I (Supplementary A1.3) proves, by Taylor expansion of the doubly robust estimating equation and the Neyman orthogonality condition, that the pseudo-outcome mean and variance converge at the required rates (tilde_mu = mu + o_p(n^{-1/2}), tilde_sigma^2 = sigma^2 + o_p(n^{-1/2})); this is a standard double-machine-learning argument, not a rearrangement of the conclusion. Step II applies the strategic central limit theorem of Chen, Feng and Zhang (2022) and Chen, Yan and Zhang (2023) to the self-normalized bandit sum; that theorem is a published, parameter-free limit theorem whose assumptions do not include this paper's type-I error or power claims, so citing it (even with overlapping author Yan) is independent support, not circularity. Step III derives the validity of the permutation p-value aggregation from the validity of the individual p-values, which is a general resampling argument. The real-data and simulation results are out-of-sample evaluations, not predictions generated by fitted constants. The skeptic's objection—that only marginal moment convergence is verified and the SCLT is not shown to hold for the cross-fitted, dependent pseudo-outcome process—is a proof gap and a correctness risk, not a circular reduction: the claimed theorem does not reduce to its inputs by construction. Hence the circularity score is 0.

Assumptions & free parameters 2 free parameters · 8 assumptions · 0 invented entities

The central claim rests on four groups of assumptions: standard causal identification, an MDP model for dynamic settings, rate conditions on nuisance estimators, and the strategic CLT imported from prior work. There are no invented entities. The method itself introduces two tuning parameters (B and K) but no fitted constants.

free parameters (2)
  • B (number of permutations) = not specified in the paper
    User-selected; the Cauchy combination approximation depends on B, but no default or sensitivity analysis is reported.
  • K (cross-fitting folds) = 5 in i.i.d. simulations, 2 in dynamic simulations
    User-selected; standard cross-fitting choice, not fitted to data.
assumptions (8)
  • domain assumption Consistency, unconfoundedness, and positivity (Assumptions 1-3)
    Identifies the ATE from observed data in i.i.d. settings.
  • domain assumption Markov and conditional mean independence (Assumption 4)
    Models the dynamic setting as an MDP and captures carryover effects through state transitions.
  • domain assumption Nuisance estimation rates (Assumptions A2 and A5)
    Requires product or individual L2 errors to vanish faster than n^{-1/2}; standard for double ML but unverifiable with small n.
  • domain assumption Bounded rewards (Assumption A3)
    Used in the DRL variance bound; may fail for heavy-tailed revenue distributions.
  • ad hoc to paper Basis function approximation (Assumption A6)
    Requires value and MIS ratio functions to be uniformly approximable at o(n^{-1/4}); specific to the proof of Theorem 2.
  • standard math Strategic CLT and bandit limit distribution (Chen et al. 2022, 2023)
    Provides the null distribution and optimality of the oracle bandit test; the paper relies on it without reproving it.
  • standard math Cauchy combination validity under arbitrary dependence (Liu and Xie 2020)
    Used to aggregate permutation p-values; requires each p-value to be valid.
  • ad hoc to paper External dataset existence (Assumptions A1 and A4)
    Theorems 1 and 2 are proved under external nuisance estimators; the paper notes sample splitting can relax this, but does not fully rework the proofs.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Two-armed Bandit Framework for A/B Testing." pith.science (2026). https://pith.science/paper/OQBX77Y3

@misc{pith2026250718118,
  author       = {Pith},
  title        = {Pith review of: A Two-armed Bandit Framework for A/B Testing},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OQBX77Y3}},
  note         = {Machine review of arXiv:2507.18118}
}
abstract

A/B testing is widely used in modern technology companies for policy evaluation and product deployment, with the goal of comparing the outcomes under a newly-developed policy against a standard control. Various causal inference and reinforcement learning methods developed in the literature are applicable to A/B testing. This paper introduces a two-armed bandit framework designed to improve the power of existing approaches. The proposed procedure consists of three main steps: (i) employing doubly robust estimation to generate pseudo-outcomes, (ii) utilizing a two-armed bandit framework to construct the test statistic, and (iii) applying a permutation-based method to compute the $p$-value. We demonstrate the efficacy of the proposed method through asymptotic theories, numerical experiments and real-world data from a ridesharing company, showing its superior performance in comparison to existing methods.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

98 extracted references · 71 canonical work pages

  1. [1]

    Imbens , Guido W G

    barticle [author] Abadie , Alberto A. Imbens , Guido W G. W. ( 2011 ). Bias-corrected matching estimators for average treatment effects . Journal of Business & Economic Statistics 29 1--11 . barticle

    2011

  2. [2]

    , Samaranayake , Samitha S

    barticle [author] Alonso-Mora , Javier J. , Samaranayake , Samitha S. , Wallar , Alex A. , Frazzoli , Emilio E. Rus , Daniela D. ( 2017 ). On-demand high-capacity ride-sharing via dynamic trip-vehicle assignment . Proceedings of the National Academy of Sciences 114 462--467 . barticle

    2017

  3. [3]

    , Imbens , Guido W G

    barticle [author] Athey , Susan S. , Imbens , Guido W G. W. Wager , Stefan S. ( 2018 ). Approximate residual balancing: debiased inference of average treatment effects in high dimensions . Journal of the Royal Statistical Society Series B: Statistical Methodology 80 597--623 . barticle

    2018

  4. [4]

    , Bickel , Peter J P

    barticle [author] Athey , Susan S. , Bickel , Peter J P. J. , Chen , Aiyou A. , Imbens , Guido W G. W. Pollmann , Michael M. ( 2023 ). Semi-parametric estimation of treatment effects in randomised experiments . Journal of the Royal Statistical Society Series B: Statistical Methodology 85 1615--1638 . barticle

    2023

  5. [5]

    , Burdick , Brian B

    barticle [author] Bajari , Patrick P. , Burdick , Brian B. , Imbens , Guido W G. W. , Masoero , Lorenzo L. , McQueen , James J. , Richardson , Thomas T. Rosen , Ido M I. M. ( 2021 ). Multiple randomization designs . arXiv preprint arXiv:2112.13495 . barticle

    arXiv 2021

  6. [6]

    Robins , James M J

    barticle [author] Bang , Heejung H. Robins , James M J. M. ( 2005 ). Doubly robust estimation in missing data and causal inference models . Biometrics 61 962--973 . barticle

    2005

  7. [7]

    binproceedings [author] Basse , Guillaume W G. W. , Soufiani , Hossein Azari H. A. Lambert , Diane D. ( 2016 ). Randomization and the pernicious effects of limited budgets on auction experiments . In Artificial Intelligence and Statistics 1412--1420 . PMLR . binproceedings

    2016

  8. [8]

    bbook [author] Berger , Roger L R. L. Casella , George G. ( 2001 ). Statistical inference . Duxbury . bbook

    2001

Show all 98 references
  1. [9]

    Shephard , Neil N

    barticle [author] Bojinov , Iavor I. Shephard , Neil N. ( 2019 ). Time series experiments and causal estimands: exact randomization tests and trading . Journal of the American Statistical Association 114 1665--1682 . barticle

    2019

  2. [10]

    , Simchi-Levi , David D

    barticle [author] Bojinov , Iavor I. , Simchi-Levi , David D. Zhao , Jinglong J. ( 2023 ). Design and analysis of switchback experiments . Management Science 69 3759--3777 . barticle

    2023

  3. [11]

    barticle [author] Bradtke , Steven J S. J. Barto , Andrew G A. G. ( 1996 ). Linear least-squares algorithms for temporal difference learning . Machine learning 22 33--57 . barticle

    1996

  4. [12]

    , Fan , Yingying Y

    barticle [author] Candes , Emmanuel E. , Fan , Yingying Y. , Janson , Lucas L. Lv , Jinchi J. ( 2018 ). Panning for gold:‘model-X’knockoffs for high dimensional controlled variable selection . Journal of the Royal Statistical Society Series B: Statistical Methodology 80 551--5...

    2018

  5. [13]

    , Feng , Shui S

    barticle [author] Chen , Zengjing Z. , Feng , Shui S. Zhang , Guodong G. ( 2022 ). Strategy-driven limit theorems associated bandit problems . arXiv preprint arXiv:2204.04442 . barticle

    2022 arXiv

  6. [14]

    , Simchi-Levi , David D

    barticle [author] Chen , Shuze S. , Simchi-Levi , David D. Wang , Chonghuan C. ( 2024 ). Experimenting on markov decision processes with local treatments . arXiv preprint arXiv:2407.19618 . barticle

    2024 arXiv

  7. [15]

    barticle [author] Chen , Z. Z. , Yan , X. X. Zhang , G. G. ( 2023 ). Strategic two-sample test via the two-armed bandit process. Journal of the Royal Statistical Society Series B: Statistical Methodology qkad061 . barticle

    2023

  8. [16]

    , Wang , Xingyan X

    barticle [author] Chen , Fang F. , Wang , Xingyan X. , Jang , Seon-Kyeong S.-K. , Quach , Bryan C B. C. , Weissenkampen , J Dylan J. D. , Khunsriraksakul , Chachrit C. , Yang , Lina L. , Sauteraud , Renan R. , Albert , Christine M C. M. , Allred , Nicholette DD N. D. et al. ( ...

    2023

  9. [17]

    barticle [author] Chernozhukov , V. V. , Chetverikov , D. D. , Demirer , M. M. , Duflo , E. E. , Hansen , C. C. Newey , W. W. ( 2017 ). Double/debiased/neyman machine learning of treatment effects. American Economic Review 107 261-265 . barticle

    2017

  10. [18]

    barticle [author] Chernozhukov , V. V. , Chetverikov , D. D. , Demirer , E. E. M. andDuflo , Hansen , C. C. , Newey , W. W. Robins , J. J. ( 2018 ). Double/debiased machine learning for treatment and structural parameters. The Econometrics Journal C1-C68 . barticle

    2018

  11. [19]

    , Karabulut , Erdem E

    barticle [author] Dag , Osman O. , Karabulut , Erdem E. Alpar , Reha R. ( 2019 ). GMDH2: Binary classification via GMDH-type neural network algorithms—R package and web-based tool . International Journal of Computational Intelligence Systems 12 649--660 . barticle

    2019

  12. [20]

    ( 2020 )

    barticle [author] D \' az , Iv \'a n I. ( 2020 ). Machine learning in the estimation of causal effects: targeted minimum loss-based estimation and double/debiased machine learning . Biostatistics 21 353--358 . barticle

    2020

  13. [21]

    , Erhan , Dumitru D

    barticle [author] Dud \' k , Miroslav M. , Erhan , Dumitru D. , Langford , John J. Li , Lihong L. ( 2014 ). Doubly Robust Policy Evaluation and Optimization . Statistical Science 29 485--511 . barticle

    2014

  14. [22]

    , Li , Andrew A

    barticle [author] Farias , Vivek V. , Li , Andrew A. , Peng , Tianyi T. Zheng , Andrew A. ( 2022 ). Markovian interference in experiments . Advances in Neural Information Processing Systems 35 535--549 . barticle

    2022

  15. [23]

    bbook [author] Fisher , Ronald Aylmer R. A. ( 1928 ). Statistical methods for research workers 5 . Oliver and Boyd . bbook

    1928

  16. [24]

    barticle [author] Glynn , Peter W P. W. , Johari , Ramesh R. Rasouli , Mohammad M. ( 2020 ). Adaptive experimental design with temporal interference: A maximum likelihood approach . Advances in Neural Information Processing Systems 33 15054--15064 . barticle

    2020

  17. [25]

    Wright , Julian J

    barticle [author] Hagiu , Andrei A. Wright , Julian J. ( 2019 ). The status of workers and platforms in the sharing economy . Journal of Economics & Management Strategy 28 97--108 . barticle

    2019

  18. [26]

    Heyde , Christopher C C

    bbook [author] Hall , Peter P. Heyde , Christopher C C. C. ( 2014 ). Martingale limit theory and its application . Academic press . bbook

    2014

  19. [27]

    ( 1999 )

    barticle [author] Hartung , Joachim J. ( 1999 ). A note on combining dependent tests of significance . Biometrical Journal: Journal of Mathematical Methods in Biosciences 41 849--855 . barticle

    1999

  20. [28]

    , Li , Shuangning S

    barticle [author] Hu , Yuchen Y. , Li , Shuangning S. Wager , Stefan S. ( 2022 ). Average direct and indirect causal effects under interference . Biometrika 109 1165--1172 . barticle

    2022

  21. [29]

    bbook [author] Imbens , Guido W G. W. Rubin , Donald B D. B. ( 2015 ). Causal inference in statistics, social, and biomedical sciences . Cambridge university press . bbook

    2015

  22. [30]

    Li , Lihong L

    binproceedings [author] Jiang , Nan N. Li , Lihong L. ( 2016 ). Doubly robust off-policy value evaluation for reinforcement learning . In International conference on machine learning 652--661 . PMLR . binproceedings

    2016

  23. [31]

    , Li , Jingyi J

    binproceedings [author] Jin , Zhu Z. , Li , Jingyi J. , Zhou , Hongyi H. , Lin , Yinan Y. , Lin , Zhenhua Z. , Shi , Chengchun C. , Tang , Niansheng N. , Zhu , Hongtu H. et al. ( 2025 ). Balancing Interference and Correlation in Spatial Experimental Designs: A Causal Graph Cut...

    2025

  24. [32]

    , Pekelis , Leo L

    barticle [author] Johari , Ramesh R. , Pekelis , Leo L. Walsh , David J D. J. ( 2015 ). Always valid inference: Bringing sequential analysis to A/B testing . arXiv preprint arXiv:1512.04922 . barticle

    2015 arXiv

  25. [33]

    Uehara , Masatoshi M

    barticle [author] Kallus , Nathan N. Uehara , Masatoshi M. ( 2020 ). Double reinforcement learning for efficient off-policy evaluation in markov decision processes . Journal of Machine Learning Research 21 1--63 . barticle

    2020

  26. [34]

    Uehara , Masatoshi M

    barticle [author] Kallus , Nathan N. Uehara , Masatoshi M. ( 2022 ). Efficiently breaking the curse of horizon in off-policy evaluation with double reinforcement learning . Operations Research 70 3282--3302 . barticle

    2022

  27. [35]

    Zhou , Angela A

    binproceedings [author] Kallus , Nathan N. Zhou , Angela A. ( 2018 ). Policy evaluation and optimization with continuous treatments . In International conference on artificial intelligence and statistics 1243--1251 . PMLR . binproceedings

    2018

  28. [36]

    barticle [author] Lai , Tze Leung T. L. ( 1987 ). Adaptive treatment allocation and the multi-armed bandit problem . The annals of statistics 1091--1114 . barticle

    1987

  29. [37]

    , Stallrich , Jonathan J

    barticle [author] Larsen , Nicholas N. , Stallrich , Jonathan J. , Sengupta , Srijan S. , Deng , Alex A. , Kohavi , Ron R. Stevens , Nathaniel T N. T. ( 2024 ). Statistical challenges in online controlled experiments: A review of a/b testing methodology . The American Statisti...

    2024

  30. [38]

    , Voloshin , Cameron C

    binproceedings [author] Le , Hoang H. , Voloshin , Cameron C. Yue , Yisong Y. ( 2019 ). Batch policy learning under constraints . In International Conference on Machine Learning 3703--3712 . PMLR . binproceedings

    2019

  31. [39]

    barticle [author] Leung , Michael P M. P. ( 2022 ). Rate-optimal cluster-randomized designs for spatial interference . The Annals of Statistics 50 3064--3087 . barticle

    2022

  32. [40]

    , Morgan , Kari Lock K

    barticle [author] Li , Fan F. , Morgan , Kari Lock K. L. Zaslavsky , Alan M A. M. ( 2018 ). Balancing covariates via propensity score weighting . Journal of the American Statistical Association 113 390--400 . barticle

    2018

  33. [41]

    , Shi , Chengchun C

    barticle [author] Li , Ting T. , Shi , Chengchun C. , Wang , Jianing J. , Zhou , Fan F. et al. ( 2023 ). Optimal treatment allocation for efficient policy evaluation in sequential decision making . Advances in Neural Information Processing Systems 36 48890--48905 . barticle

    2023

  34. [42]

    , Shi , Chengchun C

    barticle [author] Li , Ting T. , Shi , Chengchun C. , Lu , Zhaohua Z. , Li , Yi Y. Zhu , Hongtu H. ( 2024 a). Evaluating dynamic conditional quantile treatment effects with applications in ridesharing . Journal of the American Statistical Association 119 1736--1750 . barticle

    2024

  35. [43]

    , Shi , Chengchun C

    binproceedings [author] Li , Ting T. , Shi , Chengchun C. , Wen , Qianglin Q. , Sui , Yang Y. , Qin , Yongli Y. , Lai , Chunbo C. Zhu , Hongtu H. ( 2024 b). Combining Experimental and Historical Data for Policy Evaluation . In International Conference on Machine Learning 28630...

    2024

  36. [44]

    , Choi , Young-Geun Y.-G

    barticle [author] Liang , Muxuan M. , Choi , Young-Geun Y.-G. , Ning , Yang Y. , Smith , Maureen A M. A. Zhao , Ying-Qi Y.-Q. ( 2022 ). Estimation and inference on high-dimensional individualized treatment rule in observational data using split-and-pooled de-correlated score ....

    2022

  37. [45]

    , Qi , Zhengling Z

    barticle [author] Liao , Peng P. , Qi , Zhengling Z. , Wan , Runzhe R. , Klasnja , Predrag P. Murphy , Susan A S. A. ( 2022 ). Batch policy learning in average reward markov decision processes . Annals of statistics 50 3364 . barticle

    2022

  38. [46]

    , Mao , Jialiang J

    barticle [author] Liu , Min M. , Mao , Jialiang J. Kang , Kang K. ( 2020 ). Trustworthy online marketplace experimentation with budget-split design . arXiv preprint arXiv:2012.08724 . barticle

    2020 arXiv

  39. [47]

    barticle [author] Liu , Y Y. Xie , J. J. ( 2020 ). Cauchy combination test: a powerful test with analytic p-value calculation under arbitrary dependency structures. Journal of the American Statistical Association 115 393–402 . barticle

    2020

  40. [48]

    , Li , Lihong L

    barticle [author] Liu , Qiang Q. , Li , Lihong L. , Tang , Ziyang Z. Zhou , Dengyong D. ( 2018 ). Breaking the curse of horizon: Infinite-horizon off-policy estimation . Advances in neural information processing systems 31 . barticle

    2018

  41. [49]

    barticle [author] Luckett , Daniel J D. J. , Laber , Eric B E. B. , Kahkoska , Anna R A. R. , Maahs , David M D. M. , Mayer-Davis , Elizabeth E. Kosorok , Michael R M. R. ( 2020 ). Estimating dynamic treatment regimes in mobile health using v-learning . Journal of the american...

    2020

  42. [50]

    , Yang , Ying Y

    barticle [author] Luo , Shikai S. , Yang , Ying Y. , Shi , Chengchun C. , Yao , Fang F. , Ye , Jieping J. Zhu , Hongtu H. ( 2024 ). Policy evaluation for temporal and/or spatial dependent experiments . Journal of the Royal Statistical Society Series B: Statistical Methodology ...

    2024

  43. [51]

    , Ramdas , Aaditya A

    barticle [author] Martinez Taboada , Diego D. , Ramdas , Aaditya A. Kennedy , Edward E. ( 2024 ). An efficient doubly-robust test for the kernel treatment effect . Advances in Neural Information Processing Systems 36 . barticle

    2024

  44. [52]

    barticle [author] McCaw , Zachary R Z. R. , Lane , Jacqueline M J. M. , Saxena , Richa R. , Redline , Susan S. Lin , Xihong X. ( 2020 ). Operating characteristics of the rank-based inverse normal transformation for quantitative trait analysis in genome-wide association studies...

    2020

  45. [53]

    , Meier , Lukas L

    barticle [author] Meinshausen , Nicolai N. , Meier , Lukas L. B \"u hlmann , Peter P. ( 2009 ). P-values for high-dimensional regression . Journal of the American Statistical Association 104 1671--1681 . barticle

    2009

  46. [54]

    , Kanagawa , Motonobu M

    barticle [author] Muandet , Krikamol K. , Kanagawa , Motonobu M. , Saengkyongam , Sorawit S. Marukatat , Sanparith S. ( 2021 ). Counterfactual mean embeddings . Journal of Machine Learning Research 22 1--71 . barticle

    2021

  47. [55]

    barticle [author] Neyman , J. J. ( 1959 ). Optimal asymptotic tests of composite statistical hypotheses. In U. Grenander (Ed.), Probability and Statistics, 416–44. New York, NY: Wiley. barticle

    1959

  48. [56]

    barticle [author] Neyman , J. J. ( 1979 ). c( ) tests and their use . Sankhya 1–21 . barticle

    1979

  49. [57]

    Bojinov , Iavor I

    barticle [author] Ni , Tu T. Bojinov , Iavor I. ( 2025 ). Enhancing Efficiency and Robustness for Switchback Experiments: A Practical Model-assisted Framework . Available at SSRN 5229804 . barticle

    2025

  50. [58]

    ( 2000 )

    binproceedings [author] Precup , D D. ( 2000 ). Eligibility traces for off-policy policy evaluation . In In Proceedings of the 17th International Conference on Machine Learning . binproceedings

    2000

  51. [59]

    bbook [author] Puterman , Martin L M. L. ( 2014 ). Markov decision processes: discrete stochastic dynamic programming . John Wiley & Sons . bbook

    2014

  52. [60]

    , Weyns , Danny D

    barticle [author] Quin , Federico F. , Weyns , Danny D. , Galster , Matthias M. Silva , Camila Costa C. C. ( 2024 ). A/B testing: A systematic literature review . Journal of Systems and Software 211 112011 . barticle

    2024

  53. [61]

    , Li , Yuantong Y

    barticle [author] Ramprasad , Pratik P. , Li , Yuantong Y. , Yang , Zhuoran Z. , Wang , Zhaoran Z. , Sun , Will Wei W. W. Cheng , Guang G. ( 2023 ). Online bootstrap inference for policy evaluation in reinforcement learning . Journal of the American Statistical Association 118...

    2023

  54. [62]

    ( 1986 )

    barticle [author] Robins , James J. ( 1986 ). A new approach to causal inference in mortality studies with a sustained exposure period—application to control of the healthy worker survivor effect . Mathematical modelling 7 1393--1512 . barticle

    1986

  55. [63]

    barticle [author] Rosenbaum , Paul R. P. R. Rubin , Donald B. D. B. ( 1983 a). The central role of the propensity score in observational studies for causal effects . Biometrika 70 41-55 . barticle

    1983

  56. [64]

    barticle [author] Rosenbaum , Paul R P. R. Rubin , Donald B D. B. ( 1983 b). The central role of the propensity score in observational studies for causal effects . Biometrika 70 41--55 . barticle

    1983

  57. [65]

    barticle [author] Rubin , Donald B D. B. ( 1979 ). Using multivariate matched sampling and regression adjustment to control bias in observational studies . Journal of the American Statistical Association 74 318--328 . barticle

    1979

  58. [66]

    ( 2009 )

    barticle [author] Rysman , Marc M. ( 2009 ). The economics of two-sided markets . Journal of economic perspectives 23 125--143 . barticle

    2009

  59. [67]

    barticle [author] Scharfstein , Daniel O D. O. , Rotnitzky , Andrea A. Robins , James M J. M. ( 1999 ). Adjusting for nonignorable drop-out using semiparametric nonresponse models . Journal of the American Statistical Association 94 1096--1120 . barticle

    1999

  60. [68]

    ( 2025 )

    barticle [author] Shi , Chengchun C. ( 2025 ). Statistical inference in reinforcement learning: A selective survey . arXiv preprint arXiv:2502.16195 . barticle

    2025 arXiv

  61. [69]

    Li , Lexin L

    barticle [author] Shi , Chengchun C. Li , Lexin L. ( 2022 ). Testing mediation effects using logic of boolean matrices . Journal of the American Statistical Association 117 2014--2027 . barticle

    2022

  62. [70]

    , Zhang , Sheng S

    barticle [author] Shi , Chengchun C. , Zhang , Sheng S. , Lu , Wenbin W. Song , Rui R. ( 2022 ). Statistical inference of the value function for reinforcement learning in infinite-horizon settings . Journal of the Royal Statistical Society Series B: Statistical Methodology 84 ...

    2022

  63. [71]

    , Wang , Xiaoyu X

    barticle [author] Shi , Chengchun C. , Wang , Xiaoyu X. , Luo , Shikai S. , Zhu , Hongtu H. , Ye , Jieping J. Song , Rui R. ( 2023 ). Dynamic causal effects evaluation in a/b testing with a reinforcement learning framework . Journal of the American Statistical Association 118 ...

    2023

  64. [72]

    , VanderWeele , Tyler T

    binproceedings [author] Shpitser , Ilya I. , VanderWeele , Tyler T. Robins , James M J. M. ( 2010 ). On the validity of covariate adjustment for estimating causal effects . In Proceedings of the Twenty-Sixth Conference on Uncertainty in Artificial Intelligence 527--536 . binpr...

    2010

  65. [73]

    barticle [author] Sobel , Michael E M. E. Lindquist , Martin A M. A. ( 2014 ). Causal inference for fMRI time series data with systematic errors of measurement in a balanced on/off study of social evaluative threat . Journal of the American Statistical Association 109 967--976...

    2014

  66. [74]

    The probable error of a mean

    barticle [author] Student ( 1908 ). The probable error of a mean . Biometrika 1--25 . barticle

    1908

  67. [75]

    , Kong , Linglong L

    barticle [author] Sun , Ke K. , Kong , Linglong L. , Zhu , Hongtu H. Shi , Chengchun C. ( 2024 ). ARMA-Design: Optimal Treatment Allocation Strategies for A/B Testing in Partially Observable Time Series Experiments . arXiv preprint arXiv:2408.05342 . barticle

    2024 arXiv

  68. [76]

    bbook [author] Sutton , Richard S R. S. Barto , Andrew G A. G. ( 2018 ). Reinforcement learning: An introduction . MIT press . bbook

    2018

  69. [77]

    , Qin , Zhiwei Z

    binproceedings [author] Tang , Xiaocheng X. , Qin , Zhiwei Z. , Zhang , Fan F. , Wang , Zhaodong Z. , Xu , Zhe Z. , Ma , Yintai Y. , Zhu , Hongtu H. Ye , Jieping J. ( 2019 ). A deep value-network based approach for multi-driver order dispatching . In Proceedings of the 25th AC...

    2019

  70. [78]

    , Theocharous , Georgios G

    binproceedings [author] Thomas , Philip P. , Theocharous , Georgios G. Ghavamzadeh , Mohammad M. ( 2015 ). High-confidence off-policy evaluation . In Proceedings of the AAAI Conference on Artificial Intelligence 29 . binproceedings

    2015

  71. [79]

    bbook [author] Tsiatis , Anastasios A A. A. ( 2006 ). Semiparametric theory and missing data 4 . Springer . bbook

    2006

  72. [80]

    , Shi , Chengchun C

    barticle [author] Uehara , Masatoshi M. , Shi , Chengchun C. Kallus , Nathan N. ( 2022 ). A review of off-policy evaluation in reinforcement learning . arXiv preprint arXiv:2212.06355 . barticle

    2022 arXiv

  73. [81]

    , Karrer , Brian B

    binproceedings [author] Ugander , Johan J. , Karrer , Brian B. , Backstrom , Lars L. Kleinberg , Jon J. ( 2013 ). Graph cluster randomization: Network exposure to multiple universes . In Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and dat...

    2013

  74. [82]

    , Lei , Lihua L

    barticle [author] Viviano , Davide D. , Lei , Lihua L. , Imbens , Guido G. , Karrer , Brian B. , Schrijvers , Okke O. Shi , Liang L. ( 2023 ). Causal clustering: design of cluster experiments under network interference . arXiv preprint arXiv:2310.14983 . barticle

    2023 arXiv

  75. [83]

    ( 1975 )

    barticle [author] Wahba , Grace G. ( 1975 ). Smoothing noisy data with spline functions . Numerische mathematik 24 383--393 . barticle

    1975

  76. [84]

    barticle [author] Wang , Y. Y. Shah , R. D. R. D. ( 2020 ). Debiased inverse propensity score weighting for estimation of average treatment effects with high-dimensional confounders. arXiv preprint arXiv:2011.08661 . barticle

    2020 arXiv

  77. [85]

    Yang , Hai H

    barticle [author] Wang , Hai H. Yang , Hai H. ( 2019 ). Ridesourcing systems: A framework and review . Transportation Research Part B: Methodological 129 122--155 . barticle

    2019

  78. [86]

    , Wu , Lili L

    barticle [author] Waudby-Smith , Ian I. , Wu , Lili L. , Ramdas , Aaditya A. , Karampatziakis , Nikos N. Mineiro , Paul P. ( 2024 ). Anytime-valid off-policy inference for contextual bandits . ACM/IMS Journal of Data Science 1 1--42 . barticle

    2024

  79. [87]

    , Shi , Chengchun C

    binproceedings [author] Wen , Qianglin Q. , Shi , Chengchun C. , Tang , Niansheng N. , Zhu , Hongtu H. et al. ( 2025 ). Unraveling the Interplay between Carryover Effects and Reward Autocorrelations in Switchback Experiments . In Forty-second International Conference on Machin...

    2025

  80. [88]

    barticle [author] Wu , Chien-Fu Jeff C.-F. J. et al. ( 1986 ). Jackknife, bootstrap and other resampling methods in regression analysis . the Annals of Statistics 14 1261--1295 . barticle

    1986

  81. [89]

    , Chin , Alex A

    barticle [author] Xiong , Ruoxuan R. , Chin , Alex A. Taylor , Sean J S. J. ( 2024 ). Data-driven switchback experiments: Theoretical tradeoffs and empirical bayes designs . arXiv preprint arXiv:2406.06768 . barticle

    2024 arXiv

  82. [90]

    , Li , Zhixin Z

    binproceedings [author] Xu , Zhe Z. , Li , Zhixin Z. , Guan , Qingwen Q. , Zhang , Dingshui D. , Li , Qiang Q. , Nan , Junxiao J. , Liu , Chunyang C. , Bian , Wei W. Ye , Jieping J. ( 2018 ). Large-scale order dispatch in on-demand ride-hailing platforms: A learning and planni...

    2018

  83. [91]

    barticle [author] Yang , S. S. Ding , P. P. ( 2018 ). Asymptotic inference of causal effects with observational studies trimmed by the estimated propensity scores. Biometrika 105 487-493 . barticle

    2018

  84. [92]

    , Shao , Jun J

    barticle [author] Ye , Ting T. , Shao , Jun J. , Yi , Yanyao Y. Zhao , Qingyuan Q. ( 2023 ). Toward better practice of covariate adjustment in analyzing randomized clinical trials . Journal of the American Statistical Association 118 2370--2382 . barticle

    2023

  85. [93]

    , Tsiatis , Anastasios A A

    barticle [author] Zhang , Baqun B. , Tsiatis , Anastasios A A. A. , Laber , Eric B E. B. Davidian , Marie M. ( 2012 ). A robust method for estimating optimal treatment regimes . Biometrics 68 1010--1018 . barticle

    2012

  86. [94]

    , Tsiatis , Anastasios A A

    barticle [author] Zhang , Baqun B. , Tsiatis , Anastasios A A. A. , Laber , Eric B E. B. Davidian , Marie M. ( 2013 ). Robust estimation of optimal dynamic treatment regimes for sequential treatment decisions . Biometrika 100 10--1093 . barticle

    2013

  87. [95]

    , Zhang , Zhiwei Z

    barticle [author] Zhou , Jie J. , Zhang , Zhiwei Z. , Li , Zhaohai Z. Zhang , Jun J. ( 2015 ). Coarsened propensity scores and hybrid estimators for missing data and causal inference . International Statistical Review 83 449--471 . barticle

    2015

  88. [96]

    , Luo , Shikai S

    barticle [author] Zhou , Fan F. , Luo , Shikai S. , Qie , Xiaohu X. , Ye , Jieping J. Zhu , Hongtu H. ( 2021 ). Graph-based equilibrium metrics for dynamic supply--demand systems with applications to ride-sourcing platforms . Journal of the American Statistical Association 116...

    2021

  89. [97]

    , Hanna , Josiah P

    binproceedings [author] Zhou , Hongyi H. , Hanna , Josiah P. J. P. , Zhu , Jin J. , Yang , Ying Y. Shi , Chengchun C. ( 2025 ). Demystifying the Paradox of IS with an Estimated History-Dependent Behavior Policy in OPE . In International conference on machine learning . PMLR . ...

    2025

  90. [98]

    write newline

    " write newline "" before.all 'output.state := FUNCTION format.url url empty "" url if FUNCTION article output.bibitem format.authors "author" output.check author format.key output output.year.check new.block format.title "title" output.check new.block crossref missing format....

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.