Synthesizing Evidence: Data-Pooling as a Tool for Treatment Selection in Online Experiments

Chengzhang Li; Renyu Zhang; Ying Rong; Zhenkang Peng

REVIEW 3 major objections 3 minor 68 references

Synthesizing Evidence: Data-Pooling as a Tool for Treatment Selection in Online Experiments

T0 review · 3 major / 3 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper introduces and validates DPTR, a data-pooling framework that claims lower treatment-effect estimation error than difference-in-means or per-experiment OLS in non-overlapping online experiments, with the advantage growing as the nu

desk verdict Worth a referee: practical data-pooling idea with an honest scope, but the headline dominance claim likely hides a cross-experiment homogeneity condition that isn't stated. read the letter →

arxiv 2508.10331 v3 pith:PDJEYQQU submitted 2025-08-14 stat.ME

Zhenkang Peng , Chengzhang Li , Ying Rong , Renyu Zhang This is my paper

classification stat.ME

keywords datapoolingtreatmentroll-outrandomizedexperimentsonlineplatformsheterogeneouseffectsoverlappingdifference-in-meansordinaryleastsquares

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to show that online platforms can make better treatment roll-out decisions by pooling data across many experiments instead of estimating each experiment separately. It introduces DPTR, a framework that links treatment effects to covariates through a linear model and borrows strength across experiments to reduce estimation variability. Under non-overlapping traffic with a linear specification, DPTR is proven to beat difference-in-means and ordinary least squares, and the gap widens as the experiment count grows. Simulations extend the evidence to overlapping traffic, rich covariates, and nonlinear specifications, and two real platform datasets illustrate subgroup-level and cross-experiment roll-out.

What carries the argument

The machinery is the DPTR estimator, which pools observations across experiments through a shared linear model linking expected treatment effects to covariates, then uses the pooled fit to produce per-experiment or per-subgroup treatment estimates for roll-out. The theoretical engine is a variance-reduction argument: under the correct linear specification, pooling shrinks estimation noise, and the paper proves this yields lower error than difference-in-means and OLS in non-overlapping experiments.

What would settle it

Simulate $K$ non-overlapping experiments whose true treatment effects follow a nonlinear function of a covariate, e.g., $\tau_k = \beta_0 + \beta_1 x_k + \beta_2 x_k^2$, estimate with DPTR (which assumes linearity) and with per-experiment OLS, and compare mean squared error of the treatment-effect estimates; if DPTR does not beat OLS for large $K$, the theoretical dominance does not extend beyond the linear specification.

Watch

Extended reading notes

Core claim

The central claim is that DPTR dominates difference-in-means and per-experiment OLS in the non-overlapping linear setting: pooling data from many experiments reduces mean squared estimation error for treatment effects, and the advantage is largest when the number of experiments is large. The paper also claims practical adaptability — overlapping traffic, nonlinear specifications, and covariate-rich subgroups — supported by simulation and by two real-world experimental datasets. The result matters because it turns a platform's accumulated experiment logs into a single, lower-noise estimate for each roll-out decision, rather than leaving each experiment to speak alone.

Load-bearing premise

The claimed gains depend on the linear model that links treatment effects to covariates across experiments being correctly specified; if that model is wrong, the pooled estimator's bias could outweigh the variance reduction and make per-experiment OLS better.

Editorial extensions

If this is right

Platforms running many small concurrent experiments can adopt DPTR to reduce roll-out decision errors, with the benefit increasing as more experiments are pooled.
In overlapping-experiment settings, DPTR still provides usable treatment-selection guidance, per simulations, enabling coordinated policy deployment across experiments.
Within a single experiment, DPTR supports customized roll-out for covariate-defined subgroups rather than a one-size-fits-all treatment choice.
The theoretical dominance over difference-in-means and OLS is established for non-overlapping linear settings; for overlapping traffic, rich covariates, and nonlinear specifications, the paper's evidence is simulation- and data-based.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A natural next test is to run DPTR against hierarchical Bayesian or empirical-Bayes pooling estimators on the same platform logs; the comparison would show whether the linear link function or the pooling itself drives the gains.
If the linear cross-experiment model is only mildly misspecified, DPTR might still win in small-per-experiment samples because the variance reduction can outweigh bias; this bias-variance trade-off is not characterized in the paper and is worth quantifying.
The framework could be extended to sequential roll-out, where pooled estimates are updated as new experiments finish, making the method adaptive over time rather than a one-shot pooling exercise.
Platforms with heavy overlap should check DPTR's covariance adjustments carefully, since overlapping traffic violates independence assumptions that make the non-overlapping proof clean.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The practical idea is worth taking seriously: pool data across experiments instead of analyzing each one separately, and use the pooled estimates to make roll-out decisions. The authors are honest about the theory—it covers non-overlapping experiments with linear specifications only—and they back it with simulations and two real datasets. That is a solid structure, and the problem (high-variance treatment-effect estimates on platforms) is real.

The novelty is moderate. Pooling across studies is old (meta-analysis, hierarchical models), but the specific framing of roll-out decisions and the handling of overlapping traffic on online platforms is a reasonable new synthesis. The paper would be stronger with a brief positioning against those literatures; the abstract cites no prior work.

The main concern is the one the stress-test note flags. The headline claim is that DPTR beats difference-in-means and OLS under non-overlapping experiments, especially when the number of experiments is large. That only makes sense if the treatment-effect surface is shared, or if the estimand is an average across experiments. If each experiment has its own CATE tau_e(x) = gamma_e^T x and the pooled model forces a common gamma, the bias from (gamma_e - gamma)^T x does not vanish as K grows; it just trades variance for bias. So the 'surpasses' claim needs a stated homogeneity condition or an explicit aggregate estimand. The abstract mixes two goals: guiding roll-out within a single experiment versus coordinating deployments across experiments. These have different target parameters. This is not a fatal flaw if the theory actually conditions properly—but the abstract doesn't say so, and it is the load-bearing part of the paper.

Minor: 'surpasses' is vague—MSE? bias? Which baseline? A precise comparison is needed. Also the empirical results are only mentioned, not shown in the accessible text.

Net: this is a paper for researchers and practitioners in online experimentation. The idea is practically motivated, the validation plan is appropriate, and the authors are not overselling the theory's scope. The central theoretical claim needs a careful look at cross-experiment homogeneity. It deserves a serious referee—send it to review, and make sure the reviewers check that assumption. I wouldn't cite it as a reliable method until that condition is clarified, but it is worth reading.

Referee Report

3 major / 3 minor

Summary. The paper proposes the Data-Pooling Treatment Roll-out (DPTR) framework, which pools data across randomized experiments to reduce estimation variability and support treatment selection / policy roll-out on online platforms. The abstract claims three forms of validation: (a) a theoretical result that DPTR outperforms difference-in-means and ordinary least squares under non-overlapping experiments, especially with many experiments; (b) synthetic simulations covering overlapping traffic, rich covariates, and nonlinear specifications; and (c) empirical applications to two real-world experimental datasets. The manuscript text visible in the submission states that formal theoretical guarantees are limited to non-overlapping experiments under linear specifications, with overlapping and nonlinear cases evaluated only by simulation.

Significance. If the theoretical superiority claim holds under clearly stated conditions, DPTR would be a practically valuable contribution: it offers a principled way to borrow strength across experiments, potentially lowering the variance of treatment-effect estimates used for roll-out decisions. A notable strength is that the full text explicitly restricts the formal theory to non-overlapping, linear settings, which is more honest than the abstract alone. The paper also targets a real industrial problem and includes empirical demonstrations. However, the central theoretical claim, as stated in the abstract, is not yet accompanied by the conditions needed to make it correct; in particular, the role of cross-experiment effect homogeneity is unresolved. The significance therefore hinges on whether the authors can either prove the claim under weaker, explicitly stated assumptions or reframe the claim around an average estimand.

major comments (3)

[Abstract and theoretical claim] The abstract states: 'theoretical analysis shows that DPTR surpasses the traditional difference-in-mean and ordinary least squares methods under non-overlapping experiments, particularly when the number of experiments is large.' This claim appears to require a condition that is not stated anywhere in the accessible text. If each experiment e has a CATE tau_e(x)=gamma_e^T x and DPTR fits a common gamma across experiments, then the pooled estimate of gamma_e incurs bias (gamma_e-gamma)^T x that does not vanish as the number of experiments K grows; increasing K only reduces variance. Per-experiment OLS remains unbiased for gamma_e. Thus DPTR's MSE advantage is not unconditional and can fail when the gamma_e are heterogeneous. Please state the precise assumption (e.g., homogeneous coefficients, or a transfer-learning model) under which the theorem is proved, and ensure the abstract reflects
[Abstract and Introduction] The target estimand is ambiguous. The abstract promises both 'guiding customized policy roll-outs for subgroups within a single experiment' and 'coordinating policy deployments across multiple experiments.' These are different inferential targets: the former requires per-experiment conditional average treatment effects, while the latter may justify an average or pooled effect. The theoretical result should specify whether DPTR estimates each experiment's own CATE or a cross-experiment average. This ambiguity is load-bearing because the bias-variance tradeoff changes entirely depending on the estimand.
[Abstract and full-text scope statement] The full text explicitly says formal theoretical guarantees are for 'non-overlapping experiments under linear specifications,' which is a strength. However, the abstract's wording—'regardless of linear or nonlinear model specifications' and 'effectively accommodate both overlapping and non-overlapping traffic scenarios'—could be read as claiming theoretical support for those settings. Please clarify in the abstract that the overlapping and nonlinear results come from simulations only, reserving the word 'surpasses' for the setting where a theorem is actually proved.

minor comments (3)

[Abstract] The phrase 'difference-in-mean' appears where 'difference-in-means' is standard; please use consistent terminology.
[Abstract and intro] 'Key words' should be capitalized as 'Keywords' for the metadata block.
[Notation] When the theoretical result is introduced, please define the symbols used for the number of experiments (K), per-experiment sample sizes (n_e), and the treatment-effect coefficient (gamma_e) before the main theorem, so that the conditions are unambiguous.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found in the accessible portion; the core claim is an estimator comparison against external baselines, and no derivation in the provided text reduces to its inputs.

full rationale

The provided manuscript text (abstract and first page of the introduction) contains no equations, no fitted parameters secretly renamed as predictions, and no load-bearing self-citation. The central claim is that the proposed DPTR framework outperforms difference-in-means and ordinary least squares under non-overlapping experiments, with simulations and real-data applications as additional validation. This is a comparative claim against standard external baselines, not a result that is true by construction. The potential concern that the theoretical guarantee may require an unstated cross-experiment homogeneity condition is a correctness or robustness issue, not a circularity issue: it concerns whether the assumptions match reality, not whether the derivation assumes its own conclusion. No step in the accessible text defines DPTR in terms of the quantities it claims to predict, nor does it fit a parameter to a subset of data and then call the fitted value a prediction. No self-citation is visible in the accessible portion. Therefore the appropriate finding is no significant circularity, score 0.

Assumptions & free parameters 0 free parameters · 2 assumptions · 0 invented entities

Full text not available; free parameters cannot be enumerated from the abstract alone. The axioms listed are the minimal background assumptions stated in the abstract.

assumptions (2)

standard math OLS and difference-in-means provide unbiased per-experiment treatment effect estimators.
The abstract compares DPTR against these baselines; this standard fact is assumed.
domain assumption The outcome model is linear in covariates for the theoretical analysis.
The theoretical guarantee is stated only under linear specifications (abstract).

how reviews work

0 comments

Cite this review

Pith. "Pith review of Synthesizing Evidence: Data-Pooling as a Tool for Treatment Selection in Online Experiments." pith.science (2026). https://pith.science/paper/PDJEYQQU

@misc{pith2026250810331,
  author       = {Pith},
  title        = {Pith review of: Synthesizing Evidence: Data-Pooling as a Tool for Treatment Selection in Online Experiments},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PDJEYQQU}},
  note         = {Machine review of arXiv:2508.10331}
}

read the original abstract

Randomized experiments are the gold standard for causal inference but face significant challenges in business applications, including limited traffic allocation, the need for heterogeneous treatment effect estimation, and the complexity of managing overlapping experiments. These factors lead to high variability in treatment effect estimates, making data-driven policy roll out difficult. To address these issues, we introduce the data pooling treatment roll-out (DPTR) framework, which enhances policy roll-out by pooling data across experiments rather than focusing narrowly on individual ones. DPTR can effectively accommodate both overlapping and non-overlapping traffic scenarios, regardless of linear or nonlinear model specifications. We demonstrate the framework's robustness through a three-pronged validation: (a) theoretical analysis shows that DPTR surpasses the traditional difference-in-mean and ordinary least squares methods under non-overlapping experiments, particularly when the number of experiments is large; (b) synthetic simulations confirm its adaptability in complex scenarios with overlapping traffic, rich covariates and nonlinear specifications; and (c) empirical applications to two experimental datasets from real world platforms, demonstrating its effectiveness in guiding customized policy roll-outs for subgroups within a single experiment, as well as in coordinating policy deployments across multiple experiments with overlapping scenarios. By reducing estimation variability to improve decision-making effectiveness, DPTR provides a scalable, practical solution for online platforms to better leverage their experimental data in today's increasingly complex business environments.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

68 extracted references · 59 canonical work pages

[1]

, " * write output.state after.block = add.period write newline

ENTRY address author booktitle chapter doi edition editor eid howpublished institution isbn issn journal key month note number organization pages publisher school series title type url volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.all := #1...
[2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in "" FUNCTION format.date year ...
[3]

arXiv preprint arXiv:2306.13681

Abadie A, Agarwal A, Imbens G, Jia S, McQueen J, Stepaniants S (2023) Estimating the value of evidence-based decision making. arXiv preprint arXiv:2306.13681

arXiv 2023
[4]

Amemiya T (1985) Advanced econometrics (Harvard university press)

work page 1985
[5]

Angrist JD, Pischke JS (2009) Mostly harmless econometrics: An empiricist's companion (Princeton university press)

work page 2009
[6]

://arxiv.org/abs/2006.09676

Athey S, Chetty R, Imbens G (2025) Using experiments to correct for selection in observational studies. ://arxiv.org/abs/2006.09676

arXiv 2025
[7]

Proceedings of the National Academy of Sciences 113(27):7353--7360, ://dx.doi.org/10.1073/pnas.1510489113

Athey S, Imbens G (2016) Recursive partitioning for heterogeneous causal effects. Proceedings of the National Academy of Sciences 113(27):7353--7360, ://dx.doi.org/10.1073/pnas.1510489113

doi:10.1073/pnas.1510489113 2016
[8]

Management Science 67(5):2964--2984

Bastani H (2021) Predicting with proxies: Transfer learning in high dimension. Management Science 67(5):2964--2984

work page 2021

Show all 68 references

[9]

Management Science 69(7):3759--3777

Bojinov I, Simchi-Levi D, Zhao J (2023) Design and analysis of switchback experiments. Management Science 69(7):3759--3777

2023
[10]

https://partner.booking.com/en-us/click-magazine/industry-perspectives/role-experimentation-bookingcom, retrieved on June 7, 2025

Booking (2019) The role of experimentation at booking.com. https://partner.booking.com/en-us/click-magazine/industry-perspectives/role-experimentation-bookingcom, retrieved on June 7, 2025

2019
[11]

(1978) Statistics for experimenters, volume 664 (John Wiley and sons New York)

Box GE, Hunter WH, Hunter S, et al. (1978) Statistics for experimenters, volume 664 (John Wiley and sons New York)

1978
[12]

The Annals of Mathematical Statistics 42(3):855--903

Brown LD (1971) Admissible estimators, recurrent diffusions, and insoluble boundary value problems. The Annals of Mathematical Statistics 42(3):855--903

1971
[13]

Chicago Booth Research Paper (21-17)

Candogan O, Chen C, Niazadeh R (2021) Correlated cluster-based randomized experiments: Robust variance minimization. Chicago Booth Research Paper (21-17)

2021
[14]

Machine learning 28:41--75

Caruana R (1997) Multitask learning. Machine learning 28:41--75

1997
[15]

https://blog.statsig.com/embracing-overlapping-a-b-tests-and-the-danger-of-isolating-experiments-cb0a69e09d3, retrieved on August 1, 2025

Chan T (2021) Embrace overlapping a/b tests and avoid the dangers of isolating experiments. https://blog.statsig.com/embracing-overlapping-a-b-tests-and-the-danger-of-isolating-experiments-cb0a69e09d3, retrieved on August 1, 2025

2021
[16]

Available at SSRN 4878048

Chen Y, Cui X, Li A, Wu B, Yang L (2024) The role of digital platforms in data markets: How data sharing through advanced analytics empowers small business innovation. Available at SSRN 4878048

2024
[17]

The Econometrics Journal C1--C68

Chernozhukov V, Chetverikov D, Demirer M, Duflo E, Hansen C, Newey W, Robins J (2018) Double/debiased machine learning for treatment and structural parameters. The Econometrics Journal C1--C68

2018
[18]

Operations Research 65(6):1722--1731

Cheung WC, Simchi-Levi D, Wang H (2017) Dynamic pricing and demand learning with limited price experimentation. Operations Research 65(6):1722--1731

2017
[19]

Available at SSRN 4244086

Chitla S, Jagabathula S, Venkataraman A (2025) Improving demand prediction by accounting for unobserved factors. Available at SSRN 4244086

2025
[20]

Management Science 66(3):1071--1094

Cui R, Li J, Zhang DJ (2020) Reducing discrimination with reviews in the sharing economy: Evidence from field experiments on airbnb. Management Science 66(3):1071--1094

2020
[21]

CXL (2020) Can you run multiple a/b tests at the same time? https://cxl.com/blog/can-you-run-multiple-ab-tests-at-the-same-time/?gad_source=1&gad_campaignid=22595223963&gbraid=0AAAAADD4CBfgn732DU6wl5src48M5r9kc&gclid=CjwKCAjwy7HEBhBJEiwA5hQNopKI8gUQZBcywPpG2B3hvAHpgT7RxnCsNfHi...

2020
[22]

Journal of the Royal Statistical Society Series B: Statistical Methodology 77(4):727--753

Dasgupta T, Pillai NS, Rubin DB (2015) Causal inference from 2k factorial designs by using potential outcomes. Journal of the Royal Statistical Society Series B: Statistical Methodology 77(4):727--753

2015
[23]

Proceedings of the AdKDD and TargetAd Workshop, KDD, London,United Kingdom, August, 20, 2018 (ACM)

Diemert E, Artem B, Renaudin C, Massih-Reza A (2018) A large scale benchmark for uplift modeling. Proceedings of the AdKDD and TargetAd Workshop, KDD, London,United Kingdom, August, 20, 2018 (ACM)

2018
[24]

Scientific American 236(5):119--127

Efron B, Morris C (1977) Stein's paradox in statistics. Scientific American 236(5):119--127

1977
[25]

arXiv preprint arXiv:2010.14694

Farrell MH, Liang T, Misra S (2020) Deep learning for individual heterogeneity: An automatic inference framework. arXiv preprint arXiv:2010.14694

2020 arXiv
[26]

working paper

Feng Q, Li L, Shanthikumar JG (2023) Transfer learning, cross learning and co-learning across newsvendor systems with operational data analytics (ODA) . working paper

2023
[27]

Operations Research 71(3):876--894

Gallino S, Karacaoglu N, Moreno A (2023) Need for speed: The impact of in-process delays on customer behavior in online retail. Operations Research 71(3):876--894

2023
[28]

Greene WH (2003) Econometric analysis (Pearson education india)

2003
[29]

Marketing Science 43(2):378--391

Gui GZ (2024) Combining observational and experimental data to improve efficiency using imperfect instruments. Marketing Science 43(2):378--391

2024
[30]

Management Science 68(3):1595--1615

Gupta V, Kallus N (2022) Data pooling in stochastic optimization. Management Science 68(3):1595--1615

2022
[31]

Bayesian Analysis 15(3):965--1056

Hahn PR, Murray JS, Carvalho CM (2020) Bayesian regression tree models for causal inference: Regularization, confounding, and heterogeneous effects (with discussion). Bayesian Analysis 15(3):965--1056

2020
[32]

Journal of the Royal Statistical Society Series B: Statistical Methodology 87(2):362--388

Imbens G, Kallus N, Mao X, Wang Y (2025) Long-term causal inference under persistent confounding via data combination. Journal of the Royal Statistical Society Series B: Statistical Methodology 87(2):362--388

2025
[33]

Imbens GW, Rubin DB (2015) Causal inference in statistics, social, and biomedical sciences (Cambridge university press)

2015
[34]

Operations Research

Jiang Z, Li J (2025) Instrumenting while experimenting: An empirical method for competitive pricing at scale. Operations Research

2025
[35]

Management Science 68(10):7069--7089

Johari R, Li H, Liskovich I, Weintraub GY (2022) Experimental design in two-sided platforms: An analysis of bias. Management Science 68(10):7069--7089

2022
[36]

Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining, 1168--1176

Kohavi R, Deng A, Frasca B, Walker T, Xu Y, Pohlmann N (2013) Online controlled experiments at large scale. Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining, 1168--1176

2013
[37]

Kohavi R, Tang D, Xu Y (2020) Trustworthy online controlled experiments: A practical guide to a/b testing (Cambridge University Press)

2020
[38]

Harvard business review 95(5):74--82

Kohavi R, Thomke S (2017) The surprising power of online experiments. Harvard business review 95(5):74--82

2017
[39]

199–213, EC '19 (New York, NY, USA: Association for Computing Machinery), ISBN 9781450367929, ://dx.doi.org/10.1145/3328526.3329558

Lada A, Peysakhovich A, Aparicio D, Bailey M (2019) Observational data for heterogeneous treatment effects with application to recommender systems. 199–213, EC '19 (New York, NY, USA: Association for Computing Machinery), ISBN 9781450367929, ://dx.doi.org/10.1145/3328526.3329558

2019
[40]

Manufacturing & Service Operations Management

Lei D, Qi Y, Liu S, Geng D, Zhang J, Hu H, Shen ZJM (2024) Pooling and boosting for demand prediction in retail: A transfer learning approach. Manufacturing & Service Operations Management

2024
[41]

The Quarterly Journal of Economics 130(4):1941--1973

Lewis RA, Rao JM (2015) The unfavorable economics of measuring the returns to advertising. The Quarterly Journal of Economics 130(4):1941--1973

2015
[42]

Luca M, Bazerman MH (2021) The power of experiments: Decision making in a data-driven world (Mit Press)

2021
[43]

https://www.microsoft.com/en-us/research/articles/a-b-interactions-a-call-to-relax/, retrieved on August 1, 2025

Microsoft (2023) A/b interactions: A call to relax. https://www.microsoft.com/en-us/research/articles/a-b-interactions-a-call-to-relax/, retrieved on August 1, 2025

2023
[44]

Management Science 68(3):1737--1755

Nabi S, Nassif H, Hong J, Mamani H, Imbens G (2022) Bayesian meta-prior learning using empirical bayes. Management Science 68(3):1737--1755

2022
[45]

Advances in Neural Information Processing Systems 34:6466--6477

Nandy P, Venugopalan D, Lo C, Chatterjee S (2021) A/b testing for recommender systems in a two-sided marketplace. Advances in Neural Information Processing Systems 34:6466--6477

2021
[46]

Operations research 59(3):713--728

Natarajan K, Teo CP, Zheng Z (2011) Mixed 0-1 linear programs under objective uncertainty: A completely positive representation. Operations research 59(3):713--728

2011
[47]

Available at SSRN 5245482

Ni T (2025) Decision analytics of switchback experiments: A robust optimization approach. Available at SSRN 5245482

2025
[48]

Available at SSRN 4466598

Ni T, Bojinov I, Zhao J (2023) Design of panel experiments with spatial and temporal interference. Available at SSRN 4466598

2023
[49]

Canadian Journal of Statistics 51(2):444--468

Pashley NE, Bind MAC (2023) Causal inference for multiple treatments using fractional factorial designs. Canadian Journal of Statistics 51(2):444--468

2023
[50]

Sociological methodology 111--163

Raftery AE (1995) Bayesian model selection in social research. Sociological methodology 111--163

1995
[51]

Biometrics 79(4):2961--2973

Rosenman ET, Basse G, Owen AB, Baiocchi M (2023) Combining observational and experimental datasets using shrinkage estimators. Biometrics 79(4):2961--2973

2023
[52]

Management Science

Simester D, Timoshenko A, Zoumpoulis SI (2025) A sample size calculation for training and certifying targeting policies. Management Science

2025
[53]

Management Science 70(8):5115--5130

Song Y, Sun T (2024) Ensemble experiments to optimize interventions along the customer journey: A reinforcement learning approach. Management Science 70(8):5115--5130

2024
[54]

Stein C (1956) Inadmissibility of the usual estimator for the mean of a multivariate normal distribution. Proceedings of the Third Berkeley Symposium on Mathematical Statistics and Probability, Volume 1: Contributions to the Theory of Statistics, volume 3, 197--207 (USA: Unive...

1956
[55]

Proceedings of the 16th ACM SIGKDD international conference on Knowledge discovery and data mining, 17--26

Tang D, Agarwal A, O'Brien D, Meyer M (2010) Overlapping experiment infrastructure: More, better, faster experimentation. Proceedings of the 16th ACM SIGKDD international conference on Knowledge discovery and data mining, 17--26

2010
[56]

Technical report, cemmap working paper

Tetenov A (2016) An economic theory of statistical testing. Technical report, cemmap working paper

2016
[57]

Marketing Science 37(4):530--552

Ursu RM (2018) The power of rankings: Quantifying the effect of rankings on online consumer search and purchase decisions. Marketing Science 37(4):530--552

2018
[58]

arXiv preprint arXiv:1011.3027

Vershynin R (2010) Introduction to the non-asymptotic analysis of random matrices. arXiv preprint arXiv:1011.3027

2010 arXiv
[59]

Wager S (2024) Causal inference: A statistical learning approach (In preparation), ://web.stanford.edu/swager/causal_inf_book.pdf

2024
[60]

Cambridge Series in Statistical and Probabilistic Mathematics (Cambridge University Press), ://dx.doi.org/10.1017/9781108627771.002

Wainwright MJ (2019) Basic tail and concentration bounds, 21–57. Cambridge Series in Statistical and Probabilistic Mathematics (Cambridge University Press), ://dx.doi.org/10.1017/9781108627771.002

2019 doi
[61]

Wu CJ, Hamada MS (2011) Experiments: planning, analysis, and optimization (John Wiley & Sons)

2011
[62]

The KDD'23 Workshop on Causal Discovery, Prediction and Decision, 115--131 (PMLR)

Xiong R, Chin A, Taylor S (2023) Bias-variance tradeoffs for designing simultaneous temporal experiments. The KDD'23 Workshop on Causal Discovery, Prediction and Decision, 115--131 (PMLR)

2023
[63]

Xiong T, Wang Y, Zheng S (2020) Orthogonal traffic assignment in online overlapping a/b tests

2020
[64]

Management Science 69(7):3838--3860

Ye Z, Zhang DJ, Zhang H, Zhang R, Chen X, Xu Z (2022) Cold start to improve market thickness on online advertising platforms: Data-driven algorithms and field experiments. Management Science 69(7):3838--3860

2022
[65]

Management Science, Forthcoming

Ye Z, Zhang Z, Zhang D, Zhang H, Zhang RP (2025) Deep-learning-based causal inference for large-scale combinatorial experiments: Theory and empirical evidence. Management Science, Forthcoming

2025
[66]

Management Science 69(9):5189--5208

Zeng Z, Dai H, Zhang DJ, Zhang H, Zhang R, Xu Z, Shen ZJM (2023) The impact of social nudges on user-generated content for social network platforms. Management Science 69(9):5189--5208

2023
[67]

arXiv preprint arXiv:2406.14380

Zhan R, Han S, Hu Y, Jiang Z (2024) Estimating treatment effects under recommender interference: A structured neural networks approach. arXiv preprint arXiv:2406.14380

2024
[68]

Available at SSRN

Zhang Z, Zeng Z, Zhan R, Zhang D (2025) Personalized policy learning through discrete experimentation: Theory and empirical evidence. Available at SSRN

2025

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.

[1] [1]

, " * write output.state after.block = add.period write newline

ENTRY address author booktitle chapter doi edition editor eid howpublished institution isbn issn journal key month note number organization pages publisher school series title type url volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.all := #1...

[2] [2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in "" FUNCTION format.date year ...

[3] [3]

arXiv preprint arXiv:2306.13681

Abadie A, Agarwal A, Imbens G, Jia S, McQueen J, Stepaniants S (2023) Estimating the value of evidence-based decision making. arXiv preprint arXiv:2306.13681

arXiv 2023

[4] [4]

Amemiya T (1985) Advanced econometrics (Harvard university press)

work page 1985

[5] [5]

Angrist JD, Pischke JS (2009) Mostly harmless econometrics: An empiricist's companion (Princeton university press)

work page 2009

[6] [6]

://arxiv.org/abs/2006.09676

Athey S, Chetty R, Imbens G (2025) Using experiments to correct for selection in observational studies. ://arxiv.org/abs/2006.09676

arXiv 2025

[7] [7]

Proceedings of the National Academy of Sciences 113(27):7353--7360, ://dx.doi.org/10.1073/pnas.1510489113

Athey S, Imbens G (2016) Recursive partitioning for heterogeneous causal effects. Proceedings of the National Academy of Sciences 113(27):7353--7360, ://dx.doi.org/10.1073/pnas.1510489113

doi:10.1073/pnas.1510489113 2016

[8] [8]

Management Science 67(5):2964--2984

Bastani H (2021) Predicting with proxies: Transfer learning in high dimension. Management Science 67(5):2964--2984

work page 2021