Pith. sign in

REVIEW 3 major objections 4 minor 46 references

Robust Causal Inference for Incremental Return on Ad Spend with Randomized Paired Geo Experiments

T0 review · 3 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Randomized paired geo experiments turn iROAS estimation into a robust symmetry problem, and the Trimmed Match estimator solves it by discarding poorly matched pairs.

desk verdict A clean, practical estimator for a real advertising measurement problem, with one load-bearing assumption that the paper verifies only weakly. read the letter →

arxiv 1908.02922 v3 pith:APWPWXBF submitted 2019-08-08 stat.ME

Aiyou Chen , Timothy C. Au This is my paper · ORCID
classification stat.ME MSC 62G0562G3562G20
keywords incrementalreturnonadspendgeoexperimentscausalinferenceinterferencetrimmedmeaneffectratiodistribution-freeestimationheavy-taileddata
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper is trying to establish that in a randomized paired geo experiment, the incremental return on ad spend (iROAS) can be estimated without modeling the heavy-tailed distribution of ad spend and response. Its key reduction is Proposition 2: under two assumptions, the residual difference $Y_i - \theta^* X_i$ between the treated and control geo in each pair is independent across pairs and symmetric about zero. That turns the ratio-estimation problem into a one-dimensional location problem, and solving a trimmed-mean version of that location equation gives the Trimmed Match estimator, which discards the worst-matched pairs. The authors argue that this estimator is distribution-free and interpretable, and that it is often more efficient than the empirical, sign-test, and Wilcoxon estimators when geo sizes are highly heterogeneous; the claim is supported by simulations and three real experiments.

What carries the argument

The machinery is the residual identity $\epsilon_i(\theta) = Y_i - \theta X_i = (Z_{i1} - Z_{i2}) A_i$, where $A_i$ is the fair-coin sign deciding which geo in pair $i$ receives treatment. Because $Z_{i1}$ and $Z_{i2}$ are non-random under Assumption 0, multiplication by $A_i$ makes $\epsilon_i(\theta^*)$ symmetric about zero and independent across pairs; this licenses sign-test, Wilcoxon, and trimmed-mean estimating equations for $\theta^*$. The trimmed-mean equation $\bar{\epsilon}_{n\lambda}(\theta) = 0$ then does double duty: it defines the estimator as a ratio of trimmed sums of $Y_i$ and $X_i$, and it identifies poorly matched pairs through large $|\epsilon_i|$ values for removal. Lemma 2, stating that the ordering of the residuals changes only at $\theta_{ij} = (y_j - y_i)/(x_j - x_i)$, is what makes the computation $O(n^2 \log n)$.

What would settle it

One direct test: simulate a randomized paired geo experiment with known heterogeneous unit-level iROAS values (alternating $\theta_g = \theta_0(1 \pm \delta)$ with $\delta$ near 1, as the paper's own sensitivity analysis does), compute the residuals $\epsilon_i(\theta^*)$ at the true $\theta^*$, and apply a Wilcoxon signed-rank symmetry test at the paper's sample sizes; the residual distribution will be detectably asymmetric, and the trimmed-mean estimate will shift with the realized assignment, showing that both the symmetry claim and the estimator's target depend on Assumption 1 holding.

Watch

Extended reading notes

Core claim

The central claim is Proposition 2: with randomized paired assignment, Assumption 0 (the uninfluenced response $Z_g = R_g - \theta^* S_g$ is invariant to all treatment assignments, so budget-constraint interference enters only through observed spend) and Assumption 1 (unit-level iROAS is constant, $\theta_g = \theta^*$ for all geos) imply that the residuals $\epsilon_i(\theta^*) = Y_i - \theta^* X_i$ are mutually independent and symmetrically distributed about zero. Consequently $\theta^*$ is the root of the trimmed-mean equation $\bar{\epsilon}_{n\lambda}(\theta) = 0$, and the Trimmed Match estimator $\hat{\theta}^{(\mathrm{trim})}_\lambda$ is defined as the root that minimizes symmetric deviation; when it exists it equals the ratio of the sums of $Y_i$ and $X_i$ over the untrimmed pairs. The paper also supplies a data-driven trim rate chosen by minimizing confidence-interval width, proves existence when the trimmed sum of $X_i$ is nonzero, and provides an $O(n^2 \log n)$ algorithm.

Load-bearing premise

The load-bearing premise is Assumption 1, that every geographic unit has the same true incremental return on ad spend; if unit-level returns differ, the residual symmetry that powers the whole estimator breaks and the method estimates an assignment-dependent weighted average rather than the population ratio, while the paper's own check of this premise is indirect because it tests residuals computed with the estimated value of that ratio.

Editorial extensions

If this is right

  • Under the paper's two assumptions, iROAS inference becomes distribution-free: any symmetry-based estimating equation applied to $\epsilon_i(\theta)$ yields valid point and interval estimates without modeling the joint spend-response distribution.
  • Trimmed Match can be more efficient than the empirical ratio, the sign-test estimator, and the Wilcoxon estimator when geo sizes are heavy-tailed; the simulated log-normal and half-Cauchy scenarios show the largest gains.
  • The data-driven trim rate makes the method adaptive: in settings where trimming does not reduce variance, the method can select a trim rate of zero and reduce to the empirical estimator, as in case B of the real studies.
  • The same framework applies to other matched-pairs effect-ratio problems, such as incremental cost-effectiveness ratios, because only the symmetry of residuals is used.
  • An $O(n^2 \log n)$ algorithm and a studentized trimmed-mean $t$ approximation for confidence intervals make the method practical for routine advertiser experiments.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the symmetry reduction is as general as Proposition 2 suggests, other robust location estimators with higher efficiency at heavy tails—Huber-type M-estimators or adaptively weighted trimmed means—could be ported to effect-ratio estimation and would likely beat Trimmed Match in the same simulation designs.
  • The paper's residual-based check of Assumption 1 is indirect, because it tests residuals built from the estimated $\hat{\theta}^{\mathrm{(trim)}}_{\hat{\lambda}}$; a direct test requires a holdout estimate of $\theta^*$ or a permutation distribution that accounts for the estimation step, and until then the constant-iROAS assumption is supported only indirectly.
  • When Assumption 1 fails, Trimmed Match does not collapse but targets an assignment-dependent weighted average of geo-level iROAS values; a natural extension would let $\theta_g$ depend on geo covariates and use the same residual symmetry as a diagnostic instead of an assumption.
  • Because the trim rate is chosen by minimizing confidence-interval width, the method implicitly trades coverage for power; with small $n$, reported intervals should be treated as optimistic unless calibrated by simulation, a point the paper itself raises when discussing undercoverage.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper develops a framework and estimator for the incremental return on ad spend (iROAS) in randomized paired geo experiments. It introduces Assumption 0, that an unobserved 'uninfluenced response' Z_g is invariant to treatment assignments even under budget-induced interference, and Assumption 1, that the unit-level iROAS is constant across geos. Under these assumptions, Proposition 2 shows that the residual ϵ_i(θ*) = Y_i - θ* X_i is symmetric about zero for each pair, reducing iROAS estimation to a univariate location problem. The proposed Trimmed Match estimator solves the trimmed mean equation ϵ̅_{nλ}(θ)=0, with a data-driven choice of trim rate based on confidence interval width. The paper reports simulations showing efficiency gains over empirical, sign-test, and Wilcoxon rank-based estimators, three real case studies, an O(n^2 log n) algorithm for computation, and an existence theorem for the estimator.

Significance. If the two assumptions hold, the paper makes a useful contribution: Proposition 2 is cleanly derived, the estimator is distribution-free and interpretable (it trims poorly matched pairs), and the simulation evidence supports efficiency gains in heavy-tailed settings. The connection to Rosenbaum's instrumental variable framework is insightful, and the availability of a Python implementation is a practical strength. However, the entire inferential edifice rests on Assumption 1, and the manuscript's verification of that assumption is circular, while the sensitivity analysis in Section 7.2 changes the estimand rather than addressing the identification failure. These issues materially affect whether the estimator can be claimed to target the θ* of equation (1.2) in realistic heterogeneous-geo settings.

major comments (3)
  1. [Section 8, footnote 4] The only empirical check of Assumption 1 is a Wilcoxon signed-rank test applied to residuals ϵ_i(θ̂_trim) computed at the estimated parameter. Footnote 4 concedes that these p-values may not be accurate because θ* is estimated, so the test is circular: the residuals are by construction centered at the estimator's root, and the test does not account for estimation uncertainty. This is not a valid verification of symmetry of the true residuals under Assumption 1. The authors should either use a split-sample or resampling procedure that accounts for parameter estimation, or present the Wilcoxon result only as an informal diagnostic, and explicitly state that Assumption 1 remains an untestable identifying assumption.
  2. [Section 7.2] The sensitivity analysis redefines the 'true' θ* via a virtual experiment in which all geos are assigned to treatment with a doubled total incremental budget. This is a different estimand from the θ* defined in equation (1.2) for the actual paired experiment, because under budget-constrained interference the average incremental response and spend in (1.2) are assignment-dependent. Consequently, the robustness claims in Section 7.2 and the abstract, that estimates remain reliable when Assumption 1 is violated, do not apply to the original target. The manuscript should either adopt the virtual-experiment θ* as the explicit parameter of interest throughout, or provide sensitivity results for a well-defined assignment-conditional parameter, with a formal statement of what the estimating equation identifies under heterogeneity.
  3. [Proposition 2 and equation (5.2)] When Assumption 1 is violated, the residual ϵ_i(θ*) includes the additional terms (θ_{i1}-θ*)S_{i1}A_i and (θ_{i2}-θ*)S_{i2}A_i, with roles depending on A_i, which depend on realized ad spend and treatment assignment. These terms are not generally symmetric about zero, so Proposition 2 fails and the trimmed mean equation (5.2) does not identify the population θ* of (1.2); instead, the root is an assignment-dependent weighted average of heterogeneous θ_g. This is a load-bearing gap because the paper's main claim, that Trimmed Match robustly estimates the overall iROAS, requires Assumption 1. The authors should provide a formal analysis of the bias as a function of the degree of heterogeneity and the spend distribution, or explicitly restrict the target to a random-coefficient model in which Assumption 1 is replaced by a defined aggregation.
minor comments (4)
  1. [Section 5.1] The set of untrimmed indices I in equation (5.5) depends on the estimated θ̂, but the manuscript does not specify how ties in ϵ_i(θ) at the trimming boundaries are handled in the point estimator; a tie-breaking rule would make the estimator fully defined.
  2. [Section 6] The data-driven choice of trim rate λ by minimizing confidence interval width is presented as a contribution of independent interest, but no theoretical justification is given for this criterion or for the recommended α0=0.5; the discussion would benefit from at least a heuristic argument or a small asymptotic analysis.
  3. [Appendix A] The proofs of Lemma 2 and Theorem 1 are omitted. Since the algorithm and the existence result are central to the computational and inferential claims, these proofs should be provided in the appendix or a supplementary file rather than referenced as 'straightforward' and 'omitted for conciseness'.
  4. [Section 1, Lemma 1] Lemma 1 is a direct rearrangement of the definition of θ_g in (1.1), so describing it as the basis of a 'novel statistical framework' is somewhat overstated; the novelty lies in the robust estimation strategy under Assumptions 0 and 1, not in the lemma itself.

Circularity Check

0 steps flagged · score 3.0 of 10

Central estimator is not circular, but the real-data check of the key homogeneity assumption is circular.

full rationale

Proposition 2 is a direct consequence of Assumptions 0 and 1, and the Trimmed Match estimator is derived from that proposition by solving a trimmed-mean estimating equation; it is not a renamed input or a fitted parameter masquerading as a prediction. The simulations compare estimators to a known simulation ground truth, so the central efficiency claims do not reduce to the paper's assumptions. The one genuine circularity is localized to Section 8: Assumption 1 is validated by testing symmetry of residuals at the estimated θ*, which is itself fit on the same data, and the paper's footnote admits the p-values may be inaccurate. The Section 7.2 sensitivity analysis changes the estimand (defining θ* through a virtual all-treated experiment when Assumption 1 is violated), which is a target-miss concern rather than a circularity. The data-driven trim-rate selection also uses the same data, but the paper explicitly discloses the resulting undercoverage risk; this is model-selection overfitting, not a circular derivation. Overall, the estimator's mathematical derivation is self-contained; the circularity affects only the empirical check of the most fragile assumption.

Assumptions & free parameters 2 free parameters · 5 assumptions · 1 invented entities

The central method rests on two domain assumptions (constant unit-level iROAS and invariance of uninfluenced responses), a standard asymptotic approximation for the studentized trimmed mean, and a technical tie-breaking device. The free parameters are the data-driven trim rate λ and the hand-chosen selection level α0 = 0.5. The only invented entity is the latent uninfluenced response Z_g, which is a reparameterization rather than a new physical or empirical quantity.

free parameters (2)
  • trim rate λ = data-driven, varies by dataset
    λ is selected by minimizing the width of a 100(1-α0)% confidence interval computed from the data (Section 6), so it is a data-fitted tuning parameter whose uncertainty is not accounted for in the final interval.
  • α0 (level for trim-rate selection) = 0.5
    Pre-specified level for the confidence interval width used to choose λ; the paper recommends α0 = 0.5 based on numerical simulations (Section 6), but this is a hand-chosen constant.
assumptions (5)
  • domain assumption Assumption 0: The uninfluenced response Z_g is invariant to both its own treatment assignment and the treatment assignment of other geos.
    Introduced in Section 3; it is essential for the symmetry of residualized pair differences and holds trivially under no interference or under specific budget constraints, but it excludes treatment effects on response that do not operate through ad spend.
  • domain assumption Assumption 1: The unit-level iROAS is constant, θ_g = θ* for all geos g.
    Introduced in Section 3; the symmetry of ϵ_i(θ*) requires this homogeneity. The paper does not rigorously verify it, relying instead on sensitivity analysis and an indirect Wilcoxon test in Section 8.
  • domain assumption Spillover interference across geo boundaries is ignorable.
    Section 1 states that spillover can be controlled by using larger geographically clustered units; the paper assumes this form of interference is negligible without formal justification.
  • standard math The studentized trimmed mean statistic T_{nλ}(θ*) approximately follows a t-distribution with n-2m-1 degrees of freedom under mild conditions.
    Invoked in Section 5.2 to set the threshold c for confidence intervals; this is a standard asymptotic result for location problems, assumed to transfer to this estimating-equation context.
  • ad hoc to paper The x_i values are distinct, or ties are broken by adding a small amount of random noise.
    Appendix A, Lemma 2 relies on a strict ordering of x_i to enumerate all possible orderings of residuals; the paper handles ties by random perturbation, which introduces an additional source of randomness.
invented entities (1)
  • Uninfluenced response Z_g
    purpose: A latent intercept in the linear decomposition R_g = Z_g + θ_g S_g, used to reduce the bivariate causal inference problem to a univariate location problem.
    Z_g is a definitional latent variable with no direct falsifiable handle outside the model assumptions; it is not observed independently and its invariance is imposed by Assumption 0.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Robust Causal Inference for Incremental Return on Ad Spend with Randomized Paired Geo Experiments." pith.science (2026). https://pith.science/paper/APWPWXBF

@misc{pith2026190802922,
  author       = {Pith},
  title        = {Pith review of: Robust Causal Inference for Incremental Return on Ad Spend with Randomized Paired Geo Experiments},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/APWPWXBF}},
  note         = {Machine review of arXiv:1908.02922}
}
read the original abstract

Evaluating the incremental return on ad spend (iROAS) of a prospective online marketing strategy (i.e., the ratio of the strategy's causal effect on some response metric of interest relative to its causal effect on the ad spend) has become increasingly more important. Although randomized ``geo experiments'' are frequently employed for this evaluation, obtaining reliable estimates of iROAS can be challenging as oftentimes only a small number of highly heterogeneous units are used. Moreover, advertisers frequently impose budget constraints on their ad spends, which further complicates causal inference by introducing interference between the experimental units. In this paper, we formulate a novel statistical framework for inferring the iROAS of online advertising from randomized paired geo experiment which further motivates and provides new insights into Rosenbaum's arguments on instrumental variables, and we propose and develop a robust, distribution-free and interpretable estimator ``Trimmed Match'', as well as a data-driven choice of the tuning parameter which may be of independent interest. We investigate the sensitivity of Trimmed Match to some violations of its assumptions and show that it can be more efficient than some alternative estimators based on simulated data. We then demonstrate its practical utility with real case studies.

Figures

Figures reproduced from arXiv: 1908.02922 by the authors.

Figure 1
Figure 1. Comparison of each estimator’s performance in terms of a scaled RMSE, where the [PITH_FULL_IMAGE:figures/full_fig_p014_1.png] view at source ↗
Figure 2
Figure 2. The scatter plot of (X, Y ) for each of the three real case studies, where the hori￾zontal and vertical lines pass the origin but the detailed scales of both X and Y are removed to anonymize the experiments. TABLE 4 Summary of the three real case studies in terms of the kurtosis (rounded to the nearest 10) of the empirical distributions, the point estimates and confidence intervals obtained from different estimators… view at source ↗
Figure 3
Figure 3. The Trimmed Match point estimates and confidence intervals as a function of the [PITH_FULL_IMAGE:figures/full_fig_p016_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Boxplots of the residuals, where the residuals are rescaled by their maximum abso [PITH_FULL_IMAGE:figures/full_fig_p016_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

46 extracted references · 42 canonical work pages

  1. [1]

    barticle [author] Angrist , Joshua D. J. D. , Imbens , Guido W. G. W. Rubin , Donald B. D. B. ( 1996 ). Identification of causal effects using instrumental variables . Journal of the American statistical Association 91 444--455 . barticle

    work page 1996

  2. [2]

    , Eckles , Dean D

    barticle [author] Athey , Susan S. , Eckles , Dean D. Imbens , Guido W. G. W. ( 2018 ). Exact p-Values for Network Interference . Journal of the American Statistical Association 113 230-240 . barticle

    work page 2018

  3. [3]

    , Small , Dylan S

    barticle [author] Baiocchi , Mike M. , Small , Dylan S. D. S. , Lorch , Scott S. Rosenbaum , Paul R. P. R. ( 2010 ). Building a stronger instrument in an observational study of perinatal care for premature infants . Journal of the American Statistical Association 105 1285--1296 . barticle

    work page 2010

  4. [4]

    Zhao , Hongwei H

    barticle [author] Bang , Heejung H. Zhao , Hongwei H. ( 2014 ). Cost-effective analysis: a proposal of new reporting standards in statistical analysis . Journal of Biopharmaceutical Statistics 24 443 -- 460 . barticle

    work page 2014

  5. [5]

    barticle [author] Blair , Margaret H M. H. Kuse , Allan R A. R. ( 2004 ). Better practices in advertising can change a cost of doing business to wise investments in the business . Journal of Advertising Research 44 71--89 . barticle

    work page 2004

  6. [6]

    , Nosko , Chris C

    barticle [author] Blake , Thomas T. , Nosko , Chris C. Tadelis , Steven S. ( 2015 ). Consumer Heterogeneity and Paid Search Effectiveness: A Large - Scale Field Experiment . Econometrica 83 155--174 . barticle

    work page 2015

  7. [7]

    , Liu , Hanzhong H

    barticle [author] Bloniarz , Adam A. , Liu , Hanzhong H. , Zhang , Cun-Hui C.-H. , Sekhon , Jasjeet S. J. S. Yu , Bin B. ( 2016 ). Lasso adjustments of treatment effect estimates in randomized experiments . Proceedings of the National Academy of Sciences 113 7383--7390 . barticle

    work page 2016

  8. [8]

    barticle [author] Brodersen , Kay H. K. H. , Gallusser , Fabian F. , Koehler , Jim J. , Remy , Nicolas N. Scott , Steven L. S. L. ( 2015 ). Inferring causal impact using Bayesian structural time-series models . Annals of Applied Statistics 9 247--274 . barticle

    work page 2015

Show all 46 references
  1. [9]

    IAB Internet Advertising Report: 2017 Full Year Results

    bmisc [author] Interactive Advertising Bureau ( 2018 ). IAB Internet Advertising Report: 2017 Full Year Results . bmisc

    2018

  2. [10]

    barticle [author] Chaudhary , Mohammad A. M. A. Stearns , Sally C. S. C. ( 1996 ). Estimating confidence intervals for cost-effectiveness ratios: an example from a randomized trial . Statistics in Medicine 15 1447--1458 . barticle

    1996

  3. [11]

    , Longfils , Marco M

    bmisc [author] Chen , Aiyou A. , Longfils , Marco M. Best , Christoph C. ( 2020 ). The Python library for Trimmed Match and Trimmed Match Design . https://github.com/google/trimmed_match . [Online; accessed 21-April-2021] . bmisc

    2020

  4. [12]

    , Chan , David D

    btechreport [author] Chen , Aiyou A. , Chan , David D. , Perry , Mike M. , Jin , Yuxue Y. , Sun , Yunting Y. , Wang , Yueqing Y. Koehler , Jim J. ( 2018 ). Bias correction for paid search in media mix modeling Technical Report , Google Inc. https://ai.google/research/pubs/pub4...

    2018

  5. [13]

    barticle [author] Dhar , Subhra Sankar S. S. Chaudhuri , Probal P. ( 2012 ). On the derivatives of the trimmed mean . Statistica Sinica 22 655--679 . barticle

    2012

  6. [14]

    , Feller , Avi A

    barticle [author] Ding , Peng P. , Feller , Avi A. Miratrix , Luke L. ( 2016 ). Randomization inference for treatment effect variation . Journal of the Royal Statistical Society: Series B 78 655--671 . barticle

    2016

  7. [15]

    Tucker , Catherine C

    bincollection [author] Goldfarb , Avi A. Tucker , Catherine C. ( 2011 ). Online Advertising . In Advances in Computers , ( Marvin V. M. V. Zelkowitz , ed.) 81 6 , 289--315 . Elsevier . bincollection

    2011

  8. [16]

    , Zettelmeyer , Florian F

    barticle [author] Gordon , Brett B. , Zettelmeyer , Florian F. , Bhargava , Neha N. Chapsky , Dan D. ( 2019 ). A Comparison of Approaches to Advertising Measurement: Evidence from Big Field Experiments at F acebook . Marketing Science 38 193--225 . barticle

    2019

  9. [17]

    ( 1981 )

    barticle [author] Hall , Peter P. ( 1981 ). Large sample properties of Jaeckel's adaptive trimmed mean . Annals of the Institute of Statistical Mathematics 33 449--462 . barticle

    1981

  10. [18]

    bbook [author] Huber , Peter J. P. J. Ronchetti , Elvezio E. ( 2009 ). Robust statistics . John Wiley & Sons, Inc. bbook

    2009

  11. [19]

    barticle [author] Hudgens , Michael G M. G. Halloran , M Elizabeth M. E. ( 2008 ). Toward causal inference with interference . Journal of the American Statistical Association 103 832--842 . barticle

    2008

  12. [20]

    bbook [author] Imbens , Guido W. G. W. Rubin , Donald B. D. B. ( 2015 ). Causal Inference for Statistics, Social, and Biomedical Sciences: An Introduction . Cambridge University Press , New York, NY, USA . bbook

    2015

  13. [21]

    barticle [author] Jaeckel , Louis A. L. A. ( 1971 ). Some flexible estimates of location . The Annals of Mathematical Statistics 42 1540--1552 . barticle

    1971

  14. [22]

    barticle [author] Johnson , Garrett A. G. A. , Lewis , Randall A. R. A. Nubbemeyer , Elmar I. E. I. ( 2017 ). Ghost ads: Improving the economics of measuring online ad effectiveness . Journal of Marketing Research 54 867--884 . barticle

    2017

  15. [23]

    , McAteer , John J

    barticle [author] Kalyanam , Kirthi K. , McAteer , John J. , Marek , Jonathan J. , Hodges , James J. Lin , Lifeng L. ( 2018 ). Cross channel effects of search engine advertising on brick & mortar retail sales: Meta analysis of large scale field experiments on Google.com . Quan...

    2018

  16. [24]

    , Wang , Peng P

    btechreport [author] Kerman , Jouni J. , Wang , Peng P. Vaver , Jon J. ( 2017 ). Estimating Ad Effectiveness using Geo Experiments in a Time-Based Regression Framework Technical Report , Google, Inc. https://ai.google/research/pubs/pub45950 . btechreport

    2017

  17. [25]

    barticle [author] Klaassen , Chris AJ C. A. ( 1987 ). Consistent estimation of the influence function of locally asymptotically linear estimators . The Annals of Statistics 1548--1562 . barticle

    1987

  18. [26]

    barticle [author] Kunzel , Soren R. S. R. , Sekhon , Jasjeet S. J. S. , Bickel , Peter J. P. J. Yu , Bin B. ( 2019 ). Metalearners for estimating heterogeneous treatment effects using machine learning . Proceedings of the National Academy of Sciences 116 4156--4165 . barticle

    2019

  19. [27]

    bbook [author] Lehmann , Erich Leo E. L. ( 2006 ). Nonparametrics: statistical methods based on ranks . Springer . bbook

    2006

  20. [28]

    barticle [author] Lewis , Randall A. R. A. Rao , Justin M. J. M. ( 2015 ). The Unfavorable Economics of Measuring the Returns to Advertising . The Quarterly Journal of Economics 130 1941-1973 . barticle

    2015

  21. [29]

    ( 2013 )

    barticle [author] Lin , Winston W. ( 2013 ). Agnostic notes on regression adjustments to experimental data: Reexamining F reedman's critique . The Annals of Applied Statistics 7 295--318 . barticle

    2013

  22. [30]

    , Small , Dylan S

    barticle [author] Luo , Xi X. , Small , Dylan S. D. S. , Li , Chiang-Shan R. C.-S. R. Rosenbaum , Paul R. P. R. ( 2012 ). Inference With Interference Between Units in an fMRI Experiment of Motor Inhibition . Journal of the American Statistical Association 107 530-541 . barticle

    2012

  23. [31]

    Wager , Stefan S

    barticle [author] Nie , Xinkun X. Wager , Stefan S. ( 2017 ). Quasi-oracle estimation of heterogeneous treatment effects . arXiv preprint arXiv:1712.04912 . barticle

    2017 arXiv

  24. [32]

    , Aydin , Kevin K

    binproceedings [author] Rolnick , David D. , Aydin , Kevin K. , Pouget-Abadie , Jean J. , Kamali , Shahab S. , Mirrokni , Vahab V. Najmi , Amir A. ( 2019 ). Randomized Experimental Design via Geographic Clustering . In Proceedings of the 25th ACM International Conference on Kn...

    2019

  25. [33]

    barticle [author] Rosenbaum , Paul R. P. R. ( 1996 ). Identification of causal effects using instrumental variables: Comment . Journal of the American Statistical Association 91 444--444 . barticle

    1996

  26. [34]

    barticle [author] Rosenbaum , Paul R. P. R. ( 2002 ). Covariance adjustment in randomized experiments and observational studies . Statistical Science 17 286--327 . barticle

    2002

  27. [35]

    barticle [author] Rosenbaum , Paul R P. R. ( 2007 ). Interference Between Units in Randomized Experiments . Journal of the American Statistical Association 102 191-200 . barticle

    2007

  28. [36]

    bbook [author] Rosenbaum , Paul R. P. R. ( 2020 ). Design of Observational Studies, 2nd Edition . Springer Series in Statistics . bbook

    2020

  29. [37]

    R andomization A nalysis of E xperimental D ata in the F isher R andomization T est

    barticle [author] Rubin , Donald B. D. B. ( 1980 ). Discussion of " R andomization A nalysis of E xperimental D ata in the F isher R andomization T est" by B asu . Journal of the American Statistical Association 75 591-593 . barticle

    1980

  30. [38]

    , Vaver , Jon J

    btechreport [author] Sapp , Stephanie S. , Vaver , Jon J. , Schuringa , Jon J. Dropsho , Steven S. ( 2017 ). Near impressions for observational causal ad impact Technical Report , Google Inc. https://ai.google/research/pubs/pub46418 . btechreport

    2017

  31. [39]

    Rosenbaum , Paul R

    barticle [author] Small , Dylan D. Rosenbaum , Paul R. P. R. ( 2008 ). War and Wages: The Strength of Instrumental Variables and Their Sensitivity to Unobserved Biases . Journal of the American Statistical Association 103 924-933 . barticle

    2008

  32. [40]

    barticle [author] Tukey , John W. J. W. McLaughlin , Donald H. D. H. ( 1963 ). Less vulnerable confidence and significance procedures for location based on a single sample: Trimming/Winsorization 1 . Sankhy \=a : The Indian Journal of Statistics, Series A 25 331--352 . barticle

    1963

  33. [41]

    barticle [author] Varian , Hal R. H. R. ( 2009 ). Online Ad Auctions . American Economic Review 99 430-34 . 10.1257/aer.99.2.430 barticle

    2009 doi

  34. [42]

    barticle [author] Varian , Hal R. H. R. ( 2016 ). Causal inference in economics and marketing . Proceedings of the National Academy of Sciences 113 7310--7315 . barticle

    2016

  35. [43]

    Koehler , Jim J

    btechreport [author] Vaver , Jon J. Koehler , Jim J. ( 2011 ). Measuring Ad Effectiveness Using Geo Experiments Technical Report , Google Inc. https://ai.google/research/pubs/pub38355 . btechreport

    2011

  36. [44]

    Athey , Susan S

    barticle [author] Wager , Stefan S. Athey , Susan S. ( 2018 ). Estimation and Inference of Heterogeneous Treatment Effects using Random Forests . Journal of the American Statistical Association 113 1228-1242 . 10.1080/01621459.2017.1319839 barticle

    2018

  37. [45]

    ( 2016 )

    bbook [author] Wickham , Hadley H. ( 2016 ). ggplot2: Elegant Graphics for Data Analysis . Springer-Verlag New York . bbook

    2016

  38. [46]

    , Malik , Saarthak S

    binproceedings [author] Ye , Quinn Q. , Malik , Saarthak S. , Chen , Ji J. Zhu , Haijun H. ( 2016 ). The Seasonality Of Paid Search Effectiveness From A Long Running Field Test . In Proceedings of the 2016 ACM Conference on Economics and Computation . EC '16 515--530 . ACM , N...

    2016

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.