pith. sign in

arxiv: 2605.23706 · v1 · pith:WXEIHIE4new · submitted 2026-05-22 · 💰 econ.EM

Algorithm or Creative? A Three-Arm Experimental Design for Decomposing Algorithmic Bias in Platform A/B Tests

Pallavi Pal , Anjana Susarla This is my paper

Pith reviewed 2026-05-25 02:22 UTC · model grok-4.3

classification 💰 econ.EM
keywords algorithmic biasA/B testingonline advertisingmediation analysiscausal inferenceplatform experimentsadvertising platforms
0
0 comments X

The pith

A three-arm design separates an advertisement's creative effect from the platform algorithm's targeting response.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Standard two-arm A/B tests on ad platforms mix the creative's effect with the algorithm's reallocation of impressions based on predicted performance. The paper introduces a three-arm design that feeds treatment metadata to the algorithm in one arm while showing users the control creative, allowing separate identification of the algorithmic indirect effect and the creative direct effect. This avoids bias from conditioning on post-treatment audience without relying on sequential ignorability. Applied to a Meta campaign with a women-targeted text fragment, the algorithm increased female impression share by 2.07 percentage points while the creative decreased it by 0.68 percentage points, accounting for about three-quarters of the total reallocation. Conventional two-arm tests understated the algorithmic channel by a factor of two.

Core claim

The three-arm experimental design adds an arm that exposes the delivery algorithm to the treatment metadata while keeping the user-facing creative the same as in the control arm. This point-identifies the natural indirect effect through the algorithm and the direct effect of the creative without sequential ignorability assumptions. In the Meta campaign, the algorithmic channel raises female impression share by +2.07 percentage points and the creative channel moves it by -0.68 percentage points, with roughly three-quarters of the absolute reallocation attributable to the algorithm. A standard two-arm test understates the algorithmic contribution by a factor of two. The design isolates the平台算法

What carries the argument

The three-arm design, which introduces a metadata-only treatment arm to isolate the algorithm's response while holding creative content fixed.

If this is right

  • The algorithmic channel can be substantially larger than the creative channel in shifting audience composition.
  • Conventional two-arm A/B tests can understate the algorithmic effect by a factor of two.
  • The decomposition holds without sequential ignorability assumptions on mediators.
  • The platform algorithm's targeting can be measured separately from the ad creative's direct impact.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • This approach could apply to other algorithmic platforms to quantify bias in content delivery systems.
  • Campaigns may achieve demographic targeting more through algorithm predictions than through explicit creative design.
  • Extensions could test which specific metadata elements drive the algorithmic reallocation.

Load-bearing premise

The third arm successfully exposes the algorithm to the treatment metadata while holding the user-facing creative identical to control without introducing new selection or measurement artifacts.

What would settle it

If the audience composition in the third arm matches the control arm rather than the treatment arm, this would indicate that the metadata exposure failed to trigger the algorithmic response.

Figures

Figures reproduced from arXiv: 2605.23706 by Anjana Susarla, Pallavi Pal.

Figure 1
Figure 1. Figure 1: The mediator S lies on the causal pathway between treatment D and outcome Y . The total effect of D on Y decomposes into the direct effect D → Y and the indirect effect D → S → Y . Because S is itself a response to treatment, S takes value S(A) under A and value S(B) under B, and the two distributions generally differ. We adopt the potential-outcomes framework of Neyman [1990], Rubin [1974], and write Yi(a… view at source ↗
Figure 2
Figure 2. Figure 2: The three experimental arms as rendered to users. Arms 1 and 2 are pixel-identical to [PITH_FULL_IMAGE:figures/full_fig_p013_2.png] view at source ↗
read the original abstract

Online advertising platforms host hundreds of thousands of A/B tests, but the platform's delivery algorithm routes each creative to the audience it predicts will engage. Every two-arm test therefore conflates the creative's effect with the algorithm's targeting response, and adjusting for the realized audience is biased because audience is a post-treatment mediator. We propose a three-arm design that adds an arm exposing the algorithm to the treatment metadata while holding the user-facing creative identical to control, point-identifying the natural indirect (algorithmic) and direct (creative) effects without sequential ignorability. In a live Meta campaign with a women-targeted text fragment, the algorithmic channel raises female impression share by +2.07 ppt while the creative channel moves it by -0.68 ppt; roughly three-quarters of the absolute reallocation is algorithmic, and a conventional two-arm test understates the algorithmic channel by a factor of two. The design isolates the contribution of platform's algorithm to the outcome which is separable from creative content.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The paper proposes a three-arm experimental design to decompose effects in platform A/B tests into a direct creative effect and an indirect algorithmic effect. The third arm exposes the delivery algorithm to treatment metadata while holding the user-facing creative fixed at the control version, allowing point identification of natural direct and indirect effects without sequential ignorability. In a live Meta campaign featuring a women-targeted text fragment, the algorithmic channel raises female impression share by +2.07 ppt, the creative channel lowers it by -0.68 ppt, and the algorithmic channel accounts for roughly three-quarters of the absolute reallocation; a standard two-arm test is shown to understate the algorithmic component by a factor of two.

Significance. If the identification assumptions hold, the design supplies a practical, parameter-free method for separating algorithmic targeting responses from creative content in the high-volume setting of platform experiments. The empirical decomposition quantifies the relative contribution of each channel and demonstrates that conventional designs can materially misattribute effects, with direct implications for advertisers measuring bias and for platforms designing experiments.

major comments (2)
  1. [§3] §3 (three-arm identification): the point identification of the natural indirect effect requires that the third arm alters only the metadata visible to the algorithm while leaving user-facing creative and all delivery mechanics (auction eligibility, impression logging, user selection) unchanged. The manuscript provides no explicit implementation details on the metadata fields modified, balance checks on observables, or tests confirming that creative content itself is not used by the delivery algorithm, leaving the central identifying assumption unverified.
  2. [§5] Empirical results (abstract and §5): the reported decomposition (algorithmic +2.07 ppt, creative -0.68 ppt, three-quarters algorithmic) and the claim that two-arm tests understate the algorithmic channel by a factor of two are load-bearing on the validity of the third arm; without robustness checks or sensitivity analysis to plausible violations of the metadata-only assumption, the quantitative conclusions cannot be taken as point-identified.
minor comments (2)
  1. [Abstract] The abstract states specific point estimates but omits the total number of impressions or campaigns, which would help assess precision.
  2. [§2] Notation for natural direct and indirect effects should be aligned with the standard mediation literature (e.g., Pearl or Robins) to facilitate comparison.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive comments, which help clarify the requirements for substantiating the identification strategy and empirical claims. We will revise the manuscript to incorporate additional details and analyses as outlined below.

read point-by-point responses

  1. Referee: [§3] §3 (three-arm identification): the point identification of the natural indirect effect requires that the third arm alters only the metadata visible to the algorithm while leaving user-facing creative and all delivery mechanics (auction eligibility, impression logging, user selection) unchanged. The manuscript provides no explicit implementation details on the metadata fields modified, balance checks on observables, or tests confirming that creative content itself is not used by the delivery algorithm, leaving the central identifying assumption unverified.

    Authors: We agree that explicit verification of the identifying assumption is essential. In the revised manuscript we will expand §3 with a dedicated implementation subsection that specifies the exact metadata fields altered (the treatment indicator passed to the delivery algorithm), presents balance tables on pre-experiment user observables across all three arms, and documents the platform's separation of metadata processing from creative-content inspection. These elements were confirmed during experiment setup to ensure delivery mechanics remained unchanged. revision: yes

  2. Referee: [§5] Empirical results (abstract and §5): the reported decomposition (algorithmic +2.07 ppt, creative -0.68 ppt, three-quarters algorithmic) and the claim that two-arm tests understate the algorithmic channel by a factor of two are load-bearing on the validity of the third arm; without robustness checks or sensitivity analysis to plausible violations of the metadata-only assumption, the quantitative conclusions cannot be taken as point-identified.

    Authors: We concur that sensitivity checks would bolster confidence in the quantitative results. The revised §5 will include a new robustness subsection containing bounding exercises and alternative specifications that assess the sensitivity of the +2.07 ppt algorithmic and -0.68 ppt creative estimates to plausible partial violations of the metadata-only assumption. These additions will show that the conclusion that the algorithmic channel accounts for roughly three-quarters of the reallocation remains stable under moderate departures from the assumption, while preserving the point-identification result under the maintained conditions. revision: yes

Circularity Check

0 steps flagged

No significant circularity; identification rests on experimental design rather than fitted quantities or self-referential steps

full rationale

The paper's core contribution is a three-arm experimental design that claims to point-identify natural direct (creative) and indirect (algorithmic) effects without sequential ignorability. This is achieved by adding an arm that exposes the algorithm to treatment metadata while keeping user-facing creative identical to control. No equations, predictions, or decompositions in the abstract or described structure reduce by construction to fitted parameters, self-defined quantities, or prior self-citations. The reported effects (+2.07 ppt algorithmic, -0.68 ppt creative) are presented as outputs of the design applied to a live campaign, not as quantities defined by the design itself. The approach is self-contained against external benchmarks of identification via randomization; the skeptic concern targets assumption validity, not circular reduction. Score 0 is appropriate as no load-bearing step matches the enumerated circularity patterns.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 0 invented entities

The identification relies on the platform algorithm using treatment metadata for delivery decisions and on the third arm successfully isolating that channel; these are domain assumptions rather than fitted parameters or new invented entities.

axioms (2)
  • domain assumption Platform delivery algorithm conditions on treatment metadata when deciding audience exposure.
    Stated in the abstract as the mechanism that creates the conflation problem.
  • domain assumption The third arm can expose the algorithm to treatment metadata while keeping the user-facing creative identical to control.
    Central to the design; if this fails the decomposition is invalid.

pith-pipeline@v0.9.0 · 5707 in / 1402 out tokens · 15765 ms · 2026-05-25T02:22:17.145900+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

76 extracted references · 76 canonical work pages

  1. [1]

    Proceedings of the Seventeenth Conference on Uncertainty in Artificial Intelligence (UAI) , pages =

    Pearl, Judea , title =. Proceedings of the Seventeenth Conference on Uncertainty in Artificial Intelligence (UAI) , pages =. 2001 , publisher =

    work page 2001

  2. [2]

    and Greenland, Sander , title =

    Robins, James M. and Greenland, Sander , title =. Epidemiology , volume =

    work page

  3. [3]

    , title =

    Robins, James M. , title =. Highly Structured Stochastic Systems , editor =

    work page

  4. [4]

    Psychological Methods , volume =

    Imai, Kosuke and Keele, Luke and Tingley, Dustin , title =. Psychological Methods , volume =

    work page

  5. [5]

    Statistical Science , volume =

    Imai, Kosuke and Keele, Luke and Yamamoto, Teppei , title =. Statistical Science , volume =

    work page

  6. [6]

    Journal of the Royal Statistical Society, Series A , volume =

    Imai, Kosuke and Tingley, Dustin and Yamamoto, Teppei , title =. Journal of the Royal Statistical Society, Series A , volume =

    work page

  7. [7]

    , title =

    VanderWeele, Tyler J. , title =

    work page

  8. [8]

    and Kenny, David A

    Baron, Reuben M. and Kenny, David A. , title =. Journal of Personality and Social Psychology , volume =

    work page

  9. [9]

    American Political Science Review , volume =

    Acharya, Avidit and Blackwell, Matthew and Sen, Maya , title =. American Political Science Review , volume =

    work page

  10. [10]

    and Pinto, Rodrigo , title =

    Heckman, James J. and Pinto, Rodrigo , title =. Econometric Reviews , volume =

    work page

  11. [11]

    and Mullainathan, Sendhil , title =

    Ludwig, Jens and Kling, Jeffrey R. and Mullainathan, Sendhil , title =. Journal of Economic Perspectives , volume =

    work page

  12. [12]

    and Ha, Shang E

    Green, Donald P. and Ha, Shang E. and Bullock, John G. , title =. Annals of the American Academy of Political and Social Science , volume =

    work page

  13. [13]

    Statistical Science , volume =

    Neyman, Jerzy , title =. Statistical Science , volume =. 1990 , note =

    work page 1990

  14. [14]

    , title =

    Rubin, Donald B. , title =. Journal of Educational Psychology , volume =

    work page

  15. [15]

    and Green, Donald P

    Gerber, Alan S. and Green, Donald P. , title =

    work page

  16. [16]

    and Imbens, Guido W

    Angrist, Joshua D. and Imbens, Guido W. and Rubin, Donald B. , title =. Journal of the American Statistical Association , volume =

    work page

  17. [17]

    Annals of Applied Statistics , volume =

    Lin, Winston , title =. Annals of Applied Statistics , volume =

    work page

  18. [18]

    and Samii, Cyrus , title =

    Aronow, Peter M. and Samii, Cyrus , title =. Annals of Applied Statistics , volume =

    work page

  19. [19]

    Quarterly Journal of Economics , volume =

    Bertrand, Marianne and Duflo, Esther and Mullainathan, Sendhil , title =. Quarterly Journal of Economics , volume =

    work page

  20. [20]

    Colin and Gelbach, Jonah B

    Cameron, A. Colin and Gelbach, Jonah B. and Miller, Douglas L. , title =. Review of Economics and Statistics , volume =

    work page

  21. [21]

    and Nielsen, Morten

    MacKinnon, James G. and Nielsen, Morten. Cluster-Robust Inference: A Guide to Empirical Practice , journal =

    work page

  22. [22]

    and White, Halbert , title =

    MacKinnon, James G. and White, Halbert , title =. Journal of Econometrics , volume =

    work page

  23. [23]

    , title =

    Liu, Regina Y. , title =. Annals of Statistics , volume =

    work page

  24. [24]

    Annals of Statistics , volume =

    Wu, Chien-Fu Jeff , title =. Annals of Statistics , volume =

    work page

  25. [25]

    Journal of Econometrics , volume =

    Davidson, Russell and Flachaire, Emmanuel , title =. Journal of Econometrics , volume =

    work page

  26. [26]

    and Wolf, Michael , title =

    Romano, Joseph P. and Wolf, Michael , title =. Econometrica , volume =

    work page

  27. [27]

    and Shaikh, Azeem M

    List, John A. and Shaikh, Azeem M. and Xu, Yang , title =. Experimental Economics , volume =

    work page

  28. [28]

    , title =

    Robins, James M. , title =. Mathematical Modelling , volume =

    work page

  29. [29]

    and Hern

    Robins, James M. and Hern. Marginal Structural Models and Causal Inference in Epidemiology , journal =

    work page

  30. [30]

    Causal Inference: What If , publisher =

    Hern. Causal Inference: What If , publisher =

    work page

  31. [31]

    and Zhan, Ruohan and Wager, Stefan and Athey, Susan , title =

    Hadad, Vitor and Hirshberg, David A. and Zhan, Ruohan and Wager, Stefan and Athey, Susan , title =. Proceedings of the National Academy of Sciences , volume =

    work page

  32. [32]

    and Vytlacil, Edward , title =

    Heckman, James J. and Vytlacil, Edward , title =. Econometrica , volume =

    work page

  33. [34]

    Discrimination Through Optimization: How

    Ali, Muhammad and Sapie. Discrimination Through Optimization: How. Proceedings of the ACM on Human-Computer Interaction , volume =

    work page

  34. [35]

    Management Science , volume =

    Lambrecht, Anja and Tucker, Catherine , title =. Management Science , volume =

    work page

  35. [36]

    Management Science , volume =

    Aral, Sinan and Walker, Dylan , title =. Management Science , volume =

    work page

  36. [37]

    and Zettelmeyer, Florian and Bhargava, Neha and Chapsky, Dan , title =

    Gordon, Brett R. and Zettelmeyer, Florian and Bhargava, Neha and Chapsky, Dan , title =. Marketing Science , volume =

    work page

  37. [38]

    , title =

    Tucker, Catherine E. , title =. Journal of Marketing Research , volume =

    work page

  38. [39]

    Matz, S. C. and Kosinski, M. and Nave, G. and Stillwell, D. J. , title =. Proceedings of the National Academy of Sciences , volume =

    work page

  39. [40]

    Potential for Discrimination in Online Targeted Advertising , booktitle =

    Speicher, Till and Ali, Muhammad and Venkatadri, Giridhari and Ribeiro, Filipe Nunes and Arvanitakis, George and Benevenuto, Fabr. Potential for Discrimination in Online Targeted Advertising , booktitle =

    work page

  40. [41]

    Working paper, arXiv:2501.00xxxx , year =

    Imana, Basileal and Korolova, Aleksandra and Heidemann, John , title =. Working paper, arXiv:2501.00xxxx , year =

    work page

  41. [42]

    Explaining causal findings without bias: Detecting and assessing direct effects

    Avidit Acharya, Matthew Blackwell, and Maya Sen. Explaining causal findings without bias: Detecting and assessing direct effects. American Political Science Review, 110 0 (3): 0 512--529, 2016

    work page 2016

  42. [43]

    Discrimination through optimization: How Facebook 's ad delivery can lead to biased outcomes

    Muhammad Ali, Piotr Sapie \.z y \'n ski, Miranda Bogen, Aleksandra Korolova, Alan Mislove, and Aaron Rieke. Discrimination through optimization: How Facebook 's ad delivery can lead to biased outcomes. Proceedings of the ACM on Human-Computer Interaction, 3 0 (CSCW): 0 1--30, 2019

    work page 2019

  43. [44]

    Angrist, Guido W

    Joshua D. Angrist, Guido W. Imbens, and Donald B. Rubin. Identification of causal effects using instrumental variables. Journal of the American Statistical Association, 91 0 (434): 0 444--455, 1996

    work page 1996

  44. [45]

    Creating social contagion through viral product design: A randomized trial of peer influence in networks

    Sinan Aral and Dylan Walker. Creating social contagion through viral product design: A randomized trial of peer influence in networks. Management Science, 57 0 (9): 0 1623--1639, 2011

    work page 2011

  45. [46]

    Aronow and Cyrus Samii

    Peter M. Aronow and Cyrus Samii. Estimating average causal effects under general interference, with application to a social network experiment. Annals of Applied Statistics, 11 0 (4): 0 1912--1947, 2017

    work page 1912

  46. [47]

    Baron and David A

    Reuben M. Baron and David A. Kenny. The moderator-mediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations. Journal of Personality and Social Psychology, 51 0 (6): 0 1173--1182, 1986

    work page 1986

  47. [48]

    How much should we trust differences-in-differences estimates? Quarterly Journal of Economics, 119 0 (1): 0 249--275, 2004

    Marianne Bertrand, Esther Duflo, and Sendhil Mullainathan. How much should we trust differences-in-differences estimates? Quarterly Journal of Economics, 119 0 (1): 0 249--275, 2004

    work page 2004

  48. [49]

    Gordon, Poppy Zhang, and Shawndra Hill

    Gordon Burtch, Robert Moakler, Brett R. Gordon, Poppy Zhang, and Shawndra Hill. Characterizing and minimizing divergent delivery in Meta advertising experiments. Working paper, arXiv:2508.21251, 2025

    work page arXiv 2025

  49. [50]

    Colin Cameron, Jonah B

    A. Colin Cameron, Jonah B. Gelbach, and Douglas L. Miller. Bootstrap-based improvements for inference with clustered errors. Review of Economics and Statistics, 90 0 (3): 0 414--427, 2008

    work page 2008

  50. [51]

    The wild bootstrap, tamed at last

    Russell Davidson and Emmanuel Flachaire. The wild bootstrap, tamed at last. Journal of Econometrics, 146 0 (1): 0 162--169, 2008

    work page 2008

  51. [52]

    Gerber and Donald P

    Alan S. Gerber and Donald P. Green. Field Experiments: Design, Analysis, and Interpretation. W.\,W.\ Norton, 2012

    work page 2012

  52. [53]

    Gordon, Florian Zettelmeyer, Neha Bhargava, and Dan Chapsky

    Brett R. Gordon, Florian Zettelmeyer, Neha Bhargava, and Dan Chapsky. A comparison of approaches to advertising measurement: Evidence from big field experiments at Facebook . Marketing Science, 38 0 (2): 0 193--225, 2019

    work page 2019

  53. [54]

    Green, Shang E

    Donald P. Green, Shang E. Ha, and John G. Bullock. Enough already about ``black box'' experiments: Studying mediation is more difficult than most scholars suppose. Annals of the American Academy of Political and Social Science, 628 0 (1): 0 200--208, 2010

    work page 2010

  54. [55]

    Heckman and Rodrigo Pinto

    James J. Heckman and Rodrigo Pinto. Econometric mediation analyses: Identifying the sources of treatment effects from experimentally estimated production technologies with unmeasured and mismeasured inputs. Econometric Reviews, 34 0 (1--2): 0 6--31, 2015

    work page 2015

  55. [56]

    A general approach to causal mediation analysis

    Kosuke Imai, Luke Keele, and Dustin Tingley. A general approach to causal mediation analysis. Psychological Methods, 15 0 (4): 0 309--334, 2010 a

    work page 2010

  56. [57]

    Identification, inference and sensitivity analysis for causal mediation effects

    Kosuke Imai, Luke Keele, and Teppei Yamamoto. Identification, inference and sensitivity analysis for causal mediation effects. Statistical Science, 25 0 (1): 0 51--71, 2010 b

    work page 2010

  57. [58]

    Experimental designs for identifying causal mechanisms

    Kosuke Imai, Dustin Tingley, and Teppei Yamamoto. Experimental designs for identifying causal mechanisms. Journal of the Royal Statistical Society, Series A, 176 0 (1): 0 5--51, 2013

    work page 2013

  58. [59]

    Apparent algorithmic discrimination and real-time algorithmic learning in digital search advertising

    Basileal Imana, Aleksandra Korolova, and John Heidemann. Apparent algorithmic discrimination and real-time algorithmic learning in digital search advertising. Working paper, arXiv:2501.00xxxx, 2025

    work page 2025

  59. [60]

    Algorithmic bias? An empirical study of apparent gender-based discrimination in the display of STEM career ads

    Anja Lambrecht and Catherine Tucker. Algorithmic bias? An empirical study of apparent gender-based discrimination in the display of STEM career ads. Management Science, 65 0 (7): 0 2966--2981, 2019

    work page 2019

  60. [61]

    Agnostic notes on regression adjustments to experimental data: Reexamining Freedman 's critique

    Winston Lin. Agnostic notes on regression adjustments to experimental data: Reexamining Freedman 's critique. Annals of Applied Statistics, 7 0 (1): 0 295--318, 2013

    work page 2013

  61. [62]

    List, Azeem M

    John A. List, Azeem M. Shaikh, and Yang Xu. Multiple hypothesis testing in experimental economics. Experimental Economics, 22 0 (4): 0 773--793, 2019

    work page 2019

  62. [63]

    Regina Y. Liu. Bootstrap procedures under some non-i.i.d.\ models. Annals of Statistics, 16 0 (4): 0 1696--1708, 1988

    work page 1988

  63. [64]

    Kling, and Sendhil Mullainathan

    Jens Ludwig, Jeffrey R. Kling, and Sendhil Mullainathan. Mechanism experiments and policy evaluations. Journal of Economic Perspectives, 25 0 (3): 0 17--38, 2011

    work page 2011

  64. [65]

    MacKinnon and Halbert White

    James G. MacKinnon and Halbert White. Some heteroskedasticity-consistent covariance matrix estimators with improved finite sample properties. Journal of Econometrics, 29 0 (3): 0 305--325, 1985

    work page 1985

  65. [66]

    MacKinnon, Morten rregaard Nielsen, and Matthew D

    James G. MacKinnon, Morten rregaard Nielsen, and Matthew D. Webb. Cluster-robust inference: A guide to empirical practice. Journal of Econometrics, 232 0 (2): 0 272--299, 2023

    work page 2023

  66. [67]

    S. C. Matz, M. Kosinski, G. Nave, and D. J. Stillwell. Psychological targeting as an effective approach to digital mass persuasion. Proceedings of the National Academy of Sciences, 114 0 (48): 0 12714--12719, 2017

    work page 2017

  67. [68]

    On the application of probability theory to agricultural experiments

    Jerzy Neyman. On the application of probability theory to agricultural experiments. Essay on principles. Section 9. Statistical Science, 5 0 (4): 0 465--472, 1990. Translation of Neyman (1923)

    work page 1990

  68. [69]

    Direct and indirect effects

    Judea Pearl. Direct and indirect effects. In Proceedings of the Seventeenth Conference on Uncertainty in Artificial Intelligence (UAI), pages 411--420. Morgan Kaufmann, 2001

    work page 2001

  69. [70]

    James M. Robins. Semantics of causal DAG models and the identification of direct and indirect effects. In Peter J. Green, Nils Lid Hjort, and Sylvia Richardson, editors, Highly Structured Stochastic Systems, pages 70--81. Oxford University Press, 2003

    work page 2003

  70. [71]

    Robins and Sander Greenland

    James M. Robins and Sander Greenland. Identifiability and exchangeability for direct and indirect effects. Epidemiology, 3 0 (2): 0 143--155, 1992

    work page 1992

  71. [72]

    Romano and Michael Wolf

    Joseph P. Romano and Michael Wolf. Stepwise multiple testing as formalized data snooping. Econometrica, 73 0 (4): 0 1237--1282, 2005

    work page 2005

  72. [73]

    Donald B. Rubin. Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of Educational Psychology, 66 0 (5): 0 688--701, 1974

    work page 1974

  73. [74]

    Gummadi, Patrick Loiseau, and Alan Mislove

    Till Speicher, Muhammad Ali, Giridhari Venkatadri, Filipe Nunes Ribeiro, George Arvanitakis, Fabr \'i cio Benevenuto, Krishna P. Gummadi, Patrick Loiseau, and Alan Mislove. Potential for discrimination in online targeted advertising. In Conference on Fairness, Accountability and Transparency (FAT*), pages 5--19, 2018

    work page 2018

  74. [75]

    Catherine E. Tucker. Social networks, personalized advertising, and privacy controls. Journal of Marketing Research, 51 0 (5): 0 546--562, 2014

    work page 2014

  75. [76]

    VanderWeele

    Tyler J. VanderWeele. Explanation in Causal Inference: Methods for Mediation and Interaction. Oxford University Press, 2015

    work page 2015

  76. [77]

    Jackknife, bootstrap and other resampling methods in regression analysis

    Chien-Fu Jeff Wu. Jackknife, bootstrap and other resampling methods in regression analysis. Annals of Statistics, 14 0 (4): 0 1261--1295, 1986

    work page 1986