pith. sign in

arxiv: 2606.21840 · v1 · pith:2ZVURYA6new · submitted 2026-06-20 · 📊 stat.ME · econ.EM· math.ST· stat.AP· stat.TH

A Test for Treatment Heterogeneity under a Distributional Difference-in-Difference Framework

Satarupa Bhattacharjee , Bing Li , Lingzhou Xue This is my paper

Pith reviewed 2026-06-26 12:11 UTC · model grok-4.3

classification 📊 stat.ME econ.EMmath.STstat.APstat.TH
keywords distributional difference-in-differencesoptimal transportmaximum mean discrepancytreatment heterogeneitynonparametric testcounterfactual distributionasymptotic power analysis
0
0 comments X

The pith

Optimal transport maps control-group drifts to create counterfactuals for testing full distributional treatment effects in DiD designs.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The authors construct a no-treatment counterfactual for the treated group by using optimal transport to estimate the pre-to-post distributional change observed in the control group and applying that change to the treated pre-treatment outcomes. They test whether this counterfactual equals the actual treated post-treatment distribution with a maximum mean discrepancy statistic computed in a reproducing kernel Hilbert space. The resulting omnibus test is designed to detect shifts in location, scale, shape, or tails. Asymptotic theory gives the null limiting distribution as a Gaussian quadratic form and characterizes power under local alternatives. The method is shown to flag distributional effects in the Card-Krueger minimum-wage data that standard mean-based DiD misses.

Core claim

By leveraging optimal transport to estimate the untreated distributional drift from the control group and applying it to the treated group's pre-treatment baseline, the authors construct a counterfactual distribution. They frame the null of no treatment effect as equality between this counterfactual and the observed treated post-treatment distribution, and test it with an MMD statistic in an RKHS. The nonparametric test is sensitive to location, scale, shape, and tail changes; under the null the statistic converges to a Gaussian quadratic form, while under local alternatives a unified power analysis establishes Pitman local power and moderate-deviation consistency. Theory also shows how dete

What carries the argument

The counterfactual distribution obtained by applying the control group's optimal transport map of pre-to-post distributional change to the treated group's pre-treatment distribution.

If this is right

  • The test detects treatment heterogeneity in location, scale, shape, and tail behavior simultaneously.
  • Under the null the statistic converges to a Gaussian quadratic form that supports valid inference.
  • Power is characterized uniformly for local alternatives, including Pitman and moderate-deviation regimes.
  • Detectability depends on the interaction between the transport map and the chosen RKHS geometry.
  • In the Card-Krueger minimum-wage application the test identifies distributional shifts missed by mean-based DiD.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same transport construction could be adapted to staggered adoption or multiple post periods by estimating period-specific drifts.
  • Different kernel choices in the RKHS may change which parts of the distribution are most easily detected, suggesting a practical robustness check.
  • Policy evaluations using this test could reveal whether treatments compress or stretch outcome distributions even when averages are unchanged.

Load-bearing premise

The control group supplies the correct estimate of how the treated group's outcome distribution would have evolved from pre to post period without any treatment.

What would settle it

In repeated simulations where the treated post-treatment distribution is generated exactly by transporting the control drift onto the treated pre-treatment distribution, the test statistic exceeds its critical value at a rate substantially above the nominal level.

Figures

Figures reproduced from arXiv: 2606.21840 by Bing Li, Lingzhou Xue, Satarupa Bhattacharjee.

Figure 1
Figure 1. Figure 1: An illustration of OT maps in the space of measures and corresponding measurable [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Finite-sample calibration across the three simulation scenarios. Columns corre [PITH_FULL_IMAGE:figures/full_fig_p023_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Empirical power as a function of the treatment sample size [PITH_FULL_IMAGE:figures/full_fig_p024_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Card–Krueger minimum-wage study. Left: estimated asymptotic null distribution [PITH_FULL_IMAGE:figures/full_fig_p025_4.png] view at source ↗
read the original abstract

We develop a novel distributional Difference-in-Differences (DiD) framework to capture treatment heterogeneity across outcome distributions. By leveraging optimal transport, we use the control group to estimate the untreated distributional drift from the pre- to post-treatment period and apply it to the treated group's pre-treatment baseline, constructing a counterfactual distribution under the assumption of no treatment effect. We frame the null hypothesis as a distributional equality between the transported counterfactual distribution and the observed treated post-treatment distribution, and test it using a maximum mean discrepancy statistic in a reproducing kernel Hilbert space (RKHS). The resulting nonparametric omnibus test is sensitive to changes in location, scale, shape, and tail behavior. Under the null, we derive the asymptotic Gaussian quadratic-form limit of the test statistic, while under local alternatives, we provide a unified characterization of power that establishes its Pitman local power and moderate-deviation consistency. Our theory reveals how detectability is shaped by the interaction between transport-induced drift and RKHS geometry. Simulations and an application to the Card--Krueger minimum-wage data demonstrate that the proposed method identifies key distributional treatment effects missed by classical mean-based DiD.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

1 major / 0 minor

Summary. The paper develops a distributional difference-in-differences framework that leverages optimal transport to estimate untreated distributional drift from the control group and applies it to the treated pre-treatment distribution to form a counterfactual. The null of no treatment effect is tested via equality between this counterfactual and the observed treated post-treatment distribution using a maximum mean discrepancy (MMD) statistic in an RKHS. The resulting omnibus test is claimed to detect changes in location, scale, shape, and tails. Asymptotic Gaussian quadratic-form limits are derived under the null, with unified power characterizations (Pitman local power and moderate-deviation consistency) under local alternatives; simulations and a Card-Krueger minimum-wage application are provided.

Significance. If the central asymptotic claims hold after accounting for transport-map estimation, the work would supply a nonparametric omnibus test for distributional treatment heterogeneity in DiD designs, extending beyond mean-based methods and offering explicit power results shaped by transport drift and RKHS geometry. The empirical illustration on Card-Krueger data is a concrete strength if the test indeed detects effects missed by classical DiD.

major comments (1)
  1. [Theory section / abstract claim on asymptotic limit] Abstract and theory section: the derivation of the asymptotic Gaussian quadratic-form limit of the MMD statistic under the null must explicitly incorporate the estimation error of the optimal transport map from the control group. Treating the map as known (or showing only o_p(1) convergence without an influence-function expansion) risks additional variance or bias terms that would alter the stated limit and invalidate the subsequent Pitman and moderate-deviation power characterizations.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for the thorough review and the insightful comment on the asymptotic theory. We address the point below.

read point-by-point responses

  1. Referee: Abstract and theory section: the derivation of the asymptotic Gaussian quadratic-form limit of the MMD statistic under the null must explicitly incorporate the estimation error of the optimal transport map from the control group. Treating the map as known (or showing only o_p(1) convergence without an influence-function expansion) risks additional variance or bias terms that would alter the stated limit and invalidate the subsequent Pitman and moderate-deviation power characterizations.

    Authors: We agree that a complete asymptotic analysis requires an explicit accounting of the estimation error in the optimal transport map. The current derivation establishes o_p(1) consistency of the estimated map and derives the limiting distribution of the MMD statistic conditional on the map, but does not yet provide the full influence-function expansion that jointly accounts for map estimation. We will revise the theory section to derive the correct Gaussian quadratic-form limit under the null by incorporating the map's estimation error via its influence function. The Pitman local power and moderate-deviation consistency results will be updated to reflect the joint asymptotics. These changes will be made without altering the paper's main claims or conclusions. revision: yes

Circularity Check

0 steps flagged

No circularity: derivation relies on external OT and RKHS theory

full rationale

The paper constructs a counterfactual via OT map estimated from the control group, frames the null as equality to the observed post-treatment treated distribution, and applies an MMD statistic whose asymptotic quadratic-form limit and local-alternative power are derived from standard RKHS and empirical-process arguments. No step reduces a claimed prediction or limit to a fitted quantity inside the same paper by construction, nor does any load-bearing premise rest on a self-citation chain. The derivation chain is therefore self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 0 invented entities

The method rests on the domain assumption that distributional parallel trends hold so that control-group drift can be transported to the treated group; standard RKHS and optimal-transport existence conditions are also required. No free parameters or invented entities are explicitly named in the abstract.

axioms (2)
  • domain assumption The distributional drift estimated from the control group applies unchanged to the treated group in the absence of treatment (parallel distributional trends).
    Invoked when the control drift is applied to construct the treated counterfactual.
  • standard math The reproducing kernel Hilbert space is rich enough for the MMD to detect the relevant distributional differences.
    Required for the omnibus property of the test statistic.

pith-pipeline@v0.9.1-grok · 5743 in / 1360 out tokens · 24987 ms · 2026-06-26T12:11:23.253047+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

300 extracted references · 6 canonical work pages

  1. [1]

    A two-sample test for multivariate distributions , author=. J. Am. Stat. Assoc. , volume=. 1994 , publisher=

    1994

  2. [2]

    1980 , publisher=

    Approximation Theorems of Mathematical Statistics , author=. 1980 , publisher=

    1980

  3. [3]

    Random Structures & Algorithms , volume=

    A sharp concentration inequality with applications , author=. Random Structures & Algorithms , volume=. 2000 , publisher=

    2000

  4. [4]

    The Annals of Mathematical Statistics , pages=

    Asymptotic minimax character of the sample distribution function and of the classical multinomial estimator , author=. The Annals of Mathematical Statistics , pages=. 1956 , publisher=

    1956

  5. [5]

    Statistical optimal transport posed as learning kernel embedding , author=. Adv. Neural Inf. Process. Syst. , volume=

  6. [6]

    Proceedings of the 26th International Conference on Machine Learning , pages=

    Hilbert space embeddings of conditional distributions with applications to dynamical systems , author=. Proceedings of the 26th International Conference on Machine Learning , pages=

  7. [7]

    2019 , publisher=

    Computational Optimal Transport , author=. 2019 , publisher=

    2019

  8. [8]

    2015 , publisher=

    Causal Inference in Statistics, Social, and Biomedical Sciences , author=. 2015 , publisher=

    2015

  9. [9]

    2016 , publisher=

    Causal Inference in Statistics: A Primer , author=. 2016 , publisher=

    2016

  10. [10]

    , author=

    Estimating causal effects of treatments in randomized and nonrandomized studies. , author=. Journal of Educational Psychology , volume=. 1974 , publisher=

    1974

  11. [11]

    Causal inference using potential outcomes: Design, modeling, decisions , author=. J. Am. Stat. Assoc. , volume=. 2005 , publisher=

    2005

  12. [12]

    Statistica Neerlandica , volume=

    On conditional density estimation , author=. Statistica Neerlandica , volume=. 2003 , publisher=

    2003

  13. [13]

    arXiv preprint arXiv:1903.00954 , year=

    Conditional density estimation with neural networks: Best practices and benchmarks , author=. arXiv preprint arXiv:1903.00954 , year=

    Pith/arXiv arXiv 1903

  14. [14]

    Doklady Akademii Nauk , volume=

    On the transfer of masses (in Russian) , author=. Doklady Akademii Nauk , volume=

  15. [15]

    Foundations and Trends in Machine Learning , volume=

    Kernel mean embedding of distributions: A review and beyond , author=. Foundations and Trends in Machine Learning , volume=. 2017 , publisher=

    2017

  16. [16]

    A kernel two-sample test , author=. J. Mach. Learn. Res. , volume=. 2012 , publisher=

    2012

  17. [17]

    Proceedings of the 21st Conference on Learning Theory , pages=

    Injective Hilbert space embeddings of probability measures , author=. Proceedings of the 21st Conference on Learning Theory , pages=

  18. [18]

    Management Science, in press , year=

    A Graphical Point Process Framework for Understanding Removal Effects in Multi-Touch Attribution , author=. Management Science, in press , year=

  19. [19]

    Air pollution and

    Wu, Xiao and Nethery, Rachel C and Sabath, M Benjamin and Braun, Danielle and Dominici, Francesca , journal=. Air pollution and. 2020 , publisher=

    2020

  20. [20]

    Proceedings of the 29th International Conference on Machine Learning , pages=

    Hypothesis testing using pairwise distances and associated kernels , author=. Proceedings of the 29th International Conference on Machine Learning , pages=

  21. [21]

    New England Journal of Medicine , volume=

    Air pollution and mortality at the intersection of race and social class , author=. New England Journal of Medicine , volume=. 2023 , publisher=

    2023

  22. [22]

    Kernel choice and classifiability for RKHS embeddings of probability distributions , author=. Adv. Neural Inf. Process. Syst. , volume=

  23. [23]

    2010 IEEE International Symposium on Information Theory , pages=

    Non-parametric estimation of integral probability metrics , author=. 2010 IEEE International Symposium on Information Theory , pages=. 2010 , organization=

    2010

  24. [24]

    , author=

    Universality, Characteristic Kernels and RKHS Embedding of Measures. , author=. J. Mach. Learn. Res. , volume=

  25. [25]

    On the influence of the kernel on the consistency of support vector machines , author=. J. Mach. Learn. Res. , volume=

  26. [26]

    2019 , publisher=

    A Graduate Course on Statistical Inference , author=. 2019 , publisher=

    2019

  27. [27]

    2010 , publisher=

    Causal Inference , author=. 2010 , publisher=

    2010

  28. [28]

    arXiv preprint arXiv:2406.19604 , year=

    Geodesic Causal Inference , author=. arXiv preprint arXiv:2406.19604 , year=

    arXiv

  29. [29]

    Biometrics , volume=

    Doubly robust estimation in missing data and causal inference models , author=. Biometrics , volume=. 2005 , publisher=

    2005

  30. [30]

    Causal inference on distribution functions , author=. J. R. Stat. Soc. Ser. B , volume=. 2023 , publisher=

    2023

  31. [31]

    2015 , publisher=

    Theoretical foundations of functional data analysis, with an introduction to linear operators , author=. 2015 , publisher=

    2015

  32. [32]

    Nonlinear global

    Bhattacharjee, Satarupa and Li, Bing and Xue, Lingzhou , journal=. Nonlinear global

  33. [33]

    Electronic Journal of Statistics , volume=

    Concurrent object regression , author=. Electronic Journal of Statistics , volume=. 2022 , publisher=

    2022

  34. [34]

    Journal of Multivariate Analysis , volume=

    Nonlinear sufficient dimension reduction for distribution-on-distribution regression , author=. Journal of Multivariate Analysis , volume=. 2024b , publisher=

  35. [35]

    Dimension Reduction for

    Zhang, Qi and Xue, Lingzhou and Li, Bing , journal=. Dimension Reduction for. 2024 , publisher=

    2024

  36. [36]

    Journal of Nonparametric Statistics, in press , year=

    Sparse kernel sufficient dimension reduction , author=. Journal of Nonparametric Statistics, in press , year=

  37. [37]

    Journal of Business & Economic Statistics, in press , year=

    Model-Based Co-Clustering in Customer Targeting Utilizing Large-Scale Online Product Rating Networks , author=. Journal of Business & Economic Statistics, in press , year=

  38. [38]

    An additive graphical model for discrete data , author=. J. Am. Stat. Assoc. , volume=. 2024 , publisher=

    2024

  39. [39]

    Journal of Computational and Graphical Statistics , volume=

    Envelope Model for Function-on-Function Linear Regression , author=. Journal of Computational and Graphical Statistics , volume=. 2023 , publisher=

    2023

  40. [40]

    frechet: statistical analysis for random objects and

    Chen, Yaqing and Gajardo, Alvaro and Fan, Jianing and Zhong, Qixian and Dubey, Paromita and Han, Kyunghee and Bhattacharjee, Satarupa and M. frechet: statistical analysis for random objects and. R package version 0.2. 0 , year=

  41. [41]

    Single Index

    Bhattacharjee, Satarupa and M. Single Index. Ann. Stat. , volume=. 2023 , publisher=

    2023

  42. [42]

    The Annals of Applied Statistics , volume=

    A latent variable mixture model for composition-on-composition regression with application to chemical recycling , author=. The Annals of Applied Statistics , volume=. 2024 , publisher=

    2024

  43. [43]

    Statistics and Computing , volume =

    Delicado, Pedro and Huerta, Mario and Serna, Susana , title =. Statistics and Computing , volume =. 2017 , publisher =

    2017

  44. [44]

    Bernoulli , volume =

    Le Gouic, Thibaut and Loubes, Jean-Michel , title =. Bernoulli , volume =. 2017 , publisher =

    2017

  45. [45]

    2009 , publisher =

    Dryden, Ian and Mardia, Kanti , title =. 2009 , publisher =

    2009

  46. [46]

    Annals of Statistics , volume =

    Bhattacharya, Rabi and Patrangenaru, Victor , title =. Annals of Statistics , volume =

  47. [47]

    Journal of the Royal Statistical Society: Series C (Applied Statistics) , volume =

    Huckemann, Stephan , title =. Journal of the Royal Statistical Society: Series C (Applied Statistics) , volume =

  48. [48]

    Annales de l'institut Henri Poincaré , volume =

    Fréchet, Maurice , title =. Annales de l'institut Henri Poincaré , volume =

  49. [49]

    Hein, Matthias and Bousquet, Olivier , title =. J. Mach. Learn. Res. , volume =

  50. [50]

    Petersen, Alex and Müller, Hans-Georg , title =. Ann. Stat. , volume =

  51. [51]

    Journal of Causal Inference , volume=

    An optimal transport approach to estimating causal effects via nonlinear difference-in-differences , author=. Journal of Causal Inference , volume=. 2024 , publisher=

    2024

  52. [52]

    2009 , publisher=

    Mostly Harmless Econometrics: An Empiricist's Companion , author=. 2009 , publisher=

    2009

  53. [53]

    Journal of Economic Literature , volume=

    Recent developments in the econometrics of program evaluation , author=. Journal of Economic Literature , volume=

  54. [54]

    The Review of Economics and Statistics , volume=

    Using the longitudinal structure of earnings to estimate the effect of training programs , author=. The Review of Economics and Statistics , volume=

  55. [55]

    Foundations and Trends in Econometrics , volume=

    The estimation of causal effects by difference-in-difference methods , author=. Foundations and Trends in Econometrics , volume=

  56. [56]

    American Economic Journal: Economic Policy , volume=

    No child left behind: Subsidized child care and children's long-run outcomes , author=. American Economic Journal: Economic Policy , volume=

  57. [57]

    Proceedings of the National Academy of Sciences , volume=

    Causal inference in genetic trio studies , author=. Proceedings of the National Academy of Sciences , volume=

  58. [58]

    Nature Communications , volume=

    Causal network models of SARS-CoV-2 expression and aging to identify candidates for drug repurposing , author=. Nature Communications , volume=

  59. [59]

    2008 , publisher=

    Mostly harmless econometrics: An empiricist's companion , author=. 2008 , publisher=

    2008

  60. [60]

    Annual Review of Law and Social Science , volume=

    Credible causal inference for empirical legal studies , author=. Annual Review of Law and Social Science , volume=. 2011 , publisher=

    2011

  61. [61]

    2015 , publisher=

    Causal inference in statistics, social, and biomedical sciences , author=. 2015 , publisher=

    2015

  62. [62]

    American Journal of Public Health , volume=

    Causation and causal inference in epidemiology , author=. American Journal of Public Health , volume=

  63. [63]

    Proceedings of the National Academy of Sciences , volume=

    Methods for causal inference from gene perturbation experiments and validation , author=. Proceedings of the National Academy of Sciences , volume=

  64. [64]

    arXiv preprint arXiv:2011.03127 , year=

    Causal imputation via synthetic interventions , author=. arXiv preprint arXiv:2011.03127 , year=

    arXiv 2011

  65. [65]

    Statistics and causal inference , author=. J. Am. Stat. Assoc. , volume=

  66. [66]

    Journal of Environmental Economics and Management , volume=

    Adaptation to an irrigation water restriction imposed through local governance , author=. Journal of Environmental Economics and Management , volume=

  67. [67]

    Using difference-in-differences to identify causal effects of

    Goodman-Bacon, Andrew and Marcus, Jan , journal=. Using difference-in-differences to identify causal effects of

  68. [68]

    Econometrica , volume=

    Identification and inference in nonlinear difference-in-differences models , author=. Econometrica , volume=

  69. [69]

    2024 , institution=

    Difference-in-differences with a continuous treatment , author=. 2024 , institution=

    2024

  70. [70]

    SIAM Journal on Matrix Analysis and Applications , volume=

    Riemannian geometry of symmetric positive definite matrices via Cholesky decomposition , author=. SIAM Journal on Matrix Analysis and Applications , volume=. 2019 , publisher=

    2019

  71. [71]

    Biology Methods and Protocols , volume=

    Novel metric for hyperbolic phylogenetic tree embeddings , author=. Biology Methods and Protocols , volume=. 2021 , publisher=

    2021

  72. [72]

    arXiv preprint arXiv:2307.05726 , year=

    Geodesic mixed effects models for repeatedly observed/longitudinal random objects , author=. arXiv preprint arXiv:2307.05726 , year=

    arXiv

  73. [73]

    1955 , publisher=

    General Topology , author=. 1955 , publisher=

    1955

  74. [74]

    Advances in Applied Probability , volume=

    Geometry of the space of phylogenetic trees , author=. Advances in Applied Probability , volume=. 2001 , publisher=

    2001

  75. [75]

    2018 , publisher=

    Double/debiased machine learning for treatment and structural parameters , author=. 2018 , publisher=

    2018

  76. [76]

    arXiv preprint arXiv:2004.03036 , year=

    Double debiased machine learning nonparametric inference with continuous treatments , author=. arXiv preprint arXiv:2004.03036 , year=

    arXiv 2004

  77. [77]

    Causal inference in statistics: An overview , author=

  78. [78]

    2016 , publisher=

    Causal inference in statistics: A primer , author=. 2016 , publisher=

    2016

  79. [79]

    Statistics and causal inference , author=. J. Am. Stat. Assoc. , volume=. 1986 , publisher=

    1986

  80. [80]

    Annual review of sociology , volume=

    Causal inference in sociological research , author=. Annual review of sociology , volume=. 2010 , publisher=

    2010

Showing first 80 references.