REVIEW 3 major objections 4 minor 13 references
Using Pre-Trends for Inference in Difference-in-Differences
T0 review · 3 major / 4 minor · reviewed 2026-08-01 · deepseek-v4-flash
Pith's one-line read A difference-in-differences test that uses the historical distribution of pre-treatment differential trends as its reference distribution, without requiring parallel trends, is asymptotically exact.
desk verdict New DID inference method that uses absolute pre-trends as a reference distribution, with a correct proof and an explicit but strong identifying assumption that limits its practical reach. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the untreated differential trend U_t(0), defined as the change in the treated group's untreated outcome minus the contemporaneous change in the control group's untreated outcome. The paper assumes its absolute value |U_t(0)| has the same continuous distribution F at every date (condition 2), and that the empirical distribution of the observed adjusted absolute trends converges uniformly to F (condition 3). The test ranks the post-treatment absolute DID among its pre-treatment counterparts; the probability integral transform converts this rank into an asymptotically uniform variable. A stationary and ergodic joint increment process is offered as a sufficient condition fo
What would settle it
Simulate a DID panel in which the variance of the untreated differential trend increases monotonically over time (e.g., shocks scale with time); compute the pre-treatment empirical distribution of absolute DIDs from the early periods and apply the test to a later post-treatment period with a zero treatment effect. If the rejection frequency exceeds the nominal α substantially, the test's validity depends on the time-invariance of the absolute-trend distribution in a way that a researcher can check empirically.
Extended reading notes
Core claim
The core claim is that the distribution of the absolute pre-treatment difference-in-differences can be used as the reference distribution for the absolute post-treatment difference-in-differences, once the hypothesized treatment effect is subtracted. The test statistic is the empirical CDF of the adjusted absolute DIDs evaluated at the post-treatment value. Under the null, the probability integral transform makes this statistic asymptotically uniform, so rejecting when it exceeds 1−α gives an asymptotically exact test. This identifying restriction is distributional stability of |U_t(0)|, not parallel trends; stable nonzero, time-varying, or sign-changing differential trends are permitted.
Load-bearing premise
The load-bearing premise is condition (2): the distribution of the absolute untreated differential trend is identical at every date, so the historical spread of pre-treatment differential trends is the correct reference for the post-treatment period.
Editorial extensions
If this is right
- Applied DID studies with many pre-periods can conduct inference on the treatment effect without invoking parallel trends, as long as the magnitude of differential trend shocks is stable over time.
- Confidence sets for the treatment effect are obtained by inverting the test, which is simple to compute from the historical distribution of absolute DIDs.
- The method permits a stable nonzero differential trend between groups, a common concern in observational panels.
- Because only absolute differential trends are used, the distribution of the sign of pre-trends is unrestricted; trends may oscillate in sign over time.
- The test is closely tied to a calibration principle already used in the literature, but the DID-specific predictor used here changes the identifying assumption from parallel trends to distributional stability of absolute differential trends.
Reading between the lines
- If the assumption of a stable absolute-trend distribution is violated—for instance, if the variance of differential shocks grows over time—the test's size will not be controlled, and the pre-treatment distribution will be a misleading reference for the post-treatment period.
- The same ranking logic could be extended to settings with multiple treated groups or multiple post-treatment periods, by pooling absolute differential trends across units and dates under an appropriate stability condition.
- The test may have limited power when pre-treatment variability is large relative to plausible effect sizes, since the post-treatment DID must exceed a historical quantile to be flagged.
- A direct, testable extension would be to check the stability of the absolute differential-trend distribution by using earlier sub-samples of the pre-period to predict the distribution of later pre-period differential trends.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a nonparametric, rank-based test for the average treatment effect on the treated in the last period of a two-group difference-in-differences design. The test uses the empirical distribution of absolute pre-treatment differential trends as a reference distribution: for a candidate effect x, the post-treatment differential trend adjusted by x is compared with the T pre-treatment absolute differential trends. The main theoretical result, Proposition 1, shows that under an assumption that the absolute untreated differential trend has a constant continuous distribution over time (condition (2)) and the empirical CDF is uniformly consistent (condition (3)), the test is asymptotically exact. Confidence sets are obtained by inverting the test. The paper also positions the method relative to conformal inference and the parallel-trends literature, arguing that it relaxes parallel trends while imposing a different, explicit stationarity-type restriction.
Significance. If the result is taken at face value, the paper contributes a very simple and transparent inference procedure for DiD settings with long pre-treatment panels, with no parallel-trends requirement. The proof of Proposition 1 is short, clear, and mathematically correct. The paper is also honest about the substantive nature of condition (2). However, the practical value is not demonstrated: there are no simulations or empirical applications, and the load-bearing assumption (2)—time-invariance of the entire distribution of |U_t(0)|—is strong. The novelty relative to existing conformal methods, while real, is incremental: it is a specific predictor for DiD within the Chernozhukov, Wüthrich, and Zhu (2021) framework. The paper is better viewed as a useful theoretical note than as a fully developed inference method.
major comments (3)
- [§3.2, condition (2)] The asymptotic exactness of the test and the validity of the confidence set depend entirely on the assumption that P(|U_t(0)| ≤ u) = F(u) for every t, including the post-treatment period under H0. The paper's advertised advantage over parallel-trends-based methods is that it allows stable violations of parallel trends, but condition (2) is itself a strong restriction: it requires the entire distribution of absolute differential trends, including tails and quantiles, to be time-invariant. The manuscript provides no quantification of how rejection rates or coverage degrade if this distribution changes over time—for example, if the variance of differential trends grows or shrinks. Because this is the central identifying restriction, the paper should include a formal sensitivity analysis (e.g., a bound under a perturbation of F) or a simulation study showing the method's behavior under reali
- [§3.3, Proposition 1] The test uses the empirical CDF \widehat F that includes the post-treatment observation itself, so the p-value is the in-sample rank of the post-period statistic among T observations, not a leave-one-out conformal p-value. While the asymptotic argument is correct, for finite T the actual rejection probability can differ from α by O(1/T) and can be notably higher than α for small T (e.g., with T=50 and α=0.05, the in-sample rank can cause the level to be about 0.06). The paper contains no finite-sample analysis or simulation evidence to show how large T must be before the asymptotic approximation is reliable. This is a practical concern because the method is explicitly motivated by settings with 'many' pre-treatment periods, and the paper should provide guidance on what 'many' means.
- [§3.2, sufficient conditions for (3)] The sufficient condition for condition (3)—strict stationarity and ergodicity of the joint increment process—is strong and often implausible in long panels where group composition, measurement, or the economic environment changes. The paper offers no discussion of how a researcher could assess the plausibility of condition (2) from the pre-treatment data, nor the implications of conducting a pre-test for (2) on the final inference (which would itself be a form of pre-testing with its own distortions, as in Roth 2022). A practical recommendation, even a tentative one, would help close the gap between the theorem and its application.
minor comments (4)
- [§2, notation] The indicator function in the definition of \widehat F(u;x) is written as '1{...}', which is standard but could be typeset as \mathbf{1}{...} for clarity, especially since '1{t=T}' appears later as the same indicator notation.
- [§4.1] The claim that the proposed predictor 'does not imply parallel trends' should be phrased more precisely: it does not require parallel trends, but it does require a different and equally substantive distributional restriction. The current phrasing could be read as suggesting the method is assumption-free, which it is not.
- [§4.2] The discussion of Rambachan and Roth (2023) is brief. Given that the paper's method is a special case of extrapolating pre-trends via distributional stability, a fuller comparison of the restrictions imposed by condition (2) with the restrictions imposed by their sensitivity-analysis approach would be helpful.
- [General] The paper has no conclusion or limitations paragraph. A brief final section summarizing the identifying assumption, the scope of applicability, and open directions (e.g., multiple treated periods, covariates, cluster dependence) would make the note more self-contained.
Circularity Check
No significant circularity: the inference procedure is a self-contained test under explicit assumptions; no fitted parameter is relabeled as a prediction and no load-bearing self-citation appears.
full rationale
The derivation is self-contained. Proposition 1 is a standard probability result under the explicit assumptions (1)-(3): the statistic \hat F(|U_T-x|;x) is built from the data, and the proof uses only uniform consistency of the empirical CDF and the probability integral transform. Condition (2)—that P(|U_t(0)| ≤ u) = F(u) for every t—is an explicit stability assumption on the untreated differential trend, not a fitted parameter and not a consequence of the test. No parameter is estimated before testing; the confidence set is obtained by inverting the asymptotically exact test. Citations to conformal inference are contextual and explicitly acknowledge the relationship; the DID-specific predictor is defined, not borrowed as an unverified black box. The paper itself notes that the distributional stability condition is substantive, and the reviewer's concern about possible failure of (2) is an applicability caveat, not circularity.
Assumptions & free parameters
assumptions (3)
- domain assumption Potential-outcomes model: units have well-defined untreated outcomes Y_{g,t}(0); no interference; treatment is applied only to group 2 at time T; the treatment effect is non-stochastic and additive: U_T = U_T(0) + tau.
- domain assumption Distributional stability: for all t and u>=0, P(|U_t(0)| <= u) = F(u), with F continuous.
- domain assumption Uniform consistency: sup_u |\hat F(u;x) - F(u)| = o_P(1) under H0; satisfied if the joint increment process is strictly stationary and ergodic (or strongly mixing).
Cite this review
Pith. "Pith review of Using Pre-Trends for Inference in Difference-in-Differences." pith.science (2026). https://pith.science/paper/PP4XX3OM
@misc{pith2026260721312,
author = {Pith},
title = {Pith review of: Using Pre-Trends for Inference in Difference-in-Differences},
year = {2026},
howpublished = {\url{https://pith.science/paper/PP4XX3OM}},
note = {Machine review of arXiv:2607.21312}
}
read the original abstract
Difference-in-differences (DID) are sometimes estimated with many pre-treatment periods. In such settings, the observed pre-treatment outcome evolutions provide direct information about the magnitude of shocks that could also occur after treatment. This paper proposes a simple inference procedure that uses those pre-trends as the reference distribution for the post-treatment DID. The procedure is closely related to existing conformal inference procedures, but its DID-specific predictor leads to a distinct identifying restriction. Existing procedures assume parallel trends, while this paper's procedure does not require parallel trends.
Reference graph
Works this paper leans on
-
[1]
Andrews, D. W. K. (2003). End-of-sample instability tests.Econometrica, 71(6), 1661–1694
2003
-
[2]
Bilinski, A., and Hatfield, L. A. (2018). Nothing to see here? Non-inferiority approaches to parallel trends and other model assumptions.arXiv preprint arXiv:1805.03273. 6
arXiv 2018
-
[3]
Chernozhukov, V., W¨ uthrich, K., and Zhu, Y. (2018). Exact and robust conformal inference methods for predictive machine learning with dependent data. InProceedings of the 31st Conference on Learning Theory, 732–749
2018
-
[4]
Chernozhukov, V., W¨ uthrich, K., and Zhu, Y. (2021). An exact and robust conformal inference method for counterfactual and synthetic controls.Journal of the American Statistical Association, 116(536), 1849–1864
2021
-
[5]
difference in differences
Conley, T. G., and Taber, C. R. (2011). Inference with “difference in differences” with a small number of policy changes.Review of Economics and Statistics, 93(1), 113–125
2011
-
[6]
Dette, H., and Schumann, M. (2024). Testing for equivalence of pre-trends in difference-in-differences estimation.Journal of Business & Economic Statistics, forthcoming
2024
-
[7]
Ferman, B., and Pinto, C. (2019). Inference in differences-in-differences with few treated groups and heteroskedasticity.Review of Economics and Statistics, 101(3), 452–467
2019
-
[8]
Lei, L., and Cand` es, E. J. (2021). Conformal inference of counterfactuals and individual treatment effects.Journal of the Royal Statistical Society: Series B, 83(5), 911–938
2021
Show all 13 references
-
[9]
J., and Wasserman, L
Lei, J., G’Sell, M., Rinaldo, A., Tibshirani, R. J., and Wasserman, L. (2018). Distribution-free predictive inference for regression.Journal of the American Statistical Association, 113(523), 1094–1111
2018
-
[10]
Rambachan, A., and Roth, J. (2023). A more credible approach to parallel trends.Review of Economic Studies, 90(5), 2555–2591
2023
-
[11]
Roth, J. (2022). Pretest with caution: Event-study estimates after testing for parallel trends. American Economic Review: Insights, 4(3), 305–322
2022
-
[12]
(2005).Algorithmic Learning in a Random World
Vovk, V., Gammerman, A., and Shafer, G. (2005).Algorithmic Learning in a Random World. Springer
2005
-
[13]
Xu, C., and Xie, Y. (2021). Conformal prediction interval for dynamic time-series. InProceedings of the 38th International Conference on Machine Learning, 11559–11569. 7
2021
Reviewed August 1, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.