pith. sign in

arxiv: 2511.03236 · v2 · submitted 2025-11-05 · 💰 econ.EM · stat.ME

Unbiased Regression-Adjusted Estimation of Average Treatment Effects in Randomized Controlled Trials

Alberto Abadie , Mehrdad Ghadiri , Ali Jadbabaie , Mahyar JafariNodeh This is my paper

Pith reviewed 2026-05-18 01:57 UTC · model grok-4.3

classification 💰 econ.EM stat.ME
keywords average treatment effectsregression adjustmentrandomized controlled trialsfinite-sample biasunbiased estimationexact varianceleave-one-out methodscausal inference
0
0 comments X

The pith

Leave-one-out regression adjustment removes finite-sample bias from average treatment effect estimates in randomized trials.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper develops leave-one-out regression adjustment, called LOORA, to estimate average treatment effects when covariates are available in randomized controlled trials. Standard regression adjustment can produce bias in small samples even when treatment is randomized. By fitting the regression while leaving out the observation being adjusted, LOORA restores exact unbiasedness under the randomization distribution alone. The construction also delivers closed-form exact variance formulas for the adjusted Horvitz-Thompson and difference-in-means estimators. Ridge regularization is added to limit the effect of high-leverage units, and two within-subject experiments illustrate that the method cuts bias and brings confidence-interval coverage close to nominal levels.

Core claim

The leave-one-out regression adjustment (LOORA) estimator is formed by predicting the untreated outcome for each treated unit (and vice versa) using a regression fitted to all other units, which removes the finite-sample bias that arises when the same units are used both to fit the regression and to compute the adjustment. This yields an estimator that remains exactly unbiased for the average treatment effect under randomization alone. The same construction supplies exact finite-sample variance formulas for both the regression-adjusted Horvitz-Thompson estimator and the regression-adjusted difference-in-means estimator. Ridge regularization is introduced to stabilize the adjustment when high

What carries the argument

Leave-one-out regression adjustment (LOORA), the device of fitting the covariate-outcome regression on every observation except the one whose potential outcome is being predicted, which carries the unbiasedness and exact-variance claims.

If this is right

  • The estimator is exactly unbiased for the average treatment effect under the randomization distribution alone, without requiring correct specification of the outcome model.
  • Exact closed-form variance formulas are available for the regression-adjusted Horvitz-Thompson and difference-in-means versions, permitting finite-sample inference.
  • Ridge regularization bounds the influence of high-leverage observations and improves stability when sample size is small.
  • In large samples the estimator matches the asymptotic variance of the efficient regression-adjusted estimator of Lin (2013) while retaining exact unbiasedness.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Similar leave-one-out adjustments could be explored for other covariate-adjustment techniques in randomized or quasi-experimental settings to restore finite-sample unbiasedness.
  • In fields that routinely run small randomized trials, routine use of LOORA might improve the reliability of reported treatment-effect estimates without requiring larger samples.
  • The exact variance formulas open the door to studying higher-order properties or Edgeworth corrections that remain intractable for biased estimators.

Load-bearing premise

The leave-one-out construction preserves randomization-based unbiasedness and does not create new dependence that would invalidate the exact variance derivations.

What would settle it

Simulate many randomizations of treatment assignment on the same fixed potential outcomes and check whether the Monte Carlo average of the LOORA estimator equals the true average treatment effect while the conventional regression-adjusted estimator does not.

read the original abstract

This article introduces a leave-one-out regression adjustment (LOORA) for estimating average treatment effects in randomized controlled trials. In finite samples, LOORA removes the bias of conventional regression adjustment and yields exact variance formulas for regression-adjusted Horvitz-Thompson and difference-in-means estimators. Ridge regularization curbs the influence of high-leverage observations, improving stability and precision in small samples. In large samples, LOORA matches the variance of the regression-adjusted estimator in Lin (2013) while remaining exactly unbiased. Two within-subject experimental applications, each providing a realistic joint distribution of potential outcomes as ground truth, show that LOORA removes substantial bias and achieves confidence interval coverage close to the nominal level.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The paper introduces leave-one-out regression adjustment (LOORA) for estimating average treatment effects in RCTs. It claims that LOORA eliminates the finite-sample bias of conventional regression adjustment and delivers exact variance formulas for the regression-adjusted Horvitz-Thompson and difference-in-means estimators. Ridge regularization is added for stability in small samples. Asymptotically, LOORA matches the variance of Lin (2013) while remaining exactly unbiased under randomization. Two within-subject experiments with realistic potential-outcome distributions are used to illustrate bias reduction and near-nominal coverage.

Significance. If the unbiasedness and exact-variance claims hold, the contribution would be substantial for design-based inference in RCTs, offering a practical route to unbiased regression adjustment with closed-form inference. The ridge regularization and the use of within-subject experiments that supply ground-truth joint distributions of potential outcomes are clear strengths. The work directly targets a known finite-sample limitation of regression adjustment without sacrificing asymptotic efficiency.

major comments (2)
  1. [§3] §3 (LOORA definition): The leave-one-out construction defines the regression coefficient vector for unit i on the sample excluding i. This couples every adjustment term to the outcomes and treatments of all other units, creating dependence across the per-unit contributions that is absent in full-sample regression adjustment. The subsequent variance derivations must therefore incorporate the resulting covariance structure under the finite-population randomization distribution.
  2. [§4] §4 (variance formulas for regression-adjusted HT and DiM): The paper asserts exact closed-form variance expressions. Because the leave-one-out coefficients are random and shared across units, the variance of the sum includes non-zero cross terms. If the derivation conditions on the leave-one-out coefficients or treats them as independent, the formulas are not exact under randomization alone; an explicit accounting of these covariances is required for the central claim to hold.
minor comments (2)
  1. [§2] The ridge regularization parameter is introduced without a clear default choice or sensitivity analysis in the main text; a brief discussion of its practical selection would improve reproducibility.
  2. [§3] Notation for the leave-one-out estimator (e.g., distinguishing the full-sample versus leave-one-out coefficient vectors) could be made more explicit to avoid confusion with conventional regression adjustment.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the detailed and insightful comments on our paper. We have carefully considered the points raised regarding the dependence structure in the leave-one-out regression adjustment and the exact variance derivations. Our responses are provided below, and we have made revisions to enhance the clarity of the variance calculations in the manuscript.

read point-by-point responses

  1. Referee: [§3] §3 (LOORA definition): The leave-one-out construction defines the regression coefficient vector for unit i on the sample excluding i. This couples every adjustment term to the outcomes and treatments of all other units, creating dependence across the per-unit contributions that is absent in full-sample regression adjustment. The subsequent variance derivations must therefore incorporate the resulting covariance structure under the finite-population randomization distribution.

    Authors: We agree with the referee that the leave-one-out approach introduces dependence between the adjustment terms for different units. Our variance formulas are derived under the finite-population randomization distribution and explicitly include all covariance terms arising from this dependence. Specifically, the variance of the LOORA estimator is computed by taking the expectation of the squared deviation over all possible treatment assignments, which accounts for the joint distribution of the leave-one-out coefficients and the outcomes. This ensures the expressions are exact without conditioning on the coefficients. revision: partial

  2. Referee: [§4] §4 (variance formulas for regression-adjusted HT and DiM): The paper asserts exact closed-form variance expressions. Because the leave-one-out coefficients are random and shared across units, the variance of the sum includes non-zero cross terms. If the derivation conditions on the leave-one-out coefficients or treats them as independent, the formulas are not exact under randomization alone; an explicit accounting of these covariances is required for the central claim to hold.

    Authors: The derivations in the paper do not condition on the leave-one-out coefficients or assume independence. Instead, we derive the closed-form variances by direct calculation under the randomization distribution, incorporating the cross terms through combinatorial enumeration of treatment vectors. To address this concern, we have expanded the appendix with a more detailed derivation that highlights how the covariances are accounted for in the final expressions. We believe this clarifies that the formulas remain exact. revision: yes

Circularity Check

0 steps flagged

No significant circularity: LOORA is a procedural definition with claimed exact properties under randomization

full rationale

The paper defines LOORA as a new leave-one-out regression adjustment procedure that removes bias from conventional regression adjustment while providing exact variance formulas under the finite-population randomization distribution. The abstract presents this as a direct construction that matches the variance of the Lin (2013) estimator in large samples and remains exactly unbiased. No equations or steps are shown that reduce the unbiasedness claim or variance derivations to a tautology, self-fit, or self-citation chain; the leave-one-out step is introduced as an independent procedural choice rather than a quantity defined in terms of the target estimand. The derivation chain is therefore self-contained against external randomization-based benchmarks.

Axiom & Free-Parameter Ledger

1 free parameters · 1 axioms · 0 invented entities

The method rests on standard randomization-based inference plus the new leave-one-out construction; no new particles or forces are postulated.

free parameters (1)
  • ridge regularization parameter
    Introduced to limit influence of high-leverage observations; its selection rule is not specified in the abstract.
axioms (1)
  • domain assumption Randomization alone is sufficient to establish unbiasedness once the leave-one-out adjustment is applied.
    Invoked when the authors claim exact unbiasedness for the LOORA estimator.

pith-pipeline@v0.9.0 · 5654 in / 1359 out tokens · 46135 ms · 2026-05-18T01:57:05.914014+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

2 extracted references · 2 canonical work pages

  1. [1]

    Evaluating behaviorally motivated policy: Experimental evidence from the lightbulb market,

    [31] ALLCOTT, HUNT ANDDMITRYTAUBINSKY(2015): “Evaluating behaviorally motivated policy: Experimental evidence from the lightbulb market,”American Economic Review, 105 (8), 2501–2538. [27] ARMSTRONG, TIMOTHYBANDMICHALKOLESÁR(2021): “Finite-Sample Optimal Estima- tion and Inference on Average Treatment Effects Under Unconfoundedness,”Economet- rica, 89 (3),...

    work page arXiv 2015

  2. [2]

    On the application of probability theory to agricultural experiments: essay on principles, Section 9,

    [32, 33] NEYMAN, J (1923): “On the application of probability theory to agricultural experiments: essay on principles, Section 9,”Statistical Science, 5, 465–480. [5] RAO, C. RADHAKRISHNA(1973):Linear Statistical Inference and its Applications, Wiley. [39] RUBIN, DONALDB (1974): “Estimating causal effects of treatments in randomized and nonrandomized stud...

    work page 1923