Optimal Experimental Design and Estimation when Potential Outcomes are Bounded

Peter Hull

Optimal Experimental Design and Estimation when Potential Outcomes are Bounded

T0 review · 0 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that when potential outcomes are known to lie in $[L,U]$, the minimax way to estimate an average treatment effect randomizes each unit independently and uses a midpoint-centered recentered regression; balanced designs…

desk verdict A clean finite-sample minimax result showing independent randomization beats complete randomization for bounded outcomes; the proofs check out, and the main caveat is the author's own worst-case framing. read the letter →

arxiv 2608.09812 v1 pith:JE7XY2XQ submitted 2026-08-10 econ.EM

Peter Hull This is my paper · ORCID

classification econ.EM MSC 62K0562C2062D05

keywords boundedpotentialoutcomesaveragetreatmenteffectminimaxexperimentaldesignindependentrandomassignmentcompleterandomizationpairedfinitepopulationrecenteredregression

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper studies how to run and analyze a randomized experiment when each unit's two potential outcomes are known to lie in a fixed interval $[L,U]$, as with binary outcomes. It claims that under a worst-case mean-squared-error criterion, the optimal design is independent random assignment with a random treatment share, not balanced complete randomization, because randomness in the treated share becomes informative once the outcome level is anchored. The optimal estimator in a broad affine class is an unconventional regression of the support-midpoint-centered outcome on the recentered treatment, with no intercept, attaining worst-case MSE $(U-L)^2/N$. Complete randomization costs at least $(U-L)^2/(N-1)$, and paired randomization with pair-fixed effects has twice the optimum. If these results are right, the usual advice to balance treatment shares needs a bounded-outcome caveat: balance is minimax only when outcome levels are unrestricted.

What carries the argument

The engine is the no-effect endpoint adversary: for each unit, the worst case can be taken to have both potential outcomes equal to either $L$ or $U$, so the outcome vector is $M + r s$ with $s\in\{-1,1\}^N$ and $r=(U-L)/2$. Under such a schedule, the risk of an equivariant affine estimator becomes $r^2\|b(D)\|^2$, and the equivariance condition plus Cauchy-Schwarz forces $\|b(D)\|^2\ge 4/N$. Independent randomization makes the assignment pairs uncorrelated, so no pattern of endpoint signs can align with assignment dependence to inflate risk; any negative pairwise dependence (as in complete or paired randomization) gives the adversary extra leverage. For the unrestricted measurable-estimator problem, the same endpoint reduction is carried by a symmetrization argument: averaging any design over coordinate swaps and permutations yields independent randomization and a symmetric estimator, and the minimax value $\kappa_N$ is the optimum of the convex program (4) over the endpoint states $(k,m)$.

What would settle it

For a small fixed $N$ (say $N=4$), enumerate all assignment distributions on $\{0,1\}^N$ and all affine estimators satisfying equivariance condition (2), and evaluate each at all $2^N$ no-effect endpoint configurations; if any pair's worst-case squared error is strictly below $(U-L)^2/N$, Proposition 1 is false. For the unrestricted claim, compute $\kappa_N$ from (4) and search over measurable estimators via the least-favorable-prior representation; if any design-estimator pair beats $(U-L)^2\kappa_N$, Theorem 1 is false. A separate unresolved check is the paper's conjecture that complete randomization is strictly suboptimal for general estimators: finding an even $N$ where complete randomization attains the independent-randomization value would weaken that subclaim but not the main theorem.

Watch

Extended reading notes

Core claim

Formally, fix $N$ units with $Y_i(0),Y_i(1)\in[L,U]$ and target the finite-population average treatment effect $\beta = N^{-1}\sum_i(Y_i(1)-Y_i(0))$. Over all assignment mechanisms and affine estimators satisfying the equivariance condition $b(D)'(D-\tfrac12\mathbf{1})=1$ (which holds for difference-in-means, OLS with controls or fixed effects, and matching estimators), the minimax worst-case MSE is $(U-L)^2/N$. It is attained by independent Bernoulli$(1/2)$ assignment and $\hat\beta^* = \frac{2}{N}\sum_i(2D_i-1)(Y_i-M)$, where $M=(U+L)/2$: the no-intercept regression of $Y_i-M$ on $D_i-1/2$. Complete randomization with any equivariant affine estimator has worst-case MSE at least $(U-L)^2/(N-1)$, and difference-in-means attains that bound; paired randomization with pair-fixed effects has worst-case MSE $(U-L)^2/J$, exactly twice the optimum. The paper then removes the affine restriction: over all measurable estimators the minimax value is $(U-L)^2\kappa_N$, where $\kappa_N$ is the value of a finite convex program with quadratic constraints, and independent randomization remains optimal, with a generally nonlinear estimator that has a least-favorable-prior representation. Restricting to affine estimators without the equivariance condition, the best procedure shrinks $\hat\beta^*$ by $\sqrt{N}/(\sqrt{N}+1)$, giving minimax MSE $(U-L)^2/(\sqrt{N}+1)^2$.

Load-bearing premise

The result depends on evaluating designs against the worst possible configuration of bounded potential outcomes, with no restriction that units in the same pair or stratum have similar outcomes; the paper's own conclusion (Section 4) notes that this is 'likely overly pessimistic' when stratification uses informative covariates.

Editorial extensions

If this is right

Within the equivariant affine class, any completely randomized experiment has worst-case MSE at least $N/(N-1)$ times the minimax value, and paired randomization with pair-fixed effects has exactly twice the minimax value.
The minimax affine estimator is the no-intercept regression of the midpoint-centered outcome on the recentered treatment; difference-in-means is optimal only under complete randomization, where it attains the larger $(U-L)^2/(N-1)$ bound.
When the estimator is unrestricted, the minimax value is $(U-L)^2\kappa_N$ with independent assignment, and the optimal estimator is nonlinear; numerically it reduces worst-case MSE by 15-30% for moderate $N$ relative to the best affine estimator.
Without the equivariance restriction, the best affine estimator shrinks $\hat\beta^*$ by $\sqrt{N}/(\sqrt{N}+1)$, with minimax MSE $(U-L)^2/(\sqrt{N}+1)^2$, and a further nonlinear correction helps for every $N>2$.
The paper does not claim balance is always bad: when strata are chosen using informative covariates, it concludes that stratified or paired designs 'are likely to dominate' independent randomization, so the reversal applies to worst-case comparisons with uninformative strata.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If bounds were unit-specific, the same midpoint-centering derivation would replace the common midpoint $M$ by each unit's own $(L_i+U_i)/2$ and likely preserve the optimality of independent assignment; the paper does not develop this case.
Adding a mild within-stratum homogeneity or Lipschitz bound to the minimax problem would create a threshold where paired randomization overtakes independent randomization; locating that threshold would quantify the cost of balance that the paper leaves open.
The numerical decline of the nonlinear estimator's gain with $N$ suggests the correction matters most for experiments with a few hundred units; for large samples the simple no-intercept recentered regression is nearly minimax. This extrapolation follows from the paper's Figure 1, not from a stated theorem.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The headline result is real. Hull proves that for bounded potential outcomes, independent randomization and a recentered, intercept-free regression estimator achieve minimax MSE among all assignment mechanisms and a broad class of affine estimators. Complete randomization is strictly worse by an N/(N-1) factor, and paired randomization is twice as bad. Extending to arbitrary estimators, independent randomization remains optimal and the optimal estimator is characterized by a convex program. This is genuinely new relative to Aronow and Lopatto (2026), who fix marginal inclusion probabilities and restrict to design-unbiased estimators, and to Harshaw et al. (2024), who fix the Horvitz-Thompson estimator. The paper also connects cleanly to the Hodges and Lehmann (1982) sampling result. The proof structure is solid: adversarial averaging gives the lower bounds, symmetrization reduces to independent assignment, and the separate-convexity argument pins down the unrestricted minimax value. I checked the main steps and they balance; the appendix is self-contained and does not lean on the companion papers. The main soft spot is not technical but interpretive. The minimax criterion lets the adversary choose any configuration in [L,U]^2N, with no structure linking Y(0) and Y(1). That is the natural worst case, but the author himself notes in Section 4 that stratification based on informative observables will likely dominate in practice, and the gain over complete randomization is asymptotically negligible. So the practical prescription is less sharp than the abstract suggests. The numerical figure has no code, but the program is precisely specified and easily reproducible, so this is minor. Self-citations are present but not circular; they provide context, not premises. The paper deserves a serious referee. It is a well-argued, technically clean contribution to the minimax design literature. I would send it out and expect acceptance after minor revision, with the main discussion focused on how much weight to put on the pure worst-case criterion.

Referee Report

0 major / 4 minor

Summary. The paper studies minimax estimation of the finite-population average treatment effect when all potential outcomes are known to lie in a bounded interval [L,U]. The main results are: (i) Proposition 1 shows that, among all assignment mechanisms and affine estimators satisfying the equivariance condition (2), independent Bernoulli(1/2) randomization paired with the no-intercept regression of the support-midpoint-centered outcome on the recentered treatment attains the minimax worst-case MSE (U-L)^2/N; (ii) Proposition 2 shows balanced complete randomization has worst-case MSE at least (U-L)^2/(N-1), attained by the difference-in-means estimator; (iii) Proposition 3 shows paired randomization has worst-case MSE at least (U-L)^2/J = 2(U-L)^2/N; (iv) Theorem 1 extends the design optimality to all measurable estimators, giving the minimax value (U-L)^2 κ_N from the convex program (4) and characterizing a generally nonlinear optimal estimator; (v) Corollary 1 solves the unrestricted affine problem, yielding value (U-L)^2/(√N+1)^2 via a shrunk version of the Proposition 1 estimator. The paper explicitly notes in Section 4 that the pure worst-case criterion may be pessimistic for stratified designs when strata are prognostically informative, and that such designs may dominate in practice.

Significance. If correct, the paper provides a sharp and surprising decision-theoretic reversal: with bounded potential outcomes, the usual prescription of balanced complete randomization and difference-in-means is not minimax, and independent randomization with a random treated share is optimal. The proof strategy is elegant, connecting the experiment-design problem to Hodges-Lehmann-style least-favorable-prior arguments, and the convex program (4) makes the optimal nonlinear estimator computable. The appendix proofs are self-contained and the main derivations balance; the reader's stress-test concern about the worst-case modeling choice is explicitly acknowledged in the paper itself and is a modeling limitation rather than a technical flaw. The paper also credits and builds on the recent recentering literature, and the claimed novelty relative to Aronow and Lopatto (2026) and Harshaw et al. (2024) is clearly delineated.

minor comments (4)

[Abstract and Section 2] The abstract's phrase "Among all assignment mechanisms and a broad class of affine estimators" could be misread as covering all affine estimators; Proposition 1 applies to affine estimators satisfying the equivariance condition (2), while Corollary 1 covers all affine estimators with a different minimax value and a shrunk estimator. Please clarify this distinction in the abstract or at the end of the introduction.
[Corollary 1] The symbol \hat\beta^*_{NL} is used for the shrunk affine estimator in Corollary 1 as well as for the nonlinear estimator in Theorem 1. Since the two estimators are different objects, a distinct notation for the affine shrunk estimator would avoid confusion.
[Proposition 2] Proposition 2 assumes N is even, which is natural for balanced complete randomization, but the paper does not state what happens for odd N; a brief remark on the odd-N case would make the scope of the strict suboptimality claim precise.
[Theorem 1] The notation p_{k,m}(x) in (4) and p_x(z) immediately after Theorem 1 are visually close; renaming one of them, for example q_{k,m}(x) or r_x(z), would improve readability.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the minimax theorems are proved from the stated worst-case assumptions and do not depend on the self-cited companion papers as premises.

full rationale

I walked the paper's derivation chain. The central claims are Propositions 1-3 and Theorem 1/Corollary 1, all of which are mathematical results proved in the appendix from the model defined in Section 2. Proposition 1's lower bound uses averaging over adversarial endpoint schedules and Cauchy-Schwarz applied to the equivariance restriction, and its upper bound computes the exact MSE of the proposed estimator under independent Bernoulli assignment; no fitted data or cited result is used. Propositions 2 and 3 similarly derive adversarial lower bounds from the exact assignment-distribution moments and exhibit estimators attaining the bounds. Theorem 1 is a self-contained minimax argument: symmetrization reduces to independent randomization, the endpoint restriction defines a finite-dimensional convex program (4), the lower bound is obtained by restricting to vertex potential-outcome configurations, and the upper bound uses a Bernstein extension whose maximum is shown, by separate convexity, to occur at a vertex. Corollary 1 is derived directly from the affine risk formula (A.1). None of these steps imports its conclusion as an assumption. The self-citations to Borusyak and Hull (2023, 2026) and Borusyak et al. (2026) appear only in the Introduction as motivation, terminology, and related literature; they are not invoked in any proof, so they are not load-bearing. The paper also explicitly flags its own limitations: it states the minimax analysis is 'likely to be overly pessimistic' for paired randomization, concedes that stratification based on informative observables 'is likely to dominate' independent randomization in practice, and admits the conjecture that complete randomization is suboptimal for general estimators is not proved ('I have verified this numerically in small samples and conjecture it is true, but as of now have not proved it'). These are scope limitations or missing proofs, not circularity. The central derivation is therefore self-contained and conditional on the stated worst-case criterion.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

No parameters are fitted to data. The unknown objects are potential outcomes, not free parameters of the method. The axioms are the usual minimax setup plus the equivariance restriction for the affine-class results; the general theorem uses only standard mathematical tools and the known-bounds domain assumption.

assumptions (5)

domain assumption Potential outcomes are fixed and lie in a known common interval [L,U] for every unit.
Stated in Section 2; the midpoint M=(U+L)/2 anchors all estimators and is essential to the random-share intuition.
domain assumption The analyst evaluates procedures by worst-case MSE over all configurations in [L,U]^(2N).
Defines the minimax problem (1); the adversary can place both potential outcomes at endpoints, which is what makes balanced designs vulnerable.
ad hoc to paper Affine estimators must satisfy equivariance condition (2): b(D)' (D - (1/2)1) = 1 almost surely.
Used in Propositions 1-3 to make the lower-bound averaging work; many common estimators satisfy it, but it is an extra restriction. Corollary 1 and Theorem 1 relax or remove it.
domain assumption The parameter space is invariant under swaps of (A_i,C_i) and under unit relabeling.
Used in Theorem 1 Step 1 to symmetrize designs and estimators; any prior restriction on the joint distribution of potential outcomes would invalidate this step.
standard math The minimax theorem for finite zero-sum games, Cauchy-Schwarz, and separate convexity of the Bernstein extension.
Invoked in the appendix proofs of Theorem 1 and Corollary 1.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Optimal Experimental Design and Estimation when Potential Outcomes are Bounded." pith.science (2026). https://pith.science/paper/JE7XY2XQ

@misc{pith2026260809812,
  author       = {Pith},
  title        = {Pith review of: Optimal Experimental Design and Estimation when Potential Outcomes are Bounded},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JE7XY2XQ}},
  note         = {Machine review of arXiv:2608.09812}
}

read the original abstract

I study the optimal design and analysis of randomized experiments for estimating finite-population average treatment effects when potential outcomes are known to be bounded, as with binary outcomes. Among all assignment mechanisms and a broad class of affine estimators, worst-case mean-squared error (MSE) is minimized by independent random assignment and an unconventional regression of the support-midpoint-centered outcome on the recentered treatment, with no intercept. This contrasts with the usual prescription of balanced complete randomization and difference-in-means estimation: when outcomes are bounded, randomness in the realized treatment share is informative. The worst-case gain over full-sample complete randomization is asymptotically small, but gains can be first-order relative to other designs: complete within-pair randomization and pair-fixed-effect regression have twice the worst-case MSE. I extend the result to allow for arbitrary estimators. Independent random assignment remains optimal, and the generally-nonlinear optimal estimator can meaningfully reduce worst-case MSE.

Figures

Figures reproduced from arXiv: 2608.09812 by the authors.

**Figure 1.** Nonlinear minimax risk 0 500 1,000 1,500 2,000 2,500 Population size, N 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 Relative worst-case MSE Unrestricted minimax Best affine Notes: The solid blue line plots NκN against the actual population size N, where κN is the minimax MSE from Theorem 1 divided by (U − L) 2 . Thus, it reports the worst-case MSE of the unrestricted minimax estimator as a fraction of the worst-case MSE of … view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

21 extracted references · 20 canonical work pages

[1]

Journal of Econometrics , year =

Bai, Yuehao , title =. Journal of Econometrics , year =

work page
[2]

Journal of the Royal Statistical Society Series B: Statistical Methodology , year =

Kallus, Nathan , title =. Journal of the Royal Statistical Society Series B: Statistical Methodology , year =

work page
[3]

Econometrica , year =

Borusyak, Kirill and Hull, Peter , title =. Econometrica , year =

work page
[4]

Hodges, J. L. and Lehmann, E. L. , title =. Statistics and Probability: Essays in Honor of C. R. Rao , editor =. 1982 , pages =

work page 1982
[5]

Godambe, V. P. , title =. Journal of the Royal Statistical Society Series B: Methodological , year =

work page
[6]

Godambe, V. P. and Joshi, V. M. , title =. The Annals of Mathematical Statistics , year =

work page
[7]

Bickel, P. J. and Lehmann, E. L. , title =. The Annals of Statistics , year =

work page
[8]

Joshi, V. M. , title =. The Annals of Statistics , year =

work page

Show all 21 references

[9]

The Annals of Statistics , year =

Cheng, Ching-Shui and Li, Ker-Chau , title =. The Annals of Statistics , year =
[10]

Metrika , year =

Gabler, Siegfried , title =. Metrika , year =
[11]

1990 , doi =

Gabler, Siegfried , title =. 1990 , doi =

1990
[12]

The Annals of Statistics , year =

Stenger, Horst , title =. The Annals of Statistics , year =
[13]

and Lopatto, Patrick , title =

Aronow, Peter M. and Lopatto, Patrick , title =. arXiv preprint arXiv:2605.20572 , year =

arXiv
[14]

Wu, C. F. , title =. The Annals of Statistics , year =
[15]

and Herzberg, Agnes M

Bickel, Peter J. and Herzberg, Agnes M. , title =. The Annals of Statistics , year =
[16]

Trading-off Bias and Variance in Stratified Experiments and in Matching Studies, Under a Boundedness Condition on the Magnitude of the Treatment Effect , journal =

de Chaisemartin, Cl. Trading-off Bias and Variance in Stratified Experiments and in Matching Studies, Under a Boundedness Condition on the Magnitude of the Treatment Effect , journal =
[17]

American Economic Review , year =

Bai, Yuehao , title =. American Economic Review , year =
[18]

Journal of Business & Economic Statistics , year =

Hahn, Jinyong and Hirano, Keisuke and Karlan, Dean , title =. Journal of Business & Economic Statistics , year =
[19]

The Review of Economic Studies , year =

Tabord-Meehan, Max , title =. The Review of Economic Studies , year =
[20]

Balancing Covariates in Randomized Experiments with the

Harshaw, Christopher and S. Balancing Covariates in Randomized Experiments with the. Journal of the American Statistical Association , year =
[21]

arXiv preprint arXiv:2607.18601 , year =

Borusyak, Kirill and Hull, Peter and Munro, Evan , title =. arXiv preprint arXiv:2607.18601 , year =

arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.

[1] [1]

Journal of Econometrics , year =

Bai, Yuehao , title =. Journal of Econometrics , year =

work page

[2] [2]

Journal of the Royal Statistical Society Series B: Statistical Methodology , year =

Kallus, Nathan , title =. Journal of the Royal Statistical Society Series B: Statistical Methodology , year =

work page

[3] [3]

Econometrica , year =

Borusyak, Kirill and Hull, Peter , title =. Econometrica , year =

work page

[4] [4]

Hodges, J. L. and Lehmann, E. L. , title =. Statistics and Probability: Essays in Honor of C. R. Rao , editor =. 1982 , pages =

work page 1982

[5] [5]

Godambe, V. P. , title =. Journal of the Royal Statistical Society Series B: Methodological , year =

work page

[6] [6]

Godambe, V. P. and Joshi, V. M. , title =. The Annals of Mathematical Statistics , year =

work page

[7] [7]

Bickel, P. J. and Lehmann, E. L. , title =. The Annals of Statistics , year =

work page

[8] [8]

Joshi, V. M. , title =. The Annals of Statistics , year =

work page