Causal Inference from Possibly Unbalanced Split-Plot Designs: A Randomization-based Perspective

Rahul Mukerjee; Tirthankar Dasgupta

arxiv: 1906.08420 · v1 · pith:CDOJETNUnew · submitted 2019-06-20 · 📊 stat.ME

Causal Inference from Possibly Unbalanced Split-Plot Designs: A Randomization-based Perspective

Rahul Mukerjee , Tirthankar Dasgupta This is my paper

Pith reviewed 2026-05-25 19:54 UTC · model grok-4.3

classification 📊 stat.ME

keywords split-plot designscausal inferencerandomization inferencevariance estimationunbalanced designstreatment contrastsminimax biasmatrix analysis

0 comments

The pith

A new variance estimator for treatment contrasts in unbalanced split-plot designs is unbiased under milder conditions.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper develops methods for causal inference based on randomization in split-plot experiments that may be unbalanced. Extending the balanced case gives a variance expression and a conservative variance estimator, but this estimator stays biased even if treatment effects are additive. A detailed matrix analysis produces a new variance estimator that is unbiased under weaker conditions. The paper also provides a way to construct such an estimator with the smallest possible maximum bias.

Core claim

The central claim is that in possibly unbalanced split-plot designs, a matrix analysis of the randomization yields a variance estimator for treatment contrast estimators that becomes unbiased under milder conditions, along with a construction procedure for an estimator with minimax bias.

What carries the argument

The matrix analysis of the sampling variance expression under the randomization distribution in unbalanced split-plot designs.

If this is right

The sampling variance of a treatment contrast estimator can be estimated without bias under milder conditions than before.
A conservative estimator obtained by direct extension from the balanced case has bias that does not vanish even under strict additivity.
A procedure exists to generate a variance estimator with minimax bias.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

This allows experimenters to use unbalanced designs without losing the ability to get unbiased variance estimates for causal effects.
The matrix approach may apply to other experimental designs with randomization restrictions.
Researchers could test the new estimator in real split-plot experiments to check its performance.

Load-bearing premise

The matrix analysis accurately reflects the randomization restrictions and the structure of imbalance in the split-plot design.

What would settle it

If the proposed variance estimator shows non-zero expected bias in an unbalanced split-plot design where treatment effects are additive, the claim of unbiasedness under milder conditions would be false.

Figures

Figures reproduced from arXiv: 1906.08420 by Rahul Mukerjee, Tirthankar Dasgupta.

**Figure 1.** Figure 1: Boxplots of ∆ and ∆ for populations III-VIII e Populations IV through VIII differ only with respect to the correlation parameters that lead to different types of treatment effect heterogeneity. These include all zero correlations in population VI, all negative correlations in population VII, and a mix of positive and negative correlations in population VIII. From each population, 200 sets of potential outc… view at source ↗

read the original abstract

Split-plot designs find wide applicability in multifactor experiments with randomization restrictions. Practical considerations often warrant the use of unbalanced designs. This paper investigates randomization based causal inference in split-plot designs that are possibly unbalanced. Extension of ideas from the recently studied balanced case yields an expression for the sampling variance of a treatment contrast estimator as well as a conservative estimator of the sampling variance. However, the bias of this variance estimator does not vanish even when the treatment effects are strictly additive. A careful and involved matrix analysis is employed to overcome this difficulty, resulting in a new variance estimator, which becomes unbiased under milder conditions. A construction procedure that generates such an estimator with minimax bias is proposed.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

The paper fixes bias in the conservative variance estimator for unbalanced split-plots by deriving a new one via matrix analysis that is unbiased under milder conditions, plus a minimax construction for it.

read the letter

The main takeaway is that this paper fixes bias in the conservative variance estimator for unbalanced split-plots by deriving a new one via matrix analysis that is unbiased under milder conditions, plus a minimax construction for it. They start from the balanced-case randomization results, extend the sampling variance expression and the usual conservative estimator to the unbalanced setting, and show that bias remains even under additive effects. The matrix work then produces an estimator whose bias vanishes under weaker assumptions, and they give a procedure to generate versions with smallest maximum bias. This addresses a practical gap since unbalanced split-plots arise often from real constraints. The randomization-based framing keeps the causal claims tied to the design without extra modeling assumptions, which is a strength. The minimax construction also gives a concrete way to pick an estimator when exact unbiasedness is not feasible. The soft spot is that the abstract calls the matrix analysis careful and involved but supplies no steps, no explicit milder conditions, and no indication of computational cost or how the imbalance structure is encoded in the matrices. Without those details it is impossible to check whether the derivation fully respects the split-plot randomization restrictions or whether the new estimator is simple enough for routine use. The claim of improvement therefore rests on the matrix algebra holding up exactly as described. This is for people working on randomization inference and experimental design who encounter unbalanced split-plots. A reader focused on variance estimation in such designs would get direct value from the new estimator and the construction method. It deserves peer review because it identifies a real limitation in the balanced-case carry-over and offers a targeted technical fix, even though the matrix details will need referee verification.

Referee Report

0 major / 1 minor

Summary. The paper develops randomization-based causal inference methods for split-plot designs that may be unbalanced. It extends the balanced-case results to obtain an expression for the sampling variance of a treatment contrast estimator along with a conservative variance estimator. The bias of the latter does not vanish even under strict additivity. A matrix analysis is used to construct a new variance estimator that is unbiased under milder conditions, together with a procedure that produces an estimator having minimax bias.

Significance. If the matrix derivations are correct, the work supplies practical tools for variance estimation in unbalanced split-plot experiments under a randomization-inference framework. The minimax construction offers a principled selection criterion among candidate estimators. These contributions address a common practical limitation of balanced-design theory.

minor comments (1)

[Abstract] Abstract: the phrase 'milder conditions' is left undefined; a brief parenthetical or footnote indicating the precise relaxation (e.g., constancy of certain interaction terms) would improve immediate readability.

Simulated Author's Rebuttal

0 responses · 0 unresolved

We thank the referee for the positive review and the recommendation to accept the manuscript. The comments confirm the practical value of the randomization-based approach and the minimax construction for unbalanced split-plot designs.

Circularity Check

0 steps flagged

No significant circularity identified

full rationale

The abstract describes an extension of ideas from the balanced case via matrix analysis to produce a new variance estimator that is unbiased under milder conditions, along with a minimax bias construction. No specific equations, definitions, or self-citations are provided that reduce any claimed prediction or result to its inputs by construction, nor is there evidence of fitted inputs renamed as predictions or load-bearing self-citation chains. The derivation is presented as independent matrix work overcoming bias in the conservative estimator, making the central claim self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

Abstract provides no explicit free parameters or invented entities; the central claim rests on the randomization framework and matrix algebra for unbalanced designs.

axioms (1)

domain assumption Randomization-based perspective for causal inference in split-plot designs extends from balanced to unbalanced cases
Invoked when extending ideas from the balanced case to obtain the sampling variance expression.

pith-pipeline@v0.9.0 · 5640 in / 994 out tokens · 32200 ms · 2026-05-25T19:54:44.539885+00:00 · methodology

discussion (0)

Reference graph

Works this paper leans on

17 extracted references · 17 canonical work pages

[1]

Box, G. E. P., Hunter, J. S., and Hunter, W. G. (2005). Statistics for Experimenters: Design, Innovation, and Discovery . John Wiley & Sons, Hoboken, New Jersey, 2nd edition

work page 2005
[2]

Cochran, W. G. (1977). Sampling Techniques . John Wiley & Sons: New York

work page 1977
[3]

Cochran, W. G. and Cox, G. M. (1957). Experimental Designs . John Wiley & Sons, Hoboken, New Jersey, 2nd edition

work page 1957
[4]

S., and Rubin, D

Dasgupta, T., Pillai, N. S., and Rubin, D. B. (2015). Causal inference for 2^ K factorial designs by using potential outcomes. Journal of the Royal Statistical Society, Series B , 77(4):727--753

work page 2015
[5]

Fisher, R. A. (1925). Statistical Methods for Research Workers . Oliver & Boyd, Edinburgh, Scotland

work page 1925
[6]

Fisher, R. A. (1935). The Design of Experiments . Oliver & Boyd, Oxford, England, 1st edition

work page 1935
[7]

Freedman, D. A. (2006). Statistical models for causation: What inferential leverage do they provide? Evaluation Review , 30:691--713

work page 2006
[8]

Freedman, D. A. (2008). On regression adjustments to experimental data. Advances in Applied Mathematics , 40:180--193

work page 2008
[9]

Kirk, R. E. (1982). Experimental Design: Procedures for the Behavioral Sciences . Brooks/Cole, Monterey, CA

work page 1982
[10]

Mukerjee, R., Dasgupta, T., and Rubin, D. B. (2018). Randomization-based causal inference from split-plot designs. Journal of the American Statistical Association , 113:868--881

work page 2018
[11]

Neyman, J. (1923). On the application of probability theory to agricultural experiments. E ssay on principles. S ection 9. Statistical Science , 5:465--472

work page 1923
[12]

Rubin, D. B. (1974). Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of Educational Psychology , 66:688--701

work page 1974
[13]

Rubin, D. B. (1978). Bayesian inference for causal effects: The role of randomization. The Annals of Statistics , 6:34--58

work page 1978
[14]

Rubin, D. B. (2005). Causal inference using potential outcomes: Design, modeling, decisions. Journal of the American Statistical Association , 100:322--331

work page 2005
[15]

Wu, C. F. J. and Hamada, M. S. (2009). Experiments: Planning, Analysis, and Optimization . John Wiley & Sons, Hoboken, New Jersey, 2nd edition

work page 2009
[16]

Yates, F. (1935). Complex experiments. Supplement to the Journal of the Royal Statistical Society , 2:181--247

work page 1935
[17]

Zhao, A., Ding, P., Mukerjee, R., and Dasgupta, T. (2018). Randomization-based causal inference from split-plot designs. Annals of Statistics , 46:1876--1903

work page 2018

[1] [1]

Box, G. E. P., Hunter, J. S., and Hunter, W. G. (2005). Statistics for Experimenters: Design, Innovation, and Discovery . John Wiley & Sons, Hoboken, New Jersey, 2nd edition

work page 2005

[2] [2]

Cochran, W. G. (1977). Sampling Techniques . John Wiley & Sons: New York

work page 1977

[3] [3]

Cochran, W. G. and Cox, G. M. (1957). Experimental Designs . John Wiley & Sons, Hoboken, New Jersey, 2nd edition

work page 1957

[4] [4]

S., and Rubin, D

Dasgupta, T., Pillai, N. S., and Rubin, D. B. (2015). Causal inference for 2^ K factorial designs by using potential outcomes. Journal of the Royal Statistical Society, Series B , 77(4):727--753

work page 2015

[5] [5]

Fisher, R. A. (1925). Statistical Methods for Research Workers . Oliver & Boyd, Edinburgh, Scotland

work page 1925

[6] [6]

Fisher, R. A. (1935). The Design of Experiments . Oliver & Boyd, Oxford, England, 1st edition

work page 1935

[7] [7]

Freedman, D. A. (2006). Statistical models for causation: What inferential leverage do they provide? Evaluation Review , 30:691--713

work page 2006

[8] [8]

Freedman, D. A. (2008). On regression adjustments to experimental data. Advances in Applied Mathematics , 40:180--193

work page 2008

[9] [9]

Kirk, R. E. (1982). Experimental Design: Procedures for the Behavioral Sciences . Brooks/Cole, Monterey, CA

work page 1982

[10] [10]

Mukerjee, R., Dasgupta, T., and Rubin, D. B. (2018). Randomization-based causal inference from split-plot designs. Journal of the American Statistical Association , 113:868--881

work page 2018

[11] [11]

Neyman, J. (1923). On the application of probability theory to agricultural experiments. E ssay on principles. S ection 9. Statistical Science , 5:465--472

work page 1923

[12] [12]

Rubin, D. B. (1974). Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of Educational Psychology , 66:688--701

work page 1974

[13] [13]

Rubin, D. B. (1978). Bayesian inference for causal effects: The role of randomization. The Annals of Statistics , 6:34--58

work page 1978

[14] [14]

Rubin, D. B. (2005). Causal inference using potential outcomes: Design, modeling, decisions. Journal of the American Statistical Association , 100:322--331

work page 2005

[15] [15]

Wu, C. F. J. and Hamada, M. S. (2009). Experiments: Planning, Analysis, and Optimization . John Wiley & Sons, Hoboken, New Jersey, 2nd edition

work page 2009

[16] [16]

Yates, F. (1935). Complex experiments. Supplement to the Journal of the Royal Statistical Society , 2:181--247

work page 1935

[17] [17]

Zhao, A., Ding, P., Mukerjee, R., and Dasgupta, T. (2018). Randomization-based causal inference from split-plot designs. Annals of Statistics , 46:1876--1903

work page 2018