Factorial clinical trials in the presence and absence of plausible statistical interactions between treatment factors. A historical review of the methodological literature

Alan A. Montgomery; Alexandra Wright-Hughes; Pritaporn Kingkaew; Rebecca E.A. Walwyn; Steven G. Gilmour

Factorial trial guidance only looks conflicting once you put objectives first: estimands and estimator properties follow from that choice.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-15 01:44 UTC pith:LY5MQBVB

load-bearing objection Abstract-only synthesis that restates a standard point—objectives first, then estimands—to reconcile two factorial-trial literatures; useful clarification, not a new result. the 2 major comments →

arxiv 2607.12991 v1 pith:LY5MQBVB submitted 2026-07-14 stat.ME

Factorial clinical trials in the presence and absence of plausible statistical interactions between treatment factors. A historical review of the methodological literature

Rebecca E.A. Walwyn , Pritaporn Kingkaew , Alan A. Montgomery , Alexandra Wright-Hughes , Steven G. Gilmour This is my paper

classification stat.ME MSC 62K1562P10

keywords factorial trialsstatistical interactionsestimandstreatment contrastsk-in-1 designsdesign of experimentsclinical trial methodologyhistorical review

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This methodological review argues that two long-running schools of thought on factorial clinical trials—one treating them as efficient k-in-1 single-factor studies when interactions are not expected, the other treating them as Design-of-Experiments tools built to estimate interactions—have only appeared to give conflicting advice. The authors show that the apparent conflict dissolves once trialists state their scientific objectives clearly. From those objectives the relevant estimands and treatment contrasts follow, and the statistical properties of the corresponding estimators are then fixed by that choice. Using the historical debate since 1935 and an empirical illustration of alternative analyses, they conclude that objective-first specification is what reconciles design and analysis recommendations for factorial trials run either with or without anticipated interactions.

Core claim

The paper establishes that the long-standing tension between k-in-1 factorial guidance (no-interaction setting) and classical Design-of-Experiments guidance (interaction-estimating setting) is only apparent: once the trial’s scientific objectives are specified, the estimands of interest, the treatment contrasts, and the properties of their estimators are dictated by that choice, thereby reconciling the two literatures.

What carries the argument

An objective-first specification of estimands: the trialist’s scientific goals determine which treatment contrasts are of interest and therefore which estimators are appropriate, whether or not statistical interactions are anticipated.

Load-bearing premise

That surveying the methodological literature since 1935 plus one empirical example is enough to show that the two schools’ advice only seems to conflict and that stating objectives first fully reconciles design and analysis recommendations.

What would settle it

A carefully chosen factorial trial dataset in which two different, equally defensible objective statements produce incompatible recommendations for sample size, randomisation, or primary analysis that cannot be resolved by re-expressing estimands.

Watch this falsifier — get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

Abstract-only synthesis that restates a standard point—objectives first, then estimands—to reconcile two factorial-trial literatures; useful clarification, not a new result.

read the letter

This is a historical review of factorial clinical-trial methodology. From the abstract alone, the punchline is clear and modest: the k-in-1 (no-interaction) literature that dominates UK trial practice and the classical DoE (interaction-estimating) literature look like they conflict, but they mostly don’t once you fix the trial’s objectives first. Estimands and estimator properties then follow. That is standard causal/estimand reasoning applied to a real source of confusion for trialists.

What the paper does well, on its face, is bring the two schools into one place, go back to 1935, and use an empirical example to show how analysis choices change with the objective. The claim that “objectives dictate estimands” is coherent and not circular; it is conceptual rather than a fitted identity. No free parameters or invented entities appear. For multi-arm trial design and funders, a clean restatement of when factorial designs are efficient with vs without interactions is practically useful even if it is not a new theorem or design.

The soft spots are exactly those of an abstract-only review. We cannot check the sampling frame for the 1935–present literature, inclusion criteria, fairness of the historical narrative, or how the empirical example was chosen and analysed. The novelty score is low by design: synthesis and clarification, not a new estimator. The reader’s weakest-assumption flag is right—sufficiency of one example plus the review is asserted, not demonstrated here. The stress-test correctly finds no further technical flaw isolable from the abstract; the central claim is internally coherent.

Who it is for: methodologists and trialists who still treat the two literatures as opposed camps. A serious referee should see the full text. I would not cite it yet without the paper, but I would send it to peer review rather than desk-reject; the topic is live and the framing is honest. Reading-group value is maybe—only if someone wants a short discussion of estimand-first design in multi-factor trials.

Referee Report

2 major / 2 minor

Summary. This methodological review addresses apparent conflicts in guidance on factorial clinical trials between the k-in-1 literature (no anticipated interactions; k single-factor questions with the sample size of one parallel-group trial) and classical Design of Experiments (interactions to be robustly estimated). The authors summarise rationales, treatment contrasts of interest, and estimator properties for both schools; recount the debate from 1935 onward; and use one empirical example to illustrate analysis impacts. They conclude that trialists must carefully specify objectives first; estimands and treatment contrasts then follow, with estimator properties dictated by that choice.

Significance. If the historical synthesis is comprehensive and the empirical illustration well chosen, the paper would usefully clarify conflicting advice that currently confuses trialists. The central recommendation—that objectives dictate estimands—is standard causal/estimand reasoning applied to a practically important design choice, and bringing the two literatures into dialogue is a service to the field. Significance is primarily conceptual and historical rather than a novel statistical method; it rests on the quality and fairness of the review and on whether the reconciliation fully resolves residual design and analysis tensions.

major comments (2)

The central claim that the two schools are only seemingly conflicting, and that objective-first specification fully reconciles design and analysis recommendations, rests on the historical review (1935–present) and a single empirical example. The abstract does not state the sampling frame, inclusion criteria, or search strategy for the methodological literature, nor how the empirical example was selected. These are load-bearing for the reconciliation claim; without transparent, non-selective coverage the conclusion that the literatures are reconciled cannot be verified from the abstract alone.
The abstract asserts that estimands and treatment contrasts follow from objectives, with estimator properties then dictated by that choice. Without the full mapping (in the manuscript body) of how bias, variance, power, and sample-size implications under interaction map to each stated objective, it remains unclear whether residual practical conflicts remain—for example when interactions are plausible but the scientific objective is still main-effect focused. That mapping is load-bearing for the claim that the conflict is resolved rather than merely relocated.

minor comments (2)

Abstract only was available for this review; figure/table clarity, notation consistency, and reference completeness cannot be assessed.
The abstract’s phrasing ‘k-in-1 factorial trial answers k single-factor questions with the same number of units as one parallel-group trial if there are no interactions’ is clear; ensuring the full text defines ‘no interactions’ (absence of interaction on the scale of the estimand, or approximate negligibility) would avoid ambiguity for applied readers.

Circularity Check

0 steps flagged

No significant circularity: abstract-only methodological review with conceptual reconciliation, not fitted predictions or self-definitional claims.

full rationale

This is an abstract-only review of a historical methodological paper. The abstract presents a literature synthesis (k-in-1 factorial vs classical DoE schools, debate from 1935) and a conceptual recommendation: carefully specify trial objectives first so that estimands, treatment contrasts, and estimator properties follow. No equations, fitted parameters, uniqueness theorems, or self-citation chains appear in the available text. There is no claim that a quantity is derived or predicted from data while being definitionally equivalent to an input. The single empirical example is described only as an illustration of analysis-approach impact, not as a fitted result re-labeled as a prediction. Self-citation risk cannot be assessed without the full text or reference list, but nothing load-bearing reduces by construction to the paper's own inputs. Per the hard rules, an honest non-finding is required: score 0, empty steps. The residual concerns flagged by the Reader (sampling of the historical literature; selection of the empirical example) are about completeness and generalizability, not circularity.

Axiom & Free-Parameter Ledger

0 free parameters · 3 axioms · 0 invented entities

Abstract-only review paper. No free parameters are fitted in the abstract. Background axioms are standard clinical-trial and DoE assumptions (interaction definitions, estimand frameworks, historical guidance as stated). No new physical or statistical entities are invented; the contribution is synthesis of prior guidance.

axioms (3)

domain assumption Statistical interaction between treatment factors is a well-defined property that can be present or absent and that changes the interpretation of main-effect contrasts.
Abstract treats presence/absence of interactions as the fork between k-in-1 efficiency claims and classical DoE interaction estimation.
domain assumption A k-in-1 factorial trial answers k single-factor questions with the same number of units as one parallel-group trial if there are no interactions.
Stated as the dominant UK trialist framing that the review contrasts with classical DoE guidance.
standard math Objectives determine estimands, which determine treatment contrasts and the relevant properties of estimators.
Core conclusion of the abstract; standard in modern estimand frameworks (e.g., ICH E9(R1)-style reasoning) though not formally proved here.

pith-pipeline@v1.1.0-grok45 · 6141 in / 2204 out tokens · 20251 ms · 2026-07-15T01:44:05.411147+00:00 · methodology

0 comments

read the original abstract

Factorial trials can be conducted when statistical interactions between two or more treatment factors are not anticipated, but also when they are. A k-in-1 factorial trial answers k single-factor questions with the same number of units as one parallel-group trial if there are no interactions. Literature on k-in-1 factorial trials has dominated trialists understanding of factorial trials in the UK. However, factorial experiments originated from the Design of Experiments field to enable interactions to be robustly estimated. Seemingly conflicting guidance from these literatures poses a source of confusion and misunderstanding for trialists. We bring these literatures together to provide clarity on the arguments that have been used to recommend use of factorial trials in the presence and absence of interactions. We outline motivating examples. We summarise the rationales for using factorial trials, the treatment contrasts of interest, and the properties of their estimators, for the two schools of thought. We describe the debate, going back to 1935, and use an empirical example to illustrate the impact of different analysis approaches. We conclude that it is vital that trialists carefully and clearly specify their objectives. Estimands of interest and treatment contrasts follow, with properties of estimators dictated by this choice.

Factorial clinical trials in the presence and absence of plausible statistical interactions between treatment factors. A historical review of the methodological literature

Core claim

What carries the argument

Load-bearing premise

What would settle it

discussion (0)