Bayesian design and analysis of external pilot trials for complex interventions

Amanda J. Farrin; Duncan T. Wilson; James M. S. Wason; Julia Brown; Rebecca E. A. Walwyn

REVIEW 3 major objections 5 minor 44 references

Bayesian design and analysis of external pilot trials for complex interventions

T0 review · 3 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper shows that external pilot trial progression decisions can be made by minimising a three-parameter piecewise-constant loss function, with operating characteristics evaluable at the design stage.

desk verdict A genuinely useful Bayesian framework for pilot-trial progression decisions, but the printed core equations have fixable typos and the paper needs a revision before anyone should implement from it. read the letter →

arxiv 1908.05955 v2 pith:7INPBUAU submitted 2019-08-16 stat.ME

Duncan T. Wilson , James M. S. Wason , Julia Brown , Amanda J. Farrin , Rebecca E. A. Walwyn This is my paper · ORCID

classification stat.ME MSC 62C1062F1562P10

keywords Bayesiandecisiontheorypilottrialscomplexinterventionsprogressioncriterialossfunctionoperatingcharacteristicsassurancesamplesizedetermination

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

External pilot trials of complex interventions — treatments made of several interacting components — collect small samples to inform whether a confirmatory trial should go ahead, yet the thresholds used to decide this are usually chosen without assessing their statistical properties. This paper argues that the progression decision should be treated as a Bayesian decision problem: after the pilot, choose the action that minimises expected loss, where loss is defined over the whole parameter space so that trade-offs among recruitment, retention, adherence and potential efficacy can be made explicit. To keep this feasible, the loss is piecewise constant and summarised by three cost parameters, and a design-stage simulation uses a design prior to compute operating characteristics, so error probabilities and sample-size implications can be examined before data are collected. The method is demonstrated by revisiting two external pilot trials, including a cluster-randomised pilot of an intervention to increase physical activity in care home residents, and by showing how the loss parameters can be chosen to obtain a desired balance of errors.

What carries the argument

The load-bearing object is an additive, piecewise-constant loss function over the substantive parameter space, together with a partition of that space into three hypotheses $\Phi_R$, $\Phi_A$, $\Phi_G$ corresponding to the ideal decisions. Its additivity makes the expected loss of each decision a linear function of the posterior probabilities $p_R,p_A,p_G$, so the minimising decision follows directly from an MCMC posterior sample; its piecewise constancy is what allows the whole preference structure to be collapsed into the three costs $c_1,c_2,c_3$. Around this sits a nested Monte Carlo scheme that draws parameters from a design prior, simulates pilot data, and repeats the Bayesian analysis, producing unconditional error probabilities that can be used to choose the sample size and to select loss parameters by multi-objective optimisation.

What would settle it

For a simple conjugate model with two independent binomial outcomes, compute the paper's operating characteristics both by nested Monte Carlo and by exact enumeration of all possible pilot data sets; disagreement beyond Monte Carlo error would indicate a flaw in the algorithm. To test the preference assumption, ask a decision maker the two indifference gambles the paper describes and a third logically equivalent gamble; if the implied cost parameters differ, the additive three-parameter loss does not capture the stated preferences.

Watch

Extended reading notes

Core claim

The central claim is that progression decisions in external pilot trials can and should be made by minimising expected loss rather than by comparing point estimates to arbitrary thresholds. The substantive parameter space is partitioned into regions corresponding to the red, amber and green decisions; three error types are defined — proceeding to an infeasible main trial, discarding a promising intervention, and making unnecessary adjustments — and a loss function $L(d,\phi)=c_1E_1+c_2E_2+c_3E_3$ assigns costs to each. Because the loss is piecewise constant, the expected loss of each action depends only on the posterior probabilities of the three hypothesis regions, which can be computed by MCMC even for complex multilevel models. At the design stage, sampling from a design prior and repeating the analysis yields unconditional probabilities of each error type, and a search over the cost parameters reveals the achievable trade-offs. In the worked examples, the resulting pilot designs have error probabilities near conventional levels, and one cost parameter — the cost of discarding a promising intervention — dominates the operating characteristics.

Load-bearing premise

The decision maker's preferences can be summarised by three fixed costs, one for each type of wrong decision, with no additional dependence on how far wrong the decision is or on which other wrong decisions occur at the same time.

Editorial extensions

If this is right

Progression criteria become decision rules with known error rates, so pilot sample size can be chosen to control the probability of a wrong go/no-go decision.
Trade-offs between feasibility endpoints — such as accepting lower adherence when potential efficacy is higher — can be articulated through the hypothesis partition instead of being ignored by independent thresholds.
Complex multilevel models with small samples can drive the decision, because only posterior probabilities of hypothesis regions are needed and these come from MCMC rather than closed-form criteria.
A multi-objective search over the three costs lets sponsors view the frontier of achievable error probabilities and select a design matching their preferences rather than a conventional default.
Using an informative prior for a parameter with very little pilot data lowers expected loss while shifting the error balance, making prior choice a substantive design decision.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The same decision machinery could be applied to internal pilot or seamless phase II/III designs, with the amber action modelled concretely as an adaptive change to the main trial rather than left as an unmodelled option.
Because the three-cost loss assumes the cost of discarding an intervention does not depend on how effective it is, a natural test is to elicit indifference gambles at different points within a hypothesis and check whether the implied costs remain constant.
If the three costs were derived from the health-economic consequences of each error rather than from expert judgement, the operating-characteristic frontier could be converted into expected net benefit and compared directly with value-of-information analyses.
The finding that one cost parameter dominates the operating characteristics suggests a diagnostic: designs in which a single error dominates may need different hypotheses, larger samples, or different endpoints before costs are worth eliciting.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Short version: this paper has a genuinely useful idea — Bayesian decision-theoretic progression rules for external pilot trials — but the printed equations in the core derivation have real typos. The framework is salvageable and the examples are informative, but don't implement from the paper as it stands.

What's new: the authors adapt Bayesian decision theory to the traffic-light (red/amber/green) progression decisions that pilot trials actually use, with a piecewise-constant loss over the parameter space. That lets you trade off multiple endpoints (recruitment, adherence, follow-up) rather than applying independent thresholds, and you can choose the loss parameters by looking at operating characteristics rather than by pure elicitation. This goes beyond standard assurance calculations and is a sensible adaptation of admissible-design ideas from phase II. The two worked examples — TIGA-CUB and REACH — demonstrate the method on realistic, messy problems, and the code is on GitHub. The authors also flag the main conceptual limitation themselves: the additive piecewise-constant loss may not capture actual preferences.

Soft spots: Section 2.2 has a systematic c2/c3 swap in equations (3)–(5). Table 1 is correct, but the expected loss formulas printed don't follow from it. Equation (10) has the indicator reversed: they say decision g is optimal when pG > c1, then write I(pG < c1). These are mechanical errors, not a flaw in the framework — the REACH example uses the correct logic — but they sit at the center of the method and would mislead a careful implementer. There are also two smaller label errors: the Figure 6 caption gives a cost vector (0.069, 0.116, 0.815) that doesn't match the point-a values (0.07, 0.9, 0.03) in the text and Table 3, and Section 4.2.2 swaps OC2 and OC3 when describing the INA results. The nested Monte Carlo computation is heavy; the paper acknowledges that and points to possible speedups, but it's a real barrier to uptake.

The circularity concern doesn't land. Using a design prior to compute operating characteristics is standard Bayesian design, and the cost parameters are used to explore a Pareto front, not fitted to force a conclusion.

Verdict: the central claim holds up after corrections. This deserves peer review — I think a good referee would send it back for revision rather than reject it. It would be a solid contribution to the pilot-trial design literature once the equations are fixed. I'd bring it to a reading group mainly to talk about whether the loss function is flexible enough; the typos are annoying but not the interesting part.

Referee Report

3 major / 5 minor

Summary. The manuscript develops a Bayesian decision-theoretic framework for designing and analysing external pilot trials of complex interventions. The pilot data update an analysis prior; a red/amber/green progression decision is then chosen by minimising posterior expected loss under a piecewise-constant, additive loss parameterised by three costs c1,c2,c3. At the design stage, operating characteristics are defined as unconditional error probabilities with respect to a design prior and estimated by nested Monte Carlo simulation; the loss parameters can be varied to approximate a Pareto frontier of operating characteristics, and this is used to support sample-size choice. The method is illustrated on the TIGA-CUB and REACH pilot trials, and the authors provide code and simulated data on GitHub.

Significance. If the technical errors below are corrected, the paper makes a useful contribution to pilot-trial methodology. It addresses a genuine and under-studied problem: progression criteria for external pilots are routinely pre-specified but their operating characteristics are rarely evaluated. The proposed framework is coherent, handles multiple endpoints, small samples, multi-level models, and nuisance parameters within a single Bayesian workflow, and is demonstrated on two real trials. The use of a subjective design prior for assurance-type operating characteristics is standard Bayesian design rather than circularity, and the authors are appropriately explicit about the practical burden of prior and hypothesis elicitation. The GitHub repository containing code and simulated data strengthens reproducibility. The main risks are internal inconsistencies in the printed equations and notation rather than the conceptual framework; these need to be fixed before the method can be implemented from the paper as written.

major comments (3)

[Section 2.2, Eqs. (3)-(5)] The expected-loss equations are inconsistent with Table 1. From Table 1, the correct expressions are E[L(r)] = (p_A + p_G)c2, E[L(a)] = p_R(c1+c3) + p_G c3, and E[L(g)] = p_R c1 + p_A(c1+c2). The printed equations interchange c2 and c3 in every term. Because c2 and c3 differ in the REACH illustration (for point a, c2=0.9 and c3=0.03), a reader implementing the printed equations can obtain a different argmin and hence a different progression decision. These equations are the core decision rule and must be corrected; the fact that the special two-decision case in Section 3 is derived correctly suggests the numerical work may have used the right formulas, but the printed derivation is wrong.
[Section 3, Eq. (10)] The indicator in Eq. (10) is reversed. The text states that decision g is optimal whenever p_G > c1, so the probability of proceeding to an infeasible trial should be computed using I(p_G > c1 | x_f, x_a, n), not I(p_G < c1 | x_f, x_a, n). As printed, Eq. (10) counts the opposite decision and would give incorrect operating characteristics for OC1. This is a load-bearing formula for the TIGA-CUB example and needs correction.
[Section 3, Eq. (8) and surrounding text] The meaning of the sample size n is inconsistent. The likelihood is written with a per-arm sample size n, giving 2n follow-up observations and n adherence observations, yet Eq. (8) gives posterior Beta parameters (1+x_f, 1+n-x_f) and (1+x_a, 1+n/2-x_a), which correspond instead to a total sample size n with n/2 adherence observations. The paragraph also says 'given a total sample size n', while later text says 'n = 30 per arm'. Depending on the intended convention, the posterior probabilities and the operating characteristics in Figures 1-2 change. The paper should adopt one convention consistently and ensure Eq. (8), Eq. (10), and the simulation code all match it.

minor comments (5)

[Section 4.2.2] The sentence reporting that the INA analysis prior leads to 'larger probabilities of an infeasible trial (OC1) and of unnecessary adjustment (OC2), while reducing the probability of discarding a promising intervention (OC3)' swaps the definitions of OC2 and OC3 from Section 2.3. It should say 'unnecessary adjustment (OC3)' and 'discarding a promising intervention (OC2)'.
[Figure 6 caption] The caption states that the operating characteristics are evaluated at (c1,c2,c3) = (0.069, 0.116, 0.815), whereas the text says the parameters are set to point a of Table 3, (0.07, 0.9, 0.03). These are different cost vectors; the caption or the text must be corrected.
[Section 2.4] In the dominance definition, the phrase 'If there exist c, c′ ∈ C*' should refer to two vectors in the sampled set C, not in C*, since C* is the set of non-dominated parameters being constructed.
[Section 2.2] There is a typo in the sentence 'the our preferences for any one of the attributes E1,E2,E3 are independent...'; 'the our' should be 'our'.
[Section 5] The Discussion correctly acknowledges that the piecewise-constant loss function may not adequately represent the decision maker's preferences. This is an important scope condition and should be stated at the point the loss function is introduced in Section 2.2, not only in the discussion.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the operating characteristics are simulation-based design outputs, not fitted predictions; the sole overlapping-author citation is background, and the printed equation typos are correctness issues outside this pass.

full rationale

The paper's derivation chain is self-contained and non-circular. The decision rule is obtained by minimizing posterior expected loss with respect to a user-specified piecewise-constant loss function (Section 2.2); the loss parameters are either elicited from preferences or selected by viewing the Pareto front of Monte Carlo simulated operating characteristics (Sections 2.3-2.4), so no fitted quantity is presented as a prediction. The operating characteristics are unconditional simulation outputs from the design prior and the same decision rule, not independent empirical outcomes, and the paper transparently labels them as design-stage evaluations. The only overlapping-author citation [16] is background support for the existence of methodological challenges, not a load-bearing theorem or uniqueness assertion. The apparent swaps of c2 and c3 in Eqs (3)-(5) and the reversed indicator in Eq (10) are internal arithmetic/indexing errors that would affect implementation, but they do not constitute circularity: they are correctness defects, not reductions of outputs to inputs.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The method relies on a set of subjective inputs: cost parameters, hypothesis boundaries, design priors, and analysis priors. These are not fitted to data but are chosen by the decision maker. The simulation-based evaluation is transparent about these inputs, but the operating characteristics are only meaningful if the inputs are correct. No new physical or conceptual entities are introduced.

free parameters (6)

Cost parameters (c1, c2, c3) with c1+c2+c3=1 = c1=0.2, 0.36, 0.5 in TIGA-CUB; various vectors in REACH, e.g., (0.07, 0.9, 0.03)
Define the trade-off between the three error types. They are chosen by the decision maker or explored over a grid, not fitted to data.
TIGA-CUB hypothesis thresholds = pf >= 0.8 and pa >= 0.7 for green
These partition the parameter space into Phi_G and Phi_R. Chosen by the authors for illustration; in practice they would be set by clinical considerations.
REACH hypothesis boundaries = Phi_i: pf < 0.6 or 20 - 15pf > muc for red; pf > 0.66 and 22 - 15pf < muc for green.
Arbitrary but explicitly elicited linear trade-offs chosen by the authors. The operating characteristics depend on these boundaries.
TIGA-CUB design prior hyperparameters = pf ~ Beta(40,10), pa ~ Beta(11.2,4.8)
Subjective priors reflecting expected ranges 62-92% for follow-up and 40-95% for adherence.
REACH design prior hyperparameters = muc ~ N(10, sigma^2/6), sigma^2 ~ Inv-Gamma(20,39); pf ~ Beta(22.4,9.6); pa ~ Beta(28.8,3.2); mu ~ N(0.2, 0.25^2)…
Subjective priors elicited from the study team.
Analysis prior hyperparameters = TIGA-CUB: Beta(1,1); REACH: weakly informative prior detailed in the (missing) appendix
Used for posterior computation; the paper recommends weakly informative analysis priors to let data dominate.

assumptions (5)

domain assumption The sampling model p(x|theta) is correctly specified.
The operating characteristics are computed with respect to this model; if the model is wrong, the OCs are of the wrong quantities.
domain assumption The additive loss function correctly represents preferences (independence of attributes).
Explicitly stated in Section 2.2: 'the additive form of the loss function implies that the preferences for any one of the attributes E1,E2,E3 are independent of the values taken by the others'. The paper flags this as a potential limitation.
domain assumption The design prior p_D(theta) is a fully subjective representation of the decision maker's knowledge.
The method requires this for unconditional probabilities to be meaningful (Section 2.1).
domain assumption The hypothesis partition (Phi_R, Phi_A, Phi_G) is known and specifies the true optimal decision for each parameter value.
The loss is defined over these subspaces; if the partition is wrong, the errors are mis-specified.
standard math Standard Bayesian computation (MCMC) produces samples from the posterior.
The paper uses Stan and assumes the MCMC samples are reliable enough after burn-in.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Bayesian design and analysis of external pilot trials for complex interventions." pith.science (2026). https://pith.science/paper/7INPBUAU

@misc{pith2026190805955,
  author       = {Pith},
  title        = {Pith review of: Bayesian design and analysis of external pilot trials for complex interventions},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7INPBUAU}},
  note         = {Machine review of arXiv:1908.05955}
}

read the original abstract

External pilot trials of complex interventions are used to help determine if and how a confirmatory trial should be undertaken, providing estimates of parameters such as recruitment, retention and adherence rates. The decision to progress to the confirmatory trial is typically made by comparing these estimates to pre-specified thresholds known as progression criteria, although the statistical properties of such decision rules are rarely assessed. Such assessment is complicated by several methodological challenges, including the simultaneous evaluation of multiple endpoints, complex multi-level models, small sample sizes, and uncertainty in nuisance parameters. In response to these challenges, we describe a Bayesian approach to the design and analysis of external pilot trials. We show how progression decisions can be made by minimising the expected value of a loss function, defined over the whole parameter space to allow for preferences and trade-offs between multiple parameters to be articulated and used in the decision making process. The assessment of preferences is kept feasible by using a piecewise constant parameterisation of the loss function, the parameters of which are chosen at the design stage to lead to desirable operating characteristics. We describe a flexible, yet computationally intensive, nested Monte Carlo algorithm for estimating operating characteristics. The method is used to revisit the design of an external pilot trial of a complex intervention designed to increase the physical activity of care home residents.

Figures

Figures reproduced from arXiv: 1908.05955 by the authors.

**Figure 2.** Probabilities of an infeasible main trial ( [PITH_FULL_IMAGE:figures/full_fig_p011_2.png] view at source ↗

**Figure 3.** Marginal hypotheses over parameters for (a) follow-up [PITH_FULL_IMAGE:figures/full_fig_p014_3.png] view at source ↗

Figures from the paper (4 more)

**Figure 4.** Figure 4: Operating characteristics of the example pilot trial for a r [PITH_FULL_IMAGE:figures/full_fig_p016_4.png]

**Figure 5.** Figure 5: Relationships between the three loss parameters ( [PITH_FULL_IMAGE:figures/full_fig_p017_5.png]

**Figure 6.** Figure 6: Operating characteristics of the REACH trial for per-ar [PITH_FULL_IMAGE:figures/full_fig_p018_6.png]

**Figure 7.** Figure 7: Operating characteristics and expected utilities for weak [PITH_FULL_IMAGE:figures/full_fig_p019_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

44 extracted references · 38 canonical work pages

[1]

Developing and evaluating complex intervention s: the new medical research council guidance

Peter Craig, Paul Dieppe, Sally Macintyre, Susan Michie, Irwin Naz areth, and Mark Petticrew. Developing and evaluating complex intervention s: the new medical research council guidance. BMJ: British Medical Journal , 337, 9 2008

work page 2008
[2]

Eldridge, Gillian A

Sandra M. Eldridge, Gillian A. Lancaster, Michael J. Campbell, Leha na Thabane, Sally Hopewell, Claire L. Coleman, and Christine M. Bond. Deﬁning feasibility and pilot studies in preparation for randomised con - trolled trials: Development of a conceptual framework. PLOS ONE , 11(3):e0150205, mar 2016

work page 2016
[3]

Lancaster, Susanna Dodd, and Paula R

Gillian A. Lancaster, Susanna Dodd, and Paula R. Williamson. Design a nd analysis of pilot studies: recommendations for good practice. Journal of Evaluation in Clinical Practice , 10(2):307–312, 2004

work page 2004
[4]

Research for patient beneﬁt (rfpb) programme guidance on applying for feasibility studies, 2017

National Institute for Health Research. Research for patient beneﬁt (rfpb) programme guidance on applying for feasibility studies, 2017

work page 2017
[5]

CONSORT 2010 statement: extension to randomised pilot and feasibility trials

Sandra M Eldridge, Claire L Chan, Michael J Campbell, Christine M Bon d, Sally Hopewell, Lehana Thabane, and Gillian A Lancaster. CONSORT 2010 statement: extension to randomised pilot and feasibility trials. BMJ, page i5239, oct 2016

work page 2010
[6]

Informing eﬃcient randomised controlled trials: exp lo- ration of challenges in developing progression criteria for internal p ilot stud- ies

Kerry N L Avery, Paula R Williamson, Carrol Gamble, Elaine O’Connell Francischetto, Chris Metcalfe, Peter Davidson, Hywel Williams, and Jane M Blazeby. Informing eﬃcient randomised controlled trials: exp lo- ration of challenges in developing progression criteria for internal p ilot stud- ies. BMJ Open , 7(2):e013537, feb 2017

work page 2017
[7]

A framework for prospectively deﬁning progression rules for inter nal pilot studies monitoring recruitment

Lisa V Hampson, Paula R Williamson, Martin J Wilby, and Thomas Jaki. A framework for prospectively deﬁning progression rules for inter nal pilot studies monitoring recruitment. Statistical Methods in Medical Research , 0(0):0962280217708906, 2017. PMID: 28589752

work page 2017
[8]

Richard H. Browne. On the use of a pilot sample for sample size dete rmi- nation. Statistics in Medicine , 14(17):1933–1940, 1995. 22

work page 1933

Show all 44 references

[9]

Steven A. Julious. Sample size of 12 per group rule of thumb for a p ilot study. Pharmaceutical Statistics, 4(4):287–291, 2005

2005
[10]

The size of a pilot study for a clinical tr ial should be calculated in relation to considerations of precision and eﬃc iency

Julius Sim and Martyn Lewis. The size of a pilot study for a clinical tr ial should be calculated in relation to considerations of precision and eﬃc iency. Journal of Clinical Epidemiology , 65(3):301–308, mar 2012

2012
[11]

Sample size requirements to estimate k ey de- sign parameters from external pilot randomised controlled trials: a simula- tion study

M Teare, Munyaradzi Dimairo, Neil Shephard, Alex Hayman, Amy White- head, and Stephen Walters. Sample size requirements to estimate k ey de- sign parameters from external pilot randomised controlled trials: a simula- tion study. Trials, 15(1):264, 2014

2014
[12]

How big should the pilot study for my cluste r randomised trial be? Statistical Methods in Medical Research , 2015

Sandra M Eldridge, Ceire E Costelloe, Brennan C Kahan, Gillian A Lan - caster, and Sally M Kerry. How big should the pilot study for my cluste r randomised trial be? Statistical Methods in Medical Research , 2015

2015
[13]

Estimating the sample size for a pilot randomised trial to min- imise the overall trial sample size for the external pilot and main trial for a continuous outcome variable

Amy L Whitehead, Steven A Julious, Cindy L Cooper, and Michael J Campbell. Estimating the sample size for a pilot randomised trial to min- imise the overall trial sample size for the external pilot and main trial for a continuous outcome variable. Statistical Methods in Medica...

2015
[14]

A simple formula for the calculation of sa mple size in pilot studies

Wolfgang Viechtbauer, Luc Smits, Daniel Kotz, Luc Bud´ e, Mark Spigt, Jan Serroyen, and Rik Crutzen. A simple formula for the calculation of sa mple size in pilot studies. Journal of Clinical Epidemiology , 68(11):1375–1379, nov 2015

2015
[15]

Are pilot trials useful for predicting randomisat ion and attrition rates in deﬁnitive studies: A review of publicly funded tr ials

Cindy L Cooper, Amy Whitehead, Edward Pottrill, Steven A Julious , and Stephen J Walters. Are pilot trials useful for predicting randomisat ion and attrition rates in deﬁnitive studies: A review of publicly funded tr ials. Clinical Trials, 0(0):1740774517752113, 2018. PMID: 29361833

2018
[16]

D. T. Wilson, R. E. Walwyn, J. Brown, A. J. Farrin, and S. R. Brow n. Statistical challenges in assessing potential eﬃcacy of complex inte rven- tions in pilot or feasibility studies. Statistical Methods in Medical Research, 25(3):997–1009, jun 2015

2015
[17]

Stevens, and Michael J

Anthony O’Hagan, John W. Stevens, and Michael J. Campbell. As surance in clinical trial design. Pharmaceutical Statistics, 4(3):187–201, 2005

2005
[18]

Pract ical experiences of adopting assurance as a quantitative framework t o support decision making in drug development

Adam Crisp, Sam Miller, Douglas Thompson, and Nicky Best. Pract ical experiences of adopting assurance as a quantitative framework t o support decision making in drug development. Pharmaceutical Statistics, 0(0), 2018

2018
[19]

Fei Wang and Alan E. Gelfand. A simulation-based approach to ba yesian sample size determination for performance under a given model and for separating models. Statistical Science, 17(2):pp. 193–208, 2002. 23

2002
[20]

Walley, Claire L

Rosalind J. Walley, Claire L. Smith, Jeremy D. Gale, and Phil Woodwa rd. Advantages of a wholly bayesian approach to assessing eﬃcacy in ea rly drug development: a case study. Pharmaceutical Statistics, 14(3):205–215, apr 2015

2015
[21]

A weakly informative default prior distribution for logistic and other re gres- sion models

Andrew Gelman, Aleks Jakulin, Maria Grazia Pittau, and Yu-Sung S u. A weakly informative default prior distribution for logistic and other re gres- sion models. The Annals of Applied Statistics , 2(4):1360–1383, dec 2008

2008
[22]

Spiegelhalter

David J. Spiegelhalter. Bayesian methods for cluster randomize d trials with continuous responses. Statistics in Medicine , 20(3):435–452, 2001

2001
[23]

Better decisio n mak- ing in drug development through adoption of formal prior elicitation

Nigel Dallow, Nicky Best, and Timothy H Montague. Better decisio n mak- ing in drug development through adoption of formal prior elicitation. Phar- maceutical Statistics, 0(0), 2018

2018
[24]

Buck, Alireza Daneshkhah, J

Anthony O’Hagan, Caitlin E. Buck, Alireza Daneshkhah, J. Richar d Eiser, Paul H. Garthwaite, David J. Jenkinson, Jeremy E. Oakley, and Tim Rakow. Uncertain Judgements: Eliciting Experts’ Probabilities. John Wiley and Sons, 2006

2006
[25]

Statistical Decision Theory

Simon French and David Rios Insua. Statistical Decision Theory. Number 9 in Kendall’s Library of Statistics. Oxford University Press, 2000

2000
[26]

Keeney and Howard Raiﬀa

Ralph L. Keeney and Howard Raiﬀa. Decisions with multiple objectives: preferences and value tradeoﬀs . John Wiley & Sons, 1976

1976
[27]

Research exploring physical activity in care homes (REACH): study pro- tocol for a randomised controlled trial

Anne Forster, , Jennifer Airlie, Karen Birch, Robert Cicero, Bo nnie Cundill, Alison Ellwood, Mary Godfrey, Liz Graham, John Green, Claire Hulme, Rebecca Lawton, Vicki McLellan, Nicola McMaster, and Amanda Farr in. Research exploring physical activity in care homes (REACH): st...

2017
[28]

RStan: the R interface to Stan, 2016

Stan Development Team. RStan: the R interface to Stan, 2016 . R package version 2.14.1

2016
[29]

Sin-Ho Jung, Taiyeong Lee, Kyung Mann Kim, and Stephen L. Geo rge. Admissible two-stage designs for phase II cancer clinical trials. Statistics in Medicine, 23(4):561–569, 2004

2004
[30]

Mander, James M.S

Adrian P. Mander, James M.S. Wason, Michael J. Sweeting, and S imon G. Thompson. Admissible two-stage designs for phase II cancer clinica l trials that incorporate the expected sample size under the alternative h ypothesis. Pharmaceutical Statistics, 11(2):91–96, 2012

2012
[31]

Current sample size conventions: Flaws, har ms, and al- ternatives

Peter Bacchetti. Current sample size conventions: Flaws, har ms, and al- ternatives. BMC Medicine, 8(1):17, Mar 2010. 24

2010
[32]

Sutton, Nicola J

Alexander J. Sutton, Nicola J. Cooper, David R. Jones, Paul C. Lambert, John R. Thompson, and Keith R. Abrams. Evidence-based sample siz e calculations based upon updated meta-analysis. Statistics in Medicine , 26(12):2479–2500, 2007

2007
[33]

Oakley, and Alan Brennan

Mark Strong, Jeremy E. Oakley, and Alan Brennan. Estimating m ultipa- rameter partial expected value of perfect information from a pro babilistic sensitivity analysis sample. Medical Decision Making , 34(3):311–326, apr 2014

2014
[34]

Es- timating the expected value of sample information using the probabilis tic sensitivity analysis sample: a fast, nonparametric regression-bas ed method

Mark Strong, Jeremy E Oakley, Alan Brennan, and Penny Breez e. Es- timating the expected value of sample information using the probabilis tic sensitivity analysis sample: a fast, nonparametric regression-bas ed method. Medical Decision Making , 35(5):570–583, 2015

2015
[35]

Donald R. Jones. A taxonomy of global optimization methods bas ed on response surfaces. Journal of Global Optimization , 21(4):345–383, 2001

2001
[36]

DiceKriging , DiceOptim: Two R packages for the analysis of computer experiment s by kriging-based metamodeling and optimization

Olivier Roustant, David Ginsbourger, and Yves Deville. DiceKriging , DiceOptim: Two R packages for the analysis of computer experiment s by kriging-based metamodeling and optimization. Journal of Statistical Software, 51(1):1–55, 2012

2012
[37]

Dennis V. Lindley. The choice of sample size. Journal of the Royal Statis- tical Society: Series D (The Statistician) , 46(2):129–138, 1997

1997
[38]

Lawrence Joseph and David B. Wolfson. Interval-based versu s decision the- oretic criteria for the choice of sample size. Journal of the Royal Statistical Society: Series D (The Statistician) , 46(2):145–149, 1997

1997
[39]

McCulloch, and Mark R

Peter Bacchetti, Charles E. McCulloch, and Mark R. Segal. Simple , defen- sible sample sizes based on cost eﬃciency. Biometrics, 64(2):577–585, jun 2008

2008
[40]

Bayesian sample size for exploratory clinical trials incorporatin g his- torical data

John Whitehead, Elsa Vald´ es-M´ arquez, Patrick Johnson, and Gordon Gra- ham. Bayesian sample size for exploratory clinical trials incorporatin g his- torical data. Statistics in Medicine , 27(13):2307–2327, 2008

2008
[41]

Optimal sample sizes for phase II clinical trials and pilot studies

Nigel Stallard. Optimal sample sizes for phase II clinical trials and pilot studies. Statistics in Medicine , 31(11-12):1031–1042, 2012

2012
[42]

James M. S. Wason, Thomas Jaki, and Nigel Stallard. Planning mult i- arm screening studies within the context of a drug developmentpro gram. Statistics in Medicine , 32(20):3424–3435, 2013

2013
[43]

Sam- ple size planning for phase II trials based on success probabilities for phase III

Heiko Gtte, Armin Schler, Marietta Kirchner, and Meinhard Kiese r. Sam- ple size planning for phase II trials based on success probabilities for phase III. Pharmaceutical Statistics, 14(6):515–524, sep 2015. 25

2015
[44]

Utility-based optimization of phase II/III programs

Marietta Kirchner, Meinhard Kieser, Heiko Gtte, and Armin Schle r. Utility-based optimization of phase II/III programs. Statist. Med. , 35(2):305–316, aug 2015. 26

2015

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [1]

Developing and evaluating complex intervention s: the new medical research council guidance

Peter Craig, Paul Dieppe, Sally Macintyre, Susan Michie, Irwin Naz areth, and Mark Petticrew. Developing and evaluating complex intervention s: the new medical research council guidance. BMJ: British Medical Journal , 337, 9 2008

work page 2008

[2] [2]

Eldridge, Gillian A

Sandra M. Eldridge, Gillian A. Lancaster, Michael J. Campbell, Leha na Thabane, Sally Hopewell, Claire L. Coleman, and Christine M. Bond. Deﬁning feasibility and pilot studies in preparation for randomised con - trolled trials: Development of a conceptual framework. PLOS ONE , 11(3):e0150205, mar 2016

work page 2016

[3] [3]

Lancaster, Susanna Dodd, and Paula R

Gillian A. Lancaster, Susanna Dodd, and Paula R. Williamson. Design a nd analysis of pilot studies: recommendations for good practice. Journal of Evaluation in Clinical Practice , 10(2):307–312, 2004

work page 2004

[4] [4]

Research for patient beneﬁt (rfpb) programme guidance on applying for feasibility studies, 2017

National Institute for Health Research. Research for patient beneﬁt (rfpb) programme guidance on applying for feasibility studies, 2017

work page 2017

[5] [5]

CONSORT 2010 statement: extension to randomised pilot and feasibility trials

Sandra M Eldridge, Claire L Chan, Michael J Campbell, Christine M Bon d, Sally Hopewell, Lehana Thabane, and Gillian A Lancaster. CONSORT 2010 statement: extension to randomised pilot and feasibility trials. BMJ, page i5239, oct 2016

work page 2010

[6] [6]

Informing eﬃcient randomised controlled trials: exp lo- ration of challenges in developing progression criteria for internal p ilot stud- ies

Kerry N L Avery, Paula R Williamson, Carrol Gamble, Elaine O’Connell Francischetto, Chris Metcalfe, Peter Davidson, Hywel Williams, and Jane M Blazeby. Informing eﬃcient randomised controlled trials: exp lo- ration of challenges in developing progression criteria for internal p ilot stud- ies. BMJ Open , 7(2):e013537, feb 2017

work page 2017

[7] [7]

A framework for prospectively deﬁning progression rules for inter nal pilot studies monitoring recruitment

Lisa V Hampson, Paula R Williamson, Martin J Wilby, and Thomas Jaki. A framework for prospectively deﬁning progression rules for inter nal pilot studies monitoring recruitment. Statistical Methods in Medical Research , 0(0):0962280217708906, 2017. PMID: 28589752

work page 2017

[8] [8]

Richard H. Browne. On the use of a pilot sample for sample size dete rmi- nation. Statistics in Medicine , 14(17):1933–1940, 1995. 22

work page 1933