Pith. sign in

REVIEW 2 major objections 6 minor 56 references

Efficient and flexible simulation-based sample size determination for clinical trials with multiple design parameters

T0 review · 2 major / 6 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that efficient optimisation algorithms based on surrogate models can solve complex clinical trial sample size problems with several design parameters, constraints, and conflicting objectives, returning a set of…

desk verdict A genuinely useful assembly of GP surrogates and EGO for multi-objective simulation-based trial design, with reproducible code and honest discussion; the EI criterion ignores re-classification of old points, but the stress-test overstates the damage. read the letter →

arxiv 1908.07769 v1 pith:7GBH6MFO submitted 2019-08-21 stat.ME

Duncan T. Wilson , Rebecca E. A. Walwyn , Richard Hooper , Julia Brown , Amanda J. Farrin This is my paper · ORCID
classification stat.ME MSC 62K0562P1065C05
keywords simulation-basedsamplesizedeterminationclinicaltrialsGaussianprocessregressionefficientglobaloptimisationmulti-objectiveParetooptimalityexpectedimprovementoperatingcharacteristics
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Simulation is a flexible way to estimate clinical trial power, but until now it has been practical only for problems with a single design parameter. This paper claims that by wrapping the Monte Carlo power calculation in a Gaussian-process surrogate model, sample size determination can be treated as a multi-objective optimisation over several design parameters and constraints at once. The method returns a set of Pareto-optimal designs, so a trial team can see trade-offs between, say, total patients and number of therapists or doctors instead of being handed one number. That matters because complex trials with clustering, co-primary endpoints, and small-sample pilot settings often have no closed-form power formula, and previous approaches either fixed one parameter or fell back on unrealistic assumptions.

What carries the argument

The machinery is a Gaussian process regression model used as a surrogate for the unknown constraint function, such as the type II error rate, with Monte Carlo noise modelled as normal with variance estimated from the simulation count. Around the surrogate sits an expected-improvement acquisition function: the gain in dominated hypervolume that evaluating a candidate design would bring, multiplied by the probability, read from the GP's predictive quantile, that the candidate will be feasible after evaluation. The quantile feasibility rule lets the user demand a chosen level of confidence that a constraint is satisfied despite Monte Carlo error.

What would settle it

Take a design problem with an analytic power formula, run the surrogate search using Monte Carlo estimates, and compare the returned Pareto set with the true Pareto curve; if the approximation set systematically excludes feasible designs or includes infeasible ones after precise re-estimation, the Gaussian process noise model or expected-improvement rule is the cause.

Watch

Extended reading notes

Core claim

The central claim is that a surrogate-model optimisation loop can solve complex simulation-based sample size determination problems. Given a program that simulates a proposed trial and returns a binary rejection indicator, the method estimates power or type I error at a few initial designs, fits a Gaussian process to those noisy Monte Carlo estimates, then repeatedly evaluates the design that maximises an expected improvement measure that balances gains in dominated hypervolume against the probability the design will satisfy all operating-characteristic constraints. The paper demonstrates the loop on three increasingly complex problems, covering multilevel clustering with therapists and doctors, two correlated co-primary endpoints, and a small pilot trial with five design parameters including the nominal type I error rate, and in each case obtains an approximation set of nondominated designs whose constraints are confirmed by higher-precision simulation.

Load-bearing premise

The load-bearing assumption is that each Monte Carlo estimate of a constraint function can be treated as the true value plus a normal error of known variance; with only one hundred simulations per design, that approximation can be unreliable, especially when power is close to 0 or 1, and a bad noise model can make the search certify designs that are actually underpowered.

Editorial extensions

If this is right

  • Trial designers can ask for the full trade-off curve between conflicting objectives, such as number of participants versus number of care providers, rather than minimising one quantity with the others fixed.
  • Complex designs that previously had to be simplified to make analytic power formulas tractable can be simulated as actually planned, including cases where the analysis model sometimes fails to converge.
  • If a target power or type I error rate is revised after seeing preliminary results, the search can continue from existing Monte Carlo estimates instead of restarting from scratch.
  • The same recipe applies to almost any problem for which power can be estimated by simulation, including novel trial designs for which no dedicated sample-size software exists.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same surrogate loop could be aimed at other expensive trial-design objectives, such as recruitment duration, expected cost, or expected sample size under adaptive stopping rules, since the simulation program and objective definitions are the only problem-specific pieces.
  • The algorithm's explicit noise model suggests a natural extension: allocate Monte Carlo effort adaptively, running more simulations where the surrogate is uncertain instead of using a fixed number per design.
  • The feasibility confidence parameter is effectively a dial for conservatism, and a systematic study of how it interacts with the number of simulations per design would give trial designers practical guidance on setting both.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 6 minor

Summary. The paper proposes a Gaussian-process-based sequential optimization framework for simulation-based sample size determination (SSD) in clinical trials. The SSD problem is formulated as a constrained multi-objective optimization over design parameters, with constraint functions estimated by Monte Carlo. The authors use a GP surrogate for each constraint, an EGO-style acquisition function based on hypervolume improvement multiplied by the predictive probability of feasibility, and illustrate the method on three PACE-inspired examples of increasing complexity. They also provide R code and simulated data.

Significance. If the central claim holds, the paper offers a practical and general tool for trial design problems where analytic power formulas are unavailable and where several design parameters and objectives must be balanced. The authors demonstrate concrete use cases, make the code available, and validate final designs with larger Monte Carlo samples. The main limitation is that the acquisition criterion does not account for the stochastic re-classification of previously evaluated points, so the theoretical link between the criterion and the stated objective (maximizing hypervolume) is incomplete; in addition, the efficiency claim is supported mainly by the examples rather than by comparison with alternative search strategies.

major comments (2)
  1. [§4.3, Eq. (15); §5.1] The expected improvement is defined as [H(A*) − H(A)] times the probability that the candidate is feasible, but the authors acknowledge in §5.1 that evaluating a new point can cause previously evaluated points to be reclassified as infeasible and removed from A. This means H(A*) in Eq. (15) is not the hypervolume that will actually be obtained after updating the GP; the expression overstates the expected improvement because it ignores the joint revision of the approximation set. Since the acquisition function is the core of the algorithm, this is a load-bearing gap. I recommend either (i) explicitly treating EI as a heuristic and providing an empirical justification, for example comparing with random or Sobol-based search with the same budget, or with a benchmark multi-objective optimizer on a small problem, or (ii) deriving an acquisition criterion that accounts for the joint posterior of all evaluated points. At minimum, the paper should report hypervolume trajectories and final sets from several independent runs.
  2. [§4.3, Eqs. (12)–(14)] The noise model treats the Monte Carlo estimate as Gaussian with variance ω² = m(1−m)/N, where m is the GP predictive mean and N = 100 in the examples. For binary outcomes and probabilities near the constraint boundary (0.1 or 0.2), the normal approximation and the plug-in variance can be inaccurate; this directly affects the feasibility quantile q(x), the predictive distribution of q+(x), and the EI value. The authors should provide diagnostics, such as empirical coverage of the GP predictive intervals against repeated MC estimates, or move to a more appropriate noise model, such as a logit transform or an explicit binomial likelihood. This is not a purely theoretical concern because Table 3 contains N = 100 estimates that exceed the constraint while the points are nevertheless included in the final set.
minor comments (6)
  1. [§2.2] The first sentence contains a typo: 'An in PACE' should read 'As in PACE'.
  2. [§5.2] The line 'ρW = ρT = ρD = ρD = 0.9' contains a duplicated subscript; it should be 'ρW = ρT = ρD = 0.9'.
  3. [§4.3, §5] The confidence level p used in Eq. (12) is not stated for any example, although it determines whether a candidate is considered feasible; please provide these values, since they affect the feasibility rule and thus the final sets.
  4. [§5.1] The comparison labeled 'MLPowSim' is actually a custom Sobol-sequence procedure inspired by MLPowSim; please describe it as such to avoid overclaiming.
  5. [Table 3] Several rows report N = 100 estimates of α exceeding 0.2 (e.g., 0.21, 0.27) while still appearing in the final approximation set; an explanation is needed, e.g., that feasibility is based on the GP posterior rather than the individual MC estimate.
  6. [§4.4] The stopping rule is described only as budget exhaustion or lack of improvement; the examples should state the realized budget in a consistent way, such as total number of MC evaluations, not just wall-clock time.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the GP surrogate is an approximation tool, and the reported designs are validated by independent, larger Monte Carlo samples.

full rationale

The derivation chain is not circular. The operating characteristics beta(x) and alpha(x) are defined by user-written simulation programs with fixed hypotheses and nuisance parameters (Sections 2, 3.1, 5); they are not defined by the GP model or by the optimisation algorithm. The GP is used only as a surrogate: Equations (8)-(10) are standard kriging fitted to MC estimates, and Equation (15) combines an existing hypervolume improvement criterion [22,24] with an existing feasibility-quantile rule from the cited literature. The candidate designs returned in Tables 1-3 are then re-evaluated with N=50^4 independent MC samples, so the reported type I/II error rates are not the same N=100 estimates that drove the search. The self-citations (SimSam [4], Wilson et al. [13], Walwyn and Roberts [29]) provide motivation, software context, and modelling structure; they are not invoked as uniqueness theorems or as the source of the predicted power values. The observation in Section 5.1 that H(A_i) is not strictly increasing because newly evaluated points can cause previously evaluated points to be reclassified as infeasible identifies a limitation of the EI criterion, but it is an optimisation-consistency caveat rather than a reduction of outputs to inputs. No fitted parameter is renamed as a prediction, and no claimed result is equivalent to its own assumption. Hence no significant circularity.

Assumptions & free parameters 4 free parameters · 6 assumptions · 0 invented entities

The central method rests on standard Gaussian-process machinery and on several domain assumptions about the trial simulation and the error structure. No new physical entities are introduced. The free parameters listed are user-chosen inputs for the illustrative examples or internal GP hyperparameters, not parameters fitted to an external target result.

free parameters (4)
  • GP kernel hyperparameters (sigma, lambda_1,...,lambda_D) = estimated per run via maximum marginal likelihood
    Fitted to the MC estimates using Eq. (10). These are internal to the surrogate model; the central claim does not depend on their specific values, but the quality of the GP predictions does.
  • Variance components sigma^2_T (therapist) and sigma^2_D (doctor) = 0.19 and 0.37
    Chosen by hand in Section 5.1 to give VPCs of 0.05 and 0.095 in the illustrative examples. They are not fitted to any external target; they are user-specified inputs.
  • Correlation rho_W = rho_T = rho_D = 0.9
    Assumed in Section 5.2 for co-primary endpoints; not fitted, chosen to reflect similar endpoints.
  • MC sample size N during optimization = 100
    Chosen in Section 5 for computational tractability. It directly affects the noise variance in Eq. (3) and the reliability of the GP predictive distributions.
assumptions (6)
  • standard math The MC estimator is unbiased and asymptotically normal with variance omega^2 = Var[f(z_i)]/N (Eqs. 2-3).
    Used to model MC estimates as noisy observations of the true constraint function in Section 4.2.
  • standard math Conditional on observed noisy estimates, the value of the constraint function at a new point has a normal predictive distribution (Rasmussen & Williams).
    Foundation of the GP surrogate used for the expected improvement computation.
  • domain assumption The constraint functions (type I and type II error rates) are smooth enough to be well-approximated by a stationary Gaussian process with squared exponential kernel.
    Needed for the GP regression to give useful predictions about unevaluated designs; the paper acknowledges checking model fit but provides no formal validation.
  • domain assumption The predictive distribution of the revised feasibility quantile q+(x*) follows N(m+, s^2_+) with m+ and s^2_+ as in Eqs. (13)-(14) (Picheny & Ginsbourger).
    This is the basis for the constraint-satisfaction probability in the EI criterion in Eq. (15).
  • domain assumption In Examples 1 and 2, the likelihood ratio test statistic follows a chi-squared distribution with 1 degree of freedom and controls type I error at the nominal level.
    Used to define power without simulating the null hypothesis; only Example 3 explicitly simulates type I error because of small samples.
  • ad hoc to paper For a new point x*, the improvement I = H(A*) - H(A) is deterministic given that x* is feasible; the stochastic re-classification of previously evaluated points is ignored.
    Eq. (15) multiplies this deterministic improvement by the probability that x* is feasible. Section 5.1 and Figure 5 show H(A_i) is not monotonic because adding a point can change the feasibility of existing points, so the EI criterion is an approximation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Efficient and flexible simulation-based sample size determination for clinical trials with multiple design parameters." pith.science (2026). https://pith.science/paper/7GBH6MFO

@misc{pith2026190807769,
  author       = {Pith},
  title        = {Pith review of: Efficient and flexible simulation-based sample size determination for clinical trials with multiple design parameters},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7GBH6MFO}},
  note         = {Machine review of arXiv:1908.07769}
}
read the original abstract

Simulation offers a simple and flexible way to estimate the power of a clinical trial when analytic formulae are not available. The computational burden of using simulation has, however, restricted its application to only the simplest of sample size determination problems, minimising a single parameter (the overall sample size) subject to power being above a target level. We describe a general framework for solving simulation-based sample size determination problems with several design parameters over which to optimise and several conflicting criteria to be minimised. The method is based on an established global optimisation algorithm widely used in the design and analysis of computer experiments, using a non-parametric regression model as an approximation of the true underlying power function. The method is flexible, can be used for almost any problem for which power can be estimated using simulation, and can be implemented using existing statistical software packages. We illustrate its application to three increasingly complicated sample size determination problems involving complex clustering structures, co-primary endpoints, and small sample considerations.

Figures

Figures reproduced from arXiv: 1908.07769 by the authors.

Figure 1
Figure 1. Multilevel structure of the SMC and APT arms of our examp [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Example Pareto front Xp and approximation set A for a cluster ran￾domised trial design problem. The dominated hypervolume of the approxima￾tion set with respect to a reference point (cross) is the shaded area. with a single constraint function, denoted g(x), which must be estimated us￾ing simulation. The more general case of several constraints will follow. The initial step is to select a number of potential solutio… view at source ↗
Figure 3
Figure 3. A Gaussian process model of a power function over a one- [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: The proposed method has led to solutions of higher quality w [PITH_FULL_IMAGE:figures/full_fig_p015_4.png]
Figure 4
Figure 4. Figure 4: Objective values of solutions in the initial set [PITH_FULL_IMAGE:figures/full_fig_p016_4.png]
Figure 5
Figure 5. Figure 5: Quality of the approximation set obtained as the algorithm p [PITH_FULL_IMAGE:figures/full_fig_p016_5.png]
Figure 6
Figure 6. Figure 6: Objective values of solutions in the initial set [PITH_FULL_IMAGE:figures/full_fig_p018_6.png]
Figure 7
Figure 7. Figure 7: Objective values of the approximation set obtained followin [PITH_FULL_IMAGE:figures/full_fig_p020_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

56 extracted references · 56 canonical work pages

  1. [1]

    Simulation methods to estimate design power: an overview for ap- plied research

    Benjamin F Arnold, Daniel R Hogan, John M Colford, and Alan E Hub- bard. Simulation methods to estimate design power: an overview for ap- plied research. BMC Med Res Methodol , 11(1), 6 2011

    work page 2011

  2. [2]

    Sample size and power calculations for medical studies by simulation when closed form expressions are not a vail- able

    Sabine Landau and Daniel Stahl. Sample size and power calculations for medical studies by simulation when closed form expressions are not a vail- able. Statistical Methods in Medical Research , 22(3):324–345, 2013

    work page 2013

  3. [3]

    Ziding Feng and James E. Grizzle. Correlated binomial variates: Pr op- erties of estimator of intraclass correlation and its effect on sample size calculation. Statistics in Medicine , 11(12):1607–1614, 1992

    work page 1992

  4. [4]

    Versatile sample-size calculation using simulation

    Richard Hooper. Versatile sample-size calculation using simulation. The STATA Journal, 13(1):21–38, 2013

    work page 2013

  5. [5]

    Schoenfeld and Michael Borenstein

    David A. Schoenfeld and Michael Borenstein. Calculating the powe r or sample size for the logistic and proportional hazards models. Journal of Statistical Computation and Simulation , 75(10):771–785, oct 2005

    work page 2005

  6. [6]

    Grieve and Shah-Jalal Sarker

    Andrew P. Grieve and Shah-Jalal Sarker. Simulation-based samp le-sizing and power calculations in logistic regression with partial prior informa tion. Pharmaceutical Statistics, 15(6):507–516, sep 2016

    work page 2016

  7. [7]

    Sutton, Nicola J

    Alexander J. Sutton, Nicola J. Cooper, David R. Jones, Paul C. L ambert, John R. Thompson, and Keith R. Abrams. Evidence-based sample siz e calculations based upon updated meta-analysis. Statistics in Medicine , 26(12):2479–2500, 2007

    work page 2007

  8. [8]

    Simulation-based power calculations for mixed effects mode ling: ipdpower in stata

    Evangelos Kontopantelis, David A Springate, Rosa Parisi, and Dav id Reeves. Simulation-based power calculations for mixed effects mode ling: ipdpower in stata. Journal of Statistical Software , 74(12), 2016

    work page 2016

Show all 56 references
  1. [9]

    The design of multicentre trials

    Valerii Fedorov and Byron Jones. The design of multicentre trials . Statis- tical Methods in Medical Research , 14(3):205–248, 2005. PMID: 15969302. 23

    2005

  2. [10]

    Sample size calculation for a stepped wed ge trial

    Gianluca Baio, Andrew Copas, Gareth Ambler, James Hargreave s, Emma Beard, and Rumana Z Omar. Sample size calculation for a stepped wed ge trial. Trials, 16(1), aug 2015

    2015

  3. [11]

    Sample size calculation for stepped wedge and other longitudinal clus ter randomised trials

    Richard Hooper, Steven Teerenstra, Esther de Hoop, and Sa ndra Eldridge. Sample size calculation for stepped wedge and other longitudinal clus ter randomised trials. Statistics in Medicine , 35(26):4718–4728, jun 2016

    2016

  4. [12]

    Reich, Jessica A

    Nicholas G. Reich, Jessica A. Myers, Daniel Obeng, Aaron M. Milst one, and Trish M. Perl. Empirical power and sample size calculations for clus ter- randomized and cluster-randomized crossover studies. PLOS ONE , 7(4):1– 7, 04 2012

    2012

  5. [13]

    D. T. Wilson, R. E. Walwyn, J. Brown, A. J. Farrin, and S. R. Brow n. Statistical challenges in assessing potential efficacy of complex inte rven- tions in pilot or feasibility studies. Statistical Methods in Medical Research, 25(3):997–1009, jun 2015

    2015

  6. [14]

    K. Deb, A. Pratap, S. Agarwal, and T. Meyarivan. A fast and elit ist multi- objective genetic algorithm: NSGA-II. IEEE Transactions on Evolutionary Computation, 6(2):182–197, apr 2002

    2002

  7. [15]

    mco: Multiple Criteria Optimization Algorithms and Re- lated Functions, 2014

    Olaf Mersmann. mco: Multiple Criteria Optimization Algorithms and Re- lated Functions, 2014. R package version 1.0-15.1

    2014

  8. [16]

    Welch, Toby J

    Jerome Sacks, William J. Welch, Toby J. Mitchell, and Henry P. Wynn . Design and analysis of computer experiments. Statistical Science, 4(4):409– 423, 1989

    1989

  9. [17]

    Santner, Brian J

    Thomas J. Santner, Brian J. Williams, and William I. Notz. The Design and Analysis of Computer Experiments . Springer-Verlag New York, Inc., 2003

    2003

  10. [18]

    D.G. Krige. A statistical approach to some basic mine valuation pr oblems on the witwatersrand. Journal of the Southern African Institute of Mining and Metallurgy , 52(6):119–139, 1951

    1951

  11. [19]

    Carl Edward Rasmussen and Christopher K. I. Williams. Gaussian Pro- cesses for Machine Learning . MIT Press, 2006

    2006

  12. [20]

    DiceKriging , DiceOptim: Two R packages for the analysis of computer experiment s by kriging-based metamodeling and optimization

    Olivier Roustant, David Ginsbourger, and Yves Deville. DiceKriging , DiceOptim: Two R packages for the analysis of computer experiment s by kriging-based metamodeling and optimization. Journal of Statistical Software, 51(1):1–55, 2012

    2012

  13. [21]

    Donald R. Jones. A taxonomy of global optimization methods bas ed on response surfaces. Journal of Global Optimization , 21(4):345–383, 2001

    2001

  14. [22]

    Sasena, Panos Papalambros, and Pierre Goovaerts

    Michael J. Sasena, Panos Papalambros, and Pierre Goovaerts . Exploration of metamodeling sampling criteria for constrained global optimization . En- gineering Optimization , 34(3):263–278, jan 2002. 24

    2002

  15. [23]

    Hypervolume-based expected improvement: Monotonicity proper ties and exact computation

    Michael TM Emmerich, Andr´ e H Deutz, and Jan Willem Klinkenberg. Hypervolume-based expected improvement: Monotonicity proper ties and exact computation. In Evolutionary Computation (CEC), 2011 IEEE Congress on , pages 2147–2154. IEEE, 2011

    2011

  16. [24]

    Noisy kriging-based optim ization methods: A unified implementation within the DiceOptim package

    Victor Picheny and David Ginsbourger. Noisy kriging-based optim ization methods: A unified implementation within the DiceOptim package. Com- putational Statistics & Data Analysis , 71:1035–1053, mar 2014

    2014

  17. [25]

    A Guide to Sample Size Calculations for Random Effect Models vi a Simula- tion and the MLPowSim Software Package , 2009

    William J Browne, Mousa Golalizadeh Lahi, and Richard MA Parker. A Guide to Sample Size Calculations for Random Effect Models vi a Simula- tion and the MLPowSim Software Package , 2009

    2009

  18. [26]

    Peter White, Michael Sharpe, Trudie Chalder, Julia DeCesare, R ebecca Walwyn, and the PACE trial group. Protocol for the pace trial: A ra n- domised controlled trial of adaptive pacing, cognitive behaviour the rapy, and graded exercise as supplements to standardised specialist...

    2007

  19. [27]

    Comparison of adaptive pacing therapy, cognitive behav iour therapy, graded exercise therapy, and specialist medical care fo r chronic fa- tigue syndrome (pace): a randomised trial

    PD White, KA Goldsmith, AL Johnson, L Potts, R Walwyn, JC DeCe- sare, HL Baber, M Burgess, L V Clark, DL Cox, J Bavinton, BJ Angus , G Murphy, M Murphy, H O’Dowd, D Wilks, P McCrone, T Chalder, and M Sharpe. Comparison of adaptive pacing therapy, cognitive behav iour therapy, ...

    2011

  20. [28]

    Development of a fatigue scale

    T Chalder, G Berelowitz, S Hirsch, T Pawlikowska, P Wallace, and S W es- sely. Development of a fatigue scale. Journal of Psychometric Research , 37(2):147–153, 1993

    1993

  21. [29]

    Rebecca E. A. Walwyn and Chris Roberts. Therapist variation wit hin randomised trials of psychotherapy: implications for precision, inte rnal and external validity. Statistical Methods in Medical Research , 19(3):291–315, 2010

    2010

  22. [30]

    Chris Roberts, Evridiki Batistatou, and Stephen A. Roberts. Design and analysis of trials with a partially nested design and a binary outcome me a- sure. Statistics in Medicine , 35(10):1616–1636, 2015

    2015

  23. [31]

    McHorney, John E

    Colleen A. McHorney, John E. Ware, and Anastasia E. Raczek. T he mos 36- item short-form health survey (sf-36): Ii. psychometric and clinic al tests of validity in measuring physical and mental health constructs. Medical Care, 31(3):247–263, 1993

    1993

  24. [32]

    Power and sample size when mult iple endpoints are considered

    Stephen Senn and Frank Bretz. Power and sample size when mult iple endpoints are considered. Pharmaceut. Statist. , 6(3):161–170, 2007. 25

    2007

  25. [33]

    Jones, Matthias Schonlau, and William J

    Donald R. Jones, Matthias Schonlau, and William J. Welch. Efficient g lobal optimization of expensive black-box functions. Journal of Global Optimiza- tion, 13(4):455–492, 1998

    1998

  26. [34]

    pso: Particle Swarm Optimization , 2012

    Claus Bendtsen. pso: Particle Swarm Optimization , 2012. R package version 1.0.3

    2012

  27. [35]

    Picheny, D

    V. Picheny, D. Ginsbourger, and Y. Richet. Noisy expected impr ovement and on-line computation time allocation for the optimization of simula- tors with tunable fidelity. In 2nd International Conference on Engineering Optimization, 2010

    2010

  28. [36]

    randtoolbox: Generating and Testing Random Numbers, 2015

    Christophe Dutang and Petr Savicky. randtoolbox: Generating and Testing Random Numbers, 2015. R package version 1.17

    2015

  29. [37]

    Kr igInv: An efficient and user-friendly implementation of batch-sequential in version strategies based on kriging

    Cl´ ement Chevalier, Victor Picheny, and David Ginsbourger. Kr igInv: An efficient and user-friendly implementation of batch-sequential in version strategies based on kriging. Computational Statistics & Data Analysis , 71:1021–1034, mar 2014

    2014

  30. [38]

    Fitting linear mixed-effects models using lme4

    Douglas Bates, Martin M¨ achler, Ben Bolker, and Steve Walker. Fitting linear mixed-effects models using lme4. Journal of Statistical Software , 67(1):1–48, 2015

    2015

  31. [39]

    Multilevel Statistical Models

    Harvey Goldstein. Multilevel Statistical Models . Arnold, 3rd edition, 2003

    2003

  32. [40]

    How to design efficient cluster randomised trials

    K Hemming, S Eldridge, G Forbes, C Weijer, and M Taljaard. How to design efficient cluster randomised trials. BMJ, 358, 2017

    2017

  33. [41]

    Design and Analysis of Cluster Randomization Trials in Health Research

    Allan Donner and Neil Klar. Design and Analysis of Cluster Randomization Trials in Health Research . London Arnold Publishers, 2000

    2000

  34. [42]

    Sample size calculations for cluster randomised controlled tria ls with a fixed number of clusters

    Karla Hemming, Alan Girling, Alice Sitch, Jennifer Marsh, and Richar d Lil- ford. Sample size calculations for cluster randomised controlled tria ls with a fixed number of clusters. BMC Medical Research Methodology, 11(1):102, 2011

    2011

  35. [43]

    How big should the pilot study for my cluste r randomised trial be? Statistical Methods in Medical Research , 2015

    Sandra M Eldridge, Ceire E Costelloe, Brennan C Kahan, Gillian A Lan - caster, and Sally M Kerry. How big should the pilot study for my cluste r randomised trial be? Statistical Methods in Medical Research , 2015

    2015

  36. [44]

    Multilevel Analysis: Techniques and Applications

    Joop Hox. Multilevel Analysis: Techniques and Applications . Lawrence Erlbaum Associates, Inc., 2002

    2002

  37. [45]

    Tom A. B. Snijders and Roel J. Bosker. Standard errors and s ample sizes for two-level research. Journal of Educational and Behavioral Statistics , 18(3):237–259, 1993

    1993

  38. [46]

    Raudenbush and Xiaofeng Liu

    Stephen W. Raudenbush and Xiaofeng Liu. Statistical power an d optimal design for multisite randomized trials. Psychological Methods, 5(2):199–213, 2000. 26

    2000

  39. [47]

    van Breukelen and Math J.J.M

    Gerard J.P. van Breukelen and Math J.J.M. Candel. Calculating sam ple sizes for cluster randomized trials: We can keep it simple and efficient! Journal of Clinical Epidemiology , 65(11):1212 – 1218, 2012

    2012

  40. [48]

    Teerenstra, M

    S. Teerenstra, M. Moerbeek, T. van Achterberg, B. J. Pelze r, and G. F. Borm. Sample size calculations for 3-level cluster randomized trials. Clinical Trials, 5(5):486–495, sep 2008

    2008

  41. [49]

    Lawrence Joseph and David B. Wolfson. Interval-based versu s decision the- oretic criteria for the choice of sample size. Journal of the Royal Statistical Society: Series D (The Statistician) , 46(2):145–149, 1997

    1997

  42. [50]

    Smith and Andrea Marshall

    Mike K. Smith and Andrea Marshall. Importance of protocols for simu- lation studies in clinical drug development. Statistical Methods in Medical Research, 2010

    2010

  43. [51]

    Altman, Patrick Royston, and Roge r L

    Andrea Burton, Douglas G. Altman, Patrick Royston, and Roge r L. Holder. The design of simulation studies in medical statistics. Statistics in Medicine, 25(24):4279–4292, 2006

    2006

  44. [52]

    Kennedy and Anthony O’Hagan

    Marc C. Kennedy and Anthony O’Hagan. Bayesian calibration of c om- puter models. Journal of the Royal Statistical Society: Series B (Statist ical Methodology), 63(3):425–464, 2001

    2001

  45. [53]

    Stevens, and Michael J

    Anthony O’Hagan, John W. Stevens, and Michael J. Campbell. As surance in clinical trial design. Pharmaceutical Statistics, 4(3):187–201, 2005

    2005

  46. [54]

    Jack Lee, and Susan Alber

    Jing Cao, J. Jack Lee, and Susan Alber. Comparison of bayesian sample size criteria: ACC, ALC, and WOC. Journal of Statistical Planning and Inference, 139(12):4111 – 4122, 2009

    2009

  47. [55]

    Oakley, Alan Brennan, Paul Tappenden, and Jim Chilco tt

    Jeremy E. Oakley, Alan Brennan, Paul Tappenden, and Jim Chilco tt. Sim- ulation sample sizes for monte carlo partial evpi calculations. Journal of Health Economics, 29(3):468 – 477, 2010

    2010

  48. [56]

    James M. S. Wason and Thomas Jaki. Optimal design of multi-arm m ulti- stage trials. Statistics in Medicine , 31(30):4269–4279, jul 2012. 27

    2012

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.