Pith. sign in

REVIEW 2 major objections 5 minor 1 cited by

An Efficient Approach to Design Bayesian Platform Trials

T0 review · 2 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Simulating a Bayesian platform trial at just two interim sample sizes can recover its power and family-wise error curves, the paper claims.

desk verdict Solid extension of the two-anchor SSD trick to realistic platform trial constraints, honestly validated on SSTARLET; the asymptotic theorem is motivation rather than proof of finite-sample accuracy, but the empirical case is strong enough to warrant serious review. read the letter →

arxiv 2507.12647 v1 pith:MS6AMA6Q submitted 2025-07-16 stat.ME math.STstat.TH

Luke Hagar , Lara Maleyeff , Shirin Golchi , Dick Menzies This is my paper · ORCID
classification stat.MEmath.STstat.TH MSC 62F1562L0562P10
keywords Bayesianplatformtrialssamplesizedeterminationposteriorprobabilitiesfamily-wiseerrorrateinterimanalysisresponse-adaptiverandomizationMAPpriorstuberculosispreventivetherapy
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Bayesian platform trials, which test several treatments against a shared control while adding and dropping arms, must be checked by simulation before launch, and regulators expect those operating characteristics to be reported. Rerunning the simulation at every candidate sample size is slow when interim decisions are complex. The paper claims that the joint sampling distribution of the posterior probabilities driving those decisions has quantiles whose log-odds are nearly linear in the interim sample size, and proves this for a large-sample proxy distribution. That justifies simulating at just two anchor sample sizes and interpolating in between. Applied to the SSTARLET tuberculosis trial, the method selects an interim sample size of 674 and a final size of 1,685, matching full-grid simulations while cutting design computation from hours to minutes.

What carries the argument

The load-bearing mechanism is two-anchor interpolation of the logits of posterior-probability quantiles: simulate synthetic trials at two interim sample sizes $n_a$ and $n_b$, compute all posterior probabilities, sort their logits, pair the order statistics, and linearly interpolate or extrapolate to any requested $n$. Theorem 1 justifies this by showing that in the proxy distribution formed by substituting asymptotically normal maximum-likelihood estimates into a Bernstein-von Mises posterior, every conditional quantile's logit has a constant limiting slope in $n$, equal to $(0.5 - I\{\delta^*_i \ge \delta_{m,i}\})(\delta_{m,i} - \delta^*_i)^2 / \Sigma_{i,i}$. Because the slope is independent of the quantile level, interpolation of the marginal quantiles also tracks the conditional structure when the replicate order statistics are paired by rank. The algorithm enumerates all possible active arm sets, evaluates power and family-wise error from the interpolated joint distribution, and finds the smallest $n$ that meets the power constraint.

What would settle it

Pick an intermediate sample size between the two anchors (for SSTARLET, say $n_1 = 800$ between 600 and 1,000), run the full Monte Carlo simulation of the joint sampling distribution, and compare the simulated power and family-wise error against the interpolated values at that point. Repeating this for a scenario in which an arm is barely acceptable or the null lies at the margin, where discrepancies were most visible in the paper, would settle how much curvature the linearity assumption can tolerate.

Watch

Extended reading notes

Core claim

The paper establishes a theorem: under the regularity conditions for maximum-likelihood normality and the Bernstein-von Mises theorem, the limiting derivative of the logit of each conditional quantile of the posterior-probability sampling distribution with respect to the interim sample size is a constant that does not depend on the sample size or the quantile level. Those logits can therefore be treated as linear functions of the sample size, so the true joint sampling distribution can be estimated by Monte Carlo at two anchor sizes and linearly interpolated. The implementation pairs the order statistics of the logits at the two anchors, which preserves the dependence across endpoints, trial stages, and the possible sets of active arms even when the active set changes with the sample size. In the SSTARLET design this reproduces the simulated power and family-wise error curves across all four outcome scenarios and recommends $n_1 = 674$, $n_2 = 1{,}685$.

Load-bearing premise

The method rests on assuming that the log-odds (logit) of each posterior-probability quantile is nearly a straight line in the sample size, so interpolation between just two simulated sizes tracks the true curves; if those curves bend, the resulting power and error-rate estimates drift.

Editorial extensions

If this is right

  • A platform trial with the same structure can be re-evaluated for different thresholds or sample sizes without new simulations: the two anchor distributions are reused and only the interpolation target changes.
  • Power and family-wise error control are approximately preserved across the whole examined sample-size range, so the recommended sample size and thresholds come with operating-characteristic curves rather than a single point estimate.
  • The framework extends to other platform designs as long as each arm's stage-wise sample sizes are linear functions of the interim sample size, a condition the paper states as a prerequisite.
  • When the null model sits exactly at the non-inferiority margin, the limiting slopes vanish, so thresholds chosen at one anchor approximately maintain the family-wise error rate at all sample sizes.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An implication left implicit is that the two-anchor scheme should extend recursively to trials with more than two interim analyses, with each analysis pair interpolated in turn; that extension would need its own linearity check at every window.
  • The rank-pairing mechanism suggests the method could transfer beyond binomial outcomes to endpoints whose posterior summaries are approximately normal, such as time-to-event or continuous endpoints, provided a suitable variance proxy exists.
  • The paper selects several decision thresholds heuristically; a direct follow-up would optimize all thresholds jointly against the interpolated operating-characteristic surfaces instead of tuning only the final non-inferiority threshold.
  • A low-cost robustness check, not pursued in the paper, is to add a third anchor sample size whenever prior information or boundary-null scenarios are expected to bend the quantile curves, turning the linear interpolation into a diagnostic.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper proposes an efficient sample size determination (SSD) method for Bayesian platform trials. The method simulates the joint sampling distribution of posterior probabilities at only two interim sample sizes, then linearly interpolates the logits of posterior-probability order statistics to obtain operating characteristics at other sample sizes. The authors prove an asymptotic result (Theorem 1) on the limiting derivative of conditional logit quantiles, present Algorithm 1 as the SSD procedure, and demonstrate the approach on the SSTARLET tuberculosis trial, a realistic platform design with arm addition, a fixed outcome delay, variable allocation, and robust MAP priors.

Significance. If the proposed method generalizes as claimed, it would reduce the computational burden of calibrating Bayesian platform trials by replacing full-simulation grids with simulations at two sample sizes. The paper includes a substantial real-world case study, reports strong agreement between the SSD estimates and direct simulations in Table 3, and provides publicly available code, which are notable strengths. However, the theoretical support is asymptotic rather than finite-sample, and the empirical validation is limited to a single platform design, so the breadth of the central claim exceeds what is currently demonstrated.

major comments (2)
  1. [§4.1, Theorem 1 and Algorithm 1] Theorem 1 proves that the limiting derivative of the conditional logit quantiles of the proxy sampling distribution is constant as n tends to infinity. This does not establish that the logit quantiles are linear in n over a finite working range, nor does it provide an error bound for the two-point interpolation in Algorithm 1 (Lines 8-10). The manuscript itself reports discrepancies at smaller sample sizes in Figure 3B and 3D, and the simulation study considers only the SSTARLET design. The abstract's statement that the method models the joint sampling distribution 'using simulations conducted at only two sample sizes' for Bayesian platform trials is therefore stronger than what the theorem and the evidence support. I recommend either tempering the generality claims and explicitly stating that finite-sample accuracy is an empirical assumption to be verified for each design, or adding corroborating simulation studies for platform trials with different structures (for example, no outcome lag, a different number of arms, or different endpoint types) together with a discussion of conditions under which the approximation is expected to fail.
  2. [Algorithm 1, Lines 6-10] The rank-aligned pairing of the d-th order statistics across components imposes a particular copula between components at the interpolated sample size. Theorem 1 concerns conditional quantiles of the proxy distribution, not the invariance of the joint dependence structure in n, so this pairing is not theoretically justified by the theorem. The issue is load-bearing for the FWER in Equation (4), which depends on the joint distribution over arms. The authors should either provide a sensitivity analysis of this copula assumption or clearly acknowledge that the joint dependence is approximated by the rank-aligned construction.
minor comments (5)
  1. [§3.3] The sentence 'We set w = 0.5' should be 'We set w1 = 0.5' to match the notation used in the robust MAP prior specification.
  2. [Figures 1-3] The simulated curves would be easier to assess if they included pointwise Monte Carlo confidence bands or standard error shading; as presented, it is hard to separate approximation error from simulation noise.
  3. [Table 3] The caption should state explicitly that the 'Estimated' rows use Algorithm 1 with R = 10,000 replicates and the same decision thresholds as the 'Simulated' rows, so that the comparison is unambiguous.
  4. [§4.1] The phrase 'prove theoretical results that allow us to model' in the abstract and the word 'guarantees' in the text are stronger than the asymptotic result supports; replacing them with 'motivates' or 'provides asymptotic support for' would better reflect the finite-sample role of the empirical validation.
  5. [Appendix B] The proof of Theorem 1 is deferred to the online supplement, which is not included in the arXiv submission; please ensure the supplement is available to reviewers and consider including a proof sketch in the main text.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: Algorithm 1's two-anchor interpolation is validated against independent simulations; Theorem 1 is an asymptotic motivation, not an input-output identity.

full rationale

The derivation chain is not circular. The operating-characteristic curves produced by Algorithm 1 are anchored at Monte Carlo estimates of the true sampling distribution of posterior probabilities at two sample sizes (na = 600 and nb = 1000), and the interpolated logit quantiles are checked against an independent full-grid simulation in Section 5 (black dotted curves in Figures 1–3). The recommended design is then verified at the selected sample size in Table 3 by direct simulation. Theorem 1 provides an asymptotic slope result for conditional quantiles of a large-sample proxy distribution; it motivates the linear interpolation but is not used to define the target power or sample size, so the interpolation is a modeling approximation rather than a quantity that is equivalent to its inputs by construction. The discrepancies noted in Figure 3B and 3D are validation errors at small sample sizes, not circularity. The paper cites prior work by the same authors for the general framework and related asymptotic results, but these citations are not used as a substitute for the present derivation: Theorem 1 is proved in Appendix B from standard maximum-likelihood and Bernstein-von Mises conditions, and the method is empirically benchmarked against conventional simulation. Threshold selection in Algorithm 1 (Line 2) calibrates decision cutoffs at the anchor sample size and relies on a separate asymptotic flatness argument, not on renaming the fitted threshold as a prediction. No fitted parameter is relabeled as a predicted quantity, and no definition ties the output sample size to the inputs of the algorithm. The central claim therefore has independent content and is not forced by self-citation or by construction.

Assumptions & free parameters 5 free parameters · 6 assumptions · 0 invented entities

The central claim depends on standard asymptotic statistics (MLE normality, BvM), on the structural condition that all arm sample sizes are linear in n, and on design choices specific to SSTARLET (degenerate design prior, exchangeable historical data, individual-level analysis). None of these are invented entities; they are stated assumptions the reader can check.

free parameters (5)
  • Anchor interim sample sizes na and nb = 600 and 1000
    Chosen by the authors in Section 5 as the two sample sizes at which sampling distributions are simulated. The interpolation accuracy depends on these choices.
  • Second-stage ratio c2 = 2.5
    Sets final sample size n2 = c2 n1. Design choice in the SSTARLET example; the method requires it to be constant.
  • Decision thresholds gamma and kappa = gamma=(0.2,0.5,0.5), kappa=(0.975,0.99,0.99)
    Selected in Section 5: kappa1 is tuned to control FWER at 0.05; the other thresholds are set heuristically to balance interim stopping behavior. They determine the operating characteristics the method must estimate.
  • MAP prior overall weight w1 = 0.5
    Set to the recommended value in Section 3.3; weights the informative historical component against the uniform robust component.
  • Allocation ratios = 1:2:2 initial, 50% to new arm, equal split among remaining after interim
    From the SSTARLET protocol (Section 2); these linear functions of n define the sample-size structure the method relies on.
assumptions (6)
  • domain assumption Regularity conditions for asymptotic normality of the MLE (van der Vaart, 1998, Thm 5.39) hold for the binomial likelihoods and the design prior.
    Used in Section 4.1 to justify the normal approximation for the MLE in (5).
  • domain assumption Bernstein-von Mises conditions hold: the analysis prior is continuous with positive density at the true parameter value.
    Used in Section 4.1 to justify the normal posterior approximation in (6). The robust MAP prior is a mixture of Beta densities that is positive on (0,1).
  • domain assumption Sample sizes for all treatments in all trial stages are linear functions of the interim sample size n (not necessarily proportional).
    Required for Theorem 1; verified for SSTARLET in Appendix A but must be checked when applying the method to other platform trials.
  • domain assumption The design prior Psi is degenerate (point mass) at specified reference parameter values.
    Stated in Section 3.4; sacrifices design-prior uncertainty to align with how SSTARLET is designed. Non-degenerate Psi would require additional guidance (Hagar et al., 2025).
  • domain assumption The historical studies used for the MAP prior are exchangeable with the current trial and use consistent endpoint definitions.
    Justifies borrowing via robust MAP priors in Section 3.3. The paper supports this with similarities in regimens, eligibility, and adjudication.
  • domain assumption Individual-level analysis of household-clustered data is acceptable because 87% of clusters are single-member households.
    Assumed in Section 3.3 to justify ignoring within-cluster correlation for the main analysis.

how reviews work

0 comments
Cite this review

Pith. "Pith review of An Efficient Approach to Design Bayesian Platform Trials." pith.science (2026). https://pith.science/paper/MS6AMA6Q

@misc{pith2026250712647,
  author       = {Pith},
  title        = {Pith review of: An Efficient Approach to Design Bayesian Platform Trials},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MS6AMA6Q}},
  note         = {Machine review of arXiv:2507.12647}
}
read the original abstract

Platform trials evaluate multiple experimental treatments against a common control group (and/or against each other), which often reduces the trial duration and sample size. Bayesian platform designs offer several practical advantages, including the flexible addition or removal of experimental arms using posterior probabilities and the incorporation of prior/external information. Regulatory agencies require that the operating characteristics of Bayesian designs are assessed by estimating the sampling distribution of posterior probabilities via Monte Carlo simulation. It is computationally intensive to repeat this simulation process for all design configurations considered, particularly for platform trials with complex interim decision procedures. In this paper, we propose an efficient method to assess operating characteristics and determine sample sizes as well as other design parameters for Bayesian platform trials. We prove theoretical results that allow us to model the joint sampling distribution of posterior probabilities across multiple endpoints and trial stages using simulations conducted at only two sample sizes. This work is motivated by design complexities in the SSTARLET trial, an ongoing Bayesian adaptive platform trial for tuberculosis preventive therapies (ClinicalTrials.gov ID: NCT06498414). Our proposed design method is not only computationally efficient but also capable of accommodating intricate, real-world trial constraints like those encountered in SSTARLET.

Figures

Figures reproduced from arXiv: 2507.12647 by the authors.

Figure 1
Figure 1. Estimated (gray solid) vs simulated (black dotted) non-inferiority probabilities [PITH_FULL_IMAGE:figures/full_fig_p014_1.png] view at source ↗
Figure 2
Figure 2. Estimated (gray solid) vs simulated (black dotted) non-inferiority probabilities [PITH_FULL_IMAGE:figures/full_fig_p016_2.png] view at source ↗
Figure 3
Figure 3. Estimated (gray solid) and simulated (black dotted) non-inferiority probabilities [PITH_FULL_IMAGE:figures/full_fig_p017_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Frequentist-calibrated Bayesian group sequential design with dynamic borrowing

    stat.ME 2026-07 conditional novelty 5.0 of 10

    A Bayesian group sequential design provides, at each interim, an evidential threshold exactly matching the frequentist UMP test and a second threshold for dynamic borrowing of historical data.

Reference graph

Works this paper leans on

33 extracted references · 26 canonical work pages · cited by 1 Pith paper

  1. [1]

    and Kieser, M

    Bauer, P. and Kieser, M. (1999). Combining different phases in the development of medical treatments within a single trial. Statistics in medicine , 18(14):1833--1848

    work page 1999

  2. [2]

    M., Carlin, B

    Berry, S. M., Carlin, B. P., Lee, J. J., and Muller, P. (2010). Bayesian adaptive methods for clinical trials . CRC press

    2010

  3. [3]

    Brutti, P., De Santis, F., and Gubbiotti, S. (2014). Bayesian-frequentist sample size determination: a game of two priors. Metron , 72(2):133--151

    work page 2014

  4. [4]

    Burnett, T., K \"o nig, F., and Jaki, T. (2024). Adding experimental treatment arms to multi-arm multi-stage platform trials in progress. Statistics in Medicine , 43(18):3447--3462

    work page 2024

  5. [5]

    De Santis, F. (2007). Using historical data for B ayesian sample size determination. Journal of the Royal Statistical Society: Series A (Statistics in Society) , 170(1):95--113

    work page 2007

  6. [6]

    Dunnett, C. W. (1955). A multiple comparison procedure for comparing several treatments with a control. Journal of the American Statistical Association , 50(272):1096--1121

    work page 1955

  7. [7]

    Dye, C., Glaziou, P., Floyd, K., and Raviglione, M. (2013). Prospects for tuberculosis elimination. Annual Review of Public Health , 34:271--286

    work page 2013

  8. [8]

    Golchi, S. (2022). Estimating design operating characteristics in B ayesian adaptive clinical trials. Canadian Journal of Statistics , 50(2):417--436

    2022

Show all 33 references
  1. [9]

    and Willard, J

    Golchi, S. and Willard, J. J. (2024). Estimating the sampling distribution of posterior decision summaries in B ayesian clinical trials. Biometrical Journal , 66(8):e70002

    2024

  2. [10]

    Greenstreet, P., Jaki, T., Bedding, A., Harbron, C., and Mozgunov, P. (2024). A multi-arm multi-stage platform design that allows preplanned addition of arms while still controlling the family-wise error. Statistics in Medicine , 43(19):3613--3632

    2024

  3. [11]

    and Golchi, S

    Hagar, L. and Golchi, S. (2025). Design of bayesian clinical trials with clustered data and multiple endpoints. arXiv preprint arXiv:2501.13218

    2025

  4. [12]

    Hagar, L., Golchi, S., and Klein, M. B. (2025). Sequential design with posterior and posterior predictive probabilities. arXiv preprint arXiv:2504.00856

    2025 arXiv

  5. [13]

    and Stevens, N

    Hagar, L. and Stevens, N. T. (2024). Fast power curve approximation for posterior analyses . Bayesian Analysis , pages 1 -- 26 doi.org/10.1214/24--BA1469

    2024 doi

  6. [14]

    and Stevens, N

    Hagar, L. and Stevens, N. T. (2025). An economical approach to design posterior analyses. Journal of the American Statistical Association . Published online April 16, 2025

    2025

  7. [15]

    and Turnbull, B

    Jennison, C. and Turnbull, B. W. (2000). Group Sequential Methods with Applications to Clinical Trials . CRC Press

    2000

  8. [16]

    Magirr, D., Jaki, T., and Whitehead, J. (2012). A generalized dunnett test for multi-arm multi-stage clinical studies with treatment selection. Biometrika , 99(2):494--501

    2012

  9. [17]

    Menzies, D., Adjobimey, M., Ruslami, R., and et al. (2018). Four months of rifampin or nine months of isoniazid for latent tuberculosis in adults. New England Journal of Medicine , 379(5):440--453

    2018

  10. [18]

    Menzies, D., Dion, M.-J., Rabinovitch, B., Mannix, S., Brassard, P., and Schwartzman, K. (2004). Treatment completion and costs of a randomized trial of rifampin for 4 months versus isoniazid for 9 months. American Journal of Respiratory and Critical Care Medicine , 170(4):445--449

    2004

  11. [19]

    Menzies, D., Long, R., Trajman, A., Dion, M.-J., Yang, J., Al Jahdali, H., Memish, Z., Khan, K., Gardam, M., Hoeppner, V., Benedetti, A., and Schwartzman, K. (2008). Adverse events with 4 months of rifampin therapy or 9 months of isoniazid therapy for latent tuberculosis infec...

    2008

  12. [20]

    Neuenschwander, B., Capkun‐Niggli, G., Branson, M., and Spiegelhalter, D. J. (2010). Summarizing historical information on controls in clinical trials. Clinical Trials , 7(1):5--18

    2010

  13. [21]

    K., Barthel, F

    Parmar, M. K., Barthel, F. M.-S., Sydes, M., Langley, R., Kaplan, R., Eisenhauer, E., Brady, M., James, N., Bookman, M. A., Swart, A.-M., et al. (2008). Speeding up the evaluation of new agents in cancer. Journal of the National Cancer Institute , 100(17):1204--1214

    2008

  14. [22]

    K., and Qian, W

    Royston, P., Parmar, M. K., and Qian, W. (2003). Novel designs for multi-arm clinical trials with survival outcomes with an application in ovarian cancer. Statistics in medicine , 22(14):2239--2256

    2003

  15. [23]

    J., Fisher, D., Flores, E., Fox, G

    Ruslami, R., Fregonese, F., Apriani, L., Barss, L., Bedingfield, N., Chiang, V., Cook, V. J., Fisher, D., Flores, E., Fox, G. J., Johnston, J., Lim, R. K., Long, R., Paulsen, C., Nguyen, T. A., Nhung, N. V., Gibson, D., Valiquette, C., Benedetti, A., and Menzies, D. (2024). Hi...

    2024

  16. [24]

    Schmidli, H., Gsteiger, S., Roychoudhury, S., O’Hagan, A., Spiegelhalter, D., and Neuenschwander, B. (2014). Robust meta‐analytic‐predictive priors in clinical trials with historical control information. Biometrics , 70(4):1023--1032

    2014

  17. [25]

    J., Abrams, K

    Spiegelhalter, D. J., Abrams, K. R., and Myles, J. P. (2004). Bayesian approaches to clinical trials and health-care evaluation . John Wiley & Sons

    2004

  18. [26]

    and Todd, S

    Stallard, N. and Todd, S. (2003). Sequential designs for phase iii clinical trials incorporating treatment selection. Statistics in medicine , 22(5):689--703

    2003

  19. [27]

    F., Simon, R., and Estey, E

    Thall, P. F., Simon, R., and Estey, E. H. (1995). Bayesian sequential monitoring designs for single‐arm clinical trials with multiple outcomes. Statistics in Medicine , 14(4):357--379

    1995

  20. [28]

    Uplekar, M., Weil, D., Lonnroth, K., and et al. (2015). Who's new end tb strategy. Lancet , 385(9979):1799--1801

    2015

  21. [29]

    Food and Drug Administration (2010)

    U.S. Food and Drug Administration (2010). Guidance for the use of bayesian statistics in medical device clinical trials. https://www.fda.gov/media/71512/download

    2010

  22. [30]

    Food and Drug Administration (2019)

    U.S. Food and Drug Administration (2019). Adaptive designs for clinical trials of drugs and biologics: Guidance for industry. https://www.fda.gov/media/78495/download

    2019

  23. [31]

    van der Vaart, A. W. (1998). Asymptotic Statistics . Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge University Press

    1998

  24. [32]

    and Gelfand, A

    Wang, F. and Gelfand, A. E. (2002). A simulation-based approach to B ayesian sample size determination for performance under a given model and for separating models. Statistical Science , 17(2):193--208

    2002

  25. [33]

    Global tuberculosis report 2022

    World Health Organization (2022). Global tuberculosis report 2022

    2022

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.