REVIEW 3 major objections 7 minor 31 references
Randomized Basket Trial with an Interim Analysis (RaBIt) and Applications in Mental Health
T0 review · 3 major / 7 minor · reviewed 2026-08-12 · deepseek-v4-flash
Pith's one-line read RaBIt generalizes randomized basket trials to unequal basket sizes and effect sizes while preserving overall type 1 error, and in a worked example shortens expected trial duration by about 17.5 months at a power loss of roughly 0.0025.
desk verdict A sound, useful generalization of Chen et al.'s randomized basket trial to unequal baskets; the statistics hold up, but the write-up has a few inconsistencies that need cleaning. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the weighted pooled statistic $V_m$ and the weights $w_i = p_i/(m \cdot p)$, which define how a pruned basket's sample is redistributed to the baskets that remain. Because the weights are proportional to the original basket proportions, the final pooled analysis stays aligned with the planned unequal design. The paper's calibration step uses the independent-increments correlation $\mathrm{corr}(Y_{i1}, Y_{i2}) = \sqrt{t\,(m \cdot p)}$ together with the weight correlation $\mathrm{corr}(Y_{i2}, V_m) = w_i / \sqrt{\sum w_i^2}$ to express each configuration's rejection probability, then solves numerically for the final threshold $\alpha^*$ from $\alpha = \sum_{m\in M} \Pr_{H_0}(V_m \mid \alpha^*, \alpha_t, m)$. This converts the equal-basket combinatorics of the earlier design into a simple sum over pruning configurations.
What would settle it
Simulate the worked three-basket trial under a misspecified correlation or with non-normal endpoints and check whether the empirical type 1 error matches the nominal $\alpha = 0.025$; separately, recompute expected duration under the practical policy of a single interim analysis after all baskets finish stage 1. If the empirical type 1 error deviates materially, or the 17-month duration gap closes, the claims are conditional on those assumptions.
Extended reading notes
Core claim
The central claim is that a randomized basket trial can prune and pool baskets of different sizes and different effect sizes without inflating the overall type 1 error. For each possible set of baskets that survives the interim, the final test statistic is the weighted Stouffer combination $V_m = (\sum_{i \in \mathrm{id}(m)} w_i Y_{i2}) / \sqrt{\sum_{i \in \mathrm{id}(m)} w_i^2}$, with weights $w_i = p_i/(m \cdot p)$ that reallocate the sample mass of pruned baskets in proportion to each surviving basket's original size. The paper derives the correlation between the interim statistic and this final statistic, and obtains $\alpha^*$ by requiring the sum of rejection probabilities over all pruning configurations to equal the nominal $\alpha$. Under this calibration, power also has a closed-form sum. The authors report that equal baskets recover the D2 design's powers almost exactly, while unequal allocation makes the final threshold less stringent at a small power cost; in the worked example, proportional-to-accrual allocation reduces expected duration by roughly 17.5 months at a power loss near 0.0025.
Load-bearing premise
The load-bearing premise is that the interim and final test statistics follow the normal, known-variance, independent-increments correlation structure in equation (7); the trial-duration numbers also rely on constant accrual and on running interim analyses as soon as each basket reaches its target, a strategy the paper calls 'fastest possible, though impractical.'
Editorial extensions
If this is right
- A phase 3 basket trial can plan basket sizes to match expected accrual without losing type 1 error control; in the paper's three-basket example, this reduces expected duration from about 61 to about 44 months with a power difference of about 0.0025.
- More unequal basket allocation makes the final threshold $\alpha^*$ less stringent, for example $\alpha^* = 0.0100$ at equal sizes versus $0.0152$ at the most unequal allocation tested for three baskets, while lowering power modestly from 0.879 to 0.837.
- If effect sizes are unequal but their average is fixed, concentrating the larger effect in one basket increases overall power; for average effect 0.5, increasing one basket's effect from 0.5 to 1.1 raises power from 0.879 to 0.969.
- When baskets are equal, the generalized formulas reproduce the earlier D2 design's power within $\pm 0.3\%$, so RaBIt is a backward-compatible extension.
- A frequentist, prior-free randomized basket design is available for confirmatory mental-health trials where accrual differs by indication.
Reading between the lines
- The 17-month duration saving is tied to the paper's 'fastest possible, though impractical' assumption that the interim analysis is run as soon as each basket reaches its target; under the more realistic policy of one interim analysis after all baskets finish stage 1, the duration advantage could shrink or disappear.
- The $\alpha^*$ calibration assumes normally distributed interim and final statistics with known variance and the independent-increments correlation of equation (7); a misspecified correlation, or non-normal endpoints, would require a simulation check before the thresholds could be trusted in practice.
- The same weighting idea could be carried to umbrella or platform designs where pruning decisions are made per subgroup and final inference is shared through a common control; the paper does not develop that direction.
- A natural stress test would compare RaBIt's frequentist operating characteristics with Bayesian hierarchical basket designs under prior misspecification, since RaBIt deliberately avoids information sharing between baskets.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes RaBIt, an extension of Chen et al.'s D2 randomized basket trial design. The key generalization is to allow baskets to have different planned sample sizes and different anticipated effect sizes. The design prunes baskets at an interim analysis based on a common threshold and then combines the remaining baskets using a weighted Stouffer statistic, with weights determined by the initial allocation proportions conditionally on the set of baskets retained. The authors derive an expression for the overall type 1 error as a sum over all possible retention sets, solve numerically for the final critical value alpha*, and derive the corresponding power formula. They validate the implementation by reproducing Chen et al.'s power values for equal-sized baskets, examine how alpha* and power vary with allocation imbalance (measured by a Gini impurity), and compute expected trial duration and sample size under constant accrual rates. In a worked example, allocating baskets proportionally to accrual is reported to shorten expected duration by about 17.5 months with a power loss of 0.25 percentage points.
Significance. The statistical derivation appears sound and offers a useful, practical extension of a published confirmatory basket trial design. The paper ships code and validates against Chen et al.'s published results, which is a concrete strength that supports reproducibility. The design is relevant to mental health and other settings where basket accrual rates and anticipated effect sizes are heterogeneous. If the duration result is robust, it has clear logistical value. The main caveat concerns the interim-timing assumption underlying the duration comparisons, which needs to be surfaced and tested.
major comments (3)
- [Section 3.5 / Table 3 / Abstract] The headline duration saving of about 17.5 months is computed under the 'fastest possible, though impractical' assumption stated in Appendix B.1, namely that each basket's interim analysis is performed as soon as that basket reaches its interim target sample size. This assumption is not disclosed in the abstract or in Section 3.5, where the duration reduction is presented as a design benefit. Since a conventional trial would conduct a single interim analysis only after all baskets complete stage 1 accrual, and the authors themselves note this simpler strategy 'will increase the trial duration', the reported saving may not be realized in practice. Please provide expected durations under the single-interim-time model as well, or prominently qualify the claim in the abstract and Section 3.5.
- [Equation (4)] The displayed event for pruned baskets is written as \cap_{l \notin id(m)} Y_{l1} > Z_{1-\alpha_t}, which would require the pruned baskets to also exceed the interim threshold. This is inconsistent with the factorization in equation (5), which multiplies by (1-\alpha_t)^{K-|id(m)|} for those baskets, and with the corresponding event in equation (11), where pruned baskets satisfy Y_{j1} < Z_{1-\alpha_t}. The inequality in equation (4) should be corrected; as printed, the event is empty whenever any basket is pruned, which would make the subsequent formula unintelligible.
- [Section 2.3, power decomposition] The sentence 'the product of baskets accurately getting pruned away (let there be R of them) and baskets inaccurately getting pruned away' is reversed relative to the formula that follows. The product over id(g)\id(j) corresponds to active baskets that are incorrectly pruned, while (1-\alpha_t)^R is the contribution of inactive baskets that are correctly pruned. Please reword the explanation so that the text matches the displayed expression.
minor comments (7)
- [Equation (8)] The rendering of corr(Y_{i2}, V_m) as 'w_i qP_m i=1 w^2_i' is garbled; it should read w_i / sqrt(\sum_{i\in id(m)} w_i^2). Please fix the typesetting.
- [Abstract] There is a missing word: 'consistent the prior methods' should be 'consistent with the prior methods'.
- [Section 3.1] The statement that power values 'only deviate ±0.3%' is not supported by Table 1, where all absolute differences are on the order of 10^-4 (i.e., roughly 0.01 percentage points). Please report the actual maximum deviation.
- [Section 2.1] The text defines t as the information time but writes N \cdot p_i \cdot t_i for each basket; the subsequent formulas (e.g., equation (7)) use a common t. Clarify that a single information time is assumed for all baskets.
- [Section 2.2 / Equation (10)] The sum over m in equation (10) should state explicitly that terms with m = 0 (no baskets retained) contribute zero probability to the overall type 1 error, since no final test is performed in that case.
- [Section 2.4 / Figure 2] The 'Gini Impurity' used here, 1 - \sum p_i^2, is not the usual Gini coefficient; higher values indicate more equal allocation. A one-line explanation of the measure's interpretation would help avoid confusion.
- [Section 4.1] The heuristic explanation for why unequal allocation leads to a less stringent alpha* refers to 'interim power' under the alternative, whereas the alpha* calibration is derived under H0 where sample size does not affect the marginal distribution of each interim z-statistic. The heuristic may be confusing and should be reformulated in terms of the correlations in equation (9).
Circularity Check
No significant circularity: RaBIt's alpha* is calibrated, not predicted, and the only self-citation is for software tooling.
full rationale
The derivation chain is self-contained. The paper defines the final pooled statistic V_m by a weighted Stouffer combination (eq. 2), derives the correlation structure (eqs. 6-9), sums over all interim outcome patterns to express the overall type 1 error (eq. 10), and then solves numerically for alpha*. This is calibration: alpha* is the threshold that makes the type 1 error equal to the prespecified alpha, and the power calculation (eq. 12) is a downstream consequence of that threshold, not a fitted input recycled as a prediction. The validation against Chen et al.'s published code is an external benchmark for the equal-allocation special case, and the reported consistency within ±0.3% supports the generalization rather than begging the question. The only self-citation, Chen et al. (2024), points to a Shiny software tool for RaBIt and is not used to justify any load-bearing mathematical premise. The trial-duration results are explicitly model-based under stated assumptions in Appendix B.1 about constant accrual and interim timing; they are presented as design calculations, not as empirical predictions. No circular step reducing a claimed result to its own input was found.
Assumptions & free parameters
assumptions (4)
- domain assumption The interim and final test statistics are normally distributed with known variance: Y_{i1} ~ N(Delta_i sqrt(N p_i t)/4, 1) under H1 and N(0,1) under H0.
- domain assumption Baskets are mutually independent, so joint probabilities factor across baskets.
- domain assumption The correlation between interim and final test statistics for the same basket is sqrt(t * m.p), following from independent increments with overlapping samples.
- ad hoc to paper Accrual rates are constant over time, and the interim analysis is conducted as soon as each basket reaches its interim sample size.
Cite this review
Pith. "Pith review of Randomized Basket Trial with an Interim Analysis (RaBIt) and Applications in Mental Health." pith.science (2026). https://pith.science/paper/DZX5Y4JX
@misc{pith2026241113692,
author = {Pith},
title = {Pith review of: Randomized Basket Trial with an Interim Analysis (RaBIt) and Applications in Mental Health},
year = {2026},
howpublished = {\url{https://pith.science/paper/DZX5Y4JX}},
note = {Machine review of arXiv:2411.13692}
}
read the original abstract
Basket trials can efficiently evaluate a single treatment across multiple diseases with a common shared target. Prior methods for randomized basket trials required baskets to have the same sample and effect sizes. To that end, we developed a general randomized basket trial with an interim analysis (RaBIt) that allows for unequal sample sizes and effect sizes per basket. RaBIt is characterized by pruning at an interim stage and then analyzing a pooling of the remaining baskets. We derived the analytical power and type 1 error for the design. We first show that our results are consistent with the prior methods when the sample and effect sizes were the same across baskets. As we adjust the sample allocation between baskets, our threshold for the final test statistic becomes more stringent in order to maintain the same overall type 1 error. Finally, we notice that if we fix a sample size for the baskets proportional to their accrual rate, then at the cost of an almost negligible amount of power, the trial overall is expected to take substantially less time than the non-generalized version.
Figures
Reference graph
Works this paper leans on
-
[1]
Scott M. Berry, Kristine R. Broglio, Susan Groshen, and Donald A. Berry. Bayesian hierarchical modeling of patient subpopulations: efficient designs of Phase II oncology clinical trials. Clinical Trials (London, England), 10 0 (5): 0 720--734, October 2013. ISSN 1740-7753. doi:10.1177/1740774513497539
-
[2]
Cong Chen, Xiaoyun (Nicole) Li, Shuai Yuan, Zoran Antonijevic, Rasika Kalamegham, and Robert A. Beckman. Statistical Design and Considerations of a Phase 3 Basket Trial for Simultaneous Investigation of Multiple Tumor Types in One Study . Statistics in Biopharmaceutical Research, 8 0 (3): 0 248--257, July 2016. ISSN null. doi:10.1080/19466315.2016.1193044...
arXiv 2016
-
[3]
Desmond Chen, Sahil Patel, Aoqi Xie, Jiayin Chen, David Castle, and Clement Ma. Rabit:an effective visualization-driven tool for power and sample size estimation in two-stage general randomized basket trial designs. medRxiv, 2024
work page 2024
-
[4]
A Bayesian Basket Trial Design Using a Calibrated Bayesian Hierarchical Model
Yiyi Chu and Ying Yuan. A Bayesian Basket Trial Design Using a Calibrated Bayesian Hierarchical Model . Clinical trials (London, England), 15 0 (2): 0 149--158, April 2018. ISSN 1740-7745. doi:10.1177/1740774518755122. URL https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5891374/
-
[5]
The role of basket trials in drug development for neurodegenerative disorders
Jeffrey Cummings, Arturo Montes, Sana Kamboj, and Jorge Fonseca Cacho. The role of basket trials in drug development for neurodegenerative disorders. Alzheimer's Research & Therapy, 14 0 (1): 0 73, May 2022. ISSN 1758-9193. doi:10.1186/s13195-022-01015-6
-
[6]
Alan K. Davis. Examining the Safety and Clinical Efficacy of Psilocybin Therapy for Veterans With PTSD : An Open - Label Proof -of- Concept Trial . Clinical trial registration NCT05554094, clinicaltrials.gov, March 2023. URL https://clinicaltrials.gov/study/NCT05554094. submitted: September 21, 2022
work page 2023
-
[7]
James Downar. PSilocybin for psYCHological and Existential Distress in PALliative Care ( PSYCHED - PAL ): A Multi -site, Open -label, Single Arm Phase I / II Proof -of-concept, Dose -finding, and Feasibility Clinical Trial . Clinical trial registration NCT04754061, clinicaltrials.gov, January 2024. URL https://clinicaltrials.gov/study/NCT04754061. submitt...
work page 2024
-
[8]
Psilocybin for Treatment of Alcohol Use Disorder : a Feasibility Study
Anders Fink-Jensen. Psilocybin for Treatment of Alcohol Use Disorder : a Feasibility Study . Clinical trial registration NCT04718792, clinicaltrials.gov, July 2023. URL https://clinicaltrials.gov/study/NCT04718792. submitted: January 8, 2021
work page 2023
Show all 31 references
-
[9]
Obsessive-compulsive disorder and related disorders: a comprehensive survey
Michele Fornaro, Filippo Gabrielli, Claudio Albano, Stefania Fornaro, Salvatore Rizzato, Chiara Mattei, Paola Solano, Valentina Vinciguerra, and Pantaleo Fornaro. Obsessive-compulsive disorder and related disorders: a comprehensive survey. Annals of General Psychiatry, 8: 0 13...
2009 doi
-
[10]
Hobbs, Roberto Carmagnani Pestana, Emily C
Brian P. Hobbs, Roberto Carmagnani Pestana, Emily C. Zabor, Alexander M. Kaizer, and David S. Hong. Basket Trials : Review of Current Practice and Innovations for Future Trials . Journal of Clinical Oncology, 40 0 (30): 0 3520--3528, October 2022. ISSN 0732-183X. doi:10.1200/J...
2022 doi
-
[11]
Effects of Psilocybin in Obsessive Compulsive Disorder
Johns Hopkins University . Effects of Psilocybin in Obsessive Compulsive Disorder . Clinical trial registration NCT05546658, clinicaltrials.gov, April 2023. URL https://clinicaltrials.gov/study/NCT05546658. submitted: September 15, 2022
2023
-
[12]
Joshi and Gregory A
Yash B. Joshi and Gregory A. Light. Using eeg-guided basket and umbrella trials in psychiatry: A precision medicine approach for cognitive impairment in schizophrenia. Frontiers in Psychiatry, 9: 0 554, November 2018. ISSN 1664-0640. doi:10.3389/fpsyt.2018.00554
2018
-
[13]
Adetayo Kasim, Nathan Bean, Sarah Jo Hendriksen, Tai-Tsang Chen, Helen Zhou, and Matthew A. Psioda. Basket trials in oncology: a systematic review of practices and methods, comparative analysis of innovative methods, and an appraisal of a missed opportunity. Frontiers in Oncol...
2023
-
[14]
Rodas, M
Nicole Ledwos, Justyne D. Rodas, M. Ishrat Husain, Jamie D. Feusner, and David J. Castle. Therapeutic uses of psychedelics for eating disorders and body dysmorphic disorder. Journal of Psychopharmacology (Oxford, England), 37 0 (1): 0 3--13, January 2023. ISSN 1461-7285. doi:1...
2023 doi
-
[15]
Trial of pimavanserin in dementia-related psychosis
Tariot Pierre N., Cummings Jeffrey L., Soto-Martin Maria E., Ballard Clive, Erten-Lyons Deniz, Sultzer David L., Devanand Davangere P., Weintraub Daniel, McEvoy Bradley, Youakim James M., Stankovic Srdjan, and Foff Erin P. Trial of pimavanserin in dementia-related psychosis. N...
2021 doi
-
[16]
Bayesian Additive Regression Trees ( BART ) with covariate adjusted borrowing in subgroup analyses
Jane Pan, Veronica Bunn, Bradley Hupf, and Jianchang Lin. Bayesian Additive Regression Trees ( BART ) with covariate adjusted borrowing in subgroup analyses. Journal of Biopharmaceutical Statistics, 32 0 (4): 0 613--626, July 2022. ISSN 1054-3406. doi:10.1080/10543406.2022.208...
2022
-
[17]
Jay J. H. Park, Ellie Siden, Michael J. Zoratti, Louis Dron, Ofir Harari, Joel Singer, Richard T. Lester, Kristian Thorlund, and Edward J. Mills. Systematic review of basket trials, umbrella trials, and platform trials: a landscape analysis of master protocols. Trials, 20 0 (1...
2019 doi
-
[18]
Direct comparisons of anorexia nervosa and body dysmorphic disorder: A systematic review
Andrea Phillipou, David Jonathan Castle, and Susan Lee Rossell. Direct comparisons of anorexia nervosa and body dysmorphic disorder: A systematic review. Psychiatry Research, 274: 0 129--137, April 2019. ISSN 1872-7123. doi:10.1016/j.psychres.2019.01.106
2019 doi
-
[19]
Bayesian adaptive basket trial design using model averaging
Matthew A Psioda, Jiawei Xu, Qi Jiang, Chunlei Ke, Zhao Yang, and Joseph G Ibrahim. Bayesian adaptive basket trial design using model averaging. Biostatistics, 22 0 (1): 0 19--34, January 2021. ISSN 1465-4644. doi:10.1093/biostatistics/kxz014. URL https://doi.org/10.1093/biost...
2021 doi
-
[20]
L. A. Renfro and D. J. Sargent. Statistical controversies in clinical research: basket trials, umbrella trials, and other master protocols: a review and examples. Annals of Oncology, 28 0 (1): 0 34--43, January 2017. ISSN 0923-7534. doi:10.1093/annonc/mdw413. URL https://www.n...
2017 doi
-
[21]
Stouffer, Edward A
Samuel A. Stouffer, Edward A. Suchman, Leland C. Devinney, Shirley A. Star, and Robin M. Williams Jr. The American soldier: Adjustment during army life. (Studies in social psychology in World War II), Vol. 1. The American soldier: Adjustment during army life. (Studies in socia...
1949
-
[22]
Sun, Cong Chen, and Kamlesh Patel
Linda Z. Sun, Cong Chen, and Kamlesh Patel. Optimal two-stage randomized multinomial designs for phase ii oncology trials. Journal of Biopharmaceutical Statistics, 19 0 (3): 0 485–493, April 2009. ISSN 1054-3406. doi:10.1080/10543400902802417
2009 doi
-
[23]
Christophe Le Tourneau, Jean-Pierre Delord, Anthony Gonçalves, Céline Gavoille, Coraline Dubot, Nicolas Isambert, Mario Campone, Olivier Trédan, Marie-Ange Massiani, Cécile Mauborgne, Sebastien Armanet, Nicolas Servant, Ivan Bièche, Virginie Bernard, David Gentien, Pascal Jeze...
2015
-
[24]
Twenge, A
Jean M. Twenge, A. Bell Cooper, Thomas E. Joiner, Mary E. Duffy, and Sarah G. Binau. Age, period, and cohort trends in mood disorder indicators and suicide-related outcomes in a nationally representative dataset, 2005-2017. Journal of Abnormal Psychology, 128 0 (3): 0 185--199...
2005 doi
-
[25]
An Open Label Study of Single - Dose Psilocybin for Major Depressive Disorder With Co -occurring Borderline Personality Disorder
University of Chicago . An Open Label Study of Single - Dose Psilocybin for Major Depressive Disorder With Co -occurring Borderline Personality Disorder . Clinical trial registration NCT05399498, clinicaltrials.gov, September 2023. URL https://clinicaltrials.gov/study/NCT05399...
2023
-
[26]
Safety and Feasibility of Psilocybin in Methamphetamine Use Disorder in a Community - Based Sample
University of Wisconsin, Madison . Safety and Feasibility of Psilocybin in Methamphetamine Use Disorder in a Community - Based Sample . Clinical trial registration NCT05322954, clinicaltrials.gov, July 2023. URL https://clinicaltrials.gov/study/NCT05322954. submitted: April 4, 2022
2023
-
[27]
A Randomized , Double - Blind , Support -of- Concept Phase 2 Study of Single - Dose Psilocybin for Major Depressive Disorder ( MDD )
Usona Institute . A Randomized , Double - Blind , Support -of- Concept Phase 2 Study of Single - Dose Psilocybin for Major Depressive Disorder ( MDD ). Clinical trial registration NCT03866174, clinicaltrials.gov, May 2023. URL https://clinicaltrials.gov/study/NCT03866174. subm...
2023
-
[28]
Weintraub
Marc J. Weintraub. Psilocybin-assisted Cognitive Behavioral Therapy for Depression . Clinical trial registration NCT05227612, clinicaltrials.gov, July 2023. URL https://clinicaltrials.gov/study/NCT05227612. submitted: January 26, 2022
2023
-
[29]
Janet Woodcock and Lisa M. LaVange. Master Protocols to Study Multiple Therapies , Multiple Diseases , or Both . New England Journal of Medicine, 377 0 (1): 0 62--70, July 2017. ISSN 0028-4793. doi:10.1056/NEJMra1510062. URL https://doi.org/10.1056/NEJMra1510062
2017 doi
-
[30]
An Open - Label Pilot Study Examining the Feasibility , Safety , and Effectiveness of Psilocybin Therapy for Depression in Bipolar II Disorder
Joshua Woolley. An Open - Label Pilot Study Examining the Feasibility , Safety , and Effectiveness of Psilocybin Therapy for Depression in Bipolar II Disorder . Clinical trial registration NCT05065294, clinicaltrials.gov, April 2023. URL https://clinicaltrials.gov/study/NCT050...
2023
-
[31]
Psilocybin Therapy for Depression and Anxiety in Parkinson 's Disease : a Pilot Study
Joshua Woolley. Psilocybin Therapy for Depression and Anxiety in Parkinson 's Disease : a Pilot Study . Clinical trial registration NCT04932434, clinicaltrials.gov, January 2024. URL https://clinicaltrials.gov/study/NCT04932434. submitted: May 5, 2021
2024
Reviewed August 12, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.