{"id":"0cb077f2-cbb7-433c-8998-52df4e486b83","arxiv_id":"2508.03948","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A BART-based emulation of the variance term in the Bernstein-von Mises approximation gives an analytic power function for Bayesian designs with nuisance parameters, for both fixed and group sequential designs.","lead":"This paper proposes a method that uses large-sample posterior normality (Bernstein-von Mises) plus a machine learning surrogate (BART) to quickly compute power and sample size for Bayesian experiments, even when the model has nuisance parameters. The authors show its use on a logistic regression design and a group sequential clinical trial design, claiming large computational savings over full Monte Carlo.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central approximation is only as good as the BvM and constant-λ assumptions at planned n; the paper's own Discussion concedes this, and the GSD extension compounds it by using λ trained at the first interim for all stages.","rationale":"The reader's weakest_assumption is the same condition I would stress. The BvM approximation and the λ̂ ≈ λ* simplification are the only routes from the posterior probability (2) to the closed-form power (8); BART only estimates λ. In the GSD extension, the additional assumption that λ is constant across stages and can be learned at one sample size makes the approximation more brittle. The authors are transparent about this in the Discussion, which is a point in their favor, but it means the central claim of 'instant' accurate evaluation is conditional on an asymptotic regime that is not characterized. My concrete test would probe this directly. I do not see an internal algebraic error in Eq. (8) or Eq. (13); the derivation is standard conditional on the asymptotics. I therefore do not change the reader's CONDITIONAL verdict.","tokens_in":13059,"tokens_out":12282,"duration_ms":166253,"concrete_test":"Retrain the BART model for design D1 using λ estimates obtained at the final-stage sample size n = 700 rather than n = 350, and compare the resulting cumulative assurance and IESS from Eq. (13) to (a) the paper's n = 350-trained values and (b) a full Monte Carlo simulation with, say, 5,000 replicates per θ. If the maximum deviation exceeds the reported ~0.04, or if the assurance/IESS values move outside Monte Carlo confidence intervals, the constant-λ / single-training-n assumption fails at these planned sample sizes.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Eq. (8) follows from the Bernstein-von Mises approximation plus the simplification λ̂ ≈ λ* after Eq. (6); Eq. (13) adds the joint canonical distribution with a λ that is constant across stages. The load-bearing condition is that these asymptotic approximations are accurate at the planned sample sizes and that λ estimated at one n (n = 350 in the PLATINUM-CAN example, Supplement A.2) transfers to the other stages (n = 500, 700). The BART surrogate cannot remove this bias; it only predicts the asymptotic-scale λ. If the BvM normal approximation or the λ̂ ≈ λ* simplification fails, Eq. (8) and Eq. (13) are biased regardless of how well BART fits. The Discussion explicitly states that the satisfactory performance is not broadly generalizable and may be poor for small samples and more complex models. The two LOOCV examples (Section 3 and Supplement A.2) validate only those specific settings, not the general claim of instant accurate operating characteristics.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes BART-BvM, an approximation to Bayesian operating characteristics for fixed and group sequential designs in the presence of nuisance parameters. The authors combine the Bernstein-von Mises theorem with asymptotic normality of the MLE to approximate the probit-transformed posterior probability of success as approximately √n/λ(ψ(θ)-ψ0)+ε, where λ is a parameter-dependent variance scale learned by Bayesian additive regression trees (BART) from a small Monte Carlo training set. This yields analytic power, assurance, and stopping probabilities. The approach is demonstrated on a logistic regression example and on a hypothetical Bayesian group sequential design based on the PLATINUM-CAN trial, with leave-one-out cross-validation deviations below 0.04. The paper includes a public code repository.","tokens_in":13390,"tokens_out":11683,"duration_ms":144072,"significance":"If the issues below are resolved, the method has clear practical value: it offers a fast approximation to Monte Carlo-based Bayesian design, especially for nuisance-parameter settings, and it extends naturally to group sequential designs through a joint canonical distribution. The public code and the LOOCV validation on two examples are strengths. However, the central equations contain a definitional inconsistency in the scale parameter λ, and the claim of instantaneous power curves over a range of sample sizes is not fully supported by the validation. The approach is explicitly acknowledged by the authors to be not broadly generalizable, which further tempers the advertised scope.","major_comments":[{"comment":"The definition of λ is internally inconsistent. The text states 'let λ̂/n be the variance in (5) and λ*/n be the variance in (6)', so λ is the asymptotic variance of √n ψ̂. Under that definition, ε = √n/λ(ψ̂_n - ψ*) has variance 1/λ, not 1, and the approximation γ_n ≈ √n/λ(ψ* - ψ0) + ε, and hence Eq. (8), does not follow. The group sequential covariance calculation in Section 2.3 uses Var(ψ̂_k) = λ²/n_k, and the empirical construction in Section 3 ('√n times the standard deviation of this sampling distribution') corresponds to treating λ as the standard-deviation scale (Var = λ²/n). The paper should consistently define λ²/n as the variance and re-derive the equations, or explicitly state that λ denotes the standard-deviation scale. As printed, a literal reader of (5)-(8) will obtain a power function that is off by a factor of √λ.","section":"Section 2.2, Eqs. (5)-(8) and Section 2.3, Eq. (13)"},{"comment":"The claim of 'instant estimation of integrated power curves for any sample size' is not validated. The BART model is trained at n = 500 and the LOOCV in Figure 1 is only at n = 500. Figure 2 shows BART-BvM power curves over a range of sample sizes, but the Monte Carlo points are only for n = 500. If the finite-sample scale λ depends on n, the curves at other sample sizes will be biased. The GSD example partially checks this (n = 350, 500, 700), but the fixed-design sample-size sweep is not empirically supported. The paper should add Monte Carlo checks at several sample sizes or restrict the claim to sample sizes near the training n.","section":"Section 3, Figure 2"},{"comment":"The assumption that λ is constant across stages is load-bearing for Eq. (13), but it is justified only by asymptotic argument and by one LOOCV exercise (design D1). The reported maximum deviation below 0.04 is reassuring but does not establish general validity for other stage spacings, sample sizes, or analysis models. The authors themselves state in the Discussion that the satisfactory performance is not broadly generalizable. A sensitivity analysis that trains λ at different interim sample sizes, or a diagnostic for assessing constancy of λ across stages, would substantially strengthen the GSD claim.","section":"Section 2.3 and Supplement A.2"}],"minor_comments":[{"comment":"Typo: 'identifiablity' should be 'identifiability'.","section":"Section 2.2"},{"comment":"The symbol I is called an 'identity function' but it is used as an indicator function; please correct the terminology.","section":"Equation (16)"},{"comment":"Please state explicitly that the yellow dots are Monte Carlo estimates obtained at n = 500 only, to avoid the impression that the displayed curves are validated at all sample sizes.","section":"Figure 2 caption"},{"comment":"The caveat that the approximation 'may be poor for small sample sizes and in more complex models' is important and should appear earlier (e.g., in the abstract or when Eq. (8) is introduced), not only in the discussion.","section":"Section 5"},{"comment":"Several author names contain accent-mark LaTeX artifacts (e.g., 'Fl´ orez Rivera'); please fix the encoding.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The paper is a useful incremental contribution to Bayesian design computation, but the main formula's notation must be made consistent before publication. The authors should also temper the 'any sample size' claim and provide additional validation for the across-n extrapolation and the GSD constant-λ assumption. The self-citation pattern is understandable given the direct line of work but should not substitute for external validation."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core idea here is genuinely new and well motivated: instead of simulating the full sampling distribution of a posterior probability, use Bernstein-von Mises to reduce it to a normal with an unknown scale λ, and train BART to predict λ across the parameter space, nuisance parameters included. That combination is not in the earlier GP-based emulator work, and it is a sensible way to make Bayesian design calculations much faster. The math in Eqs. (8) and (13) is standard BvM plus the canonical joint distribution, and it is correctly derived. The validation is honest: leave-one-out comparison against Monte Carlo, max deviation under 0.04 in the group sequential example, public code. That counts for something.\n\nThe soft spots are real but not fatal. Two examples—one logistic, one piecewise-exponential with censoring—are a thin basis for a general method. The whole edifice depends on BvM being accurate at the planned sample sizes and on λ being roughly constant across stages; if those asymptotic assumptions fail, the BART surrogate cannot fix the bias. The paper's own Discussion concedes exactly this, which I respect, but it means the abstract's 'wide adoption' claim is too strong. The GSD extension trains λ at the first interim and assumes it transfers to later stages; the one example does not stress this. And the computational efficiency claim is vague: no baseline comparison of run time or number of Monte Carlo iterations saved.\n\nThese are addressable in revision. The authors should either benchmark against Monte Carlo in a setting where BvM is known to be slow, or explicitly frame the method as an exploratory screening tool for design comparison. They already nod at that in the Discussion, so the fix is not philosophical; it is about scope discipline.\n\nWho is this for? Someone working on Bayesian clinical trial design or computational design of experiments will get a transferable idea and a working implementation. It deserves peer review. A good referee would ask for more scenarios and a cost benchmark, but the core contribution is sound and the limitations are stated rather than hidden.","headline":"A useful, honest computational trick—BvM plus BART to emulate the variance scale—that works in two examples, but the general claims outrun the evidence.","tokens_in":13799,"tokens_out":1711,"would_cite":true,"duration_ms":24435,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62K05","62F15","62L10"],"pacs":[],"model":"deepseek-v4-flash","headline":"BART-BvM turns Bayesian design calculations into a closed-form normal approximation, with nuisance parameters handled by one learned scale factor.","keywords":["Bayesian design of experiments","assurance","nuisance parameters","sample size determination","Bernstein-von Mises","Bayesian additive regression trees","group sequential design","decision theory"],"falsifier":"In the logistic-regression example, at a planned sample size such as $n=100$, repeatedly simulate data sets at a fixed $\\theta$ in the design-prior support, compute the posterior probability $\\tau$, and histogram $\\Phi^{-1}(\\tau)$. If the histogram's variance is not close to 1 or its shape is not normal, equation (8) with BART-predicted $\\lambda$ cannot give correct power; equivalently, compare equation (8) power against empirical rejection rates and look for deviations larger than Monte Carlo error.","tokens_in":13007,"feed_emoji":"📊","tokens_out":7231,"duration_ms":88552,"temperature":0.7,"pith_summary":"Designing a Bayesian study usually requires Monte Carlo estimation of the sampling distribution of posterior summaries for every candidate sample size, and nuisance parameters make the search even heavier because they must be integrated out under the design prior. This paper claims the whole calculation can be collapsed into a closed-form approximation: after a one-time BART fit that learns how the asymptotic variance parameter varies over the parameter space, power, assurance, expected sample size, and expected cost follow by plugging values into a normal probability formula. The method is justified by the Bernstein-von Mises theorem, so it inherits both the speed of asymptotic formulas and the flexibility to average over uncertainty in every parameter. The authors caution that the approximation is asymptotic, so it is positioned as an efficient way to screen designs, with Monte Carlo kept for a few final options.","feed_headline":"One formula replaces Monte Carlo for Bayesian trial power","feed_subtitle":"One BART-emulated scale term folds nuisance-parameter uncertainty into a single power equation, making design comparisons instant.","key_machinery":"Equation (8), the analytic power formula with a BART-emulated scale parameter $\\lambda(\\theta)$. Here $\\lambda$ is the per-observation variance appearing in both the approximate posterior and the sampling distribution of the MLE; the empirical step estimates $\\lambda$ from simulated posterior variances at a few points, and the regression step uses BART/SoftBART to interpolate it across the design-prior region. This single emulation carries the entire burden of nuisance-parameter dependence, because the mean structure $\\sqrt{n}(\\psi-\\psi_0)/\\lambda$ and the standard normal error are otherwise parameter-free.","core_discovery":"The core object is an analytic power function. Under the Bernstein-von Mises theorem, the probit of the posterior probability that $\\psi(\\theta)>\\psi_0$ is approximately $\\sqrt{n}/\\lambda(\\theta)\\,(\\psi(\\theta)-\\psi_0)$ plus a standard normal error, so power is $1-\\Phi(\\Phi^{-1}(u)-\\sqrt{n}/\\lambda(\\theta)(\\psi(\\theta)-\\psi_0))$. The only unknown is $\\lambda(\\theta)$, the per-observation variance scale of the posterior and of the MLE, which depends on nuisance parameters through the Fisher information. The paper estimates $\\lambda$ by simulation at a modest set of parameter values and trains a Bayesian additive regression tree to predict $\\lambda$ anywhere in the support of the design prior;","pith_inferences":["The paper leaves implicit that the same fitted $\\lambda$ should only be trusted within the training region: if the design prior is widened or shifted substantially, the BART model must be retrained on the new support.","Because the joint distribution in (13) is threshold-agnostic, futility boundaries or two-sided decision rules could be added by changing only the limits of integration, without any new BART fit; the examples stop only for efficacy.","A practical diagnostic suggested by the approach but not developed is to run full Monte Carlo at one or two design points, such as a small and a moderate sample size, as a calibration check before trusting Equation (8) across the whole parameter space.","The method's small-sample weakness could potentially be mitigated by the small-sample corrections the authors mention, but such corrections are not part of the current procedure."],"forward_implications":["Once $\\lambda$ is fitted, integrated power (Bayesian assurance) becomes a numerical integral of a closed-form normal probability, so comparing many candidate sample sizes and design priors no longer requires repeated posterior sampling.","For group sequential designs, the joint distribution in (13) gives stage-wise stopping probabilities and expected sample sizes directly from one fitted $\\lambda$, making adaptive-design screening much cheaper.","Decision-theoretic criteria such as integrated expected cost become easy to evaluate across many designs, enabling optimization over thresholds, sample sizes, and the number and spacing of interim analyses.","Because the method is parameterized only by $\\lambda$, the same BART fit can be reused to evaluate different decision thresholds or different candidate designs without additional training.","The paper states the approach generalizes from one-sided to interval hypotheses, extending its use to equivalence and other two-sided design problems."],"supporting_citations":[{"why":"Supplies the Bernstein-von Mises theorem that yields the approximate normal posterior with variance $\\lambda/n$.","marker":"van der Vaart (1998)"},{"why":"Supplies the asymptotic normality and consistency of the MLE used for the sampling distribution of $\\hat\\psi_n$.","marker":"Lehmann and Casella (1998)"},{"why":"Introduces BART, the regression model used to predict $\\lambda$ across the parameter space.","marker":"Chipman et al. (2010)"},{"why":"Supplies SoftBART, which adapts to smoothness and sparsity and is the specific BART implementation used for training and prediction.","marker":"Linero and Yang (2018)"},{"why":"Provides the joint canonical distribution that gives the multivariate normal joint sampling distribution of stage-wise decision statistics in group sequential designs.","marker":"Jennison and Turnbull (1999)"},{"why":"Earlier Gaussian-process emulation of sampling distributions that the paper extends, noting GPs do not handle insensitive nuisance parameters well.","marker":"Golchi (2022)"},{"why":"Parametric large-sample approach for posterior decision summaries, which does not handle nuisance parameters efficiently and motivates the BART-based formulation.","marker":"Golchi and Willard (2024)"}],"fun_headline_variants":["BART turns nuisance variance into instant trial power","No more Monte Carlo: BART predicts trial power instantly","Analytic power curve from a BART-learned nuisance scale","Posterior power without simulations: BART does the heavy lifting","Folding nuisance uncertainty into a single power formula"],"cache_read_input_tokens":2816,"weakest_assumption_plain":"The load-bearing premise is that the planned sample sizes are large enough for the Bernstein-von Mises normal approximation and for $\\hat\\lambda\\approx\\lambda^*$ to hold, so the probit of the posterior probability is approximately a standard normal plus a drift; at small sample sizes the power formula is biased, and the BART-emulated $\\lambda$ cannot repair that bias.","fun_headline_variants_meta":{"raw":{"variants":["BART turns nuisance variance into instant trial power","No more Monte Carlo: BART predicts trial power instantly","Analytic power curve from a BART-learned nuisance scale","Posterior power without simulations: BART does the heavy lifting","Folding nuisance uncertainty into a single power formula"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00017,"raw_usage":{"total_tokens":1112,"prompt_tokens":759,"completion_tokens":353,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":503,"completion_tokens_details":{"reasoning_tokens":274}},"tokens_in":503,"tokens_out":353,"duration_ms":4872,"temperature":1.0,"reasoning_tokens":274,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T01:00:34.418717+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"In the logistic-regression example, at a planned sample size such as $n=100$, repeatedly simulate data sets at a fixed $\\theta$ in the design-prior support, compute the posterior probability $\\tau$, and histogram $\\Phi^{-1}(\\tau)$. If the histogram's variance is not close to 1 or its shape is not normal, equation (8) with BART-predicted $\\lambda$ cannot give correct power; equivalently, compare equation (8) power against empirical rejection rates and look for deviations larger than Monte Carlo error.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the asymptotic normality and consistency of the MLE used for the sampling distribution of $\\hat\\psi_n$."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies SoftBART, which adapts to smoothness and sparsity and is the specific BART implementation used for training and prediction."},{"cited_title":"and Turnbull, B","cited_arxiv_id":null,"evidence_quote":"Provides the joint canonical distribution that gives the multivariate normal joint sampling distribution of stage-wise decision statistics in group sequential designs."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Earlier Gaussian-process emulation of sampling distributions that the paper extends, noting GPs do not handle insensitive nuisance parameters well."},{"cited_title":"and Willard, J","cited_arxiv_id":null,"evidence_quote":"Parametric large-sample approach for posterior decision summaries, which does not handle nuisance parameters efficiently and motivates the BART-based formulation."}],"review_version":1}