pith. sign in

arxiv: 2604.13709 · v1 · submitted 2026-04-15 · 📊 stat.ME

Adaptive Sample Size Simulations with R package adsasi

Skerdi Haviari This is my paper

Pith reviewed 2026-05-10 13:11 UTC · model grok-4.3

classification 📊 stat.ME
keywords sample size determinationsimulation-based power analysisR packageclinical trial designadaptive designsprobit regressionMonte Carlo methodsbootstrap sampling
0
0 comments X

The pith

The R package adsasi finds the sample size delivering target power for any experiment that can be simulated in short time.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper introduces the adsasi R package for determining sample sizes in experiments that lack closed-form solutions, such as adaptive or factorial clinical trials and A/B tests. The user supplies a function that takes a sample size, runs a simulation of the experiment, and returns a success or failure indicator. The package then iteratively calls this function at different sizes, using a modified probit regression fitted to nearby simulations to converge on the size that achieves the desired success rate. It also computes standard errors via the Cramér-Rao bound from a custom analytical Hessian. This matters because many practical designs, including those that bootstrap from existing medical data, cannot rely on traditional formulas, making simulation-based sizing a direct route to feasible planning.

Core claim

adsasi performs simulation-first sample size calculations by accepting a user-defined function that maps sample size to a binary success outcome, then applying adsasi_0d or adsasi_1d to iteratively search for the size yielding nominal power. The search relies on probit regression updated with simulations around the current estimate and supplies standard errors at each step. For simple cases the package recovers results within Monte Carlo variance of known closed-form expressions; for intractable cases it enables sizing where analytic methods fail, including bootstrap-based designs drawn from real cohorts.

What carries the argument

The adsasi_0d and adsasi_1d functions that iteratively invoke the user simulation function and fit a modified probit regression, using a custom analytical Hessian to obtain Cramér-Rao standard errors on the estimated sample size.

If this is right

  • Any trial design expressible as a short simulation can be sized without deriving a closed-form power formula.
  • Bootstrapping analyses from existing medical cohorts become a routine basis for determining new study sizes.
  • Standard errors on the computed sample size quantify the precision of the simulation-based estimate.
  • The same machinery can empirically map a design parameter to the required sample size.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Integration with existing simulation libraries could let researchers size adaptive designs without writing custom code for each case.
  • The method could be extended to search over multiple design parameters simultaneously, enabling joint optimization of sample size and other choices.
  • If the monotonicity assumption fails in practice, users could add safeguards such as multiple random starts to detect non-convergence.

Load-bearing premise

Power rises monotonically with sample size so that the iterative search reliably converges.

What would settle it

Apply adsasi to a simulation function whose success probability does not increase with sample size and observe whether the search diverges or returns a size far from the true target power.

Figures

Figures reproduced from arXiv: 2604.13709 by Skerdi Haviari.

Figure 1
Figure 1. Figure 1: Accuracy of adsasi_0d. (left) Empirically found sample sizes versus analytical value. (right) Closed-form power of the returned sample sizes, versus Monte Carlo variability with the same number of draws, in a true proportion of 90%, as adsasi_0d is using to make its inference. use their default values unless specified, in particular f1 = 0.5 which means the arms have the same size. In order to evaluate the… view at source ↗
Figure 2
Figure 2. Figure 2: Empirical estimate of the relationship between allocation fraction for one arm and [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: In-run diagnostics as shown by adsasi_0d (top) after 250 simulations and (bot￾tom) at the end of the run, using the first call from Section 4.1. Left panels show simulated sample sizes as a function of run progression (batches are visible as lines with consecutive simulations). Right panels show the regression, with successful simulations at the top (blue) and unsuccessful ones at the bottom (orange), with… view at source ↗
Figure 4
Figure 4. Figure 4: In-run diagnostics as shown by adsasi_1d. (left) Location of simulations on the design parameter × size surface, with successful ones in blue and unsuccessful ones in orange. The current estimate of the relationship between sample size and parameter of interest is overlayed. (middle) Location of the next batch of the simulations, with the same overlay. (right) Relationship between slope e s(v) and paramete… view at source ↗
Figure 5
Figure 5. Figure 5: Simulations for the ranking-based trial. (left) Partial graphical output from [PITH_FULL_IMAGE:figures/full_fig_p015_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Observational logistic model fitted on the IST cohort. The [PITH_FULL_IMAGE:figures/full_fig_p016_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Simulations from an existing patient cohort. (left) Partial graphical output from [PITH_FULL_IMAGE:figures/full_fig_p017_7.png] view at source ↗
read the original abstract

Planning empirical experiments such as clinical trials or A/B tests requires sample size determination, which in many interesting cases has no closed-form solution (e.g. factorial or adaptive designs). adsasi is a new R package that enables simulations-first sample size calculations for any trial that can be simulated in short compute time. First, the user specifies as a function that takes a sample size as argument, simulates the experiment, and returns a boolean for success/failure. Then, adsasi functions adsasi_0d and adsasi_1d iteratively call it on different sample sizes and progressively home in on the one with nominal success rate (power), assuming that increasing sample size increases power. adsasi_1d can also draw, purely empirically, the relationship between a design parameter and sample size. The implementation uses a modified probit regression (with success/failure as the dependent variable), informed by simulations conducted around the target size, and provides standard errors at each stage using the Cram\'er-Rao bound derived from a custom analytical Hessian matrix. Simple examples are first presented, yielding results within Monte Carlo variance of their closed-form expressions, then intractable ones (including bootstrapping from an existing medical cohort). adsasi will hopefully facilitate the funding and conduct of interesting, highly complex experimental designs by making their sizing straightforward.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 0 minor

Summary. The manuscript introduces the R package adsasi for simulation-based sample size determination in experimental designs (e.g., clinical trials, A/B tests) lacking closed-form solutions. Users supply a function that takes sample size n and returns a binary success indicator from a simulation; adsasi_0d and adsasi_1d then iteratively evaluate this function at varying n values, fitting a modified probit regression to locate the n achieving target power while assuming monotonic increase of power with n. Standard errors are obtained via the Cramér-Rao bound from a custom analytical Hessian. Simple cases are validated against closed forms (within Monte Carlo error), and the package is demonstrated on intractable examples including bootstrapping from a medical cohort.

Significance. If the monotonicity assumption holds and convergence is reliable, the package would provide a practical, accessible tool for power analysis in complex adaptive or simulation-heavy designs, lowering barriers to funding and conducting such experiments by replacing ad-hoc or infeasible analytic calculations with a standardized simulation workflow.

major comments (2)
  1. [Abstract] Abstract and algorithm description: The iterative search in adsasi_0d and adsasi_1d explicitly depends on the assumption that power is monotonically increasing with sample size. No analysis of convergence behavior, failure modes (e.g., plateaus from discrete effects, non-monotonicity from adaptive stopping rules, or high Monte Carlo variance in cohort bootstrapping), or robustness checks is provided, despite this being load-bearing for the central claim that the package reliably locates the target n.
  2. [Abstract] Abstract: Validation is described only qualitatively as 'within Monte Carlo variance' for closed-form cases, with no quantitative error metrics, full implementation details of the modified probit regression or Hessian, or extensive benchmarks against alternatives. This limits assessment of practical accuracy for the intractable examples.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive feedback on our manuscript introducing the adsasi R package. We address the major comments point by point below and will revise the manuscript accordingly to improve the discussion of assumptions and the quantitative presentation of validation results.

read point-by-point responses

  1. Referee: [Abstract] Abstract and algorithm description: The iterative search in adsasi_0d and adsasi_1d explicitly depends on the assumption that power is monotonically increasing with sample size. No analysis of convergence behavior, failure modes (e.g., plateaus from discrete effects, non-monotonicity from adaptive stopping rules, or high Monte Carlo variance in cohort bootstrapping), or robustness checks is provided, despite this being load-bearing for the central claim that the package reliably locates the target n.

    Authors: We agree that the monotonicity assumption is central to the algorithm and is explicitly stated in the manuscript and package documentation. We acknowledge that the current version provides limited analysis of convergence behavior and failure modes. In the revised manuscript we will add a new subsection on 'Assumptions, Convergence, and Limitations' that (i) derives basic convergence properties under the monotonicity assumption, (ii) discusses realistic failure modes including plateaus from discrete outcomes, non-monotonicity arising from adaptive stopping rules, and effects of high Monte Carlo variance in bootstrapped cohorts, and (iii) supplies practical robustness diagnostics (e.g., repeated runs with diagnostic power-curve plots and a user-facing convergence flag). These additions will clarify when the method can be expected to succeed and how users can detect problems. revision: yes

  2. Referee: [Abstract] Abstract: Validation is described only qualitatively as 'within Monte Carlo variance' for closed-form cases, with no quantitative error metrics, full implementation details of the modified probit regression or Hessian, or extensive benchmarks against alternatives. This limits assessment of practical accuracy for the intractable examples.

    Authors: We accept that the validation section would benefit from greater quantitative detail. The revised manuscript will include (i) explicit quantitative error metrics (mean absolute deviation and root-mean-square error of the estimated sample size relative to closed-form solutions, computed over 100 independent replications), (ii) the precise functional form of the modified probit regression together with the analytic Hessian used for the Cramér-Rao standard errors, and (iii) direct runtime and accuracy comparisons against simple grid search and bisection-based alternatives on both the closed-form and intractable examples. For the bootstrapped medical-cohort case we will also report the empirical standard deviation of the estimated sample size across repeated package runs to quantify practical variability. revision: yes

Circularity Check

0 steps flagged

No significant circularity detected

full rationale

The paper presents an R package that accepts an arbitrary user-supplied simulation function returning success/failure, then applies standard iterative search with modified probit regression and the Cramér-Rao bound (via analytical Hessian) to locate the sample size achieving target power. No equations in the described method reduce the estimated n to a fitted parameter or input by construction; the regression is performed on fresh Monte Carlo draws at each step, and the monotonicity assumption is an explicit modeling choice rather than a definitional tautology. No self-citations appear as load-bearing premises, and the simple-case validations compare against independent closed-form expressions. The derivation chain therefore remains self-contained and externally falsifiable via the user-provided simulator.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The method depends on the domain assumption of monotonic power increase with sample size to guarantee convergence of the search; no free parameters are introduced beyond standard regression fitting inside the algorithm, and no new entities are postulated.

axioms (1)
  • domain assumption Power is a monotonically increasing function of sample size.
    Invoked to justify the iterative homing procedure in adsasi_0d and adsasi_1d.

pith-pipeline@v0.9.0 · 5524 in / 1219 out tokens · 41651 ms · 2026-05-10T13:11:18.263362+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

2 extracted references · 2 canonical work pages · 1 internal anchor

  1. [1]

    URLhttps://cran

    Anderson K, Zhao Y, Blischak J, Xiao N, Zhang Y, Yang J, Ling L, Li X, Wang R, Cui Y, Yang P, Zhu Y, Zhou H, Shirazi A, Manschot C, Leon L, Merck & Co, Inc, Rahway, NJ, USA and its affiliates (2025).simtrial: Clinical Trial Simulation. URLhttps://cran. r-project.org/web/packages/simtrial/index.html. Bates D, Maechler M, Bolker B, Walker S, Christensen RHB...

    work page 2025

  2. [2]

    Adam: A Method for Stochastic Optimization

    RD, Ly A (2026).lme4: Linear Mixed-Effects Models Using ’Eigen’ and S4. URLhttps: //cran.r-project.org/web/packages/lme4/index.html. 20Adaptive Sample Size Simulations withadsasi Duarte K, Ferreira JP (2020).WinRatio: Win Ratio for Prioritized Outcomes and 95% Con- fidence Interval. URLhttps://cran.r-project.org/web/packages/WinRatio/index. html. Haviari ...

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.1186/s12874-024-02191-9 2026