REVIEW 5 major objections 6 minor 22 references
Adjusting SPRT for an Efficient Procedure with Finite Number of Applications of Less Effective Treatment
T0 review · 5 major / 6 minor · reviewed 2026-08-03 · deepseek-v4-flash
Pith's one-line read This paper claims that a likelihood-ratio-driven adaptive sequential test can make the expected number of allocations to the less effective treatment a finite, explicitly computable constant, while preserving the efficiency of the classical
desk verdict Promising formula and idea, but the proof is invalid and the paper's own simulations contradict the claimed error control; major revision needed. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central mechanism is the adaptive allocation rule, 'Method M': at each step, look at the stream with the larger current count n_max, form the cumulative log-likelihood ratio of the product of f0 to f1 on those observations, and allocate the next observation to the same stream if the ratio is positive, otherwise to the other stream. This rule biases sampling toward the population that appears superior, and the probability of an incorrect allocation decays like the normal tail 1−Φ((η/σ)√n). Summing these tail probabilities over n gives the closed-form constant in Theorem 3.1. The rule is coupled with classical SPRT stopping boundaries, giving error control and an average sample number with
What would settle it
Run the adaptive rule under a distribution where the log-likelihood ratio has infinite variance (for example, a heavy-tailed log-likelihood) and check whether the empirical expected number of inferior allocations keeps growing as n increases rather than stabilizing; alternatively, in the normal case, use a very small separation Δ and compare the empirical E(N1,n) to 4/Δ² — if the estimate systematically diverges from the formula as Δ → 0, the normal-tail approximation is the culprit.
Extended reading notes
Core claim
The core discovery is that a likelihood-ratio-driven adaptive allocation rule makes the number of allocations to the inferior population a finite random variable with a closed-form expected value. The allocation rule (Method M) tracks the stream that currently has the larger sample size, computes the cumulative log-likelihood ratio of f0 versus f1 on that stream, and draws the next observation from that stream if the ratio is positive and from the other stream if it is negative. The theorem states that for large n, E(N1,n) ≈ ½(σx²/ηx² + σy²/ηy²), a constant depending only on the mean and standard deviation of the log-likelihood ratios of the two streams. The proof approximates the probabilit
Load-bearing premise
The whole derivation rests on approximating the probability of an incorrect allocation by a normal tail computed from the current sample size, even though the event that a stream is the larger one is correlated with its own cumulative log-likelihood, and on treating the allocation indicators as independent Bernoulli variables.
Editorial extensions
If this is right
- The expected number of allocations to the inferior treatment is bounded by a finite constant that does not depend on the stopping boundary or the total sample size.
- All moments of the inferior-allocation count are finite, implying N1,n/N0,n converges to 0 in probability as n grows.
- For the normal example with separation Δ, the formula becomes 4/Δ², giving a simple rule of thumb for ethical exposure.
- The adaptive SPRT's average sample number is approximately the classical SPRT's ASN minus the finite expected inferior count, so the ethical improvement costs only a constant number of additional samples.
- Under the symmetric normal case with means ±θ0, the allocation rule becomes parameter-free, depending only on the running sample mean and sample size.
Reading between the lines
- The formula suggests that the expected inferior count is governed by the variance-to-squared-mean ratio of the log-likelihood ratio; distributions with heavy tails that drive the variance to infinity would likely violate the finite bound, so a natural extension is to prove or check the result under a finite-variance assumption.
- The same 'sample from the current best' rule could generalize to k > 2 treatments, with the expected count of each non-best treatment bounded by an analogous sum of tail probabilities; this is hinted at but not developed in the paper.
- In a composite-hypothesis setting, the authors suggest a worst-case bound by maximizing the formula over the parameter space; a practical extension would estimate the parameters and plug them in, yielding an adaptive design whose ethical guarantee holds approximately.
- One can interpret the expected inferior count as a regret measure for a two-armed bandit; the paper's result implies regret that stays bounded rather than growing logarithmically, under the normality approximation.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes an adaptive sequential probability ratio test (SPRT) that allocates observations to the treatment with the currently larger cumulative log-likelihood ratio (Method M), aiming to concentrate sampling on the superior population while preserving the error control and asymptotic average sample number of the classical SPRT. The central theoretical claim, Theorem 3.1, is that for large n the expected number of allocations to the less effective treatment satisfies E(N1,n) ≈ ½(σx²/ηx² + σy²/ηy²), a finite constant depending only on the first two moments of the log-likelihood ratio statistics; consequently all moments of N1,n are bounded and N1,n/N0,n → 0 in probability. Theorem 3.2 claims the adaptive SPRT is asymptotically efficient relative to the classical SPRT. The paper supports these claims with simulation studies for Normal, Poisson, and Asymmetric Laplace distributions, and compares inferior allocations with the classical SPRT.
Significance. If the finiteness and closed-form claims were rigorously established, the paper would make a practically valuable contribution: an explicit, easily computed benchmark for the expected number of inferior allocations in a likelihood-ratio–driven adaptive design, with potential relevance to clinical trial ethics. The simulation evidence is extensive and shows that the proposed procedure substantially reduces inferior allocations relative to the classical SPRT while maintaining high probability of correct selection. However, the theoretical derivation contains load-bearing gaps: the conditional nature of the allocation event is ignored, the alleged independence of allocation indicators is false, and the moment argument is incomplete. These issues currently prevent the manuscript from delivering its main promise, although the underlying phenomenon appears plausible and the simulation agreement is suggestive.
major comments (5)
- [§3, Eq. (1) and Theorem 3.1] The derivation of E(N1,n) as a sum of unconditional normal tails is not justified. The event that U corresponds to the larger-count stream X is determined by the same cumulative log-likelihood ratios that appear in the tail probability. At n=2, (N0,N1)=(2,1) requires the initial X log-ratio to be positive, so the conditional distribution of the X cumulative sum given U~X is stochastically shifted from its unconditional law. Lemma 2.1 is invoked to claim conditional i.i.d.ness, but that lemma is false for this reason. Additionally, when the larger stream has a negative cumulative log-likelihood, the rule draws from the smaller stream and re-evaluates the same statistic until the counts tie, producing consecutive dependent inferior allocations. Thus the representation as a sum over i and j of unconditional tail probabilities does not follow.
- [Theorem 3.1, moment claim] The statement 'as N1,n is the sum of independent Bernoulli variables, it follows that all the moments of N1,n are bounded' is false: the allocation indicators are dependent, as noted above. Moreover, even a uniform bound on E(N1,n) would not imply a bound on all moments. The subsequent MGF argument ('finite for finite domain by a constant function not depending on n') is not developed and does not establish uniform moment bounds. This is load-bearing because the proof that N1,n/N0,n → 0 in probability uses the second moment via Markov's inequality.
- [Lemma 2.1] The proof of Lemma 2.1 incorrectly factors f(X1,...,X_N0,n | path leading to (N0,n,N1,n)) as the product of the marginal densities, treating the path event as if it were independent of the X-values. The path event contains inequalities involving these same observations, so the factorization is circular. A concrete counterexample at n=2 shows the conditional law is not i.i.d. Because the lemma underlies the normal tail approximation in Theorem 3.1, its failure is a central gap.
- [§3.2, Theorem 3.2] The efficiency claim that ASN/ASN_K0 → 1 (or ASN/ASN_K1 → 1) is not proven. Equation (3) refers to running the SPRT on the X-data stream or the Y-data stream, whereas the adaptive rule updates U1,U2,... from the current nmax stream, a data-dependent mixture. The distribution of this U-stream is not that of an i.i.d. sample from f0 or f1, so the standard SPRT ASN formulas in Eq. (2) do not directly apply. The simulations also show P_CS values systematically different from 1-α, so the claimed exact efficiency requires a formal argument not present in the manuscript.
- [§3 vs §4.2] The main theorem explicitly restricts to continuous densities ('If they are not continuous, we need to adjust a little bit (not shown in the paper)'), yet Section 4.2 applies the unadjusted formula N1* to Poisson distributions and reports agreement. The discrete case requires a separate derivation; without it, the Poisson simulation results are not evidence for Theorem 3.1. Either the theorem must be extended to discrete families with a proper proof, or the simulation claims must be qualified accordingly.
minor comments (6)
- [Abstract] The abstract promises 'real data analyses', but the paper contains only simulations. Either add a real-data example or revise the abstract.
- [§2.1] The sentence 'When N0,n=N1,n, we assign them to nmax and nmin with probability 1/2 each' is ambiguous: the values are equal, so assigning them to labels should be clarified (e.g., randomly assign the label of X to nmax with probability 1/2).
- [Eq. (1)] The summation indices are written as 'nx' and 'ny' but the objects summed over are i and j; the definitions of nx and ny should be stated explicitly before the display.
- [Theorem 3.1 proof] The transition from the sums of Φ(-η/σ√i) to the integrals with √(-t) is not derived; the change of variable and the error bounds are omitted. This is not merely cosmetic, as the approximation's accuracy is part of the claimed closed form.
- [Remark 5] For the symmetric normal case the paper states N1* ≈ 1/θ², but the earlier formula gives 4/(θ0-θ1)² ; with the parameterization used, the constants should be checked for consistency.
- [Introduction] There is a typo in the first paragraph: 'SP RThenceforth' should be 'SPRT'. Also, some references in the introduction are cited repeatedly without page numbers; the Rao (1973) citation lacks a specific page in the reference list despite the text citing a page.
Circularity Check
No significant circularity: the closed-form expression is derived from moment-based normal-tail approximations, not fitted to simulations; self-citations are motivational and not load-bearing.
full rationale
The central derivation, Theorem 3.1, computes E(N1,n) as a sum of tail probabilities, replaces each tail by the normal approximation 1−Φ(η/σ√n) or Φ(η/σ√n), and then approximates the resulting infinite sum by an integral to obtain 1/2(σx²/ηx² + σy²/ηy²). The only inputs are the true first two moments of the log-likelihood ratios; no parameter is fitted to the simulation output. The simulations use the same theoretical N*1 as an external benchmark and show agreement, but the formula is not derived from those simulations. The paper does cite the authors' own earlier work (Bhandari et al. 2007, 2009; Kundu et al. 2025) as background and motivation, e.g., 'Building on that theoretical foundation' and 'the existence of a finite bound was hinted at in our earlier work (Kundu et al., 2025)'. However, those citations are not used to prove Theorem 3.1; the finiteness and closed form are argued directly from the normal-tail sum and the convergence of the Gaussian tail series. Thus there is no self-citation chain that forces the result. The main mathematical weakness is the assumption that N1,n is a sum of independent Bernoulli variables and that the unconditional normal tail applies to the larger stream; this is a correctness or rigor concern about the derivation, not a circular reduction of the conclusion to its inputs. No equation is defined in terms of the target quantity, and no fitted input is relabeled as a prediction.
Assumptions & free parameters
assumptions (4)
- domain assumption The log-likelihood-ratio increments have finite mean and variance, and a CLT applies to their partial sums at the random times n_max.
- ad hoc to paper f0 and f1 are continuous densities; the discrete case requires an adjustment 'not shown in the paper'.
- domain assumption The adaptive sampling does not spoil the SPRT error-control properties; the U-stream behaves like an i.i.d. sequence for large n.
- ad hoc to paper The allocation indicators are independent when bounding moments.
Cite this review
Pith. "Pith review of Adjusting SPRT for an Efficient Procedure with Finite Number of Applications of Less Effective Treatment." pith.science (2026). https://pith.science/paper/OJEVW7GM
@misc{pith2026251120061,
author = {Pith},
title = {Pith review of: Adjusting SPRT for an Efficient Procedure with Finite Number of Applications of Less Effective Treatment},
year = {2026},
howpublished = {\url{https://pith.science/paper/OJEVW7GM}},
note = {Machine review of arXiv:2511.20061}
}
read the original abstract
We propose an adaptive Sequential Probability Ratio Test (SPRT) which allocates a finite number of applications to the less effective treatment. In the classical SPRT framework, patients are assigned to the two competing treatments one by one until the stopping criterion, based on breaching the boundary values which are pre-determined using the Type-I and Type-II error probabilities, is met. This ensures the control of errors at the cost of ethical efficiency as the exposure to the less effective treatment is large. We begin with proposing an adaptive sequential framework for testing two simple hypotheses that analytically ensures finite exposure to the less effective treatment. Our proposed procedure employs a likelihood ratio driven adaptive allocation rule, dynamically concentrating sampling effort on the superior population while preserving asymptotic efficiency (in terms of average sample number), comparable to the classical SPRT. We derive an explicit closed-form expression for the expected number of allocations to the inferior treatment. Extensive simulation studies and real data analyses substantiate the theoretical results, evincing a significant reduction in inferior allocations compared to the classical SPRT. The proposed design thus offers a balanced method between statistical precision and ethical responsibility, aligning inferential reliability with patient safety.
Reference graph
Works this paper leans on
-
[1]
Bandyopadhyay, U., Mukherjee, S., and Biswas, A. (2020). Adaptive two-treatment three-period crossover design for normal responses . Brazilian Journal of Probability and Statistics , 34(2):291 -- 303. https://doi.org/10.1214/18-BJPS427
-
[2]
Berry, D. A. and Fristedt, B. (1985). Bandit Problems: Sequential Allocation of Experiments . Chapman and Hall, London
1985
-
[3]
Bhandari, S. K., De, S. K., Mandal, S., Pradhan, S., and Ghosh, B. (2009). Study of optimal adaptive rule in testing composite hypothesis. Sequential Analysis , 28(3):394--405. https://doi.org/10.1080/07474940903041852
-
[4]
K., Dutta, R., and Niyogi, R
Bhandari, S. K., Dutta, R., and Niyogi, R. G. (2007). Study of optimal adaptive rule in testing problem. In Advances in Multivariate Statistical Methods . World Scientific
2007
-
[5]
Biswas, A., Bandyopadhyay, U., and Bhattacharya, R. (2007). Response-Adaptive Designs in Phase III Clinical Trials. In Statistical Advances in the Biomedical Sciences , pages 33--53. John Wiley & Sons, Ltd
2007
-
[6]
Biswas, A., Bhattacharya, R., and Das, S. (2020). A multi-treatment response adaptive design for ordinal categorical responses. Statistical Methods in Medical Research , 29(3):827--836
2020
-
[7]
Das, R. (2024a). An optimal sequential design in ethical allocation with an adaptive interim analysis. Journal of Nonparametric Statistics , 36(3):547--572. https://doi.org/10.1080/10485252.2023.2223322
arXiv 2023
-
[8]
Das, S. (2024b). A two armed optimal response adaptive randomization for ordinal categorical responses with possible misclassification. Sequential Analysis , 43(2):131--146. https://doi.org/10.1080/07474946.2023.2295264
arXiv 2023
Show all 22 references
-
[9]
Das, S., Bhattacharya, R., and Biswas, A. (2023). Multi-arm covariate adjusted response adaptive designs for ordinal outcome clinical trials. Statistical Methods in Medical Research , 32(1):88--99
2023
-
[10]
M., Furgberg, C
Friedman, L. M., Furgberg, C. D., and DeMets, D. L. (2010). Fundamentals of Clinical Trials . Springer
2010
-
[11]
and Rosenberger, W
Ivanova, A. and Rosenberger, W. F. (2000). A comparison of urn designs for randomized clinical trials of k > 2 treatments. Journal of Biopharmaceutical Statistics , 10(1):93--107
2000
-
[12]
and Jain, R
Jain, S. and Jain, R. (1994). An application of sprt for detecting change point in a reliability growth model. Microelectronics Reliability , 34(5):811--814. https://www.sciencedirect.com/science/article/pii/002627149490006X
1994
-
[13]
J., and Podg \'o rski, K
Kotz, S., Kozubowski, T. J., and Podg \'o rski, K. (2001). Asymmetric Laplace Distributions. In The Laplace Distribution and Generalizations: A Revisit with Applications to Communications, Economics, Engineering, and Finance , pages 133--178. Birkh \"a user Boston. https://doi...
2001 doi
-
[14]
Kundu, S., Jha, J., and Bhandari, S. K. (2025). To study properties of a known procedure in adaptive sequential sampling design. Sequential Analysis , pages 1--13. https://doi.org/10.1080/07474946.2025.2533328
2025
-
[15]
and Nenes, G
Li, C. and Nenes, G. (2024). Economic modelling and optimisation of sprt-based quality control schemes for individual observations. International Journal of Production Research , 62(24):8766--8789. https://doi.org/10.1080/00207543.2024.2349252
2024
-
[16]
X., Jeske, D
Li, J. X., Jeske, D. R., and Klein, J. A. (2012). Sequential analysis methodology for a poisson glmm with applications to multicenter randomized clinical trials. Journal of Statistical Planning and Inference , 142(12):3225--3234. https://www.sciencedirect.com/science/article/p...
2012
-
[17]
Martens, M. J. and Logan, B. R. (2024). Statistical rules for safety monitoring in clinical trials. Clinical Trials , 21(2):152--161
2024
-
[18]
Rao, C. R. (1973). Linear Statistical Inference and its Applications: Second Editon , pages 444--515. Wiley Series in Probability and Statistics. Wiley. https://doi.org/10.1002/9780470316436.ch7
1973 doi
-
[19]
and Alinezhad, E
Rasay, H. and Alinezhad, E. (2022). Developing an adaptable sequential probability ratio test applicable for lifetime analysis of different continuous distributions. Quality Technology & Quantitative Management , 19(4):511--530. https://doi.org/10.1080/16843703.2021.2020954
2022
-
[20]
F., Stallard, N., Ivanova, A., Harper, C
Rosenberger, W. F., Stallard, N., Ivanova, A., Harper, C. N., and Ricks, M. L. (2001). Optimal adaptive designs for binary response trials. Biometrics , 57(3):909--913. https://doi.org/10.1111/j.0006-341X.2001.00909.x
2001 arXiv
-
[21]
B., Castagliola, P., and Moy, W
Teoh, J., Teoh, W., Khoo, M. B., Castagliola, P., and Moy, W. (2022). On designing an optimal sprt control chart with estimated process parameters under guaranteed in-control performance. Computers & Industrial Engineering , 174:108806. https://www.sciencedirect.com/science/ar...
2022
-
[22]
Wald, A. (1947). Sequential Analysis . Wiley Mathematical Statistics Series. John Wiley & Sons
1947
Reviewed August 3, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.