Multi-Armed Sequential Hypothesis Testing by Betting

Ian Waudby-Smith; Michael I. Jordan; Ricardo J. Sandoval

REVIEW 2 major objections 1 minor 1 cited by

Multi-Armed Sequential Hypothesis Testing by Betting

T0 review · 2 major / 1 minor · reviewed 2026-07-13 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Sequential tests that pick among multiple data sources can match an oracle that already knows which source gives the strongest evidence against a global null.

desk verdict Wrong manuscript was cached for 2603.17925; only the abstract of the multi-armed betting paper is available, so the oracle-matching claims cannot be audited. read the letter →

arxiv 2603.17925 v2 pith:ZPHKBYHF submitted 2026-03-18 stat.ME cs.LGmath.STstat.TH

Ricardo J. Sandoval , Ian Waudby-Smith , Michael I. Jordan This is my paper

classification stat.MEcs.LGmath.STstat.TH MSC 62L1062F0362C10

keywords sequentialtestingbybettinge-processesmulti-armedcompositenulllog-optimalityKellycapitalgrowthupperconfidencebound

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper studies sequential hypothesis testing by betting when the statistician must choose, at each step, which of several data sources (arms) to observe. The global null says that every arm is null; the alternative is that at least one arm is non-null. The authors ask for e-processes and sequential tests that remain as strong as if an oracle already knew which arm generates the most evidence against the null—even when several arms are actually non-null. They formalize this by extending log-optimality and expected rejection-time optimality to the multi-arm setting, and they prove matching lower and upper bounds for both criteria. The constructive upper bounds come from a modified upper-confidence-bound algorithm that tracks unobservable but estimable wealth-growth rewards, together with new nonasymptotic concentration inequalities for Kelly-optimal growth rates.

What carries the argument

A modified upper-confidence-bound-style algorithm for unobservable but sufficiently estimable rewards (the per-arm optimal wealth growth rates in Kelly’s sense), supported by new nonasymptotic concentration inequalities for those growth rates; the algorithm is the device that turns oracle-style optimality into an implementable multi-arm procedure.

What would settle it

Construct a multi-arm composite null/alternative pair in which the per-arm Kelly growth rates are not sufficiently estimable from the observed bets, and check whether any sequential e-process still attains the claimed oracle-matching log-optimality or expected rejection time; a systematic gap would refute the upper bounds.

Watch

Extended reading notes

Core claim

Even when several arms are non-null, there exist e-processes and sequential tests whose log-optimality and expected rejection-time performance match those of an oracle that knows which arm produces the most evidence against the composite global null, and these performance guarantees are tight: the paper supplies matching lower and upper bounds for both optimality notions.

Load-bearing premise

The analysis needs the best-arm wealth growth rates to be unobservable yet still estimable well enough that a UCB-like tracker can follow the strongest source of evidence; if that estimability fails, the oracle-matching upper bounds need not hold.

Editorial extensions

If this is right

Multi-arm sequential tests for a global null can be designed to ignore weak arms without paying an asymptotic price relative to an oracle that already knows the strongest arm.
Log-optimality and expected rejection-time optimality both admit matching lower and upper bounds once the multi-arm setting is formalized, so the desideratum is tight rather than merely aspirational.
The same concentration tools for Kelly growth rates can be reused wherever sequential wealth processes must be tracked without direct observation of the optimal growth rate.
Composite alternatives of the form “at least one arm is non-null” become amenable to e-process methods with explicit oracle-matching guarantees.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The estimability condition may fail in heavy-tailed or highly dependent arm models, suggesting a natural next boundary for the theory.
The multi-arm betting formulation is close to sequential adaptive experimental design; the same oracle-matching idea could organize dosage or treatment-arm selection under a global null of no effect.
Nonasymptotic Kelly-growth concentration may be of independent use in portfolio-style sequential inference beyond hypothesis testing.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The only usable material for Multi-Armed Sequential Hypothesis Testing by Betting is its abstract. The full text that was supplied is a completely different paper (clavicle CT legal age estimation, 2603.17926). So we cannot check theorems, the multi-arm log-optimality or expected-rejection-time definitions, the modified UCB for unobservable rewards, or the Kelly-growth concentration inequalities.

From the abstract alone the contribution looks real and well-scoped: sequential testing by betting with multiple arms, a composite global null that all arms are null, and an explicit oracle-matching desideratum—even when several arms are non-null, match the e-process and expected rejection time of an oracle that knows the best arm. They claim matching lower and upper bounds for both optimality notions, plus nonasymptotic concentration for Kelly optimal wealth growth rates that may stand alone. That package sits cleanly in anytime-valid inference and multi-stream monitoring (multi-dose trials, etc.). The framing is not circular; the estimability condition for the rewards is stated up front as the key technical device.

Soft spots are almost entirely about missing evidence, not about visible flaws. We cannot verify whether the estimability condition holds for the relevant wealth-growth rates, whether the UCB-like algorithm actually tracks the best arm, or whether the bounds are tight under realistic composite alternatives. Those are the load-bearing pieces, and they are simply not here. Novelty and significance look solid on the abstract’s terms; soundness is uncheckable.

This is for people who already work on e-processes, sequential testing by betting, or adaptive multi-arm monitoring. It is not a general-audience paper. A serious editor should send the correct manuscript to peer review once it is available—the abstract is important enough and formally ambitious enough to deserve referee time. Until the right PDF is in hand, do not cite and do not put it in reading group. Get the actual paper and re-evaluate.

Referee Report

2 major / 1 minor

Summary. The manuscript claims to study multi-armed sequential hypothesis testing by betting: at each step the statistician selects one of several data sources (arms) and seeks e-processes and sequential tests that reject a composite global null (all arms null) in favor of a composite alternative (at least one arm non-null). The central optimality desideratum is that performance should match an oracle that knows which arm generates the most evidence against the null, even when several arms are non-null. The abstract asserts matching lower and upper bounds for generalized multi-arm log-optimality and expected rejection-time optimality, obtained via a modified UCB-like algorithm for unobservable but sufficiently estimable rewards, together with nonasymptotic concentration inequalities for Kelly optimal wealth growth rates.

Significance. If the claimed matching bounds and concentration inequalities hold under the stated estimability condition, the work would supply a clean oracle-matching theory for multi-source sequential testing by betting and would extend classical single-stream e-process optimality to an arm-selection setting of clear practical relevance (e.g., multi-dosage trials). The nonasymptotic Kelly-growth concentration bounds would be of independent interest. However, the supplied full-text body is an unrelated manuscript on clavicle CT legal age estimation (arXiv 2603.17926), so none of the theorems, definitions, algorithms, or inequalities can be verified from the material provided for review.

major comments (2)

The full manuscript text supplied for review is a completely different paper (clavicle CT legal-age estimation, arXiv 2603.17926). No definitions of multi-arm log-optimality or expected rejection-time optimality, no statement of the estimability condition, no description of the modified UCB algorithm, and no concentration inequalities for Kelly growth rates appear. The abstract's central claims of matching lower/upper bounds are therefore un-auditable; the load-bearing technical content of arXiv 2603.17925 is missing.
Because the correct manuscript is absent, it is impossible to check whether the 'unobservable but sufficiently estimable rewards' assumption (the key technical device named in the abstract) is stated with sufficient precision, holds for the intended hypothesis classes, or is used correctly in the upper-bound constructions. Without that material the oracle-matching claim cannot be assessed for correctness.

minor comments (1)

The abstract alone is well written and the informal optimality desideratum is clear, but without the body no further presentation issues can be evaluated.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity identifiable: supplied full text is a mismatched paper (clavicle CT age estimation) unrelated to the multi-armed betting abstract, so no derivation chain, optimality bounds, or estimability claims can be inspected or reduced.

full rationale

The abstract of arXiv:2603.17925 posits an external oracle-matching desideratum for multi-arm e-processes and sequential tests (log-optimality and expected rejection time) with matching lower/upper bounds, plus a modified UCB device for estimable rewards and Kelly-growth concentration inequalities. None of those objects, definitions, theorems, algorithms, or proofs appear in the provided full manuscript text, which is instead the unrelated clavicle-CT legal-age paper (arXiv:2603.17926). Because no equations, constructions, or self-citations from the claimed derivation chain are present, no reduction of a prediction or first-principles claim to its own inputs can be exhibited. Per the hard rules, circularity is only flagged when a concrete quote-and-reduction is possible; the honest finding is therefore score 0 with empty steps. (The medical paper itself is a standard empirical pipeline with held-out MAE, ablations, and reimplemented baselines; it likewise exhibits no definitional loops or fitted-as-prediction circularity, but that is irrelevant to the queried betting paper.)

Assumptions & free parameters 0 free parameters · 4 assumptions · 2 invented entities

Abstract-only review of a theoretical sequential-testing paper. Load-bearing structure is the composite global null that all arms are null, a composite alternative that at least one is non-null, the testing-by-betting / e-process framework, and the estimability condition needed for the modified UCB on unobservable rewards. No numerical free parameters are stated. Invented technical objects are the multi-arm optimality notions and the modified UCB for estimable rewards; independent evidence outside this paper is not assessable from the abstract.

assumptions (4)

domain assumption Composite global null P: every arm is null in a specified sense; alternative Q: at least one arm is non-null.
Defines the multi-armed testing problem in the abstract; all optimality claims are relative to this hypothesis structure.
domain assumption Testing-by-betting / e-process framework for sequential evidence accumulation and rejection.
The paper positions itself as a variant of sequential testing by betting; wealth growth and e-processes are the performance language.
ad hoc to paper Arm rewards (optimal wealth growth rates) are unobservable but sufficiently estimable for a modified UCB-style algorithm.
Abstract identifies this as the key technical device for the optimality analysis; without it the oracle-matching construction may not go through.
standard math Kelly (1956) optimal wealth growth rates are the right log-optimality benchmark.
Abstract explicitly ties concentration inequalities and optimality to Kelly’s growth-rate notion.

invented entities (2)

Multi-arm log-optimality and expected rejection-time optimality (oracle-matching desideratum)
purpose: Formalize performance as strong as knowing the arm that generates the most evidence against P.
Abstract presents these generalized optimality notions as the paper’s formal target; independent prior codification not verifiable here.
Modified UCB-like algorithm for unobservable but estimable rewards
purpose: Select arms online so that wealth growth tracks the best arm without observing true rewards directly.
Described as the key technical device enabling the upper bounds; existence and properties not checkable without the manuscript.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Multi-Armed Sequential Hypothesis Testing by Betting." pith.science (2026). https://pith.science/paper/ZPHKBYHF

@misc{pith2026260317925,
  author       = {Pith},
  title        = {Pith review of: Multi-Armed Sequential Hypothesis Testing by Betting},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZPHKBYHF}},
  note         = {Machine review of arXiv:2603.17925}
}

abstract

We consider a variant of sequential testing by betting where, at each time step, the statistician is presented with multiple data sources (arms) and obtains data by choosing one of the arms. We consider the composite global null hypothesis $\mathscr{P}$ that all arms are null in a certain sense (e.g. all dosages of a treatment are ineffective) and we are interested in rejecting $\mathscr{P}$ in favor of a composite alternative $\mathscr{Q}$ where at least one arm is non-null (e.g. there exists an effective treatment dosage). We posit an optimality desideratum that we describe informally as follows: even if several arms are non-null, we seek $e$-processes and sequential tests whose performance are as strong as the ones that have oracle knowledge about which arm generates the most evidence against $\mathscr{P}$. Formally, we generalize notions of log-optimality and expected rejection time optimality to more than one arm, obtaining matching lower and upper bounds for both. A key technical device in this optimality analysis is a modified upper-confidence-bound-like algorithm for unobservable but sufficiently "estimable" rewards. In the design of this algorithm, we derive nonasymptotic concentration inequalities for optimal wealth growth rates in the sense of Kelly [1956]. These may be of independent interest.

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Mitigating the Winner's Curse While Controlling Multiplicity: e-Process Methods for Anytime-Valid Inference in Dose-Ranging Trials
stat.ME 2026-07 conditional novelty 6.0 of 10

The paper derives an anytime-valid global test for dose-ranging trials that subtracts a predictable 'selection charge' from the running maximum dose effect to correct winner's curse bias while controlling Type I error.

Pith tools

Reviewed July 13, 2026 · model on record in the stance chip above.