pith. sign in

arxiv: 2606.00839 · v1 · pith:AWY2A3Y2new · submitted 2026-05-30 · 📊 stat.ME

Sequential multiple testing with multiple hypotheses and prior information on the hypothesis configuration

Yiming Xing This is my paper

Pith reviewed 2026-06-28 18:14 UTC · model grok-4.3

classification 📊 stat.ME
keywords sequential multiple testingfamilywise error controlprior informationhypothesis configurationasymptotic optimalitydata streamsmultiple hypotheses
0
0 comments X

The pith

Prior information on hypothesis configurations enables a sequential multiple testing procedure that controls all familywise error rates while achieving asymptotic optimality in expected sample size.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper studies the sequential testing of marginal distributions across multiple independent data streams, where each stream has several candidate hypotheses and prior knowledge about the overall configuration of true hypotheses is available. It develops a procedure that incorporates this prior information to decide when to stop sampling. The method is shown to keep every type of familywise error probability below any chosen threshold, to make decisions by examining only the smallest relevant sets of alternative configurations, and to reach the smallest possible expected sample size among all procedures that meet the error bounds, in the limit as those bounds approach zero. Readers should care because the approach turns side information into fewer total observations while preserving reliability across many simultaneous tests.

Core claim

The designed procedure is reliable by controlling all types of familywise error probabilities below arbitrary user-specified levels, computationally efficient by focusing on minimal sets of alternative hypothesis configurations in making decisions, and asymptotically optimal by achieving the minimum expected sample size among all reliable procedures as the error levels go to zero.

What carries the argument

A sequential stopping rule that incorporates prior information on the unknown hypothesis configuration by restricting attention to minimal sets of alternative configurations.

If this is right

  • The procedure applies directly to settings with a known number of streams per hypothesis or with exclusive hypotheses.
  • All familywise error rates remain below the chosen thresholds for any true configuration.
  • The expected number of samples approaches the minimum possible among error-controlling rules as the error tolerances shrink to zero.
  • Decisions focus only on the smallest relevant sets of alternative configurations, keeping computation feasible.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same structure could be tested for robustness when streams exhibit mild dependence.
  • Real-time monitoring applications with streaming data may become feasible because only minimal configuration sets are examined at each step.
  • The asymptotic optimality result points to potential gains in multi-endpoint clinical trials or sensor networks where prior configuration knowledge is available.

Load-bearing premise

The data streams are independent and accurate prior information on the hypothesis configuration can be incorporated without invalidating the error control.

What would settle it

A numerical check or analytic counterexample in which, for vanishing error levels, the expected sample size of the procedure exceeds the information-theoretic lower bound while still controlling the errors, or in which the realized error rate exceeds the nominal level for some configuration consistent with the prior.

Figures

Figures reproduced from arXiv: 2606.00839 by Yiming Xing.

Figure 1
Figure 1. Figure 1: Expected sample sizes (left) and ratios of expected sample sizes to their asymptotic approximations (right) of the proposed procedure against equal [PITH_FULL_IMAGE:figures/full_fig_p013_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Negative base-10 logarithm of the error probabilities (left) and the relative errors of the estimation (right). The four curves, viewed from top to bottom on the far right of the first sub-figure, correspond to the proposed procedure in (17), the theoretical upper bound on its error probability in Theorem IV.1, the proposed procedure in (17) with the first three criteria neglected, and the proposed procedu… view at source ↗
read the original abstract

In this work, we study the problem of testing the marginal distributions of multiple independent, sequentially observed data streams, where for each stream there are multiple candidate hypotheses to select from, in the presence of prior information on the unknown hypothesis configuration. The goal is to understand the benefit of such information and to design a sequential testing procedure that effectively leverages it. We start with arbitrary prior information and specialize to concrete examples, including known number or known lower bound on the number of streams following each hypothesis, and the presence of exclusive hypotheses. The designed procedure is three-fold: (i) reliable, i.e., controlling all types of familywise error probabilities below arbitrary user-specified levels, (ii) computationally efficient, i.e., focusing on minimal sets of alternative hypothesis configurations in making decisions, and (iii) asymptotically optimal, i.e., achieving the minimum expected sample size among all reliable procedures asymptotically as the error levels go to zero. Numerical studies are presented for illustration.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

0 major / 3 minor

Summary. The manuscript develops a sequential multiple testing procedure for independent data streams where each stream has multiple candidate hypotheses, incorporating arbitrary prior information on the unknown hypothesis configuration (with specializations to known numbers or lower bounds on streams per hypothesis and exclusive hypotheses). The central claims are that the procedure (i) controls all types of familywise error probabilities at arbitrary user-specified levels, (ii) achieves computational efficiency by restricting attention to minimal sets of alternative configurations, and (iii) attains asymptotic optimality by achieving the minimal expected sample size among all reliable procedures as the error levels tend to zero. Numerical studies are included for illustration.

Significance. If the derivations and proofs hold under the stated assumptions of independent streams and accurate priors, the work provides a concrete framework for leveraging prior configuration information in sequential multiple testing without sacrificing error control or asymptotic efficiency. The emphasis on minimal alternative sets for computational tractability and the extension to multiple hypotheses per stream are practical strengths; the asymptotic optimality result would be a notable contribution if rigorously established.

minor comments (3)
  1. [Abstract / Introduction] The abstract and introduction should explicitly define or reference the precise familywise error probabilities being controlled (e.g., via a dedicated subsection or equation in §2) to make the reliability claim immediately verifiable.
  2. [Numerical studies] In the numerical studies, add details on the exact simulation parameters, number of replications, and how the prior information is encoded to improve reproducibility and allow readers to assess the efficiency gains.
  3. Notation for the minimal sets of alternative configurations should be introduced with a clear example early on to aid readability when the efficiency claim is discussed.

Simulated Author's Rebuttal

0 responses · 0 unresolved

We thank the referee for the careful reading and positive assessment of the manuscript, including the recommendation for minor revision. No specific major comments were raised in the report.

Circularity Check

0 steps flagged

No significant circularity detected

full rationale

The paper designs a new sequential multiple testing procedure that incorporates prior information on hypothesis configurations for independent data streams. It claims error control (all familywise error probabilities below user levels), computational efficiency (via minimal alternative sets), and asymptotic optimality (minimum expected sample size as errors approach zero). These rest on explicit assumptions (independence, accurate priors) and are presented as derived properties of the procedure, not as self-referential definitions or reductions to fitted inputs. No load-bearing self-citations, ansatzes smuggled via prior work, or renamings of known results are indicated in the provided text. The central claims have independent content relative to the inputs.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract provides no explicit free parameters, axioms, or invented entities; insufficient detail to populate the ledger.

pith-pipeline@v0.9.1-grok · 5686 in / 996 out tokens · 18436 ms · 2026-06-28T18:14:27.647111+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

169 extracted references · 16 canonical work pages

  1. [1]

    American Mathematical Society

    Mathematics Into Type . American Mathematical Society. [Online]. Available: https://www.ams.org/arc/styleguide/mit-2.pdf

  2. [2]

    T. W. Chaundy, P. R. Barrett and C. Batey, The Printing of Mathematics . London, U.K., Oxford Univ. Press, 1954

    1954

  3. [3]

    Mittelbach and M

    F. Mittelbach and M. Goossens, The Companion , 2nd ed. Boston, MA, USA: Pearson, 2004

    2004

  4. [4]

    Gr\"atzer, More Math Into LaTeX , New York, NY, USA: Springer, 2007

    G. Gr\"atzer, More Math Into LaTeX , New York, NY, USA: Springer, 2007

    2007

  5. [5]

    Letourneau and J

    M. Letourneau and J. W. Sharp, AMS-StyleGuide-online.pdf, American Mathematical Society, Providence, RI, USA, [Online]. Available: http://www.ams.org/arc/styleguide/index.html

  6. [6]

    Sira-Ramirez, ``On the sliding mode control of nonlinear systems,'' Syst

    H. Sira-Ramirez, ``On the sliding mode control of nonlinear systems,'' Syst. Control Lett., vol. 19, pp. 303--312, 1992

    1992

  7. [7]

    Levant, ``Exact differentiation of signals with unbounded higher derivatives,'' in Proc

    A. Levant, ``Exact differentiation of signals with unbounded higher derivatives,'' in Proc. 45th IEEE Conf. Decis. Control, San Diego, CA, USA, 2006, pp. 5585--5590. DOI: 10.1109/CDC.2006.377165

    work page doi:10.1109/cdc.2006.377165 2006

  8. [8]

    Fliess, C

    M. Fliess, C. Join, and H. Sira-Ramirez, ``Non-linear estimation is easy,'' Int. J. Model., Ident. Control, vol. 4, no. 1, pp. 12--27, 2008

    2008

  9. [9]

    Ortega, A

    R. Ortega, A. Astolfi, G. Bastin, and H. Rodriguez, ``Stabilization of food-chain systems using a port-controlled Hamiltonian description,'' in Proc. Amer. Control Conf., Chicago, IL, USA, 2000, pp. 2245--2249

    2000

  10. [10]

    Sequential Multiple Testing with Three Hypotheses and Known Number of Streams Following Each Hypothesis , year=

    Xing, Yiming and Chen, Yifan and Qu, Tianyi , booktitle=. Sequential Multiple Testing with Three Hypotheses and Known Number of Streams Following Each Hypothesis , year=

  11. [11]

    2026 , journal=

    Sequential Multiple Testing: A Second-Order Asymptotic Analysis , author=. 2026 , journal=

    2026

  12. [12]

    Efficient Best Arm Identification in Stochastic Bandits: Beyond -Optimality , year=

    Mukherjee, Arpan and Tajer, Ali , journal=. Efficient Best Arm Identification in Stochastic Bandits: Beyond -Optimality , year=

  13. [13]

    arXiv preprint arXiv:2509.14596 , year=

    Efficient Importance Sampling for Wrong Exit Probabilities over Combinatorially Many Rare Regions , author=. arXiv preprint arXiv:2509.14596 , year=

    arXiv

  14. [14]

    Sequential Anomaly Identification Under Sampling Constraints for Generalized Error Metrics , year=

    Tsopelakos, Aristomenis and Fellouris, Georgios , journal=. Sequential Anomaly Identification Under Sampling Constraints for Generalized Error Metrics , year=

  15. [15]

    Journal of Machine Learning Research , volume=

    Mixture martingales revisited with applications to sequential tests and confidence intervals , author=. Journal of Machine Learning Research , volume=

  16. [16]

    Sequential Analysis , volume =

    Aurélien Garivier and Emilie Kaufmann , title =. Sequential Analysis , volume =. 2021 , publisher =. doi:10.1080/07474946.2021.1847965 , URL =

    work page doi:10.1080/07474946.2021.1847965 2021

  17. [17]

    Adaptive 3-Stage Procedures for Multi-Hypothesis Testing , year=

    Xing, Yiming and Fellouris, Georgios , booktitle=. Adaptive 3-Stage Procedures for Multi-Hypothesis Testing , year=

  18. [18]

    2025 , eprint=

    Signal Detection under Composite Hypotheses with Identical Distributions for Signals and for Noises , author=. 2025 , eprint=

    2025

  19. [19]

    High-Dimensional Sequential Testing of Multiple Hypotheses , year=

    Xing, Yiming and Yan, Shen and Wang, Ziming , booktitle=. High-Dimensional Sequential Testing of Multiple Hypotheses , year=

  20. [20]

    and Tartakovsky, A.G

    Draglia, V.P. and Tartakovsky, A.G. and Veeravalli, V.V. , journal=. Multihypothesis sequential probability ratio tests .I. Asymptotic optimality , year=

  21. [21]

    Sequential multiple hypothesis testing and efficient fault detection-isolation in stochastic systems , year=

    Tze Leung Lai , journal=. Sequential multiple hypothesis testing and efficient fault detection-isolation in stochastic systems , year=

  22. [22]

    IEEE signal processing magazine , volume=

    Spectrum sensing for cognitive radio: State-of-the-art and recent advances , author=. IEEE signal processing magazine , volume=. 2012 , publisher=

    2012

  23. [23]

    A stable sequential multiple test for Koopman–Darmois family , journal =

    Shuaiyu Chen and Yan Li and Xiaolong Pu and Dongdong Xiang , keywords =. A stable sequential multiple test for Koopman–Darmois family , journal =. 2023 , issn =. doi:https://doi.org/10.1016/j.jspi.2023.01.006 , url =

    work page doi:10.1016/j.jspi.2023.01.006 2023

  24. [24]

    2024 IEEE International Symposium on Information Theory (ISIT) , pages=

    Joint sequential detection and isolation of anomalies under composite hypotheses , author=. 2024 IEEE International Symposium on Information Theory (ISIT) , pages=. 2024 , organization=

    2024

  25. [25]

    and Gongguo Tang and Nowak, Robert D

    Malloy, Matthew L. and Gongguo Tang and Nowak, Robert D. , booktitle=. Quickest search for a rare distribution , year=

  26. [26]

    52nd IEEE Conference on Decision and Control , pages=

    Unstructured sequential testing in sensor networks , author=. 52nd IEEE Conference on Decision and Control , pages=. 2013 , organization=

    2013

  27. [27]

    IEEE Transactions on Information Theory , volume=

    Quickest search over multiple sequences , author=. IEEE Transactions on Information Theory , volume=. 2011 , publisher=

    2011

  28. [28]

    , journal=

    Heydari, Javad and Tajer, Ali and Vincent Poor, H. , journal=. Quickest Linear Search over Correlated Sequences , year=

  29. [29]

    Advances in neural information processing systems , volume=

    Combinatorial pure exploration of multi-armed bandits , author=. Advances in neural information processing systems , volume=

  30. [30]

    1985 , issn =

    Asymptotically efficient adaptive allocation rules , journal =. 1985 , issn =. doi:https://doi.org/10.1016/0196-8858(85)90002-8 , url =

    work page doi:10.1016/0196-8858(85)90002-8 1985

  31. [31]

    29th Annual Conference on Learning Theory , pages =

    Optimal Best Arm Identification with Fixed Confidence , author =. 29th Annual Conference on Learning Theory , pages =. 2016 , editor =

    2016

  32. [32]

    Data Science and Pattern Recognition , volume=

    A survey of sequential pattern mining , author=. Data Science and Pattern Recognition , volume=

  33. [33]

    Nature Reviews Methods Primers , volume=

    Genome-wide association studies , author=. Nature Reviews Methods Primers , volume=. 2021 , publisher=

    2021

  34. [34]

    Expert systems With applications , volume=

    Financial fraud: a review of anomaly detection techniques and recent advances , author=. Expert systems With applications , volume=. 2022 , publisher=

    2022

  35. [35]

    Bolton and David J

    Richard J. Bolton and David J. Hand , title =. Statistical Science , number =. 2002 , doi =

    2002

  36. [36]

    Quickest Sequential Multiband Spectrum Sensing With Mixed Observations , year=

    Geng, Jun and Xu, Weiyu and Lai, Lifeng , journal=. Quickest Sequential Multiband Spectrum Sensing With Mixed Observations , year=

  37. [37]

    , journal=

    Xu, Qunzhi and Mei, Yajun and Moustakides, George V. , journal=. Optimum Multi-Stream Sequential Change-Point Detection With Sampling Control , year=

  38. [38]

    and Georgios Fellouris and Moustakides, George V

    Veeravalli, Venugopal V. and Georgios Fellouris and Moustakides, George V. Quickest Change Detection with Controlled Sensing. IEEE Journal on Selected Areas in Information Theory. 2024. doi:10.1109/JSAIT.2024.3362324

    work page doi:10.1109/jsait.2024.3362324 2024

  39. [39]

    Sequential analysis , volume=

    Sequential tests of multiple hypotheses controlling false discovery and nondiscovery rates , author=. Sequential analysis , volume=. 2020 , publisher=

    2020

  40. [40]

    Anomaly Search With Multiple Plays Under Delay and Switching Costs , year=

    Lambez, Tidhar and Cohen, Kobi , journal=. Anomaly Search With Multiple Plays Under Delay and Switching Costs , year=

  41. [41]

    Optimal Index Policies for Anomaly Localization in Resource-Constrained Cyber Systems , year=

    Cohen, Kobi and Zhao, Qing and Swami, Ananthram , journal=. Optimal Index Policies for Anomaly Localization in Resource-Constrained Cyber Systems , year=

  42. [42]

    Sequential Anomaly Detection Under a Nonlinear System Cost , year=

    Gurevich, Andrey and Cohen, Kobi and Zhao, Qing , journal=. Sequential Anomaly Detection Under a Nonlinear System Cost , year=

  43. [43]

    Sequential Test for the Lowest Mean: From Thompson to Murphy Sampling , url =

    Kaufmann, Emilie and Koolen, Wouter M and Garivier, Aur\'. Sequential Test for the Lowest Mean: From Thompson to Murphy Sampling , url =. Advances in Neural Information Processing Systems , editor =

  44. [44]

    The Journal of Machine Learning Research , volume=

    On the complexity of best-arm identification in multi-armed bandit models , author=. The Journal of Machine Learning Research , volume=. 2016 , publisher=

    2016

  45. [45]

    2024 , eprint=

    Sequential anomaly identification with observation control under generalized error metrics , author=. 2024 , eprint=

    2024

  46. [46]

    Sequential anomaly detection with observation control under a generalized error metric , year=

    Tsopelakos, Aristomenis and Fellouris, Georgios , booktitle=. Sequential anomaly detection with observation control under a generalized error metric , year=

  47. [47]

    IEEE Transactions on Information Theory , volume=

    Decentralized sequential hypothesis testing using asynchronous communication , author=. IEEE Transactions on Information Theory , volume=. 2010 , publisher=

    2010

  48. [48]

    The annals of statistics , volume=

    The positive false discovery rate: a Bayesian interpretation and the q-value , author=. The annals of statistics , volume=. 2003 , publisher=

    2003

  49. [49]

    2024 , eprint=

    Hypothesis testing with e-values , author=. 2024 , eprint=

    2024

  50. [50]

    Statistical Science , number =

    Aaditya Ramdas and Peter Gr. Statistical Science , number =. 2023 , doi =

    2023

  51. [51]

    2024 IEEE International Symposium on Information Theory (ISIT) , pages=

    Asymptotically optimal multistage tests for multihypothesis testing , author=. 2024 IEEE International Symposium on Information Theory (ISIT) , pages=. 2024 , organization=

    2024

  52. [52]

    2007 , publisher=

    Clinical decision support systems , author=. 2007 , publisher=

    2007

  53. [53]

    Artificial intelligence in medicine , volume=

    Reinforcement learning for intelligent healthcare applications: A survey , author=. Artificial intelligence in medicine , volume=. 2020 , publisher=

    2020

  54. [54]

    IEEE transactions on systems, man, and cybernetics , volume=

    Optimal search strategies in dynamic hypothesis testing , author=. IEEE transactions on systems, man, and cybernetics , volume=. 1995 , publisher=

    1995

  55. [55]

    IEEE Transactions on Signal Processing , volume=

    Sensor scheduling for energy-efficient target tracking in sensor networks , author=. IEEE Transactions on Signal Processing , volume=. 2011 , publisher=

    2011

  56. [56]

    Sequential Analysis , volume=

    Universal sequential outlier hypothesis testing , author=. Sequential Analysis , volume=. 2017 , publisher=

    2017

  57. [57]

    IEEE Transactions on Signal Processing , volume=

    Anomaly search over discrete composite hypotheses in hierarchical statistical models , author=. IEEE Transactions on Signal Processing , volume=. 2023 , publisher=

    2023

  58. [58]

    Conference on Learning Theory , pages=

    Optimal best arm identification with fixed confidence , author=. Conference on Learning Theory , pages=. 2016 , organization=

    2016

  59. [59]

    2020 , publisher=

    Bandit algorithms , author=. 2020 , publisher=

    2020

  60. [60]

    Advances in Neural Information Processing Systems , volume=

    Optimal best-arm identification in linear bandits , author=. Advances in Neural Information Processing Systems , volume=

  61. [61]

    Sequential Design of Experiments , urldate =

    Herman Chernoff , journal =. Sequential Design of Experiments , urldate =

  62. [62]

    Theory and applications of the sequential design of experiments, k-actions and infinitely many experiments: Part I--Theory , author=. Appl. Math. Statist. Lab., Stanford Univ., Stanford, CA, USA, Tech. Rep , volume=

  63. [63]

    The Annals of Mathematical Statistics , volume=

    The sequential design of experiments for infinitely many states of nature , author=. The Annals of Mathematical Statistics , volume=. 1961 , publisher=

    1961

  64. [64]

    Kiefer and J

    J. Kiefer and J. Sacks , title =. The Annals of Mathematical Statistics , number =. 1963 , doi =

    1963

  65. [65]

    The Annals of Statistics , pages=

    Second order efficiency in the sequential design of experiments , author=. The Annals of Statistics , pages=. 1984 , publisher=

    1984

  66. [66]

    The Annals of Probability , pages=

    A control problem arising in the sequential design of experiments , author=. The Annals of Probability , pages=. 1986 , publisher=

    1986

  67. [67]

    IEEE Transactions on automatic control , volume=

    Controlled sensing for multihypothesis testing , author=. IEEE Transactions on automatic control , volume=. 2013 , publisher=

    2013

  68. [68]

    Sequential Analysis , volume=

    Controlled sensing for sequential multihypothesis testing with controlled Markovian observations and non-uniform control cost , author=. Sequential Analysis , volume=. 2015 , publisher=

    2015

  69. [69]

    Sequential Analysis , volume=

    Sequential controlled sensing for composite multihypothesis testing , author=. Sequential Analysis , volume=. 2021 , publisher=

    2021

  70. [70]

    The Annals of Statistics , number =

    Mohammad Naghshvar and Tara Javidi , title =. The Annals of Statistics , number =. 2013 , doi =

    2013

  71. [71]

    The Annals of Probability , volume=

    The optional sampling theorem for martingales indexed by directed sets , author=. The Annals of Probability , volume=. 1980 , publisher=

    1980

  72. [72]

    Baum–Katz–Nagaev type results for martingales , journal =

    George Stoica , keywords =. Baum–Katz–Nagaev type results for martingales , journal =. 2007 , issn =. doi:https://doi.org/10.1016/j.jmaa.2007.03.012 , url =

    work page doi:10.1016/j.jmaa.2007.03.012 2007

  73. [73]

    MULTIPLE HYPOTHESIS TESTS CONTROLLING GENERALIZED ERROR RATES FOR SEQUENTIAL DATA , urldate =

    Jay Bartroff , journal =. MULTIPLE HYPOTHESIS TESTS CONTROLLING GENERALIZED ERROR RATES FOR SEQUENTIAL DATA , urldate =

  74. [74]

    Statistica Sinica , volume=

    Asymptotically optimal multistage tests for non-iid data , author=. Statistica Sinica , volume=

  75. [75]

    2006 , publisher=

    Measure theory and probability theory , author=. 2006 , publisher=

    2006

  76. [76]

    Statistical Methodology , volume=

    Sequential tests controlling generalized familywise error rates , author=. Statistical Methodology , volume=. 2015 , publisher=

    2015

  77. [77]

    Bernoulli , volume=

    Asymptotically optimal sequential multiple testing with asynchronous decisions , author=. Bernoulli , volume=. 2025 , publisher=

    2025

  78. [78]

    Logarithmically efficient simulation for misclassification probabilities in sequential multiple testing , year=

    Song, Yanglei and Fellouris, Georgios , booktitle=. Logarithmically efficient simulation for misclassification probabilities in sequential multiple testing , year=

  79. [79]

    Statistics in Medicine , volume =

    Ding, Yuxin and Markatou, Marianthi and Ball, Robert , title =. Statistics in Medicine , volume =. doi:https://doi.org/10.1002/sim.8447 , url =. https://onlinelibrary.wiley.com/doi/pdf/10.1002/sim.8447 , abstract =

    work page doi:10.1002/sim.8447

  80. [80]

    1999 , publisher=

    Group sequential methods with applications to clinical trials , author=. 1999 , publisher=

    1999

Showing first 80 references.