REVIEW 4 major objections 5 minor 26 references
Asymptotics in Multiple Hypotheses Testing under Dependence: beyond Normality
T0 review · 4 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash
Pith's one-line read This paper establishes that under positive equicorrelation and superexponentially light-tailed margins, the Bonferroni FWER tends to zero as the number of hypotheses grows, and extends this to all step-down procedures and positively…
desk verdict The zero-FWER theorem for superexponential equicorrelated statistics is correct; the positive-correlation extension and the positive-limit example are not, as written. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The argument is carried by an explicit upper bound on the Bonferroni FWER (Theorem 3.1): for every $d\in(0,1)$, $\mathrm{FWER}_{\mathrm{Bon}}(n,\alpha,\rho) \le 1 - G(0)F^n(c_{\mathrm{Bon}}/\sqrt{1-\rho}) - [G(c_{\mathrm{Bon}}(1-d)/\sqrt{\rho}) - G(0)]F^n(d\,c_{\mathrm{Bon}}/\sqrt{1-\rho})$, obtained by conditioning on the common factor $U$ and splitting the integral at 0. Lemma 3.1 then evaluates $\log\lim_n F^n(c_{\mathrm{Bon}}/\sqrt{1-\rho})$ as $-\alpha$ times the limit of a density ratio $f(c_{\mathrm{Bon}}/\sqrt{1-\rho}) / \int f((c_{\mathrm{Bon}}-\sqrt{\rho}u)/\sqrt{1-\rho})g(u)\,du$. The superexponential tail condition (ii), $f(x)/f(x-b)\to 0$ for all $b>0$, forces this ratio to zero, so $F^n(\cdot)\to 1$ and the upper bound collapses to zero. The same tail condition is the engine behind the elliptical and step-down extensions, via the quadrant-probability comparison of DasGupta et al. and the cut-off optimality of Holm's procedure.
What would settle it
Simulate the equicorrelated model with $Z_i$ i.i.d. standard exponential (standardized to mean 0, variance 1) and $U\sim N(0,1)$, set $\alpha=0.05$, $\rho=0.5$, and estimate $\mathrm{FWER}_{\mathrm{Bon}}$ by Monte Carlo for $n=10^3,10^4,10^5,10^6$. Since $f(x)/f(x-b)=e^{-b}$ does not tend to 0, condition (ii) fails; if the estimated FWER stays above, say, $10^{-3}$ rather than trending to 0, the paper's claim that a wide class of exponential-family distributions satisfies its conditions is refuted, and the zero-limit theorem is not applicable to Laplace or exponential margins.
Extended reading notes
Core claim
The central claim is Theorem 3.3 (with Theorem 3.2 for the global null): in the equicorrelated model $X_i = \sqrt{1-\rho}Z_i + \sqrt{\rho}U$, where $Z_i \sim F(\mu_i/\sqrt{1-\rho},1)$ are independent and $U\sim G(0,1)$ is common, if $G(a)<1$ for some $a>0$ and the density $f$ of $F$ is non-increasing on $\mathbb{R}_+$ with $f(x)/f(x-b)\to 0$ for every $b>0$, then $\lim_{n\to\infty} \mathrm{FWER}_{\mathrm{Bon}}(n,\alpha,\rho)=0$ for any $\alpha\in(0,1)$, $\rho\in(0,1)$, and any configuration of true and false null hypotheses. The proof rests on an upper bound obtained by conditioning on the common factor $U$, and on a lemma showing that under the tail condition the bound's key factor $F^n(c_{\mathrm{Bon}}/\sqrt{1-\rho})$ converges to 1. The same mechanism extends Theorem 4.1 to elliptical densities with $\liminf \rho_{ij}>0$, and Theorem 5.3 to all step-down FWER-controlling procedures: the probability of rejecting at least one hypothesis goes to zero. The positive-limit construction (Theorem 3.4) uses a Pareto-type density with variance 1, for which $F^n(\sqrt{n/(\gamma(1-\rho))})$ stays below 1, yielding $\liminf \mathrm{FWER}>0$.
Load-bearing premise
The whole zero-limit result hinges on the assumption that the density of the idiosyncratic component has superexponentially light tails: $f$ must be non-increasing on $\mathbb{R}_+$ and satisfy $f(x)/f(x-b)\to 0$ for every fixed shift $b>0$. This excludes exponential, gamma, and polynomial-tailed distributions; if it fails, the lemma that drives $F^n(\cdot)\to 1$ no longer applies, and the FWER limit is not known to be zero.
Editorial extensions
If this is right
- In the equicorrelated normal case, the known zero-limit results are recovered as special cases, with the new proof covering arbitrary non-Gaussian light-tailed margins.
- Any step-down FWER-controlling procedure, including Holm's method, makes no rejection (of true or false nulls) in the limit: both FWER and AnyPwr vanish when $\liminf \rho_{ij}>0$ and the tail condition holds.
- The upper bound of Theorem 3.1 is asymptotically sharp: it converges to zero exactly when the tail condition holds, and the Pareto example shows the limit can be strictly positive when it fails.
- With positive correlation, the Bonferroni threshold becomes increasingly conservative as $n$ grows: the probability of even one false rejection goes to zero, so power, not error control, becomes the binding constraint in large-scale testing.
Reading between the lines
- If the result is right, the practical regime where Bonferroni is 'too conservative' under positive dependence is even broader than the Gaussian case: any light-tailed data with a common factor will have vanishing false-rejection probability as $n$ grows, so researchers can safely use Bonferroni to suppress false positives even at enormous hypothesis counts.
- The tail condition (ii) appears to be a sharp phase boundary: it separates distributions for which the proof gives a zero limit from those, like Pareto or exponential, where the limit may be positive. A natural next step is to map the exact critical tail index for intermediate distributions such as Weibull with shape between 0 and 1.
- The same conditioning argument could be re-run for two-sided tests or for false-discovery-rate procedures; the zero-limit phenomenon may extend to FDR control, where the analogue would be that the FDP collapses to zero under positive equicorrelation and light tails.
- The elliptically contoured extension suggests the result is not an artifact of the additive common-factor structure; any positive dependence that keeps all pairwise correlations bounded below by $\delta>0$ may inherit the zero limit, so the phenomenon is robust to the precise form of dependence.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper studies the asymptotic family-wise error rate (FWER) of Bonferroni and step-down multiple testing procedures for correlated test statistics under non-normal distributions. In the equicorrelated factor model X_i = sqrt(1-ρ)Z_i + sqrt(ρ)U, the paper proves an upper bound for the Bonferroni FWER (Theorem 3.1), derives a zero limit under a superexponential tail condition on the density f of the idiosyncratic components (Theorems 3.2–3.3), and attempts to construct a Pareto example with a strictly positive FWER limit (Section 3.2, Theorem 3.4). It then claims an extension to general positively correlated elliptically contoured distributions (Theorem 4.1) and to step-down procedures (Theorem 5.3).
Significance. If the central zero-limit theorem is correct, it is a genuine generalization of the known equicorrelated-normal results to a class of light-tailed distributions, and the proof in Section 3 is largely self-contained, built on conditioning and standard inequalities rather than on fitted parameters. The claimed extensions to general positive correlation and to all step-down procedures would considerably broaden the applicability of the results. However, as submitted, the positive-limit example and the two advertised extensions contain load-bearing gaps. The core zero-limit result for the equicorrelated factor model appears defensible after correcting typos, so the paper has real potential, but it is not yet in publishable form.
major comments (4)
- [Section 3.2, Theorem 3.4 and the Pareto example] The proof of the positive-limit example is incorrect. The displayed identity F^n( sqrt(n/(γ(1-ρ))) ) = (1 - d/n^{1+δ/2})^n is followed by the lower bound (1 - d/n^{1+δ/2})^{n^{1+δ/2}} -> e^{-d}. But the left-hand side actually tends to 1, because n/n^{1+δ/2} -> 0; a lower bound by e^{-d} does not establish that the limit of F^n at this threshold is below 1, which is what the positive-FWER conclusion requires. Moreover, condition (ii) of Theorem 3.4 cannot hold for any F with finite second moment, which the model assumes: for c_n = sqrt(n/(γ(1-ρ))), finite variance implies c_n^2(1-F(c_n)) -> 0, hence n(1-F(c_n)) -> 0 and F^n(c_n) -> 1. The claimed Pareto phenomenon is in fact true, but it must be shown using the actual quantile c_Bon: one obtains F^n(c_Bon/sqrt(1-ρ)) = (1 - α(1-ρ)^{1+δ/2}/n)^n -> exp(-α(1-ρ)^{1+δ/2}) < 1. This section therefore needs to be rewritten with a correct calculation and a corrected sufficient condition.
- [Section 4, proof of Theorem 4.1] The assertion that 'Since lim inf ρ_ij = δ > 0, we also have n - |M_n| is finite' is false under the standard double-array reading of liminf. For example, set ρ_{1j} = 1/2 for all j ≥ 2 and ρ_{ij} = 1 for all i,j ≥ 2. Then every pair with both indices tending to infinity has correlation 1, so the liminf is 1, but for every n the set M_n = {i : ρ_{ij} ≥ 1 for all j ≠ i} is empty; hence n - |M_n| = n is unbounded. The reduction to Theorem 3.3 therefore fails at a load-bearing step. The proof needs a different construction of a large subset with all pairwise correlations bounded below by some δ' > 0, rather than the set M_n as currently defined.
- [Section 4, Theorems 4.1 and 5.3] The statements assume an elliptically contoured joint density but formulate the sufficient conditions in terms of densities f and g from the factor model X_i = sqrt(1-ρ)Z_i + sqrt(ρ)U of Section 2.1, and the proofs invoke Theorem 3.3, which is proved only for that factor model. An equicorrelated elliptical distribution need not admit the representation with independent Z_i and U outside the normal case, so the meaning of f and g in Theorems 4.1 and 5.3 is not defined and the reduction to Theorem 3.3 is incomplete. The authors need either to prove the zero-limit result directly for equicorrelated elliptical laws or to state explicit conditions on the radial density or on the marginal distribution.
- [Section 5, proof of Theorem 5.3] The proof uses the inequality P_{Σ_n}(X_(n) ≥ c_Bon) ≤ P_{Γ_n(δ)}(X_(n) ≥ c_Bon), which requires ρ_{ij} ≥ δ for all i,j. The theorem only assumes liminf ρ_{ij} = δ, so the same issue as in Theorem 4.1 arises. In addition, the final step asserts that E_U[F^n((c_Bon - sqrt(ρ)U - μ*)/sqrt(1-ρ))] -> 1 with the explanation that the proof is exactly similar to Theorem 2 of Dey and Bhandari (2023b), but no argument is supplied for why the fixed shift μ* is compatible with condition (ii). Since this is the step that transfers the equicorrelated zero-limit to arbitrary configurations with bounded means, the details need to be written out.
minor comments (5)
- [Theorems 3.2, 3.3, 3.4] The statements read 'lim_{n→0}' in several places; this should be 'lim_{n→∞}'.
- [Proof of Theorem 3.1] In the displayed chain of inequalities, the integration upper limit 'c_Bon · (1-α)/sqrt(ρ)' should be 'c_Bon · (1-d)/sqrt(ρ)', and the expression 'c_Bon · 1-d/sqrt(ρ)' is missing parentheses; it should be 'c_Bon(1-d)/sqrt(ρ)'.
- [Section 3.2] The constant d in the Pareto calculation is not defined correctly: the expression (1 - d/n^{1+δ/2})^n would require d = η^{2+δ}(γ(1-ρ))^{1+δ/2}, not d = η^{1+δ/2}. This is part of the larger algebraic error described in the major comments.
- [Theorem 3.2, condition (ii)] The claim that condition (ii) is satisfied by 'a wide class of distributions from the exponential family' is not demonstrated and is false for the standard natural exponential family on R+, where f(x)/f(x-b) is typically a constant. The authors should state precisely which parametric families satisfy the condition.
- [Section 4] The notation liminf ρ_{ij} for a triangular array of correlations is ambiguous; the authors should define whether it means the limit over pairs with both indices tending to infinity or the limit of the minimum off-diagonal entry of Σ_n.
Circularity Check
No significant circularity: the core limiting results are derived in-text from conditioning, l'Hopital's rule, and external inequalities; self-citations are pointers or proof analogies, not definitional reductions.
full rationale
The derivation chain is essentially self-contained. Theorem 3.1 is proved from the representation X_i = sqrt(1-rho)Z_i + sqrt(rho)U by conditioning on U and using monotonicity of F. Lemma 3.1 computes the limit of F^n(c_Bon/sqrt(1-rho)) via l'Hopital's rule and the convolution formula for f-star, so the limit is not assumed. Theorem 3.2 combines Lemma 3.1 with the explicit tail conditions on f and g; Theorem 3.3 extends to arbitrary null configurations by a monotonicity argument on the true-null indices. No parameter is fitted to data, and no theorem reduces by construction to an input quantity. The positive-limit result in Theorem 3.4 uses Cantelli's inequality and an explicit Pareto example, so it is independent of the zero-limit theorem. Section 5 uses the external Gordon-Salzman optimality theorem and reduces any step-down procedure to Holm, an argument independent of the authors' prior work. One remaining convergence assertion in the proof of Theorem 5.3 is stated to be 'exactly similar' to Theorem 2 of Dey and Bhandari (2023b); this is an omitted proof and a self-citation, but it is an analogy to a prior proof rather than a reduction of the target claim to a definitionally identical or fitted quantity, so it does not constitute circularity under the stated standard. Self-citations to Dey (2022, 2024a, 2024b) and Dey and Bhandari (2023b) are pointers to prior normal-case results and are not load-bearing for the principal in-text derivations. For completeness, a separate non-circularity concern is that the proof of Theorem 4.1 asserts that liminf rho_ij = delta > 0 implies n - |M_n| is finite, which is false for infinite arrays such as rho_1j = 1/2 and rho_ij = 1 for i,j >= 2; this is a correctness gap, not a circularity gap.
Assumptions & free parameters
assumptions (5)
- domain assumption The test statistics admit the representation Xi = sqrt(1-rho) Zi + sqrt(rho) U with Zi independent and U independent of the Zi's, each having densities f and g.
- domain assumption The density f is non-increasing on R+ and satisfies lim_{x to infinity} f(x)/f(x-b)=0 for every b>0 (Theorem 3.2 condition ii).
- domain assumption The joint density of the test statistics is elliptically contoured (Section 4, Theorem 4.1).
- standard math Cantelli's inequality, the comparison inequality for elliptically contoured quadrant probabilities (DasGupta et al. 1972), and the Gordon-Salzman bound on step-down critical values are invoked.
- standard math L'Hopital's rule is applied to the sequence n log F(t cBon) in Lemma 3.1.
Cite this review
Pith. "Pith review of Asymptotics in Multiple Hypotheses Testing under Dependence: beyond Normality." pith.science (2026). https://pith.science/paper/JML4SVQX
@misc{pith2026241112119,
author = {Pith},
title = {Pith review of: Asymptotics in Multiple Hypotheses Testing under Dependence: beyond Normality},
year = {2026},
howpublished = {\url{https://pith.science/paper/JML4SVQX}},
note = {Machine review of arXiv:2411.12119}
}
read the original abstract
Correlated observations are ubiquitous phenomena in a plethora of scientific avenues. Tackling this dependence among test statistics has been one of the pertinent problems in simultaneous inference. However, very little literature exists that elucidates the effect of correlation on different testing procedures under general distributional assumptions. In this work, we address this gap in a unified way by considering the multiple testing problem under a general correlated framework. We establish an upper bound on the family-wise error rate(FWER) of Bonferroni's procedure for equicorrelated test statistics. Consequently, we find that for a quite general class of distributions, Bonferroni FWER asymptotically tends to zero when the number of hypotheses approaches infinity. We extend this result to general positively correlated elliptically contoured setups. We also present examples of distributions for which Bonferroni FWER has a strictly positive limit under equicorrelation. We extend the limiting zero results to the class of step-down procedures under quite general correlated setups. Specifically, the probability of rejecting at least one hypothesis approaches zero asymptotically for any step-down procedure. The results obtained in this work generalize existing results for correlated Normal test statistics and facilitate new insights into the performances of multiple testing procedures under dependence.
Reference graph
Works this paper leans on
-
[1]
F. Abramovich, Y. Benjamini, D. L. Donoho, and I. M. Johnstone. Special invited lecture: Adapting to unknown sparsity by controlling the false discovery rate. The Annals of Statistics, 34 0 (2): 0 584--653, 2006. URL http://www.jstor.org/stable/25463431
-
[2]
Y. Benjamini and W. Liu. A distribution-free multiple test procedure that controls the false discovery rate . Technical report, Tel Aviv University, Technical report RP-SOR-99-3., 1999 a
work page 1999
-
[3]
Y. Benjamini and W. Liu. A step-down multiple hypotheses testing procedure that controls the false discovery rate under independence. Journal of Statistical Planning and Inference, 82 0 (1): 0 163--170, 1999 b . URL https://www.sciencedirect.com/science/article/pii/S0378375899000403
work page 1999
-
[4]
M. Bogdan, A. Chakrabarti, F. Frommlet, and J. K. Ghosh. Asymptotic Bayes-optimality under sparsity of some multiple testing procedures . The Annals of Statistics, 39 0 (3): 0 1551 -- 1579, 2011. URL https://doi.org/10.1214/10-AOS869
-
[5]
Z. Chi, A. Ramdas, and R. Wang. Multiple testing under negative dependence, 2024. URL https://arxiv.org/abs/2212.09706
arXiv 2024
- [6]
-
[7]
S. DasGupta, M. L. Eaton, I. Olkin, M. Perlman, L. J. Savage, and M. Sobel. Inequalitites on the probability content of convex regions for elliptically contoured distributions. In Proceedings of the Sixth Berkeley Symposium on Mathematical Statistics and Probability, Vol. II: Probability theory, pp 241–265, University of California Press, Berkeley, CA, 1972
work page 1972
-
[8]
S. Delattre and E. Roquain. On the false discovery proportion convergence under gaussian equi-correlation. Statistics & Probability Letters, 81 0 (1): 0 111--115, 2011. URL https://www.sciencedirect.com/science/article/pii/S0167715210002750
work page 2011
Show all 26 references
-
[9]
M. Dey. Behavior of fwer in normal distributions. Communications in Statistics - Theory and Methods, 0 0 (0): 0 1--15, 2022. URL https://doi.org/10.1080/03610926.2022.2150826
2022
-
[10]
M. Dey. Some Contributions to Multiple Hypotheses Testing under Dependence. PhD thesis, Indian Statistical Institute, 2024 a
2024
-
[11]
M. Dey. On limiting behaviors of stepwise multiple testing procedures. Statistical Papers, 2024 b . URL https://doi.org/10.1007/s00362-024-01613-6
2024 doi
-
[12]
Dey and S
M. Dey and S. K. Bhandari. Bounds on generalized family-wise error rates for normal distributions. Statistical Papers, 2023 a . URL https://doi.org/10.1007/s00362-023-01487-0
2023 doi
-
[13]
Dey and S.K
M. Dey and S.K. Bhandari. Fwer goes to zero for correlated normal. Statistics & Probability Letters, 193: 0 109700, 2023 b . URL https://www.sciencedirect.com/science/article/pii/S0167715222002139
2023
-
[14]
Dickhaus
T. Dickhaus. False Discovery Rate and Asymptotics, PhD Thesis. Heinrich-Heine-University Düsseldorf, 2008
2008
-
[15]
Donoho and J
D. Donoho and J. Jin. Higher criticism for detecting sparse heterogeneous mixtures. The Annals of Statistics, 32 0 (3): 0 962--994, 2004. URL https://doi.org/10.1214/009053604000000265
2004 doi
-
[16]
Dudoit and Mark J
S. Dudoit and Mark J. Laan. Multiple Testing Procedures with Applications to Genomics. Springer Series in Statistics. Springer, 2008
2008
-
[17]
Finner and M
H. Finner and M. Roters. Asymptotic sharpness of product-type inequalities for maxima of random variables with applications in multiple comparisons. Journal of Statistical Planning and Inference, 98 0 (1): 0 39--56, 2001. URL https://www.sciencedirect.com/science/article/pii/S...
2001
-
[18]
Finner, T
H. Finner, T. Dickhaus, and M. Roters. Dependency and false discovery rate: Asymptotics . The Annals of Statistics, 35 0 (4): 0 1432--1455, 2007. URL https://www.jstor.org/stable/25464546
2007
-
[19]
Gasparin, R
M. Gasparin, R. Wang, and A. Ramdas. Combining exchangeable p-values, 2024. URL https://arxiv.org/abs/2404.03484
2024 arXiv
-
[20]
A. Y. Gordon and P. Salzman. Optimality of the H olm procedure among general step-down multiple testing procedures. Statistics & Probability Letters, 78 0 (13): 0 1878--1884, 2008. URL https://www.sciencedirect.com/science/article/pii/S0167715208000540
2008
-
[21]
S. Holm. A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics, 6 0 (2): 0 65--70, 1979. URL http://www.jstor.org/stable/4615733
1979
-
[22]
M. A. Proschan and P. A. Shaw. Asymptotics of B onferroni for dependent normal test statistics. Statistics & Probability Letters, 81 0 (7): 0 739--748, 2011. URL https://www.sciencedirect.com/science/article/pii/S0167715210003329. Statistics in Biological and Medical Sciences
2011
-
[23]
Roy and S
R. Roy and S. K. Bhandari. Asymptotic bayes’ optimality under sparsity for exchangeable dependent multivariate normal test statistics. Statistics & Probability Letters, 207: 0 110030, 2024. URL https://www.sciencedirect.com/science/article/pii/S0167715223002535
2024
-
[24]
S. K. Sarkar. On controlling the false discovery rate in multiple testing of the means of correlated normals against two-sided alternatives, 2023. URL https://arxiv.org/abs/2304.05261
2023 arXiv
-
[25]
P. H. Westfall, R. D. Tobias, D. Rom, R. D. Wolfinger, and Y. Hochberg. Multiple Comparisons and Multiple Testing Using the SAS System. SAS Institute, Inc., 1999
1999
-
[26]
Xu and A
Z. Xu and A. Ramdas. More powerful multiple testing under dependence via randomization, 2024. URL https://arxiv.org/abs/2305.11126
2024
Reviewed August 12, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.