Pith. sign in

REVIEW 4 major objections 5 minor 21 references

Bound on FWER for correlated normal distribution

T0 review · 4 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper proves that, for equicorrelated normal test statistics, the Bonferroni family-wise error rate is asymptotically bounded by alpha(1-rho).

desk verdict A simple and probably true bound for FWER under equicorrelated normality, but the proof has a real derivative error and sketched lemmas; worth a serious referee, not acceptance as-is. read the letter →

arxiv 1908.02193 v2 pith:6ZHP5I3Q submitted 2019-08-06 math.ST stat.TH

Nabaneet Das , Subir K.Bhandari This is my paper · ORCID
classification math.STstat.TH MSC 62F0362E2062H20
keywords family-wiseerrorrateBonferroniprocedureequicorrelatednormalmultipletestingconvexityasymptoticboundcorrelationsimultaneoushypothesis
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that when normally distributed test statistics are positively equicorrelated, the family-wise error rate of a Bonferroni multiple-testing procedure shrinks as the correlation grows, and is asymptotically bounded by alpha(1-rho) rather than alpha. If true, this means the classical Bonferroni correction is conservative under positive dependence, and that a correlation-aware correction is needed to avoid over-correction. The argument runs through the probability H(rho) that no null is rejected: showing H is asymptotically concave in rho makes FWER convex, so it is controlled by the straight line between the independence and perfect-dependence endpoints.

What carries the argument

The central object is H(rho), the probability that all n standardized test statistics stay below the common cutoff c. The paper rewrites equicorrelated normals as X_i = $\theta$ + Z_i, where $\theta$ is a common N(0,rho) component independent of the i.i.d. N(0,1-rho) noise Z_i, so that H(rho) = E[Phi^n((c + $\sqrt$(rho)Z)/$\sqrt$(1-rho))]. Differentiating twice under the integral, it shows the terms in H''(rho) vanish except on the tail region alpha_1 < 1/n, where alpha_1 = Phi(-d) is the single-test tail probability, and that the remaining term is non-positive. This asymptotic concavity of H is the load-bearing identity: it converts the two endpoint values into a linear upper bound on FWER.

What would settle it

Numerically integrate H(rho) = E[Phi^n((c + $\sqrt$(rho)Z)/$\sqrt$(1-rho))] for, say, n=$10^{4}$ and $\alpha$=0.05 over a fine grid of rho in [0,1], computing H''(rho) by quadrature; one interval with H''(rho) > 0 at large n would refute the claimed asymptotic concavity, and hence the $\alpha$(1-rho) bound.

Watch

Extended reading notes

Core claim

For n null hypotheses X_i ~ N(0,1) with common correlation rho, testing each at level alpha_n with n*alpha_n -> alpha, the paper claims that H(rho) = P(max_i X_i <= c) is asymptotically concave in rho. Consequently FWER = 1 - H is asymptotically convex in rho and lies below the chord joining the independence endpoint (rho=0, FWER ~ n*alpha_n) to the perfect-dependence endpoint (rho=1, FWER=alpha_n). With Bonferroni's choice alpha_n = alpha/n, this gives the asymptotic bound FWER(rho) <= alpha(1-rho), showing that positive correlation reduces the effective family-wise error rate below the nominal level.

Load-bearing premise

The proof depends on the assumption that, for large n, the second derivative of H(rho) is controlled by the tail region alpha_1 < 1/n and that this localization is uniform in rho; if that fails, the convexity of FWER and the alpha(1-rho) bound are not established.

Editorial extensions

If this is right

  • For Bonferroni's rule alpha_n = alpha/n, the family-wise error rate is asymptotically no larger than alpha(1-rho), so positive correlation makes the procedure more conservative than the nominal level suggests.
  • FWER as a function of rho is asymptotically convex, so its worst case over rho in [0,1] occurs at rho=0, meaning independence is the most error-prone configuration.
  • The bound remains bounded and meaningful as n grows, unlike the distribution-free bound in Tong (2014), which becomes unbounded as the number of hypotheses increases.
  • In the parallel-system lifetime model, the result gives a lower bound on the c.d.f. of the failure time when component lifetimes are exchangeable normal variables.
  • The result indicates that Bonferroni-type procedures under positive correlation need a correlation correction to avoid being overly conservative.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct extension the authors leave implicit is a correlation-correction recipe: to hold FWER near alpha under positive correlation, a Bonferroni test could inflate its marginal level to roughly alpha/(1-rho), although the paper does not propose such a procedure.
  • If the asymptotic concavity could be established non-asymptotically, the same chord argument would give a finite-sample bound; the natural numerical check is whether H''(rho) <= 0 already holds at moderate n.
  • The proof's localization to the tail region alpha_1 < 1/n suggests the result is driven by rare single-test events, so a similar bound might hold for other exchangeable distributions, but any extension would need a new integral representation because the Gaussian identity is used explicitly.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper studies the family-wise error rate (FWER) of a Bonferroni-type procedure when the n test statistics are equicorrelated standard normal variables. The central object is H(ρ) = P(max_i X_i ≤ c), and the authors claim that as n → ∞ with n α_n → α, H is concave in ρ ∈ [0,1], so FWER is convex and lies below the chord joining the independence and perfect-correlation endpoints. For Bonferroni's choice α_n = α/n this yields the asymptotic bound FWER ≤ α(1−ρ). The proof represents H as an expectation over a common normal factor, differentiates twice under the integral, and then analyzes the sign of H'' by splitting the integration region according to α_1 = Φ(−d) relative to 1/n. Simulations in Table 1 are reported to support the bound for n = 10000 across several α and ρ.

Significance. If the claim is correct, the paper provides a very clean, parameter-free asymptotic bound that quantifies how positive equicorrelation makes Bonferroni's procedure more conservative, and it identifies a simple correction factor. The bound is falsifiable and the simulation evidence in Section 7 is broadly supportive, with all reported estimates below α(1−ρ). The representation of H as an expectation and the reduction of the problem to the sign of H'' are natural and potentially useful. However, the proof as written contains a miscomputed derivative, an incorrect constant in the location of z0, and asymptotic lemmas that are only sketched; these issues directly affect the claimed concavity in the small-ρ regime, which is essential for the final bound. The result is plausible, but the manuscript does not currently establish it rigorously.

major comments (4)
  1. [Eq. (1) and Section 3.1] The derivative G in Eq. (1) is incorrect. With d = (c + √ρ Z)/√(1−ρ), the correct derivative is ∂d/∂ρ = (c + Z/√ρ)/(2(1−ρ)^{3/2}), not (c + Z√ρ)/(2(1−ρ)^{3/2}) as stated. This is not a harmless typo: under the correct G, the first term in H'' acquires an extra 1/ρ factor, the term bG acquires an extra 1/ρ^{3/2} factor (since b ∼ 1/ρ), and the estimates in Lemmas 6.1 and 6.2 acquire additional singularities as ρ → 0. The proof's localization argument relies on bounds such as sup_{z≤z0} dG²φ(z) → 0 as n → ∞; with the correct G this sup is not bounded uniformly in ρ, and the claim that the integrals over {α1 ≥ 1/n} vanish is not established in the small-ρ regime. Since Theorem 3.1 asserts concavity on all of [0,1], and the small-ρ regime is exactly where the bound is nontrivial, the central proof does not currently go through.
  2. [Section 4.1, location of z0] The derivation after c − d(z0) → 0 concludes that z0 + cT(ρ) → 0 with T(ρ) = 1/(1 + √(1−ρ)). Solving z0 = (√(1−ρ) d(z0) − c)/√ρ with d(z0) ≈ c gives z0 ≈ −c√ρ/(1 + √(1−ρ)), so T(ρ) should be √ρ/(1 + √(1−ρ)). This quantity tends to 0 as ρ → 0. The subsequent estimates in Lemma 6.1 (e.g., c e^{−z0²} → 0) require |z0| to grow like a constant times c; for small ρ this fails. Thus the control of the middle region in Lemma 6.1 is not uniform in ρ, and the localization argument on which Step 2 of the main proof depends is not valid as ρ → 0.
  3. [Eq. (2) and Lemma 6.2] The definition of a in Eq. (2) is inconsistent with the approximation used in the proof. The paper defines a = (n−1)φ(d) − dΦ(−d), but Lemma 6.2 uses the identity |a − d(nα1 − 1)| = (n−1)|φ(d) − dΦ(−d)|. This identity holds only if a = (n−1)φ(d) − dΦ(d), i.e., the argument of Φ should be d, not −d. As written, a − d(nα1 − 1) equals (n−1)φ(d) − dα1 − dnα1 + d, not (n−1)(φ(d) − dα1). Since Lemma 6.2 is the step that replaces a by d(nα1 − 1) and thereby determines the sign of the dominant term, this inconsistency is load-bearing. Correcting it to Φ(d) is necessary, and the proof of Lemma 6.2 would need to be redone with the corrected formula.
  4. [Section 4 and Appendix, Lemmas 6.1 and 6.2] The asymptotic localization is not established with sufficient rigor. Lemma 6.1 explicitly proves only the third term and states that the second 'follows similarly'; Lemma 6.2 is a sketch that says 'an idea similar to the proof of lemma 6.1 will tell us' and considers only the region {1/(n(log n)^3) ≤ α1 ≤ 8 log n/n}. In view of the ρ-singularities introduced by the correct derivative G, those 'similar' arguments are not routine and require explicit uniform bounds in ρ and n. Moreover, the final conclusion of Theorem 3.1 requires H_n''(ρ) ≤ 0 for all ρ ∈ [0,1] for large n; a pointwise statement for each fixed ρ would not justify the chord bound in Corollary 3.1.1. The manuscript needs to state and prove uniform versions of the estimates, or restrict the theorem to a setting where uniformity is not needed.
minor comments (5)
  1. [Section 4, notation] The symbol c is used both for the critical value and then again inside formula (2); this is confusing. Please distinguish the critical value, e.g., c_n, from other uses.
  2. [Section 4, asymptotic relation] The text writes 'n ∼ α√(2π c) e^{c²/2}'; the correct relation from α_n ∼ φ(c)/c is n ∼ α √(2π) c e^{c²/2}. Please fix the typo.
  3. [Lemma 6.1] The inequality φ(d) ≤ d³ α1 for large d is weaker than the standard φ(d) ∼ d α1; using the sharper asymptotic would simplify the proof and avoid unnecessary logarithmic factors. Please clarify.
  4. [Table 1] The table header is visually confusing: the row label 'ρ α' followed by the column labels 0.01, 0.05, ... without separating the α values from the ρ values. Please reformat the table so the two factors are clear.
  5. [Abstract and Introduction] The paper says the Tong (2014) bound is 'not a bounded quantity as n gets larger'; please clarify what is meant, since a probability is always in [0,1]. Presumably the comparison is about the sharpness of the upper bound as n grows, not its lack of finite value.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the FWER bound is derived from a self-contained asymptotic analysis of an explicit integral representation.

full rationale

The paper's derivation chain is self-contained. H(rho) is written as H(rho) = E[Phi^n((c + sqrt(rho) Z)/sqrt(1 - rho))], and the main theorem is proved by differentiating this representation twice and analyzing the asymptotic sign of H''(rho) via a partition of the range of alpha_1 at alpha_1 = 1/n. No parameter is fitted to data, no external result is imported as a load-bearing assumption, and no prior work of the authors is invoked to force the conclusion. Tong (2014) is cited only as motivation for why a distribution-free bound is too crude; it is not used in the proof. Lemma 6.1 and Lemma 6.2 are proved in the appendix, albeit with one part stated as 'the other one follows similarly' and with possible calculational errors. A proof gap or an incorrect derivative, if it exists, is a correctness or rigor concern, not circularity. Corollary 3.1.1 is a direct convexity/chord consequence of Theorem 3.1, and Theorem 5.1 is the corresponding endpoint interpolation; neither is a renamed input. The simulation section is empirical verification and is not used in the derivation. The alleged errors concerning G and T(rho) affect uniformity and validity of the asymptotic localization near rho = 0, but they do not make any asserted result equivalent to its own assumptions by construction. Therefore the paper exhibits no circularity under the criteria of this analysis.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The paper introduces no free parameters fitted to data and no new postulated entities. It relies on the standard Gaussian common-factor representation, the tail expansion for the normal distribution, and repeated use of dominated convergence. The main burden is on unverified regularity conditions and the localized asymptotic analysis of H''.

assumptions (3)
  • domain assumption The sequence (X_i) under the intersection null is exchangeable normal with equicorrelation ρ, representable as X_i = θ + Z_i with θ ~ N(0,ρ), Z_i iid N(0,1-ρ).
    Section 3.1; this common-factor representation underlies the expression for H(ρ) and all derivative computations.
  • standard math The threshold c satisfies nΦ(-c) → α, so c^2 = Θ(log n), and the asymptotic tail expansion Φ(-x) ~ φ(x)/x holds for large x.
    Section 4; used repeatedly to locate z0 and to bound integrals in the proof.
  • standard math Dominated convergence can be applied to differentiate H(ρ) inside the expectation and to exchange limits with integrals.
    Equations (1)-(2); the paper invokes DCT without verifying the required uniform integrability, which is nontrivial because the integrands depend on n through c.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Bound on FWER for correlated normal distribution." pith.science (2026). https://pith.science/paper/6ZHP5I3Q

@misc{pith2026190802193,
  author       = {Pith},
  title        = {Pith review of: Bound on FWER for correlated normal distribution},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6ZHP5I3Q}},
  note         = {Machine review of arXiv:1908.02193}
}
read the original abstract

In this paper,our main focus is to obtain an asymptotic bound on the family wise error rate (FWER) for Bonferroni-type procedure in the simultaneous hypotheses testing problem when the observations corresponding to individual hypothesis are correlated. In particular, we have considered the sequence of null hypotheses H_{0i} : X_i follows N(0,1) , (i=1,2,....,n) and equicorrelated structure of the sequence (X_1,....,X_n). Distribution free bound on FWER under equicorrelated setup can be found in Tong(2014). But the upper bound provided in Tong(2014) is not a bounded quantity as the no. of hypotheses(n) gets larger and larger and as a result,FWER is highly overestimated for the choice of a particular distribution (e.g.- normal). In the equicorrelated normal setup, we have shown that FWER asymptotically is a convex function (as a function of correlation (rho)) and hence an upper bound on the FWER of Bonferroni-(alpha) procedure is alpha(1-\rho).This implies,Bonferroni's method actually controls the FWER at a much smaller level than the desired level of significance under the positively correlated case and necessitates a correlation correction.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

21 extracted references · 17 canonical work pages

  1. [1]

    Benjamini and Y

    Y. Benjamini and Y. Hochberg. Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the Royal statistical society: series B (Methodological), 57 0 (1): 0 289--300, 1995

    1995

  2. [2]

    Benjamini and W

    Y. Benjamini and W. Liu. A step-down multiple hypotheses testing procedure that controls the false discovery rate under independence. Journal of statistical planning and inference, 82 0 (1-2): 0 163--170, 1999

    work page 1999

  3. [3]

    Dudoit and M

    S. Dudoit and M. J. Van Der Laan. Multiple testing procedures with applications to genomics. Springer Science & Business Media, 2007

    work page 2007

  4. [4]

    Dudoit, J

    S. Dudoit, J. P. Shaffer, J. C. Boldrick, et al. Multiple hypothesis testing in microarray experiments. Statistical Science, 18 0 (1): 0 71--103, 2003

    work page 2003

  5. [5]

    B. Efron. Correlation and large-scale simultaneous significance testing. Journal of the American Statistical Association, 102 0 (477): 0 93--103, 2007

    work page 2007

  6. [6]

    B. Efron. Correlated z-values and the accuracy of large-scale statistical estimates. Journal of the American Statistical Association, 105 0 (491): 0 1042--1055, 2010

    work page 2010

  7. [7]

    B. Efron. Large-scale inference: empirical Bayes methods for estimation, testing, and prediction, volume 1. Cambridge University Press, 2012

    2012

  8. [8]

    Efron, R

    B. Efron, R. Tibshirani, J. D. Storey, and V. Tusher. Empirical bayes analysis of a microarray experiment. Journal of the American statistical association, 96 0 (456): 0 1151--1160, 2001

    2001

Show all 21 references
  1. [9]

    Efron et al

    B. Efron et al. Size, power and false discovery rates. The Annals of Statistics, 35 0 (4): 0 1351--1377, 2007

    2007

  2. [10]

    Hochberg

    Y. Hochberg. A sharper bonferroni procedure for multiple tests of significance. Biometrika, 75 0 (4): 0 800--802, 1988

    1988

  3. [11]

    S. Holm. A simple sequentially rejective multiple test procedure. Scandinavian journal of statistics, pages 65--70, 1979

    1979

  4. [12]

    G. Hommel. A stagewise rejective multiple test procedure based on a modified bonferroni test. Biometrika, 75 0 (2): 0 383--386, 1988

    1988

  5. [13]

    Loperfido, J

    N. Loperfido, J. Navarro, J. M. Ruiz, and C. J. Sandoval. Some relationships between skew-normal distributions and order statistics from exchangeable normal random vectors. Communications in Statistics—Theory and Methods, 36 0 (9): 0 1719--1733, 2007

    2007

  6. [14]

    J. P. Romano, A. M. Shaikh, and M. Wolf. Control of the false discovery rate under dependence using the bootstrap and subsampling. Test, 17 0 (3): 0 417, 2008

    2008

  7. [15]

    S. K. Sarkar. On methods controlling the false discovery rate. Sankhy \=a : The Indian Journal of Statistics, Series A (2008-) , pages 135--168, 2008

    2008

  8. [16]

    S. K. Sarkar et al. Some results on false discovery rate in stepwise multiple testing procedures. The Annals of Statistics, 30 0 (1): 0 239--257, 2002

    2002

  9. [17]

    S. K. Sarkar et al. Stepup procedures controlling generalized fwer and generalized fdr. The Annals of Statistics, 35 0 (6): 0 2405--2420, 2007

    2007

  10. [18]

    R. J. Simes. An improved bonferroni procedure for multiple tests of significance. Biometrika, 73 0 (3): 0 751--754, 1986

    1986

  11. [19]

    J. D. Storey. A direct approach to false discovery rates. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 64 0 (3): 0 479--498, 2002

    2002

  12. [20]

    Y. L. Tong. Probability inequalities in multivariate distributions. Academic Press, 2014

    2014

  13. [21]

    Yekutieli and Y

    D. Yekutieli and Y. Benjamini. Resampling-based false discovery rate controlling multiple test procedures for correlated test statistics. Journal of Statistical Planning and Inference, 82 0 (1-2): 0 171--196, 1999

    1999

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.