REVIEW 2 major objections 5 minor 62 references
Mediation hypotheses can be screened in a first stage and tested in a second stage using the same sample, with asymptotically independent p-values, so standard FDR control applies to the survivors.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-01 17:34 UTC pith:QWFO7ZK2
load-bearing objection Genuinely useful transformation pair for two-stage mediation testing, but the FDR guarantee is formally only under cross-hypothesis independence, which the real-data regime violates. the 2 major comments →
Two-stage Adaptive Testing of Large-scale Mediation Hypotheses
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The paper's central claim is that each mediation hypothesis's two path-specific p-values can be transformed into a screening p-value and a testing p-value that are asymptotically independent even though both come from the same data. Screening uses the transformed minimum to eliminate likely global nulls, while testing uses the rescaled p-value on survivors. Because the selection and testing inputs are asymptotically independent, no sample splitting or selection-dependent adjustment is needed, and a standard step-up FDR procedure controls the false discovery rate asymptotically. The paper implements this with an adaptive signal-proportion estimator for the screening threshold and a conservati
What carries the argument
The central objects are two transformed p-values built from the order statistics of the two path-specific p-values: the transformed minimum, p*_1/2 = 1 - (1 - p_(1))^2, and the rescaled p-value, p*_2/2 = 1 - (1 - p_(2))/(1 - p_(1)). Under the global null both are asymptotically independent standard uniforms; under a partial null the rescaled p-value remains uniform and independent of the screening input; under a full alternative both degenerate to zero. This asymptotic independence lets the first-stage screen be data-adaptive and arbitrary without invalidating the second-stage FDR procedure, and the proof uses a classical representation of ordered uniform variables to establish the separatio
Load-bearing premise
The formal FDR guarantee assumes that the null p-values across different candidate mediators are mutually independent asymptotically; the paper only shows simulation robustness for one block-exchangeable dependence pattern, so under stronger or more complex cross-hypothesis correlation the guarantee is unproven.
What would settle it
Simulate a mediation study with, say, J = 50,000 null mediators whose p-values are positively correlated beyond the paper's block design—for instance, a single-factor correlation with rho = 0.6 or an autoregressive order-1 structure—and run the procedure at a nominal FDR of 0.2. If the empirical FDR systematically exceeds 0.2, the asymptotic FDR control claim is contradicted.
If this is right
- Screening can remove the majority of global-null hypotheses, reducing the multiplicity burden in Stage 2 and increasing power, especially when signals are sparse.
- The Stage 1 threshold can be chosen data-adaptively, since FDR control is asymptotically unaffected by the choice of screening threshold.
- Any step-up FDR-controlling procedure can be used in Stage 2; substituting an adaptive Bonferroni correction yields family-wise error rate control without changing Stage 1.
- In the paper's data application, screening retained 3,556 of 484,613 candidates and the procedure identified 23 mediators at FDR 0.05, compared to 6 without screening while covering all findings of prior approaches.
- The two-stage procedure preserves the strongest signals; the screening step primarily removes unpromising hypotheses rather than sacrificing tail discoveries.
Where Pith is reading between the lines
- The two-p-value transformation may generalize to other composite-null problems with paired p-values, such as replicability analysis, but each setting must independently justify the asymptotic independence; the paper only proves it for the mediation structure.
- If cross-hypothesis dependence in real data is stronger or more long-range than the block-exchangeable pattern simulated (e.g., factor-model or autoregressive correlations), the asymptotic FDR guarantee has not been proven and could break, so practitioners should verify empirically before trusting nominal levels.
- The asymptotic nature of the result suggests finite-sample performance may degrade with small sample sizes, so users with modest n might need calibration studies before applying the procedure.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a two-stage procedure for large-scale mediation testing. For each mediator j, the two asymptotically independent path-specific p-values (p1j, p2j) are ordered and transformed into p*_{1/2}=1-(1-p_(1))^2 and p*_{2/2}=1-(1-p_(2))/(1-p_(1)). Proposition 1 shows that, under the composite mediation null, the screening p-value and the rescaled test p-value are asymptotically independent and the latter is uniform; under partial and full alternatives the screening p-value collapses to zero. Stage 1 uses an adaptive signal-missing-rate rule (adSMR) on p*_{1/2} to filter out global nulls; Stage 2 applies Storey's adaptive step-up procedure to p*_{2/2} among the survivors. Theorem 2 claims asymptotic FDR control under mutual independence of the null p-values. The paper compares the method with DACT, HDMT, and M-DACT in simulations under sparse, moderate, and dense signals, including block dependence, and applies it to the Normative Aging Study, recovering previously reported CpG sites and adding two further discoveries.
Significance. The core idea is attractive and potentially useful: exploit the composite-null structure of mediation to obtain within-hypothesis asymptotic independence between screening and testing p-values, thereby avoiding sample splitting and complicated selection-adjusted FDR procedures. The supplementary proofs are self-contained, and the order-statistic argument in Proposition 1 is transparent. The simulation study is broader than in many competing papers and covers both FDR and FWER endpoints. If the independence-based guarantee is treated as the formal result, the contribution is solid. The two concerns below concern how far that guarantee extends to the dependent, large-J settings that motivate the paper.
major comments (2)
- [Theorem 2 and S2; Remark after Theorem 2] The theorem is proved only under mutual independence of the null p-values across J. In the motivating NAS application (Section 4), the 484,613 CpG p-values are correlated within genomic regions, and the simulations in Section 3 cover only a block-exchangeable model with ρ=0.3 and block sizes 10–100. The remark that 'standard step-up FDR procedures are robust to positive regression dependency' does not transfer automatically to the selected set I(tγ): the Stage-1 event {p*_{1/2,j}≤tγ} is a complex, data-adaptive selection event that can destroy PRDS among the survivors even if the original p-values are PRDS. Because the abstract and introduction present FDR control without qualification, the manuscript should either prove a version of Theorem 2 under a stated dependence model (e.g., block dependence with explicit conditions) or delimit the formal claim to independent hypotheses and descri
- [Theorem 2 / S2 (asymptotic regime)] The proof in S2 treats J as fixed and lets n→∞: it assumes convergence of the J-dimensional vector P*(n) to P*(∞) and applies the continuous mapping theorem. The paper's stated domain is 'large-scale' testing with J on the order of 10^5, and the simulations use J=10,000. If J=J_n grows with n, the convergence and continuity argument for the FDP mapping is not defined, and no uniform-in-J control is given. The theorem should state its asymptotic regime explicitly. If the claim is only for fixed J, the 'large-scale' framing needs adjustment; if J is meant to diverge, a triangular-array or uniform convergence argument is needed.
minor comments (5)
- [Section 2, after definition of p*] Typo: 'at least at large as' should read 'as large as'; the following sentence about p*_{2/2,j} being 'neither linear nor monotone' is also awkwardly phrased.
- [Algorithm 2, step 3] \hat{s}=J\hat{\pi}_{MR} is not necessarily an integer, yet k* is defined using p*_{1/2,(\hat{s}+j)}. Please specify whether \hat{s} is rounded, or define order statistics with a real index convention.
- [Figure 1 caption] The caption states (π00, π11)=(0.9, 0), but the simulation setting described in the text uses (π00, π11)=(0.9, 0.04) with π10=π01=0.05. Please correct this apparent inconsistency.
- [Section 4, NAS analysis] The simulation-based bounding sequence c_sim_J used for the NAS data is not fully described. Since the calibration is meant to account for dependence, the block structure and simulation protocol should be specified for reproducibility.
- [Section 2, Stage 2 estimator] Storey's estimator depends on a user-chosen λ, but the paper gives no guidance on choosing λ or a sensitivity analysis for it. A brief remark or a small simulation would help practitioners.
Circularity Check
No circularity: the derivation uses external distributional facts, standard estimators, and an explicit independence assumption; the simulation-calibrated screening constant does not enter the FDR-control proof.
full rationale
The central derivation chain is self-contained. Proposition 1 is proved from the asymptotic distribution of the path-specific p-values, the joint density of order statistics, and Rényi's representation, not from the conclusion it supports. Theorem 2 is then proved by conditioning on the Stage-1 filtration and using a reverse-martingale/optional-stopping argument with Storey's estimator; the binomial step in the proof is exactly the stated mutual-independence assumption, which is an explicit hypothesis of the theorem rather than a fitted input. The simulation-calibrated bounding sequence cJ in adSMR is used only to set the Stage-1 screening threshold, and the paper explicitly states that 'the FDR or FWER guarantee is provided solely by the Stage-2 testing procedure and holds for any data-adaptive choice of Stage 1 screening threshold tγ due to martingale properties of Stage-2 procedures.' Thus the FDR conclusion does not reduce to the calibration constant. There are no self-citations that carry load-bearing weight: the cited results (Storey et al. 2004; Meinshausen and Rice 2006; Jeng et al. 2019; Liu et al. 2022; Dai et al. 2022) are external, and the transformations are attributed to Tippett (1931) and Rényi (1953). The paper openly flags its main limitation: 'the FDR guarantee in this work is asymptotic and relies on the asymptotic independence of the two transformed p-values,' and 'the theoretical results rely on mutual independence across j.' Whether cross-hypothesis dependence invalidates the real-data application is a correctness/robustness concern, not a circularity. No step in the derivation is equivalent to its own input by construction, and no fitted parameter is renamed as a prediction.
Axiom & Free-Parameter Ledger
free parameters (2)
- λ (Storey estimator tuning parameter)
- cJ (bounding sequence in Meinshausen-Rice / adSMR) =
csim_J (simulation) or cind_J (Gumbel approximation)
axioms (5)
- domain assumption The base p-values p1j and p2j are asymptotically independent under the relevant null hypotheses (score orthogonality).
- domain assumption For each base hypothesis, the p-value converges in distribution to Unif(0,1) under the null and to a point mass at 0 under the alternative.
- domain assumption Null p-values are mutually independent across hypotheses j (for Theorem 2).
- standard math Rényi's representation / order-statistics distributional facts for i.i.d. uniforms; continuous mapping theorem; optional stopping theorem.
- domain assumption The linear mediation models in (1) are correctly specified with independent errors, so the path-specific p-values are valid.
read the original abstract
In testing large-scale mediation hypotheses, the exposure-mediator and mediator-outcome path-specific p-values obtained for each hypothesis from the same sample can be combined into a pair of asymptotically independent p-values, which can then be used to test a global null hypothesis and a composite mediation null hypothesis, respectively. A first-stage screening procedure targeting the more stringent global null can effectively reduce the number of mediation hypotheses to be tested in the second stage, which in turn reduces the conservativeness of multiple comparison. The framework can incorporate any data-adaptive choice of screening threshold in Stage 1, and any step-up false discovery rate control method in Stage 2. The proposed procedure controls the false discovery rate asymptotically while being consistently well powered across a wide range of scenarios.
Figures
Reference graph
Works this paper leans on
-
[1]
Acta Mathematica Hungarica , volume=
On the theory of order statistics , author=. Acta Mathematica Hungarica , volume=. 1953 , publisher=
1953
-
[2]
The methods of statistics , author=
-
[3]
Storey , journal =
John D. Storey , journal =. A Direct Approach to False Discovery Rates , urldate =
-
[4]
Krieger and Daniel Yekutieli , title =
Yoav Benjamini and Abba M. Krieger and Daniel Yekutieli , title =. Biometrika , number =. 2006 , doi =
2006
-
[5]
Sarkar , journal =
Sanat K. Sarkar , journal =. On Methods Controlling the False Discovery Rate , urldate =
-
[6]
Storey and Jonathan E
John D. Storey and Jonathan E. Taylor and David Siegmund , journal =. Strong Control, Conservative Point Estimation and Simultaneous Conservative Consistency of False Discovery Rates: A Unified Approach , urldate =
-
[7]
Sarkar , journal =
Yulia Gavrilov and Yoav Benjamini and Sanat K. Sarkar , journal =. An Adaptive Step-Down Procedure with Proven FDR Control under Independence , urldate =
-
[8]
Sarkar and Jingjing Chen and Wenge Guo , journal =
Sanat K. Sarkar and Jingjing Chen and Wenge Guo , journal =. Multiple Testing in a Two-Stage Adaptive Design With Combination Tests Controlling FDR , urldate =
-
[9]
Sociological methodology , volume=
Asymptotic confidence intervals for indirect effects in structural equation models , author=. Sociological methodology , volume=. 1982 , publisher=
1982
-
[10]
2021 , eprint=
Detecting Multiple Replicating Signals using Adaptive Filtering Procedures , author=. 2021 , eprint=
2021
-
[11]
On the Adaptive Control of the False Discovery Rate in Multiple Testing with Independent Statistics , urldate =
Yoav Benjamini and Yosef Hochberg , journal =. On the Adaptive Control of the False Discovery Rate in Multiple Testing with Independent Statistics , urldate =
-
[12]
Sarkar , journal =
Sanat K. Sarkar , journal =. Some Results on False Discovery Rate in Stepwise Multiple Testing Procedures , urldate =
-
[13]
Large-Scale Inference: Empirical Bayes Methods for Estimation, Testing, and Prediction , publisher=
Efron, Bradley , year=. Large-Scale Inference: Empirical Bayes Methods for Estimation, Testing, and Prediction , publisher=
-
[14]
Multiple Comparisons Among Means , urldate =
Olive Jean Dunn , journal =. Multiple Comparisons Among Means , urldate =
-
[15]
Large-Scale Simultaneous Hypothesis Testing: The Choice of a Null Hypothesis , urldate =
Bradley Efron , journal =. Large-Scale Simultaneous Hypothesis Testing: The Choice of a Null Hypothesis , urldate =
-
[16]
REPLICABILITY ANALYSIS FOR GENOME-WIDE ASSOCIATION STUDIES , urldate =
Ruth Heller and Daniel Yekutieli , journal =. REPLICABILITY ANALYSIS FOR GENOME-WIDE ASSOCIATION STUDIES , urldate =
-
[17]
Biometrics , year =
Yoav Benjamini and Ruth Heller , title =. Biometrics , year =
-
[18]
Su and Chiara Sabatti and Art B
Jingshu Wang and Lin Gui and Weijie J. Su and Chiara Sabatti and Art B. Owen , doi =. The Annals of Statistics , keywords =. 2022 , bdsk-url-1 =
2022
-
[19]
Journal of the American Statistical Association , volume=
Efficient signal inclusion with genomic applications , author=. Journal of the American Statistical Association , volume=. 2019 , publisher=
2019
-
[20]
Journal of the American Statistical Association , volume=
Multiple testing in a two-stage adaptive design with combination tests controlling FDR , author=. Journal of the American Statistical Association , volume=. 2013 , publisher=
2013
-
[21]
Biometrika , volume=
A note on e-values and multiple testing , author=. Biometrika , volume=. 2025 , publisher=
2025
-
[22]
, author=
Adaptive false discovery rate control under independence and dependence. , author=. Journal of Machine Learning Research , volume=
-
[23]
, author=
A comparison of methods to test mediation and other intervening variable effects. , author=. Psychological methods , volume=. 2002 , publisher=
2002
-
[24]
Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing , urldate =
Yoav Benjamini and Yosef Hochberg , journal =. Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing , urldate =
-
[25]
Chan , title =
Zhenman Yuan and Gary K.C. Chan , title =. Biometrika , year =
-
[26]
arXiv preprint arXiv:1703.06336 , year=
Analysis of error control in large scale two-stage multiple hypothesis testing , author=. arXiv preprint arXiv:1703.06336 , year=
-
[27]
Bogomolov, Marina and Heller, Ruth , TITLE =. Biometrika , FJOURNAL =. 2018 , NUMBER =. doi:10.1093/biomet/asy029 , URL =
-
[28]
J. A. Ferreira and A. H. Zwinderman , TITLE =. The Annals of Statistics , VOLUME =. 2006 , NUMBER =. doi:10.1214/009053606000000425 , URL =
-
[29]
K. J. Friston and W. D. Penny , TITLE =. Neuroimage , VOLUME =. 2005 , NUMBER =
2005
-
[30]
Yuan and K
Z. Yuan and K. C. G. Chan , TITLE =
-
[31]
J. Y. Dai and J. L. Stanford and M. LeBlanc , doi =. A Multiple-Testing Procedure for High-Dimensional Mediation Hypotheses , url =. 2022 , bdsk-url-1 =. https://doi.org/10.1080/01621459.2020.1765785 , journal =
arXiv 2022
-
[32]
Journal of the American Statistical Association , volume=
Large-scale hypothesis testing for causal mediation effects with applications in genome-wide epigenetic studies , author=. Journal of the American Statistical Association , volume=. 2022 , publisher=
2022
-
[33]
The Annals of Statistics , volume=
Estimating the proportion of false null hypotheses among a large number of independently tested hypotheses , author=. The Annals of Statistics , volume=
-
[34]
Annual Review of Statistics and Its Application , volume=
Causal mediation analysis for integrating exposure, genomic, and phenotype data , author=. Annual Review of Statistics and Its Application , volume=. 2025 , publisher=
2025
-
[35]
The health consequences of smoking—50 years of progress: A report of the surgeon general , author=
-
[36]
Circulation: cardiovascular genetics , volume=
Epigenetic signatures of cigarette smoking , author=. Circulation: cardiovascular genetics , volume=. 2016 , publisher=
2016
-
[37]
PloS one , volume=
Tobacco smoking leads to extensive genome-wide changes in DNA methylation , author=. PloS one , volume=. 2013 , publisher=
2013
-
[38]
American journal of respiratory and critical care medicine , volume=
Pulmonary function and blood DNA methylation: a multiancestry epigenome-wide association meta-analysis , author=. American journal of respiratory and critical care medicine , volume=. 2022 , publisher=
2022
-
[39]
Chest , volume=
SmokeHaz: systematic reviews and meta-analyses of the effects of smoking on respiratory health , author=. Chest , volume=. 2016 , publisher=
2016
-
[40]
American journal of respiratory and critical care medicine , volume=
Smoking and lung function of Lung Health Study participants after 11 years , author=. American journal of respiratory and critical care medicine , volume=. 2002 , publisher=
2002
-
[41]
Aging and Human Development , volume=
The Normative Aging Study: an interdisciplinary and longitudinal study of health and aging , author=. Aging and Human Development , volume=. 1972 , publisher=
1972
-
[42]
Journal of antimicrobial chemotherapy , volume=
Analysing the composition of outpatient antibiotic use: a tutorial on compositional data analysis , author=. Journal of antimicrobial chemotherapy , volume=. 2011 , publisher=
2011
-
[43]
Environmental health perspectives , volume=
Long-term exposure to ambient fine particulate matter and renal function in older men: the veterans administration normative aging study , author=. Environmental health perspectives , volume=
-
[44]
BMC bioinformatics , volume=
A unified approach to false discovery rate estimation , author=. BMC bioinformatics , volume=. 2008 , publisher=
2008
-
[45]
Mediation analysis , author=. Annu. Rev. Psychol. , volume=. 2007 , publisher=
2007
-
[46]
, author=
The moderator--mediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations. , author=. Journal of personality and social psychology , volume=. 1986 , publisher=
1986
-
[47]
Epigenetics , volume=
Air pollution and gene-specific methylation in the Normative Aging Study: association, effect modification, and mediation analysis , author=. Epigenetics , volume=. 2014 , publisher=
2014
-
[48]
Computational and structural biotechnology journal , volume=
Statistical methods for mediation analysis in the era of high-throughput genomics: current successes and future challenges , author=. Computational and structural biotechnology journal , volume=. 2021 , publisher=
2021
-
[49]
BMC bioinformatics , volume=
Identification of trans-eQTLs using mediation analysis with multiple mediators , author=. BMC bioinformatics , volume=. 2019 , publisher=
2019
-
[50]
Genome research , volume=
Identifying cis-mediators for trans-eQTLs across many human tissues using genomic mediation analysis , author=. Genome research , volume=
-
[51]
Proceedings of the National Academy of Sciences , volume=
Independent filtering increases detection power for high-throughput experiments , author=. Proceedings of the National Academy of Sciences , volume=. 2010 , publisher=
2010
-
[52]
Journal of the American Statistical Association , volume=
Estimating the null and the proportion of nonnull effects in large-scale multiple comparisons , author=. Journal of the American Statistical Association , volume=. 2007 , publisher=
2007
-
[53]
Statistics in Medicine , volume=
Group testing in mediation analysis , author=. Statistics in Medicine , volume=. 2020 , publisher=
2020
-
[54]
Bioinformatics , volume=
Mediation analysis for survival data with high-dimensional mediators , author=. Bioinformatics , volume=. 2021 , publisher=
2021
-
[55]
Bioinformatics , volume=
CoxMKF: a knockoff filter for high-dimensional mediation analysis with a survival outcome in epigenetic studies , author=. Bioinformatics , volume=. 2022 , publisher=
2022
-
[56]
Journal of the Royal Statistical Society Series B: Statistical Methodology , volume=
Sure independence screening for ultrahigh dimensional feature space , author=. Journal of the Royal Statistical Society Series B: Statistical Methodology , volume=. 2008 , publisher=
2008
-
[57]
Biometrics , volume=
Hypothesis test of mediation effect in causal mediation model with high-dimensional continuous mediators , author=. Biometrics , volume=. 2016 , publisher=
2016
-
[58]
Biometrics , volume=
High dimensional mediation analysis with latent variables , author=. Biometrics , volume=. 2019 , publisher=
2019
-
[59]
Journal of the American Statistical Association , volume=
High-dimensional mediation analysis for selecting DNA methylation loci mediating childhood trauma and cortisol stress reactivity , author=. Journal of the American Statistical Association , volume=. 2022 , publisher=
2022
-
[60]
An adaptive step-down procedure with proven FDR control under independence , author=
-
[61]
Replicability analysis for genome-wide association studies , author=
-
[62]
2000 , publisher=
Asymptotic statistics , author=. 2000 , publisher=
2000
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.