REVIEW 3 major objections 4 minor 17 references
Subsampling-based Tests in Mediation Analysis
T0 review · 3 major / 4 minor · reviewed 2026-08-12 · deepseek-v4-flash
Pith's one-line read Splitting the sample into subsamples and studentizing per-subsample Sobel statistics yields a t-distribution null that is the same for all three composite-null cases in mediation analysis.
desk verdict Correct and elegant subsample studentization for the mediation null, but the final Cauchy combination step is not fully justified and needs a theorem or a condition check before CSMT is presented as validated. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the studentized subsample Sobel statistic, built by partitioning the data into K disjoint subsamples, computing the classical Sobel statistic S_{G_i} within each, and applying a one-sample t-statistic to the K resulting values. Because each S_{G_i} is asymptotically normal under every null scenario but with a variance that depends on which null case holds, the studentization cancels the unknown scale and leaves a pivotal t_{K−1} limit. The second piece is the Cauchy combination test, which aggregates p-values from M random splits using random positive weights; the paper rejects when the combined statistic exceeds the standard Cauchy quantile.
What would settle it
Simulate data under H00 (both coefficients zero) with large n, fix one dataset, compute p-values from M = 500 random splits, form the Cauchy-combined statistic C_m, and repeat over many datasets; if the empirical distribution of C_m deviates substantially from the standard Cauchy in the tail, the pivotal rejection region is not valid.
Extended reading notes
Core claim
The central claim is Theorem 2: under the composite null H0: αβ = 0, the studentized subsample statistic T_n = $K^{{1/2}}$ \bar{S}_K / ( (1/(K−1)) \sum (S_{G_i} − \bar{S}_K)^2 )^{1/2} converges in distribution to a t random variable with K−1 degrees of freedom. Each subsample Sobel statistic S_{G_i} is asymptotically normal with mean zero and variance either 1 (when exactly one of α and β is zero) or 1/4 (when both are zero), so the sample variance across subsamples estimates and removes the scale. The null distribution is therefore the same in all three null cases, requiring a single cutoff. Repeating the split M times and combining the resulting p-values through a weighted Cauchy combination test gives CSMT, which the paper demonstrates by simulation to control size more accurately than the Sobel test, the MaxP test, and an adaptive bootstrap test while achieving higher power at stronger signals.
Load-bearing premise
The procedure's size control ultimately depends on the Cauchy combination of p-values from different random splits following the standard Cauchy distribution even though those p-values are dependent, and the paper does not prove this.
Editorial extensions
If this is right
- A single t_{K−1} cutoff can be used for mediation testing regardless of which of the three null cases holds, eliminating the need to know the null type.
- Combining p-values from repeated random splits stabilizes the test results and raises power, as shown in the numerical studies.
- CSMT controls size more accurately than the classical Sobel and MaxP tests, which are conservative when both coefficients are zero.
- At stronger signal strengths, CSMT outperforms the adaptive bootstrap test while avoiding that test's inflated size.
- The method remains valid for the general structural equation framework covered by the assumptions, not only for linear models.
Reading between the lines
- Editorial inference: the proof of Theorem 2 only uses asymptotic normality of the per-subsample statistics, so the same subsample-studentization device could pivotailize other test statistics whose normal limit has a null-dependent variance.
- Editorial inference: the recommended choice K = ⌊0.5√n⌋ is based on empirical evidence; a data-driven or cross-validated choice of K has not been explored and could improve finite-sample behavior.
- Editorial inference: because the Cauchy combination's null distribution is assumed rather than proved under the dependence across splits, practitioners using CSMT on smaller samples should verify the empirical null by permutation before trusting nominal p-values.
- Editorial inference: the SPIRIT data analysis points to butyric, acetic, and valeric acids as candidate mediators of metformin's anti-inflammatory effect; a confirmatory study with multiple-testing adjustment would be needed to make this a robust scientific claim.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a subsampling-based test for the composite null of no mediation effect (alpha*beta = 0). The idea is to split the data into K disjoint subsamples, compute the Sobel statistic in each subsample, and then studentize their mean to obtain a statistic T_n whose asymptotic null distribution is claimed to be t with K-1 degrees of freedom under all three null configurations (H00, H01, H10). This is the content of Theorem 2. To reduce variability across different random splits and improve power, the paper then combines p-values from M random splits using the Cauchy combination test, calling the resulting procedure CSMT. Section 5 recommends K = floor(0.5 sqrt(n)), and Section 6 reports simulation comparisons with Sobel, MaxP, and an adaptive bootstrap test, followed by a real-data analysis of the SPIRIT trial.
Significance. If the main claim is correct, the paper offers a simple, pivotal null distribution for a composite-null mediation problem, which is a genuinely useful contribution because standard Sobel and MaxP tests are conservative under H00. Theorem 2 is elementary and self-contained, with no fitted quantities entering the null distribution, which is a strength. The paper also provides an R implementation and reports extensive simulations. However, the actual procedure recommended to users, CSMT, relies on a Cauchy combination of dependent p-values for which no theorem is stated or proved; this gap is load-bearing for the paper's headline claim of accurate size control. The fixed-K theory also does not directly cover the recommended growing K.
major comments (3)
- [Section 4] The claim that the combined statistic C_m follows a standard Cauchy distribution, and hence that the p-value is 0.5 - arctan(C_m)/pi, is not justified. The p-values p_m are computed from M random splits of the same data and are therefore dependent. Liu and Xie (2020) is cited, but the paper neither states the precise theorem being used nor verifies its conditions. In particular, Theorem 2 gives only marginal asymptotic t-calibration of each p_m, not finite-sample validity, and arbitrary dependence among valid p-values does not make the weighted sum of their Cauchy transforms exactly Cauchy. Unless the authors state the relevant Liu-Xie result and prove that its conditions hold (or prove an asymptotic version for their setting), the size control of CSMT is not established by the theoretical part of the manuscript.
- [Theorem 2 and Section 5] Theorem 2 is stated and proved only for fixed K, but Section 5 recommends K = floor(0.5 sqrt(n)), which grows with n. The manuscript does not provide a theorem that covers K = K_n tending to infinity, even though in this regime the subsample sizes n/K_n also grow. The authors should either extend the asymptotic result to K_n -> infinity with n/K_n -> infinity, or explicitly limit the theoretical claim to a fixed K chosen by the user and describe the recommended K as a finite-sample heuristic. As written, the recommended implementation is not covered by the stated theorem.
- [Proof of Theorem 2] The proof of Theorem 2 says the result follows by applying Theorem 1 to each subsample and the continuous mapping theorem, but this omits an essential step: one must establish joint convergence of (S_{G_1}, ..., S_{G_K}) to independent normal variables, and then verify that the sample variance of the S_{G_i} is consistent. Since the subsamples are disjoint and K is fixed, this is straightforward, but it should be stated explicitly. Additionally, in the proof of Theorem 1 under H01/H10, the variance estimator is written as \hat{\alpha}^2 s_1^2 + \hat{\beta}^2 s_2^2, whereas the Sobel statistic in equation (1) uses \hat{\alpha}^2 s_2^2 + \hat{\beta}^2 s_1^2; the subscripts appear to be swapped.
minor comments (4)
- [Section 4] The notation is inconsistent: the statistic is first written as C_m with index m ranging over splits, but later the same symbol C_m is used as the combined statistic; the index m is reused in both the denominator and the numerator of the weighted sum.
- [Section 4] The sentence 'where wms are non-negative weights satisfying that sum_{m=1}^M w_m = 1' should read 'non-negative weights w_m satisfying ...' to avoid a missing subscript.
- [Section 5] The recommendation K = floor(0.5 sqrt(n)) is based only on empirical evidence summarized in the supplementary materials; the main text should clarify whether this choice is meant to be applied as a deterministic function of n or as a practical guideline independent of the asymptotic theory.
- [Section 6] In Example 2, the text says 'the power of the ABtest decreases as the signal strength increases,' but the accompanying Figure 3 is not described in enough detail; a brief explanation of why this occurs would improve readability.
Circularity Check
No significant circularity: Theorem 2's pivotal t_{K-1} null is derived from an in-paper proof of the Sobel limit under all three null cases; the only soft spot is Section 4's unverified application of the external Liu-Xie Cauchy combination to dependent split p-values, a correctness risk rather than a circular step.
full rationale
The derivation chain is self-contained and contains no circular step. Theorem 1 (the composite-null limiting law of the Sobel statistic) is proved in the paper from Assumptions 1-2, via the delta method under H01/H10 and a polar-transform argument under H00; the Glonek (1993) citation is attribution, not load-bearing. Theorem 2 then applies Theorem 1 to each of K disjoint subsamples and studentizes: each S_Gi converges to N(0,1) under H01/H10 and to N(0,1/4) under H00, and since T_n = K^{1/2} average(S_Gi)/sd is scale-invariant, its weak limit is t_{K-1} in all three cases by the continuous mapping theorem. No parameter is fitted to produce this limiting law, and K enters only as degrees of freedom. The p-value p = P(|t_{K-1}| > |T_n|) is a standard asymptotic calibration, not a fitted quantity; the tuning choices (K = floor(0.5 sqrt(n)), M splits, random Cauchy weights) are practical and do not enter the null derivation. The one soft spot is Section 4: the paper asserts 'C follows a standard Cauchy distribution' for the combination of M p-values from different splits and rejects at the standard Cauchy quantile, citing Liu and Xie (2020), but it does not restate or verify that theorem's conditions for these dependent, only asymptotically t-calibrated p_m. That is a missing-support and correctness risk for the final CSMT size, not circularity: the combined statistic is not equivalent to its own inputs, and the cited source is an independent external result. There are no load-bearing self-citations (the reference list contains no work by the present authors), no uniqueness theorem imported from the authors' prior work, and no fitted input renamed as a prediction. Hence the honest finding is no circularity, with a minor flag on Section 4.
Assumptions & free parameters
free parameters (2)
- K (number of subsamples) =
K = floor(0.5 * sqrt(n))
- M (number of repeated splits) =
500 in all simulations
assumptions (3)
- domain assumption Assumptions 1 and 2: the estimators of the mediation coefficients are asymptotically normal with a diagonal covariance matrix and have consistent variance estimators.
- domain assumption Sobel statistics from disjoint subsamples are asymptotically independent.
- domain assumption The Cauchy combination of p-values is a valid test when the p-values are dependent across random splits.
Cite this review
Pith. "Pith review of Subsampling-based Tests in Mediation Analysis." pith.science (2026). https://pith.science/paper/GMQGE6B3
@misc{pith2026241110648,
author = {Pith},
title = {Pith review of: Subsampling-based Tests in Mediation Analysis},
year = {2026},
howpublished = {\url{https://pith.science/paper/GMQGE6B3}},
note = {Machine review of arXiv:2411.10648}
}
read the original abstract
Testing for mediation effect poses a challenge since the null hypothesis (i.e., the absence of mediation effects) is composite, making most existing mediation tests quite conservative and often underpowered. In this work, we propose a subsampling-based procedure to construct a test statistic whose asymptotic null distribution is pivotal and remains the same regardless of the three null cases encountered in mediation analysis. The method, when combined with the popular Sobel test, leads to an accurate size control under the null. We further introduce a Cauchy combination test to construct p-values from different subsample splits, which reduces variability in the testing results and increases detection power. Through numerical studies, our approach has demonstrated a more accurate size and higher detection power than the competing classical and contemporary methods.
Figures
Figures from the paper (3 more)
Reference graph
Works this paper leans on
-
[1]
Regulation of inflammation by short chain fatty acids
Marco AR Vinolo, Hosana G Rodrigues, Renato T Nachbar, and Rui Curi. Regulation of inflammation by short chain fatty acids. Nutrients, 3 0 (10): 0 858--876, 2011
work page 2011
-
[2]
Noel T Mueller, Moira K Differding, Mingyu Zhang, Nisa M Maruthur, Stephen P Juraschek, Edgar R Miller III, Lawrence J Appel, and Hsin-Chieh Yeh. Metformin affects gut microbiome composition and function and circulating short-chain fatty acids: a randomized trial. Diabetes care, 44 0 (7): 0 1462--1471, 2021
work page 2021
-
[3]
A general approach to causal mediation analysis
Kosuke Imai, Luke Keele, and Dustin Tingley. A general approach to causal mediation analysis. Psychological methods, 15 0 (4): 0 309, 2010
2010
-
[4]
Mediation analysis for survival data using semiparametric probit models
Yen-Tsung Huang and Tianxi Cai. Mediation analysis for survival data using semiparametric probit models. Biometrics, 72 0 (2): 0 563--574, 2016
work page 2016
-
[5]
Odds ratios for mediation analysis for a dichotomous outcome
Tyler J VanderWeele and Stijn Vansteelandt. Odds ratios for mediation analysis for a dichotomous outcome. American journal of epidemiology, 172 0 (12): 0 1339--1348, 2010
work page 2010
-
[6]
Causal mediation analysis with survival data
Tyler J VanderWeele. Causal mediation analysis with survival data. Epidemiology, 22 0 (4): 0 582--585, 2011
work page 2011
-
[7]
Asymptotic confidence intervals for indirect effects in structural equation models
Michael E Sobel. Asymptotic confidence intervals for indirect effects in structural equation models. Sociological methodology, 13: 0 290--312, 1982
work page 1982
-
[8]
A comparison of methods to test mediation and other intervening variable effects
David P MacKinnon, Chondra M Lockwood, Jeanne M Hoffman, Stephen G West, and Virgil Sheets. A comparison of methods to test mediation and other intervening variable effects. Psychological methods, 7 0 (1): 0 83, 2002
work page 2002
Show all 17 references
-
[9]
On the behaviour of wald statistics for the disjunction of two regular hypotheses
GFV Glonek. On the behaviour of wald statistics for the disjunction of two regular hypotheses. Journal of the Royal Statistical Society: Series B (Methodological), 55 0 (3): 0 749--755, 1993
1993
-
[10]
Cauchy combination test: a powerful test with analytic p-value calculation under arbitrary dependency structures
Yaowu Liu and Jun Xie. Cauchy combination test: a powerful test with analytic p-value calculation under arbitrary dependency structures. Journal of the American Statistical Association, 115 0 (529): 0 393--402, 2020
2020
-
[11]
Adaptive bootstrap tests for composite null hypotheses in the mediation pathway analysis
Yinqiu He, Peter X K Song, and Gongjun Xu. Adaptive bootstrap tests for composite null hypotheses in the mediation pathway analysis . Journal of the Royal Statistical Society Series B: Statistical Methodology, 86 0 (2): 0 411--434, 11 2023. ISSN 1369-7412. doi:10.1093/jrsssb/q...
2023 doi
-
[12]
H.C. Yeh, N. Maruthur, N.Y. Wang, G. Jerome, A. Dalcin, E. Tseng, K. White, E. Miller, S. Juraschek, N. Mueller, et al. A randomized controlled trial of behavioral weight loss and metformin on IGFs in cancer survivors. Obesity, 28: 0 17, 2020
2020
-
[13]
Yeh, N.M
H.C. Yeh, N.M. Maruthur, N.Y. Wang, G.J. Jerome, A.T. Dalcin, E. Tseng, K. White, E.R. Miller, S.P. Juraschek, N.T. Mueller, et al. Effects of behavioral weight loss and metformin on insulin-like growth factors in cancer survivors: A randomized trial. J. Clin. Endocrinol. Meta...
2021 doi
-
[14]
Metformin is associated with higher relative abundance of mucin-degrading akkermansia muciniphila and several short-chain fatty acid--producing microbiota in the gut
Jacobo De La Cuesta-Zuluaga, Noel T Mueller, Vanessa Corrales-Agudelo, Eliana P Vel \'a squez-Mej \' a, Jenny A Carmona, Jos \'e M Abad, and Juan S Escobar. Metformin is associated with higher relative abundance of mucin-degrading akkermansia muciniphila and several short-chai...
2017
-
[15]
Anti-inflammatory properties of the short-chain fatty acids acetate and propionate: a study with relevance to inflammatory bowel disease
Sofia Tedelind, Fredrik Westberg, Martin Kjerrulf, and Alexander Vidal. Anti-inflammatory properties of the short-chain fatty acids acetate and propionate: a study with relevance to inflammatory bowel disease. World journal of gastroenterology: WJG, 13 0 (20): 0 2826, 2007
2007
-
[16]
Increases in circulating and fecal butyrate are associated with reduced blood pressure and hypertension: results from the spirit trial
Curtis Tilves, Hsin-Chieh Yeh, Nisa Maruthur, Stephen P Juraschek, Edgar Miller, Karen White, Lawrence J Appel, and Noel T Mueller. Increases in circulating and fecal butyrate are associated with reduced blood pressure and hypertension: results from the spirit trial. Journal o...
2022
-
[17]
Xiushuang Chen, Qinghui Kong, Xiaoxiao Zhao, Chenxi Zhao, Pin Hao, Irfan Irshad, Hongjun Lei, Muhammad Fakhar-e-Alam Kulyar, Zeeshan Ahmad Bhutta, Hassan Ashfaq, et al. Sodium acetate/sodium butyrate alleviates lipopolysaccharide-induced diarrhea in mice via regulating the gut...
2022
Reviewed August 12, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.