Causal Meta-Analysis: Rethinking the Foundations of Evidence-Based Medicine
Pith reviewed 2026-05-19 12:52 UTC · model grok-4.3
The pith
Classical meta-analysis has a causal interpretation only for risk differences; new aggregation formulas restore it for risk ratios and odds ratios using aggregate data alone.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
Classical meta-analysis estimators possess a clear causal interpretation when effects are measured as risk differences, but this breaks down for nonlinear measures like the risk ratio and odds ratio; novel causal aggregation formulas restore the interpretation while remaining compatible with standard practices and requiring only aggregate study-level data.
What carries the argument
Causal aggregation formulas that combine study-level effect estimates into an overall causal effect on a specified target population.
If this is right
- Risk ratios and odds ratios can now be pooled while retaining a causal reading on the target population.
- Reanalysis of existing meta-analyses can alter conclusions about whether a treatment is beneficial or harmful.
- Evidence hierarchies that rely on meta-analysis can incorporate explicit target-population specifications.
- Public policy decisions informed by meta-analyses may require updating when discrepancies appear.
Where Pith is reading between the lines
- Software packages for meta-analysis could add the new formulas as an optional causal reporting mode.
- Guidelines for evidence synthesis might begin requiring explicit statements of the target population for each pooled estimate.
- The discrepancy cases identified in the 500 meta-analyses offer concrete candidates for further validation against individual-participant data.
Load-bearing premise
Well-defined causal effects on target populations can be recovered from aggregate study-level summaries alone without additional unverifiable assumptions about confounding or transportability.
What would settle it
Direct comparison of the causal meta-analysis result against a pooled analysis performed on the individual-level records from the same studies; systematic mismatch would show the formulas fail to recover the true causal quantity.
read the original abstract
Meta-analysis, by synthesizing effect estimates from multiple studies conducted in diverse settings, stands at the top of the evidence hierarchy in clinical research. Yet, conventional approaches based on fixed- or random-effects models lack a causal framework, which may limit their interpretability and utility for public policy. Incorporating causal inference reframes meta-analysis as the estimation of well-defined causal effects on clearly specified populations, enabling a principled approach to handling study heterogeneity. We show that classical meta-analysis estimators have a clear causal interpretation when effects are measured as risk differences. However, this breaks down for nonlinear measures like the risk ratio and odds ratio. To address this, we introduce novel causal aggregation formulas that remain compatible with standard meta-analysis practices and do not require access to individual-level data. To evaluate real-world impact, we apply both classical and causal meta-analysis methods to 500 published meta-analyses. While the conclusions often align, notable discrepancies emerge, revealing cases where conventional methods may suggest a treatment is beneficial when, under a causal lens, it is in fact harmful.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript reframes meta-analysis as the estimation of well-defined causal effects on specified target populations. It demonstrates that standard fixed- and random-effects meta-analysis estimators possess a causal interpretation when the effect measure is the risk difference. For nonlinear measures such as the risk ratio and odds ratio, this interpretation fails, prompting the development of new causal aggregation formulas. These formulas are designed to be compatible with conventional meta-analysis workflows and rely solely on study-level summary statistics. The authors evaluate the approach by re-analyzing 500 published meta-analyses, identifying instances where classical and causal methods yield conflicting conclusions regarding treatment benefits.
Significance. Should the proposed causal aggregation formulas prove valid under the stated conditions, the paper could meaningfully advance evidence synthesis in medicine by clarifying the causal meaning of meta-analytic results for policy-relevant effect measures. The scale of the empirical application to 500 meta-analyses provides a useful illustration of potential discrepancies. Strengths include the emphasis on compatibility with existing practices and the avoidance of requirements for individual participant data.
major comments (2)
- [Methods] The derivation of the causal aggregation formulas for risk ratios does not include a formal identification result or a complete list of assumptions. Specifically, it is unclear how the formulas account for potential effect modification or ensure transportability to the target population when only marginal estimates are available from each study.
- [Results] In the application to 500 meta-analyses, the paper reports notable discrepancies but does not detail the definition of the target population for each meta-analysis or conduct sensitivity analyses to assess robustness to violations of the implicit transportability assumptions.
minor comments (2)
- [Abstract] The abstract mentions 'novel causal aggregation formulas' but could briefly indicate the key difference from classical estimators for nonlinear measures.
- [Introduction] Some notation for the target population causal parameter could be introduced earlier to aid readability.
Simulated Author's Rebuttal
We thank the referee for their constructive and insightful comments on our manuscript. These have helped us identify areas where additional clarification will strengthen the presentation of the causal meta-analysis approach. We respond to each major comment below and indicate the revisions we intend to make.
read point-by-point responses
-
Referee: [Methods] The derivation of the causal aggregation formulas for risk ratios does not include a formal identification result or a complete list of assumptions. Specifically, it is unclear how the formulas account for potential effect modification or ensure transportability to the target population when only marginal estimates are available from each study.
Authors: We agree that a more explicit formal identification result and list of assumptions would improve clarity. In the revised manuscript we will add a dedicated subsection that states the identification assumptions, including no unmeasured confounding within each study, positivity, and the conditions under which the marginal study estimates can be transported to a common target population. The causal aggregation formulas are constructed to preserve a well-defined causal interpretation by re-expressing the target-population risk ratio as a function of the study-specific marginal quantities; effect modification is accommodated through the implicit standardization to the target-population covariate distribution. Because only summary statistics are used, the formulas rely on the availability of the necessary marginal estimates and do not require individual-level data. We will make these points explicit and add the corresponding identification theorem. revision: yes
-
Referee: [Results] In the application to 500 meta-analyses, the paper reports notable discrepancies but does not detail the definition of the target population for each meta-analysis or conduct sensitivity analyses to assess robustness to violations of the implicit transportability assumptions.
Authors: We acknowledge that greater transparency regarding the target population is desirable. In the revision we will add a clear description of the target population employed in the empirical study: a synthetic population whose covariate distribution is a weighted average of the baseline characteristics reported across the included studies. Because the analysis encompasses 500 distinct meta-analyses, it is not feasible to furnish a bespoke, fully enumerated target-population definition for every meta-analysis; instead we will emphasize that, in applied work, the target should be chosen according to the policy or clinical question at hand. For sensitivity analyses we will report results from a targeted sensitivity exercise performed on the subset of meta-analyses that exhibited the largest discrepancies, varying the strength of assumed transportability violations. This will illustrate the robustness of the qualitative conclusions without substantially lengthening the main text. revision: partial
Circularity Check
No circularity in derivation of causal aggregation formulas
full rationale
The paper derives novel causal aggregation formulas for nonlinear measures (RR/OR) from causal inference principles applied to aggregate study summaries, showing that classical meta-analysis has a causal interpretation only for risk differences. These formulas are presented as derived rather than fitted, and the evaluation applies both methods to 500 externally published meta-analyses rather than self-generated data. No load-bearing self-citations, self-definitional loops, fitted inputs renamed as predictions, or ansatz smuggling via prior work are identifiable. The derivation chain remains self-contained with independent content from stated causal assumptions and external validation.
Axiom & Free-Parameter Ledger
axioms (1)
- domain assumption Well-defined causal effects on specified populations can be recovered from study-level summaries without individual data
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.