Recognition: unknown
Constructing external comparator groups via transportability in mean or in effect measure
Pith reviewed 2026-05-10 01:40 UTC · model grok-4.3
The pith
Causal effects in target populations can be identified by combining index trial data with external comparators under either transportability of potential outcome means or transportability of effect measures.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
We delineate external comparator analyses under two distinct but related identification strategies. The first relies on exchangeability of potential outcome means, using information only on the treatments to be compared. The second relies on transportability in effect measure, requiring additional use of information on a third treatment common to the populations. We propose estimators for identifying observed data functionals, with a particular focus on semiparametric efficient augmented weighting estimators that incorporate models for the probability of trial participation, the probability of treatment, and conditional outcome means. We derive the asymptotic properties of these estimators,,
What carries the argument
Semiparametric efficient augmented weighting estimators incorporating models for the probability of trial participation, the probability of treatment, and conditional outcome means.
Load-bearing premise
Exchangeability of potential outcome means or transportability in effect measure must hold for the combined populations.
What would settle it
A data-generating process in which transportability fails and the augmented weighting estimators exhibit bias relative to the known true causal effect.
read the original abstract
Learning about causal effects in target populations and their subsets may be facilitated by combining information from multiple sources. One major class of study designs that combine information involves appending an index study with data from an external comparator, which may facilitate head-to-head comparisons of treatments initially studied in different populations. We delineate external comparator analyses under two distinct, but related, identification strategies. The first strategy relies on exchangeability (transportability) of potential outcome means, which uses information only on the treatments that are to be compared. The second strategy relies on transportability in effect measure, requiring additional use of information on a third treatment common to the populations that have been combined. In a time-fixed setting with a point treatment and non-failure time outcome, we examine identification and estimation under a basic setup where information from an index trial is combined with a second, and external to the index trial, data source. We propose estimators for identifying observed data functionals, with a particular focus on semiparametric efficient augmented weighting estimators that incorporate models for the probability of trial participation, the probability of treatment, and conditional outcome means. We derive the asymptotic properties of these augmented weighting estimators -- including robustness to model misspecification and slower rates of convergence for some nuisance function models -- and use simulation to compare their finite sample performance to estimators based only on outcome modeling or weighting. Last, we provide a practical demonstration of the proposed methods by combining the ACCEPT and PHOENIX 1 randomized trials to evaluate the effect of various biologic agents on plaque psoriasis, a chronic inflammatory disorder.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper delineates two identification strategies for causal effects when appending an index trial with external comparator data: transportability of potential outcome means (using only the treatments of interest) and transportability in effect measure (requiring a third common treatment). It proposes semiparametric efficient augmented weighting estimators that incorporate models for trial participation probability, treatment probability, and conditional outcome means; derives their asymptotic properties including robustness to misspecification and slower nuisance convergence rates; compares finite-sample performance to pure outcome modeling and weighting estimators via simulation; and applies the methods to combine ACCEPT and PHOENIX 1 trials for evaluating biologic agents in plaque psoriasis.
Significance. If the asymptotic robustness properties hold for both strategies, the work provides a principled and efficient framework for external comparator analyses that improves upon standard approaches by allowing partial nuisance misspecification and slower convergence rates. The simulation comparisons and real-data demonstration add practical value for clinical research settings where direct randomization in the target population is infeasible.
major comments (2)
- [§4] §4 (Estimation under transportability in effect measure) and the associated influence function derivation: the claimed robustness to partial nuisance misspecification (double or triple robustness) is not fully verified for the effect-measure strategy. The influence function must be shown to cancel bias terms when only subsets of the three nuisance models (trial participation, treatment, outcome) are correct; otherwise the practical advantage over pure outcome modeling is reduced, as noted in the stress-test concern. Please provide the explicit expansion or theorem establishing the conditions.
- [§5] Theorem on asymptotic normality (likely §5): the slower rates of convergence permitted for nuisance estimators (e.g., n^{-1/4}) must be explicitly tied to both identification strategies and verified to ensure the central limit theorem and efficiency claims hold uniformly; the current outline leaves open whether the effect-measure case requires stricter rates.
minor comments (2)
- [Abstract] The abstract states the focus on augmented weighting estimators but could more clearly contrast the differing assumption sets and robustness properties of the two transportability strategies.
- [Simulation section] In the simulation section, include the exact data-generating processes and parameter values to facilitate reproducibility of the finite-sample comparisons.
Simulated Author's Rebuttal
We thank the referee for their careful reading and valuable comments on our manuscript. We have carefully considered the major comments and provide point-by-point responses below. We will make revisions to address the concerns raised.
read point-by-point responses
-
Referee: [§4] §4 (Estimation under transportability in effect measure) and the associated influence function derivation: the claimed robustness to partial nuisance misspecification (double or triple robustness) is not fully verified for the effect-measure strategy. The influence function must be shown to cancel bias terms when only subsets of the three nuisance models (trial participation, treatment, outcome) are correct; otherwise the practical advantage over pure outcome modeling is reduced, as noted in the stress-test concern. Please provide the explicit expansion or theorem establishing the conditions.
Authors: We appreciate this observation. The manuscript claims triple robustness for the augmented weighting estimator under transportability in effect measure, but we acknowledge that the explicit bias cancellation expansion for cases where only subsets of the nuisance models are correct was not detailed in the main text or appendix. We will add a new subsection or appendix entry providing the full influence function expansion, demonstrating that the estimator is consistent if any two of the three nuisance functions (trial participation probability, treatment probability, and conditional outcome mean) are correctly specified. This will strengthen the presentation of the robustness properties and directly address the concern about practical advantages. revision: yes
-
Referee: [§5] Theorem on asymptotic normality (likely §5): the slower rates of convergence permitted for nuisance estimators (e.g., n^{-1/4}) must be explicitly tied to both identification strategies and verified to ensure the central limit theorem and efficiency claims hold uniformly; the current outline leaves open whether the effect-measure case requires stricter rates.
Authors: We thank the referee for highlighting this point. The asymptotic normality result in Theorem 5.1 is derived under conditions that apply to both identification strategies, allowing nuisance estimators to converge at rates slower than n^{-1/2} (specifically n^{-1/4} under standard regularity conditions for the influence function to be asymptotically linear). However, to ensure clarity, we will revise the theorem statement and its proof to explicitly state the conditions for both the mean transportability and effect measure transportability cases, confirming that the same rate requirements suffice for the central limit theorem to hold and that the efficiency claims are uniform across strategies. No stricter rates are needed for the effect-measure case. revision: yes
Circularity Check
No significant circularity; derivations rely on standard identification assumptions and semiparametric efficiency theory
full rationale
The paper delineates two identification strategies (transportability of potential outcome means or in effect measure) and proposes augmented weighting estimators incorporating models for trial participation, treatment probability, and conditional outcome means. Asymptotic properties, including robustness to misspecification, are derived from standard semiparametric efficiency theory rather than reducing to fitted quantities or self-citations by construction. No load-bearing steps equate predictions to inputs via definition, renaming, or author-specific uniqueness theorems. The central claims remain independent of the paper's own fitted values.
Axiom & Free-Parameter Ledger
free parameters (1)
- models for trial participation probability, treatment probability, and conditional outcome means
axioms (3)
- domain assumption Exchangeability (transportability) of potential outcome means between index and external populations conditional on covariates
- domain assumption Transportability in effect measure, requiring a common third treatment
- standard math Standard positivity and consistency assumptions for causal inference
Reference graph
Works this paper leans on
-
[1]
Ung, L., Wang, G., Haneuse, S., Hern´ an, M. A. & Dahabreh, I. J.Identification strate- gies for combining an experimental study with external data2026. arXiv: 2406.03302 [stat.ME]
work page internal anchor Pith review Pith/arXiv arXiv
-
[2]
E.et al.Comparison of ustekinumab and etanercept for moderate-to-severe psoriasis.New England Journal of Medicine362,118–128 (2010)
Griffiths, C. E.et al.Comparison of ustekinumab and etanercept for moderate-to-severe psoriasis.New England Journal of Medicine362,118–128 (2010)
2010
-
[3]
Leonardi, C. L.et al.Efficacy and safety of ustekinumab, a human interleukin-12/23 monoclonal antibody, in patients with psoriasis: 76-week results from a randomised, double-blind, placebo-controlled trial (PHOENIX 1).The Lancet371,1665–1674 (2008)
2008
-
[4]
Guyatt, G. H. & Rennie, D. Users’ guides to the medical literature.Journal of the American Medical Association270,2096–2097 (1993)
2096
-
[5]
L., Dans, L
Dans, A. L., Dans, L. F., Guyatt, G. H., Richardson, S., Group, E.-B. M. W.,et al. Users’ guides to the medical literature: XIV. How to decide on the applicability of clinical trial results to your patient.Journal of the American Medical Association279, 545–549 (1998)
1998
-
[6]
Gehan, E. A. & Freireich, E. J. Non-randomized controls in cancer clinical trials.New England Journal of Medicine290,198–203 (1974)
1974
-
[7]
Pocock, S. J. The combination of randomized and historical controls in clinical trials. Journal of Chronic Diseases29,175–188 (1976)
1976
-
[8]
C., Guyatt, G
Bucher, H. C., Guyatt, G. H., Griffith, L. E. & Walter, S. D. The results of direct and indirect treatment comparisons in meta-analysis of randomized controlled trials. Journal of Clinical Epidemiology50,683–691 (1997)
1997
-
[9]
Network meta-analysis for indirect treatment comparisons.Statistics in Medicine21,2313–2324 (2002)
Lumley, T. Network meta-analysis for indirect treatment comparisons.Statistics in Medicine21,2313–2324 (2002). 32
2002
-
[10]
M., Ades, A
Caldwell, D. M., Ades, A. & Higgins, J. Simultaneous comparison of multiple treat- ments: combining direct and indirect evidence.British Medical Journal331,897–900 (2005)
2005
-
[11]
Ioannidis, J. P. Integration of evidence from multiple meta-analyses: a primer on um- brella reviews, treatment networks and multiple treatments meta-analyses.Canadian Medical Association Journal181,488–493 (2009)
2009
-
[12]
Rubin, D. B. Estimating causal effects of treatments in randomized and nonrandomized studies.Journal of Educational Psychology66,688 (1974)
1974
-
[13]
Robins, J. M. A new approach to causal inference in mortality studies with a sus- tained exposure period – application to control of the healthy worker survivor effect. Mathematical Modelling7,1393–1512 (1986)
1986
-
[14]
On the application of probability theory to agricultural experi- ments
Splawa-Neyman, J. On the application of probability theory to agricultural experi- ments. Essay on principles. Section 9. [Translated from Splawa-Neyman, J (1923) in Roczniki Nauk Rolniczych Tom X, 1–51]. Trans. by Dabrowska, D. M. & Speed, T. P. Statistical Science5,465–472 (1990)
1923
-
[15]
E.et al.Fusing trial data for treatment comparisons: Single vs multi-span bridging.Statistics in Medicine43,793–815 (2024)
Shook-Sa, B. E.et al.Fusing trial data for treatment comparisons: Single vs multi-span bridging.Statistics in Medicine43,793–815 (2024)
2024
-
[16]
Zivich, P. N.et al.HIV prevention among men who have sex with men: tenofovir alafe- namide combination preexposure prophylaxis versus placebo.The Journal of Infectious Diseases229,1123–1130 (2024)
2024
-
[17]
M., Rotnitzky, A
Robins, J. M., Rotnitzky, A. & Zhao, L. P. Estimation of regression coefficients when some regressors are not always observed.Journal of the American Statistical Association 89,846–866 (1994)
1994
-
[18]
Rotnitzky, A., Robins, J. M. & Scharfstein, D. O. Semiparametric regression for re- peated outcomes with nonignorable nonresponse.Journal of the American Statistical Association93,1321–1339 (1998). 33
1998
-
[19]
& Robins, J
Bang, H. & Robins, J. M. Doubly robust estimation in missing data and causal inference models.Biometrics61,962–973 (2005)
2005
-
[20]
Robins, J. M. & Morgenstern, H. The foundations of confounding in epidemiology. Computers & Mathematics with Applications14,869–916 (1987)
1987
-
[21]
Robins, J. M. Confidence intervals for causal parameters.Statistics in Medicine7,773– 785 (1988)
1988
-
[22]
Robins, J. M. Covariance Adjustment in Randomized Experiments and Observational Studies: Comment.Statistical Science17,309–321 (2002)
2002
-
[23]
J., Robertson, S
Dahabreh, I. J., Robertson, S. E., Steingrimsson, J. A., Stuart, E. A. & Hern´ an, M. A. Extending inferences from a randomized trial to a new target population.Statistics in Medicine39,1999–2014 (2020)
1999
-
[24]
J.et al.Study designs for extending causal inferences from a randomized trial to a target population.American Journal of Epidemiology190,1632–1642 (2021)
Dahabreh, I. J.et al.Study designs for extending causal inferences from a randomized trial to a target population.American Journal of Epidemiology190,1632–1642 (2021)
2021
-
[25]
Robins, J. M. & Greenland, S. Causal inference without counterfactuals: comment. Journal of the American Statistical Association95,431–435 (2000)
2000
-
[26]
VanderWeele, T. J. Concerning the consistency assumption in causal inference.Epi- demiology20,880–883 (2009)
2009
-
[27]
J., Robins, J
Dahabreh, I. J., Robins, J. M., Haneuse, S. J.-P. A. & Hern´ an, M. A.Generalizing causal inferences from randomized trials: counterfactual and graphical identification
- [28]
-
[29]
Ung, L., VanderWeele, T. J. & Dahabreh, I. J. Generalizing and transporting causal inferences from randomized trials in the presence of trial engagement effects.Epidemi- ology36,500–510 (2025)
2025
-
[30]
A.Hawthorne revisited: management and the worker, its critics, and developments in human relations in industry(Cornell University, Ithaca, NY, 1958)
Landsberger, H. A.Hawthorne revisited: management and the worker, its critics, and developments in human relations in industry(Cornell University, Ithaca, NY, 1958). 34
1958
-
[31]
& Robins, J
Greenland, S., Pearl, J. & Robins, J. M. Causal diagrams for epidemiologic research. Epidemiology,37–48 (1999)
1999
-
[32]
Pearl, J.Causality2nd (Cambridge University Press, Cambridge, UK, 2009)
2009
-
[33]
Richardson, T. S. & Robins, J. M.Single world intervention graphs: a primerinSecond UAI workshop on causal structure learning, Bellevue, Washington(2013)
2013
-
[34]
& Pearl, J
Bareinboim, E. & Pearl, J. Causal inference and the data-fusion problem.Proceedings of the National Academy of Sciences113,7345–7352 (2016)
2016
-
[35]
Dahabreh, I. J., Robertson, S. E. & Hern´ an, M. A.Generalizing and transporting infer- ences about the effects of treatment assignment subject to non-adherence2022. arXiv: 2211.04876[stat.ME]
-
[36]
J., Robertson, S
Dahabreh, I. J., Robertson, S. E., Petito, L. C., Hern´ an, M. A. & Steingrimsson, J. A. Efficient and robust methods for causally interpretable meta-analysis: Transporting inferences from multiple randomized trials to a target population.Biometrics79,1057– 1072 (2023)
2023
-
[37]
& Pearl, J.Transportability of Causal Effects: Completeness Resultsin AAAI(2012), 698–704
Bareinboim, E. & Pearl, J.Transportability of Causal Effects: Completeness Resultsin AAAI(2012), 698–704
2012
-
[38]
VanderWeele, T. J. On the distinction between interaction and effect modification. Epidemiology,863–871 (2009)
2009
-
[39]
Glasziou, P. P. & Irwig, L. M. An evidence based approach to individualising treatment. British Medical Journal311,1356–1359 (1995)
1995
-
[40]
J., Robertson, S
Dahabreh, I. J., Robertson, S. E. & Steingrimsson, J. A. Learning about treatment effects in a new target population under transportability assumptions for relative effect measures.European Journal of Epidemiology,1–9 (2024). 35
2024
-
[41]
Wang, G., Levis, A., Steingrimsson, J. & Dahabreh, I.Causal inference under trans- portability assumptions for conditional relative effect measures2024. arXiv: 2402.02702 [stat.ME]
-
[42]
& Imbens, G
Athey, S. & Imbens, G. W. Identification and inference in nonlinear difference-in- differences models.Econometrica74,431–497 (2006)
2006
-
[43]
Difference-in-differences with variation in treatment timing.Jour- nal of econometrics225,254–277 (2021)
Goodman-Bacon, A. Difference-in-differences with variation in treatment timing.Jour- nal of econometrics225,254–277 (2021)
2021
-
[44]
Hampel, F. R. The influence curve and its role in robust estimation.Journal of the American Statistical Association69,383–393 (1974)
1974
-
[45]
J., Klaassen, C
Bickel, P. J., Klaassen, C. A., Wellner, J. A. & Ritov, Y.Efficient and adaptive esti- mation for semiparametric models(Johns Hopkins University Press Baltimore, 1993)
1993
-
[46]
Tsiatis, A.Semiparametric theory and missing data(Springer Science & Business Me- dia, 2007)
2007
-
[47]
Kennedy, E. H. Semiparametric doubly robust targeted double machine learning: a review.Handbook of Statistical Methods for Precision Medicine,207–236 (2024)
2024
- [48]
-
[49]
Chernozhukov, V.et al.Double/debiased/neyman machine learning of treatment ef- fects.American Economic Review107,261–65 (2017)
2017
-
[50]
Chernozhukov, V.et al.Double/debiased machine learning for treatment and structural parameters.The Econometrics Journal21,C1–C68 (2018)
2018
-
[51]
Bancroft, T. A. On biases in estimation due to the use of preliminary tests of signifi- cance.The Annals of Mathematical Statistics15,190–204 (1944)
1944
-
[52]
Giles, J. A. & Giles, D. E. Pre-test estimation and testing in econometrics: recent developments.Journal of Economic Surveys7,145–197 (1993). 36
1993
-
[53]
J., Matthews, A., Steingrimsson, J
Dahabreh, I. J., Matthews, A., Steingrimsson, J. A., Scharfstein, D. O. & Stuart, E. A. Using trial and observational data to assess effectiveness: Trial emulation, transporta- bility, benchmarking, and joint analysis.Epidemiologic Reviews,mxac011 (2023)
2023
-
[54]
Stefanski, L. A. & Boos, D. D. The calculus of M-estimation.The American Statistician 56,29–38 (2002)
2002
-
[55]
& Tibshirani, R
Efron, B. & Tibshirani, R. J.An Introduction to the Bootstrap(Chapman & Hall/CRC, 1994)
1994
-
[56]
E.et al.Double robust variance estimation with parametric working models.Biometrics81,ujaf054 (2025)
Shook-Sa, B. E.et al.Double robust variance estimation with parametric working models.Biometrics81,ujaf054 (2025)
2025
-
[57]
Wu, H.et al. Why Is the Double-Robust Estimator for Causal Inference Not Doubly Robust for Variance Estimation?2025. arXiv: 2511.17907[stat.ME]
-
[58]
E., Steingrimsson, J
Robertson, S. E., Steingrimsson, J. A. & Dahabreh, I. J. Using numerical methods to design simulations: revisiting the balancing intercept.American Journal of Epidemiol- ogy191,1283–1289 (2022)
2022
-
[59]
M.et al.Risk of myocardial infarction in patients with psoriasis.Journal of the American Medical Association296,1735–1741 (2006)
Gelfand, J. M.et al.Risk of myocardial infarction in patients with psoriasis.Journal of the American Medical Association296,1735–1741 (2006)
2006
-
[60]
E., Armstrong, A
Griffiths, C. E., Armstrong, A. W., Gudjonsson, J. E. & Barker, J. N. Psoriasis.The Lancet397,1301–1315 (2021)
2021
-
[61]
Krumholz, H. M. & Waldstreicher, J. The Yale Open Data Access (YODA) project– a mechanism for data sharing.The New England Journal of Medicine375,403–405 (2016)
2016
-
[62]
Hern´ an, M. A. & Robins, J. M. Using big data to emulate a target trial when a ran- domized trial is not available.American Journal of Epidemiology183,758–764 (2016). 37
2016
-
[63]
A., Dahabreh, I
Hern´ an, M. A., Dahabreh, I. J., Dickerman, B. A. & Swanson, S. A. The target trial framework for causal inference from observational data: why and when is it helpful? Annals of Internal Medicine178,402–407 (2025)
2025
-
[64]
& Dahabreh, I
Ung, L. & Dahabreh, I. J. Keep asking: What do I want? What do I have? What do I do? An approach for combining information to learn about a target population. European Journal of Epidemiology,1–10 (2025)
2025
-
[65]
Marginal Structural Models, 1997 Proceedings of the American Statistical Association, section on Bayesian statistical science, pp
Robins, J. Marginal Structural Models, 1997 Proceedings of the American Statistical Association, section on Bayesian statistical science, pp. 1–10 (1998)
1997
-
[66]
L.et al.Etanercept as monotherapy in patients with psoriasis.New Eng- land Journal of Medicine349,2014–2022 (2003)
Leonardi, C. L.et al.Etanercept as monotherapy in patients with psoriasis.New Eng- land Journal of Medicine349,2014–2022 (2003)
2014
-
[67]
Dahabreh, I. J., Ung, L., Hern´ an, M. A. & Chiu, Y.-H.The role of assignment in defining and identifying causal effects in randomized trials2025. arXiv: 2408.14710 [stat.ME]. 38
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.