pith. sign in

arxiv: 2605.28785 · v1 · pith:ON3YN7J2new · submitted 2026-05-27 · 📊 stat.ME

Beyond Exchangeability: Distribution-Shift-Aware Integration of External Control Data in Randomized Trials

Jiawei Shan , Yiteng Tu , Guanbo Wang , Chao Ying , Jiwei Zhao This is my paper

Pith reviewed 2026-06-29 10:23 UTC · model grok-4.3

classification 📊 stat.ME
keywords external controlsrandomized trialsdistribution shiftcalibration equationsefficient influence functionshrinkage estimatorcausal inferenceaugmented estimation
0
0 comments X

The pith

External control data augments randomized trials by balancing populations with calibration equations even when exchangeability fails.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper establishes a way to incorporate external controls into RCTs when differences in eligibility, care standards, or data procedures create distribution shifts between groups. It adapts the trial's efficient influence functions by solving calibration equations that reweight or adjust the external data to match the trial population on key covariates. This produces augmented estimators that use the external data fully without requiring the populations to be exchangeable. An adaptive shrinkage version of the estimator is shown to remain consistent for the trial effect while being more efficient than the trial-only benchmark. Synthetic experiments and a real-data example illustrate the efficiency gains under realistic shift conditions.

Core claim

Augmented estimators are constructed by adapting trial-only efficient influence functions through calibration equations that balance the trial and external populations, thereby fully exploiting the external control data even when exchangeability fails; an adaptive shrinkage estimator is developed that preserves consistency while guaranteeing efficiency dominance over the trial-only benchmark.

What carries the argument

Calibration equations that balance the trial and external populations with respect to covariates capturing distribution shifts, together with an adaptive shrinkage estimator applied to the resulting augmented influence functions.

If this is right

  • The method yields estimators that remain consistent for the trial causal effect while attaining lower asymptotic variance than trial-only estimators.
  • Efficiency gains increase as the external sample size grows, provided the calibration equations achieve balance.
  • The adaptive shrinkage step automatically down-weights external information when shifts are large, protecting against efficiency loss.
  • The framework applies directly to settings where shifts arise from eligibility criteria, standard-of-care differences, or measurement procedures.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same calibration approach could be extended to time-to-event or longitudinal outcomes by replacing the influence function accordingly.
  • In practice, the method might allow smaller RCTs when external controls are abundant but shifted, provided the balancing covariates are measured.
  • If the calibration covariates miss important effect modifiers, the efficiency gain would be incomplete but consistency for the trial population would still hold.

Load-bearing premise

Calibration equations can be constructed that balance the trial and external populations on the covariates driving the distribution shifts.

What would settle it

A dataset in which no finite set of calibration weights or equations can simultaneously match the trial and external covariate distributions on variables that affect the outcome would cause the augmented estimators to lose their efficiency advantage or introduce bias.

Figures

Figures reproduced from arXiv: 2605.28785 by Chao Ying, Guanbo Wang, Jiawei Shan, Jiwei Zhao, Yiteng Tu.

Figure 1
Figure 1. Figure 1: Empirical bias and standard deviation as the amount of validation information varies. [PITH_FULL_IMAGE:figures/full_fig_p010_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Empirical bias under four working-model misspecification scenarios. [PITH_FULL_IMAGE:figures/full_fig_p011_2.png] view at source ↗
read the original abstract

Randomized controlled trials (RCTs) are the gold standard for evaluating causal effects but are often costly and difficult to scale; consequently, they are frequently augmented with auxiliary external controls in many applications. Prior approaches for borrowing such data typically rely on exchangeability, under which the external controls are readily usable for inference in the trial population. In practice, however, differences in eligibility criteria, standard of care, and data collection procedures may induce distribution shifts between the RCT and the external controls, rendering exchangeability implausible. In this paper, we propose a novel framework for integrating external controls by explicitly modeling these distribution shifts. We construct augmented estimators by adapting trial-only efficient influence functions through calibration equations that balance the trial and external populations, thereby fully exploiting the external control data even when exchangeability fails. We further develop an adaptive shrinkage estimator that preserves consistency while guaranteeing efficiency dominance over the trial-only benchmark. Synthetic experiments and a real data application demonstrate the practical advantages of the proposed approaches.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The paper claims to develop a framework for integrating external control data into RCTs without assuming exchangeability, by adapting trial-only efficient influence functions via calibration equations that enforce balance between trial and external covariate distributions to account for shifts from eligibility, standard of care, and data collection. It further proposes an adaptive shrinkage estimator that remains consistent for the trial parameter while guaranteeing efficiency gains over the trial-only benchmark. The approach is illustrated with synthetic experiments and a real-data application.

Significance. If the calibration construction and shrinkage procedure are valid and robust, the work would offer a principled semiparametric route to borrow strength from external controls under realistic distribution shifts, potentially increasing power and reducing costs in trials where exchangeability fails. The explicit efficiency-dominance guarantee is a notable strength relative to many existing borrowing methods.

major comments (2)
  1. [Method (calibration step)] The central construction relies on calibration equations to adapt the trial EIF and balance populations (described in the abstract and method outline). No identifiability conditions, existence/uniqueness results, or regularization strategy are supplied for the case of limited covariate overlap or high-dimensional shifts induced by eligibility/SOC factors; without these, the equations may be ill-posed, producing extreme weights or non-existence and thereby threatening consistency of the augmented estimator.
  2. [Adaptive shrinkage estimator] The adaptive shrinkage estimator is asserted to preserve consistency while dominating the trial-only benchmark in efficiency. The manuscript does not appear to verify that the shrinkage step remains valid when the calibration equations only partially capture the shift (i.e., when relevant covariates are misspecified or incomplete), which is load-bearing for the efficiency claim.
minor comments (2)
  1. [Abstract/Introduction] The abstract states that the method 'fully exploits' external data even when exchangeability fails, but the precise sense in which the estimator remains consistent for the trial parameter (rather than a shifted parameter) should be stated more explicitly in the introduction.
  2. [Notation] Notation for the calibration equations and the efficient influence function adaptation could be clarified with an explicit display of the estimating equations early in the methods section.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their constructive comments, which have helped us improve the manuscript. We address each major comment below and indicate the revisions we will make.

read point-by-point responses

  1. Referee: [Method (calibration step)] The central construction relies on calibration equations to adapt the trial EIF and balance populations (described in the abstract and method outline). No identifiability conditions, existence/uniqueness results, or regularization strategy are supplied for the case of limited covariate overlap or high-dimensional shifts induced by eligibility/SOC factors; without these, the equations may be ill-posed, producing extreme weights or non-existence and thereby threatening consistency of the augmented estimator.

    Authors: We agree that the manuscript would benefit from explicit discussion of identifiability conditions for the calibration equations. In the revised version, we will add a subsection detailing the necessary assumptions for existence and uniqueness of the calibration weights, including requirements on covariate overlap. We will also introduce a regularization approach, such as a penalized calibration or truncation of weights, to handle cases of limited overlap or high-dimensional shifts. This will strengthen the theoretical foundation of the method. revision: yes

  2. Referee: [Adaptive shrinkage estimator] The adaptive shrinkage estimator is asserted to preserve consistency while dominating the trial-only benchmark in efficiency. The manuscript does not appear to verify that the shrinkage step remains valid when the calibration equations only partially capture the shift (i.e., when relevant covariates are misspecified or incomplete), which is load-bearing for the efficiency claim.

    Authors: The referee correctly identifies a potential limitation. The efficiency dominance is guaranteed under the assumption that the calibration equations adequately capture the distribution shift. In the revision, we will explicitly state this assumption and provide a discussion on the consequences of misspecification, including additional simulation studies to assess robustness when relevant covariates are omitted. We will also clarify that the consistency is preserved regardless, but efficiency gains may be reduced under partial capture of the shift. revision: partial

Circularity Check

0 steps flagged

No circularity; derivation builds on standard semiparametric EIFs and external calibration without self-referential reduction.

full rationale

The paper's central construction adapts trial-only efficient influence functions via calibration equations that balance covariate distributions between trial and external populations. This relies on established semiparametric theory for EIFs rather than defining quantities in terms of the paper's own fitted outputs or predictions. The adaptive shrinkage estimator is presented as preserving consistency while dominating the trial-only benchmark, with no indication that its properties reduce by construction to inputs defined within the paper itself. No self-citation chains, ansatzes smuggled via prior work, or renaming of known results are load-bearing in the abstract or described framework. The approach is self-contained against external benchmarks from semiparametric statistics.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

Based solely on the abstract. The approach rests on standard causal inference tools (efficient influence functions) and the domain assumption that calibration equations can be formed to handle shifts. No free parameters or invented entities are mentioned.

axioms (1)
  • domain assumption Efficient influence functions from trial-only estimators can be adapted via calibration equations to account for distribution shifts between RCT and external populations.
    This is the central modeling step described in the abstract.

pith-pipeline@v0.9.1-grok · 5709 in / 1323 out tokens · 35947 ms · 2026-06-29T10:23:01.280653+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

31 extracted references · 5 canonical work pages

  1. [1]

    D., Abad, J., Wang, G., Donhauser, K., Duch, R

    Bartolomeis, P. D., Abad, J., Wang, G., Donhauser, K., Duch, R. M., Yang, F., and Dahabreh, I. (2026). Efficient randomized experiments using foundation models. In The Thirty-ninth Annual Conference on Neural Information Processing Systems

    2026

  2. [2]

    Colnet, B., Mayer, I., Chen, G., Dieng, A., Li, R., Varoquaux, G., Vert, J.-P., Josse, J., and Yang, S. (2024). Causal inference methods for combining randomized trials and observational studies: A review. Statistical Science , 39(1):165--191

    2024

  3. [3]

    J., Robertson, S

    Dahabreh, I. J., Robertson, S. E., Tchetgen, E. J., Stuart, E. A., and Hern \'a n, M. A. (2019). Generalizing causal inferences from individuals in randomized trials to all trial-eligible individuals. Biometrics , 75(2):685--694

    2019

  4. [4]

    and Wahba, S

    Dehejia, R. and Wahba, S. (1999). Causal effects in nonexperimental studies: Reevaluating the evaluation of training programs. Journal of the American Statistical Association , 94(448):1053--1062

    1999

  5. [5]

    Gao, C., Yang, S., Shan, M., Y E, W., Lipkovich, I., and Faries, D. (2024). Improving randomized controlled trial analysis via data-adaptive borrowing. Biometrika

    2024

  6. [6]

    Ghassami, A., Yang, A., Richardson, D., Shpitser, I., and Tchetgen, E. T. (2022). Combining experimental and observational data for identification and estimation of long-term causal effects. arXiv preprint arXiv:2201.10743

    work page arXiv 2022

  7. [7]

    P., Sargent, D

    Hobbs, B. P., Sargent, D. J., and Carlin, B. P. (2012). Commensurate priors for incorporating historical information in clinical trials using general and generalized linear models. Bayesian Analysis , 7(3):639--674

    2012

  8. [8]

    Imbens, G. W. and Rubin, D. B. (2015). Causal Inference for Statistics, Social, and Biomedical Sciences: An Introduction . Cambridge University Press, Cambridge

    2015

  9. [9]

    D., Krijthe, J

    Karlsson, R., Wang, G., Bartolomeis, P. D., Krijthe, J. H., and Dahabreh, I. J. (2025). Robust integration of external control data in randomized trials. arXiv preprint arXiv:2406.17971

    work page arXiv 2025

  10. [10]

    Lalonde, R. (1986). Evaluating the econometric evaluations of training-programs with experimental-data. American Economic Review , 76(4):604--620

    1986

  11. [11]

    and Luedtke, A

    Li, S. and Luedtke, A. (2023). Efficient estimation under data fusion. Biometrika , 110(4):1041--1054

    2023

  12. [12]

    Li, X., Miao, W., Lu, F., and Zhou, X.-H. (2023). Improving efficiency of inference in clinical trials with external control data. Biometrics , 79(1):394--403

    2023

  13. [13]

    D., H jbjerre-Frandsen , E., Hubbard, A

    Liao, L. D., H jbjerre-Frandsen , E., Hubbard, A. E., and Schuler, A. (2025). Prognostic adjustment with efficient estimators to unbiasedly leverage historical data in randomized trials. The International Journal of Biostatistics , 21(1):1--15

    2025

  14. [14]

    , year =

    Lin, X., Tarp, J. M., and Evans, R. J. (2024). Data fusion for efficiency gain in ATE estimation: A practical review with simulations. arXiv preprint arXiv:2407.01186

    work page arXiv 2024

  15. [15]

    Newey, W. K. and McFadden, D. (1994). Large sample estimation and hypothesis testing. In Handbook of Econometrics , volume 4, pages 2111--2245. Elsevier

    1994

  16. [16]

    Oberst, M., D'Amour, A., Chen, M., Wang, Y., Sontag, D., and Yadlowsky, S. (2022). Understanding the risks and rewards of combining unbiased and possibly biased estimators, with applications to causal inference

    2022

  17. [17]

    Parikh, H., Morucci, M., Orlandi, V., Roy, S., Rudin, C., and Volfovsky, A. (2025). A double machine learning approach for combining experimental and observational studies. Observational Studies , 11(3):249--300

    2025

  18. [18]

    Pocock, S. J. (1976). The combination of randomized and historical controls in clinical trials. Journal of Chronic Diseases , 29(3):175--188

    1976

  19. [19]

    M., Rotnitzky, A., and Zhao, L

    Robins, J. M., Rotnitzky, A., and Zhao, L. (1994). Estimation of regression-coefficients when some regressors are not always observed. Journal of the American Statistical Association , 89(427):846--866

    1994

  20. [20]

    Rosenbaum, P. R. and Rubin, D. B. (1983). The central role of the propensity score in observational studies for causal effects. Biometrika , 70(1):41--55

    1983

  21. [21]

    T., Basse, G., Owen, A

    Rosenman, E. T., Basse, G., Owen, A. B., and Baiocchi, M. (2023). Combining observational and experimental datasets using shrinkage estimators. Biometrics , 79(4):2961--2973

    2023

  22. [22]

    Schmidli, H., Gsteiger, S., Roychoudhury, S., O'Hagan, A., Spiegelhalter, D., and Neuenschwander, B. (2014). Robust meta-analytic-predictive priors in clinical trials with historical control information. Biometrics , 70(4):1023--1032

    2014

  23. [23]

    Schuler, A., Walsh, D., Hall, D., Walsh, J., Fisher, C., Disease, f. t. C. P. f. A., Initiative, t. A. D. N., and Study, t. A. D. C. (2022). Increasing the efficiency of randomized trial estimates via linear adjustment for a prognostic score. The International Journal of Biostatistics , 18(2):329--356

    2022

  24. [24]

    R., Jonsson Funk, M., and Kosorok, M

    Valancius, M., Pang, H., Zhu, J., Cole, S. R., Jonsson Funk, M., and Kosorok, M. R. (2024). A causal inference framework for leveraging external controls in hybrid trials. Biometrics , 80(4):ujae095

    2024

  25. [25]

    G., Kinnersley, N., Lindborg, S., Micallef, S., Roychoudhury, S., and Thompson, L

    Viele, K., Berry, S., Neuenschwander, B., Amzal, B., Chen, F., Enas, N., Hobbs, B., Ibrahim, J. G., Kinnersley, N., Lindborg, S., Micallef, S., Roychoudhury, S., and Thompson, L. (2014). Use of historical control data for assessing treatment effects in clinical trials. Pharmaceutical Statistics , 13(1):41--54

    2014

  26. [26]

    Wang, G., Karlsson, R., De Bartolomeis, P., Robins, J., Tchetgen Tchetgen, E., and Dahabreh, I. J. (2024a). Robust trial augmentation using external data. arXiv preprint arXiv:2505.01201

    work page arXiv

  27. [27]

    W., Pang, M., and Koukounari, A

    Wang, G., Poulin-Costello, M., Pang, H., Zhu, J., Helms, H.-J., Reyes-Rivera, I., Platt, R. W., Pang, M., and Koukounari, A. (2024b). Evaluating hybrid controls methodology in early-phase oncology trials: A simulation study based on the morpheus-uc trial. Pharmaceutical Statistics , 23(1):31--45

  28. [28]

    and Yang, S

    Wu, L. and Yang, S. (2023). Transfer learning of individualized treatment rules from experimental to real-world data. Journal of Computational and Graphical Statistics , 32(3):1036--1045

    2023

  29. [29]

    Yang, Q., Li, J., and Wu, P. (2025). Adaptive data-borrowing for improving treatment effect estimation using external controls. arXiv preprint arXiv:2508.03282

    work page arXiv 2025

  30. [30]

    and Ding, P

    Yang, S. and Ding, P. (2020). Combining multiple observational data sources to estimate causal effects. Journal of the American Statistical Association , 115(531):1540--1554

    2020

  31. [31]

    Yang, S., Gao, C., Zeng, D., and Wang, X. (2023). Elastic integrative analysis of randomised trial and real-world data for treatment heterogeneity estimation. Journal of the Royal Statistical Society Series B: Statistical Methodology , 85(3):575--596

    2023