Target trial emulation without matching: a more efficient approach for evaluating vaccine effectiveness using observational data

David Benkeser; Elizabeth Rogawski McQuade; Emily Wu; Mats Stensrud; Razieh Nabi

arxiv: 2504.17104 · v1 · submitted 2025-04-23 · 📊 stat.ME · stat.AP

Target trial emulation without matching: a more efficient approach for evaluating vaccine effectiveness using observational data

Emily Wu , Elizabeth Rogawski McQuade , Mats Stensrud , Razieh Nabi , David Benkeser This is my paper

Pith reviewed 2026-05-22 17:37 UTC · model grok-4.3

classification 📊 stat.ME stat.AP

keywords vaccine effectivenesstarget trial emulationobservational datahazard regressioncausal estimandCOVID-19 vaccineefficiency gains

0 comments

The pith

A hazard regression approach estimates vaccine effectiveness over calendar time more efficiently than matching in observational studies.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper proposes a causal estimand for vaccine effectiveness that averages performance across calendar time, identified from observational data using two hazard regression models rather than matching vaccinated and unvaccinated individuals. A sympathetic reader would care because matching discards information and can blur the target question when vaccine uptake and infection rates shift rapidly over time. The method aims to produce the same scientific conclusions with substantially narrower uncertainty intervals from the same data. It is demonstrated in simulations and in a study of the Pfizer-BioNTech COVID-19 vaccine among children aged 5-11.

Core claim

The paper claims that a time-averaged causal estimand for vaccine effectiveness can be identified from observational data by fitting two hazard regression models—one for the vaccination process and one for the infection outcome—under the assumptions that all relevant confounders are measured and the models are correctly specified. This estimand summarizes effectiveness similarly to how randomized trials report it, and the corresponding estimator is more efficient than standard matching-based target trial emulations while yielding comparable inferences.

What carries the argument

The time-averaged estimand of vaccine effectiveness, identified and estimated via two hazard regression models that capture the time-varying nature of vaccination uptake and infection risk.

If this is right

Comparable scientific inferences about vaccine effectiveness are obtained in both simulation and real data applications.
Significant efficiency gains are achieved compared to matching-based estimators.
The approach handles time-varying vaccine uptake and changing infection dynamics naturally.
Simple implementation is possible with standard statistical software for hazard models.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

Researchers studying other time-dependent interventions in epidemiology could adapt this framework to gain precision without matching.
Smaller observational cohorts might become sufficient for detecting moderate effects if this method replaces matching.
Validation in additional disease settings beyond COVID-19 would test the robustness of the efficiency advantage.

Load-bearing premise

All confounders of both vaccination and infection are measured and the two hazard regression models are correctly specified.

What would settle it

Applying the estimator to a dataset where matching-based methods show a significant effect but this approach does not, or finding absent efficiency gains in repeated simulations under known conditions, would challenge the claim of similar inferences with better efficiency.

read the original abstract

Real-world vaccine effectiveness has increasingly been studied using matching-based approaches, particularly in observational cohort studies following the target trial emulation framework. Although matching is appealing in its simplicity, it suffers important limitations in terms of clarity of the target estimand and the efficiency or precision with which is it estimated. Scientifically justified causal estimands of vaccine effectiveness may be difficult to define owing to the fact that vaccine uptake varies over calendar time when infection dynamics may also be rapidly changing. We propose a causal estimand of vaccine effectiveness that summarizes vaccine effectiveness over calendar time, similar to how vaccine efficacy is summarized in a randomized controlled trial. We describe the identification of our estimand, including its underlying assumptions, and propose simple-to-implement estimators based on two hazard regression models. We apply our proposed estimator in simulations and in a study to assess the effectiveness of the Pfizer-BioNTech COVID-19 vaccine to prevent infections with SARS-CoV2 in children 5-11 years old. In both settings, we find that our proposed estimator yields similar scientific inferences while providing significant efficiency gains over commonly used matching-based estimators.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

Offers a hazard-model route to time-averaged vaccine effectiveness with claimed efficiency gains over matching, but the gains rest on untested correct specification of the two regressions.

read the letter

The paper's main contribution is a time-summarized causal estimand for vaccine effectiveness that averages over calendar time, paired with estimators from two hazard regressions instead of the usual matching step in target-trial work. This avoids some of the clarity and efficiency problems that come with matching when uptake and infection rates shift together. In the simulations the estimator recovers the target, and in the Pfizer data for 5-11 year olds it produces point estimates close to the matching version while showing narrower intervals. That efficiency comparison is the concrete result they deliver. The identification argument uses standard no-unmeasured-confounding plus correct specification of the vaccination and infection hazard models. The citation pattern is appropriate for the causal-inference and vaccine-effectiveness literature. The soft spot is exactly the one the stress-test flags: the paper does not run sensitivity checks that vary the functional forms or time trends in those hazard models. In real observational vaccine data those models are easy to misspecify, and under misspecification the reported precision advantage could mask bias. The simulations and application are both run under designs that match the assumed models, so they do not speak to robustness. This work is aimed at epidemiologists and causal-inference researchers who already use target-trial emulation and want a non-matching option for time-varying settings. A reader who needs a practical alternative to matching and is willing to defend the hazard specifications will find usable material here. The paper is coherent on its own terms and shows clear engagement with the matching literature, so it deserves a serious referee even though the robustness checks are missing. I would send it out for review with a request for misspecification diagnostics.

Referee Report

2 major / 2 minor

Summary. The manuscript proposes a matching-free approach to target trial emulation for observational vaccine effectiveness studies. It defines a time-averaged causal estimand that summarizes vaccine effectiveness over calendar time, identifies this estimand under standard no-unmeasured-confounding and correct-model-specification assumptions using two parametric hazard regressions (one for vaccination uptake over time and one for infection risk conditional on vaccination status), and proposes corresponding estimators. Simulations and an application to Pfizer-BioNTech COVID-19 vaccine data in children 5–11 years old are used to claim that the method produces similar scientific conclusions to matching-based estimators while delivering substantial efficiency gains.

Significance. If the identification and estimation assumptions hold, the approach could improve precision in observational vaccine studies by avoiding the sample-size reduction inherent in matching, while still targeting a scientifically interpretable time-averaged quantity analogous to RCT vaccine efficacy summaries. The simulations recover the target parameter under the assumed models and the real-data example shows comparable point estimates with reported efficiency improvements; these are concrete strengths that would support adoption if robustness to model misspecification is demonstrated.

major comments (2)

[Identification and estimation sections] Identification and estimation sections: consistency of the proposed estimator for the time-averaged estimand requires correct specification of both the vaccination hazard regression and the infection hazard regression. The simulation studies recover the target only under correctly specified models; no sensitivity analyses that vary functional forms, time trends, or omitted interactions are reported. Because the efficiency-gain claim and the “similar scientific inferences” conclusion presuppose consistency, this assumption is load-bearing and needs explicit probing before the performance advantage can be considered reliable in settings with rapidly changing infection dynamics.
[Simulation section] Simulation section: the reported efficiency gains are quantified only under the data-generating process that matches the estimation models exactly. Adding at least one misspecification scenario (e.g., misspecified baseline hazard or omitted calendar-time interaction) would directly address whether the precision advantage persists or whether bias is traded for variance reduction.

minor comments (2)

[Abstract] Abstract: the phrase “significant efficiency gains” is used without numerical illustration; adding a brief quantitative statement (e.g., “variance reduced by X%”) would improve clarity for readers who do not reach the tables.
[Notation throughout] Notation: the manuscript introduces two hazard models but does not always use distinct symbols for the vaccination-process parameters versus the outcome-process parameters; consistent subscripting or superscripting would reduce ambiguity when both models appear in the same equation.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive comments and for recognizing the potential of our approach to improve precision in observational vaccine effectiveness studies. We address each major comment point by point below, agreeing where revisions are warranted and explaining our position on the underlying assumptions.

read point-by-point responses

Referee: [Identification and estimation sections] Identification and estimation sections: consistency of the proposed estimator for the time-averaged estimand requires correct specification of both the vaccination hazard regression and the infection hazard regression. The simulation studies recover the target only under correctly specified models; no sensitivity analyses that vary functional forms, time trends, or omitted interactions are reported. Because the efficiency-gain claim and the “similar scientific inferences” conclusion presuppose consistency, this assumption is load-bearing and needs explicit probing before the performance advantage can be considered reliable in settings with rapidly changing infection dynamics.

Authors: We agree that consistency of the proposed estimators for the time-averaged estimand requires correct specification of both the vaccination uptake hazard model and the conditional infection hazard model (in addition to no unmeasured confounding). These assumptions are stated explicitly in the identification section. The simulations were constructed to confirm that the estimators recover the target when the models are correctly specified, which is a standard and necessary validation step for any parametric estimator. We acknowledge that the current version does not include sensitivity analyses under misspecification of functional forms, time trends, or interactions. To address this directly, we will add a new subsection to the simulation studies that examines performance under at least two misspecification scenarios (e.g., omitted calendar-time interactions and misspecified baseline hazard). This will allow assessment of whether efficiency gains persist or whether bias is introduced, thereby strengthening the support for the method in settings with changing infection dynamics. revision: yes
Referee: [Simulation section] Simulation section: the reported efficiency gains are quantified only under the data-generating process that matches the estimation models exactly. Adding at least one misspecification scenario (e.g., misspecified baseline hazard or omitted calendar-time interaction) would directly address whether the precision advantage persists or whether bias is traded for variance reduction.

Authors: The reported efficiency gains are shown under the correctly specified data-generating process because this setting permits a direct comparison of precision between consistent estimators. This is the appropriate benchmark for demonstrating the variance-reduction benefit of avoiding matching. We agree that examining behavior under misspecification is valuable to evaluate potential bias-variance trade-offs. In the revision we will incorporate at least one additional simulation scenario with misspecification, such as a misspecified baseline hazard or an omitted calendar-time interaction in the fitted models. We will report bias, variance, and mean squared error for both the proposed estimator and the matching-based comparator in these scenarios. revision: yes

Circularity Check

0 steps flagged

No significant circularity; central claims rest on standard causal assumptions and externally motivated hazard models.

full rationale

The paper identifies a time-averaged vaccine effectiveness estimand via two hazard regression models (one for vaccination uptake, one for infection risk) under standard no-unmeasured-confounding and correct-specification assumptions. These are invoked explicitly as identification conditions rather than derived from the paper's own equations. Efficiency comparisons to matching estimators are shown via simulation and application data, not forced by redefinition or self-citation chains. No self-definitional steps, fitted inputs renamed as predictions, or load-bearing self-citations appear in the derivation. The result is self-contained against external benchmarks and does not reduce to its inputs by construction.

Axiom & Free-Parameter Ledger

1 free parameters · 2 axioms · 0 invented entities

The method depends on standard causal assumptions for identification and on parametric hazard models whose coefficients are fitted to data; no new entities are postulated.

free parameters (1)

coefficients in the vaccination and infection hazard regression models
Estimated from the observed data to produce the final effectiveness summary.

axioms (2)

domain assumption No unmeasured confounding between vaccination timing and infection risk
Invoked for identification of the causal estimand from observational data.
domain assumption Correct specification of the two hazard regression models
Required for consistent estimation of the time-averaged effect.

pith-pipeline@v0.9.0 · 5740 in / 1362 out tokens · 58213 ms · 2026-05-22T17:37:23.431107+00:00 · methodology

discussion (0)

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

IndisputableMonolith/Cost/FunctionalEquation.lean washburn_uniqueness_aczel unclear

?

unclear
Relation between the paper passage and the cited Recognition theorem.

We describe the identification of our estimand, including its underlying assumptions, and propose simple-to-implement estimators based on two hazard regression models.
IndisputableMonolith/Foundation/AbsoluteFloorClosure.lean absolute_floor_iff_bare_distinguishability unclear

?

unclear
Relation between the paper passage and the cited Recognition theorem.

Assumption 1: Exchangeability of vaccination and censoring given covariates and vaccination history

What do these tags mean?

matches: The paper's claim is directly supported by a theorem in the formal canon.
supports: The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends: The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses: The paper appears to rely on the theorem as machinery.
contradicts: The paper's claim conflicts with a theorem or certificate in the canon.
unclear: Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.