Pith. sign in

REVIEW 2 major objections 3 minor 2 references

Dealing with positivity violations in mediation analysis via weighted controlled effects, with application to assessing immune correlates of protection in antigen-experienced participants

T0 review · 2 major / 3 minor · reviewed 2026-05-10 · grok-4.3

Challenge this review Download PDF Read on arXiv ↗

Pith's one-line read A weighted controlled risk approach recovers valid estimates of immune marker effects in populations with prior antigen exposure by focusing on achievable levels.

desk verdict The paper provides a weighted controlled risk method to address positivity violations in mediation analysis for vaccine immune correlates by targeting attainable subpopulations. read the letter →

arxiv 2604.06407 v1 submitted 2026-04-07 stat.ME

Qijia He , Bo Zhang This is my paper · ORCID
classification stat.ME
keywords causalmediationanalysispositivityviolationcontrolledeffectsimmunecorrelatesofprotectionvaccineresearchweightedestimationantigen-experiencedpopulations
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Standard controlled effects methods for mediation analysis in vaccine research assume it is possible to set any post-vaccination immune marker to a fixed level for the whole population, but this breaks down in antigen-experienced groups where baseline levels already exceed some targets. The paper develops a weighted version that restricts attention to a prespecified subpopulation with positive probability of reaching the target marker level, allowing estimation of controlled risks and contrasts within that group. This restores identification under the violated positivity assumption while preserving the original causal interpretation for the relevant subpopulation. The method is shown to work in simulations and is used to reanalyze neutralizing antibody data against Omicron variants from the COVAIL trial. By targeting the subpopulation explicitly, the approach makes mediation analysis applicable to real-world vaccine studies involving influenza, dengue, and later-stage COVID-19 participants.

What carries the argument

The weighted controlled risk estimator, which reweights observations to isolate the subpopulation with positive probability of reaching the fixed immune marker level and thereby restores identifiability.

What would settle it

In a simulation where the true controlled risk is known for the target subpopulation, the weighted estimator would be falsified if it fails to recover the correct value when positivity is violated but recovers it when positivity holds.

Watch

Extended reading notes

Core claim

The paper introduces a weighted controlled risk approach that targets a subpopulation for whom there is a prespecified probability of attaining a post-vaccination immune marker level. It further generalizes this framework to study contrasts of controlled risks for relevant subpopulations, thereby addressing positivity violations that arise in causal mediation analysis for antigen-experienced populations.

Load-bearing premise

A well-defined subpopulation exists with prespecified positive probability of attaining the target immune marker level, and the weighting scheme produces unbiased estimates of the controlled risks without introducing new bias.

Editorial extensions

If this is right

  • Controlled risk curves become estimable for the relevant subpopulation even when the full population violates positivity.
  • Contrasts between controlled risks at different marker levels can be compared within the same subpopulation.
  • Reanalysis of existing trial data, such as the COVAIL Omicron neutralizing antibody results, yields subpopulation-specific immune correlate estimates.
  • Simulation studies confirm that the weighted estimators remain consistent under the stated conditions.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The method could be applied to reanalyze legacy vaccine datasets from antigen-experienced cohorts without redesigning trials.
  • Extensions might combine the weighting with time-varying marker measurements to handle dynamic immune responses.
  • Sensitivity analyses around the choice of target probability could reveal how robust the subpopulation findings are to that tuning parameter.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 3 minor

Summary. The manuscript introduces a weighted controlled risk estimator for causal mediation analysis that re-targets inference to a subpopulation defined by a prespecified positive probability of attaining a target post-vaccination immune marker level. This modification is intended to circumvent positivity violations that arise in antigen-experienced populations (where baseline levels may already exceed the target). The framework is further extended to contrasts of controlled risks across relevant subpopulations. Validity is assessed via simulation studies, and the method is applied to re-analyze neutralizing antibody titers against Omicron BA.4/BA.5 as an immune correlate in the COVAIL trial. R code is provided on GitHub.

Significance. If the estimators are correctly derived and the weighting scheme avoids introducing new bias, the approach offers a transparent, assumption-explicit extension of standard controlled-effects mediation tools that is directly relevant to vaccine research in previously exposed populations (influenza, dengue, later-phase COVID-19). The simulation validation and open-source implementation are positive features that support reproducibility and practical adoption.

major comments (2)
  1. [§3.2, Eq. (8)] §3.2, Eq. (8): the identification result for the weighted controlled risk relies on the weighting function w(A) being correctly specified as the inverse probability of attaining the target level conditional on baseline covariates; however, the manuscript does not provide a formal proof that this weighting preserves the causal interpretation under the stated positivity relaxation, nor does it derive the asymptotic variance that accounts for estimation of the weight.
  2. [§4.1] §4.1, simulation design: the data-generating process enforces the prespecified probability exactly by construction, which may overstate finite-sample performance; a more realistic sensitivity analysis varying the degree of positivity violation and the accuracy of the estimated probability would strengthen the claim that the estimator remains unbiased in practice.
minor comments (3)
  1. The notation for the target subpopulation probability (denoted p in the text) is introduced without an explicit link to the observed data distribution; adding a short paragraph clarifying how p is chosen or estimated from the sample would improve clarity.
  2. Figure 2 (real-data application) lacks error bars or confidence bands on the controlled risk curves; including pointwise intervals would allow readers to assess precision.
  3. The abstract states that the method 'targets a subpopulation for whom there is a prespecified probability,' but the main text does not discuss how sensitive results are to the choice of this probability value; a brief sensitivity table would be useful.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their thoughtful and constructive comments. We respond to each major comment below and describe the revisions we intend to implement.

read point-by-point responses

  1. Referee: [§3.2, Eq. (8)] §3.2, Eq. (8): the identification result for the weighted controlled risk relies on the weighting function w(A) being correctly specified as the inverse probability of attaining the target level conditional on baseline covariates; however, the manuscript does not provide a formal proof that this weighting preserves the causal interpretation under the stated positivity relaxation, nor does it derive the asymptotic variance that accounts for estimation of the weight.

    Authors: We agree that a more explicit derivation would enhance the rigor of the presentation. The weighting function w(A) is defined as the inverse of the conditional probability of attaining the target marker level given baseline covariates, which allows us to re-target the inference to the subpopulation satisfying the relaxed positivity condition. This follows from re-expressing the controlled risk as an expectation over the weighted distribution. To address the comment, we will include a formal proof of identification in the supplementary materials, along with the derivation of the asymptotic variance using the influence function that incorporates the estimation of the weights via a parametric model or nonparametric approach. revision: yes

  2. Referee: [§4.1] §4.1, simulation design: the data-generating process enforces the prespecified probability exactly by construction, which may overstate finite-sample performance; a more realistic sensitivity analysis varying the degree of positivity violation and the accuracy of the estimated probability would strengthen the claim that the estimator remains unbiased in practice.

    Authors: The referee correctly notes that our current simulation enforces the probability by construction, which simplifies the setting. We will revise the simulation section to include additional scenarios that vary the degree of positivity violation (e.g., by adjusting the distribution of baseline levels relative to the target) and incorporate estimation error in the probability weights. This will involve generating data where the true probability is estimated from finite samples and assessing bias and coverage under these more realistic conditions. revision: yes

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; derivation is a direct, assumption-transparent extension of standard controlled-effects mediation

full rationale

The paper introduces a weighted controlled risk estimator that re-targets inference to a prespecified subpopulation (those with positive probability of attaining the target post-vaccination marker level) to circumvent positivity violations in antigen-experienced populations. This is presented as a modification of the existing controlled-effects framework rather than a recovery of the original full-population quantity. Validity is demonstrated via simulation studies and re-analysis of the COVAIL trial; no load-bearing step reduces by the paper's own equations to a fitted parameter, self-definition, or self-citation chain. The weighting scheme is an external, transparent adjustment with no internal reduction to inputs by construction.

Assumptions & free parameters 0 free parameters · 0 assumptions · 0 invented entities

Abstract-only review provides no explicit free parameters, axioms, or invented entities; the method presumably relies on standard causal assumptions plus a new weighting construction whose details are not visible here.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Dealing with positivity violations in mediation analysis via weighted controlled effects, with application to assessing immune correlates of protection in antigen-experienced participants." pith.science (2026). https://pith.science/paper/2604.06407

@misc{pith2026260406407,
  author       = {Pith},
  title        = {Pith review of: Dealing with positivity violations in mediation analysis via weighted controlled effects, with application to assessing immune correlates of protection in antigen-experienced participants},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2604.06407}},
  note         = {Machine review of arXiv:2604.06407}
}
read the original abstract

Causal mediation analysis has become an important and increasingly used framework for evaluating candidate immune response biomarkers in vaccine research. A controlled effects approach has been proposed to estimate controlled risk curves under a counterfactual scenario in which the entire study population is vaccinated and their post-vaccination immune responses are set to a range of fixed levels. This framework performs well when the study population is antigenically na\"ive, that is, individuals have not been previously exposed to the antigen, as is common in HIV-1 vaccine research and during the early phases of the COVID-19 pandemic. However, the controlled effects framework becomes more challenging to apply in antigen-experienced populations, where prior vaccination or infection has occurred, as in the case of influenza, dengue, and more recent phases of the COVID-19 pandemic. In such settings, a key identification assumption for valid causal mediation analysis, the positivity assumption, is violated: it is no longer plausible to conceive of a hypothetical intervention that sets a post-vaccination immune marker to a fixed level below an individual's baseline immune level. In this article, we introduce a weighted controlled risk approach that targets a subpopulation for whom there is a prespecified probability of attaining a post-vaccination immune marker level. We further generalize this framework to study contrasts of controlled risks for relevant subpopulations. We demonstrate the validity of the proposed estimators through simulation studies and apply the method to reanalyze post-vaccination neutralizing antibody titers against Omicron BA.4/BA.5 as an immune correlate of COVID-19 in the Coronavirus Variant Immunologic Landscape (COVAIL) trial. R code to implement the proposed method can be found on Github: https://github.com/Qijia-He/weighted_CVE.

Figures

Figures reproduced from arXiv: 2604.06407 by the authors.

Figure 1
Figure 1. Sampling distributions of EIF-based estimator [PITH_FULL_IMAGE:figures/full_fig_p019_1.png] view at source ↗
Figure 2
Figure 2. The joint distribution of baseline (Day 1, D1) and approximately peak (Day 15, D15) nAb [PITH_FULL_IMAGE:figures/full_fig_p022_2.png] view at source ↗
Figure 3
Figure 3. Estimated STWCRVE(1, 1, s1, s0 = 3.5) (left panel) and STWCRVE(1, 1, s1, s0 = 3.7) (right panel) with pointwise 95% confidence intervals for selected values of s1 > s0. containing vaccine and at least 10% probability of achieving a 3.5-log D15 nAb-ID50 titer when receiving the Prototype vaccine, the controlled relative VE was estimated to be 24.2% (95%: -28.0% to 55.2%; subpopulation controlled risk = 33.1% vs. 43.7… view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Controlled direct effect with pointwise 95% confidence intervals comparing Omicron [PITH_FULL_IMAGE:figures/full_fig_p024_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

2 extracted references · 2 canonical work pages

  1. [1]

    Albert, J. M. (2008). Mediation analysis via potential outcomes models.Statistics in Medicine, 27(8):1282–1304. Benkeser, D., Fong, Y., Janes, H. E., Kelly, E. J., Hirsch, I., Sproule, S., Stanley, A. M., Maaske, J., Villafana, T., Houchens, C. R., et al. (2023). Immune correlates analysis of a phase 3 trial of the AZD1222 (ChAdOx1 nCoV-19) vaccine.npj Va...

    work page arXiv 2008

  2. [2]

    Throughout, we follow the cross-fitting framework of Chernozhukov et al

    S2 Cross-fitting EIF-based estimators This section describes the cross-fitting procedure used to construct the one-step estimators for the smoothed and trimmed parameters underlying STWCR(a, s) and STWCRVE(s1, s0). Throughout, we follow the cross-fitting framework of Chernozhukov et al. (2018). LetO i = (Yi, Ai, Si, Bi,X i),i= 1, . . . , n, be an i.i.d. s...

    work page 2018

Pith tools

Reviewed May 10, 2026 · model on record in the stance chip above.