Pith. sign in

REVIEW 2 major objections 2 minor 1 cited by

Estimation and Inference for Win Measures with Multiple Ordinal Endpoints Subject to Missingness

T0 review · 2 major / 2 minor · reviewed 2026-06-29 · grok-4.3

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read IPW and AIPW estimators yield consistent win measures for ordinal endpoints with missing data

desk verdict The paper fills a practical gap with IPW and AIPW estimators for win measures under missing hierarchical ordinal endpoints, backed by simulations and an R package, but consistency still requires correct specification of the joint missingness model. read the letter →

arxiv 2605.27085 v1 pith:7NQUB7RV submitted 2026-05-26 stat.ME math.STstat.APstat.TH

Yi Liu , Huiman Barnhart , Sean O'Brien , Yuliya Lokhnygina , Roland A. Matsouaka This is my paper · ORCID
classification stat.MEmath.STstat.APstat.TH
keywords winratiomeasuresmissingdatainverseprobabilityweightingaugmentedIPWordinalendpointsclinicaltrialsestimation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper develops methods to estimate win measures such as the win ratio when multiple ordinal endpoints have missing values that may depend on treatment and covariates. The standard method of treating incomplete pairs as ties can bias results even when data are missing completely at random. The authors propose inverse probability weighting (IPW) that reweights observed pairs by their probability of being fully observed, and an augmented version (AIPW) that adds outcome modeling for better efficiency and double robustness. They derive closed-form variance estimators and demonstrate through simulations that the new estimators are consistent with good coverage, while the standard approach is not. The methods are applied to two clinical trials to show practical use.

What carries the argument

IPW estimator that reweights pairwise comparisons using estimated joint non-missingness probabilities, and its augmented version that combines with outcome regression for double robustness.

What would settle it

Simulation results or trial data where the missingness probability model is misspecified, leading to biased IPW estimates despite known true win measures.

Watch

Extended reading notes

Core claim

We develop inverse probability weighting (IPW) and augmented IPW (AIPW) estimators for win measures with hierarchical ordinal endpoints subject to missing data, allowing missingness to depend on treatment assignment and baseline covariates. The IPW estimator corrects bias by reweighting complete observed outcomes using joint non-missingness probabilities involved in estimating the joint cell probabilities that define the win measures. The AIPW estimator additionally incorporates outcome modeling, improving efficiency and achieving double robustness. For inference, we derive closed-form variance estimators for both methods based on influence functions.

Load-bearing premise

The probability of joint non-missingness for pairs can be correctly estimated from treatment assignment and baseline covariates, and the missingness mechanism is correctly specified for the IPW to be unbiased.

Editorial extensions

If this is right

  • The standard pairwise-comparison approach produces biased estimates even under MCAR.
  • IPW and AIPW estimators are consistent for the true win measures.
  • AIPW estimator is more efficient than IPW.
  • Variance estimators for IPW and AIPW achieve near-nominal coverage in simulations.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The double robustness property of AIPW protects against misspecification of either the missingness or outcome model.
  • These estimators could be extended to settings with more complex missingness patterns or additional covariates.
  • Adoption of these methods may lead to more reliable conclusions in clinical trials with incomplete ordinal data.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 2 minor

Summary. The manuscript develops IPW and AIPW estimators for win measures (win ratio, win odds, net benefit, and DOOR) defined via pairwise comparisons of multiple hierarchical ordinal endpoints when some endpoints are missing. Missingness is allowed to depend on treatment assignment and baseline covariates under a MAR assumption. The IPW reweights observed pairs by the inverse of the estimated joint non-missingness probability to recover the cell probabilities that define the win measures; AIPW augments this with outcome regression for double robustness and efficiency. Closed-form variance estimators are derived from influence functions. Simulations show the standard approach (treating incomplete pairs as ties) is biased while the proposed estimators are consistent with near-nominal coverage, AIPW being more efficient; applications to SCOUT-CAP and ACTT-1 trials are presented along with the R package WinMO.

Significance. If the derivations and simulation results hold, the work fills a clear methodological gap by extending standard IPW/AIPW theory to win measures with missing hierarchical ordinal data, a setting common in clinical trials. The double-robustness property of AIPW, the closed-form variances, and the provision of an R package for implementation are concrete strengths that support usability and reproducibility. The simulation evidence for consistency and coverage under the stated missingness mechanism is a positive feature.

major comments (2)
  1. [Methods] Methods section (IPW construction): consistency of the IPW estimator for the joint cell probabilities requires correct specification and consistent estimation of the joint non-missingness probability P(both endpoints observed | treatment, covariates). The manuscript states this assumption but does not report any sensitivity analyses or additional simulations under misspecification of the missingness model, which is load-bearing for the practical claim that the IPW estimator remains consistent when applied to real data.
  2. [Simulations] Simulation studies: all reported scenarios assume the missingness model is correctly specified with the same covariates used in estimation. This leaves open whether the reported near-nominal coverage and efficiency advantage of AIPW persist when the missingness mechanism is misspecified, which directly affects the strength of evidence for the central consistency and efficiency claims.
minor comments (2)
  1. [Abstract] The abstract and introduction could more explicitly note that the double robustness of AIPW holds only if at least one of the missingness or outcome models is correctly specified, to avoid overstatement of robustness.
  2. [Methods] Notation for the joint non-missingness probability and the cell-probability estimators could be clarified with an explicit equation linking the reweighting step to the win-measure definitions.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive comments and positive overall assessment. We address each major comment below.

read point-by-point responses

  1. Referee: [Methods] Methods section (IPW construction): consistency of the IPW estimator for the joint cell probabilities requires correct specification and consistent estimation of the joint non-missingness probability P(both endpoints observed | treatment, covariates). The manuscript states this assumption but does not report any sensitivity analyses or additional simulations under misspecification of the missingness model, which is load-bearing for the practical claim that the IPW estimator remains consistent when applied to real data.

    Authors: We agree that the consistency of the IPW estimator relies on correct specification of the missingness model, as stated in the manuscript. Sensitivity analyses under misspecification would strengthen the practical claims. We will add such simulations (including cases where the missingness model omits key covariates or uses an incorrect functional form) to the revised manuscript. Note that the AIPW estimator retains double robustness, providing protection against misspecification of either the missingness or outcome model. revision: yes

  2. Referee: [Simulations] Simulation studies: all reported scenarios assume the missingness model is correctly specified with the same covariates used in estimation. This leaves open whether the reported near-nominal coverage and efficiency advantage of AIPW persist when the missingness mechanism is misspecified, which directly affects the strength of evidence for the central consistency and efficiency claims.

    Authors: We acknowledge that the reported simulations assume correct specification of the missingness model. To directly address this, we will expand the simulation section in revision to include misspecified missingness scenarios. These will evaluate bias, coverage, and relative efficiency of IPW versus AIPW, allowing assessment of whether the efficiency advantage and near-nominal coverage persist. We anticipate AIPW will demonstrate greater robustness due to double robustness. revision: yes

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: standard IPW/AIPW applied to fixed win-measure definitions

full rationale

The derivation starts from the established pairwise win-measure definitions (WR, WO, NB, DOOR) based on hierarchical ordinal endpoints and applies textbook IPW and AIPW reweighting by the joint non-missingness probability P(both observed | treatment, covariates). This produces consistent estimators under the usual MAR assumption without any step in which a fitted quantity is renamed as a prediction, a self-citation supplies a uniqueness theorem, or an ansatz is smuggled in. The closed-form influence-function variances and the simulation results are downstream consequences of the same construction rather than inputs that force the result. The paper therefore remains self-contained against external benchmarks.

Assumptions & free parameters 1 free parameters · 2 assumptions · 0 invented entities

Relies on standard missing data assumptions and correct model specification for the weighting.

free parameters (1)
  • Parameters in the missingness model
    The IPW requires estimation of non-missingness probabilities, which are fitted from data.
assumptions (2)
  • domain assumption Missing at random conditional on treatment and covariates
    Methods allow dependence on treatment and baseline covariates.
  • domain assumption Correct specification of the weighting model for consistency
    Standard for IPW estimators.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Estimation and Inference for Win Measures with Multiple Ordinal Endpoints Subject to Missingness." pith.science (2026). https://pith.science/paper/7NQUB7RV

@misc{pith2026260527085,
  author       = {Pith},
  title        = {Pith review of: Estimation and Inference for Win Measures with Multiple Ordinal Endpoints Subject to Missingness},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7NQUB7RV}},
  note         = {Machine review of arXiv:2605.27085}
}
read the original abstract

Win measures, including the win ratio (WR), win odds (WO), net benefit (NB), and desirability of outcome ranking (DOOR), are increasingly used in randomized clinical trials with multiple hierarchical ordinal endpoints. In practice, however, one or more component endpoints may have missing data. The standard pairwise-comparison approach, which treats pairs with missing outcomes as ties, can produce biased estimates, even if the data are missing completely at random (MCAR). Although inverse probability of censoring weighting (IPCW) methods have been developed for censored survival endpoints, corresponding methods for addressing missing hierarchical ordinal endpoints are not yet available. To address this gap, we develop inverse probability weighting (IPW) and augmented IPW (AIPW) estimators for win measures with hierarchical ordinal endpoints subject to missing data, allowing missingness to depend on treatment assignment and baseline covariates. The IPW estimator corrects bias by reweighting complete observed outcomes using joint non-missingness probabilities involved in estimating the joint cell probabilities that define the win measures. The AIPW estimator additionally incorporates outcome modeling, improving efficiency and achieving double robustness. For inference, we derive closed-form variance estimators for both methods based on influence functions. Simulation studies show that the standard approach can be substantially biased, whereas the proposed IPW and AIPW estimators remain consistent with near-nominal coverage. Furthermore, the AIPW estimator is generally more efficient than IPW estimator. Applications to the SCOUT-CAP and ACTT-1 trials illustrate the practical utility of the proposed methods. An R package, WinMO, is provided for implementation.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Win-Ratio Regression for Prioritized Composite Outcomes in Observational Studies: Doubly Robust and Efficient Estimation with Future-Score Correction

    stat.ME 2026-08 conditional novelty 7.0 of 10

    A doubly robust, efficient win-ratio regression estimator that replaces censored future pairwise scores with their conditional expectation given observed history.

Reference graph

Works this paper leans on

3 extracted references · 2 canonical work pages · cited by 1 Pith paper

  1. [1]

    Covariate-adjusted win statistics in randomized clinical trials with ordinal outcomes

    doi: 10.1056/NEJMoa2007764. URLhttps://www.nejm.org/doi/full/10.1056/NEJMoa2007764. 28 Marc Buyse. Generalized pairwise comparisons of prioritized outcomes in the two-sample problem.Statistics in Medicine, 29(30):3245–3257, 2010. Marc Buyse, Johan Verbeeck, Everardo D. Saad, Micka¨ el De Backer, Vaiva Deltuvaite-Thomas, and Geert Molenberghs, editors.Hand...

    work page Pith review arXiv doi:10.1056/nejmoa2007764 2010

  2. [2]

    +P 1 1 (i1)ψ(O;P 1 0 (i′ 1)) + X i1=i′ 1 X i2>i′ 2 ψ(O;P 1:2 1 (i1, i2))P 1:2 0 (i1, i′

  3. [3]

    ∂ωk(a,X, eR1:k) ∂β(a) k {I(Y1 =i 1, . . . , Yk =i k)−µ k(a,X;i 1, . . . , ik)} # , and ∆a,k(i1, . . . , ik) =E

    +P 1:2 1 (i1, i2)ψ(O;P 1:2 0 (i1, i′ 2)) + · · ·+ X i1=···=iK−1 X iK >i′ K ψ(O;P 1:K 1 (i1, . . . , iK))P 1:K 0 (i1, . . . , i′ K) +P 1:K 1 (i1, . . . , iK)ψ(O;P 1:K 0 (i1, . . . , i′ K)) .(A.5.3) Similarly,ψ ipw(O;p L) is obtained by reversing>to<above, while ψipw(O;p T ) =−ψ ipw(O;p W )−ψ ipw(O;p L). Let bψipw(Oi;p ·) denote the empirical influence func...

    work page arXiv

Pith tools

Reviewed June 29, 2026 · model on record in the stance chip above.