Pith. sign in

REVIEW 3 major objections 5 minor 27 references

Cumulative/Dynamic Time-Dependent ROC Analysis for Left-Truncated and Right-Censored Data: Estimators and Comparison

T0 review · 3 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read New inverse-probability-weighting estimators give consistent time-dependent ROC and AUC for left-truncated, right-censored data.

desk verdict Useful extension of time-dependent ROC to LTRC data, but the unadjusted IPW estimators require a stronger conditional independence assumption than the paper states. read the letter →

arxiv 2509.05693 v1 pith:MN4Q2QTY submitted 2025-09-06 stat.ME

Kendrick Li , Mithun Kumar Acharjee This is my paper
classification stat.ME MSC 62N0162N0262P10
keywords time-dependentROCAUClefttruncationrightcensoringinverseprobabilityweightingselectionbiassurvivalanalysispredictionmodeling
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tackles a practical measurement problem: how to tell whether a risk score actually discriminates people who will have an event by a given time from those who will not, when the observed survival data are left-truncated and right-censored—that is, people can only be seen if they survived long enough to enter the study, and can be lost or still event-free at the end of follow-up. Standard time-dependent ROC and AUC estimators developed for ordinary right-censored data ignore the left-truncation selection, and the paper shows they can be substantially biased. The authors propose two routes: regression-based estimators that extend an earlier LTRC estimator to handle covariate-induced dependence, and new inverse probability weighting estimators that reweight each observation by its probability of being observed, with variants that condition on covariates. In simulations, the conditional IPW estimators are unbiased and have calibrated confidence intervals across the settings studied, provided the entry-time model is correct, while estimators that ignore truncation are not. An application to heart-failure risk scores in childhood cancer survivors illustrates the estimators on real delayed-entry data.

What carries the argument

The load-bearing object is the observation weight 1 divided by the probability that the individual is observed—i.e., the probability that study entry occurs before the event and censoring occurs after the event—either marginally or conditional on covariates. These weights remove the double selection bias of entering the study only after surviving long enough and being censored before the event. The paper constructs sample versions of these weights under two censoring scenarios, censoring after entry and censoring possibly before entry, using risk-set-adjusted survival estimates and regression models for the entry-time distribution. With these weights, sensitivity, specificity, and AUC become

What would settle it

Simulate a large cohort in which an unmeasured frailty affects both study entry and event time, so that conditional independence given the recorded covariates fails; if the proposed conditional IPW AUC keeps a bias that does not vanish as the sample size grows, the central claim is refuted. The paper's own misspecified-entry-time simulations already provide a smaller-scale version of this check.

Watch

Extended reading notes

Core claim

The central claim is that cumulative sensitivity, dynamic specificity, and the time-dependent AUC can be consistently estimated from left-truncated right-censored data by weighting each uncensored or at-risk observation by the inverse of the probability that it would have been observed—either marginally or conditional on baseline covariates. For an event at time T, the key selection probability is the probability that the study-entry time is before T and the censoring time is after T, conditional on T (or on T and covariates Z); under scenarios where censoring can or cannot occur before study entry, the paper gives explicit estimators of these weights from observed data. Sensitivity is estim

Load-bearing premise

The estimators stay correct only if, after conditioning on the covariates actually used, the study-entry time and the censoring time carry no extra information about the event time, and if the researcher's models for the entry-time and censoring distributions are correctly specified.

Editorial extensions

If this is right

  • AUC estimates for right-censored data that ignore left truncation are biased in LTRC cohorts—by as much as 0.07 in the simulations—so they should be used with caution when study entry is delayed.
  • The nonparametric regression estimator remains consistent whether censoring happens only after entry or can happen before entry, because the risk-set adjustment works in both settings.
  • Conditional IPW estimators are the recommended default under covariate-dependent left truncation: they stay unbiased when the entry-time model is correct, whereas marginal IPW and regression-type semiparametric estimators can be biased.
  • Correct specification of the left-truncation distribution is essential: when it is misspecified, both conditional IPW estimators show bias and under-covered confidence intervals.
  • In the childhood cancer survivor application, the choice of method matters most through the regression-type semiparametric estimator, which gives lower AUC than the IPW estimators, consistent with its simulated negative bias.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable implication of the simulation results is that the conditional IPW estimators should also outperform marginal IPW when censoring is covariate-dependent, not just when left truncation is; the paper mainly varies the truncation mechanism, so this extension is untested here.
  • Because the weighting step is generic, the same inverse observation-probability weights could be dropped into other discrimination metrics for LTRC data, such as the concordance index or incident/dynamic ROC, transferring the bias correction without re-deriving the weights.
  • The demonstrated failure under a misspecified entry-time model suggests investigators should treat the entry-time model as a first-class modeling choice and consider flexible alternatives or a doubly robust estimator, rather than assuming a single fitted proportional-hazards model suffices.
  • A useful practical benchmark: report both marginal and conditional IPW AUCs; divergence between the two is a warning that left truncation depends on covariates.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper develops estimators for cumulative/dynamic time-dependent ROC analysis under left-truncated and right-censored (LTRC) data. The authors extend the regression-type estimators of Li (2017) to handle covariate-induced dependent truncation and to allow right censoring before study entry, and they propose new inverse probability weighting (IPW) estimators of sensitivity, specificity, and AUC, with unadjusted versions and covariate-conditional versions (CIPW). The target quantities are defined in Eqs. (1)-(3), and the proposed estimators are given in Eqs. (17), (20)-(21), (24)-(31). The paper reports extensive simulations (Tables C1-C4) and an application to a childhood cancer survivor cohort (SJLIFE), evaluating a heart failure risk score.

Significance. If the proposed estimators are valid, the paper fills a genuine gap: time-dependent ROC methods for LTRC data are scarce, and the only prior work (Li 2017) does not cover all the scenarios considered here. The IPW approach is a natural extension of existing weighted methods for LTRC data and could be useful for evaluating risk prediction models in cohorts with delayed entry. The simulation study is comprehensive in its coverage of truncation/censoring mechanisms, and the CIPW estimators appear to perform well when nuisance models are correctly specified. The application to SJLIFE is relevant and illustrates the methods. However, the central claim that the unadjusted IPW estimators are consistent under the stated marginal independence scenarios (A1/B1) is not correct without an additional assumption, and the proofs are only referenced, not provided. This limits the paper's contribution as it stands.

major comments (3)
  1. [Section 3.2.1, Eqs. (15)-(21)] The derivation of the unadjusted IPW estimators requires the conditional independence (L,C)⊥X | T (or (L,D)⊥X | T under Scenario A1). In Eq. (15), the inner expectation conditional on T pulls the factor 1(X>c,T≤t) outside the expectation; this step is valid only if the selection indicator 1(L<T,C>T) is conditionally independent of X given T. Scenarios A1/B1, as stated, only impose (L,D)⊥T and D⊥L|T (or (L,C)⊥T and L⊥C|T). Since X=χ(Z), L may depend on Z even when L is marginally independent of T, so (L,C)⊥X|T is not implied. Consequently, SeIPW, SpIPW-1, and AUCIPW-1 need not be consistent under A1/B1. The simulation design does not expose this: L1 has L independent of Z, and L2/L3 violate A1/B1 because T also depends on Z. A simulation with T⊥Z and L dependent on Z would demonstrate the bias. The authors should either state the additional assumption explicitly, or restrict the validity
  2. [Appendix B and Section 3.1.1] The consistency of the nonparametric regression estimator F_{T,X} in Eq. (6) rests on the same missing assumption. The factorization in Appendix B, e.g., P(L<u,C≥u,T∈du,X∈dv)=P(L<u,C≥u)F_{T,X}(du,dv), requires (L,C)⊥(T,X) or at least (L,C)⊥X|T. This is not stated among the independence assumptions in Section 2. If the authors intend the nonparametric estimator to work under A1/B1, this condition must be made explicit; otherwise, the estimator is consistent only under a stronger condition than the paper claims.
  3. [Section 3.2.1, consistency claim] The statement that 'Proof of consistency and weak convergence ... directly follows the same arguments from Web Appendix D of Hartman et al. 2023' is not sufficient. The estimators here are U-statistics with estimated inverse weights K1 and K2, and the nuisance estimation is nontrivial (especially for F_{L|Z} in the CIPW versions). The transfer from concordance indices to time-dependent ROC is not automatic. The authors should provide a proof sketch or state precise regularity conditions under which the proposed estimators are consistent and asymptotically normal, particularly for the CIPW estimators in Eqs. (27)-(31).
minor comments (5)
  1. [Section 3.2.1, Eq. (20)] The notation is inconsistent: K2(t, T_i) and K2(t, eT_i) are mixed; use one symbol, e.g., T_i or eT_i, throughout.
  2. [Section 2, Scenario definitions] The phrase 'independent left truncation' is ambiguous. Since X=χ(Z), the reader may assume L is independent of Z. Clarify that Scenarios A1/B1 allow L to depend on Z as long as marginal independence with T holds, and that a further condition is needed for the unadjusted estimators.
  3. [Section 3.2.1, Eq. (24)] In SpIPW-2, the weight K1(t) appears in an intermediate expression and then cancels. State explicitly that this cancellation uses constancy of P(L<t<C|T) for T>t under A1/B1; the final unweighted estimator is not valid for covariate-induced dependent truncation.
  4. [Section 4, simulation description] The simulation scenarios use N=1,500 or 3,000 and 1,000 replications. It would be helpful to report the average effective sample size of events at each t; the tables give #at risk and #cum.events, but these are not clearly defined in the text.
  5. [Introduction] The abbreviation 'LRTC' is used once; elsewhere it is 'LTRC'. Please unify.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: target parameters are defined independently and IPW weights are estimated from nuisance distributions, not fitted to the target.

full rationale

The paper's target parameters Se(c,t), Sp(c,t), and AUC(t) are defined by population probabilities (Eqs. 1-3) independently of any estimator. The IPW estimators (Eqs. 17, 20, 21, 26, and conditional versions) are derived from explicit expectation identities (Eqs. 15-16, 18-19, 22-23) in which the weights depend only on truncation and censoring distributions, not on the target ROC quantities. The weights are estimated using standard prior-work estimators (Wang 1991 for F_L, Kaplan-Meier for S_C, Cox models for conditional survival functions), and the paper does not fit the weights to reproduce the estimated Se, Sp, or AUC. The consistency proofs are delegated to external prior work (Hartman et al. 2023; Li 2017) rather than to the present authors' own previous claims, so there is no load-bearing self-citation chain. The explicit statement that the alternative specificity estimator cSpIPW-2 is identical to Li 2017's nonparametric estimator (Eq. 24 and surrounding text) is a transparent disclosure, not a disguised renaming. The reviewer's noted concern that the unadjusted IPW derivation may require an unstated (L,C)⊥X|T condition is a correctness or assumption-specification issue, not circularity: even if A1/B1 are insufficient for consistency, the target parameters remain defined independently of the estimators. No equation or definition reduces to its own input by construction, and no fitted parameter is relabeled as a prediction.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new entities, particles, or forces. The free parameters are minimal: only the reverse-time horizon tau is an arbitrary constant. The main burden is the domain assumptions about independence and correct nuisance model specification, which the paper itself tests in simulation.

free parameters (1)
  • Reverse-time horizon tau
    Used to construct reverse entry times for estimating F_L|Z in the CIPW estimators (Section 3.2.2). The paper requires tau >= eTi for all i but does not specify a concrete choice; finite-sample results may depend on it.
assumptions (4)
  • domain assumption Independent or conditionally independent left truncation and right censoring (Scenarios A1/A2/B1/B2)
    Section 2 defines these assumptions as the basis for the probability identities that justify the IPW estimators. If truncation and censoring are not independent of event time even given covariates, the weights are invalid.
  • domain assumption Correct specification of nuisance models for F_L|Z and S_D|Z or S_C|Z
    Section 3.2.2 uses Cox models for these conditional survival functions. The paper's own simulation L3 shows bias when F_L|Z is misspecified (Tables C2, C4), so consistency is conditional on correct modeling.
  • standard math Consistency of Kaplan-Meier estimators with risk sets adjusted for left truncation
    Used throughout for bST, bSC, bSD. Standard result in survival analysis under independent censoring/truncation.
  • standard math Wang 1991 estimator for F_L under LTRC is consistent
    Used in Equation (18) and surrounding text for estimating the marginal truncation distribution under scenario A1/B1.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Cumulative/Dynamic Time-Dependent ROC Analysis for Left-Truncated and Right-Censored Data: Estimators and Comparison." pith.science (2026). https://pith.science/paper/MN4Q2QTY

@misc{pith2026250905693,
  author       = {Pith},
  title        = {Pith review of: Cumulative/Dynamic Time-Dependent ROC Analysis for Left-Truncated and Right-Censored Data: Estimators and Comparison},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MN4Q2QTY}},
  note         = {Machine review of arXiv:2509.05693}
}
read the original abstract

Time-dependent Receiver Operating Characteristics (ROC) analysis is a standard method to evaluate the discriminative performance of biomarkers or risk scores for time-to-event outcomes. Extensions of this useful method to left-truncated right-censored data have been understudied, with the exception of Li 2017. In this paper, we first extended the estimators in Li 2017 to several regression-type estimators that account for independent or covariate-induced dependent left truncation and right censoring. We further proposed novel inverse probability weighting estimators of cumulative sensitivity, dynamic specificity, and area under the ROC curve (AUC), where the weights simultaneously account for left truncation and right censoring, with or without adjusting for covariates. We demonstrated the proposed AUC estimators in simulation studies with different scenarios. We performed the proposed time-dependent ROC analysis to evaluate the predictive performance of two risk prediction models of heart failure by Chow et al. 2015 in five-year childhood cancer survivors using the St. Jude Lifetime Cohort Study.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

27 extracted references · 27 canonical work pages

  1. [1]

    Basic principles of ROC analysis

    Metz CE. Basic principles of ROC analysis. In: . 8. Elsevier. 1978:283–298

    work page 1978

  2. [2]

    An introduction to ROC analysis.Pattern recognition letters.2006;27(8):861–874

    Fawcett T. An introduction to ROC analysis.Pattern recognition letters.2006;27(8):861–874

    work page 2006

  3. [3]

    Time-dependent ROC curves for censored survival data and a diagnostic marker.Biometrics.2000;56(2):337–344

    Heagerty PJ, Lumley T, Pepe MS. Time-dependent ROC curves for censored survival data and a diagnostic marker.Biometrics.2000;56(2):337–344

    work page 2000

  4. [4]

    Time-dependent ROC curve analysis in medical research: current methods and applications.BMC medical research methodology.2017;17(1):53

    Kamarudin AN, Cox T, Kolamunnage-Dona R. Time-dependent ROC curve analysis in medical research: current methods and applications.BMC medical research methodology.2017;17(1):53

    work page 2017

  5. [5]

    General right censoring and its impact on the analysis of survival data.Biometrics.1979:139–156

    Lagakos SW. General right censoring and its impact on the analysis of survival data.Biometrics.1979:139–156

    work page 1979

  6. [6]

    Evaluating prediction rules for t-year survivors with censored regression models.Journal of the American Statistical Association.2007;102(478):527–537

    Uno H, Cai T, Tian L, Wei LJ. Evaluating prediction rules for t-year survivors with censored regression models.Journal of the American Statistical Association.2007;102(478):527–537

    work page 2007

  7. [7]

    Blanche P, Dartigues JF, Jacqmin-Gadda H. Estimating and comparing time-dependent areas under receiver operating characteristic curves for censored event times with competing risks.Statistics in medicine.2013;32(30):5381–5397

    work page 2013

  8. [8]

    Conditions for bias from differential left truncation.American journal of epidemiology.2007;165(4):444– 452

    Howards PP, Hertz-Picciotto I, Poole C. Conditions for bias from differential left truncation.American journal of epidemiology.2007;165(4):444– 452

    work page 2007

Show all 27 references
  1. [9]

    Accuracy loss due to selection bias in cohort studies with left truncation.Paediatric and perinatal epidemiology.2013;27(5):491–502

    Schisterman EF, Cole SR, Ye A, Platt RW. Accuracy loss due to selection bias in cohort studies with left truncation.Paediatric and perinatal epidemiology.2013;27(5):491–502

    2013

  2. [10]

    Recognizing the problem of delayed entry in time-to-event studies: better late than never for clinical neuroscientists

    Betensky RA, Mandel M. Recognizing the problem of delayed entry in time-to-event studies: better late than never for clinical neuroscientists. Annals of neurology.2015;78(6):839–844

    2015

  3. [11]

    Concordance indices with left-truncated and right-censored data.Biometrics.2023;79(3):1624–1634

    Hartman N, Kim S, He K, Kalbfleisch JD. Concordance indices with left-truncated and right-censored data.Biometrics.2023;79(3):1624–1634

    2023

  4. [12]

    Estimating time-dependent ROC curves using data under prevalent sampling.Statistics in Medicine.2017;36(8):1285–1301

    Li S. Estimating time-dependent ROC curves using data under prevalent sampling.Statistics in Medicine.2017;36(8):1285–1301

    2017

  5. [13]

    Assumptions regarding right censoring in the presence of left truncation.Statistics & probability letters.2014;87:12–17

    Qian J, Betensky RA. Assumptions regarding right censoring in the presence of left truncation.Statistics & probability letters.2014;87:12–17

    2014

  6. [14]

    Prospective medical assessment of adults surviving childhood cancer: study design, cohort characteristics, and feasibility of the St

    Hudson MM, Ness KK, Nolan VG, et al. Prospective medical assessment of adults surviving childhood cancer: study design, cohort characteristics, and feasibility of the St. Jude Lifetime Cohort study.Pediatric blood & cancer .2011;56(5):825–836

    2011

  7. [15]

    Cohort profile: the St

    Howell CR, Bjornard KL, Ness KK, et al. Cohort profile: the St. Jude Lifetime Cohort Study (SJLIFE) for paediatric cancer survivors.International Journal of Epidemiology.2021;50(1):39–49

    2021

  8. [16]

    Individual Prediction of Heart Failure Among Childhood Cancer Survivors.Journal of Clinical Oncology

    Chow EJ, Chen Y , Kremer LC, et al. Individual Prediction of Heart Failure Among Childhood Cancer Survivors.Journal of Clinical Oncology. 2015;33(5):394-402. PMID: 25287823doi: 10.1200/JCO.2014.56.1373 12 Li et al

    2015 doi

  9. [17]

    Nonparametric estimation of the survival distribution under covariate-induced dependent truncation.Biometrics.2022;78(4):1390–1401

    Vakulenko-Lagun B, Qian J, Chiou SH, Wang N, Betensky RA. Nonparametric estimation of the survival distribution under covariate-induced dependent truncation.Biometrics.2022;78(4):1390–1401

    2022

  10. [18]

    Nonparametric estimation from incomplete observations.Journal of the American statistical association.1958;53(282):457– 481

    Kaplan EL, Meier P. Nonparametric estimation from incomplete observations.Journal of the American statistical association.1958;53(282):457– 481

    1958

  11. [19]

    Regression models and life-tables.Journal of the Royal Statistical Society: Series B (Methodological).1972;34(2):187–202

    Cox DR. Regression models and life-tables.Journal of the Royal Statistical Society: Series B (Methodological).1972;34(2):187–202

    1972

  12. [20]

    The accelerated failure time model: a useful alternative to the Cox regression model in survival analysis.Statistics in medicine

    Wei LJ. The accelerated failure time model: a useful alternative to the Cox regression model in survival analysis.Statistics in medicine. 1992;11(14-15):1871–1879

    1992

  13. [21]

    Debiased machine learning for counterfactual survival functionals based on left-truncated right-censored data

    Morenz ER, Wolock CJ, Carone M. Debiased machine learning for counterfactual survival functionals based on left-truncated right-censored data. arXiv preprint arXiv:2411.09017.2024

    2024 arXiv

  14. [22]

    Nonparametric estimation from cross-sectional survival data.Journal of the American Statistical Association.1991;86(413):130–143

    Wang MC. Nonparametric estimation from cross-sectional survival data.Journal of the American Statistical Association.1991;86(413):130–143

    1991

  15. [23]

    Chapman and Hall/CRC, 1994

    Efron B, Tibshirani RJ.An introduction to the bootstrap. Chapman and Hall/CRC, 1994

    1994

  16. [24]

    Approach for classification and severity grading of long-term and late-onset health events among childhood cancer survivors in the St

    Hudson MM, Ehrhardt MJ, Bhakta N, et al. Approach for classification and severity grading of long-term and late-onset health events among childhood cancer survivors in the St. Jude Lifetime Cohort.Cancer epidemiology, biomarkers & prevention.2017;26(5):666–674

    2017

  17. [25]

    Nearest neighbor estimation of a bivariate distribution under random censoring.The Annals of Statistics.1994:1299–1327

    Akritas MG. Nearest neighbor estimation of a bivariate distribution under random censoring.The Annals of Statistics.1994:1299–1327

    1994

  18. [26]

    Doubly robust estimation under covariate-induced dependent left truncation.Biometrika.2024;111(3):789–808

    Wang Y , Ying A, Xu R. Doubly robust estimation under covariate-induced dependent left truncation.Biometrika.2024;111(3):789–808

    2024

  19. [27]

    Learning treatment effects under covariate dependent left truncation and right censoring.arXiv preprint arXiv:2411.18879

    Wang Y , Ying A, Xu R. Learning treatment effects under covariate dependent left truncation and right censoring.arXiv preprint arXiv:2411.18879. 2024. SUPPORTING INFORMA TION Additional supporting information may be found in the online version of the article at the publisher’s...

    2024 arXiv

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.