REVIEW 2 major objections 6 minor 49 references
Statistical Considerations in Long COVID Research
T0 review · 2 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read Long COVID research requires statistical methods tailored to a condition with no gold-standard definition.
desk verdict A clear, useful review of Long COVID statistical pitfalls, but its strongest claim about the LCRI rests on an undefended identifiability assumption, and the paper reads more like a program overview than a neutral methods survey. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the negative-unlabeled data formulation of Long COVID status: never-infected individuals are 'negative' for Long COVID, while infected individuals are 'unlabeled' because they may or may not have the condition. This formulation licenses a pseudo-label classifier — Lasso-penalized logistic regression with infection history as the outcome and symptoms as predictors — whose nonzero coefficients form the Long COVID Research Index (LCRI), a weighted symptom score thresholded to define Long COVID. The other load-bearing mechanism is repeated auxiliary-variable dependent sampling, a two-phase design in which expensive tiered tests are administered to a subset selected on cheap auxiliary variables such as symptom triggers; the paper argues that ignoring this sampling mechanism induces selection bias that can attenuate or invert associations.
What would settle it
A validation study that applies the LCRI to a population where infection history is known but Long COVID status is adjudicated by a blinded clinical panel, and shows that LCRI-positive never-infected individuals are common or that LCRI status fails to track clinically confirmed Long COVID trajectories, would falsify the pseudo-label core of the approach.
Extended reading notes
Core claim
The paper's central claim is that the defining features of Long COVID — absence of a gold standard, multiple sub-phenotypes, waxing and waning symptoms, and reliance on auxiliary-variable dependent sampling — create distinct statistical challenges that standard cohort analysis methods do not address. It argues that two moves make research possible: treating Long COVID status as negative-unlabeled data and using SARS-CoV-2 infection history as a pseudo-outcome to train classifiers such as the LCRI, and accounting for repeated auxiliary-variable dependent sampling in the analysis to avoid selection bias. The authors take the LCRI approach to be the only current strategy that rigorously defines Long COVID while minimizing misclassification of chronic conditions with other causes, and they describe structural intermittent missingness and left censoring as built into RECOVER-style designs.
Load-bearing premise
The load-bearing premise is that symptom patterns that separate people with a history of SARS-CoV-2 infection from people who were never infected also capture Long COVID itself; the paper adopts this pseudo-label assumption from earlier LCRI work without independently validating it.
Editorial extensions
If this is right
- Studies that define Long COVID with a broad symptom-based or NASEM-style definition will tend to dilute effect sizes and lose power, so research definitions should be chosen for specificity.
- Comparisons against individuals with an LCRI of 0 are preferable to comparisons against asymptomatics, because asymptomatics are healthier than the general population and bias results.
- Analyses of tiered tests must account for the sampling mechanism and for informative missingness within the phase-two sample, or associations will be biased toward the null or otherwise distorted.
- Left-censored enrollment and structural missingness around reinfections mean that time-to-recovery and trajectory analyses need methods that handle intermittent unobservability of the condition.
- Symptom counts are poor outcomes because their distribution depends on the number of symptoms in the instrument and on correlations within organ systems, as demonstrated in the paper's comparison of naive counts with the LCRI.
Reading between the lines
- If the pseudo-label assumption is wrong, the LCRI may measure general post-viral symptom burden rather than Long COVID specifically; this can be tested by external validation against clinician-adjudicated or biomarker-based diagnoses.
- The same negative-unlabeled and auxiliary-variable dependent sampling framework could be transferred to other infection-associated chronic conditions, with a clear testable extension being cross-condition classifiers trained on symptom data from multiple post-infection syndromes.
- The paper's review suggests that standardized reporting of sampling probabilities and trigger variables should become a required part of Long COVID study publications, a practice that would improve reproducibility across cohorts.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper is a perspective/review article on statistical challenges in Long COVID (LC) research. It argues that LC studies face unique data analytic difficulties because there is no gold-standard definition, the condition has multiple and time-varying presentations, and sampling designs are often auxiliary-variable dependent. Section 2 discusses definitions, focusing on a negative-unlabeled approach that uses SARS-CoV-2 infection history as a pseudo-label to train a classifier, yielding the Long COVID Research Index (LCRI). Section 3 describes sub-phenotypes, waxing/waning symptoms, left censoring, and variant-related cohort composition. Section 4 describes two-phase and repeated auxiliary-variable dependent sampling as used in RECOVER-Adult and RECOVER-Pediatrics, and outlines selection bias, informative missingness, confounding, and loss to follow-up. The Discussion describes the NIH-funded Network of Biostatisticians for RECOVER (NBR). The central claim is that these features demand careful statistical design and analysis.
Significance. The paper performs a useful service by cataloguing and organizing the statistical challenges specific to LC research, and its descriptions of the RECOVER designs and LCRI appear consistent with the cited publications. Its main contribution is synthetic: it connects negative-unlabeled learning, two-phase sampling, and longitudinal missing-data concepts to a concrete, high-impact clinical domain, and it explicitly flags false-negative limitations of the LCRI and left censoring in contemporary cohorts. If the unsupported claims in Section 2 are appropriately qualified, the paper could serve as a practical orientation for statisticians entering this area. It is not a methodological development; its value lies in its accurate survey and its emphasis on design-aware analysis.
major comments (2)
- [§2 and Figure 1] The negative-unlabeled strategy assumes that infection history A is a valid pseudo-outcome for latent LC status Y, and that symptoms X are not precursors of A and are affected by A only through Y. This identifiability condition is load-bearing for the claim that the LCRI 'rigorously defines LC' and minimizes misclassification of chronic conditions unrelated to SARS-CoV-2 infection, but it is not defended in this manuscript. Pre-existing symptoms may predict infection risk through healthcare-seeking behavior, testing access, or occupational exposure, in which case P(A|X) diverges from P(Y|X). The text acknowledges false negatives from thresholding but does not address this false-positive mechanism or provide independent validation of the pseudo-label premise beyond citing prior work. Please either provide evidence or a formal argument for this assumption, or substantially temper the claims about the LCRI.
- [Section 2] The statement that the LCRI is 'the only strategy to our knowledge that rigorously defines LC' is not supported by a systematic comparison with other proposed definitions or by a formal criterion for what qualifies as 'rigorously defines.' Because the authors are also developers of the LCRI, this superlative claim needs either a concrete argument or a more modest formulation; as written, it reads as advocacy rather than assessment.
minor comments (6)
- [Figure 1 caption] The caption contains the typo 'sstandard' and should be corrected to 'standard.'
- [Section 3] The text refers to the 'Omicon' variant; this should be 'Omicron.'
- [Section 5] The sentence beginning 'Resource-efficient cohort study designs an optimal subset of participants may be selected' is incomplete and should be reworded for clarity.
- [Section 2 and reference list] The term 'na¨ıve' contains encoding artifacts; it should read 'naive.'
- [Discussion and Table 2] Table 2 is referenced in the Discussion but its content is not shown in the manuscript; please ensure the table is included in the final version.
- [Section 2] The claim that 'individuals with no symptoms at all tend to be healthier than the general population' is an empirical assertion that would benefit from a supporting citation.
Circularity Check
No significant circularity: the paper's survey claims about statistical challenges are independent of its self-cited LCRI/RECOVER work.
full rationale
The manuscript is a methods-review/position piece rather than a derivation. Its central claims (absence of gold standard, time-varying/multiple presentations, auxiliary-variable dependent sampling creates selection bias) are supported by external references and by the RECOVER design description, not by a fitted parameter renamed as a prediction. The LCRI is presented as prior work (Thaweethai et al. 2023; Geng et al. 2024; Reeder et al. 2025) and is explicitly described as a pseudo-label approach: infection history is used as a training outcome, not concealed as an independent validation. The identifiability conditions in Figure 1 (symptoms not precursors of infection; infection affects symptoms only through latent LC) are stated assumptions, not derived results; they may be contested on subject-matter grounds, but that is a validity concern, not circularity. The 'only strategy... that rigorously defines LC' sentence is a strong literature claim and relies partly on the authors' own published work, but it is not used to derive the paper's main statistical-challenges conclusions and no equation reduces to itself. No self-definitional, fitted-input-called-prediction, or uniqueness-imported-from-authors pattern is present.
Assumptions & free parameters
assumptions (4)
- domain assumption LC status can be treated as negative-unlabeled data, with uninfected individuals serving as known negatives.
- domain assumption The LCRI threshold (e.g., 11) provides a valid classification of LC with controlled false positives.
- domain assumption Auxiliary variables used for sampling are associated with the outcome and selection probabilities are known.
- domain assumption Standard statistical methods for two-phase sampling can be extended to repeated, longitudinal auxiliary-variable dependent sampling.
Cite this review
Pith. "Pith review of Statistical Considerations in Long COVID Research." pith.science (2026). https://pith.science/paper/YHSB5R5E
@misc{pith2026260804919,
author = {Pith},
title = {Pith review of: Statistical Considerations in Long COVID Research},
year = {2026},
howpublished = {\url{https://pith.science/paper/YHSB5R5E}},
note = {Machine review of arXiv:2608.04919}
}
read the original abstract
Long COVID is a condition characterized by ongoing or relapsing symptoms attributable to SARS-CoV-2 infection that are present three or more months after infection. It represents a major clinical and public health concern as an estimated 5-10\% of individuals with a history of SARS-CoV-2 infection present with long term sequelae that range from mild to debilitating with profound impacts on quality of life. Clinical research studies of Long COVID have emerged rapidly over the past few years, and with them we are seeing several new data analytic challenges. In this manuscript, we highlight statistical challenges arising from the defining features of LC and associated study design strategies. This work is motivated by the Researching COVID to Enhance Recovery (RECOVER) Adult and Pediatric observational meta-cohort studies.
Figures
Reference graph
Works this paper leans on
-
[1]
Adjaye-Gbewonyo, Dzifa and Vahratian, Anjel and Cria G., Perrine and Bertolli, Jeanne , doi =. Long. 2023 , month =
work page 2023
-
[2]
Magnusson, Karin and Turkiewicz, Aleksandra and Flottorp, Signe Agnes and Englund, Martin , journal =. Prevalence of long. 2023 , month =. doi:10.1038/s41598-023-32636-y , issn =
-
[3]
Wang, Xiaoyu and Vel. Post-. Journal of Multimorbidity and Comorbidity , doi =. 2026 , month =
work page 2026
-
[4]
Steyerberg, E. W. and Harrell, F. E. Jr and Borsboom, G. J. and Eijkemans, M. J. and Vergouwe, Y. and Habbema, J. D. , journal =. Internal validation of predictive models: efficiency of some procedures for logistic regression analysis. , volume =. 2001 , month =. doi:10.1016/s0895-4356(01)00341-9 , issn =
-
[5]
Learning from positive and unlabeled data: a survey , volume =
Bekker, Jessa and Davis, Jesse , journal =. Learning from positive and unlabeled data: a survey , volume =. 2020 , month =. doi:10.1007/s10994-020-05877-5 , issn =
-
[6]
Liu, B. and Dai, Y. and Li, X. and Lee, W.S. and Yu, P.S. , booktitle =. Building text classifiers using positive and unlabeled examples , url =. 2003 , pages =. doi:10.1109/ICDM.2003.1250918 , isbn =
arXiv 2003
-
[7]
Liu, Siyan and Yeh, Chi-Kuang and Zhang, Xin and Tian, Qinglong and Li, Pengfei , journal =. Positive and. 2025 , month =. doi:10.1080/01621459.2025.2474266 , issn =
-
[8]
Van Engelen, Jesper E. and Hoos, Holger H. , journal =. A survey on semi-supervised learning , volume =. 2020 , month =. doi:10.1007/s10994-019-05855-6 , issn =
Show all 49 references
-
[9]
JAMA , author =
Development of a. JAMA , author =. 2023 , pages =. doi:10.1001/jama.2023.8823 , number =
2023
-
[10]
JAMA , author =
2024. JAMA , author =. doi:10.1001/jama.2024.24184 , urldate =
2024
-
[11]
and Thaweethai, Tanayott and Kleinman, Lawrence C
Gross, Rachel S. and Thaweethai, Tanayott and Kleinman, Lawrence C. and Snowden, Jessica N. and Rosenzweig, Erika B. and Milner, Joshua D. and Tantisira, Kelan G. and Rhee, Kyung E. and Jernigan, Terry L. and Kinser, Patricia A. and Salisbury, Amy L. and Warburton, David and M...
2024
-
[12]
and Thaweethai, Tanayott and Salisbury, Amy L
Gross, Rachel S. and Thaweethai, Tanayott and Salisbury, Amy L. and Kleinman, Lawrence C. and Mohandas, Sindhu and Rhee, Kyung E. and Snowden, Jessica N. and Tantisira, Kelan G. and Warburton, David and Wood, John C. and Kinser, Patricia A. and Milner, Joshua D. and Rosenzweig...
2025
-
[13]
and Easter, Rebecca and Parker, Hannah and Della Penna, Christina and Santiuste, Isabel and Schultz, Holly and Ehrenspeck, Ana and Veenhuis, Rebecca and Rubin, Leah H
Azola, Alba and Dastgheyb, Raha M. and Easter, Rebecca and Parker, Hannah and Della Penna, Christina and Santiuste, Isabel and Schultz, Holly and Ehrenspeck, Ana and Veenhuis, Rebecca and Rubin, Leah H. , journal =. Putting the. 2025 , month =. doi:10.1007/s11606-025-10042-6 , issn =
2025 doi
-
[14]
and Geng, Linda N
Erlandson, Kristine M. and Geng, Linda N. and Selvaggi, Caitlin A. and Thaweethai, Tanayott and Chen, Peter and Erdmann, Nathan B. and Goldman, Jason D. and Henrich, Timothy J. and Hornig, Mady and Karlson, Elizabeth W. and Katz, Stuart D. and Kim, C. and Cribbs, Sushma K. and...
2024
-
[15]
Wesley and Brown, Lisa M
Ely, E. Wesley and Brown, Lisa M. and Fineberg, Harvey V. , journal =. Long. 2024 , month =. doi:10.1056/NEJMsb2408466 , issn =
2024 doi
-
[16]
Annual Review of Public Health , year =
Thomas, Duncan and Stram, Daniel and Dwyer, James , title =. Annual Review of Public Health , year =
- [17]
-
[18]
A simple correction for multiple comparisons in interval mapping genome scans , volume =
Cheverud, James M , journal =. A simple correction for multiple comparisons in interval mapping genome scans , volume =. 2001 , month =. doi:10.1046/j.1365-2540.2001.00901.x , issn =
2001
-
[19]
and Ji, L
Li, J. and Ji, L. , journal =. Adjusting multiple testing in multilocus analyses using the eigenvalues of a correlation matrix , volume =. 2005 , month =. doi:10.1038/sj.hdy.6800717 , issn =
2005 doi
- [20]
-
[21]
and Thaweethai, Tanayott and Foulkes, Andrea S
Reeder, Harrison T. and Thaweethai, Tanayott and Foulkes, Andrea S. , journal =. Penalized regression with negative-unlabeled data: an approach to developing a. 2025 , month =. doi:10.1186/s12874-025-02737-5 , issn =
2025 doi
-
[22]
Davis, H. E. and McCorkell, L. and Vogel, J. M. and Topol, E. J. , journal =. Long. 2023 , month =
2023
-
[23]
Refinement of post-
Wang, Yunhe and Alcalde-Herraiz, Marta and G. Refinement of post-. eBioMedicine , doi =. 2025 , month =
2025
- [24]
-
[25]
Derrick, Ben and Dobson-Mckittrick, Anselma and Toher, Deirdre and White, Paul , title =
-
[26]
Test statistics for the comparison of means for two samples that include both paired and independent observations , volume =
Derrick, Ben and Russ, Bethan and Toher, Deirdre and White, Paul , journal =. Test statistics for the comparison of means for two samples that include both paired and independent observations , volume =. 2017 , month =. doi:10.22237/jmasm/1493597280 , issn =
2017
-
[27]
and Mohandas, Sindhu and Stein, Cheryl R
Rao, Suchitra and Gross, Rachel S. and Mohandas, Sindhu and Stein, Cheryl R. and Case, Abigail and Dreyer, Benard and Pajor, Nathan M. and Bunnell, H. Timothy and Warburton, David and Berg, Elizabeth and Overdevest, Jonathan B. and Gorelik, Mark and Milner, Joshua and Saxena, ...
2024 doi
-
[28]
and Martin, Jeffrey N
Thaweethai, Tanayott and Donohue, Sarah E. and Martin, Jeffrey N. and Hornig, Mady and Mosier, Jarrod M. and Shinnick, Daniel J. and Ashktorab, Hassan and Atieh, Ornina and Blomkalns, Andra and Brim, Hassan and Chen, Yu and Cortez, Melissa M. and Erdmann, Nathan B. and Flaherm...
2025
-
[29]
Estimation of regression coefficients when some regressors are not always observed , volume =
Robins, James M and Rotnitzky, Andrea and Zhao, Lue Ping , journal =. Estimation of regression coefficients when some regressors are not always observed , volume =. 1994 , pages =. doi:10.1080/01621459.1994.10476818 , number =
1994
-
[30]
Semiparametric regression estimation in the presence of dependent censoring , volume =
Rotnitzky, Andrea and Robins, James M , journal =. Semiparametric regression estimation in the presence of dependent censoring , volume =. 1995 , pages =
1995
-
[31]
Maximum likelihood estimation of logistic regression parameters under two-phase, outcome-dependent sampling , volume =
Breslow, Norman E and Holubkov, Richard , journal =. Maximum likelihood estimation of logistic regression parameters under two-phase, outcome-dependent sampling , volume =. 1997 , pages =
1997
-
[32]
Large sample theory for semiparametric regression models with two-phase, outcome-dependent sampling , volume =
Breslow, Norman and McNeney, Brad and Wellner, Jon A , journal =. Large sample theory for semiparametric regression models with two-phase, outcome-dependent sampling , volume =. 2003 , pages =
2003
-
[33]
Causal inference in outcome-dependent two-phase sampling designs , volume =
Wang, Weiwei and Scharfstein, Daniel and Tan, Zhiqiang and MacKenzie, Ellen J , journal =. Causal inference in outcome-dependent two-phase sampling designs , volume =. 2009 , pages =
2009
-
[34]
A targeted maximum likelihood estimator for two-stage designs , volume =
Rose, Sherri and van der Laan, Mark J , journal =. A targeted maximum likelihood estimator for two-stage designs , volume =. 2011 , pages =
2011
-
[35]
Efficient nonparametric causal inference with missing exposure information , volume =
Kennedy, Edward H , journal =. Efficient nonparametric causal inference with missing exposure information , volume =. 2020 , pages =
2020
-
[36]
Efficient nonparametric inference on the effects of stochastic interventions under two-phase sampling, with applications to vaccine efficacy trials , volume =
Hejazi, Nima S and van der Laan, Mark J and Janes, Holly E and Gilbert, Peter B and Benkeser, David C , journal =. Efficient nonparametric inference on the effects of stochastic interventions under two-phase sampling, with applications to vaccine efficacy trials , volume =. 20...
2021 doi
-
[37]
and Thaweethai, Tanayott and Rosenzweig, Erika B
Gross, Rachel S. and Thaweethai, Tanayott and Rosenzweig, Erika B. and Chan, James and Chibnik, Lori B. and Cicek, Mine S. and Elliott, Amy J. and Flaherman, Valerie J. and Foulkes, Andrea S. and Gage Witvliet, Margot and Gallagher, Richard and Gennaro, Maria Laura and Jerniga...
2024
-
[38]
and Thaweethai, Tanayott and Brosnahan, Shari B
Horwitz, Leora I. and Thaweethai, Tanayott and Brosnahan, Shari B. and Cicek, Mine S. and Fitzgerald, Megan L. and Goldman, Jason D. and Hess, Rachel and Hodder, S. L. and Jacoby, Vanessa L. and Jordan, Michael R. and Krishnan, Jerry A. and Laiyemo, Adeyinka O. and Metz, Torri...
2023
-
[39]
and Yu, Xuesong and Rotnitzky, Andrea , journal =
Gilbert, Peter B. and Yu, Xuesong and Rotnitzky, Andrea , journal =. Optimal auxiliary-covariate-based two-phase sampling design for semiparametric efficient estimation of a mean or mean difference, with application to clinical trials , volume =. 2014 , month =. doi:10.1002/si...
2014 doi
-
[40]
A case-cohort design for epidemiologic cohort studies and disease prevention trials , volume =
Prentice, Ross L , journal =. A case-cohort design for epidemiologic cohort studies and disease prevention trials , volume =. 1986 , pages =
1986
-
[41]
Statistics in epidemiology: the case-control study , volume =
Breslow, Norman E , journal =. Statistics in epidemiology: the case-control study , volume =. 1996 , pages =
1996
-
[42]
Breslow, N. E. and Lumley, T. and Ballantyne, C. M. and Chambless, L. E. and Kulich, M. , journal =. Using the. 2009 , month =. doi:10.1093/aje/kwp055 , issn =
2009 doi
-
[43]
Patient recruitment using electronic health records under selection bias: A two-phase sampling framework , volume =
Zhang, Guanghao and Beesley, Lauren J and Mukherjee, Bhramar and Shi, X U , journal =. Patient recruitment using electronic health records under selection bias: A two-phase sampling framework , volume =. 2024 , month =
2024
-
[44]
Two-phase sampling designs for data validation in settings with covariate measurement error and continuous outcome , volume =
Amorim, Gustavo and Tao, Ran and Lotspeich, Sarah and Shaw, Pamela A and Lumley, Thomas and Shepherd, Bryan E , journal =. Two-phase sampling designs for data validation in settings with covariate measurement error and continuous outcome , volume =. 2021 , pages =
2021
-
[45]
Double sampling for informatively missing data in electronic health record-based comparative effectiveness research , volume =
Levis, Alexander W and Mukherjee, Rajarshi and Wang, Rui and Fischer, Heidi and Haneuse, Sebastien , journal =. Double sampling for informatively missing data in electronic health record-based comparative effectiveness research , volume =. 2024 , pages =
2024
-
[46]
Augmented designs to assess immune response in vaccine trials , volume =
Follmann, Dean , journal =. Augmented designs to assess immune response in vaccine trials , volume =. 2006 , pages =
2006
-
[47]
Calibration weighted estimation of semiparametric transformation models for two-phase sampling , volume =
Fong, Youyi and Gilbert, Peter , journal =. Calibration weighted estimation of semiparametric transformation models for two-phase sampling , volume =. 2015 , month =
2015
-
[48]
Joint modeling of longitudinal and survival data with the
Fu, Rong and Gilbert, Peter B , journal =. Joint modeling of longitudinal and survival data with the. 2017 , month =
2017
-
[49]
Four statistical frameworks for assessing an immune correlate of protection (surrogate endpoint) from a randomized, controlled, vaccine efficacy trial , volume =
Gilbert, Peter B and Fong, Youyi and Hejazi, Nima S and Kenny, Avi and Huang, Ying and Carone, Marco and Benkeser, David and Follmann, Dean , journal =. Four statistical frameworks for assessing an immune correlate of protection (surrogate endpoint) from a randomized, controll...
2024 doi
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.