{"id":"00869f5b-e195-4757-935f-7307ec5294d1","arxiv_id":"2505.21824","paper_version":2,"verdict":"REJECT","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"An unsupervised NMF and KL-divergence method uses T2DM patients' comorbidity and medication codes to score undiagnosed patients' relative T2DM risk.","lead":"This paper builds a diabetes risk score for people without a T2DM diagnosis by finding disease and medication patterns among already diagnosed patients, using a type of machine learning that needs no confirmed healthy controls. The approach could allow insurers and health systems with claims data to flag likely undiagnosed T2DM cases for early screening.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central transfer assumption is never tested against actual T2DM outcomes; every stated validation is self-referential, so the claim that the score estimates risk in undiagnosed individuals is unsupported.","rationale":"The reader's weakest assumption is exactly the one I would stress-test. The paper's own Section 2.2 states the transfer assumption as an assumption, and Section 4.1 lists the absence of HbA1C as a limitation, but no experiment converts these into evidence. In an unsupervised setting, a risk score can still be useful if it is validated against a hard outcome; here the validations are constructed from the same score. Method 4 is the clearest example: selecting negatives by the risk score and then measuring classifier separation of those labels is a proxy for how well XGBoost can reconstruct the risk score's linear combination of covariates, not for how well the score predicts T2DM. I also checked the directional KL issue raised by the reader: Eq 6 is an unsigned D_KL, so a feature less prevalent in diagnosed patients still contributes positively in Eq 7 (e.g., cervicalgia, 6.36% diagnosed vs 11.19% undiagnosed in Table 1). That is a real internal defect, but it is secondary to the missing external outcome because even a correctly signed similarity score would not establish the central claim without follow-up or lab validation. The proposed test, linking risk strata to HbA1c or future E11 diagnoses, would settle the transfer premise. Until that is done, the current REJECT verdict is appropriate.","tokens_in":13012,"tokens_out":7417,"duration_ms":78241,"concrete_test":"On a dataset with laboratory values or follow-up claims (e.g., an EHR cohort, or MarketScan years after 2022), take a random sample of undiagnosed patients from each of the three risk strata: low (<50th percentile), moderate (50th-90th percentile), and high (>90th percentile). Define ground truth as HbA1c >= 6.5% or a first E11 diagnosis within 2 years. Compute the score's AUC and the positive predictive value of the high-risk stratum relative to the moderate and low strata. If high-risk patients do not show materially higher conversion or abnormal-HbA1c rates, the transfer premise fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim, that the score estimates T2DM risk in undiagnosed people (Abstract; Sec 3.5), rests on the premise stated in Sec 2.2: 'assuming that undiagnosed individuals with similar covariate profiles may also be at risk.' That premise is never checked against actual disease status. The four validation methods are self-referential. Method 1 only shows that diagnosed and undiagnosed groups have different score distributions, which follows from construction. Method 2 measures agreement with an SME and AI models, not with a T2DM diagnosis. Method 3 compares covariate overlap between groups defined by the same risk score. Method 4 selects 'negative' examples using the risk-score threshold, then trains an XGBoost classifier on those labels; the observed performance decline with higher thresholds is an inevitable consequence of choosing progressively noisier labels from the model's own ranking, not evidence about T2DM. Sec 4.1 concedes there is no HbA1C data, and no follow-up diagnosis is reported. Consequently, the high-risk proportion and its rough match to CDC undiagnosed prevalence is a post-hoc alignment. If the similarity-to-diagnosed-profile assumption is wrong, the score is a general multimorbidity/polypharmacy index, not a T2DM risk estimate.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes an unsupervised framework for estimating T2DM risk in undiagnosed individuals from claims data. The method applies NMF to pre-diagnosis covariates of diagnosed T2DM patients, derives rank-weighted coefficients (RWC) for features, computes KL divergence between diagnosed and undiagnosed feature prevalences, and forms a risk score as a weighted sum over selected features. Risk categories are defined by percentiles of the diagnosed score distribution. Validation comprises distribution comparisons, agreement with an SME and three AI models, Jaccard similarity between high-risk groups, and an XGBoost classifier trained with negative labels selected by risk-score thresholds. The paper reports that 3.18% of undiagnosed individuals are high-risk, which it claims aligns with CDC undiagnosed diabetes prevalence.","tokens_in":13214,"tokens_out":4628,"duration_ms":43523,"significance":"If the central claim were established, the approach would offer a label-efficient and interpretable alternative to supervised T2DM risk models in settings where confirmed negatives are unavailable, with potential applicability to other phenotypes. The authors include several strengths: a large real-world claims cohort, a clearly specified algorithmic pipeline, and reproducible descriptions of the NMF and score computations. However, the evidence for the core claim that the score estimates T2DM risk in undiagnosed individuals is self-referential, and the KL divergence formulation contains a directional error. The significance of the work therefore rests on unvalidated assumptions rather than demonstrated results.","major_comments":[{"comment":"The KL divergence in Eq. (6) is always nonnegative, so it cannot indicate whether a feature is more prevalent in diagnosed or undiagnosed individuals. The statement in Sec. 2.7 that 'A large KL divergence suggests that feature j is more prevalent among diagnosed individuals' is incorrect. For example, cervicalgia has prevalence 6.36% in diagnosed and 11.19% in undiagnosed (Table 1), yet its d_j is still positive. Because Eq. (7) sums w_j * d_j for every present feature, the risk score rewards features that are more common in undiagnosed individuals just as much as features enriched in diagnosed patients. This undermines the directional interpretation of the risk score and the central claim that it estimates T2DM risk.","section":"Sec. 2.7, Eq. (6) and Eq. (7)"},{"comment":"The negative labels in Method 4 are selected from the model's own risk-score thresholds, and then the same covariate space is used to train XGBoost. The observed decline in classification performance as the threshold increases is an expected consequence of labeling undiagnosed individuals by the model's own ranking: higher thresholds include individuals the model itself considers more T2DM-like, which mechanically makes the binary separation harder. This does not provide independent evidence about whether low-risk undiagnosed individuals are true negatives. The conclusion that the model 'can also reliably identify negative cases using a low-risk threshold' is therefore circular.","section":"Sec. 3.5, Method 4 (Table 5)"},{"comment":"The central transfer assumption, stated in Sec. 2.2 as 'assuming that undiagnosed individuals with similar covariate profiles may also be at risk,' is never tested against actual T2DM outcomes. The paper acknowledges in Sec. 4.1 the lack of HbA1C measurements and reports no follow-up diagnosis data. The only external anchor, the rough match between the 3.18% high-risk proportion and the CDC undiagnosed prevalence in the Discussion, is post-hoc: the risk thresholds were set in consultation with an SME (Sec. 2.8) before the CDC comparison. As a result, the Abstract and Sec. 3.5 claim that high-risk categories 'may in fact be undiagnosed T2DM cases' is unsupported by the presented evidence.","section":"Sec. 2.2 and Sec. 4.1"}],"minor_comments":[{"comment":"The text contains a typo: 'undiagnose individuals' should be 'undiagnosed individuals.'","section":"Sec. 2.2"},{"comment":"The typesetting of Eq. (6) is broken, with stray newline characters and unclear fraction formatting; the equation should be presented as a standard KL divergence expression.","section":"Sec. 2.7, Eq. (6)"},{"comment":"Lines 5-6 of Algorithm 3 split the two terms of the KL formula with a stray '+' and the log arguments are awkwardly nested; the pseudocode should be cleaned up for readability.","section":"Algorithm 3"},{"comment":"The caption contains a typo: 'T able' should be 'Table'; also, the phrase 'Comorbidities and medications are top covariates with high R WC values' could be clarified to specify that the listed items are examples drawn from the top RWC features.","section":"Table 1 caption"},{"comment":"The justification for choosing arctangent over sigmoid relies on informal claims about 'overconfident estimates' and 'rapid asymptotic behavior' without quantification; a brief reference to the score distribution or a comparison of transformations would be more rigorous.","section":"Sec. 2.8"}],"recommendation":"reject","confidential_remarks":"The paper studies an important problem and the general idea of transferring latent patterns from diagnosed to undiagnosed patients is worth exploring. However, the load-bearing issues are the directional error in the KL-based risk score and the absence of any external outcome validation. The validation methods are self-consistent but do not establish that the score measures T2DM risk. In its current form, the manuscript does not meet the standard for publication in a serious journal."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper introduces a specific combination - NMF rank-weighted coefficients with KL-divergence-based prevalence weighting - to score T2DM risk in undiagnosed people from claims data. That combination is new as far as I can tell, and the writing is clear. The pipeline is reproducible from the equations, and the authors are honest about their choices, including why they did not use PU learning.\n\nThe central problem is that every validation is self-referential. Method 1 shows diagnosed and undiagnosed score distributions differ, which is almost built into the construction. Method 2 is agreement with SME/AI, not with a diagnosis. Method 3 compares covariate overlap between groups selected by the same score. Method 4 picks negatives using the score thresholds and then shows classifiers trained on those labels perform better; that is expected, not evidence about T2DM. There is no HbA1C or follow-up diagnosis anywhere, and the limitations section concedes this. So the abstract's claim that the score 'estimates T2DM risk' in undiagnosed individuals is unsupported as stated.\n\nThere is also a real direction problem in Eq. 6-7. KL divergence is not directional, and the formula computes D_KL(p_d || p_u), which is not symmetric, but a feature like cervicalgia that is less common in diagnosed patients (6.36% vs 11.19%) still gets a positive d_j and contributes positively to risk. The paper never flags this. It may or may not matter much in aggregate, but it is a genuine flaw in the scoring rule.\n\nThe method is not nonsense. The idea of learning patterns from positives only and transferring to unlabeled data is coherent, and the NMF-RWC weighting is a sensible way to summarize feature importance. The scale of the data is impressive. But the evidence offered does not establish the central claim.\n\nWho is this for? People working on positive-unlabeled problems in health claims might read it as a methodological proposal, but they should be warned about the validation. A serious referee could ask for external validation - e.g., a holdout cohort with confirmed diagnosis, or HbA1C - and the paper would be substantially improved. As it stands, I would not cite it as evidence that the risk score works. But it deserves peer review rather than desk rejection, because the method is new, clearly specified, and the flaw is in the evaluation, not in the idea. I'd bring it to a reading group as a case study in circular validation.","headline":"A cleanly-written unsupervised risk-scoring method for T2DM in claims data, but the validation is circular and the score's direction has an unaddressed sign error; worth a referee to push for external validation.","tokens_in":13784,"tokens_out":1579,"would_cite":false,"duration_ms":15909,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"An unsupervised score built from diagnosed patients' comorbidity and medication patterns estimates T2DM risk in undiagnosed people without needing confirmed negatives.","keywords":["type 2 diabetes mellitus","undiagnosed diabetes","non-negative matrix factorization","Kullback-Leibler divergence","risk score","claims data","multimorbidity","unsupervised learning"],"falsifier":"Give a cohort of people scored high-risk by this method a hemoglobin A1C test (or track their future T2DM diagnoses) and compare with people scored low-risk; if incidences are equal, the score is measuring general multimorbidity rather than T2DM risk. The paper's own limitation section identifies the absence of HbA1C data as the reason this test was not performed.","tokens_in":12753,"feed_emoji":"🩺","tokens_out":11110,"duration_ms":98497,"temperature":0.7,"pith_summary":"Type 2 diabetes is frequently undiagnosed, and supervised machine-learning models are hampered because claims data provide confirmed positives but no confirmed negatives. This paper proposes an unsupervised pipeline that learns latent comorbidity and medication patterns from diagnosed T2DM patients alone, then scores every undiagnosed patient by how closely their record matches those patterns. The score combines NMF-derived rank-weighted coefficients with KL-divergence prevalence weights, and patients above the 90th percentile of the diagnosed-score distribution are labeled high risk. Applied to 17.1 million US claims records, 3.18% of undiagnosed individuals fall into the high-risk group, in line with national estimates of undiagnosed diabetes. The authors argue this yields an interpretable, scalable tool for prioritizing undiagnosed patients for screening.","feed_headline":"Model flags 3.18% of undiagnosed people as high type 2 diabetes risk","feed_subtitle":"Matching comorbidity and medication patterns to diagnosed patients ranks who should be screened first.","key_machinery":"The central object is the risk score $r_i$, built from two learned weights per covariate. $w_j$, the rank-weighted coefficient, is computed from the NMF coefficient matrix $H$ of diagnosed patients: $w_j = \\sum_i H_{ij}/r_{ij} / \\sum_i 1/r_{ij}$ over latent components, emphasizing covariates that rank high in many components. $d_j$ is the KL divergence between the covariate's prevalence in diagnosed and undiagnosed groups, with smoothing $\\epsilon=10^{-8}$. The product $w_j d_j$ assigns each covariate a single importance weight, and the score sums those weights over the covariates present in a patient's record. NMF supplies the latent comorbidity-and-polypharmacy patterns; KL divergence supplies the discrimination signal; the indicator sum makes the result interpretable as a weighted count of T2DM-associated findings.","core_discovery":"The paper's central claim is that undiagnosed patients can be ranked for T2DM risk by measuring how similar their pre-diagnosis clinical fingerprint is to that of people already diagnosed with T2DM. The fingerprint is a sparse vector of ICD-10 diagnoses and RxNorm medications; NMF factorizes the diagnosed-only matrix into latent components, and each covariate receives a rank-weighted coefficient $w_j$ that rewards consistent high contributions across components. A KL-divergence $d_j$ then measures how much more prevalent that covariate is among diagnosed than undiagnosed patients, and the raw risk score is $r_i = \\sum_{j\\in S} w_j d_j I(x_{ij}>0)$. Scores are arctangent-scaled to $[0,1)$ and converted to percentiles against the diagnosed distribution; undiagnosed patients above the 90th percentile are called high risk and are suspected to be undiagnosed T2DM cases. The paper supports this with four validations: temporal separation, expert and AI agreement, Jaccard similarity of covariate profiles, and improved classifier performance when low-risk undiagnosed patients serve as negatives.","pith_inferences":["The score is best interpreted as a similarity-to-diagnosed-T2DM index, not a calibrated probability; the paper says the arctangent transform should not be read as a probability, so percentile ranks carry the operational meaning.","The closeness of the 3.18% high-risk share to national undiagnosed-prevalence statistics is suggestive but not a direct validation, because the category thresholds were set by expert review against the diagnosed-score distribution rather than by any biomarker.","A missing experiment that would settle the transfer assumption is to follow high-risk undiagnosed patients with hemoglobin A1C tests or future diagnosis codes; the paper itself notes the claims data contain no HbA1C measurements.","The ranking could be coupled with positive-unlabeled class-proportion estimation to convert relative scores into an estimated count of undiagnosed cases in a particular region or demographic."],"forward_implications":["A health system can generate a screening-priority ranking for undiagnosed patients using only confirmed-positive cases, removing the need for hand-labeled negative examples.","Low-risk undiagnosed patients can stand in as a cleaner negative set, which the paper shows improves a downstream classifier's accuracy, Matthews correlation, AUC-ROC, and Brier score.","The same pipeline, NMF on positives, rank-weighted coefficient selection, KL-divergence weighting, and percentile thresholding, should transfer to other underdiagnosed phenotypes with reliable confirmed-case codes.","The roughly 3.18% of undiagnosed people assigned high risk become a concrete target for follow-up screening and early intervention.","Because the score is a weighted count of indicators $I(x_{ij}>0)$ rather than raw encounter counts, it is insensitive to how often a patient happens to visit the clinic."],"supporting_citations":[{"why":"Supplies the NMF algorithm that factorizes the diagnosed-patient covariate matrix into latent components, the source of the rank-weighted coefficients.","marker":"Lee and Seung, 1999"},{"why":"Supplies the KL-divergence formula used to weight each selected covariate by how differently it is distributed between diagnosed and undiagnosed groups.","marker":"Kullback and Leibler, 1951"},{"why":"Supplies the OMOP CDMv5 standardization applied to the claims database to create a uniform covariate matrix.","marker":"Voss et al., 2015"},{"why":"Defines the positive-unlabeled setting and motivates the decision to build risk scores from confirmed positives alone.","marker":"Kumar and Lambert, 2024"},{"why":"Supplies the label-noise analysis that justifies not treating undiagnosed patients as true negatives in a supervised design.","marker":"Frénay and Verleysen, 2013"},{"why":"Supplies the gradient-boosted classifier used in the balanced-dataset validation showing cleaner performance when negatives come from low-risk undiagnosed patients.","marker":"Chen and Guestrin, 2016"},{"why":"Supplies the national undiagnosed-diabetes prevalence estimate used as an external benchmark for the model's high-risk rate.","marker":"Disease Control and Prevention"}],"fun_headline_variants":["Unsupervised model flags 3.18% of undiagnosed as high T2DM risk","Pattern matching finds undiagnosed patients at 3.18% T2DM risk","NMF latent patterns rank undiagnosed patients for T2DM screening","Unsupervised approach estimates T2DM risk in undiagnosed populations","3.18% of undiagnosed flagged as high risk for type 2 diabetes"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that an undiagnosed person whose comorbidity and medication record resembles the pre-diagnosis record of diagnosed T2DM patients is genuinely at elevated risk for T2DM, a transfer the study never checks with laboratory values or follow-up diagnoses.","fun_headline_variants_meta":{"raw":{"variants":["Unsupervised model flags 3.18% of undiagnosed as high T2DM risk","Pattern matching finds undiagnosed patients at 3.18% T2DM risk","NMF latent patterns rank undiagnosed patients for T2DM screening","Unsupervised approach estimates T2DM risk in undiagnosed populations","3.18% of undiagnosed flagged as high risk for type 2 diabetes"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000809,"raw_usage":{"total_tokens":3581,"prompt_tokens":1007,"completion_tokens":2574,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":623,"completion_tokens_details":{"reasoning_tokens":2466}},"tokens_in":623,"tokens_out":2574,"duration_ms":18937,"temperature":1.0,"reasoning_tokens":2466,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T13:22:18.749768+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Give a cohort of people scored high-risk by this method a hemoglobin A1C test (or track their future T2DM diagnoses) and compare with people scored low-risk; if incidences are equal, the score is measuring general multimorbidity rather than T2DM risk. The paper's own limitation section identifies the absence of HbA1C data as the reason this test was not performed.","supporting_citations":[{"cited_title":", Seung , H.S","cited_arxiv_id":null,"evidence_quote":"Supplies the NMF algorithm that factorizes the diagnosed-patient covariate matrix into latent components, the source of the rank-weighted coefficients."},{"cited_title":", Leibler , R.A","cited_arxiv_id":null,"evidence_quote":"Supplies the KL-divergence formula used to weight each selected covariate by how differently it is distributed between diagnosed and undiagnosed groups."},{"cited_title":", Makadia , R","cited_arxiv_id":null,"evidence_quote":"Supplies the OMOP CDMv5 standardization applied to the claims database to create a uniform covariate matrix."},{"cited_title":", Guestrin , C","cited_arxiv_id":null,"evidence_quote":"Supplies the gradient-boosted classifier used in the balanced-dataset validation showing cleaner performance when negatives come from low-risk undiagnosed patients."}],"review_version":1}