REVIEW 3 major objections 5 minor 37 references
Adaptable Cardiovascular Disease Risk Prediction from Heterogeneous Data using Large Language Models
T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read A large language model fine-tuned on 467,000 UK Biobank records matches the best tabular models for 10-year CVD risk prediction while also handling incomplete, unstructured, and shifted clinical inputs.
desk verdict Solid applied LLM-for-tabular-clinical-data work with a clean UK Biobank benchmark; the 'state-of-the-art' and 'seamless text integration' claims outrun what the evidence supports. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the text prompt. Every patient is serialized into a structured description such as "Gender: Male; Age: 41 years; ...", and AdaCVD, based on Mistral-7B-Instruct and trained with LoRA (low-rank adaptation, updating about 0.13% of parameters), is fine-tuned to answer Yes or No to whether the patient will have a major cardiovascular event within ten years. Risk is read out from the normalized logits of the Yes/No answer rather than from a numeric response. Because the input is text, features can be missing, reordered, or replaced by free-text summaries without changing the model interface.
What would settle it
Collect real clinician-written notes paired with 10-year CVD outcomes for a cohort comparable to UK Biobank, then compare AdaCVD adapted on those real notes against the same model adapted on the synthetic summaries; a large AUROC drop or worse calibration would falsify the unstructured-text claim.
Extended reading notes
Core claim
The central discovery is that CVD risk prediction can be reframed as a binary text-completion problem, and a domain-fine-tuned LLM can solve it at the level of the best tabular machine-learning models while adding flexibility. On UK Biobank base risk factors alone, AdaCVD reaches AUROC 0.738, matching gradient-boosted trees; incorporating lifestyle, polygenic risk scores, ICD codes, medical history, and other categories raises this to 0.774, with well-calibrated predictions. The same model, after light additional fine-tuning, handles incomplete inputs (AdaCVD-Flex), transfers to unstructured patient summaries (AdaCVD-Text/Notes), and transfers zero-shot to the Framingham cohort at AUROC 0.687, reaching 0.693 after fine-tuning on 1,500 examples. The paper's claim is that one adaptable model can replace rigid risk scores and fixed-feature ML pipelines in realistic, heterogeneous clinical settings.
Load-bearing premise
The text-related results assume that LLM-generated summaries from structured data realistically mimic the noise, ambiguity, and information density of real clinical notes; if that assumption fails, the demonstrated transfer to free-text may not hold.
Editorial extensions
If this is right
- A single fine-tuned LLM can replace a suite of feature-group-specific expert models, with only a small AUROC penalty, when patient records vary in completeness.
- Risk prediction improves as broader patient information is included, up to +4.9% AUROC, with the largest gains from medical history, ICD codes, polygenic risk scores, and lifestyle factors.
- Structured-data fine-tuning transfers to unstructured text at up to 100 times data efficiency, meaning a model pre-adapted on structured records needs very few text examples to adapt.
- Zero-shot general LLMs are not sufficient for this task; task-specific fine-tuning on population-scale data is required.
- On a shifted population, AdaCVD works zero-shot and after 1,500 training examples matches the performance of the Framingham Risk Score.
Reading between the lines
- Beyond the paper, the unstructured-text results are evaluated on LLM-generated summaries rather than real clinical notes, so whether the 100x data-efficiency claim survives contact with actual physician documentation is an open empirical question.
- If the recipe holds, the same serialized-text plus fine-tuning approach could be carried over to other clinical outcomes and other tabular datasets without architectural changes.
- The value proposition over gradient-boosted trees is flexibility, not raw AUROC on clean tabular data; in settings where inputs are already complete and structured, the performance difference is negligible.
- A testable extension would be to measure how much prediction degrades when only a subset of feature groups is available at inference, to quantify AdaCVD-Flex's behavior in deployment.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces AdaCVD, a Mistral-7B-Instruct model fine-tuned with LoRA on 467,063 structured patient descriptions from the UK Biobank for 10-year cardiovascular disease risk prediction. The authors report AUROC 0.738 using only base risk factors, matching a LightGBM baseline, and AUROC 0.774 when all feature groups are incorporated. They further propose three adaptations: AdaCVD-Flex for variable and incomplete inputs, AdaCVD-Text/Notes for unstructured free-text patient summaries, and AdaCVD-Shift for distribution shifts evaluated on the Framingham cohort. The paper claims these adaptations are data-efficient and that the model offers a flexible alternative to rigid tabular risk models.
Significance. If the results hold, the paper provides credible evidence that a fine-tuned open-access LLM can match gradient-boosted trees on structured tabular risk factors while offering additional flexibility in input formatting. The core held-out evaluation on the UK Biobank is well designed, the test split is independent, and the public code release strengthens reproducibility. However, the significance is currently limited by two gaps: the 'state-of-the-art' claim is not supported for all-feature inputs because no tabular or AutoML baseline is trained on all feature groups, and the unstructured-text results are evaluated exclusively on LLM-generated summaries rather than real clinical notes. The latter is explicitly acknowledged as a limitation in the Discussion but remains load-bearing for the paper's central adaptability claims.
major comments (3)
- [Section 4.2.3 / Fig. 6a / Discussion] The unstructured-text adaptation results, including zero-shot AUROC 0.685, the adaptation curves, and the 100x data-efficiency claim, are evaluated solely on UKB-notes: free-text summaries generated by an LLM from structured UK Biobank data. The paper itself states in Section 4.2.3 that 'In the absence of real-world datasets ... we leveraged LLMs to generate patient descriptions that mimic realistic clinical notes,' and the Discussion emphasizes 'the need for the creation of publicly available, high-quality datasets containing real clinical text.' This is a genuine external-validity threat, not an internal inconsistency: the generated examples in Supplementary A.2 are template-like, numerically precise, and largely complete, whereas real clinical notes contain noise, abbreviations, hedging, and irrelevant content. Moreover, both the generator and AdaCVD are based on Mistral-7B, so the model may exploit style-specific shortcuts. Because the 'seamless integration of unstructured text' claim and the data-efficiency numbers rest entirely on these synthetic summaries, the claims should be substantially softened or supplemented with an external evaluation on real clinical text. At minimum, the paper should label these experiments as simulations and make clear that transfer to real notes is unvalidated.
- [Section 2 / Fig. 3b / Methods 4.4] The statement that AdaCVD 'achieved state-of-the-art performance' is not supported for the all-feature setting. Figure 3b compares AdaCVD variants against medical risk scores, but no tabular ML baseline (e.g., LightGBM, logistic regression, or AutoML) is trained on the same comprehensive feature groups. The only strong tabular baseline, LightGBM, is evaluated using base risk factors only, where AdaCVD matches it (AUROC 0.738). For the all-feature comparison (AUROC 0.774), the reader cannot tell whether this improves on a strong tabular model trained on the same inputs. I recommend adding a tabular baseline trained on all feature groups, or explicitly reframing the claim as 'superior to medical risk scores and comparable to a strong tabular baseline on base factors.' This is directly testable and should be feasible within the manuscript's scope.
- [Section 4.2.2 / Fig. 5] The AdaCVD-Flex experiment trains by randomly sampling a subset of features per patient, but the paper does not characterize this training distribution or provide evidence that it matches realistic missingness patterns in clinical records. The claim that AdaCVD-Flex 'can handle heterogeneous and incomplete patient inputs' and performs well 'in real-world settings with variable and incomplete data' depends on this assumption. Please report the number and composition of sampled subsets, compare them to actual missingness patterns in a clinical cohort if possible, and temper the external claim if such evidence is unavailable.
minor comments (5)
- [Abstract / Introduction] The phrase 'seamlessly integrates structured data and unstructured text' overstates the evidence, since all text results are obtained on synthetic summaries; suggest rewording to 'simulated clinical text' or adding an explicit caveat in the abstract.
- [Section 4.4.1] The statement that 'we report the median value and their 95% spread across 5000 bootstrapping rounds' is ambiguous; define the spread as a percentile interval or standard error, and indicate whether error bars in Figures 3 and 5 represent this quantity.
- [Section 4.2.3 / Table 1] There is an inconsistency in dataset sizes: Section 4.2.3 states that UKB-notes is generated for n = 40,000 patients, while Table 1 lists the AdaCVD-Text training dataset as 20k; clarify how many examples are used for the adaptation curve and how the subsets are split.
- [Section 2 (ICD vs. Medical History)] The comparison of ICD codes and medical history relies on median token counts (201 vs. 469); reporting only medians does not convey the overlap or variance of the distributions, so add quartiles or histograms if this comparison is meant to be substantive.
- [Figure 6b] The dotted line for the Framingham risk score is described as derived from a larger version of the same dataset; state explicitly that this is an external reference and not a matched within-study benchmark, so readers can interpret the comparison appropriately.
Circularity Check
Partial circularity on the unstructured-text axis: UKB-notes are LLM-generated from the same UK Biobank structured features used to train AdaCVD, so the 'seamless integration of unstructured text' claim rests on a synthetic re-encoding rather than independent clinical text. The core held-out structured benchmark remains non-circular.
-
renaming known result
[Section 2 'Adapting to Textual Patient Representations'; Section 4.2.3]
"In the absence of real-world datasets containing unstructured textual descriptions of patients with corresponding 10-year CVD outcomes, we leveraged LLMs to generate patient descriptions that mimic realistic clinical notes. We evaluated our model's ability to adapt to these unstructured representations using the generated free-text patient descriptions UKB-notes."
The unstructured-text target domain is defined as LLM-generated summaries of the same UK Biobank structured features on which AdaCVD was trained. The zero-shot AUROC 0.685, the few-shot AUROC 0.697, and the 100x data-efficiency claim therefore measure transfer to a synthetic paraphrase of the source distribution, not to independent real clinical notes. Because both the generator and the adapted model are LLMs and the underlying features and outcomes are identical to the structured training data, the 'seamlessly integrates unstructured text' headline claim reduces to evaluating on a re-encoding of the known structured-data prediction task.
full rationale
The core benchmark claims are independent of the model's fitted values: AdaCVD is evaluated on a held-out 20% UK Biobank test split (Section 4.3.1: 'The cohort was randomly split into a training (75%), test (20%), and validation set (5%). All reported results are computed on the test set unless stated otherwise.'), giving AUROC 0.738 with base risk factors and 0.774 with all patient information. The Framingham evaluation is an external cohort and is also non-circular. The one partially circular step is the unstructured-text axis: UKB-notes are generated by an LLM from the same structured UKB features used for training, so the demonstrated 'adaptation to unstructured text' is a test on a synthetic paraphrase of the training distribution, not on real clinician-written notes. The paper explicitly flags this in the Discussion limitation ('we emphasize the need for the creation of publicly available, high-quality datasets containing real clinical text'), which is a genuine external-validity caveat. Because this axis is one of the paper's three headline contributions, the circularity is partial but real; the central state-of-the-art structured benchmark and the Framingham shift evaluation remain independently grounded.
Assumptions & free parameters
free parameters (5)
- LoRA rank =
16
- Fine-tuning epochs per variant =
2 (AdaCVD), 1 (AdaCVD-Flex), 5 (AdaCVD-Notes), 10 (AdaCVD-Shift)
- Token length caps for generated notes =
200 tokens (base summaries), 400 tokens (detailed summaries)
- Random feature subset sampling for AdaCVD-Flex =
Not specified
- Train/validation/test split =
75%/5%/20%
assumptions (6)
- ad hoc to paper LLM-generated free-text summaries preserve enough clinical information to serve as valid proxies for real clinical notes.
- ad hoc to paper Random omission of features during AdaCVD-Flex training approximates real-world missingness in clinical records.
- domain assumption UK Biobank participants (467,063 after exclusion) are an appropriate training population for generalized CVD risk prediction.
- domain assumption The defined outcome (first CVD event via ICD-9/10, self-report, and death registry within 10 years) aligns with clinically accepted endpoints.
- domain assumption The Framingham cohort, with 3,014 complete-case participants from period three, is representative of a distribution shift that matters clinically.
- domain assumption The base risk factor list used for the controlled comparison is the appropriate reference set for clinical risk scores.
Cite this review
Pith. "Pith review of Adaptable Cardiovascular Disease Risk Prediction from Heterogeneous Data using Large Language Models." pith.science (2026). https://pith.science/paper/GWJXOIZZ
@misc{pith2026250524655,
author = {Pith},
title = {Pith review of: Adaptable Cardiovascular Disease Risk Prediction from Heterogeneous Data using Large Language Models},
year = {2026},
howpublished = {\url{https://pith.science/paper/GWJXOIZZ}},
note = {Machine review of arXiv:2505.24655}
}
read the original abstract
Cardiovascular disease (CVD) risk prediction models are essential for identifying high-risk individuals and guiding preventive actions. However, existing models struggle with the challenges of real-world clinical practice as they oversimplify patient profiles, rely on rigid input schemas, and are sensitive to distribution shifts. We developed AdaCVD, an adaptable CVD risk prediction framework built on large language models extensively fine-tuned on over half a million participants from the UK Biobank. In benchmark comparisons, AdaCVD surpasses established risk scores and standard machine learning approaches, achieving state-of-the-art performance. Crucially, for the first time, it addresses key clinical challenges across three dimensions: it flexibly incorporates comprehensive yet variable patient information; it seamlessly integrates both structured data and unstructured text; and it rapidly adapts to new patient populations using minimal additional data. In stratified analyses, it demonstrates robust performance across demographic, socioeconomic, and clinical subgroups, including underrepresented cohorts. AdaCVD offers a promising path toward more flexible, AI-driven clinical decision support tools suited to the realities of heterogeneous and dynamic healthcare environments.
Figures
Figures from the paper (9 more)
Reference graph
Works this paper leans on
-
[1]
Cardiovascular Diseases (CVDs) https://www.who.int/news-room/fact-sheets/detail/cardiovascular- diseases-(cvds)
-
[2]
Roth, G. A., Mensah, G. A. & Fuster, V. The Global Burden of Cardiovascular Diseases and Risks: A Compass for Global Action.Journal of the American College of Cardiology76, 2980–2981 (Dec. 2020)
work page 2020
-
[3]
Khan, S. S. et al. Novel Prediction Equations for Absolute Risk Assessment of Total Cardiovascu- lar Disease Incorporating Cardiovascular-Kidney-Metabolic Health: A Scientific Statement From the American Heart Association.Circulation 148, 1982–2004 (Dec. 2023)
work page 1982
-
[4]
Overview | Cardiovascular Disease: Risk Assessment and Reduction, Including Lipid Modification | Guidance | NICEhttps://www.nice.org.uk/guidance/ng238. Dec. 2023
work page 2023
-
[5]
Visseren, F. L. J.et al.2021 ESC Guidelines on Cardiovascular Disease Prevention in Clinical Practice: Developed by the Task Force for Cardiovascular Disease Prevention in Clinical Practice with Represen- tatives of the European Society of Cardiology and 12 Medical Societies With the Special Contribution of the European Association of Preventive Cardiolog...
work page 2021
-
[6]
D’Agostino, R. B. et al.General Cardiovascular Risk Profile for Use in Primary Care: The Framingham Heart Study.Circulation 117, 743–753 (Feb. 2008)
work page 2008
-
[7]
Rippe, J. M. Lifestyle Strategies for Risk Factor Reduction, Prevention, and Treatment of Cardiovas- cular Disease.American Journal of Lifestyle Medicine13, 204–212 (2019)
work page 2019
-
[8]
M., Bolton, T., Angelantonio, E
Alaa, A. M., Bolton, T., Angelantonio, E. D., Rudd, J. H. F. & van der Schaar, M. Cardiovascular Disease Risk Prediction Using Automated Machine Learning: A Prospective Study of 423,604 UK Biobank Participants.PLOS ONE 14, e0213653 (May 2019)
work page 2019
Show all 37 references
-
[9]
F., Reps, J., Kai, J., Garibaldi, J
Weng, S. F., Reps, J., Kai, J., Garibaldi, J. M. & Qureshi, N. Can Machine-Learning Improve Cardio- vascular Risk Prediction Using Routine Clinical Data?PLOS ONE 12, e0174944 (Apr. 2017)
2017
-
[10]
Bubeck, S. et al. Sparks of Artificial General Intelligence: Early experiments with GPT-4. arXiv preprints. eprint: 2303.12712 (2023)
2023 arXiv
-
[11]
GPT-4 Technical ReportMar
OpenAI et al. GPT-4 Technical ReportMar. 2024. arXiv:2303.08774 [cs]
2024 arXiv
-
[12]
Brown, T. et al. Language Models are Few-Shot Learnersin Advances in Neural Information Processing Systems (eds Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M. & Lin, H.)33 (Curran Associates, Inc., 2020), 1877–1901
2020
-
[13]
et al.Foundation Models for Generalist Medical Artificial Intelligence.Nature 616, 259–265 (Apr
Moor, M. et al.Foundation Models for Generalist Medical Artificial Intelligence.Nature 616, 259–265 (Apr. 2023)
2023
-
[14]
Thirunavukarasu, A. J. et al. Large Language Models in Medicine.Nature Medicine 29, 1930–1940 (Aug. 2023)
1930
-
[16]
Singhal, K. et al. Toward Expert-Level Medical Question Answering with Large Language Models. Nature Medicine31, 943–950 (Mar. 2025)
2025
-
[17]
& Sontag, D.Large Language Models are Few-Shot Clinical Information Extractorsin Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing(2022)
Agrawal, M., Hegselmann, S., Lang, H., Kim, Y. & Sontag, D.Large Language Models are Few-Shot Clinical Information Extractorsin Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing(2022)
2022
-
[18]
Nature Medicine30, 1134–1142 (Apr
Van Veen, D.et al.Adapted Large Language Models Can Outperform Medical Experts in Clinical Text Summarization. Nature Medicine30, 1134–1142 (Apr. 2024)
2024
-
[19]
Framingham Heart Studyhttps://www.framinghamheartstudy.org/
-
[20]
UK Biobank - UK Biobankhttps://www.ukbiobank.ac.uk. Jan. 2025. 12
2025
-
[21]
Jiang, A. Q. et al. Mistral 7BOct. 2023. arXiv:2310.06825 [cs]
2023 arXiv
-
[22]
Hu, E. J. et al. LoRA: Low-Rank Adaptation of Large Language ModelsinThe Tenth International Con- ference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022(OpenReview.net, 2022)
2022
-
[23]
Khan, S. S. et al.Development and Validation of the American Heart Association’s PREVENT Equa- tions. Circulation 149, 430–449 (Feb. 2024)
2024
-
[24]
Hippisley-Cox, J. et al.Derivation and Validation of QRISK, a New Cardiovascular Disease Risk Score for the United Kingdom: Prospective Open Cohort Study.BMJ : British Medical Journal335, 136 (July 2007)
2007
-
[25]
Arnett, D. K. et al.2019 ACC/AHA Guideline on the Primary Prevention of Cardiovascular Disease: A Report of the American College of Cardiology/American Heart Association Task Force on Clinical Practice Guidelines.Circulation 140 (Sept. 2019)
2019
-
[26]
Framingham Teaching Datasets - Public Use Datasetshttps://biolincc.nhlbi.nih.gov/teaching/
-
[27]
Singhal, K. et al. Large Language Models Encode Clinical Knowledge. Nature 620, 172–180 (Aug. 2023)
2023
-
[28]
et al.Evaluation of GPT-4 for 10-Year Cardiovascular Risk Prediction: Insights from the UK Biobank and KoGES Data.iScience 27, 109022 (Feb
Han, C. et al.Evaluation of GPT-4 for 10-Year Cardiovascular Risk Prediction: Insights from the UK Biobank and KoGES Data.iScience 27, 109022 (Feb. 2024)
2024
-
[29]
Grattafiori, A. et al. The Llama 3 Herd of ModelsNov. 2024. arXiv:2407.21783 [cs]
2024 arXiv
-
[30]
Abdin, M. et al. Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone Aug. 2024. arXiv:2404.14219 [cs]
2024 arXiv
-
[31]
Team, G. et al. Gemma 2: Improving Open Language Models at a Practical SizeOct. 2024. arXiv: 2408.00118 [cs]
2024 arXiv
-
[32]
Hegselmann, S. et al. Tabllm: Few-shot Classification of Tabular Data with Large Language Modelsin International Conference on Artificial Intelligence and Statistics(PMLR, 2023), 5549–5581
2023
-
[33]
Belyaeva, A. et al. Multimodal LLMs for Health Grounded in Individual-Specific DataJuly 2023. arXiv: 2307.09018 [cs, q-bio]
2023 arXiv
-
[34]
et al.Parameter-Efficient Fine-Tuning of Large-Scale Pre-Trained Language Models.Nature Machine Intelligence5, 220–235 (Mar
Ding, N. et al.Parameter-Efficient Fine-Tuning of Large-Scale Pre-Trained Language Models.Nature Machine Intelligence5, 220–235 (Mar. 2023)
2023
-
[35]
et al.UK Biobank: An Open Access Resource for Identifying the Causes of a Wide Range of Complex Diseases of Middle and Old Age.PLoS Medicine12, e1001779 (Mar
Sudlow, C. et al.UK Biobank: An Open Access Resource for Identifying the Causes of a Wide Range of Complex Diseases of Middle and Old Age.PLoS Medicine12, e1001779 (Mar. 2015)
2015
-
[36]
M., Bolton, T., Di Angelantonio, E., Rudd, J
Alaa, A. M., Bolton, T., Di Angelantonio, E., Rudd, J. H. F. & Van Der Schaar, M. Cardiovascular Disease Risk Prediction Using Automated Machine Learning: A Prospective Study of 423,604 UK Biobank Participants.PLOS ONE 14 (ed Aalto-Setala, K.) e0213653 (May 2019)
2019
-
[37]
SCORE2 Risk Prediction Algo- rithms: New Models to Estimate 10-Year Risk of Cardiovascular Disease in Europe.European Heart Journal 42, 2439–2454 (July 2021)
SCORE2 working group and ESC Cardiovascular risk collaboration. SCORE2 Risk Prediction Algo- rithms: New Models to Estimate 10-Year Risk of Cardiovascular Disease in Europe.European Heart Journal 42, 2439–2454 (July 2021)
2021
-
[38]
UK Biobank Showcase Homepagehttps://biobank.ndph.ox.ac.uk/showcase/index.cgi. 13 4 Methods Real-world clinical settings pose challenges to CVD risk prediction models, requiring them to handle diverse input information in varying formats and to adapt quickly to different health...
1948
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.