Pith. sign in

REVIEW 4 major objections 5 minor 34 references

MDD-LLM: Towards Accuracy Large Language Models for Major Depressive Disorder Diagnosis

T0 review · 4 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Fine-tuning Llama 3.1 on 274,348 UK Biobank records yields an MDD classifier that reaches 0.8919 AUC and beats SVM, XGBoost, and MLP baselines.

desk verdict A useful but sloppy empirical study: the AUC is plausible, but the reported PPV/NPV are mathematically inconsistent with the stated 4.6% prevalence, implying undisclosed test-set rebalancing. read the letter →

arxiv 2505.00032 v1 pith:OCR7MLA4 submitted 2025-04-28 cs.CL cs.AI

Yuyang Sha , Hongxin Pan , Wei Xu , Weiyu Meng , Gang Luo , Xinyu Du , Xiaobing Zhai , Henry H. Y. Tong
show 2 more authors
Caijuan Shi Kefeng Li
This is my paper · ORCID
classification cs.CLcs.AI
keywords majordepressivedisorderlargelanguagemodelsLLMfine-tuningLoRAUKBiobanktabulardatatransformationmedicaldiagnosisinterpretability
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

MDD-LLM is a proposal to diagnose major depressive disorder by fine-tuning a general-purpose large language model on routine tabular health records that have been converted into plain-language prompts. Using 274,348 UK Biobank participants (12,715 with MDD), the fine-tuned 70B model reaches accuracy 0.8378 and area-under-the-curve (AUC) 0.8919 (95% CI 0.8799–0.9040), which the authors report as better than SVM, random forest, LightGBM, XGBoost, CatBoost, MLP, and ResNet1D trained on the same data. The paper also argues that the LLM approach tolerates missing features better than the classical models and can explain its predictions in dialogue, addressing two known barriers to deploying MDD classifiers in practice. The motivating problem is that current diagnosis relies on symptom scales requiring patient cooperation, and existing data-driven models are hard to interpret. If the result holds, it would provide a scalable, text-based path for depression diagnosis using data already present in biobank-style records.

What carries the argument

The load-bearing machinery is the combination of three components: (1) a tabular-data-to-prompt transformation that turns each participant's fields into sentences such as "Age is 60, sex is female, ..."; (2) supervised fine-tuning of Llama 3.1 with LoRA, a parameter-efficient method that trains low-rank adapters (rank 8, alpha 16) rather than all model weights; and (3) an instruction that forces the model to answer "Yes" or "No" together with a probability, which is then used as a risk score for AUC evaluation. The language-model cross-entropy loss is the training objective, and the probability of the predicted token sequence, normalized across classes, supplies the confidence score. This design lets the model exploit pretrained linguistic and medical prior knowledge and keeps fine-tuning feasible on four H100 GPUs.

What would settle it

Apply MDD-LLM and the best classical baseline (e.g., XGBoost) to a held-out cohort where MDD status is confirmed by structured clinical interview instead of treatment codes or self-report, and compare their AUCs on the same tabular fields. If the LLM's advantage shrinks or disappears on that cohort, the reported superiority is an artefact of the UK Biobank label rather than a general diagnostic ability.

Watch

Extended reading notes

Core claim

The paper's central claim is that a parameter-efficiently fine-tuned LLM, given only text renderings of tabular health-record fields, is a superior MDD classifier to the classical machine and deep learning models usually applied to such data. MDD-LLM (70B) obtains ACC 0.8378, F1 0.8184, and AUC 0.8919 (95% CI 0.8799–0.9040); MDD-LLM (8B) obtains ACC 0.7904, F1 0.7627, and AUC 0.8566, both outperforming all tested baselines. The authors additionally report that LoRA fine-tuning is slightly more accurate and faster than QLoRA (40 vs 55 minutes) at the cost of more GPU memory; that Text Template and GPT-generated prompts outperform a List Template; and that keeping only 40% of features reduces accuracy by 9.24% and F1 by 9.50%, whereas XGBoost and MLP degrade much more sharply. The interpretability experiments show the model can state reasons for a yes/no prediction in free text. The conclusion positions MDD-LLM as the first LLM-based solution fine-tuned on an extensive real-world dataset for MDD diagnosis.

Load-bearing premise

The load-bearing premise is that the UK Biobank label for MDD, built from ICD-10 code F32 plus self-reported conditions in fields 20433 and 20434, is a true ground truth for major depressive disorder; if that label mostly captures a single depressive episode or help-seeking behaviour, the model's high accuracy partly measures consultation behaviour rather than the disease itself.

Editorial extensions

If this is right

  • Depression diagnosis could be run directly on routine electronic health-record fields, without requiring a patient to complete PHQ-9 or similar scales during the diagnostic step.
  • Because the model keeps much of its accuracy when 60% of features are missing, LLM-based diagnosis is more usable with incomplete clinical records than gradient-boosted trees or MLPs.
  • A clinician-facing tool could ask the model to justify each prediction in natural language, turning a binary classifier into an auditable dialogue.
  • Scaling the base model from 8B to 70B parameters raises accuracy by about 4.7 percentage points and AUC by about 3.5 points, suggesting that larger foundation models are a direct path to better MDD diagnosis.
  • Fine-tuning with LoRA on the 8B model completes in 40 minutes on four H100 GPUs, so the approach is reproducible at moderate hardware cost.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The UK Biobank MDD label is based on ICD-10 F32 codes plus self-reported conditions, so part of the AUC may reflect predicting who sought help or received a depression code, not necessarily a stable MDD phenotype; validation on interview-confirmed cases would settle this.
  • The same prompt-conversion and LoRA fine-tuning recipe could be applied to other ICD-10-defined psychiatric conditions in biobank-scale cohorts, though there is no evidence yet that the accuracy gain transfers.
  • Because the model produces free-text explanations, one could audit the features it cites and compare them with epidemiological risk factors, which might expose label bias or spurious correlations.
  • The baseline comparisons are against classical models trained from scratch on the same tabular fields; a fairer decomposition of the gain would compare MDD-LLM with an LLM of similar size fine-tuned on the same prompts, isolating the contribution of the base model's pretraining.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes MDD-LLM, a system that converts UK Biobank tabular data into text prompts and fine-tunes Llama 3.1 8B and 70B models with LoRA to predict major depressive disorder. The authors report that MDD-LLM 70B achieves accuracy 0.8378 and AUC 0.8919 (95% CI: 0.8799–0.9040), outperforming classical machine learning and deep learning baselines on a test set derived from an 80/20 patient-level split. They also compare prompt templates, fine-tuning methods, and robustness to missing features, and they provide interpretability examples. The central claim is that this is the first LLM-based solution fine-tuned on an extensive real-world dataset for MDD diagnosis.

Significance. If the performance claims held, the paper would provide a useful demonstration that LLMs can be applied to tabular clinical data for depression screening, with strengths including the large UK Biobank sample, public code release, exploration of prompt-format effects, and demonstration of robustness to missing features. The interpretability examples are a valuable direction for clinical adoption. However, the central performance claim is currently not trustworthy: the reported PPV/NPV values are internally inconsistent with the stated naturalistic prevalence, and the final model configuration was selected using the same test set on which performance is reported. The paper also overstates the accuracy improvement by not comparing to a trivial majority-class baseline. These issues must be resolved before the significance of the empirical results can be assessed.

major comments (4)
  1. [Section 2.4 / Table 3]
  2. [Sections 3.3 and 3.4]
  3. [Section 3.2 / Table 3]
  4. [Section 2.1 / Section 6]
minor comments (5)
  1. [Section 2.4]
  2. [Section 3.2]
  3. [Abstract and Table 3]
  4. [Throughout]
  5. [Section 3.5]

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the held-out evaluation is a genuine out-of-sample prediction; the flagged PPV/NPV and test-set-selection issues are validity concerns, not derivation-circle defects.

full rationale

The paper's derivation chain is a conventional supervised-learning pipeline. The MDD label (Section 2.1, ICD-10 F32 plus self-reported fields 20433/20434) is an independent ground truth; the tabular features are converted into prompts (Section 2.2); the model is fine-tuned with a token-level language-modeling loss (Eq. 2); and performance is measured on a 20% held-out split by patient ID (Section 2.4). Nothing in the paper defines the target label in terms of the model's output, and no fitted parameter is later renamed as a prediction. The self-citations (e.g., Refs. 8, 13, 24-26, 29) are background context and are not load-bearing for the central accuracy claim. Two reviewer concerns should be recorded as non-circular validity risks rather than circularity: (i) Sections 3.3 and 3.4 select LoRA and the Text Template by comparing accuracy on what appears to be the same held-out test set, which can optimistically bias the reported Table 3 numbers; and (ii) the PPV/NPV values in Table 3 imply a test-set case mix of roughly 43.5%, inconsistent with the stated 4.63% prevalence under the described 80/20 split, suggesting undisclosed rebalancing. Both are serious reproducibility and calibration concerns, but neither makes the model's label predictions equivalent to its inputs by construction, so the circularity score is 0.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claim rests on a standard supervised learning pipeline. The free parameters are hyperparameters, not fitted constants in a scientific derivation. The axioms concern label validity, representativeness, and the information-preserving nature of text conversion. No new physical or conceptual entities are introduced.

free parameters (5)
  • LoRA rank = 8
    Set by the authors without a sweep; affects the capacity of the low-rank adaptation.
  • LoRA alpha = 16
    Scaling factor for LoRA updates, chosen by hand.
  • Peak learning rate = 0.0003
    Used with linear warm-up and decay; not tuned across values.
  • Training epochs = 5
    Fixed at 5 with no early stopping or epoch comparison.
  • Global batch size = 192
    Selected for the available hardware; not varied.
assumptions (4)
  • domain assumption UK Biobank participants are representative of the target population for MDD screening.
    The model is trained and evaluated exclusively on UK Biobank volunteers aged 40-69, which may not generalize to younger or non-volunteer populations.
  • domain assumption ICD-10 code F32 and self-reported fields 20433/20434 correctly identify true MDD status.
    Section 2.1 defines the phenotype this way; if these labels are noisy or capture single episodes, the model's predictions are trained on weak ground truth.
  • ad hoc to paper Converting tabular features into text preserves all information relevant to the classification task.
    The text templates drop the numerical precision or ordering implicitly present in the original table, and the GPT generation template may introduce paraphrasing noise.
  • domain assumption LLM token probabilities normalized across 'Yes' and 'No' can be used as a risk score comparable to classifier probability estimates.
    The paper computes AUC from these probabilities, but no calibration analysis is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MDD-LLM: Towards Accuracy Large Language Models for Major Depressive Disorder Diagnosis." pith.science (2026). https://pith.science/paper/OCR7MLA4

@misc{pith2026250500032,
  author       = {Pith},
  title        = {Pith review of: MDD-LLM: Towards Accuracy Large Language Models for Major Depressive Disorder Diagnosis},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OCR7MLA4}},
  note         = {Machine review of arXiv:2505.00032}
}
read the original abstract

Major depressive disorder (MDD) impacts more than 300 million people worldwide, highlighting a significant public health issue. However, the uneven distribution of medical resources and the complexity of diagnostic methods have resulted in inadequate attention to this disorder in numerous countries and regions. This paper introduces a high-performance MDD diagnosis tool named MDD-LLM, an AI-driven framework that utilizes fine-tuned large language models (LLMs) and extensive real-world samples to tackle challenges in MDD diagnosis. Therefore, we select 274,348 individual information from the UK Biobank cohort to train and evaluate the proposed method. Specifically, we select 274,348 individual records from the UK Biobank cohort and design a tabular data transformation method to create a large corpus for training and evaluating the proposed approach. To illustrate the advantages of MDD-LLM, we perform comprehensive experiments and provide several comparative analyses against existing model-based solutions across multiple evaluation metrics. Experimental results show that MDD-LLM (70B) achieves an accuracy of 0.8378 and an AUC of 0.8919 (95% CI: 0.8799 - 0.9040), significantly outperforming existing machine learning and deep learning frameworks for MDD diagnosis. Given the limited exploration of LLMs in MDD diagnosis, we examine numerous factors that may influence the performance of our proposed method, such as tabular data transformation techniques and different fine-tuning strategies.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

34 extracted references · 34 canonical work pages

  1. [1]

    Comorbid depression in medical diseases

    Sm G, O K-F, R M-M, A M, Jj M, M B, et al. Comorbid depression in medical diseases. Nature reviews Disease primers. 2020;6

    work page 2020

  2. [3]

    Major depressive disorder: hypothesis, mechanism, prevention and treatment

    Cui L, Li S, Wang S, Wu X, Liu Y , Y u W, et al. Major depressive disorder: hypothesis, mechanism, prevention and treatment. Signal Transduct Target Ther. 2024;9:30

    work page 2024

  3. [4]

    Despite the substantial resources allocated to researching the causes, diagnosis, and treatment of MDD, progress in these areas remains below expectations

    forecast that by around 2030, MDD is expected to become the leading health condition worldwide. Despite the substantial resources allocated to researching the causes, diagnosis, and treatment of MDD, progress in these areas remains below expectations. In certain regions, individuals with MDD may face cultural and economic barriers that limit their access ...

    work page

  4. [5]

    Combined prevention for substance use, depression, and anxiety in adolescence: a cluster-randomised controlled trial of a digital online intervention

    Teesson M, Newton NC, Slade T, Chapman C, Birrell L, Mewton L, et al. Combined prevention for substance use, depression, and anxiety in adolescence: a cluster-randomised controlled trial of a digital online intervention. Lancet Digit Health. 2020;2:e74–84

    work page 2020

  5. [6]

    To our knowledge, the proposed MDD-LLM is the first LLM-based solution fine-tuned on an extensive real-world dataset for MDD diagnosis

    Conclusion In this study, we develop an MDD diagnosis solution based on LLMs and large-scale real-world samples, named MDD-LLM. To our knowledge, the proposed MDD-LLM is the first LLM-based solution fine-tuned on an extensive real-world dataset for MDD diagnosis. In order to demonstrate the model's performance, we conduct comprehensive comparisons with ex...

    work page 2023

  6. [7]

    Burden of disease scenarios for 204 countries and territories, 2022-2050: a forecasting analysis for the Global Burden of Disease Study

    GBD 2021 Forecasting Collaborators. Burden of disease scenarios for 204 countries and territories, 2022-2050: a forecasting analysis for the Global Burden of Disease Study

    work page 2021

  7. [8]

    Bootstrap inference and machine learning reveal core differential plasma metabolic connectome signatures in major depressive disorder

    Pan H, Sha Y , Zhai X, Luo G, Xu W, Meng W, et al. Bootstrap inference and machine learning reveal core differential plasma metabolic connectome signatures in major depressive disorder. J Affect Disord. 2025;378:281–92

    work page 2025

  8. [9]

    Prospective biomarkers of major depressive disorder: a systematic review and meta-analysis

    Kennis M, Gerritsen L, van Dalen M, Williams A, Cuijpers P , Bockting C. Prospective biomarkers of major depressive disorder: a systematic review and meta-analysis. Mol Psychiatry. 2020;25:321–38

    work page 2020

Show all 34 references
  1. [10]

    Predicting treatment response using EEG in major depressive disorder: A machine-learning meta-analysis

    Watts D, Pulice RF, Reilly J, Brunoni AR, Kapczinski F, Passos IC. Predicting treatment response using EEG in major depressive disorder: A machine-learning meta-analysis. Transl Psychiatry. 2022;12:332

    2022

  2. [12]

    Magnetic resonance imaging for individual prediction of treatment response in major depressive disorder: a systematic review and meta-analysis

    Cohen SE, Zantvoord JB, Wezenberg BN, Bockting CLH, van Wingen GA. Magnetic resonance imaging for individual prediction of treatment response in major depressive disorder: a systematic review and meta-analysis. Transl Psychiatry. 2021;11:168

    2021

  3. [13]

    Automated accurate detection of depression using twin pascal’s triangles lattice pattern with eeg signals

    Tasci G, Loh HW, Barua PD, Baygin M, Tasci B, Dogan S, et al. Automated accurate detection of depression using twin pascal’s triangles lattice pattern with eeg signals. Knowledge-Based Systems. 2023;260:110190

    2023

  4. [14]

    A textual-based featuring approach for depression detection using machine learning classifiers and social media texts

    Chiong R, Budhi GS, Dhakal S, Chiong F. A textual-based featuring approach for depression detection using machine learning classifiers and social media texts. Computers in Biology and Medicine. 2021;135:104499

    2021

  5. [15]

    Metabolic Connectome and Its Role in the Prediction, Diagnosis, and Treatment of Complex Diseases

    Meng W, Pan H, Sha Y , Zhai X, Xing A, Lingampelly SS, et al. Metabolic Connectome and Its Role in the Prediction, Diagnosis, and Treatment of Complex Diseases. Metabolites. 2024;14:93

    2024

  6. [16]

    Longitudinal pathways between childhood BMI, body dissatisfaction, and adolescent depression: an observational study using the UK Millenium Cohort Study

    Blundell E, De Stavola BL, Kellock MD, Kelly Y , Lewis G, McMunn A, et al. Longitudinal pathways between childhood BMI, body dissatisfaction, and adolescent depression: an observational study using the UK Millenium Cohort Study. Lancet Psychiatry. 2024;11:47–55

    2024

  7. [17]

    Sympathetic and blood pressure reactivity in young adults with major depressive disorder

    Darling AM, Y oung BE, Skow RJ, Dominguez CM, Saunders EFH, Fadel PJ, et al. Sympathetic and blood pressure reactivity in young adults with major depressive disorder. J Affect Disord. 2024;361:322–32

    2024

  8. [18]

    Role of age, gender and marital status in prognosis for adults with depression: An individual patient data meta-analysis

    Buckman JEJ, Saunders R, Stott J, Arundell L-L, O’Driscoll C, Davies MR, et al. Role of age, gender and marital status in prognosis for adults with depression: An individual patient data meta-analysis. Epidemiol Psychiatr Sci. 2021;30:e42

    2021

  9. [19]

    Income inequality and depression: a systematic review and meta-analysis of the association and a scoping review of mechanisms

    Patel V , Burns JK, Dhingra M, Tarver L, Kohrt BA, Lund C. Income inequality and depression: a systematic review and meta-analysis of the association and a scoping review of mechanisms. World Psychiatry. 2018;17:76–89

    2018

  10. [20]

    Testing and Evaluation of Health Care Applications of Large Language Models: A Systematic Review

    Bedi S, Liu Y , Orr-Ewing L, Dash D, Koyejo S, Callahan A, et al. Testing and Evaluation of Health Care Applications of Large Language Models: A Systematic Review. JAMA. 2025;333:319–28

    2025

  11. [21]

    Multimodal machine learning enables AI chatbot to diagnose ophthalmic diseases and provide high-quality medical responses

    Ma R, Cheng Q, Y ao J, Peng Z, Y an M, Lu J, et al. Multimodal machine learning enables AI chatbot to diagnose ophthalmic diseases and provide high-quality medical responses. NPJ Digit Med. 2025;8:64

    2025

  12. [22]

    Foundation models for generalist medical artificial intelligence

    Moor M, Banerjee O, Abad ZSH, Krumholz HM, Leskovec J, Topol EJ, et al. Foundation models for generalist medical artificial intelligence. Nature. 2023;616:259–65

    2023

  13. [24]

    MetDIT: transforming and analyzing clinical metabolomics data with convolutional neural networks

    Sha Y , Meng W, Luo G, Zhai X, Tong HHY , Wang Y , et al. MetDIT: transforming and analyzing clinical metabolomics data with convolutional neural networks. Anal Chem. 2024;96:2949–57

    2024

  14. [25]

    A novel lightweight deep learning fall detection system based on global-local attention and channel feature augmentation

    Sha Y , Zhai X, Li J, Meng W, Tong HH, Li K. A novel lightweight deep learning fall detection system based on global-local attention and channel feature augmentation. Interdisciplinary Nursing Research. 2023;2:68–75

    2023

  15. [26]

    The UK Biobank resource with deep phenotyping and genomic data

    Bycroft C, Freeman C, Petkova D, Band G, Elliott LT, Sharp K, et al. The UK Biobank resource with deep phenotyping and genomic data. Nature. 2018;562:203–9

    2018

  16. [27]

    MedChatZH: A tuning LLM for traditional Chinese medicine consultations

    Tan Y , Zhang Z, Li M, Pan F, Duan H, Huang Z, et al. MedChatZH: A tuning LLM for traditional Chinese medicine consultations. Comput Biol Med. 2024;172:108290

    2024

  17. [28]

    The UK Biobank is a large-scale prospective cohort study that recruited over 500,000 individuals (aged 40-69) between 2006 and

    provides a large amount of medical data, particularly regarding MDD diagnosis, which serves as the primary resource for experimentation and analysis in this article. The UK Biobank is a large-scale prospective cohort study that recruited over 500,000 individuals (aged 40-69) b...

    2006

  18. [29]

    HerbMet: Enhancing metabolomics data analysis for accurate identification of Chinese herbal medicines using deep learning

    Sha Y , Jiang M, Luo G, Meng W, Zhai X, Pan H, et al. HerbMet: Enhancing metabolomics data analysis for accurate identification of Chinese herbal medicines using deep learning. Phytochem Anal. 2025;36:261–72

    2025

  19. [32]

    QLoRA: Efficient Finetuning of Quantized LLMs

    Dettmers T, Pagnoni A, Holtzman A, Zettlemoyer L. QLoRA: Efficient Finetuning of Quantized LLMs. Advances in Neural Information Processing Systems. 2023;36:10088–115

    2023

  20. [36]

    Multiple imputation with multivariate imputation by chained equation (MICE) package

    Zhang Z. Multiple imputation with multivariate imputation by chained equation (MICE) package. Ann Transl Med. 2016;4:30

    2016

  21. [37]

    MissForest--non-parametric missing value imputation for mixed-type data

    Stekhoven DJ, Bühlmann P . MissForest--non-parametric missing value imputation for mixed-type data. Bioinformatics. 2012;28:112–8

    2012

  22. [38]

    M-MDD: A multi-task deep learning framework for major depressive disorder diagnosis using EEG

    Wang Y , Zhao S, Jiang H, Li S, Li T, Pan G. M-MDD: A multi-task deep learning framework for major depressive disorder diagnosis using EEG. Neurocomputing. 2025;636:130008

    2025

  23. [39]

    Harnessing multimodal approaches for depression detection using large language models and facial expressions

    Sadeghi M, Richer R, Egger B, Schindler-Gmelch L, Rupp LH, Rahimi F, et al. Harnessing multimodal approaches for depression detection using large language models and facial expressions. Npj Ment Health Res. 2024;3:66

    2024

  24. [40]

    Collaborative Enhancement of Consistency and Accuracy in US Diagnosis of Thyroid Nodules Using Large Language Models

    Wu S-H, Tong W-J, Li M-D, Hu H-T, Lu X-Z, Huang Z-R, et al. Collaborative Enhancement of Consistency and Accuracy in US Diagnosis of Thyroid Nodules Using Large Language Models. Radiology. 2024;310:e232255

    2024

  25. [41]

    Evaluation of GPT-4 for 10-year cardiovascular risk prediction: Insights from the UK Biobank and KoGES data

    Han C, Kim DW, Kim S, Chan Y ou S, Park JY , Bae S, et al. Evaluation of GPT-4 for 10-year cardiovascular risk prediction: Insights from the UK Biobank and KoGES data. iScience. 2024;27:109022

    2024

  26. [2021]

    2024;403:2204–56

    Lancet. 2024;403:2204–56

    2024

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.