Pith. sign in

REVIEW 2 major objections 5 minor 201 references

Medical specialist models win on diagnosis; general models win on dialogue and decision support, so route queries by task type.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-10 18:49 UTC pith:3WR6X4EQ

load-bearing objection Useful dual-view taxonomy plus a real 18-model stratified table; the specialist-vs-general routing claim is directionally interesting but rests on an unreleased, LLM-reconstructed benchmark. the 2 major comments →

arxiv 2607.07761 v1 pith:3WR6X4EQ submitted 2026-07-08 cs.AI

Aligning Clinical Needs and AI Capabilities: A Survey on LLMs for Medical Reasoning

Qi Peng , Jiatong Li , Sirui Huang , Yiyang Jiang , Kaisong Gong , Ronger Ding , Shijie Ye , Changmeng Zheng
show 5 more authors
Yi Cai Xiaobo Yang Jin Huang Xiao-Yong Wei Qing Li
This is my paper
classification cs.AI
keywords medical LLMsclinical reasoningMiller's Pyramiddeductive inductive abductivemedical benchmarkspecialist vs general modelshallucinationworkflow-ready systems
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This survey argues that medical reasoning with large language models needs to be judged by how well it matches real clinical competence, not by exam scores alone. The authors map clinical work onto a five-level ladder that climbs from term recognition and triage through full diagnosis, personalized recommendations, and multi-turn case management. They also classify model methods by classical reasoning types: deduction from rules, induction from patterns, abduction of likely causes, and mixtures of all three. On a new 5,000-sample benchmark covering those five levels, specialist medical models pull ahead on diagnosis-centric prediction while high-capacity general models lead on decision support, multi-round dialogue, and discharge summarization. Length-of-stay and other structured temporal outcomes remain hard for every system tested. The practical claim is that clinics should route diagnosis-heavy queries to specialist models and hand dialogue and supportive tasks to general models, while the field still has to solve data scarcity, hallucination, and grounding before systems are ready for daily workflows.

Core claim

Across five levels of medical reasoning, medical specialist models excel on diagnosis-centric tasks while high-capacity general models lead on decision support, multi-turn dialogue, and summarization; model size alone does not explain the gap, so a practical system should route diagnosis-heavy queries to specialists and supportive tasks to general models.

What carries the argument

The dual-view taxonomy: a five-level clinical competency ladder extended from Miller's Pyramid (knowledge recognition through dynamic case management) paired with classical reasoning types (deductive, inductive, abductive, mixed), used both to organize existing work and to build a balanced 5,000-sample benchmark that scores 18 models.

Load-bearing premise

The claim rests on the idea that the five-level ladder and the 5,000 curated samples (semi-automatically rebuilt from existing sources) faithfully represent real clinical competence and do not introduce reconstruction artifacts that favor one model family over another.

What would settle it

Re-run the same 18 models on a fully clinician-authored, multi-site hold-out set that keeps the five-level structure but never saw the semi-automated reconstruction pipeline; if specialist models no longer lead diagnosis or general models no longer lead dialogue and decision support, the routing claim fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. This survey proposes a dual-view framework for medical LLM reasoning that maps a five-level extension of Miller’s Pyramid (knowledge recognition → classification/triage → causal diagnosis → decision support → dynamic interaction) onto classical deductive/inductive/abductive/mixed reasoning patterns and common clinical tasks. It organizes datasets and model paradigms (CoT, Long-CoT, search-guided, RAG, multimodal, agentic) under this taxonomy, then introduces a new 5,000-sample five-level benchmark (1,000 per level) constructed by LLM-assisted restructuring of existing sources with 20% double-blind clinician review (κ=0.88). Results on 18 models (Table 5) show medical specialist models stronger on diagnosis-centric tasks and high-capacity general models stronger on decision support, multi-turn dialogue and summarization; failure cases (Table 6) and deployment routing implications are discussed, together with challenges (data scarcity, hallucination, grounding) and future directions.

Significance. If the specialist-vs-general performance pattern holds under independent scrutiny, the paper supplies a practical routing principle for clinical LLM deployment and a reusable competency hierarchy that unifies clinical education language with computational reasoning types. Strengths include the breadth of the literature synthesis (Figs. 2, 5–7), the concrete failure-mode table, reported p-values between model cohorts, and explicit construction details (de-identification, format standardization, IAA). The work is timely for a field that currently lacks shared language between clinicians and model builders.

major comments (2)
  1. §5.4.1 and Table 5: The central empirical claim (specialist models excel on diagnosis-centric tasks; general models on Levels 4–5; therefore route accordingly) rests on a 5,000-sample benchmark that the authors constructed via LLM-assisted feature extraction and input reformatting of existing sources, with only 20% clinician double-blind acceptance review (κ=0.88) and no public release of items, prompts or reconstruction code. Because the same model families later evaluated may have participated in rewriting, reconstruction artifacts (canonicalized phrasing, simplified differentials, format-friendly narratives) can systematically favor models already strong on structured clinical text. Without released items or an independent re-annotation of a substantial held-out subset, the specialist–general gap cannot be verified as genuine clinical competence rather than reconstruction bias. Releas
  2. §2.4.2 and §5.4.2: The five-level hierarchy is presented as an extension of Miller’s Pyramid, yet no external clinical validation (expert panel mapping of real tasks onto levels, inter-rater reliability of level assignment, or ablation showing that Levels 3 vs. 4 are cleanly separable under the reconstruction process) is reported. If the level boundaries are porous under the authors’ own construction pipeline, the reported specialist advantage on “diagnosis-centric” tasks and the general-model advantage on Level 4–5 tasks become confounded. A short validation study or sensitivity analysis that re-assigns a subset of items across adjacent levels would strengthen the taxonomy’s claim to organize the empirical results.
minor comments (5)
  1. Table 5 caption reports independent t-tests between general and medical cohorts (p-values for Levels 1–5) but does not state whether multiple-comparison correction was applied; a note would help interpretation.
  2. Fig. 7 heatmap is dense; a clearer legend or supplementary table listing which models reach which competency levels would improve readability.
  3. §3.2–3.3 and Table 4: several dataset URLs and sample counts appear as footnotes or incomplete entries; a uniform citation style would aid reproducibility.
  4. Occasional typographical inconsistencies (e.g., “Med-Gemma” vs. “MedGemma”, “ClinicalGPT-r1” vs. “ClinicalGPT-R1”) should be normalized.
  5. §6.2.2 chest-pain example is pedagogically useful but would benefit from an explicit link back to the Level 3 abductive-reasoning definition so the illustration is not free-standing.

Circularity Check

0 steps flagged

Survey taxonomy is definitional by design; empirical specialist-vs-general claims rest on a new external-style benchmark whose labels are inherited, not fitted or self-defined.

full rationale

This is a survey paper whose dual-view framework (five-level Miller extension + deductive/inductive/abductive mapping) is an organizational taxonomy, not a first-principles derivation that claims to predict clinical competence from axioms. The load-bearing empirical claim—that medical specialist models excel on diagnosis-centric tasks while general models lead on decision support, dialogue, and summarization—comes from Table 5 and §5.4 analysis of 18 models on a 5 000-sample benchmark the authors constructed. Labels and answers are explicitly inherited from prior gold-standard source datasets; the authors only restructure inputs via LLM-assisted feature extraction and apply a 20 % clinician double-blind acceptance review (κ=0.88). No free parameter is fitted to a subset and then re-presented as a prediction; no equation equates a claimed result to its own definition; self-citations are ordinary background (prior medical LLMs, CoT, RAG, agents) and are not load-bearing uniqueness theorems that force the routing recommendation. The hierarchy itself is definitional (authors define Level 1–5 and then measure models against those definitions), which is normal for a survey taxonomy and does not constitute circular reduction of a prediction to its inputs. Residual concerns about reconstruction artifacts or unreleased items are validity/reproducibility issues, not circularity. Score 1 reflects only the mild definitional character of any author-defined competency ladder.

Axiom & Free-Parameter Ledger

0 free parameters · 3 axioms · 1 invented entities

As a survey-plus-benchmark paper the load-bearing premises are definitional and methodological rather than physical constants. The central empirical claim rests on the validity of the five-level hierarchy, the representativeness of the curated samples, and the fairness of the 18-model selection. No continuous free parameters are fitted; the main invented entity is the hierarchy itself.

axioms (3)
  • domain assumption Miller's Pyramid (Knows / Knows How / Shows How / Does) is an appropriate and complete scaffold for grading LLM clinical competence when extended to five levels.
    Invoked throughout §2.3–2.4 and Fig. 3 as the organizing principle; never independently validated against real clinical outcomes.
  • domain assumption Deductive, inductive and abductive reasoning (Peirce) plus mixed patterns exhaustively cover the reasoning patterns required by medical tasks.
    Used to structure Fig. 2 and §3–4; alternative taxonomies (e.g., causal vs. correlational, System-1/System-2) are not compared.
  • ad hoc to paper The 5,000 samples (1,000 per level) reconstructed from existing sources with 20% clinician review (κ=0.88) are representative of the five competency levels and free of systematic reconstruction bias.
    Stated in §5.4.1; the semi-automated LLM-assisted reconstruction step is a paper-specific methodological choice whose residual error is unquantified beyond the κ score.
invented entities (1)
  • Five-level LLM medical competency hierarchy (Recognition → Classification → Reasoning → Decision → Interact) no independent evidence
    purpose: Provides the dual-view taxonomy that organizes datasets, models and the new benchmark.
    Explicitly introduced in §2.4 and Fig. 3 as an extension of Miller's Pyramid tailored to LLMs; no independent clinical validation outside this paper.

pith-pipeline@v1.1.0-grok45 · 50487 in / 2910 out tokens · 39197 ms · 2026-07-10T18:49:32.778878+00:00 · methodology

0 comments
read the original abstract

Large language models (LLMs) have emerged as important tools in healthcare, showing growing potential for clinical reasoning and patient care. This survey examines recent progress in medical LLMs, focusing on reasoning applications and requirements. We present a dual-view approach that connects clinical practice with computational methods. On the clinical side, we establish a five-level competency scheme following Miller's Pyramid, progressing from knowledge recall to dynamic case management. On the computational side, we link deductive, inductive, and abductive reasoning patterns to common medical goals and tasks. We also introduce a benchmark dataset spanning five levels of medical reasoning capability and report results on 18 state-of-the-art models, revealing that medical specialist models excel in diagnosis-centric tasks while general models lead in decision support and dialogue. We conclude by discussing current progress and open challenges, including data limitations, hallucination, and grounding issues, and outline directions toward safer, more reliable, and workflow-ready systems.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

201 extracted references · 201 canonical work pages · 55 internal anchors

  1. [1]

    Multimedia tools and applications81(18), 25877– 25911 (2022)

    Aljabri, M., AlAmir, M., AlGhamdi, M., Abdel-Mottaleb, M., Collado- Mesa, F.: Towards a better understanding of annotation tools for medical imaging: a survey. Multimedia tools and applications81(18), 25877– 25911 (2022)

    work page 2022

  2. [2]

    Briefings in bioinformatics15(2), 327–340 (2014)

    Neves, M., Leser, U.: A survey on annotation tools for the biomedical literature. Briefings in bioinformatics15(2), 327–340 (2014)

    work page 2014

  3. [3]

    Journal of digital imaging26(6), 1025–1039 (2013)

    Kumar, A., Kim, J., Cai, W., Fulham, M., Feng, D.: Content-based med- ical image retrieval: a survey of applications to multidimensional and multimodality data. Journal of digital imaging26(6), 1025–1039 (2013)

    work page 2013

  4. [4]

    European Journal of Epidemiology40(9), 1143–1159 (2025)

    Lin, Y., Yang, Y., Li, Z., Du, L., Shi, R., Shi, Q., Xu, X., Yin, G., Zhang, F., Huang, W.,et al.: Cohort profile: the west-china hospital alliance longitudinal epidemiology wellness (whale) study. European Journal of Epidemiology40(9), 1143–1159 (2025)

    work page 2025

  5. [5]

    A Survey of Large Language Models

    Zhao, W.X., Zhou, K., Li, J., Tang, T., Wang, X., Hou, Y., Min, Y., Zhang, B., Zhang, J., Dong, Z., et al.: A survey of large language models. arXiv preprint arXiv:2303.182231(2) (2023)

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  6. [6]

    Machine Intelligence Research20(2), 147–179 (2023)

    Qiu, Y., Lin, F., Chen, W., Xu, M.: Pre-training in medical data: A survey. Machine Intelligence Research20(2), 147–179 (2023)

    work page 2023

  7. [7]

    Machine Intelligence Research22(6), 1127–1137 (2025)

    Liang, Y., Yang, E., Guo, G., Cai, W., Jiang, L., Zhao, J., Wang, X.: Answer semantics-enhanced medical visual question answering. Machine Intelligence Research22(6), 1127–1137 (2025)

    work page 2025

  8. [8]

    Medical Multimodal Foundation Models in Clinical Diagnosis and Treatment: Applications, Challenges, and Future Directions

    Sun, K., Xue, S., Sun, F., Sun, H., Luo, Y., Wang, L., Wang, S., Guo, N., Liu, L., Zhao, T., et al.: Medical multimodal foundation models in clinical diagnosis and treatment: Applications, challenges, and future directions. arXiv preprint arXiv:2412.02621 (2024)

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  9. [9]

    Machine Intelligence Research 22(3), 437–451 (2025)

    Shao, H., Zeng, Q., Hou, Q., Yang, J.: Mcanet: Medical image segmenta- tion with multi-scale cross-axis attention. Machine Intelligence Research 22(3), 437–451 (2025)

    work page 2025

  10. [10]

    Machine Intelligence Research22(3), 524–538 (2025) Springer Nature 2021 LATEX template 44Survey on LLMs for Medical Reasoning

    Nie, Q., Zhang, X., Chen, C., Zhang, Z., Hu, Y., Liu, J.: Reparameter- ized multi-scale transformer for deformable retinal image registration. Machine Intelligence Research22(3), 524–538 (2025) Springer Nature 2021 LATEX template 44Survey on LLMs for Medical Reasoning

    work page 2025

  11. [11]

    Diagnostics14(5), 527 (2024)

    Baig, Z., Lawrence, D., Ganhewa, M., Cirillo, N.: Accuracy of treat- ment recommendations by pragmatic evidence search and artificial intelligence: An exploratory study. Diagnostics14(5), 527 (2024)

    work page 2024

  12. [12]

    Machine Intelligence Research, 1–17 (2026)

    Sehar, U., Xiong, J., Zhai, J., Xia, Z.: Automatic orthodontic treat- ment planning using deep learning. Machine Intelligence Research, 1–17 (2026)

    work page 2026

  13. [13]

    Journal of Medical Internet Research26, 58329 (2024)

    Seo, J., Choi, D., Kim, T., Cha, W.C., Kim, M., Yoo, H., Oh, N., Yi, Y., Lee, K.H., Choi, E.: Evaluation framework of large language models in medical documentation: Development and usability study. Journal of Medical Internet Research26, 58329 (2024)

    work page 2024

  14. [14]

    Annals of palliative medicine5(2), 832–892 (2016)

    Chow, R., Chiu, N., Bruera, E., Krishnan, M., Chiu, L., Lam, H., DeAngelis, C., Pulenzas, N., Vuong, S., Chow, E.: Inter-rater reliabil- ity in performance status assessment among health care professionals: a systematic review. Annals of palliative medicine5(2), 832–892 (2016)

    work page 2016

  15. [15]

    Academic medicine65(9), 63–7 (1990)

    Miller, G.E.: The assessment of clinical skills/competence/performance. Academic medicine65(9), 63–7 (1990)

    work page 1990

  16. [16]

    In: Proceedings of the Fourth Social Media Mining for Health Applications (# SMM4H) Workshop & Shared Task, pp

    Belousov, M., Dixon, W.G., Nenadic, G.: Mednorm: A corpus and embeddings for cross-terminology medical concept normalisation. In: Proceedings of the Fourth Social Media Mining for Health Applications (# SMM4H) Workshop & Shared Task, pp. 31–39 (2019)

    work page 2019

  17. [17]

    Ensuring Safety and Trust: Analyzing the Risks of Large Language Models in Medicine

    Yang, Y., Jin, Q., Leaman, R., Liu, X., Xiong, G., Sarfo-Gyamfi, M., Gong, C., Ferri` ere-Steinert, S., Wilbur, W.J., Li, X., et al.: Ensur- ing safety and trust: Analyzing the risks of large language models in medicine. arXiv preprint arXiv:2411.14487 (2024)

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  18. [18]

    International journal of environmental research and public health20(4), 3378 (2023)

    Hirosawa, T., Harada, Y., Yokose, M., Sakamoto, T., Kawamura, R., Shimizu, T.: Diagnostic accuracy of differential-diagnosis lists generated by generative pretrained transformer 3 chatbot for clinical vignettes with common chief complaints: a pilot study. International journal of environmental research and public health20(4), 3378 (2023)

    work page 2023

  19. [19]

    PMC-VQA: Visual Instruction Tuning for Medical Visual Question Answering

    Zhang, X., Wu, C., Zhao, Z., Lin, W., Zhang, Y., Wang, Y., Xie, W.: Pmc-vqa: Visual instruction tuning for medical visual question answering. arXiv preprint arXiv:2305.10415 (2023)

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  20. [20]

    Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Models

    Chen, Q., Qin, L., Liu, J., Peng, D., Guan, J., Wang, P., Hu, M., Zhou, Y., Gao, T., Che, W.: Towards reasoning era: A survey of long chain-of-thought for reasoning large language models. arXiv preprint arXiv:2503.09567 (2025)

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  21. [21]

    Zhang, D., Wu, J., Lei, J., Che, T., Li, J., Xie, T., Huang, X., Zhang, Springer Nature 2021 LATEX template Survey on LLMs for Medical Reasoning45 S., Pavone, M., Li, Y.,et al.: Llama-berry: Pairwise optimization for olympiad-level mathematical reasoning via o1-like monte carlo tree search. In: Proceedings of the 2025 Conference of the Nations of the Amer...

    work page 2021

  22. [22]

    Machine Learn- ing and Knowledge Extraction6(4), 2355–2374 (2024)

    Bora, A., Cuay´ ahuitl, H.: Systematic analysis of retrieval-augmented generation-based llms for medical chatbot applications. Machine Learn- ing and Knowledge Extraction6(4), 2355–2374 (2024)

    work page 2024

  23. [23]

    AUTOCT: Automating Interpretable Clinical Trial Prediction with LLM Agents

    Liu, F., Wang, H., Cho, J., Roth, D., Lo, A.W.: Autoct: Automating interpretable clinical trial prediction with llm agents. arXiv preprint arXiv:2506.04293 (2025)

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  24. [24]

    In: Proceedings of the 32nd ACM International Conference on Multimedia, pp

    Zheng, C., Liang, D., Zhang, W., Wei, X.-Y., Chua, T.-S., Li, Q.: A picture is worth a graph: A blueprint debate paradigm for multimodal reasoning. In: Proceedings of the 32nd ACM International Conference on Multimedia, pp. 419–428 (2024)

    work page 2024

  25. [25]

    Nature Reviews Cancer22(2), 114–126 (2022)

    Boehm, K.M., Khosravi, P., Vanguri, R., Gao, J., Shah, S.P.: Harness- ing multimodal data integration to advance precision oncology. Nature Reviews Cancer22(2), 114–126 (2022)

    work page 2022

  26. [26]

    IEEE Transactions on Circuits and Systems for Video Technology34(1), 561–575 (2023)

    Feng, J., Wang, G., Zheng, C., Cai, Y., Fu, Z., Wang, Y., Wei, X.- Y., Li, Q.: Towards bridged vision and language: Learning cross-modal knowledge representation for relation extraction. IEEE Transactions on Circuits and Systems for Video Technology34(1), 561–575 (2023)

    work page 2023

  27. [27]

    Advances in neural information processing systems33, 1877–1901 (2020)

    Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J.D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A.,et al.: Language models are few-shot learners. Advances in neural information processing systems33, 1877–1901 (2020)

    work page 1901

  28. [28]

    Journal of Machine Learning Research24(240), 1–113 (2023)

    Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., Barham, P., Chung, H.W., Sutton, C., Gehrmann, S.,et al.: Palm: Scaling language modeling with pathways. Journal of Machine Learning Research24(240), 1–113 (2023)

    work page 2023

  29. [29]

    GPT-4 Technical Report

    Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F.L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al.: Gpt-4 technical report. arXiv preprint arXiv:2303.08774 (2023)

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  30. [30]

    ClinicalBERT: Modeling Clinical Notes and Predicting Hospital Readmission

    Huang, K., Altosaar, J., Ranganath, R.: Clinicalbert: Modeling clinical notes and predicting hospital readmission. arXiv preprint arXiv:1904.05342 (2019) Springer Nature 2021 LATEX template 46Survey on LLMs for Medical Reasoning

    work page internal anchor Pith review Pith/arXiv arXiv 1904

  31. [31]

    Bioinformatics36(4), 1234–1240 (2020)

    Lee, J., Yoon, W., Kim, S., Kim, D., Kim, S., So, C.H., Kang, J.: Biobert: a pre-trained biomedical language representation model for biomedical text mining. Bioinformatics36(4), 1234–1240 (2020)

    work page 2020

  32. [32]

    Nature620(7972), 172–180 (2023)

    Singhal, K., Azizi, S., Tu, T., Mahdavi, S.S., Wei, J., Chung, H.W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S.,et al.: Large language models encode clinical knowledge. Nature620(7972), 172–180 (2023)

    work page 2023

  33. [33]

    Capabilities of Gemini Models in Medicine

    Saab, K., Tu, T., Weng, W.-H., Tanno, R., Stutz, D., Wulczyn, E., Zhang, F., Strother, T., Park, C., Vedadi, E., et al.: Capabilities of gemini models in medicine. arXiv preprint arXiv:2404.18416 (2024)

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  34. [34]

    ACM Transactions on Computing for Healthcare (HEALTH)3(1), 1–23 (2021)

    Gu, Y., Tinn, R., Cheng, H., Lucas, M., Usuyama, N., Liu, X., Naumann, T., Gao, J., Poon, H.: Domain-specific language model pre- training for biomedical natural language processing. ACM Transactions on Computing for Healthcare (HEALTH)3(1), 1–23 (2021)

    work page 2021

  35. [35]

    Briefings in bioinformatics23(6), 409 (2022)

    Luo, R., Sun, L., Xia, Y., Qin, T., Zhang, S., Poon, H., Liu, T.-Y.: Biogpt: generative pre-trained transformer for biomedical text generation and mining. Briefings in bioinformatics23(6), 409 (2022)

    work page 2022

  36. [36]

    ClinicalGPT: Large Language Models Finetuned with Diverse Medical Data and Comprehensive Evaluation

    Wang, G., Yang, G., Du, Z., Fan, L., Li, X.: Clinicalgpt: large lan- guage models finetuned with diverse medical data and comprehensive evaluation. arXiv preprint arXiv:2306.09968 (2023)

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  37. [37]

    Finetuned Language Models Are Zero-Shot Learners

    Wei, J., Bosma, M., Zhao, V.Y., Guu, K., Yu, A.W., Lester, B., Du, N., Dai, A.M., Le, Q.V.: Finetuned language models are zero-shot learners. arXiv preprint arXiv:2109.01652 (2021)

    work page internal anchor Pith review Pith/arXiv arXiv 2021

  38. [38]

    Advances in neural information processing systems35, 24824–24837 (2022)

    Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F., Chi, E., Le, Q.V., Zhou, D.,et al.: Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems35, 24824–24837 (2022)

    work page 2022

  39. [39]

    Schechter, J.: Deductive reasoning (2009)

    work page 2009

  40. [40]

    Abduction and induction: Essays on their relation and integration, 1–27 (2000)

    Flach, P.A., Kakas, A.C.: Abductive and inductive reasoning: back- ground and issues. Abduction and induction: Essays on their relation and integration, 1–27 (2000)

    work page 2000

  41. [41]

    Karimi, S., Metke Jimenez, A., Kemp, M., Wang, C.: CADEC. v3. CSIRO (2015). https://doi.org/10.4225/08/570FB102BDAD2. https:// doi.org/10.4225/08/570FB102BDAD2

    work page doi:10.4225/08/570fb102bdad2 2015

  42. [42]

    Nucleic acids research32(suppl 1), 267–270 (2004) Springer Nature 2021 LATEX template Survey on LLMs for Medical Reasoning47

    Bodenreider, O.: The unified medical language system (umls): integrat- ing biomedical terminology. Nucleic acids research32(suppl 1), 267–270 (2004) Springer Nature 2021 LATEX template Survey on LLMs for Medical Reasoning47

    work page 2004

  43. [43]

    In: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp

    Zhang, N., Chen, M., Bi, Z., Liang, X., Li, L., Shang, X., Yin, K., Tan, C., Xu, J., Huang, F.,et al.: Cblue: A chinese biomedical language understanding evaluation benchmark. In: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 7888–7915 (2022)

    work page 2022

  44. [44]

    Journal of biomedical informatics47, 1–10 (2014)

    Do˘ gan, R.I., Leaman, R., Lu, Z.: Ncbi disease corpus: a resource for dis- ease name recognition and concept normalization. Journal of biomedical informatics47, 1–10 (2014)

    work page 2014

  45. [45]

    In: Proceedings of the 3rd Clinical Natural Language Processing Workshop, pp

    Wen, Z., Lu, X.H., Reddy, S.: Medal: Medical abbreviation disam- biguation dataset for natural language understanding pretraining. In: Proceedings of the 3rd Clinical Natural Language Processing Workshop, pp. 130–135 (2020)

    work page 2020

  46. [46]

    Retrieved from the University Digital Conservancy

    Moon, S., Pakhomov, S., Melton, G.: Clinical abbreviation sense inven- tory (2012). Retrieved from the University Digital Conservancy

    work page 2012

  47. [47]

    Large Language Models are Few-Shot Clinical Information Extractors

    Agrawal, M., Hegselmann, S., Lang, H., Kim, Y., Sontag, D.: Large lan- guage models are few-shot clinical information extractors. arXiv preprint arXiv:2205.12689 (2022)

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  48. [48]

    Journal of the American Medical Informatics Association23(3), 596–600 (2016)

    Kass-Hout, T.A., Xu, Z., Mohebbi, M., Nelsen, H., Baker, A., Levine, J., Johanson, E., Bright, R.A.: Openfda: an innovative platform provid- ing access to a wealth of fda’s publicly available data. Journal of the American Medical Informatics Association23(3), 596–600 (2016)

    work page 2016

  49. [49]

    In: CCF International Conference on Natural Language Processing and Chinese Computing, pp

    Liu, W., Tang, J., Cheng, Y., Li, W., Zheng, Y., Liang, X.: Meddg: an entity-centric medical consultation dataset for entity-aware medical dia- logue generation. In: CCF International Conference on Natural Language Processing and Chinese Computing, pp. 447–459 (2022). Springer

    work page 2022

  50. [50]

    BMC Medical Informatics and Decision Making20(Suppl 3), 125 (2020)

    Chen, N., Su, X., Liu, T., Hao, Q., Wei, M.: A benchmark dataset and case study for chinese medical question intent classification. BMC Medical Informatics and Decision Making20(Suppl 3), 125 (2020)

    work page 2020

  51. [51]

    Advances in neural information processing systems35, 31306–31318 (2022)

    Fansi Tchango, A., Goel, R., Wen, Z., Martel, J., Ghosn, J.: Ddxplus: A new dataset for automatic medical diagnosis. Advances in neural information processing systems35, 31306–31318 (2022)

    work page 2022

  52. [52]

    Chang J.: MIMIC-IV-Ext-DiReCT

    Wang B., .Q.Y. Chang J.: MIMIC-IV-Ext-DiReCT. (version 1.0.0). Phy- sioNet. RRID:SCR-007345. (2025). https://doi.org/10.13026/yf96-kc87

    work page doi:10.13026/yf96-kc87 2025

  53. [53]

    Nightingale Open Science https://doi

    Kansal, A., Chen, E., Jin, B., Rajpurkar, P., Kim, D.: Multimodal clinical monitoring in the emergency department (mc-med). Nightingale Open Science https://doi. org/10.48815/N57P4G (2025) Springer Nature 2021 LATEX template 48Survey on LLMs for Medical Reasoning

    work page doi:10.48815/n57p4g 2025

  54. [54]

    (version 1.0.0)

    Shen Q., .G.Q.: MIMIC-IV-Ext Triage Instruction Corpus. (version 1.0.0). PhysioNet. RRID:SCR-007345. (2025). https://doi.org/10.13026/ q1nc-2e47

    work page 2025

  55. [55]

    UCI machine learning repository10, 52–4 (1988)

    Janosi, A., Steinbrunn, W., Pfisterer, M., Detrano, R.: Heart disease. UCI machine learning repository10, 52–4 (1988)

    work page 1988

  56. [56]

    ´A.A.: Covid data for shared learning (cdsl): A comprehensive, multimodal covid-19 dataset from hm hospitales (2024)

    Ritor´ e,´A., Oprescu, A.M., Bronchalo, A.E., de la Hoz, M. ´A.A.: Covid data for shared learning (cdsl): A comprehensive, multimodal covid-19 dataset from hm hospitales (2024). https://doi.org/10.13026/1176-6c44. (version 1.0.0). PhysioNet. RRID:SCR-007345

    work page doi:10.13026/1176-6c44 2024

  57. [57]

    In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp

    Wang, X., Peng, Y., Lu, L., Lu, Z., Bagheri, M., Summers, R.M.: Chestx- ray8: Hospital-scale chest x-ray database and benchmarks on weakly- supervised classification and localization of common thorax diseases. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2097–2106 (2017)

    work page 2097

  58. [58]

    In: 2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI), pp

    Liu, B., Zhan, L.-M., Xu, L., Ma, L., Yang, Y., Wu, X.-M.: Slake: A semantically-labeled knowledge-enhanced dataset for medical visual question answering. In: 2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI), pp. 1650–1654 (2021). IEEE

    work page 2021

  59. [59]

    Scientific data5(1), 1–10 (2018)

    Lau, J.J., Gayen, S., Ben Abacha, A., Demner-Fushman, D.: A dataset of clinically generated visual questions and answers about radiology images. Scientific data5(1), 1–10 (2018)

    work page 2018

  60. [60]

    PathVQA: 30000+ Questions for Medical Visual Question Answering

    He, X., Zhang, Y., Mou, L., Xing, E., Xie, P.: Pathvqa: 30000+ questions for medical visual question answering. arXiv preprint arXiv:2003.10286 (2020)

    work page internal anchor Pith review Pith/arXiv arXiv 2003

  61. [61]

    https://physionet.org/content/mitdb/1.0.0/

    Moody GB, M.R.: The impact of the MIT-BIH Arrhythmia Database. https://physionet.org/content/mitdb/1.0.0/. PMID: 11446209 (2001)

    work page 2001

  62. [62]

    Raghavan, P., Liang, J.J., Mahajan, D., Chandra, R., Szolovits, P.: emrkbqa: A clinical knowledge-base question answering dataset. (2021). Association for Computational Linguistics

    work page 2021

  63. [63]

    Jin, C., Zhang, M., Ma, X., Yujiao, L., Wang, Y., Jia, Y., Du, Y., Sun, T., Wang, H., Fan, C., Gu, J., Chi, C., Lv, X., Li, F., Xue, W., Huang, Y.: RJUA-MedDQA: A Multimodal Benchmark for Medical Document Question Answering and Clinical Reasoning (2024)

    work page 2024

  64. [64]

    In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp

    Pampari, A., Raghavan, P., Liang, J., Peng, J.: emrqa: A large corpus for question answering on electronic medical records. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 2357–2368 (2018) Springer Nature 2021 LATEX template Survey on LLMs for Medical Reasoning49

    work page 2018

  65. [65]

    MedicalAgentsBench for Complex Medical Reasoning: Comparing Internalized Reasoning Models versus Externalized Agent-based Frameworks

    Tang, X., Shao, D., Sohn, J., Chen, J., Zhang, J., Xiang, J., Wu, F., Zhao, Y., Wu, C., Shi, W., et al.: Medagentsbench: Benchmarking think- ing models and agent frameworks for complex medical reasoning. arXiv preprint arXiv:2503.07459 (2025)

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  66. [66]

    arXiv preprint arXiv:2506.09513 (2025)

    Sun, Y., Qian, X., Xu, W., Zhang, H., Xiao, C., Li, L., Rong, Y., Huang, W., Bai, Q., Xu, T.: Reasonmed: A 370k multi-agent generated dataset for advancing medical reasoning. arXiv preprint arXiv:2506.09513 (2025)

    work page arXiv 2025

  67. [67]

    Nucleic acids research52(D1), 1265–1275 (2024)

    Knox, C., Wilson, M., Klinger, C.M., Franklin, M., Oler, E., Wilson, A., Pon, A., Cox, J., Chin, N.E., Strawbridge, S.A.,et al.: Drugbank 6.0: the drugbank knowledgebase for 2024. Nucleic acids research52(D1), 1265–1275 (2024)

    work page 2024

  68. [68]

    Clinical Pharmacology & Therapeutics110(3), 563–572 (2021)

    Whirl-Carrillo, M., Huddart, R., Gong, L., Sangkuhl, K., Thorn, C.F., Whaley, R., Klein, T.E.: An evidence-based framework for evaluat- ing pharmacogenomics knowledge for personalized medicine. Clinical Pharmacology & Therapeutics110(3), 563–572 (2021)

    work page 2021

  69. [69]

    Journal of the American Medical Informatics Association18(4), 441–448 (2011)

    Nelson, S.J., Zeng, K., Kilbourne, J., Powell, T., Moore, R.: Normalized names for clinical drugs: Rxnorm at 6 years. Journal of the American Medical Informatics Association18(4), 441–448 (2011)

    work page 2011

  70. [70]

    UCI Machine Learning Repository

    Hong, Z.Q., Yang, J.Y.: Lung Cancer. UCI Machine Learning Repository. DOI: https://doi.org/10.24432/C57596 (1991)

    work page doi:10.24432/c57596 1991

  71. [71]

    UCI Machine Learning Repository10, 5230 (2014)

    Clore, J., Cios, K., DeShazo, J., Strack, B.: Diabetes 130-us hospitals for years 1999-2008. UCI Machine Learning Repository10, 5230 (2014)

    work page 1999

  72. [72]

    Mullenbach, J., Pruksachatkun, Y., Adler, S., Seale, J., Swartz, J., McK- elvey, G., Dai, H., Yang, Y., Sontag, D.: Clip: A dataset for extracting action items for physicians from hospital discharge notes. In: Proceed- ings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural La...

    work page 2021

  73. [73]

    BMC Bioinform.20(1), 511–151123 (2019)

    Ben Abacha, A., Demner-Fushman, D.: A question-entailment approach to question answering. BMC Bioinform.20(1), 511–151123 (2019)

    work page 2019

  74. [74]

    What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Exams

    Jin, D., Pan, E., Oufattole, N., Weng, W.-H., Fang, H., Szolovits, P.: What disease does this patient have? a large-scale open domain question answering dataset from medical exams. arXiv preprint arXiv:2009.13081 (2020)

    work page internal anchor Pith review Pith/arXiv arXiv 2009

  75. [75]

    In: Flores, G., Chen, G.H., Pollard, T., Ho, J.C., Naumann, T

    Pal, A., Umapathi, L.K., Sankarasubbu, M.: Medmcqa: A large-scale Springer Nature 2021 LATEX template 50Survey on LLMs for Medical Reasoning multi-subject multi-choice dataset for medical domain question answer- ing. In: Flores, G., Chen, G.H., Pollard, T., Ho, J.C., Naumann, T. (eds.) Proceedings of the Conference on Health, Inference, and Learning. Pro-...

    work page 2021

  76. [76]

    Jin, Q., Dhingra, B., Liu, Z., Cohen, W., Lu, X.: Pubmedqa: A dataset for biomedical research question answering. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 2567–2577 (2019)

    work page 2019

  77. [77]

    Nature Medicine31(3), 943–950 (2025)

    Singhal, K., Tu, T., Gottweis, J., Sayres, R., Wulczyn, E., Amin, M., Hou, L., Clark, K., Pfohl, S.R., Cole-Lewis, H.,et al.: Toward expert- level medical question answering with large language models. Nature Medicine31(3), 943–950 (2025)

    work page 2025

  78. [78]

    HealthBench: Evaluating Large Language Models Towards Improved Human Health

    Arora, R.K., Wei, J., Hicks, R.S., Bowman, P., Qui˜ nonero-Candela, J., Tsimpourlas, F., Sharman, M., Shah, M., Vallone, A., Beutel, A., et al.: Healthbench: Evaluating large language models towards improved human health. arXiv preprint arXiv:2505.08775 (2025)

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  79. [79]

    In: TREC 2017 (2017)

    Ben Abacha, A., Agichtein, E., Pinter, Y., Demner-Fushman, D.: Overview of the medical question answering task at trec 2017 liveqa. In: TREC 2017 (2017)

    work page 2017

  80. [80]

    In: MEDINFO 2019 (2019)

    Ben Abacha, A., Mrabet, Y., Sharp, M., Goodwin, T., Shooshan, S.E., Demner-Fushman, D.: Bridging the gap between consumers’ medication questions and trusted answers. In: MEDINFO 2019 (2019)

    work page 2019

Showing first 80 references.