Pith. sign in

REVIEW 3 major objections 5 minor 47 references

DynamiCare: A Dynamic Multi-Agent Framework for Interactive and Open-Ended Medical Decision-Making

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A dynamic team of AI specialists that changes as it questions a simulated patient outperforms a single-agent doctor on open-ended diagnosis.

desk verdict A dynamic-team framework and a useful benchmark design, but the evaluation leaks the answer key through the patient JSON and the headline comparisons need redoing. read the letter →

arxiv 2507.02616 v1 pith:SH2LKD3G submitted 2025-07-03 cs.AI

Tianqi Shang , Weiqing He , Charles Zheng , Lingyao Li , Li Shen , Bingxin Zhao This is my paper
classification cs.AI
keywords multi-agentLLMsystemsclinicaldecision-makinginteractivediagnosispatientsimulationelectronichealthrecordsdynamicagentteamsopen-endedmedicalbenchmark
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that clinical diagnosis is better modeled as an interactive, multi-round conversation than as a one-shot question-answer task, and that a doctor team whose composition changes as information arrives outperforms a fixed single-agent doctor. It anchors this claim in a working system: DynamiCare, a six-step loop in which a coordinating agent recruits and dismisses specialist agents while they question a simulated patient, together with MIMIC-Patient, a benchmark of 500 structured admissions derived from a public critical-care EHR database. On its own benchmark, the dynamic multi-agent system reaches a top-5 hit rate of 63.4 and a top-10 recall of 58.8 with the stronger of two LLMs, versus 58.0 and 41.7 for the single-agent variant under the same conditions, and the same ordering holds for the smaller model. The paper also claims that this is the first benchmark for dynamic clinical decision-making and that human annotators rate the simulated patients' answers as largely truthful and relevant.

What carries the argument

The load-bearing mechanism is the six-step interaction loop of DynamiCare. Step 1 initializes a visit log from admission data; Step 2 has a Central Agent form a specialist team; Step 3 has the team respond with either a diagnosis list or a follow-up question; Step 4 passes the question to the Patient System, which retrieves an answer by keyword-matching the query against sections of a per-patient JSON record and, only when no section matches, falls back to letting an LLM read the record; Step 5 appends the question-answer pair to the log; and Step 6 lets the Central Agent add or remove specialists before the next round. The loop terminates when a diagnosis is issued or the round limit is reached. The Patient System's rule-first design is what keeps answers anchored to the record, and the Central Agent's add/remove decision is what makes the process dynamic.

What would settle it

Go through the logged question-answer transcripts and count how often doctor questions ask about prior diagnoses or chart labels, and whether the patient's answer reproduces wording from the Diagnoses section of the record; if such exchanges occur and the predictions match the ground truth, the reported Hit@K and Rec@K numbers are inflated. A cleaner experiment: rerun the benchmark with the Diagnoses section removed from the patient record, and see whether the multi-agent advantage over the single agent survives.

Watch

Extended reading notes

Core claim

The central discovery is that a diagnostic agent can improve by actively deciding what to ask and who should answer it next, rather than absorbing a full case description up front. In the proposed design, the Doctor System assembles a team of specialists that propose either a ranked list of up to ten diagnoses or a follow-up question, with confidence ratings and cross-voting deciding which response goes forward; each answer gets appended to a visit log, and a central coordinator re-examines the log to change the team before the next round. The paper reports that this dynamic multi-agent setting beats its single-agent twin across Hit@5, Hit@10, Rec@5, and Rec@10 on both LLM backbones tested, and that the same system beats an existing interactive multiple-choice benchmark's baseline under identical conditions. On its own terms, the paper is establishing a first infrastructure for studying open-ended, interactive diagnostic reasoning in simulated but realistic clinical encounters.

Load-bearing premise

The evaluation assumes the simulated patient never reveals the charted diagnoses to the doctor agents, even though those same charted diagnoses are the scoring key for every reported accuracy number.

Editorial extensions

If this is right

  • Open-ended diagnosis can be decomposed into a sequence of targeted queries, so evaluation can measure information-seeking behavior rather than only final answer accuracy.
  • Structured patient-level records from real admissions make it possible to benchmark interactive agents on realistic clinical complexity, including labs, imaging reports, and discharge summaries.
  • Dynamic team adjustment, such as bringing in an imaging specialist when scans become relevant, is a transferable design pattern for any multi-agent task driven by a growing evidence log.
  • The manual truthfulness and relevance ratings imply that a rule-then-LLM patient simulator can support trustworthy multi-turn interaction, making larger interactive evaluations feasible.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's multi-versus-single comparison does not isolate the dynamic-adjustment mechanism itself; a sharper test would compare a team that may change composition across rounds against a fixed team with the same type and number of specialists, holding the LLM and round limit constant.
  • Because the per-patient record used by the Patient System contains a Diagnoses section that also serves as the scoring key, a replication should strip that section from the patient context and rerun; if Hit@5 and Rec@K drop materially, part of the reported accuracy is label retrieval rather than diagnostic reasoning.
  • The same six-step loop should transfer to other open-ended diagnostic settings, such as root-cause analysis in engineering or investigative triage, where a coordinator adds experts as evidence accumulates; the paper leaves that generalization implicit.
  • One testable extension is to measure diagnostic value per question, for example by tracking how much the ranked diagnosis list changes after each patient answer, to quantify whether interaction itself drives the improvement.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces MIMIC-Patient, a dataset of 500 patient records derived from MIMIC-III and structured as JSON files to support multi-round interactive diagnosis, and DynamiCare, a multi-agent framework in which a central agent dynamically recruits and removes specialist agents over repeated question-answer rounds. The authors compare DynamiCare's multi-agent setting with a single-agent variant on open-ended diagnosis using Hit@5, Hit@10, Rec@5, Rec@10, and average number of questions (Ave-Q), reporting that the multi-agent setting outperforms the single-agent setting for both GPT-4.1 and GPT-4o-mini (Table 2). They also report accuracy on the MEDIQ benchmark (Table 3) and a manual evaluation of Patient System responses for truthfulness and relevance (Table 4). The central claim is that dynamically adjusting specialist team composition improves interactive clinical reasoning over static single-agent baselines.

Significance. If the reported results hold, the paper would make a useful contribution: open-ended, interactive diagnosis with adaptive specialist teams is a timely problem, and the MIMIC-Patient resource could support future work. The paper has several genuine strengths: it builds patient-level records from real EHR data, uses a two-stage Patient System designed to reduce hallucination, evaluates with two backbone LLMs, and provides a cross-benchmark comparison on MEDIQ. However, the central evaluation is currently compromised by a plausible label-leakage path through the Patient System's fallback context, and the multi-agent versus single-agent comparison is confounded by a large difference in the number of questions asked. These issues are load-bearing for the paper's main claim, so the significance is conditional on a corrected re-evaluation.

major comments (3)
  1. [Section 4.1, Appendix A, Section 5.2] The evaluation has a label-leakage path. In the Patient System's fallback mode, the patient JSON is provided as LLM context with only admission and demographics information removed (Section 4.1), and the JSON example in Appendix A shows that the 'Diagnoses' field contains the same ICD-9 codes used as ground truth for Hit@K and Rec@K in Section 5.2. The Appendix B prompts do not instruct the Patient System to withhold known diagnoses, nor do they prohibit the doctor from asking for them. A doctor agent can therefore ask the simulated patient for their chart diagnoses and receive the evaluation's ground-truth labels, which would inflate Table 2 and undermine the claim that the framework measures diagnostic reasoning rather than label retrieval. The manual truthfulness scores in Table 4 cannot distinguish faithful chart summarization from direct label retrieval, so they do not address this concern. Please remove or mask the Diagnoses section from all Patient System contexts, or otherwise guarantee that ground-truth labels cannot be recovered through patient responses, and report the resulting Table 2 numbers.
  2. [Section 5.2, Table 2] The central multi-agent versus single-agent comparison is confounded by interaction volume. Table 2 reports Ave-Q of 7.55 versus 3.83 for GPT-4.1 and 2.78 versus 0.74 for GPT-4o-mini, meaning the multi-agent arm collects roughly twice as many (or more) patient responses before diagnosing. The paper attributes the performance gap to dynamic team adaptation, but more information alone could explain the improvement. Additionally, no confidence intervals, significance tests, or per-case paired analyses are reported. Please add per-case paired comparisons and confidence intervals, and include an ablation that controls for the number of questions (for example, by capping multi-agent rounds or by measuring incremental information gain per question) to separate the effect of team dynamics from the effect of additional queries.
  3. [Section 5.3, Table 3] The MEDIQ comparison reports large gaps (92.0 and 96.4 for DynamiCare versus 67.0 and 72.1 for MedIQ) but provides no confidence intervals, number of independent runs, or detailed adaptation protocol for the multiple-choice format beyond saying that prompt templates were 'appropriately adapted.' Without these details, and without knowing whether the same stopping criteria and question budgets were used, the statement that DynamiCare 'significantly outperforms' the baseline is not supported by the reported evidence. Please report the exact prompt adaptation, stopping rules, and variance estimates for both datasets.
minor comments (5)
  1. [Table 3 caption] The caption contains a typo: 'DyamiCare' should be 'DynamiCare'.
  2. [Section 5.2] Ave-Q is used in the results discussion before it is defined; please define the metric at its first occurrence in Section 5.1 or 5.2.
  3. [Section 5.5] The text refers to 'top-5 accuracy' and 'top-10 accuracy' while Section 5.2 defines Hit@5 and Hit@10; please align the terminology to avoid ambiguity.
  4. [Appendix A] The keyword mapping dictionary contains keys with trailing spaces, e.g., 'admission medications ' and 'initial meds '; this is likely to cause matching failures and should be cleaned.
  5. [Table 5] The table caption says categories with accuracy above the mean are highlighted in green, but the text version of the table does not show color highlighting; please use a text marker or restructure the table so the distinction survives all renderings.

Circularity Check

1 steps flagged · score 6.0 of 10

Ground-truth Diagnoses field remains in the Patient System fallback context, so simulated patients can leak ICD-9 labels and inflate the MIMIC-Patient Hit@K/Rec@K results that support the multi-agent advantage.

  1. other [Section 4.1 (Patient System fallback), Appendix A (patient JSON example), Section 5.2 (evaluation)]
    "In such scenarios, the patient JSON (de-identified, removing admission and demographics) is provided as context to GPT-4.1, which then generates an answer based on the broader clinical context. ... 'Diagnoses': [['4019', 'Hypertension NOS', 'Unspecified essential hypertension'],...] ... We consider a prediction correct if the first three digits of the predicted ICD-9 code match any of the ground truth codes."

    The patient JSON contains a 'Diagnoses' array holding the exact ICD-9-coded labels used as ground truth for Hit@K and Rec@K. The fallback branch removes only admission and demographics, so Diagnoses remains in the LLM context. Appendix B's question prompts forbid repeating previous questions and asking about topics answered 'not in chart,' but they do not forbid asking the simulated patient for her known diagnoses or chart labels. A doctor agent can therefore ask 'What diagnoses were you given?' and receive 'Hypertension NOS', which maps to ICD-9 401 and is scored as correct under the three-digit matching rule. The measured accuracy can thus reduce to retrieving the answer key from the input record by construction.

full rationale

The framework itself is not derivational circularity: DynamiCare's architecture, prompts, and the external MEDIQ comparison (Table 3) stand independently of any fitted parameters or self-citation. The circularity is confined to the MIMIC-Patient evaluation, but it is real and load-bearing for the paper's central claim. In the Patient System's fallback branch, the full patient JSON minus admission and demographics is placed in the LLM context, and that JSON contains a Diagnoses array holding the same ICD-9 labels later used as ground truth in Section 5.2. Nothing in the Appendix B prompts prohibits asking the simulated patient for her chart diagnoses, so a direct request can return strings like 'Hypertension NOS', which map to a three-digit ICD-9 code that counts as a hit. Thus Hit@K and Rec@K on MIMIC-Patient can reduce to retrieving the answer key from the input record. The leak does not force every reported number, because keyword matching may answer many queries before fallback and a doctor agent is not guaranteed to ask for diagnoses, but it provides a trivial success path for both arms, and the multi-agent arm's higher question count gives it more opportunities to exploit the leak. The manual truthfulness and relevance evaluation only checks that patient answers are consistent with the JSON record, which is exactly compatible with copying the Diagnoses field, so it does not remove the leak. Because the external MEDIQ experiments are self-contained and not affected by this leak, the paper is only partially circular rather than entirely forced.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

No new physical entities are introduced. The central claim rests on empirical assumptions about the dataset and evaluation rather than on a derivation, so the ledger lists hand-chosen system thresholds and domain assumptions.

free parameters (4)
  • Maximum specialist team size = 5
    Hand-chosen cap used in the triage and coordination prompts (Appendix B); the central agent can add or remove specialists but never exceeds five.
  • Specialist voting agreement threshold = not reported
    Section 4.2 says the first proposal meeting a 'predefined agreement threshold' is accepted, but the threshold value and voting rule details are not given.
  • Maximum interaction rounds = not reported
    The stopping criterion in Section 4.3 is described as a 'predefined maximum number of interaction rounds', but the value is never reported.
  • Confidence stopping threshold = not reported
    Specialists rate confidence on a 1 to 5 scale and a diagnosis is issued only if confidence is high enough; the precise threshold is not stated in the paper.
assumptions (5)
  • domain assumption MIMIC-III admission diagnoses are an appropriate and complete ground truth for open-ended diagnosis evaluation.
    Section 5.2 uses the Diagnoses field in the patient JSON as the reference for Hit@K and Rec@K.
  • ad hoc to paper The Patient System will not reveal the ground-truth Diagnoses section to the Doctor System.
    Section 4.1 fallback inference passes the full patient JSON, with only admission and demographics removed, to GPT-4.1; no prompt restricts questions about known diagnoses.
  • domain assumption Three-digit ICD-9 code matching is a clinically meaningful correctness criterion.
    Section 5.2 treats any prediction whose first three ICD-9 digits match a ground truth code as correct, which groups distinct conditions into broad categories.
  • domain assumption Manual ratings by three medical students on a 3-point scale reliably measure Patient System quality.
    Section 5.4 reports mean Truthfulness and Relevance scores without inter-annotator agreement statistics.
  • domain assumption The filtered 500 admissions are representative enough to support general claims about dynamic diagnosis.
    Section 3 excludes newborns, deceased patients, and admissions with five or more diagnoses before random selection, which may bias the benchmark toward simpler cases.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DynamiCare: A Dynamic Multi-Agent Framework for Interactive and Open-Ended Medical Decision-Making." pith.science (2026). https://pith.science/paper/SH2LKD3G

@misc{pith2026250702616,
  author       = {Pith},
  title        = {Pith review of: DynamiCare: A Dynamic Multi-Agent Framework for Interactive and Open-Ended Medical Decision-Making},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SH2LKD3G}},
  note         = {Machine review of arXiv:2507.02616}
}
read the original abstract

The rise of Large Language Models (LLMs) has enabled the development of specialized AI agents with domain-specific reasoning and interaction capabilities, particularly in healthcare. While recent frameworks simulate medical decision-making, they largely focus on single-turn tasks where a doctor agent receives full case information upfront -- diverging from the real-world diagnostic process, which is inherently uncertain, interactive, and iterative. In this paper, we introduce MIMIC-Patient, a structured dataset built from the MIMIC-III electronic health records (EHRs), designed to support dynamic, patient-level simulations. Building on this, we propose DynamiCare, a novel dynamic multi-agent framework that models clinical diagnosis as a multi-round, interactive loop, where a team of specialist agents iteratively queries the patient system, integrates new information, and dynamically adapts its composition and strategy. We demonstrate the feasibility and effectiveness of DynamiCare through extensive experiments, establishing the first benchmark for dynamic clinical decision-making with LLM-powered agents.

Figures

Figures reproduced from arXiv: 2507.02616 by the authors.

Figure 1
Figure 1. Illustration of the MIMIC-Patient dataset and DynamiCare framework. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Illustrative example of DynamiCare with a multi-specialist team. The figure presents three rounds [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

47 extracted references · 19 canonical work pages

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, and 1 others. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774

    arXiv 2023

  4. [4]

    Hanjie Chen, Zhouxiang Fang, Yash Singla, and Mark Dredze. 2025. Benchmarking large language models on answering and explaining challenging medical questions. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 3563--3599

    2025

  5. [5]

    Justin Chih-Yao Chen, Swarnadeep Saha, and Mohit Bansal. 2023. Reconcile: Round-table conference improves reasoning via consensus among diverse llms. arXiv preprint arXiv:2309.13007

    arXiv 2023

  6. [6]

    Michael Christ, Florian Grossmann, Daniela Winter, Roland Bingisser, and Elke Platz. 2010. Modern triage in the emergency department. Deutsches \"A rzteblatt International , 107(50):892

    work page 2010

  7. [7]

    Jan Clusmann, Fiona R Kolbinger, Hannah Sophie Muti, Zunamys I Carrero, Jan-Niklas Eckardt, Narmin Ghaffari Laleh, Chiara Maria Lavinia L \"o ffler, Sophie-Caroline Schwarzkopf, Michaela Unger, Gregory P Veldhuizen, and 1 others. 2023. The future landscape of large language models in medicine. Communications medicine, 3(1):141

    2023

  8. [8]

    Yilun Du, Shuang Li, Antonio Torralba, Joshua B Tenenbaum, and Igor Mordatch. 2023. Improving factuality and reasoning in language models through multiagent debate. In Forty-first International Conference on Machine Learning

    2023

Show all 47 references
  1. [9]

    Jungwei Fan. 2019. Annotating and characterizing clinical sentences with explicit why-qa cues. In Proceedings of the 2nd Clinical Natural Language Processing Workshop, pages 101--106

    2019

  2. [10]

    Zhihao Fan, Jialong Tang, Wei Chen, Siyuan Wang, Zhongyu Wei, Jun Xi, Fei Huang, and Jingren Zhou. 2024. Ai hospital: Interactive evaluation and collaboration of llms as intern doctors for clinical diagnosis. arXiv e-prints, pages arXiv--2402

    2024

  3. [11]

    Nicki Gilboy, Paula Tanabe, Debbie Travers, Alexander M Rosenau, and 1 others. 2012. Emergency severity index (esi): a triage tool for emergency department care, version 4. Implementation handbook, 2012:12--0014

    2012

  4. [12]

    Zhiyuan Hu, Chumin Liu, Xidong Feng, Yilun Zhao, See-Kiong Ng, Anh Tuan Luu, Junxian He, Pang Wei Koh, and Bryan Hooi. 2024. Uncertainty of thoughts: Uncertainty-aware planning enhances information seeking in large language models. arXiv preprint arXiv:2402.03271

    2024 arXiv

  5. [13]

    Di Jin, Eileen Pan, Nassim Oufattole, Wei-Hung Weng, Hanyi Fang, and Peter Szolovits. 2021. What disease does this patient have? a large-scale open domain question answering dataset from medical exams. Applied Sciences, 11(14):6421

    2021

  6. [14]

    Qiao Jin, Bhuwan Dhingra, Zhengping Liu, William W Cohen, and Xinghua Lu. 2019. Pubmedqa: A dataset for biomedical research question answering. arXiv preprint arXiv:1909.06146

    2019 arXiv

  7. [15]

    Qiao Jin, Zhizheng Wang, Yifan Yang, Qingqing Zhu, Donald Wright, Thomas Huang, W John Wilbur, Zhe He, Andrew Taylor, Qingyu Chen, and 1 others. 2024. Agentmd: Empowering language agents for risk prediction with large-scale clinical tool learning. arXiv preprint arXiv:2402.13225

    2024 arXiv

  8. [16]

    Alistair EW Johnson, Lucas Bulgarelli, Lu Shen, Alvin Gayles, Ayad Shammout, Steven Horng, Tom J Pollard, Sicheng Hao, Benjamin Moody, Brian Gow, and 1 others. 2023. Mimic-iv, a freely accessible electronic health record dataset. Scientific data, 10(1):1

    2023

  9. [17]

    Alistair EW Johnson, Tom J Pollard, Lu Shen, Li-wei H Lehman, Mengling Feng, Mohammad Ghassemi, Benjamin Moody, Peter Szolovits, Leo Anthony Celi, and Roger G Mark. 2016. Mimic-iii, a freely accessible critical care database. Scientific data, 3(1):1--9

    2016

  10. [18]

    Shreya Johri, Jaehwan Jeong, Benjamin A Tran, Daniel I Schlessinger, Shannon Wongvibulsin, Zhuo Ran Cai, Roxana Daneshjou, and Pranav Rajpurkar. 2024. Craft-md: A conversational evaluation framework for comprehensive assessment of clinical llms. In AAAI 2024 Spring Symposium o...

    2024

  11. [19]

    Maria Palwasha Khan and Eoin Daniel O’Sullivan. 2024. A comparison of the diagnostic ability of large language models in challenging clinical cases. Frontiers in Artificial Intelligence, 7:1379297

    2024

  12. [20]

    Yubin Kim, Chanwoo Park, Hyewon Jeong, Yik S Chan, Xuhai Xu, Daniel McDuff, Hyeonhoon Lee, Marzyeh Ghassemi, Cynthia Breazeal, and Hae W Park. 2024 a . Mdagents: An adaptive collaboration of llms for medical decision-making. Advances in Neural Information Processing Systems, 3...

    2024

  13. [21]

    Yubin Kim, Xuhai Xu, Daniel McDuff, Cynthia Breazeal, and Hae Won Park. 2024 b . Health-llm: Large language models for health prediction via wearable sensor data. arXiv preprint arXiv:2401.06866

    2024 arXiv

  14. [22]

    Sunjun Kweon, Jiyoun Kim, Heeyoung Kwak, Dongchul Cha, Hangyul Yoon, Kwang Kim, Jeewon Yang, Seunghyun Won, and Edward Choi. 2024. Ehrnoteqa: An llm benchmark for real-world clinical practice using discharge summaries. Advances in Neural Information Processing Systems, 37:1245...

    2024

  15. [23]

    Guohao Li, Hasan Hammoud, Hani Itani, Dmitrii Khizbullin, and Bernard Ghanem. 2023. Camel: Communicative agents for" mind" exploration of large language model society. Advances in Neural Information Processing Systems, 36:51991--52008

    2023

  16. [24]

    Junkai Li, Yunghwei Lai, Weitao Li, Jingyi Ren, Meng Zhang, Xinhui Kang, Siyu Wang, Peng Li, Ya-Qin Zhang, Weizhi Ma, and 1 others. 2024 a . Agent hospital: A simulacrum of hospital with evolvable medical agents. arXiv preprint arXiv:2405.02957

    2024 arXiv

  17. [25]

    Stella Li, Vidhisha Balachandran, Shangbin Feng, Jonathan Ilgen, Emma Pierson, Pang Wei W Koh, and Yulia Tsvetkov. 2024 b . Mediq: Question-asking llms and a benchmark for reliable interactive clinical reasoning. Advances in Neural Information Processing Systems, 37:28858--28888

    2024

  18. [26]

    Tian Liang, Zhiwei He, Wenxiang Jiao, Xing Wang, Yan Wang, Rui Wang, Yujiu Yang, Shuming Shi, and Zhaopeng Tu. 2023. Encouraging divergent thinking in large language models through multi-agent debate. arXiv preprint arXiv:2305.19118

    2023 arXiv

  19. [27]

    Valentin Li \'e vin, Christoffer Egeberg Hother, Andreas Geert Motzfeldt, and Ole Winther. 2024. Can large language models reason about medical questions? Patterns, 5(3)

    2024

  20. [28]

    Daniel McDuff, Mike Schaekermann, Tao Tu, Anil Palepu, Amy Wang, Jake Garrison, Karan Singhal, Yash Sharma, Shekoofeh Azizi, Kavita Kulkarni, and 1 others. 2025. Towards accurate differential diagnosis with large language models. Nature, pages 1--7

    2025

  21. [29]

    Harsha Nori, Nicholas King, Scott Mayer McKinney, Dean Carignan, and Eric Horvitz. 2023. Capabilities of gpt-4 on medical challenge problems. arXiv preprint arXiv:2303.13375

    2023 arXiv

  22. [30]

    OpenAI. 2025. Introducing gpt-4.1 in the api. https://openai.com/index/gpt-4-1/. Accessed May 2025

    2025

  23. [31]

    Ankit Pal, Logesh Kumar Umapathi, and Malaikannan Sankarasubbu. 2022. Medmcqa: A large-scale multi-subject multi-choice dataset for medical domain question answering. In Conference on health, inference, and learning, pages 248--260. PMLR

    2022

  24. [32]

    Anusri Pampari, Preethi Raghavan, Jennifer Liang, and Jian Peng. 2018. emrqa: A large corpus for question answering on electronic medical records. arXiv preprint arXiv:1809.00732

    2018 arXiv

  25. [33]

    Alvin Rajkomar, Eyal Oren, Kai Chen, Andrew M Dai, Nissan Hajaj, Michaela Hardt, Peter J Liu, Xiaobing Liu, Jake Marcus, Mimi Sun, and 1 others. 2018. Scalable and accurate deep learning with electronic health records. NPJ digital medicine, 1(1):18

    2018

  26. [34]

    Khaled Saab, Tao Tu, Wei-Hung Weng, Ryutaro Tanno, David Stutz, Ellery Wulczyn, Fan Zhang, Tim Strother, Chunjong Park, Elahe Vedadi, and 1 others. 2024. Capabilities of gemini models in medicine. arXiv preprint arXiv:2404.18416

    2024 arXiv

  27. [35]

    Sarah Sandmann, Sarah Riepenhausen, Lucas Plagwitz, and Julian Varghese. 2024. Systematic analysis of chatgpt, google search and llama 2 for clinical decision support tasks. Nature Communications, 15(1):2050

    2024

  28. [36]

    Samuel Schmidgall, Rojin Ziaei, Carl Harris, Eduardo Reis, Jeffrey Jopling, and Michael Moor. 2024. Agentclinic: a multimodal agent benchmark to evaluate ai in simulated clinical environments. arXiv preprint arXiv:2405.07960

    2024 arXiv

  29. [37]

    Haoran Shi, Pengtao Xie, Zhiting Hu, Ming Zhang, and Eric P Xing. 2017. Towards automated icd coding using deep learning. arXiv preprint arXiv:1711.04075

    2017 arXiv

  30. [38]

    Karan Singhal, Shekoofeh Azizi, Tao Tu, S Sara Mahdavi, Jason Wei, Hyung Won Chung, Nathan Scales, Ajay Tanwani, Heather Cole-Lewis, Stephen Pfohl, and 1 others. 2023. Large language models encode clinical knowledge. Nature, 620(7972):172--180

    2023

  31. [39]

    Karan Singhal, Tao Tu, Juraj Gottweis, Rory Sayres, Ellery Wulczyn, Mohamed Amin, Le Hou, Kevin Clark, Stephen R Pfohl, Heather Cole-Lewis, and 1 others. 2025. Toward expert-level medical question answering with large language models. Nature Medicine, pages 1--8

    2025

  32. [40]

    Xiangru Tang, Anni Zou, Zhuosheng Zhang, Ziming Li, Yilun Zhao, Xingyao Zhang, Arman Cohan, and Mark Gerstein. 2023. Medagents: Large language models as collaborators for zero-shot medical reasoning. arXiv preprint arXiv:2311.10537

    2023 arXiv

  33. [41]

    Daniel Truhn, Jan-Niklas Eckardt, Dyke Ferber, and Jakob Nikolas Kather. 2024. Large language models and multimodal foundation models for precision oncology. NPJ Precision Oncology, 8(1):72

    2024

  34. [42]

    Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, Jingsen Zhang, Zhiyuan Chen, Jiakai Tang, Xu Chen, Yankai Lin, and 1 others. 2024. A survey on large language model based autonomous agents. Frontiers of Computer Science, 18(6):186345

    2024

  35. [43]

    Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2022. Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171

    2022 arXiv

  36. [44]

    Patricia L Whetzel, Natalya F Noy, Nigam H Shah, Paul R Alexander, Csongor Nyulas, Tania Tudorache, and Mark A Musen. 2011. https://doi.org/10.1093/nar/gkr469 Bioportal: enhanced functionality via new web services from the national center for biomedical ontology to access and ...

    2011 doi

  37. [45]

    Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Beibin Li, Erkang Zhu, Li Jiang, Xiaoyun Zhang, Shaokun Zhang, Jiale Liu, and 1 others. 2023. Autogen: Enabling next-gen llm applications via multi-agent conversation. arXiv preprint arXiv:2308.08155

    2023 arXiv

  38. [46]

    Richard C Wuerz, Leslie W Milne, David R Eitel, Debbie Travers, and Nicki Gilboy. 2000. Reliability and validity of a new five-level triage instrument. Academic emergency medicine, 7(3):236--242

    2000

  39. [47]

    Hongjian Zhou, Fenglin Liu, Boyang Gu, Xinyu Zou, Jinfa Huang, Jinge Wu, Yiru Li, Sam S Chen, Peilin Zhou, Junling Liu, and 1 others. 2023. A survey of large language models in medicine: Progress, application, and challenge. arXiv preprint arXiv:2311.05112

    2023 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.