Pith. sign in

REVIEW 4 major objections 6 minor 30 references

The paper claims that large language models now outperform physicians at generating complex text-based differential diagnoses and can produce expert-style written and narrated presentations that blinded physicians cannot reliably distinguis

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

An LLM agent and a 10-task benchmark built from 7,102 NEJM clinicopathologic cases push medical AI evaluation beyond final diagnosis accuracy.

T0 review reviewed 2026-08-04 challenge →

load-bearing objection A valuable benchmark for AI diagnostic reasoning, but the headline 'exceeds physicians' claim rests on an unmatched historical baseline. the 4 major comments →

arxiv 2509.12194 v2 pith:VHWCY54E submitted 2025-09-15 cs.AI cs.CV

Teaching large language models to reason like expert diagnosticians

Thomas A. Buckley , Riccardo Conci , Peter G. Brodeur , Jason Gusdorf , Sourik Beltr\'an , Bita Behrouzi , Byron Crowe , Jacob Dockterman
show 18 more authors
Muzzammil Muhammad Sarah Ohnigian Andrew Sanchez James A. Diao Aashna P. Shah Daniel Restrepo Eric S. Rosenberg Andrew S. Lea Emily Glanton Kimberly LeBlanc Undiagnosed Diseases Network Marinka Zitnik Scott H. Podolsky Zahir Kanjee Raja-Elie E. Abdulnour Jacob M. Koshy Adam Rodman Arjun K. Manrai
This is my paper
classification cs.AI cs.CV
keywords differential diagnosisclinicopathologic conferenceslarge language modelsagentic AICPC-BenchDr. CaBotmedical reasoningmultimodal clinical AI
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that modern large language models can do more than list likely diagnoses: they can perform the full task of an expert discussant, from parsing a case presentation to producing a written differential and a narrated slide conference. To test this, the authors built CPC-Bench, a physician-validated benchmark drawn from 7,102 clinicopathologic conferences spanning a century, and Dr. CaBot, an agentic system that mimics expert style and searches clinical literature. On 377 recent cases, o3 ranked the final diagnosis first in 60% of cases and in its top ten in 84%, which the paper compares to a 20-physician baseline of 24% top-1 and 45% top-10. In blinded head-to-head comparisons, physicians misattributed CaBot's differential to a human expert in 74% of trials and rated it favorably. The overarching claim is that raw model scale and inference-time reasoning, not fine-tuning or prompting tricks, are driving progress in clinical AI, while image interpretation and literature retrieval trail behind.

Core claim

The central discovery is that a general-purpose reasoning model, embedded in an agentic loop that retrieves similar past cases and literature, can generate differential diagnoses that expert physicians rate as high as the published human discussant's work and often cannot tell apart. On 377 CPCs from 2015 through 2025, o3 achieved 60% top-1 and 84% top-10 accuracy, and next-test selection reached 98%. On 72 referral notes from an undiagnosed-disease network, CaBot identified the working diagnosis in 69% of cases. On 62 blinded trials, physicians correctly identified CaBot as AI only 26% of the time, and on 44 rated trials CaBot scored higher on overall quality, diagnostic justification, cita

What carries the argument

The key object is Dr. CaBot, an agentic AI discussant built on the o3 model. It receives only the case presentation, then performs up to 25 iterative searches over an embedded index of roughly 3.5 million clinical abstracts, and selects two stylistically similar past CPC differentials by embedding similarity to mimic the expert's written form. The supporting object is CPC-Bench, a ten-task benchmark built from 7,102 CPCs with a physician-validated LLM judge; its diagnostic-touchpoint annotations split cases into sequential clinical events, which is what lets the authors measure how diagnostic accuracy rises as information accumulates.

Load-bearing premise

The claim that LLMs exceed physician performance assumes the prior 20-physician baseline measured on 302 different CPCs is directly comparable to o3's scores on 377 different cases; if the baseline were collected on the same cases, the superiority result could change.

What would settle it

Run the same 20-physician protocol with unrestricted electronic search on the exact 377 CPCs used to score o3; if physicians' top-1 or top-10 accuracy reaches or exceeds o3's 60% and 84%, the paper's central claim that LLMs exceed physicians on text-based differential diagnosis falls.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Text-based differential diagnosis on curated, information-dense cases is approaching saturation: the best model now puts the correct diagnosis in its top ten 84% of the time on the evaluated set.
  • Progress in clinical AI appears to come from model scale and inference-time reasoning, not from medical fine-tuning or elaborate prompting, because the authors observed steep gains from GPT-3.5 to o3 with no domain-specific tuning.
  • An AI discussant can serve as an endlessly interrogable teaching tool in case-based learning, generating on-demand text and video case conferences for any era.
  • Image interpretation and literature retrieval remain the binding constraints: pure-image diagnosis (19% top-1) and citation recall (best 49% top-10 with retrieval) are far below text-based performance.
  • Public release of the benchmark and system enables transparent, standardized comparison of future models across all ten tasks.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A direct head-to-head trial on the same 377 cases, with physicians receiving identical inputs and scoring, would be needed to confirm the physician-superiority claim; the paper's baseline comes from a different 302-case set.
  • If the agentic search loop is what closes the gap to experts, ablating retrieval on new cases would reveal how much of CaBot's performance depends on access to the literature index versus the base model's memory.
  • The touchpoint annotation method could be turned into a clinical tool: scoring a model after each new piece of information could identify which missing findings most change the diagnostic rank, giving clinicians live feedback about what to ask next.
  • The style-matching trick of feeding two similar past cases may generalize beyond the CPC genre to other structured expert writings, such as radiology reports or surgery notes, but that is an untested extension.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper introduces CPC-Bench, a 10-task benchmark built from 7,102 NEJM Clinicopathological Conferences and 1,021 Image Challenges, together with Dr. CaBot, an agentic LLM system that generates written and narrated slide-based differential presentations. Using GPT-4.1 as an LLM judge and physician-validated annotations, the authors report that o3 achieves 60% top-1 and 84% top-10 accuracy on 377 contemporary CPCs, outperforming a previously reported 20-physician baseline; that CaBot's presentations are often mistaken for human expert discussants in blinded physician evaluation; and that LLMs still lag on image interpretation and literature search. The authors also benchmark models on historical CPCs, analyze performance as case information is added, and release leaderboards and public resources.

Significance. If the results hold, CPC-Bench is a valuable community resource: it spans a century of curated cases, covers ten cognitive and multimodal tasks, includes physician-validated annotations and LLM-judge validation, and the paper provides transparency through prompt/schema releases and public leaderboards. The historical benchmarking across decades is original and useful. CaBot is a concrete, reproducible artifact. However, the two headline claims—'exceeds physician performance' and the 50/72 UDN result—are not supported by the evidence in the submitted main text. The physician comparison relies on an unmatched historical baseline, and the UDN result is absent from the main text. These are load-bearing gaps, not presentation issues.

major comments (4)
  1. [Abstract; Main Text (UDN result)] The abstract states that 'CaBot identified the working diagnosis in 50/72 (69%) of cases from referral notes alone' for NIH Undiagnosed Diseases Network patients. The full text contains no Methods or Results describing this evaluation: no inclusion criteria for the 72 cases, no definition of 'working diagnosis,' no description of the referral-note exposure, no adjudication or physician-verification protocol, and no time period. This is a headline claim in the abstract and is entirely unsupported in the submitted manuscript. The authors must either add the complete evaluation or remove the claim from the abstract and conclusions.
  2. [RESULTS, Differential Diagnosis Generation; Figure 2B] The central claim that LLMs 'exceed physician performance' rests on comparing o3 (60% top-1, 84% top-10 on 377 CPCs) with a prior cohort of 20 physicians on 302 different CPCs (24% top-1, 45% top-10). This is not a head-to-head comparison. CPCs are curated and their difficulty varies by era and case selection; the two case sets are not shown to be comparable, and the scoring/extraction protocols used for the prior study are not described. Figure 2B explicitly labels the baseline as 'subset of 302 cases' from a prior study. The Discussion's fourth limitation concedes that several tasks lack a large human baseline, but the abstract and conclusions do not carry this caveat. Please either collect physician performance on the same 377 cases with the same ranking and extraction procedure, or revise the claim to 'outperforms a previously reported 302-case physician cohort' and temper the Conclu
  3. [METHODS, Benchmark Creation and Validation] The DDx, Diagnostic Touchpoints, Information Omission, Visual DDx, and Testing Plan metrics rely on GPT-4.1 as an LLM judge. The reported validation on 1,467 prior physician-annotated differentials (accuracy 86%, F1 89%) is useful, but it is validation against prior-study annotations, not against a random sample of this benchmark's 377-case outputs. No stratified analysis is given (e.g., by diagnosis frequency, case era, or model). Because the absolute numbers (60% top-1, 84% top-10) and the cross-model rankings depend on this judge, a systematic false-positive or false-negative tendency could affect the conclusions. Please report judge agreement on a sample of the actual outputs and perform a sensitivity analysis with a second judge or physician arbitration of disagreements.
  4. [RESULTS, AI Expert Discussant Performance; Figure 4] The emulation claim is based on 62 blinded trials from 27 cases, with 5 physicians; the rubric-rated assessment is based on 44 trials from 22 cases, and the Figure 4C legend says four physicians. The manuscript does not clarify the relation between the 62 and 44 trial subsets, whether the 44 trials are a random subset, or whether presentation order was balanced. The claim that CaBot scored 'consistently better' on all four quality dimensions is strong; please provide the trial-level breakdown, inter-rater agreement among physicians, and a statement of how the subset was selected. If the subset is not representative, the emulation conclusion is weakened.
minor comments (6)
  1. [Figure 4] The text states five physicians (two attendings, three residents) were involved, while Figure 4C legend says four physicians. Please reconcile.
  2. [Abstract / Using CPC-Bench and Dr. CaBot] The abstract says 'we release both CaBot and CPC-Bench publicly,' but the full text says the benchmark subset will be released 'upon publication.' Please clarify what is currently available and under what license.
  3. [Introduction / AI Expert Discussant] The phrase 'CaBot recently generated the first AI diagnosis published in the 100+ year history of the NEJM CPCs' has no citation or date. Add a reference or remove the claim.
  4. [METHODS, Physician Annotation] Ten physicians × 20 cases = 200 annotated case reports, but the text says the annotation covered all cases from January 2023 through February 2025 plus a uniform sample from 1991 onward. This is likely more than 200 cases. Specify the sampling scheme and the total number of unique annotated cases.
  5. [RESULTS, Benchmarking on Historical Cases] The historical comparison in Figure 5B requires knowing how the expert discussant's predicted diagnosis was extracted from the 'Differential Diagnosis' section. A discussant's differential is often a ranked or narrative list; please state the extraction rule (e.g., first listed diagnosis, LLM-extracted top diagnosis) and whether it was applied identically to humans and models.
  6. [RESULTS, Differential Diagnosis Generation] The prior physician baseline is only cited (reference 12). For the comparison to be interpretable, add a sentence in Methods describing that study's physician protocol, scoring method, case years, and whether physicians had access to the same kind of ranked-list output or free text.

Circularity Check

0 steps flagged

No significant circularity: the evaluation pipeline is externally anchored by physician labels and the physician-comparison baseline, while imperfect, is an external historical benchmark rather than a fitted input.

full rationale

The paper's central claims are empirical evaluations rather than derivations that reduce to their inputs. The DDx accuracy numbers are obtained by an LLM judge (GPT-4.1) checking whether the final CPC diagnosis appears in ranked model outputs; the judge is validated against 1,467 physician-annotated differentials from prior studies and 112 physician scores, so the evaluation is anchored to human labels rather than defined by the model itself. CaBot's style-matching uses similar CPC differentials and its quality is assessed by blinded physicians, not by a metric that is constructed from CaBot's own outputs. The main concern—that 'LLMs exceed physician performance' relies on a 20-physician baseline from 302 different CPCs—is a legitimate external-validity limitation, but it is not circularity: the baseline is not fitted from the present data, is not derived from the models being evaluated, and is acknowledged in the paper's fourth limitation. No equation-level self-definition, no fitted parameter renamed as a prediction, and no load-bearing self-citation chain were found. Self-citations to prior physician-annotation studies provide published, externally checkable evidence and do not make the current derivation circular.

Axiom & Free-Parameter Ledger

3 free parameters · 4 axioms · 0 invented entities

The claims rest on benchmark construction choices and LLM-based evaluation assumptions rather than mathematical postulates. No free parameters are fitted to data, but CaBot has several hand-set design choices, and the headline comparisons assume LLM-judge validity and transferability of a prior physician baseline.

free parameters (3)
  • Number of style-comparator CPCs for CaBot = 2
    Hard-coded choice in the CaBot prompt: the two closest Differential Diagnosis sections are used to mimic style. This affects the generated text and is not derived from data.
  • Maximum literature queries in CaBot = 25
    Cap on agentic retrieval iterations, chosen by design. It bounds runtime and could affect diagnostic and citation quality.
  • Case window for comparator selection = Modern cases from 2000 onward within 2 years of index case
    Default window for selecting style exemplars; a historical mode uses a user-specified decade. This is a tunable design choice.
axioms (4)
  • domain assumption The final diagnosis section of each CPC defines ground truth, and the GPT-4.1 LLM judge can reliably detect whether model outputs contain that diagnosis.
    All differential-based tasks depend on this judge. It was validated at 86% accuracy and F1 89% on 1,467 physician-annotated differentials, leaving residual scoring error.
  • ad hoc to paper A prior 20-physician baseline (24% top-1, 45% top-10 on 302 CPCs) is comparable to o3 performance on the present 377-case evaluation set.
    Used to conclude that LLMs exceed physician performance; the case sets and study protocols differ, so transferability is assumed.
  • ad hoc to paper Performance on post-training-cutoff CPCs controls for memorization of case answers.
    The paper states training data are unknown and cites a Supplemental post-cutoff analysis. The control is asserted rather than demonstrated in the main text.
  • domain assumption CPC images, captions, and pre-1945 PDFs are correctly parsed and annotated.
    Pre-1945 cases were parsed with a vision-language model, and VQA and image tasks depend on caption quality. No large-scale manual verification is reported.

reviewed 2026-08-04 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Teaching large language models to reason like expert diagnosticians." pith.science (2026). https://pith.science/paper/VHWCY54E

@misc{pith2026250912194,
  author       = {Pith},
  title        = {Pith review of: Teaching large language models to reason like expert diagnosticians},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VHWCY54E}},
  note         = {Machine review of arXiv:2509.12194}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Differential diagnosis is an iterative process that integrates patient information with broader medical knowledge. Clinical case series such as the NEJM Clinicopathologic Conferences (CPCs), published continuously since 1923, feature expert physicians who demonstrate diagnostic reasoning to peers, and have been used for decades to evaluate AI. However, prior AI evaluations have largely focused on final diagnostic accuracy rather than nuanced clinical reasoning. Here, we introduce Dr. CaBot, an agentic AI system that emulates an expert diagnostician by generating written and narrated slide-based presentations from an initial case description alone. CaBot recently generated the first AI diagnosis published in the 100+ year history of the NEJM CPCs. In blinded evaluations, physicians misclassified the source of the differential (CaBot vs. physician-written) in 46/62 (74%) of trials and rated them favorably across quality dimensions. When tasked with solving cases for 72 patients with undiagnosed disease from the NIH Undiagnosed Diseases Network, CaBot identified the working diagnosis in 50/72 (69%) of cases from referral notes alone. To promote transparency and research, we also developed CPC-Bench, a physician-validated benchmark based on 7,102 CPCs and 47,648 questions across 10 tasks. We show that CaBot outperforms frontier models on CPC-Bench, and release both CaBot and CPC-Bench publicly to foster progress in clinical AI.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

30 extracted references · 2 canonical work pages

  1. [1]

    Case 9431

    Cabot RC, Cabot H, Painter FM. Case 9431. Boston Med Surg J 1923;189(17):595–9

    1923

  2. [2]

    Building a community of medical learning - A century of case records of the Massachusetts general hospital in the journal

    Brinkmann R, Rosenberg E, Louis DN, Podolsky SH. Building a community of medical learning - A century of case records of the Massachusetts general hospital in the journal. N Engl J Med 2024;391(9):858–63

    2024

  3. [3]

    The clinicopathological conferences (CPCs)

    Harris NL, Scully RE, Louis DN, Young RH. The clinicopathological conferences (CPCs). Keen minds to explore the dark continents of disease A history of the pathology services at the Massachusetts General Hospital Massachusetts General Hospital and Harvard Medical School, Boston 2011;349–53

    2011

  4. [4]

    Reasoning foundations of medical diagnosis; symbolic logic, probability, and value theory aid our understanding of how physicians reason

    Ledley RS, Lusted LB. Reasoning foundations of medical diagnosis; symbolic logic, probability, and value theory aid our understanding of how physicians reason. Science 1959;130(3366):9–21

    1959

  5. [5]

    Digital computers and medical logic

    Ebald R, Lane R. Digital computers and medical logic. IRE Trans Med Electron 1960;ME-7(4):283–8

    1960

  6. [6]

    Digitizing diagnosis: Medicine, minds, and machines in twentieth-century America

    Lea AS. Digitizing diagnosis: Medicine, minds, and machines in twentieth-century America. 2023;Available from: https://books.google.com/books?hl=en&lr=&id=qPqwEAAAQBAJ&oi=fnd&pg=PP1&dq=Andrew+S.+Lea,+Digitizing+Diagnosis:+Medicine,+Minds,+and+Machines+in+Twentieth-Century+America+(Baltimore:+Johns+Hopkins+University+Press,+2023).&ots=F0u7KcAOfU&sig=5my2kI...

    2023

  7. [7]

    Internist-1, an experimental computer-based diagnostic consultant for general internal medicine

    Miller RA, Pople HE Jr, Myers JD. Internist-1, an experimental computer-based diagnostic consultant for general internal medicine. N Engl J Med 1982;307(8):468–76

    1982

  8. [8]

    Differential diagnosis generators: an evaluation of currently available computer programs

    Bond WF, Schwartz LM, Weaver KR, Levick D, Giuliano M, Graber ML. Differential diagnosis generators: an evaluation of currently available computer programs. J Gen Intern Med 2012;27(2):213–9

    2012

  9. [9]

    Evaluation of medical decision support systems (DDX generators) using real medical cases of varying complexity and origin

    Fritz P, Kleinhans A, Raoufi R, et al. Evaluation of medical decision support systems (DDX generators) using real medical cases of varying complexity and origin. BMC Med Inform Decis Mak 2022;22(1):254

    2022

  10. [10]

    Accuracy of a generative artificial intelligence model in a complex diagnostic challenge

    Kanjee Z, Crowe B, Rodman A. Accuracy of a generative artificial intelligence model in a complex diagnostic challenge. JAMA 2023;330(1):78–80

    2023

  11. [11]

    Comparison of frontier open-source and proprietary large language models for complex diagnoses

    Buckley TA, Crowe B, Abdulnour R-E, Rodman A, Manrai AK. Comparison of frontier open-source and proprietary large language models for complex diagnoses. JAMA Health Forum 2025;6(3):e250040–e250040

    2025

  12. [12]

    Towards accurate differential diagnosis with large language models

    McDuff D, Schaekermann M, Tu T, et al. Towards accurate differential diagnosis with large language models. Nature 2025;642(8067):451–7

    2025

  13. [13]

    Superhuman performance of a large language model on the reasoning tasks of a physician [Internet]

    Brodeur PG, Buckley TA, Kanjee Z, et al. Superhuman performance of a large language model on the reasoning tasks of a physician [Internet]. arXiv [cs.AI]. 2024;Available from: http://arxiv.org/abs/2412.10849

    Pith/arXiv arXiv 2024

  14. [14]

    Sequential diagnosis with language models [Internet]

    Nori H, Daswani M, Kelly C, et al. Sequential diagnosis with language models [Internet]. arXiv [cs.CL]. 2025;Available from: http://arxiv.org/abs/2506.22405

    Pith/arXiv arXiv 2025

  15. [15]

    It’s time to bench the medical exam benchmark

    Raji ID, Daneshjou R, Alsentzer E. It’s time to bench the medical exam benchmark. NEJM AI 2025;2(2):AIe2401235

    2025

  16. [16]

    When it comes to benchmarks, humans are the only way

    Rodman A, Zwaan L, Olson A, Manrai AK. When it comes to benchmarks, humans are the only way. NEJM AI 2025;2(4):AIe2500143

    2025

  17. [17]

    Representation and misdiagnosis of dark skin in a large-scale visual diagnostic challenge

    Diao JA, Adamson AS. Representation and misdiagnosis of dark skin in a large-scale visual diagnostic challenge. J Am Acad Dermatol 2022;86(4):950–1

    2022

  18. [18]

    OpenAlex: A fully-open index of scholarly works, authors, venues, institutions, and concepts [Internet]

    Priem J, Piwowar H, Orr R. OpenAlex: A fully-open index of scholarly works, authors, venues, institutions, and concepts [Internet]. arXiv [cs.DL]. 2022;Available from: http://arxiv.org/abs/2205.01833

    Pith/arXiv arXiv 2022

  19. [19]

    Hippocratic corpus

    Iniesta I. Hippocratic corpus. BMJ 2011;342(apr19 2):d688–d688

    2011

  20. [20]

    Hippocratic

    Craik EM. The “Hippocratic” corpus: Content and context [Internet]. London, England: Routledge; 2014. Available from: http://dx.doi.org/10.4324/9781315736723/hippocratic-corpus-elizabeth-craik

    work page doi:10.4324/9781315736723/hippocratic-corpus-elizabeth-craik 2014

  21. [21]

    MEDITRON-70B: Scaling medical pretraining for large language models [Internet]

    Chen Z, Cano AH, Romanou A, et al. MEDITRON-70B: Scaling medical pretraining for large language models [Internet]. arXiv [cs.CL]. 2023;Available from: http://arxiv.org/abs/2311.16079

    Pith/arXiv arXiv 2023

  22. [22]

    MedGemma Technical Report [Internet]

    Sellergren A, Kazemzadeh S, Jaroensri T, et al. MedGemma Technical Report [Internet]. arXiv [cs.AI]. 2025;Available from: http://arxiv.org/abs/2507.05201

    Pith/arXiv arXiv 2025

  23. [23]

    Limitations of learning new and updated medical knowledge with commercial fine-tuning large language models

    Wu E, Wu K, Zou J. Limitations of learning new and updated medical knowledge with commercial fine-tuning large language models. NEJM AI [Internet] 2025;2(8). Available from: http://dx.doi.org/10.1056/aics2401155

    work page doi:10.1056/aics2401155 2025

  24. [24]

    Evaluating the effectiveness of biomedical fine-tuning for large language models on clinical tasks

    Dorfner FJ, Dada A, Busch F, et al. Evaluating the effectiveness of biomedical fine-tuning for large language models on clinical tasks. J Am Med Inform Assoc 2025;32(6):1015–24

    2025

  25. [25]

    Chain-of-thought prompting elicits reasoning in large language models

    Wei J, Wang X, Schuurmans D, et al. Chain-of-thought prompting elicits reasoning in large language models. In: Proceedings of the 36th International Conference on Neural Information Processing Systems. Red Hook, NY, USA: Curran Associates Inc.; 2022. p. 24824–37

    2022

  26. [26]

    Can generalist foundation models outcompete special-purpose tuning? Case study in medicine [Internet]

    Nori H, Lee YT, Zhang S, et al. Can generalist foundation models outcompete special-purpose tuning? Case study in medicine [Internet]. arXiv [cs.CL]. 2023;Available from: http://arxiv.org/abs/2311.16452

    Pith/arXiv arXiv 2023

  27. [27]

    From Medprompt to o1: Exploration of run-time strategies for medical challenge problems and beyond [Internet]

    Nori H, Usuyama N, King N, et al. From Medprompt to o1: Exploration of run-time strategies for medical challenge problems and beyond [Internet]. arXiv [cs.CL]. 2024;Available from: http://arxiv.org/abs/2411.03590

    Pith/arXiv arXiv 2024

  28. [28]

    The Bitter Lesson [Internet]

    Sutton R. The Bitter Lesson [Internet]. Available from: https://www.cs.utexas.edu/~eunsol/courses/data/bitter_lesson.pdf

  29. [29]

    LongHealth: A question answering benchmark with long clinical documents

    Adams L, Busch F, Han T, et al. LongHealth: A question answering benchmark with long clinical documents. J Healthc Inform Res 2025;9(3):280–96

    2025

  30. [30]

    Health system-scale language models are all-purpose prediction engines

    Jiang LY, Liu XC, Nejatian NP, et al. Health system-scale language models are all-purpose prediction engines. Nature 2023;619(7969):357–62

    2023

This paper was first reviewed by deepseek-v4-flash on August 4, 2026.