Pith. sign in

REVIEW 4 major objections 6 minor 29 references

Evaluating Hierarchical Clinical Document Classification Using Reasoning-Based LLMs

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Across 1,500 MIMIC-IV discharge summaries tokenized with cTAKES, none of 11 LLMs exceeded 57% F1 on ICD-10 classification, and performance fell as codes became more specific.

desk verdict The 57% F1 ceiling on these 10 ICD-10 codes is probably real and worth knowing, but the paper's reasoning-vs-non-reasoning comparison is too confounded to support its main interpretive claim. read the letter →

arxiv 2507.03001 v1 pith:GLXLY5VX submitted 2025-07-02 cs.CL cs.AI

Akram Mustafa , Usman Naseem , Mostafa Rahimi Azghadi This is my paper
classification cs.CLcs.AI
keywords ICD-10codingclinicallargelanguagemodelsreasoningLLMsMIMIC-IVcTAKESdischargesummariesF1score
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish how close current large language models are to reliably assigning ICD-10 diagnosis codes from hospital discharge summaries. Using 1,500 MIMIC-IV summaries condensed by cTAKES and a fixed coder-style prompt, it reports that none of 11 models reaches an F1 score above 57% on primary or all-diagnosis classification at code levels 3, 4, or 5. It also claims that reasoning-classified models beat non-reasoning models on average, that Gemini 2.5 Pro performs best overall, and that accuracy declines as codes become more specific. A sympathetic reader would care because these numbers mark the current gap between LLM assistance and fully automated clinical coding.

What carries the argument

The central object is the ICD-10 hierarchy itself, treated as a five-level taxonomy and used as an evaluation ladder: the same predictions are scored at Level 3 (category), Level 4 (subcategory), and Level 5 (most specific code). The argument runs on three devices: cTAKES tokenization to reduce each discharge summary to affirmed or negated medical concepts; a fixed coder-style prompt demanding '#Primary#' and '#Secondary#' answers in ###.### format; and macro F1 computed per code at each level for both primary and all-diagnosis settings. This lets the paper attribute differences in performance to code granularity and to the reasoning/non-reasoning model dichotomy.

What would settle it

Rerun the benchmark with a single base model prompted both with and without explicit reasoning instructions on the same 1,500 summaries; if the reasoning prompt does not raise F1 at Levels 3-5, the paper's claim that reasoning drives the advantage is falsified. Also, deduplicating the per-code samples and recomputing aggregate F1 would show whether the 57% ceiling is an artifact of overlapping documents.

Watch

Extended reading notes

Core claim

On its own terms, the central discovery is a performance ceiling: after extracting medical concepts with cTAKES and asking 11 LLMs to act as clinical coders on 1,500 MIMIC-IV discharge summaries, no model achieved an F1 score above 57% for ICD-10 classification at levels 3, 4, or 5, for either primary or all diagnoses. The best single result was 56.5% for GPT o3 Mini on I25 at Level 3 primary diagnosis, and the best overall model was Gemini 2.5 Pro, whose all-diagnosis F1 reached 39% at Level 3. Chronic ischemic heart disease (I25) was the most accurately classified code, while Y92, Z51, and I13 hovered near zero as primary diagnoses. The paper reads this as evidence that LLMs can support but not replace human coders, and that structured reasoning helps mainly at coarser granularity.

Load-bearing premise

The load-bearing premise is that the reasoning/non-reasoning labels isolate reasoning ability and that the 150 summaries sampled per code are independent documents; the paper does not run an ablation that fixes the base model, and it does not state a deduplication step.

Editorial extensions

If this is right

  • If the ceiling is real, health systems should treat LLMs as assistive tools rather than autonomous coders for the foreseeable future.
  • Because F1 falls from Level 3 to Level 5, broad category grouping or triage is a more realistic short-term use than precise code assignment.
  • The per-code variation means coding benchmarks should report code-level scores; aggregates hide that I25 is tractable while Y92 and Z51 are effectively undeciperable by these models.
  • Reasoning-oriented models leading on average gives hospitals and vendors a concrete selection signal when choosing models for clinical NLP pilots.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper, a controlled test holding the base model fixed while adding or removing a reasoning prompt would separate 'reasoning helps' from 'bigger or newer models help'; the authors' grouping mixes family, size, provider, and version.
  • Beyond the paper, the near-zero scores for Z51 and Y92 suggest that injecting code definitions, inclusion or exclusion notes, or structured encounter data into the prompt, rather than only cTAKES tokens, could move those codes off the floor.
  • Beyond the paper, comparing cTAKES-tokenized inputs with raw summaries on the same models would isolate how much information the preprocessing step discards; the current design cannot separate tokenization loss from model limitation.
  • Beyond the paper, the per-code sampling of 150 summaries may select overlapping documents; a deduplicated split would give independent per-code estimates of the ceiling.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. This paper benchmarks 11 large language models (LLMs) on hierarchical ICD-10 code classification using 1,500 discharge summaries sampled from MIMIC-IV. The authors preprocess the summaries with cTAKES to extract clinical tokens, prompt the models with a standardized coder-style prompt, and evaluate F1 scores for primary and all-diagnoses classification at ICD-10 levels 3, 4, and 5. The main reported findings are that no model reaches an F1 above 57%, performance declines as code granularity increases, reasoning-classified models on average outperform non-reasoning models, and Gemini 2.5 Pro performs best overall. The paper concludes that current off-the-shelf LLMs are not yet suitable for fully automated clinical coding but may support human coders.

Significance. If the descriptive findings hold, this is a useful empirical benchmark showing that state-of-the-art LLMs still fall short of high-accuracy ICD-10 coding, and the detailed per-code and per-level tables in the appendix are a valuable resource. The study also highlights which clinical codes are particularly difficult for LLMs. However, the central comparative claim about reasoning capabilities is not convincingly supported because the comparison is confounded by model identity, scale, and provider, and no statistical significance testing is provided. The paper does not release code or data, which limits reproducibility, but the extensive result tables provide transparency for the descriptive results.

major comments (4)
  1. [§2.1] The dataset sampling procedure is underspecified regarding duplicate discharge summaries. The paper samples 150 summaries for each of the top 10 ICD-10 codes, and because a single discharge summary can have multiple ICD-10 codes, the same summary can appear in more than one code-specific sample. The authors do not state whether such overlaps were removed or accounted for, so the claimed total of 1,500 clinical documents may not represent 1,500 unique summaries. This affects the independence of the evaluation units and the validity of the aggregate F1 scores; the authors should clarify whether duplicates were de-duplicated and, if not, how the current numbers should be interpreted.
  2. [§2.3 and §3.3 (Table 2, Figures 3–4)] The main comparative claim that reasoning models outperform non-reasoning models is not established by the current design. The reasoning and non-reasoning groups differ simultaneously in model family, parameter scale, provider, version, and access platform; there is no controlled comparison that holds the base model fixed while toggling reasoning capability (e.g., DeepSeek Chat vs. DeepSeek Reasoner differ in both base model and reasoning mode, and Gemini 2.0 Flash vs. Gemini 2.5 Pro differ in generation and size as well as reasoning flag). No confidence intervals, significance tests, or repeated-run variance are reported, so the observed gaps of 3.5–4 percentage points at Level 3 may be within noise, especially given the large per-code variability shown in the appendix tables (e.g., GPT o3 Mini scores 0% on Z79 primary while DeepSeek Reasoner scores about 20%). Therefore the statements in the abstract and conclusion that reasoning models 'generally outperformed' non-reasoning models are not supported by this experimental design.
  3. [§2.5 and §3.1–3.3] The F1 aggregation procedure is not specified. It is unclear whether the overall F1 scores in Table 3 and Figures 3–4 are computed as macro-averages over the 10 per-code F1 scores (which would be implied by the appendix tables) or as micro-averages over pooled true positives, false positives, and false negatives. These two approaches yield different results under class imbalance, which is present here because the selected codes have very different frequencies. Additionally, the denominators used for the 'all diagnoses' task (total number of true codes, predicted codes, and matched codes across all documents) are not stated. Without this information, the central descriptive numbers, including the 57% ceiling and the 26% vs. 22% gap in Figure 3, cannot be reproduced or precisely interpreted.
  4. [§2.4 and §2.6] The evaluation does not report decoding parameters (temperature, max tokens, top-p) or explain how the prompt handles model-internal reasoning. The single prompt asks models to 'provide the answers directly as simple as possible,' but it does not instruct non-reasoning models to suppress chain-of-thought, and many models in the non-reasoning group may still perform implicit reasoning by default. Conversely, some reasoning models may produce verbose rationales that affect parsing. The paper should describe the decoding settings and the parsing procedure for extracting ICD-10 codes from the '#Primary#' and '#Secondary#' sections, including how invalid or out-of-range codes were handled, because these implementation details directly affect the reported F1 scores and the comparison between model groups.
minor comments (6)
  1. [Abstract and Table 9] The abstract states that no model achieved an F1 score above 57%, and the per-code Table 9 shows GPT o3 Mini at 56.5% for Level 3 primary I25, which is just below the stated ceiling; consider reporting the exact maximum value to avoid ambiguity.
  2. [Table 1] Y92 ('Place of occurrence of the external cause') is not a diagnosis in the usual clinical sense, yet the paper treats it as one of the top-10 diagnosis codes; a brief justification of its inclusion would help readers interpret the low F1 scores for this code.
  3. [Appendix Tables 4–13] There are formatting inconsistencies in the appendix tables, such as missing percentage signs (e.g., Table 5 shows '11%' in a column that otherwise uses decimal notation like '11.0%') and inconsistent capitalization of model names (e.g., 'Deepseek' vs. 'DeepSeek'); please standardize the formatting.
  4. [Figures 5–14] The appendix charts are difficult to read because the axis labels do not contain numeric values and the model ordering varies across charts; consider using labeled bar charts or annotating the actual F1 values on the bars.
  5. [Discussion, §4] The claim that 'a significant contribution of this study is the novel integration of cTAKES with LLMs for ICD-10 classification, a combination not previously explored in the literature' is overstated, since cTAKES is a well-established tool and has been combined with machine-learning classifiers in prior work; the novelty claim should be qualified or supported by a more targeted citation.
  6. [Figure 2 caption] The word 'Experemental' in the Figure 2 caption should be 'Experimental'; also check reference [5] for a formatting error in the author name ('Re´ ategui' should be 'Reátegui').

Circularity Check

0 steps flagged · score 1.0 of 10

Empirical benchmark is self-contained; the reasoning comparison is an observed between-model result, and the authors' self-citations are non-load-bearing framing.

full rationale

This paper is an empirical benchmark with no fitted parameters and no derivation chain that could reduce to its own inputs. The central claims—that no model exceeded 57% F1, that performance declines as ICD-10 granularity increases, and that reasoning-labelled models have higher average F1 than non-reasoning-labelled models—are directly reported measurements from Table 3 and the per-code appendix tables. The reasoning/non-reasoning grouping is taken from vendor designations in Section 2.3 and Table 2, not derived from the outcomes, so the comparison is not circular by construction. It may be confounded by model family, scale, provider, and prompt behavior, but confounding is a validity concern, not circularity under the stated criteria. The self-citations (refs [6], [19], [20], [29]) are used for framing, prior context, and metric definitions; none of them is invoked as evidence that forces the measured F1 values, and the paper does not rely on a uniqueness theorem or on an ansatz imported from prior work. The paper even acknowledges its own sampling limitations in Section 4, noting that 'the primary code selection and the limited sample size of 1,500 discharge summaries may still introduce selection bias and affect generalizability.' No step in the paper equates a prediction with a fitted input or renames a known result as a derivation. Therefore no circular step reaches the threshold requiring a quotation-backed reduction.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

No fitted parameters or invented entities; the study is an empirical benchmark. The main ledger entries are domain assumptions about label quality, cTAKES fidelity, output parsing, and the interpretability of the reasoning/non-reasoning comparison.

assumptions (4)
  • domain assumption MIMIC-IV diagnostic labels are gold-standard ground truth.
    Used as reference labels for F1 computation without auditing; stated in Discussion and Ethical Compliance, Sections 2.1 and 4.
  • domain assumption cTAKES tokenization preserves all information required for ICD-10 coding decisions.
    The preprocessing reconstructs summaries from only cTAKES-extracted concept tokens; if context is lost, observed low F1 is an artifact of the pipeline rather than the LLMs. Section 2.2.
  • domain assumption Model outputs can be reliably parsed into ICD-10 codes.
    Authors parse '#Primary#' and '#Secondary#' sections; parsing failures, hallucinations, or format violations are not reported. Section 2.6.
  • ad hoc to paper Reasoning vs non-reasoning is a valid controlled experimental factor.
    Models differ by provider, size, family, and version; no ablation or matched-pair design isolates reasoning capability. Sections 2.3 and 3.3.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Evaluating Hierarchical Clinical Document Classification Using Reasoning-Based LLMs." pith.science (2026). https://pith.science/paper/GLXLY5VX

@misc{pith2026250703001,
  author       = {Pith},
  title        = {Pith review of: Evaluating Hierarchical Clinical Document Classification Using Reasoning-Based LLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GLXLY5VX}},
  note         = {Machine review of arXiv:2507.03001}
}
read the original abstract

This study evaluates how well large language models (LLMs) can classify ICD-10 codes from hospital discharge summaries, a critical but error-prone task in healthcare. Using 1,500 summaries from the MIMIC-IV dataset and focusing on the 10 most frequent ICD-10 codes, the study tested 11 LLMs, including models with and without structured reasoning capabilities. Medical terms were extracted using a clinical NLP tool (cTAKES), and models were prompted in a consistent, coder-like format. None of the models achieved an F1 score above 57%, with performance dropping as code specificity increased. Reasoning-based models generally outperformed non-reasoning ones, with Gemini 2.5 Pro performing best overall. Some codes, such as those related to chronic heart disease, were classified more accurately than others. The findings suggest that while LLMs can assist human coders, they are not yet reliable enough for full automation. Future work should explore hybrid methods, domain-specific model training, and the use of structured clinical data.

Figures

Figures reproduced from arXiv: 2507.03001 by the authors.

Figure 1
Figure 1. ICD-10 Code Hierarchy (Levels 1–5): This diagram illustrates the five-level [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. The Workflow used for submitting discharge summaries to LLMs, extracting [PITH_FULL_IMAGE:figures/full_fig_p010_2.png] view at source ↗
Figure 3
Figure 3. Comparison of average F1 Scores by Model Reasoning Type (All Diagnoses). [PITH_FULL_IMAGE:figures/full_fig_p014_3.png] view at source ↗
Figures from the paper (11 more)
Figure 4
Figure 4. Figure 4: Comparison of F1 Scores by Model Reasoning Type (Primary Diagnoses). [PITH_FULL_IMAGE:figures/full_fig_p015_4.png]
Figure 5
Figure 5. Figure 5: F1 scores for ICD-10 code A41, by LLM, across ICD-10 levels 3 to 5 [PITH_FULL_IMAGE:figures/full_fig_p029_5.png]
Figure 6
Figure 6. Figure 6: F1 scores for ICD-10 code E78, by LLM, across ICD-10 levels 3 to 5 [PITH_FULL_IMAGE:figures/full_fig_p029_6.png]
Figure 7
Figure 7. Figure 7: F1 scores for ICD-10 code I10, by LLM, across ICD-10 levels 3 to 5 [PITH_FULL_IMAGE:figures/full_fig_p030_7.png]
Figure 8
Figure 8. Figure 8: F1 scores for ICD-10 code I13, by LLM, across ICD-10 levels 3 to 5 [PITH_FULL_IMAGE:figures/full_fig_p030_8.png]
Figure 9
Figure 9. Figure 9: F1 scores for ICD-10 code I21, by LLM, across ICD-10 levels 3 to 5 [PITH_FULL_IMAGE:figures/full_fig_p031_9.png]
Figure 10
Figure 10. Figure 10: F1 scores for ICD-10 code I25, by LLM, across ICD-10 levels 3 to 5 [PITH_FULL_IMAGE:figures/full_fig_p031_10.png]
Figure 11
Figure 11. Figure 11: F1 scores for ICD-10 code Y92, by LLM, across ICD-10 levels 3 to 5 [PITH_FULL_IMAGE:figures/full_fig_p032_11.png]
Figure 12
Figure 12. Figure 12: F1 scores for ICD-10 code Z51, by LLM, across ICD-10 levels 3 to 5 [PITH_FULL_IMAGE:figures/full_fig_p032_12.png]
Figure 13
Figure 13. Figure 13: F1 scores for ICD-10 code Z79, by LLM, across ICD-10 levels 3 to 5 [PITH_FULL_IMAGE:figures/full_fig_p033_13.png]
Figure 14
Figure 14. Figure 14: F1 scores for ICD-10 code Z87, by LLM, across ICD-10 levels 3 to 5 [PITH_FULL_IMAGE:figures/full_fig_p033_14.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

29 extracted references · 21 canonical work pages

  1. [1]

    Federal Institute for Drugs and Medical Devices (BfArM), Structure of tabular list - icd, https://www.bfarm.de/EN/Code-systems/ Classifications/ICD/ICD-10-GM/Tabular-list/structure_ tab-list.html, accessed: 2025-04-28 (2025)

    work page 2025

  2. [2]

    Dorsey, F

    S. Dorsey, F. Ahmed, Developments in distinguishing secondary vascular headache from primary headache disorders in clinical practice, Expert Review of Neurotherapeutics 21 (12) (2021) 1357–1369

    work page 2021

  3. [3]

    Henderson, J

    T. Henderson, J. Shepheard, V. Sundararajan, Quality of diagnosis and procedure coding in icd-10 administrative data, Medical care 44 (11) (2006) 1011–1019

    work page 2006

  4. [4]

    Y. Sun, L. Sang, D. Wu, S. He, Y. Chen, H. Duan, H. Chen, X. Lu, Enhanced icd-10 code assignment of clinical texts: A summarization- based approach, Artificial Intelligence in Medicine 156 (2024) 102967

    work page 2024

  5. [5]

    Re´ ategui, S

    R. Re´ ategui, S. Ratt´ e, Comparison of metamap and ctakes for entity ex- traction in clinical notes, BMC medical informatics and decision making 18 (2018) 13–19

    work page 2018

  6. [6]

    Mustafa, M

    A. Mustafa, M. Rahimi Azghadi, Clustered automated machine learning (caml) model for clinical coding multi-label classification, International Journal of Machine Learning and Cybernetics (2024) 1–23

    work page 2024

  7. [7]

    Panny, H

    A. Panny, H. Hegde, I. Glurich, F. A. Scannapieco, J. G. Vedre, J. J. VanWormer, J. Miecznikowski, A. Acharya, A methodological approach to validate pneumonia encounters from radiology reports using natural language processing, Methods of information in medicine 61 (01/02) (2022) 038–045

    work page 2022

  8. [8]

    R. Kaur, J. A. Ginige, O. Obst, Ai-based icd coding and classification approaches using discharge summaries: A systematic literature review, Expert Systems with Applications 213 (2023) 118997

    work page 2023

Show all 29 references
  1. [9]

    Hossain, R

    E. Hossain, R. Rana, N. Higgins, J. Soar, P. D. Barua, A. R. Pisani, K. Turner, Natural language processing in electronic health records in relation to healthcare decision-making: a systematic review, Computers in biology and medicine 155 (2023) 106649. 19

    2023

  2. [10]

    B. Bi, L. Liu, O. Perez-Concha, Adapting large language models for au- tomated summarisation of electronic medical records in clinical coding, in: Health. Innovation. Community: It Starts With Us, IOS Press, 2024, pp. 24–29

    2024

  3. [11]

    Boggavarapu, V

    L. Boggavarapu, V. Srivastava, A. M. Varanasi, Y. Lu, R. Bhaumik, Evaluating enhanced llms for precise mental health diagnosis from clin- ical notes, medRxiv (2024) 2024–12

    2024

  4. [12]

    Lehman, E

    E. Lehman, E. Hernandez, D. Mahajan, J. Wulff, M. J. Smith, Z. Ziegler, D. Nadler, P. Szolovits, A. Johnson, E. Alsentzer, Do we still need clini- cal language models?, in: Conference on health, inference, and learning, PMLR, 2023, pp. 578–597

    2023

  5. [13]

    D. Yoon, C. Han, D. W. Kim, S. Kim, S. Bae, J. A. Ryu, Y. Choi, Re- defining health care data interoperability: empirical exploration of large language models in information exchange, Journal of Medical Internet Research 26 (2024) e56614

    2024

  6. [14]

    Savage, A

    T. Savage, A. Nayak, R. Gallo, E. Rangan, J. H. Chen, Diagnostic reasoning prompts reveal the potential for large language model inter- pretability in medicine, NPJ Digital Medicine 7 (1) (2024) 20

    2024

  7. [15]

    P. Qiu, C. Wu, S. Liu, W. Zhao, Z. Chen, H. Gu, C. Peng, Y. Zhang, Y. Wang, W. Xie, Quantifying the reasoning abilities of llms on real- world clinical cases, arXiv preprint arXiv:2503.04691 (2025)

    2025 arXiv

  8. [16]

    D. Wang, S. Zhang, Large language models in medical and healthcare fields: applications, advances, and challenges, Artificial Intelligence Re- view 57 (11) (2024) 299

    2024

  9. [17]

    Sonoda, R

    Y. Sonoda, R. Kurokawa, A. Hagiwara, Y. Asari, T. Fukushima, J. Kan- zawa, W. Gonoi, O. Abe, Structured clinical reasoning prompt enhances llm’s diagnostic capabilities in diagnosis please quiz cases, Japanese Journal of Radiology 43 (4) (2025) 586–592

    2025

  10. [18]

    J. Chen, Z. Cai, K. Ji, X. Wang, W. Liu, R. Wang, J. Hou, B. Wang, Huatuogpt-o1, towards medical complex reasoning with llms, arXiv preprint arXiv:2412.18925 (2024). 20

    2024 arXiv

  11. [19]

    Mustafa, U

    A. Mustafa, U. Naseem, M. R. Azghadi, Can reasoning llms en- hance clinical document classification?, arXiv preprint arXiv:2504.08040 (2025)

    2025

  12. [20]

    Mustafa, U

    A. Mustafa, U. Naseem, M. R. Azghadi, Large language models vs hu- man for classifying clinical documents, International Journal of Medical Informatics (2025) 105800

    2025

  13. [21]

    Naliyatthaliyazchayil, R

    P. Naliyatthaliyazchayil, R. Muthyala, S. Purkayastha, J. W. Gichoya, Evaluating reasoning capabilities of large language models for medi- cal coding and hospital readmission risk stratification with zero shot prompting, Available at SSRN 5172464 (2025)

    2025

  14. [22]

    E. Bai, X. Luo, Z. Zhang, K. Adelgais, H. Ali, J. Finkelstein, J. Kutzin, Assessment and integration of large language models for automated elec- tronic health record documentation in emergency medical services, Jour- nal of Medical Systems 49 (1) (2025) 1–19

    2025

  15. [23]

    A. Zou, W. Yu, H. Zhang, K. Ma, D. Cai, Z. Zhang, H. Zhao, D. Yu, Docbench: A benchmark for evaluating llm-based document reading systems, arXiv preprint arXiv:2407.10701 (2024)

    2024 arXiv

  16. [24]

    M. Wang, L. Chen, C. Fu, S. Liao, X. Zhang, B. Wu, H. Yu, N. Xu, L. Zhang, R. Luo, et al., Leave no document behind: Bench- marking long-context llms with extended multi-doc qa, arXiv preprint arXiv:2406.17419 (2024)

    2024 arXiv

  17. [25]

    S. Dai, W. Liu, Y. Zhou, L. Pang, R. Ruan, G. Wang, Z. Dong, J. Xu, J.-R. Wen, Cocktail: A comprehensive information retrieval benchmark with llm-generated documents integration, arXiv preprint arXiv:2405.16546 (2024)

    2024 arXiv

  18. [26]

    Luo, Y.-F

    L. Luo, Y.-F. Li, G. Haffari, S. Pan, Reasoning on graphs: Faith- ful and interpretable large language model reasoning, arXiv preprint arXiv:2310.01061 (2023)

    2023 arXiv

  19. [27]

    Barabucci, V

    G. Barabucci, V. Shia, E. Chu, B. Harack, K. Laskowski, N. Fu, Combin- ing multiple large language models improves diagnostic accuracy, NEJM AI 1 (11) (2024) AIcs2400502. 21

    2024

  20. [28]

    W. Ma, J. He, C. Snell, T. Griggs, S. Min, M. Zaharia, Reasoning mod- els can be effective without thinking, arXiv preprint arXiv:2504.09858 (2025)

    2025 arXiv

  21. [29]

    Mustafa, M

    A. Mustafa, M. Rahimi Azghadi, Automated machine learning for healthcare and clinical notes analysis, Computers 10 (2) (2021) 24. 22 A. Appendix A.1. Tables Table 4: F1 Scores for A41 Primary and All Diagnoses (Levels 3-5) by LLM Model Level 3 Primary Level 4 Primary Level 5 P...

    2021

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.