Pith. sign in

REVIEW 4 major objections 4 minor 37 references

RESPClinBench: Benchmarking Multimodal Clinical Decision-Making and Longitudinal Disease Management in Respiratory Specialty Care

T0 review · 4 major / 4 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read RESPClinBench, a real-world benchmark of 623 respiratory cases, finds that no single large language model leads both pulmonary nodule management and longitudinal COPD care, with top scores near 71/100 and frequent safety flags.

desk verdict Solid benchmark concept, but the headline numbers are provisional until the LLM judge is validated against human clinicians. read the letter →

arxiv 2608.04514 v2 pith:43TFNR4Z submitted 2026-08-05 cs.CL

Mouxiao Bian , Zhi Chen , Ruiyao Chen , Lu Lu , Hengrui Liang , Chaoyi Huang , Yiluo Lin , Jingru Ding
show 3 more authors
Yun Zhong Yueming Su Jie Xu
This is my paper · ORCID
classification cs.CL
keywords RESPClinBenchrespiratoryclinicaldecision-makinglargelanguagemodelevaluationCOPDlongitudinalmanagementpulmonarynoduleassessmentmultimodalLLMsafetyflagsLLM-as-a-Judge
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

RESPClinBench is a new benchmark that evaluates large language models on real respiratory clinical work rather than static exam questions. It contains 623 cases: 427 longitudinal COPD management cases and 196 pulmonary nodule cases that pair chest CT images with clinical data. Seven models generated 4,361 responses, and the mean final score was 68.58 out of 100, with no model leading both datasets: Qwen3.5-397B-A17B led nodule management at 72.48, while Qwen3.6-27B led COPD management at 71.11. The paper also reports that 31.85% of nodule responses contained imaging hallucinations, 8.16% carried serious medical risk, and 26.93% of COPD responses carried medication-safety risk. The authors argue that respiratory competence is task-dependent and that evaluation should combine explicit clinical-action coverage, holistic rubric scoring, and independent safety flags.

What carries the argument

The central mechanism is a dual-component automated scoring framework. Each case carries an expert reference answer decomposed into atomic clinical-action points; the first score is the fraction of those points the model response explicitly covers. The second score comes from a rubric-based LLM-as-a-Judge, rated on dataset-specific dimensions such as lesion-nature assessment, longitudinal consistency, medication safety, and clinical usability. The final score is the arithmetic mean of the two components, and scenario-specific safety flags (imaging hallucination, overdiagnosis, medication-safety risk, serious medical risk) are generated separately rather than folded into the total. This design is what lets the paper separate sounding right from doing the required clinical work.

What would settle it

Take a random sample of 100 RESPClinBench responses, 50 from each dataset, have two independent respiratory physicians score them on the same atomic action points and safety flags, then compare physician consensus with the automated judge; if agreement is poor or re-scoring changes which model ranks first, the central claim that the benchmark measures clinically meaningful task-specific competence would need revision.

Watch

Extended reading notes

Core claim

The central claim is that no single model is the best respiratory decision-maker; performance is tied to the specific workflow. On the multimodal PNBIM dataset, Qwen3.5-397B-A17B ranked first with 72.48, while on the longitudinal AECOPD-PIM dataset, Qwen3.6-27B ranked first with 71.11, and cross-dataset rank correlation was moderate but not statistically significant (Spearman rho=0.643, P=0.119). A persistent gap between rubric-based quality and atomic-action recall (19.36 points in PNBIM, 32.45 points in AECOPD-PIM) shows that fluent, coherent answers frequently omit concrete management actions. Safety flags were common even among high-scoring models, and serious-risk responses scored about 17 to 20 points lower, indicating that the scoring captures part of the risk. The benchmark's contribution is a reproducible evaluation design that separates what a model says from what a model actually does in a clinical pathway.

Load-bearing premise

The load-bearing premise is that the automated judge scores responses and safety flags accurately enough that the reported rankings and risk rates reflect clinical quality, and the paper reports no human-agreement measurement.

Editorial extensions

If this is right

  • Model selection for respiratory AI should be workflow-specific, since the best pulmonary-nodule model was not the best COPD-management model.
  • Holistic quality scores should not be interpreted as evidence that required clinical actions were performed: action coverage and rubric quality correlated weakly, and the gap exceeded 30 points in 55.07% of COPD responses.
  • Independent safety flags are necessary because 31.85% of nodule responses included imaging hallucinations and 26.93% of COPD responses included medication-safety risk, even among leading models.
  • Aggregate scores are insufficient for deployment decisions; dimension profiles and risk flags should accompany any reported total.
  • No evaluated model is near autonomous use: the highest mean score was 71.22, and serious-risk responses scored 17 to 20 points lower than unflagged ones.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An editorial inference: because the gap between the top two models is only 1.10 points, even a small judge bias could reorder the leaderboard, which the paper's own limitation discussion leaves open.
  • An editorial inference: if the 31.85% imaging-hallucination rate holds on other CT cohorts, current multimodal LLMs are not yet safe for image-grounded nodule triage without a separate vision-grounding check.
  • An editorial inference: the same dual-component evaluation could be applied to other longitudinal specialties, where fluent-but-action-incomplete responses are likely to be a common failure mode.
  • An editorial inference: rescoring a subset of the 623 cases with independent clinicians would show whether the 68.58 mean reflects clinically acceptable performance or is partly an artifact of automated judging.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper introduces RESPClinBench, a scenario-based benchmark for evaluating LLMs in respiratory specialty care, with two datasets: AECOPD-PIM (427 longitudinal COPD management cases, text and structured data) and PNBIM (196 pulmonary nodule cases combining chest CT with clinical data). Seven models were evaluated through standardized API inference, and responses were scored by an automated pipeline that averages atomic-action recall with an LLM-as-a-Judge rubric score, supplemented by safety flags. The headline findings are that Qwen3.6-27B ranks first overall, Qwen3.5-397B-A17B leads PNBIM, Qwen3.6-27B leads AECOPD-PIM, no model leads both datasets, and that safety risks (e.g., 31.85% imaging hallucination in PNBIM) are frequent.

Significance. If the reported measurements are trustworthy, the benchmark addresses a genuine gap: most medical LLM benchmarks rely on examination-style questions and ignore longitudinal, multimodal, and safety-critical aspects of specialty care. The paper has several strengths: cases were adapted from real-world data and curated by attending physicians with senior specialist adjudication; inference was standardized (temperature 0, fixed token limit, independent requests); and the statistical analysis is detailed, including bootstrap confidence intervals, Friedman tests, and Holm-adjusted pairwise comparisons. The benchmark is publicly available on MedBench, and the authors provide dimension-level and safety analyses that go beyond a single aggregate score. The central message that respiratory competence is task-dependent is plausible and clinically relevant. However, the entire measurement pipeline rests on an unvalidated LLM judge, with no human agreement data, no judge-model identity, and no specification of the atomic-action matching procedure; until these are supplied, the rankings and safety rates remain provisional rather than established.

major comments (4)
  1. [Sections 2.7, 2.8, and 4.5] The central measurement premise—that the automated LLM-as-a-Judge pipeline correctly scores atomic-action coverage, rubric quality, and safety flags—is not validated against human judgment. The paper explicitly concedes in Section 4.5 that "the evaluation agent, although structured, can still introduce systematic bias; human validation remains important," yet no human agreement coefficients, judge prompts, judge model identity, or calibration analyses are reported. Since the headline rankings (including the 1.10-point margin between first and second place) and the safety rates (31.85% imaging hallucination, 8.16% serious medical risk in PNBIM; 26.93% medication-safety risk in AECOPD-PIM) are all produced by this pipeline, a modest systematic bias in the judge could alter the rankings and risk estimates. I recommend that the authors report a human validation study using a random sample of responses, with agreement statistics (e.g., Cohen's kappa or precision/recall against expert annotations) for the atomic-action matching, rubric scores, and each safety flag, and that they disclose the judge model and prompts.
  2. [Section 2.7] The method for atomic-action matching is not specified. The paper states that "every prespecified answer point was assessed independently and assigned a binary value of 1 when explicitly covered and 0 when absent," but it does not describe whether this is exact string matching, semantic similarity, LLM-based judgment, or a rule-based procedure. This matters because atomic-action recall contributes 50% of the final score and is highly correlated with the composite score (Spearman rho = 0.926 in PNBIM and 0.912 in AECOPD-PIM), so any error in this component propagates directly into the main results. The authors should specify the matching algorithm, including thresholds or prompts, and should provide evidence of its reliability (e.g., human-judge agreement on a subset).
  3. [Section 4.5] The statement "Removing a single unsupported model series and recalculating all results ensured that rankings and risk estimates reflected a consistent seven-model comparison" introduces a substantial data-exclusion step that is not described anywhere in the Methods. Which model series was removed, why was it "unsupported," and at what point in the analysis was it removed? Because the reported rankings and risk estimates are explicitly the result of this removal, the exclusion must be documented transparently in the Methods section; otherwise, readers cannot assess whether the exclusion was pre-specified or whether it materially influenced the conclusions.
  4. [Section 2.8] The safety flag definitions are qualitative and the generation process is underspecified. For example, "serious medical risk" is defined as "recommendations with plausible potential for major harm," and "overdiagnosis" as "excessive diagnostic certainty or disproportionate escalation beyond the available evidence." These criteria are open to subjective interpretation, and the paper does not state whether the flags are produced by the same LLM judge, by separate rules, or by a combination. Given that the safety rates are headline findings and are used to draw clinical conclusions, the authors should operationalize each flag with concrete criteria or examples and validate the flag annotations against expert clinicians, including reporting inter-annotator agreement.
minor comments (4)
  1. [Section 3.1] The cross-dataset Spearman correlation is reported as rho = 0.643 with P = 0.119 based on only seven models. The non-significant result should be interpreted with caution, and the text "performance in multimodal nodule management did not fully predict performance in longitudinal COPD care" could be strengthened by acknowledging the low statistical power.
  2. [Section 3.3] Figure citations are inconsistent: the text refers to "Figure 4A and Figure 4C" for PNBIM and "Figure 4 and Figure 4D" for AECOPD-PIM; these should be made uniform and accurate.
  3. [Section 4.1] The claim that "fluent presentation could not compensate for incomplete action coverage" is imprecise: since the final score is an arithmetic mean of atomic-action recall and the rubric score, a high rubric score can compensate for low atomic-action recall by exactly 50%. The intended meaning is presumably that the equal-weight design prevents full compensation, but the phrasing should be corrected to avoid overstating the result.
  4. [Full text header] The title in the running header appears as "BenchmarkingMultimodal" without a space; this is a typographical error that should be corrected.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the benchmark scores are empirical measurements against physician-authored reference standards, not derivations that reduce to their own inputs.

full rationale

The paper's central claims are empirical measurement results: 623 respiratory cases with physician-authored reference answers and atomic clinical-action points (Sections 2.2 and 2.5), model outputs generated through standardized API inference, and scores computed as the arithmetic mean of atomic-action recall and an LLM-as-a-Judge rubric score (Section 2.7). The final score is a defined aggregation of observed outputs, not a fitted parameter renamed as a prediction. Model rankings, cross-dataset rank stability, and safety-flag rates are all reported as measurements against these external reference standards. No equation in the paper equates a predicted quantity to an input by construction, and no parameter is fit to a subset and then used to predict a closely related quantity. The use of an LLM judge without reported human calibration is a validity limitation that the paper explicitly acknowledges in Section 4.5 ('the evaluation agent, although structured, can still introduce systematic bias; human validation remains important'), but this is a measurement-validity concern, not circularity: the reference standards themselves are clinician-validated and independent of the tested models. Citations to prior work on evaluator LLMs and physician-validated benchmarks are supporting external evidence, not load-bearing self-citations, and no uniqueness theorem or ansatz is imported from the authors' own prior work. The benchmark is self-contained as an evaluation study, so no circular step is exhibited.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The central scoring results rest on expert-curated reference standards and on an unvalidated LLM judge. No numeric parameters were fitted to data, but the 50/50 composite weighting is a hand-set design choice. The most consequential axioms are domain assumptions about expert ground truth and judge reliability; if the judge assumption fails, the headline safety rates are not trustworthy. No invented entities are introduced.

free parameters (1)
  • Composite-score weighting (50% atomic-action recall, 50% judge rubric) = 0.5 / 0.5
    Hand-set in Section 2.7; the two components correlate only weakly, so the chosen weights materially affect final rankings and the reported component gaps.
assumptions (4)
  • domain assumption Expert reference answers and atomic clinical-action lists are a complete and correct ground truth for each case.
    Used as the scoring standard in Sections 2.2 and 2.5; rests on the clinical judgment of three attending physicians and one senior specialist rather than on validated outcome data.
  • domain assumption The LLM-as-a-Judge produces valid quality scores and safety flags without requiring human adjudication.
    Invoked in Sections 2.7 and 2.8; the paper notes in Section 4.5 that the evaluation agent may carry systematic bias and that human validation is still needed.
  • domain assumption De-identified source records, after standardization into benchmark cases, preserve the clinical information needed for the task.
    Assumed in Section 2.2; Section 4.5 concedes some histories are standardized and more coherent than routine records.
  • domain assumption A single selected chest CT image is an adequate representation of the radiological input for each nodule case.
    Assumed in Section 2.4; Section 4.5 notes PNBIM uses selected CT images rather than the complete interactive image stack available to radiologists.

how reviews work

0 comments
Cite this review

Pith. "Pith review of RESPClinBench: Benchmarking Multimodal Clinical Decision-Making and Longitudinal Disease Management in Respiratory Specialty Care." pith.science (2026). https://pith.science/paper/43TFNR4Z

@misc{pith2026260804514,
  author       = {Pith},
  title        = {Pith review of: RESPClinBench: Benchmarking Multimodal Clinical Decision-Making and Longitudinal Disease Management in Respiratory Specialty Care},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/43TFNR4Z}},
  note         = {Machine review of arXiv:2608.04514}
}
read the original abstract

Background: Respiratory specialty care requires multimodal interpretation, longitudinal risk assessment, guideline-concordant intervention, and whole-course management, which are poorly represented by examination-oriented medical benchmarks. Objective: To develop RESPClinBench, a real-world scenario-based benchmark for respiratory clinical decision-making, and evaluate seven contemporary large language models across AECOPD-PIM and PNBIM. Methods: RESPClinBench cases were adapted from de-identified respiratory clinical data. Three attending-level respiratory physicians revised cases, reference answers, and atomic clinical-action points, while one senior respiratory specialist performed cross-review and final adjudication. AECOPD-PIM comprised 427 open-ended COPD cases, and PNBIM comprised 196 multimodal pulmonary nodule cases combining chest CT with structured clinical information. Seven models generated 4,361 responses through standardized API inference with temperature 0 and a maximum output length of 8192 tokens. An automated framework calculated the final score as the arithmetic mean of atomic-action recall and rubric-based LLM-as-a-Judge assessment. Results: Across 623 cases, the mean final score was 68.58. Qwen3.6-27B ranked first overall at 71.22, Qwen3.5-397B-A17B led PNBIM at 72.48, and Qwen3.6-27B led AECOPD-PIM at 71.11. Imaging hallucination and serious medical risk occurred in 31.85% and 8.16% of PNBIM responses; medication-safety risk and serious medical risk occurred in 26.93% and 1.44% of AECOPD-PIM responses. Conclusions: RESPClinBench identifies task-specific limitations in multimodal pulmonary nodule assessment and longitudinal COPD management. Combining explicit clinical-action coverage, holistic evaluation, and independent safety flags provides a clinically grounded basis for model selection and prospective validation.

Figures

Figures reproduced from arXiv: 2608.04514 by the authors.

Figure 1
Figure 1. Overall and dataset-specific model performance in RESPClinBench. (A) Case-weighted final scores across [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. Divergence between guideline-derived clinical-action coverage and rubric-based clinical quality. (A) Case [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Dimension-level clinical performance across RESPClinBench models. (A) PNBIM rubric dimensions for [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Safety, reliability, and performance impact of scenario-specific risk flags. (A) Model-specific rates of imaging [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

37 extracted references · 1 linked inside Pith

  1. [1]

    Large language models encode clinical knowledge.Nature, 620(7972):172–180, 2023

    Karan Singhal, Shekoofeh Azizi, Tao Tu, S Sara Mahdavi, Jason Wei, Hyung Won Chung, Nathan Scales, Ajay Tanwani, Heather Cole-Lewis, Stephen Pfohl, et al. Large language models encode clinical knowledge.Nature, 620(7972):172–180, 2023

    2023

  2. [2]

    Toward expert-level medical question answering with large language models.Nature medicine, 31(3):943–950, 2025

    Karan Singhal, Tao Tu, Juraj Gottweis, Rory Sayres, Ellery Wulczyn, Mohamed Amin, Le Hou, Kevin Clark, Stephen R Pfohl, Heather Cole-Lewis, et al. Toward expert-level medical question answering with large language models.Nature medicine, 31(3):943–950, 2025

    2025

  3. [3]

    Towards conversational diagnostic artificial intelligence.Nature, 642(8067):442–450, 2025

    Tao Tu, Mike Schaekermann, Anil Palepu, Khaled Saab, Jan Freyberg, Ryutaro Tanno, Amy Wang, Brenna Li, Mohamed Amin, Yong Cheng, et al. Towards conversational diagnostic artificial intelligence.Nature, 642(8067):442–450, 2025

    2025

  4. [4]

    Towards accurate differential diagnosis with large language models.Nature, 642(8067):451–457, 2025

    Daniel McDuff, Mike Schaekermann, Tao Tu, Anil Palepu, Amy Wang, Jake Garrison, Karan Singhal, Yash Sharma, Shekoofeh Azizi, Kavita Kulkarni, et al. Towards accurate differential diagnosis with large language models.Nature, 642(8067):451–457, 2025. 10

    2025

  5. [5]

    Testing and evaluation of health care applications of large language models: a systematic review.Jama, 333(4):319–328, 2025

    Suhana Bedi, Yutong Liu, Lucy Orr-Ewing, Dev Dash, Sanmi Koyejo, Alison Callahan, Jason A Fries, Michael Wornow, Akshay Swaminathan, Lisa Soleymani Lehmann, et al. Testing and evaluation of health care applications of large language models: a systematic review.Jama, 333(4):319–328, 2025

    2025

  6. [6]

    Evaluation and mitigation of the limitations of large language models in clinical decision-making.Nature medicine, 30(9):2613–2622, 2024

    Paul Hager, Friederike Jungmann, Robbie Holland, Kunal Bhagat, Inga Hubrecht, Manuel Knauer, Jakob Vielhauer, Marcus Makowski, Rickmer Braren, Georgios Kaissis, et al. Evaluation and mitigation of the limitations of large language models in clinical decision-making.Nature medicine, 30(9):2613–2622, 2024

    2024

  7. [7]

    Large language model influence on diagnostic reasoning: a randomized clinical trial.JAMA network open, 7(10):e2440969, 2024

    Ethan Goh, Robert Gallo, Jason Hom, Eric Strong, Yingjie Weng, Hannah Kerman, Joséphine A Cool, Zahir Kanjee, Andrew S Parsons, Neera Ahuja, et al. Large language model influence on diagnostic reasoning: a randomized clinical trial.JAMA network open, 7(10):e2440969, 2024

    2024

  8. [8]

    An evaluation framework for clinical use of large language models in patient interaction tasks.Nature medicine, 31(1):77–86, 2025

    Shreya Johri, Jaehwan Jeong, Benjamin A Tran, Daniel I Schlessinger, Shannon Wongvibulsin, Leandra A Barnes, Hong-Yu Zhou, Zhuo Ran Cai, Eliezer M Van Allen, David Kim, et al. An evaluation framework for clinical use of large language models in patient interaction tasks.Nature medicine, 31(1):77–86, 2025

    2025

Show all 37 references
  1. [9]

    Large language model performance and clinical reasoning tasks.JAMA Network Open, 9(4):e264003, 2026

    Arya S Rao, Kaiz P Esmail, Richard S Lee, Sharon Jiang, Bianca Arraiza Carlo, Jasleen Gill, Praneet Khanna, Ezra Kalmowitz, Basile Montagnese, Kimia Heydari, et al. Large language model performance and clinical reasoning tasks.JAMA Network Open, 9(4):e264003, 2026

    2026

  2. [10]

    Limitations of large language models in clinical diagnostic reasoning.JAMA Network Open, 9(4):e264014, 2026

    Mickael Tordjman and Xueyan Mei. Limitations of large language models in clinical diagnostic reasoning.JAMA Network Open, 9(4):e264014, 2026

    2026

  3. [11]

    Quantifying the reasoning abilities of llms on clinical cases.Nature Communica- tions, 16(1):9799, 2025

    Pengcheng Qiu, Chaoyi Wu, Shuyu Liu, Yanjie Fan, Weike Zhao, Zhuoxia Chen, Hongfei Gu, Chuanjin Peng, Ya Zhang, Yanfeng Wang, et al. Quantifying the reasoning abilities of llms on clinical cases.Nature Communica- tions, 16(1):9799, 2025

    2025

  4. [12]

    The evaluation illusion of large language models in medicine.npj Digital Medicine, 8(1):600, 2025

    Monica Agrawal, Irene Y Chen, Freya Gulamali, and Shalmali Joshi. The evaluation illusion of large language models in medicine.npj Digital Medicine, 8(1):600, 2025

    2025

  5. [13]

    Automating expert-level medical reasoning evaluation of large language models

    Shuang Zhou, Wenya Xie, Jiaxi Li, Zaifu Zhan, Meijia Song, Han Yang, Cheyenna Espinoza, Lindsay Welton, Xinnie Mai, Yanwei Jin, et al. Automating expert-level medical reasoning evaluation of large language models. NPJ Digital Medicine, 9(1):34, 2025

    2025

  6. [14]

    Evaluating clinical ai summaries with large language models as judges.npj Digital Medicine, 8(1):640, 2025

    Emma Croxford, Yanjun Gao, Elliot First, Nicholas Pellegrino, Miranda Schnier, John Caskey, Madeline Oguss, Graham Wills, Guanhua Chen, Dmitriy Dligach, et al. Evaluating clinical ai summaries with large language models as judges.npj Digital Medicine, 8(1):640, 2025

    2025

  7. [15]

    Llmeval-med: A real-world clinical benchmark for medical llms with physician validation.arXiv preprint arXiv:2506.04078, 2025

    Ming Zhang, Yujiong Shen, Zelin Li, Huayu Sha, Binze Hu, Yuhui Wang, Chenhao Huang, Shichun Liu, Jingqi Tong, Changhao Jiang, et al. Llmeval-med: A real-world clinical benchmark for medical llms with physician validation.arXiv preprint arXiv:2506.04078, 2025

    2025 arXiv

  8. [16]

    A novel evaluation benchmark for medical llms illuminating safety and effectiveness in clinical domains.npj Digital Medicine, 9(1):91, 2025

    Shirui Wang, Zhihui Tang, Huaxia Yang, Qiuhong Gong, Tiantian Gu, Hongyang Ma, Yongxin Wang, Wubin Sun, Zeliang Lian, Kehang Mao, et al. A novel evaluation benchmark for medical llms illuminating safety and effectiveness in clinical domains.npj Digital Medicine, 9(1):91, 2025

    2025

  9. [17]

    Future-ai: international consensus guideline for trustworthy and deployable artificial intelligence in healthcare.bmj, 388, 2025

    Karim Lekadir, Alejandro F Frangi, Antonio R Porras, Ben Glocker, Celia Cintas, Curtis P Langlotz, Eva Weicken, Folkert W Asselbergs, Fred Prior, Gary S Collins, et al. Future-ai: international consensus guideline for trustworthy and deployable artificial intelligence in healt...

    2025

  10. [18]

    The stard-ai reporting guideline for diagnostic accuracy studies using artificial intelligence.Nature medicine, 31(10):3283–3289, 2025

    Viknesh Sounderajah, Ahmad Guni, Xiaoxuan Liu, Gary S Collins, Alan Karthikesalingam, Sheraz R Markar, Robert M Golub, Alastair K Denniston, Shravya Shetty, David Moher, et al. The stard-ai reporting guideline for diagnostic accuracy studies using artificial intelligence.Natur...

    2025

  11. [19]

    Global strategy for prevention, diagnosis and management of copd: 2026 report

    Global Initiative for Chronic Obstructive Lung Disease. Global strategy for prevention, diagnosis and management of copd: 2026 report. Report, Global Initiative for Chronic Obstructive Lung Disease, Fontana, WI, 2026

    2026

  12. [20]

    Ai in respiratory care: findings from the gold report.Journal of Translational Medicine, 2026

    Ali Al-Waqeerah, Ahmed Bashah, Eslam Ghaleb, Huanyu He, and Lili Gao. Ai in respiratory care: findings from the gold report.Journal of Translational Medicine, 2026

    2026

  13. [21]

    Large language models in respiratory care: Safety and clinical integration.ERJ Open Research, 2026

    Alessandro Porcella, Antonio Fabozzi, Matteo Bonini, and Paolo Palange. Large language models in respiratory care: Safety and clinical integration.ERJ Open Research, 2026

    2026

  14. [22]

    Hayriye Bekta¸ s Aksoy, Selda Günaydın, ¸ Saban Melih ¸ Sim¸ sek, and Ruhsel Cörüt. Evaluation of large language models in a pulmonology outpatient clinic using structured clinical data and chest radiographs: a single-center prospective observational study.Frontiers in Medicin...

    2026

  15. [23]

    Spirollm: Finetuning pretrained llms to understand spirogram time series with clinical validation in copd reporting.PLOS Digital Health, 5(3):e0001300, 2026

    Shuhao Mei, Yongchao Long, Xiaoyu Xiao, Shan Cao, Xiaobo Han, Shijia Geng, Jinbo Sun, Yuxi Zhou, and Shenda Hong. Spirollm: Finetuning pretrained llms to understand spirogram time series with clinical validation in copd reporting.PLOS Digital Health, 5(3):e0001300, 2026. 11

    2026

  16. [24]

    Evaluation of large language models in generating pulmonary nodule follow-up recommendations.European Journal of Radiology Open, 14:100655, 2025

    Junzhe Wen, Wanyue Huang, Huzheng Yan, Jie Sun, Mengshi Dong, Chao Li, and Jie Qin. Evaluation of large language models in generating pulmonary nodule follow-up recommendations.European Journal of Radiology Open, 14:100655, 2025

    2025

  17. [25]

    Assessments of lung nodules by an artificial intelligence chatbot using longitudinal ct images.Cell Reports Medicine, 6(3), 2025

    Yuqiang Mao, Nan Xu, Yanan Wu, Lu Wang, Hongtao Wang, Qianqian He, Tianqi Zhao, Shuangchun Ma, Meihong Zhou, Hongjie Jin, et al. Assessments of lung nodules by an artificial intelligence chatbot using longitudinal ct images.Cell Reports Medicine, 6(3), 2025

    2025

  18. [26]

    Leveraging artificial intelligence as a safety net for incidentally identified lung nodules at a tertiary center.Journal of the American College of Surgeons, 240(4):417, 2025

    Palina Woodhouse, Rafael Paez, Patrick Meyers, Rob J Lentz, Samira Shoajaee, Kenneth Sharp, Nikki Baldi, Fabien Maldonado, and Eric L Grogan. Leveraging artificial intelligence as a safety net for incidentally identified lung nodules at a tertiary center.Journal of the America...

    2025

  19. [27]

    Guidelines for management of incidental pulmonary nodules detected on ct images: from the fleischner society 2017.Radiology, 284(1):228–243, 2017

    Heber MacMahon, David P Naidich, Jin Mo Goo, Kyung Soo Lee, Ann NC Leung, John R Mayo, Atul C Mehta, Yoshiharu Ohno, Charles A Powell, Mathias Prokop, et al. Guidelines for management of incidental pulmonary nodules detected on ct images: from the fleischner society 2017.Radio...

    2017

  20. [28]

    Screening for lung cancer: Chest guideline and expert panel report.Chest, 153(4):954–985, 2018

    Peter J Mazzone, Gerard A Silvestri, Sheena Patel, Jeffrey P Kanne, Linda S Kinsinger, Renda Soylemez Wiener, Guy Soo Hoo, and Frank C Detterbeck. Screening for lung cancer: Chest guideline and expert panel report.Chest, 153(4):954–985, 2018

    2018

  21. [29]

    Chinese Thoracic Society, Chinese Medical Association, Chinese Alliance Against Lung Cancer Expert Group, et al. Chinese expert consensus on diagnosis and treatment of pulmonary nodules (2024).Zhonghua jie he he hu xi za zhi= Zhonghua jiehe he huxi zazhi= Chinese journal of tu...

    2024

  22. [30]

    Haiquan Chen, Anthony W Kim, Michael Hsin, Joseph B Shrager, Ashley E Prosper, Momen M Wahidi, Dennis A Wigle, Carol C Wu, James Huang, Kazuhiro Yasufuku, et al. The 2023 american association for thoracic surgery (aats) expert consensus document: management of subsolid lung no...

    2023

  23. [31]

    Acr lung-rads v2022: assessment categories and management recommendations.Journal of the American College of Radiology, 21(3):473–488, 2024

    Jared Christensen, Ashley Elizabeth Prosper, Carol C Wu, Jonathan Chung, Elizabeth Lee, Brett Elicker, Andetta R Hunsaker, Milena Petranovic, Kim L Sandler, Brendon Stiles, et al. Acr lung-rads v2022: assessment categories and management recommendations.Journal of the American...

    2024

  24. [32]

    Multimodal large language models in medical imaging: current state and future directions.Korean Journal of Radiology, 26(10):900, 2025

    Yoojin Nam, Dong Yeong Kim, Sunggu Kyung, Jinyoung Seo, Jeong Min Song, Jimin Kwon, Jihyun Kim, Wooyoung Jo, Hyungbin Park, Jimin Sung, et al. Multimodal large language models in medical imaging: current state and future directions.Korean Journal of Radiology, 26(10):900, 2025

    2025

  25. [33]

    Multimodal generative ai for medical image interpretation.Nature, 639(8056):888–896, 2025

    Vishwanatha M Rao, Michael Hla, Michael Moor, Subathra Adithan, Stephen Kwak, Eric J Topol, and Pranav Rajpurkar. Multimodal generative ai for medical image interpretation.Nature, 639(8056):888–896, 2025

    2025

  26. [34]

    Management of copd exacerbations: a european respiratory society/american thoracic society guideline.European Respiratory Journal, 49(3):1600791, 2017

    Jadwiga A Wedzicha, Marc Miravitlles, John R Hurst, Peter MA Calverley, Richard K Albert, Antonio Anzueto, Gerard J Criner, Alberto Papi, Klaus F Rabe, David Rigau, et al. Management of copd exacerbations: a european respiratory society/american thoracic society guideline.Euro...

    2017

  27. [35]

    Reporting guideline for the early-stage clinical evaluation of decision support systems driven by artificial intelligence: Decide-ai.Nature medicine, 28(5):924–933, 2022

    Baptiste Vasey, Myura Nagendran, Bruce Campbell, David A Clifton, Gary S Collins, Spiros Denaxas, Alastair K Denniston, Livia Faes, Bart Geerts, Mudathir Ibrahim, et al. Reporting guideline for the early-stage clinical evaluation of decision support systems driven by artificia...

    2022

  28. [36]

    Tripod+ ai statement: updated guidance for reporting clinical prediction models that use regression or machine learning methods.bmj, 385, 2024

    Gary S Collins, Karel GM Moons, Paula Dhiman, Richard D Riley, Andrew L Beam, Ben Van Calster, Marzyeh Ghassemi, Xiaoxuan Liu, Johannes B Reitsma, Maarten Van Smeden, et al. Tripod+ ai statement: updated guidance for reporting clinical prediction models that use regression or ...

    2024

  29. [37]

    Tina Hernandez-Boussard, Selen Bozkurt, John PA Ioannidis, and Nigam H Shah. Minimar (minimum information for medical ai reporting): Developing reporting standards for artificial intelligence in health care.Journal of the American Medical Informatics Association, 27(12):2011–2...

    2011

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.