Pith. sign in

REVIEW 4 major objections 4 minor 1 cited by

Multi-Modal Explainable Medical AI Assistant for Trustworthy Human-AI Collaboration

T0 review · 4 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that XMedGPT, a single 8-billion-parameter medical multimodal model, can simultaneously ground its findings in images, score its own uncertainty, and predict survival, outperforming prior generalist medical AI and GPT-4o…

desk verdict Ambitious medical MLLM integration, but the central uncertainty formula is wrong and the reported AUCs don't evaluate the proposed reliability index. read the letter →

arxiv 2505.06898 v1 pith:6OH4C5OU submitted 2025-05-11 cs.CV cs.CL

Honglong Yang , Shanshan Song , Yi Qin , Lehan Wang , Haonan Wang , Xinpeng Ding , Qixiang Zhang , Bodong Du
show 1 more author
Xiaomeng Li
This is my paper · ORCID
classification cs.CVcs.CL
keywords generalistmedicalAImultimodalexplainabilityvisualgroundingsemanticentropyuncertaintyquantificationprognosticpredictionVQAradiologyreportgeneration
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

XMedGPT is a generalist medical AI that aims to do more than perceive: the authors build a single 8-billion-parameter multimodal model that writes diagnoses and reports, points to the anatomical regions behind each statement, attaches a reliability score to its own outputs, and predicts survival and recurrence from imaging plus clinical data. The paper reports that this one model outperforms prior generalist medical AI and GPT-4o across diagnosis, visual question answering, captioning, region grounding, and prognosis, with external validation on hospital data covering eleven lesion types. A sympathetic reader would care because the proposal is a concrete route from opaque medical language models toward tools a clinician can audit: every claim carries a location and a confidence, and the model itself flags when it is unsure. If the reported numbers hold, generalist medical AI becomes a decision-support layer rather than a perception-only system.

What carries the argument

The load-bearing machinery is the pairing of a grounded report generator with a reliability index. The generator follows a five-step chain of thought: identify anatomy, localize it, describe each region, integrate patient context, then write the report, so every sentence can be traced to detected boxes. The reliability index converts each generated sentence into several yes/no visual question-answer pairs, compares the answers with the sentence, and computes a reformulated semantic entropy over consistency clusters; equations (5)-(7) define that entropy, with equation (5) intended to correct underestimation when all sampled answers fall in one cluster. This index is what turns raw outputs into high, medium, or low reliability flags for the clinician.

What would settle it

Recompute both sides of Equation (5) on the paper's own VQA examples: $-\hat{P}(c)\log \hat{P}(c)$ versus $\sum_{s\in c} \hat{p}_s\log \hat{p}_s$, with $\hat{p}_s$ the probability of sequence $s$ normalized within cluster $c$. They differ whenever the sequences in a cluster have unequal probabilities; then re-estimate the uncertainty AUC using $H=-\sum_c \hat{P}(c)\log \hat{P}(c)$ and compare with the reported 0.862 and 0.764. A material drop would mean the reliability advantage is an artifact of the mislabeled estimator.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that perception, grounding, uncertainty quantification, and prognosis can live in one medical multimodal model. XMedGPT is fine-tuned from a vision-language transformer on over seven million image-text pairs, including 1.6 million with pixel-level annotations, and then post-trained for chain-of-thought reasoning, audio input, and region-level interaction. The authors claim it achieves 0.703 IoU for lesion localization across 141 anatomical regions, an uncertainty estimation AUC of 0.862 on visual question answering and 0.764 on sentence-level report generation, and survival and recurrence AUCs that beat the prior task-specific models by 26.9% on average and GPT-4o by 25.0%. These results are presented as evidence that explainability, reliability scoring, and prognostic modeling can be unified rather than bolted on.

Load-bearing premise

The uncertainty-scoring scheme rests on the claim in Equation (5) that the entropy contributed by a cluster of consistent answers equals the entropy of the individual answer probabilities inside that cluster; that identity is not true, so if it gives way the reported reliability scores are not scores of the proposed reliability index.

Editorial extensions

If this is right

  • If the claims hold, a single 8B model can replace separate tools for diagnosis, VQA, captioning, grounding, audio interaction, and prognosis in one clinical workflow.
  • Grounded reporting with 0.703 IoU across 141 regions means each textual finding carries spatial evidence a clinician can check against the image.
  • A reliability index with 0.862 VQA AUC and 0.764 report AUC would let clinicians triage which AI statements need scrutiny and which can be acted on.
  • Survival and recurrence predictions that beat prior task-specific models by 26.9% would extend generalist medical AI from reading images to supporting treatment decisions.
  • Detecting 83% of unanswerable image-query mismatches would give the system a practical guardrail against confident hallucinations.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's uncertainty numbers depend on Eq. (5), which identifies the entropy contribution of a cluster with the entropy of the sequence probabilities inside that cluster; the two expressions are not generally equal, so the reported AUCs likely evaluate a related but different estimator.
  • The prognostic gains come from aggregating slice-level probabilities on external cohorts of 49 to 163 patients; a patient-level, calibration-focused validation on larger cohorts would be needed to know whether the margin over GPT-4o and task-specific models transfers.
  • The observed improvement in uncertainty AUC as the number of generated questions per sentence rises (0.676 to 0.764) suggests that most of the gain comes from spending more test-time computation, leaving the choice of question budget an open design variable.
  • The Kendall tau-b of 0.479 between reasoning-path quality and final report quality shows the two co-vary, but it does not establish that the visible reasoning is what produced the correct report; an intervention study would be needed to test causation.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The manuscript introduces XMedGPT, an 8B-parameter multimodal medical assistant built on InternVL-2 and trained on roughly 7 million image-text pairs. It is evaluated across five core tasks (single-label and multi-label diagnosis, VQA, multiple-choice reasoning, and captioning), region-level grounding, uncertainty quantification, audio-input interaction, and survival/recurrence prediction. The paper claims state-of-the-art results over prior GMAI models, an anatomical-grounding IoU of 0.703, uncertainty-estimation AUCs of 0.862 (VQA) and 0.764 (report generation), and a 26.9% improvement over prior leading models in prognosis. A reliability-index mechanism is proposed that computes semantic entropy over consistency-based question-answer checks, and the paper also reports external validation on in-house clinical data and a senior-clinician human evaluation.

Significance. If the contributions were as claimed, XMedGPT would be a substantial advance: an 8B-parameter generalist medical model with visual grounding, sentence-level uncertainty, audio and region-based interaction, and prognostic modeling, validated on public and external clinical data. The paper's strengths are its breadth (40 modalities, 141 anatomical regions, 18 clinical tasks), the explicit guardrail evaluation, and the inclusion of a senior-clinician human evaluation in the collaboration study. However, the central uncertainty estimator is built on an algebraically false identity, and the reported results contain unresolved numerical contradictions across the abstract, the main text, and the tables. As a result, the headline uncertainty, grounding, and prognostic claims are not currently supported. The absence of released code or checkpoints at submission further limits verification of the benchmark numbers.

major comments (4)
  1. [Methodology, 'Formulation for robust uncertainty estimation', Eq. (5)] Equation (5) is algebraically false. With P(c|x,q) defined in Eq. (2) as the sum over s in c of P(s|x,q), the left side is -P(c)logP(c), while the right side is the within-cluster entropy -sum_{s in c} [P(s)/P(c)] log(P(s)/P(c)). These two quantities are equal only in degenerate cases, e.g., when a single sequence carries all cluster mass. Consequently, Eq. (6) is not a valid semantic-entropy estimator: if H_c is read literally as -P(c)logP(c), then Eq. (6) evaluates -sum_i P(C_i)^2 logP(C_i), which is not semantic entropy; if H_c is read as the within-cluster entropy on the right side of Eq. (5), then Eq. (6) becomes a weighted average of within-cluster entropies and discards the between-cluster uncertainty term entirely. Since the reliability index is defined through this entropy, the reported AUC values of 0.862 on VQA and 0.764 on report generation are not measurements of the proposed estimator. Equation (7) inherits the same error.
  2. [Abstract; Results regional-performance section; Table 6; Introduction] The paper contains several mutually inconsistent headline numbers that must be reconciled. The abstract reports an IoU of 0.703 across 141 anatomical regions, and the Results section repeats the IoU 0.703 for anatomical localization, but Table 6 reports an overall lesion-detection IoU of 0.9019. The Results section states a region-recognition F1-score of 0.774, whereas Table 6 lists the overall region-recognition F1 as 0.6265. The number of evaluation datasets is given as 347 in the abstract, 23 public benchmarks in the Introduction and Methods, and 38 benchmarks in the Discussion, with no definition of which count is being used. Finally, the abstract and conclusion claim a 26.9% average improvement in survival and recurrence prediction, but the per-task AUC improvements reported in the Results are 16.7% for PFS, 18.1% for NSCLC OS, and 2.7% for GBM OS, whose average is approximately 12.3%.
  3. [Results, 'XMedGPT is A Clinician-Centric AI Assistant...'; Table 10; Methodology, 'Formulation for medical reasoning'] The evaluation setup creates a potential closed loop that weakens the claimed superiority over prior models. The principal GMAI comparator, MedRegA, is reference [80], which is the authors' own prior work; this self-comparison is not disclosed in the main text. In Table 10, the free-text region-VQA responses are scored by GPT-4o as judge, and the reasoning dataset used for training is also generated with GPT-4o. Without a human-validated LLM judge or an independent evaluation protocol, the 'accuracy', 'consistency', and 'clinical relevance' scores in Table 10 are not established as objective measurements, and the improvement margins over MedRegA and MedDr should be interpreted with caution.
  4. [Methodology, 'Implementation for prognostic decision making'; Discussion] The prognostic claims rest on a modeling choice that is not validated. Patient-level survival probabilities are obtained by averaging slice-level predictions across CT slices, but no analysis is provided to justify this aggregation, and the external cohorts are small (n=49, 58, 133, and 163). The Discussion acknowledges the small-cohort limitation, but the manuscript does not report calibration or decision-curve analyses for the survival models. Given that prognostic modeling is one of the four pillars of the paper, the reported improvement over task-specific models should be supported by an evaluation of the aggregation procedure and by calibration measures, not only by AUC and accuracy on small external sets.
minor comments (4)
  1. [Abstract and throughout] The abstract contains 'external validation sans four anatomical systems', which should read 'across four anatomical systems'; a typographical pass is needed for similar issues (e.g., 'Generalists Medical AI' in the Methodology, 'Direct Policy Optimization' for DPO, and 'Bradeley-Terry' for Bradley-Terry).
  2. [Table 10] The 'Mean' row in Table 10 appears misaligned with the column headers: the listed mean values (7.98, 6.68, 7.89, 3.22, 4.58, 8.61) do not correspond in order to MedDr, MedRegA, LLaVA-Med, MedFlamingo, RadFM, and Ours. The authors should verify that each column's mean is placed under the correct model.
  3. [Figure 4 caption] Figure 4e mentions Steps 1, 2, and 4 of the chain-of-thought evaluation, while the text describes a five-step process; the caption should be aligned with the step numbering used in the main text.
  4. [Code Availability] The code is promised 'after the manuscript is published', which is insufficient for a benchmark-heavy paper with many reported state-of-the-art numbers; providing code, checkpoints, or at least a detailed evaluation protocol at submission would substantially improve verifiability.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: benchmark claims are independently grounded on public datasets; self-citation to MedRegA and GPT-4o judging are concerns, not circular reductions.

full rationale

The paper's main results are evaluated against public benchmarks (SLAKE, VQA-RAD, PathVQA, OmniMedVQA, GMAI-MMbench, MIMIC-CXR, IU-Xray, MedGRIT, LUNG1/RADIO, UPENN/BraTS) with fixed labels and external baselines, so the reported IoU, F1, and prognostic AUC values are not forced by the paper's own definitions. The repeated comparison to MedRegA [80] is a self-comparison because that reference shares authors with the present paper, but it functions as a baseline rather than a derivation step, and the paper also compares against MedDr, BiRD, GPT-4o, and other public systems. The GPT-4o-as-judge protocol in the in-house region-VQA evaluation (Table 10) creates a stylistic loop with the GPT-4o-generated reasoning data, but the judge is a fixed external model and the primary benchmark claims do not depend on it. The Discussion properly acknowledges the small external prognostic cohorts (n=49-163) and the pulmonary embolism performance drop; these are correctness limitations, not circularity. Separately, Eq. (5) is algebraically invalid: the left-hand side -P(c)logP(c) is not equal to the within-cluster conditional entropy on the right, so the reported uncertainty AUCs may not evaluate the stated reliability estimator. That is a serious correctness risk, but no prediction in the paper reduces to a fitted parameter or to a self-citation chain, so no circular step is established.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The central claim depends on several unverified assumptions and hand-picked thresholds. The free parameters are mostly clinical endpoints and sampling settings; the axioms include the base model, the NLI-based clustering, GPT-4o as generator and judge, and slice-level averaging. No code is released, so none of these can be independently checked. The mathematical error in the entropy estimator invalidates the reliability-index component.

free parameters (6)
  • PFS threshold = 14 months
    Binary endpoint for TKI progression-free survival, chosen from median follow-up; changes the task definition.
  • NSCLC OS threshold = 2 years
    Binary endpoint for overall survival in LUNG1/RADIO, following prior studies.
  • GBM OS threshold = 1 year
    Binary endpoint for glioblastoma survival in UPENN/BraTS.
  • VQA questions per sentence M = 5 (implied)
    Report-generation uncertainty AUC depends on M; reported 0.764 comes from the largest tested M, suggesting tuning on the evaluation subset.
  • Sampling temperature and top-p = temperature 1.0, top-p 0.9
    Used for all semantic-entropy samplings; no ablation on these values.
  • Reliability level cutoffs = unspecified
    The three-level high/medium/low index requires thresholds on the entropy score that are not given.
assumptions (5)
  • domain assumption InternVL-2 provides a sufficiently strong visual-linguistic base for medical tasks
    The model is a one-epoch fine-tune of InternVL-2; no evidence is given that the base generalizes across 40 modalities.
  • domain assumption DeBERTa-Large-MNLI entailment correctly identifies semantic equivalence
    All semantic-entropy clustering relies on this NLI model; errors directly affect entropy values.
  • domain assumption GPT-4o-generated reasoning and scores are clinically reliable
    Reasoning data for SFT/DPO and the free-text VQA scores (Table 10) are produced or judged by GPT-4o without radiologist verification.
  • domain assumption Slice-level averaging is a valid aggregation for patient-level survival
    OS predictions average probabilities over tumor-bearing slices; no justification that this approximates whole-volume reading.
  • domain assumption Chest-ImaGenome automated labels are accurate enough for reasoning training
    The reasoning dataset uses ROI annotations from an automated pipeline, which may contain noise.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Multi-Modal Explainable Medical AI Assistant for Trustworthy Human-AI Collaboration." pith.science (2026). https://pith.science/paper/6OH4C5OU

@misc{pith2026250506898,
  author       = {Pith},
  title        = {Pith review of: Multi-Modal Explainable Medical AI Assistant for Trustworthy Human-AI Collaboration},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6OH4C5OU}},
  note         = {Machine review of arXiv:2505.06898}
}
read the original abstract

Generalist Medical AI (GMAI) systems have demonstrated expert-level performance in biomedical perception tasks, yet their clinical utility remains limited by inadequate multi-modal explainability and suboptimal prognostic capabilities. Here, we present XMedGPT, a clinician-centric, multi-modal AI assistant that integrates textual and visual interpretability to support transparent and trustworthy medical decision-making. XMedGPT not only produces accurate diagnostic and descriptive outputs, but also grounds referenced anatomical sites within medical images, bridging critical gaps in interpretability and enhancing clinician usability. To support real-world deployment, we introduce a reliability indexing mechanism that quantifies uncertainty through consistency-based assessment via interactive question-answering. We validate XMedGPT across four pillars: multi-modal interpretability, uncertainty quantification, and prognostic modeling, and rigorous benchmarking. The model achieves an IoU of 0.703 across 141 anatomical regions, and a Kendall's tau-b of 0.479, demonstrating strong alignment between visual rationales and clinical outcomes. For uncertainty estimation, it attains an AUC of 0.862 on visual question answering and 0.764 on radiology report generation. In survival and recurrence prediction for lung and glioma cancers, it surpasses prior leading models by 26.9%, and outperforms GPT-4o by 25.0%. Rigorous benchmarking across 347 datasets covers 40 imaging modalities and external validation spans 4 anatomical systems confirming exceptional generalizability, with performance gains surpassing existing GMAI by 20.7% for in-domain evaluation and 16.7% on 11,530 in-house data evaluation. Together, XMedGPT represents a significant leap forward in clinician-centric AI integration, offering trustworthy and scalable support for diverse healthcare applications.

Figures

Figures reproduced from arXiv: 2505.06898 by the authors.

Figure 2
Figure 2. a, XMedGPT consistently outperforms other generalist biomedical models across five core tasks: single-label diagnosis, multi-label diagnosis, visual question answering (VQA), multiple-choice reasoning, and image captioning. b, Overall task performance relative to model scale. c, Performance on single- and multi-label diagnosis tasks, evaluated using F1-score and micro-F1-score, respectively. d, Accuracy on the OmniM… view at source ↗
Figure 4
Figure 4. a, Comparison of our method with conventional semantic entropy (SE) and discrete entropy (DE) on four VQA tasks using AUROC to assess error prediction. b, Performance on image captioning tasks, showing our method AUC improves with more generated questions per sentence. c, Overview of curated datasets for supervised fine-tuning (10k samples) and direct preference optimization (8k samples); see [PITH_FULL_IMAGE:figur… view at source ↗
Figure 5
Figure 5. a, XMedGPT is an all-in-one biomedical model capable of processing image, text, audio, and region-level inputs, enabling tasks such as region recognition, region-based VQA, and lesion localization. b, List of anatomical regions recognizable by XMedGPT, which supports localization of 141 lesion types (see Figure 5b for details). c, Performance on region-related tasks: F1-score for region recognition, IoU for lesion l… view at source ↗
Figures from the paper (11 more)
Figure 6
Figure 6. Figure 6: a, Evaluation on the external validation data across diverse biomedical tasks (Multiple￾choice, Region Recognition and Lesion Localization). Accuracy is used as the evaluation metric for multiple-choice, F1-score is used for region recognition, and Intersection over Un…
Figure 7
Figure 7. Figure 7: Overview of uncertainty estimation in visual question answering and report generation. [PITH_FULL_IMAGE:figures/full_fig_p037_7.png]
Figure 8
Figure 8. Figure 8: Ablation results on four VQA datasets demonstrating the impact of sampling frequency [PITH_FULL_IMAGE:figures/full_fig_p038_8.png]
Figure 9
Figure 9. Figure 9: Schematic overview of the data curation pipeline for reasoning tasks. [PITH_FULL_IMAGE:figures/full_fig_p039_9.png]
Figure 10
Figure 10. Figure 10: Illustration of the prompt design used for lesion detection task. [PITH_FULL_IMAGE:figures/full_fig_p040_10.png]
Figure 11
Figure 11. Figure 11: Illustration of the prompt design used for region recognition task. [PITH_FULL_IMAGE:figures/full_fig_p040_11.png]
Figure 12
Figure 12. Figure 12: Illustration of the prompt design used for reasoning data curation. [PITH_FULL_IMAGE:figures/full_fig_p041_12.png]
Figure 13
Figure 13. Figure 13: Qualitative comparison between XmedGPT and a state-of-the-art Generalist Medical AI [PITH_FULL_IMAGE:figures/full_fig_p042_13.png]
Figure 14
Figure 14. Figure 14: Qualitative comparison between XmedGPT and a state-of-the-art Generalist Medical AI [PITH_FULL_IMAGE:figures/full_fig_p043_14.png]
Figure 15
Figure 15. Figure 15: Representative example illustrating the uncertainty estimation pipeline and its correspond [PITH_FULL_IMAGE:figures/full_fig_p044_15.png]
Figure 16
Figure 16. Figure 16: Representative example of trustworthy report generation with integrated visual and textual [PITH_FULL_IMAGE:figures/full_fig_p045_16.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Unifying Biomedical Vision-Language Expertise: Towards a Generalist Foundation Model via Multi-CLIP Knowledge Distillation

    cs.CV 2025-06 conditional novelty 6.0 of 10

    A student CLIP model distilled from nine medical CLIP teachers outperforms its teachers across most of 58 biomedical benchmarks.

Reference graph

Works this paper leans on

95 extracted references · 36 canonical work pages · cited by 1 Pith paper

  1. [80]

    Interpretable bilingual multimodal large language model for diverse biomedical tasks.arXiv preprint arXiv:2410.18387, 2024

    Lehan Wang, Haonan Wang, Honglong Yang, Jiaji Mao, Zehong Yang, Jun Shen, and Xiaomeng Li. Interpretable bilingual multimodal large language model for diverse biomedical tasks.arXiv preprint arXiv:2410.18387, 2024

    arXiv 2024

  2. [1]

    Multimodal biomedical ai.Nature medicine, 28(9):1773–1784, 2022

    Julián N Acosta, Guido J Falcone, Pranav Rajpurkar, and Eric J Topol. Multimodal biomedical ai.Nature medicine, 28(9):1773–1784, 2022

    2022

  3. [2]

    Dataset of breast ultrasound images.Data in brief, 28:104863, 2020

    Walid Al-Dhabyani, Mohammed Gomaa, Hussien Khaled, and Aly Fahmy. Dataset of breast ultrasound images.Data in brief, 28:104863, 2020

    2020

  4. [3]

    The medical segmentation decathlon.Nature communications, 13(1):4128, 2022

    Michela Antonelli, Annika Reinke, Spyridon Bakas, Keyvan Farahani, Annette Kopp-Schneider, Bennett A Landman, Geert Litjens, Bjoern Menze, Olaf Ronneberger, Ronald M Summers, et al. The medical segmentation decathlon.Nature communications, 13(1):4128, 2022

    2022

  5. [4]

    Speecht5: Unified-modal encoder-decoder pre-training for spoken language processing.arXiv preprint arXiv:2110.07205, 2021

    Junyi Ao, Rui Wang, Long Zhou, Chengyi Wang, Shuo Ren, Yu Wu, Shujie Liu, Tom Ko, Qing Li, Yu Zhang, et al. Speecht5: Unified-modal encoder-decoder pre-training for spoken language processing.arXiv preprint arXiv:2110.07205, 2021

    arXiv

  6. [5]

    Bach: Grand challenge on breast cancer histology images.Medical image analysis, 56:122–139, 2019

    Guilherme Aresta, Teresa Araújo, Scotty Kwok, Sai Saketh Chennamsetty, Mohammed Safwan, Varghese Alex, Bahram Marami, Marcel Prastawa, Monica Chan, Michael Donovan, et al. Bach: Grand challenge on breast cancer histology images.Medical image analysis, 56:122–139, 2019

    2019

  7. [6]

    Meteor: An automatic metric for mt evaluation with improved correlation with human judgments

    Satanjeev Banerjee and Alon Lavie. Meteor: An automatic metric for mt evaluation with improved correlation with human judgments. InProceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization, pages 65–72, 2005

    2005

  8. [7]

    Vqa-med: Overview of the medical visual question answering task at imageclef 2019

    Asma Ben Abacha, Sadid A Hasan, Vivek V Datla, Dina Demner-Fushman, and Henning Müller. Vqa-med: Overview of the medical visual question answering task at imageclef 2019. InProceedings of CLEF (Conference and Labs of the Evaluation Forum) 2019 Working Notes. 9-12 September 2019, 2019

    2019

Show all 95 references
  1. [8]

    Towards a general- purpose foundation model for computational pathology.Nature Medicine, 30(3):850–862, 2024

    Richard J Chen, Tong Ding, Ming Y Lu, Drew FK Williamson, Guillaume Jaume, Andrew H Song, Bowen Chen, Andrew Zhang, Daniel Shao, Muhammad Shaban, et al. Towards a general- purpose foundation model for computational pathology.Nature Medicine, 30(3):850–862, 2024

    2024

  2. [9]

    Med3d: Transfer learning for 3d medical image analysis.arXiv preprint arXiv:1904.00625, 2019

    Sihong Chen, Kai Ma, and Yefeng Zheng. Med3d: Transfer learning for 3d medical image analysis.arXiv preprint arXiv:1904.00625, 2019

    1904 arXiv

  3. [10]

    How far are we to gpt-4v? closing the gap to commercial multimodal models with open-source suites.arXiv preprint arXiv:2404.16821, 2024

    Zhe Chen, Weiyun Wang, Hao Tian, Shenglong Ye, Zhangwei Gao, Erfei Cui, Wenwen Tong, Kongzhi Hu, Jiapeng Luo, Zheng Ma, et al. How far are we to gpt-4v? closing the gap to commercial multimodal models with open-source suites.arXiv preprint arXiv:2404.16821, 2024

    2024 arXiv

  4. [11]

    Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks

    Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al. Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks. InProceedings of the IEEE/CVF conference on computer visi...

    2024

  5. [12]

    Rouge: A package for automatic evaluation of summaries

    Lin Chin-Yew. Rouge: A package for automatic evaluation of summaries. InProceedings of the Workshop on Text Summarization Branches Out, 2004, 2004

    2004

  6. [13]

    Vision–language foundation model for echocardiogram interpretation.Nature Medicine, 30(5):1481–1488, 2024

    Matthew Christensen, Milos Vukadinovic, Neal Yuan, and David Ouyang. Vision–language foundation model for echocardiogram interpretation.Nature Medicine, 30(5):1481–1488, 2024

    2024

  7. [14]

    Bcn20000: Dermoscopic lesions in the wild.arXiv preprint arXiv:1908.02288, 2019

    Marc Combalia, Noel CF Codella, Veronica Rotemberg, Brian Helba, Veronica Vilaplana, Ofer Reiter, Cristina Carrera, Alicia Barreiro, Allan C Halpern, Susana Puig, et al. Bcn20000: Dermoscopic lesions in the wild.arXiv preprint arXiv:1908.02288, 2019

    1908 arXiv

  8. [15]

    Preventing unrestricted and unmonitored ai experimentation in healthcare through transparency and accountability.npj Digital Medicine, 8(1):42, 2025

    Donnella S Comeau, Danielle S Bitterman, and Leo Anthony Celi. Preventing unrestricted and unmonitored ai experimentation in healthcare through transparency and accountability.npj Digital Medicine, 8(1):42, 2025

    2025

  9. [16]

    Pitvis-2023 challenge: Workflow recognition in videos of endoscopic pituitary surgery.arXiv preprint arXiv:2409.01184, 2024

    Adrito Das, Danyal Z Khan, Dimitrios Psychogyios, Yitong Zhang, John G Hanrahan, Fran- cisco Vasconcelos, You Pang, Zhen Chen, Jinlin Wu, Xiaoyang Zou, et al. Pitvis-2023 challenge: Workflow recognition in videos of endoscopic pituitary surgery.arXiv preprint arXiv:2409.01184, 2024

    2023 arXiv

  10. [17]

    Preparing a collection of radiology examinations for distribution and retrieval.Journal of the American Medical Informatics Association, 23(2):304–310, 2016

    Dina Demner-Fushman, Marc D Kohli, Marc B Rosenman, Sonya E Shooshan, Laritza Ro- driguez, Sameer Antani, George R Thoma, and Clement J McDonald. Preparing a collection of radiology examinations for distribution and retrieval.Journal of the American Medical Informatics Associa...

    2016

  11. [18]

    Kexue Deng, Lu Wang, Yuchan Liu, Xin Li, Qiuyang Hou, Mulan Cao, Nathan Norton Ng, Huan Wang, Huanhuan Chen, Kristen W Yeom, et al. A deep learning-based system for survival benefit prediction of tyrosine kinase inhibitors and immune checkpoint inhibitors in stage iv non-small...

    2022

  12. [19]

    Crossmoda 2021 chal- lenge: Benchmark of cross-modality domain adaptation techniques for vestibular schwannoma and cochlea segmentation.Medical Image Analysis, 83:102628, 2023

    Reuben Dorent, Aaron Kujawa, Marina Ivory, Spyridon Bakas, Nicola Rieke, Samuel Joutard, Ben Glocker, Jorge Cardoso, Marc Modat, Kayhan Batmanghelich, et al. Crossmoda 2021 chal- lenge: Benchmark of cross-modality domain adaptation techniques for vestibular schwannoma and coch...

    2021

  13. [20]

    Detecting hallucinations in large language models using semantic entropy.Nature, 630(8017):625–630, 2024

    Sebastian Farquhar, Jannik Kossen, Lorenz Kuhn, and Yarin Gal. Detecting hallucinations in large language models using semantic entropy.Nature, 630(8017):625–630, 2024

    2024

  14. [21]

    Multiparametric mri along with machine learning predicts prognosis and treatment response in pediatric low-grade glioma.Nature Communications, 16(1):340, 2025

    Anahita Fathi Kazerooni, Adam Kraya, Komal S Rathi, Meen Chul Kim, Arastoo V ossough, Nastaran Khalili, Ariana M Familiar, Deep Gandhi, Neda Khalili, Varun Kesherwani, et al. Multiparametric mri along with machine learning predicts prognosis and treatment response in pediatric...

    2025

  15. [22]

    Palm: Pathologic myopia challenge, 2019

    Huazhu Fu, Fei Li, José Ignacio Orlando, Hrvoje Bogunovi ´c, Xu Sun, Jingan Liao, Yanwu Xu, Shaochong Zhang, and Xiulan Zhang. Palm: Pathologic myopia challenge, 2019. URL https://dx.doi.org/10.21227/55pk-8z03

    2019 doi

  16. [23]

    Pannuke dataset extension, insights and baselines.arXiv preprint arXiv:2003.10778, 2020

    Jevgenij Gamper, Navid Alemi Koohbanani, Ksenija Benes, Simon Graham, Mostafa Jahanifar, Syed Ali Khurram, Ayesha Azam, Katherine Hewitt, and Nasir Rajpoot. Pannuke dataset extension, insights and baselines.arXiv preprint arXiv:2003.10778, 2020

    2003 arXiv

  17. [24]

    Bayeseg: Bayesian modeling for medical image segmentation with interpretable generalizability.Medical Image Analysis, 89: 102889, 2023

    Shangqi Gao, Hangqi Zhou, Yibo Gao, and Xiahai Zhuang. Bayeseg: Bayesian modeling for medical image segmentation with interpretable generalizability.Medical Image Analysis, 89: 102889, 2023

    2023

  18. [25]

    A whole-body fdg-pet/ct dataset with manually annotated tumor lesions.Scientific Data, 9(1):601, 2022

    Sergios Gatidis, Tobias Hepp, Marcel Früh, Christian La Fougère, Konstantin Nikolaou, Christina Pfannenberg, Bernhard Schölkopf, Thomas Küstner, Clemens Cyran, and Daniel Rubin. A whole-body fdg-pet/ct dataset with manually annotated tumor lesions.Scientific Data, 9(1):601, 2022

    2022

  19. [26]

    Octid: Optical coherence tomography image database.Computers & Electrical Engineering, 81:106532, 2020

    Peyman Gholami, Priyanka Roy, Mohana Kuppuswamy Parthasarathy, and Vasudevan Laksh- minarayanan. Octid: Optical coherence tomography image database.Computers & Electrical Engineering, 81:106532, 2020

    2020

  20. [27]

    The llama 3 herd of models.arXiv preprint arXiv:2407.21783, 2024

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models.arXiv preprint arXiv:2407.21783, 2024

    2024 arXiv

  21. [28]

    Evalu- ation and mitigation of the limitations of large language models in clinical decision-making

    Paul Hager, Friederike Jungmann, Robbie Holland, Kunal Bhagat, Inga Hubrecht, Manuel Knauer, Jakob Vielhauer, Marcus Makowski, Rickmer Braren, Georgios Kaissis, et al. Evalu- ation and mitigation of the limitations of large language models in clinical decision-making. Nature m...

    2024

  22. [29]

    Meddr: Diagnosis-guided bootstrapping for large-scale medical vision-language learning.arXiv preprint arXiv:2404.15127, 2024

    Sunan He, Yuxiang Nie, Zhixuan Chen, Zhiyuan Cai, Hongmei Wang, Shu Yang, and Hao Chen. Meddr: Diagnosis-guided bootstrapping for large-scale medical vision-language learning.arXiv preprint arXiv:2404.15127, 2024. 23

    2024 arXiv

  23. [30]

    Pathvqa: 30000+ questions for medical visual question answering.arXiv preprint arXiv:2003.10286, 2020

    Xuehai He, Yichen Zhang, Luntian Mou, Eric Xing, and Pengtao Xie. Pathvqa: 30000+ questions for medical visual question answering.arXiv preprint arXiv:2003.10286, 2020

    2003 arXiv

  24. [31]

    The kits21 challenge: Automatic segmentation of kidneys, renal tumors, and renal cysts in corticomedullary-phase ct

    Nicholas Heller, Fabian Isensee, Dasha Trofimova, Resha Tejpaul, Zhongchen Zhao, Huai Chen, Lisheng Wang, Alex Golts, Daniel Khapun, Daniel Shats, et al. The kits21 challenge: Automatic segmentation of kidneys, renal tumors, and renal cysts in corticomedullary-phase ct. arXiv ...

    2023 arXiv

  25. [32]

    Isles 2022: A multi-center magnetic resonance imaging stroke lesion segmentation dataset

    Moritz R Hernandez Petzsche, Ezequiel de la Rosa, Uta Hanning, Roland Wiest, Waldo Valenzuela, Mauricio Reyes, Maria Meyer, Sook-Lei Liew, Florian Kofler, Ivan Ezhov, et al. Isles 2022: A multi-center magnetic resonance imaging stroke lesion segmentation dataset. Scientific da...

    2022

  26. [33]

    Omnimed- vqa: A new large-scale comprehensive evaluation benchmark for medical lvlm

    Yutao Hu, Tianbin Li, Quanfeng Lu, Wenqi Shao, Junjun He, Yu Qiao, and Ping Luo. Omnimed- vqa: A new large-scale comprehensive evaluation benchmark for medical lvlm. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22170–22183, 2024

    2024

  27. [34]

    A refer-and-ground multimodal large language model for biomedicine

    Xiaoshuang Huang, Haifeng Huang, Lingdong Shen, Yehui Yang, Fangxin Shang, Junwei Liu, and Jia Liu. A refer-and-ground multimodal large language model for biomedicine. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 399–409. Spr...

    2024

  28. [35]

    Gpt-4o system card.arXiv preprint arXiv:2410.21276, 2024

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card.arXiv preprint arXiv:2410.21276, 2024

    2024 arXiv

  29. [36]

    Segmentation of glioma tumors in brain using deep convolutional neural network.Neurocomputing, 282:248–261, 2018

    Saddam Hussain, Syed Muhammad Anwar, and Muhammad Majid. Segmentation of glioma tumors in brain using deep convolutional neural network.Neurocomputing, 282:248–261, 2018

    2018

  30. [37]

    Quilt-1m: One million image-text pairs for histopathology.Advances in neural information processing systems, 36: 37995–38017, 2023

    Wisdom Ikezogwo, Saygin Seyfioglu, Fatemeh Ghezloo, Dylan Geva, Fatwir Sheikh Mo- hammed, Pavan Kumar Anand, Ranjay Krishna, and Linda Shapiro. Quilt-1m: One million image-text pairs for histopathology.Advances in neural information processing systems, 36: 37995–38017, 2023

    2023

  31. [38]

    Chexpert: A large chest radiograph dataset with uncertainty labels and expert comparison

    Jeremy Irvin, Pranav Rajpurkar, Michael Ko, Yifan Yu, Silviana Ciurea-Ilcus, Chris Chute, Henrik Marklund, Behzad Haghgoo, Robyn Ball, Katie Shpanskaya, et al. Chexpert: A large chest radiograph dataset with uncertainty labels and expert comparison. InProceedings of the AAAI c...

    2019

  32. [39]

    Kvasir-seg: A segmented polyp dataset

    Debesh Jha, Pia H Smedsrud, Michael A Riegler, Pål Halvorsen, Thomas De Lange, Dag Johansen, and Håvard D Johansen. Kvasir-seg: A segmented polyp dataset. InMultiMedia modeling: 26th international conference, MMM 2020, Daejeon, South Korea, January 5–8, 2020, proceedings, part...

    2020

  33. [40]

    Amos: A large-scale abdominal multi-organ benchmark for versatile medical image segmentation.Advances in neural information processing systems, 35:36722–36732, 2022

    Yuanfeng Ji, Haotian Bai, Chongjian Ge, Jie Yang, Ye Zhu, Ruimao Zhang, Zhen Li, Lingyan Zhanng, Wanling Ma, Xiang Wan, et al. Amos: A large-scale abdominal multi-organ benchmark for versatile medical image segmentation.Advances in neural information processing systems, 35:367...

    2022

  34. [41]

    Mimic-cxr, a de-identified publicly available database of chest radiographs with free-text reports.Scientific data, 6(1):317, 2019

    Alistair EW Johnson, Tom J Pollard, Seth J Berkowitz, Nathaniel R Greenbaum, Matthew P Lungren, Chih-ying Deng, Roger G Mark, and Steven Horng. Mimic-cxr, a de-identified publicly available database of chest radiographs with free-text reports.Scientific data, 6(1):317, 2019

    2019

  35. [42]

    Why we should not mistake accuracy of medical ai for efficiency.NPJ Digital Medicine, 7(1):57, 2024

    Karin Rolanda Jongsma, Martin Sand, and Megan Milota. Why we should not mistake accuracy of medical ai for efficiency.NPJ Digital Medicine, 7(1):57, 2024

    2024

  36. [43]

    Multi-class texture analysis in colorectal cancer histology.Scientific reports, 6(1):1–11, 2016

    Jakob Nikolas Kather, Cleo-Aron Weis, Francesco Bianconi, Susanne M Melchers, Lothar R Schad, Timo Gaiser, Alexander Marx, and Frank Gerrit Zöllner. Multi-class texture analysis in colorectal cancer histology.Scientific reports, 6(1):1–11, 2016. 24

    2016

  37. [44]

    Chaos challenge- combined (ct-mr) healthy abdominal organ segmentation.Medical image analysis, 69:101950, 2021

    A Emre Kavur, N Sinem Gezer, Mustafa Barı¸ s, Sinem Aslan, Pierre-Henri Conze, Vladimir Groza, Duc Duy Pham, Soumick Chatterjee, Philipp Ernst, Sava¸ s Özkan, et al. Chaos challenge- combined (ct-mr) healthy abdominal organ segmentation.Medical image analysis, 69:101950, 2021

    2021

  38. [45]

    Transparent medical image ai via an image–text foundation model grounded in medical literature.Nature Medicine, 30(4):1154–1165, 2024

    Chanwoo Kim, Soham U Gadgil, Alex J DeGrave, Jesutofunmi A Omiye, Zhuo Ran Cai, Roxana Daneshjou, and Su-In Lee. Transparent medical image ai via an image–text foundation model grounded in medical literature.Nature Medicine, 30(4):1154–1165, 2024

    2024

  39. [46]

    A caution against customized ai in healthcare.npj Digital Medicine, 8(1):13, 2025

    Kristin M Kostick-Quenet. A caution against customized ai in healthcare.npj Digital Medicine, 8(1):13, 2025

    2025

  40. [47]

    Paras Lakhani, John Mongan, Chinmay Singhal, Quan Zhou, Katherine P Andriole, William F Auffermann, PM Prasanna, Theresa X Pham, Michael Peterson, Peter J Bergquist, et al. The 2021 siim-fisabio-rsna machine learning covid-19 challenge: Annotation and standard exam classificat...

    2021

  41. [48]

    Emidec: a database usable for the automatic evaluation of myocardial infarction from delayed-enhancement cardiac mri.Data, 5(4):89, 2020

    Alain Lalande, Zhihao Chen, Thomas Decourselle, Abdul Qayyum, Thibaut Pommier, Luc Lor- gis, Ezequiel de La Rosa, Alexandre Cochet, Yves Cottin, Dominique Ginhac, et al. Emidec: a database usable for the automatic evaluation of myocardial infarction from delayed-enhancement ca...

    2020

  42. [49]

    Miccai multi-atlas labeling beyond the cranial vault–workshop and challenge

    Bennett Landman, Zhoubing Xu, J Igelsias, Martin Styner, T Langerak, and Arno Klein. Miccai multi-atlas labeling beyond the cranial vault–workshop and challenge. InProc. MICCAI Multi-Atlas Labeling Beyond Cranial Vault—Workshop Challenge, volume 5, page 12, 2015

    2015

  43. [50]

    A dataset of clinically generated visual questions and answers about radiology images.Scientific data, 5(1): 1–10, 2018

    Jason J Lau, Soumya Gayen, Asma Ben Abacha, and Dina Demner-Fushman. A dataset of clinically generated visual questions and answers about radiology images.Scientific data, 5(1): 1–10, 2018

    2018

  44. [51]

    Llava-med: Training a large language-and-vision assistant for biomedicine in one day.Advances in Neural Information Processing Systems, 36, 2024

    Chunyuan Li, Cliff Wong, Sheng Zhang, Naoto Usuyama, Haotian Liu, Jianwei Yang, Tristan Naumann, Hoifung Poon, and Jianfeng Gao. Llava-med: Training a large language-and-vision assistant for biomedicine in one day.Advances in Neural Information Processing Systems, 36, 2024

    2024

  45. [52]

    Pmc-clip: Contrastive language-image pre-training using biomedical documents

    Weixiong Lin, Ziheng Zhao, Xiaoman Zhang, Chaoyi Wu, Ya Zhang, Yanfeng Wang, and Weidi Xie. Pmc-clip: Contrastive language-image pre-training using biomedical documents. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 525–536. S...

    2023

  46. [53]

    Evaluation of prostate segmentation algorithms for mri: the promise12 challenge.Medical image analysis, 18 (2):359–373, 2014

    Geert Litjens, Robert Toth, Wendy Van De Ven, Caroline Hoeks, Sjoerd Kerkstra, Bram Van Gin- neken, Graham Vincent, Gwenael Guillard, Neil Birbeck, Jindang Zhang, et al. Evaluation of prostate segmentation algorithms for mri: the promise12 challenge.Medical image analysis, 18 ...

    2014

  47. [54]

    Deepseek-v3 technical report.arXiv preprint arXiv:2412.19437, 2024

    Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report.arXiv preprint arXiv:2412.19437, 2024

    2024 arXiv

  48. [55]

    Slake: A semantically- labeled knowledge-enhanced dataset for medical visual question answering

    Bo Liu, Li-Ming Zhan, Li Xu, Lin Ma, Yan Yang, and Xiao-Ming Wu. Slake: A semantically- labeled knowledge-enhanced dataset for medical visual question answering. In2021 IEEE 18th international symposium on biomedical imaging (ISBI), pages 1650–1654. IEEE, 2021

    2021

  49. [56]

    Visual instruction tuning.Advances in neural information processing systems, 36:34892–34916, 2023

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning.Advances in neural information processing systems, 36:34892–34916, 2023

    2023

  50. [57]

    A visual-language foundation model for computational pathology.Nature Medicine, 30(3):863–874, 2024

    Ming Y Lu, Bowen Chen, Drew FK Williamson, Richard J Chen, Ivy Liang, Tong Ding, Guil- laume Jaume, Igor Odintsov, Long Phi Le, Georg Gerber, et al. A visual-language foundation model for computational pathology.Nature Medicine, 30(3):863–874, 2024

    2024

  51. [58]

    Word: A large scale dataset, benchmark and clinical applicable study for abdominal organ segmentation from ct image

    Xiangde Luo, Wenjun Liao, Jianghong Xiao, Jieneng Chen, Tao Song, Xiaofan Zhang, Kang Li, Dimitris N Metaxas, Guotai Wang, and Shaoting Zhang. Word: A large scale dataset, benchmark and clinical applicable study for abdominal organ segmentation from ct image. Medical Image Ana...

    2022

  52. [59]

    A new dataset of computed-tomography angiography images for computer-aided detection of pulmonary embolism.Scientific Data, 5 (180180), 2018

    Mojtaba Masoudi, Hamid-Reza Pourreza, Mahdi Saadatmand-Tarzjan, Noushin Eftekhari, Fateme Shafiee Zargar, and Masoud Pezeshki Rad. A new dataset of computed-tomography angiography images for computer-aided detection of pulmonary embolism.Scientific Data, 5 (180180), 2018

    2018

  53. [60]

    Christian Matek, Sebastian Krappe, Christian Münzenmayer, Torsten Haferlach, and Carsten Marr. Highly accurate differentiation of bone marrow cell morphologies using deep neural networks on a large image data set.Blood, The Journal of the American Society of Hematology, 138(20...

    1917

  54. [61]

    Md Ashraful Alam Milton. Automated skin lesion classification using ensemble of deep neural networks in isic 2018: Skin lesion analysis towards melanoma detection challenge.arXiv preprint arXiv:1901.10802, 2019

    2018 arXiv

  55. [62]

    Knee osteoarthritis detection and severity classification using residual neural networks on preprocessed x-ray images.Diagnostics, 13(8):1380, 2023

    Abdul Sami Mohammed, Ahmed Abul Hasanaath, Ghazanfar Latif, and Abul Bashar. Knee osteoarthritis detection and severity classification using residual neural networks on preprocessed x-ray images.Diagnostics, 13(8):1380, 2023

    2023

  56. [63]

    Med-flamingo: a multimodal medical few-shot learner

    Michael Moor, Qian Huang, Shirley Wu, Michihiro Yasunaga, Yash Dalmia, Jure Leskovec, Cyril Zakka, Eduardo Pontes Reis, and Pranav Rajpurkar. Med-flamingo: a multimodal medical few-shot learner. InMachine Learning for Health (ML4H), pages 353–367. PMLR, 2023

    2023

  57. [64]

    Brset: a brazilian multilabel ophthalmo- logical dataset of retina fundus photos.PLOS Digital Health, 3(7):e0000454, 2024

    Luis Filipe Nakayama, David Restrepo, João Matos, Lucas Zago Ribeiro, Fernando Korn Malerbi, Leo Anthony Celi, and Caio Saito Regatieri. Brset: a brazilian multilabel ophthalmo- logical dataset of retina fundus photos.PLOS Digital Health, 3(7):e0000454, 2024

    2024

  58. [65]

    Vindr-cxr: An open dataset of chest x-rays with radiologist’s annotations.Scientific Data, 9(1):429, 2022

    Ha Q Nguyen, Khanh Lam, Linh T Le, Hieu H Pham, Dat Q Tran, Dung B Nguyen, Dung D Le, Chi M Pham, Hang TT Tong, Diep H Dinh, et al. Vindr-cxr: An open dataset of chest x-rays with radiologist’s annotations.Scientific Data, 9(1):429, 2022

    2022

  59. [66]

    Vindr-spinexr: A deep learning framework for spinal lesions detection and classification from radiographs

    Hieu T Nguyen, Hieu H Pham, Nghia T Nguyen, Ha Q Nguyen, Thang Q Huynh, Minh Dao, and Van Vu. Vindr-spinexr: A deep learning framework for spinal lesions detection and classification from radiographs. InMedical Image Computing and Computer Assisted Intervention–MICCAI 2021: 24...

    2021

  60. [67]

    Vindr-mammo: A large-scale benchmark dataset for computer-aided diagnosis in full-field digital mammography.Scientific Data, 10(1):277, 2023

    Hieu T Nguyen, Ha Q Nguyen, Hieu H Pham, Khanh Lam, Linh T Le, Minh Dao, and Van Vu. Vindr-mammo: A large-scale benchmark dataset for computer-aided diagnosis in full-field digital mammography.Scientific Data, 10(1):277, 2023

    2023

  61. [68]

    Pad-ufes-20: A skin lesion dataset composed of patient data and clinical images collected from smartphones.Data in brief, 32:106221, 2020

    Andre GC Pacheco, Gustavo R Lima, Amanda S Salomao, Breno Krohling, Igor P Biral, Gabriel G de Angelo, Fábio CR Alves Jr, José GM Esgario, Alana C Simora, Pedro BC Castro, et al. Pad-ufes-20: A skin lesion dataset composed of patient data and clinical images collected from sma...

    2020

  62. [69]

    Foundation model for cancer imaging biomarkers.Nature machine intelligence, 6(3):354–367, 2024

    Suraj Pai, Dennis Bontempi, Ibrahim Hadzic, Vasco Prudente, Mateo Sokaˇc, Tafadzwa L Chaun- zwa, Simon Bernatz, Ahmed Hosny, Raymond H Mak, Nicolai J Birkbak, et al. Foundation model for cancer imaging biomarkers.Nature machine intelligence, 6(3):354–367, 2024

    2024

  63. [70]

    Retinal fundus multi-disease image dataset (rfmid) 2.0: a dataset of frequently and rarely identified diseases.Data, 8(2):29, 2023

    Sachin Panchal, Ankita Naik, Manesh Kokare, Samiksha Pachade, Rushikesh Naigaonkar, Prerana Phadnis, and Archana Bhange. Retinal fundus multi-disease image dataset (rfmid) 2.0: a dataset of frequently and rarely identified diseases.Data, 8(2):29, 2023

    2023

  64. [71]

    Bleu: a method for automatic evaluation of machine translation

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for automatic evaluation of machine translation. InProceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311–318, 2002

    2002

  65. [72]

    Vindr-pcxr: An open, large-scale pediatric chest x-ray dataset for interpretation of common thoracic diseases.PhysioNet (version 1.0

    H Hieu Pham, T Thanh Tran, and Ha Quy Nguyen. Vindr-pcxr: An open, large-scale pediatric chest x-ray dataset for interpretation of common thoracic diseases.PhysioNet (version 1.0. 0), 10(2), 2022

    2022

  66. [73]

    Mura: Large dataset for abnormality detection in musculoskeletal radiographs.arXiv preprint arXiv:1712.06957, 2017

    Pranav Rajpurkar, Jeremy Irvin, Aarti Bagul, Daisy Ding, Tony Duan, Hershel Mehta, Brandon Yang, Kaylie Zhu, Dillon Laird, Robyn L Ball, et al. Mura: Large dataset for abnormality detection in musculoskeletal radiographs.arXiv preprint arXiv:1712.06957, 2017. 26

    2017 arXiv

  67. [74]

    Ai in health and medicine

    Pranav Rajpurkar, Emma Chen, Oishi Banerjee, and Eric J Topol. Ai in health and medicine. Nature medicine, 28(1):31–38, 2022

    2022

  68. [75]

    Multimodal generative ai for medical image interpretation.Nature, 639(8056):888–896, 2025

    Vishwanatha M Rao, Michael Hla, Michael Moor, Subathra Adithan, Stephen Kwak, Eric J Topol, and Pranav Rajpurkar. Multimodal generative ai for medical image interpretation.Nature, 639(8056):888–896, 2025

    2025

  69. [76]

    Brain hemorrhage extended (bhx): Bounding box extrapolation from thick to thin slice ct images.PhysioNet, 101(23):e215–20, 2020

    Eduardo Pontes Reis, Felipe Nascimento, Mateus Aranha, Fernando Mainetti Secol, Birajara Machado, Marcelo Felix, Anouk Stein, and Edson Amaro. Brain hemorrhage extended (bhx): Bounding box extrapolation from thick to thin slice ct images.PhysioNet, 101(23):e215–20, 2020

    2020

  70. [77]

    Ct-org, a new dataset for multiple organ segmentation in computed tomography.Scientific Data, 7(1): 381, 2020

    Blaine Rister, Darvin Yi, Kaushik Shivakumar, Tomomi Nobashi, and Daniel L Rubin. Ct-org, a new dataset for multiple organ segmentation in computed tomography.Scientific Data, 7(1): 381, 2020

    2020

  71. [78]

    A large annotated medical image dataset for the development and evaluation of segmentation algorithms.arXiv preprint arXiv:1902.09063, 2019

    Amber L Simpson, Michela Antonelli, Spyridon Bakas, Michel Bilello, Keyvan Farahani, Bram Van Ginneken, Annette Kopp-Schneider, Bennett A Landman, Geert Litjens, Bjoern Menze, et al. A large annotated medical image dataset for the development and evaluation of segmentation alg...

    1902 arXiv

  72. [79]

    Chexbert: combining automatic labelers and expert annotations for accurate radiology report labeling using bert.arXiv preprint arXiv:2004.09167, 2020

    Akshay Smit, Saahil Jain, Pranav Rajpurkar, Anuj Pareek, Andrew Y Ng, and Matthew P Lungren. Chexbert: combining automatic labelers and expert annotations for accurate radiology report labeling using bert.arXiv preprint arXiv:2004.09167, 2020

    2004 arXiv

  73. [81]

    Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution.arXiv preprint arXiv:2409.12191, 2024

    Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, et al. Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution.arXiv preprint arXiv:2409.12191, 2024

    2024 arXiv

  74. [82]

    The extreme cardiac mri analysis challenge under respiratory motion (cmrxmotion), 2022

    Shuo Wang, Chen Qin, Chengyan Wang, Kang Wang, Haoran Wang, Chen Chen, Cheng Ouyang, Xutong Kuang, Chengliang Dai, Yuanhan Mo, Zhang Shi, Chenchen Dai, Xinrong Chen, He Wang, and Wenjia Bai. The extreme cardiac mri analysis challenge under respiratory motion (cmrxmotion), 2022

    2022

  75. [83]

    Totalseg- mentator: robust segmentation of 104 anatomic structures in ct images.Radiology: Artificial Intelligence, 5(5):e230024, 2023

    Jakob Wasserthal, Hanns-Christian Breit, Manfred T Meyer, Maurice Pradella, Daniel Hinck, Alexander W Sauter, Tobias Heye, Daniel T Boll, Joshy Cyriac, Shan Yang, et al. Totalseg- mentator: robust segmentation of 104 anatomic structures in ct images.Radiology: Artificial Intel...

    2023

  76. [84]

    Towards generalist foundation model for radiology.arXiv preprint arXiv:2308.02463, 2023

    Chaoyi Wu, Xiaoman Zhang, Ya Zhang, Yanfeng Wang, and Weidi Xie. Towards generalist foundation model for radiology.arXiv preprint arXiv:2308.02463, 2023

    2023 arXiv

  77. [85]

    Chest imagenome dataset for clinical reasoning.arXiv preprint arXiv:2108.00316, 2021

    Joy T Wu, Nkechinyere N Agu, Ismini Lourentzou, Arjun Sharma, Joseph A Paguio, Jasper S Yao, Edward C Dee, William Mitchell, Satyananda Kashyap, Andrea Giovannini, et al. Chest imagenome dataset for clinical reasoning.arXiv preprint arXiv:2108.00316, 2021

    2021 arXiv

  78. [86]

    Gamma challenge: glaucoma grading from multi-modality images.Medical Image Analysis, 90:102938, 2023

    Junde Wu, Huihui Fang, Fei Li, Huazhu Fu, Fengbin Lin, Jiongcheng Li, Yue Huang, Qinji Yu, Sifan Song, Xinxing Xu, et al. Gamma challenge: glaucoma grading from multi-modality images.Medical Image Analysis, 90:102938, 2023

    2023

  79. [87]

    A vision–language foundation model for precision oncology.Nature, pages 1–10, 2025

    Jinxi Xiang, Xiyue Wang, Xiaoming Zhang, Yinghua Xi, Feyisope Eweje, Yijiang Chen, Yuchen Li, Colin Bergstrom, Matthew Gopaulchan, Ted Kim, et al. A vision–language foundation model for precision oncology.Nature, pages 1–10, 2025

    2025

  80. [88]

    Medmnist v2-a large-scale lightweight benchmark for 2d and 3d biomedical image classification.Scientific Data, 10(1):41, 2023

    Jiancheng Yang, Rui Shi, Donglai Wei, Zequan Liu, Lin Zhao, Bilian Ke, Hanspeter Pfister, and Bingbing Ni. Medmnist v2-a large-scale lightweight benchmark for 2d and 3d biomedical image classification.Scientific Data, 10(1):41, 2023. 27

    2023

  81. [89]

    Sa-med2d-20m dataset: Segment anything in 2d medical imaging with 20 million masks.arXiv preprint arXiv:2311.11969, 2023

    Jin Ye, Junlong Cheng, Jianpin Chen, Zhongying Deng, Tianbin Li, Haoyu Wang, Yanzhou Su, Ziyan Huang, Jilong Chen, Lei Jiang, et al. Sa-med2d-20m dataset: Segment anything in 2d medical imaging with 20 million masks.arXiv preprint arXiv:2311.11969, 2023

    2023 arXiv

  82. [90]

    Gmai-mmbench: A comprehensive multimodal evaluation benchmark towards general medical ai.Advances in Neural Information Processing Systems, 37:94327–94427, 2024

    Jin Ye, Guoan Wang, Yanjun Li, Zhongying Deng, Wei Li, Tianbin Li, Haodong Duan, Ziyan Huang, Yanzhou Su, Benyou Wang, et al. Gmai-mmbench: A comprehensive multimodal evaluation benchmark towards general medical ai.Advances in Neural Information Processing Systems, 37:94327–94...

    2024

  83. [91]

    A generalist vision–language foundation model for diverse biomedical tasks.Nature Medicine, pages 1–13, 2024

    Kai Zhang, Rong Zhou, Eashan Adhikarla, Zhiling Yan, Yixin Liu, Jun Yu, Zhengliang Liu, Xun Chen, Brian D Davison, Hui Ren, et al. A generalist vision–language foundation model for diverse biomedical tasks.Nature Medicine, pages 1–13, 2024

    2024

  84. [92]

    Multi-site, multi-domain airway tree modeling.Medical image analysis, 90:102957, 2023

    Minghui Zhang, Yangqian Wu, Hanxiao Zhang, Yulei Qin, Hao Zheng, Wen Tang, Corey Arnold, Chenhao Pei, Pengxin Yu, Yang Nan, et al. Multi-site, multi-domain airway tree modeling.Medical image analysis, 90:102957, 2023

    2023

  85. [93]

    A multimodal biomedical foundation model trained from fifteen million image–text pairs.NEJM AI, 2(1):AIoa2400640, 2025

    Sheng Zhang, Yanbo Xu, Naoto Usuyama, Hanwen Xu, Jaspreet Bagga, Robert Tinn, Sam Preston, Rajesh Rao, Mu Wei, Naveen Valluri, et al. A multimodal biomedical foundation model trained from fifteen million image–text pairs.NEJM AI, 2(1):AIoa2400640, 2025

    2025

  86. [94]

    Pmc-vqa: Visual instruction tuning for medical visual question answering.arXiv preprint arXiv:2305.10415, 2023

    Xiaoman Zhang, Chaoyi Wu, Ziheng Zhao, Weixiong Lin, Ya Zhang, Yanfeng Wang, and Weidi Xie. Pmc-vqa: Visual instruction tuning for medical visual question answering.arXiv preprint arXiv:2305.10415, 2023

    2023 arXiv

  87. [95]

    segmenting small targets in a large background

    Yukun Zhou, Mark A Chia, Siegfried K Wagner, Murat S Ayhan, Dominic J Williamson, Robbert R Struyven, Timing Liu, Moucheng Xu, Mateo G Lozano, Peter Woodward-Court, et al. A foundation model for generalizable disease detection from retinal images.Nature, 622 (7981):156–163, 20...

    2023

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.