Pith. sign in

REVIEW 4 major objections 6 minor 127 references

A cascaded multi-encoder vision stack plus region-grounded scoring lets a medical multimodal model match radiologists more closely than prior open systems and several proprietary APIs.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-31 06:16 UTC pith:C5CBTD7Y

load-bearing objection Solid systems paper with a real multi-encoder+native-3D design and a useful radiologist study; flagship report SOTA numbers are soft until train/test decontamination is shown. the 4 major comments →

arxiv 2607.24743 v2 pith:C5CBTD7Y submitted 2026-07-27 cs.CV cs.AIcs.CL

ClinFusion: A Vision-Centric Multimodal LLM System for Holistic Medical Understanding

Hangjie Yuan , Yichen Qian , Zhiwei Tang , Xianzhe Xu , Lirong Wu , Sicheng Yang , Jinwang Wang , Pengju Wang
show 16 more authors
Zhitao Zeng Yizeng Han Yan Xing Shengxuan Luo Tao Feng Qing Xie Weigen Yao Yi Yang Zuozhu Liu Jiasheng Tang Shaocheng Wang Jitao Wang Jiahong Dong Weihua Chen Feng Xu Fan Wang
This is my paper
classification cs.CV cs.AIcs.CL
keywords multimodal large language modelsmedical imagingcompositional vision encoderCaSL Fusion3D CT/MRI understandingreport generation evaluationinstruction followingagentic tool use
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

ClinFusion argues that medical multimodal models fail in the clinic mainly because they see poorly and are scored poorly. A single vision backbone cannot absorb heterogeneous 2D and native 3D medical images, and standard report metrics reward wording rather than factual, region-level diagnosis. The authors build a compositional vision encoder that starts from a strong base vision transformer and progressively fuses specialist 2D features, then anchors native 3D features to those 2D representations through Cascade Spatial-Aware Locality Fusion. They also introduce MedIF-Bench for medical instruction-following and an RoI-grounded report protocol that conditions generation on clinical indication and anatomical focus, then scores matched, missed, and hallucinated findings with an LLM judge. On a large suite of 2D/3D VQA, report, instruction-following, and text medical tasks, ClinFusion leads most open medical models and many proprietary multimodal comparisons; six radiologists ranking 300 cases put its reports first, and the RoI metric correlates best with their judgment. Agentic retrieval and specialist perception tools further lift performance in realistic workflows.

Core claim

Holistic medical understanding is achieved by treating the problem as vision-centric: a cascaded compositional encoder with CaSL Fusion unifies diverse 2D and native 3D medical images into a progressively enriched, language-aligned representation, and RoI-grounded evaluation plus MedIF-Bench make instruction-following and report quality measurable in clinically meaningful, factual terms—yielding SOTA open-model results, competitive proprietary multimodal performance, top radiologist rankings, and the automatic metric most aligned with experts.

What carries the argument

Cascade Spatial-Aware Locality Fusion (CaSL Fusion): an asymmetric, cascaded local cross-attention operator that keeps a foundational well-aligned vision representation as the query backbone, enriches it from specialist 2D encoders in a spatial neighborhood, and, for volumes, uses 2D-anchored depth-aware fusion so native 3D features align through those 2D anchors.

Load-bearing premise

That clinical context pulled from ground-truth reports without leaking findings, plus an LLM judge labeling matched, missed, and hallucinated claims, is a fair enough yardstick of report quality and model ranking.

What would settle it

A blinded multi-radiologist study on held-out cases where RoI context is independently verified for leakage, claim labels are audited against expert adjudication, and ClinFusion no longer ranks first or the RoI metric no longer best correlates with expert rankings would overturn the central clinical claims.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Medical MLLMs should keep a strong base encoder and cascade specialist and native 3D features rather than forcing all volumes into 2D slice stacks or a single monolithic encoder.
  • Report generation benchmarks should condition on indication and anatomical focus and score factual matched/missed/hallucinated findings instead of only lexical or closed-schema entity overlap.
  • Instruction-following must be measured after medical fine-tuning; format failure alone can make knowledge look weaker than it is.
  • Agentic retrieval and specialist perception tools can raise both text and multimodal clinical utility without replacing the core vision stack.
  • Radiologist ranking studies can validate which automatic metrics actually track clinical quality.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If RoI extraction becomes standard, public leaderboards may reorder models that today look strong only under context-free or surface-matching scores.
  • The 2D-anchored 3D design suggests future modalities (e.g., dynamic ultrasound or multi-phase MRI) could plug in as additional specialists without full re-alignment of the language model.
  • Preserving instruction-following through medical adaptation may matter as much as raw VQA points for electronic-health-record integration.
  • Hospital-scale pseudo-reports from segmentation tools imply that volumetric supervision can grow faster than expert-written reports if quality control holds.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper presents ClinFusion, a medical MLLM built on Qwen3-VL (8B/32B) with a compositional vision encoder: specialist 2D encoders (DINOv2, ConvNeXt) and a native 3D encoder (PE-3D) are fused into the Qwen ViT representation via a cascaded local cross-attention operator (CaSL Fusion), including a 2D-anchored depth-aware variant for volumetric data. A five-stage progressive training recipe on 22.2M curated samples is described. On the evaluation side, the paper introduces MedIF-Bench (900 regex-verified instruction-following items) and an RoI-grounded report-generation protocol: clinical indication and areas of focus are extracted from the ground-truth report with a leakage-blocking prompt, the model generates a focused report, and an LLM judge (GPT-4.1) decomposes findings into matched/missed/hallucinated sets yielding P/R/F1. Headline claims: SOTA on 20/24 benchmarks vs. open-source medical MLLMs, better than GPT-5.2/Gemini-3-Flash on 13/16 multimodal benchmarks, top-ranked reports in a blinded six-radiologist study (300 cases), and the highest expert correlation (τ=0.511) among eleven automatic metrics. An agentic RAG + perception-tool extension gives further gains. Code and weights are released.

Significance. If the results hold, the paper makes three useful contributions: (i) a compositional 2D/3D vision encoder with a concrete, ablated fusion operator (CaSL) that beats parallel fusion and global cross-attention under controlled comparisons; (ii) a clinically motivated report-generation protocol whose design choices (context conditioning, judge-based claim decomposition) are directly ablated against RadGraph-F1, including a length-bias stress test; and (iii) MedIF-Bench, released with code. Particularly creditable are the unified re-benchmarking of all baselines under one kit, the blinded six-radiologist study on 300 cases with inter-annotator agreement reporting, the architecture ablations, and the release of model weights (8B/32B) and code. The 'strongest correlation with expert judgment' claim is supported by a real human study rather than asserted. The agentic extension is incremental but honestly framed as an extension.

major comments (4)
  1. [§2.2, §4.3; Extended Data Tables 1–2] The flagship report-generation and VQA margins are measured on benchmarks whose source datasets appear in the training corpus, and the manuscript contains no decontamination statement. Extended Data Table 1 lists IU-Xray and open-i (Stage 1), CheXpert Plus, PMC-VQA, SLAKE, VQA-RAD, PathVQA and their recaptions (Stage 3), CT-Rate-report and AMOS-MM-report (Stage 4), and 3D-RAD (Stage 5) as training data; Extended Data Table 2 then evaluates on CheXpert-Plus (200 cases), IU-Xray (296), CT-Rate Report (3,039), AMOS-MM Report (400), 3D-RAD, PMC-VQA, SLAKE, PathVQA, and VQA-RAD, with OmniMedVQA partially subsuming SLAKE/VQA-RAD/PathVQA. If training ingested full datasets (or recaptions/pseudo-labels derived from them) without restricting to official train splits and excluding evaluation IDs, the model may have seen the reference reports against which the RoI-grounded F1 is computed, inflating
  2. [§4.2.3, Extended Data Fig. 6] The RoI-grounded protocol conditions report generation on clinical context extracted from the ground-truth report itself, and the only leakage safeguard is a prompt-level 'CRITICAL SECURITY CONSTRAINT' (Extended Data Fig. 6a) with no quantitative audit. Two concrete risks: (i) the extracted 'Areas of Focus' enumerate the anatomical systems the GT discusses, which is itself signal about where findings live; (ii) nothing verifies that GPT-4.1's extraction never smuggles finding content (e.g., the worked example's indication 'known or suspected infectious disease' already biases toward consolidation-type findings present in that GT). Because the same extracted context is given to all compared models the comparison is internally fair, but the absolute F1 numbers — and the claim that this protocol is 'clinically aligned' — depend on leakage being negligible. Please add an audit: e.g., human o
  3. [§2.3, §2.4.1, Fig. 5b–c] The metric-validation argument is partially circular: the authors introduce the RoI-grounded F1, use it as the primary report metric, and then conclude it 'best reflects expert clinical judgment' (§2.3, Fig. 5b). The six-radiologist study is a genuine mitigating strength, but the correlation analysis (Kendall's τ = 0.511, Top-1 accuracy 55.5%) is computed over only four systems (ClinFusion+agent, ClinFusion, Gemini-3-Flash, Hulu-Med), two of which are the authors' own model; with such few items and system-level rankings, discriminating among eleven metrics is noisy, and the per-case τ distribution (Fig. 5c) needs confidence intervals or a bootstrap test to support 'strongest correlation'. Relatedly, the LLM judge's TP/FN/FP labels are never validated against human extraction: please report a human spot-check of the judge's claim decomposition on a sample (per-category precision/recall of
  4. [§2.3, Fig. 5a] In the radiologist study (Fig. 5a) significance is reported only for ClinFusion+Agent vs. Hulu-Med and vs. Gemini-3-Flash on the Overall dimension; the bracket in the caption mentions 'n.s.' without saying which comparison it refers to (presumably standalone ClinFusion vs. ClinFusion+Agent, or a per-dimension test). Since the paper's top-line claim is 'highest-ranked reports,' please clarify all pairwise tests (standalone vs. baselines, standalone vs. agentic) across the three dimensions, and state whether the agentic system's tool outputs (which include external segmentation/classification models) were available to radiologists as evidence or folded into the report text — this affects what exactly was being ranked.
minor comments (6)
  1. [§4.1.1–4.1.2, Eqs. (4), (6), (10)] Cross-reference errors in the Methods: §4.1.1 ('CaSL Fusion with stochastic residual regularization') refers to 'the residual connection in the fusion block (equation (10))', but the residual connection is Eq. (4); Eq. (10) is defined later in §4.1.2. Similarly §4.1.2 says 'residual connections defined in equation (10)' while defining Eq. (10) itself — presumably Eq. (4)/(6) is meant.
  2. [Extended Data Figs. 1, 4] Internal editing artifacts remain in the Extended Data: Chinese annotations such as '2D3D Case 合并（最终选择使⽤）', '⾄威的合并图1', and '下⾯这个被弃⽤了' ('the one below is deprecated') appear in/around Extended Data Figs. 1 and 4. These should be removed before publication.
  3. [§2.3, Fig. 5a] Terminology inconsistency: §2.3 defines the third ranking dimension as 'Clinical Utility', while Fig. 5a and its caption use 'Operability'. Please unify.
  4. [§2.1.2] Reproducibility of proprietary-model comparisons: API snapshot dates/versions for GPT-5.2, Gemini-3-Flash, and Claude-Sonnet-4.5 are not stated, nor is how 3D volumes were provided to them (slice count, montaging). Given the '13 of 16' claim against proprietary models, these details should be documented.
  5. [Extended Data Fig. 6a] Extended Data Fig. 6a's example GT report is internally contradictory ('bilateral minimal pleural effusion... No pleural or pericardial effusion was detected'). If this reflects source-data noise, a sentence on how noisy references affect the F1 protocol would strengthen the evaluation section; a cleaner example would help the reader.
  6. [Abstract, §2.2] The abstract and §2.2 state '20 out of 24 benchmarks' without identifying the four losses; a compact win/loss breakdown (possibly in Supplementary Tables) would make the claim auditable.

Circularity Check

0 steps flagged

No derivation-circularity: empirical MLLM claims rest on external benchmarks and a blinded radiologist study, not on results true by construction.

full rationale

ClinFusion is an empirical systems paper (compositional vision encoder + CaSL Fusion, MedIF-Bench, RoI-grounded report metric, agentic tools). Its load-bearing claims are comparative performance numbers and a six-radiologist blinded ranking on 300 cases, not a first-principles derivation in which outputs equal fitted or definitional inputs. Architecture choices are supported by ablations against alternative fusion/encoders on the same protocol; SOTA tables compare against independent open-source and proprietary models under a unified kit. The RoI-grounded F1 is author-proposed and used as a primary report metric, but its “strongest correlation with experts” claim is an empirical rank-correlation against external radiologist consensus (vs. eleven other metrics), not a quantity forced by how the metric is defined—and ClinFusion’s clinical advantage is separately corroborated by the blinded human ranking. No self-definitional identity, fitted-parameter-as-prediction, load-bearing self-cited uniqueness theorem, or renamed known law appears in the argument chain. Train/test overlap risks (if any) are contamination/correctness issues outside this circularity rubric. Finding: no significant circularity.

Axiom & Free-Parameter Ledger

5 free parameters · 6 axioms · 4 invented entities

This is an empirical systems paper. The claim rests on standard deep-learning practice, medical-data assumptions, and several paper-specific modeling and evaluation choices rather than a formal derivation. The largest external dependencies are the Qwen3-VL stack, chosen specialist encoders, synthetic/pseudo labels, and LLM judges.

free parameters (5)
  • CaSL local window sizes (k=3 DINOv2/3D; k=5 ConvNeXt) = k=3 / k=5
    Hand-chosen locality widths that define how much specialist context each foundation token can see; ablated only coarsely via mechanism comparisons.
  • Stochastic residual drop probability p_drop = 0.1
    Training-only Bernoulli gate to force use of specialist paths; default 0.1 without extensive sensitivity analysis in the main text.
  • 2D anchor slice count S and 3D depth tokens D_vol = S=4, D_vol=4
    Controls how much of a volume is represented via 2D anchors versus native 3D features (S=4, D_vol=4 from 32-frame inputs).
  • Progressive training schedule (stages, epochs, LR, batch sizes) = LR 1e-5; stages 1–5 epochs 1/1/3/1/3; batch 128–256
    Curriculum and optimization hyperparameters that materially affect whether fusion modules and the 3D encoder learn useful features.
  • LLM-as-judge model choice and prompts for open VQA and RoI factuality = gpt-4.1-2025-04-14
    Primary scoring machinery for open answers and report F1 depends on a specific proprietary judge (gpt-4.1-2025-04-14) and long prompts.
axioms (6)
  • domain assumption Cross-attention over projected, spatially resized encoder maps can enrich a language-aligned foundation ViT without destroying vision-language alignment.
    Core premise of CaSL Fusion and DeepStack integration (Methods 4.1); supported by ablations but not theoretically guaranteed.
  • domain assumption A small set of 2D slices plus a pre-aligned 3D encoder suffices to transfer 2D semantics into volumetric understanding.
    2D-anchored depth-aware fusion (Methods 4.1.2); 3D-off ablation shows drops but does not prove clinical sufficiency.
  • ad hoc to paper Radiology reports can be fairly scored by restricting generation to RoIs inferred from the reference report and counting finding-level TP/FN/FP with an LLM judge.
    Defines the paper’s preferred report metric (Methods 4.2.3); central to report SOTA and expert-correlation claims.
  • ad hoc to paper Format compliance on MedIF-Bench is a valid prerequisite signal for clinical usability and benchmark evaluability.
    Introduced to explain degraded medical fine-tunes and low scores of models like MedGemma on other suites (§2.2.3).
  • domain assumption Segmentation-tool pseudo-reports and dual-VLM dense captions are good enough supervision for perception gains at scale.
    Data synthesis pipelines (Section 4.3 / Supp S1) underpin much of the 3D and caption training signal.
  • standard math Standard transformer training, contrastive 3D pre-alignment, and benchmark accuracy/F1 comparisons are appropriate evidence for ‘holistic medical understanding.’
    Ordinary ML evaluation assumptions used throughout Results; no formal proof obligations.
invented entities (4)
  • CaSL Fusion (Cascade Spatial-Aware Locality Fusion), including 2D-anchored depth-aware variant independent evidence
    purpose: Progressively fuse specialist 2D and native 3D features into a foundation ViT representation with local spatial bias.
    Named operator is the architectural centerpiece; independent evidence is empirical ablations and downstream benchmarks, not external theory.
  • MedIF-Bench independent evidence
    purpose: Measure medical instruction/format following separately from medical correctness.
    New 900-sample benchmark constructed by the authors; usefulness depends on regex tasks reflecting real deployment needs.
  • RoI-grounded report generation evaluation independent evidence
    purpose: Context-condition report generation and score matched/missed/hallucinated abnormalities via LLM judge.
    Paper-defined metric used for primary report tables; partially validated by radiologist correlation study.
  • ClinFusion agentic tool-use extension (hybrid RAG + perception experts) independent evidence
    purpose: Add retrieval and specialist vision tools on top of the base MLLM for verifiable clinical workflows.
    System-level wrapper rather than a new physical entity; gains reported mainly as add-on experiments.

pith-pipeline@v1.2.0-grok45-kimik3 · 57940 in / 4489 out tokens · 100931 ms · 2026-07-31T06:16:38.477296+00:00 · methodology

0 comments
read the original abstract

Multimodal large language models (MLLMs) hold immense potential to revolutionize clinical practice, yet deploying them in the medical domain is fundamentally a vision-centric challenge: models must absorb knowledge from heterogeneous 2D and 3D medical images, and evaluation protocols must align with radiologists' clinical practice and provide an accurate, fine-grained and factualness-driven assessment. In this paper, we introduce ClinFusion, a vision-centric MLLM designed for holistic medical understanding that systematically addresses these limitations. We propose a compositional and cascaded vision encoder architecture featuring a Cascade Spatial-Aware Locality Fusion operator that unifies diverse 2D and native 3D medical image understanding within a fused encoder. We further introduce a vision-grounded evaluation framework, including MedIF-Bench for instruction-following assessment and a region-of-interest-grounded method for clinically aligned and factualness-driven report generation evaluation. We show that ClinFusion sets a new state-of-the-art across a comprehensive suite of 2D and 3D multimodal medical benchmarks---spanning visual question answering, report generation, and instruction following---as well as textual medical tasks, outperforming leading open-source medical MLLMs (\textit{e.g.}, Hulu-Med, Lingshu) on 20 out of 24 benchmarks and demonstrating multimodal capabilities better than powerful proprietary models such as GPT-5.2 and Gemini-3-Flash on 13 out of 16 benchmarks, and can be further augmented with agentic tool use for retrieval-augmented and tool-assisted clinical workflows. A blinded evaluation by board-certified radiologists confirms that ClinFusion produces the highest-ranked reports, and validates our RoI-grounded metric as achieving the strongest correlation with expert judgment among all automatic evaluation metrics examined.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

127 extracted references · 5 canonical work pages

  1. [1]

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report.arXiv preprint arXiv:2502.13923, 2025

    Pith/arXiv arXiv 2025

  2. [2]

    Qwen3-vl technical report.arXiv preprint arXiv:2511.21631, 2025

    Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, Wenbin Ge, Zhifang Guo, Qidong Huang, Jie Huang, Fei Huang, Binyuan Hui, Shutong Jiang, Zhaohai Li, Mingsheng Li, Mei Li, Kaixin Li, Zicheng Lin, Junyang Lin, Xuejing Liu, Jiawei Liu, Chenglong Liu, Yang Liu, Dayiheng Liu, Shixuan ...

    Pith/arXiv arXiv 2025

  3. [3]

    Visual instruction tuning.Advances in neural information processing systems, 36:34892–34916, 2023

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning.Advances in neural information processing systems, 36:34892–34916, 2023

    2023

  4. [4]

    Gpt-4o system card.arXiv preprint arXiv:2410.21276, 2024

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card.arXiv preprint arXiv:2410.21276, 2024

    Pith/arXiv arXiv 2024

  5. [5]

    Improved baselines with visual instruction tuning

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 26296–26306, 2024

    2024

  6. [6]

    Lingshu: A generalist foundation model for unified multimodal medical understanding and reasoning.arXiv preprint arXiv:2506.07044, 2025

    Weiwen Xu, Hou Pong Chan, Long Li, Mahani Aljunied, Ruifeng Yuan, Jianyu Wang, Chenghao Xiao, Guizhen Chen, Chaoqun Liu, Zhaodonghui Li, et al. Lingshu: A generalist foundation model for unified multimodal medical understanding and reasoning.arXiv preprint arXiv:2506.07044, 2025. 20

    Pith/arXiv arXiv 2025

  7. [7]

    Hulu-med: A transparent generalist model towards holistic medical vision-language understanding.arXiv preprint arXiv:2510.08668, 2025

    Songtao Jiang, Yuan Wang, Sibo Song, Tianxiang Hu, Chenyi Zhou, Bin Pu, Yan Zhang, Zhibo Yang, Yang Feng, Joey Tianyi Zhou, et al. Hulu-med: A transparent generalist model towards holistic medical vision-language understanding.arXiv preprint arXiv:2510.08668, 2025

    arXiv 2025

  8. [8]

    Medgemma technical report.arXiv preprint arXiv:2507.05201, 2025

    Andrew Sellergren, Sahar Kazemzadeh, Tiam Jaroensri, Atilla Kiraly, Madeleine Traverse, Timo Kohlberger, Shawn Xu, Fayaz Jamil, Cían Hughes, Charles Lau, et al. Medgemma technical report.arXiv preprint arXiv:2507.05201, 2025

    Pith/arXiv arXiv 2025

  9. [10]

    Medvlm-r1: Incentivizing medical reasoning capability of vision-language models (vlms) via reinforcement learning

    Jiazhen Pan, Che Liu, Junde Wu, Fenglin Liu, Jiayuan Zhu, Hongwei Bran Li, Chen Chen, Cheng Ouyang, and Daniel Rueckert. Medvlm-r1: Incentivizing medical reasoning capability of vision-language models (vlms) via reinforcement learning. InInternational Conference on Medical Image Computing and Computer-Assisted Intervention, pages 337–347. Springer, 2025

    2025

  10. [11]

    Med- r1: Reinforcement learning for generalizable medical reasoning in vision-language models.arXiv preprint arXiv:2503.13939, 2025

    Yuxiang Lai, Jike Zhong, Ming Li, Shitian Zhao, Yuheng Li, Konstantinos Psounis, and Xiaofeng Yang. Med- r1: Reinforcement learning for generalizable medical reasoning in vision-language models.arXiv preprint arXiv:2503.13939, 2025

    arXiv 2025

  11. [12]

    Huatuogpt-vision, towards injecting medical visual knowledge into multimodal llms at scale.arXiv preprint arXiv:2406.19280, 2024

    Junying Chen, Chi Gui, Ruyi Ouyang, Anningzhe Gao, Shunian Chen, Guiming Hardy Chen, Xidong Wang, Ruifei Zhang, Zhenyang Cai, Ke Ji, et al. Huatuogpt-vision, towards injecting medical visual knowledge into multimodal llms at scale.arXiv preprint arXiv:2406.19280, 2024

    Pith/arXiv arXiv 2024

  12. [13]

    A generalist vision–language foundation model for diverse biomedical tasks.Nature Medicine, 30(11):3129–3141, 2024

    Kai Zhang, Rong Zhou, Eashan Adhikarla, Zhiling Yan, Yixin Liu, Jun Yu, Zhengliang Liu, Xun Chen, Brian D Davison, Hui Ren, et al. A generalist vision–language foundation model for diverse biomedical tasks.Nature Medicine, 30(11):3129–3141, 2024

    2024

  13. [14]

    Omn- imedvqa: A new large-scale comprehensive evaluation benchmark for medical lvlm

    Yutao Hu, Tianbin Li, Quanfeng Lu, Wenqi Shao, Junjun He, Yu Qiao, and Ping Luo. Omn- imedvqa: A new large-scale comprehensive evaluation benchmark for medical lvlm. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 22170– 22183, June 2024. URLhttps://openaccess.thecvf.com/content/CVPR2024/papers/Hu_OmniMedV...

    2024

  14. [15]

    Slake: A semantically-labeled knowledge- enhanced dataset for medical visual question answering

    Bo Liu, Li-Ming Zhan, Li Xu, Lin Ma, Yan Yang, and Xiao-Ming Wu. Slake: A semantically-labeled knowledge- enhanced dataset for medical visual question answering. In2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI), pages 1650–1654, 2021. URLhttps://ieeexplore.ieee.org/document/9434010

    arXiv 2021

  15. [16]

    Generalist foundation models from a multimodal dataset for 3d computed tomography.Nature Biomedical Engineering, pages 1–19, 2026

    Ibrahim Ethem Hamamci, Sezgin Er, Chenyu Wang, Furkan Almas, Ayse Gulnihan Simsek, Sevval Nil Esirgun, Irem Dogan, Omer Faruk Durugol, Benjamin Hou, Suprosanna Shit, et al. Generalist foundation models from a multimodal dataset for 3d computed tomography.Nature Biomedical Engineering, pages 1–19, 2026

    2026

  16. [17]

    Measuring massive multitask language understanding

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. InInternational Conference on Learning Representations,

  17. [18]

    Medxpertqa: Benchmarking expert-level medical reasoning and understanding

    Yuxin Zuo, Shang Qu, Yifei Li, Zhang-Ren Chen, Xuekai Zhu, Ermo Hua, Kaiyan Zhang, Ning Ding, and Bowen Zhou. Medxpertqa: Benchmarking expert-level medical reasoning and understanding. InInternational Conference on Machine Learning, pages 80961–80990. PMLR, 2025

    2025

  18. [19]

    Pubmedqa: A dataset for biomedical research question answering

    Qiao Jin, Bhuwan Dhingra, Zhengping Liu, William Cohen, and Xinghua Lu. Pubmedqa: A dataset for biomedical research question answering. InProceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP), pages 2567–2577, 2019

    2019

  19. [20]

    Towards a holistic framework for multimodal LLM in 3D brain CT radiology report generation.Nature Communications, 16(1):2258, 2025

    Cheng-Yi Li, Kao-Jung Chang, Cheng-Fu Yang, Hsin-Yu Wu, Wenting Chen, Hritik Bansal, Ling Chen, Yi-Ping Yang, Yu-Chun Chen, Shih-Pin Chen, et al. Towards a holistic framework for multimodal LLM in 3D brain CT radiology report generation.Nature Communications, 16(1):2258, 2025. doi: 10.1038/s41467-025-57426-0

    work page doi:10.1038/s41467-025-57426-0 2025

  20. [21]

    Med3dvlm: An efficient vision-language model for 3d medical image analysis.IEEE Journal of Biomedical and Health Informatics, 2025

    Yu Xin, Gorkem Can Ates, Kuang Gong, and Wei Shao. Med3dvlm: An efficient vision-language model for 3d medical image analysis.IEEE Journal of Biomedical and Health Informatics, 2025

    2025

  21. [22]

    Towards generalist foundation model for radiology by leveraging web-scale 2d&3d medical data.Nature Communications, 16(1):7866, 2025

    Chaoyi Wu, Xiaoman Zhang, Ya Zhang, Hui Hui, Yanfeng Wang, and Weidi Xie. Towards generalist foundation model for radiology by leveraging web-scale 2d&3d medical data.Nature Communications, 16(1):7866, 2025. 21

    2025

  22. [23]

    Chaunzwa, Simon Bernatz, Ahmed Hosny, Raymond H

    Suraj Pai, Dennis Bontempi, Ibrahim Hadzic, Vasco Prudente, Mateo Sokač, Tafadzwa L. Chaunzwa, Simon Bernatz, Ahmed Hosny, Raymond H. Mak, Nicolai J. Birkbak, and Hugo J. W. L. Aerts. Foundation model for cancer imaging biomarkers.Nature Machine Intelligence, 6(3):354–367, 2024. doi: 10.1038/s42256-024-00807-9

    doi:10.1038/s42256-024-00807-9 2024

  23. [24]

    Eagle: Exploring the design space for multimodal llms with mixture of encoders

    Min Shi, Fuxiao Liu, Shihao Wang, Shijia Liao, Subhashree Radhakrishnan, Yilin Zhao, De-An Huang, Hongxu Yin, Karan Sapra, Yaser Yacoob, et al. Eagle: Exploring the design space for multimodal llms with mixture of encoders. InInternational Conference on Learning Representations, volume 2025, pages 92628–92646, 2025

    2025

  24. [25]

    Eagle-2: Faster inference of language models with dynamic draft trees

    Yuhui Li, Fangyun Wei, Chao Zhang, and Hongyang Zhang. Eagle-2: Faster inference of language models with dynamic draft trees. InProceedings of the 2024 conference on empirical methods in natural language processing, pages 7421–7432, 2024

    2024

  25. [26]

    Cambrian-1: A fully open, vision-centric exploration of multimodal llms.Advances in Neural Information Processing Systems, 37:87310–87356, 2024

    Peter Tong, Ellis Brown, Penghao Wu, Sanghyun Woo, Adithya Jairam Vedagiri IYER, Sai Charitha Akula, Shusheng Yang, Jihan Yang, Manoj Middepogu, Ziteng Wang, et al. Cambrian-1: A fully open, vision-centric exploration of multimodal llms.Advances in Neural Information Processing Systems, 37:87310–87356, 2024

    2024

  26. [27]

    Mini-gemini: Mining the potential of multi-modality vision language models.IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025

    Yanwei Li, Yuechen Zhang, Chengyao Wang, Zhisheng Zhong, Yixin Chen, Ruihang Chu, Shaoteng Liu, and Jiaya Jia. Mini-gemini: Mining the potential of multi-modality vision language models.IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025

    2025

  27. [28]

    Prismatic vlms: Investigating the design space of visually-conditioned language models

    Siddharth Karamcheti, Suraj Nair, Ashwin Balakrishna, Percy Liang, Thomas Kollar, andDorsa Sadigh. Prismatic vlms: Investigating the design space of visually-conditioned language models. InForty-first International Conference on Machine Learning, 2024

    2024

  28. [29]

    Feast your eyes: Mixture- of-resolution adaptation for multimodal large language models

    Gen Luo, Yiyi Zhou, Yuxin Zhang, Xiawu Zheng, Xiaoshuai Sun, and Rongrong Ji. Feast your eyes: Mixture- of-resolution adaptation for multimodal large language models. InInternational Conference on Learning Representations, volume 2025, pages 84491–84506, 2025

    2025

  29. [30]

    Rouge: A package for automatic evaluation of summaries

    Chin-Yew Lin. Rouge: A package for automatic evaluation of summaries. InText summarization branches out, pages 74–81, 2004

    2004

  30. [31]

    Meteor: An automatic metric for mt evaluation with improved correlation with human judgments

    Satanjeev Banerjee and Alon Lavie. Meteor: An automatic metric for mt evaluation with improved correlation with human judgments. InProceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization, pages 65–72, 2005

    2005

  31. [32]

    Cider: Consensus-based image description evaluation

    Ramakrishna Vedantam, C Lawrence Zitnick, and Devi Parikh. Cider: Consensus-based image description evaluation. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 4566–4575, 2015

    2015

  32. [33]

    Weinberger, and Yoav Artzi

    Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. Bertscore: Evaluating text generation with bert. InInternational Conference on Learning Representations, 2020

    2020

  33. [34]

    Improving the factual correctness of radiology report generation with semantic rewards

    Jean-Benoit Delbrouck, Pierre Chambon, Christian Bluethgen, Emily Tsai, Omar Almusa, and Curtis Langlotz. Improving the factual correctness of radiology report generation with semantic rewards. InFindings of the Association for Computational Linguistics: EMNLP 2022, pages 4348–4360, Abu Dhabi, United Arab Emirates, December 2022. Association for Computati...

    2022

  34. [35]

    Ratescore: A metric for radiology report generation

    Weike Zhao, Chaoyi Wu, Xiaoman Zhang, Ya Zhang, Yanfeng Wang, and Weidi Xie. Ratescore: A metric for radiology report generation. InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 15004–15019, 2024

    2024

  35. [36]

    Green: Generative radiology report evaluation and error notation

    Sophie Ostmeier, Justin Xu, Zhihong Chen, Maya Varma, Louis Blankemeier, Christian Bluethgen, Arne Edward Michalson Md, Michael Moseley, Curtis Langlotz, Akshay S Chaudhari, et al. Green: Generative radiology report evaluation and error notation. InFindings of the association for computational linguistics: EMNLP 2024, pages 374–390, 2024

    2024

  36. [37]

    Jianing Qiu, Kyle Lam, Guohao Li, Amish Acharya, Tien Yin Wong, Ara Darzi, Wu Yuan, and Eric J. Topol. Llm-based agentic systems in medicine and healthcare.Nature Machine Intelligence, 6(12):1418–1420, 2024

    2024

  37. [38]

    A survey of llm-based agents in medicine: How far are we from baymax?Findings of the Association for Computational Linguistics: ACL 2025, pages 10345–10359, 2025

    Wenxuan Wang, Zizhan Ma, Zheng Wang, Chenghan Wu, Jiaming Ji, Wenting Chen, Xiang Li, and Yixuan Yuan. A survey of llm-based agents in medicine: How far are we from baymax?Findings of the Association for Computational Linguistics: ACL 2025, pages 10345–10359, 2025

    2025

  38. [39]

    Medagent- pro: Towards evidence-based multi-modal medical diagnosis via reasoning agentic workflow.arXiv preprint arXiv:2503.18968, 2025

    Ziyue Wang, Junde Wu, Linghan Cai, Chang Han Low, Xihong Yang, Qiaxuan Li, and Yueming Jin. Medagent- pro: Towards evidence-based multi-modal medical diagnosis via reasoning agentic workflow.arXiv preprint arXiv:2503.18968, 2025. 22

    Pith/arXiv arXiv 2025

  39. [40]

    Mmedagent: Learning to use medical tools with multi-modal agent

    Binxu Li, Tiankai Yan, Yuanting Pan, Jie Luo, Ruiyang Ji, Jiayuan Ding, Zhe Xu, Shilong Liu, Haoyu Dong, Zihao Lin, et al. Mmedagent: Learning to use medical tools with multi-modal agent. InFindings of the Association for Computational Linguistics: EMNLP 2024, pages 8745–8760, 2024

    2024

  40. [41]

    Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models.arXiv preprint arXiv:2504.10479, 2025

    Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Hao Tian, Yuchen Duan, Weijie Su, Jie Shao, et al. Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models.arXiv preprint arXiv:2504.10479, 2025

    Pith/arXiv arXiv 2025

  41. [42]

    Bimedix2: Bio-medical expert lmm for diverse medical modalities

    Sahal Shaji Mullappilly, Mohammed Irfan Kurpath, Sara Pieri, Saeed Yahya Alseiari, Shanavas Cholakkal, Khaled Aldahmani, Fahad Khan, Rao Anwer, Salman Khan, Timothy Baldwin, et al. Bimedix2: Bio-medical expert lmm for diverse medical modalities. InFindings of the association for computational linguistics: EMNLP 2025, pages 14051–14071, 2025

    2025

  42. [43]

    Healthgpt: A medical large vision-language model for unifying comprehension and generation via heterogeneous knowledge adaptation

    Tianwei Lin, Wenqiao Zhang, Sijing Li, Yuqian Yuan, Binhe Yu, Haoyuan Li, Wanggui He, Hao Jiang, Mengze Li, Song Xiaohui, et al. Healthgpt: A medical large vision-language model for unifying comprehension and generation via heterogeneous knowledge adaptation. InInternational Conference on Machine Learning, pages 37975–37995. PMLR, 2025

    2025

  43. [44]

    Gsco: Towards generalizable ai in medicine via generalist-specialist collaboration

    Sunan He, Yuxiang Nie, Hongmei Wang, Shu Yang, Yihui Wang, Zhiyuan Cai, Zhixuan Chen, Yingxue Xu, Luyang Luo, Huiling Xiang, et al. Gsco: Towards generalizable ai in medicine via generalist-specialist collaboration. arXiv preprint arXiv:2404.15127, 2024

    Pith/arXiv arXiv 2024

  44. [45]

    Ryutaro Tanno, David G. T. Barrett, Andrew Sellergren, Sumedh Ghaisas, Sumanth Dathathri, Abigail See, Johannes Welbl, Charles Lau, Tao Tu, Shekoofeh Azizi, et al. Collaboration between clinicians and vision–language models in radiology report generation.Nature Medicine, 31(2):599–608, 2025. doi: 10.1038/s41591-024-03302-1

    doi:10.1038/s41591-024-03302-1 2025

  45. [47]

    3d-rad: A comprehensive 3d radiology med-vqa dataset with multi-temporal analysis and diverse diagnostic tasks.arXiv preprint arXiv:2506.11147, 2025

    Xiaotang Gai, Jiaxiang Liu, Yichen Li, Zijie Meng, Jian Wu, and Zuozhu Liu. 3d-rad: A comprehensive 3d radiology med-vqa dataset with multi-temporal analysis and diverse diagnostic tasks.arXiv preprint arXiv:2506.11147, 2025

    arXiv 2025

  46. [48]

    Amos: A large-scale abdominal multi-organ benchmark for versatile medical image segmentation.Advances in neural information processing systems, 35:36722–36732, 2022

    Yuanfeng Ji, Haotian Bai, Chongjian Ge, Jie Yang, Ye Zhu, Ruimao Zhang, Zhen Li, Lingyan Zhanng, Wanling Ma, Xiang Wan, et al. Amos: A large-scale abdominal multi-organ benchmark for versatile medical image segmentation.Advances in neural information processing systems, 35:36722–36732, 2022

    2022

  47. [49]

    Bleu: a method for automatic evaluation of machine translation

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for automatic evaluation of machine translation. InProceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311–318, 2002

    2002

  48. [50]

    RadGraph-XL: A large-scale expert-annotated dataset for entity and relation extraction from radiology reports

    Jean-Benoit Delbrouck, Pierre Chambon, Zhihong Chen, Maya Varma, Andrew Johnston, Louis Blankemeier, Dave Van Veen, Tan Bui, Steven Truong, and Curtis Langlotz. RadGraph-XL: A large-scale expert-annotated dataset for entity and relation extraction from radiology reports. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors,Findings of the Association...

    2024

  49. [51]

    Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity.Journal of Machine Learning Research, 23(120):1–39, 2022

    William Fedus, Barret Zoph, and Noam Shazeer. Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity.Journal of Machine Learning Research, 23(120):1–39, 2022

    2022

  50. [52]

    Position: Compositional generative modeling: A single model is not all you need

    Yilun Du and Leslie Pack Kaelbling. Position: Compositional generative modeling: A single model is not all you need. InInternational Conference on Machine Learning, pages 11721–11732. PMLR, 2024

    2024

  51. [53]

    A convnet for the 2020s.Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022

    Zhuang Liu, Hanzi Mao, Chao-Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie. A convnet for the 2020s.Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022

    2022

  52. [54]

    Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy V. Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel HAZIZA, Francisco Massa, Alaaeldin El-Nouby, Mido Assran, Nicolas Ballas, Wojciech Galuba, Russell Howes, Po-Yao Huang, Shang-Wen Li, Ishan Misra, Michael Rabbat, Vasu Sharma, Gabriel Synnaeve, Hu Xu, Herve Jegou, Julien Mairal, Patrick Lab...

    2024

  53. [55]

    Exploring scalable medical image encoders beyond text supervision.Nature Machine Intelligence, 7(1):119–130, 2025

    Fernando Pérez-García, Harshita Sharma, Sam Bond-Taylor, Kenza Bouzid, Valentina Salvatelli, Maximilian Ilse, Shruthi Bannur, Daniel C Castro, Anton Schwaighofer, Matthew P Lungren, et al. Exploring scalable medical image encoders beyond text supervision.Nature Machine Intelligence, 7(1):119–130, 2025

    2025

  54. [56]

    A multimodal biomedical foundation model trained from fifteen million image–text pairs.Nejm Ai, 2(1):AIoa2400640, 2025

    Sheng Zhang, Yanbo Xu, Naoto Usuyama, Hanwen Xu, Jaspreet Bagga, Robert Tinn, Sam Preston, Rajesh Rao, Mu Wei, Naveen Valluri, et al. A multimodal biomedical foundation model trained from fifteen million image–text pairs.Nejm Ai, 2(1):AIoa2400640, 2025

    2025

  55. [57]

    Visionzip: Longer is better but not necessary in vision language models

    Senqiao Yang, Yukang Chen, Zhuotao Tian, Chengyao Wang, Jingyao Li, Bei Yu, and Jiaya Jia. Visionzip: Longer is better but not necessary in vision language models. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 19792–19802, 2025

    2025

  56. [58]

    Sparsevlm: Visual token sparsification for efficient vision-language model inference

    Yuan Zhang, Chun-Kai Fan, Junpeng Ma, Wenzhao Zheng, Tao Huang, Kuan Cheng, Denis A Gudovskiy, Tomoyuki Okuno, Yohei Nakata, Kurt Keutzer, et al. Sparsevlm: Visual token sparsification for efficient vision-language model inference. InInternational Conference on Machine Learning, pages 74840–74857. PMLR, 2025

    2025

  57. [59]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. InNeurIPS, pages 5998–6008, 2017

    2017

  58. [60]

    Deepstack: Deeply stacking visual tokens is surprisingly simple and effective for lmms.Advances in Neural Information Processing Systems, 37:23464–23487, 2024

    Lingchen Meng, Jianwei Yang, Rui Tian, Xiyang Dai, Zuxuan Wu, Jianfeng Gao, and Yu-Gang Jiang. Deepstack: Deeply stacking visual tokens is surprisingly simple and effective for lmms.Advances in Neural Information Processing Systems, 37:23464–23487, 2024

    2024

  59. [61]

    Perception encoder: The best visual embeddings are not at the output of the network.arXiv preprint arXiv:2504.13181, 2025

    Daniel Bolya, Po-Yao Huang, Peize Sun, Jang Hyun Cho, Andrea Madotto, Chen Wei, Tengyu Ma, Jiale Zhi, Jathushan Rajasegaran, Hanoona Rasheed, et al. Perception encoder: The best visual embeddings are not at the output of the network.arXiv preprint arXiv:2504.13181, 2025

    Pith/arXiv arXiv 2025

  60. [62]

    Generalized radiograph representation learning via cross-supervision between images and free-text radiology reports.Nature Machine Intelligence, 4(1):32–40, 2022

    Hong-Yu Zhou, Xiaoyu Chen, Yinghao Zhang, Ruibang Luo, Liansheng Wang, and Yizhou Yu. Generalized radiograph representation learning via cross-supervision between images and free-text radiology reports.Nature Machine Intelligence, 4(1):32–40, 2022. doi: 10.1038/s42256-021-00425-9

    work page doi:10.1038/s42256-021-00425-9 2022

  61. [63]

    Huatuogpt, towards taming language model to be a doctor

    Hongbo Zhang, Junying Chen, Feng Jiang, Fei Yu, Zhihong Chen, Guiming Chen, Jianquan Li, Xiangbo Wu, Zhang Zhiyi, Qingying Xiao, et al. Huatuogpt, towards taming language model to be a doctor. InFindings of the association for computational linguistics: EMNLP 2023, pages 10859–10885, 2023

    2023

  62. [64]

    State of what art? a call for multi-prompt llm evaluation.Transactions of the Association for Computational Linguistics, 12: 933–949, 2024

    Moran Mizrahi, Guy Kaplan, Dan Malkin, Rotem Dror, Dafna Shahaf, and Gabriel Stanovsky. State of what art? a call for multi-prompt llm evaluation.Transactions of the Association for Computational Linguistics, 12: 933–949, 2024

    2024

  63. [65]

    Surveillance, epidemiology, and end results (seer) program (www.seer.cancer.gov), 2025

    National Cancer Institute, Surveillance Research Program. Surveillance, epidemiology, and end results (seer) program (www.seer.cancer.gov), 2025. URLhttps://seer.cancer.gov/data/. SEER database; data analyzed using SEER*Stat. Visit https://seer.cancer.gov/data/

    2025

  64. [66]

    Look again, think slowly: Enhancing visual reflection in vision-language models

    Pu Jian, Junhong Wu, Wei Sun, Chen Wang, Shuo Ren, and Jiajun Zhang. Look again, think slowly: Enhancing visual reflection in vision-language models. InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 9262–9281, 2025

    2025

  65. [67]

    Meng, and Bo Zhao

    Fan Bai, Yuxin Du, Tiejun Huang, Max Q.-H. Meng, and Bo Zhao. M3d: Advancing 3d medical image analysis with multi-modal large language models.arXiv preprint arXiv:2404.00578, 2024

    Pith/arXiv arXiv 2024

  66. [68]

    Merlin: a computed tomography vision–language foundation model and dataset.Nature, pages 1–11, 2026

    Louis Blankemeier, Ashwin Kumar, Joseph Paul Cohen, Jiaming Liu, Longchao Liu, Dave Van Veen, Syed Jamal Safdar Gardezi, Hongkun Yu, Magdalini Paschali, Zhihong Chen, et al. Merlin: a computed tomography vision–language foundation model and dataset.Nature, pages 1–11, 2026

    2026

  67. [69]

    Inspect: A multimodal dataset for patient outcome prediction of pulmonary embolisms

    Shih-Cheng Huang, Zepeng Huo, Ethan Steinberg, Chia-Chun Chiang, Curtis Langlotz, Matthew Lungren, Serena Yeung, Nigam Shah, and Jason Fries. Inspect: A multimodal dataset for patient outcome prediction of pulmonary embolisms. InAdvances in Neural Information Processing Systems 36 (NeurIPS), Datasets and Benchmarks Track, pages 17742–17772, 2023. URLhttps...

    2023

  68. [70]

    Vista3d: A unified segmentation foundation model for 3d medical imaging

    Yufan He, Pengfei Guo, Yucheng Tang, Andriy Myronenko, Vishwesh Nath, Ziyue Xu, Dong Yang, Can Zhao, Benjamin Simon, Mason Belue, et al. Vista3d: A unified segmentation foundation model for 3d medical imaging. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 20863–20873, 2025. 24

    2025

  69. [71]

    Totalsegmentator: robust segmentation of 104 anatomic structures in ct images.Radiology: Artificial Intelligence, 5(5):e230024, 2023

    Jakob Wasserthal, Hanns-Christian Breit, Manfred T Meyer, Maurice Pradella, Daniel Hinck, Alexander W Sauter, Tobias Heye, Daniel T Boll, Joshy Cyriac, Shan Yang, et al. Totalsegmentator: robust segmentation of 104 anatomic structures in ct images.Radiology: Artificial Intelligence, 5(5):e230024, 2023

    2023

  70. [72]

    Dmqr-rag: Diverse multi-query rewriting for rag.arXiv preprint arXiv:2411.13154, 2024

    Zhicong Li, Jiahao Wang, Zhishu Jiang, Hangyu Mao, Zhongxia Chen, Jiazhen Du, Yuanxing Zhang, Fuzheng Zhang, Di Zhang, and Yong Liu. Dmqr-rag: Diverse multi-query rewriting for rag.arXiv preprint arXiv:2411.13154, 2024

    Pith/arXiv arXiv 2024

  71. [73]

    UMLS knowledge sources, release 2024aa, 2024.http://www.nlm.nih.gov/ research/umls/licensedcontent/umlsknowledgesources.html(accessed 15 July 2024)

    National Library of Medicine (US). UMLS knowledge sources, release 2024aa, 2024.http://www.nlm.nih.gov/ research/umls/licensedcontent/umlsknowledgesources.html(accessed 15 July 2024)

    2024

  72. [74]

    Building a knowledge graph to enable precision medicine

    Payal Chandak, Kexin Huang, and Marinka Zitnik. Building a knowledge graph to enable precision medicine. Scientific Data, 10(1):67, 2023. doi: 10.1038/s41597-023-01960-3

    doi:10.1038/s41597-023-01960-3 2023

  73. [75]

    Sayers, Evan E

    Eric W. Sayers, Evan E. Bolton, J. Rodney Brister, et al. Database resources of the national center for biotechnology information.Nucleic Acids Research, 52(D1):D33–D43, 2024. doi: 10.1093/nar/gkad1044

    work page doi:10.1093/nar/gkad1044 2024

  74. [76]

    StatPearls Publishing, Treasure Island, FL, 2026

    StatPearls Publishing.StatPearls [Internet]. StatPearls Publishing, Treasure Island, FL, 2026. Available from: https://www.ncbi.nlm.nih.gov/books/NBK430685/(accessed 27 February 2026)

    2026

  75. [77]

    What disease does this patient have? a large-scale open domain question answering dataset from medical exams.Applied Sciences, 11 (14):6421, 2021

    Di Jin, Eileen Pan, Nassim Oufattole, Wei-Hung Weng, Hanyi Fang, and Peter Szolovits. What disease does this patient have? a large-scale open domain question answering dataset from medical exams.Applied Sciences, 11 (14):6421, 2021. doi: 10.3390/app11146421

    doi:10.3390/app11146421 2021

  76. [78]

    Xiong, Q

    G. Xiong, Q. Jin, Z. Lu, et al. Benchmarking retrieval-augmented generation for medicine. InFindings of the Association for Computational Linguistics: ACL 2024, pages 6233–6251, 2024

    2024

  77. [79]

    Multi-modal ai for opportunistic screening, staging and progression risk stratification of steatotic liver disease.Nature Communications, 17(1):1562, 2026

    Yuan Gao, Chunli Li, Wanxing Chang, Bai Du, Xianghua Ye, Yee Hui Yeo, Yingda Xia, Heng Guo, Xiaoming Zhang, Wei Liu, et al. Multi-modal ai for opportunistic screening, staging and progression risk stratification of steatotic liver disease.Nature Communications, 17(1):1562, 2026

    2026

  78. [80]

    Effective lymph nodes detection in ct scans using location debiased query selection and contrastive query representation in transformer

    Qinji Yu, Yirui Wang, Ke Yan, Haoshen Li, Dazhou Guo, Li Zhang, Na Shen, Qifeng Wang, Xiaowei Ding, Le Lu, et al. Effective lymph nodes detection in ct scans using location debiased query selection and contrastive query representation in transformer. InEuropean Conference on Computer Vision, pages 180–198. Springer, 2024

    2024

  79. [81]

    On the limits of cross-domain generalization in automated x-ray prediction

    Joseph Paul Cohen, Mohammad Hashir, Rupert Brooks, and Hadrien Bertrand. On the limits of cross-domain generalization in automated x-ray prediction. InMedical Imaging with Deep Learning (MIDL), volume 121 ofProceedings of Machine Learning Research, pages 136–155. PMLR, 2020. URLhttps://proceedings.mlr. press/v121/cohen20a.html

    2020

  80. [82]

    Kalra, and Pingkun Yan

    Zefan Yang, Xuanang Xu, Jiajin Zhang, Ge Wang, Mannudeep K. Kalra, and Pingkun Yan. Chest x-ray foundation model with global and local representations integration.IEEE Transactions on Medical Imaging, 44 (12):4787–4799, 2025. doi: 10.1109/TMI.2025.3581907

    arXiv 2025

Showing first 80 references.