REVIEW 17 cited by
GSCo: Towards Generalizable AI in Medicine via Generalist-Specialist Collaboration
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Generalist foundation models (GFMs) are renowned for their exceptional capability and flexibility in effectively generalizing across diverse tasks and modalities. In the field of medicine, while GFMs exhibit superior generalizability based on their extensive intrinsic knowledge as well as proficiency in instruction following and in-context learning, specialist models excel in precision due to their domain knowledge. In this work, for the first time, we explore the synergy between the GFM and specialist models, to enable precise medical image analysis on a broader scope. Specifically, we propose a cooperative framework, Generalist-Specialist Collaboration (GSCo), which consists of two stages, namely the construction of GFM and specialists, and collaborative inference on downstream tasks. In the construction stage, we develop MedDr, the largest open-source GFM tailored for medicine, showcasing exceptional instruction-following and in-context learning capabilities. Meanwhile, a series of lightweight specialists are crafted for downstream tasks with low computational cost. In the collaborative inference stage, we introduce two cooperative mechanisms, Mixture-of-Expert Diagnosis and Retrieval-Augmented Diagnosis, to harvest the generalist's in-context learning abilities alongside the specialists' domain expertise. For a comprehensive evaluation, we curate a large-scale benchmark featuring 28 datasets and about 250,000 images. Extensive results demonstrate that MedDr consistently outperforms state-of-the-art GFMs on downstream datasets. Furthermore, GSCo exceeds both GFMs and specialists across all out-of-domain disease diagnosis datasets. These findings indicate a significant paradigm shift in the application of GFMs, transitioning from separate models for specific tasks to a collaborative approach between GFMs and specialists, thereby advancing the frontiers of generalizable AI in medicine.
Forward citations
Cited by 17 Pith papers
-
ClinFusion: A Vision-Centric Multimodal LLM System for Holistic Medical Understanding
A cascaded multi-encoder medical MLLM with native 3D fusion and RoI-grounded report metrics claims SOTA on most 2D/3D medical benchmarks and highest radiologist report rankings.
-
OralAgent: Integrating Reasoning, Tools, and Knowledge for Interactive Dental Image Analysis
OralAgent, a ReAct-style dental agent with 22 vision tools and a 134.8M-token textbook RAG corpus, reaches SOTA on MMOral-Uni, MMOral-OPG, and OralQA-ZH.
-
Agentic Visual Reasoning in Whole-Slide Pathology Images via Active Perception
AdaptivePath trains a reinforcement-learning navigator on pathologist-reviewed abnormality labels to acquire sparse multi-scale observations, then uses language models to interpret, deliberate, and adjudicate answers ...
-
PathSelect: Sequential Token Selection for Whole Slide Pathology
A learnable sequential token router with noise-gated Soft Top-K training and Hard Top-K inference cuts WSI visual context ~36.6× while holding 74% SlideBench accuracy on frozen SlideChat.
-
Super-Generalist: Towards Comprehensive and Accurate Medical Image Understanding via Generalist-Specialist Synergy
Injecting multi-expert anatomy/lesion segmentation priors into vision–language alignment and calibrating text attention with lesion masks yields broad CT diagnosis plus specialist-level tumor performance and lesion grounding.
-
Med-RewardBench: Benchmarking Reward Models and Judges for Medical Multimodal Large Language Models
The authors construct a 1,026-case medical benchmark with doctor preference labels across six quality dimensions and show current multimodal LLMs, including medical-specific ones, are only moderately aligned with expe...
-
Constructing Ophthalmic MLLM for Positioning-diagnosis Collaboration Through Clinical Cognitive Chain Reasoning
FundusExpert, an 8B ophthalmic MLLM trained on region-grounded cognitive-chain instructions, reports state-of-the-art QA and report-generation results, with a fitted data-scaling exponent of 0.068.
-
Semantically Informed Salient Regions Guided Radiology Report Generation
SISRNet identifies medically salient regions in chest X-rays via cross-modal alignment and uses them to guide both masked image modeling and report generation, improving clinical accuracy metrics on IU-Xray and MIMIC-CXR.
-
MedBookVQA: A Systematic and Comprehensive Medical Benchmark Derived from Open-Access Book
MedBookVQA is a new 5,000-question, textbook-derived multimodal benchmark for testing medical AI systems, with labels for imaging modality, body anatomy, and clinical specialty.
-
DDaTR: Dynamic Difference-aware Temporal Residual Network for Longitudinal Radiology Report Generation
A longitudinal radiology report generator that aligns prior image-text features and fuses difference-aware features at every encoder stage to improve clinical efficacy scores.
-
MCA-RG: Enhancing LLMs with Medical Concept Alignment for Radiology Report Generation
MCA-RG uses concept alignment, contrastive learning, matching loss, and feature gating to generate radiology reports, reporting SOTA on MIMIC-CXR and CheXpert Plus.
-
EndoChat: Grounded Multimodal Large Language Model for Endoscopic Surgery
EndoChat is a grounded multimodal LLM for endoscopic surgery, trained on the new Surg-396K dataset and reported to outperform prior MLLMs, though its evaluation is confounded by training-data overlap.
-
GMAI-VL & GMAI-VL-5.5M: A Large Vision-Language Model and A Comprehensive Multimodal Dataset Towards General Medical AI
GMAI-VL-5.5M is a new 5.5M-sample medical image-text dataset built from 219 datasets via GPT-4o annotation-guided generation, and GMAI-VL is a three-stage LLaVA-style model reporting SOTA numbers, though the evaluatio...
-
Anatomy-Guided Radiology Report Generation with Pathology-Aware Regional Prompts
A report generation pipeline that uses detected pathologies mapped to anatomical regions as prompt tokens improves several NLG and clinical efficacy metrics on MIMIC-CXR.
-
Real-Time Hardware-Free HIFU Interference Suppression via Teacher-Student Diffusion Framework
An image-domain latent diffusion model suppresses HIFU-induced ultrasound interference at up to 15 frames per second, beating a Notch Filter on an 18,802-pair in vitro/ex vivo/in vivo dataset.
-
Multi-Modal Explainable Medical AI Assistant for Trustworthy Human-AI Collaboration
A fine-tuned 8B medical vision-language model that claims explainable grounding, uncertainty estimates and cancer prognosis, but its core uncertainty formula is mathematically inconsistent and key comparisons use the ...
-
Spatial navigation in preclinical Alzheimer's disease: A review
Spatial navigation performance, particularly path integration and wayfinding, correlates with AD biomarkers such as p-tau in cognitively unimpaired at-risk individuals and may enable earlier detection than episodic me...
Discussion (0). Continue with ORCID to comment.