REVIEW 31 cited by
HealthGPT: A Medical Large Vision-Language Model for Unifying Comprehension and Generation via Heterogeneous Knowledge Adaptation
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
We present HealthGPT, a powerful Medical Large Vision-Language Model (Med-LVLM) that integrates medical visual comprehension and generation capabilities within a unified autoregressive paradigm. Our bootstrapping philosophy is to progressively adapt heterogeneous comprehension and generation knowledge to pre-trained large language models (LLMs). This is achieved through a novel heterogeneous low-rank adaptation (H-LoRA) technique, which is complemented by a tailored hierarchical visual perception approach and a three-stage learning strategy. To effectively learn the HealthGPT, we devise a comprehensive medical domain-specific comprehension and generation dataset called VL-Health. Experimental results demonstrate exceptional performance and scalability of HealthGPT in medical visual unified tasks. Our project can be accessed at https://github.com/DCDmllm/HealthGPT.
Forward citations
Cited by 31 Pith papers
-
MMRareBench: A Rare-Disease Multimodal and Multi-Image Medical Benchmark
MMRareBench provides 1,756 QA pairs and 7,958 images from PMC rare-disease cases to evaluate 23 MLLMs, revealing low treatment-planning scores and medical models underperforming general models on multi-image tasks due...
-
MMRareBench: A Rare-Disease Multimodal and Multi-Image Medical Benchmark
MMRareBench is the first rare-disease benchmark for multimodal and multi-image clinical evaluation of MLLMs, revealing fragmented capabilities, low treatment-planning scores, and medical models underperforming general...
-
Do All Individual Layers Help? An Empirical Study of Task-Interfering Layers in Vision-Language Models
Empirical analysis identifies task-interfering layers in VLMs and proposes TaLo, a test-time method to bypass them for improved performance without training.
-
OralAgent: Integrating Reasoning, Tools, and Knowledge for Interactive Dental Image Analysis
OralAgent, a ReAct-style dental agent with 22 vision tools and a 134.8M-token textbook RAG corpus, reaches SOTA on MMOral-Uni, MMOral-OPG, and OralQA-ZH.
-
PathScale-R1: Cross-scale Reasoning for Pathological Image Analysis
PathScale-R1 and PathScale-VQA raise cross-scale pathology VQA accuracy to 83% by training on expert multi-magnification paths with anti-shortcut curation and scale-aware RL.
-
Can Multimodal Large Language Models Understand OCT?
OCT-Bench, a 20-task benchmark across 10,076 questions, shows current MLLMs score up to 62% overall but only 43% on clinical reasoning over OCT images.
-
VisualThink-VLA: Visual Intermediate Reasoning for Effective and Low-Latency Vision-Language-Action Policies
VISUALTHINK-VLA uses visual evidence tokens and selective routing to reach top success rates on VLA benchmarks while cutting reasoning latency from multi-second to sub-second levels.
-
InstructSAM: Segment Any Instance with Any Instructions
InstructSAM uses learnable queries in a VLM to condition SAM3 for single-pass multi-instance segmentation from arbitrary instructions, with a new Inst2Seg benchmark.
-
MedRCube: A Multidimensional Framework for Fine-Grained and In-Depth Evaluation of MLLMs in Medical Imaging
MedRCube is a new fine-grained evaluation framework that benchmarks 33 MLLMs on medical imaging, ranks Lingshu-32B highest, and finds a significant positive link between shortcut behaviors and diagnostic performance.
-
Deep Expert Injection for Anchoring Retinal VLMs with Domain-Specific Knowledge
EyExIn, a 7B retinal VLM with dual-stream expert encoding and adaptive deep-layer visual injection, reports state-of-the-art F1 on four fundus VQA benchmarks, exceeding larger proprietary models.
-
Epistemic-aware Vision-Language Foundation Model for Fetal Ultrasound Interpretation
A multi-view vision-language model trained on 20,000 fetal ultrasound reports generates clinical text and diagnoses, reportedly outperforming general and medical baselines.
-
Towards Better Dental AI: A Multimodal Benchmark and Instruction Dataset for Panoramic X-ray Analysis
MMOral is a large new dental X-ray instruction dataset and benchmark, but the proposed model's 24.73% improvement is from fine-tuning and then testing on the same data pool.
-
HeartcareGPT: A Unified Multimodal ECG Suite for Dual Signal-Image Modeling and Understanding
HeartcareGPT proposes Dual Stream Projection Alignment (DSPA) on a structure-aware tokenizer for unified ECG signal-image modeling, supported by Heartcare-400K dataset and Heartcare-Bench.
-
Clinical Cognition Alignment for Gastrointestinal Diagnosis with Multimodal LLMs
Hierarchical clinical-reasoning SFT plus counterfactual GRPO yields SoTA diagnostic accuracy for multimodal LLMs on gastrointestinal endoscopy benchmarks.
-
Rad-JEPA 3D: Radiology Joint-Embedding Predictive Model for 3D Computed Tomography
A 19M-parameter JEPA-style 3D-CT encoder with a routed Mamba+GQA hybrid and orthogonal hidden-state regularization gives a 4B total model the best mean accuracy on M3D-VQA closed-ended questions and the best average o...
-
APRIL-MedSeg: A Modular Medical Image Segmentation Toolbox Embracing Modern Paradigms
APRIL-MedSeg is a new open-source modular toolbox that uses YAML configuration and component registries to unify multiple advanced paradigms for medical image segmentation.
-
E-MRL: Cross-view Aligned Evidence-driven Multimodal Reinforcement Learning for Reliable 3D Tumor Analysis
E-MRL trains VLMs via RL on a diagnosis-localization-verification MDP with a novel cross-view consistency reward to ground 3D tumor reports in verifiable CT slices.
-
Case-Aware Medical Image Classification with Multimodal Knowledge Graphs and Reliability-Guided Refinement
A framework constructs multimodal knowledge graphs from adaptively retrieved similar cases, propagates knowledge via an image-centric Graph Attention Network, injects features with bidirectional cross-modal attention,...
-
Case-Aware Medical Image Classification with Multimodal Knowledge Graphs and Reliability-Guided Refinement
The paper presents a case-aware multimodal knowledge graph approach for medical image classification that retrieves similar cases, propagates knowledge via graph attention, and refines predictions with reliability estimates.
-
EgoCoT-Bench: Benchmarking Grounded and Verifiable Operation-Centric Chain of Thought Reasoning for MLLMs
EgoCoT-Bench provides 3,172 verifiable QA pairs across perception, anticipation, and reasoning tasks on egocentric videos, revealing that many MLLMs give answer-correct but evidence-inconsistent explanations.
-
Regulating Anatomy-Aware Rewards via Trajectory-Integral Feedback for Volumetric Computed Tomography Analysis
TIF-GRPO uses integral feedback on pseudo-temporal trajectories to regulate anatomy-aware rewards in RL for clinical faithfulness in volumetric CT analysis.
-
CrossView Suite: Harnessing Cross-view Spatial Intelligence of MLLMs with Dataset, Model and Benchmark
CrossView Suite supplies a 1.6M-sample dataset, scene-disjoint benchmark, and explicit-alignment framework to advance MLLMs from single-view perception to cross-view spatial intelligence.
-
Clinical Cognition Alignment for Gastrointestinal Diagnosis with Multimodal LLMs
CogAlign uses hierarchical supervised fine-tuning on clinical cognition data plus counterfactual RL to align MLLMs with expert diagnostic pathways and enforce causal lesion grounding for GI endoscopy diagnosis.
-
M3CoTBench: Benchmark Chain-of-Thought of MLLMs in Medical Image Understanding
A medical-image benchmark that scores the step-by-step reasoning chains of multimodal LLMs shows current models explain poorly and chain-of-thought prompting frequently reduces diagnostic accuracy.
-
Nirvana: A Specialized Generalist Model With Task-Aware Memory Mechanism
Nirvana adds a task-aware memory trigger and updater to specialized generalist models, achieving strong general benchmark results, lowest perplexity in biomedicine/finance/law, and improved MRI reconstruction fidelity.
-
The Path to Self-Evolving Clinical Systems: Scaling Medical Agents from Assistance to Autonomy
Medical agents should be scaled mainly by richer clinical environments and self-evolution loops, not parameter growth alone, under a three-level autonomy taxonomy.
-
APRIL-MedSeg: A Modular Medical Image Segmentation Toolbox Embracing Modern Paradigms
Presents APRIL-MedSeg, a modular YAML-configurable toolbox for 2D medical image segmentation integrating semi-supervised, domain adaptation, distillation, weakly supervised, text-guided, and foundation model paradigms...
-
UniReason-Med: A Shared Grounded Reasoning Interface for 2D-to-3D Transfer in Medical VQA
UniReason-Med introduces a unified framework for 2D and 3D medical VQA with shared grounded reasoning, trained on a 220K dataset, claiming that joint 2D+3D supervision improves 3D performance over 3D-only training.
-
ScatterPrism: convergence for generative simulation and inverse problems in particle and nuclear physics
Standard Conditional Flow Matching loss is a misleading early plateau; physics-informed metrics keep improving, so ScatterPrism and multi-metric diagnostics are needed for kinematic fidelity.
-
ScatterPrism: convergence for generative simulation and inverse problems in particle and nuclear physics
CFM training loss plateaus while physics-informed metrics continue improving; ScatterPrism and a multi-metric protocol are proposed to restore kinematic fidelity without memorization.
-
LMMs Meet Object-Centric Vision: Understanding, Segmentation, Editing and Generation
This review organizes literature on large multimodal models and object-centric vision into four themes—understanding, referring segmentation, editing, and generation—while summarizing paradigms, strategies, and challe...
Discussion (0). Sign in to comment.