REVIEW 10 cited by
ELIXR: Towards a general purpose X-ray artificial intelligence system through alignment of large language models and radiology vision encoders
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
In this work, we present an approach, which we call Embeddings for Language/Image-aligned X-Rays, or ELIXR, that leverages a language-aligned image encoder combined or grafted onto a fixed LLM, PaLM 2, to perform a broad range of chest X-ray tasks. We train this lightweight adapter architecture using images paired with corresponding free-text radiology reports from the MIMIC-CXR dataset. ELIXR achieved state-of-the-art performance on zero-shot chest X-ray (CXR) classification (mean AUC of 0.850 across 13 findings), data-efficient CXR classification (mean AUCs of 0.893 and 0.898 across five findings (atelectasis, cardiomegaly, consolidation, pleural effusion, and pulmonary edema) for 1% (~2,200 images) and 10% (~22,000 images) training data), and semantic search (0.76 normalized discounted cumulative gain (NDCG) across nineteen queries, including perfect retrieval on twelve of them). Compared to existing data-efficient methods including supervised contrastive learning (SupCon), ELIXR required two orders of magnitude less data to reach similar performance. ELIXR also showed promise on CXR vision-language tasks, demonstrating overall accuracies of 58.7% and 62.5% on visual question answering and report quality assurance tasks, respectively. These results suggest that ELIXR is a robust and versatile approach to CXR AI.
Forward citations
Cited by 10 Pith papers
-
Toward a Vision-Language Foundation Model for Medical Data: Multimodal Dataset and Benchmarks for Vietnamese PET/CT Report Generation
A new Vietnamese PET/CT-report dataset improves medical VLM report generation and VQA, but clinical F1 scores remain modest.
-
DocCHA: Towards LLM-Augmented Interactive Online diagnosis System
DocCHA, a confidence-scored three-module LLM pipeline, reports improved diagnostic accuracy and information recall over direct-prompting LLMs on two Chinese consultation datasets, but evaluation gaps weaken the claim.
-
Harrison.Rad 1.5 Technical Report: A radiology foundation model that can draft reports from images, priors and clinical context
Harrison.Rad 1.5 is a radiology-specific multimodal LLM that passes simulated FRCR 2B Short Case examinations and outperforms general-purpose frontier models on plain-film radiography reporting tasks.
-
Benchmarking Foundation Models with Multimodal Public Electronic Health Records
A standardized MIMIC-IV benchmark comparing eight unimodal and multimodal foundation models shows multimodal inputs improve predictive performance without adding bias, while medical LVLMs underperform on length-of-sta...
-
CXR-TFT: Multi-Modal Temporal Fusion Transformer for Predicting Chest X-ray Trajectories
CXR-TFT predicts future chest X-ray embeddings hour-by-hour from clinical time series and prior X-rays, claiming 95% accuracy for abnormal findings 12 hours before the next scan.
-
Multimodal, Multi-Disease Medical Imaging Foundation Model (MerMED-FM)
A self-supervised vision model with a memory module reports AUROC from 0.86 to 0.99 across seven medical imaging modalities, often matching or beating specialty foundation models.
-
Fair-MoE: Fairness-Oriented Mixture of Experts in Vision-Language Models
Fair-MoE reports improved accuracy and fairness on Harvard-FairVLMed for some protected attributes by adding sparse mixture-of-experts layers and a variance-based fairness loss to CLIP, but the all-attribute improveme...
-
Health AI Developer Foundations
Health AI Developer Foundations packages six domain-specific medical embedding models into one platform, claiming large data and compute savings for downstream health ML tasks.
-
Gla-AI4BioMed at RRG24: Visual Instruction-tuned Adaptation for Radiology Report Generation
A LLaVA-style radiology report generator using LoRA fine-tuning and stitched chest X-ray inputs placed fourth in the RRG24 shared task.
-
Multimodal Large Language Models for Medicine: A Comprehensive Survey
A comprehensive review cataloging medical MLLMs, their uses in report generation, diagnosis, and treatment, and the challenges of accuracy, hallucination, fairness, privacy, and deployment.
Discussion (0). Continue with ORCID to comment.