REVIEW 4 cited by
Direct Preference Optimization for Suppressing Hallucinated Prior Exams in Radiology Report Generation
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Recent advances in generative vision-language models (VLMs) have exciting potential implications for AI in radiology, yet VLMs are also known to produce hallucinations, nonsensical text, and other unwanted behaviors that can waste clinicians' time and cause patient harm. Drawing on recent work on direct preference optimization (DPO), we propose a simple method for modifying the behavior of pretrained VLMs performing radiology report generation by suppressing unwanted types of generations. We apply our method to the prevention of hallucinations of prior exams, addressing a long-established problem behavior in models performing chest X-ray report generation. Across our experiments, we find that DPO fine-tuning achieves a 3.2-4.8x reduction in lines hallucinating prior exams while maintaining model performance on clinical accuracy metrics. Our work is, to the best of our knowledge, the first work to apply DPO to medical VLMs, providing a data- and compute- efficient way to suppress problem behaviors while maintaining overall clinical accuracy.
Forward citations
Cited by 4 Pith papers
-
TaskGalaxy: Scaling Multi-modal Instruction Fine-tuning with Tens of Thousands Vision Task Types
The authors release TaskGalaxy, a 413k-sample multimodal instruction dataset with 19,227 hierarchically organized task types, and report consistent benchmark gains when it is added to LLaVA-v1.5 and InternVL-Chat-v1.0.
-
Linguistically-Aligned and Visually-Grounded Preference Optimization for Clinically-Augmented Medical Report Generation
DPO-Clin improves medical report generation by focusing preference optimization on clinical findings, adding visual-context preference inversion and counterfactual training for uncertain predictions.
-
MMedPO: Aligning Medical Vision-Language Models with Clinical-Aware Multimodal Preference Optimization
MMedPO weights preference-optimization training samples by clinical relevance scores, combining hallucinated text answers and locally noised lesion images, and reports improved medical VQA and report generation metrics.
-
Libra: Leveraging Temporal Images for Biomedical Radiology Analysis
Libra introduces a Temporal Alignment Connector for multimodal LLMs that fuses current and prior chest X-ray features and reports improved radiology report generation on MIMIC-CXR.
Discussion (0). Continue with ORCID to comment.