REVIEW 3 cited by
Self-supervised vision-language pretraining for Medical visual question answering
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Medical image visual question answering (VQA) is a task to answer clinical questions, given a radiographic image, which is a challenging problem that requires a model to integrate both vision and language information. To solve medical VQA problems with a limited number of training data, pretrain-finetune paradigm is widely used to improve the model generalization. In this paper, we propose a self-supervised method that applies Masked image modeling, Masked language modeling, Image text matching and Image text alignment via contrastive learning (M2I2) for pretraining on medical image caption dataset, and finetunes to downstream medical VQA tasks. The proposed method achieves state-of-the-art performance on all the three public medical VQA datasets. Our codes and models are available at https://github.com/pengfeiliHEU/M2I2.
Forward citations
Cited by 3 Pith papers
-
HSCR: Hierarchical Self-Contrastive Rewarding for Aligning Medical Vision Language Models
HSCR uses visual token dropout and logit contrast to construct self-generated dispreferred answers, then trains a medical VLM with explicit and implicit preference losses, improving zero-shot Rad-VQA, SLAKE, and PathV...
-
Knowing or Guessing? Robust Medical Visual Question Answering via Joint Consistency and Contrastive Learning
RoMed and CCL: a 144k-question perturbation benchmark for medical VQA and a consistency-plus-contrastive training method that improves LLaVA-Med's accuracy and reduces answer variation.
-
Uncertainty-Driven Expert Control: Enhancing the Reliability of Medical Vision-Language Models
Expert-CFG combines entropy-based uncertainty selection with classifier-free guidance over expert-highlighted text to refine MedVLM outputs, reporting gains on VQA-RAD, SLAKE, and PathVQA.
Discussion (0). Continue with ORCID to comment.