Knowing or Guessing? Robust Medical Visual Question Answering via Joint Consistency and Contrastive Learning

Jian Wu; Sibo Song; Songtao Jiang; Yang Feng; Yan Zhang; Yeying Jin; Yuxi Chen; Zuozhu Liu

REVIEW 4 major objections 4 minor 32 references

Knowing or Guessing? Robust Medical Visual Question Answering via Joint Consistency and Contrastive Learning

T0 review · 4 major / 4 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Medical VQA models answer the same question differently when it is rephrased, and the paper's joint consistency-and-contrastive training objective largely closes that gap.

desk verdict Useful robustness benchmark and training recipe for Med-VQA, with real gains, but the headline 50% consistency claim is overstated and the evaluation is in-distribution with the perturbation pipeline. read the letter →

arxiv 2508.18687 v1 pith:6ALCHAXZ submitted 2025-08-26 cs.CL

Songtao Jiang , Yuxi Chen , Sibo Song , Yan Zhang , Yeying Jin , Yang Feng , Jian Wu , Zuozhu Liu This is my paper

classification cs.CL

keywords medicalvisualquestionansweringvision-languagemodelsanswerconsistencyperturbationrobustnesslearningcontrastiveRoMeddatasetimageanalysis

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Medical visual question answering (VQA)—models that answer questions about clinical images—should give the same answer when a question is merely reworded. The paper shows they often do not: word-, sentence-, and semantic-level rephrasings can flip the answer, and adding more training data barely helps. The proposed explanation is that standard fine-tuning rewards surface phrasing rather than shared medical meaning, and that the training data contain syntactic shortcuts. To make the problem measurable, the paper builds RoMed, a dataset of 144k perturbed questions with consistency metrics (mean absolute deviation and coefficient of variation) alongside accuracy. It then proposes Consistency and Contrastive Learning (CCL), which makes a model answer all variants of a question consistently while contrasting unrelated questions, and reports state-of-the-art accuracy on three medical VQA benchmarks plus a roughly 50% improvement in answer consistency on RoMed—the difference between a model that knows and one that guesses.

What carries the argument

The load-bearing object is RoMed, a perturbation dataset in which each original medical VQA question is expanded into clusters of word-, sentence-, and semantic-level rephrasings generated by a multi-agent pipeline and validated for semantic equivalence. Consistency is quantified by Mean Absolute Deviation (MAD) and Coefficient of Variation (CV) of the model's answers within each cluster. The corrective mechanism is the CCL training objective: a knowledge-anchored consistency loss that applies the standard next-token prediction loss to original plus perturbed questions, and a bias-aware contrastive loss that treats original and perturbed versions as positive pairs and other questions in the

What would settle it

Take a sample of RoMed clusters and have human clinicians label whether each perturbed question means the same thing and has the same answer as the original; then generate new rephrasings with a different pipeline, or have humans write them, and test whether CCL-trained models stay consistent. If consistency collapses on human-verified or independently generated paraphrases, the reported robustness is an artifact of the original perturbation distribution rather than a general property of the model.

Watch

Extended reading notes

Core claim

The paper's central claim is that fragility under rephrasing is an objective problem, not a data-quantity problem: the standard supervised fine-tuning loss teaches a model to predict the next token for one surface form, so it never learns that different wordings of the same clinical question should converge to the same answer. CCL replaces that single objective with a joint one. A knowledge-anchored consistency loss applies the same next-token answer-generation loss to the original question and to word-, sentence-, and semantic-level perturbations, forcing the model to treat them as one task. A bias-aware contrastive loss pulls the hidden representations of original and perturbed questions t

Load-bearing premise

The load-bearing premise is that RoMed's perturbed questions are semantically equivalent to the originals and share the same answer—a property verified only by a single general-purpose validation model—and separately, the introduction's data-diversity argument contains an unresolved citation placeholder that makes part of the motivation untraceable.

Editorial extensions

If this is right

Accuracy on original benchmark questions is not enough to certify a medical VQA model; robustness reporting should include consistency metrics such as MAD and CV over rephrased question clusters.
Adding perturbation diversity to the training set alone does not robustify a model; the training objective itself must reward invariance to rephrasing.
CCL is a plug-and-play objective that can be added on top of existing medical VLM fine-tuning and improves both open-ended and closed-ended accuracy on Rad-VQA, SLAKE, and PathVQA.
The benefit of adding more perturbations per question levels off after one variation per level, so the consistency gains are not simply a matter of more training data.
Under CCL, representations of different question variants cluster together, which is the mechanism behind the improved robustness.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If consistency across rephrasings is a valid proxy for understanding, then conventional accuracy benchmarks systematically overstate medical VQA reliability; the same perturbation-and-consistency evaluation could be applied to other high-stakes multimodal settings such as clinical note QA or surgical video QA.
Because one proprietary general-purpose model both generates and validates the perturbations, the evaluation is only as trustworthy as that validator; sampling clusters for human review, or generating rephrasings with several independent models, would test whether the consistency gains transfer to the way clinicians actually rephrase questions.
CCL's contrastive component reshapes hidden representations rather than simply expanding data, so it could combine with other robustness techniques such as preference optimization or chain-of-thought medical reasoning; the paper does not test those combinations.
The observed diminishing returns at two variations per level suggest an economical annotation budget for building robustness datasets in other medical domains: one well-designed perturbation per level captures most of the benefit.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Short version: this paper earns its keep. It builds a large perturbed-question benchmark for medical VQA (RoMed, 144k questions with word/sentence/semantic variations) and a training method (CCL) that combines consistency-style training on paraphrases with supervised contrastive learning. On three standard benchmarks and on RoMed itself, the method moves accuracy up noticeably across two different LLaVA-Med backbones, and the ablation shows both loss terms matter. The SFT comparison is the most convincing piece: training on the same perturbed data with the ordinary objective gives almost nothing, while CCL gives a big jump. That supports the claim that the objective, not just the extra data, is doing the work.

Where I'd push back: the abstract says "improves answer consistency by 50%," but Table 2 shows CV reductions of roughly 15–20%. The 50% refers to the drop in recall on the original benchmark, which is a different and weaker statement. That is a wording problem, not necessarily a hidden failure, but it should be fixed. More substantively, MAD and CV are computed on text answers, and the paper never says how answers are mapped to numbers. Without that encoding, those metrics are undefined as written. The dataset validation also rests entirely on GPT-4o, with no human check on a sample; combined with the fact that train and test perturbations come from the same multi-agent pipeline, the evaluation is measuring robustness to this particular perturbation style rather than robustness to real clinical rephrasings. The gains on the standard benchmarks are reassuring, but the RoMed test is in-distribution by construction.

Minor points: no error bars or seed variance, though the gains are large enough that this is probably not masking a null result. The contrastive learning uses in-batch negatives; the paper reports batch size 2 per GPU, but the effective batch (with 8 GPUs) is never stated, so it is hard to tell how strong that supervision is. The "knowledge-anchored" description oversells what is actually just a language-model loss on knowledge-generated paraphrases.

Who should read it: anyone building or evaluating medical VQA systems, and people who care about robustness benchmarks for multimodal LLMs. It deserves a serious referee: the empirical contribution is real, the flaws are addressable, and the dataset could be reused. Send it to review with requests for the answer encoding, error bars, and a small human validation of the perturbation pipeline.

Referee Report

4 major / 4 minor

Summary. The paper addresses the fragility of medical vision-language models (Med-VLMs) under semantically equivalent question rephrasings. The authors construct RoMed, a dataset of 144k perturbed questions derived from Rad-VQA, SLAKE, and PathVQA using a multi-agent pipeline (HuatuoGPT-Vision, HuatuoGPT-o1, GPT-4o) with word-, sentence-, and semantic-level perturbations. They report accuracy drops of existing Med-VLMs on RoMed and propose Consistency and Contrastive Learning (CCL), which adds a consistency loss (standard autoregressive LM loss on perturbed inputs) and a contrastive loss on hidden representations. Experiments compare LLaVA-Med backbones with and without CCL on the three base benchmarks and on RoMed, reporting state-of-the-art accuracy and improved robustness as measured by MAD and CV. The paper also includes ablations, a comparison to vanilla SFT, representation visualizations, and a data-scaling study.

Significance. The paper targets a real and important limitation of Med-VQA systems: inconsistent answers across equivalent phrasings. If the RoMed construction and the consistency metrics were properly validated, the dataset and the CCL training strategy would be useful contributions to trustworthy medical AI. The paper gives explicit credit to the multi-agent perturbation design, the inclusion of both accuracy and consistency metrics, and the ablation separating consistency and contrastive components. The main experiments show consistent improvements over the LLaVA-Med baseline. However, the central robustness claim depends on the validity of the perturbation pipeline and on the numerical treatment of free-form answers; both currently have gaps that prevent the evidence from being fully load-bearing.

major comments (4)

[§2.1 (RoMed construction)] The semantic equivalence of the perturbed questions is validated only by GPT-4o, i.e., the same model family that generated them. No human evaluation, inter-annotator agreement, or independent verification is reported. Moreover, the RoMed train and test sets are generated by the same multi-agent perturbation process, so the test distribution is in-distribution with respect to the perturbation style. Consequently, the consistency gains on RoMed could reflect fitting that particular perturbation distribution rather than general robustness to real clinical rephrasings. Please add human or independent-model validation, report agreement, and evaluate on a held-out perturbation source (e.g., human-written rephrasings or a different LLM generator).
[§2.1 (MAD and CV definitions)] MAD and CV are defined as mean absolute deviation and coefficient of variation over model answers x_i, but the paper does not specify how free-form text answers and binary (yes/no) answers are converted to numeric values. Without a defined numeric encoding and a rule for handling non-numeric or unanswerable outputs, these metrics are not well-defined, and the reported reductions in CV/MAD in Table 2 are difficult to interpret. Please specify the encoding, the treatment of open-ended vs. closed-ended answers, and whether the metrics are computed per cluster or across all questions.
[Abstract and Table 2] The abstract claims 'improves answer consistency by 50%' on RoMed, but Table 2 shows CV reductions of roughly 9–30% (e.g., LLaVA-Med StableLM: RAD CV 83.9→68.3, SLAKE 65.3→57.6, Path 96.0→67.7). The 50% figure appears to refer instead to the reduction in the accuracy drop between the original benchmark and RoMed, which is a different, less direct measure. The headline is therefore ambiguous and should be re-stated with the exact metric, and both the accuracy-drop reduction and the CV/MAD changes should be reported consistently.
[§3 (Experiments)] All tables report a single run per configuration, with no error bars, multiple seeds, or statistical significance tests. Given the batch size of 2 and the joint loss with new hyperparameters (temperature τ and loss weighting), training stochasticity is non-trivial. The claim of 'notably improves' consistency and accuracy would be stronger with at least 3–5 seeds and reporting of mean±std, especially for the small gains in Table 1 and the ablation in Table 3.

minor comments (4)

[§2.2] The 'knowledge-anchored' consistency loss is simply the standard autoregressive LM loss applied to the original and perturbed inputs. The term overstates the mechanism; no explicit medical knowledge source is injected. Similarly, 'bias-aware' contrastive learning is not supported by any direct measurement of bias.
[Eq. for total loss] The total loss L = (L_ctr + L_consistency)/2 weights L_consistency, which already contains four LM terms, more heavily than L_ctr. This weighting choice is not motivated or ablated; please clarify the intended contribution of each term.
[Figure 5 / Table 4] The comparison between SFT and CCL is informative, but the caption of Table 4 says 'CCL++ indicates training with doubled dataset size' while the text says 'generating two additional variations per level'. Please align the description with the actual data construction.
[General presentation] There are typographical and consistency issues: 'Sinapore' (affiliation), 'RAD-VQA' vs 'Rad-VQA' inconsistent usage, and references [20] and [21] appear to be the same work. Also, Fig. 4(e,f) shows t-SNE-like visualizations without a quantitative separation measure; please report a numeric embedding-distance metric.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: CCL's reported gains are empirical and evaluated on a held-out RoMed test split; train/test generation overlap is a generalizability concern, not a definitional reduction.

full rationale

The paper's central claims are empirical: CCL combines a consistency loss (Section 2.2, L_consistency) and a contrastive loss (Section 2.2, L_ctr) and is evaluated on held-out RoMed test clusters and on three standard benchmarks. The consistency loss directly supervises same-answer prediction across perturbations, and the evaluation measures answer agreement across perturbations; the two are related but not identical by construction—a model could in principle optimize likelihood without yielding lower CV/MAD. The fact that RoMed train and test perturbations are generated by the same multi-agent pipeline means the robustness gains are demonstrated in-distribution with respect to the perturbation process; this limits external generalization but does not make the evaluation circular. No parameter is fitted to the test set, no prediction reduces to an input by construction, and the self-citations (refs 10-14) are contextual rather than load-bearing. The manuscript does contain a missing citation ('[?]' in Section 1) and relies on GPT-4o-only validation for semantic equivalence (Section 2.1); these are limitations in support, not circularity. Accordingly, no circular steps are identified and the score is 0.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The central claim rests on the validity of the perturbation-generation pipeline and the assumption that answers to semantically equivalent questions have a numeric encoding for MAD/CV. The contrastive temperature and loss weights are hyperparameters chosen without reported sensitivity analysis.

free parameters (3)

contrastive temperature tau
Introduced in the contrastive loss (Sec. 2.2); the value is not reported.
loss weighting L = (L_ctr + L_consistency)/2 = 0.5 each
Equal weighting is chosen by hand; no sensitivity analysis is reported.
perturbation level weights = 1.0 each
In L_consistency, each of the three perturbation losses is summed with equal weight; no ablations on these weights.

assumptions (3)

domain assumption Perturbed questions preserve semantic equivalence and share the same answer as the original
Stated in Sec. 2.1; relied on for both training and testing. Validity is checked only by GPT-4o, which is fallible.
domain assumption The multi-agent generation pipeline produces a representative distribution of clinical query variations
No evidence that the three perturbation levels cover the space of real-world rephrasings; the test set is built from the same pipeline.
ad hoc to paper MAD and CV are well-defined on mixed text and binary answers
The definitions in Sec. 2.1 apply to numeric values; the paper does not specify how answers are encoded to numbers.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Knowing or Guessing? Robust Medical Visual Question Answering via Joint Consistency and Contrastive Learning." pith.science (2026). https://pith.science/paper/6ALCHAXZ

@misc{pith2026250818687,
  author       = {Pith},
  title        = {Pith review of: Knowing or Guessing? Robust Medical Visual Question Answering via Joint Consistency and Contrastive Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6ALCHAXZ}},
  note         = {Machine review of arXiv:2508.18687}
}

read the original abstract

In high-stakes medical applications, consistent answering across diverse question phrasings is essential for reliable diagnosis. However, we reveal that current Medical Vision-Language Models (Med-VLMs) exhibit concerning fragility in Medical Visual Question Answering, as their answers fluctuate significantly when faced with semantically equivalent rephrasings of medical questions. We attribute this to two limitations: (1) insufficient alignment of medical concepts, leading to divergent reasoning patterns, and (2) hidden biases in training data that prioritize syntactic shortcuts over semantic understanding. To address these challenges, we construct RoMed, a dataset built upon original VQA datasets containing 144k questions with variations spanning word-level, sentence-level, and semantic-level perturbations. When evaluating state-of-the-art (SOTA) models like LLaVA-Med on RoMed, we observe alarming performance drops (e.g., a 40\% decline in Recall) compared to original VQA benchmarks, exposing critical robustness gaps. To bridge this gap, we propose Consistency and Contrastive Learning (CCL), which integrates two key components: (1) knowledge-anchored consistency learning, aligning Med-VLMs with medical knowledge rather than shallow feature patterns, and (2) bias-aware contrastive learning, mitigating data-specific priors through discriminative representation refinement. CCL achieves SOTA performance on three popular VQA benchmarks and notably improves answer consistency by 50\% on the challenging RoMed test set, demonstrating significantly enhanced robustness. Code will be released.

Figures

Figures reproduced from arXiv: 2508.18687 by the authors.

**Figure 1.** A simple perturbation experiment demonstrates that current Med-VLMs exhibit inconsistencies in VQA tasks, raising concerns about the robustness of Med-VQA. remarkable progress in Medical Visual Question Answering (Med-VQA) [14, 13, 11, 10, 22]. Through supervised fine-tuning (SFT) on Med-VQA training sets, these models achieve strong performance on downstream tasks. However, as illustrated in [PITH_FULL_IMAGE:figu… view at source ↗

**Figure 2.** Overview of RoMed. RoMed is a comprehensive VQA dataset spanning diverse organs and modalities (CT, MRI, X-Ray, Pathology), with dual evaluations for accuracy and robustness ensuring a holistic assessment [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 3.** Overview of our CCL pipeline. Our framework consists of two key components: (a) constructing the RoMed dataset through medical multi-agent collaboration; (b) joint knowledge-anchored consistency learning for medical expertise alignment and bias-aware contrastive learning to reduce inherent representation biases. learning [6], CCL provides explicit supervision signals to ensure the model delivers correct answers ac… view at source ↗

Figures from the paper (2 more)

**Figure 4.** Figure 4: (a)–(d) Performance degradation under varied VQA questions, significantly mitigated by CCL; (e)–(f) Representation embeddings of multi-level VQA variations. LLaVA-Med (Phi2) + SFT + CCL 60 70 80 90 100 Accuracy (%) 63.9% 65.0% 81.4% (a) RoMed-radvqa Accuracy Recall LLa…

**Figure 5.** Figure 5: Comparison between SFT and CCL. SFT yields minimal performance gains. representations under varied perturbations become more robust, suggesting that the model learns more low-level, generalizable features across different levels of perturbations. This makes it better s…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

32 extracted references · 11 canonical work pages

[1]

arXiv preprint arXiv:2404.14219 (2024)

Abdin, M., Jacobs, S.A., Awan, A.A., Aneja, J., Awadallah, A., Awadalla, H., Bach, N., Bahree, A., Bakhtiari, A., Behl, H., et al.: Phi-3 technical report: A highly capable language model locally on your phone. arXiv preprint arXiv:2404.14219 (2024)

arXiv 2024
[2]

Bioengineering (2023)

Bazi, Y., Rahhal, M.M.A., Bashmal, L., Zuair, M.: Vision–language model for visual question answering in medical imagery. Bioengineering (2023)

work page 2023
[3]

arXiv preprint arXiv:2402.17834 (2024)

Bellagente, M., Tow, J., Mahan, D., Phung, D., Zhuravinskyi, M., Adithyan, R., Baicoianu, J., Brooks, B., Cooper, N., Datta, A., et al.: Stable lm 2 1.6 b technical report. arXiv preprint arXiv:2402.17834 (2024)

arXiv 2024
[4]

Chen, J., Cai, Z., Ji, K., Wang, X., Liu, W., Wang, R., Hou, J., Wang, B.: Huatuogpt-o1, towards medical complex reasoning with llms (2024), https://arxiv.org/abs/2412.18925

arXiv 2024
[5]

Chen, J., Ouyang, R., Gao, A., Chen, S., Chen, G.H., Wang, X., Zhang, R., Cai, Z., Ji, K., Yu, G., Wan, X., Wang, B.: Huatuogpt-vision, towards injecting medical visual knowledge into multimodal llms at scale (2024), https://arxiv.org/abs/2406.19280

arXiv 2024
[6]

In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (June 2019)

Dwibedi, D., Aytar, Y., Tompson, J., Sermanet, P., Zisserman, A.: Temporal cycle- consistency learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (June 2019)

work page 2019
[7]

Eslami, S., Meinel, C., De Melo, G.: Pubmedclip: How much does clip benefit visual question answering in the medical domain? In: Findings of the Association for Computational Linguistics: EACL 2023. pp. 1151–1163 (2023)

work page 2023
[8]

arXiv preprint arXiv:2003.10286 (2020)

He, X., Zhang, Y., Mou, L., Xing, E., Xie, P.: Pathvqa: 30000+ questions for medical visual question answering. arXiv preprint arXiv:2003.10286 (2020)

arXiv 2003

Show all 32 references

[9]

arXiv preprint arXiv:2410.21276 (2024)

Hurst, A., Lerer, A., Goucher, A.P., Perelman, A., Ramesh, A., Clark, A., Os- trow, A., Welihinda, A., Hayes, A., Radford, A., et al.: Gpt-4o system card. arXiv preprint arXiv:2410.21276 (2024)

2024 arXiv
[10]

arXiv preprint arXiv:2506.12849 (2025)

Jiang, S., Wang, Y., Chen, R., Zhang, Y., Luo, R., Lei, B., Song, S., Feng, Y., Sun, J., Wu, J., et al.: Capo: Reinforcing consistent reasoning in medical decision- making. arXiv preprint arXiv:2506.12849 (2025)

2025 arXiv
[11]

arXiv preprint arXiv:2504.14692 (2025)

Jiang, S., Wang, Y., Song, S., Zhang, Y., Meng, Z., Lei, B., Wu, J., Sun, J., Liu, Z.: Omniv-med: Scaling medical vision-language model for universal visual understanding. arXiv preprint arXiv:2504.14692 (2025)

2025 arXiv
[12]

arXiv preprint arXiv:2410.15334 (2024) 10 Songtao Jiang et al

Jiang, S., Zhang, Y., Chen, R., Jin, Y., Liu, Z.: Modality-fair preference optimiza- tion for trustworthy mllm alignment. arXiv preprint arXiv:2410.15334 (2024) 10 Songtao Jiang et al

2024 arXiv
[13]

arXiv preprint arXiv:2506.00805 (2025)

Jiang, S., Zhang, Y., Jin, Y., Tang, Z., Wu, Y., Feng, Y., Wu, J., Liu, Z.: Hscr: Hierarchical self-contrastive rewarding for aligning medical vision language models. arXiv preprint arXiv:2506.00805 (2025)

2025 arXiv
[14]

In: Find- ingsoftheAssociationforComputationalLinguistics:EMNLP2024.pp.3843–3860 (2024)

Jiang, S., Zheng, T., Zhang, Y., Jin, Y., Yuan, L., Liu, Z.: Med-moe: Mixture of domain-specific experts for lightweight medical vision-language models. In: Find- ingsoftheAssociationforComputationalLinguistics:EMNLP2024.pp.3843–3860 (2024)

2024
[15]

Advances in neural information processing systems33, 18661–18673 (2020)

Khosla, P., Teterwak, P., Wang, C., Sarna, A., Tian, Y., Isola, P., Maschinot, A., Liu, C., Krishnan, D.: Supervised contrastive learning. Advances in neural information processing systems33, 18661–18673 (2020)

2020
[16]

Scientific data 5(1), 1–10 (2018)

Lau, J.J., Gayen, S., Ben Abacha, A., Demner-Fushman, D.: A dataset of clinically generated visual questions and answers about radiology images. Scientific data 5(1), 1–10 (2018)

2018
[17]

Advances in Neural Information Processing Systems36 (2024)

Li, C., Wong, C., Zhang, S., Usuyama, N., Liu, H., Yang, J., Naumann, T., Poon, H., Gao, J.: Llava-med: Training a large language-and-vision assistant for biomedicine in one day. Advances in Neural Information Processing Systems36 (2024)

2024
[18]

arXiv preprint arXiv:2211.13594 (2022)

Li, P., Liu, G., Tan, L., Liao, J., Zhong, S.: Self-supervised vision-language pre- training for medical visual question answering. arXiv preprint arXiv:2211.13594 (2022)

2022 arXiv
[19]

IEEE (2021)

Liu, B., Zhan, L.M., Xu, L., Ma, L., Yang, Y., Wu, X.M.: Slake: A semantically- labeledknowledge-enhanceddatasetformedicalvisualquestionanswering.In:2021 IEEE18thInternationalSymposiumonBiomedicalImaging(ISBI).pp.1650–1654. IEEE (2021)

2021
[20]

In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

Liu, H., Li, C., Li, Y., Lee, Y.J.: Improved baselines with visual instruction tuning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 26296–26306 (June 2024)

2024
[21]

In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

Liu, H., Li, C., Li, Y., Lee, Y.J.: Improved baselines with visual instruction tuning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 26296–26306 (2024)

2024
[22]

arXiv preprint arXiv:2412.13736 (2024)

Liu, J., Wang, Y., Du, J., Zhou, J.T., Liu, Z.: Medcot: Medical chain of thought via hierarchical expert. arXiv preprint arXiv:2412.13736 (2024)

2024 arXiv
[23]

arXiv preprint arXiv:2304.01611 (2023)

Liu, Y., Wang, Z., Xu, D., Zhou, L.: Q2atransformer: Improving medical vqa via an answer querying decoder. arXiv preprint arXiv:2304.01611 (2023)

2023 arXiv
[24]

In: Machine Learning for Health (ML4H)

Moor, M., Huang, Q., Wu, S., Yasunaga, M., Dalmia, Y., Leskovec, J., Zakka, C., Reis, E.P., Rajpurkar, P.: Med-flamingo: a multimodal medical few-shot learner. In: Machine Learning for Health (ML4H). pp. 353–367. PMLR (2023)

2023
[25]

arXiv preprint arXiv:1909.04696 (2019)

Ray, A., Sikka, K., Divakaran, A., Lee, S., Burachas, G.: Sunny and dark outside?! improving answer consistency in vqa through entailed question generation. arXiv preprint arXiv:1909.04696 (2019)

1909 arXiv
[26]

arXiv preprint arXiv:2305.09617 (2023)

Singhal, K., Tu, T., Gottweis, J., Sayres, R., Wulczyn, E., Hou, L., Clark, K., Pfohl, S., Cole-Lewis, H., Neal, D., et al.: Towards expert-level medical question answering with large language models. arXiv preprint arXiv:2305.09617 (2023)

2023 arXiv
[27]

arXiv preprint arXiv:2303.05977 (2023)

van Sonsbeek, T., Derakhshani, M.M., Najdenkoska, I., Snoek, C.G., Worring, M.: Open-ended medical visual question answering through prefix tuning of language models. arXiv preprint arXiv:2303.05977 (2023)

2023 arXiv
[28]

arXiv preprint arXiv:2406.19973 (2024) Title Suppressed Due to Excessive Length 11

Sun, G., Qin, C., Fu, H., Wang, L., Tao, Z.: Stllava-med: Self-training large language and vision assistant for medical question-answering. arXiv preprint arXiv:2406.19973 (2024) Title Suppressed Due to Excessive Length 11

2024 arXiv
[29]

In: European Conference on Computer Vision

Sun, G., Qin, C., Wang, J., Chen, Z., Xu, R., Tao, Z.: Sq-llava: Self-questioning for large vision-language assistant. In: European Conference on Computer Vision. pp. 156–172. Springer (2025)

2025
[30]

arXiv preprint arXiv:2305.17100 (2023)

Zhang, K., Yu, J., Yan, Z., Liu, Y., Adhikarla, E., Fu, S., Chen, X., Chen, C., Zhou, Y., Li, X., et al.: Biomedgpt: A unified and generalist biomedical generative pre-trained transformer for vision, language, and multimodal tasks. arXiv preprint arXiv:2305.17100 (2023)

2023 arXiv
[31]

arXiv preprint arXiv:2303.00915 (2023)

Zhang, S., Xu, Y., Usuyama, N., Bagga, J., Tinn, R., Preston, S., Rao, R., Wei, M., Valluri, N., Wong, C., et al.: Large-scale domain-specific pretraining for biomedical vision-language processing. arXiv preprint arXiv:2303.00915 (2023)

2023 arXiv
[32]

arXiv preprint arXiv:2305.10415 (2023)

Zhang, X., Wu, C., Zhao, Z., Lin, W., Zhang, Y., Wang, Y., Xie, W.: Pmc-vqa: Visual instruction tuning for medical visual question answering. arXiv preprint arXiv:2305.10415 (2023)

2023 arXiv

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.

[1] [1]

arXiv preprint arXiv:2404.14219 (2024)

Abdin, M., Jacobs, S.A., Awan, A.A., Aneja, J., Awadallah, A., Awadalla, H., Bach, N., Bahree, A., Bakhtiari, A., Behl, H., et al.: Phi-3 technical report: A highly capable language model locally on your phone. arXiv preprint arXiv:2404.14219 (2024)

arXiv 2024

[2] [2]

Bioengineering (2023)

Bazi, Y., Rahhal, M.M.A., Bashmal, L., Zuair, M.: Vision–language model for visual question answering in medical imagery. Bioengineering (2023)

work page 2023

[3] [3]

arXiv preprint arXiv:2402.17834 (2024)

Bellagente, M., Tow, J., Mahan, D., Phung, D., Zhuravinskyi, M., Adithyan, R., Baicoianu, J., Brooks, B., Cooper, N., Datta, A., et al.: Stable lm 2 1.6 b technical report. arXiv preprint arXiv:2402.17834 (2024)

arXiv 2024

[4] [4]

Chen, J., Cai, Z., Ji, K., Wang, X., Liu, W., Wang, R., Hou, J., Wang, B.: Huatuogpt-o1, towards medical complex reasoning with llms (2024), https://arxiv.org/abs/2412.18925

arXiv 2024

[5] [5]

Chen, J., Ouyang, R., Gao, A., Chen, S., Chen, G.H., Wang, X., Zhang, R., Cai, Z., Ji, K., Yu, G., Wan, X., Wang, B.: Huatuogpt-vision, towards injecting medical visual knowledge into multimodal llms at scale (2024), https://arxiv.org/abs/2406.19280

arXiv 2024

[6] [6]

In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (June 2019)

Dwibedi, D., Aytar, Y., Tompson, J., Sermanet, P., Zisserman, A.: Temporal cycle- consistency learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (June 2019)

work page 2019

[7] [7]

Eslami, S., Meinel, C., De Melo, G.: Pubmedclip: How much does clip benefit visual question answering in the medical domain? In: Findings of the Association for Computational Linguistics: EACL 2023. pp. 1151–1163 (2023)

work page 2023

[8] [8]

arXiv preprint arXiv:2003.10286 (2020)

He, X., Zhang, Y., Mou, L., Xing, E., Xie, P.: Pathvqa: 30000+ questions for medical visual question answering. arXiv preprint arXiv:2003.10286 (2020)

arXiv 2003