Pith. sign in

REVIEW 3 major objections 58 references

Medical vision-language models often answer from text shortcuts; a hard-negative contrastive objective raises accuracy and cuts visual hallucination.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-12 00:56 UTC pith:OMJVU42N

load-bearing objection Usable medical-VLM grounding audit plus a contrastive fix that moves accuracy and VHR; train/eval overlap softens the headline numbers but does not erase the contribution. the 3 major comments →

arxiv 2607.03647 v1 pith:OMJVU42N submitted 2026-07-04 cs.CV

Do Medical Vision Language Models Actually See? A Counterfactual Grounding Framework and Hard-Negative Contrastive Training for Visually-Reliant Medical VLMs

Anas Zafar , Leema Krishna Murali , Siddhant Bharadwaj , Ashish Vashist , Jia Wu This is my paper
classification cs.CV
keywords medical vision-language modelsvisual groundingcounterfactual evaluationhard-negative contrastive learningVisual Hallucination Ratemedical VQAlanguage priors
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Medical vision-language models can score well on visual question answering while barely looking at the image, relying instead on language patterns in the question. This paper builds a counterfactual test that replaces the real image with blank, shuffled, missing, or confusable substitutes and measures how much the answer actually depends on visual content. It then trains CORAL, a 7-billion-parameter model, with a Contrastive Grounding Objective that rewards the model only when its answer changes under a hard-negative image swap. On four medical VQA benchmarks the method lifts accuracy by 6.7 points and cuts the rate of confidently wrong visual answers by 8 points relative to the matched base model, while two reinforcement-learning variants do not. The work matters because high accuracy alone can hide a model that would fail when the image truly matters for diagnosis.

Core claim

Across four closed-form medical VQA benchmarks, a LoRA fine-tune of Qwen2.5-VL-7B trained with a Contrastive Grounding Objective that penalises answer invariance under CLIP-retrieved hard-negative image swaps improves macro accuracy by +6.7 percentage points and reduces Visual Hallucination Rate by 8.0 points over the matched base; neither of two accuracy-only RL checkpoints achieves a significant gain on either metric.

What carries the argument

The Contrastive Grounding Objective (CGO): an additive reward term that is positive only when the model’s extracted answer changes between the real image and a CLIP-similar but label-different hard negative, thereby making language-prior shortcuts high-loss at training time.

Load-bearing premise

The method assumes that CLIP nearest-neighbour images with different labels are hard enough visual confusers to force genuine visual reliance rather than just answer-format sensitivity.

What would settle it

Re-run the same paired n=400 evaluation after replacing CLIP hard negatives with images that human clinicians rate as visually confusable on the same clinical feature; if the accuracy and VHR gains disappear, the claim that CGO induces visual reliance fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 0 minor

Summary. The paper argues that medical VLMs often exploit textual shortcuts rather than visual evidence, and proposes (i) a counterfactual evaluation suite that substitutes real images with blank, shuffled, absent, and CLIP hard-negative surrogates, yielding metrics such as VRS and VHR, and (ii) CORAL, a LoRA fine-tune of Qwen2.5-VL-7B trained with a Contrastive Grounding Objective (CGO) that rewards answer change under hard-negative image swaps. On a paired n=400 evaluation across PathVQA, PMC-VQA, SLAKE, and VQA-RAD, CORAL is reported to raise macro accuracy by +6.7 pp and cut VHR by 8.0 pp relative to the matched base, while two MedVLThinker RL checkpoints do not; cross-domain diagnostics further show much smaller image-substitution costs on medical than on general VQA. The authors release framework, code, and weights and openly list train/eval overlap and underpowered secondary metrics as limitations.

Significance. If the accuracy and VHR gains hold under clean held-out evaluation, the work supplies both a practical audit protocol for clinical VLMs and a concrete training-time intervention that targets language-prior shortcuts—directly relevant to reliability and SaMD-style failure-mode analysis. Strengths include carefully paired bootstrap/McNemar inference, an explicit multi-condition metric suite, open discussion of underpowered VRS/BD and format-dependent IS, and planned release of code and weights. The medical-vs-general grounding gap (≤6.5 pp vs 48–61 pp) is a useful diagnostic contribution even independent of CORAL.

major comments (3)
  1. [Table 1, Table 3, Table 4, §3.1, §3.5] Table 1 and §3.1/§3.5: CGO hard-negative pools are mined from the same benchmark sources used in Phase-2 evaluation (2,000 PMC-VQA and 3,362 PathVQA-closed pairs among 5,778 total). Table 4 shows the only accuracy CI that cleanly excludes zero is PMC-VQA (+15 pp), the richest training pool, while PathVQA accuracy regresses. The headline macro in Table 3 and the abstract therefore pools in-domain items with SLAKE/VQA-RAD. Because the paper does not report decontamination (overlap of the seed-42 100-item eval sets with the training pools, or near-duplicate image/question rates), the central +6.7 pp / −8.0 pp claims are not cleanly out-of-distribution. Please either (a) remove overlapping items and re-report macro CIs, or (b) present a fully held-out macro (SLAKE+VQA-RAD only, or an external set) as the primary result and demote the pooled macro.
  2. [Table 3, §4.1, §3.3 Eq. (2)] Table 3, ISpred column and §4.1 “IS caveat”: CORAL has the worst (highest) extracted-answer invariance under shuffle (ISpred=0.698 vs base 0.518). The paper’s operational definition of visual reliance (§1, §3.3) is that predictions should change when the image is made uninformative. A model that keeps the same multiple-choice letter ~70% of the time under shuffle is not more visually reliant on this primary behavioural criterion, even if VHR falls. The accuracy/VHR gains can still be real, but the framing that CGO produces a “visually-reliant” model is not supported by ISpred and should be revised; either strengthen the contrastive term (as the authors suggest for future work) until ISpred improves, or restate claims as “reduced VHR / higher accuracy under hard-negative training” without the stronger reliance claim.
  3. [§3.1 Eq. (1), Table 1] §3.1 Eq. (1) and Table 1: Hard negatives are defined as CLIP-ViT-L/14 nearest neighbours with differing ground-truth labels (mean cosine 0.96–0.98). There is no independent check that these pairs are clinically or visually confusable (e.g., human rating, shared pathology labels, or feature-level overlap beyond CLIP). If many “hard” negatives differ only in non-visual metadata or answer format while looking nearly identical, CGO may primarily penalise answer-format invariance rather than force use of diagnostic visual content. A short validation (sample of pairs with clinical feature annotation or human confusability) or an ablation with weaker/random negatives would make the mechanism claim load-bearing rather than assumed.

Circularity Check

1 steps flagged

No load-bearing circularity in the derivation chain; metrics and CGO are definitionally independent of the measured gains, which remain empirical. Mild non-central self-citation and openly flagged train/eval source overlap do not reduce claims by construction.

specific steps
  1. self citation load bearing [§2.6 and References (Zafar et al. 2026)]
    "Anas Zafar, Leema Krishna Murali, and Ashish Vashist. 2026. Beyond accuracy: Evaluating visual grounding in multimodal medical reasoning. arXiv preprint arXiv:2603.03437."

    Authors cite their own concurrent arXiv on visual grounding metrics. This is non-load-bearing (appears only in Related Work; the present metrics, CGO, and Phase-2 results are fully defined and measured independently here), so it raises the score only to 1 rather than higher.

full rationale

The paper's core claims are empirical measurements under a fully specified counterfactual protocol, not first-principles derivations that collapse to their inputs. VRS, BD, VBR, VHR, IS_pred, HN-GAP etc. are defined directly from binary correctness indicators under controlled image conditions (real/blank/shuffled/no-image/HN) with no free parameters fitted to the target quantities (Eqs. 2-4 and surrounding text). CGO (Eq. 1) is an additive reward term that penalises answer invariance under CLIP hard-negatives; it is optimised during LoRA training and then evaluated post-hoc, so the reported +6.7 pp accuracy and -8.0 pp VHR are not tautological. Hard-negative pools (Table 1) and Phase-2 items share benchmark sources, and the paper itself lists train/eval overlap as a limitation; this is a standard leakage/generalisation concern, not a definitional reduction of the form 'fitted parameter renamed as prediction'. Self-citations (Zafar et al. 2026 and related) appear only in Related Work and do not supply uniqueness theorems, ansätze, or load-bearing premises for the metrics or the CORAL results. No uniqueness is imported, no ansatz is smuggled, and no known result is merely renamed. The evaluation is self-contained against external benchmarks with paired bootstrap CIs and McNemar tests. Score 1 reflects only the minor, non-central self-citation and acknowledged source overlap; the central derivation chain does not reduce by construction.

Axiom & Free-Parameter Ledger

3 free parameters · 4 axioms · 2 invented entities

The central empirical claims rest on a small set of free hyperparameters chosen by validation sweep, standard ML training assumptions, and the operational definition that visual reliance equals answer change under image substitution. No new physical entities are postulated; the invented constructs are evaluation metrics and a training reward.

free parameters (3)
  • CGO contrastive weight λ = 0.5
    Selected on a held-out validation sweep; set to 0.5. Directly scales the answer-change reward that drives the reported VHR reduction.
  • LoRA rank r / alpha = r=16, α=32
    r=16, α=32 chosen by authors; controls capacity of the adapter whose weights produce the claimed gains.
  • GRPO KL coefficient β and group size G = β=0.01, G=8
    β=0.01, G=8; standard RL hyperparameters that affect how strongly the contrastive signal is optimized.
axioms (4)
  • ad hoc to paper A model is visually reliant if and only if its extracted answer changes when the image is replaced by an uninformative or hard-negative surrogate.
    Operational definition introduced in §1 and §3.2–3.3; underpins every grounding metric (VRS, VHR, IS_pred, etc.).
  • domain assumption CLIP-ViT-L/14 cosine similarity with a label-difference constraint yields clinically meaningful hard negatives.
    Used for both training pools (Table 1) and evaluation HN-GAP; never validated against human or clinical feature similarity.
  • domain assumption Extracted multiple-choice letter or normalized free-text token is a sufficient proxy for the model’s decision (Ans(·)).
    All metrics and the CGO reward operate on Ans(·) rather than full generation strings (§3.1, Eq. 1–2).
  • standard math Paired bootstrap resampling items within each benchmark yields valid 95 % CIs for macro-mean differences.
    Standard non-parametric inference (§3.4); B=10 000, seed 42.
invented entities (2)
  • Visual Reliance Score (VRS) and Visual Hallucination Rate (VHR) no independent evidence
    purpose: Quantify net visual advantage and the asymmetric failure of being wrong on real images yet correct on shuffled images.
    Defined in §3.3; primary reported outcomes. Independent evidence is limited to the synthetic signal control mentioned but not detailed.
  • Contrastive Grounding Objective (CGO) no independent evidence
    purpose: Training reward that adds a binary answer-change term under hard-negative image swap to standard accuracy reward.
    Eq. 1; the sole training intervention claimed to produce the accuracy and VHR gains.

pith-pipeline@v1.1.0-grok45 · 17542 in / 3243 out tokens · 28126 ms · 2026-07-12T00:56:35.039029+00:00 · methodology

0 comments
read the original abstract

Large vision language models (VLMs) report strong accuracy on medical question-answering, yet it remains unclear whether they reason from visual evidence or exploit textual shortcuts. We introduce a counterfactual evaluation framework that decouples visual and textual contributions by substituting input images with controlled surrogates blank, pixel-shuffled, image-absent, and CLIP-retrieved hard negatives and derive a suite of grounding metrics including the Visual Reliance Score (VRS) and Visual Hallucination Rate (VHR). We further introduce CORAL (COntrastive Retrieval-Augmented Learning), a 7B-parameter LoRA fine-tune of Qwen2.5-VL-7B trained with a Contrastive Grounding Objective (CGO) that penalises answer invariance under hard-negative image swaps. On a paired controlled evaluation across four closed-form medical VQA benchmarks (PathVQA, PMC-VQA, SLAKE, VQA-RAD; n=400 total), CORAL improves macro accuracy by +6.7 pp (P(Delta>0)=0.988) and reduces VHR by 8.0 pp (P<0.001) over the matched Qwen2.5-VL-7B base; neither MedVLThinker RL variant achieves a significant gain on either metric. Cross-domain diagnostics further reveal that image substitution costs only <=6.5 pp on medical benchmarks versus 48-61 pp on general-domain tasks, situating the grounding gap that CGO targets. We discuss evaluation limitations openly including train/eval benchmark overlap and underpowered secondary metrics and release our framework, training code, and model weights to support reproducible grounding audits of medical VLMs.

Figures

Figures reproduced from arXiv: 2607.03647 by Anas Zafar, Ashish Vashist, Jia Wu, Leema Krishna Murali, Siddhant Bharadwaj.

Figure 1
Figure 1. Figure 1: Overview of the proposed counterfactual grounding framework. Given an image and question, the [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Blank Drop (BD = Accreal − Accblank) across eight benchmarks for the four MedVLM fam￾ilies (n=500 per benchmark). Each axis spans [−0.07, +0.13]. The dashed octagon indicates the ap￾proximate BD=0 reference region; values inside sug￾gest language-shortcut behaviour, while values outside indicate that removing the image reduces accuracy. 4.1.1 CORAL Per-Benchmark Behaviour [PITH_FULL_IMAGE:figures/full_fig… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

58 extracted references · 10 linked inside Pith

  1. [1]

    Aho and Jeffrey D

    Alfred V. Aho and Jeffrey D. Ullman , title =. 1972

    1972

  2. [2]

    Publications Manual , year = "1983", publisher =

    1983

  3. [3]

    Chandra and Dexter C

    Ashok K. Chandra and Dexter C. Kozen and Larry J. Stockmeyer , year = "1981", title =. doi:10.1145/322234.322243

    doi:10.1145/322234.322243 1981

  4. [4]

    arXiv preprint arXiv:2603.03437 , year=

    Beyond accuracy: Evaluating visual grounding in multimodal medical reasoning , author=. arXiv preprint arXiv:2603.03437 , year=

    arXiv

  5. [5]

    arXiv preprint arXiv:2604.08815 , year=

    Towards Responsible Multimodal Medical Reasoning via Context-Aligned Vision-Language Models , author=. arXiv preprint arXiv:2604.08815 , year=

    Pith/arXiv arXiv

  6. [6]

    Scalable training of

    Andrew, Galen and Gao, Jianfeng , booktitle=. Scalable training of

  7. [7]

    Dan Gusfield , title =. 1997

    1997

  8. [8]

    Tetreault , title =

    Mohammad Sadegh Rasooli and Joel R. Tetreault , title =. Computing Research Repository , volume =. 2015 , url =

    2015

  9. [9]

    A Framework for Learning Predictive Structures from Multiple Tasks and Unlabeled Data , Volume =

    Ando, Rie Kubota and Zhang, Tong , Issn =. A Framework for Learning Predictive Structures from Multiple Tasks and Unlabeled Data , Volume =. Journal of Machine Learning Research , Month = dec, Numpages =

  10. [10]

    Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

    Learning to exploit temporal structure for biomedical vision-language processing , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

  11. [11]

    Proceedings of the Asian conference on computer vision , pages=

    Medblip: Bootstrapping language-image pre-training from 3d medical images and texts , author=. Proceedings of the Asian conference on computer vision , pages=

  12. [12]

    Advances in Neural Information Processing Systems , volume=

    Llava-med: Training a large language-and-vision assistant for biomedicine in one day , author=. Advances in Neural Information Processing Systems , volume=

  13. [13]

    Zhang, Kai and Zhou, Rong and Adhikarla, Eashan and Yan, Zhiling and Liu, Yixin and Yu, Jun and Liu, Zhengliang and Chen, Xun and Davison, Brian D. and Ren, Hui and Huang, Jing and Chen, Chen and Zhou, Yuyin and Fu, Sunyang and Liu, Wei and Liu, Tianming and Li, Xiang and Chen, Yong and He, Lifang and Zou, James and Li, Quanzheng and Liu, Hongfang and Sun...

    doi:10.1038/s41591-024-03185-2

  14. [14]

    2021 IEEE 18th international symposium on biomedical imaging (ISBI) , pages=

    Slake: A semantically-labeled knowledge-enhanced dataset for medical visual question answering , author=. 2021 IEEE 18th international symposium on biomedical imaging (ISBI) , pages=. 2021 , organization=

    2021

  15. [15]

    Scientific data , volume=

    A dataset of clinically generated visual questions and answers about radiology images , author=. Scientific data , volume=. 2018 , publisher=

    2018

  16. [16]

    arXiv preprint arXiv:2003.10286 , year=

    Pathvqa: 30000+ questions for medical visual question answering , author=. arXiv preprint arXiv:2003.10286 , year=

    Pith/arXiv arXiv 2003

  17. [17]

    arXiv preprint arXiv:1611.07450 , year=

    Grad-CAM: Why did you say that? , author=. arXiv preprint arXiv:1611.07450 , year=

    Pith/arXiv arXiv

  18. [18]

    Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

    Transformer interpretability beyond attention visualization , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

  19. [19]

    Qwen2.5-VL , url =

    Qwen Team , month =. Qwen2.5-VL , url =

  20. [20]

    Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

    Gqa: A new dataset for real-world visual reasoning and compositional question answering , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

  21. [21]

    Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=

    Breaking common sense: Whoops! a vision-and-language benchmark of synthetic and compositional images , author=. Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=

  22. [22]

    Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=

    Llava-cot: Let vision language models reason step-by-step , author=. Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=

  23. [23]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  24. [24]

    arXiv preprint arXiv:2305.10415 , year=

    Pmc-vqa: Visual instruction tuning for medical visual question answering , author=. arXiv preprint arXiv:2305.10415 , year=

    Pith/arXiv arXiv

  25. [25]

    Findings of the Association for Computational Linguistics: ACL 2024 , pages=

    Aligning large multimodal models with factually augmented rlhf , author=. Findings of the Association for Computational Linguistics: ACL 2024 , pages=

    2024

  26. [26]

    arXiv preprint arXiv:2203.02155 , year=

    Training language models to follow instructions with human feedback , author=. arXiv preprint arXiv:2203.02155 , year=

    Pith/arXiv arXiv

  27. [27]

    arXiv preprint arXiv:2204.05862 , year=

    Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback , author=. arXiv preprint arXiv:2204.05862 , year=

    Pith/arXiv arXiv

  28. [28]

    arXiv preprint arXiv:1707.06347 , year=

    Proximal policy optimization algorithms , author=. arXiv preprint arXiv:1707.06347 , year=

    Pith/arXiv arXiv

  29. [29]

    arXiv preprint arXiv:2402.03300 , year=

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models , author=. arXiv preprint arXiv:2402.03300 , year=

    Pith/arXiv arXiv

  30. [30]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Rlhf-v: Towards trustworthy mllms via behavior alignment from fine-grained correctional human feedback , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  31. [31]

    Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

    Rlaif-v: Open-source ai feedback leads to super gpt-4v trustworthiness , author=. Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

  32. [32]

    arXiv preprint arXiv:2501.18362 , year=

    Medxpertqa: Benchmarking expert-level medical reasoning and understanding , author=. arXiv preprint arXiv:2501.18362 , year=

    Pith/arXiv arXiv

  33. [33]

    Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

    Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

  34. [34]

    Proceedings of the IEEE conference on computer vision and pattern recognition , pages=

    Making the v in vqa matter: Elevating the role of image understanding in visual question answering , author=. Proceedings of the IEEE conference on computer vision and pattern recognition , pages=

  35. [35]

    Advances in neural information processing systems , volume=

    Learn to explain: Multimodal reasoning via thought chains for science question answering , author=. Advances in neural information processing systems , volume=

  36. [36]

    Findings of the association for computational linguistics: ACL 2022 , pages=

    Chartqa: A benchmark for question answering about charts with visual and logical reasoning , author=. Findings of the association for computational linguistics: ACL 2022 , pages=

    2022

  37. [37]

    European Conference on Computer Vision , pages=

    Reliable visual question answering: Abstain rather than answer incorrectly , author=. European Conference on Computer Vision , pages=. 2022 , organization=

    2022

  38. [38]

    Scientific data , volume=

    Aci-bench: a novel ambient clinical intelligence dataset for benchmarking automatic visit note generation , author=. Scientific data , volume=. 2023 , publisher=

    2023

  39. [39]

    Proceedings of the 2016 conference on empirical methods in natural language processing , pages=

    Analyzing the behavior of visual question answering models , author=. Proceedings of the 2016 conference on empirical methods in natural language processing , pages=

    2016

  40. [40]

    International Workshop on Machine Learning in Medical Imaging , pages=

    Benchmarking dependence measures to prevent shortcut learning in medical imaging , author=. International Workshop on Machine Learning in Medical Imaging , pages=. 2024 , organization=

    2024

  41. [41]

    2021 , month =

    Artificial Intelligence/Machine Learning (. 2021 , month =

    2021

  42. [42]

    Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers) , pages=

    Annotation artifacts in natural language inference data , author=. Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers) , pages=

    2018

  43. [43]

    Nature Machine Intelligence , volume=

    Shortcut learning in deep neural networks , author=. Nature Machine Intelligence , volume=. 2020 , publisher=

    2020

  44. [44]

    Proceedings of the IEEE conference on computer vision and pattern recognition , pages=

    Don't just assume; look and answer: Overcoming priors for visual question answering , author=. Proceedings of the IEEE conference on computer vision and pattern recognition , pages=

  45. [45]

    Proceedings of the second workshop on shortcomings in vision and language , pages=

    Adversarial regularization for visual question answering: Strengths, shortcomings, and side effects , author=. Proceedings of the second workshop on shortcomings in vision and language , pages=

  46. [46]

    Proceedings of the 2020 conference on empirical methods in natural language processing (EMNLP) , pages=

    Learning to contrast the counterfactual samples for robust visual question answering , author=. Proceedings of the 2020 conference on empirical methods in natural language processing (EMNLP) , pages=

    2020

  47. [47]

    Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages =

    Shekhar, Ravi and Pezzelle, Sandro and Klimovich, Yauhen and Herbelot, Aur. Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages =. 2017 , publisher =

    2017

  48. [48]

    Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages =

    Parcalabescu, Letitia and Cafagna, Michele and Muradjan, Lilitta and Frank, Anette and Calixto, Iacer and Gatt, Albert , title =. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages =. 2022 , publisher =

    2022

  49. [49]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages =

    Thrush, Tristan and Jiang, Ryan and Bartolo, Max and Singh, Amanpreet and Williams, Adina and Kiela, Douwe and Ross, Candace , title =. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages =. 2022 , doi =

    2022

  50. [50]

    International Conference on Learning Representations (ICLR) , year =

    Yuksekgonul, Mert and Bianchi, Federico and Kalluri, Pratyusha and Jurafsky, Dan and Zou, James , title =. International Conference on Learning Representations (ICLR) , year =

  51. [51]

    Proceedings of the 2024 conference on empirical methods in natural language processing , pages=

    Towards injecting medical visual knowledge into multimodal llms at scale , author=. Proceedings of the 2024 conference on empirical methods in natural language processing , pages=

    2024

  52. [52]

    and Le, Quoc V

    Wei, Jason and Wang, Xuezhi and Schuurmans, Dale and Bosma, Maarten and Ichter, Brian and Xia, Fei and Chi, Ed H. and Le, Quoc V. and Zhou, Denny , title =. Advances in Neural Information Processing Systems (NeurIPS) , volume =

  53. [53]

    arXiv preprint arXiv:2302.00923 , year =

    Zhang, Zhuosheng and Zhang, Aston and Li, Mu and Zhao, Hai and Karypis, George and Smola, Alex , title =. arXiv preprint arXiv:2302.00923 , year =

    Pith/arXiv arXiv

  54. [54]

    2025 , eprint=

    Qwen2.5-VL Technical Report , author=. 2025 , eprint=

    2025

  55. [55]

    2021 , journal =

    Learning Transferable Visual Models From Natural Language Supervision , author =. 2021 , journal =

    2021

  56. [56]

    CoRR , volume =

    Letitia Parcalabescu and Albert Gatt and Anette Frank and Iacer Calixto , title =. CoRR , volume =. 2021 , url =

    2021

  57. [57]

    ArXiv , year=

    Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting , author=. ArXiv , year=

  58. [58]

    ArXiv , year=

    Human-Adversarial Visual Question Answering , author=. ArXiv , year=