Pith. sign in

REVIEW 5 major objections 5 minor 35 references

What's Missing in Vision-Language Models? Probing Their Struggles with Causal Order Reasoning

T0 review · 5 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Twelve vision-language models, including CLIP, BLIP, LLaVA, and Qwen variants, perform at or near random chance when asked to pick which of two image captions states the correct causal order, despite scoring well on object and activity…

desk verdict Useful new benchmarks and a solid negative result, but the central claim that VLMs lack visual causal reasoning is only as strong as the unverified assumption that VQA/VCR 'why' answers are visually grounded causes. read the letter →

arxiv 2506.00869 v3 pith:XB4HURHN submitted 2025-06-01 cs.CL

Zhaotian Weng , Haoxuan Li , Xin Eric Wang , Kuan-Hao Huang , Jieyu Zhao This is my paper
classification cs.CL
keywords causalreasoningvision-languagemodelsorderbenchmarkvisualquestionansweringhardnegativefine-tuningtrainingdatascarcitycontrastivelearning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that current vision-language models (VLMs) lack a basic form of causal reasoning: telling which event in a scene caused another. To show this, the authors build VQA-Causal and VCR-Causal, two benchmarks that pair each image with two captions identical except for the direction of a causal relation (for example, "the woman's holding an umbrella is caused by the rain" versus "the rain is caused by the woman holding an umbrella"). On this forced-choice task, ten of the twelve evaluated VLMs score no better than 52%, barely above the 50% random baseline and far below an estimated 98% human performance. The authors trace the failure to training data: explicit causal language appears in roughly 0.08% of sampled LAION-400M captions and 0.01% of MSCOCO captions. They then show that fine-tuning a CLIP model on hard-negative caption pairs that differ only in causal order raises accuracy into the high 50s to low 60s on both in-domain and out-of-domain benchmarks while leaving retrieval performance largely intact.

What carries the argument

The load-bearing mechanism is the counterfactual caption-pair: each test instance is one image plus two captions that are word-for-word identical except that the causal relation between two events is reversed, generated from VQA and VCR "why" questions and answers using twelve causal conjunctions (is due to, is caused by, is a result of, is the effect of, is the consequence of, because, owe to, result in, cause, lead to, give rise to, bring about to). Score-based models are evaluated by the matching score between image and caption, while generative models are scored by the probability of "yes" to the reformulated question "Does it reflect the proper causal relationship?". The same pair format doubles as training machinery: fine-tuning extends CLIP's contrastive objective with hard-negative captions that flip causal order, which is what yields CausalCLIP's improvement.

What would settle it

Take a sample of VQA-Causal instances and ask independent annotators to point to both events in the image and say which caused which, with no access to the original answer text; if their agreement on the intended direction is near chance or clearly below the reported human accuracy, the benchmark's gold labels are not visually determined, and near-chance model scores would reflect label ambiguity rather than missing causal reasoning. Conversely, feed models a subset of pairs whose cause and effect are both unambiguously visible and mechanically ordered, such as a hammer striking and a pane shattering; high accuracy there would show VLMs can do visual causal order when the image actually shows it.

Watch

Extended reading notes

Core claim

On the paper's terms, the central discovery is that the gap between low-level visual understanding and causal reasoning is large and systematic: the same models that reach roughly 90% accuracy on a controlled object-and-activity test fall to near chance when the only difference between the correct and incorrect caption is the causal direction between two events. Across twelve models, including score-based contrastive models and instruction-tuned generative models, accuracy on VQA-Causal and VCR-Causal clusters around 50%, with a few models reaching the mid-to-high 50s and Qwen3-VL dropping below chance to about 42-44%. The authors attribute this to a data-level scarcity: causal expressions are vanishingly rare in large image-text corpora, and even the VQA and VCR benchmarks that nominally include causal questions can often be solved by object detection or activity recognition without any genuine causal reasoning. As a remedy, the paper shows that a contrastive model fine-tuned with hard negatives, captions differing only in causal order, improves on both the in-domain VQA-Causal set and the out-of-domain VCR-Causal set, generalizing to causal conjunctions it never saw during fine-tuning.

Load-bearing premise

The load-bearing premise is that each "why" question and its original answer really do identify a visually grounded cause-and-effect direction between two events in the image, so that reversing the order yields a caption that only visual causal reasoning can reject.

Editorial extensions

If this is right

  • Existing benchmark scores that claim strong VLM reasoning should be re-examined, because a large share of causal-looking questions in VQA and VCR can be answered by recognizing objects and activities alone, so high accuracy there does not establish causal reasoning.
  • The failure is at least partly a training-distribution problem: hard-negative contrastive fine-tuning lifts one model from roughly 51% to 61% on VQA-Causal and from 51% to 57% on VCR-Causal, showing that targeted data can move the needle without architectural changes.
  • Reported causal accuracy depends on surface syntax: most models do better when the result precedes the cause (is caused by, is due to) and worse when the cause precedes the result (cause, lead to), so any causal benchmark must control for conjunction direction.
  • Because the fine-tuned model generalizes to causal conjunctions excluded from fine-tuning and to a different source dataset, the learned signal is about causal direction rather than memorized templates.
  • The training-data scarcity numbers imply that simply adding explicit causal language to large image-text corpora could be a cheap, scalable way to improve causal reasoning across many model families.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The benchmark's near-chance results may partly reflect gold-label ambiguity rather than pure causal blindness: many original VQA answers (for example, "to keep from being washed away") describe purposes or external facts that are not visually verifiable from the image, so the correct causal direction is not always grounded in what a model can see; the paper's own human check reports caption fluenc
  • A sharper test would restrict instances to causal pairs whose two events are both visible and mechanically ordered, such as a hand pushing and a cup falling; the paper does not isolate this subset, and doing so would separate "cannot judge visual causality" from "cannot parse this kind of causal sentence".
  • The data-scarcity story suggests a cheap intervention beyond fine-tuning: harvesting or synthesizing captions that explicitly state causal relations between visible events, at scale, may transfer to other fine-grained reasoning skills such as spatial and temporal order reasoning.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper introduces two new benchmarks, VQA-Causal and VCR-Causal, designed to isolate and evaluate vision-language models' (VLMs) ability to reason about causal order in images. The benchmarks are constructed from existing VQA and VCR 'why' questions by converting each question-answer pair into two captions that differ only in the direction of the causal relation. The authors evaluate a diverse set of 12–14 models, reporting that most perform near chance on the causal-order task while achieving high accuracy on a control object/activity understanding test. They further report that explicit causal expressions are rare in common pretraining datasets (LAION-400M, MSCOCO) and that fine-tuning on GPT-4-generated hard-negative causal captions improves performance on both in-domain and out-of-domain benchmarks while largely preserving retrieval performance. The central claim is that current VLMs lack robust visual causal reasoning, and that the benchmarks provide a reliable instrument for measuring this deficiency.

Significance. If the benchmarks are valid, this paper makes a timely and useful contribution: the causal-order task with hard negatives is a clean experimental design that is not easily shortcut by object or activity recognition, the evaluation covers a broad range of open models, and the fine-tuning recipe is potentially actionable. The release of code and data is a strength, as is the explicit control experiment showing strong object/activity recognition in the same models. However, the central validity assumption—that the constructed captions require visual causal reasoning—is not established, and the human baseline and fine-tuning controls are insufficiently documented. The main conclusion is therefore currently uninterpretable.

major comments (5)
  1. [Section 2.1, Figure 2] The benchmark construction assumes that every 'why' answer specifies a visually grounded causal direction that can be determined from the image alone. This assumption is violated by many instances, including the paper's own example in Figure 2: 'To keep from being washed away' is a purpose or external fact, not a visually present cause. For such instances, the 'correct' caption cannot be validated by looking at the image, and the task reduces to judging linguistic plausibility or world knowledge rather than visual causality. Because the headline finding is that VLMs fail at visual causal reasoning, this issue is load-bearing. Please provide evidence that the causal direction is recoverable from the image, for example by reporting a human study with and without image access, or by filtering to a subset of instances where the cause is visually verified.
  2. [Section 5, Appendix A] The fine-tuning data in Section 5 is generated by GPT-4-turbo using a prompt that is nearly identical in structure to the prompt used to construct the benchmarks in Section 2.1. The reported improvement of CausalCLIP may therefore reflect learning the stylistic pattern of GPT-4-generated captions rather than improved visual causal reasoning. A control experiment is needed, for example fine-tuning on hard negatives that are not causally ordered, or evaluating on a benchmark constructed with a different generation method, to rule out format overfitting.
  3. [Section 2.3, Table 1] The 'Human Estimate' of 99.17% and 98.17% is reported in Table 1 and used to conclude that models are 'significantly below human performance,' but no protocol is described: number of annotators, whether images were shown, how they were instructed, and how ties were resolved are all unspecified. The human verification described in Section 2.1 evaluates fluency and reasonableness, not causal correctness, so it does not support this estimate. Please provide the full human evaluation protocol or remove the 'Human Estimate' rows from Tables 1, 5, and 6.
  4. [Section 2.2, Table 1] There is a model-count inconsistency that affects the accuracy of the central empirical claim. Section 2.2 says 'four generation-based models' but lists five (Vicuna, LLaVA1.5, LLaVA1.6, Qwen2.5-VL, Qwen3-VL), and Table 1 contains 14 rows, while the abstract and Section 1 say 12 models and 'ten out of twelve' perform no better than 52% on VQA-Causal. Please reconcile the numbers and ensure the claim references a clearly defined model set (for example, excluding the text-only Vicuna from the '12 VLMs' count).
  5. [Section 4] The data-level explanation in Section 4 relies on the frequency of explicit causal keywords (because, cause, lead to, etc.) and concludes that the scarcity of causal expressions explains the models' failure. However, causal relations are often expressed implicitly, and the absence of explicit causal vocabulary does not establish that the training signal lacks causal information. This is not fatal to the paper, but it is an overreach as currently stated; the analysis should be presented as measuring explicit causal language, with appropriate caveats.
minor comments (5)
  1. [Throughout] There are several typos and minor errors: 'avaiable' in the abstract footnote, 'foundamental' in Section 6, 'focuns' and 'foucs' in the Limitations section, and inconsistent use of 'winth' in the text. A careful proofread is needed.
  2. [Figure 1] Figure 1 is dense and the legend explaining the red/blue/green nodes in the scene graph is not clearly integrated into the caption. Please enlarge the figure and describe the color coding directly in the caption.
  3. [Section 2.2] For text-generation models, the reformulation into yes/no questions ('Does it reflect the proper causal relationship?') may introduce response bias. Please report the exact prompt template, decoding settings (temperature, top-p), and the number of trials averaged over if any.
  4. [Section 4] The sampling details for LAION-400M and MSCOCO are incomplete: it is unclear whether the reported counts are at the caption level or unique-image level, and whether the sampled captions are from the training split actually used by the evaluated models. Please clarify.
  5. [Tables 5 and 6] The 'Human Estimate' rows in the appendix tables report per-conjunction values such as 100.00 and 96.00 that appear to be based on very small samples. Please specify the sample size per conjunction or present the human baseline as an aggregate.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central negative result is an external zero-shot evaluation, and the labeled benchmarks are constructed before and independently of the models tested.

full rationale

The paper's main claim—that twelve VLMs perform near chance on VQA-Causal and VCR-Causal—is a direct empirical measurement on datasets whose ground-truth labels are fixed by original VQA/VCR 'why' answers and GPT-4-turbo generation, not by any evaluated model's parameters or outputs. No fitted parameter is later relabeled as a prediction. The fine-tuning section is explicitly an intervention study: CausalCLIP is trained on hard-negative caption pairs and then evaluated on both an in-domain benchmark and an out-of-domain, zero-shot benchmark (VCR-Causal), which provides an external check against format overfitting. The paper contains no load-bearing self-citations; the evaluation format is adopted from Kamath et al. (2023) and the fine-tuning recipe from Yuksekgonul et al. (2023), both external prior work. The data-scarcity analysis in Section 4 is a correlational observation about pretraining corpora, not a derivation of the benchmark results. A construct-validity concern remains—some 'why' answers may express purposes or world knowledge rather than visually grounded causes, so the benchmark may not perfectly isolate visual causal reasoning—but that is a benchmark validity issue, not a circular reduction of the paper's conclusion to its inputs.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The paper is an empirical benchmark study, not a derivation with fitted constants. Its central claims rest on domain assumptions about the validity of GPT-generated captions and about the interpretability of VQA 'why' answers as visual causal relations. No new theoretical entities or free parameters are introduced.

assumptions (4)
  • domain assumption The original VQA 'why' question and answer encode the correct causal order between two events in the image.
    Introduced in Section 2.1: each VQA 'why' question is transformed into a correct causal caption and its reversal. If the original answer is a purpose, intention, or external fact, the causal direction may not be visually verifiable.
  • domain assumption Reversing the causal order with the 12 chosen conjunctions always yields an incorrect caption that is clearly false for the image.
    Used throughout the benchmark construction (Section 2.1). The paper does not verify each of the 5,458 instances; it only checks fluency and coherence on a 200-pair sample per dataset.
  • domain assumption The keyword list (because, cause, lead to, reason, etc.) captures the presence of explicit causal expressions in training corpora.
    Used in Section 4 to measure causal-expression scarcity in LAION-400M and MSCOCO. Implicit causal relations or paraphrases are not counted, which may understate the true prevalence.
  • domain assumption The human accuracy estimates (99.17% on VQA-Causal, 98.17% on VCR-Causal) are reliable ground-truth baselines.
    Reported in Table 1 and Section 2.3, but the human evaluation protocol is not described; the Appendix only mentions a fluency/coherence check, not an accuracy task.

how reviews work

0 comments
Cite this review

Pith. "Pith review of What's Missing in Vision-Language Models? Probing Their Struggles with Causal Order Reasoning." pith.science (2026). https://pith.science/paper/XB4HURHN

@misc{pith2026250600869,
  author       = {Pith},
  title        = {Pith review of: What's Missing in Vision-Language Models? Probing Their Struggles with Causal Order Reasoning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XB4HURHN}},
  note         = {Machine review of arXiv:2506.00869}
}
read the original abstract

Despite the impressive performance of vision-language models (VLMs) on downstream tasks, their ability to understand and reason about causal relationships in visual inputs remains unclear. Robust causal reasoning is fundamental to solving complex high-level reasoning tasks, yet existing benchmarks often include a mixture of reasoning questions, and VLMs can frequently exploit object recognition and activity identification as shortcuts to arrive at the correct answers, making it challenging to truly assess their causal reasoning abilities. To bridge this gap, we introduce VQA-Causal and VCR-Causal, two new benchmarks specifically designed to isolate and rigorously evaluate VLMs' causal reasoning abilities. Our findings reveal that while VLMs excel in object and activity recognition, they perform poorly on causal reasoning tasks, often only marginally surpassing random guessing. Further analysis suggests that this limitation stems from a severe lack of causal expressions in widely used training datasets, where causal relationships are rarely explicitly conveyed. We additionally explore fine-tuning strategies with hard negative cases, showing that targeted fine-tuning can improve model's causal reasoning while maintaining generalization and downstream performance. Our study highlights a key gap in current VLMs and lays the groundwork for future work on causal understanding.

Figures

Figures reproduced from arXiv: 2506.00869 by the authors.

Figure 1
Figure 1. Examples from the VQA-Causal test and the [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. The VCR dataset fails to genuinely evaluate [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

35 extracted references · 20 canonical work pages

  1. [1]

    Stanislaw Antol, Aishwarya Agrawal, Jiasen Lu, Margaret Mitchell, Dhruv Batra, C Lawrence Zitnick, and Devi Parikh. 2015. Vqa: Visual question answering. In Proceedings of the IEEE international conference on computer vision, pages 2425--2433

    2015

  2. [2]

    Boyuan Chen, Zhuo Xu, Sean Kirmani, Brain Ichter, Dorsa Sadigh, Leonidas Guibas, and Fei Xia. 2024. Spatialvlm: Endowing vision-language models with spatial reasoning capabilities. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14455--14465

    work page 2024

  3. [3]

    An-Chieh Cheng, Hongxu Yin, Yang Fu, Qiushan Guo, Ruihan Yang, Jan Kautz, Xiaolong Wang, and Sifei Liu. 2024. Spatialrgpt: Grounded spatial reasoning in vision-language models. In NeurIPS

    work page 2024

  4. [4]

    Mehdi Cherti, Romain Beaumont, Ross Wightman, Mitchell Wortsman, Gabriel Ilharco, Cade Gordon, Christoph Schuhmann, Ludwig Schmidt, and Jenia Jitsev. 2023. Reproducible scaling laws for contrastive language-image learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2818--2829

    work page 2023

  5. [5]

    Gonzalez, Ion Stoica, and Eric P

    Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. 2023. https://lmsys.org/blog/2023-03-30-vicuna/ Vicuna: An open-source chatbot impressing gpt-4 with 90\

    2023

  6. [6]

    Lisa Anne Hendricks and Aida Nematzadeh. 2021. https://doi.org/10.18653/v1/2021.findings-acl.318 Probing image-language transformers for verb understanding . In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 3635--3644, Online. Association for Computational Linguistics

    work page doi:10.18653/v1/2021.findings-acl.318 2021

  7. [7]

    Drew A Hudson and Christopher D Manning. 2019. Gqa: A new dataset for real-world visual reasoning and compositional question answering. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 6700--6709

    2019

  8. [8]

    Gabriel Ilharco, Mitchell Wortsman, Ross Wightman, Cade Gordon, Nicholas Carlini, Rohan Taori, Achal Dave, Vaishaal Shankar, Hongseok Namkoong, John Miller, Hannaneh Hajishirzi, Ali Farhadi, and Ludwig Schmidt. 2021. https://doi.org/10.5281/zenodo.5143773 Openclip . If you use this software, please cite it as below

    doi:10.5281/zenodo.5143773 2021

Show all 35 references
  1. [9]

    Amita Kamath, Jack Hessel, and Kai-Wei Chang. 2023. What’s “up” with vision-language models? investigating their struggle with spatial reasoning. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 9161--9175

    2023

  2. [10]

    Brenden M Lake, Tomer D Ullman, Joshua B Tenenbaum, and Samuel J Gershman. 2017. Building machines that learn and think like people. Behavioral and brain sciences, 40:e253

    2017

  3. [11]

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. 2023. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In International conference on machine learning, pages 19730--19742. PMLR

    2023

  4. [12]

    Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. 2022 a . Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation. In International conference on machine learning, pages 12888--12900. PMLR

    2022

  5. [13]

    Liunian Harold Li, Mark Yatskar, Da Yin, Cho-Jui Hsieh, and Kai-Wei Chang. 2019. Visualbert: A simple and performant baseline for vision and language. arXiv preprint arXiv:1908.03557

    2019 arXiv

  6. [14]

    Liunian Harold Li, Pengchuan Zhang, Haotian Zhang, Jianwei Yang, Chunyuan Li, Yiwu Zhong, Lijuan Wang, Lu Yuan, Lei Zhang, Jenq-Neng Hwang, and 1 others. 2022 b . Grounded language-image pre-training. In Proceedings of the IEEE/CVF conference on computer vision and pattern rec...

    2022

  7. [15]

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll \'a r, and C Lawrence Zitnick. 2014. Microsoft coco: Common objects in context. In Computer vision--ECCV 2014: 13th European conference, zurich, Switzerland, September 6-12, 2014, ...

    2014

  8. [16]

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023. Visual instruction tuning. Advances in neural information processing systems, 36:34892--34916

    2023

  9. [17]

    Kenneth Marino, Mohammad Rastegari, Ali Farhadi, and Roozbeh Mottaghi. 2019. Ok-vqa: A visual question answering benchmark requiring external knowledge. In Proceedings of the IEEE/cvf conference on computer vision and pattern recognition, pages 3195--3204

    2019

  10. [18]

    Roni Paiss, Ariel Ephrat, Omer Tov, Shiran Zada, Inbar Mosseri, Michal Irani, and Tali Dekel. 2023. Teaching clip to count to ten. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3170--3180

    2023

  11. [19]

    Letitia Parcalabescu, Albert Gatt, Anette Frank, and Iacer Calixto. 2021. https://aclanthology.org/2021.mmsr-1.4/ Seeing past words: Testing the cross-modal capabilities of pretrained V & L models on counting tasks . In Proceedings of the 1st Workshop on Multimodal Semantic Re...

    2021

  12. [20]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, and 1 others. 2021. Learning transferable visual models from natural language supervision. In International conference on machine...

    2021

  13. [21]

    Christian Schlarmann, Naman Deep Singh, Francesco Croce, and Matthias Hein. 2024. Robust clip: Unsupervised adversarial fine-tuning of vision embeddings for robust large vision-language models. In International Conference on Machine Learning, pages 43685--43704. PMLR

    2024

  14. [22]

    Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade W Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, Patrick Schramowski, Srivatsa R Kundurthy, Katherine Crowson, Ludwig Schmidt, Robert Kaczmarczyk, and Jenia Jitsev. 2...

    2022

  15. [23]

    Christoph Schuhmann, Robert Kaczmarczyk, Aran Komatsuzaki, Aarush Katta, Richard Vencu, Romain Beaumont, Jenia Jitsev, Theo Coombes, and Clayton Mullis. 2021. Laion-400m: Open dataset of clip-filtered 400 million image-text pairs. In NeurIPS Workshop Datacentric AI, FZJ-2022-0...

    2021

  16. [24]

    Amanpreet Singh, Ronghang Hu, Vedanuj Goswami, Guillaume Couairon, Wojciech Galuba, Marcus Rohrbach, and Douwe Kiela. 2022. Flava: A foundational language and vision alignment model. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 15...

    2022

  17. [25]

    Yingtian Tang, Yutaro Yamada, Yoyo Zhang, and Ilker Yildirim. 2023. When are lemons purple? the concept association bias of vision-language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 14333--14348

    2023

  18. [26]

    Jiayu Wang, Yifei Ming, Zhenmei Shi, Vibhav Vineet, Xin Wang, Sharon Li, and Neel Joshi. 2024. Is a picture worth a thousand words? delving into spatial reasoning for vision language models. Advances in Neural Information Processing Systems, 37:75392--75421

    2024

  19. [27]

    Zhenhailong Wang, Ansel Blume, Sha Li, Genglin Liu, Jaemin Cho, Zineng Tang, Mohit Bansal, and Heng Ji. 2023. Paxion: Patching action knowledge in video-language foundation models. Advances in Neural Information Processing Systems, 36:20729--20749

    2023

  20. [28]

    Peter Young, Alice Lai, Micah Hodosh, and Julia Hockenmaier. 2014. From image descriptions to visual denotations: New similarity metrics for semantic inference over event descriptions. Transactions of the association for computational linguistics, 2:67--78

    2014

  21. [29]

    M Yuksekgonul, F Bianchi, P Kalluri, D Jurafsky, J Zou, and 1 others. 2023. When and why vision-language models behave like bags-of-words, and what to do about it? In 11th International Conference on Learning Representations, ICLR 2023. International Conference on Learning Rep...

    2023

  22. [30]

    Rowan Zellers, Yonatan Bisk, Ali Farhadi, and Yejin Choi. 2019. From recognition to cognition: Visual commonsense reasoning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 6720--6731

    2019

  23. [31]

    Haotian Zhang, Pengchuan Zhang, Xiaowei Hu, Yen-Chun Chen, Liunian Li, Xiyang Dai, Lijuan Wang, Lu Yuan, Jenq-Neng Hwang, and Jianfeng Gao. 2022. Glipv2: Unifying localization and vision-language understanding. Advances in Neural Information Processing Systems, 35:36067--36080

    2022

  24. [32]

    Tiancheng Zhao, Tianqi Zhang, Mingwei Zhu, Haozhan Shen, Kyusong Lee, Xiaopeng Lu, and Jianwei Yin. 2022. Vl-checklist: Evaluating pre-trained vision-language models with objects, attributes and relations. arXiv preprint arXiv:2207.00221

    2022 arXiv

  25. [33]

    P Xing, Hao Zhang, Joseph E

    Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric. P Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. 2023. https://arxiv.org/abs/2306.05685 Judging llm-as-a-judge with mt-bench and chatbot arena . P...

    2023 arXiv

  26. [34]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  27. [35]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.