Pith. sign in

REVIEW 3 major objections 6 minor 2 cited by

SpaRE: Enhancing Spatial Reasoning in Vision-Language Models with Synthetic Data

T0 review · 3 major / 6 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Synthetic question-answer pairs distilled from hyper-detailed image captions improve spatial reasoning in vision-language models by up to 49 percentage points on the What's Up benchmark, without hurting general task performance.

desk verdict A genuinely useful synthetic spatial-QA dataset, but the headline What's Up gains are suspect until image overlap with training data is ruled out. read the letter →

arxiv 2504.20648 v1 pith:LQB3ZA4E submitted 2025-04-29 cs.CV cs.AI

Michael Ogezi , Freda Shi This is my paper · ORCID
classification cs.CVcs.AI
keywords spatialreasoningvision-languagemodelssyntheticdatavisualquestionansweringlong-tailrelationsimagecaptionsfine-tuningbenchmarks
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Vision-language models can name objects but often cannot say which object is left, right, above, or facing another. The paper's diagnosis is that their training data is spatially impoverished: in the popular instruction-tuning set analyzed, a handful of relations like left, right, and on dominate while most relations form a tiny long tail. The remedy is a synthetic dataset that turns hyper-detailed human-written image descriptions into 3.4 million spatial question-answer pairs using an instruction-tuned language model, then fine-tunes 2B and 7B VLMs on those pairs. The resulting SpaRE models raise spatial benchmark accuracy by up to 49 percentage points on What's Up A, with several-point gains on VSR, 3DSRBench, and RealWorldQA and no meaningful loss on MMMU, MMBench, or TextVQA. If those gains are genuine rather than memorization of overlapping benchmark images, this is a cheap and scalable recipe for teaching space to VLMs.

What carries the argument

The load-bearing mechanism is the caption-to-QA distillation pipeline. Hyper-detailed image captions, typically 100–200 words of human-written description, are first filtered for explicit spatial content, then fed to an instruction-tuned LLM that must return a JSON list of questions and answers restricted to spatial relations such as positions, orientations, distances, layers, boundaries, shadows, overlap, and scale. A cascade of automated checks then removes duplicate questions, references to the description, answers not grounded in the caption, QA pairs whose image-question semantic alignment score is low, and pairs that are not spatial; the surviving 455k samples with 3.4M QA pairs become the training set for SpaRE. The pipeline matters because it converts a relatively small amount of human spatial knowledge, already present in captions, into a large and diverse training signal without rendering synthetic scenes or collecting new annotations.

What would settle it

Take the test images from VSR and What's Up and split them into those with exact or near-duplicate images in Localized Narratives, DOCCI, or PixMo-Cap and those with no overlap; if SpaRE's advantage over the base model shrinks sharply or disappears on the non-overlapping subset, the claim that the model learned general spatial reasoning is falsified. A second check is to build a held-out spatial benchmark with unseen objects and layouts and measure whether the gain transfers.

Watch

Extended reading notes

Core claim

The paper's central claim is that the spatial reasoning deficit of VLMs is largely a data-distribution problem, not an architectural one, and that it can be fixed by distilling spatial questions from captions that already describe real images in detail. It first quantifies the scarcity: in the supervised fine-tuning set of a leading open VLM family, the top 17% of spatial-relation types account for over 90% of relation occurrences, leaving rare relations such as facing, opposite, and between almost unseen. It then builds a synthetic dataset of 455k samples and 3.4M QA pairs by prompting an LLM to extract position, orientation, distance, foreground/background, overlap, and scale questions from captions in Localized Narratives, DOCCI, and PixMo-Cap, followed by automated filtering for duplicates, ungrounded answers, non-spatial questions, and image-question mismatch. Fine-tuning Qwen2-VL-2B and Qwen2-VL-7B on this data yields SpaRE models whose average spatial accuracy rises by roughly 9 and 3 points respectively; on What's Up A the 2B model moves from 44.6 to 93.4, and general benchmarks stay stable. The authors interpret this as evidence that synthetic caption-derived QA data narrows the gap between human and VLM spatial reasoning.

Load-bearing premise

The load-bearing premise is that the evaluation benchmarks do not overlap with the training images, so the reported gains reflect learned spatial reasoning instead of the model recognizing pictures it has already seen; the paper does not report an overlap analysis or deduplication between its training sources, which include COCO, Flickr30k, ADE20K, and Open Images, and benchmarks such as VSR and What's Up that use COCO-derived images.

Editorial extensions

If this is right

  • Adding more examples of rare spatial relations is likely more valuable than adding more examples of common ones, since the paper shows the distribution is a long tail.
  • Real-world spatial tasks can be improved without architectural changes: the 2B SpaRE model gains 4.9 points on RealWorldQA.
  • Spatial fine-tuning can be layered onto existing general-purpose VLMs without sacrificing MMMU, MMBench, or TextVQA performance, making it a safe add-on.
  • The same caption-to-QA recipe can absorb any future hyper-detailed caption dataset, so the approach scales with data rather than with annotation effort.
  • The technique does not solve frame-of-reference ambiguity: 3DSRBench gains are smaller, and the paper's own error analysis shows egocentric perspectives remain hard.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: because the training captions come from sources that include COCO, Flickr30k, ADE20K, and Open Images, and two of the evaluation benchmarks use COCO-derived images, part of the headline gain could be image-level memorization; a held-out-image split would separate genuine spatial generalization from familiarity.
  • Beyond the paper: the pipeline only generates positive assertions about observed relations; adding negative QA pairs that ask whether a wrong relation holds would likely sharpen the contrastive signal and could be tested directly.
  • Beyond the paper: the same caption-to-QA template could generate other long-tail reasoning skills, such as temporal order or part-whole relations, simply by changing the prompt taxonomy, since the bottleneck identified is data distribution rather than architecture.
  • Beyond the paper: a controlled test could scramble the spatial phrases in the captions before QA generation; if the gains vanish, the active ingredient is the linguistic description rather than the visual content of the images.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper introduces SpaRE, a synthetic spatial-reasoning VQA dataset built by prompting Qwen2.5-3B to extract QA pairs from hyper-detailed image descriptions in Localized Narratives, DOCCI, and PixMo-Cap. The authors fine-tune Qwen2-VL-2B and Qwen2-VL-7B on 455k samples containing 3.4M QA pairs and report large improvements on spatial reasoning benchmarks, including an increase from 44.6 to 93.4 on What's Up A for the 2B model, while general benchmarks remain stable. The central claim is that caption-derived synthetic QA data transfers spatial reasoning to unseen settings without harming general VL performance.

Significance. If the results hold, this is a valuable contribution: it suggests that a simple caption-to-QA pipeline can address spatial-reasoning data scarcity at scale. The paper is strong in empirical scope, including multiple seeds, a range of spatial and general benchmarks, human evaluation of data quality, and ablations of the generation model. The headline improvement on COCO-derived benchmarks, however, is threatened by the lack of any overlap analysis with the training sources, which include COCO images via Localized Narratives. The paper's machine-checked experiments and reproducible recipes (open weights, open data planned) are assets, but the contamination risk must be resolved before the claim is fully supported.

major comments (3)
  1. [Section 3.1.1, 4.2.1, Table 3] Localized Narratives is built from COCO, Flickr30k, ADE20K, and Open Images images, and the VSR and What's Up benchmarks evaluate on COCO-derived images. The paper reports no overlap analysis between the 455k training images and these evaluation images, and the deduplication in Section 3.3 only removes duplicate QA pairs within a sample, not duplicate images across the train/test boundary. Because the QA pairs are synthesized from captions that state spatial relations for a given image, any COCO image appearing in both training and evaluation allows the model to exploit image-level cues. The abstract's headline claim of a 49% gain on What's Up A therefore requires either a demonstrated absence of overlap or a re-evaluation on a disjoint image set.
  2. [Section 4.3, Appendix F] The GPT-4o and GPT-4o-mini rows in Table 3 are computed on 100-example subsamples of VSR, What's Up A, and What's Up B, while the SpaRE models and other baselines are evaluated on the full benchmarks. The main text does not disclose this subsampling, making the comparison rows appear to be on equal footing. The paper should either run these models on the full benchmarks or clearly mark the subsample-based rows in the table and state the associated sampling error.
  3. [Section 3.1.2, Table 2] The analysis that motivates the paper is framed as showing that spatial relations are generally rare in VQA datasets, but Table 2 reports Objects365-YorN as 94.35% spatial-related samples, and lists VSR as one of the 'VQA datasets in the supervised fine-tuning set used by InternVL2.' VSR is described in Section 4.2.1 as an evaluation benchmark, not part of an SFT training set. This inconsistency makes it hard to interpret the scarcity claim; the authors should clarify what Objects365-YorN is, why it has such a high ratio, and whether VSR's inclusion in this table is a labeling error.
minor comments (6)
  1. [Abstract] The phrase '49% performance gain' is ambiguous; Table 3 shows a 48.8-point absolute increase on What's Up A for the 2B model (44.6 to 93.4), not a 49% relative gain. Please specify whether the reported number is in percentage points or relative percent.
  2. [Section 4.5.1] The text says the average accuracy of the 2B and 7B models 'increase by around 9% and 3%.' From Table 3, the 2B average increases by 17.6 points (59.8 to 77.4) and the 7B average by 3.1 points (79.2 to 82.3), so the stated percentages appear inconsistent with the table.
  3. [Section 3.3] The CLIP deduplication cutoff of 0.95 and the CLIPScore image-question alignment cutoff of 0.25 are chosen based on manual inspection of only 25 and 100 samples, respectively; a small sensitivity analysis would help establish that the results are not sensitive to these thresholds.
  4. [Section 3.2.2] The phrase 'the show image' should be 'the shown image'.
  5. [Section 3.3 item 1] The sentence 'we employ full-string matching on the questions. and CLIP semantic similarity' contains an errant period after 'questions.'
  6. [Table 1] The dataset name 'Pixmo-Cap' is spelled inconsistently; elsewhere it is 'PixMo-Cap'.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the contributions are empirical and are evaluated against external benchmarks; no predicted quantity reduces by construction to a fitted input.

full rationale

SpaRE is an empirical data-generation and fine-tuning paper; its claims are benchmark accuracies, not quantities derived from equations whose inputs already contain the outputs. The synthetic QA pairs are generated by an LLM from hyper-detailed captions, and the models are then evaluated on external spatial-reasoning benchmarks (VSR, What's Up, 3DSRBench, RealWorldQA) and general VL benchmarks (MMMU, MMBench, etc.); none of these evaluation scores is defined in terms of the training objective or the generating LLM. The self-citation to Zhang et al. (2024), which shares an author, is used only to explain weaker 3DSRBench performance and is not load-bearing for the main claim. The paper does not report image-level de-duplication against VSR/What's Up, and Localized Narratives does draw on COCO and other image sources; this is a potential data-contamination concern for the headline gain, not a circularity, because the paper's own text does not establish that the benchmark images coincide with training images, and the transfer evidence on 3DSRBench and RealWorldQA is independent. No mathematical derivation or fitted parameter is renamed as a prediction, and no uniqueness theorem is imported from the authors' prior work. Therefore the circularity score is 0.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim rests on caption fidelity, LLM-filtering reliability, and an unexamined benchmark-overlap assumption. No new physical or theoretical entities are introduced.

free parameters (3)
  • CLIP semantic similarity deduplication cutoff = 0.95
    Selected by manually testing a sample of QA pairs from 25 sample images (Section 3.3). This directly affects how many generated QA pairs survive.
  • CLIPScore image-question alignment cutoff = 0.25
    Selected by manually testing a sample of 100 QA pairs (Section 3.3). Determines whether a question is kept for a given image.
  • LLM chosen for QA generation and filtering = Qwen2.5-3B-Instruct
    Ablation on 100 samples in Appendix B; the choice of 3B over 1.5B/7B affects dataset size (17 QA pairs per sample) and quality, but downstream benchmark impact is not ablated.
assumptions (4)
  • domain assumption Hyper-detailed captions faithfully describe the spatial layout of the corresponding images.
    The entire QA supervision signal is derived from captions; if a caption is inaccurate or omits spatial layout, the generated answer is wrong for the image (Sections 3.1, 3.2).
  • domain assumption QA pairs generated from captions are answerable from the image alone during training, so the model learns visual grounding rather than text-based shortcuts.
    Training presents image+question and expects the caption-derived answer; the paper provides no verification that answers are visually grounded (Section 3.7).
  • ad hoc to paper Benchmark images (VSR, What's Up) do not overlap with the images used for training data.
    No deduplication or overlap analysis is reported despite Localized Narratives covering COCO, the likely source of VSR and What's Up images (Sections 3.1.1, 4.2.1).
  • domain assumption The Qwen2.5-3B model's spatial-relation classification (used for filtering and analysis) is sufficiently accurate.
    Filtering and dataset stats rely on LLM judgments; ablations use only 100 samples and report unusual precision values (Appendix B, Table 7).

how reviews work

0 comments
Cite this review

Pith. "Pith review of SpaRE: Enhancing Spatial Reasoning in Vision-Language Models with Synthetic Data." pith.science (2026). https://pith.science/paper/LQB3ZA4E

@misc{pith2026250420648,
  author       = {Pith},
  title        = {Pith review of: SpaRE: Enhancing Spatial Reasoning in Vision-Language Models with Synthetic Data},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LQB3ZA4E}},
  note         = {Machine review of arXiv:2504.20648}
}
read the original abstract

Vision-language models (VLMs) work well in tasks ranging from image captioning to visual question answering (VQA), yet they struggle with spatial reasoning, a key skill for understanding our physical world that humans excel at. We find that spatial relations are generally rare in widely used VL datasets, with only a few being well represented, while most form a long tail of underrepresented relations. This gap leaves VLMs ill-equipped to handle diverse spatial relationships. To bridge it, we construct a synthetic VQA dataset focused on spatial reasoning generated from hyper-detailed image descriptions in Localized Narratives, DOCCI, and PixMo-Cap. Our dataset consists of 455k samples containing 3.4 million QA pairs. Trained on this dataset, our Spatial-Reasoning Enhanced (SpaRE) VLMs show strong improvements on spatial reasoning benchmarks, achieving up to a 49% performance gain on the What's Up benchmark, while maintaining strong results on general tasks. Our work narrows the gap between human and VLM spatial reasoning and makes VLMs more capable in real-world tasks such as robotics and navigation.

Figures

Figures reproduced from arXiv: 2504.20648 by the authors.

Figure 1
Figure 1. Our synthetic data generation pipeline: Hyper [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Comparison of answers provided by different [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Ambiguity of spatial relations without an [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: For our qualitative analysis, each sub-figure contains an image, a corresponding question, different models’ [PITH_FULL_IMAGE:figures/full_fig_p015_4.png]
Figure 5
Figure 5. Figure 5: An example from DOCCI, one of the hyper-detailed image-captioning datasets that we extract QA pairs [PITH_FULL_IMAGE:figures/full_fig_p016_5.png]
Figure 6
Figure 6. Figure 6: An example from Localized Narratives, one of the hyper-detailed image-captioning datasets that we [PITH_FULL_IMAGE:figures/full_fig_p017_6.png]
Figure 7
Figure 7. Figure 7: An example from PixMo-Cap, one of the hyper-detailed image-captioning datasets that we extract QA [PITH_FULL_IMAGE:figures/full_fig_p017_7.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. The N-Body Problem: Parallel Execution from Single-Person Egocentric Video

    cs.CV 2025-12 conditional novelty 6.0 of 10

    Structured prompting for a VLM, fed with ground-truth spatial zone schedules, can generate more feasible multi-agent parallel executions from single-person egocentric videos.

  2. SpatialThinker: Reinforcing Scene Graph-Grounded Spatial Reasoning via Dense Rewards

    cs.CV 2025-11 conditional novelty 6.0 of 10

    Dense scene-graph-grounded rewards let a 7B multimodal LLM trained on 7K synthetic questions beat SFT and sparse-RL baselines and outscore GPT-4o on average across 12 spatial/real-world benchmarks.

Reference graph

Works this paper leans on

49 extracted references · 14 canonical work pages · cited by 2 Pith papers

  1. [1]

    Marah Abdin, Sam Ade Jacobs, Ammar Ahmad Awan, Jyoti Aneja, Ahmed Awadallah, Hany Awadalla, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Harkirat Behl, et al. 2024. Phi-3 technical report: A highly capable language model locally on your phone. arXiv preprint arXiv:2404.14219

    arXiv 2024

  2. [2]

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774

    arXiv 2023

  3. [3]

    Palaash Agrawal, Haidi Azaman, and Cheston Tan. 2023. Stupd: A synthetic dataset for spatial and temporal relation reasoning. arXiv preprint arXiv:2309.06680

    work page Pith review arXiv 2023

  4. [4]

    Balakrishnan, M.Syed Shahul Hameed, Kavya Venkatesan, and G Aswin

    S. Balakrishnan, M.Syed Shahul Hameed, Kavya Venkatesan, and G Aswin. 2021. https://api.semanticscholar.org/CorpusID:235340798 Interaction of spatial computing in augmented reality . 2021 7th International Conference on Advanced Computing and Communication Systems (ICACCS), 1:1900--1904

    work page 2021

  5. [5]

    James Betker, Gabriel Goh, Li Jing, † TimBrooks, Jianfeng Wang, Linjie Li, † LongOuyang, † JuntangZhuang, † JoyceLee, † YufeiGuo, † WesamManassra, † PrafullaDhariwal, † CaseyChu, † YunxinJiao, and Aditya Ramesh. 2023. https://api.semanticscholar.org/CorpusID:264403242 Improving image generation with better captions

    work page 2023

  6. [6]

    Boyuan Chen, Zhuo Xu, Sean Kirmani, Brain Ichter, Dorsa Sadigh, Leonidas Guibas, and Fei Xia. 2024 a . Spatialvlm: Endowing vision-language models with spatial reasoning capabilities. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14455--14465

    2024

  7. [7]

    Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Conghui He, Jiaqi Wang, Feng Zhao, and Dahua Lin. 2025. Sharegpt4v: Improving large multi-modal models with better captions. In European Conference on Computer Vision, pages 370--387. Springer

    2025

  8. [8]

    Zhe Chen, Weiyun Wang, Hao Tian, Shenglong Ye, Zhangwei Gao, Erfei Cui, Wenwen Tong, Kongzhi Hu, Jiapeng Luo, Zheng Ma, et al. 2024 b . How far are we to gpt-4v? closing the gap to commercial multimodal models with open-source suites. Science China Information Sciences, 67(12):220101

    2024

Show all 49 references
  1. [9]

    Jaemin Cho, Abhaysinh Zala, and Mohit Bansal. 2022. https://api.semanticscholar.org/CorpusID:253510037 Dall-eval: Probing the reasoning skills and social biases of text-to-image generation models . 2023 IEEE/CVF International Conference on Computer Vision (ICCV), pages 3020--3031

    2022

  2. [10]

    Smith, Hannaneh Hajishirzi, Ross Girshick, Ali Farhadi, and Aniruddha Kembhavi

    Matt Deitke, Christopher Clark, Sangho Lee, Rohun Tripathi, Yue Yang, Jae Sung Park, Mohammadreza Salehi, Niklas Muennighoff, Kyle Lo, Luca Soldaini, Jiasen Lu, Taira Anderson, Erin Bransom, Kiana Ehsani, Huong Ngo, YenSung Chen, Ajay Patel, Mark Yatskar, Chris Callison-Burch,...

    2024 arXiv

  3. [11]

    Haodong Duan, Junming Yang, Yuxuan Qiao, Xinyu Fang, Lin Chen, Yuan Liu, Xiaoyi Dong, Yuhang Zang, Pan Zhang, Jiaqi Wang, et al. 2024. Vlmevalkit: An open-source toolkit for evaluating large multi-modality models. In Proceedings of the 32nd ACM international conference on mult...

    2024

  4. [12]

    Tejas Gokhale, Hamid Palangi, Besmira Nushi, Vibhav Vineet, Eric Horvitz, Ece Kamar, Chitta Baral, and Yezhou Yang. 2023. https://arxiv.org/abs/2212.10015 Benchmarking spatial relationships in text-to-image generation . Preprint, arXiv:2212.10015

    2023 arXiv

  5. [13]

    Yash Goyal, Tejas Khot, Douglas Summers-Stay, Dhruv Batra, and Devi Parikh. 2017. Making the v in vqa matter: Elevating the role of image understanding in visual question answering. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 6904--6913

    2017

  6. [14]

    Tianrui Guan, Fuxiao Liu, Xiyang Wu, Ruiqi Xian, Zongxia Li, Xiaoyu Liu, Xijun Wang, Lichang Chen, Furong Huang, Yaser Yacoob, Dinesh Manocha, and Tianyi Zhou. 2024. Hallusionbench: An advanced diagnostic suite for entangled language hallucination and visual illusion in large ...

    2024

  7. [15]

    Suriya Gunasekar, Yi Zhang, Jyoti Aneja, Caio C \'e sar Teodoro Mendes, Allie Del Giorno, Sivakanth Gopi, Mojan Javaheripi, Piero Kauffmann, Gustavo de Rosa, Olli Saarikivi, et al. 2023. Textbooks are all you need. arXiv preprint arXiv:2306.11644

    2023 arXiv

  8. [16]

    Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. 2021. Clipscore: A reference-free evaluation metric for image captioning. arXiv preprint arXiv:2104.08718

    2021 arXiv

  9. [17]

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685

    2021 arXiv

  10. [18]

    Hudson and Christopher D

    Drew A. Hudson and Christopher D. Manning. 2019. Gqa: A new dataset for real-world visual reasoning and compositional question answering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    2019

  11. [19]

    Justin Johnson, Bharath Hariharan, Laurens Van Der Maaten, Li Fei-Fei, C Lawrence Zitnick, and Ross Girshick. 2017. Clevr: A diagnostic dataset for compositional language and elementary visual reasoning. In Proceedings of the IEEE conference on computer vision and pattern reco...

    2017

  12. [20]

    Amita Kamath, Jack Hessel, and Kai-Wei Chang. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.568 What ' s '' up '' with vision-language models? investigating their struggle with spatial reasoning . In Proceedings of the 2023 Conference on Empirical Methods in Natural Langua...

    2023 doi

  13. [21]

    Douwe Kiela, Hamed Firooz, Aravind Mohan, Vedanuj Goswami, Amanpreet Singh, Pratik Ringshia, and Davide Testuggine. 2020. The hateful memes challenge: Detecting hate speech in multimodal memes. Advances in neural information processing systems, 33:2611--2624

    2020

  14. [22]

    Alina Kuznetsova, Hassan Rom, Neil Alldrin, Jasper Uijlings, Ivan Krasin, Jordi Pont-Tuset, Shahab Kamali, Stefan Popov, Matteo Malloci, Alexander Kolesnikov, et al. 2020. The open images dataset v4: Unified image classification, object detection, and visual relationship detec...

    2020

  15. [23]

    Walter, and D

    Christian Landsiedel, Verena Rieser, Matthew R. Walter, and D. Wollherr. 2017. https://doi.org/10.1080/01691864.2016.1277554 A review of spatial reasoning and interaction for real-world robotics . Advanced Robotics, 31:222 -- 242

    2017

  16. [24]

    Levinson

    Stephen C. Levinson. 2003. Space in Language and Cognition: Explorations in Cognitive Diversity. Language Culture and Cognition. Cambridge University Press

    2003

  17. [25]

    Yuanzhi Li, S \'e bastien Bubeck, Ronen Eldan, Allie Del Giorno, Suriya Gunasekar, and Yin Tat Lee. 2023. Textbooks are all you need ii: phi-1.5 technical report. arXiv preprint arXiv:2309.05463

    2023 arXiv

  18. [26]

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll \'a r, and C Lawrence Zitnick. 2014. Microsoft coco: Common objects in context. In Computer Vision--ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, ...

    2014

  19. [27]

    Fangyu Liu, Guy Emerson, and Nigel Collier. 2023 a . https://doi.org/10.1162/tacl_a_00566 Visual spatial reasoning . Transactions of the Association for Computational Linguistics, 11:635--651

    2023 doi

  20. [28]

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023 b . https://proceedings.neurips.cc/paper_files/paper/2023/file/6dcf277ea32ce3288914faf369fe6de0-Paper-Conference.pdf Visual instruction tuning . In Advances in Neural Information Processing Systems, volume 36, pages...

    2023

  21. [29]

    Yangzhou Liu, Yue Cao, Zhangwei Gao, Weiyun Wang, Zhe Chen, Wenhai Wang, Hao Tian, Lewei Lu, Xizhou Zhu, Tong Lu, et al. 2024 a . Mminstruct: A high-quality multi-modal instruction tuning dataset with extensive diversity. Science China Information Sciences, 67(12):1--16

    2024

  22. [30]

    Yuan Liu, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Wangbo Zhao, Yike Yuan, Jiaqi Wang, Conghui He, Ziwei Liu, et al. 2024 b . Mmbench: Is your multi-modal model an all-around player? In European conference on computer vision, pages 216--233. Springer

    2024

  23. [31]

    Wufei Ma, Haoyu Chen, Guofeng Zhang, Celso M de Melo, Alan Yuille, and Jieneng Chen. 2024. 3dsrbench: A comprehensive 3d spatial reasoning benchmark. arXiv preprint arXiv:2412.07825

    2024

  24. [32]

    Kenneth Marino, Mohammad Rastegari, Ali Farhadi, and Roozbeh Mottaghi. 2019. Ok-vqa: A visual question answering benchmark requiring external knowledge. In Proceedings of the IEEE/cvf conference on computer vision and pattern recognition, pages 3195--3204

    2019

  25. [33]

    Newcombe, Janellen, Huttenlocher, I

    Nora S. Newcombe, Janellen, Huttenlocher, I. Campari, Nora S. Janellen Huttenlocher, and Janellen Huttenlocher. 2000. https://api.semanticscholar.org/CorpusID:128513118 Making space: The development of spatial representation and reasoning

    2000

  26. [34]

    Yasumasa Onoe, Sunayana Rane, Zachary Berger, Yonatan Bitton, Jaemin Cho, Roopal Garg, Alexander Ku, Zarana Parekh, Jordi Pont-Tuset, Garrett Tanzer, Su Wang, and Jason Baldridge. 2024. DOCCI: Descriptions of Connected and Contrasting Images . In ECCV

    2024

  27. [35]

    Jordi Pont-Tuset, Jasper Uijlings, Soravit Changpinyo, Radu Soricut, and Vittorio Ferrari. 2020. Connecting vision and language with localized narratives. In ECCV

    2020

  28. [36]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. In International conference on machine learni...

    2021

  29. [37]

    Viresh Ranjan, Udbhav Sharma, Thu Nguyen, and Minh Hoai. 2021. Learning to count everything. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3394--3403

    2021

  30. [38]

    Shuai Shao, Zeming Li, Tianyuan Zhang, Chao Peng, Gang Yu, Xiangyu Zhang, Jing Li, and Jian Sun. 2019. Objects365: A large-scale, high-quality dataset for object detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)

    2019

  31. [39]

    Oleksii Sidorov, Ronghang Hu, Marcus Rohrbach, and Amanpreet Singh. 2020. Textcaps: a dataset for image captioning with reading comprehension. In Computer Vision--ECCV 2020: 16th European Conference, Glasgow, UK, August 23--28, 2020, Proceedings, Part II 16, pages 742--758. Springer

    2020

  32. [40]

    Amanpreet Singh, Vivek Natarajan, Meet Shah, Yu Jiang, Xinlei Chen, Dhruv Batra, Devi Parikh, and Marcus Rohrbach. 2019. Towards vqa models that can read. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    2019

  33. [41]

    Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, et al. 2024. Qwen2-vl: Enhancing vision-language model's perception of the world at any resolution. arXiv preprint arXiv:2409.12191

    2024 arXiv

  34. [42]

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. 2024. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115

    2024 arXiv

  35. [43]

    Shukang Yin, Chaoyou Fu, Sirui Zhao, Ke Li, Xing Sun, Tong Xu, and Enhong Chen. 2023. A survey on multimodal large language models. arXiv preprint arXiv:2306.13549

    2023 arXiv

  36. [44]

    Peter Young, Alice Lai, Micah Hodosh, and Julia Hockenmaier. 2014. https://doi.org/10.1162/tacl_a_00166 From image descriptions to visual denotations: New similarity metrics for semantic inference over event descriptions . Transactions of the Association for Computational Ling...

    2014 doi

  37. [45]

    Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, et al. 2024. Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi. In Proceedings of the IEEE/CVF Conferenc...

    2024

  38. [46]

    Zheyuan Zhang, Fengyuan Hu, Jayjun Lee, Freda Shi, Parisa Kordjamshidi, Joyce Chai, and Ziqiao Ma. 2024. https://arxiv.org/abs/2410.17385 Do vision-language models represent space and how? evaluating spatial frame of reference under ambiguities . Preprint, arXiv:2410.17385

    2024 arXiv

  39. [47]

    Bolei Zhou, Hang Zhao, Xavier Puig, Tete Xiao, Sanja Fidler, Adela Barriuso, and Antonio Torralba. 2019. Semantic understanding of scenes through the ade20k dataset. International Journal of Computer Vision, 127:302--321

    2019

  40. [48]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  41. [49]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.