Pith. sign in

REVIEW 4 major objections 6 minor 4 cited by

Perspective-Aware Reasoning in Vision-Language Models via Mental Imagery Simulation

T0 review · 4 major / 6 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Allocentric spatial questions can be converted into egocentric ones with a coarse 3D scene abstraction, lifting a vision-language model's left/right accuracy from near chance to 89.7%.

desk verdict A clever abstraction-based perspective-taking system for VLMs with large reported gains, but the missing oracle-abstraction ablation leaves the central claim underdetermined. read the letter →

arxiv 2504.17207 v1 pith:U774CFTY submitted 2025-04-24 cs.CV

Phillip Y. Lee , Jihyeon Je , Chanho Park , Mikaela Angelina Uy , Leonidas Guibas , Minhyuk Sung This is my paper · ORCID
classification cs.CV
keywords vision-languagemodelsspatialreasoningperspective-takingallocentric3Dsceneabstractionmentalimageryvisualpromptingnovelviewsynthesis
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper is trying to establish that vision-language models can solve perspective-aware spatial reasoning—left/right, closer, visibility, and facing judgments from another object's viewpoint—without being taught perspective-taking at all. Its strategy is to build a sparse 3D abstraction of the objects named in the question, using off-the-shelf detection, segmentation, depth, and orientation models, then rotate that abstraction into the reference object's egocentric coordinate frame and feed it back to the VLM as a prompt. On the synthetic COMFORT++ benchmark and the real-image 3DSRBench benchmark, the visual-prompt version reaches 89.67% and 72.78% respectively on left/right tasks, where the same VLMs alone hover at or below chance. If correct, this reframes allocentric reasoning as a scene-abstraction problem rather than a missing cognitive skill in the model.

What carries the argument

The load-bearing object is a scene abstraction $\mathcal{S}_E = \{ (t_i, c_i, p_i) \}$, in which each object of interest has a textual name $t_i$, a 3D position $c_i$ taken as the median coordinate of its unprojected, depth-filtered segmentation pixels, and a unit orientation vector $p_i$ giving its frontal facing direction. The perspective change is a rigid coordinate transformation into the reference viewer's frame—origin at the viewer, facing direction along the $z$-axis—and the transformed abstraction is delivered to the VLM in two alternative forms: a numerical prompt listing the coordinates, or a visual prompt in which equal-sized colored cubes are rendered from the reference viewpoint. This machinery turns an allocentric problem into the egocentric form that VLMs can already handle, and it is what lets the framework avoid synthesizing photorealistic novel views.

What would settle it

Hand-correct every object orientation and centroid on a subset of 3DSRBench and rerun APC's visual prompt; if visibility accuracy does not rise clearly above the reported 67.44%, the paper's stated cause of the real-image gap would be wrong and the bottleneck would lie elsewhere in the pipeline.

Watch

Extended reading notes

Core claim

APC's central claim is that an allocentric question—such as "from the man's perspective, is the dog on the left or right?"—can be answered accurately by an unmodified VLM once the scene has been reduced to an abstracted set of 3D object locations and facing directions and re-expressed from the reference viewpoint. The framework has the VLM name the objects of interest, recovers each object's position as the median of unprojected depths inside its segmentation mask, estimates each object's frontal orientation, applies a rigid coordinate transform so the reference object sits at the origin facing along the positive z-axis, and then hands the VLM either a coordinate list or a rendered image of colored cubes. Across both benchmarks the visual prompt consistently outperforms every pure VLM, every spatially fine-tuned model tested, and both dense-reconstruction baselines, and the perspective-awareness probe shows the gain is not confined to small angular offsets from the camera: APC stays accurate even when the reference viewer faces the camera.

Load-bearing premise

The framework's accuracy rests on the assumption that a coarse abstraction—median 3D positions from monocular depth and estimated front-facing orientations—faithfully represents the scene from the reference viewer's perspective, and the paper itself attributes its real-image visibility shortfall to noise in exactly those estimates.

Editorial extensions

If this is right

  • A prompt-level transformation, not fine-tuning or architectural changes, is enough to remove the egocentric bias that makes VLMs fail at allocentric questions.
  • The visual prompt beats the numerical prompt on visibility and facing tasks by a wide margin, because the abstract rendering bypasses the logical errors VLMs make with raw coordinates.
  • The approach transfers from synthetic scenes to real images, although real-image visibility accuracy drops to 67.44% because of noise in detected orientations and centroids.
  • Accuracy stays high across the full range of camera-to-reference angular offsets, including near 180 degrees, where baseline VLMs collapse to chance.
  • Dense reconstruction and novel-view synthesis are not needed for perspective change; a sparse abstraction delivers higher accuracy at a fraction of the inference time.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not test whether correcting its off-the-shelf orientation and depth estimates with ground truth would close the real-image gap; a natural follow-up is to rerun APC on 3DSRBench visibility with hand-corrected centroids and orientations.
  • Because all cubes are rendered the same size, the visual prompt's closer and facing judgments rely on the prompt-side convention that a larger rendered cube is closer; this convention deserves a direct ablation to see how much of the visual-prompt advantage it carries.
  • A testable extension is to apply the same allocentric-to-egocentric conversion in embodied settings, asking questions from a robot's gripper or a future waypoint rather than from a named object.
  • If the central claim is right, any future improvement in egocentric spatial reasoning—depth perception, grounding, or reasoning—should transfer automatically to allocentric questions through this conversion, making scene fidelity the real bottleneck.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes APC, a training-free framework that converts allocentric perspective questions into egocentric ones by first constructing a coarse 3D abstraction of the scene using off-the-shelf modules (GroundingDINO, SAM, DepthPro, OrientAnything) and then prompting a VLM either with numerical 3D coordinates or with a rendered abstract visual prompt from the reference viewpoint. Evaluations on COMFORT++ (including newly constructed visibility and facing tasks) and on 3DSRBench show large accuracy gains over pure VLMs, grounded spatial-reasoning models, and dense-reconstruction baselines; for example, APC-Vis reaches 89.67% on COMFORT++ left/right versus 55.33% for the best pure VLM, and 72.78% on 3DSRBench left/right versus 50.10% for SpatialPIN*. The paper includes failure analyses of dense reconstruction baselines, an inference-time comparison, and an extensive appendix with prompts and implementation details.

Significance. If the reported results are reproducible, APC is a practical and inexpensive approach to improving allocentric spatial reasoning in VLMs without fine-tuning or dense 3D reconstruction. The paper makes several solid engineering contributions: a VLM-based detection refinement stage, depth outlier filtering, two complementary prompt representations, and a COMFORT++ extension with new task types. The method is motivated by a clear diagnosis of egocentric bias in VLMs, and the comparison with dense reconstruction baselines is informative. The main scientific claim, however, depends on separating the quality of the scene abstraction from the VLM's reasoning ability; the current experimental design does not yet provide that separation, which limits the strength of the attribution.

major comments (4)
  1. [Sec. 4.2–4.3, Table 1] The central claim—that converting allocentric tasks into egocentric ones over a coarse abstraction improves VLM reasoning—is underdetermined without an oracle-abstraction ablation. On COMFORT++ the ground-truth 3D positions and orientations are available by construction (Sec. D), so the authors should run APC with oracle positions and orientations and report the resulting accuracies, alongside variants with corrupted or ablated abstraction inputs. This would partition the total error into abstraction noise versus VLM reasoning capability. The need for this ablation is reinforced by the authors' own statement in Sec. 4.3 attributing the lower 3DSRBench visibility accuracy to 'noise in the scene abstraction phase, particularly errors in detected orientations or centroids.'
  2. [Sec. 4.2, Table 1] All reported accuracies are single-run point estimates without error bars, confidence intervals, or significance tests. Since many COMFORT++ tasks contain only 300–320 items and VLM decoding is stochastic (especially with CircularEval over answer permutations), the reader cannot assess whether the large gaps are stable. I request repeated runs or bootstrap confidence intervals, plus pairwise significance tests for the key APC-versus-baseline comparisons.
  3. [Sec. 4.1, Table 1] APC is evaluated with a single backbone, Qwen2.5-VL-7B. Because the method's motivation is to leverage the egocentric reasoning abilities that VLMs already possess, the framework should be tested with at least one or two additional backbones (for example, a LLaVA variant or GPT-4o) to demonstrate that the gains are not an artifact of one model's prompt-following behavior. This is necessary to support the general claim that APC 'empowers VLMs' rather than only Qwen2.5-VL.
  4. [Appendix C.3] The visual-prompt rendering parameters are not fully specified: zmin, zmax, and d* are referenced but never assigned numerical values, and the 'shift the camera backward along the z-axis' step for 3DSRBench left/right does not state the shift distance. These choices directly affect which objects appear in the visual prompt and therefore influence the reported APC-Vis accuracies. The authors should provide exact values or release code, and should ablate the camera-shift distance to show the sensitivity of the left/right results.
minor comments (6)
  1. [Sec. 1] Typo: 'In constrast' should be 'In contrast'.
  2. [Sec. 3.3] Typo: 'astract scene information' should be 'abstract scene information' in the text near Fig. 5.
  3. [References] References [76] and [77] are duplicate entries for OrientAnything; retain one with the arXiv identifier.
  4. [Sec. 4.3] Typo: 'the different between' should be 'the difference between' near the SpatialPIN* discussion.
  5. [Fig. 8] The x-axis is labeled 'Angle' but the caption defines θ; labeling the axis as 'θ (degrees)' would improve readability.
  6. [Appendix C.1] The sentence 'We then assign the coordinate-wise median of the remaining points in the remaining points as the 3D position' contains a duplicated phrase; it should be 'the coordinate-wise median of the remaining points'.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: APC is an empirical systems pipeline evaluated on external benchmarks with off-the-shelf modules and no fitted parameters.

full rationale

The paper contains no derivation chain in which a predicted quantity is constructed from the same data it is supposed to predict. APC is a fixed pipeline: GroundingDINO, SAM, DepthPro, OrientAnything, and a frozen Qwen2.5-VL are used to build a scene abstraction, transform coordinates to a reference viewer, and re-prompt the VLM. The reported accuracies in Table 1 are measurements against pre-existing benchmarks with known ground truth; the authors' extension of COMFORT++ with additional Blender scenes is transparently described and does not define the answer in terms of the method. There is no fitted parameter that is then renamed as a prediction, and no equation reduces the output to the input by construction. Self-citations, such as the reference to SpatialVLM, are used as baselines or prior work, not as load-bearing justification for APC's central claim. The method's dependence on coarse abstraction quality is acknowledged in Sec. 4.3 as an error source, which is a correctness or robustness concern, not circularity. Therefore the circularity score is 0.

Assumptions & free parameters 3 free parameters · 6 assumptions · 0 invented entities

The framework has no fitted parameters in the derivation sense, but several hand-set thresholds (s, k, depth filter range) control the abstraction quality. The load-bearing modeling assumptions are that off-the-shelf depth, detection, segmentation, and orientation estimates are faithful enough, and that an equal-size-cube rendering preserves the spatial relations needed to answer the questions. No new physical entities are introduced.

free parameters (3)
  • GroundingDINO confidence threshold s = 0.15
    Set by hand in Sec. C.1 to filter detection candidates; affects which object crops are passed to the VLM for refinement.
  • Top-k candidate count k = 5
    Set by hand in Sec. C.1 to limit the number of crops in the VLM selection grid; affects detection accuracy.
  • Depth outlier filter range factor = [0.9, 1.1] times mode depth
    Set by hand in Sec. C.1 to remove background pixels when computing object 3D position; affects centroid accuracy.
assumptions (6)
  • domain assumption The median of unprojected pixels inside the SAM mask approximates the object's true 3D position.
    Invoked in Sec. 3.1; if the depth map or masks are inaccurate, the perspective transform inherits the error.
  • domain assumption OrientAnything's estimated frontal direction matches the facing direction implied by benchmark questions.
    Invoked in Sec. 3.1; facing and left/right tasks directly depend on this orientation.
  • domain assumption VLMs are sufficiently stronger at egocentric than allocentric reasoning that the conversion is beneficial.
    Invoked in Secs. 1 and 3, based on prior work [90]; if this gap were small, APC's gain would shrink.
  • domain assumption A rendering of equal-sized colored cubes preserves left/right, closer, visibility, and facing relations from the reference viewpoint.
    Invoked in Sec. 3.3; normalization and the rule that larger rendered objects are closer encode the abstraction's semantics.
  • ad hoc to paper For 3DSRBench left/right, shifting the rendering camera backward along the z-axis yields the correct notion of left/right for objects behind the reference viewer.
    Appendix C.3; this benchmark-specific adjustment is applied only to that task and affects the reported visual-prompt accuracy.
  • domain assumption Objects with z <= 0 in the reference frame are not visible from that perspective.
    Appendix C.3; used to define visibility in the visual prompt.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Perspective-Aware Reasoning in Vision-Language Models via Mental Imagery Simulation." pith.science (2026). https://pith.science/paper/U774CFTY

@misc{pith2026250417207,
  author       = {Pith},
  title        = {Pith review of: Perspective-Aware Reasoning in Vision-Language Models via Mental Imagery Simulation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/U774CFTY}},
  note         = {Machine review of arXiv:2504.17207}
}
read the original abstract

We present a framework for perspective-aware reasoning in vision-language models (VLMs) through mental imagery simulation. Perspective-taking, the ability to perceive an environment or situation from an alternative viewpoint, is a key benchmark for human-level visual understanding, essential for environmental interaction and collaboration with autonomous agents. Despite advancements in spatial reasoning within VLMs, recent research has shown that modern VLMs significantly lack perspective-aware reasoning capabilities and exhibit a strong bias toward egocentric interpretations. To bridge the gap between VLMs and human perception, we focus on the role of mental imagery, where humans perceive the world through abstracted representations that facilitate perspective shifts. Motivated by this, we propose a framework for perspective-aware reasoning, named Abstract Perspective Change (APC), that effectively leverages vision foundation models, such as object detection, segmentation, and orientation estimation, to construct scene abstractions and enable perspective transformations. Our experiments on synthetic and real-image benchmarks, compared with various VLMs, demonstrate significant improvements in perspective-aware reasoning with our framework, further outperforming fine-tuned spatial reasoning models and novel-view-synthesis-based approaches.

Figures

Figures reproduced from arXiv: 2504.17207 by the authors.

Figure 1
Figure 1. We introduce Abstract Perspective Change (APC), a framework that empowers VLMs to adopt arbitrary perspectives for spatial reasoning. As demonstrated by the examples above, APC significantly enhances VLM’s ability to imagine a scene from alternative viewpoints, overcoming the inherent egocentric bias that constrains the spatial reasoning of existing VLMs to the camera’s viewpoint. Abstract We present a framework for… view at source ↗
Figure 2
Figure 2. Egocentric vs. Allocentric. While VLMs perform well when questions are asked from an egocentric (i.e. camera’s) per￾spective, they struggle when the same questions are posed from an allocentric perspective, showing a strong bias toward egocentric reasoning. age and a perspective-based question as input and operates through three key stages. First, by simulating the mental imagery process, it builds an abstract repre… view at source ↗
Figure 3
Figure 3. Mental Imagery Simulation. Inspired by how humans employ mental imagery to reason from across different perspec￾tives (left), we propose a similar process for VLMs, by construct￾ing an explicit abstraction of the input scene and using it as a foun￾dation for perspective changes (right). 3 [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Pipeline Overview of APC. Our proposed framework consists of three stages. 1) Scene Abstraction (Sec. 3.1): APC first detects the objects of interest and build a coarse 3D abstraction of the scene using off-the-shelf vision foundation models. 2) Perspective Change (Sec…
Figure 5
Figure 5. Figure 5: Perspective Prompt Samples. We explore two variations of perspective prompting, numerical (left) and visual (right). Numerical (textual) prompt is generated by directly utilizing the 3D coordinate and orientation information. To generate the Visual prompt, we first pla…
Figure 6
Figure 6. Figure 6: Benchmark Visualization. Example image-question pairs from 3DSRBench [54] and COMFORT++ [90] benchmarks. The tasks probe spatial reasoning across left-right relations, object visibility, closenss, and the facing direction of objects. tial reasoning tasks that require a…
Figure 7
Figure 7. Figure 7 [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: Perspective Awareness. Each plot shows accuracy versus the angular offset θ between the camera and the refer￾ence viewpoint. While baselines show clear degradation at certain ranges of θ, APC retains robust accuracy across all angles, demon￾strating strong perspective-…
Figure 9
Figure 9. Figure 9: Dense Reconstruction Baseline Examples. Novel views synthesized by SpatialPIN∗ [53] and ViewCrafter [87] both display noisy and inaccurate objects and scene structures lacking the original context of the input image, thereby leading to low accuracy when VLMs are fed th…
Figure 10
Figure 10. Figure 10 [PITH_FULL_IMAGE:figures/full_fig_p014_10.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. VIEW2SPACE: Studying Multi-View Visual Reasoning from Sparse Observations

    cs.CV 2026-03 conditional novelty 6.5 of 10

    Sparse multi-view reasoning is largely unsolved for VLMs; grounded CoT with visual evidence improves moderate cases and transfers to real data, but deep multi-hop reasoning still scales poorly.

  2. SpaceTools: Tool-Augmented Spatial Reasoning via Double Interactive RL

    cs.CV 2025-12 conditional novelty 6.0 of 10

    A two-phase interactive RL framework (DIRL) lets a 3B VLM learn to coordinate multiple vision and robot tools, reaching top benchmark scores and 86% real-robot pick-and-place success.

  3. Dense360: Dense Understanding from Omnidirectional Panoramas

    cs.CV 2025-06 reject novelty 6.0 of 10

    Introduces a 160K-panorama auto-annotated dataset, a dense captioning and grounding benchmark, and ERP-RoPE; fine-tuning Qwen2.5VL on the data lifts benchmark scores.

  4. OrientSAM: Mitigating Camera-Centric Shortcut in Multimodal Spatial Reasoning via Orientation-Aware Spatial Alignment

    cs.AI 2026-07 conditional novelty 5.0 of 10

    OrientSAM injects Fourier-encoded object orientation into a vision-language model and uses curriculum training, improving reference-centric spatial reasoning and reducing camera-centric shortcut behavior.

Reference graph

Works this paper leans on

93 extracted references · 53 canonical work pages · cited by 4 Pith papers

  1. [1]

    Flamingo: a visual language model for few-shot learning

    Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al. Flamingo: a visual language model for few-shot learning. In NeurIPS,

  2. [2]

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923, 2025. 2, 3, 6, 7, 8

    arXiv 2025

  3. [3]

    Visual question answering from another perspective: Clevr mental rotation tests

    Christopher Beckham, Martin Weiss, Florian Golemo, Sina Honari, Derek Nowrouzezahrai, and Christopher Pal. Visual question answering from another perspective: Clevr mental rotation tests. Pattern Recognition, 2023. 3

    2023

  4. [4]

    Depth pro: Sharp monocular metric depth in less than a second

    Aleksei Bochkovskii, Ama ¨el Delaunoy, Hugo Germain, Marcel Santos, Yichao Zhou, Stephan R Richter, and Vladlen Koltun. Depth pro: Sharp monocular metric depth in less than a second. In ICLR, 2025. 4

    2025

  5. [5]

    Omni3d: A large benchmark and model for 3d object detection in the wild

    Garrick Brazil, Abhinav Kumar, Julian Straub, Nikhila Ravi, Justin Johnson, and Georgia Gkioxari. Omni3d: A large benchmark and model for 3d object detection in the wild. In CVPR, 2023. 13

    2023

  6. [6]

    Spatial memory: how egocentric and allocen- tric combine

    Neil Burgess. Spatial memory: how egocentric and allocen- tric combine. Trends in cognitive sciences, 2006. 2

    2006

  7. [7]

    Spatialbot: Pre- cise spatial understanding with vision language models

    Wenxiao Cai, Iaroslav Ponomarenko, Jianhao Yuan, Xiaoqi Li, Wankou Yang, Hao Dong, and Bo Zhao. Spatialbot: Pre- cise spatial understanding with vision language models. In ICRA, 2025. 2, 3

    2025

  8. [8]

    Monoscene: Monoc- ular 3d semantic scene completion

    Anh-Quan Cao and Raoul De Charette. Monoscene: Monoc- ular 3d semantic scene completion. In CVPR, 2022. 13

    2022

Show all 93 references
  1. [9]

    End-to- end object detection with transformers

    Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to- end object detection with transformers. In ECCV, 2020. 2

    2020

  2. [10]

    Spatialvlm: Endow- ing vision-language models with spatial reasoning capabili- ties

    Boyuan Chen, Zhuo Xu, Sean Kirmani, Brain Ichter, Dorsa Sadigh, Leonidas Guibas, and Fei Xia. Spatialvlm: Endow- ing vision-language models with spatial reasoning capabili- ties. In CVPR, 2024. 1, 2, 3, 6, 7

    2024

  3. [11]

    Visual chain- of-thought prompting for knowledge-based visual reasoning

    Zhenfang Chen, Qinhong Zhou, Yikang Shen, Yining Hong, Zhiqing Sun, Dan Gutfreund, and Chuang Gan. Visual chain- of-thought prompting for knowledge-based visual reasoning. In AAAI, 2024. 3

    2024

  4. [12]

    Spatial- rgpt: Grounded spatial reasoning in vision language model

    An-Chieh Cheng, Hongxu Yin, Yang Fu, Qiushan Guo, Rui- han Yang, Jan Kautz, Xiaolong Wang, and Sifei Liu. Spatial- rgpt: Grounded spatial reasoning in vision language model. In NeurIPS, 2024. 1, 2, 3, 6, 7

    2024

  5. [13]

    Clark and Susan E

    Herbert H. Clark and Susan E. Brennan. Grounding in communication. Perspectives on Socially Shared Cognition,

  6. [14]

    A return of mental imagery: The pictorial theory of visual perspective-taking

    Geoff G Cole, Steven Samuel, and Madeline J Eacott. A return of mental imagery: The pictorial theory of visual perspective-taking. Consciousness and Cognition, 2022. 2

    2022

  7. [15]

    Blender - a 3d modelling and rendering package, 2018

    Blender Online Community. Blender - a 3d modelling and rendering package, 2018. 6

    2018

  8. [16]

    Panoptic 3d scene reconstruction from a single rgb image

    Manuel Dahnert, Ji Hou, Matthias Nießner, and Angela Dai. Panoptic 3d scene reconstruction from a single rgb image. In NeurIPS, 2021. 13

    2021

  9. [17]

    Dawson-Haggerty et al. trimesh. 14

  10. [18]

    Objaverse-xl: A universe of 10m+ 3d objects

    Matt Deitke, Ruoshi Liu, Matthew Wallingford, Huong Ngo, Oscar Michel, Aditya Kusupati, Alan Fan, Christian Laforte, Vikram V oleti, Samir Yitzhak Gadre, et al. Objaverse-xl: A universe of 10m+ 3d objects. In NeurIPS, 2023. 15

    2023

  11. [19]

    Molmo and pixmo: Open weights and open data for state-of-the-art multimodal models

    Matt Deitke, Christopher Clark, Sangho Lee, Rohun Tri- pathi, Yue Yang, Jae Sung Park, Mohammadreza Salehi, Niklas Muennighoff, Kyle Lo, Luca Soldaini, et al. Molmo and pixmo: Open weights and open data for state-of-the-art multimodal models. arXiv preprint arXiv:2409.17146, 2...

    2024 arXiv

  12. [20]

    Viewpoint dependence in scene recognition

    Vaibhav A Diwadkar and Timothy P McNamara. Viewpoint dependence in scene recognition. Psychological science ,

  13. [21]

    Vlmevalkit: An open-source toolkit for evaluating large multi-modality models

    Haodong Duan, Junming Yang, Yuxuan Qiao, Xinyu Fang, Lin Chen, Yuan Liu, Xiaoyi Dong, Yuhang Zang, Pan Zhang, Jiaqi Wang, et al. Vlmevalkit: An open-source toolkit for evaluating large multi-modality models. In ACM MM, 2024. 15

    2024

  14. [22]

    Minedojo: Building open-ended embodied agents with internet-scale knowledge

    Linxi Fan, Guanzhi Wang, Yunfan Jiang, Ajay Mandlekar, Yuncong Yang, Haoyi Zhu, Andrew Tang, De-An Huang, Yuke Zhu, and Anima Anandkumar. Minedojo: Building open-ended embodied agents with internet-scale knowledge. In NeurIPS, 2022. 2

    2022

  15. [23]

    Principles of mental imagery, 1989

    RA Finke. Principles of mental imagery, 1989. 2, 3, 4

    1989

  16. [24]

    Blink: Multimodal large language models can see but not perceive

    Xingyu Fu, Yushi Hu, Bangzheng Li, Yu Feng, Haoyu Wang, Xudong Lin, Dan Roth, Noah A Smith, Wei-Chiu Ma, and Ranjay Krishna. Blink: Multimodal large language models can see but not perceive. In ECCV, 2024. 3

    2024

  17. [25]

    Seeing through their eyes: Evaluating visual per- spective taking in vision language models

    Gracjan G ´oral, Alicja Ziarko, Michal Nauman, and Maciej Wołczyk. Seeing through their eyes: Evaluating visual per- spective taking in vision language models. arXiv preprint arXiv:2409.12969, 2024. 2, 3

    2024 arXiv

  18. [26]

    Visual program- ming: Compositional visual reasoning without training

    Tanmay Gupta and Aniruddha Kembhavi. Visual program- ming: Compositional visual reasoning without training. In CVPR, 2023. 3

    2023

  19. [27]

    Omni-rgpt: Unifying image and video region-level understanding via token marks

    Miran Heo, Min-Hung Chen, De-An Huang, Sifei Liu, Sub- hashree Radhakrishnan, Seon Joo Kim, Yu-Chiang Frank Wang, and Ryo Hachiuma. Omni-rgpt: Unifying image and video region-level understanding via token marks. In CVPR,

  20. [28]

    Visual sketchpad: Sketching as a visual chain of thought for multimodal language models

    Yushi Hu, Weijia Shi, Xingyu Fu, Dan Roth, Mari Osten- dorf, Luke Zettlemoyer, Noah A Smith, and Ranjay Krishna. Visual sketchpad: Sketching as a visual chain of thought for multimodal language models. In NeurIPS, 2024. 3

    2024

  21. [29]

    Visual program distillation: Distilling tools and programmatic reasoning into vision-language models

    Yushi Hu, Otilia Stretcu, Chun-Ta Lu, Krishnamurthy Viswanathan, Kenji Hata, Enming Luo, Ranjay Krishna, and Ariel Fuxman. Visual program distillation: Distilling tools and programmatic reasoning into vision-language models. In CVPR, 2025. 2

    2025

  22. [30]

    Gpt-4o system card

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perel- man, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Weli- hinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276, 2024. 2, 3, 6, 7

    2024 arXiv

  23. [31]

    Perspective fields for single image camera calibration

    Linyi Jin, Jianming Zhang, Yannick Hold-Geoffroy, Oliver Wang, Kevin Blackburn-Matzen, Matthew Sticha, and David F Fouhey. Perspective fields for single image camera calibration. In CVPR, 2023. 3 9

    2023

  24. [32]

    What’s “up” with vision-language models? investigating their strug- gle with spatial reasoning

    Amita Kamath, Jack Hessel, and Kai-Wei Chang. What’s “up” with vision-language models? investigating their strug- gle with spatial reasoning. In EMNLP, 2023. 3

    2023

  25. [33]

    Berg, Wan-Yen Lo, Piotr Doll ´ar, and Ross Girshick

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C. Berg, Wan-Yen Lo, Piotr Doll ´ar, and Ross Girshick. Segment anything. In ICCV, 2023. 2

    2023

  26. [34]

    Segment any- thing

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C Berg, Wan-Yen Lo, et al. Segment any- thing. In ICCV, 2023. 4, 13

    2023

  27. [35]

    S. M. Kosslyn, T. M. Ball, and B. J. Reiser. Visual images preserve metric spatial information: Evidence from studies of image scanning. Journal of Experimental Psychology: Human Perception and Performance, 1978. 2

    1978

  28. [36]

    Scaffolding coordinates to promote vision-language coordination in large multi-modal models

    Xuanyu Lei, Zonghan Yang, Xinrui Chen, Peng Li, and Yang Liu. Scaffolding coordinates to promote vision-language coordination in large multi-modal models. arXiv preprint arXiv:2402.12058, 2024. 3

    2024 arXiv

  29. [37]

    Llava-onevision: Easy visual task transfer

    Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Zi- wei Liu, et al. Llava-onevision: Easy visual task transfer. arXiv preprint arXiv:2408.03326, 2024. 2, 3, 6, 7

    2024 arXiv

  30. [38]

    Topviewrs: Vision-language models as top-view spatial reasoners

    Chengzu Li, Caiqi Zhang, Han Zhou, Nigel Collier, Anna Korhonen, and Ivan Vuli ´c. Topviewrs: Vision-language models as top-view spatial reasoners. In EMNLP, 2024. 3

    2024

  31. [39]

    Imag- ine while reasoning in space: Multimodal visualization-of- thought

    Chengzu Li, Wenshan Wu, Huanyu Zhang, Yan Xia, Shaoguang Mao, Li Dong, Ivan Vuli´c, and Furu Wei. Imag- ine while reasoning in space: Multimodal visualization-of- thought. arXiv preprint arXiv:2501.07542, 2025. 3

    2025 arXiv

  32. [40]

    Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation

    Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation. In ICML,

  33. [41]

    Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In ICML,

  34. [42]

    Microsoft coco: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll´ar, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In ECCV, 2014. 6

    2014

  35. [43]

    The 3d-pc: a benchmark for visual per- spective taking in humans and machines

    Drew Linsley, Peisen Zhou, Alekh Karkada Ashok, Akash Nagaraj, Gaurav Gaonkar, Francis E Lewis, Zygmunt Pizlo, and Thomas Serre. The 3d-pc: a benchmark for visual per- spective taking in humans and machines. In ICLR, 2025. 2, 3

    2025

  36. [44]

    Coarse corre- spondence elicit 3d spacetime understanding in multimodal language model

    Benlin Liu, Yuhao Dong, Yiqin Wang, Yongming Rao, Yan- song Tang, Wei-Chiu Ma, and Ranjay Krishna. Coarse corre- spondence elicit 3d spacetime understanding in multimodal language model. In CVPR, 2025. 3

    2025

  37. [45]

    Visual spatial reasoning

    Fangyu Liu, Guy Emerson, and Nigel Collier. Visual spatial reasoning. In EMNLP, 2023. 3

    2023

  38. [46]

    Visual instruction tuning

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. In NeurIPS, 2023. 2, 3

    2023

  39. [47]

    Llava-next: Im- proved reasoning, ocr, and world knowledge, 2024

    Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. Llava-next: Im- proved reasoning, ocr, and world knowledge, 2024. 3, 6, 7

    2024

  40. [48]

    One-2-3-45: Any single image to 3d mesh in 45 seconds without per-shape optimiza- tion

    Minghua Liu, Chao Xu, Haian Jin, Linghao Chen, Mukund Varma T, Zexiang Xu, and Hao Su. One-2-3-45: Any single image to 3d mesh in 45 seconds without per-shape optimiza- tion. In NeurIPS, 2023. 13

    2023

  41. [49]

    One-2-3-45++: Fast single image to 3d objects with consistent multi-view generation and 3d diffusion

    Minghua Liu, Ruoxi Shi, Linghao Chen, Zhuoyang Zhang, Chao Xu, Xinyue Wei, Hansheng Chen, Chong Zeng, Ji- ayuan Gu, and Hao Su. One-2-3-45++: Fast single image to 3d objects with consistent multi-view generation and 3d diffusion. In CVPR, 2024. 3, 13

    2024

  42. [50]

    Grounding dino: Marrying dino with grounded pre-training for open-set object detection

    Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Qing Jiang, Chunyuan Li, Jianwei Yang, Hang Su, et al. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. In ECCV, 2024. 2, 4, 13, 14

    2024

  43. [51]

    Mmbench: Is your multi-modal model an all-around player? In ECCV, 2024

    Yuan Liu, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Wangbo Zhao, Yike Yuan, Jiaqi Wang, Conghui He, Ziwei Liu, et al. Mmbench: Is your multi-modal model an all-around player? In ECCV, 2024. 15

    2024

  44. [52]

    Spatialcot: Advancing spatial reasoning through coordinate alignment and chain-of-thought for embodied task planning

    Yuecheng Liu, Dafeng Chi, Shiguang Wu, Zhanguang Zhang, Yaochen Hu, Lingfeng Zhang, Yingxue Zhang, Shuang Wu, Tongtong Cao, Guowei Huang, et al. Spatialcot: Advancing spatial reasoning through coordinate alignment and chain-of-thought for embodied task planning. arXiv preprint...

    2025 arXiv

  45. [53]

    Spatialpin: Enhancing spatial reason- ing capabilities of vision-language models through prompt- ing and interacting 3d priors

    Chenyang Ma, Kai Lu, Ta-Ying Cheng, Niki Trigoni, and Andrew Markham. Spatialpin: Enhancing spatial reason- ing capabilities of vision-language models through prompt- ing and interacting 3d priors. In NeurIPS, 2024. 1, 2, 3, 6, 7, 13

    2024

  46. [54]

    3dsrbench: A compre- hensive 3d spatial reasoning benchmark

    Wufei Ma, Haoyu Chen, Guofeng Zhang, Celso M de Melo, Alan Yuille, and Jieneng Chen. 3dsrbench: A compre- hensive 3d spatial reasoning benchmark. arXiv preprint arXiv:2412.07825, 2024. 2, 3, 6, 7, 14, 15

    2024

  47. [55]

    Visual agentic ai for spatial reasoning with a dynamic api

    Damiano Marsili, Rohun Agrawal, Yisong Yue, and Geor- gia Gkioxari. Visual agentic ai for spatial reasoning with a dynamic api. In CVPR, 2025. 3

    2025

  48. [56]

    Mental imagery

    Bence Nanay. Mental imagery. The Stanford Encyclopedia of Philosophy, 2021. 2, 3

    2021

  49. [57]

    Total3dunderstanding: Joint lay- out, object pose and mesh reconstruction for indoor scenes from a single image

    Yinyu Nie, Xiaoguang Han, Shihui Guo, Yujian Zheng, Jian Chang, and Jian Jun Zhang. Total3dunderstanding: Joint lay- out, object pose and mesh reconstruction for indoor scenes from a single image. In CVPR, 2020. 13

    2020

  50. [58]

    Learning 3d scene priors with 2d supervision

    Yinyu Nie, Angela Dai, Xiaoguang Han, and Matthias Nießner. Learning 3d scene priors with 2d supervision. In CVPR, 2023. 13

    2023

  51. [59]

    A. Paivio. Imagery and Verbal Processes (1st ed.). Psychol- ogy Press, 1979. 2

    1979

  52. [60]

    Vision language models are blind

    Pooyan Rahmanzadehgervi, Logan Bolton, Moham- mad Reza Taesiri, and Anh Totti Nguyen. Vision language models are blind. In ACCV, 2024. 3

    2024

  53. [61]

    Does spatial cognition emerge in frontier models? In ICLR, 2025

    Santhosh Kumar Ramakrishnan, Erik Wijmans, Philipp Kraehenbuehl, and Vladlen Koltun. Does spatial cognition emerge in frontier models? In ICLR, 2025. 3

    2025

  54. [62]

    Sam 2: Segment anything in images and videos

    Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman 10 R¨adle, Chloe Rolland, Laura Gustafson, Eric Mintun, Junt- ing Pan, Kalyan Vasudev Alwala, Nicolas Carion, Chao- Yuan Wu, Ross Girshick, Piotr Doll´ar, and Christoph F...

    2024

  55. [63]

    Sat: Spa- tial aptitude training for multimodal language models

    Arijit Ray, Jiafei Duan, Reuben Tan, Dina Bashkirova, Rose Hendrix, Kiana Ehsani, Aniruddha Kembhavi, Bryan A Plummer, Ranjay Krishna, Kuo-Hao Zeng, et al. Sat: Spa- tial aptitude training for multimodal language models. arXiv preprint arXiv:2412.07755, 2024. 3

    2024

  56. [64]

    Visual chain of thought: bridging logical gaps with multimodal infillings

    Daniel Rose, Vaishnavi Himakunthala, Andy Ouyang, Ryan He, Alex Mei, Yujie Lu, Michael Saxon, Chinmay Sonar, Diba Mirza, and William Yang Wang. Visual chain of thought: bridging logical gaps with multimodal infillings. arXiv preprint arXiv:2305.02317, 2023. 3

    2023 arXiv

  57. [65]

    Visual perspective-taking in complex natural scenes

    Paola Del Sette, Markus Bindemann, and Heather J Fergu- son. Visual perspective-taking in complex natural scenes. Quarterly Journal of Experimental Psychology, 2022. 3

    2022

  58. [66]

    Visual cot: Advancing multi-modal language models with a comprehen- sive dataset and benchmark for chain-of-thought reasoning

    Hao Shao, Shengju Qian, Han Xiao, Guanglu Song, Zhuofan Zong, Letian Wang, Yu Liu, and Hongsheng Li. Visual cot: Advancing multi-modal language models with a comprehen- sive dataset and benchmark for chain-of-thought reasoning. NeurIPS, 2024. 3

    2024

  59. [67]

    Mental rotation of three-dimensional objects

    Roger N Shepard and Jacqueline Metzler. Mental rotation of three-dimensional objects. Science, 171(3972):701–703,

  60. [68]

    An empirical analysis on spatial reason- ing capabilities of large multimodal models

    Fatemeh Shiri, Xiao-Yu Guo, Mona Far, Xin Yu, Reza Haf, and Yuan-Fang Li. An empirical analysis on spatial reason- ing capabilities of large multimodal models. In EMNLP,

  61. [69]

    What does clip know about a red circle? visual prompt engineering for vlms

    Aleksandar Shtedritski, Christian Rupprecht, and Andrea Vedaldi. What does clip know about a red circle? visual prompt engineering for vlms. In ICCV, 2023. 3

    2023

  62. [70]

    Robospatial: Teaching spatial understanding to 2d and 3d vision-language models for robotics

    Chan Hee Song, Valts Blukis, Jonathan Tremblay, Stephen Tyree, Yu Su, and Stan Birchfield. Robospatial: Teaching spatial understanding to 2d and 3d vision-language models for robotics. In CVPR, 2025. 3

    2025

  63. [71]

    Vipergpt: Vi- sual inference via python execution for reasoning

    D ´ıdac Sur´ıs, Sachit Menon, and Carl V ondrick. Vipergpt: Vi- sual inference via python execution for reasoning. In ICCV,

  64. [72]

    Sparkle: Mastering basic spatial capabilities in vision language models elicits gen- eralization to composite spatial reasoning

    Yihong Tang, Ao Qu, Zhaokai Wang, Dingyi Zhuang, Zhaofeng Wu, Wei Ma, Shenhao Wang, Yunhan Zheng, Zhan Zhao, and Jinhua Zhao. Sparkle: Mastering basic spatial capabilities in vision language models elicits gen- eralization to composite spatial reasoning. arXiv preprint arXiv:2...

  65. [73]

    Gemini 1.5: Unlocking multimodal understanding across millions of tokens of con- text

    Gemini Team, Petko Georgiev, Ving Ian Lei, Ryan Burnell, Libin Bai, Anmol Gulati, Garrett Tanzer, Damien Vincent, Zhufeng Pan, Shibo Wang, et al. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of con- text. arXiv preprint arXiv:2403.05530, 2024. 2, 3, 6, 7

    2024 arXiv

  66. [74]

    Cambrian-1: A fully open, vision-centric ex- ploration of multimodal llms

    Peter Tong, Ellis Brown, Penghao Wu, Sanghyun Woo, Adithya Jairam Vedagiri IYER, Sai Charitha Akula, Shusheng Yang, Jihan Yang, Manoj Middepogu, Ziteng Wang, et al. Cambrian-1: A fully open, vision-centric ex- ploration of multimodal llms. In NeurIPS, 2024. 1, 2, 3, 7, 8

    2024

  67. [75]

    Is a picture worth a thou- sand words? delving into spatial reasoning for vision lan- guage models

    Jiayu Wang, Yifei Ming, Zhenmei Shi, Vibhav Vineet, Xin Wang, Sharon Li, and Neel Joshi. Is a picture worth a thou- sand words? delving into spatial reasoning for vision lan- guage models. In NeurIPS, 2024. 3

    2024

  68. [76]

    Orient anything: Learning robust object orientation estimation from rendering 3d mod- els

    Zehan Wang, Ziang Zhang, Tianyu Pang, Chao Du, Heng- shuang Zhao, and Zhou Zhao. Orient anything: Learning robust object orientation estimation from rendering 3d mod- els. arXiv, 2024. 13

    2024

  69. [77]

    Orient anything: Learning robust object orientation estimation from rendering 3d mod- els

    Zehan Wang, Ziang Zhang, Tianyu Pang, Chao Du, Heng- shuang Zhao, and Zhou Zhao. Orient anything: Learning robust object orientation estimation from rendering 3d mod- els. arXiv preprint arXiv:2412.18605, 2024. 2, 4

    2024 arXiv

  70. [78]

    Visual prompting in multi- modal large language models: A survey

    Junda Wu, Zhehao Zhang, Yu Xia, Xintong Li, Zhaoyang Xia, Aaron Chang, Tong Yu, Sungchul Kim, Ryan A Rossi, Ruiyi Zhang, et al. Visual prompting in multi- modal large language models: A survey. arXiv preprint arXiv:2409.15310, 2024. 3

    2024 arXiv

  71. [79]

    Mind’s eye of llms: Visualization-of-thought elicits spatial reasoning in large language models

    Wenshan Wu, Shaoguang Mao, Yadong Zhang, Yan Xia, Li Dong, Lei Cui, and Furu Wei. Mind’s eye of llms: Visualization-of-thought elicits spatial reasoning in large language models. In NeurIPS, 2024. 3

    2024

  72. [80]

    Det- toolchain: A new prompting paradigm to unleash detection ability of mllm

    Yixuan Wu, Yizhou Wang, Shixiang Tang, Wenhao Wu, Tong He, Wanli Ouyang, Philip Torr, and Jian Wu. Det- toolchain: A new prompting paradigm to unleash detection ability of mllm. In ECCV, 2024. 3

    2024

  73. [81]

    How2comm: Communication-efficient and collaboration-pragmatic multi- agent perception

    Dingkang Yang, Kun Yang, Yuzheng Wang, Jing Liu, Zhi Xu, Rongbin Yin, Peng Zhai, and Lihua Zhang. How2comm: Communication-efficient and collaboration-pragmatic multi- agent perception. In NeurIPS, 2023. 2

    2023

  74. [82]

    Set-of-mark prompting unleashes extraordinary visual grounding in gpt-4v

    Jianwei Yang, Hao Zhang, Feng Li, Xueyan Zou, Chunyuan Li, and Jianfeng Gao. Set-of-mark prompting unleashes extraordinary visual grounding in gpt-4v. arXiv preprint arXiv:2310.11441, 2023. 3

    2023 arXiv

  75. [83]

    Thinking in space: How mul- timodal large language models see, remember, and recall spaces

    Jihan Yang, Shusheng Yang, Anjali W Gupta, Rilyn Han, Li Fei-Fei, and Saining Xie. Thinking in space: How mul- timodal large language models see, remember, and recall spaces. In CVPR, 2025. 3

    2025

  76. [84]

    Fine-grained visual prompting

    Lingfeng Yang, Yueze Wang, Xiang Li, Xinlong Wang, and Jian Yang. Fine-grained visual prompting. InNeurIPS, 2023. 3

    2023

  77. [85]

    Guiding long-horizon task and motion planning with vision language models

    Zhutian Yang, Caelan Garrett, Dieter Fox, Tom ´as Lozano- P´erez, and Leslie Pack Kaelbling. Guiding long-horizon task and motion planning with vision language models. In ICRA,

  78. [86]

    Open vocabulary monocular 3d object detection

    Jin Yao, Hao Gu, Xuweiyi Chen, Jiayun Wang, and Zezhou Cheng. Open vocabulary monocular 3d object detection. arXiv preprint arXiv:2411.16833, 2024. 13

    2024

  79. [87]

    Viewcrafter: Taming video diffusion models for high-fidelity novel view synthesis.arXiv preprint arXiv:2409.02048, 2024

    Wangbo Yu, Jinbo Xing, Li Yuan, Wenbo Hu, Xiaoyu Li, Zhipeng Huang, Xiangjun Gao, Tien-Tsin Wong, Ying Shan, and Yonghong Tian. Viewcrafter: Taming video diffusion models for high-fidelity novel view synthesis.arXiv preprint arXiv:2409.02048, 2024. 6, 7, 13

    2024 arXiv

  80. [88]

    Robopoint: A vision-language model for spatial affordance prediction for robotics

    Wentao Yuan, Jiafei Duan, Valts Blukis, Wilbert Pumacay, Ranjay Krishna, Adithyavairavan Murali, Arsalan Mousa- vian, and Dieter Fox. Robopoint: A vision-language model for spatial affordance prediction for robotics. In CORL,

  81. [89]

    Sphere: A hierarchical evaluation on spatial perception and reasoning for vision- language models

    Wenyu Zhang, Wei En Ng, Lixin Ma, Yuwen Wang, Jungqi Zhao, Boyang Li, and Lu Wang. Sphere: A hierarchical evaluation on spatial perception and reasoning for vision- language models. arXiv preprint arXiv:2412.12693 , 2024. 2, 3

    2024 arXiv

  82. [90]

    Do vision- language models represent space and how? evaluating spatial frame of reference under ambiguities

    Zheyuan Zhang, Fengyuan Hu, Jayjun Lee, Freda Shi, Parisa Kordjamshidi, Joyce Chai, and Ziqiao Ma. Do vision- language models represent space and how? evaluating spatial frame of reference under ambiguities. In ICLR, 2025. 2, 3, 4, 6, 7, 15

    2025

  83. [91]

    Cot-vla: Visual chain-of-thought rea- soning for vision-language-action models

    Qingqing Zhao, Yao Lu, Moo Jin Kim, Zipeng Fu, Zhuoyang Zhang, Yecheng Wu, Zhaoshuo Li, Qianli Ma, Song Han, Chelsea Finn, et al. Cot-vla: Visual chain-of-thought rea- soning for vision-language-action models. In CVPR, 2025. 3

    2025

  84. [92]

    Image-of-thought prompting for visual reasoning refinement in multimodal large language models

    Qiji Zhou, Ruochen Zhou, Zike Hu, Panzhong Lu, Siyang Gao, and Yue Zhang. Image-of-thought prompting for visual reasoning refinement in multimodal large language models. arXiv preprint arXiv:2405.13872, 2024. 3

    2024 arXiv

  85. [93]

    Select the image that best fits the description: ‘man in white shirt’. Please return its index

    Xizhou Zhu, Weijie Su, Lewei Lu, Bin Li, Xiaogang Wang, and Jifeng Dai. Deformable detr: Deformable transformers for end-to-end object detection. In ICLR, 2021. 2 12 Appendix In this appendix, we first discuss the limitations of our work and potential directions for future wor...

    2021

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.