Pith. sign in

REVIEW 3 major objections 4 minor 9 cited by

Current multimodal large language models are constrained less by spatial training data volume than by how positional encoding is handled in the visual encoder, and data scaling alone will not close the spatial reasoning gap.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Spatial understanding in multimodal LLMs plateaus quickly as training data grows, and position encoding in the visual encoder is the more influential factor.

T0 review reviewed 2026-08-05 challenge →

load-bearing objection Good benchmark and data scaling study, but the VE-over-LLM conclusion depends on a confounded ablation that needs re-running. the 3 major comments →

arxiv 2509.02359 v1 pith:7LNV4V7U submitted 2025-09-02 cs.CV

Why Do MLLMs Struggle with Spatial Understanding? A Systematic Analysis from Data to Architecture

Wanyue Zhang , Yibin Huang , Yangbin Xu , JingJing Huang , Helu Zhi , Shuo Ren , Wang Xu , Jiajun Zhang This is my paper
classification cs.CV
keywords spatial understandingmultimodal large language modelspositional encodingdata scalingmulti-view benchmarkmachine reasoningRoPEvisual reasoning
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks whether current multimodal large language models fail at spatial understanding because they lack training data or because their architecture cannot represent where things are. Using a new multi-view benchmark, MulSeT, plus single-view and video tasks, it finds that fine-tuning on more spatial data improves performance only modestly and plateaus quickly, with the lowest ceiling on tasks that demand spatial imagination. Ablations of positional encodings show that the vision encoder's 2D position signals are the dominant carrier of spatial ability, while disturbing the language model's position tokens has comparatively small effects. The paper concludes that future gains require architectural changes and targeted reasoning prompts, not just data scaling.

Core claim

The central claim is that current MLLMs' poor spatial understanding is not primarily a data-scaling problem: fine-tuning on 1k to 90k spatial samples produces gains that plateau quickly, and the ceiling is lowest precisely for tasks that require spatial imagination, such as azimuth transfer. On the architecture side, the paper argues that the visual encoder's positional encoding is the main carrier of spatial ability. In Qwen2.5-VL, LLaVA-OneVision, and Mono-InternVL, shuffling or masking the vision encoder's 2D-RoPE height and width dimensions sharply degrades spatial tasks, while perturbing the LLM's position tokens does much less; direction-specific ablations show the width dimension cont

What carries the argument

The central object is positional encoding, specifically the vision encoder's 2D rotary position embedding (2D-RoPE) with separate height and width dimensions, contrasted with the language model's M-RoPE or 1D-RoPE. The argument is carried by controlled ablations—mask, shuffle, and constant-value strategies—applied to one modality and one PE dimension at a time. This machinery shows that spatial performance tracks the integrity of the vision encoder's height and width components, while LLM positional perturbations matter much less, establishing the visual encoder's positional signal as the structural bottleneck.

Load-bearing premise

The load-bearing premise is that the PE ablations measure spatial information specifically; in fact the harshest ablations also destroy every non-spatial ordering cue, so the reported collapses are not clean evidence of a spatial-only bottleneck, and the 'low upper bound' on imagination tasks is partly set by items humans themselves answer at chance.

What would settle it

Run the same PE ablations on a non-spatial control task and check whether the strongest text-position perturbation also collapses non-spatial accuracy; if it does, the bottleneck is not specifically spatial. Separately, re-score Azimuth Transfer with items where humans are evaluated on wider angular separations; if human accuracy rises well above 51%, the low model ceiling is partly a benchmark artifact.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Fine-tuning on synthetic multi-view data or video spatial data yields fast-saturating returns, so adding more spatial data alone will not close the gap on spatial reasoning.
  • The performance ceiling depends on task type: semantic-matching tasks like occlusion restoration respond to data, while tasks requiring spatial imagination, such as azimuth transfer and route planning, stay low.
  • Architectural changes that preserve or strengthen visual-encoder positional signals should matter more than larger datasets or larger models.
  • Reasoning injection helps mainly on harder logical spatial tasks, and a multi-view consistency prompt outperforms generic stepwise prompting; explicit chain-of-thought can harm performance by diffusing attention.
  • Model size gives diminishing fine-tuning gains for specialized spatial tasks, as seen across the 3B, 7B, 32B, and 72B Qwen2.5-VL family.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the visual encoder's positional encoding is the true bottleneck, then MLLMs that feed the LLM only patch tokens without explicit 2D or 3D position information should continue to fail at spatial integration; this is testable by comparing vision encoders with and without strong 2D-RoPE while holding data and model size constant.
  • The paper's strongest positional ablation also collapses non-spatial VG-Attribution (from 90.79 to 1.63), so the reported catastrophic drops likely reflect destruction of all ordering information rather than spatial-specific encoding; a cleaner experiment would perturb only the spatial dimensions while preserving token order.
  • The low human accuracy on azimuth transfer (51.25%) suggests the benchmark's imagination tasks may be ambiguous even for people; a cleaned subset with larger angular separations would give a fairer estimate of the true model ceiling.
  • The reasoning-injection results point to a potentially data-free gain: implicit step-by-step prompting with multi-view consistency may improve spatial tasks without fine-tuning, and explicit CoT should be evaluated against attention-ground-truth tasks where distraction can be measured directly.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper presents a systematic analysis of why MLLMs struggle with spatial understanding, across single-view, multi-view, and video settings. It introduces MulSeT, a synthetic multi-view benchmark with three tasks (occlusion restoration, distance comparison, azimuth transfer), and uses it to study how fine-tuning data scale affects spatial performance. The paper also ablates positional encodings in the vision encoder (VE) and language model (LLM) of several MLLMs (Qwen2.5-VL-7B, LLaVA-OneVision-7B, Mono-InternVL-2B), concluding that VE positional encoding is substantially more critical than LLM positional encoding for spatial understanding. Finally, it evaluates reasoning-injection prompts and proposes a multi-view consistency prompt that outperforms stepwise and explicit-CoT variants.

Significance. If the architectural conclusion holds, it would be an important and actionable finding: current MLLMs have a structural bottleneck in visual position representation, not merely a data-hunger problem. The paper is commendable for releasing MulSeT and code, for using controlled synthetic scenes, for evaluating on external benchmarks (What'sUp, COCO-QA, VSI-Bench), and for including a non-spatial control task (VG-Attribution). The data-scaling analysis across multiple model sizes and tasks is also a useful empirical contribution. However, the PE ablation protocol does not isolate spatial information, and the data-scaling claims lack statistical support. The core conclusions are plausible but not established by the evidence as presented.

major comments (3)
  1. [Sec. 4.2 / Table 2] The central architectural claim that VE positional encoding is more critical than LLM positional encoding rests on an asymmetric and confounded ablation. Shuffle-LLM-txy(*) modifies both text and vision tokens across all PE dimensions, and it collapses even the non-spatial VG-Attribution task from 90.79 to 1.63 (Table 2; same in Table 6). This shows the manipulation destroys all token-order information needed for any language task, not selectively spatial cues. By contrast, Shuffle-VE-hw only perturbs two spatial dimensions in the image modality. The observed 'VE more critical' difference could therefore reflect ablation breadth and task-agnostic order destruction rather than a spatial-specific bottleneck. The paper's own text-only LLM ablations (Shuffle-LLM-txy) produce small or even positive changes on MulSeT Azimuth (+6.53), so the conclusion cannot be read off the data as presented.
  2. [Sec. 3.1 / Figures 4 and 5] The data-scaling conclusion is supported by single runs with no error bars or significance tests, and the protocol explicitly records the best result across epochs ('We save and evaluate the model for every epoch, and the best results are recorded'). This post-hoc selection inflates scores and obscures trends. Small differences such as −0.4, +0.4, and +1.8 in Figure 4 are interpreted as saturation or non-improvement. Please report at least 3 seeds with standard deviations, and either justify best-epoch selection with a validation-based stopping rule or report the final-epoch results for the main trend. Also, Table 5 shows non-monotonic fluctuations in Route Plan and other tasks; a formal trend analysis or curve-fit with confidence intervals would strengthen the 'diminishing returns' claim.
  3. [Table 1 / Sec. 2.1] The human evaluation on MulSeT reports only 51.25% accuracy for Azimuth Transfer. This low human ceiling suggests that the task may contain ambiguous or under-specified items, so the 'low upper bound for spatial imagination' may reflect benchmark properties rather than a fixed model ceiling. Please provide per-item human agreement (e.g., majority-vote consistency or inter-annotator agreement), and report model accuracy on the subset of items with high human consensus. If the model still performs poorly on unambiguous items, the architectural conclusion is strengthened; if not, the benchmark's difficulty should be reinterpreted.
minor comments (4)
  1. [Throughout] Minor naming inconsistencies: 'LLaV A-OV-7B' and 'LLaV A-OneVision' are used interchangeably; please standardize. Also, some abbreviations in table captions (e.g., 'txy', 'hw') are explained only in the main text; adding a compact legend to each table would improve readability.
  2. [Sec. 3.2] The 'Model size matters' paragraph is under-specified: it mentions SpaceR-151k and a consistent subset of 10,000 samples but does not give LoRA hyperparameters or training epochs for the 3B/32B/72B runs. Please add these details to the appendix.
  3. [Fig. 7 / Sec. 5.1] The attention visualization analysis is qualitative. Consider adding a quantitative metric, such as attention concentration or overlap with the relevant objects, to support the claim that explicit CoT diffuses attention.
  4. [Sec. 3.1] The phrase 'performance converges quickly' would benefit from a formal saturation criterion (e.g., slope below a threshold over a window) rather than a visual inspection of the curves.

Circularity Check

0 steps flagged

No significant circularity: the paper's central claims are empirical results from held-out and external benchmarks, not consequences of the benchmark construction or of fitted parameters.

full rationale

The paper's two main conclusions (data scaling gives diminishing returns; visual-encoder positional encoding matters more than language-model positional encoding) are derived from controlled experiments rather than by construction. Data scaling is measured on a held-out split of the newly introduced MulSeT benchmark and on the external VSI-Bench, with fine-tuning starting from an untuned baseline; no free parameter is fitted to the test set, and the conclusion is not baked into the benchmark definition. The architectural claim is supported by interventions on model weights (masking/shuffling positional-encoding dimensions) evaluated on external single-view benchmarks and the MulSeT held-out set, with the non-spatial VG-Attribution task serving as a control; the comparison between VE and LLM ablations is an empirical contrast, even if the ablation protocol has interpretation caveats. The directional ablation in Appendix A.3 is a sanity check rather than a derivation of the main result. The paper contains no load-bearing self-citation: the cited attention-visualization methodology is not by the present authors, and no uniqueness theorem or ansatz is smuggled in via prior work. The reader's concern that the PE ablations may not perfectly isolate spatial information is a validity and interpretation issue, not a circularity in the sense of the target result being equivalent to the input by definition. Therefore, no circular step can be exhibited with a specific reduction, and the honest finding is no significant circularity.

Axiom & Free-Parameter Ledger

5 free parameters · 3 axioms · 0 invented entities

No new physical or mathematical entities are postulated. The load-bearing assumptions are about measurement validity: that synthetic QA accuracy captures spatial understanding, and that PE ablations selectively remove spatial information. Both are partially contradicted by the paper's own controls and human baselines.

free parameters (5)
  • LoRA rank and alpha = rank 8, alpha 32 for multi-view; rank 256, alpha 512 for video
    Training hyperparameters chosen by hand. They control the scaling curves and are not fitted to the target result, but they affect where saturation appears.
  • Learning rate and epochs = 1e-5, 10 epochs for multi-view; 5e-6, 1 epoch for video
    Hand-chosen training settings that influence whether performance plateaus at a low ceiling rather than continuing to improve.
  • Best-epoch selection = per-run chosen epoch
    The authors save and evaluate every epoch and record the best results, effectively selecting a free epoch parameter that maximizes scores and confounds the scaling analysis.
  • MulSeT azimuth angle threshold = 15 degrees
    Construction filter that enforces angular separation for azimuth transfer; chosen by the authors and affects task difficulty.
  • MulSeT min area ratio = not reported
    Visibility filter in the construction pipeline; the specific ratio is not given, which affects which objects appear in questions and thus task difficulty.
axioms (3)
  • domain assumption Multiple-choice QA on synthetic rendered images measures spatial understanding.
    The data-centric and benchmark conclusions treat MulSeT accuracy as a proxy for spatial reasoning. Human baselines are low, especially Azimuth Transfer at 51.25%, so the benchmark itself limits the 'low upper bound' interpretation.
  • ad hoc to paper Perturbing positional indices in a dimension removes only that position-information channel.
    The Mask/Shuffle/Constant ablation interpretation in Sec 4.2 depends on this. Table 2 shows Shuffle-LLM-txy(*) collapses even non-spatial VG-Attribution to near zero, indicating non-selective disruption.
  • domain assumption LoRA fine-tuning with best-epoch selection isolates the effect of data quantity on spatial ability.
    Used in Sec 3.1. Best-epoch selection and differing LoRA configurations across multi-view and video settings confound the scaling curves, so observed saturation may not be purely due to data quantity.

reviewed 2026-08-05 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Why Do MLLMs Struggle with Spatial Understanding? A Systematic Analysis from Data to Architecture." pith.science (2026). https://pith.science/paper/7LNV4V7U

@misc{pith2026250902359,
  author       = {Pith},
  title        = {Pith review of: Why Do MLLMs Struggle with Spatial Understanding? A Systematic Analysis from Data to Architecture},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7LNV4V7U}},
  note         = {Machine review of arXiv:2509.02359}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Spatial understanding is essential for Multimodal Large Language Models (MLLMs) to support perception, reasoning, and planning in embodied environments. Despite recent progress, existing studies reveal that MLLMs still struggle with spatial understanding. However, existing research lacks a comprehensive and systematic evaluation of these limitations, often restricted to isolated scenarios, such as single-view or video. In this work, we present a systematic analysis of spatial understanding from both data and architectural perspectives across three representative scenarios: single-view, multi-view, and video. We propose a benchmark named MulSeT (Multi-view Spatial Understanding Tasks), and design a series of experiments to analyze the spatial reasoning capabilities of MLLMs. From the data perspective, the performance of spatial understanding converges quickly as the training data increases, and the upper bound is relatively low, especially for tasks that require spatial imagination. This indicates that merely expanding training data is insufficient to achieve satisfactory performance. From the architectural perspective, we find that spatial understanding relies more heavily on the positional encoding within the visual encoder than within the language model, in both cascaded and native MLLMs. Moreover, we explore reasoning injection and envision future improvements through architectural design to optimize spatial understanding. These insights shed light on the limitations of current MLLMs and suggest new directions for improving spatial reasoning capabilities through data scaling and architectural tuning.

Figures

Figures reproduced from arXiv: 2509.02359 by Helu Zhi, Jiajun Zhang, Jingjing Huang, Shuo Ren, Wang Xu, Wanyue Zhang, Yangbin Xu, Yibin Huang.

Figure 1
Figure 1. Figure 1: Comparison of MLLMs and human performance [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overview of our study on spatial reasoning in MLLMs. The terms t, x, y, h, and w denote different dimensions of [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Overview of MulSeT. is currently a lack of high-quality benchmarks tailored for multi-view spatial understanding tasks (Yeh et al. 2025; Jia et al. 2025), which are crucial for assessing models’ ability to integrate spatial cues across different viewpoints or com￾plementary images. To fill this gap, we introduce MulSeT, a simulation-based dataset designed to systematically eval￾uate MLLMs’ spatial understa… view at source ↗
Figure 4
Figure 4. Figure 4: Performance trends under increasing training data scales for multi-view scenarios. Here, [PITH_FULL_IMAGE:figures/full_fig_p004_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Performance trends under increasing training data [PITH_FULL_IMAGE:figures/full_fig_p004_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Scores for different models (Base Score + Fine [PITH_FULL_IMAGE:figures/full_fig_p005_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Case study of attention visualization. Each column [PITH_FULL_IMAGE:figures/full_fig_p006_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Case study of attention visualization. A.4 Comprehensive Attention Visualization This section provides a comprehensive, layer-by-layer visu￾alization of the model’s attention maps under different in￾ference methods, supplementing the targeted analysis pre￾sented in the main body of the paper. While the main text highlights the most illustrative layers (e.g., 21st and 27th), here we present the complete att… view at source ↗
Figure 9
Figure 9. Figure 9: Layer-wise attention visualization of image0 in Qwen2.5-VL-7B for the case input with the vanilla prompt. [PITH_FULL_IMAGE:figures/full_fig_p015_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Layer-wise attention visualization of image0 in Qwen2.5-VL-7B for the case input with the Implicit Stepwise [PITH_FULL_IMAGE:figures/full_fig_p015_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Layer-wise attention visualization of image0 in Qwen2.5-VL-7B for the case input with the Explicit Stepwise CoT [PITH_FULL_IMAGE:figures/full_fig_p016_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Layer-wise attention visualization of image0 in Qwen2.5-VL-7B for the case input with the Implicit Multi-view [PITH_FULL_IMAGE:figures/full_fig_p016_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Layer-wise attention visualization of image0 in Qwen2.5-VL-7B for the case input with the Explicit Multi-view CoT [PITH_FULL_IMAGE:figures/full_fig_p017_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: Layer-wise attention visualization of image1 in Qwen2.5-VL-7B for the case input with the vanilla prompt. [PITH_FULL_IMAGE:figures/full_fig_p017_14.png] view at source ↗
Figure 15
Figure 15. Figure 15: Layer-wise attention visualization of image1 in Qwen2.5-VL-7B for the case input with the Implicit Stepwise [PITH_FULL_IMAGE:figures/full_fig_p018_15.png] view at source ↗
Figure 16
Figure 16. Figure 16: Layer-wise attention visualization of image1 in Qwen2.5-VL-7B for the case input with the Explicit Stepwise CoT [PITH_FULL_IMAGE:figures/full_fig_p018_16.png] view at source ↗
Figure 17
Figure 17. Figure 17: Layer-wise attention visualization of image1 in Qwen2.5-VL-7B for the case input with the Implicit Multi-view [PITH_FULL_IMAGE:figures/full_fig_p019_17.png] view at source ↗
Figure 18
Figure 18. Figure 18: Layer-wise attention visualization of image1 in Qwen2.5-VL-7B for the case input with the Explicit Multi-view CoT [PITH_FULL_IMAGE:figures/full_fig_p019_18.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 9 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. SSMNBench: Diagnosing Image-based Cross-View Human-Object Understanding via Single-View Sufficiency and Multi-View Necessity

    cs.CV 2026-06 unverdicted novelty 7.0

    SSMNBench shows that MLLMs suffer distraction degradation on single-view-sufficient tasks and fail to integrate geometric evidence across views, instead relying on semantic averaging and view preference.

  2. Why Far Looks Up: Probing Spatial Representation in Vision-Language Models

    cs.CV 2026-05 conditional novelty 7.0

    VLMs exhibit consistent vertical-distance entanglement in embeddings from perspective bias in natural images, producing accuracy gaps that a new synthetic benchmark SpatialTunnel exposes as model-intrinsic.

  3. MPDocBench-Parse: Benchmarking Practical Multi-page Document Parsing

    cs.AI 2026-05 unverdicted novelty 7.0

    MPDocBench-Parse provides a 3,246-page benchmark and evaluation protocol for multi-page document parsing that tests text/table/formula extraction, merging, figure handling, reading order, and heading hierarchy.

  4. Why MLLMs Struggle to Determine Object Orientations

    cs.CV 2026-04 accept novelty 7.0

    Orientation information is recoverable from MLLM visual encoder embeddings via linear regression, contradicting the hypothesis that failures originate in the encoders.

  5. SCP: Spatial Causal Prediction in Video

    cs.CV 2026-03 unverdicted novelty 7.0

    SCP defines a new benchmark task for predicting spatial causal outcomes beyond direct observation and shows that 23 leading models lag far behind humans on it.

  6. Self in Space: Benchmarking Self-Awareness and Spatial Cognition in UAV Embodied Intelligence

    cs.CV 2026-07 unverdicted novelty 6.0

    SIS-Bench shows MLLMs lag on UAV self-awareness versus spatial tasks, and motion-aware optical-flow fusion improves perception and memory.

  7. Self in Space: Benchmarking Self-Awareness and Spatial Cognition in UAV Embodied Intelligence

    cs.CV 2026-07 conditional novelty 6.0

    SIS-Bench, a 4,856-question UAV benchmark, shows video MLLMs are weaker at self-awareness than spatial cognition, and optical-flow fusion improves perception and memory.

  8. SVoT: State-aware Visualization-of-Thought for Spatial Reasoning via Reinforcement Learning

    cs.AI 2026-06 unverdicted novelty 6.0

    SVoT uses RL with GRPO to train MLLMs on interleaved textual and visual reasoning chains for multi-hop spatial tasks, achieving up to 65% accuracy gains on new domains with quantitative state verification.

  9. MPDocBench-Parse: Benchmarking Practical Multi-page Document Parsing

    cs.AI 2026-05 unverdicted novelty 6.0

    MPDocBench-Parse provides 433 annotated multi-page documents and an evaluation protocol covering text/table/formula extraction, merging, figure extraction, reading order, and heading hierarchy for realistic document parsing.

Reference graph

Works this paper leans on

49 extracted references · 18 canonical work pages · cited by 7 Pith papers · 1 internal anchor

  1. [1]

    L.; Almeida, D.; Altenschmidt, J.; Altman, S.; Anadkat, S.; et al

    Achiam, J.; Adler, S.; Agarwal, S.; Ahmad, L.; Akkaya, I.; Aleman, F. L.; Almeida, D.; Altenschmidt, J.; Altman, S.; Anadkat, S.; et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774

    Pith/arXiv arXiv 2023

  2. [2]

    Alayrac, J.-B.; Donahue, J.; Luc, P.; Miech, A.; Barr, I.; Hasson, Y.; Lenc, K.; Mensch, A.; Millican, K.; Reynolds, M.; et al. 2022. Flamingo: a visual language model for few-shot learning. Neural Information Processing Systems

    work page 2022

  3. [3]

    Bai, S.; Chen, K.; Liu, X.; Wang, J.; Ge, W.; Song, S.; Dang, K.; Wang, P.; Wang, S.; Tang, J.; Zhong, H.; Zhu, Y.; Yang, M.; Li, Z.; Wan, J.; Wang, P.; Ding, W.; Fu, Z.; Xu, Y.; Ye, J.; Zhang, X.; Xie, T.; Cheng, Z.; Zhang, H.; Yang, Z.; Xu, H.; and Lin, J. 2025. Qwen2.5-VL Technical Report. arXiv:2502.13923

    Pith/arXiv arXiv 2025

  4. [4]

    C.; Geva, M.; He, J.; Wu, J.; and Li, M

    Chen, S.; Zhu, T.; Zhou, R.; Zhang, J.; Gao, S.; Niebles, J. C.; Geva, M.; He, J.; Wu, J.; and Li, M. 2025. Why Is Spatial Reasoning Hard for VLMs? An Attention Mechanism Perspective on Focus Areas. In Proceedings of the International Conference on Machine Learning

    work page 2025

  5. [5]

    F.; and Chai, J

    Chen, X.; Ma, Z.; Zhang, X.; Xu, S.; Qian, S.; Yang, J.; Fouhey, D. F.; and Chai, J. 2024. Multi-Object Hallucination in Vision-Language Models. arXiv:2407.06192

    Pith/arXiv arXiv 2024

  6. [6]

    Cheng, Z.; Tu, Y.; Li, R.; Dai, S.; Hu, J.; Hu, S.; Li, J.; Shi, Y.; Yu, T.; Chen, W.; Shi, L.; and Sun, M. 2025. EmbodiedEval: Evaluate Multimodal LLMs as Embodied Agents. In The Annual Meeting of the Association for Computational Linguistics

    work page 2025

  7. [7]

    Colan, J.; Davila, A.; and Hasegawa, Y. 2025. Assessing the Value of Visual Input: A Benchmark of Multimodal Large Language Models for Robotic Path Planning. arXiv:2507.12391

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  8. [8]

    Dai, W.; Li, J.; Li, D.; Tiong, A. M. H.; Zhao, J.; Wang, W.; Li, B.; Fung, P.; and Hoi, S. 2023. InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning. arXiv:2305.06500

    Pith/arXiv arXiv 2023

  9. [9]

    Diao, H.; Li, X.; Cui, Y.; Wang, Y.; Deng, H.; Pan, T.; Wang, W.; Lu, H.; and Wang, X. 2025. EVEv2: Improved Baselines for Encoder-Free Vision-Language Models. arXiv preprint arXiv:2502.06788

    Pith/arXiv arXiv 2025

  10. [10]

    Dosovitskiy, A.; Beyer, L.; Kolesnikov, A.; Weissenborn, D.; Zhai, X.; Unterthiner, T.; Dehghani, M.; Minderer, M.; Heigold, G.; Gelly, S.; Uszkoreit, J.; and Houlsby, N. 2021. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. arXiv:2010.11929

    Pith/arXiv arXiv 2021

  11. [11]

    Du, M.; Wu, B.; Li, Z.; Huang, X.; and Wei, Z. 2024. E mb S patial-Bench: Benchmarking Spatial Understanding for Embodied Tasks with Large Vision-Language Models. In Ku, L.-W.; Martins, A.; and Srikumar, V., eds., Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), 346--355. Bangkok, Thailand: ...

    work page 2024

  12. [12]

    Fei, H.; Wu, S.; Ji, W.; Zhang, H.; Zhang, M.; Lee, M.-L.; and Hsu, W. 2024. Video-of-Thought: Step-by-Step Video Reasoning from Perception to Cognition. In Proceedings of the International Conference on Machine Learning

    work page 2024

  13. [13]

    Feng, K.; Gong, K.; Li, B.; Guo, Z.; Wang, Y.; Peng, T.; Wu, J.; Zhang, X.; Wang, B.; and Yue, X. 2025. Video-R1: Reinforcing Video Reasoning in MLLMs. arXiv:2503.21776

    Pith/arXiv arXiv 2025

  14. [14]

    Heo, B.; Park, S.; Han, D.; and Yun, S. 2024. Rotary position embedding for vision transformer. In European Conference on Computer Vision, 289--305. Springer

    work page 2024

  15. [15]

    J.; Shen, Y.; Wallis, P.; Allen-Zhu, Z.; Li, Y.; Wang, S.; Wang, L.; Chen, W.; et al

    Hu, E. J.; Shen, Y.; Wallis, P.; Allen-Zhu, Z.; Li, Y.; Wang, S.; Wang, L.; Chen, W.; et al. 2022. Lora: Low-rank adaptation of large language models. International Conference on Learning Representations, 1(2): 3

    work page 2022

  16. [16]

    P.; Perelman, A.; Ramesh, A.; Clark, A.; Ostrow, A.; Welihinda, A.; Hayes, A.; Radford, A.; et al

    Hurst, A.; Lerer, A.; Goucher, A. P.; Perelman, A.; Ramesh, A.; Clark, A.; Ostrow, A.; Welihinda, A.; Hayes, A.; Radford, A.; et al. 2024. GPT-4o System Card. arXiv preprint arXiv:2410.21276

    Pith/arXiv arXiv 2024

  17. [17]

    Jia, M.; Qi, Z.; Zhang, S.; Zhang, W.; Yu, X.; He, J.; Wang, H.; and Yi, L. 2025. OmniSpatial: Towards Comprehensive Spatial Reasoning Benchmark for Vision Language Models. arXiv:2506.03135

    arXiv 2025

  18. [18]

    Well, Keep Thinking

    Jin, H.; Yeom, J. W.; Bae, S.; and Kim, T. 2025. "Well, Keep Thinking": Enhancing LLM Reasoning with Adaptive Injection Decoding. arXiv:2503.10167

    Pith/arXiv arXiv 2025

  19. [19]

    Kamath, A.; Hessel, J.; and Chang, K.-W. 2023. What ' s ``up'' with vision-language models? Investigating their struggle with spatial reasoning. In Bouamor, H.; Pino, J.; and Bali, K., eds., Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, 9161--9175. Singapore: Association for Computational Linguistics

    work page 2023

  20. [20]

    Kolve, E.; Mottaghi, R.; Han, W.; VanderBilt, E.; Weihs, L.; Herrasti, A.; Deitke, M.; Ehsani, K.; Gordon, D.; Zhu, Y.; et al. 2017. Ai2-thor: An interactive 3d environment for visual ai. arXiv preprint arXiv:1712.05474

    Pith/arXiv arXiv 2017

  21. [21]

    A.; et al

    Krishna, R.; Zhu, Y.; Groth, O.; Johnson, J.; Hata, K.; Kravitz, J.; Chen, S.; Kalantidis, Y.; Li, L.-J.; Shamma, D. A.; et al. 2017. Visual genome: Connecting language and vision using crowdsourced dense image annotations. International journal of computer vision, 123(1): 32--73

    work page 2017

  22. [22]

    Li, B.; Zhang, Y.; Guo, D.; Zhang, R.; Li, F.; Zhang, H.; Zhang, K.; Li, Y.; Liu, Z.; and Li, C. 2024. Llava-onevision: Easy visual task transfer. arXiv preprint arXiv:2408.03326

    Pith/arXiv arXiv 2024

  23. [23]

    Li, C.; Wu, W.; Zhang, H.; Xia, Y.; Mao, S.; Dong, L.; Vulić, I.; and Wei, F. 2025. Imagine while Reasoning in Space: Multimodal Visualization-of-Thought. In Proceedings of the International Conference on Machine Learning

    work page 2025

  24. [24]

    Li, X.; Zhang, M.; Geng, Y.; Geng, H.; Long, Y.; Shen, Y.; Zhang, R.; Liu, J.; and Dong, H. 2023. ManipLLM: Embodied Multimodal Large Language Model for Object-Centric Robotic Manipulation. arXiv:2312.16217

    Pith/arXiv arXiv 2023

  25. [25]

    Liao, Z.; Xie, Q.; Zhang, Y.; Kong, Z.; Lu, H.; Yang, Z.; and Deng, Z. 2025. Improved Visual-Spatial Reasoning via R1-Zero-Like Training. arXiv:2504.00883

    Pith/arXiv arXiv 2025

  26. [26]

    Lin, T.; Li, G.; Zhong, Y.; Zou, Y.; and Zhao, B. 2025. Evo-0: Vision-Language-Action Model with Implicit Spatial Understanding. arXiv preprint arXiv:2507.00416

    arXiv 2025

  27. [27]

    Lin, T.-Y.; Maire, M.; Belongie, S.; Hays, J.; Perona, P.; Ramanan, D.; Doll \'a r, P.; and Zitnick, C. L. 2014. Microsoft coco: Common objects in context. In European conference on computer vision, 740--755. Springer

    2014

  28. [28]

    Liu, H.; Li, C.; Wu, Q.; and Lee, Y. J. 2023. Visual instruction tuning. Advances in neural information processing systems, 36: 34892--34916

    2023

  29. [29]

    Liu, H.; Yan, W.; Zaharia, M.; and Abbeel, P. 2024. World model on million-length video and language with ringattention. arXiv preprint arXiv:2402.08268

    Pith/arXiv arXiv 2024

  30. [30]

    Liu, Y.; Chi, D.; Wu, S.; Zhang, Z.; Hu, Y.; Zhang, L.; Zhang, Y.; Wu, S.; Cao, T.; Huang, G.; Huang, H.; Tian, G.; Qiu, W.; Quan, X.; Hao, J.; and Zhuang, Y. 2025. SpatialCoT: Advancing Spatial Reasoning through Coordinate Alignment and Chain-of-Thought for Embodied Task Planning. arXiv:2501.10074

    Pith/arXiv arXiv 2025

  31. [31]

    Luo, G.; Yang, X.; Dou, W.; Wang, Z.; Liu, J.; Dai, J.; Qiao, Y.; and Zhu, X. 2025. Mono-internvl: Pushing the boundaries of monolithic multimodal large language models with endogenous visual pre-training. In Proceedings of the Computer Vision and Pattern Recognition Conference, 24960--24971

    work page 2025

  32. [32]

    Ouyang, K.; Liu, Y.; Wu, H.; Liu, Y.; Zhou, H.; Zhou, J.; Meng, F.; and Sun, X. 2025. SpaceR: Reinforcing MLLMs in Video Spatial Reasoning. arXiv:2504.01805

    Pith/arXiv arXiv 2025

  33. [33]

    W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agarwal, S.; Sastry, G.; Askell, A.; Mishkin, P.; Clark, J.; Krueger, G.; and Sutskever, I

    Radford, A.; Kim, J. W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agarwal, S.; Sastry, G.; Askell, A.; Mishkin, P.; Clark, J.; Krueger, G.; and Sutskever, I. 2021. Learning Transferable Visual Models From Natural Language Supervision. arXiv:2103.00020

    Pith/arXiv arXiv 2021

  34. [34]

    Su, J.; Ahmed, M.; Lu, Y.; Pan, S.; Bo, W.; and Liu, Y. 2024. Roformer: Enhanced transformer with rotary position embedding. Neurocomputing, 568: 127063

    work page 2024

  35. [35]

    Tong, S.; Liu, Z.; Zhai, Y.; Ma, Y.; LeCun, Y.; and Xie, S. 2024. Eyes wide shut? exploring the visual shortcomings of multimodal llms. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 9568--9578

    work page 2024

  36. [36]

    V.; Zhou, D.; et al

    Wei, J.; Wang, X.; Schuurmans, D.; Bosma, M.; Xia, F.; Chi, E.; Le, Q. V.; Zhou, D.; et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35: 24824--24837

    2022

  37. [37]

    Wei, X.; Liu, X.; Zang, Y.; Dong, X.; Zhang, P.; Cao, Y.; Tong, J.; Duan, H.; Guo, Q.; Wang, J.; Qiu, X.; and Lin, D. 2025. VideoRoPE: What Makes for Good Video Rotary Position Embedding? arXiv:2502.05173

    Pith/arXiv arXiv 2025

  38. [38]

    Wu, D.; Liu, F.; Hung, Y.-H.; and Duan, Y. 2025 a . Spatial-MLLM: Boosting MLLM Capabilities in Visual-based Spatial Intelligence. arXiv:2505.23747

    Pith/arXiv arXiv 2025

  39. [39]

    Wu, Q.; Yang, X.; Zhou, Y.; Fang, C.; Song, B.; Sun, X.; and Ji, R. 2025 b . Grounded Chain-of-Thought for Multimodal Large Language Models. arXiv:2503.12799

    Pith/arXiv arXiv 2025

  40. [40]

    Xiong, C.; Shen, C.; Li, X.; Zhou, K.; Liu, J.; Wang, R.; and Dong, H. 2024. AIC MLLM: Autonomous Interactive Correction MLLM for Robust Robotic Manipulation. In The Conference on Robot Learning

    work page 2024

  41. [41]

    J.; Tao, X.; and Wang, F

    Xu, L.; Xie, H.; Qin, S.-Z. J.; Tao, X.; and Wang, F. L. 2023. Parameter-efficient fine-tuning methods for pretrained language models: A critical review and assessment. arXiv preprint arXiv:2312.12148

    Pith/arXiv arXiv 2023

  42. [42]

    W.; Han, R.; Fei-Fei, L.; and Xie, S

    Yang, J.; Yang, S.; Gupta, A. W.; Han, R.; Fei-Fei, L.; and Xie, S. 2025. Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spaces. arXiv:2412.14171

    Pith/arXiv arXiv 2025

  43. [43]

    Yeh, C.-H.; Wang, C.; Tong, S.; Cheng, T.-Y.; Wang, R.; Chu, T.; Zhai, Y.; Chen, Y.; Gao, S.; and Ma, Y. 2025. Seeing from Another Perspective: Evaluating Multi-View Understanding in MLLMs. In Neural Information Processing Systems

    work page 2025

  44. [44]

    Zhang, J.; Khayatkhoei, M.; Chhikara, P.; and Ilievski, F. 2025. MLLM s Know Where to Look: Training-free Perception of Small Visual Details with Multimodal LLM s. In The Thirteenth International Conference on Learning Representations

    work page 2025

  45. [45]

    Zheng, D.; Huang, S.; and Wang, L. 2025. Video-3D LLM: Learning Position-Aware Video Representation for 3D Scene Understanding. arXiv:2412.00493

    Pith/arXiv arXiv 2025

  46. [46]

    Zhou, X.; He, J.; Ke, Y.; Zhu, G.; Gutierrez Basulto, V.; and Pan, J. 2024. An Empirical Study on Parameter-Efficient Fine-Tuning for M ulti M odal Large Language Models. In Ku, L.-W.; Martins, A.; and Srikumar, V., eds., Findings of the Association for Computational Linguistics, 10057--10084. Bangkok, Thailand: Association for Computational Linguistics

    work page 2024

  47. [47]

    Zhu, J.; Wang, W.; Chen, Z.; Liu, Z.; Ye, S.; Gu, L.; Tian, H.; Duan, Y.; Su, W.; Shao, J.; Gao, Z.; Cui, E.; Wang, X.; Cao, Y.; Liu, Y.; Wei, X.; Zhang, H.; Wang, H.; Xu, W.; Li, H.; Wang, J.; Deng, N.; Li, S.; He, Y.; Jiang, T.; Luo, J.; Wang, Y.; He, C.; Shi, B.; Zhang, X.; Shao, W.; He, J.; Xiong, Y.; Qu, W.; Sun, P.; Jiao, P.; Lv, H.; Wu, L.; Zhang, ...

    Pith/arXiv arXiv 2025

  48. [48]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

  49. [49]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

This paper was first reviewed by deepseek-v4-flash on August 5, 2026.