Pith. sign in

REVIEW 4 major objections 7 minor 43 references

Enhancing Spatial Reasoning through Visual and Textual Thinking

T0 review · 4 major / 7 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read SpatialVTS trains a vision-language model to first locate helpful regions, then reason over them in text, lifting average spatial-benchmark accuracy to 82.03 percent without depth or mask inputs.

desk verdict A plausible training recipe and a real data-cleaning effort, but the headline gains over baselines are unverified because the paper never documents whether baselines were scored under the same LLM-judge protocol. read the letter →

arxiv 2507.20529 v1 pith:S3IM6H3B submitted 2025-07-28 cs.CV cs.AI

Xun Liang , Xin Guo , Zhongming Jin , Weihang Pan , Penghui Shang , Deng Cai , Binbin Lin , Jieping Ye This is my paper · ORCID
classification cs.CVcs.AI
keywords spatialreasoningvisionlanguagemodelsregiontokenschain-of-thoughtdatasetreconstructionvisualthinkingtextualVQA
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that spatial reasoning in vision-language models can be substantially improved without feeding in depth maps or segmentation masks: instead, the model is trained to think twice, first by emitting location tokens for both obvious and hidden reference targets, then by producing step-by-step textual reasoning before the answer. The authors build this capability by reconstructing the SpatialRGPT and VPT training data, replacing mask and depth placeholders with plain region labels, manually fixing incorrect labels, and generating rationales. On five qualitative benchmarks the trained model averages 82.03 percent, above general VLMs and dedicated spatial models, and on quantitative distance benchmarks it lands near SpatialRGPT, which trains with extra depth and mask inputs. A sympathetic reading is that explicit visual cue discovery plus textual reasoning over those cues is what carries the gain.

What carries the argument

The load-bearing mechanism is a two-phase generation loop. First, spatial visual thinking: the model divides the image into an 8x8 grid and outputs special tokens such as <x_3><y_6> for the top-left and bottom-right cells of each target region, covering both objects named in the question and potential reference objects. Second, spatial textual thinking: the selected regions are cropped, passed through the same vision encoder, and the model must reason step by step from those visual cues before giving the final answer. Around this loop, the paper reconstructs the training data so that region labels replace mask and depth placeholders, incorrect labels are manually corrected, and a long reasoning process is generated by a 'seeking the cause by grasping the result' strategy, which feeds the problem, visual cues, and answer into a public VLM to produce a correct rationale.

What would settle it

Re-run all baselines under the exact protocol, using the same judge, same threshold, same answer prompt, and same $\delta \le 2$ acceptance, then compare the averages; if the gap shrinks or reverses, the reported advantage is an artifact of scoring rather than spatial reasoning. A judge-free check would replace the LLM grader with exact-match or human scoring on a random subset.

Watch

Extended reading notes

Core claim

The central claim is that a VLM trained with SpatialVTS reaches better spatial understanding than existing open models using only RGB images and text. During spatial visual thinking, the model emits discrete region tokens (cell indices on an 8x8 grid) for the objects in the question and for additional potential reference objects; during spatial textual thinking, those crops are re-encoded and the model must produce a rationale and then the answer. The paper reports an average score of 82.03 across CoT, VSR, SRGPT-QUAL, WhatsUP, and BLINK-Spatial, compared with 74.47 for VPT, 67.26 for Qwen2.5-VL, 59.00 for SpatialRGPT, and 57.55 for SpaceThinker, and near-parity with SpatialRGPT on quantitative benchmarks. It further claims the reconstructed dataset and the chain-of-thought component are responsible for the gains: removing the textual thinking drops accuracy by 3 to 7 percentage points, and a 250k-sample subset already rivals VPT.

Load-bearing premise

The load-bearing premise is that the baseline scores in Table 1 and Fig. 6 were obtained with the same evaluation protocol as SpatialVTS, namely the DeepSeek-V3-0324 judge at a 0.5 threshold for qualitative tasks and the $\delta \le 2$ rule for quantitative ones; the paper does not state that the baselines were re-run under this protocol.

Editorial extensions

If this is right

  • Spatial reasoning can be improved with only RGB input, so depth sensors or mask annotations are not prerequisites for VLM spatial competence.
  • The explicit emission of potential-reference-target tokens gives a mechanism by which a model can use everyday reference scales, such as a building's height or a TV's width, to answer distance questions.
  • Training with textual rationales is directly measurable: adding CoT raises performance by about 3 points on SRGPT-QUAN at $\delta \le 1.25$ and 7 points on Q-Spatial++ at $\delta \le 2$.
  • A much smaller reconstructed dataset of 250k samples is enough to match VPT's level, suggesting that data quality and input format may matter more than raw training scale.
  • The same two-phase recipe could transfer to other perception tasks that need measuring or comparing objects, not just spatial VQA.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the gain really comes from naming potential reference objects, then a testable extension is to evaluate whether the model's region tokens are themselves interpretable: removing the 'potential' targets and keeping only question-mentioned objects should close most of the gap.
  • The reported numbers may depend on the DeepSeek-V3-0324 judge accepting verbose, well-structured answers; a judge-free or human-scored replication would tell whether the CoT format inflates qualitative scores.
  • The near-parity with SpatialRGPT on quantitative tasks, despite no depth, suggests that reference-object scales can substitute for metric depth in many indoor scenes; applying the method to outdoor or aerial images, where reference scales are rarer, would stress-test the mechanism.
  • The manual dataset correction step is expensive; a follow-up could automate label correction and rationale generation while keeping the same input format, testing whether the gains survive without human cleaning.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper introduces SpatialVTS, a two-stage training pipeline for enhancing spatial reasoning in vision-language models (VLMs). In the spatial visual thinking stage, the model is trained to emit discrete region-location tokens identifying both explicitly mentioned and potentially informative objects. In the spatial textual thinking stage, the model receives the cropped candidate regions as visual cues and is trained to produce step-by-step rationales before the final answer. The authors reconstruct a training set from the SpatialRGPT and VPT datasets by manually correcting labels, replacing mask/depth placeholders with region labels, and generating CoT rationales via a 'seeking the cause by grasping the result' strategy that takes the ground-truth answer as input. They fine-tune Qwen2-VL-7B and report results on VSR, WhatsUP, BLINK-spatial, SRGPT-QUAL, and quantitative benchmarks, claiming a significant average improvement over existing open VLMs and spatial-reasoning-specific models without using masks or depth.

Significance. If the empirical results hold, the paper provides a practical and scalable recipe for improving spatial reasoning in open VLMs using only RGB images and text. The combination of discrete region tokens with CoT rationales is a sensible synthesis of existing ideas, and the manually corrected dataset with rationales is a potentially valuable community resource. The paper's explicit focus on 'potential objects' beyond those named in the question is a useful insight, and the ablations on text-thinking and data scale help isolate the contributions. The design also avoids the need for depth sensors or segmentation masks, which broadens applicability. However, the central claim of significance depends on the comparability of the baseline numbers and on the statistical reliability of the reported advantages, both of which are currently not established.

major comments (4)
  1. [Section 4.1, Metrics; Table 1] The evaluation protocol uses an LLM judge (DeepSeek-V3-0324) that scores each answer in [0,1] with acceptance above 0.5. The manuscript never states whether the baseline numbers in Table 1 and Fig. 6 were obtained by re-running all baselines under this exact protocol or are quoted from the original publications. This is load-bearing: different judges, different thresholds, or exact-match criteria can shift scores by several points, and the reported gaps (e.g., 82.03 vs. 57.55 average) are much larger than the ablation gains in Table 2. The authors must re-evaluate every baseline with the same judge, the same prompt template, and the same threshold, and disclose the judge prompt and version. They should also report a small human-validation subset to show the judge is not systematically biased toward SpatialVTS's answer style, which is a real risk because the model was trained on rationales that resemble the judge's expected output format.
  2. [Tables 1-3 and Fig. 6] No error bars, confidence intervals, or significance tests are reported anywhere. The ablation study in Table 2 shows text thinking contributes about 3-7 points on the quantitative benchmarks, and Table 3 shows an 8-point difference from data scale on VSR. Without variance estimates, statements such as 'significantly improved' and 'almost on par' are not supported. Please report standard errors or bootstrap intervals across benchmark items, and if the LLM judge is stochastic, repeat scoring multiple times and report the judge's variance. This is necessary to assess whether the reported margins exceed noise, especially given the modest sizes of some of the qualitative benchmarks.
  3. [Section 3.2, Fig. 4] The rationale generation strategy 'seeking the cause by grasping the result' feeds the ground-truth answer into the large model when constructing the reasoning process. This means the rationales are post-hoc justifications constrained to end at the correct answer, not independent derivations. The paper should acknowledge this limitation explicitly and provide evidence about the faithfulness of the rationales, for example a human evaluation of whether the rationales correspond to the visual evidence actually used by the model. Without this, it is unclear whether the improved accuracy comes from genuine spatial reasoning or from the model learning to produce confident-sounding text that the LLM judge accepts.
  4. [Section 4.1 and full paper] The manuscript states 'We promise to publish our model and datasets after the review is completed.' The manually corrected dataset and the region/CoT annotations are central to the method and to verifying the claimed improvements. Since the paper is currently under review, this is acceptable as a condition, but the final version should commit to a concrete release plan (e.g., model weights, dataset, and evaluation scripts) and should additionally specify which public VLM was used to generate the reasoning processes and which manual annotation guidelines were followed.
minor comments (7)
  1. [Table 1] The table formatting is corrupted: entries such as '79.0058.45' and '69.1785.3175.96' should be split into separate numbers (e.g., '79.00 58.45' and '69.17 85.31 75.96'). Please also verify the row/column alignment for VPT and SpatialVTS.
  2. [Section 2.1 heading] The heading contains a typo: 'Benckmarks' should be 'Benchmarks'.
  3. [Fig. 5] The corrected answer example contains a typo: 'Inorrect' should be 'Incorrect'.
  4. [Section 4.2] 'SRPGTBench' appears to be a typo for 'SRGPT-Bench'.
  5. [Section 5] The conclusion refers to the 'Visual Textual Thinking' phase, whereas the body consistently uses 'Spatial Textual Thinking'; please align the terminology.
  6. [Section 3.3 and References] The reference [43] for SpaceThinker is listed as 'Vqasynth' by Remyx AI, which does not appear to be the correct source for SpaceThinker. Please cite the actual SpaceThinker publication.
  7. [Section 3.1] The grid size k is a free parameter set to k=8 without sensitivity analysis. A brief ablation or justification for this choice would strengthen the paper.

Circularity Check

1 steps flagged · score 2.0 of 10

Answer-conditioned CoT generation creates a non-load-bearing circularity in the 'thinking' narrative; the benchmark comparison itself is external and not circular.

  1. other [Section 3.2 (Spatial Textual Thinking), Fig. 4(a); dataset construction Section 3.3]
    "In particular, we employ the ’seeking the cause by grasping the result’ strategy to ensure the rationality and correctness of the generating reasoning process. The question, visual cues, and answer are inputted into the large model simultaneously to find the correct reasoning path. During the training and inference phase, the model is trained to provide rationales and answers only based on the problem and visual cues."

    The rationales are generated by conditioning on the ground-truth answer, so the CoT supervision is a post-hoc justification of the label rather than an independent reasoning trace derived from visual cues alone. This makes the 'textual thinking' evidence partially self-confirming: the model is trained to reproduce answer-fitted rationales, so success on benchmarks cannot cleanly be attributed to genuinely emergent reasoning from visual cues. However, because evaluation is done on held-out benchmarks where the answer is not provided, the central performance comparison is not circular; this step is a methodological caveat, not a derivation-level equivalence.

full rationale

The central claim (improved average spatial reasoning without masks/depth) is tested against external benchmarks (WhatsUP, VSR, BLINK-spatial, SRGPT-QUAL, Q-Spatial++, SRGPT-QUAN). The region-token representation is explicitly inherited from VPT and the dataset is rebuilt from SpatialRGPT/VPT with manual corrections; no self-citation chain or imported uniqueness theorem is load-bearing. The single in-paper circular element is the 'seeking the cause by grasping the result' CoT construction (Sec. 3.2), where training rationales are produced with the answer as input. This is a known data-construction technique and does not by itself force the benchmark numbers, but it weakens the interpretive claim that the model's rationales demonstrate autonomous long-term thinking. The baseline-protocol uncertainty (whether Table 1 numbers were re-scored with the DeepSeek-V3-0324 judge and 0.5 threshold) is a correctness/comparability risk, not a circularity. Overall, no load-bearing circularity: score 2 reflects the minor answer-conditioned CoT generation rather than a circular central derivation.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The central claim rests on data curation and evaluation choices rather than a mathematical derivation. No fitted parameters are used in the main comparisons, but the grid resolution k=8 is a hand-chosen design parameter. The main load-bearing assumptions are the reliability of the AI judge, the correctness of manual data corrections, and the validity of the generated reasoning chains.

free parameters (1)
  • Grid size k for region location tokens = 8
    Hand-chosen, following VPT (Section 3.1). The image is divided into k by k cells; k=8 sets the resolution of the region tokens the model outputs. No sweep or data-dependent tuning is reported.
assumptions (4)
  • domain assumption The DeepSeek-V3-0324 judge correctly evaluates the correctness of free-form spatial answers with a 0.5 threshold.
    Section 4.1 Metrics: all qualitative benchmark scores are assigned by an AI judge; if the judge is biased toward the style or content of the model's answers, the accuracy numbers may not reflect true spatial understanding.
  • domain assumption Manually corrected labels and generated rationales are correct and improve training.
    Section 3.2-3.3: the training data is corrected and augmented with rationales generated by a public VLM using the answer as a prior. The paper does not provide inter-annotator agreement or verification that these rationales are valid derivations.
  • domain assumption The original SpatialRGPT and VPT datasets are suitable for sampling after correction, and the manual corrections fix errors without introducing new biases.
    Section 3.3: the paper claims a considerable proportion of answers are incorrect and many questions are uncorrelated; the corrected dataset is not released, so this cannot be independently checked.
  • domain assumption Performance on the selected benchmarks measures spatial reasoning.
    Section 4.2 acknowledges benchmarks are 'vague and biased' and answers can be guessed; still the central claim is based on these benchmark averages.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Enhancing Spatial Reasoning through Visual and Textual Thinking." pith.science (2026). https://pith.science/paper/S3IM6H3B

@misc{pith2026250720529,
  author       = {Pith},
  title        = {Pith review of: Enhancing Spatial Reasoning through Visual and Textual Thinking},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/S3IM6H3B}},
  note         = {Machine review of arXiv:2507.20529}
}
read the original abstract

The spatial reasoning task aims to reason about the spatial relationships in 2D and 3D space, which is a fundamental capability for Visual Question Answering (VQA) and robotics. Although vision language models (VLMs) have developed rapidly in recent years, they are still struggling with the spatial reasoning task. In this paper, we introduce a method that can enhance Spatial reasoning through Visual and Textual thinking Simultaneously (SpatialVTS). In the spatial visual thinking phase, our model is trained to generate location-related specific tokens of essential targets automatically. Not only are the objects mentioned in the problem addressed, but also the potential objects related to the reasoning are considered. During the spatial textual thinking phase, Our model conducts long-term thinking based on visual cues and dialogues, gradually inferring the answers to spatial reasoning problems. To effectively support the model's training, we perform manual corrections to the existing spatial reasoning dataset, eliminating numerous incorrect labels resulting from automatic annotation, restructuring the data input format to enhance generalization ability, and developing thinking processes with logical reasoning details. Without introducing additional information (such as masks or depth), our model's overall average level in several spatial understanding tasks has significantly improved compared with other models.

Figures

Figures reproduced from arXiv: 2507.20529 by the authors.

Figure 1
Figure 1. Our model analyzes the original problem and the image to find both the evident and potential [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. The framework of SpatialVTS. Our model contains two phases: the Spatial Visual Thinking [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Bounding box representation. We use the indices of the cells containing the top-left and [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Construct the reasoning process via ’seeking the cause by grasping the result’. [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: The original Q&A from SpatialRGPT and our corrected Q&A. [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: Performance comparison on quantitative spatial QA benchmarks. The white hatch means [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Some reasoning examples. The special regions are bounded with red boxes. SpatialVTS [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

43 extracted references · 8 canonical work pages

  1. [38]

    Relational inductive biases, deep learning, and graph networks.arXiv preprint arXiv:1806.01261, 2018

    Peter W Battaglia, Jessica B Hamrick, Victor Bapst, Alvaro Sanchez-Gonzalez, Vinicius Zambaldi, Mateusz Malinowski, Andrea Tacchetti, David Raposo, Adam Santoro, Ryan Faulkner, et al. Relational inductive biases, deep learning, and graph networks.arXiv preprint arXiv:1806.01261, 2018

    arXiv 2018

  2. [43]

    Vqasynth, 2023

    Remyx AI. Vqasynth, 2023. 12

    work page 2023

  3. [1]

    Spatialrgpt: Grounded spatial reasoning in vision language models.arXiv preprint arXiv:2406.01584, 2024

    An-Chieh Cheng, Hongxu Yin, Yang Fu, Qiushan Guo, Ruihan Yang, Jan Kautz, Xiaolong Wang, and Sifei Liu. Spatialrgpt: Grounded spatial reasoning in vision language models.arXiv preprint arXiv:2406.01584, 2024

    arXiv 2024

  4. [2]

    Danny Driess, Fei Xia, Mehdi S. M. Sajjadi, Corey Lynch, Aakanksha Chowdhery, Brian Ichter, Ayzaan Wahid, Jonathan Tompson, Quan Vuong, Tianhe Yu, Wenlong Huang, Yevgen Chebotar, Pierre Sermanet, Daniel Duckworth, Sergey Levine, Vincent Vanhoucke, Karol Hausman, Marc Toussaint, Klaus Greff, Andy Zeng, Igor Mordatch, and Pete Florence. Palm-e: An embodied ...

    arXiv 2023

  5. [3]

    Spatialvlm: Endowing vision-language models with spatial reasoning capabilities

    Boyuan Chen, Zhuo Xu, Sean Kirmani, Brain Ichter, Dorsa Sadigh, Leonidas Guibas, and Fei Xia. Spatialvlm: Endowing vision-language models with spatial reasoning capabilities. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14455–14465, 2024

    2024

  6. [4]

    Visual instruction tuning, 2023

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning, 2023

    2023

  7. [5]

    Improved baselines with visual instruction tuning, 2023

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning, 2023

    2023

  8. [6]

    Llava-next-interleave: Tackling multi-image, video, and 3d in large multimodal models

    Feng Li, Renrui Zhang, Hao Zhang, Yuanhan Zhang, Bo Li, Wei Li, Zejun Ma, and Chunyuan Li. Llava-next-interleave: Tackling multi-image, video, and 3d in large multimodal models. arXiv preprint arXiv:2407.07895, 2024

    arXiv 2024

Show all 43 references
  1. [7]

    Llava-onevision: Easy visual task transfer.arXiv preprint arXiv:2408.03326, 2024

    Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, et al. Llava-onevision: Easy visual task transfer.arXiv preprint arXiv:2408.03326, 2024

    2024 arXiv

  2. [8]

    Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond.arXiv preprint arXiv:2308.12966, 2023

    Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond.arXiv preprint arXiv:2308.12966, 2023

    2023 arXiv

  3. [9]

    Qwen2-vl: Enhancing vision- language model’s perception of the world at any resolution.arXiv preprint arXiv:2409.12191, 2024

    Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Yang Fan, Kai Dang, Mengfei Du, Xuancheng Ren, Rui Men, Dayiheng Liu, Chang Zhou, Jingren Zhou, and Junyang Lin. Qwen2-vl: Enhancing vision- language model’s p...

    2024 arXiv

  4. [10]

    Qwen2.5-vl technical report

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Han...

    2025 arXiv

  5. [11]

    Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling.arXiv preprint arXiv:2412.05271, 2024

    Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shen- glong Ye, Hao Tian, Zhaoyang Liu, et al. Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling.arXiv preprint arXiv:2412.05271, 2024

    2024 arXiv

  6. [12]

    How far are we to gpt-4v? closing the gap to commercial multimodal models with open-source suites.arXiv preprint arXiv:2404.16821, 2024

    Zhe Chen, Weiyun Wang, Hao Tian, Shenglong Ye, Zhangwei Gao, Erfei Cui, Wenwen Tong, Kongzhi Hu, Jiapeng Luo, Zheng Ma, et al. How far are we to gpt-4v? closing the gap to commercial multimodal models with open-source suites.arXiv preprint arXiv:2404.16821, 2024

    2024 arXiv

  7. [13]

    Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks

    Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al. Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks. InProceedings of the IEEE/CVF Conference on Computer Visi...

    2024

  8. [14]

    Gpt-4 technical report.arXiv preprint arXiv:2303.08774, 2023

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report.arXiv preprint arXiv:2303.08774, 2023. 10

    2023 arXiv

  9. [15]

    Gpt-4o system card.arXiv preprint arXiv:2410.21276, 2024

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card.arXiv preprint arXiv:2410.21276, 2024

    2024 arXiv

  10. [16]

    The llama 3 herd of models.arXiv preprint arXiv:2407.21783, 2024

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models.arXiv preprint arXiv:2407.21783, 2024

    2024 arXiv

  11. [17]

    Phi-3 technical report: A highly capable language model locally on your phone.arXiv preprint arXiv:2404.14219, 2024

    Marah Abdin, Jyoti Aneja, Hany Awadalla, Ahmed Awadallah, Ammar Ahmad Awan, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Jianmin Bao, Harkirat Behl, et al. Phi-3 technical report: A highly capable language model locally on your phone.arXiv preprint arXiv:2404.14219, 2024

    2024 arXiv

  12. [18]

    Visual spatial reasoning.Transactions of the Association for Computational Linguistics, 11:635–651, 2023

    Fangyu Liu, Guy Emerson, and Nigel Collier. Visual spatial reasoning.Transactions of the Association for Computational Linguistics, 11:635–651, 2023

    2023

  13. [19]

    Navigating to objects in the real world.Science Robotics, 8(79):eadf6991, 2023

    Theophile Gervet, Soumith Chintala, Dhruv Batra, Jitendra Malik, and Devendra Singh Chaplot. Navigating to objects in the real world.Science Robotics, 8(79):eadf6991, 2023

    2023

  14. [20]

    Iqa: Visual question answering in interactive environments

    Daniel Gordon, Aniruddha Kembhavi, Mohammad Rastegari, Joseph Redmon, Dieter Fox, and Ali Farhadi. Iqa: Visual question answering in interactive environments. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 4089–4098, 2018

    2018

  15. [21]

    Learning spatial- semantic representations from natural language descriptions and scene classifications

    Sachithra Hemachandra, Matthew R Walter, Stefanie Tellex, and Seth Teller. Learning spatial- semantic representations from natural language descriptions and scene classifications. In2014 IEEE international conference on robotics and automation (ICRA), pages 2623–2630. IEEE, 2014

    2014

  16. [22]

    Scene graph reasoning for visual question answering.arXiv preprint arXiv:2007.01072, 2020

    Marcel Hildebrandt, Hang Li, Rajat Koner, V olker Tresp, and Stephan Günnemann. Scene graph reasoning for visual question answering.arXiv preprint arXiv:2007.01072, 2020

    2007 arXiv

  17. [23]

    Learning 3d semantic scene graphs from 3d indoor reconstructions

    Johanna Wald, Helisa Dhamo, Nassir Navab, and Federico Tombari. Learning 3d semantic scene graphs from 3d indoor reconstructions. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3961–3970, 2020

    2020

  18. [24]

    Learning semantic maps from natural language descriptions

    Matthew R Walter, Sachithra Hemachandra, Bianca Homberg, Stefanie Tellex, and Seth J Teller. Learning semantic maps from natural language descriptions. InRobotics: science and systems, volume 2, 2013

    2013

  19. [25]

    Llava-o1: Let vision language models reason step-by-step.arXiv preprint arXiv:2411.10440, 2024

    Guowei Xu, Peng Jin, Li Hao, Yibing Song, Lichao Sun, and Li Yuan. Llava-o1: Let vision language models reason step-by-step.arXiv preprint arXiv:2411.10440, 2024

    2024 arXiv

  20. [26]

    Visual-rft: Visual reinforcement fine-tuning.arXiv preprint arXiv:2503.01785, 2025

    Ziyu Liu, Zeyi Sun, Yuhang Zang, Xiaoyi Dong, Yuhang Cao, Haodong Duan, Dahua Lin, and Jiaqi Wang. Visual-rft: Visual reinforcement fine-tuning.arXiv preprint arXiv:2503.01785, 2025

    2025 arXiv

  21. [27]

    Imagine while reasoning in space: Multimodal visualization-of-thought.arXiv preprint arXiv:2501.07542, 2025

    Chengzu Li, Wenshan Wu, Huanyu Zhang, Yan Xia, Shaoguang Mao, Li Dong, Ivan Vuli ´c, and Furu Wei. Imagine while reasoning in space: Multimodal visualization-of-thought.arXiv preprint arXiv:2501.07542, 2025

    2025 arXiv

  22. [28]

    Llamav-o1: Rethinking step-by-step visual reasoning in llms.arXiv preprint arXiv:2501.06186, 2025

    Omkar Thawakar, Dinura Dissanayake, Ketan More, Ritesh Thawkar, Ahmed Heakl, Noor Ahsan, Yuhao Li, Mohammed Zumri, Jean Lahoud, Rao Muhammad Anwer, et al. Llamav-o1: Rethinking step-by-step visual reasoning in llms.arXiv preprint arXiv:2501.06186, 2025

    2025 arXiv

  23. [29]

    Visual cot: Unleashing chain-of-thought reasoning in multi-modal language models.arXiv e-prints, pages arXiv–2403, 2024

    Hao Shao, Shengju Qian, Han Xiao, Guanglu Song, Zhuofan Zong, Letian Wang, Yu Liu, and Hongsheng Li. Visual cot: Unleashing chain-of-thought reasoning in multi-modal language models.arXiv e-prints, pages arXiv–2403, 2024

    2024

  24. [30]

    Llava-cot: Let vision language models reason step-by-step, 2025

    Guowei Xu, Peng Jin, Hao Li, Yibing Song, Lichao Sun, and Li Yuan. Llava-cot: Let vision language models reason step-by-step, 2025

    2025

  25. [31]

    Introducing visual perception token into multi- modal large language model.arXiv preprint arXiv:2502.17425, 2025

    Runpeng Yu, Xinyin Ma, and Xinchao Wang. Introducing visual perception token into multi- modal large language model.arXiv preprint arXiv:2502.17425, 2025. 11

    2025 arXiv

  26. [32]

    Regiongpt: Towards region understanding vision language model

    Qiushan Guo, Shalini De Mello, Hongxu Yin, Wonmin Byeon, Ka Chun Cheung, Yizhou Yu, Ping Luo, and Sifei Liu. Regiongpt: Towards region understanding vision language model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13796–13806, 2024

    2024

  27. [33]

    Osprey: Pixel understanding with visual instruction tuning

    Yuqian Yuan, Wentong Li, Jian Liu, Dongqi Tang, Xinjie Luo, Chi Qin, Lei Zhang, and Jianke Zhu. Osprey: Pixel understanding with visual instruction tuning. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 28202–28211, 2024

    2024

  28. [34]

    Openai o1 system card.arXiv preprint arXiv:2412.16720, 2024

    Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, et al. Openai o1 system card.arXiv preprint arXiv:2412.16720, 2024

    2024 arXiv

  29. [35]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025

    2025 arXiv

  30. [36]

    Qvq: To see the world with wisdom, December 2024

    Qwen Team. Qvq: To see the world with wisdom, December 2024

    2024

  31. [37]

    Dinov2: Learning robust visual features without supervision.arXiv preprint arXiv:2304.07193, 2023

    Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision.arXiv preprint arXiv:2304.07193, 2023

    2023 arXiv

  32. [39]

    What’s “up” with vision-language models? investigating their struggle with spatial reasoning

    Amita Kamath, Jack Hessel, and Kai-Wei Chang. What’s “up” with vision-language models? investigating their struggle with spatial reasoning. InEMNLP, 2023

    2023

  33. [40]

    Blink: Multimodal large language models can see but not perceive.arXiv preprint arXiv:2404.12390, 2024

    Xingyu Fu, Yushi Hu, Bangzheng Li, Yu Feng, Haoyu Wang, Xudong Lin, Dan Roth, Noah A Smith, Wei-Chiu Ma, and Ranjay Krishna. Blink: Multimodal large language models can see but not perceive.arXiv preprint arXiv:2404.12390, 2024

    2024 arXiv

  34. [41]

    Reasoning paths with reference objects elicit quantitative spatial reasoning in large vision-language models, 2024

    Yuan-Hong Liao, Rafid Mahmood, Sanja Fidler, and David Acuna. Reasoning paths with reference objects elicit quantitative spatial reasoning in large vision-language models, 2024

    2024

  35. [42]

    Deepseek-v3 technical report, 2024

    DeepSeek-AI. Deepseek-v3 technical report, 2024

    2024

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.