Pith. sign in

REVIEW 3 major objections 5 minor 56 references

SCOUT: Unlocking Enhanced Spatial Reasoning via Structured Chain-of-Thought and Multi-Objective Process Reward

T0 review · 3 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A 7-billion-parameter vision-language model, SCOUT-7B, surpasses GPT-4o on spatial reasoning benchmarks by training with structured, depth-aware chain-of-thought and multi-objective process rewards.

desk verdict A well-engineered spatial-reasoning RL recipe whose main numbers are currently not trustworthy because the paper never shows train/eval disjointness and tunes hyperparameters on the test benchmarks. read the letter →

arxiv 2608.12220 v1 pith:PQQ4SUMU submitted 2026-08-12 cs.CV cs.AI

Zile Zhou , Huining Yuan , Weichen Zhang , Xinlei Chen , Xiao-ping Zhang This is my paper · ORCID
classification cs.CVcs.AI
keywords spatialreasoningvision-languagemodelsreinforcementlearningprocessrewardschain-of-thoughtdepthperception3Dsceneunderstandingcreditassignment
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper aims to establish that the spatial-reasoning bottleneck of vision-language models can be broken by forcing the model to externalize 3D perception—bounding boxes plus depth—inside a structured chain of thought, then reinforcing each stage with its own reward rather than a single final-answer reward. It builds a synthesized spatial QA dataset, SCOUT-24k, and an RL algorithm that assigns token-level credit to scene perception, analysis, and the final answer. If the claims hold, a small open-weight model can outperform a proprietary frontier model on general and complex spatial benchmarks, and spatial skills learned on single images transfer to multi-image and video questions. The paper's central bet is that fine-grained process supervision, not raw scale, is what unlocks 3D reasoning in VLMs.

What carries the argument

The central mechanism is the tagged reasoning trace—<think><caption>…</caption><scene>…</scene><analyze>…</analyze></think><answer>…</answer>—which turns an otherwise opaque chain of thought into three separately verifiable modules. Bounding-box and depth values in the <scene> module make the model's 3D perception explicit, so RL can reward accurate object localization and depth with a Hungarian-matched grounding reward and a continuous depth reward. A blind verifier reward on <analyze> ensures the reasoning alone entails the answer, while the advantage estimator z-score normalizes the five rewards, blends local process advantages with the global outcome advantage at a weight of 0.3, and assigns those advantages token-wise by tag boundaries before the clipped policy update. This setup is what lets the model receive different gradient signals for perceiving, reasoning, and answering.

What would settle it

Take held-out images with known 3D ground truth, such as RGB-D scans with metric depth, and compare SCOUT-7B's predicted <scene> depth values and relative-distance answers against both the true depths and Depth-Anything-3's estimates; if the model tracks the estimator's predictions but diverges from true metric depth on the same distribution, the reward has taught estimator-following rather than 3D understanding.

Watch

Extended reading notes

Core claim

On its own terms, the paper reports that SCOUT-7B, initialized from Qwen2.5-VL-7B and trained with an SFT cold-start followed by process-supervised RL, reaches 79.66% on the general spatial benchmark average, 4.28 points above GPT-4o, and 61.79% overall on complex spatial reasoning, 0.87 points above GPT-4o; the 3B version improves its base model by 16.85 points and 6.3 points on the two suites. The load-bearing components are a depth-aware structured CoT—<caption>, <scene> with bounding boxes and depths, <analyze>, <answer>—and a multi-objective process-reward RL with normalized rewards for grounding, depth, reasoning consistency, format, and accuracy, aggregated into scene, analysis, and outcome advantages and assigned to tokens by tag position. The paper also claims out-of-domain transfer from single-image training to ViewSpatial multi-image and VSI-Bench multiple-choice video questions, with gains of 2.46 and 3.13 points over the base model. Ablations show that removing the perception advantage stops the grounding and depth rewards from improving, and removing the analysis advantage collapses the consistency reward and drops RoboSpatial accuracy, supporting the claim that fine-grained credit assignment is what makes the pipeline work.

Load-bearing premise

The training pipeline assumes that the monocular depth estimates from Depth-Anything-3, sampled at each bounding box's center, are accurate enough to serve as ground truth for the dataset and as the RL depth reward, so any systematic bias in that estimator is what the model is trained to reproduce.

Editorial extensions

If this is right

  • If the central claim is right, a compact open-weight model can beat a proprietary frontier model on spatial benchmarks, so frontier-level spatial reasoning no longer requires closed systems.
  • The ablations imply that outcome-only RL or vanilla CoT leaves large gains on the table: the full method's 67.94% average beats GRPO with vanilla CoT by 4.04 points and beats process-free or credit-free variants by about 2.7 points.
  • The observed transfer from single-image training to multi-image and video tasks suggests the model learns abstract 3D structure rather than image-format-specific shortcuts, at least for relative-direction and multiple-choice video questions.
  • Because gains appear in both general and complex benchmarks, the depth-aware structured format plus process rewards appears to improve perception and reasoning jointly, not just format following.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: since the depth reward and dataset labels both come from a single monocular depth estimator, the method's ceiling is tied to that estimator's accuracy; training with multiple depth predictors or metric-depth supervision would reveal whether the gains are genuine 3D knowledge or estimator imitation.
  • Editorial inference: the token-level advantage decomposition suggests a general recipe for other structured visual outputs—declare a verifiable grammar, reward each module, then blend local and global advantages—which could transfer to detection, planning, or embodied question answering.
  • Editorial inference: because the method requires a strictly structured CoT format, the same RL machinery may underperform on free-form reasoning; a natural test is to relax the tags after training and measure how much spatial accuracy survives.
  • Editorial inference: the paper's own VSI-Bench numerical-question results drop sharply, which suggests temporal and absolute quantitative estimation is not transferred; adding video-depth or tracking pseudo-labels to the reward set would be a targeted extension beyond the paper's single-image scope.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes SCOUT, a two-stage training pipeline for vision-language models that combines a structured chain-of-thought (caption, scene, analyze, answer) with multi-objective process rewards and token-level advantage estimation. The authors build a 24k-example dataset (SCOUT-24k) from EmbSpatial and STVQA images using Qwen-VL-Max annotations and Depth-Anything-3 depth estimates, cold-start with SFT, and then apply RL with rewards for grounding, depth, reasoning consistency, answer accuracy, and format. Experiments on six single-image benchmarks and two multi-image/video benchmarks report that SCOUT-3B and SCOUT-7B outperform Qwen2.5-VL baselines and that SCOUT-7B surpasses GPT-4o on the average of the single-image benchmarks, with additional gains on multi-image and video multiple-choice questions.

Significance. If the empirical claims were clean, SCOUT would be a useful contribution: the reward decomposition is concrete, the token-level advantage assignment in Eqs. (5)-(7) is a sensible way to address credit assignment in RLVR, and the ablation study in Table 4 attempts to isolate the contribution of each reward. The two major risks are (i) possible train/eval overlap because SCOUT-24k is built from EmbSpatial and CV-Bench-derived material while those benchmarks are used for evaluation, and (ii) test-set hyperparameter selection in Table 5. Both concerns are load-bearing for the headline comparisons, so the significance of the paper cannot be assessed until they are resolved.

major comments (3)
  1. [§2.4, §3.1, Appendix A.2] SCOUT-24k is constructed from source images in EmbSpatial and STVQA (Section 2.4), and the evaluation in Section 3.1 uses EmbSpatial and CV-Bench, among others; the paper never states that the training/RL items are disjoint from the evaluation items. Appendix A.2 refers to an 'EmbSpatial training set' without defining its relationship to the EmbSpatial benchmark evaluation split, and Section 2.4 says that samples with existing answer annotations are 'reserved for reinforcement learning.' If any EmbSpatial images or QA pairs used in RL appear in the evaluation, the headline margins over GPT-4o (4.28%) and over baselines (16.85%) are not evidence for the proposed method but for train/test leakage. The authors must report exact image- and QA-level overlap statistics and re-run the evaluation on a strictly disjoint split, and clarify whether CV-Bench-derived templates or items were used for evaluation questions.
  2. [§3.4, Table 5] The mixing coefficients α1 and α2 in Eq. (6) are chosen by evaluating α∈{0.3,0.5,0.7} on the same six benchmarks that later constitute the main results (Table 5). Selecting α=0.3 as the best configuration on the evaluation set is test-set model selection; it invalidates the reported averages as unbiased estimates and inflates the apparent advantage of SCOUT. The sensitivity analysis should be conducted on a held-out validation split, and the test evaluation should be reported only for the final configuration.
  3. [§2.4, Eq. (3)] The depth reward (Eq. 3) and the dataset's depth labels both treat Depth-Anything-3 estimates, sampled at the center of each bounding box, as ground truth. The paper provides no validation of this monocular estimator on the image distributions of the evaluation benchmarks. If the estimator is systematically biased, the RL reward teaches the model to reproduce the estimator's errors, so gains in depth-related columns (e.g., BLINK depth and 3D in Table 1) may reflect fitting to the estimator rather than genuine 3D understanding. Please quantify the estimator's error against available metric-depth ground truth in the benchmarks and, if possible, verify that the reward correlates with true depth.
minor comments (5)
  1. [Abstract and §1] The reported improvements of 16.85% and 6.3% are absolute percentage-point differences in average accuracy (e.g., Table 1: 60.71 to 77.56); please use 'percentage points' or report relative improvements.
  2. [§2.4] The dataset 'STVQA [4]' is cited with reference [4], but reference [4] in the bibliography is Batra et al., SpatialThinker; no STVQA dataset entry is provided anywhere in the reference list.
  3. [§2.3, Eq. (4)] The z-score normalization divides by the group standard deviation σ_k, which is zero whenever all samples in a group have the same binary reward; please state how zero-variance groups are handled in the implementation.
  4. [Table 3] The claim of 'robust out-of-domain generalization' is weakened by the large drops on VSI-Bench numerical questions (25.03→19.26 for 3B and 40.52→29.35 for 7B); the text acknowledges this, but the abstract and Section 1 should qualify the generalization claim accordingly.
  5. [Appendix C.1, Table 1] The definitions of the 'BLINK', 'Overall 2D', and '3D' columns in Table 1 are not fully specified; please state which BLINK subset is used and how the average columns are computed.

Circularity Check

1 steps flagged · score 6.0 of 10

The headline benchmark gains are partly selected on the evaluation benchmarks themselves: the saved checkpoint and the α hyperparameter are chosen by best accuracy on the same six benchmarks used for the reported improvements, so the central 'predictions' are partly fitted rather than independent.

  1. fitted input called prediction [Section 3.1 (Implementation Details, checkpoint selection) and Section 3.4 (Sensitivity analysis of α; Table 5)]
    "Checkpoints are saved every 50 steps, and we report the performance of the best-performing checkpoint. We further conduct a sensitivity analysis for the factors by setting α1 = α2. As shown in Tab. 5, α=0.3 achieves the best overall performance, while the average accuracy steadily degrades as α increases."

    The reported headline results (e.g., the 16.85% and 9.51% gains over baseline groups, the 67.94% SCOUT-3B average, and the 4.28% margin over GPT-4o) are measured on the same six benchmarks that were used to select the α values and the saved checkpoint. The paper describes no held-out validation split for these choices; 'best-performing checkpoint' and 'α=0.3 achieves the best overall performance' therefore mean that the published numbers are the maximum of a selection procedure over the evaluation set itself. A performance figure obtained by selecting hyperparameters and checkpoints on the test benchmarks is a fitted quantity, not an out-of-sample prediction of the method, so the central quantitative claims are partially circular.

full rationale

The strongest demonstrated circularity is selection on the evaluation benchmarks: Section 3.1 chooses the reported checkpoint by best performance, and Section 3.4 chooses α=0.3 by best overall accuracy on the six benchmarks that later constitute Tables 1, 2, and 4. This makes part of the headline improvement a fitted maximum rather than an independent prediction. I do not count the EmbSpatial/CV-Bench data-construction overlap as a demonstrated circular step: the paper says QA generation is done on the EmbSpatial training set and it evaluates on EmbSpatial/CV-Bench, but it never states a disjoint split, so actual item overlap is not established; this is a missing-support correctness risk rather than a quoted reduction. The depth reward is not self-definitional: r_depth is computed against Depth-Anything-3 pseudo-labels, while the benchmark answers are independent, or at least are not shown to be generated by that estimator; training to imitate a teacher is ordinary distillation. The only apparent self-citation ([38], MM-Eureka, sharing author Z. Zhou) appears in a broad RLVR related-work list and is not load-bearing. Independent content remains in the ablations and in the out-of-domain ViewSpatial/VSI-Bench results, which were not used for the α and checkpoint selection on the six main benchmarks, so the paper is only partially circular rather than entirely forced.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

No new physical entities, forces, or conserved quantities are introduced; the structured CoT tags are output-format devices, not theoretical entities. The main load-bearing assumptions are about the reliability of pseudo-labels (monocular depth, Qwen-VL-Max generation), the validity of the blind verifier, and the absence of training/evaluation contamination. Several hand-chosen coefficients in the reward design are free parameters, with the alpha mixing coefficients explicitly selected on test benchmarks.

free parameters (4)
  • Grounding cost weights lambda_sem, lambda_iou, lambda_dep = 2.0, 3.0, 0.5
    Hand-chosen coefficients in Equation 1 balancing semantic similarity, Efficient IoU, and depth consistency in the Hungarian matching cost; no sensitivity analysis is reported for these weights.
  • Over-generation penalty eta = 0.2
    Hand-chosen penalty in Equation 2 to discourage bounding box proliferation; no sensitivity analysis is reported.
  • Advantage mixing coefficients alpha1, alpha2 = 0.3, 0.3
    Equation 6 blends process rewards with the outcome advantage. Table 5 selects alpha = 0.3 because it gives the best average on the six evaluation benchmarks, meaning the value is effectively tuned on test data.
  • Equidistance threshold in dataset construction = 0.03 meters
    Appendix A.1 labels two candidate objects as equidistant when depth differences are below 0.03 meters; this hand-chosen threshold affects ground-truth labels for Relative Distance Prediction data.
assumptions (5)
  • domain assumption Monocular depth estimates from Depth-Anything-3, sampled at bounding box centers, are accurate enough to serve as ground truth for dataset labels and the depth reward.
    Section 2.4 uses Depth-Anything-3 to produce center coordinates (u, v, z) that become the perception ground truth and the depth reward target. If the estimator is biased on evaluation distributions, the model learns those biases.
  • domain assumption Qwen-VL-Max generated captions, scene metadata, and answer verifications are reliable enough to build correct reasoning chains.
    Section 2.4 and Appendix A.2 rely on Qwen-VL-Max for captions, answer generation, and answer filtering; errors in these steps propagate into SCOUT-24k and into the consistency verification.
  • domain assumption The blind verifier, a base model given only the question and reasoning text, measures logical entailment.
    Section 2.2 defines r_consistency by feeding question plus reasoning text to the base model without the image. For multiple-choice questions with patterned templates, the verifier may guess or exploit answer leakage in the text, so the reward may not truly measure consistency.
  • domain assumption The model's structured scene output can be parsed reliably and matched to ground-truth objects by the Hungarian algorithm.
    Equation 2 and the grounding/depth rewards depend on parsing model-generated scene JSON and matching predicted objects to ground truth. Parsing failure rates and matching degeneracies are not reported.
  • domain assumption Evaluation benchmarks, especially EmbSpatial and CV-Bench, are not contaminated by SCOUT-24k training data.
    Section 2.4 constructs SCOUT-24k from EmbSpatial and templates adapted from CV-Bench, while Section 3.1 evaluates on EmbSpatial and CV-Bench. The paper only mentions using the EmbSpatial training set and never documents a disjoint evaluation split.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SCOUT: Unlocking Enhanced Spatial Reasoning via Structured Chain-of-Thought and Multi-Objective Process Reward." pith.science (2026). https://pith.science/paper/PQQ4SUMU

@misc{pith2026260812220,
  author       = {Pith},
  title        = {Pith review of: SCOUT: Unlocking Enhanced Spatial Reasoning via Structured Chain-of-Thought and Multi-Objective Process Reward},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PQQ4SUMU}},
  note         = {Machine review of arXiv:2608.12220}
}
read the original abstract

Existing Vision-Language Models (VLMs) exhibits a critical bottleneck in robust spatial reasoning. Recent reinforcement learning (RL) methods aim to close this gap with verifiable outcomes, yet they suffer from poor credit assignment across intermediate reasoning steps. Concurrently, structured reasoning approaches overlook the critical depth perception necessary for comprehensive 3D understanding. To address these challenges, we propose SCOUT (Structured Chain-Of-Thought Utilizing Process-Supervised RL Training). Specifically, we design a structured Chain-of-Thought (CoT) framework that explicitly models 3D environmental perception to ensure robust spatial understanding and reasoning. Furthermore, we introduce a novel RL algorithm featuring multi-objective process rewards and a tailored advantage estimation method, facilitating fine-grained credit assignment across distinct segments of the reasoning trajectory. To support our framework, we develop SCOUT-24k, a structured spatial reasoning CoT dataset synthesized through a customized pipeline. Extensive evaluations demonstrate that SCOUT-3B improves upon baseline models by 16.85% and 6.3% on general spatial benchmarks and complex spatial reasoning tasks respectively. Notably, our larger SCOUT-7B even outperforms GPT-4o by a margin of 4.28%. Moreover, despite being trained exclusively on single image, SCOUT-7B exhibits robust out-of-domain generalization to multi-image and video scenarios. These empirical results render SCOUT as a critical step towards next generation of spatially-aware VLMs.

Figures

Figures reproduced from arXiv: 2608.12220 by the authors.

Figure 1
Figure 1. We present SCOUT, a novel spatial reasoning framework for VLMs, equipped with a structured CoT optimized via process-supervised RL. (a) Qualitative ex￾ample: the baseline produces an incorrect answer due to a lack of spatial awareness, whereas our model generates accurate prediction by explicitly analyzing object depth and scene information. (b) Quantitative evaluations on multiple spatial reasoning benchmarks demon… view at source ↗
Figure 2
Figure 2. Method overview of SCOUT. Stage 1 initializes the training through a SFT cold-start using our curated CoT data. Stage 2 employs reinforcement learning with multi-objective rewards and fine-grained advantage estimation to effectively assign credit across different tokens and update the policy model. are fundamentally grounded in physical reality. More importantly, by partition￾ing the CoT into independently extractab… view at source ↗
Figure 3
Figure 3. Overview of SCOUT-24k dataset construction pipeline [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Training dynamics under different reward configurations. [PITH_FULL_IMAGE:figures/full_fig_p012_4.png]
Figure 5
Figure 5. Figure 5: Qualitative examples of the reasoning process in SCOUT. [PITH_FULL_IMAGE:figures/full_fig_p014_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

56 extracted references · 28 canonical work pages

  1. [1]

    AI, R., Mayorquin, S.: Spacellava models.https://huggingface.co/remyxai/ SpaceLLaVA(2024), accessed: 2026-02-03 9

    work page 2024

  2. [2]

    AI, R., Mayorquin, S.: Spaceom models.https://huggingface.co/remyxai/ SpaceOm(2025), accessed: 2026-02-03 9

    work page 2025

  3. [3]

    5-vl technical report

    Bai, S., Chen, K., Liu, X., Wang, J., Ge, W., Song, S., Dang, K., Wang, P., Wang, S., Tang, J., et al.: Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923 (2025) 1, 3, 8, 9, 13

    arXiv 2025

  4. [4]

    arXiv preprint arXiv:2511.07403 (2025) 2, 5, 8, 9, 10, 14

    Batra, H., Tu, H., Chen, H., Lin, Y., Xie, C., Clark, R.: Spatialthinker: Re- inforcing 3d reasoning in multimodal llms via spatial rewards. arXiv preprint arXiv:2511.07403 (2025) 2, 5, 8, 9, 10, 14

    arXiv 2025

  5. [5]

    In: 2025 IEEE International Conference on Robotics and Automation (ICRA)

    Cai, W., Ponomarenko, I., Yuan, J., Li, X., Yang, W., Dong, H., Zhao, B.: Spa- tialbot: Precise spatial understanding with vision language models. In: 2025 IEEE International Conference on Robotics and Automation (ICRA). pp. 9490–9498. IEEE (2025) 2, 9, 13

    work page 2025

  6. [6]

    Advances in Neural Information Processing Systems37, 53168–53197 (2024) 1

    Chandrasegaran, K., Gupta, A., Hadzic, L.M., Kota, T., He, J., Eyzaguirre, C., Durante, Z., Li, M., Wu, J., Fei-Fei, L.: Hourvideo: 1-hour video-language under- standing. Advances in Neural Information Processing Systems37, 53168–53197 (2024) 1

    work page 2024

  7. [7]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog- nition

    Chen, B., Xu, Z., Kirmani, S., Ichter, B., Sadigh, D., Guibas, L., Xia, F.: Spa- tialvlm: Endowing vision-language models with spatial reasoning capabilities. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog- nition. pp. 14455–14465 (2024) 2, 9, 13

    work page 2024

  8. [8]

    Advances in Neural Information Processing Systems37, 135062–135093 (2024) 2, 13

    Cheng, A.C., Yin, H., Fu, Y., Guo, Q., Yang, R., Kautz, J., Wang, X., Liu, S.: Spatialrgpt: Grounded spatial reasoning in vision-language models. Advances in Neural Information Processing Systems37, 135062–135093 (2024) 2, 13

    work page 2024

Show all 56 references
  1. [9]

    arXiv preprint arXiv:2501.17161 (2025) 2

    Chu, T., Zhai, Y., Yang, J., Tong, S., Xie, S., Schuurmans, D., Le, Q.V., Levine, S., Ma, Y.: Sft memorizes, rl generalizes: A comparative study of foundation model post-training. arXiv preprint arXiv:2501.17161 (2025) 2

    2025 arXiv

  2. [10]

    arXiv preprint arXiv:2507.06261 (2025) 1, 13

    Comanici, G., Bieber, E., Schaekermann, M., Pasupat, I., Sachdeva, N., Dhillon, I., Blistein, M., Ram, O., Zhang, D., Rosen, E., et al.: Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. arXiv prepr...

    2025 arXiv

  3. [11]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision

    Daxberger, E., Wenzel, N., Griffiths, D., Gang, H., Lazarow, J., Kohavi, G., Kang, K., Eichner, M., Yang, Y., Dehghan, A., et al.: Mm-spatial: Exploring 3d spatial understanding in multimodal llms. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp...

    2025

  4. [12]

    In: Proceedings of the Computer Vision and Pattern Recognition Conference

    Deitke, M., Clark, C., Lee, S., Tripathi, R., Yang, Y., Park, J.S., Salehi, M., Muen- nighoff, N., Lo, K., Soldaini, L., et al.: Molmo and pixmo: Open weights and open data for state-of-the-art vision-language models. In: Proceedings of the Computer Vision and Pattern Recognit...

    2025

  5. [13]

    In: Conference on robot learning

    Driess, D., Ha, J.S., Toussaint, M., Tedrake, R.: Learning models as functionals of signed-distance fields for manipulation planning. In: Conference on robot learning. pp. 245–255. PMLR (2022) 1

    2022

  6. [14]

    In: Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)

    Du, M., Wu, B., Li, Z., Huang, X.J., Wei, Z.: Embspatial-bench: Benchmarking spatial understanding for embodied tasks with large vision-language models. In: Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). pp. 34...

    2024

  7. [15]

    In: European Conference on Computer Vision

    Fu, X., Hu, Y., Li, B., Feng, Y., Wang, H., Lin, X., Roth, D., Smith, N.A., Ma, W.C., Krishna, R.: Blink: Multimodal large language models can see but not per- ceive. In: European Conference on Computer Vision. pp. 148–166. Springer (2024) 9

    2024

  8. [16]

    arXiv preprint arXiv:2501.12948 (2025) 14

    Guo, D., Yang, D., Zhang, H., Song, J., Zhang, R., Xu, R., Zhu, Q., Ma, S., Wang, P., Bi, X., et al.: Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948 (2025) 14

    2025 arXiv

  9. [17]

    Advances in Neural Information Processing Systems36, 20482–20494 (2023) 13

    Hong, Y., Zhen, H., Chen, P., Zheng, S., Du, Y., Chen, Z., Gan, C.: 3d-llm: In- jecting the 3d world into large language models. Advances in Neural Information Processing Systems36, 20482–20494 (2023) 13

    2023

  10. [18]

    Iclr1(2), 3 (2022) 8

    Hu, E.J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W., et al.: Lora: Low-rank adaptation of large language models. Iclr1(2), 3 (2022) 8

    2022

  11. [19]

    arXiv preprint arXiv:2311.12871 (2023) 13

    Huang, J., Yong, S., Ma, X., Linghu, X., Li, P., Wang, Y., Li, Q., Zhu, S.C., Jia, B., Huang, S.: An embodied generalist agent in 3d world. arXiv preprint arXiv:2311.12871 (2023) 13

    2023 arXiv

  12. [20]

    arXiv preprint arXiv:2507.23478 (2025) 2, 14

    Huang, T., Zhang, Z., Tang, H.: 3d-r1: Enhancing reasoning in 3d vlms for unified scene understanding. arXiv preprint arXiv:2507.23478 (2025) 2, 14

    2025 arXiv

  13. [21]

    arXiv preprint arXiv:2409.01652 (2024) 1

    Huang, W., Wang, C., Li, Y., Zhang, R., Fei-Fei, L.: Rekep: Spatio-temporal rea- soning of relational keypoint constraints for robotic manipulation. arXiv preprint arXiv:2409.01652 (2024) 1

    2024 arXiv

  14. [22]

    arXiv preprint arXiv:2503.06749 (2025) 14

    Huang, W., Jia, B., Zhai, Z., Cao, S., Ye, Z., Zhao, F., Xu, Z., Hu, Y., Lin, S.: Vision-r1: Incentivizing reasoning capability in multimodal large language models. arXiv preprint arXiv:2503.06749 (2025) 14

    2025 arXiv

  15. [23]

    arXiv preprint arXiv:2410.21276 (2024) 1, 3, 9

    Hurst, A., Lerer, A., Goucher, A.P., Perelman, A., Ramesh, A., Clark, A., Os- trow, A., Welihinda, A., Hayes, A., Radford, A., et al.: Gpt-4o system card. arXiv preprint arXiv:2410.21276 (2024) 1, 3, 9

    2024 arXiv

  16. [24]

    Naval research logistics quarterly2(1-2), 83–97 (1955) 5

    Kuhn, H.W.: The hungarian method for the assignment problem. Naval research logistics quarterly2(1-2), 83–97 (1955) 5

    1955

  17. [25]

    arXiv preprint arXiv:2505.21500 (2025) 4, 10

    Li, D., Li, H., Wang, Z., Yan, Y., Zhang, H., Chen, S., Hou, G., Jiang, S., Zhang, W., Shen, Y., et al.: Viewspatial-bench: Evaluating multi-perspective spatial lo- calization in vision-language models. arXiv preprint arXiv:2505.21500 (2025) 4, 10

    2025

  18. [26]

    arXiv preprint arXiv:2510.08531 (2025) 2, 9, 14

    Li, H., Li, D., Wang, Z., Yan, Y., Wu, H., Zhang, W., Shen, Y., Lu, W., Xiao, J., Zhuang, Y.: Spatialladder: Progressive training for spatial reasoning in vision- language models. arXiv preprint arXiv:2510.08531 (2025) 2, 9, 14

    2025

  19. [27]

    arXiv preprint arXiv:2511.10647 (2025) 8

    Lin, H., Chen, S., Liew, J.H., Chen, D.Y., Li, Z., Shi, G., Feng, J., Kang, B.: Depth anything 3: Recovering the visual space from any views. arXiv preprint arXiv:2511.10647 (2025) 8

    2025 arXiv

  20. [28]

    Transactions of the Association for Computational Linguistics11, 635–651 (2023) 13

    Liu, F., Emerson, G., Collier, N.: Visual spatial reasoning. Transactions of the Association for Computational Linguistics11, 635–651 (2023) 13

    2023

  21. [29]

    In: Proceedings of the IEEE/CVF International Con- ference on Computer Vision

    Liu, S., Zhang, H., Qi, Y., Wang, P., Zhang, Y., Wu, Q.: Aerialvln: Vision-and- language navigation for uavs. In: Proceedings of the IEEE/CVF International Con- ference on Computer Vision. pp. 15384–15394 (2023) 1

    2023

  22. [30]

    arXiv preprint arXiv:2505.12448 (2025) 13

    Liu, Y., Ma, M., Yu, X., Ding, P., Zhao, H., Sun, M., Huang, S., Wang, D.: Ssr: Enhancing depth perception in vision-language models via rationale-guided spatial reasoning. arXiv preprint arXiv:2505.12448 (2025) 13

    2025

  23. [31]

    arXiv preprint arXiv:2510.27606 (2025) 8 SCOUT 17

    Liu, Y., Zhang, B., Zang, Y., Cao, Y., Xing, L., Dong, X., Duan, H., Lin, D., Wang, J.: Spatial-ssrl: Enhancing spatial understanding via self-supervised reinforcement learning. arXiv preprint arXiv:2510.27606 (2025) 8 SCOUT 17

    2025

  24. [32]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Liu, Z., Zhu, L., Shi, B., Zhang, Z., Lou, Y., Yang, S., Xi, H., Cao, S., Gu, Y., Li, D., et al.: Nvila: Efficient frontier visual language models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 4122– 4134 (2025) 13

    2025

  25. [33]

    arXiv preprint arXiv:2503.01785 (2025) 14

    Liu, Z., Sun, Z., Zang, Y., Dong, X., Cao, Y., Duan, H., Lin, D., Wang, J.: Visual- rft: Visual reinforcement fine-tuning. arXiv preprint arXiv:2503.01785 (2025) 14

    2025 arXiv

  26. [34]

    arXiv preprint arXiv:2601.01984 (2026) 2, 14

    Ma, W., Sun, S., Yu, T., Wang, R., Chua, T.S., Bian, J.: Thinking with blueprints: Assisting vision-language models in spatial reasoning via structured object repre- sentation. arXiv preprint arXiv:2601.01984 (2026) 2, 14

    2026

  27. [35]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision

    Ma, W., Chen, H., Zhang, G., Chou, Y.C., Chen, J., de Melo, C., Yuille, A.: 3dsr- bench: A comprehensive 3d spatial reasoning benchmark. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 6924–6934 (2025) 9

    2025

  28. [36]

    In: Advances in Neural Information Processing Systems

    Ma, W., Chou, Y.C., Liu, Q., Wang, X., Melo, C.M.d., Xie, J., Yuille, A.: Spatial- reasoner: Towards explicit and generalizable 3d spatial reasoning. In: Advances in Neural Information Processing Systems. vol. 38 (2025) 2, 9, 14

    2025

  29. [37]

    In: Proceed- ings of the Computer Vision and Pattern Recognition Conference

    Ma, W., Ye, L., de Melo, C.M., Yuille, A., Chen, J.: Spatialllm: A compound 3d- informed design towards spatially-intelligent large multimodal models. In: Proceed- ings of the Computer Vision and Pattern Recognition Conference. pp. 17249–17260 (2025) 2, 13

    2025

  30. [38]

    arXiv preprint arXiv:2503.07365 (2025) 14

    Meng, F., Du, L., Liu, Z., Zhou, Z., Lu, Q., Fu, D., Han, T., Shi, B., Wang, W., He, J., et al.: Mm-eureka: Exploring the frontiers of multimodal reasoning with rule-based reinforcement learning. arXiv preprint arXiv:2503.07365 (2025) 14

    2025 arXiv

  31. [39]

    arXiv preprint arXiv:2504.01805 (2025) 2, 14

    Ouyang, K., Liu, Y., Wu, H., Liu, Y., Zhou, H., Zhou, J., Meng, F., Sun, X.: Spacer: Reinforcing mllms in video spatial reasoning. arXiv preprint arXiv:2504.01805 (2025) 2, 14

    2025 arXiv

  32. [40]

    arXiv preprint arXiv:2402.03300 (2024) 6

    Shao, Z., Wang, P., Zhu, Q., Xu, R., Song, J., Bi, X., Zhang, H., Zhang, M., Li, Y., Wu, Y., et al.: Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300 (2024) 6

    2024 arXiv

  33. [41]

    arXiv preprint arXiv:2504.07615 (2025) 14

    Shen, H., Liu, P., Li, J., Fang, C., Ma, Y., Liao, J., Shen, Q., Zhang, Z., Zhao, K., Zhang, Q., et al.: Vlm-r1: A stable and generalizable r1-style large vision-language model. arXiv preprint arXiv:2504.07615 (2025) 14

    2025 arXiv

  34. [42]

    In: Proceedings of the Computer Vision and Pattern Recognition Conference

    Song, C.H., Blukis, V., Tremblay, J., Tyree, S., Su, Y., Birchfield, S.: Robospatial: Teaching spatial understanding to 2d and 3d vision-language models for robotics. In: Proceedings of the Computer Vision and Pattern Recognition Conference. pp. 15768–15780 (2025) 9

    2025

  35. [43]

    arXiv preprint arXiv:2402.12289 (2024) 1

    Tian, X., Gu, J., Li, B., Liu, Y., Wang, Y., Zhao, Z., Zhan, K., Jia, P., Lang, X., Zhao, H.: Drivevlm: The convergence of autonomous driving and large vision- language models. arXiv preprint arXiv:2402.12289 (2024) 1

    2024 arXiv

  36. [44]

    Advances in Neural Information Processing Systems 37, 87310–87356 (2024) 8, 9

    Tong, P., Brown, E., Wu, P., Woo, S., IYER, A.J.V., Akula, S.C., Yang, S., Yang, J., Middepogu, M., Wang, Z., et al.: Cambrian-1: A fully open, vision-centric ex- ploration of multimodal llms. Advances in Neural Information Processing Systems 37, 87310–87356 (2024) 8, 9

    2024

  37. [45]

    CoRR (2024) 1

    Wang, S., Yu, Z., Jiang, X., Lan, S., Shi, M., Chang, N., Kautz, J., Li, Y., Alvarez, J.M.: Omnidrive: A holistic llm-agent framework for autonomous driving with 3d perception, reasoning and planning. CoRR (2024) 1

    2024

  38. [46]

    arXiv preprint arXiv:2508.18265 (2025) 9

    Wang, W., Gao, Z., Gu, L., Pu, H., Cui, L., Wei, X., Liu, Z., Jing, L., Ye, S., Shao, J., et al.: Internvl3.5: Advancing open-source multimodal models in versatility, reasoning, and efficiency. arXiv preprint arXiv:2508.18265 (2025) 9

    2025 arXiv

  39. [47]

    arXiv preprint arXiv:2505.23747 (2025) 2, 14 18 Z

    Wu, D., Liu, F., Hung, Y.H., Duan, Y.: Spatial-mllm: Boosting mllm capabilities in visual-based spatial intelligence. arXiv preprint arXiv:2505.23747 (2025) 2, 14 18 Z. Zhou et al

    2025 arXiv

  40. [48]

    In: Pro- ceedings of the Computer Vision and Pattern Recognition Conference

    Yang, J., Yang, S., Gupta, A.W., Han, R., Fei-Fei, L., Xie, S.: Thinking in space: How multimodal large language models see, remember, and recall spaces. In: Pro- ceedings of the Computer Vision and Pattern Recognition Conference. pp. 10632– 10643 (2025) 4, 10

    2025

  41. [49]

    arXiv preprint arXiv:2503.10615 (2025) 14

    Yang, Y., He, X., Pan, H., Jiang, X., Deng, Y., Yang, X., Lu, H., Yin, D., Rao, F., Zhu, M., et al.: R1-onevision: Advancing generalized multimodal reasoning through cross-modal formalization. arXiv preprint arXiv:2503.10615 (2025) 14

    2025 arXiv

  42. [50]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Yang, Y., Sun, F.Y., Weihs, L., VanderBilt, E., Herrasti, A., Han, W., Wu, J., Haber, N., Krishna, R., Liu, L., et al.: Holodeck: Language guided generation of 3d embodied ai environments. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. p...

    2024

  43. [51]

    In: 2024 IEEE International Con- ference on Robotics and Automation (ICRA)

    Yokoyama, N., Ha, S., Batra, D., Wang, J., Bucher, B.: Vlfm: Vision-language frontier maps for zero-shot semantic navigation. In: 2024 IEEE International Con- ference on Robotics and Automation (ICRA). pp. 42–48. IEEE (2024) 1

    2024

  44. [52]

    arXiv preprint arXiv:2504.07954 (2025) 14

    Yu, E., Lin, K., Zhao, L., Yin, J., Wei, Y., Peng, Y., Wei, H., Sun, J., Han, C., Ge, Z., et al.: Perception-r1: Pioneering perception policy with reinforcement learning. arXiv preprint arXiv:2504.07954 (2025) 14

    2025 arXiv

  45. [53]

    arXiv preprint arXiv:2511.01618 (2025) 2

    Zhan, X., Huang, W., Sun, H., Fu, X., Ma, C., Cao, S., Jia, B., Lin, S., Yin, Z., Bai, L., et al.: Actial: Activate spatial reasoning ability of multimodal large language models. arXiv preprint arXiv:2511.01618 (2025) 2

    2025

  46. [54]

    Neurocomputing506, 146–157 (2022) 5

    Zhang, Y.F., Ren, W., Zhang, Z., Jia, Z., Wang, L., Tan, T.: Focal and efficient iou loss for accurate bounding box regression. Neurocomputing506, 146–157 (2022) 5

    2022

  47. [55]

    arXiv preprint arXiv:2510.25760 (2025) 1

    Zheng, X., Dongfang, Z., Jiang, L., Zheng, B., Guo, Y., Zhang, Z., Albanese, G., Yang, R., Ma, M., Zhang, Z., et al.: Multimodal spatial reasoning in the large model era: A survey and benchmarks. arXiv preprint arXiv:2510.25760 (2025) 1

    2025

  48. [56]

    label":

    Zitkovich, B., Yu, T., Xu, S., Xu, P., Xiao, T., Xia, F., Wu, J., Wohlhart, P., Welker, S., Wahid, A., et al.: Rt-2: Vision-language-action models transfer web knowledge to robotic control. In: Conference on Robot Learning. pp. 2165–2183. PMLR (2023) 1 SCOUT 19 A Additional Da...

    2023

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.