Pith. sign in

REVIEW 3 major objections 5 minor 35 references

SpatialQuery: Benchmarking Geometry-Grounded Multi-Instance Spatial Reasoning in Vision-Language Models

T0 review · 3 major / 5 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A training-free pipeline answers 'nearest instance to X' queries from one RGB photo at 0.259 m mean floor-plane error, beating fine-tuned spatial specialists on a new million-question benchmark.

desk verdict The CIDQ benchmark and geometry pipeline are a real contribution, but the evaluation is circular: the ground truth is filtered by the same detector the method uses at inference, so the headline numbers don't stand as reported. read the letter →

arxiv 2608.01709 v1 pith:TLCTVMVG submitted 2026-08-03 cs.CV

Hai Nguyen , Tung Vu , Cong Tran This is my paper
classification cs.CV
keywords closest-instancedistancequerymulti-instancespatialreasoningvision-languagemodelsmonocularmetricdepthbird's-eyeviewrepresentationuncertainty-awarepromptingbenchmarkzero-shotgeometricgrounding
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper studies the Closest-Instance Distance Query (CIDQ): given one RGB photo and a question such as 'how far is the nearest chair to the TV?', a model must assemble every visible chair, decide which one is closest to the TV, and report the gravity-aligned floor-plane distance in metres. Its central claim is that CIDQ can be solved with no spatial fine-tuning at all, provided the scene is rebuilt as a canonical top-down map in which each detected instance appears as an equal-size, category-coded block, and the VLM is prompted with geometry-derived per-instance uncertainty. To test this, the paper builds SPATIALQUERY-1M, over a million RGB-only question-answer pairs from 200 indoor scenes, and reports that the pipeline with an 8-billion-parameter VLM reaches 0.259 m mean absolute floor-plane error and 84.18% proximity-decision accuracy under a strict zero-shot protocol, ahead of fine-tuned spatial specialists and closed-source models. A sympathetic reader would care because the result locates the bottleneck of metric spatial reasoning not in the size of the model but in whether the model is given geometry in a form it can actually compare.

What carries the argument

Two devices carry the argument. Scene Cubifying is the visual interface: after tilt-corrected metric floor-plane positions are recovered per instance, all instances are rendered on a canonical top-down canvas as uniformly sized, category-coded cuboids, with the reference given a contrasting border. It suppresses texture, perspective, and object-scale cues while preserving exactly the relative layout the query needs; metric values are read from the underlying coordinates, not from the image. UA-CoT (Uncertainty-Aware Chain-of-Thought) is the reasoning interface: the prompt supplies the BEV canvas plus a per-instance record of box, category, floor-plane coordinate, and dispersion score (root t

What would settle it

Re-score the method on the fixed 5,000-query sets using complete ground-truth candidate sets — every visible instance from the 3D annotations projected into the image — instead of the detector-filtered sets, or generate candidates with a different detector. If accuracy on complete candidate sets falls well below the reported 84.18% proximity accuracy and 0.259 m Floor-MAE, the claim that the pipeline handles candidate-set aggregation is contradicted. Second check: a trivial baseline that always reports sigma = 0.5 m should match Unc-Acc@0.3 m = 90.5% if the geometry-derived uncertainty adds no

Watch

Extended reading notes

Core claim

The paper's discovery claim is that Closest-Instance Distance Query reasoning — detecting all candidates of a class, selecting the nearest to a unique reference, and estimating the metric distance — is achievable without task-specific fine-tuning or architectural change. The recipe is Scene Cubifying: detected instances are back-projected from monocular depth, filtered and pitch-corrected to a gravity-aligned floor plane, then rendered on a 512x512 top-down canvas as uniformly sized, category-coded blocks, with the reference highlighted. Distance answers are computed from the retained floor-plane coordinates, never from pixels, and the VLM's answer is conditioned on a UA-CoT prompt that carr

Load-bearing premise

The benchmark's ground-truth candidate set is filtered by the same open-vocabulary detector the method uses to find candidates: any visible instance the detector misses is removed from both the correct answer and the model's input, so the hardest part of the task — simply not overlooking a candidate — is never scored.

Editorial extensions

If this is right

  • A fixed metric floor-plane representation, not larger spatial fine-tuning, is the key enabler: the full model beats both pure VLMs and fine-tuned spatial specialists on the same 5,000-query evaluation set without any task-specific training.
  • The geometric components are complementary and order-dependent: UA-CoT alone raises Acc@0.2 m to 49.5% but explodes Floor-MAE to 16.341 m; adding tilt-corrected MAD-RANSAC refinement brings it to 0.263 m, showing that structured prompting without stable geometry can reinforce errors.
  • The same geometry estimate transfers from distance regression (T1) to threshold-based proximity decisions (T2), gaining 24.28 points over the pure Qwen3-VL-8B backbone on T2.
  • The geometry-derived uncertainty tracks difficulty: the high-dispersion query subset has roughly 60% higher Floor-MAE than the low-dispersion subset, so the dispersion score can flag geometrically unreliable predictions.
  • The benchmark itself, 1,064,022 QA pairs with thresholds from 0.5 to 3 m across 315 categories, is a reusable resource for evaluating and training multi-instance spatial reasoning beyond fixed object pairs.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The benchmark's candidate set and the method's input share the same open-vocabulary detector: Stage 3 keeps only annotations matched to a GroundingDINO detection with IoU >= 0.5, and Phase I generates candidates with that same detector. A verifiable consequence is that any instance the detector misses vanishes from both ground truth and input, so candidate-set recall failures are not scored on the
  • A clean ablation would isolate the visual interface from the coordinates: feeding the same floor-plane coordinates and uncertainties to the VLM as text (or as a marked image with numbered dots) versus the cubified BEV would show how much of the gain is due to the block abstraction itself rather than to geometry being available at all.
  • Because Unc-Acc adds any positive uncertainty to the acceptance tolerance, large dispersion values automatically inflate coverage; the claim that sigma carries information therefore rests on the stratification analysis. A calibration-style companion metric, such as coverage at a fixed nominal probability or a proper scoring rule on the reported uncertainty, would sharpen comparisons across methods
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper defines the Closest-Instance Distance Query (CIDQ): given a single RGB image and a natural-language query, the model must identify the nearest visible instance of a candidate category to a unique reference object and estimate their gravity-aligned floor-plane distance, or decide whether that distance is within a threshold. The authors propose SPATIALQUERY, a training-free pipeline combining Qwen-VL query parsing, GroundingDINO/SAM instance grounding, Depth Pro metric depth and camera estimation, MAD-RANSAC depth filtering, pitch-corrected floor-plane localization, a Bird's-Eye View 'Scene Cubifying' representation, and Uncertainty-Aware Chain-of-Thought (UA-CoT) prompting. They also introduce SPATIALQUERY-1M, a benchmark with over one million QA pairs synthesized from MM-Spatial/CA-1M indoor annotations. On fixed 5,000-query T1 and T2 evaluation sets, SPATIALQUERY with Qwen3-VL-8B reports Floor-MAE 0.259 m, Unc-Acc@0.3 m 90.5%, and T2 accuracy 84.18%, outperforming fine-tuned spatial specialists and closed-source models.

Significance. If the benchmark were valid, this would be a valuable contribution: CIDQ is a natural and underexplored task that combines candidate-set aggregation, metric reconstruction, and closest-instance selection, and the proposed pipeline is transparent and clearly specified. The scale of the benchmark and the offline use of ground-truth geometry are strengths, as is the honest acknowledgment that the uncertainty metric is not a calibrated confidence interval. However, the benchmark construction shares its detector with the method's inference pipeline, which compromises the central empirical claims. The formal CIDQ formulation, the ablation structure, and the attempt to propagate geometric uncertainty into VLM reasoning are useful, but the evaluation protocol must be reconstructed with an independent candidate-source before the reported accuracies can be accepted as measuring the intended capability.

major comments (3)
  1. [Section V-A.3, Eq. (22), and Section IV-B] The benchmark construction is circular with the method's inference pipeline. Stage 3 retains only MM-Spatial annotations that have a class-consistent GroundingDINO detection with IoU ≥ 0.5, and Phase I uses GroundingDINO to build SPATIALQUERY's candidate set. Any instance GroundingDINO misses is removed from the ground-truth annotation set (after which Stage 1 eligibility is re-applied) and is also absent from the method's input. The evaluation therefore excludes exactly the candidate-set aggregation failures that CIDQ is designed to expose. All headline metrics are conditional on GroundingDINO having succeeded, which also disadvantages baselines that do not use this detector. No GroundingDINO recall on unfiltered annotations is reported, and no evaluation set includes missed instances; the Limitations section is silent on this dependency. The benchmark needs an independent candidate sou
  2. [Section III-B, Eq. (5)] Unc-Acc@δ uses tolerance δ_q + u_q with u_q ≥ 0, so uncertainty-conditioned accuracy is monotone in u_q and is at least Acc@δ by construction. The reported gain (Acc@0.2m 58.50% → Unc-Acc@0.2m 87.50%; Unc-Acc@0.3m 90.50%) is therefore partly mechanical and does not validate σ_q as an informative uncertainty measure. The paper's own caveat that the interval is not calibrated is appropriate, but the RQ2 claim that geometry-derived uncertainty 'identifies unreliable predictions' relies on the supplementary stratified analysis (Floor-MAE 0.349 vs 0.218 for high/low dispersion). That analysis, or a calibration/reliability evaluation, needs to be in the main text before the uncertainty claim is supported.
  3. [Section VI-A and Eq. (4)] Floor-MAE is computed only over valid original-image distance predictions, whereas all accuracy-style metrics use the full 5,000-query denominator and assign zero credit to invalid outputs. If the number of valid predictions differs across methods, Floor-MAE and Acc@0.2m are not directly comparable across rows of Table III, and a method with many invalid outputs can look better in Floor-MAE than in accuracy. Please report per-method valid-prediction counts and evaluate Floor-MAE on a common set when possible.
minor comments (5)
  1. [Section VI-A] The black-image sanity filter gives credit only when a method is wrong on the black image and penalizes methods that refuse to answer or return invalid outputs on the black image. This should be justified and its effect on each method reported separately.
  2. [Section V-A.3] GroundingDINO version, confidence threshold, and the sensitivity of Eq. (22) to θ_match = 0.5 are not reported. Since this threshold directly controls which queries enter the benchmark, a sensitivity analysis is needed.
  3. [Section IV-D] The hyperparameters ε_num, λ_MAD, and K_iter are deferred to the supplementary material. Please provide their values in the main text and report sensitivity of the final metrics to reasonable variations.
  4. [Section V-A.2] Stage 2 uses GPT-4o to filter annotations by visibility with threshold θ_v = 0.7. This may bias the benchmark toward objects that a commercial VLM can recognize; the threshold is not validated.
  5. [Table III] Decimal precision is inconsistent across rows (e.g., 0.5354 vs. 0.259). Use uniform rounding for readability.

Circularity Check

2 steps flagged · score 6.0 of 10

Benchmark ground truth is filtered by the same GroundingDINO detector the method uses at inference, so candidate-set misses are removed from the evaluation and the reported gains are not fully independent.

  1. self definitional [Section V-A.3 (Stage 3, Eq. 22) and Section IV-B (Phase I)]
    "GroundingDINO [23] is queried using the categories represented in \tilde{B}_f, producing detection boxes b^det_{f,p} ... A match (j,p) is admissible only if label(B_j)=\ell^det_{f,p} and IoU(b^proj_{f,j}, b^det_{f,p}) \ge \theta_match, \theta_match=0.5. Annotations with admissible one-to-one matches form \hat{B}_f. ... GroundingDINO is used only to determine annotation retention; the semantic labels, gravity-aligned centroids, and metric quantities of \hat{B}_f remain inherited from MM-Spatial."

    The benchmark's ground-truth candidate set \hat{B}_f is defined as the subset of MM-Spatial annotations that match GroundingDINO detections at IoU \ge 0.5. In Phase I, SPATIALQUERY's candidate set B_q is produced by the same GroundingDINO detector. Thus the candidate instances that define d*_q and the candidate instances the method compares are both determined by the same detector. A visible instance that GroundingDINO misses is removed from the ground truth before Eq. (1) selects the closest candidate, so the method is never penalized for missing it. Since the CIDQ task is specifically designed to test candidate-set aggregation, filtering the ground truth by the method's own detector removes exactly the failure mode the benchmark claims to measure. Other methods do not receive GroundingDI

  2. self definitional [Section III-B, Eq. (5); Section VI-B, RQ2]
    "Unc-Acc@\delta = 1/|Q^T1_eval| \sum_{q\in Q^T1_eval} 1[ |\hat{d}_q - d*_q| \le \delta_q + \hat{u}_q ]. ... Importantly, increased coverage alone does not establish that \sigma_q is an informative reliability signal, since any positive uncertainty margin enlarges the acceptance interval."

    The uncertainty-aware accuracy metric is defined with acceptance tolerance \delta_q + \hat{u}_q. For any non-negative \hat{u}_q, the threshold is strictly larger than the base tolerance \delta_q, so Unc-Acc is automatically at least the corresponding Acc for the same predictions. The paper's RQ2 result that uncertainty increases Acc@0.2 m from 58.50% to Unc-Acc@0.2 m of 87.50% is therefore a necessary consequence of the metric definition, not an empirical demonstration that the uncertainty estimates are informative. The authors disclose this caveat, but the headline Unc-Acc@0.3 m of 90.5% remains a self-definitional quantity rather than independent evidence of uncertainty quality.

full rationale

SPATIALQUERY's geometric derivation itself is not circular: \hat{d}_q is computed from Depth Pro and GroundingDINO outputs and is not fit to the benchmark labels. The load-bearing circularity is in the evaluation design. Stage 3 of benchmark construction retains only MM-Spatial annotations that match GroundingDINO detections at IoU >= 0.5, while Phase I of the method obtains its candidate boxes from the same GroundingDINO detector. The ground-truth candidate set and the method's candidate set are therefore defined by the same component, so candidate-set misses are excluded from the ground truth and cannot be scored as failures. This removes the central CIDQ challenge from the benchmark and biases comparisons against methods that do not use GroundingDINO as privileged input. The Unc-Acc metric is also partly self-definitional because Eq. (5) adds the method's uncertainty to the tolerance, although the paper explicitly acknowledges that this alone does not establish informativeness and provides a stratified analysis. No load-bearing self-citation or imported uniqueness theorem is present. Overall, the central performance claim is partially circular, giving a score of 6.

Assumptions & free parameters 6 free parameters · 6 assumptions · 0 invented entities

The central claim rests on off-the-shelf components (Depth Pro, GroundingDINO, SAM, Qwen-VL, GPT-4o) plus benchmark-construction thresholds. The most consequential premise is the coupling between Stage 3 detector filtering and Phase I detection: it removes detector failures from the evaluation. The free parameters listed are construction and refinement hyperparameters, none of which are fitted to the benchmark labels, but several are not reported with values in the main text.

free parameters (6)
  • N_min = 2
    Stage 1 frame filtering requires at least 2 instances of the query category and exactly 1 reference instance (Eq. 21).
  • theta_v = 0.7
    GPT-4o visibility score threshold for retaining annotations in Stage 2.
  • theta_dup = 0.85
    Class-wise NMS IoU threshold in Stage 3 for deduplicating GroundingDINO detections.
  • theta_match = 0.5
    IoU threshold for admissible Hungarian matches between projected annotations and detections (Eq. 22).
  • RANSAC constants = lambda_MAD, K_iter, epsilon_num unspecified
    Depth-mode consensus parameters in Eqs. (12)-(13); values are deferred to the supplementary material.
  • T2 thresholds = {0.5, 1.0, 1.5, 2.0, 3.0} m
    Decision thresholds for proximity labels in Eq. (23), sampled across distance strata to balance labels.
assumptions (6)
  • domain assumption Depth Pro provides accurate zero-shot metric depth and camera intrinsics.
    Phase II uses Depth Pro output as the metric basis for all floor-plane coordinates; global depth error directly propagates to distance errors.
  • domain assumption GroundingDINO detects all visible candidates with high recall.
    Phase I relies on GroundingDINO for the candidate set; misses cannot be recovered downstream.
  • domain assumption The horizon-row and pitch estimator recovers the gravity direction.
    Eq. (8) computes camera pitch from horizon; wrong pitch corrupts floor-plane projections.
  • domain assumption MM-Spatial and CA-1M annotations are accurate ground truth.
    All benchmark labels inherit gravity-aligned centroids and camera parameters from these datasets.
  • domain assumption GPT-4o zero-shot visibility scores correctly discard occluded or truncated instances.
    Stage 2 uses thresholded LLM scores as the only visibility filter before geometric refinement.
  • ad hoc to paper Benchmark Stage 3 keeps exactly the annotations that the method's own detector can find.
    The Hungarian IoU matching (Eq. 22) with GroundingDINO boxes defines the final annotation set, coupling the ground truth to the inference detector.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SpatialQuery: Benchmarking Geometry-Grounded Multi-Instance Spatial Reasoning in Vision-Language Models." pith.science (2026). https://pith.science/paper/TLCTVMVG

@misc{pith2026260801709,
  author       = {Pith},
  title        = {Pith review of: SpatialQuery: Benchmarking Geometry-Grounded Multi-Instance Spatial Reasoning in Vision-Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TLCTVMVG}},
  note         = {Machine review of arXiv:2608.01709}
}
read the original abstract

Vision-language models (VLMs) achieve strong semantic understanding but remain unreliable in metric spatial reasoning, particularly when queries require comparing multiple instances of the same object category. We study this problem through the Closest-Instance Distance Query (CIDQ), where a model must identify the nearest visible candidate to a unique reference object and estimate their gravity-aligned floor-plane distance. We introduce SPATIALQUERY, a training- free framework for CIDQ reasoning from a single RGB image, together with SPATIALQUERY-1M, a benchmark containing over one million RGB-only question-answer pairs from 200 indoor scenes. SPATIALQUERY recovers instance-level metric geometry and transforms it into a canonical Bird's-Eye View through Scene Cubifying, which represents objects as uniformly sized, category-coded blocks to emphasize their relative floor- plane locations. We further propose Uncertainty-Aware Chain-of-Thought (UA-CoT) prompting, which incorporates geometry- derived per-instance uncertainty into the VLM reasoning process. Without task-specific fine-tuning or architectural modification, SPATIALQUERY with Qwen3-VL-8B achieves a Floor-MAE of 0.259 m, an Unc-Acc@0.3 m of 90.5%, and a proximity-decision accuracy of 84.18%, outperforming fine-tuned spatial specialists, general-purpose VLMs, and closed-source frontier models. Code, benchmark resources, and an interactive demo are available at https://namhai1810.github.io/SpatialQuery/.

Figures

Figures reproduced from arXiv: 2608.01709 by the authors.

Figure 1
Figure 1. SPATIALQUERY Pipeline. Phase I: Qwen-VL [22] identifies entity roles; GroundingDINO [23] and SAM [24] localize and segment all instances. Phase II: Depth Pro [9] estimates zero-shot metric depth and camera intrinsics; a geometric estimator recovers the camera pitch angle ϕˆ; instance pixels are back-projected into metric camera-frame point clouds. Phase III: MAD-based RANSAC filters unreliable depth support; the ref… view at source ↗
Figure 2
Figure 2. Four-stage QA synthesis pipeline for SPATIALQUERY-1M. Indoor frames from MM-Spatial pass through: Stage 1 (Frame Filtering) — scenes are retained based on object frequency analysis, requiring at least one unique reference instance and at least one category with at least two co-occurring instances; Stage 2 (LLM Verification) — ChatGPT zero-shot prompting assigns probabilistic visibility scores to each instance, disca… view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

35 extracted references · 25 canonical work pages

  1. [1]

    MineDojo: Building open- ended embodied agents with internet-scale knowledge,

    L. Fan, G. Wang, Y . Jiang, A. Mandlekar, Y . Yang, H. Zhu, A. Tang, D.-A. Huang, Y . Zhu, and A. Anandkumar, “MineDojo: Building open- ended embodied agents with internet-scale knowledge,” inProc. Adv. Neural Inf. Process. Syst. (NeurIPS), New Orleans, LA, USA, Nov. 2022

    work page 2022

  2. [2]

    Guiding long-horizon task and motion planning with vision language models,

    Z. Yang, C. Garrett, D. Fox, T. Lozano-Pérez, and L. P. Kaelbling, “Guiding long-horizon task and motion planning with vision language models,” inProc. IEEE Int. Conf. Robot. Autom. (ICRA), Atlanta, GA, USA, May 2025

    work page 2025

  3. [3]

    RoboSpatial: Teaching spatial understanding to 2D and 3D vision- language models for robotics,

    C. H. Song, V . Blukis, J. Tremblay, S. Tyree, Y . Su, and S. Birchfield, “RoboSpatial: Teaching spatial understanding to 2D and 3D vision- language models for robotics,” inProc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Nashville, TN, USA, Jun. 2025

    work page 2025

  4. [4]

    GPT-4o system card,

    A. Hurstet al., “GPT-4o system card,”arXiv preprint arXiv:2410.21276, Oct. 2024. [Online]. Available: https://arxiv.org/abs/2410.21276

    arXiv 2024

  5. [5]

    Qwen2.5-VL technical report,

    S. Bai, K. Chen, X. Liuet al., “Qwen2.5-VL technical report,”arXiv preprint arXiv:2502.13923, Feb. 2025

    arXiv 2025

  6. [6]

    Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context,

    Gemini Team, P. Georgiev, V . I. Leiet al., “Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context,”arXiv preprint arXiv:2403.05530, Mar. 2024

    arXiv 2024

  7. [7]

    SpatialVLM: Endowing vision-language models with spatial reasoning capabilities,

    B. Chen, Z. Xu, S. Kirmani, B. Ichter, D. Sadigh, L. Guibas, and F. Xia, “SpatialVLM: Endowing vision-language models with spatial reasoning capabilities,” inProc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Seattle, W A, USA, Jun. 2024

    work page 2024

  8. [8]

    SpatialRGPT: Grounded spatial reasoning in vision language models,

    A.-C. Cheng, H. Yin, Y . Fu, Q. Guo, R. Yang, J. Kautz, X. Wang, and S. Liu, “SpatialRGPT: Grounded spatial reasoning in vision language models,” inProc. Adv. Neural Inf. Process. Syst. (NeurIPS), Vancouver, BC, Canada, Dec. 2024

    work page 2024

Show all 35 references
  1. [9]

    Depth Pro: Sharp monocular metric depth in less than a second,

    A. Bochkovskii, A. Delaunoy, H. Germain, M. Santos, Y . Zhou, S. R. Richter, and V . Koltun, “Depth Pro: Sharp monocular metric depth in less than a second,” inProc. Int. Conf. Learn. Represent. (ICLR), Singapore, Apr. 2025

    2025

  2. [10]

    SpatialPIN: Enhancing spatial reasoning capabilities of vision-language models through prompting and interacting 3D priors,

    C. Ma, K. Lu, T.-Y . Cheng, N. Trigoni, and A. Markham, “SpatialPIN: Enhancing spatial reasoning capabilities of vision-language models through prompting and interacting 3D priors,” inProc. Adv. Neural Inf. Process. Syst. (NeurIPS), Vancouver, BC, Canada, Dec. 2024

    2024

  3. [11]

    Spatial reasoning with vision-language models in ego-centric multi-view scenes,

    M. Gholami, A. Rezaei, Z. Weimin, S. Mao, S. Zhou, Y . Zhang, and M. Akbari, “Spatial reasoning with vision-language models in ego-centric multi-view scenes,” Sep. 2025. [Online]. Available: https://arxiv.org/abs/2509.06266

    2025

  4. [12]

    Talk2BEV: Language-enhanced bird’s-eye view maps for autonomous driving,

    T. Choudhary, V . Dewangan, S. Chandhok, S. Priyadarshan, A. Jain, A. K. Singh, S. Srivastava, K. M. Jatavallabhula, and K. M. Krishna, “Talk2BEV: Language-enhanced bird’s-eye view maps for autonomous driving,” inProc. IEEE Int. Conf. Robot. Autom. (ICRA), Yokohama, Japan, May 2024

    2024

  5. [13]

    BLINK: Multimodal large language models can see but not perceive,

    X. Fu, Y . Hu, B. Li, Y . Feng, H. Wang, X. Lin, D. Roth, N. A. Smith, W.- C. Ma, and R. Krishna, “BLINK: Multimodal large language models can see but not perceive,” inProc. Eur. Conf. Comput. Vis. (ECCV), Milan, Italy, Sep. 2024

    2024

  6. [14]

    Does spatial cognition emerge in frontier models?

    S. K. Ramakrishnan, E. Wijmans, P. Krähenbühl, and V . Koltun, “Does spatial cognition emerge in frontier models?” inProc. Int. Conf. Learn. Represent. (ICLR), Singapore, Apr. 2025

    2025

  7. [15]

    3DSRBench: A comprehensive 3D spatial reasoning bench- mark,

    W. Ma, H. Chen, G. Zhang, Y .-C. Chou, J. Chen, C. de Melo, and A. Yuille, “3DSRBench: A comprehensive 3D spatial reasoning bench- mark,” inProc. IEEE/CVF Int. Conf. Comput. Vis. (ICCV), Honolulu, HI, USA, Oct. 2025, pp. 6924–6934

    2025

  8. [16]

    Do vision-language models represent space and how? evaluating spatial frame of reference under ambiguities,

    Z. Zhang, F. Hu, J. Lee, F. Shi, P. Kordjamshidi, J. Chai, and Z. Ma, “Do vision-language models represent space and how? evaluating spatial frame of reference under ambiguities,” inProc. Int. Conf. Learn. Represent. (ICLR), Singapore, Apr. 2025

    2025

  9. [17]

    Seeing through their eyes: Evaluating visual perspective taking in vision language models,

    G. Goral, A. Ziarko, M. Nauman, and M. Wołczyk, “Seeing through their eyes: Evaluating visual perspective taking in vision language models,” arXiv preprint arXiv:2409.12969, Sep. 2024

    2024 arXiv

  10. [18]

    Perspective- aware reasoning in vision-language models via mental imagery simula- tion,

    P. Y . Lee, J. Je, C. Park, M. A. Uy, L. Guibas, and M. Sung, “Perspective- aware reasoning in vision-language models via mental imagery simula- tion,” inProc. IEEE/CVF Int. Conf. Comput. Vis. (ICCV), Honolulu, HI, USA, Oct. 2025, pp. 9241–9251

    2025

  11. [19]

    Chain-of-thought prompting elicits reasoning in large language models,

    J. Wei, X. Wang, D. Schuurmans, M. Bosma, F. Xia, E. Chi, Q. V . Le, D. Zhouet al., “Chain-of-thought prompting elicits reasoning in large language models,” inProc. Adv. Neural Inf. Process. Syst. (NeurIPS), New Orleans, LA, USA, Nov. 2022

    2022

  12. [20]

    Set-of-mark prompting unleashes extraordinary visual grounding in GPT-4V,

    J. Yang, H. Zhang, F. Li, X. Zou, C. Li, and J. Gao, “Set-of-mark prompting unleashes extraordinary visual grounding in GPT-4V,”arXiv preprint arXiv:2310.11441, Oct. 2023

    2023 arXiv

  13. [21]

    Visual sketchpad: Sketching as a visual chain of thought for multimodal language models,

    Y . Hu, W. Shi, X. Fu, D. Roth, M. Ostendorf, L. Zettlemoyer, N. A. Smith, and R. Krishna, “Visual sketchpad: Sketching as a visual chain of thought for multimodal language models,” inProc. Adv. Neural Inf. Process. Syst. (NeurIPS), Vancouver, BC, Canada, Dec. 2024

    2024

  14. [22]

    Qwen-VL: A versatile vision-language model for un- derstanding, localization, text reading, and beyond,

    J. Baiet al., “Qwen-VL: A versatile vision-language model for un- derstanding, localization, text reading, and beyond,”arXiv preprint arXiv:2308.12966, Aug. 2023

    2023 arXiv

  15. [23]

    Grounding DINO: Marrying DINO with grounded pre- training for open-set object detection,

    S. Liuet al., “Grounding DINO: Marrying DINO with grounded pre- training for open-set object detection,” inProc. Eur. Conf. Comput. Vis. (ECCV), Milan, Italy, Sep. 2024

    2024

  16. [24]

    SAM 3: Segment anything with concepts,

    N. Carionet al., “SAM 3: Segment anything with concepts,” Nov

  17. [25]

    MM-Spatial: Exploring 3D spatial understanding in multimodal LLMs,

    E. Daxberger, N. Wenzel, D. Griffithset al., “MM-Spatial: Exploring 3D spatial understanding in multimodal LLMs,” inProc. IEEE/CVF Int. Conf. Comput. Vis. (ICCV), Honolulu, HI, USA, Oct. 2025

    2025

  18. [26]

    Cubify anything: Scaling indoor 3D object detection,

    J. Lazarow, D. Griffiths, G. Kohavi, F. Crespo, and A. Dehghan, “Cubify anything: Scaling indoor 3D object detection,” inProc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Nashville, TN, USA, Jun. 2025, pp. 22 225–22 233

    2025

  19. [27]

    Visual spatial reasoning,

    F. Liu, G. Emerson, and N. Collier, “Visual spatial reasoning,”Trans. Assoc. Comput. Linguist. (TACL), vol. 11, pp. 635–651, 2023

    2023

  20. [28]

    SQA3D: Situated question answering in 3D scenes,

    X. Ma, S. Yong, Z. Zheng, Q. Li, Y . Liang, S.-C. Zhu, and S. Huang, “SQA3D: Situated question answering in 3D scenes,” inProc. Int. Conf. Learn. Represent. (ICLR), Kigali, Rwanda, May 2023. [Online]. Available: https://openreview.net/forum?id=IDJx97BC38

    2023

  21. [29]

    EmbodiedScan: A holistic multi-modal 3D perception suite towards embodied AI,

    T. Wanget al., “EmbodiedScan: A holistic multi-modal 3D perception suite towards embodied AI,” inProc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Seattle, W A, USA, Jun. 2024. [Online]. Available: https://arxiv.org/abs/2312.16170

    2024 arXiv

  22. [30]

    GPT-4o mini: Advancing cost-efficient intelligence,

    OpenAI, “GPT-4o mini: Advancing cost-efficient intelligence,” https: //openai.com/index/gpt-4o-mini-advancing-cost-efficient-intelligence/, Jul. 2024

    2024

  23. [31]

    Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities,

    G. Comaniciet al., “Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities,”arXiv preprint arXiv:2507.06261, Jul. 2025. [Online]. Available: https://arxiv.org/abs/2507.06261

    2025 arXiv

  24. [32]

    SpaceThinker-Qwen2.5VL-3B: A thinking/reasoning VLM for quantitative spatial reasoning,

    Remyx AI, “SpaceThinker-Qwen2.5VL-3B: A thinking/reasoning VLM for quantitative spatial reasoning,” https://huggingface.co/remyxai/ SpaceThinker-Qwen2.5VL-3B, 2025

    2025

  25. [33]

    SpaceOm: Spatial reasoning with extended thinking traces,

    RemyxAI, “SpaceOm: Spatial reasoning with extended thinking traces,” https://huggingface.co/remyxai/SpaceOm, 2025

    2025

  26. [34]

    Spatial-SSRL: Enhancing spatial understanding via self- supervised reinforcement learning,

    Y . Liu, B. Zhang, Y . Zang, Y . Cao, L. Xing, X. Dong, H. Duan, D. Lin, and J. Wang, “Spatial-SSRL: Enhancing spatial understanding via self- supervised reinforcement learning,” inProc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2026, pp. 9570–9581

    2026

  27. [2025]

    Available: https://arxiv.org/abs/2511.16719

    [Online]. Available: https://arxiv.org/abs/2511.16719

    arXiv

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.