Pith. sign in

REVIEW 2 major objections 4 minor 36 references

This paper claims that image-generation models are genuinely competitive with text-output VLMs on spatial tasks when answers can be drawn directly in pixels, while text models keep the edge when conclusions require multi-step reasoning.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 08:32 UTC pith:B42LD3RD

load-bearing objection A genuinely useful and unusually self-audited framework for scoring image-generation models on text-oriented spatial benchmarks, though the 'clear text advantage in reasoning' claim is softer than the abstract suggests once you look at the unnamed Prediction parser and the overlapping confidence intervals. the 2 major comments →

arxiv 2607.21072 v1 pith:B42LD3RD submitted 2026-07-23 cs.CV

Show, Don't Tell: Evaluating Spatial Cognition in Generative Pixels Rather Than LLM Text

Xu Wang , Kaixiang Yao , Miao Pan , Xiaohe Zhou , Xuanyu Liu , Wenqi Zhang , Xuhong Zhang This is my paper
classification cs.CV
keywords spatial cognitionimage generationvision-language modelsbenchmarkvisual answer interfacepixel-space externalizationspatial reasoningprotocolized evaluation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper argues that the way a spatial answer is expressed changes what we can measure: image-generation models can be fairly compared with text-output VLMs if they are allowed to draw, mark, or color the answer directly in the image, and the drawing is then parsed back into the benchmark's original scoring format. To do this it introduces ProVisE, a protocol system where each task gets a fixed visual response format and a deterministic parser, plus an agentic builder that writes such protocols for new benchmarks, and SpatialGen-Bench, 470 curated samples across 14 spatial tasks. The headline finding is complementary strengths: the best visual-answering model matches or beats the best text model on depth and relationship questions, where the answer stays visible in pixels, while text models keep a clear lead on counting, size, feasibility, prediction, and compositional reasoning. If the parsing is faithful, the study shows pixel-space externalization is a real answer interface, not a degraded version of text answering.

Core claim

On the paper's own terms, the discovery is that image-generation models can be fairly scored on text-oriented spatial benchmarks when their answers are drawn: with protocols frozen before evaluation, the best visual system (GPT Image 2) scores 54.49 overall versus 61.04 for the best text system (GPT-5.4) on the same 470 items, and it leads on relative depth and relationship verification while rescuing 71 items that the best text model misses. Text models keep a clear advantage on counting, size, geometric feasibility, state prediction, and multi-hop reasoning. The paper argues this pattern shows complementary strengths — pixel-space output preserves continuous geometry and lets the answer st

What carries the argument

A visual protocol: a fixed guidance prompt that tells the image model how to express its answer in pixels (green marks for counting, a blue grid cell for direction, a grayscale depth field, a binary region mask, a drawn trajectory, and so on), paired with a parser that deterministically converts the rendered image into the structured prediction the original benchmark expects. The parser pool includes color thresholding, region masking, CLIP state matching, trajectory skeletonization, and, where no deterministic operator exists, a Fallback route that reads only the generated image under a fixed contract. The same protocol is frozen and shared by all models, so differences in scores reflect mo

Load-bearing premise

The results assume the parsers — especially the fixed auxiliary VLM on the Prediction task — recover the visual answer exactly as a human reader would, so that score gaps come from model spatial ability, not from the reading of the images; the paper itself flags Fallback parser variability as a limitation.

What would settle it

Re-annotate the 275 Prediction generated images: have several independent human readers record the circled hole index in each trajectory image and compare with the fixed auxiliary VLM's readings; if human-VLM agreement is poor on valid visual answers, the reported text advantage on Reasoning is partly a parsing artifact. Also rerun visual Reasoning scores with human readings substituted.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Pixel-space externalization is a usable answer interface: with shared protocols, image-generation models can be scored by the original benchmark metrics, so visual answering is not merely a qualitative artifact.
  • Visual answering solves items missed by every text system tested, so the two interfaces are complementary rather than one being a subset of the other.
  • A universal black-box parser changes both scores and model rankings, meaning the decoding of generated evidence is part of the measurement and cannot be assumed neutral.
  • Most visual failures (88%) are valid but incorrect spatial predictions, not generation, protocol, or parser failures — so the bottleneck for image generators is spatial reasoning itself.
  • Agentic protocol construction transfers to six external benchmarks, so the method generalizes beyond SpatialGen-Bench.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A natural extension the paper leaves implicit is a routing system that sends each task or sample to the interface most likely to solve it; the complementarity data suggest such a hybrid would outperform either interface alone.
  • Because the Prediction task's parser is an auxiliary VLM reading generated images, the measured text advantage in Reasoning should be read with that caveat; if human agreement with that reader is ever measured, the visual side could prove stronger than reported.
  • The same protocolized-answer idea could extend to video and embodied tasks, where answers are actions rather than static images; the paper names this as future work, so this is an inference, not a result.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. This paper proposes ProVisE, a benchmark-agnostic framework for evaluating image-generation models on spatial benchmarks by eliciting protocol-constrained visual answers and parsing them into metric-compatible predictions, together with SpatialGen-Bench, a 470-sample diagnostic benchmark spanning 14 spatial subtasks across four capability levels. The authors evaluate 20 text-output VLMs and 11 visual-answering systems under fixed cross-model protocols, report complementary strengths (visual advantages on Depth and Relationship, text advantages on Size, Feasibility, Prediction, and Counting), validate Agentic protocol construction on six external benchmarks, and show that substituting universal VLM parsers changes model scores and rankings (e.g., top-model identity flips). The central claim is that image-generation models are competitive when spatial answers can be externalized directly in pixel space, while text-output VLMs retain a clear advantage in compositional spatial reasoning.

Significance. If the result holds, the paper provides a useful, falsifiable framework for comparing image-generation models and text-output VLMs under shared task semantics. The design is careful in several respects: protocols are frozen before evaluation, deterministic parsers have explicit invalid-output rules, bootstrap confidence intervals are reported for all systems, the parser-sensitivity experiment honestly demonstrates ranking instability under VLM-based scoring, and the external pilots test end-to-end transfer. The central risk is concentrated in the Prediction-task scorer, which relies on an unnamed fixed auxiliary VLM with a high parser-failure rate, and in the interpretation of task-level failure attribution. These issues are fixable with additional analysis, so the paper merits a major revision rather than rejection.

major comments (2)
  1. [Appendix C.1/C.2, Table 8, Table 15, §4.6, Eq. (3)] This is the same comment.
  2. [Table 15, §4.6, Fig. 6] This is a distinct concern.
minor comments (4)
  1. [Title] The running header contains 'Show, Don’t T ell' with a space in 'Tell'; fix typographical spacing.
  2. [Appendix E.1] The Human reference row is used to support the 'substantially behind humans' claim, but no details are given about the number of raters, their instructions, or inter-rater agreement. Please provide these methodological details.
  3. [Table 5] The ProVisE column for GPT Image 2 (51.63) differs from the main result in Table 2 (54.49). The availability mask and 36 missing artifacts are explained in the text and Appendix C.6, but a footnote in Table 5 would prevent confusion for readers comparing the two tables.
  4. [§4.6, Fig. 7] The aggregate statement that 88.03% of non-correct outcomes are valid but incorrect predictions aggregates over tasks with very different noncompliance rates (e.g., Size and Feasibility at 27–38%). Consider reporting the breakdown by capability or noting the task-level heterogeneity in the main text.

Circularity Check

0 steps flagged

No significant circularity: the evaluation pipeline is frozen before model comparison, parsers are deterministic or fixed, and the central claim is an empirical comparison under shared metrics.

full rationale

The paper does not exhibit a circular derivation. ProVisE's protocols and parsers are fixed before target-model evaluation (Sec. 2.3: 'these elements are fixed before model evaluation'; Sec. 4.1: 'Each task's visual protocol is frozen before target-model evaluation'), and smoke validation explicitly excludes correctness ('Correctness is diagnostic rather than an acceptance gate,' Appendix C.3). The main benchmark scores come from deterministic parsers (HSV thresholding, geometry, CLIP similarity, grayscale sampling) plus one fixed auxiliary VLM for the Prediction task; none of these are fitted to the model outputs or to the benchmark answers, and the auxiliary VLM reads only the generated image, not the source image or original question (Sec. 2.3). The six external benchmarks are out-of-sample transfer pilots with protocols shared across models. The central claim (visual advantage on directly externalizable tasks, text advantage in compositional reasoning) is an empirical result under a common metric, not an identity or a fitted parameter renamed as a prediction. The paper explicitly acknowledges protocol-construction bias and provider-dependent Fallback variability in Sec. 4.7; these are validity limitations, not circularity. The overlap between construction backends and evaluated models (GPT-5.4/GPT Image 2) is a transparency caveat, and because construction does not use correctness feedback, it does not make the evaluation tautological. Concerns about the Relationship protocol measuring grounding rather than relation reasoning are measurement-validity issues, not circular reductions; the paper's Reasoning-level results rely on protocols where the model directly selects or renders the answer.

Axiom & Free-Parameter Ledger

3 free parameters · 5 axioms · 0 invented entities

No new physical or ontological entities are postulated. 'Visual protocols' are methodological artifacts (prompt + parser + invalid-output rules), not entities, and they carry no independent falsifiable handle outside the benchmark results. The free parameters above are hand-chosen evaluation thresholds rather than fitted model parameters; the most consequential are the Prediction partial-credit rule and the DFD threshold.

free parameters (3)
  • DFD success threshold = 0.4
    Trajectory Planning score uses Discrete Fréchet Distance < 0.4 as binary success (Eq. 4). No sensitivity analysis is reported for this threshold, and it gates one of the four Interaction tasks.
  • Prediction adjacent-hole partial credit = 0.8 − 1/H
    Eq. 3 assigns 1.0 for exact endpoint, 0.8 − 1/H for adjacent-hole errors, 0 otherwise. These constants are chosen by hand and affect the Reasoning-level aggregate where text models lead.
  • Parser color/morphology thresholds = e.g., H=35–90, S/V≥80; ≥50 blue; ≥30 magenta pixels
    Table 8 encodes hand-set HSV ranges and minimum pixel/solidity cutoffs for every deterministic parser; invalid-output boundaries determine zero-scoring rates (e.g., Prediction 24.4% parser failures, Table 15).
axioms (5)
  • domain assumption A protocol-constrained generated image expresses the model's spatial judgment, and deterministic parsing recovers it without imposing the answer.
    The whole framework depends on this (§2.1–2.3). The paper validates it partially via failure attribution: 88.03% of non-correct outcomes are valid-but-wrong predictions, so wrong answers do pass the parsers (Fig. 7).
  • domain assumption Text answers (A/B, labels, coordinates) and protocol-rendered visual answers measure the same underlying spatial competence under 'shared task semantics'.
    Central to the complementarity claim (§2.4, §4.3). The interfaces change the required operation (choose closer point vs. render a depth map with a brighter-is-closer convention), so equal scores do not strictly imply equal spatial competence; the paper itself hedges 'not a causal modality effect' (§4.3, §4.7).
  • domain assumption Fixed CLIP ViT-B/32 cosine similarity is a reliable state/candidate matcher.
    Used in State Matching and Marked-Object Choice parsers (Table 8); no independent validation of CLIP's separation reliability on the specific task domains.
  • domain assumption Source benchmark annotations and metrics are valid ground truth.
    SpatialGen-Bench and the six external benchmarks are taken as ground truth; curation filters ambiguous or multi-answer items (Appendix B), but the residual annotation quality is inherited from source datasets.
  • domain assumption Benchmark-provided A/B coordinates for Depth do not leak which point is closer.
    Depth parser samples grayscale at benchmark-supplied coordinates (§C.5); the paper states the coordinates 'identify where to read the generated depth map without revealing which point is closer.' This is a claim about the source benchmark's annotation design.

pith-pipeline@v1.3.0-alltime-deepseek · 35669 in / 21940 out tokens · 223927 ms · 2026-08-01T08:32:52.589192+00:00 · methodology

0 comments
read the original abstract

Spatial intelligence is essential for agents to move from static semantic understanding toward interacting with the physical world. Many spatial tasks are grounded in continuous visual scenes, where locations, regions, and paths are more naturally expressed by pointing, marking, or drawing than by reporting precise coordinates or discrete textual symbols. Yet existing spatial reasoning benchmarks usually require coordinates, options, or text, creating an answer-interface mismatch for image-generation models. This makes it difficult to evaluate image-generation models under the same task semantics as text-output VLMs, despite their ability to externalize spatial judgments directly in pixel space. We propose ProVisE (Protocolized Visual Evaluation), a benchmark-agnostic framework that elicits protocol-constrained visual answers from image-generation models and parses them into structured predictions compatible with original metrics. ProVisE also includes an Agentic builder that constructs and validates task-specific protocols for new benchmarks. We further introduce SpatialGen-Bench, a curated diagnostic benchmark of 470 samples across 14 spatial subtasks, four capability levels, and diverse answer forms. We evaluate representative text-output VLMs and image-generation models in a unified setting and validate Agentic protocol construction on six external spatial benchmarks. Results show that image-generation models are competitive when spatial answers can be externalized directly in pixel space, while text-output VLMs retain a clear advantage in compositional spatial reasoning. These findings reveal complementary strengths of pixel-space expression and text-based reasoning and establish a metric-compatible testbed for studying spatial cognition in image-generation models.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

36 extracted references · 19 linked inside Pith

  1. [1]

    Gpt-4 technical report.arXiv preprint arXiv:2303.08774,

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report.arXiv preprint arXiv:2303.08774,

    Pith/arXiv arXiv

  2. [4]

    Wei Chow, Jiageng Mao, Boyi Li, Daniel Seita, Vitor Guizilini, and Yue Wang

    https://arxiv.org/abs/2310.18235. Wei Chow, Jiageng Mao, Boyi Li, Daniel Seita, Vitor Guizilini, and Yue Wang. Physbench: Benchmarking and enhancing vision-language models for physical world understanding, 2025.https://arxiv.org/abs/2501.16411. Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Bliste...

    Pith/arXiv arXiv 2025

  3. [5]

    Yufeng Cui, Honghao Chen, Haoge Deng, Xu Huang, Xinghang Li, Jirong Liu, Yang Liu, Zhuoyan Luo, Jinsheng Wang, Wenxuan Wang, et al. Emu3. 5: Native multimodal models are world learners.arXiv preprint arXiv:2510.26583 ,

    Pith/arXiv arXiv

  4. [8]

    Xingyu Fu, Yushi Hu, Bangzheng Li, Yu Feng, Haoyu Wang, Xudong Lin, Dan Roth, Noah A Smith, Wei-Chiu Ma, and Ranjay Krishna

    https://arxiv.org/abs/2505.20279. Xingyu Fu, Yushi Hu, Bangzheng Li, Yu Feng, Haoyu Wang, Xudong Lin, Dan Roth, Noah A Smith, Wei-Chiu Ma, and Ranjay Krishna. Blink: Multimodal large language models can see but not perceive. InEuropean Conference on Computer Vision , pages 148–166. Springer,

    Pith/arXiv arXiv

  5. [9]

    Image generators are generalist vision learners.arXiv preprint arXiv:2604.20329,

    Valentin Gabeur, Shangbang Long, Songyou Peng, Paul Voigtlaender, Shuyang Sun, Yanan Bao, Karen Truong, Zhicheng Wang, Wenlei Zhou, Jonathan T Barron, et al. Image generators are generalist vision learners.arXiv preprint arXiv:2604.20329,

    Pith/arXiv arXiv

  6. [12]

    Robobrain: A unified brain model for robotic manipulation from abstract to concrete.arXiv preprint arXiv:2502.21257,

    Yuheng Ji, Huajie Tan, Jiayu Shi, Xiaoshuai Hao, Yuan Zhang, Hengyuan Zhang, Pengwei Wang, Mengdi Zhao, Yao Mu, Pengju An, et al. Robobrain: A unified brain model for robotic manipulation from abstract to concrete.arXiv preprint arXiv:2502.21257,

    Pith/arXiv arXiv

  7. [13]

    Omnispatial: Towards comprehensive spatial reasoning benchmark for vision language models.arXiv preprint arXiv:2506.03135 ,

    Mengdi Jia, Zekun Qi, Shaochen Zhang, Wenyao Zhang, Xinqiang Yu, Jiawei He, He Wang, and Li Yi. Omnispatial: Towards comprehensive spatial reasoning benchmark for vision language models.arXiv preprint arXiv:2506.03135 ,

    arXiv

  8. [14]

    Joyai-image: Awakening spatial intelligence in unified multimodal understanding and generation

    Joy Future Academy, JD. Joyai-image: Awakening spatial intelligence in unified multimodal understanding and generation. Technical report, 2026.https://joyai-image.s3.cn-north-1.jdcloud-oss.com/JoyAI-Image.pdf . Ji Hyeok Jung, Eun Tae Kim, Seoyeon Kim, Joo Ho Lee, Bumsoo Kim, and Buru Chang. Is ‘right’right? enhancing object orientation understanding in mu...

    2026

  9. [15]

    Black Forest Labs, Stephen Batifol, Andreas Blattmann, Frederic Boesel, Saksham Consul, Cyril Diagne, Tim Dockhorn, Jack English, Zion English, Patrick Esser, et al. Flux. 1 kontext: Flow matching for in-context image generation and editing in latent space.arXiv preprint arXiv:2506.15742 ,

    Pith/arXiv arXiv

  10. [16]

    Genai-bench: Evaluating and improving compositional text-to-visual generation.arXiv preprint arXiv:2406.13743,

    Baiqi Li, Zhiqiu Lin, Deepak Pathak, Jiayao Li, Yixin Fei, Kewen Wu, Tiffany Ling, Xide Xia, Pengchuan Zhang, Graham Neubig, et al. Genai-bench: Evaluating and improving compositional text-to-visual generation.arXiv preprint arXiv:2406.13743,

    Pith/arXiv arXiv

  11. [17]

    Viewspatial-bench: Evaluating multi-perspective spatial localization in vision-language models

    Dingming Li, Hongxing Li, Zixuan Wang, Yuchen Yan, Hang Zhang, Siqi Chen, Guiyang Hou, Shengpei Jiang, Wenqi Zhang, Yongliang Shen, et al. Viewspatial-bench: Evaluating multi-perspective spatial localization in vision-language models. arXiv preprint arXiv:2505.21500 , 2025a. 15 Hongxing Li, Dingming Li, Zixuan Wang, Yuchen Yan, Hang Wu, Wenqi Zhang, Yongl...

    arXiv 2024

  12. [18]

    Ssr: Enhancing depth perception in vision-language models via rationale-guided spatial reasoning.arXiv preprint arXiv:2505.12448,

    Yang Liu, Ming Ma, Xiaomin Yu, Pengxiang Ding, Han Zhao, Mingyang Sun, Siteng Huang, and Donglin Wang. Ssr: Enhancing depth perception in vision-language models via rationale-guided spatial reasoning.arXiv preprint arXiv:2505.12448,

    arXiv

  13. [20]

    Roni Paiss, Ariel Ephrat, Omer Tov, Shiran Zada, Inbar Mosseri, Michal Irani, and Tali Dekel

    https://arxiv.org/abs/2504.01805. Roni Paiss, Ariel Ephrat, Omer Tov, Shiran Zada, Inbar Mosseri, Michal Irani, and Tali Dekel. Teaching clip to count to ten. In Proceedings of the IEEE/CVF international conference on computer vision , pages 3170–3180,

    Pith/arXiv arXiv

  14. [21]

    Sat: Dynamic spatial aptitude training for multimodal language models

    Arijit Ray, Jiafei Duan, Ellis Brown, Reuben Tan, Dina Bashkirova, Rose Hendrix, Kiana Ehsani, Aniruddha Kembhavi, Bryan A Plummer, Ranjay Krishna, et al. Sat: Dynamic spatial aptitude training for multimodal language models. arXiv preprint arXiv:2412.07755 ,

    arXiv

  15. [22]

    Seedream 4.0: Toward next-generation multimodal image generation.arXiv preprint arXiv:2509.20427,

    Team Seedream, Yunpeng Chen, Yu Gao, Lixue Gong, Meng Guo, Qiushan Guo, Zhiyao Guo, Xiaoxia Hou, Weilin Huang, Yixuan Huang, et al. Seedream 4.0: Toward next-generation multimodal image generation.arXiv preprint arXiv:2509.20427,

    Pith/arXiv arXiv

  16. [23]

    Huajie Tan, Enshen Zhou, Zhiyu Li, Yijie Xu, Yuheng Ji, Xiansheng Chen, Cheng Chi, Pengwei Wang, Huizhu Jia, Yulong Ao, et al

    https://arxiv.org/abs/2503.19707. Huajie Tan, Enshen Zhou, Zhiyu Li, Yijie Xu, Yuheng Ji, Xiansheng Chen, Cheng Chi, Pengwei Wang, Huizhu Jia, Yulong Ao, et al. Robobrain 2.5: Depth in sight, time in mind. arXiv preprint arXiv:2601.14352 ,

    Pith/arXiv arXiv

  17. [24]

    Yingbo Tang, Lingfeng Zhang, Shuyi Zhang, Yinuo Zhao, and Xiaoshuai Hao

    https://arxiv.org/abs/2601.14352. Yingbo Tang, Lingfeng Zhang, Shuyi Zhang, Yinuo Zhao, and Xiaoshuai Hao. Roboafford: A dataset and benchmark for enhancing object and spatial affordance learning in robot manipulation. InProceedings of the 33rd ACM International Conference on Multimedia , pages 12706–12713,

    arXiv

  18. [25]

    org/10.1145/3746027.3758209

    doi: 10.1145/3746027.3758209.https://doi. org/10.1145/3746027.3758209. V Team, Wenyi Hong, Wenmeng Yu, Xiaotao Gu, Guo Wang, Guobing Gan, Haomiao Tang, Jiale Cheng, Ji Qi, Junhui Ji, Lihang Pan, Shuaiqi Duan, Weihan Wang, Yan Wang, Yean Cheng, Zehai He, Zhe Su, Zhen Yang, Ziyang Pan, Aohan Zeng, Baoxu Wang, Bin Chen, Boyan Shi, Changyu Pang, Chenhui Zhang...

    arXiv

  19. [26]

    16 Shengbang Tong, Ellis Brown, Penghao Wu, Sanghyun Woo, Manoj Middepogu, Sai C Akula, Jihan Yang, Shusheng Yang, Adithya Iyer, Xichen Pan, et al

    https://arxiv.org/abs/2507.01006. 16 Shengbang Tong, Ellis Brown, Penghao Wu, Sanghyun Woo, Manoj Middepogu, Sai C Akula, Jihan Yang, Shusheng Yang, Adithya Iyer, Xichen Pan, et al. Cambrian-1: A fully open, vision-centric exploration of multimodal llms. Advances in Neural Information Processing Systems , 37:87310–87356,

    Pith/arXiv arXiv

  20. [27]

    Mindcube: Spatial mental modeling from limited views, 2026a.https://arxiv.org/abs/2506.21458

    Qineng Wang, Baiqiao Yin, Pingyue Zhang, Jianshu Zhang, Kangrui Wang, Zihan Wang, Jieyu Zhang, Keshigeyan Chandrasegaran, Han Liu, Ranjay Krishna, Saining Xie, Jiajun Wu, Li Fei-Fei, and Manling Li. Mindcube: Spatial mental modeling from limited views, 2026a.https://arxiv.org/abs/2506.21458. Weiyun Wang, Zhangwei Gao, Lixin Gu, Hengjun Pu, Long Cui, Xingg...

    arXiv 2025

  21. [28]

    Diankun Wu, Fangfu Liu, Yi-Hsin Hung, and Yueqi Duan

    https://arxiv.org/abs/2410.13848. Diankun Wu, Fangfu Liu, Yi-Hsin Hung, and Yueqi Duan. Spatial-mllm: Boosting mllm capabilities in visual-based spatial intelligence. arXiv preprint arXiv:2505.23747 , 2025b. Haoning Wu, Xiao Huang, Yaohui Chen, Ya Zhang, Yanfeng Wang, and Weidi Xie. Spatialscore: Towards comprehensive evaluation for spatial intelligence.a...

    Pith/arXiv arXiv

  22. [29]

    Qiucheng Wu, Handong Zhao, Michael Saxon, Trung Bui, William Yang Wang, Yang Zhang, and Shiyu Chang

    https://arxiv.org/abs/2601.19834. Qiucheng Wu, Handong Zhao, Michael Saxon, Trung Bui, William Yang Wang, Yang Zhang, and Shiyu Chang. Vsp: Diagnosing the dual challenges of perception and reasoning in spatial planning tasks for mllms. InProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) , pages 2270–2280, October 2025d. Shitao...

    arXiv 2024

  23. [30]

    Spatialbench: Benchmarking multimodal large language models for spatial cognition, 2026.https://arxiv.org/abs/2511.21471

    Peiran Xu, Sudong Wang, Yao Zhu, Jianing Li, Gege Qi, and Yunjian Zhang. Spatialbench: Benchmarking multimodal large language models for spatial cognition, 2026.https://arxiv.org/abs/2511.21471. Jihan Yang, Shusheng Yang, Anjali W Gupta, Rilyn Han, Li Fei-Fei, and Saining Xie. Thinking in space: How multimodal large language models see, remember, and reca...

    Pith/arXiv arXiv 2026

  24. [31]

    Sphere: Unveiling spatial blind spots in vision-language models through hierarchical evaluation.ACL, 2025a

    Wenyu Zhang, Wei En Ng, Lixin Ma, Yuwen Wang, Junqi Zhao, Allison Koenecke, Boyang Li, and Lu Wang. Sphere: Unveiling spatial blind spots in vision-language models through hierarchical evaluation.ACL, 2025a. Yuyou Zhang, Radu Corcodel, Chiori Hori, Anoop Cherian, and Ding Zhao. Spinbench: Perspective and rotation as a lens on spatial reasoning in vlms.arX...

    arXiv

  25. [32]

    Xu Zheng, Zihao Dongfang, Lutao Jiang, Boyuan Zheng, Yulong Guo, Zhenquan Zhang, Giuliano Albanese, Runyi Yang, Mengjiao Ma, Zixin Zhang, et al

    https://arxiv.org/abs/2605.28548. Xu Zheng, Zihao Dongfang, Lutao Jiang, Boyuan Zheng, Yulong Guo, Zhenquan Zhang, Giuliano Albanese, Runyi Yang, Mengjiao Ma, Zixin Zhang, et al. Multimodal spatial reasoning in the large model era: A survey and benchmarks. arXiv preprint arXiv:2510.25760 ,

    Pith/arXiv arXiv

  26. [33]

    Roborefer: Towards spatial referring with reasoning in vision-language models for robotics

    Enshen Zhou, Jingkun An, Cheng Chi, Yi Han, Shanyu Rong, Chi Zhang, Pengwei Wang, Zhongyuan Wang, Tiejun Huang, Lu Sheng, et al. Roborefer: Towards spatial referring with reasoning in vision-language models for robotics. arXiv preprint arXiv:2506.04308 ,

    arXiv

  27. [34]

    • Appendix B documents data sources, schema, and quality control

    18 Appendix Overview The appendix provides supporting context, implementation details, and complete results: • Appendix A reviews spatial and generative-model evaluation. • Appendix B documents data sources, schema, and quality control. • Appendix C specifies parser rules, Agentic construction, and validation analyses. • Appendix D presents representative...

    2023

  28. [35]

    GPT-5.4 OpenAI-compatible chat API; request model gpt-5.4; direct structured answer; deterministic request settings where supported

    Local BF16 inference; deterministic decoding. GPT-5.4 OpenAI-compatible chat API; request model gpt-5.4; direct structured answer; deterministic request settings where supported. Cosmos3-Nano; RynnBrain-8B; RoboBrain2.5-8B-NV (NVIDIA, 2026; Dang et al., 2026; Tan et al.,

    2026

  29. [36]

    VLM-3R-7B; SpatialRGPT-8B; Spatial-MLLM; SpatialBot-3B (Fan et al., 2025; Cheng et al., 2024; Wu et al., 2025b; Cai et al.,

    SpaceR; Cambrian-S-7B; GEM-2B; SpatialThinker-3B (Ouyang et al., 2025; Yang et al., 2025b; Zhao et al., 2026; Batra et al., 2025); SpaceOm Local spatial-specialized VLM inference; released checkpoint and processor; deterministic decoding. VLM-3R-7B; SpatialRGPT-8B; Spatial-MLLM; SpatialBot-3B (Fan et al., 2025; Cheng et al., 2024; Wu et al., 2025b; Cai et al.,

    2025

  30. [37]

    SenseNova-Vision-7B-MoT; Janus-Pro-7B; Janus-1.3B (Han et al., 2026; Chen et al., 2025; Wu et al.,

    Local spatial-specialized VLM inference; released checkpoint and task-compatible prompt format. SenseNova-Vision-7B-MoT; Janus-Pro-7B; Janus-1.3B (Han et al., 2026; Chen et al., 2025; Wu et al.,

    2026

  31. [1024]

    32 Table 17 continued

    FLUX.2 [klein] 9B Local image editing with the 9B checkpoint; one protocol-constrained image per sample. 32 Table 17 continued. Model Runtime setting SenseNova-Vision-7B-MoT; BAGEL-7B-MoT (Han et al., 2026; Deng et al.,

    2026

  32. [2022]

    Vision as unified multimodal generation, 2026.https://arxiv.org/abs/2607.06560

    Xiaoyang Han, Jianhua Li, Kewang Deng, Zukai Chen, Xuanke Shi, Sihan Wang, Boxuan Li, Linyan Wang, Siyi Xie, Xin You, et al. Vision as unified multimodal generation, 2026.https://arxiv.org/abs/2607.06560. Xiaoshuai Hao, Yingbo Tang, Lingfeng Zhang, Yanbiao Ma, Yunfeng Diao, Ziyu Jia, Wenbo Ding, Hangjun Ye, and Long Chen. Roboafford++: A generative ai-enh...

    Pith/arXiv arXiv 2026

  33. [2023]

    Benchmarking spatial relationships in text-to-image generation.arXiv preprint arXiv:2212.10015 ,

    Tejas Gokhale, Hamid Palangi, Besmira Nushi, Vibhav Vineet, Eric Horvitz, Ece Kamar, Chitta Baral, and Yezhou Yang. Benchmarking spatial relationships in text-to-image generation.arXiv preprint arXiv:2212.10015 ,

    Pith/arXiv arXiv

  34. [2024]

    Janus-pro: Unified multimodal understanding and generation with data and model scaling.arXiv preprint arXiv:2501.17811 ,

    Xiaokang Chen, Zhiyu Wu, Xingchao Liu, Zizheng Pan, Wen Liu, Zhenda Xie, Xingkai Yu, and Chong Ruan. Janus-pro: Unified multimodal understanding and generation with data and model scaling.arXiv preprint arXiv:2501.17811 ,

    Pith/arXiv arXiv

  35. [2025]

    https://arxiv.org/abs/2509.23661. Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, Wenbin Ge, Zhifang Guo, Qidong Huang, Jie Huang, Fei Huang, Binyuan Hui, Shutong Jiang, Zhaohai Li, Mingsheng Li, Mei Li, Kaixin Li, Zicheng Lin, Junyang Lin, Xuejing Liu, Jiawei Liu, Chenglong Liu...

    Pith/arXiv arXiv 2023

  36. [2026]

    14 Chaorui Deng, Deyao Zhu, Kunchang Li, Chenhui Gou, Feng Li, Zeyu Wang, Shu Zhong, Weihao Yu, Xiaonan Nie, Ziang Song, et al

    https://arxiv.org/abs/2602.14979. 14 Chaorui Deng, Deyao Zhu, Kunchang Li, Chenhui Gou, Feng Li, Zeyu Wang, Shu Zhong, Weihao Yu, Xiaonan Nie, Ziang Song, et al. Emerging properties in unified multimodal pretraining.arXiv preprint arXiv:2505.14683 ,

    arXiv