Pith. sign in

REVIEW 3 major objections 4 minor 31 references

GST-Bench: Can VLMs Develop Global Spatial Awareness from Video?

T0 review · 3 major / 4 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read GST-Bench shows current vision-language models cannot assemble a globally consistent spatial map from long egocentric video, with the strongest zero-shot model scoring 42.68 against a human baseline of 79.08.

desk verdict A genuinely useful new benchmark for global spatial awareness in video, but the aggregate score overstates the integration claim because the scene-matching subtasks are partly single-frame solvable, and the human baseline is too thin to carry the headline gap. read the letter →

arxiv 2608.05747 v1 pith:SJMAZRVG submitted 2026-08-06 cs.CV

Qifeng Zhang , Kaixiang Huang , Heng Dong , Huang Fang , Junting Chen , Junjie Zhu , Yonghang Chen , Zhiyu Zhang
show 1 more author
Wei Li
This is my paper · ORCID
classification cs.CV
keywords globalspatialawarenessvideounderstandingvision-languagemodelsembodiedAIreasoningbenchmarktop-downmappingcross-frameintegrationegocentric
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper introduces GST-Bench, a video question-answering benchmark that tests whether vision-language models can build a global spatial understanding of a scene from a long egocentric exploration video. Across 22 models, the best zero-shot score is 42.68 versus 79.08 for human evaluators, a gap the authors trace to cross-frame integration rather than single-image perception. A controlled local variant shows that proprietary models improve sharply when the target object is visible in the current view, while open-source models improve little or even degrade. The paper also releases GST-Train, and fine-tuning a small open model on it raises its GST-Bench score from 25.89 to 53.52, above all zero-shot proprietary models. The finding matters because embodied agents need exactly this ability to navigate, remember, and retrieve objects after losing sight of them.

What carries the argument

The benchmark itself is the machinery: synthetically generated egocentric exploration videos from simulated indoor scenes, with query viewpoints sampled off the video trajectory and, for object-localization tasks, a hard guarantee that the target object is invisible from the query view. Three abstraction levels of top-down images (photo-realistic bird's-eye view, occupancy-style map, and bare floor plan) let the benchmark test whether egocentric observations can be aligned to an explicit global representation. The controlled local variants (Global, Local-Video, and Local-Image) isolate whether a model's failure is perceptual or integrative by progressively removing the need for cross-frame reasoning.

What would settle it

A concrete test would be to take GST-Bench's object-localization questions and run a model that receives only one randomly sampled frame from the exploration video plus the current view: if it still scores far above the random baseline on those questions, then the target-invisible constraint is not preventing single-frame solutions. Similarly, re-running Top-Down Selection with the exploration video replaced by a single frame would show whether that subtask measures cross-frame integration at all; if single-frame accuracy matches video accuracy, the subtask should be excluded from the global-reasoning claim.

Watch

Extended reading notes

Core claim

The central discovery is a measurable, systematic failure: current VLMs do not maintain a globally consistent spatial representation from long-horizon egocentric video. GST-Bench asks models to localize a novel viewpoint on a top-down map, infer where a target object is when it is absent from the current view, and match egocentric motion to a global trajectory. The strongest zero-shot model reaches 42.68, less than 55% of the human score of 79.08, and most open-source models sit near random guessing. Ablations that make the target visible in the query view lift proprietary models by up to 39 points, showing that their bottleneck is cross-frame integration, whereas open-source models stay weak even in the local setting. Targeted fine-tuning on GST-Train lifts Qwen3-VL-8B from 25.89 to 53.52, surpassing all zero-shot proprietary models.

Load-bearing premise

The benchmark's central claim rests on the assumption that every GST-Bench question truly requires integrating information across video frames and cannot be answered from a single frame or by retrieving a matching frame; for Top-Down Selection and Trajectory Selection this is not fully enforced, and the human baseline is estimated from only 20 samples per task.

Editorial extensions

If this is right

  • If the gap is real, current VLMs cannot reliably support embodied tasks that require remembering object locations after they leave view, such as household retrieval.
  • The disentanglement result implies that improving cross-frame spatial integration, not just single-image perception, is the priority for proprietary-scale models.
  • Open-source models need gains at both local perception and integration; local benchmarks alone will not predict global performance.
  • GST-Train demonstrates that targeted supervision can narrow the gap, so future training data design should include explicit global map supervision.
  • The low human score on metric distance suggests absolute distance estimation is intrinsically hard and should be scored with tolerance-aware metrics.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct extension the paper does not pursue is whether the same gap appears in real-world egocentric video, since GST-Bench is entirely synthetic; a real-video probe would test whether simulation captures the bottleneck.
  • Top-Down Selection and Trajectory Selection subtasks may be solvable by matching a single frame's room layout to a candidate map, so the claim that every GST-Bench task strictly requires cross-frame integration likely overstates the benchmark's uniformity; re-analyzing per-task single-frame solvability would sharpen the claim.
  • If the integration bottleneck is architectural, explicit spatial memory or mapping modules may close the gap faster than simply scaling context length.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. GST-Bench introduces a synthetic video-based VQA benchmark for global spatial awareness, spanning 12 subtasks across self-localization, object localization, and scene structure understanding. The data are generated in OmniGibson/BEHAVIOR-1K with off-trajectory current views, target-invisible constraints for object-localization tasks, and top-down map variants, yielding 2,762 human-verified questions. The authors evaluate 22 VLMs and report that the strongest zero-shot model, Gemini-3-Pro, scores 42.68 versus a human score of 79.08. They also construct local variants that remove the cross-frame requirement and attribute the proprietary-model failure mainly to cross-frame integration, and they show that fine-tuning Qwen3-VL-8B on GST-Train improves its score from 25.89 to 53.52.

Significance. If the central claim holds, the benchmark fills a real gap: it targets cross-view spatial integration rather than single-image spatial relations, uses precise numerical metrics, and leverages simulator geometry for exact labels, avoiding the circularity risk of model-generated ground truth. The human verification stage, the broad model coverage, and the local/global diagnostic design are further strengths. The central result, however, depends on two load-bearing assumptions: that every subtask genuinely requires cross-frame integration, and that the human baseline is a reliable estimate. Both assumptions need additional evidence before the headline gap can be interpreted as measuring global spatial awareness rather than a mixture of task formats.

major comments (3)
  1. [Section 3.2 / Eq. (5)] The claim in Section 1 that 'every task requires integrating information across views' is only enforced for object-localization tasks. Section 3.2 shows that Top-Down Selection (easy/medium/hard) takes an exploration video plus four candidate top-down images; a single distinctive frame can reveal the room layout or object arrangement needed to choose the correct map. Section 4.2 itself states that the open-source advantage is 'mainly driven by the Top-Down Selection (easy/medium) subtasks' and describes these as 'coarse scene identification,' which effectively concedes that these subtasks are not diagnostics of cross-frame integration. Because the overall Score in Eq. (5) is the arithmetic mean over all 12 subtasks, the headline gap (42.68 vs. 79.08) mixes single-frame-solvable matching tasks with genuinely integrative tasks. Please recompute the aggregate score and the human/model comparison on the set of tasks that strictly require cross-frame integration, and empirically test single-frame-solvability of Top-Down Selection and Trajectory Selection by feeding only one frame (or a short clip) to models.
  2. [Table 1] The human baseline of 79.08 is reported as 'Human Level (20 samples/task)' with no protocol details, no participant information, no selection criteria, and no error bars. Per-task human scores such as EDv=75.00, EDist=41.50, and Ori=85.00 are proportions over only 20 samples, so the reported 36.4-point gap relative to Gemini-3-Pro may be statistically fragile; for example, on EDistv the human score (41.50) is essentially matched by Gemini-2.5-Pro (42.00). Please provide the full human evaluation protocol, sample sizes per task, and confidence intervals, and verify that the headline gap remains statistically significant after restricting the comparison to tasks that genuinely require cross-frame integration.
  3. [Section 4.3 / Table 2] The local-vs-global comparison is not fully controlled. In Local-Video and Local-Image, the current view is replaced by a view in which the target is visible; this changes both target visibility and the particular viewpoint/camera pose of the query. Improvements from Global to Local-Video can therefore reflect a more favorable query view rather than the mere removal of the cross-frame requirement. In addition, Table 2 covers only the two semantic tasks EDs and EDists, so the broader conclusion that proprietary models 'mainly fail at cross-frame spatial integration' is not directly tested for the visual modality, global position, orientation, or top-down tasks. Please hold the query viewpoint fixed while adding/removing the target through controlled rendering, and extend the local variants to additional subtasks to support the stated diagnosis.
minor comments (4)
  1. [Section 4.4] Because GST-Train is generated with the same simulation pipeline and asset family as GST-Bench, the 27.63-point fine-tuning gain partly reflects distribution matching; the authors disclose that the scenes are disjoint, but they should explicitly state this limitation and report performance on held-out scenes from different simulators or with different rendering conditions.
  2. [Equations (1)-(4)] The threshold sets C_d, C_a, and C_p are fixed without a sensitivity analysis; since the overall Score is the mean over these thresholds, a short robustness check would strengthen the claim that the reported ranking is not an artifact of threshold choice.
  3. [Section 3.3] Human verification is described as checking answerability and target identifiability, but not as independently verifying the automatically generated geometric labels; the paper should state this explicitly, since label correctness is otherwise guaranteed only by simulator geometry.
  4. [Table 1] The random-guessing baselines for the distance, angle, and point metrics are not derived in the text; please describe how the per-subtask random baselines are computed, including the assumed answer distributions.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: benchmark answers derive from simulator geometry, not from model outputs or fitted parameters.

full rationale

GST-Bench's claimed predictions (model scores and the human gap) are not derived from the benchmark's inputs by construction. Labels for all twelve subtasks come from simulator geometry: camera-object poses, top-down projections, and scene or trajectory identities, with automatic filtering and human verification. The central comparison (42.68 vs 79.08) is an external evaluation of 22 VLMs against ground truth generated independently of those models; no parameter of the benchmark is fitted to model outputs, and no 'prediction' is a renamed fit. The only pipeline-sharing element is GST-Train, which is generated by the same simulation pipeline as GST-Bench; this is disclosed in Section 4.4 and affects the fine-tuned model's distribution match, but it does not alter the benchmark labels or the zero-shot evaluations, so it is at most a training-data caveat rather than circular reasoning. Self-citations (e.g., Robix [5]) appear only as evaluated baselines and are not load-bearing for the benchmark's validity. The concern that Top-Down Selection tasks may be solvable from a single frame would weaken construct validity of the aggregate score, but it is not a circularity: the task labels still come from scene identity and are not defined in terms of the models' answers. Human baseline imprecision (20 samples per task) is a measurement limitation, not circularity. Overall, the derivation chain from task construction to evaluation is self-contained and externally grounded.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim is an empirical benchmark finding, so it rests mainly on the benchmark's construct validity and the human baseline, not on fitted model parameters. The metric thresholds are hand-chosen evaluation constants, not fit parameters.

free parameters (3)
  • Distance confidence thresholds C_d = {0.50, 0.55, ..., 0.95}
    Chosen by hand following the MRA protocol from VSI-Bench; affects distance scores but not the qualitative model/human ordering.
  • Angle tolerance thresholds C_a = {15, 30, 45} degrees
    Hand-chosen tolerances for angle accuracy; different tolerances would shift scores.
  • Point distance thresholds C_p = {100, 150, 200, 250, 300} pixels
    Hand-chosen pixel thresholds for point prediction accuracy on top-down images.
assumptions (4)
  • domain assumption Simulation scenes from BEHAVIOR-1K, HyperSim, and ArtVIP are representative enough of real indoor environments for conclusions to transfer.
    Section 3.3: the benchmark is entirely synthetic; no real-world validation is provided.
  • domain assumption Human verification guarantees answerability and lack of shortcuts.
    Section 3.3: annotators confirm target identification and overlap; there is no reported inter-annotator agreement or protocol detail, yet this verification is used to certify every sample.
  • standard math The ground-truth geometry recorded from the simulator is error-free and the QA templates map perfectly to it.
    Section 3.3: answers are computed directly from simulator geometry; no error analysis is given.
  • domain assumption The evaluated models are representative of state-of-the-art VLMs and their API responses are deterministic under greedy decoding.
    Section 4.1: 22 models evaluated; API models may be stochastic and no repeated-run variance is reported.

how reviews work

0 comments
Cite this review

Pith. "Pith review of GST-Bench: Can VLMs Develop Global Spatial Awareness from Video?." pith.science (2026). https://pith.science/paper/SJMAZRVG

@misc{pith2026260805747,
  author       = {Pith},
  title        = {Pith review of: GST-Bench: Can VLMs Develop Global Spatial Awareness from Video?},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SJMAZRVG}},
  note         = {Machine review of arXiv:2608.05747}
}
read the original abstract

Spatial intelligence is fundamental to embodied agents, yet existing benchmarks focus on local spatial perception from single or few viewpoints, overlooking global spatial awareness over continuous, long-horizon visual streams. To address this limitation, we introduce the Global-Spatial-Temporal Benchmark (GST-Bench), a VQA benchmark for global spatial intelligence in video understanding, comprising human-verified questions derived from 6,790 minutes of synthetically generated video. It requires models to perform accurate spatial inference from novel viewpoints unseen in the input video and to map egocentric observations onto global top-down images. A comprehensive evaluation of 22 state-of-the-art VLMs exposes a striking gap between models and humans: the strongest zero-shot model attains only 42.68, far below the human score of 79.08. To probe the cause of this gap, we construct GST-Bench-Local and find that models, despite strong local spatial understanding under the same task formulation, still fail to consolidate long-horizon observations into a globally consistent scene representation. We further provide GST-Train, a dataset for global spatial reasoning, as a complementary resource to facilitate future research on this challenge.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

31 extracted references · 11 canonical work pages

  1. [1]

    Llava-onevision-1.5: Fully open framework for democratized multimodal training

    Xiang An, Yin Xie, Kaicheng Yang, Wenkang Zhang, Xiuwei Zhao, Zheng Cheng, Yirui Wang, Songcen Xu, Changrui Chen, Didi Zhu, et al. Llava-onevision-1.5: Fully open framework for democratized multimodal training. arXiv preprint arXiv:2509.23661, 2025

    arXiv 2025

  2. [2]

    Qwen3-vl technical report.arXiv preprint arXiv:2511.21631, 2025

    Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, et al. Qwen3-vl technical report.arXiv preprint arXiv:2511.21631, 2025

    arXiv 2025

  3. [3]

    Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities.arXiv preprint arXiv:2507.06261, 2025

    Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel 12 Blistein, Ori Ram, Dan Zhang, Evan Rosen, et al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities.arXiv preprint arXiv:2507.06261, 2025

    arXiv 2025

  4. [4]

    Mm-spatial: Exploring 3d spatial understanding in multimodal llms

    Erik Daxberger, Nina Wenzel, David Griffiths, Haiming Gang, Justin Lazarow, Gefen Kohavi, Kai Kang, Marcin Eichner, Yinfei Yang, Afshin Dehghan, et al. Mm-spatial: Exploring 3d spatial understanding in multimodal llms. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 7395–7408, 2025

    work page 2025

  5. [5]

    Robix: A unified model for robot interaction, reasoning and planning.arXiv preprint arXiv:2509.01106, 2025

    Huang Fang, Mengxi Zhang, Heng Dong, Wei Li, Zixuan Wang, Qifeng Zhang, Xueyun Tian, Yucheng Hu, and Hang Li. Robix: A unified model for robot interaction, reasoning and planning.arXiv preprint arXiv:2509.01106, 2025

    arXiv 2025

  6. [6]

    Blink: Multimodal large language models can see but not perceive

    Xingyu Fu, Yushi Hu, Bangzheng Li, Yu Feng, Haoyu Wang, Xudong Lin, Dan Roth, Noah A Smith, Wei-Chiu Ma, and Ranjay Krishna. Blink: Multimodal large language models can see but not perceive. InEuropean Conference on Computer Vision, pages 148–166. Springer, 2024

    2024

  7. [7]

    Gemini 3: Introducing the latest gemini ai model from google.https://blog.google/products/gemini/ gemini-3/, 2025

    Google. Gemini 3: Introducing the latest gemini ai model from google.https://blog.google/products/gemini/ gemini-3/, 2025. Accessed: 2026-06-12

    work page 2025

  8. [8]

    Gpt-4o system card.arXiv preprint arXiv:2410.21276, 2024

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card.arXiv preprint arXiv:2410.21276, 2024

    arXiv 2024

Show all 31 references
  1. [9]

    Artvip: Articulated digital assets of visual realism, modular interaction, and physical fidelity for robot learning.arXiv preprint arXiv:2506.04941, 2025

    Zhao Jin, Zhengping Che, Tao Li, Zhen Zhao, Kun Wu, Yuheng Zhang, Yinuo Zhao, Zehui Liu, Qiang Zhang, Xiaozhu Ju, et al. Artvip: Articulated digital assets of visual realism, modular interaction, and physical fidelity for robot learning.arXiv preprint arXiv:2506.04941, 2025

    2025

  2. [10]

    What’s “up” with vision-language models? investigating their struggle with spatial reasoning

    Amita Kamath, Jack Hessel, and Kai-Wei Chang. What’s “up” with vision-language models? investigating their struggle with spatial reasoning. InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 9161–9175, 2023

    2023

  3. [11]

    Behavior-1k: A benchmark for embodied ai with 1,000 everyday activities and realistic simulation

    Chengshu Li, Ruohan Zhang, Josiah Wong, Cem Gokmen, Sanjana Srivastava, Roberto Martín-Martín, Chen Wang, Gabrael Levine, Michael Lingelbach, Jiankai Sun, et al. Behavior-1k: A benchmark for embodied ai with 1,000 everyday activities and realistic simulation. InConference on R...

    2023

  4. [12]

    Viewspatial-bench: Evaluating multi-perspective spatial localization in vision-language models

    Dingming Li, Hongxing Li, Zixuan Wang, Yuchen Yan, Hang Zhang, Siqi Chen, Guiyang Hou, Shengpei Jiang, Wenqi Zhang, Yongliang Shen, et al. Viewspatial-bench: Evaluating multi-perspective spatial localization in vision-language models. arXiv preprint arXiv:2505.21500, 2025

    2025

  5. [13]

    Sti-bench: Are mllms ready for precise spatial-temporal world understanding? InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 5622–5632, 2025

    Yun Li, Yiming Zhang, Tao Lin, XiangRui Liu, Wenxiao Cai, Zheng Liu, and Bo Zhao. Sti-bench: Are mllms ready for precise spatial-temporal world understanding? InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 5622–5632, 2025

    2025

  6. [14]

    Mmsi-video-bench: A holistic benchmark for video-based spatial intelligence.arXiv preprint arXiv:2512.10863, 2025

    Jingli Lin, Runsen Xu, Shaohao Zhu, Sihan Yang, Peizhou Cao, Yunlong Ran, Miao Hu, Chenming Zhu, Yiman Xie, Yilin Long, et al. Mmsi-video-bench: A holistic benchmark for video-based spatial intelligence.arXiv preprint arXiv:2512.10863, 2025

    2025

  7. [15]

    Ost-bench: Evaluating the capabilities of mllms in online spatio-temporal scene understanding

    Jingli Lin, Chenming Zhu, Runsen Xu, Xiaohan Mao, Xihui Liu, Tai WANG, and Jiangmiao Pang. Ost-bench: Evaluating the capabilities of mllms in online spatio-temporal scene understanding. In D. Belgrave, C. Zhang, H. Lin, R. Pascanu, P. Koniusz, M. Ghassemi, and N. Chen, editors...

    2025

  8. [16]

    Visual spatial reasoning.Transactions of the Association for Computational Linguistics, 11:635–651, 2023

    Fangyu Liu, Guy Emerson, and Nigel Collier. Visual spatial reasoning.Transactions of the Association for Computational Linguistics, 11:635–651, 2023

    2023

  9. [17]

    Nvila: Efficient frontier visual language models

    Zhijian Liu, Ligeng Zhu, Baifeng Shi, Zhuoyang Zhang, Yuming Lou, Shang Yang, Haocheng Xi, Shiyi Cao, Yuxian Gu, Dacheng Li, et al. Nvila: Efficient frontier visual language models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4122...

    2025

  10. [18]

    3dsrbench: A comprehensive 3d spatial reasoning benchmark

    Wufei Ma, Haoyu Chen, Guofeng Zhang, Yu-Cheng Chou, Jieneng Chen, Celso de Melo, and Alan Yuille. 3dsrbench: A comprehensive 3d spatial reasoning benchmark. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 6924–6934, 2025

    2025

  11. [19]

    Openeqa: Embodied question answering in the era of foundation models

    Arjun Majumdar, Anurag Ajay, Xiaohan Zhang, Pranav Putta, Sriram Yenamandra, Mikael Henaff, Sneha Silwal, Paul Mcvay, Oleksandr Maksymets, Sergio Arnaud, et al. Openeqa: Embodied question answering in the era of foundation models. InProceedings of the IEEE/CVF conference on co...

    2024

  12. [20]

    Cosmos-reason2

    NVIDIA. Cosmos-reason2. https://docs.nvidia.com/cosmos/latest/reason2/index.html, 2026. NVIDIA Cosmos documentation

    2026

  13. [21]

    Hypersim: A photorealistic synthetic dataset for holistic indoor scene understanding

    Mike Roberts, Jason Ramapuram, Anurag Ranjan, Atulit Kumar, Miguel Angel Bautista, Nathan Paczan, Russ Webb, and Joshua M Susskind. Hypersim: A photorealistic synthetic dataset for holistic indoor scene understanding. InProceedings of the IEEE/CVF internationalconference on co...

    2021

  14. [22]

    Bytedance Seed. Seed1. 8 model card: Towards generalized real-world agency.arXiv preprint arXiv:2603.20633, 2026

    2026 arXiv

  15. [23]

    Openai gpt-5 system card.arXiv preprint arXiv:2601.03267, 2025

    Aaditya Singh, Adam Fry, Adam Perelman, Adam Tart, Adi Ganesh, Ahmed El-Kishky, Aidan McLaughlin, Aiden Low, AJ Ostrow, Akhila Ananthram, et al. Openai gpt-5 system card.arXiv preprint arXiv:2601.03267, 2025

    2025 arXiv

  16. [24]

    Robobrain 2.5: Depth in sight, time in mind.arXiv preprint arXiv:2601.14352, 2026

    Huajie Tan, Enshen Zhou, Zhiyu Li, Yijie Xu, Yuheng Ji, Xiansheng Chen, Cheng Chi, Pengwei Wang, Huizhu Jia, Yulong Ao, et al. Robobrain 2.5: Depth in sight, time in mind.arXiv preprint arXiv:2601.14352, 2026

    2026

  17. [25]

    Cambrian-1: A fully open, vision-centric exploration of multimodal llms

    Shengbang Tong, Ellis Brown, Penghao Wu, Sanghyun Woo, Manoj Middepogu, Sai C Akula, Jihan Yang, Shusheng Yang, Adithya Iyer, Xichen Pan, et al. Cambrian-1: A fully open, vision-centric exploration of multimodal llms. Advances in Neural Information Processing Systems, 37:87310...

    2024

  18. [26]

    Internvl3

    Weiyun Wang, Zhangwei Gao, Lixin Gu, Hengjun Pu, Long Cui, Xingguang Wei, Zhaoyang Liu, Linglin Jing, Shenglong Ye, Jie Shao, et al. Internvl3. 5: Advancing open-source multimodal models in versatility, reasoning, and efficiency.arXiv preprint arXiv:2508.18265, 2025

    2025 arXiv

  19. [27]

    Spatial457: A diagnostic benchmark for 6d spatial reasoning of large mutimodal models

    Xingrui Wang, Wufei Ma, Tiezheng Zhang, Celso M de Melo, Jieneng Chen, and Alan Yuille. Spatial457: A diagnostic benchmark for 6d spatial reasoning of large mutimodal models. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 24669–24679, 2025

    2025

  20. [28]

    Thinking in space: How multimodal large language models see, remember, and recall spaces

    Jihan Yang, Shusheng Yang, Anjali W Gupta, Rilyn Han, Li Fei-Fei, and Saining Xie. Thinking in space: How multimodal large language models see, remember, and recall spaces. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 10632–10643, 2025

    2025

  21. [29]

    Cambrian-s: Towards spatial supersensing in video

    Shusheng Yang, Jihan Yang, Pinzhi Huang, Ellis L Brown II, Zihao Yang, Yue Yu, Shengbang Tong, Zihan Zheng, Yifan Xu, Muhan Wang, et al. Cambrian-s: Towards spatial supersensing in video. InThe Fourteenth International Conference on Learning Representations, 2026

    2026

  22. [30]

    Mmsi-bench: A benchmark for multi-image spatial intelligence

    Sihan Yang, Runsen Xu, Yiman Xie, Sizhe Yang, Mo Li, Jingli Lin, Chenming Zhu, Xiaochen Chen, Haodong Duan, Xiangyu Yue, et al. Mmsi-bench: A benchmark for multi-image spatial intelligence. arXiv preprint arXiv:2505.23764, 2025

    2025 arXiv

  23. [31]

    From flatland to space: Teach- ing vision-language models to perceive and reason in 3d

    Jiahui Zhang, Yurui Chen, Yueming Xu, Ze Huang, Jilin Mei, Chunhui Chen, Yanpeng Zhou, Yu-Jie Yuan, Xinyue Cai, Guowei Huang, Xingyue Quan, Hang Xu, and Li Zhang. From flatland to space: Teach- ing vision-language models to perceive and reason in 3d. In D. Belgrave, C. Zhang, ...

    2025

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.