Computer Vision and Pattern Recognition · arXiv:2610.29232 v1 · submitted 21 Oct 2026 · 64 references
Think-Capture: Active 3D Geometry Acquisition for Spatial Reasoning in Vision-Language Models
The paper says
“Vision-Language Models (VLMs) excel at 2D recognition but often fail to maintain consistent 3D relations across sparse viewpoints, leading to spatial hallucinations under occlusion and perspective changes.”
more of the abstract
Existing methods commonly fuse all frames with dense geometry, incurring substantial computation and context overhead while encouraging blind reliance on imperfect estimators. We present Think-Capture, a query-conditioned framework that first identifies task-critical objects and then acquires their geometry on demand. A 3D toolkit lifts zero-shot masks into a common coordinate frame, and a Symbolic 3D State Abstraction compresses object and camera states into structured text. Hierarchical Group Relative Policy Optimization (GRPO) trains the VLM on cases in which geometry either helps or misleads, teaching it when to use tool evidence and when to fall back on visual cues. Think-Capture-4B reaches 54.4% on MindCube-Tiny (+21.5 points over its backbone), 39.1% on the MMSI-Bench positional subset (+7.9), and 42.8% on ViewSpatial (+5.2). It also remains robust under coordinate and mask perturbations. The full stack fits one 24 GB GPU and serves as an asynchronous high-level reasoner on an Agibot G1. Our code is available at https://github.com/819346675/Think-Capture.
from the abstract
Pith says
Machine review · deepseek-v4-flash · 22 Oct 2026 · what was read was not recorded · not checked by a person
It can be wrong. Challenge this review or reply to it.
Holds up. Solid paper on active 3D acquisition for spatial reasoning—the evaluation is thorough, the limitations are honestly stated, and the central claim holds up. This line is wrong
5 minor
How often objections from reviews like this one are right: not yet measured
Not checked: figures, code, what the 64 references say.
5 minor objections
The paper says, at Section 3.4
Pith says: minor objection 1 of 5
The term 'hierarchical GRPO' is used to describe the sample-conditioned reward distinction between positive and negative subsets. While the concept is clear, the term 'hierarchical' could be misleading, as the reward structure does not involve multiple levels of abstraction. Consider either renaming it (e.g., 'sample-conditioned GRPO') or adding a brief justification for the term.
The paper says, at Section 3.2, Eq. (2)
Pith says: minor objection 2 of 5
The set comprehension notation for P^{(k)}_{world} is correct but could be clarified. It may help to explicitly denote the masked local points as P^{(k)}_{local}[M^{(k)}>0] before applying the rigid transformation, to avoid any ambiguity about whether the mask is applied after or before the transformation.
show 3 more
The paper says, at Section 4.10
Pith says: minor objection 3 of 5
The paper mentions an empty-target fallback in fewer than 1.6% of MindCube-Tiny queries. Reporting what fraction of queries across all benchmarks trigger the fallback (or providing a qualitative sense of when it occurs) would make the limitation discussion more concrete.
The paper says, at Figure 3
Pith says: minor objection 4 of 5
The reasoning trace in Figure 3 mentions 'image 4' for the seating bench, but Figure 3 only shows two images (Image 1 and Image 2). Adding a brief note in the caption about which image corresponds to which view in the example would improve readability.
The paper says, at Abstract and Table 1
Pith says: minor objection 5 of 5
Minor typos: In the abstract, 'presentThink-Capture' should be 'present Think-Capture'. In Table 1, 'OverallCam-Cam' likely needs a space: 'Overall Cam-Cam', and similar for other concatenated column headers.
What would settle it
Run the full Think-Capture-4B pipeline on 50 random MindCube-Tiny samples with a different zero-shot segmentation model (e.g., Grounding DINO + SAM2 instead of SAM3). If the accuracy on these 50 samples drops by more than 5 points compared to the reported 54.4%, then the method's success is partially tied to SAM3's specific segmentation quality. If accuracy stays within 3 points, the framework generalizes across segmentation backends.
from the skeptic's reading
How the review went
4 readings by deepseek-v4-flash · 22 Oct 2026
- Reader holds up
- Skeptic agrees
- Editor wrote the verdict above
- Referee minor revision: 5 minor
Not run on this paper: the simulated authors' defence.
Pith did not check this review's objections against the paper.
The review in full
The review in brief, machine-written
Thanks for sending this over. I gave Think-Capture a careful read and it holds up well. The core idea—query-conditioned geometry acquisition instead of dense fusion—is sensible, and the execution is clean. The main results are as advertised: +21.5 on MindCube-Tiny, +7.9 on MMSI-Bench positional, +5.2 on ViewSpatial. Gains are consistent across 2B, 4B, and 8B backbones, and the zero-shot transfer to unseen benchmarks suggests the method isn't just fitting the training distribution.
What I like most: the evaluation is unusually thorough. Three benchmarks, ablation of each component (tools, GRPO, symbolic format), a matched 2D-box control that isolates the value of 3D state, perturbation tests on tool outputs, and a real robot demo. The authors honestly state limitations in Section 4.10—dependence on perception tools, symbolic information loss, latency—and these match what I saw. Code is available.
The novel bits that matter: (1) query-conditioned target selection, which differs from prior tool-augmented methods like Think3D that don't condition on the query; (2) the symbolic 3D state that compresses object positions and camera poses into structured text; (3) hierarchical GRPO with separate positive/negative rewards to train selective trust. The ablation shows that tools alone (39.9%) plus GRPO (54.4%) is more than additive.
I don't see any critical flaws. The weakest assumption is that the zero-shot perception tools (SAM3, Pi3X) stay accurate across diverse scenes. The paper acknowledges this and tests with Gaussian noise and mask dropout. Those tests show graceful degradation, though Gaussian noise may not capture systematic biases in scale or pose. That's a minor concern, not a dealbreaker. The empty-target fallback covers the <1.6% of cases where tools return nothing.
A couple of minor points: the training data is only 3,032 examples from MindCube, so the gains on unseen benchmarks (5–8 points) are modest compared to MindCube's 21.5 points. The hierarchical reward (+1 for positive, +2 for negative) works empirically but the motivation could be sharper—the ablation confirms it helps, but why the asymmetry? These are soft spots, not fatal.
Who should read this? Anyone working on spatial reasoning in VLMs, embodied AI, or tool-augmented reasoning. It's a practical method that fits on a 24 GB GPU. I'd accept it for peer review: the paper is sound, well-evaluated, and honestly limited. I would cite it.
The strongest objection
The paper's central claim—that query-conditioned geometry acquisition, symbolic 3D state, and hierarchical GRPO significantly improve spatial reasoning—is well-supported by consistent benchmark gains, careful ablations isolating each component, controlled perturbation tests, and honest discussion of limitations. The held-out evaluation on MMSI-Bench and ViewSpatial-Bench demonstrates positive transfer from MindCube training, mitigating concerns about dataset-specific overfitting. The weakest assumption (perception tool accuracy) is acknowledged by the authors in Section 4.10 and bounded by the perturbation experiments and the empty-target fallback mechanism. I find no critical unaddressed internal inconsistency or evidentiary gap that undermines the central argument.
The reader identified perception tool accuracy as the weakest assumption. The paper acknowledges this limitation and provides robustness experiments (Table 2) and a fallback mechanism. The held-out benchmarks and consistent backbone scaling further support generalization. I find no novel concern that the reader missed. The concrete test above would provide additional evidence for the framework's independence from the specific segmentation model, strengthening the claim without contradicting the current verdict.
The referee's report
If the reported results hold, Think-Capture makes three important contributions. First, it introduces a query-conditioned acquisition paradigm that replaces the common 'fuse everything' approach with on-demand geometry, reducing computation and irrelevant context. Second, it provides a compact symbolic interface (templated object and camera streams) that bridges the gap between raw point maps and VLM language understanding. Third, the hierarchical GRPO training explicitly teaches the VLM when to trust and when to ignore imperfect tool evidence, which is critical for robust deployment. The consistent gains across multiple benchmarks and backbone sizes, together with the ablation and robustness experiments, suggest that active, query-conditioned geometry acquisition is a practical and effective alternative to dense fusion for spatial reasoning in VLMs. The real-robot deployment on a single RTX 4090 demonstrates potential for embodied applications.
Pith's reading of the paper (a machine's summary, not the authors')
Think-Capture replaces the common practice of densely reconstructing the entire scene with a query-conditioned strategy: the VLM first names which objects it needs, a 3D toolkit then captures masks, point clouds, and poses for just those targets, and the results are distilled into a compact symbolic text state. A hierarchical GRPO reward trains the VLM to use this geometry when it helps and to ignore it when the tool output is misleading. On three spatial reasoning benchmarks—MindCube-Tiny, MMSI-Bench, and ViewSpatial-Bench—the 4B model consistently outperforms its backbone by large margins while remaining robust to coordinate noise and mask dropout. The full stack fits on a single 24 GB GPU and has been demonstrated as an asynchronous high-level reasoner on an Agibot G1 robot.
What carries the argument. The framework combines (1) query-conditioned target selection, where the VLM emits semantic objects to segment; (2) a 3D Capturing Toolkit that wraps SAM3 for zero-shot segmentation and π3 (Pi3X) for metric point maps and camera poses, lifting masks into a common world frame; (3) a Symbolic 3D State Abstraction that compresses object positions, scales, and pairwise relations along with camera poses into structured text (~100 tokens per query); and (4) Hierarchical Group Relative Policy Optimization (GRPO) that uses disagreement-mined positive and negative examples to train the VLM to exploit helpful geometry and reject misleading evidence.
Core claim. The central discovery is that a two-stage Reason-then-Act loop—where the VLM first plans which geometry to acquire (Stage 1) and then reasons over the original images together with the captured symbolic state (Stage 2)—can be trained with hierarchical GRPO to combine visual and 3D evidence selectively. The key insight is that dense scene geometry is often unnecessary and can mislead; acquiring only task-relevant objects on demand reduces computation, cuts context length by roughly half, and makes the geometric evidence explicit enough for the VLM to learn when to trust it. The result is a practical, GPU-efficient framework that matches or approaches much larger proprietary models on multi-view spatial understanding.
The premise it rests on. The zero-shot perception tools (SAM3 for segmentation and π3 for point maps and poses) produce sufficiently accurate geometry across the diversity of scenes in the evaluated benchmarks; systematic errors such as depth bias or pose drift that are not captured by the tested Gaussian noise or mask dropout could lead the symbolic state astray even with GRPO training.
If the paper is right (Pith's inferences, not the authors')
VLMs can achieve grounded 3D spatial reasoning without dense reconstruction, cutting per-query context and computation.
The method transfers across backbone sizes (2B, 4B, 8B) and to unseen benchmarks without retraining, suggesting the learned integration policy is architecture-agnostic.
The symbolic state bottleneck (~100 tokens) makes the 3D evidence directly inspectable and editable by the VLM, unlike fused 3D features.
The full VLM+SAM3+π3 stack fits on a single 24GB GPU, lowering the hardware barrier for 3D-aware embodied systems.
The real-robot deployment demonstrates that an asynchronous high-level spatial reasoner can couple with a low-level motor policy on consumer hardware.
The approach implies that explicit, task-conditioned geometric state is more sample-efficient than implicit learning of spatial relations from images alone, especially for cross-view and egocentric transformations.
The disagreement-mined training strategy—selecting cases where the vision-only and tool-augmented models disagree—could generalize to other tool-use scenarios, providing a general recipe for teaching VLMs to calibrate trust in external sensors.
The symbolic bottleneck might serve as a design pattern for grounding VLMs in other structured modalities (e.g., audio, force) where a compact text representation can carry task-relevant information without the modality gap of raw signals.
The perturbation experiments suggest that learned integration (GRPO) is more critical than raw tool accuracy: even under 0.4m coordinate noise, the full system retains 46.2% accuracy versus 26.7% for naive tool use, showing that the VLM learns to hedge against imperfect geometry.
Check this review
Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.
How often reviews like this one are right
Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.
How often objections from reviews like this one are right: not yet measured
Reviews by people
This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.
The paper
Abstract
Vision-Language Models (VLMs) excel at 2D recognition but often fail to maintain consistent 3D relations across sparse viewpoints, leading to spatial hallucinations under occlusion and perspective changes. Existing methods commonly fuse all frames with dense geometry, incurring substantial computation and context overhead while encouraging blind reliance on imperfect estimators. We present Think-Capture, a query-conditioned framework that first identifies task-critical objects and then acquires their geometry on demand. A 3D toolkit lifts zero-shot masks into a common coordinate frame, and a Symbolic 3D State Abstraction compresses object and camera states into structured text. Hierarchical Group Relative Policy Optimization (GRPO) trains the VLM on cases in which geometry either helps or misleads, teaching it when to use tool evidence and when to fall back on visual cues. Think-Capture-4B reaches 54.4% on MindCube-Tiny (+21.5 points over its backbone), 39.1% on the MMSI-Bench positional subset (+7.9), and 42.8% on ViewSpatial (+5.2). It also remains robust under coordinate and mask perturbations. The full stack fits one 24 GB GPU and serves as an asynchronous high-level reasoner on an Agibot G1. Our code is available at https://github.com/819346675/Think-Capture.
The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.
abstract compiled · 2026-10-22
The authors' main claim, as the classifier picked it out 0.89
“Think-Capture-4B reaches 54.4% on MindCube-Tiny (+21.5 points over its backbone), 39.1% on the MMSI-Bench positional subset (+7.9), and 42.8% on ViewSpatial (+5.2).”
How the main claim is stated: asserted 1.00
What the abstract itself states, as probabilities: claims first 0.07 · states limitation 0.22 · human participants 0.04 · controlled comparison 0.82 · randomized assignment 0.02 · states quantitative result 0.99 · compares with prior results 0.92 · mentions code or data availability 0.98
The abstract, sentence by sentence
This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.
| Sentence | Role | The authors' own work | States the main conclusion | Strength |
|---|---|---|---|---|
| Vision-Language Models (VLMs) excel at 2D recognition but often fail to maintain consistent 3D relations across sparse viewpoints, leading to spatial hallucinations under occlusion and perspective changes. | background 0.94 | 0.08 | 0.03 | asserted 0.96 |
| Existing methods commonly fuse all frames with dense geometry, incurring substantial computation and context overhead while encouraging blind reliance on imperfect estimators. | background 0.76 | 0.06 | 0.02 | asserted 0.99 |
| We present Think-Capture, a query-conditioned framework that first identifies task-critical objects and then acquires their geometry on demand. | contribution 0.95 | 0.91 | 0.16 | asserted 1.00 |
| A 3D toolkit lifts zero-shot masks into a common coordinate frame, and a Symbolic 3D State Abstraction compresses object and camera states into structured text. | method 0.99 | 0.89 | 0.03 | asserted 1.00 |
| Hierarchical Group Relative Policy Optimization (GRPO) trains the VLM on cases in which geometry either helps or misleads, teaching it when to use tool evidence and when to fall back on visual cues. | method 0.99 | 0.93 | 0.08 | asserted 1.00 |
| Think-Capture-4B reaches 54.4% on MindCube-Tiny (+21.5 points over its backbone), 39.1% on the MMSI-Bench positional subset (+7.9), and 42.8% on ViewSpatial (+5.2). | result 1.00 | 0.96 | 0.31 | asserted 1.00 |
| It also remains robust under coordinate and mask perturbations. | result 1.00 | 0.91 | 0.06 | asserted 1.00 |
| The full stack fits one 24 GB GPU and serves as an asynchronous high-level reasoner on an Agibot G1. | result 0.38 | 0.89 | 0.05 | asserted 1.00 |
| Our code is available at https://github.com/819346675/Think-Capture. | contribution 0.75 | 0.62 | 0.02 | asserted 0.99 |
What kind of paper the classifier reports this is: method or tool 1.00 · evidence from observation or experiment 0.97
What was examined, and what was not
Sources read
- arXiv listing abstract v1
Not examined
- full text
- figures and tables
- references and citing sentences
- relations to other papers
Levels run
- P abstract
Not run
- S sections
- T body sentences
- C citations
- R cross-paper relations
How the labels were made
Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-10-22: 1 request, 7,048 input tokens.
Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON
Figures (5)
Figures are the authors', reproduced from the paper.
References (64)
- [1] Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, Wenbin Ge, Zhifang Guo, Qidong Huang, Jie Huang, Fei Huang, Binyuan Hui, Shutong Jiang, Zhao- hai Li, Mingsheng Li, Mei Li, Kaixin Li, Zicheng Lin, Jun... on PithPith's reviewarXiv
- [2] Kevin Black, Noah Brown, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Lachy Groom, Karol Hausman, Brian Ichter, Szymon Jakubczak, Tim Jones, Liyiming Ke, Sergey Levine, Adrian Li-Bell, Mo- hith Mothukuri, Suraj Nair, Karl Pertsch, Lucy Xiaoyang Shi, J... on PithPith's reviewarXiv
- [3] Nicolas Carion, Laura Gustafson, Yuan-Ting Hu, Shoub- hik Debnath, Ronghang Hu, Didac Suris, Chaitanya Ryali, Kalyan Vasudev Alwala, Haitham Khedr, Andrew Huang, Jie Lei, Tengyu Ma, Baishan Guo, Arpit Kalla, Markus Marks, Joseph Greer, Meng Wang, Peize Sun, Roman R¨adle, Trian... on PithPith's reviewarXiv
- [4] Yihang Chen, Pin Qian, Su Wang, Chong Peng, Huan Xu, Shuaiting Li, and Yiqi Sun. Explicit State Elicitation Is Not Enough: A Controlled Audit of Memory-Policy Classification. arXiv preprint arXiv:2608.17247, 2026 on PithPith's reviewarXiv
- [5] Zhangquan Chen, Manyuan Zhang, Xinlei Yu, Xufang Luo, Mingze Sun, Zihao Pan, Xiang An, Yan Feng, Peng Pei, Xunliang Cai, and Ruqi Huang. Think with 3D: Geomet- ric Imagination Grounded Spatial Reasoning from Limited Views.arXiv preprint arXiv:2510.18632, 2025 on PitharXiv
- [6] Erik Daxberger, Nina Wenzel, David Griffiths, Haiming Gang, Justin Lazarow, Gefen Kohavi, Kai Kang, Marcin Eichner, Yinfei Yang, Afshin Dehghan, and Peter Grasch. MM-Spatial: Exploring 3D Spatial Understanding in Multimodal LLMs. InProceedings of the IEEE/CVF International Con... on Pith
- [7] Jiajun Deng, Tianyu He, Li Jiang, Tianyu Wang, Feras Day- oub, and Ian Reid. 3D-LLaV A: Towards Generalist 3D LMMs with Omni Superpoint Transformer. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 3772–3782, 2025 on Pith
- [8] Zhiwen Fan, Jian Zhang, Renjie Li, Junge Zhang, Run- jin Chen, Hezhen Hu, Kevin Wang, Peihao Wang, Huaizhi Qu, Shijie Zhou, Dilin Wang, Zhicheng Yan, Hongyu Xu, Justin Theiss, Tianlong Chen, Jiachen Li, Zhengzhong Tu, Zhangyang Wang, and Rakesh Ranjan. VLM-3R: Vision- Language... on Pith
- [9] Kaituo Feng, Kaixiong Gong, Bohao Li, Zonghao Guo, Yibing Wang, Tianshuo Peng, Junfei Wu, Xiaoying Zhang, Benyou Wang, and Xiangyu Yue. Video-R1: Rein- forcing Video Reasoning in MLLMs.arXiv preprint arXiv:2503.21776, 2025 on PithPith's reviewarXiv
- [10] Gemini Team, Google. Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities.arXiv preprint arXiv:2507.06261, 2025 on PithPith's reviewarXiv
- [11] GLM-4.5 Team. GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Models.arXiv preprint arXiv:2508.06471, 2025 on PithPith's reviewarXiv
- [12] Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Peiyi Wang, Qihao Zhu, Runxin Xu, Ruoyu Zhang, Shirong Ma, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei Feng, Chengda ... on Pith
- [13] Yuheng Ji, Huajie Tan, Jiayu Shi, Xiaoshuai Hao, Yuan Zhang, Hengyuan Zhang, Pengwei Wang, Mengdi Zhao, Yao Mu, Pengju An, Xinda Xue, Qinghang Su, Huaihai Lyu, Xiao- long Zheng, Jiaming Liu, Zhongyuan Wang, and Shanghang Zhang. RoboBrain: A Unified Brain Model for Robotic Mani... on Pith
- [14] P. N. Johnson-Laird. Mental Models in Cognitive Science. Cognitive Science, 4(1):71–115, 1980 on Pith
- [15] Philip N. Johnson-Laird.Mental Models: Towards a Cog- nitive Science of Language, Inference, and Consciousness. Harvard University Press, 1983 on Pith
- [16] JoyAI-RA Team. JoyAI-RA 0.5: Scaling Robot Manipula- tion Learning via Dual Action Alignment.arXiv preprint arXiv:2608.05674, 2026 on PithPith's reviewarXiv
- [17] Nikhil Keetha, Norman M ¨uller, Johannes Sch ¨onberger, Lorenzo Porzi, Yuchen Zhang, Tobias Fischer, Arno Knapitsch, Duncan Zauss, Ethan Weber, Nelson Antunes, Jonathon Luiten, Manuel Lopez-Antequera, Samuel Rota Bul`o, Christian Richardt, Deva Ramanan, Sebastian Scherer, and ... on PithPith's reviewarXiv
- [18] Kimi Team, Angang Du, Bohong Yin, Bowei Xing, Bowen Qu, Bowen Wang, Cheng Chen, Chenlin Zhang, Chenzhuang Du, Chu Wei, Congcong Wang, Dehao Zhang, Dikang Du, Dongliang Wang, Enming Yuan, Enzhe Lu, Fang Li, Flood Sung, Guangda Wei, Guokun Lai, Han Zhu, Hao Ding, Hao Hu, Hao Yan... on PithPith's reviewarXiv
- [19] Phillip Y . Lee, Jihyeon Je, Chanho Park, Mikaela Angelina Uy, Leonidas Guibas, and Minhyuk Sung. Perspective-Aware Reasoning in Vision-Language Models via Mental Imagery Simulation. InProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 9241–9... on Pith
- [20] Dingming Li, Hongxing Li, Zixuan Wang, Yuchen Yan, Hang Zhang, Siqi Chen, Guiyang Hou, Shengpei Jiang, Wenqi Zhang, Yongliang Shen, Weiming Lu, and Yueting Zhuang. ViewSpatial-Bench: Evaluating Multi-perspective Spatial Localization in Vision-Language Models.arXiv preprint arX... on PitharXiv
show 44 more
- [21] Hongxing Li, Dingming Li, Zixuan Wang, Yuchen Yan, Hang Wu, Wenqi Zhang, Yongliang Shen, Weiming Lu, Jun Xiao, and Yueting Zhuang. SpatialLadder: Progressive Training for Spatial Reasoning in Vision-Language Models. InInterna- tional Conference on Learning Representations, 2026 on Pith
- [22] Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Qing Jiang, Chunyuan Li, Jianwei Yang, Hang Su, Jun Zhu, and Lei Zhang. Grounding DINO: Marry- ing DINO with Grounded Pre-training for Open-Set Object Detection. InComputer Vision – ECCV 2024, pages 38–55, 2024 on Pith
- [23] Wufei Ma, Yu-Cheng Chou, Qihao Liu, Xingrui Wang, Celso de Melo, Jianwen Xie, and Alan Yuille. SpatialReasoner: Towards Explicit and Generalizable 3D Spatial Reasoning. In Advances in Neural Information Processing Systems, 2025 on Pith
- [24] Wufei Ma, Luoxin Ye, Celso M de Melo, Alan Yuille, and Jieneng Chen. SpatialLLM: A Compound 3D-Informed De- sign towards Spatially-Intelligent Large Multimodal Models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 17249–17260, 2025 on Pith
- [25] Yue Ma, Yingqing He, Xiaodong Cun, Xintao Wang, Siran Chen, Xiu Li, and Qifeng Chen. Follow Your Pose: Pose- Guided Text-to-Video Generation Using Pose-Free Videos. In 10 Proceedings of the AAAI Conference on Artificial Intelligence, pages 4117–4125, 2024 on Pith
- [26] Yue Ma, Yulong Liu, Qiyuan Zhu, Ayden Yang, Kunyu Feng, Xinhua Zhang, Zexuan Yan, Zhifeng Li, Sirui Han, Chenyang Qi, and Qifeng Chen. Follow-Your-Motion: Video Motion Transfer via Efficient Spatial-Temporal Decoupled Finetuning. arXiv preprint arXiv:2506.05207, 2025 on PitharXiv
- [27] Yue Ma, Jiangming Wang, Yucheng Wang, Xilai Wang, Zhiyuan Li, Xinyu Wang, Hongyu Liu, Ruofan Liang, Songchun Zhang, Yuxuan Xue, and Qifeng Chen. LiveLight: Real-time Streaming Video Relighting with Interactive Con- trol.arXiv preprint arXiv:2608.01771, 2026 on PithPith's reviewarXiv
- [28] Runqing Miao, Sheng Xu, Runyi Zhao, Wai Kin (Victor) Chan, and Guiliang Liu. Uncertainty-aware Preference Align- ment for Diffusion Policies. InAdvances in Neural Informa- tion Processing Systems, 2025 on Pith
- [29] OpenAI. GPT-4o System Card.arXiv preprint arXiv:2410.21276, 2024 on PithPith's reviewarXiv
- [30] Kun Ouyang, Yuanxin Liu, Haoning Wu, Yi Liu, Hao Zhou, Jie Zhou, Fandong Meng, and Xu Sun. SpaceR: Reinforc- ing MLLMs in Video Spatial Reasoning.arXiv preprint arXiv:2504.01805, 2025 on PithPith's reviewarXiv
- [31] Physical Intelligence, Ali Amin, Raichelle Aniceto, Ashwin Balakrishna, Kevin Black, Ken Conley, Grace Connors, James Darpinian, Karan Dhabalia, Jared DiCarlo, Danny Driess, Michael Equi, Adnan Esmail, Yunhao Fang, Chelsea Finn, Catherine Glossop, Thomas Godden, Ivan Goryachev... on PithPith's reviewarXiv
- [32] Physical Intelligence, Kevin Black, Noah Brown, James Darpinian, Karan Dhabalia, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Manuel Y . Gal- liker, Dibya Ghosh, Lachy Groom, Karol Hausman, Brian Ichter, Szymon Jakubczak, Tim Jones, Liyiming Ke, Devin... on PithPith's reviewarXiv
- [33] Zhangyang Qi, Zhixiong Zhang, Ye Fang, Jiaqi Wang, and Hengshuang Zhao. GPT4Scene: Understand 3D Scenes from Videos with Vision-Language Models.arXiv preprint arXiv:2501.01428, 2025 on PithPith's reviewarXiv
- [34] Santhosh Kumar Ramakrishnan, Erik Wijmans, Philipp Kraehenbuehl, and Vladlen Koltun. Does Spatial Cog- nition Emerge in Frontier Models?arXiv preprint arXiv:2410.06468, 2024 on PithPith's reviewarXiv
- [35] Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y . K. Li, Y . Wu, and Daya Guo. DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models.arXiv preprint arXiv:2402.03300, 2024 on PithPith's reviewarXiv
- [36] StarVLA Community. StarVLA: A Lego-like Codebase for Vision-Language-Action Model Developing.arXiv preprint arXiv:2604.05014, 2026 on PithPith's reviewarXiv
- [37] Zhaochen Su, Linjie Li, Mingyang Song, Yunzhuo Hao, Zhengyuan Yang, Jun Zhang, Guanjie Chen, Jiawei Gu, Jun- tao Li, Xiaoye Qu, and Yu Cheng. OpenThinkIMG: Learning to Think with Images via Visual Tool Reinforcement Learn- ing.arXiv preprint arXiv:2505.08617, 2025 on PithPith's reviewarXiv
- [38] Zhaochen Su, Peng Xia, Hangyu Guo, Zhenhua Liu, Yan Ma, Xiaoye Qu, Jiaqi Liu, Yanshu Li, Kaide Zeng, Zhengyuan Yang, Linjie Li, Yu Cheng, Heng Ji, Junxian He, and Yi R. Fung. Thinking with Images for Multimodal Reasoning: Foundations, Methods, and Future Frontiers.arXiv prepri... on PithPith's reviewarXiv
- [39] Jianyuan Wang, Minghao Chen, Nikita Karaev, Andrea Vedaldi, Christian Rupprecht, and David Novotny. VGGT: Visual Geometry Grounded Transformer. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 5294–5306, 2025 on Pith
- [40] Qineng Wang, Baiqiao Yin, Pingyue Zhang, Jianshu Zhang, Kangrui Wang, Zihan Wang, Jieyu Zhang, Keshigeyan Chan- drasegaran, Han Liu, Ranjay Krishna, Saining Xie, Jiajun Wu, Li Fei-Fei, and Manling Li. MindCube: Spatial Mental Mod- eling from Limited Views.arXiv preprint arXiv:... on PitharXiv
- [41] Xinyu Wang, Chongbo Zhao, Fangneng Zhan, and Yue Ma. LiveEdit: Towards Real-Time Diffusion-Based Streaming Video Editing.arXiv preprint arXiv:2606.26740, 2026 on PithPith's reviewarXiv
- [42] Yifan Wang, Jianjun Zhou, Haoyi Zhu, Wenzheng Chang, Yang Zhou, Zizun Li, Junyi Chen, Jiangmiao Pang, Chunhua Shen, and Tong He. π3: Permutation-Equivariant Visual Geometry Learning.arXiv preprint arXiv:2507.13347, 2025 on PithPith's reviewarXiv
- [43] Yongzhong Wang, Keyu Zhu, Yong Zhong, Liqiong Wang, Jinyu Yang, and Feng Zheng. RADAR: Closed-Loop Robotic Data Generation via Semantic Planning and Autonomous Causal Environment Reset.arXiv preprint arXiv:2603.11811, 2026 on PithPith's reviewarXiv
- [44] Diankun Wu, Fangfu Liu, Yi-Hsin Hung, and Yueqi Duan. Spatial-MLLM: Boosting MLLM Capabilities in Visual- based Spatial Intelligence. InAdvances in Neural Information Processing Systems, 2025 on Pith
- [45] Runsen Xu, Weiyao Wang, Hao Tang, Xingyu Chen, Xi- aodong Wang, Fu-Jen Chu, Matt Feiszli, and Kevin J. Liang. Multi-SpatialMLLM: Multi-Frame Spatial Understanding with Multi-Modal Large Language Models.arXiv preprint arXiv:2505.17015, 2025 on PithPith's reviewarXiv
- [46] Sheng Xu and Guiliang Liu. Robust Inverse Constrained Reinforcement Learning under Model Misspecification. In Proceedings of the 41st International Conference on Machine Learning, pages 55162–55185. PMLR, 2024. 11 on Pith
- [47] Sheng Xu and Guiliang Liu. Uncertainty-aware Constraint Inference in Inverse Constrained Reinforcement Learning. In International Conference on Learning Representations, 2024 on Pith
- [48] Sheng Xu, Bo Yue, Hongyuan Zha, and Guiliang Liu. A Distributional Approach to Uncertainty-Aware Preference Alignment Using Offline Demonstrations. InInternational Conference on Learning Representations, 2025 on Pith
- [49] Sheng Xu, Ruixing Jin, Huayi Zhou, Bo Yue, Guanren Qiao, Yunxin Tai, Yueci Deng, Kui Jia, and Guiliang Liu. From Reaction to Anticipation: Proactive Failure Recovery through Agentic Task Graph for Robotic Manipulation.arXiv preprint arXiv:2605.11951, 2026 on PithPith's reviewarXiv
- [50] Jihan Yang, Shusheng Yang, Anjali W. Gupta, Rilyn Han, Li Fei-Fei, and Saining Xie. Thinking in Space: How Multi- modal Large Language Models See, Remember, and Recall Spaces. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 106... on Pith
- [51] Rui Yang, Hanyang Chen, Junyu Zhang, Mark Zhao, Cheng Qian, Kangrui Wang, Qineng Wang, Teja Venkat Koripella, Marziyeh Movahedi, Manling Li, Heng Ji, Huan Zhang, and Tong Zhang. EmbodiedBench: Comprehensive Benchmark- ing Multi-modal Large Language Models for Vision-Driven Emb... on PithPith's reviewarXiv
- [52] Shusheng Yang, Jihan Yang, Pinzhi Huang, Ellis Brown, Zi- hao Yang, Yue Yu, Shengbang Tong, Zihan Zheng, Yifan Xu, Muhan Wang, Daohan Lu, Rob Fergus, Yann LeCun, Li Fei-Fei, and Saining Xie. Cambrian-S: Towards Spatial Supersensing in Video.arXiv preprint arXiv:2511.04670, 2025 on PithPith's reviewarXiv
- [53] Sihan Yang, Runsen Xu, Yiman Xie, Sizhe Yang, Mo Li, Jingli Lin, Chenming Zhu, Xiaochen Chen, Haodong Duan, Xiangyu Yue, Dahua Lin, Tai Wang, and Jiangmiao Pang. MMSI-Bench: A Benchmark for Multi-Image Spatial Intelli- gence. InInternational Conference on Learning Representa- ... on Pith
- [54] Chun-Hsiao Yeh, Chenyu Wang, Shengbang Tong, Ta-Ying Cheng, Ruoyu Wang, Tianzhe Chu, Yuexiang Zhai, Yubei Chen, Shenghua Gao, and Yi Ma. Seeing from Another Per- spective: Evaluating Multi-View Understanding in MLLMs. arXiv preprint arXiv:2504.15280, 2025 on PithPith's reviewarXiv
- [55] Songsong Yu, Yuxin Chen, Hao Ju, Lianjie Jia, Fuxi Zhang, Shaofei Huang, Yuhan Wu, Rundi Cui, Binghao Ran, Zaibin Zhang, Zhedong Zheng, Zhipeng Zhang, Yi- fan Wang, Lin Song, Lijun Wang, Yanwei Li, Ying Shan, and Huchuan Lu. How Far are VLMs from Visual Spatial Intel- ligence?... on PitharXiv
- [56] Zhihao Yuan, Xu Yan, Yinghong Liao, Ruimao Zhang, Sheng Wang, Zhen Li, and Shuguang Cui. InstanceRefer: Cooper- ative Holistic Understanding for Visual Grounding on Point Clouds through Instance Multi-level Contextual Referring. In2021 IEEE/CVF International Conference on Comp... on Pith
- [57] Zhihao Yuan, Shuyi Jiang, Chun-Mei Feng, Yaolun Zhang, Shuguang Cui, Zhen Li, and Na Zhao. Scene-R1: Video- Grounded Large Language Models for 3D Scene Reasoning without 3D Annotations.arXiv preprint arXiv:2506.17545, 2025 on PithPith's reviewarXiv
- [58] Jirong Zha, Yuxuan Fan, Xiao Yang, Chen Gao, and Xinlei Chen. How to Enable LLM with 3D Capacity? A Survey of Spatial Reasoning in LLM.arXiv preprint arXiv:2504.05786, 2025 on PitharXiv
- [59] Tianle Zhang, Zhihao Yuan, Dafeng Chi, Peidong Liu, Dong- wei Li, Kejun Hu, Likui Zhang, Junnan Nie, Ziming Wei, Zengjue Chen, Yili Tang, Jiayi Li, Zhiyuan Xiang, Mingyang Li, Tianci Luo, Hanwen Wan, Ao Li, Linbo Zhai, Zhihao Zhan, Xiaodong Bai, Jiakun Cai, Peng Cao, Kangliang... on PithPith's reviewarXiv
- [60] Zaibin Zhang, Yuhan Wu, Lianjie Jia, Yifan Wang, Zhongbo Zhang, Yijiang Li, Binghao Ran, Fuxi Zhang, Zhuohan Sun, Yizhuang Peng, Zhenfei Yin, Lijun Wang, and Huchuan Lu. Think3D: Thinking with Space for Spatial Reasoning.arXiv preprint arXiv:2601.13029, 2026 on PithPith's reviewarXiv
- [61] Runyi Zhao, Sheng Xu, Ruixing Jin, Yueci Deng, Yunxin Tai, Kui Jia, and Guiliang Liu. Sim2Real VLA: Zero-Shot Gener- alization of Synthesized Skills to Realistic Manipulation. In International Conference on Learning Representations, 2026 on Pith
- [62] Duo Zheng, Shijia Huang, Yanyang Li, and Liwei Wang. Learning from Videos for 3D World: Enhancing MLLMs with 3D Vision Geometry Priors. InAdvances in Neural Information Processing Systems, 2025 on Pith
- [63] Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shen- glong Ye, Lixin Gu, Hao Tian, Yuchen Duan, Weijie Su, Jie Shao, Zhangwei Gao, Erfei Cui, Xuehui Wang, Yue Cao, Yangzhou Liu, Xingguang Wei, Hongjie Zhang, Haomin Wang, Weiye Xu, Hao Li, Jiahao Wang, Nianchen Deng, Songze L... on PithPith's reviewarXiv
- [64] Filippo Ziliotto, Tommaso Campari, Luciano Serafini, and Lamberto Ballan. TANGO: Training-free Embodied AI Agents for Open-world Tasks. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 24603–24613, 2025. 12 Supplementary Materia... on Pith
Challenges and discussion
Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.
Come back by email
A short list of the week's newly reviewed papers. Nothing is sent until you confirm from your inbox, and every email has a link that stops it. No account.
No comments yet.
Sign in to comment