Pith. sign in

REVIEW 3 major objections 6 minor 46 references

Projecting robot state into images boosts manipulation 8.7%

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · glm-5.2

2026-07-09 20:03 UTC pith:VVN7G56F

load-bearing objection Solid empirical contribution: geometric grounding of proprioception in visual feature space yields consistent gains, but the 'consistent enhancement' claim is overstated given 5/15 tasks degrade in expanded results. the 3 major comments →

arxiv 2607.07101 v1 pith:VVN7G56F submitted 2026-07-08 cs.RO cs.AI

GeoProp: Grounding Robot State in Vision for Generalist Manipulation

Guoyang Zhao , Quanhao Qian , Gongjie Zhang , Wenhao Li , Jiuniu Wang , Xiaowei Lu , Deli Zhao , Ran Xu This is my paper
classification cs.RO cs.AI
keywords proprioceptionvisual groundingrobot manipulationgeometric projectionfeature modulationdiffusion policyvision-language-action
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The author is trying to establish that the representational gap between 3D robot kinematics and 2D visual features is not a minor engineering inconvenience but a structural limitation that can degrade policy performance below vision-only baselines. The central mechanism is geometric projection: take the end-effector's 3D position, project it through known camera parameters onto the image plane, and sample the visual feature at that pixel. This produces a state token that is spatially co-located with the visual evidence the robot needs to act on, rather than a global vector that the policy must learn to associate with the right image region from scratch. The paper shows this works as a plug-and-play adapter across two architecturally distinct policy families — diffusion-based controllers and vision-language-action models — suggesting the problem and its geometric solution are framework-independent. The gains are largest on precision tasks involving small objects, where localizing the interaction region in visual space matters most, and the method adds only 2–3% parameter overhead. A secondary mechanism, predictive kinematic sampling, extrapolates a short-horizon future waypoint from recent motion history and samples features there, giving the policy a look-ahead visual context that captures motion intent.

Core claim

The paper claims that standard practice — feeding robot proprioception as an isolated, ungrounded vector into manipulation policies — is not just suboptimal but can be actively harmful, sometimes producing worse performance than using vision alone. The fix is to project the robot's 3D end-effector position onto the 2D image plane and sample the visual feature at that exact location, creating a grounded state token that inherits scene semantics from the same latent space as the visual representation. This geometric projection, combined with localized feature modulation at the projected cell and a short-horizon predictive sample, gives the policy an explicit correspondence between robot kinem×

What carries the argument

The central object is the grounded state token, constructed by: (1) projecting the 3D end-effector position through camera intrinsics and extrinsics to obtain a 2D image coordinate; (2) mapping that coordinate to the visual backbone's feature grid; (3) applying FiLM modulation — channel-wise affine transformation conditioned on proprioceptive state — only at the feature cell aligned with the projected location; (4) aggregating multi-scale features via an FPN; and (5) bilinear-sampling the aggregated feature at the projected coordinate to produce the token. A second token is sampled at a polynomially extrapolated future waypoint from recent end-effector history. Both tokens are concatenated (

Load-bearing premise

The method assumes that the projected end-effector position lands on a visually informative, unoccluded region of the image. When the manipulated object covers the gripper — as in closing a box — the sampled feature carries noise rather than useful scene semantics, and the method's gains can shrink or invert.

What would settle it

If one could show that simply providing the projected 2D coordinate as a dense heatmap or relative-coordinate encoding (without co-located feature sampling) matches GeoProp's gains, the core claim that localized feature sampling is the active mechanism would be undermined. The paper addresses this with the Heatmap and RC-PE baselines, which underperform, but the test is only on 15 MetaWorld tasks.

Watch this falsifier — get emailed when new claim-graph text bears on it.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If geometric correspondence between 3D state and 2D features is the key inductive bias, then policies operating in multi-view or moving-camera settings could benefit from projecting state onto multiple image planes simultaneously, creating redundant grounded tokens that are robust to single-view occlusion.
  • The super-additive effect of motion sampling and FiLM modulation suggests these components address distinct failure modes — motion intent and feature conditioning, respectively — which implies further gains may be available from additional geometrically-motivated conditioning sites such as contact-point or object-center projection.
  • The fact that a parameter-matched control recovers almost no gain suggests the improvement is genuinely architectural rather than capacity-driven, which would predict that gains persist even as backbone scale increases — a pattern partially confirmed by the π0 results but not yet tested at larger model scales.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The manuscript introduces GeoProp, a lightweight adapter that aligns proprioceptive state with visual features by projecting the 3D end-effector position onto the 2D image plane, sampling co-located visual features via bilinear interpolation, and applying localized FiLM modulation. A predictive kinematic sampling component additionally samples features at a short-horizon extrapolated waypoint to capture motion intent. The method is evaluated across 67 tasks spanning MetaWorld, RLBench, and RoboTwin in simulation, plus 4 real-world tasks on Mobile ALOHA, using two policy families (Diffusion Policy and π0) and two vision backbones (ResNet-18, ViT-B). The central empirical claim is that explicit geometric grounding of proprioception yields consistent gains (8.7% for DP in simulation, 4.0% for π0 on RoboTwin, 10.6% real-world average) at 2–3% parameter overhead.

Significance. The paper addresses a genuine and underexplored representational gap in robot learning: the implicit and often ineffective fusion of low-dimensional proprioception with high-dimensional visual tokens. The core idea—projecting the end-effector into image space and sampling co-located features—is geometrically principled, backbone-agnostic, and simple to implement. The experimental breadth is commendable: 67 tasks, two distinct policy architectures (shallow-concatenation DP and deep-attention VLA), parameter-matched controls, component ablations, alternative spatial-grounding baselines (Heatmap, RC-PE), and calibration robustness tests. The finding that ungrounded proprioception can underperform vision-only baselines is a valuable empirical observation for the community. The method's plug-and-play nature and low overhead strengthen its practical relevance.

major comments (3)
  1. The central claim of 'consistent enhancement' is not fully supported by the expanded results in Table B.5 (15-task RoboTwin with π0). GeoProp underperforms Vanilla π0 on 5 of 15 tasks (dump_bin_bigbin, pick_diverse_bottles, place_shoe, scan_object, stack_blocks_two, each −4pp). This is one-third of the expanded suite where the method actively hurts. The manuscript attributes these failures to occlusion and visual ambiguity (§6, §4.3) but provides only a single anecdotal example (Box Close, §4.3) rather than a systematic analysis correlating failure with specific geometric conditions. Without a principled characterization of when projection-based grounding helps versus hurts, the 'consistent' qualifier in the abstract and §1 is overstated. The authors should either soften the claim to reflect task-dependence or provide a systematic failure-mode analysis (e.g., correlating per-task gains/​
  2. The real-world headline result (10.6% average gain, Table 4) rests on only 20 trials per task across 4 tasks. At this sample size, a per-task difference of 4 successes out of 20 (e.g., Coffee Retrieval 35%→55%) is within the 95% confidence interval of a binomial with n=20 (approx. ±20pp). The simulation results are more robust, but the real-world claim is statistically fragile. The authors should either (a) report confidence intervals or binomial test p-values for the real-world results, or (b) explicitly acknowledge the statistical limitation in §5.2 rather than presenting the gains as definitive. This is load-bearing because the real-world transfer claim is a key part of the paper's contribution.
  3. The occlusion failure mode (§4.3, §6) is acknowledged but not systematically addressed. The manuscript states that 'gains shrink or invert when the manipulated object occludes the projected pixel,' yet the method has no mechanism to detect or mitigate this condition. For tasks where the gripper is frequently occluded by the manipulated object (e.g., Box Close, or tasks involving large objects), the grounded token may carry noise rather than useful scene semantics. The authors should discuss whether a confidence-weighted or occlusion-aware sampling mechanism could be incorporated, or at minimum provide a quantitative breakdown of how frequently the projected end-effector is occluded across the evaluated tasks and how this correlates with performance. Without this, the practical applicability of the method remains unclear for a significant class of manipulation tasks.
minor comments (6)
  1. §3.1, Eq. (1): The extrinsic notation uses (R, t) but the projection writes Π_K(R r_t + t). It would help to clarify whether R is the rotation from the robot/world frame to the camera frame and t is the corresponding translation, or the inverse convention, as this affects reproducibility.
  2. Table 1: The 'Overall' column averages across MetaWorld (50 tasks), RLBench (6 tasks), and RoboTwin (7 tasks) with equal weight per benchmark. This gives MetaWorld 50/63 weight in the 'Overall' number. Consider clarifying whether this is intended or reporting a simple mean of means.
  3. §4.1: MetaWorld uses per-task training while RLBench and RoboTwin use multi-task training. This difference is noted but its implications for the comparison are not discussed. Does GeoProp's benefit differ between multi-task and single-task settings?
  4. Table 2: The π0 results on RoboTwin show GeoProp underperforming Vanilla on 2 of 7 tasks (Beat Block Hammer: 88 vs 100; Stack Two Blocks: 60 vs 48). The text states GeoProp outperforms on 5/7, but the per-task variance is not discussed. A brief note on task-level variance would strengthen the claim.
  5. Appendix B.4, Fig. B.3: The calibration robustness results are informative but the y-axis values are not labeled with specific numbers in the description. Including explicit success rate values on the axes would make the degradation curves more interpretable and the claim more concrete.
  6. The project page URL (https://alibaba-damo-academy.github.io/GeoProp/) is referenced but the manuscript does not mention whether code or trained models are released. Given the plug-and-play nature of the method, a statement about code availability would be appropriate, even if only on the project page.

Circularity Check

0 steps flagged

No circularity found: GeoProp's derivation is self-contained and empirically validated against external benchmarks

full rationale

GeoProp's derivation chain is straightforward and non-circular. The method projects 3D end-effector positions to 2D image coordinates using standard camera geometry (Eq. 1), samples visual features at those coordinates via bilinear interpolation (Eq. 2-3), applies FiLM modulation conditioned on proprioceptive state at the projected location (Eq. 4-5), and constructs a predictive token via polynomial extrapolation of recent kinematics (Eq. 6). Each step is a forward computation: state → projection → feature sampling → token construction. No step reduces to its own input by definition. The central claim—that geometric grounding of proprioception improves manipulation—is validated empirically against external benchmarks (MetaWorld, RLBench, RoboTwin, Mobile ALOHA) using standard policy architectures (Diffusion Policy, π0) that the authors did not develop. Self-citations (e.g., Qian et al. [41] for MetaWorld task difficulty splits and ablation task selection) are methodological conveniences that do not bear on the core derivation. The FiLM parameters are generated from proprioceptive state through a learned MLP, but this is standard conditional modulation, not a self-defitional loop. The predictive waypoint is computed via polynomial extrapolation from a 4-frame history, which is a standard signal processing operation, not a fitted parameter renamed as prediction. The paper is self-contained against external benchmarks with no circular reasoning in its derivation chain.

Axiom & Free-Parameter Ledger

4 free parameters · 3 axioms · 0 invented entities

GeoProp introduces no new physical entities or postulated objects. The method is constructed from known components (FiLM modulation, FPN, bilinear sampling, polynomial extrapolation) applied in a novel configuration. The free parameters are architectural choices rather than fitted physical constants.

free parameters (4)
  • FiLM MLP hidden dimension = Not explicitly stated; two-layer MLP
    The FiLM generator architecture size is not fully specified in the main text.
  • Kinematic history window length = 4 frames
    Chosen for polynomial extrapolation; stated in implementation details.
  • Extrapolation polynomial order = Quadratic
    Used for predicting future waypoint; stated in implementation details.
  • FPN output stride = 16 (ResNet) / 14 (ViT)
    Determines feature map resolution for sampling; stated in Appendix A.1.
axioms (3)
  • domain assumption Camera intrinsics K and extrinsics (R,t) are known and accurate at deployment.
    Required for Eq. (1) to produce meaningful projections. The paper tests robustness to calibration drift (Appendix B.4) but the method fundamentally depends on this calibration being available.
  • domain assumption The projected 2D end-effector location corresponds to a visually informative region in the feature map.
    The grounded state token (Eq. 3) is only useful if the sampled visual feature contains task-relevant information. The paper acknowledges this fails under occlusion (§6).
  • domain assumption Short-horizon end-effector motion is locally smooth and predictable from recent history.
    Predictive kinematic sampling (§3.3) uses polynomial extrapolation, which assumes smooth trajectories. The paper notes abrupt direction changes can produce off-task look-ahead points (§6).

pith-pipeline@v1.1.0-glm · 20994 in / 2531 out tokens · 234746 ms · 2026-07-09T20:03:47.311086+00:00 · methodology

0 comments
read the original abstract

Proprioception is fundamental to robotic manipulation, yet standard fusion methods often treat it as an isolated vector lacking explicit alignment with visual tokens. Without a direct correspondence between 3D kinematics and 2D feature maps, manipulation policies struggle to ground the robot's state within the scene, frequently underperforming even vision-only baselines. To address this, we introduce GeoProp, a lightweight, plug-and-play adapter that aligns proprioception with vision through explicit geometric grounding and spatial feature sampling. GeoProp projects the robot state onto the image plane to sample localized visual features, constructing a grounded state token. It then injects state-derived spatial priors into the corresponding visual features via FiLM modulation. To capture motion intent, GeoProp further samples features at a short-horizon predicted coordinate derived from recent kinematics, providing look-ahead visual context. Across 67 tasks, GeoProp improves Diffusion Policy by 8.7% on 63 simulation tasks and pi_0 by 4.0% on the RoboTwin subset, and yields a 10.6% average gain across both policy families in the real world, while adding only 2-3% to the parameter count. These results demonstrate that GeoProp is a simple yet high-impact inductive bias for generalist embodied policies. Project page: https://alibaba-damo-academy.github.io/GeoProp/.

Figures

Figures reproduced from arXiv: 2607.07101 by Deli Zhao, Gongjie Zhang, Guoyang Zhao, Jiuniu Wang, Quanhao Qian, Ran Xu, Wenhao Li, Xiaowei Lu.

Figure 1
Figure 1. Figure 1: Proprioceptive-to-image attention in π0: GeoProp concentrates attention on the gripper and manipulated objects, while vanilla attention is diffuse. Despite the diversification of robot learning architectures—spanning diffusion-based con￾trollers [1, 2, 3], transformer-based action predic￾tors [4, 5], and large-scale vision–language–action (VLA) systems [6, 7, 8, 9]—a persistent represen￾tational limitation… view at source ↗
Figure 2
Figure 2. Figure 2: GeoProp projects end-effector and look-ahead waypoints to image features, producing [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

46 extracted references · 46 canonical work pages · 11 internal anchors

  1. [1]

    C. Chi, S. Feng, Y . Du, Z. Xu, E. Cousineau, B. Burchfiel, and S. Song. Diffusion policy: Visuomotor policy learning via action diffusion. InRobotics: Science and Systems, 2023. URL https://doi.org/10.15607/RSS.2023.XIX.026

    work page doi:10.15607/rss.2023.xix.026 2023

  2. [2]

    Y . Ze, G. Zhang, K. Zhang, C. Hu, M. Wang, and H. Xu. 3d diffusion policy: Generalizable visuomotor policy learning via simple 3d representations. InICRA 2024 Workshop on 3D Visual Representations for Robot Manipulation, 2024

    work page 2024

  3. [3]

    Dasari, O

    S. Dasari, O. Mees, S. Zhao, M. K. Srirama, and S. Levine. The ingredients for robotic diffusion transformers. In2025 IEEE International Conference on Robotics and Automation (ICRA), pages 15617–15625. IEEE, 2025

    work page 2025

  4. [4]

    T. Zhao, V . Kumar, S. Levine, and C. Finn. Learning fine-grained bimanual manipulation with low-cost hardware.Robotics: Science and Systems XIX, 2023

    work page 2023

  5. [5]

    Gervet, Z

    T. Gervet, Z. Xian, N. Gkanatsios, and K. Fragkiadaki. Act3d: 3d feature field transformers for multi-task robotic manipulation. InCoRL, 2023

    work page 2023

  6. [6]

    RT-1: Robotics Transformer for Real-World Control at Scale

    A. Brohan, N. Brown, J. Carbajal, Y . Chebotar, J. Dabis, C. Finn, K. Gopalakrishnan, K. Haus- man, A. Herzog, J. Hsu, et al. Rt-1: Robotics transformer for real-world control at scale.arXiv preprint arXiv:2212.06817, 2022

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  7. [7]

    Zitkovich, T

    B. Zitkovich, T. Yu, S. Xu, P. Xu, T. Xiao, F. Xia, J. Wu, P. Wohlhart, S. Welker, A. Wahid, et al. Rt-2: Vision-language-action models transfer web knowledge to robotic control. InConference on Robot Learning, pages 2165–2183. PMLR, 2023

    work page 2023

  8. [8]

    M. J. Kim, K. Pertsch, S. Karamcheti, T. Xiao, A. Balakrishna, S. Nair, R. Rafailov, E. P. Foster, P. R. Sanketi, Q. Vuong, et al. Openvla: An open-source vision-language-action model. In Conference on Robot Learning, pages 2679–2713. PMLR, 2025

    work page 2025

  9. [9]

    $\pi_0$: A Vision-Language-Action Flow Model for General Robot Control

    K. Black, N. Brown, D. Driess, A. Esmail, M. Equi, C. Finn, N. Fusai, L. Groom, K. Hausman, B. Ichter, et al. π0: A vision-language-action flow model for general robot control. corr, abs/2410.24164, 2024. doi: 10.48550.arXiv preprint ARXIV .2410.24164, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  10. [10]

    E. Jang, A. Irpan, M. Khansari, D. Kappler, F. Ebert, C. Lynch, S. Levine, and C. Finn. Bc-z: Zero-shot task generalization with robotic imitation learning. InConference on Robot Learning, pages 991–1002. PMLR, 2022

    work page 2022

  11. [11]

    M. J. Kim, C. Finn, and P. Liang. Fine-tuning vision-language-action models: Optimizing speed and success.arXiv preprint arXiv:2502.19645, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  12. [12]

    O. Mees, D. Ghosh, K. Pertsch, K. Black, H. R. Walke, S. Dasari, J. Hejna, T. Kreiman, C. Xu, J. Luo, et al. Octo: An open-source generalist robot policy. InFirst Workshop on Vision-Language Models for Navigation and Manipulation at ICRA 2024, 2024

    work page 2024

  13. [13]

    GR00T N1: An Open Foundation Model for Generalist Humanoid Robots

    Nvidia, J. Bjorck, F. Castaneda, N. Cherniadev, X. Da, R. Ding, LinxiJimFan, Y . Fang, D. Fox, F. Hu, S. Huang, J. Jang, Z. Jiang, J. Kautz, K. Kundalia, L. Lao, Z. Li, Z. Lin, K. Lin, G. Liu, E. Llontop, L. Magne, A. Mandlekar, A. Narayan, S. Nasiriany, S. Reed, Y . L. Tan, G. Wang, Z. Wang, J. Wang, Q. Wang, J. Xiang, Y . Xie, Y . Xu, Z.-T. Xu, S. Ye, Z...

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  14. [14]

    Perez, F

    E. Perez, F. Strub, H. De Vries, V . Dumoulin, and A. Courville. Film: Visual reasoning with a general conditioning layer. InProceedings of the AAAI Conference on Artificial Intelligence, 2018

    work page 2018

  15. [15]

    T. Yu, D. Quillen, Z. He, R. Julian, K. Hausman, C. Finn, and S. Levine. Meta-world: A benchmark and evaluation for multi-task and meta reinforcement learning. InConference on robot learning, pages 1094–1100. PMLR, 2020

    work page 2020

  16. [16]

    James, Z

    S. James, Z. Ma, D. R. Arrojo, and A. J. Davison. Rlbench: The robot learning benchmark & learning environment.IEEE Robotics and Automation Letters, 5(2):3019–3026, 2020

    work page 2020

  17. [17]

    Y . Mu, T. Chen, Z. Chen, S. Peng, Z. Lan, Z. Gao, Z. Liang, Q. Yu, Y . Zou, M. Xu, et al. Robotwin: Dual-arm robot benchmark with generative digital twins. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 27649–27660, 2025

    work page 2025

  18. [18]

    Z. Fu, T. Z. Zhao, and C. Finn. Mobile aloha: Learning bimanual mobile manipulation using low-cost whole-body teleoperation. In8th Annual Conference on Robot Learning, 2024

    work page 2024

  19. [19]

    T. P. Lillicrap. Continuous control with deep reinforcement learning.arXiv preprint arXiv:1509.02971, 2015

    work page internal anchor Pith review Pith/arXiv arXiv 2015

  20. [20]

    Proximal Policy Optimization Algorithms

    J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov. Proximal policy optimization algorithms.arXiv preprint arXiv:1707.06347, 2017

    work page internal anchor Pith review Pith/arXiv arXiv 2017

  21. [21]

    Majumdar, K

    A. Majumdar, K. Yadav, S. Arnaud, J. Ma, C. Chen, S. Silwal, A. Jain, V .-P. Berges, T. Wu, J. Vakil, et al. Where are we in the search for an artificial visual cortex for embodied intelligence? Advances in Neural Information Processing Systems, 36:655–677, 2023

    work page 2023

  22. [22]

    Shang, K

    J. Shang, K. Schmeckpeper, B. B. May, M. V . Minniti, T. Kelestemur, D. Watkins, and L. Herlant. Theia: Distilling diverse vision foundation models for robot learning. In8th Annual Conference on Robot Learning, 2024. URLhttps://openreview.net/forum?id=ylZHvlwUcI

    work page 2024

  23. [23]

    J. Lu, W. Xia, Y . Wu, Z. Lu, and D. Hu. When would vision-proprioception policy fail in robotic manipulation?arXiv preprint, 2025

    work page 2025

  24. [24]

    J. Zhao, W. Lu, D. Zhang, Y . Liu, Y . Liang, T. Zhang, Y . Cao, J. Xie, Y . Hu, S. Wang, et al. Do you need proprioceptive states in visuomotor policies?arXiv preprint arXiv:2509.18644, 2025

    work page arXiv 2025

  25. [25]

    T. Kim, J. Lee, M. Koo, D. Kim, K. Lee, C. Kim, Y . Seo, and J. Shin. Contrastive representation regularization for vision-language-action models.arXiv preprint arXiv:2510.01711, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  26. [26]

    S. Fan, K. Wu, Z. Che, X. Wang, D. Wu, F. Liao, N. Liu, Y . Zhang, Z. Zhao, Z. Xu, et al. Xr-1: Towards versatile vision-language-action models via learning unified vision-motion representations.arXiv preprint arXiv:2511.02776, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  27. [27]

    Suzuki, S

    K. Suzuki, S. Shimizu, and T. Ogata. Proprioception enhances vision language model in generating captions and subtask segmentations for robot task.arXiv preprint arXiv:2512.20876, 2025

    work page arXiv 2025

  28. [28]

    B. Li, S. He, H. Xu, H. Yuan, X. Xu, Y . Zang, L. Hu, J. Yue, Z. Jiang, P. Hu, et al. To- wards proprioception-aware embodied planning for dual-arm humanoid robots.arXiv preprint arXiv:2510.07882, 2025

    work page arXiv 2025

  29. [29]

    D. Niu, Y . Sharma, G. Biamby, J. Quenum, Y . Bai, B. Shi, T. Darrell, and R. Herzig. Llarva: Vision-action instruction tuning enhances robot learning. InConference on Robot Learning, 2024

    work page 2024

  30. [30]

    Y . Li, Y . Deng, J. Zhang, J. Jang, M. Memmel, R. Yu, C. R. Garrett, F. Ramos, D. Fox, A. Li, A. Gupta, and A. Goyal. Hamster: Hierarchical action models for open-world robot manipulation.arXiv preprint arXiv:2502.05485, 2025. 10

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  31. [31]

    Zhang, M

    J. Zhang, M. Memmel, K. Kim, D. Fox, J. Thomason, F. Ramos, E. Bıyık, A. Gupta, and A. Li. Peek: Guiding and minimal image representations for zero-shot generalization of robot manipulation policies. InIEEE International Conference on Robotics and Automation, 2025

    work page 2025

  32. [32]

    Y . Dai, J. Lee, Y . Zhang, Z. Ma, J. Yang, A. Zadeh, C. Li, N. Fazeli, and J. Chai. Aimbot: A simple auxiliary visual cue to enhance spatial awareness of visuomotor policies.arXiv preprint arXiv:2508.08113, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  33. [33]

    Zheng, S

    D. Zheng, S. Huang, Y . Li, and L. Wang. Learning from videos for 3d world: Enhancing mllms with 3d vision geometry priors.arXiv preprint arXiv:2505.24625, 2025

    work page arXiv 2025

  34. [34]

    X. Miao, H. Duan, Q. Qian, J. Wang, Y . Long, L. Shao, D. Zhao, R. Xu, and G. Zhang. Towards scalable spatial intelligence via 2d-to-3d data lifting. InProceedings of the IEEE/CVF International Conference on Computer Vision, 2025

    work page 2025

  35. [35]

    Zhang, W

    G. Zhang, W. Li, Q. Qian, J. Wang, D. Zhao, S. Lu, and R. Xu. On the generalization capacities of MLLMs for spatial intelligence. InThe Fourteenth International Conference on Learning Representations, 2026. URLhttps://openreview.net/forum?id=DE5ZJtR4bg

    work page 2026

  36. [36]

    Shridhar, L

    M. Shridhar, L. Manuelli, and D. Fox. Perceiver-actor: A multi-task transformer for robotic manipulation. InConference on Robot Learning, pages 785–799. PMLR, 2023

    work page 2023

  37. [37]

    Goyal, V

    A. Goyal, V . Blukis, J. Xu, Y . Guo, Y .-W. Chao, and D. Fox. Rvt-2: Learning precise ma- nipulation from few demonstrations. InRSS 2024 Workshop: Data Generation for Robotics, 2024

    work page 2024

  38. [38]

    Zhang, Z

    Q. Zhang, Z. Liu, H. Fan, G. Liu, B. Zeng, and S. Liu. Flowpolicy: Enabling fast and robust 3d flow-based policy via consistency flow matching for robot manipulation. InProceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 14754–14762, 2025

    work page 2025

  39. [39]

    F. Li, W. Song, H. Zhao, J. Wang, P. Ding, D. Wang, L. Zeng, and H. Li. Spatial forcing: Implicit spatial representation alignment for vision-language-action model.arXiv preprint arXiv:2510.12276, 2025

    work page arXiv 2025

  40. [40]

    J. Qian, B. Han, C. Shi, L. Xiao, L. Yang, S. Shi, and L. Jiang. Geopredict: Leveraging predictive kinematics and 3d gaussian geometry for precise vla manipulation.arXiv preprint arXiv:2512.16811, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  41. [41]

    Q. Qian, G. Zhao, G. Zhang, J. Wang, R. Xu, J. Gao, and D. Zhao. Gp3: A 3d geometry-aware policy with multi-view images for robotic manipulation.arXiv preprint arXiv:2509.15733, 2025

    work page arXiv 2025

  42. [42]

    T.-Y . Lin, P. Dollár, R. Girshick, K. He, B. Hariharan, and S. Belongie. Feature pyramid networks for object detection. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 2117–2125, 2017

    work page 2017

  43. [43]

    Y . Jia, J. Liu, S. Chen, C. Gu, Z. Wang, L. Luo, X. Li, P. Wang, Z. Wang, R. Zhang, et al. Lift3d policy: Lifting 2d foundation models for robust 3d robotic manipulation. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 17347–17358, 2025

    work page 2025

  44. [44]

    Y . Fang, K. Ranasinghe, L. Xue, H. Zhou, J. Tan, R. Xu, S. Heinecke, C. Xiong, S. Savarese, D. Szafir, et al. Robotic vla benefits from joint learning with motion image diffusion.arXiv preprint arXiv:2512.18007, 2025

    work page arXiv 2025

  45. [45]

    K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770– 778, 2016. 11

    work page 2016

  46. [46]

    Dosovitskiy, L

    A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. InInternational Conference on Learning Representations, 2021. URLhttps://openreview.net/forum?id=YicbFdNTTy. A Implementa...

    work page 2021