Pith. sign in

REVIEW 2 major objections 5 minor 58 references

Robot-centric pointmaps close the frame gap between camera views and robot actions in vision-language-action models.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-14 05:10 UTC pith:YS7JRFGQ

load-bearing objection Clean geometric fix for VLA viewpoint mismatch: robot-frame EE-centered pointmaps fuse into pretrained 2D VLAs and the gains hold in sim and real, with the main limit being calibration dependence they already state. the 2 major comments →

arxiv 2607.11498 v1 pith:YS7JRFGQ submitted 2026-07-13 cs.RO cs.AI

See like a Robot: Robot-Centric Pointmaps for Vision-Language-Action Models

Byungkun Lee , Dongyoon Hwang , Dongjin Kim , Hojoon Lee , Minho Park , Jaegul Choo This is my paper
classification cs.RO cs.AI
keywords VLArobot manipulationpointmap3D geometrycamera viewpoint variationvision-language-actionend-effector centering
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Vision-language-action models must output motions in the robot’s own 3D frame, yet they usually see the world only through camera images. When training data mixes many camera placements, the policy has to learn a different observation-to-action map for each viewpoint. This paper argues that the better fix is to change the observation itself: build an image whose every pixel holds the 3D coordinates of the corresponding scene point already expressed in the robot frame. That robot-centric pointmap keeps the dense image grid pretrained 2D vision towers expect, so it slots into existing VLAs with a twin encoder and simple token addition. On a multi-task simulation suite and on a real robot, the change raises success over RGB-only policies and over camera-aware and 3D baselines, and the margin grows when the camera is moved to a placement never seen in training.

Core claim

Expressing each observed scene point as a robot-frame, image-form pointmap—especially one re-centered on the current end effector—and fusing it by element-wise addition of a separate visual encoder’s tokens onto RGB tokens improves pretrained vision-language-action models under camera viewpoint variation. Pre-computing the camera-to-robot transform, preserving the H×W grid, and end-effector centering are the design choices that drive the gain over leaving the mapping for the policy to learn or using sparse point clouds.

What carries the argument

Robot-centric pointmaps: dense H×W images whose pixels store 3D scene coordinates in the robot (or end-effector-centered) frame, encoded by a separate visual tower initialized from the RGB encoder and added element-wise to RGB tokens so the pretrained 2D pathway absorbs robot-frame geometry without a point-cloud encoder or extra tokens.

Load-bearing premise

Accurate camera calibration and usable depth must be available at both training and test time so every pixel can be lifted into the robot frame.

What would settle it

Corrupt camera extrinsics or replace depth with noise while keeping the same backbone and training recipe; if the pointmap variant then loses its success-rate edge over RGB and other 3D baselines under viewpoint shift, the central claim fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Multi-institution robot datasets with mixed camera setups become easier for VLAs to use once observations are converted to robot-centric pointmaps.
  • At deployment, a held-out camera placement hurts an RGB-only policy more than a pointmap policy.
  • Conditioning on camera parameters or attaching a separate point-cloud module is less effective than pre-computing robot-frame geometry on the image grid.
  • End-effector centering makes the same grasp motion look similar across distant workspace locations, aiding multi-task learning.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • As foundation robot models absorb more cross-lab data, standardized robot-frame geometry inputs may matter as much as standardized action spaces.
  • The same construction could aid cross-embodiment transfer if each robot’s base frame is used consistently when building the pointmap.
  • Where hardware calibration is unavailable, monocular geometry estimators could be substituted, but that would reintroduce the very camera-to-robot inference burden the method removes.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper addresses the observation-to-action frame mismatch in vision-language-action (VLA) models: actions are defined in the robot base frame while observations are typically camera-frame RGB (or depth). Under multi-view training data this mismatch forces the policy to learn a viewpoint-dependent mapping. The authors propose robot-centric pointmaps—dense H imes W images whose pixels store 3D coordinates of the corresponding scene points after the camera-to-robot transform (Eqs. 1–2), optionally re-centered at the current end-effector (Eq. 3). A separate visual tower, initialized from the RGB encoder, encodes the pointmap; its tokens are added element-wise to the RGB tokens (Eq. 4), preserving the pretrained 2D pathway with no extra tokens or point-cloud encoder. Controlled ablations on a fixed π-style backbone isolate pre-computation of robot-frame geometry versus Plücker+depth cues (Table 1), image-form pointmaps with add fusion versus point clouds (Table 2), base versus EE origin (Table 3), and robustness to increasing training-time camera jitter (Fig. 6). Full fine-tunes of π0.5 and SmolVLA on RoboCasa improve 24-task averages by +7.6 and +4.2 points over RGB-only and outperform reimplemented camera-aware and 3D-augmented baselines (Table 4). Real-robot Franka experiments show the margin over RGB widening from +5.0 at a seen external-camera placement to +11.7 at a held-out placement (Table 5).

Significance. If the results hold, the work supplies a simple, architecture-light way to inject metric robot-frame geometry into pretrained 2D VLAs precisely when large multi-institution datasets introduce viewpoint diversity. The controlled RQ studies cleanly separate pre-computation of the camera-to-robot transform, preservation of the image grid with element-wise fusion, and end-effector centering; the main tables show consistent gains on two backbones of different scale and a real Franka setup whose advantage grows at an unseen camera. The method reuses existing visual towers, requires no generative view synthesis or dedicated 3D expert at inference, and is therefore immediately usable by practitioners who already have calibrated RGB-D. The explicit limitation that calibration and depth must be available at train and test time is stated in §6 and does not undermine the internal claims. The contribution is therefore a practical, well-isolated design principle rather than a new foundation model.

major comments (2)
  1. §5.1 / Appendix B.4: The camera-aware and 3D-augmented baselines (KYC, OC-VLA, GeoVLA, PointVLA) are reimplemented on π0.5 because official code is unavailable. While the authors describe the reimplementations carefully, the paper should report a short validation that the reimplemented versions recover the qualitative behavior claimed in the original works (or at least match published numbers on a shared subset) so that the ranking in Table 4 can be trusted as a fair comparison rather than an artifact of reimplementation choices.
  2. §4.2 / Table 2: The point-cloud baselines use a fixed sampling budget (1024 points for the MLP, and an unspecified but fixed budget for PTv3). The authors note in §6 that a larger budget could narrow the gap. Because the central claim is that image-form pointmaps are preferable to point clouds for pretrained VLAs, a sensitivity sweep over point count (or a denser voxel/grid baseline) is needed to confirm that the 34.7 vs 32.8 gap is not an artifact of under-sampling the point cloud.
minor comments (5)
  1. Fig. 1 caption and surrounding text: the three-column comparison (Depth / Pointcloud / Pointmap) is clear, but the figure itself would benefit from an explicit legend indicating which cells are robot-centric versus camera-centric.
  2. Eq. (3) and §4.3: the end-effector-centered pointmap is well motivated, yet the text never states whether the orientation of the end-effector is also used or only its translation t_EE; a one-sentence clarification would prevent mis-implementation.
  3. Table 5: success rates are reported from 15 rollouts per task per camera condition. Adding binomial standard errors or a simple significance test would help readers judge the +11.7 unseen margin.
  4. Appendix B.3: the real-robot pointmap is built from RealSense stereo depth and a one-time hand-eye calibration; a short quantitative note on typical depth noise or calibration residual would make the sim-to-real gap more transparent.
  5. Related Work §2.2: recent concurrent pointmap diffusion-policy papers [45,46] are cited; a sentence distinguishing the present contribution (pretrained VLA + robot-frame + EE centering) from those from-scratch policies would sharpen novelty.

Circularity Check

0 steps flagged

No circularity: geometric construction plus external task-success evaluation; nothing reduces to a fitted free parameter or self-citation chain.

full rationale

This is an empirical robotics method paper. The load-bearing chain is: (i) define robot-centric (optionally EE-centered) pointmaps by the fixed camera-to-robot lifts in Eqs. 1–3; (ii) fuse them into pretrained VLAs by element-wise addition of a separate image-encoder stream; (iii) measure success rates on RoboCasa atomic tasks and real Franka rollouts under seen/unseen cameras. Success is an external binary/task criterion, not a quantity recovered from a fit. Controlled ablations (Tables 1–3, Fig. 6) change only how 3D is supplied while holding backbone and recipe fixed; main results (Tables 4–5) compare against reimplemented baselines on the same data. Pointmaps are a standard 3D-vision representation cited from external work; the paper does not invoke a self-authored uniqueness theorem, does not fit a free parameter and rename it a prediction, and does not define the reported metric in terms of the input construction. Calibration dependence (§6) is a scope limitation, not circularity. Score 0 is therefore appropriate.

Axiom & Free-Parameter Ledger

2 free parameters · 4 axioms · 1 invented entities

The claim rests on standard robotics geometry (known K, R, t, depth) and the domain premise that actions live in the robot/end-effector frame. No new physical entities are postulated. Training hyperparameters are ordinary optimization choices, not free parameters that define the reported success rates.

free parameters (2)
  • Training schedule (steps, LR, batch size) = π0.5: 20k steps; SmolVLA: 60k; controlled: 30k
    20k/60k steps, 1e-4 AdamW, batch 64, etc., chosen by authors; affect absolute success rates but are shared across compared variants in the controlled studies.
  • Point-cloud sampling budget in ablations = 1024 / 4096 points
    1024 points (MLP) / fixed budget for PTv3 and DP3 (4096); acknowledged as a single-budget comparison that could narrow the image-form gap.
axioms (4)
  • domain assumption Robot actions are defined in the robot base / end-effector task space, so robot-frame geometry is the natural action coordinate system.
    Stated in §1 and used throughout; standard for the VLA setups considered.
  • domain assumption Per-pixel depth and camera extrinsics/intrinsics are available so Eqs. 1–3 can be evaluated.
    Required for pointmap construction; listed as a limitation in §6.
  • ad hoc to paper A vision encoder of the same architecture as the RGB tower can usefully encode XYZ images when initialized from RGB weights.
    Design choice in §3; supported empirically but not derived.
  • ad hoc to paper Element-wise addition of pointmap tokens to RGB tokens preserves spatial correspondence better than concatenation for this task.
    Ablated in §4.2 (Table 2); empirical, not proven.
invented entities (1)
  • Robot-centric (end-effector-centered) pointmap as VLA visual input independent evidence
    purpose: Provide metric robot-frame 3D geometry while keeping the dense HxW grid expected by pretrained 2D VLAs.
    Representation is assembled from known depth and calibration; novelty is the frame choice and integration pattern, not a new physical object. Independent evidence is the reported task-success improvements.

pith-pipeline@v1.1.0-grok45 · 23219 in / 3040 out tokens · 32045 ms · 2026-07-14T05:10:59.793310+00:00 · methodology

0 comments
read the original abstract

Vision-language-action (VLA) models predict robot actions from visual observations and language instructions. These actions are defined in the robot's own 3D coordinate frame, yet most VLAs observe the scene in the camera frame, creating a frame mismatch between where the scene is observed and where actions are defined. The mismatch is benign under a fixed viewpoint, where the policy can memorize a single observation-to-action mapping, but grows harder as large-scale datasets aggregate demonstrations across diverse camera setups and the policy must generalize this mapping across viewpoints. We address this mismatch with robot-centric pointmaps, images whose pixels store the 3D coordinates of scene points in the robot frame. Pointmaps provide robot-frame 3D geometry while preserving the dense H x W grid expected by pretrained 2D VLAs, so they integrate into existing VLAs with minimal architectural change. On RoboCasa, pointmaps improve both pi0.5 and SmolVLA and outperform representative camera-viewpoint and 3D-aware baselines. In real-robot experiments, their advantage over an RGB-only policy widens when the camera is moved to a placement unseen during training.

Figures

Figures reproduced from arXiv: 2607.11498 by Byungkun Lee, Dongjin Kim, Dongyoon Hwang, Hojoon Lee, Jaegul Choo, Minho Park.

Figure 1
Figure 1. Figure 1: Robot-centric pointmaps provide 3D geometry aligned with robot-frame actions. Large-scale training data is collected from diverse camera viewpoints around the robot. Robot￾centric pointmaps preserve the dense H ×W grid expected by pretrained 2D VLAs while providing robot-centric 3D geometry. the robot frame. A point cloud can provide robot-centric 3D geometry, but it discards the regular H × W image grid u… view at source ↗
Figure 2
Figure 2. Figure 2: Observations from different cam￾era frames are expressed in the coordinate frame where robot actions are defined. In this paper, we implement this idea with robot￾centric pointmaps as inputs to pretrained VLAs. A robot-centric pointmap is an image whose pixels store the 3D coordinates of the corresponding scene points in the robot frame [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: RGB-D observations are lifted into camera-frame pointmaps and transformed into robot-frame pointmaps. The robot-frame pointmap P R c has the same H × W grid as the RGB image, with one 3D coordinate per pixel. The same physical scene point receives the same robot-frame coordinate regardless of the camera viewpoint, although the pixel (u, v) where it appears may change. We additionally re-center the pointmap… view at source ↗
Figure 4
Figure 4. Figure 4: Pointmap-RGB fusion. Addition preserves spatial correspondence, while con￾catenation treats pointmap tokens as a sepa￾rate sequence [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: End-effector centering concentrates interaction targets near a common origin. (a) Target coordinates are broadly distributed in the robot-base frame but cluster near the origin in the end-effector-centered frame. (b) Two example tasks show that targets with different robot-base coordinates lie near the end-effector-centered origin at grasp. ing making the input less dependent on the target’s absolute works… view at source ↗
Figure 6
Figure 6. Figure 6: Effect of training-time cam￾era viewpoint variation. RGB performance drops as viewpoint variation increases, while RGB + Pointmap remains stable. The previous sections establish our pointmap de￾sign. We now test whether this design remains ro￾bust as training-time camera viewpoint variation in￾creases. To isolate this factor, we vary only the level of camera randomization in RoboCasa while hold￾ing all oth… view at source ↗
Figure 7
Figure 7. Figure 7: Real robot setup. (a) A Franka Research 3 with a fixed wrist camera and one external camera, placed at three training positions (red) and one held-out test position (blue). (b) Tasks: pick￾and-place, stack blocks, open/close drawer. while we physically reposition the D435i to three placements, giving three camera configurations (Fig. 7a). Under each configuration, we collect 15 demonstrations per task on f… view at source ↗
Figure 8
Figure 8. Figure 8: visualizes the camera-pose distributions as frustums in the robot-base frame [PITH_FULL_IMAGE:figures/full_fig_p015_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Qualitative examples across variation levels. For six tasks, the same frame is rendered under each training level for the left and right third-person cameras. Under No variation, the view￾point is identical across demonstrations; under Low and High, growing per-demonstration jitter shifts and rotates the scene within the image. The robot-centric pointmap reduces this camera-induced variation by expressing … view at source ↗
Figure 10
Figure 10. Figure 10: Model architecture. Each RGB-D camera observation is converted into an end-effector￾centered robot-centric pointmap. RGB and pointmap inputs are encoded by separate SigLIP vision towers [56], and pointmap tokens are added element-wise to the corresponding RGB tokens before entering the pretrained VLA. The VLA predicts an action chunk from the fused visual tokens, lan￾guage instruction, and proprioception … view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

58 extracted references · 13 linked inside Pith

  1. [1]

    Zitkovich et al

    B. Zitkovich et al. Rt-2: Vision-language-action models transfer web knowledge to robotic control. InCoRL, 2023. 1

    2023

  2. [2]

    M. J. Kim et al. Openvla: An open-source vision-language-action model. InCoRL, 2024

    2024

  3. [3]

    Black et al.π 0 : A vision-language-action flow model for general robot control.arXiv preprint arXiv:2410.24164, 2024

    K. Black et al.π 0 : A vision-language-action flow model for general robot control.arXiv preprint arXiv:2410.24164, 2024. 4

    Pith/arXiv arXiv 2024

  4. [4]

    Black, N

    K. Black, N. Brown, et al. pi0.5: a vision-language-action model with open-world generaliza- tion. InCoRL, 2025. 8, 9, 15, 20

    2025

  5. [5]

    Bjorck et al

    J. Bjorck et al. GR00T N1: An open foundation model for generalist humanoid robots.arXiv preprint arXiv:2503.14734, 2025. 1

    Pith/arXiv arXiv 2025

  6. [6]

    O’Neill et al

    A. O’Neill et al. Open x-embodiment: Robotic learning datasets and rt-x models: Open x- embodiment collaboration. InICRA, 2024. 1, 3

    2024

  7. [7]

    Bu et al

    Q. Bu et al. Agibot world colosseo: A large-scale manipulation platform for scalable and in- telligent embodied systems. In2025 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2025

    2025

  8. [8]

    Khazatsky et al

    A. Khazatsky et al. DROID: A large-scale in-the-wild robot manipulation dataset.arXiv preprint arXiv:2403.12945, 2024. 1, 3

    Pith/arXiv arXiv 2024

  9. [9]

    Nasiriany, A

    S. Nasiriany, A. Maddukuri, L. Zhang, A. Parikh, A. Lo, A. Joshi, A. Mandlekar, and Y . Zhu. Robocasa: Large-scale simulation of everyday tasks for generalist robots. InRobotics: Science and Systems (RSS), 2024. 2, 4, 16

    2024

  10. [10]

    S. Fei, S. Wang, J. Shi, Z. Dai, J. Cai, P. Qian, L. Ji, X. He, S. Zhang, Z. Fei, J. Fu, J. Gong, and X. Qiu. Libero-plus: In-depth robustness analysis of vision-language-action models.arXiv preprint arXiv:2510.13626, 2025. 2

    Pith/arXiv arXiv 2025

  11. [11]

    Z. Wang, Z. Zhou, J. Song, Y . Huang, Z. Shu, and L. Ma. Vlatest: Testing and evaluating vision-language-action models for robotic manipulation.Proc. ACM Softw. Eng., 2(FSE), June

  12. [12]

    URLhttps://doi.org/10.1145/3729343

    doi:10.1145/3729343. URLhttps://doi.org/10.1145/3729343. 2

    doi:10.1145/3729343

  13. [13]

    Walke, K

    H. Walke, K. Black, A. Lee, M. J. Kim, M. Du, C. Zheng, T. Zhao, P. Hansen-Estruch, Q. Vuong, A. He, V . Myers, K. Fang, C. Finn, and S. Levine. Bridgedata v2: A dataset for robot learning at scale. InConference on Robot Learning (CoRL), 2023. 3

    2023

  14. [14]

    A. Xie, L. Lee, T. Xiao, and C. Finn. Decomposing the generalization gap in imitation learning for visual robotic manipulation. InIEEE International Conference on Robotics and Automation (ICRA), 2024. 3

    2024

  15. [15]

    Morgan, P

    J. Morgan, P. Vijay, H. Oh, J. Song, A. Arora, A. Du, G. Sukhatme, J. Thomason, and I. Singh. Colosseum v2: Benchmarking generalization for vision language action models.arXiv preprint arXiv:2605.27759, 2026. 3

    Pith/arXiv arXiv 2026

  16. [16]

    H. Lin, H. Yu, J. Huang, H. Zhang, Y . Ling, P. Tan, X. Xue, and Y . Fu. Universal pose pre- training for generalizable vision-language-action policies. InRobotics: Science and Systems (RSS), 2026. 3

    2026

  17. [17]

    Jiang, J

    T. Jiang, J. Ji, X. Tan, J. Fang, A. Bhattad, V . Guizilini, and M. R. Walter. Do you know where your camera is? View-invariant policy learning with camera conditioning. InIEEE International Conference on Robotics and Automation (ICRA), 2026. 8, 18, 20

    2026

  18. [18]

    Zhang, Y

    J. Zhang, Y . Chen, Y . Xu, Z. Huang, Y . Zhou, Y .-J. Yuan, X. Cai, G. Huang, X. Quan, H. Xu, and L. Zhang. 4d-VLA: Spatiotemporal vision-language-action pretraining with cross-scene calibration. InProc. the Advances in Neural Information Processing Systems (NeurIPS), 2025. 11

    2025

  19. [19]

    D. Qu, H. Song, Q. Chen, Y . Yao, X. Ye, Y . Ding, Z. Wang, J. Gu, B. Zhao, D. Wang, et al. Spatialvla: Exploring spatial representations for visual-language-action model.arXiv preprint arXiv:2501.15830, 2025. 3

    Pith/arXiv arXiv 2025

  20. [20]

    Zhang, H

    T. Zhang, H. Duan, H. Hao, Y . Qiao, J. Dai, and Z. Hou. Grounding actions in camera space: Observation-centric vision-language-action policy. InProc. the AAAI Conference on Artificial Intelligence (AAAI), 2026. 3, 8, 18, 20

    2026

  21. [21]

    S. Tian, B. Wulfe, K. Sargent, K. Liu, S. Zakharov, V . C. Guizilini, and J. Wu. View-invariant policy learning via zero-shot novel view synthesis. InConference on Robot Learning (CoRL),

  22. [22]

    L. Y . Chen, C. Xu, K. Dharmarajan, R. Cheng, K. Keutzer, M. Tomizuka, Q. Vuong, and K. Goldberg. Rovi-aug: Robot and viewpoint augmentation for cross-embodiment robot learn- ing. InConference on Robot Learning (CoRL), 2024. 3

    2024

  23. [23]

    L. Y . Chen, K. Hari, K. Dharmarajan, C. Xu, Q. Vuong, and K. Goldberg. Mirage: Cross- embodiment zero-shot policy transfer with cross-painting. InRobotics: Science and Systems (RSS), 2024. 3

    2024

  24. [24]

    H. Heo, S. Woo, S. M. Kim, J. Kim, J. Lee, Y . Lee, and Y . M. Kim. Anycamvla: Zero-shot camera adaptation for viewpoint robust vision-language-action models, 2026. 3

    2026

  25. [25]

    F. Liu, F. Yan, L. Zheng, Y . Huang, C. Feng, and L. Ma. Robouniview: Visual-language model with unified view representation for robotic manipulation.arXiv preprint arXiv:2406.18977,

    Pith/arXiv arXiv

  26. [26]

    Wilcox, M

    A. Wilcox, M. Ghanem, M. Moghani, P. Barroso, B. Joffe, and A. Garg. Adapt3r: Adaptive 3d scene representation for domain transfer in imitation learning. InConference on Robot Learning (CoRL), 2025. 3

    2025

  27. [27]

    L. Sun, B. Xie, Y . Liu, H. Shi, T. Wang, and J. Cao. Geovla: Empowering 3d representations in vision-language-action models. InarXiv preprint arXiv:2508.09071, 2025. 3, 8, 17, 20

    Pith/arXiv arXiv 2025

  28. [28]

    C. Li, J. Wen, Y . Peng, Y . Peng, F. Feng, and Y . Zhu. Pointvla: Injecting the 3d world into vision-language-action models.IEEE Robotics and Automation Letters, 11(3):2506–2513,

  29. [29]

    Bhat, Y .-H

    V . Bhat, Y .-H. Lan, P. Krishnamurthy, R. Karri, and F. Khorrami. 3d cavla: Leveraging depth and 3d context to generalize vision language action models for unseen tasks. InarXiv preprint arXiv:2505.05800, 2025

    arXiv 2025

  30. [30]

    T. Yuan, Y . Liu, C. Lu, Z. Chen, T. Jiang, and H. Zhao. Depthvla: Enhancing vision-language- action models with depth-aware spatial reasoning. InarXiv preprint arXiv:2510.13375, 2025

    arXiv 2025

  31. [31]

    X. Fan, S. Deng, X. Wu, Y . Lu, Z. Li, M. Yan, Y . Zhang, Z. Zhang, H. Wang, and H. Zhao. Any3d-vla: Enhancing vla robustness via diverse point clouds.arXiv preprint arXiv:2602.00807, 2026

    Pith/arXiv arXiv 2026

  32. [32]

    S. Chen, P. Pacaud, and C. Schmid. Pointact: Vision-language-action models with multi-scale point-action interaction, 2026. URLhttps://arxiv.org/abs/2605.21414. 3

    Pith/arXiv arXiv 2026

  33. [33]

    P. Li, Y . Chen, H. Wu, X. Ma, X. Wu, Y . Huang, L. Wang, T. Kong, and T. Tan. Bridgevla: Input-output alignment for efficient 3d manipulation learning with vision-language models. In Proc. the Advances in Neural Information Processing Systems (NeurIPS), 2025. 3

    2025

  34. [34]

    Singh, A

    I. Singh, A. Goyal, S. Birchfield, D. Fox, A. Garg, and V . Blukis. Og-vla: Orthographic im- age generation for 3d-aware vision-language action model.arXiv preprint arXiv:2506.01196,

    arXiv

  35. [35]

    H. Zhen, X. Qiu, P. Chen, J. Yang, X. Yan, Y . Du, Y . Hong, and C. Gan. 3d-vla: 3d vision- language-action generative world model.Proc. the International Conference on Machine Learning (ICML), 2024. 3

    2024

  36. [36]

    T. Lin, G. Li, Y . Zhong, Y . Zou, and B. Zhao. Evo-0: Vision-language-action model with implicit spatial understanding.arXiv preprint arXiv:2507.00416, 2025. 3

    arXiv 2025

  37. [37]

    F. Li, W. Song, H. Zhao, J. Wang, P. Ding, D. Wang, L. Zeng, and H. Li. Spatial forc- ing: Implicit spatial representation alignment for vision-language-action model. InProc. the International Conference on Learning Representations (ICLR), 2026. URLhttps: //openreview.net/forum?id=euMVC1DO4k. 3

    2026

  38. [38]

    S. Wang, V . Leroy, Y . Cabon, B. Chidlovskii, and J. Revaud. Dust3r: Geometric 3d vision made easy. InProc. of the IEEE conference on computer vision and pattern recognition (CVPR), pages 20697–20709, June 2024. 3

    2024

  39. [39]

    Leroy, Y

    V . Leroy, Y . Cabon, and J. Revaud. Grounding image matching in 3d with mast3r. InProc. of the European Conference on Computer Vision (ECCV), 2024

    2024

  40. [40]

    J. Wang, M. Chen, N. Karaev, A. Vedaldi, C. Rupprecht, and D. Novotny. Vggt: Visual geom- etry grounded transformer. InProc. of the IEEE conference on computer vision and pattern recognition (CVPR), pages 5294–5306, June 2025

    2025

  41. [41]

    R. Wang, S. Xu, C. Dai, J. Xiang, Y . Deng, X. Tong, and J. Yang. Moge: Unlocking accurate monocular geometry estimation for open-domain images with optimal training supervision. InProc. of the IEEE conference on computer vision and pattern recognition (CVPR), pages 5261–5271, 2025

    2025

  42. [42]

    R. Wang, S. Xu, Y . Dong, Y . Deng, J. Xiang, Z. Lv, G. Sun, X. Tong, and J. Yang. Moge-2: Accurate monocular geometry with metric scale and sharp details. InProc. the Advances in Neural Information Processing Systems (NeurIPS), 2025

    2025

  43. [43]

    Wang*, Y

    Q. Wang*, Y . Zhang*, A. Holynski, A. A. Efros, and A. Kanazawa. Continuous 3d perception model with persistent state. InProc. of the IEEE conference on computer vision and pattern recognition (CVPR), 2025

    2025

  44. [44]

    J. Yang, A. Sax, K. J. Liang, M. Henaff, H. Tang, A. Cao, J. Chai, F. Meier, and M. Feiszli. Fast3r: Towards 3d reconstruction of 1000+ images in one forward pass. InProc. of the IEEE conference on computer vision and pattern recognition (CVPR), June 2025

    2025

  45. [45]

    Y . Wang, J. Zhou, H. Zhu, W. Chang, Y . Zhou, Z. Li, J. Chen, J. Pang, C. Shen, and T. He. $\pi^3$: Permutation-equivariant visual geometry learning. InProc. the International Confer- ence on Learning Representations (ICLR), 2026. 3

    2026

  46. [46]

    X. Jia, Q. Wang, A. Wang, H. A. Wang, B. Gyenes, E. Gospodinov, X. Jiang, G. Li, H. Zhou, W. Liao, X. Huang, M. Beck, M. Reuss, R. Lioutikov, and G. Neumann. Pointmappolicy: Structured point cloud processing for multi-modal imitation learning. InProc. the Advances in Neural Information Processing Systems (NeurIPS), 2025. 3

    2025

  47. [47]

    X. Yang, R. Wu, J. Liu, and X. Li. Remap-dp: Reprojected multi-view aligned pointmaps for diffusion policy, 2026. 3

    2026

  48. [48]

    Beyer et al

    L. Beyer et al. Paligemma: A versatile 3b vlm for transfer.arXiv preprint arXiv:2407.07726,

    Pith/arXiv arXiv

  49. [49]

    X. Wu, L. Jiang, P.-S. Wang, Z. Liu, X. Liu, Y . Qiao, W. Ouyang, T. He, and H. Zhao. Point transformer v3: Simpler, faster, stronger. InProc. of the IEEE conference on computer vision and pattern recognition (CVPR), 2024. 6, 19 13

    2024

  50. [50]

    M. Liu, X. Li, Z. Ling, Y . Li, and H. Su. Frame mining: a free lunch for learning robotic manipulation from 3d point clouds. InConference on Robot Learning (CoRL), 2022. 6

    2022

  51. [51]

    L. Ke, J. Wang, T. Bhattacharjee, B. Boots, and S. Srinivasa. Grasping with chopsticks: Com- bating covariate shift in model-free imitation learning for fine manipulation. InIEEE Interna- tional Conference on Robotics and Automation (ICRA), page 6185–6191. IEEE Press, 2021. doi:10.1109/ICRA48506.2021.9561662. URLhttps://doi.org/10.1109/ICRA48506. 2021.9561662

    doi:10.1109/icra48506.2021.9561662 2021

  52. [52]

    K. Rana, J. Abou-Chakra, S. Garg, R. Lee, I. Reid, and N. Suenderhauf. Learning from 10 demos: Generalisable and sample-efficient policy learning with oriented affordance frames. In Conference on Robot Learning (CoRL), 2025. URLhttps://openreview.net/forum?id= 1K3kjo91Q1

    2025

  53. [53]

    Zhang, Z

    Z. Zhang, Z. Xu, J. N. Lakamsani, and Y . She. Canonical policy: Learning canonical 3d representation for equivariant policy.arXiv preprint arXiv:2505.18474, 2025. 6

    arXiv 2025

  54. [54]

    Shukor, D

    M. Shukor, D. Aubakirova, F. Capuano, P. Kooijmans, S. Palma, A. Zouitine, M. Aractingi, C. Pascal, M. Russi, A. Marafioti, et al. Smolvla: A vision-language-action model for afford- able and efficient robotics.arXiv preprint arXiv:2506.01844, 2025. 8, 15, 20

    Pith/arXiv arXiv 2025

  55. [55]

    R. Yang, G. Chen, C. Wen, and Y . Gao. Fp3: A 3d foundation policy for robotic manipulation,

  56. [56]

    8, 18, 20

    URLhttps://arxiv.org/abs/2503.08950. 8, 18, 20

    Pith/arXiv arXiv

  57. [57]

    Y . Ze, G. Zhang, K. Zhang, C. Hu, M. Wang, and H. Xu. 3d diffusion policy: Generalizable visuomotor policy learning via simple 3d representations. InRobotics: Science and Systems (RSS), 2024. 9, 19

    2024

  58. [58]

    X. Zhai, B. Mustafa, A. Kolesnikov, and L. Beyer. Sigmoid loss for language image pre- training. InProceedings of the IEEE/CVF international conference on computer vision, pages 11975–11986, 2023. 17 14 Supplementary Material A Details of Training-Time Camera Viewpoint Variation Section 4.4 of the main paper varies how much the third-person camera poses c...

    2023