Pith. sign in

REVIEW 3 major objections 6 minor 45 references

RayViT: Ray-Conditioned Visual Representations for Viewpoint-Robust Imitation Learning

T0 review · 3 major / 6 minor · reviewed 2026-08-03 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that injecting per-pixel camera rays into a pretrained vision transformer keeps imitation-learning policies nearly intact when test cameras shift, while RGB-only policies collapse.

desk verdict Plausible architecture with consistent robustness numbers, but the central claim that ray geometry is the operative mechanism is not yet established — the missing mismatched-ray-map ablation is the key hole. read the letter →

arxiv 2607.29622 v1 pith:3GSJK6GS submitted 2026-07-31 cs.RO cs.CV

Qian Wang , Longrui Chen , Peiran Sun , Aleksandar Taranovic , Niklas Freymuth , Ge Li , Weiran Liao , C. F. Maximilian Nagy
show 3 more authors
Yucheng Tan Tao Chen Gerhard Neumann
This is my paper · ORCID
classification cs.ROcs.CV
keywords imitationlearningcameraperturbationrobustnessPlückerraymapvisiontransformerview-conditionedrepresentationrobotmanipulationgatedcross-attentioncross-viewconsistency
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that robot policies trained by imitation from RGB images can be made robust to camera movements without extra sensors or multi-view data augmentation. The proposed architecture, RayViT, encodes camera geometry as a Plücker ray map—a per-pixel six-vector of ray direction and moment—and feeds it into a fully fine-tuned pretrained ViT through a gated cross-attention class token and patch-aligned ray positional embeddings. An auxiliary cosine loss aligns class tokens across views, using the gripper view as anchor. On a 16-task simulated kitchen benchmark, the method reduces the average success-rate drop under camera perturbation from roughly 18.5 points (RGB baseline) to about 2.5 points, and on four real-robot tasks it lifts average completed stages in the perturbed setting from 0.91 to 2.69.

What carries the argument

Plücker ray map: a per-pixel 6-dimensional encoding (viewing direction plus moment, derived from camera intrinsics and extrinsics) that is spatially aligned with the RGB image. RayViT injects it via (1) a gated cross-attention module that reads the ray grid to produce a geometry-aware class token replacing the ViT's [CLS], and (2) patch-aligned ray features added to patch tokens as geometric positional embeddings on top of the existing rotary embeddings. An auxiliary cross-view cosine loss, applied at an intermediate layer with the gripper camera as anchor, regularizes the class tokens.

What would settle it

Run the variant-setting evaluation with random Gaussian noise added to the camera extrinsics (e.g., rotations of a few degrees and translations of a few centimeters) before computing the ray map; if the success rate drops to RGB-baseline levels, the claim depends on perfect pose knowledge.

Watch

Extended reading notes

Core claim

RayViT's central claim is that camera geometry, injected as Plücker ray features alongside image patches and summarized into a ray-conditioned class token, gives a pretrained ViT the ability to explain appearance changes due to viewpoint shifts, so the policy encodes the task rather than the camera pose. Concretely, the method computes a ray map from camera intrinsics and extrinsics, average-pools it to patch resolution, adds it as positional embeddings, and replaces the ViT's class token with a token produced by two gated cross-attention blocks over the ray grid. A cosine-similarity loss on intermediate class tokens, anchored on the gripper view, further aligns representations across camera

Load-bearing premise

The evaluation assumes that at test time, for every observation including perturbed cameras, the camera intrinsics and extrinsics are known exactly; if they are noisy or estimated, the injected geometry is wrong and the claimed robustness is not covered by the results.

Editorial extensions

If this is right

  • If the results hold, imitation-learning policies can become viewpoint-robust without collecting multi-view demonstrations or adding depth sensors; the only extra requirement is calibrated camera poses at train and test time.
  • The small remaining drop (about 2.5 points) suggests the learned representation is close to viewpoint-invariant on the tested tasks, so further gains may come from residual appearance shifts rather than geometry.
  • The cosine-similarity regularizer is only effective when tokens already carry geometry, implying that view-consistency objectives should be applied to geometry-aware representations, not raw RGB features.
  • RayViT's gains persist when the backbone is trained from scratch and when a different pretrained ViT is used, indicating the mechanism transfers beyond one pretrained encoder.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct test: add pose noise, calibration error, or false intrinsics to the ray map at evaluation; if robustness collapses, the method's gains in practice depend on exact pose knowledge, which the paper does not quantify.
  • The gripper-view anchor for the cosine loss is a task-relevant choice; in tasks with no gripper camera or with heavy occlusion of the hand, the loss may need a learned or multiple anchors, a regime the paper does not explore.
  • Because ray maps are cheap and deterministic, RayViT could serve as a drop-in geometric conditioning layer for other pretrained vision encoders or vision-language-action models, not just the policy head tested here.
  • The method might combine fruitfully with multi-view augmentation: geometric conditioning could reduce the number of synthetic cameras needed, since the encoder no longer must infer viewpoint purely from appearance.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. RayViT proposes to make imitation-learning policies robust to camera perturbations by injecting Plücker ray maps into a pretrained ViT encoder. The ray map, computed from camera intrinsics/extrinsics via Eq. (1)–(2), is used in two ways: patch-aligned ray features added as positional embeddings, and a ray-conditioned class token produced by gated cross-attention. A cosine-similarity loss aligns class tokens of different viewpoints, using the gripper camera as anchor. The paper evaluates on 16 RoboCasa tasks and 4 real-robot tasks. Reported results show that RayViT degrades only ~2.5 percentage points on RoboCasa under camera perturbations versus ~18.5 points for a RGB-only baseline, and improves real-world completed stages from 0.91 to 2.69, while remaining competitive in default settings.

Significance. If the central claim holds, RayViT is a useful contribution: it adds explicit camera geometry to a pretrained ViT without extra sensors, point-cloud reconstruction, or multi-view data augmentation, and it reports consistent gains across simulation and real robots. The paper's projective-geometry equations are standard and correct, and the ablation studies (cosine loss, layer choice, backbone transfer, from-scratch training) are a strength. The main risk is that the experiments do not currently demonstrate that the ray map is the operative mechanism behind the robustness gains, rather than RGB appearance or camera-identity shortcuts. The exact-pose assumption and the choice of anchor view also need more scrutiny. These are fixable with additional experiments, so the contribution is potentially significant but not yet fully supported.

major comments (3)
  1. [§4 and §5 (variant evaluation protocol)] The central robustness claim is not backed by evidence that the ray map is the operative mechanism. In RoboCasa, the two static cameras are fixed during training; only the gripper camera moves. In the real-robot study, the perturbed right camera is introduced only at test time. Thus the model never observes, during training, how a third-person view's ray map changes with camera pose. The network could instead rely on RGB appearance and a camera-identity shortcut, with the ray map acting as a mostly uninformative side input. The paper provides no ablation that corrupts or mismatches the ray map at test time. I ask for a decisive experiment: feed perturbed images with the default (unperturbed) ray map, or with a ray map from a different camera pose. If mismatched rays do not sharply degrade performance, the geometric mechanism is not established; if they do, the model is geometry-sensitive
  2. [§3 Eq. (1)–(2) and §4 experimental setup] The paper implicitly assumes that at test time, for every observation including perturbed cameras, the exact camera intrinsics and extrinsics (K, R, t) are known, because Eq. (1)–(2) compute the ray map from them. This assumption is never stated as such. The perturbation ranges are described only as 'randomly rotated within bounded azimuth and elevation ranges ... shifted in distance' with no numerical values, and there is no experiment measuring sensitivity to calibration noise or estimated poses. In deployment, exact poses are often unavailable or noisy. Please state the perturbation ranges, add a test-time ray-map noise/corruption sweep (e.g., adding error to K, R, t or using an estimated pose), and report how the robustness benefit degrades with pose error. Without this, the claimed robustness is only demonstrated in an idealized setting where geometry is perfect.
  3. [§6 Limitations and §3 Cross-view Alignment Objective] The cross-view consistency objective relies on a task-relevant anchor view, always the gripper camera in these experiments (Eq. 3). The paper acknowledges this in §6, but it interacts with the robustness claim in a nontrivial way. In the real-robot variant, only the right camera is replaced, while the gripper anchor remains fixed; in RoboCasa, the static cameras are fixed during training. The reported gains could therefore partly reflect the policy learning to rely on the anchor view rather than achieving genuinely camera-invariant representations of the third-person views. Please ablate the anchor choice (e.g., use a third-person view as anchor, or average pair-wise losses) and report per-view performance, at least for the real-robot variant. This would clarify whether the benefit is a property of the representation or of a privileged camera.
minor comments (6)
  1. [§4 experimental setup] The perturbation ranges for the variant simulation environment are not quantified. Please give concrete numbers (e.g., degrees of azimuth/elevation rotation, distance shift) so that the difficulty of the evaluation is reproducible.
  2. [Table 1] Header uses 'RayVit-cls' while the text uses 'RayViT-cls'; please unify. Also, the bold/highlight convention in the table caption is not clearly explained in the table itself.
  3. [Appendix C.1] The sentence 'Table 4 shows an overview of our real-world tasks' appears to be a cross-reference error; the overview is likely Figure 4 or Table 7, not Table 4.
  4. [Abstract and §4.1] The abstract says the method 'consistently improve[s] the performance' but Table 2 shows RayViT default (39.8) below RGB default (42.5); only RayViT-cls and the variant setting show gains. Please soften the wording or clarify that 'consistently' refers to robustness, not default performance.
  5. [§3 Eq. (3)] The notation for the reference view is slightly inconsistent: 'g' is used as a view label and as an index in 'c_g'. Also, V is defined as an unordered set of size N−1; using an ordered list would make the summation clearer.
  6. [General] The paper does not state whether code and models will be released. Given the empirical nature of the work, releasing the implementation would materially strengthen reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: ray map is an independent geometric input and the robustness results are held-out empirical evaluations, not derived from fitted inputs.

full rationale

This paper is an empirical architecture and evaluation contribution rather than a formal derivation, and I find no step where a claimed result reduces by construction to its inputs. The ray map is computed independently from camera intrinsics and extrinsics via Eqs. (1)–(2); it is not fitted to the downstream task, nor is the reported robustness a restatement of any training objective. The headline numbers (RoboCasa variant success, real-world completed stages) are measured on perturbed camera configurations not seen during training, so they are genuine held-out evaluations rather than predictions forced by fitted parameters. The baselines are either external methods or controlled ablations (RGB-only variant), and the self-citations (PMP, xLSTM action head) are implementation or comparison choices, not load-bearing justifications of the central robustness claim. The paper explicitly acknowledges the reliance on a task-relevant anchor view in §6, which is a scope limitation, not a circular step. The unstated assumption that test-time camera poses are known exactly for computing the ray map is an evaluation/correctness concern about out-of-distribution or noisy pose estimation, but it does not make any result equivalent to its inputs. Overall, the claim chain is self-contained against the stated evaluation protocol.

Assumptions & free parameters 5 free parameters · 6 assumptions · 0 invented entities

The central claim rests on one standard math block (Plücker ray encoding) and several domain assumptions about calibration availability and pretrained-representation compatibility. The most consequential is the untested premise that perturbed camera poses are known exactly at test time; the evaluation protocol effectively assumes it, and the paper never quantifies the perturbation schedule. No invented entities are introduced; the learned ray-conditioned token is a network component, not a postulated physical object. The free-parameter load is modest (a few tuned hyperparameters), but the undisclosed perturbation bounds are a real entry on this ledger.

free parameters (5)
  • cosine-loss weight λ = 0.01 (Table 9)
    Hand-set auxiliary-loss weight; no sweep reported; the claim that the loss 'consistently improves' robustness depends on this value.
  • cosine-loss layer (10th of 12) = 10 (Table 9)
    Layer choice for the cross-view alignment token; Figure 2 right shows layer depth changes results markedly, so this is a tuned choice affecting the gains.
  • camera perturbation ranges = unspecified ('bounded azimuth/elevation, shifted distance', §4)
    Bounds of the variant evaluation are never quantified; they set the difficulty of the robustness test and thus the size of the headline 13pp gain.
  • gated cross-attention size (2 blocks, 4 heads) = 2 blocks / 4 heads / 384-d (Table 9)
    Architecture capacity chosen by hand; no ablation on module size versus robustness.
  • anchor view for cosine loss = gripper camera, always (§3)
    A task-relevant anchor is required; the paper notes in §6 that this limits the method when no such view exists.
assumptions (6)
  • standard math Plücker coordinates (Eq. 1–2) correctly encode each pixel's viewing ray from intrinsics K and extrinsics R, t
    Textbook projective geometry, cited to [14,37,38]; not in dispute.
  • domain assumption Exact camera intrinsics and extrinsics are available at test time for every observation, including perturbed cameras
    The ray map — the entire geometric signal — is computed from K, R, t (§3); the §4 variant evaluation therefore presupposes the perturbed poses are known. Never stated as an assumption; no experiments with noisy or estimated poses.
  • domain assumption Additively adding ray features to patch tokens does not destroy the pretrained DINOv3 feature distribution
    Central design bet of §3 'Ray Map Positional Embedding'; supported indirectly by default-setting performance parity (Table 1) but not by a representation-quality analysis.
  • domain assumption Average-pooling the ray map to patch resolution preserves ray semantics
    §3 'Patch-aligned Ray Features' argues non-parametric pooling is better than learned convolution; no ablation compares choices.
  • domain assumption Pulling class tokens of different views together (Eq. 3) transfers to policy robustness
    Empirically supported by Table 2 and Figure 2 only when tokens already carry geometry; the paper itself shows this premise fails for the RGB-only representation.
  • standard math Score-based diffusion action head (Eq. 4, Appendix A) is a valid policy parameterization
    Borrowed framework from the diffusion policy literature; used identically across all compared methods.

how reviews work

0 comments
Cite this review

Pith. "Pith review of RayViT: Ray-Conditioned Visual Representations for Viewpoint-Robust Imitation Learning." pith.science (2026). https://pith.science/paper/3GSJK6GS

@misc{pith2026260729622,
  author       = {Pith},
  title        = {Pith review of: RayViT: Ray-Conditioned Visual Representations for Viewpoint-Robust Imitation Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3GSJK6GS}},
  note         = {Machine review of arXiv:2607.29622}
}
read the original abstract

Visual imitation learning enables robots to acquire visuomotor skills directly from images, yet RGB observations lack explicit geometric cues, making learned policies brittle to camera perturbations. To address this, we propose \textbf{Ray-conditioned Vision Transformer Encoder (RayViT)}, a lightweight architecture that injects camera geometry into pretrained ViT backbones. RayViT represents camera geometry as a Pl\"ucker ray map, patchifies it into ray features, and uses gated cross-attention to produce a ray-conditioned class token. These ray features are added as dense positional embeddings, while the ray class token replaces the original ViT class token to provide a geometry-aware summary representation. We combine this approach with an auxiliary cosine similarity loss to consistently improve the performance and robustness for geometry-aware tokens. Experiments on sim- and real-robot tasks demonstrate that RayViT improves robustness by approximately 13 percentage points under camera perturbations in multi-task RoboCasa benchmark and by 1.78 average completed stages in real-world multi-task success rate compared to baselines.

Figures

Figures reproduced from arXiv: 2607.29622 by the authors.

Figure 1
Figure 1. Overview of RayViT. Camera geometry is injected through a ray-map–conditioned class token and patch-aligned ray embeddings. An intermediate cross-view consistency loss regularizes the visual representation. Visual tokens, frozen CLIP [19] language tokens, and noisy action tokens are processed by a decoder-only xLSTM policy [41, 42]. Stacking r(u, v) over all pixels yields a ray map spatially aligned with the input i… view at source ↗
Figure 2
Figure 2. RGB-variant and RayViT-cls success rates when applying the auxiliary cosine-similarity [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Real-robot experimental setup. Vi￾sion system uses three cameras: left-, right-, and gripper-views and adapts a variant right-side view Task Max RGB RayViT-cls RayViT Default Variant Default Variant Default Variant Drawer 4 2.95 0.45 1.80 1.45 3.45 2.80 Insertion 3 1.55 1.25 2.20 0.60 1.70 1.50 Pick Place 4 1.85 0.90 1.60 1.50 2.60 3.10 Cup Stack 4 3.40 1.05 3.45 3.25 3.50 3.35 Completed Stages↑ 2.44 0.91 2.26 1.7 2… view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Overview of real robot tasks 15 [PITH_FULL_IMAGE:figures/full_fig_p015_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

45 extracted references · 9 linked inside Pith

  1. [1]

    Sim ´eoni, H

    O. Sim ´eoni, H. V . V o, M. Seitzer, F. Baldassarre, M. Oquab, C. Jose, V . Khalidov, M. Szafraniec, S. Yi, M. Ramamonjisoa, F. Massa, D. Haziza, L. Wehrstedt, J. Wang, T. Darcet, T. Moutakanni, L. Sentana, C. Roberts, A. Vedaldi, J. Tolan, J. Brandt, C. Couprie, J. Mairal, H. J´egou, P. Labatut, and P. Bojanowski. DINOv3.arXiv preprint arXiv:2508.10104,...

    arXiv 2025

  2. [2]

    J. Pang, N. Tang, K. Li, Y . Tang, X.-Q. Cai, Z.-Y . Zhang, G. Niu, M. Sugiyama, and Y . Yu. Learning view-invariant world models for visual robotic manipulation. InInterna- tional Conference on Learning Representations (ICLR), 2025. URLhttps://openreview. net/forum?id=vJwjWyt4Ed

    2025

  3. [3]

    L. Y . Chen, C. Xu, K. Dharmarajan, M. Z. Irshad, R. Cheng, K. Keutzer, M. Tomizuka, Q. Vuong, and K. Goldberg. RoVi-Aug: Robot and viewpoint augmentation for cross- embodiment robot learning. InConference on Robot Learning (CoRL), 2024. Oral

    2024

  4. [4]

    S. Tian, B. Wulfe, K. Sargent, K. Liu, S. Zakharov, V . Guizilini, and J. Wu. View-invariant policy learning via zero-shot novel view synthesis. InConference on Robot Learning (CoRL), 2024

    2024

  5. [5]

    Jiang, J

    T. Jiang, J. Ji, X. Tan, J. Fang, A. Bhattad, V . Guizilini, and M. R. Walter. Do you know where your camera is? view-invariant policy learning with camera conditioning, 2025. URL https://arxiv.org/abs/2510.02268

    arXiv 2025

  6. [6]

    Goyal, J

    A. Goyal, J. Xu, Y . Guo, V . Blukis, Y .-W. Chao, and D. Fox. RVT: Robotic view transformer for 3D object manipulation. InConference on Robot Learning (CoRL), 2023. Oral

    2023

  7. [7]

    Goyal, V

    A. Goyal, V . Blukis, J. Xu, Y . Guo, Y .-W. Chao, and D. Fox. RVT-2: Learning precise manip- ulation from few demonstrations. InRobotics: Science and Systems (RSS), 2024

    2024

  8. [8]

    T.-W. Ke, N. Gkanatsios, and K. Fragkiadaki. 3D diffuser actor: Policy diffusion with 3D scene representations. InConference on Robot Learning (CoRL), 2024

    2024

Show all 45 references
  1. [9]

    Shridhar, L

    M. Shridhar, L. Manuelli, and D. Fox. Perceiver-actor: A multi-task transformer for robotic manipulation. InConference on Robot Learning, pages 785–799. PMLR, 2023

    2023

  2. [10]

    X. Jia, Q. Wang, A. Wang, H. A. Wang, B. Gyenes, E. Gospodinov, X. Jiang, G. Li, H. Zhou, W. Liao, X. Huang, M. Beck, M. Reuss, R. Lioutikov, and G. Neumann. PointMapPol- icy: Structured point cloud processing for multi-modal imitation learning.arXiv preprint arXiv:2510.20406,...

    2025

  3. [11]

    H. Wang, K. Ye, Y . Li, W. Chen, and B. Chen. The less you depend, the more you learn: Synthesizing novel views from sparse, unposed images without any 3D knowledge.arXiv preprint arXiv:2506.09885, 2025. URLhttps://arxiv.org/abs/2506.09885

    2025 arXiv

  4. [12]

    Szymanowicz, M

    S. Szymanowicz, M. Chen, J. Wang, C. Rupprecht, and A. Vedaldi. LagerNVS: Latent geom- etry for fully neural real-time novel view synthesis.arXiv preprint arXiv:2603.20176, 2026. URLhttps://arxiv.org/abs/2603.20176. To appear in CVPR 2026

    2026 arXiv

  5. [13]

    F. Wang, Z. Shang, G. Liu, and Y . Yan. RayMap3R: Inference-time RayMap for dynamic 3D reconstruction.arXiv preprint arXiv:2603.20588, 2026. URLhttps://arxiv.org/abs/ 2603.20588. 9

    2026

  6. [14]

    Cheng, X

    Z. Cheng, X. Zhang, D. Di, C. Wei, H. Li, and X. Yang. Moca: Modeling object consistency for 3d camera control in video generation. InThe F ourteenth International Conference on Learn- ing Representations, 2026. URLhttps://openreview.net/forum?id=DZcpnudp7f

    2026

  7. [15]

    Z. Qiu, Z. Wang, B. Zheng, Z. Huang, K. Wen, S. Yang, R. Men, L. Yu, F. Huang, S. Huang, D. Liu, J. Zhou, and J. Lin. Gated attention for large language models: Non-linearity, sparsity, and attention-sink-free. InAdvances in Neural Information Processing Systems (NeurIPS),

  8. [16]

    S. Nair, A. Rajeswaran, V . Kumar, C. Finn, and A. Gupta. R3m: A universal visual repre- sentation for robot manipulation. InConference on Robot Learning, pages 892–909. PMLR, 2023

    2023

  9. [17]

    T. Xiao, I. Radosavovic, T. Darrell, and J. Malik. Masked visual pre-training for motor control. arXiv preprint arXiv:2203.06173, 2022

    2022 arXiv

  10. [18]

    Majumdar, K

    A. Majumdar, K. Yadav, S. Arnaud, J. Ma, C. Chen, S. Silwal, A. Jain, V .-P. Berges, T. Wu, J. Vakil, et al. Where are we in the search for an artificial visual cortex for embodied intelli- gence?Advances in Neural Information Processing Systems, 36:655–677, 2023

    2023

  11. [19]

    Radford, J

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, et al. Learning transferable visual models from natural language supervi- sion. InInternational conference on machine learning, pages 8748–8763. PmLR, 2021

    2021

  12. [20]

    Kirillov, E

    A. Kirillov, E. Mintun, N. Ravi, H. Mao, C. Rolland, L. Gustafson, T. Xiao, S. Whitehead, A. C. Berg, W.-Y . Lo, et al. Segment anything. InProceedings of the IEEE/CVF international conference on computer vision, pages 4015–4026, 2023

    2023

  13. [21]

    J. Wang, M. Chen, N. Karaev, A. Vedaldi, C. Rupprecht, and D. Novotny. VGGT: Visual geometry grounded transformer. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025. Best Paper Award

    2025

  14. [22]

    Abouzeid, M

    A. Abouzeid, M. Mansour, Q. Sun, Z. Sun, and D. Song. GeoAware-VLA: Implicit geometry aware vision-language-action model.arXiv preprint arXiv:2509.14117, 2025. URLhttps: //arxiv.org/abs/2509.14117

    2025

  15. [23]

    A. D. Vuong, M. N. Vu, and I. Reid. Improving robotic manipulation with efficient geometry- aware vision encoder.arXiv preprint arXiv:2509.15880, 2025. URLhttps://arxiv.org/ abs/2509.15880

    2025

  16. [24]

    Y . Zhu, Z. Jiang, P. Stone, and Y . Zhu. Learning generalizable manipulation policies with object-centric 3d representations. InConference on Robot Learning, pages 3418–3433. PMLR, 2023

    2023

  17. [25]

    S. Fei, S. Wang, Z. Shi, lee2025classai, J. Cai, P. Qian, L. Ji, X. He, S. Zhang, Z. Fei, et al. Libero-plus: In-depth robustness analysis of vision-language-action models.arXiv preprint arXiv:2510.13626, 2025

    2025 arXiv

  18. [26]

    Simeonov, Y

    A. Simeonov, Y . Du, Y .-C. Lin, A. R. Garcia, L. P. Kaelbling, T. Lozano-P´erez, and P. Agrawal. Se (3)-equivariant relational rearrangement with neural descriptor fields. InConference on Robot Learning, pages 835–846. PMLR, 2023

    2023

  19. [27]

    Zhang, Z

    Z. Zhang, Z. Xu, J. N. Lakamsani, and Y . She. Canonical policy: Learning canonical 3d representation for se (3)-equivariant policy.arXiv preprint arXiv:2505.18474, 2025

    2025

  20. [28]

    X. Zhu, Y . Qi, Y . Zhu, R. Walters, and R. Platt. Equact: An se (3)-equivariant multi-task transformer for open-loop robotic manipulation.arXiv preprint arXiv:2505.21351, 2025. 10

    2025 arXiv

  21. [29]

    Tobin, R

    J. Tobin, R. Fong, A. Ray, J. Schneider, W. Zaremba, and P. Abbeel. Domain randomization for transferring deep neural networks from simulation to the real world. In2017 IEEE/RSJ international conference on intelligent robots and systems (IROS), pages 23–30. IEEE, 2017

    2017

  22. [30]

    Laskin, K

    M. Laskin, K. Lee, A. Stooke, L. Pinto, P. Abbeel, and A. Srinivas. Reinforcement learning with augmented data.Advances in neural information processing systems, 33:19884–19895, 2020

    2020

  23. [31]

    Yarats, I

    D. Yarats, I. Kostrikov, and R. Fergus. Image augmentation is all you need: Regularizing deep reinforcement learning from pixels. InInternational conference on learning representations, 2020

    2020

  24. [32]

    S.-W. Lee, X. Kang, B. Yang, and Y .-L. Kuo. CLASS: Contrastive learning via action sequence supervision for robot manipulation. InConference on Robot Learning (CoRL), volume 305 of Proceedings of Machine Learning Research, pages 4743–4766. PMLR, 2025

    2025

  25. [33]

    Y . Ze, G. Zhang, K. Zhang, C. Hu, M. Wang, and H. Xu. 3D diffusion policy: Generalizable visuomotor policy learning via simple 3D representations. InRobotics: Science and Systems (RSS), 2024

    2024

  26. [34]

    Gervet, Z

    T. Gervet, Z. Xian, N. Gkanatsios, and K. Fragkiadaki. Act3d: 3d feature field transformers for multi-task robotic manipulation. InConference on Robot Learning, pages 3949–3965. PMLR, 2023

    2023

  27. [35]

    Gyenes, E

    B. Gyenes, E. Gospodinov, J. Frieling, E. Krohmer, X. Jia, N. Schreiber, N. Freymuth, and G. Neumann. Fourier features let agents learn high precision policies with imitation learn- ing. InF orty-third International Conference on Machine Learning, 2026. URLhttps: //openreview....

    2026

  28. [36]

    Wilcox, M

    A. Wilcox, M. Ghanem, M. Moghani, P. Barroso, B. Joffe, and A. Garg. Adapt3r: Adaptive 3d scene representation for domain transfer in imitation learning. InConference on Robot Learning, pages 1486–1514. PMLR, 2025

    2025

  29. [37]

    Sitzmann, S

    V . Sitzmann, S. Rezchikov, B. Freeman, J. Tenenbaum, and F. Durand. Light field networks: Neural scene representations with single-evaluation rendering.Advances in Neural Informa- tion Processing Systems, 34:19313–19325, 2021

    2021

  30. [38]

    Y .-B. Jia. Pl ¨ucker coordinates for lines in the space.Problem Solver Techniques for Applied Computer Science, Com-S-477/577 Course Handout, 3:3, 2020

    2020

  31. [39]

    H. Lin, S. Chen, J. H. Liew, D. Y . Chen, Z. Li, G. Shi, J. Feng, and B. Kang. Depth anything 3: Recovering the visual space from any views.arXiv preprint arXiv:2511.10647, 2025. URL https://arxiv.org/abs/2511.10647

    2025 arXiv

  32. [40]

    Aether Team, H. Zhu, Y . Wang, J. Zhou, W. Chang, Y . Zhou, Z. Li, J. Chen, C. Shen, J. Pang, and T. He. Aether: Geometric-aware unified world modeling. InProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2025

    2025

  33. [41]

    M. Beck, K. P ¨oppel, M. Spanring, A. Auer, O. Prudnikova, M. Kopp, G. Klambauer, J. Brand- stetter, and S. Hochreiter. xlstm: Extended long short-term memory, 2024. URLhttps: //arxiv.org/abs/2405.04517

    2024 arXiv

  34. [42]

    X. Jia, A. Donat, X. Huang, X. Zhao, D. Blessing, H. Zhou, H. A. Wang, H. Zhang, Q. Wang, R. Lioutikov, and G. Neumann. X-il: Exploring the design space of imitation learning policies,

  35. [43]

    Nasiriany, A

    S. Nasiriany, A. Maddukuri, L. Zhang, A. Parikh, A. Lo, A. Joshi, A. Mandlekar, and Y . Zhu. Robocasa: Large-scale simulation of everyday tasks for generalist robots. InRobotics: Science and Systems (RSS), 2024. 11

    2024

  36. [44]

    C. Zhu, S. Suri, C. Jose, M. Oquab, M. Szafraniec, W. Wen, Y . Xiong, P. Labatut, P. Bo- janowski, R. Krishnamoorthi, and V . Chandra. Efficient universal perception encoder.arXiv preprint arXiv:2603.22387, 2026. URLhttps://arxiv.org/abs/2603.22387. 12 A Score-based Diffusion ...

    2026

  37. [2025]

    URLhttps://arxiv.org/abs/2502.12330

    arXiv

Pith tools

Reviewed August 3, 2026 · model on record in the stance chip above.