Pith. sign in

REVIEW 2 major objections 5 minor 57 references

A Bayesian shared-control system for full SE(3) grasping lets low-bandwidth joystick users succeed more often with less effort.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-13 03:55 UTC pith:XN2UA6MT

load-bearing objection Solid SE(3) extension of RT-V2 with a clean 21-subject study; the grasp-set assumption is the real limit, and the authors already own it. the 2 major comments →

arxiv 2607.09315 v1 pith:XN2UA6MT submitted 2026-07-10 cs.RO

Robot Trajectron V3: A Probabilistic Shared Control Framework for SE(3) Manipulation

Pinhao Song , Zhongxi Li , Ze Fu , Federico Ulloa Rios , Renaud Detry This is my paper
classification cs.RO
keywords shared controlSE(3) manipulationBayesian intent inferencetrajectory predictionassistive teleoperationgrasp affordanceslow-bandwidth interfaces
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Teleoperating a multi-joint robot arm with a low-bandwidth device such as a joystick is slow, fatiguing, and error-prone because the user must constantly command six degrees of freedom while also avoiding collisions. RT-V3 treats the problem as Bayesian inference: it first learns a prior over the trajectories a skilled operator would produce given the current scene geometry and a dense cloud of candidate grasp poses, then continuously updates that prior with the noisy commands the user actually issues. The resulting posterior supplies real-time assistance, and an asynchronous mode lets the robot continue on the most likely trajectory when the user pauses. In simulation and with 21 novice participants the method raises success rates, cuts joystick inputs, shortens trajectories, and lowers NASA-TLX mental and effort scores relative to pure teleoperation and a classical hindsight-optimization baseline, while still giving users a sense of control.

Core claim

Shared control for multi-object, multi-affordance SE(3) grasping can be realized as continuous Bayesian posterior estimation over future trajectories: a transformer-conditioned generative prior that reasons over point clouds and grasp candidates is fused at every step with a likelihood derived from the observed user twist, producing assistance that is both intent-aware and collision-aware.

What carries the argument

The factorized CVAE-GMM prior: a transformer encodes scene point clouds and grasp-pose clouds, an LSTM tracks robot dynamics, a discrete latent variable captures multi-modal maneuver classes, and the six-dimensional action is split into a translational Gaussian whose sample then conditions a rotational Gaussian, yielding a tractable density that remains stable in closed-loop execution.

Load-bearing premise

The off-the-shelf grasp planner must already contain the exact grasp pose the user intends; if that pose is missing, the learned prior cannot recover it and the assistance actively fights the user.

What would settle it

Replace the grasp planner with one that systematically omits the instructed side or orientation of each object and measure whether success rate and user-reported control drop below pure teleoperation.

Watch this falsifier — get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper presents Robot Trajectron V3 (RT-V3), a Bayesian shared-control framework for SE(3) grasping under low-bandwidth interfaces. User intent is modeled as a distribution over future end-effector trajectories conditioned on past dynamics, point clouds, and candidate grasp poses from an off-the-shelf planner. A transformer-based CVAE-GMM prior with factorized translation-then-rotation decoding is trained via ELBO on planner trajectories; at runtime a Gaussian interface-noise likelihood yields a closed-form posterior that is used for continuous assistance, including an asynchronous mode that executes the prior when the user is silent. Evaluation comprises trajectory-prediction ablations, closed-loop planning against CuRobo and NeoSS, large-scale simulated-user shared-control trials, and a 21-participant real-world Xbox-joystick study reporting higher success rate, fewer inputs, and lower NASA-TLX scores than pure teleoperation and Hindsight Optimization.

Significance. If the reported gains hold, RT-V3 supplies a practical, modular recipe for lifting probabilistic shared autonomy from SE(2)/R^3 navigation to full SE(3) multi-object multi-affordance grasping. The combination of a transformer context encoder, translation-conditioned rotation factorization, and asynchronous assistance is a concrete engineering advance that demonstrably reduces cognitive and physical load for novice users on low-bandwidth devices. Code release, extensive ablations, and a properly powered user study with non-parametric statistics strengthen the claim and make the work immediately usable by the assistive-robotics community. The explicit surface of the grasp-planner completeness assumption further aids reproducibility and future improvement.

major comments (2)
  1. Methodology Overview and Sec. V-B state the load-bearing assumption that the dense candidate set G produced by the off-the-shelf grasp planner contains the user’s intended grasp. When this fails (explicitly illustrated by the failure case in Fig. 8b and quantified by unmatched/time-out rates), the prior cannot recover the correct intent and posterior assistance becomes actively harmful. While the paper already acknowledges the limitation and shows RT-V3 is less brittle than HO under the same incomplete G, the real-world claim (86.9 % SR) is conditioned on the planner’s coverage of the 12 tested scenes. A short quantitative sensitivity analysis—e.g., randomly dropping a fraction of ground-truth grasps from G and re-running the simulated-user suite—would make the robustness boundary explicit and strengthen the central claim.
  2. Sec. VII-A / Table I: the “6D pred” ablation achieves the best open-loop ADE/FDE yet collapses to 3.4 % planning success. The authors attribute this to compounding error and claim the translation-conditioned factorization acts as a regularizer. The explanation is plausible but remains post-hoc; a short diagnostic (e.g., closed-loop state-distribution divergence or per-step rotation error conditioned on translation error) would confirm that the factorization, rather than other hyper-parameters, is the decisive factor for closed-loop stability.
minor comments (5)
  1. Eq. (5) and surrounding text: the maneuver-class posterior q(z|h_H) is used only at training time; a one-sentence clarification that inference always samples from the prior p(z|h_0) would prevent reader confusion.
  2. Fig. 6 caption and text: “adjusted” trajectory lengths exclude autonomous segments; the exact distance threshold used to disable asynchronous mode should be stated numerically for reproducibility.
  3. Appendix B: the manifold distance scales Euler angles by 0.3/π; a brief justification relative to the 0.3 m workspace size is already present, but units of the resulting ADE/FDE numbers (cm? rad?) should be made explicit in Table I.
  4. Typographical: “forSE(3)” missing space in title and abstract; “CV AE” inconsistently spaced; “maneuver class” sometimes hyphenated, sometimes not.
  5. Sec. VII-D protocol: 19 male / 2 female participants; a short note on whether gender imbalance was considered in the statistical analysis would be good practice.

Circularity Check

0 steps flagged

No significant circularity: standard CVAE prior + Bayesian update trained and evaluated empirically; self-cites supply lineage only.

full rationale

RT-V3 learns a trajectory prior p(i|c) via CVAE/ELBO (Eqs. 2–5) on planner-generated SE(3) trajectories, then forms the posterior by a standard Gaussian-interface likelihood (Eqs. 22–34). All headline claims—ADE/FDE, planning SR, and the 21-user study gains in success rate, joystick inputs, and NASA-TLX—are measured on held-out or real data against Direct/HO baselines, not obtained by algebraic rearrangement of fitted constants. Self-citations to RT-V1/RT-V2 supply the 2-D navigation architecture that is extended, but the SE(3) factorization, transformer encoders, asynchronous mechanism, and new experimental numbers stand independently. The grasp-set assumption is an explicit modeling premise with acknowledged failure modes (Fig. 8b, Limitations), not a definitional loop. No equation reduces a claimed prediction to its own input by construction.

Axiom & Free-Parameter Ledger

4 free parameters · 6 axioms · 1 invented entities

The central claims rest on a small set of modeling assumptions that are standard in probabilistic robotics plus a few free parameters that control the Bayesian update and the asynchronous switch. No new physical entities are postulated; the 'maneuver class' latent is a conventional discrete CVAE variable. The most consequential domain assumptions are the completeness of the grasp-pose set and the transferability of planner-generated trajectories to human users.

free parameters (4)
  • β (ELBO weight)
    Standard CVAE hyper-parameter that trades reconstruction against KL regularization of the maneuver-class posterior; value not reported but required for training stability.
  • Σ_sys, Γ_sys (interface noise covariances)
    Diagonal covariances of the Gaussian likelihood p(u|i) that appear in the closed-form Kalman-style posterior update (Eqs. 24–31); chosen by the authors to match the joystick interface.
  • asynchronous disable distance
    Hand-chosen proximity threshold at which autonomous execution is turned off so the user can perform fine grasp adjustments; directly affects command count and success rate.
  • number of maneuver classes |Z|
    Cardinality of the discrete latent that induces the GMM action distribution; controls multi-modality capacity and is selected by the authors.
axioms (6)
  • domain assumption The dense set of candidate grasp poses generated by an off-the-shelf planner contains the grasp the user intends to execute.
    Stated explicitly in Methodology Overview and Sec. V-B; if false the prior cannot place mass on the true intent.
  • domain assumption Environment is static during a trial (point cloud and grasp set do not change).
    Used to factor the state into dynamics and fixed context (Sec. V).
  • domain assumption Latent dynamics are Markovian: h_t depends only on h_{t-1} and a_{t-1}.
    Enables the factorization of the trajectory likelihood in Eq. 3–4.
  • domain assumption User-command noise is independent Gaussian with known covariance (p(u|i)=N(u|i,Σ_sys)).
    Yields the closed-form posterior mean and covariance in Eqs. 23–31.
  • domain assumption All poses, velocities and point clouds are expressed in the current end-effector frame, which is assumed to confer SE(3) equivariance.
    Stated in Sec. V and proved in Appendix A; underpins generalization across workspace locations.
  • standard math Standard CVAE ELBO with discrete latent is a valid training objective for multi-modal trajectory densities.
    Invoked via citations to Sohn et al. and Higgins et al.; no new derivation required.
invented entities (1)
  • maneuver-class latent z no independent evidence
    purpose: Low-dimensional discrete variable that induces a GMM over future SE(3) actions, capturing multi-modal grasp intents.
    Standard CVAE construction already used in RT-V2; not a new physical or mathematical object, merely re-used for SE(3).

pith-pipeline@v1.1.0-grok45 · 29520 in / 3281 out tokens · 36218 ms · 2026-07-13T03:55:37.276082+00:00 · methodology

0 comments
read the original abstract

We aim to address the challenge of teleoperating robotic arms for high-degree-of-freedom (high-DoF) manipulation tasks, which is cognitively demanding and error-prone, particularly when relying on low-bandwidth interfaces. We propose Robot Trajectron V3 (RT-V3), a probabilistic shared control framework designed for $SE(3)$ grasping tasks. RT-V3 formulates shared control as Bayesian inference by learning a prior over user intent and combining it with real-time user commands to estimate the posterior intent distribution. The prior models user intent as a distribution over future trajectories conditioned on past robot dynamics and visual scene context. The intent prior is parameterized by a transformer-based conditional generative model that reasons over point clouds and candidate grasp poses, together with a factorized translation-rotation representation that improves learning efficiency in high-dimensional action spaces. During execution, RT-V3 continuously estimates the posterior distribution over future trajectories by combining the learned intent prior with a user-command likelihood derived from the observed control input, enabling continuous intent refinement and shared assistance. Comprehensive experiments demonstrate that RT-V3 achieves high accuracy in trajectory prediction and competitive performance in reactive planning. Furthermore, real-world user studies indicate that RT-V3 significantly outperforms baseline methods in terms of success rate and efficiency, while substantially reducing the user's physical and mental workload.

Figures

Figures reproduced from arXiv: 2607.09315 by Federico Ulloa Rios, Pinhao Song, Renaud Detry, Ze Fu, Zhongxi Li.

Figure 1
Figure 1. Figure 1: Graphical models of the probabilistic decision-making process in the RT-V3 framework. Trajectories are represented [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: The overall architecture of Robot Trajectron V3 (RT-V3). [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Simulation experiment setup. [Left] Simulation setup with a 30 × 30 × 30 cm3 . [Right] The sampled point clouds and candidate grasp poses of the scene. • Shared Control (Sec. VII-C and VII-D): To demonstrate generalizability, we first conduct large-scale testing with diverse simulated agents, followed by real-world trials with human users. These experiments validate RT-V3’s assistive ability across varied … view at source ↗
Figure 4
Figure 4. Figure 4: Visualization of RT-V3’s predicted trajectories. For clarity, we visualize every four iterations. The past trajectories are [PITH_FULL_IMAGE:figures/full_fig_p009_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Real-world experimental setup with a Franka Research [PITH_FULL_IMAGE:figures/full_fig_p010_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Some objective results for the real-world experiment. (a) Total joystick inputs. (b) Completion time. (c1) Translational [PITH_FULL_IMAGE:figures/full_fig_p012_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: The user study for each method across all participants for the real-world shared control experiment. The plots are bar [PITH_FULL_IMAGE:figures/full_fig_p013_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Visualization of task processes and entropy in successful and failed scenarios. (a) A successful demonstration of grasping [PITH_FULL_IMAGE:figures/full_fig_p014_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Twelve tasks in the real-world shared autonomy experiment. During the experiment, these tasks were separated into 3 [PITH_FULL_IMAGE:figures/full_fig_p016_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Visualizations of four different demonstrations. (a) Task1: Grasp the facial cleanser from the right. (b) Task3: Grasp [PITH_FULL_IMAGE:figures/full_fig_p017_10.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

57 extracted references · 12 linked inside Pith

  1. [1]

    Mobile aloha: Learning bimanual mobile manipulation with low-cost whole-body teleoperation,

    Z. Fu, T. Z. Zhao, and C. Finn, “Mobile aloha: Learning bimanual mobile manipulation with low-cost whole-body teleoperation,”arXiv preprint arXiv:2401.02117, 2024

    Pith/arXiv arXiv 2024

  2. [2]

    Open-television: Teleoperation with immersive active visual feedback,

    X. Cheng, J. Li, S. Yang, G. Yang, and X. Wang, “Open-television: Teleoperation with immersive active visual feedback,”arXiv preprint arXiv:2407.01512, 2024

    Pith/arXiv arXiv 2024

  3. [3]

    Telemotion: A realtime humanoid teleoperation system with motion capture,

    J. Gan, S. Guo, Z. Li, and X. Shi, “Telemotion: A realtime humanoid teleoperation system with motion capture,” inInternational Conference on Extended Reality. Springer, 2024, pp. 31–45

    2024

  4. [4]

    Rulik, M

    I. Rulik, M. S. H. Sunny, J. D. Sanjuan De Caro, M. I. I. Zarif, B. Brahmi, S. I. Ahamed, K. Schultz, I. Wang, T. Leheng, J. P. Longxiang,et al., “Control of a wheelchair-mounted 6dof assistive robot 𝑡=0.0 𝑡=1.9𝑡=3.3 𝑡=9.3𝑡=6.3 𝑡=19.4𝑡=13.4 𝑡=5.7 𝑡=10.2 (a) 𝑡=0.0 𝑡=4.3 𝑡=9.8𝑡=11.9𝑡=14.9 𝑡=19.0𝑡=17.8 (b) 𝑡=0.0 𝑡=2.6 𝑡=14.9𝑡=8.0 𝑡=9.9 𝑡=13.6 (c) Fig. 10: Vi...

    2022

  5. [5]

    Controlling an automated wheelchair via joystick/head-joystick supported by smart driving assistance,

    T. Rofer, C. Mandel, and T. Laue, “Controlling an automated wheelchair via joystick/head-joystick supported by smart driving assistance,” in 2009 IEEE international conference on rehabilitation robotics. IEEE, 2009, pp. 743–748

    2009

  6. [6]

    Reach and grasp by people with tetraplegia using a neurally controlled robotic arm,

    L. R. Hochberg, D. Bacher, B. Jarosiewicz, N. Y . Masse, J. D. Simeral, J. V ogel, S. Haddadin, J. Liu, S. S. Cash, P. Van Der Smagt,et al., “Reach and grasp by people with tetraplegia using a neurally controlled robotic arm,”Nature, vol. 485, no. 7398, pp. 372–375, 2012

    2012

  7. [7]

    Stabilization–responsiveness trade-offs in continuous shared-control for invasive brain–computer interfaces,

    O. Saussus, P. Song, S. De Schrijver, I. Caprara, T. Decramer, R. Detry, and P. Janssen, “Stabilization–responsiveness trade-offs in continuous shared-control for invasive brain–computer interfaces,”bioRxiv, pp. 2026–03, 2026

    2026

  8. [8]

    Assistive planning in complex, dynamic environments: a probabilistic approach,

    P. Trautman, “Assistive planning in complex, dynamic environments: a probabilistic approach,” in2015 IEEE International Conference on Systems, Man, and Cybernetics. IEEE, 2015, pp. 3072–3078

    2015

  9. [9]

    An intention prediction based shared control system for point-to-point navigation of a robotic wheelchair,

    Z. Lei, B. Y . Tan, N. P. Garg, L. Li, A. Sidarta, and W. T. Ang, “An intention prediction based shared control system for point-to-point navigation of a robotic wheelchair,”IEEE Robotics and Automation Letters, vol. 7, no. 4, pp. 8893–8900, 2022

    2022

  10. [10]

    Shared control in robot teleoperation with improved potential fields,

    A. Gottardi, S. Tortora, E. Tosello, and E. Menegatti, “Shared control in robot teleoperation with improved potential fields,”IEEE Transactions on Human-Machine Systems, vol. 52, no. 3, pp. 410–422, 2022

    2022

  11. [11]

    A policy-blending formalism for shared control,

    A. D. Dragan and S. S. Srinivasa, “A policy-blending formalism for shared control,”The International Journal of Robotics Research, vol. 32, no. 7, pp. 790–805, 2013

    2013

  12. [12]

    Contin- uous shared control for robotic arm reaching driven by a hybrid gaze- brain machine interface,

    Y . Wang, G. Xu, A. Song, B. Xu, H. Li, C. Hu, and H. Zeng, “Contin- uous shared control for robotic arm reaching driven by a hybrid gaze- brain machine interface,” in2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2018, pp. 4462–4467

    2018

  13. [13]

    Natural gradient shared control,

    Y . Oh, S.-W. Wu, M. Toussaint, and J. Mainprice, “Natural gradient shared control,” in2020 29th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN). IEEE, 2020, pp. 1223–1229

    2020

  14. [14]

    Brain–computer interface control with artificial intelligence copilots,

    J. Y . Lee, S. Lee, A. Mishra, X. Yan, B. McMahan, B. Gaisford, C. Kobashigawa, M. Qu, C. Xie, and J. C. Kao, “Brain–computer interface control with artificial intelligence copilots,”Nature machine intelligence, vol. 7, no. 9, pp. 1510–1523, 2025

    2025

  15. [15]

    A shared control strategy for reach and grasp of multiple objects using robot vision and noninvasive brain–computer interface,

    Y . Xu, H. Zhang, L. Cao, X. Shu, and D. Zhang, “A shared control strategy for reach and grasp of multiple objects using robot vision and noninvasive brain–computer interface,”IEEE Transactions on Automa- tion Science and Engineering, vol. 19, no. 1, pp. 360–372, 2020

    2020

  16. [16]

    Robot trajectron: Trajectory prediction-based shared control for robot manipulation,

    P. Song, P. Li, E. Aertbeli ¨en, and R. Detry, “Robot trajectron: Trajectory prediction-based shared control for robot manipulation,” inProceedings Of IEEE International Conference on Robotics and Automation, 2024

    2024

  17. [17]

    Robot trajectron v2: A probabilistic shared control framework for navigation,

    P. Song, Y . Du, O. Saussus, S. De Schrijver, I. Caprara, P. Janssen, and R. Detry, “Robot trajectron v2: A probabilistic shared control framework for navigation,”arXiv preprint arXiv:2509.19954, 2025

    arXiv 2025

  18. [18]

    Model predictive-based shared control for brain- controlled driving,

    Y . Lu, L. Bi, and H. Li, “Model predictive-based shared control for brain- controlled driving,”IEEE Transactions on Intelligent Transportation Systems, vol. 21, no. 2, pp. 630–640, 2019

    2019

  19. [19]

    Shared control for intelligent wheelchairs: an implicit estimation of the user intention,

    D. Vanhooydonck, E. Demeester, M. Nuttin, and H. Van Brussel, “Shared control for intelligent wheelchairs: an implicit estimation of the user intention,” inProceedings of the 1st international workshop on advances in service robotics (ASER’03), 2003, pp. 176–182

    2003

  20. [20]

    Assistron: Bayesian shared auton- omy with off-the-shelf vision-language-action models,

    P. Song, Z. Fu, Y . Hu, and R. Detry, “Assistron: Bayesian shared auton- omy with off-the-shelf vision-language-action models,”arXiv preprint arXiv:2606.23147, 2026

    Pith/arXiv arXiv 2026

  21. [21]

    Shared autonomy via hindsight optimization for teleoperation and teaming,

    S. Javdani, H. Admoni, S. Pellegrinelli, S. S. Srinivasa, and J. A. Bagnell, “Shared autonomy via hindsight optimization for teleoperation and teaming,”The International Journal of Robotics Research, vol. 37, no. 7, pp. 717–742, 2018

    2018

  22. [22]

    Autonomy infused teleoperation with application to brain computer interface controlled manipulation,

    K. Muelling, A. Venkatraman, J.-S. Valois, J. E. Downey, J. Weiss, S. Javdani, M. Hebert, A. B. Schwartz, J. L. Collinger, and J. A. Bagnell, “Autonomy infused teleoperation with application to brain computer interface controlled manipulation,”Autonomous Robots, vol. 41, pp. 1401–1422, 2017

    2017

  23. [23]

    Shared autonomy of a robotic manipulator for grasping under human intent uncertainty using pomdps,

    J.-A. Yow, N. P. Garg, and W. T. Ang, “Shared autonomy of a robotic manipulator for grasping under human intent uncertainty using pomdps,” IEEE Transactions on Robotics, vol. 40, pp. 332–350, 2023

    2023

  24. [24]

    Tasc: Task-aware shared control for teleoperated manipulation,

    Z. Fu, P. Song, Y . Hu, and R. Detry, “Tasc: Task-aware shared control for teleoperated manipulation,” 2025. [Online]. Available: https://arxiv.org/abs/2509.10416

    Pith/arXiv arXiv 2025

  25. [25]

    Survey of imitation learning for robotic manipulation,

    B. Fang, S. Jia, D. Guo, M. Xu, S. Wen, and F. Sun, “Survey of imitation learning for robotic manipulation,”International Journal of Intelligent Robotics and Applications, vol. 3, pp. 362–369, 2019

    2019

  26. [26]

    A survey on imitation learning techniques for end-to-end autonomous vehicles,

    L. Le Mero, D. Yi, M. Dianati, and A. Mouzakitis, “A survey on imitation learning techniques for end-to-end autonomous vehicles,” IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 9, pp. 14 128–14 147, 2022

    2022

  27. [27]

    Open x- embodiment: Robotic learning datasets and rt-x models,

    A. Padalkar, A. Pooley, A. Jain, A. Bewley, A. Herzog, A. Ir- pan, A. Khazatsky, A. Rai, A. Singh, A. Brohan,et al., “Open x- embodiment: Robotic learning datasets and rt-x models,”arXiv preprint arXiv:2310.08864, 2023

    Pith/arXiv arXiv 2023

  28. [28]

    Learning fine- grained bimanual manipulation with low-cost hardware,

    T. Z. Zhao, V . Kumar, S. Levine, and C. Finn, “Learning fine- grained bimanual manipulation with low-cost hardware,”arXiv preprint arXiv:2304.13705, 2023

    Pith/arXiv arXiv 2023

  29. [29]

    Implicit behavioral cloning,

    P. Florence, C. Lynch, A. Zeng, O. A. Ramirez, A. Wahid, L. Downs, A. Wong, J. Lee, I. Mordatch, and J. Tompson, “Implicit behavioral cloning,” inConference on robot learning. PMLR, 2022, pp. 158–168

    2022

  30. [30]

    Diffusion policy: Visuomotor policy learning via ac- tion diffusion,

    C. Chi, Z. Xu, S. Feng, E. Cousineau, Y . Du, B. Burchfiel, R. Tedrake, and S. Song, “Diffusion policy: Visuomotor policy learning via ac- tion diffusion,”The International Journal of Robotics Research, p. 02783649241273668, 2023

    2023

  31. [31]

    Rlbench: The robot learning benchmark & learning environment,

    S. James, Z. Ma, D. R. Arrojo, and A. J. Davison, “Rlbench: The robot learning benchmark & learning environment,”IEEE Robotics and Automation Letters, vol. 5, no. 2, pp. 3019–3026, 2020

    2020

  32. [32]

    Libero: Benchmarking knowledge transfer for lifelong robot learning,

    B. Liu, Y . Zhu, C. Gao, Y . Feng, Q. Liu, Y . Zhu, and P. Stone, “Libero: Benchmarking knowledge transfer for lifelong robot learning,”Advances in Neural Information Processing Systems, vol. 36, pp. 44 776–44 791, 2023

    2023

  33. [33]

    Calvin: A benchmark for language-conditioned policy learning for long-horizon robot manipulation tasks,

    O. Mees, L. Hermann, E. Rosete-Beas, and W. Burgard, “Calvin: A benchmark for language-conditioned policy learning for long-horizon robot manipulation tasks,”IEEE Robotics and Automation Letters, vol. 7, no. 3, pp. 7327–7334, 2022

    2022

  34. [34]

    Dinov2: Learning robust visual features without supervision,

    M. Oquab, T. Darcet, T. Moutakanni, H. V o, M. Szafraniec, V . Khalidov, P. Fernandez, D. Haziza, F. Massa, A. El-Nouby,et al., “Dinov2: Learning robust visual features without supervision,”arXiv preprint arXiv:2304.07193, 2023

    Pith/arXiv arXiv 2023

  35. [35]

    Openvla: An open- source vision-language-action model,

    M. J. Kim, K. Pertsch, S. Karamcheti, T. Xiao, A. Balakrishna, S. Nair, R. Rafailov, E. Foster, G. Lam, P. Sanketi,et al., “Openvla: An open- source vision-language-action model,”arXiv preprint arXiv:2406.09246, 2024

    Pith/arXiv arXiv 2024

  36. [36]

    Pointnet: Deep learning on point sets for 3d classification and segmentation,

    C. R. Qi, H. Su, K. Mo, and L. J. Guibas, “Pointnet: Deep learning on point sets for 3d classification and segmentation,” inProceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 652–660

    2017

  37. [37]

    Point transformer,

    H. Zhao, L. Jiang, J. Jia, P. H. Torr, and V . Koltun, “Point transformer,” inProceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 16 259–16 268

    2021

  38. [38]

    Point transformer v2: Grouped vector attention and partition-based pooling,

    X. Wu, Y . Lao, L. Jiang, X. Liu, and H. Zhao, “Point transformer v2: Grouped vector attention and partition-based pooling,”Advances in Neural Information Processing Systems, vol. 35, pp. 33 330–33 342, 2022

    2022

  39. [39]

    Point transformer v3: Simpler faster stronger,

    X. Wu, L. Jiang, P.-S. Wang, Z. Liu, X. Liu, Y . Qiao, W. Ouyang, T. He, and H. Zhao, “Point transformer v3: Simpler faster stronger,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2024, pp. 4840–4851

    2024

  40. [40]

    Equivariant diffusion policy,

    D. Wang, S. Hart, D. Surovik, T. Kelestemur, H. Huang, H. Zhao, M. Yeatman, J. Wang, R. Walters, and R. Platt, “Equivariant diffusion policy,” in8th Annual Conference on Robot Learning, 2024

    2024

  41. [41]

    Bilateral control-based imitation learning for velocity- controlled robot,

    S. Sakaino, “Bilateral control-based imitation learning for velocity- controlled robot,” in2021 IEEE 30th International Symposium on Industrial Electronics (ISIE). IEEE, 2021, pp. 1–6

    2021

  42. [42]

    3d diffuser ac- tor: Policy diffusion with 3d scene representations,

    T.-W. Ke, N. Gkanatsios, and K. Fragkiadaki, “3d diffuser ac- tor: Policy diffusion with 3d scene representations,”arXiv preprint arXiv:2402.10885, 2024

    Pith/arXiv arXiv 2024

  43. [43]

    Mini diffuser: Fast multi-task diffusion policy training using two-level mini-batches,

    Y . Hu, P. Song, K. Wen, and R. Detry, “Mini diffuser: Fast multi-task diffusion policy training using two-level mini-batches,”arXiv preprint arXiv:2505.09430, 2025

    Pith/arXiv arXiv 2025

  44. [44]

    Perceiver-actor: A multi- task transformer for robotic manipulation,

    M. Shridhar, L. Manuelli, and D. Fox, “Perceiver-actor: A multi- task transformer for robotic manipulation,” inProceedings of The 6th Conference on Robot Learning, ser. Proceedings of Machine Learning Research, K. Liu, D. Kulic, and J. Ichnowski, Eds., vol

  45. [45]

    PMLR, 14–18 Dec 2023, pp. 785–799. [Online]. Available: https://proceedings.mlr.press/v205/shridhar23a.html

    2023

  46. [46]

    Transformer-based deep im- itation learning for dual-arm robot manipulation,

    H. Kim, Y . Ohmura, and Y . Kuniyoshi, “Transformer-based deep im- itation learning for dual-arm robot manipulation,” in2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2021, pp. 8965–8972

    2021

  47. [47]

    Implicit grasp diffusion: Bridging the gap between dense prediction and sampling-based grasping,

    P. Song, P. Li, and R. Detry, “Implicit grasp diffusion: Bridging the gap between dense prediction and sampling-based grasping,” in8th Annual Conference on Robot Learning, 2024

    2024

  48. [48]

    Equivariant volumetric grasping,

    P. Song, Y . Hu, P. Li, and R. Detry, “Equivariant volumetric grasping,” arXiv preprint arXiv:2507.18847, 2025

    Pith/arXiv arXiv 2025

  49. [49]

    A review of intent detection, arbitration, and communication aspects of shared control for physical human–robot interaction,

    D. P. Losey, C. G. McDonald, E. Battaglia, and M. K. O’Malley, “A review of intent detection, arbitration, and communication aspects of shared control for physical human–robot interaction,”Applied Mechanics Reviews, vol. 70, no. 1, p. 010804, 2018

    2018

  50. [50]

    Learning structured output represen- tation using deep conditional generative models,

    K. Sohn, H. Lee, and X. Yan, “Learning structured output represen- tation using deep conditional generative models,”Advances in neural information processing systems, vol. 28, 2015

    2015

  51. [51]

    beta-vae: Learning basic visual concepts with a constrained variational framework,

    I. Higgins, L. Matthey, A. Pal, C. Burgess, X. Glorot, M. Botvinick, S. Mohamed, and A. Lerchner, “beta-vae: Learning basic visual concepts with a constrained variational framework,” inInternational conference on learning representations, 2016

    2016

  52. [52]

    Stochastic tra- jectory prediction via motion indeterminacy diffusion,

    T. Gu, G. Chen, J. Li, C. Lin, Y . Rao, J. Zhou, and J. Lu, “Stochastic tra- jectory prediction via motion indeterminacy diffusion,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 17 113–17 122

    2022

  53. [53]

    The trajectron: Probabilistic multi-agent trajectory modeling with dynamic spatiotemporal graphs,

    B. Ivanovic and M. Pavone, “The trajectron: Probabilistic multi-agent trajectory modeling with dynamic spatiotemporal graphs,” inProceed- ings of the IEEE/CVF International Conference on Computer Vision, 2019, pp. 2375–2384

    2019

  54. [54]

    Curobo: Parallelized collision-free robot motion generation,

    B. Sundaralingam, S. K. S. Hari, A. Fishman, C. Garrett, K. Van Wyk, V . Blukis, A. Millane, H. Oleynikova, A. Handa, F. Ramos,et al., “Curobo: Parallelized collision-free robot motion generation,” in2023 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2023, pp. 8112–8119

    2023

  55. [55]

    Efficient collision detection framework for enhancing collision-free robot motion,

    X. Zhu, Y . Xin, S. Li, H. Liu, C. Xia, and B. Liang, “Efficient collision detection framework for enhancing collision-free robot motion,” in2025 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2025, pp. 16 162–16 168

    2025

  56. [56]

    Shared autonomy via deep reinforcement learning,

    S. Reddy, A. D. Dragan, and S. Levine, “Shared autonomy via deep reinforcement learning,”arXiv preprint arXiv:1802.01744, 2018

    Pith/arXiv arXiv 2018

  57. [57]

    Development of nasa-tlx (task load index): Results of empirical and theoretical research,

    S. Hart, “Development of nasa-tlx (task load index): Results of empirical and theoretical research,”Human mental workload/Elsevier, 1988

    1988