Pith. sign in

REVIEW 4 major objections 5 minor 55 references

With just 15 AR-teleoperated demonstrations, dexterous-grasping RL trains about four times faster and reaches higher success rates than standard actor-critic methods.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 03:37 UTC pith:NUMTPH6R

load-bearing objection A plausible two-phase BC+contrastive-SAC pipeline for dexterous grasping with an AR teleop system, but the 4x speedup and collapse-prevention claims rest on a single unseeded comparison and 15 unanalyzed demonstrations. the 4 major comments →

arxiv 2602.07341 v2 pith:NUMTPH6R submitted 2026-02-07 cs.LG cs.RO

Scalable Dexterous Robot Learning with AR-based Remote Human-Robot Interactions

Yicheng Yang , Ruijiao Li , Lifeng Wang , Shuai Zheng , Shunzheng Ma , Keyu Zhang , Tuoyu Sun , Chenyun Dai
show 2 more authors
Jie Ding Zhuo Zou
This is my paper
classification cs.LG cs.RO
keywords augmented reality teleoperationbehavior cloningcontrastive learningreinforcement learningdexterous manipulationsoft actor-criticsim-to-real transferimitation learning
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper claims that the bottleneck of dexterous manipulation RL — long training time and policy collapse — can be broken by combining two ingredients: a behavior-cloning pretraining phase built from a handful of teleoperated demonstrations, and a contrastive auxiliary loss that keeps the RL policy's actions close to expert actions for the same states. The demonstrations are collected through an AR-based remote teleoperation system, so the data collection is meant to be scalable. In PyBullet simulations and real-world trials, the proposed two-phase algorithm converges in roughly a quarter of the training time of plain soft actor-critic, reaches about 92% success on ball and bottle grasping versus about 84% for SAC, and avoids the policy collapse that the authors observe in imitation-pretrained RL baselines. A sympathetic reader would take away that a very small amount of expert data can be leveraged much more strongly than by simple BC warm-starting.

Core claim

Starting from 15 expert trajectories collected by an AR teleoperation setup, the authors first train the actor by regression (behavior cloning), then fine-tune a SAC agent while a projection head projects (state, action) pairs of both the expert and the current actor into a shared latent space; a contrastive loss maximizes agreement between the expert's pair and the actor's pair for the same state. This contrastive constraint is what distinguishes the method from BC+SAC, and the ablation indicates it prevents the policy from collapsing during RL fine-tuning. The paper reports roughly 4x faster convergence than vanilla SAC (55-75 minutes vs 270-300 minutes), higher mean rewards and success ra

What carries the argument

The load-bearing mechanism is the projection head: a two-layer MLP that maps concatenated expert and actor (state, action) pairs into a 128-dimensional latent space. A temperature-scaled InfoNCE-style contrastive loss (Eq. 12) treats the expert's and the actor's representation for the same expert state as a positive pair, pulling the two together. This operates as a regularizer on the SAC policy objective (Eq. 11), added with weight 0.5, and is what the ablation isolates when comparing BC+SAC against the full method. The first phase, BC pretraining, provides the initialization; the contrastive term is the mechanism that prevents collapse and accelerates convergence.

Load-bearing premise

The 15 expert demonstrations are good enough — representative, correct, and sufficiently diverse — to serve both as the BC pretraining target and as the contrastive anchors; if they are noisy or biased, the policy is actively pulled toward suboptimal behavior.

What would settle it

Ablate demonstration quality: take the same 15 trajectories, corrupt a random 30% of the actions with noise (or replace them with random actions), and retrain. If the proposed method still reaches ~90% success and 4x speedup, then the contrastive mechanism must be doing the work regardless of demo fidelity; if performance collapses, the claim's dependence on high-quality demos is confirmed. Also, train BC+SAC without contrastive learning on 15 demos and then with 100 demos: if the contrastive gain disappears when demos are plentiful, the mechanism's value is specifically in the low-data regime

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Behavior cloning pretraining accounts for most of the training-time reduction; contrastive learning adds a smaller but consistent performance gain.
  • With a tiny demonstration set (15 trajectories), the method outperforms both PPO and SAC in success rate on two grasping tasks, suggesting demo-efficient policy learning is feasible.
  • The projection head can be discarded after RL training, so inference time is not increased by the auxiliary network.
  • The event-driven reward (success/collision/contact) that accompanies the contrastive loss is part of why the learned policy behaves safely in the real world.
  • The method is presented as task-agnostic within arm-hand manipulation, so the same pipeline could transfer to other dexterous skills beyond ball and bottle grasping.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the contrastive regularizer is the real collapse-preventer, a stripped-down version that uses only BC pretraining plus a simple L2 penalty on actor-expert action distance might achieve similar gains; the paper does not test this, so the projection head's specific design is not proven necessary.
  • The scalability claim rests on the AR teleoperation pipeline; a testable extension would be to vary demonstration count (e.g., 5, 15, 50) and see whether the contrastive gain shrinks as demos increase, which would clarify whether the method is demo-efficient or just under-powered at 15 demos.
  • The same two-phase recipe could apply to other off-policy RL algorithms than SAC, since neither the contrastive loss nor BC pretraining is SAC-specific; a direct test would be BC+contrastive with TD3 or DDPG.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes a two-phase learning framework for dexterous arm-hand grasping: Phase 1 performs behavior cloning (BC) pretraining on 15 expert trajectories collected through a HoloLens 2 based AR teleoperation system; Phase 2 trains a Soft Actor-Critic (SAC) policy with an auxiliary contrastive loss defined through a projection head over expert state-action pairs. The method is evaluated in PyBullet on ball and bottle grasping against PPO, SAC, and a BC+SAC ablation, plus 50-trial real-world experiments. The central claims are that the proposed method trains about four times faster than SAC, achieves the highest success rates, and that the contrastive component prevents 'policy collapse.'

Significance. If substantiated, the combination of a low-cost AR teleoperation interface, BC pretraining, and a contrastive auxiliary loss would be a practically useful recipe for reducing RL training time in high-DoF dexterous manipulation. The paper also provides a genuine ablation (BC+SAC without contrastive learning) and reports both simulated and physical experiments. However, the central quantitative claims are not currently established: the empirical comparison rests on single training runs without variance or statistical tests, the notion of convergence is left undefined, and the sensitivity of the method to the single 15-trajectory demonstration set is not examined. The contribution is therefore promising but needs substantially stronger empirical support before the advertised speed and robustness claims can be accepted.

major comments (4)
  1. [Section IV-A, Table III] The success-rate and time comparisons are reported from a single training run per algorithm. Table III reports means over 10^4 evaluation trials, but no standard deviations, confidence intervals, or number of seeds. The gap between BC+SAC and Ours on ball grasping (85.49% vs 91.80%) and the identical convergence time (75 min) cannot be distinguished from seed noise without multi-seed experiments. Please report mean ± std over at least 5 seeds and include statistical significance tests (e.g., paired bootstrap) for the success-rate and time differences.
  2. [Section IV-A, Table III / Fig. 6] The terms 'convergence' and 'Number of iterations until convergence' are never defined. Fig. 6 shows reward curves but no convergence criterion, no error bands, and no definition of when training is deemed converged. Moreover, the time-consumption column does not state whether the Phase 1 BC pretraining time is included. Since the 'approximately four times faster than SAC' claim is a headline result, please define the convergence rule explicitly and report wall-clock time excluding and including Phase 1 pretraining.
  3. [Section III, Eqs. (11)-(12) and Algorithm 1] The claim that contrastive learning 'overcomes policy collapse' is asserted repeatedly but never defined or demonstrated. No metric for policy collapse (e.g., action entropy, effective parameter norm, Q-value dispersion) is provided, and Fig. 6 does not compare collapse behavior. In addition, the contrastive loss draws positive pairs exclusively from the 15 expert trajectories. The paper does not analyze demonstration quality, diversity, or sensitivity to the number of demonstrations; if the demonstrations are noisy or biased, L_CL would actively attract the policy toward suboptimal behavior. Please add a formal or operational definition of policy collapse, quantitative evidence, and a demonstration-count/quality sensitivity study.
  4. [Section IV-A, Fig. 8] The real-world results are based on 50 trials per method with no confidence intervals or error bars. The discussion of 'marginal sim-to-real gap' for bottle grasping is not supported by any numerical comparison or statistical test. Please report exact success counts, binomial confidence intervals, and, if applicable, a test comparing the proposed method with BC+SAC, which is the closest ablation.
minor comments (5)
  1. [Eq. (6) and surrounding text] Typo: 'hand pose adjustment award' should be 'reward'. Also, Eq. (6) has a piecewise structure with Z4/Z5 but the text does not explain the intuition for the two regimes or how Λ_th was chosen.
  2. [Algorithm 1, lines 6-7] The update order is: projection head updated with L_CL, then the actor updated with L_πθ that includes ξ4 L_CL. Please clarify whether the projection head's parameters are treated as fixed or updated during the actor update, since L_CL depends on φ through h_A.
  3. [References] Reference [29] contains garbled characters ('Bj?rkman') and should be corrected. Also, the arXiv date (Feb 2026) and the use of very recent references (e.g., [53]-[55]) should be double-checked for correctness.
  4. [Section II-B] The statement that the system can 'capture diverse expert demonstrations' is not supported by any quantitative description of the 15 trajectories (e.g., length, variance across trajectories, number of successes/failures). A brief descriptive summary of the dataset would strengthen the paper.
  5. [Table I] The table lists α=1, but SAC's temperature is usually learned. If α is fixed, state so; if learned, explain how the value 1 was obtained. Similarly, the large number of hand-chosen reward weights (Z1-Z5, ξ1-ξ4, Λ_th) should be justified or at least motivated.

Circularity Check

0 steps flagged

No significant circularity: the central claim is an empirical comparison, and the only self-citation is a non-load-bearing background reference.

full rationale

The paper's central claim is an empirical result: in Table III and Section IV-A, the proposed BC-pretrained, contrastive-augmented SAC is reported to converge in about 75/55 minutes versus 300/270 minutes for SAC, with higher success rates on ball and bottle grasping. No step of the derivation reduces to its own inputs by construction. Phase 1 fits a behavior-cloning regression L_BC to expert demonstrations; Phase 2 adds a contrastive loss L_CL over the same demonstrations, but the reported success rates are evaluated on 10^4 simulated trials and 50 real-world trials, not on the training demonstrations themselves. The reward function in Eqs. (4)-(6) is hand-shaped with constants from Table I, but the reward is the optimization objective rather than a fitted predictor of the reported success-rate gaps, and the same task metrics are used for all algorithms. The only identifiable self-citation is reference [23], a co-author's 360-degree video delivery paper, used as background for the statement that 'in 6G, extended reality (XR) ... is one of the most transformative services'; it is not load-bearing for the proposed method or the evaluation. No uniqueness theorem, ansatz-by-citation, or renaming pattern appears. The absence of seed variance, error bars, and demonstration-quality analysis is a statistical robustness limitation, not circularity.

Axiom & Free-Parameter Ledger

12 free parameters · 4 axioms · 0 invented entities

The central claim rests on a heavily hand-tuned reward function, the quality of 15 AR-collected demonstrations, and the unproved efficacy of the contrastive projection head. No new physical entities are introduced. The free parameters are numerous and case-specific, limiting generality.

free parameters (12)
  • α (entropy temperature) = 1
    Temperature in the entropy-augmented reward (Eq. 3); set to 1 in Table I without sensitivity analysis.
  • ξ1 (smooth/progress reward weight) = 1000
    Weight on r_smooth in Eq. (4), chosen by hand; affects the shape of the reward and therefore the learned policy.
  • ξ2 (event reward weight) = 1
    Weight on r_succ in Eq. (4), chosen by hand.
  • ξ3 (pose reward weight) = 1
    Weight on r_pose in Eq. (4), chosen by hand.
  • ξ4 (contrastive loss weight) = 0.5
    Weight on L_CL in Eq. (11), chosen by hand; directly controls the influence of the contrastive term.
  • Z1 (success reward) = 1000
    Large positive event reward in Eq. (5); chosen by hand and directly boosts success signal.
  • Z2 (collision penalty) = 100
    Negative event reward in Eq. (5); chosen by hand.
  • Z3 (hand-object contact penalty) = 60
    Negative event reward in Eq. (5); chosen by hand.
  • Z4 (pose reward coefficient, lower regime) = 7
    Coefficient in Eq. (6) when cosψ ≤ Λ_th; chosen by hand.
  • Z5 (pose reward coefficient, upper regime) = 80
    Coefficient in Eq. (6) otherwise; chosen by hand.
  • Λ_th (geometric reasoning threshold) = 0.75
    Threshold in Eq. (6) for hand-object alignment; chosen by hand.
  • τ_CL (contrastive temperature) = 0.1
    Temperature in the InfoNCE loss (Eq. 12); chosen by hand.
axioms (4)
  • domain assumption The 15 expert demonstrations collected via AR are high-quality, representative, and sufficient for behavior cloning and contrastive anchoring.
    The entire two-phase method depends on the quality and representativeness of the demonstrations; no analysis of demo quality or sensitivity to the number of demos is provided.
  • ad hoc to paper The hand-crafted reward function (Eqs. 4-6) with its weights correctly encodes task success, safety, and desirable hand-object alignment.
    The weights are chosen without sensitivity analysis, and the reward directly shapes the measured success rates.
  • ad hoc to paper The contrastive loss with the projection head (Eqs. 11-12) prevents policy collapse and improves sample efficiency.
    This is a core design claim, but it is neither theoretically derived nor quantitatively demonstrated—the ablation shows only a small gap between BC+SAC and the full method.
  • domain assumption A policy trained in PyBullet transfers to the real robot with acceptable sim-to-real gap for the considered tasks.
    The real-world trials use the simulation-trained policy; the paper acknowledges a success-rate drop but does not quantify transfer robustness across task variations.

pith-pipeline@v1.3.0-alltime-deepseek · 12009 in / 12949 out tokens · 123913 ms · 2026-08-03T03:37:09.239759+00:00 · methodology

0 comments
read the original abstract

This paper focuses on the scalable robot learning for manipulation in the dexterous robot arm-hand systems, where the remote human-robot interactions via augmented reality (AR) are established to collect the expert demonstration data for improving efficiency. In such a system, we present a novel method to address the general manipulation task problem. Specifically, the proposed method consists of two phases: i) In the first phase for pretraining, the policy is created in a behavior cloning (BC) manner, through leveraging the learning data from our AR-based remote human-robot interaction system; ii) In the second phase, a contrastive learning empowered reinforcement learning (RL) method is developed to obtain more efficient and robust policy than the BC, and thus a projection head is designed to accelerate the learning progress. An event-driven augmented reward is adopted for enhancing the safety. To validate the proposed method, both the physics simulations via PyBullet and real-world experiments are carried out. The results demonstrate that compared to the baselines, our method not only significantly speeds up the training process, but also achieves much better performance in terms of the success rate for fulfilling the manipulation tasks. By conducting the ablation study, it is confirmed that the proposed RL with contrastive learning overcomes policy collapse. Supplementary demonstrations are available at https://cyberyyc.github.io/.

Figures

Figures reproduced from arXiv: 2602.07341 by Chenyun Dai, Jie Ding, Keyu Zhang, Lifeng Wang, Ruijiao Li, Shuai Zheng, Shunzheng Ma, Tuoyu Sun, Yicheng Yang, Zhuo Zou.

Figure 1
Figure 1. Figure 1: Illustration of a learning data collection system via AR: The expert [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Design structure of the proposed algorithm: Differing from the existing algorithms, the expert demonstration data are leveraged for behavior cloning [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: The proposed projection head for improving the sample-efficiency and preventing the policy collapse in the actor network. [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Dexterous robot manipulation tasks in the simulation environment and real-world. [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: The rollouts of object grasping via the proposed algorithm: The functional grasps are achieved in the end. [PITH_FULL_IMAGE:figures/full_fig_p006_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: The convergence of using different RL algorithms for object grasping. [PITH_FULL_IMAGE:figures/full_fig_p007_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: AR-based visual interface overlaid onto the real-world. [PITH_FULL_IMAGE:figures/full_fig_p008_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: The performance comparisons in the real-world trials. [PITH_FULL_IMAGE:figures/full_fig_p008_8.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

55 extracted references · 3 linked inside Pith

  1. [1]

    A roadmap for AI in robotics,

    A. Billard, A. Albu-Schaeffer, M. Beetzet al., “A roadmap for AI in robotics,”Nat. Mach. Intell., vol. 7, pp. 818–824, 2025

    2025

  2. [2]

    A survey of embodied AI: From simulators to research tasks,

    J. Duan, S. Yu, H. L. Tan, H. Zhu, and C. Tan, “A survey of embodied AI: From simulators to research tasks,”IEEE Trans. Emerg. Top. Comput. Intell., vol. 6, no. 2, pp. 230–244, Apr. 2022

    2022

  3. [3]

    DexPilot: Vision-based teleoper- ation of dexterous robotic hand-arm system,

    A. Handa, K. Van Wyk, W. Yang, J. Liang, Y .-W. Chao, Q. Wan, S. Birchfield, N. Ratliff, and D. Fox, “DexPilot: Vision-based teleoper- ation of dexterous robotic hand-arm system,” inIEEE Int. Conf.. Robot. Autom. (ICRA), 2020, pp. 9164–9170

    2020

  4. [4]

    A review of robot learning for manipulation: Challenges, representations, and algorithms,

    O. Kroemer, S. Niekum, and G. Konidaris, “A review of robot learning for manipulation: Challenges, representations, and algorithms,”J. Mach. Learn. Res., vol. 22, no. 1, Jan. 2021

    2021

  5. [5]

    Learning thin deformable object manipulation with a multisensory integrated soft hand,

    C. Zhao, C. Jiang, L. Luo, S. Yuan, Q. Chen, and H. Yu, “Learning thin deformable object manipulation with a multisensory integrated soft hand,”IEEE Trans. Robot., vol. 41, pp. 4606–4621, 2025

    2025

  6. [6]

    Enhancing dexterity in confined spaces: Real-time motion planning for multifingered in-hand manipulation,

    X. Gao, K. Yao, F. Khadivar, and A. Billard, “Enhancing dexterity in confined spaces: Real-time motion planning for multifingered in-hand manipulation,”IEEE Robot. Autom. Mag., vol. 31, no. 4, pp. 100–112, Sept. 2024

    2024

  7. [7]

    Deep imitation learning for complex manipulation tasks from virtual reality teleoperation,

    T. Zhang, Z. McCarthy, O. Jow, D. Lee, X. Chen, K. Goldberg, and P. Abbeel, “Deep imitation learning for complex manipulation tasks from virtual reality teleoperation,” inIEEE Int. Conf. Robot. Autom. (ICRA), 2018, pp. 5628–5635

    2018

  8. [8]

    Coarse-to-fine imitation learning: Robot manipulation from a single demonstration,

    E. Johns, “Coarse-to-fine imitation learning: Robot manipulation from a single demonstration,” inIEEE Int. Conf. Robot. Autom. (ICRA), 2021, pp. 4613–4619

    2021

  9. [9]

    HG-DAgger: Interactive imitation learning with human experts,

    M. Kelly, C. Sidrane, K. Driggs-Campbell, and M. J. Kochenderfer, “HG-DAgger: Interactive imitation learning with human experts,” inInt. Conf. Robot. Autom. (ICRA), 2019, pp. 8077–8083

    2019

  10. [10]

    Multistage cable routing through hierarchical imitation learning,

    J. Luo, C. Xu, X. Geng, G. Feng, K. Fang, L. Tan, S. Schaal, and S. Levine, “Multistage cable routing through hierarchical imitation learning,”IEEE Trans. Robot., vol. 40, pp. 1476–1491, Jan. 2024

    2024

  11. [11]

    Goal-conditioned dual-action imitation learning for dexterous dual-arm robot manipulation,

    H. Kim, Y . Ohmura, and Y . Kuniyoshi, “Goal-conditioned dual-action imitation learning for dexterous dual-arm robot manipulation,”IEEE Trans. Robot., vol. 40, pp. 2287–2305, Mar. 2024

    2024

  12. [12]

    Learning fine-grained bimanual manipulation with low-cost hardware,

    T. Z. Zhao, V . Kumar, S. Levine, and C. Finn, “Learning fine-grained bimanual manipulation with low-cost hardware,” inProc. Robot.: Sci. Syst., 2023

    2023

  13. [13]

    Behavior cloning-based active scene recognition via generated expert data with revision and prediction for domestic robots,

    S. Liu, C. Huang, and H. Huang, “Behavior cloning-based active scene recognition via generated expert data with revision and prediction for domestic robots,”IEEE Trans. Robot., vol. 41, pp. 4180–4194, 2025

    2025

  14. [14]

    Learning complex dexterous manipulation with deep reinforcement learning and demonstrations,

    A. Rajeswaran, V . Kumar, A. Gupta, G. Vezzani, J. Schulman, E. Todorov, and S. Levine, “Learning complex dexterous manipulation with deep reinforcement learning and demonstrations,” InRobot.: Sci. Syst. (RSS), 2018

    2018

  15. [15]

    DexRepNet: Learning dexterous robotic grasping network with geometric and spatial hand-object representations,

    Q. Liuet al., “DexRepNet: Learning dexterous robotic grasping network with geometric and spatial hand-object representations,” inIEEE Int. Conf. Intell. Rob. Syst., 2023, pp. 3153–3160

    2023

  16. [16]

    Teleoperation of humanoid robots: A survey,

    K. Darvish, L. Penco, J. Ramos, R. Cisneros, J. Pratt, E. Yoshida, S. Ivaldi, and D. Pucci, “Teleoperation of humanoid robots: A survey,” IEEE Trans. Robot., vol. 39, no. 3, pp. 1706–1727, June 2023

    2023

  17. [17]

    De- terministic networking empowered robotic teleoperation,

    C. Yang, H. Yu, Q. Guo, T. Taleb, J. C. Requena, and K. Tammi, “De- terministic networking empowered robotic teleoperation,”IEEE Netw., vol. 39, no. 4, pp. 280–289, 2025

    2025

  18. [18]

    Incremental learning of robotic manipulation tasks through virtual reality demonstrations,

    G. Rauso, R. Caccavale, and A. Finzi, “Incremental learning of robotic manipulation tasks through virtual reality demonstrations,” inIEEE/RSJ Int. Conf. Intell. Robot. Syst. (IROS), 2024, pp. 5176–5181. 9

    2024

  19. [19]

    Robot teleoperation with augmented reality virtual surrogates,

    M. E. Walker, H. Hedayati, and D. Szafir, “Robot teleoperation with augmented reality virtual surrogates,” inACM/IEEE Int. Conf. Human- Robot Interaction (HRI), 2019, pp. 202–210

    2019

  20. [20]

    Bunny-VisionPro: Real-time bimanual dexterous teleoperation for im- itation learning,

    R. Ding, Y . Qin, J. Zhu, C. Jia, S. Yang, R. Yang, X. Qi, and X. Wang, “Bunny-VisionPro: Real-time bimanual dexterous teleoperation for im- itation learning,”arXiv preprint arXiv:2407.03162, 2024

    Pith/arXiv arXiv 2024

  21. [21]

    Augmented reality and humancrobot collaboration framework for percutaneous nephrolithotomy: System de- sign, implementation, and performance metrics,

    J. Fu, M. Pecorella, E. Iovene, M. C. Palumbo, A. Rota, A. Redaelli, G. Ferrigno, and E. De Momi, “Augmented reality and humancrobot collaboration framework for percutaneous nephrolithotomy: System de- sign, implementation, and performance metrics,”IEEE Robot. Autom. Mag., vol. 31, no. 3, pp. 25–37, 2024

    2024

  22. [22]

    E-BTS: Event-based tactile sensor for haptic teleoperation in aug- mented reality,

    D. Mukashev, S. Seitzhan, J. Chumakov, S. Khajikhanov, M. Yergibay, N. Zhaniyar, R. Chibar, A. Mazhitov, M. Rubagotti, and Z. Kappassov, “E-BTS: Event-based tactile sensor for haptic teleoperation in aug- mented reality,”IEEE Trans. Robot., vol. 41, pp. 450–463, 2025

    2025

  23. [23]

    Two-tier 360-degree video delivery control in multiuser immersive communications systems,

    M. Hu, L. Wang, B. Tan, and S. Jin, “Two-tier 360-degree video delivery control in multiuser immersive communications systems,”IEEE Trans. V eh. Technol., vol. 72, no. 3, pp. 4119–4123, 2023

    2023

  24. [24]

    A survey of imitation learning: Algorithms, recent developments, and challenges,

    M. Zare, P. M. Kebria, A. Khosravi, and S. Nahavandi, “A survey of imitation learning: Algorithms, recent developments, and challenges,” IEEE Trans. Cybern., vol. 54, no. 12, pp. 7173–7186, Dec. 2024

    2024

  25. [25]

    Dexterous manipulation with deep reinforcement learning: Efficient, general, and low-cost,

    H. Zhu, A. Gupta, A. Rajeswaran, S. Levine, and V . Kumar, “Dexterous manipulation with deep reinforcement learning: Efficient, general, and low-cost,” inInt. Conf. Robot. Autom. (ICRA), 2019, pp. 3651–3657

    2019

  26. [26]

    Deep reinforcement learning for vision-based robotic grasping: A simulated comparative evaluation of off-policy methods,

    D. Quillen, E. Jang, O. Nachum, C. Finn, J. Ibarz, and S. Levine, “Deep reinforcement learning for vision-based robotic grasping: A simulated comparative evaluation of off-policy methods,” inIEEE Int. Conf. Robot. Autom. (ICRA), 2018, pp. 6284–6291

    2018

  27. [27]

    Scal- able deep reinforcement learning for vision based robotic manipulation,

    D. Kalashnikov, A. Irpan, P. Pastor, J. Ibarz, A. Herzog, E. Jang, D. Quillen, E. Holly, M. Kalakrishnan, V . Vanhoucke, and S. Levine, “Scal- able deep reinforcement learning for vision based robotic manipulation,” inProc. PMLR, 2018, pp. 651–673

    2018

  28. [28]

    Towards safe human-robot collaboration using deep reinforcement learning,

    M. El-Shamouty, X. Wu, S. Yang, M. Albus, and M. F. Huber, “Towards safe human-robot collaboration using deep reinforcement learning,” in IEEE Int. Conf. Robot. Autom. (ICRA), 2020, pp. 4899–4905

    2020

  29. [29]

    Human-centered collaborative robots with deep reinforcement learn- ing,

    A. Ghadirzadeh, X. Chen, W. Yin, Z. Yi, M. Bj?rkman, and D. Kragic, “Human-centered collaborative robots with deep reinforcement learn- ing,”IEEE Robot. Autom. Lett., vol. 6, no. 2, pp. 566–571, 2021

    2021

  30. [30]

    Task-oriented deep reinforcement learning for robotic skill acquisition and control,

    G. Xiang and J. Su, “Task-oriented deep reinforcement learning for robotic skill acquisition and control,”IEEE Trans. Cybern., vol. 51, no. 2, pp. 1056–1069, 2021

    2021

  31. [31]

    Dexterous pre-grasp manipulation for human-like functional categorical grasping: Deep reinforcement learning and grasp representations,

    D. Pavlichenko and S. Behnke, “Dexterous pre-grasp manipulation for human-like functional categorical grasping: Deep reinforcement learning and grasp representations,”IEEE Trans. Autom. Sci. Eng., early access, pp. 1–14, 2025

    2025

  32. [32]

    Reinforce- ment learning methods for assistive and rehabilitation robotic systems: A survey,

    M. Sharifi, S. Tripathi, Y . Chen, Q. Zhang, and M. Tavakoli, “Reinforce- ment learning methods for assistive and rehabilitation robotic systems: A survey,”IEEE Trans. Syst. Man. Cybern.: Syst., vol. 55, no. 7, pp. 4534–4551, 2025

    2025

  33. [33]

    Learning visual robotic control efficiently with contrastive pre-training and data augmentation,

    A. Zhan, R. Zhao, L. Pinto, P. Abbeel, and M. Laskin, “Learning visual robotic control efficiently with contrastive pre-training and data augmentation,” inIEEE/RSJ Int. Conf. Intell. Robot. Syst. (IROS), 2022, pp. 4040–4047

    2022

  34. [34]

    Masked world models for visual control,

    Y . Seo, D. Hafner, H. Liu, F. Liu, S. James, K. Lee, and P. Abbeel, “Masked world models for visual control,” inConf. Robot Learn. (CoRL), 2022

    2022

  35. [35]

    MoDem: Accelerating visual model-based reinforcement learning with demonstrations,

    N. Hansen, Y . Lin, H. Su, X. Wang, V . Kumar, and A. Rajeswaran, “MoDem: Accelerating visual model-based reinforcement learning with demonstrations,”Int. Conf. Learn. Rep. (ICLR), 2023

    2023

  36. [36]

    Imitation bootstrapped rein- forcement learning ,

    H. Hu, S. Mirchandani, and D. Sadigh, “Imitation bootstrapped rein- forcement learning ,” InRobot.: Sci. Syst. (RSS), 2024

    2024

  37. [37]

    Bootstrapping reinforcement learning with imitation for vision-based agile flight,

    J. Xing, A. Romero, L. Bauersfeld, and D. Scaramuzza, “Bootstrapping reinforcement learning with imitation for vision-based agile flight,”Conf. Robot Learn. (CoRL), 2024

    2024

  38. [38]

    Contrastive learning as goal-conditioned reinforcement learning,

    B. Eysenbach, T. Zhang, S. Levine, and R. Salakhutdinov, “Contrastive learning as goal-conditioned reinforcement learning,” inNeural Inf. Process. Syst. (NeurIPS), A. H. Oh, A. Agarwal, D. Belgrave, and K. Cho, Eds., 2022

    2022

  39. [39]

    Representation learning with contrastive predictive coding,

    A. van den Oord, Y . Li, and O. Vinyals, “Representation learning with contrastive predictive coding,” 2019

    2019

  40. [40]

    A simple framework for contrastive learning of visual representations,

    T. Chen, S. Kornblith, M. Norouzi, and G. Hinton, “A simple framework for contrastive learning of visual representations,” inInt. Conf. Mach. Learn. (ICML), 2020

    2020

  41. [41]

    CURL: Contrastive unsupervised representations for reinforcement learning,

    M. Laskin, A. Srinivas, and P. Abbeel, “CURL: Contrastive unsupervised representations for reinforcement learning,” inInt. Conf. Mach. Learn. (ICML), 2020, pp. 5639–5650

    2020

  42. [42]

    Masked contrastive representation learning for reinforcement learning,

    J. Zhu, Y . Xia, L. Wu, J. Deng, W. Zhou, T. Qin, T.-Y . Liu, and H. Li, “Masked contrastive representation learning for reinforcement learning,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 45, no. 3, pp. 3421–3433, 2023

    2023

  43. [43]

    Return-based contrastive representation learning for reinforcement learning,

    G. Liu, C. Zhang, L. Zhao, T. Qin, J. Zhu, L. Jian, N. Yu, and T.-Y . Liu, “Return-based contrastive representation learning for reinforcement learning,” inInt. Conf. Learn. Rep. (ICLR), 2021

    2021

  44. [44]

    Contrastive learning methods for deep reinforce- ment learning,

    D. Wang and M. Hu, “Contrastive learning methods for deep reinforce- ment learning,”IEEE Access, vol. 11, pp. 97 107–97 117, 2023

    2023

  45. [45]

    Contrastive UCB: Prov- ably efficient contrastive self-supervised learning in online reinforcement learning,

    S. Qiu, L. Wang, C. Bai, Z. Yang, and Z. Wang, “Contrastive UCB: Prov- ably efficient contrastive self-supervised learning in online reinforcement learning,”Int. Conf. Mach. Learn. (ICML), 2022

    2022

  46. [46]

    Contrastive energy prediction for exact energy-guided diffusion sampling in offline reinforcement learning,

    C. Lu, H. Chen, J. Chen, H. Su, C. Li, and J. Zhu, “Contrastive energy prediction for exact energy-guided diffusion sampling in offline reinforcement learning,”Int. Conf. Mach. Learn. (ICML), 2023

    2023

  47. [47]

    ContraDiff: Planning towards high return states via contrastive learn- ing,

    Y . Shan, Z. Zhu, T. Long, L. Qifan, Y . Chang, W. Zhang, and L. Yin, “ContraDiff: Planning towards high return states via contrastive learn- ing,” inInt. Conf. Learn. Rep. (ICLR), 2025

    2025

  48. [48]

    Stabilizing Contrastive RL: Techniques for robotic goal reaching from offline data,

    C. Zheng, B. Eysenbach, H. Walke, P. Yin, K. Fang, R. Salakhutdinov, and S. Levine, “Stabilizing Contrastive RL: Techniques for robotic goal reaching from offline data,” inInt. Conf. Learn. Rep. (ICLR), 2024

    2024

  49. [49]

    On-robot reinforcement learning with goal-contrastive rewards,

    O. Biza, T. Weng, L. Sun, K. Schmeckpeper, T. Kelestemur, Y . J. Ma, R. Platt, J. van de Meent, and L. L. S. Wong, “On-robot reinforcement learning with goal-contrastive rewards,” inIEEE Int. Conf. Robot. Autom. (ICRA), 2025

    2025

  50. [50]

    Prox- imal policy optimization algorithms,

    J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Prox- imal policy optimization algorithms,”arXiv preprint arXiv:1707.06347, 2017

    Pith/arXiv arXiv 2017

  51. [51]

    Soft Actor-Critic: Off- policy maximum entropy deep reinforcement learning with a stochastic actor,

    T. Haarnoja, A. Zhou, P. Abbeel, and S. Levine, “Soft Actor-Critic: Off- policy maximum entropy deep reinforcement learning with a stochastic actor,” inInt. Conf. Mach. Learn. (ICML), 2018, pp. 1861–1870

    2018

  52. [52]

    Real-time safe and smooth visual servoing for robot manipulators via reinforcement learning,

    J. Mao, J. Hu, X. Zhou, C. Zhang, J. Yang, and H. Wang, “Real-time safe and smooth visual servoing for robot manipulators via reinforcement learning,”IEEE Trans. Ind. Electron., pp. 1–12, 2025

    2025

  53. [53]

    Embedding high-resolution touch across robotic hands enables adaptive human-like grasping,

    Z. Zhao, W. Li, Y . Liet al., “Embedding high-resolution touch across robotic hands enables adaptive human-like grasping,”Nat. Mach. Intell., vol. 7, pp. 889–900, 2025

    2025

  54. [54]

    Polysim: Bridging the sim-to-real gap for humanoid control via multi-simulator dynamics randomization,

    Z. Lei, Z. Zhou, S. Yin, Y . Chen, Q. Xu, W. Li, Y . Wang, B. Tang, W. Jing, and S. Chen, “Polysim: Bridging the sim-to-real gap for humanoid control via multi-simulator dynamics randomization,”arXiv preprint arXiv:2510.01708, 2025

    arXiv 2025

  55. [55]

    Visual-geometry diffusion policy: Robust generalization via complementarity-aware mul- timodal fusion,

    Y . Tang, H. Geng, S. Zang, P. Abbeel, and J. Malik, “Visual-geometry diffusion policy: Robust generalization via complementarity-aware mul- timodal fusion,”arXiv preprint arXiv:2511.22445, 2025

    Pith/arXiv arXiv 2025