Pith. sign in

REVIEW 3 major objections 6 minor 32 references

RL post-training of action-chunking robot policies raises contact success to 100% and cuts unsafe force events by 46× without adding latency or memory cost.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-13 01:54 UTC pith:RHH4BRIG

load-bearing objection Solid sim systems paper: chunk-level PPO on pretrained ACT with hybrid prior and encoder-value critic; force-safety gains are real in MuJoCo but reward-shaped and untransferred. the 3 major comments →

arxiv 2607.09590 v1 pith:RHH4BRIG submitted 2026-07-10 cs.RO cs.AI

PAC-ACT: Post-training Actor-Critic for Action Chunking Transformers

Yujie Pang , Zudong Li This is my paper
classification cs.RO cs.AI
keywords visuomotor controlaction chunkingreinforcement learning fine-tuningimitation learningprecision contact manipulationforce safetyactor-criticbehavior prior
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Industrial robots that must touch metal and rubber parts under pose error need policies that finish the job, stay force-safe, and run fast on modest hardware. Pure behavior-cloned action-chunking models learn smooth multi-step motions from demos but still fail under distribution shift and can produce dangerous contact spikes. PAC-ACT takes a pretrained Action Chunking Transformer, reframes learning as chunk-level decisions so credit matches how actions are generated, reuses the backbone as actor and critic, and adds hybrid KL-style priors that keep the policy near the expert manifold while online RL optimizes success and force. On Contour and related contact tasks the method lifts success from 60% to 100%, shortens completion, and reduces the share of force readings above 60 N by 46 times, all while keeping ~88 ms inference and ~2.3 GB GPU memory—far leaner than large vision-language-action models.

Core claim

A pretrained ACT policy can be stably fine-tuned with chunk-level PPO by transferring its encoder/decoder structure into an actor-critic, removing the CVAE, and constraining updates with hybrid behavior-prior terms; the resulting PAC-ACT policy improves task success, contact stability, and force safety on precision industrial contact benchmarks while preserving the low latency and low memory of the original vision-action chunking model.

What carries the argument

PAC-ACT: chunk-level MDP reformulation (execute c steps as one decision) plus an ACT-transferred actor-critic and hybrid KL/behavior-prior constraint that keeps online updates near the pretrained action manifold.

Load-bearing premise

The force-safety and robustness gains measured in simulation with force used only in the reward (not as a policy input) and with hand-tuned prior coefficients will transfer to real industrial contact without major redesign.

What would settle it

Deploy the same Contour policy on a physical arm with ±2 cm pose randomization and a force/torque sensor; if peak-force histograms and >60 N event rates do not improve over the pretrained ACT baseline under matched evaluation, the central safety claim fails.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. PAC-ACT is an RL post-training framework for pretrained Action Chunking Transformer (ACT) policies aimed at precision contact manipulation. The method reformulates PPO at the chunk level (c-step action chunks with inter-chunk discount Γ=γ^c), builds an ACT-transferred actor-critic that reuses the ACT visual/Transformer backbone while removing the CVAE and using an encoder-pool-MLP critic, and applies a hybrid behavior prior: adjacent-update KL (β1) plus a frozen-baseline MSE reward penalty (β2). On MuJoCo Metal Touch (Diamond/Cross/Contour) and robomimic Square Assembly, PAC-ACT raises Contour success from 60% to 100% and Square from 51.2% to 98.2% versus BC ACT, reduces completion steps ~2.8×, and on Contour cuts the fraction of force readings >60 N by ~46× (4.6%→0.1%) while keeping ~88 ms mean inference and ~2.3 GB GPU memory. Architecture ablations (flat-decoder critic, CVAE-retained actor) and a sparse-reward KL ablation with trajectory-structure diagnosis support the design choices.

Significance. If the results hold under the stated simulation protocol, the paper offers a practical, deployment-oriented path for task-specific refinement of lightweight vision-action chunking policies without adopting large VLA inference costs. Strengths include: (i) a clear structural alignment of ACT chunk generation with PPO updates; (ii) multi-task quantitative tables for success, efficiency, and force statistics over 50-episode evaluations; (iii) multi-seed notes (22/32/42); (iv) targeted architecture ablations and a sparse-reward ablation that diagnoses behavior-structure collapse when the KL prior is removed; and (v) explicit latency/memory comparison against ACT, Diffusion Policy, and π0.5. The work is relevant to industrial contact settings where BC distribution shift and force safety matter, provided claims about real-world force safety are appropriately scoped.

major comments (3)
  1. [Abstract, §IV-G, Table IV, Eq. (10)] Abstract, §I, and §IV-G / Table IV: the headline industrial “force safety” claim (e.g., 46× reduction of readings >60 N, median peak 105.4 N→20.74 N) is obtained under the dense reward of Eq. (10), which includes an explicit force-control penalty p_force whenever f_total leaves the safety band. Force/torque is never a policy input (only dual RGB + joint angles; §§IV-A–B)—it is a training-time reward signal and a post-hoc metric. The learned policy therefore acquires open-loop visual-motor associations that avoid high-force states inside the MuJoCo contact model. Discussion correctly flags unvalidated sim-to-real transfer, but the abstract and introduction still frame force safety as an industrial result. Either substantially qualify those claims to “simulation force-safety under reward shaping,” or provide real-robot / closed-loop force evidence; as written, the industrial force-safety c
  2. [§III-A–B, §IV-I] §III-A–B motivate chunk-level MDP reformulation from a “structural mismatch” between step-wise PPO ratios/advantages and ACT’s temporally coupled chunks (Eqs. 1–3). Architecture ablations (§IV-I) test CVAE retention and critic decoder design, but there is no load-bearing ablation of chunk-level PPO versus step-wise PPO (or step-wise GAE with chunk actions) on the same pretrained ACT backbone and reward. Without that comparison, it remains unclear whether the reported gains require the chunk-level reformulation or would arise from standard PPO fine-tuning plus the hybrid prior alone. A controlled step-wise baseline (or a clear failure case of step-wise updates) is needed to support the central methodological claim.
  3. [§III-D, Eq. (8)–(9), §IV-B] §III-D and Algorithm 1 fix hybrid prior coefficients at β1=3.0 and β2=2.0 (and task-specific log σ init) with limited sensitivity analysis in the main dense-reward setting. The sparse-reward ablation shows that removing β1 degrades structured contour behavior, which is useful, but does not establish robustness of the main Contour/Square results to β1/β2 or to the dense reward component weights in Eq. (10). Because the hybrid prior is presented as essential for stable fine-tuning near the BC manifold, a brief sensitivity or grid study (or at least multi-seed variation of β1/β2) is needed so that force-safety and success gains are not tied to a single hand-chosen regularizer setting.
minor comments (6)
  1. [Table I] Table I footnote notes that ACT and Diffusion Policy both report 60% on Contour as coincidence; consider reporting confidence intervals or bootstrap intervals for all success rates (n=50) so readers can judge separation from π0.5 (79%) and PAC-ACT (100%).
  2. [§IV-B, Eq. (10)] §IV-B: reward component weights (touch, dist, smooth, orient, time, force, success) are described qualitatively but not fully specified numerically in the main text; a short table or appendix listing coefficients would aid reproducibility.
  3. [Fig. 4, Fig. 6] Fig. 4/6 success curves would benefit from explicit seed-averaged mean ± std (or min–max) rather than only local fluctuation shading, consistent with the three-seed statement elsewhere.
  4. [§II-B] Related work (§II-B) correctly distances concurrent chunk RL (Chunking the Critic, Q-Chunking, AC3) as non-direct baselines; a short qualitative table of setting differences (from-scratch vs pretrained, offline data, reward sparsity) would make that positioning easier to scan.
  5. [§III-B–D] Notation: probability ratio is written both ρτ(θ) and rt(θ)/ρt(θ) across §III; unify symbols for the PPO ratio to avoid confusion with reward rt.
  6. [Throughout / §II / §IV-F] Typographical: “CV AE” appears with a space throughout; standardize to “CVAE”. “RELATEDWORK” heading spacing; “F . Analysis” extra space after F.

Circularity Check

0 steps flagged

No significant circularity: empirical RL post-training results measured on environment rollouts, not derived by redefining targets as fits or self-citations.

full rationale

PAC-ACT is a systems/empirical paper. The claimed gains (Contour success 60%→100%, force readings >60 N reduced 46×, completion steps ~2.8× lower, latency/memory retained) are obtained by running chunk-level PPO on a frozen-pretrained ACT actor plus encoder-value critic, then evaluating deterministic rollouts under randomized initial poses in MuJoCo/robosuite. The hybrid prior (β1 KL between successive policies + β2 MSE to the frozen BC baseline in the reward) is ordinary trust-region/behavior regularization; it does not make the measured success or force statistics equal the inputs by construction. The dense reward of Eq. (10) does contain an explicit p_force term that targets the safety band used in Table IV, but that is reward shaping, not a tautology: the policy never observes force (only RGB + joints), RL can fail to improve or can trade off other terms, and the same metrics are reported for pure-BC baselines that never saw p_force. Sparse-reward ablations further show the prior is necessary for structured exploration rather than defining the outcome. No uniqueness theorems, ansatzes, or load-bearing results are imported via self-citation; references are standard external literature (ACT, PPO, Diffusion Policy, etc.). No parameters are fitted to a subset and then “predicted” on a near-identical quantity. The derivation chain is therefore self-contained experimental comparison, not circular. (Sim-to-real and open-loop force claims are validity concerns outside the circularity criteria.)

Axiom & Free-Parameter Ledger

6 free parameters · 5 axioms · 3 invented entities

Load-bearing content is mostly engineering choices and sim assumptions, not new physics entities. Free parameters (β1, β2, logσ init, chunk length, learning rates, dense reward weights) are tuned for stable fine-tuning and directly affect success/force outcomes. Domain axioms include standard MDP/PPO validity, MuJoCo contact fidelity as proxy for industrial force safety, and that a frozen BC prior plus MSE/KL penalties keep exploration safe without force feedback in the policy. Invented entities are methodological constructs (PAC-ACT stack, hybrid behavior-prior constraint, ACT-transferred critic), not new physical objects; independent evidence is only the paper’s own sim ablations.

free parameters (6)
  • β1 (adjacent PPO KL coefficient)
    Set to 3.0; controls trust-region strength in high-dimensional chunk action space; sparse-reward ablation shows removing related prior collapses structure.
  • β2 (baseline MSE behavior-prior reward penalty)
    Set to 2.0; subtracts ||aθ−abase||² from reward; central to keeping policy near pretrained manifold during online fine-tuning.
  • logσ initialization
    Hand-set to −5.5 (Metal Touch) or −3.0 (Square Assembly) to match action scales and start narrow exploration around pretrained means.
  • chunk execution length c
    Changed from pretrained c0=10 to c=8 for RL; alters temporal unit of MDP and exploration difficulty.
  • dense reward component weights (touch, dist, smooth, orient, time, force, success)
    Milestone and force-penalty design drives reported efficiency and force-safety gains; sparse ablation retains only success=12.0.
  • actor/critic learning rates and PPO epochs
    1e-6 / 1e-5, 8 epochs, minibatch 32, 16 envs—standard but claim-sensitive training knobs without automatic selection.
axioms (5)
  • domain assumption Chunk-level MDP with Rτ = sum of c step rewards and Γ=γ^c correctly aligns credit assignment with ACT chunk generation for PPO.
    §III-A/B reformulates step-wise RL; assumed sufficient without formal optimality proof vs step-wise alternatives.
  • domain assumption PPO clipped surrogate plus empirical KL is a valid stable update rule in c×da continuous chunk action space.
    Standard RL assumption invoked throughout Algorithm 1 and §III-D.
  • domain assumption MuJoCo Metal Touch / robosuite Square contact forces and ±2 cm pose randomization adequately proxy industrial pose error and force safety.
    All main claims rest on these sims; Discussion states sim-to-real unvalidated.
  • ad hoc to paper Removing CVAE and modeling a diagonal Gaussian over chunk actions preserves useful pretrained temporal structure while enabling RL exploration.
    §III-C argues CVAE conflicts with β2 penalty; supported by ablation but is a design axiom of the method.
  • domain assumption Force/torque may be used in reward during training but omitted from policy observations at inference without invalidating force-safety claims.
    §IV-A: sensors for reward/eval only; force safety is therefore reward-shaped, not closed-loop force control.
invented entities (3)
  • PAC-ACT framework (chunk-level PPO + ACT-transferred actor-critic + hybrid behavior prior) no independent evidence
    purpose: Name and package the post-training stack claimed to fix BC distribution shift for industrial contact while preserving ACT efficiency.
    Methodological composite; evidence is internal sim experiments, not an external independent construct.
  • Hybrid KL-regularized objective (adjacent KL on updates + frozen-baseline MSE reward penalty) no independent evidence
    purpose: Prevent drift from pretrained action manifold during online fine-tuning, especially under sparse rewards.
    Combination is paper-specific; ablations support necessity of the prior under sparse reward, but no external theory or third-party replication.
  • Encoder-value Critic (ACT encoder + pool + MLP, no decoder) no independent evidence
    purpose: Lightweight state-value estimate reusing pretrained ACT features without action decoding.
    Architectural choice validated by ablation vs flat-decoder critic inside this paper only.

pith-pipeline@v1.1.0-grok45 · 21461 in / 4229 out tokens · 46465 ms · 2026-07-13T01:54:46.186827+00:00 · methodology

0 comments
read the original abstract

Precision industrial contact manipulation requires reliable robot policies under pose perturbations and contact-force constraints. Vision-language-action models offer broad generalization but often introduce high inference latency and GPU-memory cost, while vision-action chunking policies are more suitable for real-time industrial control. However, these policies are usually trained by behavior cloning and suffer from distribution shift in contact-rich tasks. This paper proposes PAC-ACT, a reinforcement-learning post-training framework for pretrained Action Chunking Transformer policies. PAC-ACT reformulates policy optimization at the chunk level, constructs an ACT-transferred actor-critic architecture, and introduces a hybrid behavior-prior constraint to preserve the pretrained action distribution during online fine-tuning. Experiments on industrial precision-contact benchmarks show that PAC-ACT improves task success, contact stability, and force safety while retaining low latency and low GPU-memory usage. On the Contour task, PAC-ACT significantly reduces peak contact force and decreases the proportion of force readings above 60 N by 46 times. Sparse-reward ablations further show that the proposed behavior-prior constraint enables effective exploration under randomized initial poses.

Figures

Figures reproduced from arXiv: 2607.09590 by Yujie Pang, Zudong Li.

Figure 1
Figure 1. Figure 1: Overall network architecture of PAC-ACT. The PPO post-training framework is built from a pretrained ACT model. The Actor branch preserves the [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Metal Touch precision contact task environment and randomized ini [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Success-rate curves of ACT behavior cloning under different demon [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Success rates of different policies across manipulation tasks and the convergence process of PAC-ACT. (a) Task-level success rate comparison [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Comparison between ACT and PAC-ACT in force control and task [PITH_FULL_IMAGE:figures/full_fig_p009_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Training-process comparison for architecture ablations. Curves show [PITH_FULL_IMAGE:figures/full_fig_p010_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Training success rate and end-effector trajectory structure diagnosis [PITH_FULL_IMAGE:figures/full_fig_p011_7.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

32 extracted references · 1 canonical work pages

  1. [1]

    Visual servo control. I. Basic ap- proaches,

    F. Chaumette and S. Hutchinson, “Visual servo control. I. Basic ap- proaches,”IEEE Robotics & Automation Magazine, vol. 13, no. 4, pp. 82–90, 2006, doi: 10.1109/MRA.2006.250573

    doi:10.1109/mra.2006.250573 2006

  2. [2]

    Measurement errors in visual servoing,

    V . Kyrki, D. Kragic, and H. I. Christensen, “Measurement errors in visual servoing,”Robotics and Autonomous Systems, vol. 54, no. 10, pp. 815–827, 2006, doi: 10.1016/j.robot.2006.05.002

    work page doi:10.1016/j.robot.2006.05.002 2006

  3. [3]

    Learning fine-grained bimanual manipulation with low-cost hardware,

    T. Z. Zhao, V . Kumar, S. Levine, and C. Finn, “Learning fine-grained bimanual manipulation with low-cost hardware,” inProc. Robotics: Science and Systems (RSS), 2023

    2023

  4. [4]

    A reduction of imitation learning and structured prediction to no-regret online learning,

    S. Ross, G. Gordon, and D. Bagnell, “A reduction of imitation learning and structured prediction to no-regret online learning,” inProc. Inter- national Conference on Artificial Intelligence and Statistics (AISTATS), PMLR, vol. 15, pp. 627–635, 2011

    2011

  5. [5]

    What matters in learning from offline human demonstrations for robot manipulation,

    A. Mandlekar, D. Xu, J. Wong, S. Nasiriany, C. Wang, R. Kulkarni, L. Fei-Fei, et al., “What matters in learning from offline human demonstrations for robot manipulation,” inProc. Conference on Robot Learning (CoRL), PMLR, vol. 164, pp. 1678–1690, 2022

    2022

  6. [6]

    A survey of robot manipulation in contact,

    M. Suomalainen, Y . Karayiannidis, and V . Kyrki, “A survey of robot manipulation in contact,”Robotics and Autonomous Systems, vol. 156, Art. no. 104224, 2022, doi: 10.1016/j.robot.2022.104224

    doi:10.1016/j.robot.2022.104224 2022

  7. [7]

    Impedance control: An approach to manipulation: Part I— Theory,

    N. Hogan, “Impedance control: An approach to manipulation: Part I— Theory,”Journal of Dynamic Systems, Measurement, and Control, vol. 107, no. 1, pp. 1–7, 1985, doi: 10.1115/1.3140702

    doi:10.1115/1.3140702 1985

  8. [8]

    Chunking the critic: A transformer-based soft actor-critic with n-step returns,

    D. Tian, O. Celik, and G. Neumann, “Chunking the critic: A transformer-based soft actor-critic with n-step returns,” arXiv preprint arXiv:2503.03660, 2025

    Pith/arXiv arXiv 2025

  9. [9]

    Reinforcement learning with action chunking,

    Q. Li, Z. Zhou, and S. Levine, “Reinforcement learning with action chunking,” inProc. Advances in Neural Information Processing Systems (NeurIPS), 2025

    2025

  10. [10]

    LeRobot: An open-source library for end-to-end robot learning,

    R. Cadene, S. Alibert, F. Capuano, M. Aractingi, A. Zouitine, P. Kooijmans, et al., “LeRobot: An open-source library for end-to-end robot learning,” inProc. International Conference on Learning Rep- resentations (ICLR), 2026

    2026

  11. [11]

    Learning structured output representation using deep conditional generative models,

    K. Sohn, H. Lee, and X. Yan, “Learning structured output representation using deep conditional generative models,” inProc. Advances in Neural Information Processing Systems (NeurIPS), 2015

    2015

  12. [12]

    Diffusion policy: Visuomotor policy learning via action diffusion,

    C. Chi, S. Feng, Y . Du, Z. Xu, E. Cousineau, B. Burchfiel, and S. Song, “Diffusion policy: Visuomotor policy learning via action diffusion,” in Proc. Robotics: Science and Systems (RSS), 2023

    2023

  13. [13]

    Behavior transformers: Cloningkmodes with one stone,

    N. M. Shafiullah, Z. Cui, A. Altanzaya, and L. Pinto, “Behavior transformers: Cloningkmodes with one stone,” inProc. Advances in Neural Information Processing Systems (NeurIPS), 2022

    2022

  14. [14]

    RT-2: Vision-language-action models transfer web knowledge to robotic control,

    B. Zitkovich et al., “RT-2: Vision-language-action models transfer web knowledge to robotic control,” inProc. Conference on Robot Learning (CoRL), PMLR, vol. 229, pp. 2165–2183, 2023

    2023

  15. [15]

    π0.5: a vision-language-action model with open-world generalization,

    Physical Intelligence, K. Black, N. Brown, J. Darpinian, et al., “π0.5: a vision-language-action model with open-world generalization,” arXiv preprint arXiv:2504.16054, 2025

    Pith/arXiv arXiv 2025

  16. [16]

    Implicit behavioral cloning,

    P. Florence, C. Lynch, A. Zeng, et al., “Implicit behavioral cloning,” in Proc. Conference on Robot Learning (CoRL), 2021

    2021

  17. [17]

    Coarse-to-fine Q-network with action se- quence for data-efficient reinforcement learning,

    Y . Seo and P. Abbeel, “Coarse-to-fine Q-network with action se- quence for data-efficient reinforcement learning,” arXiv preprint arXiv:2411.12155, 2024

    arXiv 2024

  18. [18]

    Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor,

    T. Haarnoja et al., “Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor,” inProc. International Conference on Machine Learning (ICML), 2018

    2018

  19. [19]

    Actor-critic for continuous action chunks: A reinforcement learning framework for long-horizon robotic manipulation with sparse reward,

    J. Yang, B. Zhu, J. Chen, and Y .-G. Jiang, “Actor-critic for continuous action chunks: A reinforcement learning framework for long-horizon robotic manipulation with sparse reward,” inProc. AAAI Conference on Artificial Intelligence (AAAI), vol. 40, no. 22, pp. 18692–18700, 2026, doi: 10.1609/aaai.v40i22.38937

    doi:10.1609/aaai.v40i22.38937 2026

  20. [20]

    Conservative Q- learning for offline reinforcement learning,

    A. Kumar, A. Zhou, G. Tucker, and S. Levine, “Conservative Q- learning for offline reinforcement learning,” inProc. Advances in Neural Information Processing Systems (NeurIPS), 2020

    2020

  21. [21]

    Offline reinforcement learning with implicit Q-learning,

    I. Kostrikov, A. Nair, and S. Levine, “Offline reinforcement learning with implicit Q-learning,” inProc. International Conference on Learning Representations (ICLR), 2022

    2022

  22. [22]

    Safe learning in robotics: From learning-based control to safe reinforcement learning,

    L. Brunke, M. Greeff, A. W. Hall, Z. Yuan, S. Zhou, J. Panerati, and A. P. Schoellig, “Safe learning in robotics: From learning-based control to safe reinforcement learning,”Annual Review of Control, Robotics, and Autonomous Systems, vol. 5, pp. 411–444, 2022, doi: 10.1146/annurev- control-042920-020211

    doi:10.1146/annurev- 2022

  23. [23]

    Open X-Embodiment: Robotic learning datasets and RT-X models,

    Open X-Embodiment Collaboration, A. O’Neill, A. Rehman, et al., “Open X-Embodiment: Robotic learning datasets and RT-X models,” inProc. IEEE International Conference on Robotics and Automation (ICRA), pp. 6892–6903, 2024, doi: 10.1109/ICRA57147.2024.10611477

    doi:10.1109/icra57147.2024.10611477 2024

  24. [24]

    Diffusion policy policy optimization,

    A. Z. Ren, J. Lidard, L. L. Ankile, A. Simeonov, P. Agrawal, A. Majumdar, et al., “Diffusion policy policy optimization,” arXiv preprint arXiv:2409.00588, 2024

    Pith/arXiv arXiv 2024

  25. [25]

    Deep residual learning for image recognition,

    K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” inProc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2016

    2016

  26. [26]

    Attention is all you need,

    A. Vaswani et al., “Attention is all you need,” inProc. Advances in Neural Information Processing Systems (NeurIPS), 2017

    2017

  27. [27]

    R3M: A universal visual representation for robot manipulation,

    S. Nair, A. Rajeswaran, V . Kumar, C. Finn, and A. Gupta, “R3M: A universal visual representation for robot manipulation,” inProc. Conference on Robot Learning (CoRL), PMLR, vol. 205, pp. 892–909, 2023

    2023

  28. [28]

    Prox- imal policy optimization algorithms,

    J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Prox- imal policy optimization algorithms,” arXiv preprint arXiv:1707.06347, 2017

    Pith/arXiv arXiv 2017

  29. [29]

    Trust re- gion policy optimization,

    J. Schulman, S. Levine, P. Abbeel, M. Jordan, and P. Moritz, “Trust re- gion policy optimization,” inProc. International Conference on Machine Learning (ICML), 2015

    2015

  30. [30]

    MuJoCo: A physics engine for model-based control,

    E. Todorov, T. Erez, and Y . Tassa, “MuJoCo: A physics engine for model-based control,” inProc. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2012

    2012

  31. [31]

    robosuite: A modular simulation framework and benchmark for robot learning,

    Y . Zhu, J. Wong, A. Mandlekar, R. Martin-Martin, A. Joshi, K. Lin, et al., “robosuite: A modular simulation framework and benchmark for robot learning,” arXiv preprint arXiv:2009.12293, 2020

    Pith/arXiv arXiv 2009

  32. [32]

    DeeR-VLA: Dynamic inference of multimodal large language models for efficient robot execution,

    Y . Yue, Y . Wang, B. Kang, Y . Han, S. Wang, S. Song, J. Feng, and G. Huang, “DeeR-VLA: Dynamic inference of multimodal large language models for efficient robot execution,” inProc. Advances in Neural Information Processing Systems (NeurIPS), 2024

    2024