Pith. sign in

REVIEW 3 major objections 7 minor 50 references

GPA-RAM: Grasp-Pretraining Augmented Robotic Attention Mamba for Spatial Task Learning

T0 review · 3 major / 7 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read GPA-RAM claims that injecting frozen grasp-detector features into robot policies plus an attention-Mamba backbone raises average manipulation success to 87.5% on RLBench, 98% on ALOHA cube transfer, and 56% on bimanual insertion while…

desk verdict Solid imitation-learning engineering with a real but underspecified grasp-prior module; the SOTA headline is fragile because RAM alone sits below ARP+ and the detector's training data is undisclosed. read the letter →

arxiv 2504.19683 v4 pith:IKFELWPQ submitted 2025-04-28 cs.RO

Juyi Sheng , Yangjun Liu , Sheng Xu , Zhixin Yang , Tiantian Xu , Mengyuan Liu This is my paper · ORCID
classification cs.RO
keywords robotlearningimitation3Dmanipulationmulti-modalperceptiongrasppretrainingstatespacemodelsbimanualreal-timeroboticcontrol
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that end-to-end imitation policies can be made substantially more precise at grasping by re-using the grasp features already latent in their own task demonstrations. The mechanism is a Grasp-Pretraining Augmentation (GPA) module that fuses features from a pretrained grasp pose detector into the policy's spatial features, together with a hybrid Robotic Attention Mamba (RAM) backbone that keeps the added capacity from destroying real-time speed. On the 18-task RLBench benchmark the combined system reports an 87.5% average success rate, beating RVT2 by 8.2 points and ARP+ by 2.6; on ALOHA it reports 98% on cube transfer and 56% on bimanual insertion, up 12 and 40 points over ACT, at roughly 71 frames per second.

What carries the argument

The load-bearing machinery is the pair (GPA, RAM). GPA, Grasp-Pretraining Augmentation, is a module built around a pretrained grasp pose detector (M2T2 on RLBench, ResNet on ALOHA) whose output head is removed and whose body remains frozen or fine-tunable; its grasping features are concatenated with RAM's spatially aligned features in the spatial dimension and fused through a self-attention block called Pre-trained Location Fusion (PLoFusion). RAM, Robotic Attention Mamba, is a hybrid backbone that interleaves single-view self-attention blocks with Mamba selective state-space layers, processing virtual multi-view RGB-D images in a coarse-to-fine heatmap pipeline for discrete keyframe prediction and in a one-stage form for continuous action generation. The Mamba layers provide linear-time global context aggregation while the attention blocks preserve local spatial alignment, which is what lets the combined model keep both precision and real-time throughput.

What would settle it

Retrain the grasp pose detector on demonstrations of a held-out set of objects and tasks, then run the full GPA-RAM evaluation on the original RLBench and ALOHA tasks. If the average success rate falls back to the RAM-only 84.7% level (or below the 84.9% ARP+ baseline), the grasp-pretraining augmentation is memorizing its own training benchmark rather than supplying a transferable grasping prior.

Watch

Extended reading notes

Core claim

The central claim, on the paper's own terms, is that grasp precision is a bottleneck in fine-grained manipulation and that it can be fixed without new annotations or a separate grasping stage. GPA takes a pretrained grasp pose detector, removes its prediction head, and uses the remaining feature extractor (frozen or fine-tuned) to produce grasping features that are concatenated with the policy's spatially aligned features and fused by self-attention; this supplies detailed affordance cues that deep Transformer layers tend to wash out. RAM, the companion architecture, combines single-view attention with Mamba selective state-space layers to aggregate global scene context in linear time and refines a coarse heatmap into a fine position, so the augmented policy stays fast enough for real-time control. The reported consequences are the benchmark numbers cited in the abstract plus real-robot demonstrations on UR5 and ARX R5 systems.

Load-bearing premise

The gain depends on the pretrained grasp detector providing a generalizable grasp prior; the paper does not disclose whether the detector's training demonstrations overlap with the 100-demonstration task sets it is evaluated on, so if they do, the margin over RVT2 and ARP+ may reflect learned benchmark familiarity rather than transferable grasping skill.

Editorial extensions

If this is right

  • An existing Transformer-based policy can be upgraded with GPA by a feature-concatenation change rather than by adding a separate grasping stage, gaining precision without new grasp-pose annotations.
  • Attention-Mamba hybrids can replace pure Transformer encoders in discrete and continuous manipulation policies, preserving or improving accuracy while avoiding quadratic sequence cost; RAM alone already beats RVT2 by 5.4 points on RLBench.
  • On ALOHA, the policy's 71 FPS throughput is fast enough for closed-loop servoing at 50 Hz, so the gains in bimanual insertion (56% versus 16% for ACT) are achieved with real-time execution rather than offline planning.
  • Because the same GPA recipe works with two different detector backbones (M2T2 on RLBench, ResNet on ALOHA), the benefit is not tied to a particular grasp detector architecture.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension not pursued in the paper: pretrain the grasp detector on demonstrations whose objects and scenes are disjoint from the RLBench evaluation tasks; if the gains persist, the augmentation is transferable, and if not, part of the +2.6-point margin over ARP+ is benchmark familiarity rather than a general grasping prior.
  • Another implication left implicit: since GPA helps most on tasks where initial grasp accuracy dominates (insertions, stacking, sorting), the same feature-fusion recipe should transfer to other policy families such as diffusion policies, where gripper-conditioned generation could absorb the grasp prior differently.
  • The slight regression on placement-flexibility tasks suggests a phase-dependent or confidence-weighted fusion of GPA features could remove the trade-off, turning the reported limitation into a design handle for future work.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. The manuscript proposes GPA-RAM, a framework for vision-based imitation learning in fine-grained robotic manipulation. The Grasp-Pretraining Augmentation (GPA) module injects features from a pretrained grasp-pose detector into an imitation policy through a fusion module called PLoFusion, with the stated goal of improving initial grasping without additional grasp-pose data or annotation. The Robotic Attention Mamba (RAM) architecture replaces Transformer encoders with a hybrid attention/state-space model to achieve real-time inference. The paper reports an average 87.5% success rate on 18 RLBench tasks, outperforming RVT2 (79.3%) and ARP+ (84.9%), and 98% and 56% success on ALOHA cube transfer and bimanual insertion, respectively, at approximately 71 FPS. It also presents real-robot experiments on UR5 and ARX R5 systems. Ablations compare GPA-RAM against RAM, C-RAM, GPA-C-RAM, GPA-RVT2, and GPA-ACT across RLBench and ALOHA.

Significance. If the headline results hold under a clean pretraining protocol, the paper would be a solid empirical contribution: the GPA module is architecture-agnostic, the RAM module addresses a real efficiency bottleneck, and the evaluation spans two sim benchmarks and two physical robot platforms. The measured inference-speed advantage over ACT (71 FPS vs 61 FPS) and the modular ablations are concrete strengths. The central SOTA claim is conditional, however, on two load-bearing issues: the pretrained grasp detector may have been trained on the same benchmark tasks used for evaluation, and the model-selection protocol appears to choose checkpoints on test episodes. These issues directly affect whether the 2.6-point margin over ARP+ on RLBench and the GPA-induced gains on ALOHA are due to transferable grasp priors or to benchmark-specific leakage and favorable selection.

major comments (3)
  1. [Section III.D and IV.C, Eq. (15)] The GPA feature F_GPA = phi_GPA(O_t, F_L, F_RAM) in Eq. (15) comes from a 'pre-trained grasp pose detector,' but the paper never discloses the detector's training data. Section IV.C states that M2T2 is used for RLBench and a single-task ResNet for ALOHA; M2T2 is a model trained on RLBench-style pick-and-place demonstrations, and the ResNet is trained on ALOHA demonstrations. If these detectors were pretrained on the same task distribution (or the same demonstrations) used for the 100-demo RLBench and 50-demo ALOHA policy training sets, then the GPA gains, including the +2.8% over RAM on RLBench and the +24% on ALOHA insertion, may reflect task-specific grasp knowledge rather than a transferable grasp prior. The paper must report: (i) the exact detector training split and its overlap with the policy training sets; (ii) whether the detector is frozen or fine-tuned in the main experiments (Section III.D says it 'can be frozen or fine-tuned,' without specifying); and (iii) an experiment with a grasp detector pretrained on object sets or tasks disjoint from the evaluation benchmark. Without this, the 'without additional grasp-pose data' framing is not established, and the SOTA margin over ARP+ is confounded.
  2. [Section IV.C, model selection] The implementation details state: 'Each task is trained on 100 demonstrations and tested four times on 25 unseen scenarios, using models from the final 30 training epochs. The best-performing model is selected for the final success rate comparison.' Selecting the checkpoint with the highest success on the test episodes is a form of test-set model selection, which inflates the reported success rate relative to a protocol that chooses a checkpoint on held-out validation episodes. Because the headline claim is the 2.6-point gap between GPA-RAM (87.5%) and ARP+ (84.9%), this selection procedure must be changed or justified. Please re-evaluate all methods under a validation-based early-stopping rule, or report the mean and standard deviation over all runs with an identical selection rule for every baseline and for all ablations.
  3. [Section IV.E and Table I] The paper does not quantify the uncertainty of the key comparisons. Given the reported protocol of four runs of 25 unseen scenarios per task, the per-task success rates are based on 100 episodes, and the paper reports no confidence intervals, standard errors, or hypothesis tests for the average success rates. The 2.6-point difference between GPA-RAM and ARP+ is small relative to the sampling noise from this protocol, and the model-selection issue in Section IV.C makes the point estimate additionally biased. Please report the number of independent training runs, the standard deviation across runs, the number of test episodes per task, and, where possible, paired comparisons across the 18 tasks so that the SOTA claim has a defensible statistical basis.
minor comments (7)
  1. [Eqs. (7)-(11)] The state-space model equations are not written correctly: Eq. (7) appears to omit the time derivative and should be h'(t) = A h(t) + B x(t); Eqs. (9)-(10) also conflate the continuous-time matrix A with the discretized matrix. Please correct the notation.
  2. [Table II and Section IV.F] The column header 'Inference Time (fps)' is inconsistent with the reported values, which are frame rates, not times. Rename it to 'Inference Speed (FPS)' and specify whether the FPS is measured for the policy forward pass only or includes image rendering.
  3. [Abstract and Section III.D] The claim that GPA works 'without additional grasp-pose data or annotation' appears to conflict with Section III.D, where the detector is pretrained using grasp poses extracted from expert demonstrations. Clarify whether this means no new data collection beyond the existing demonstrations, or no additional labels of any kind.
  4. [Section III.D, Eq. (15)] The module name 'PLoFusion' is introduced but never defined concretely; please provide the exact concatenation and attention operations, including feature dimensions, so that Eq. (15) is reproducible.
  5. [Section IV.C and Table VI] The selection rule differs between benchmarks: for RLBench, 'the best-performing model is selected' from the final 30 epochs, while for ALOHA, 'the best model during training is chosen.' Specify whether these procedures were applied identically to all baselines and ablations, and describe how the 50 test trials per ALOHA task are used for model selection.
  6. [Tables III and IV] The real-world results are based on only 10 trials per task; please report exact counts (for example, 7/10) and, if used, any confidence intervals, so the reader can assess the reliability of the real-robot comparisons.
  7. [Introduction, contributions] The introduction reports a '26.0%' improvement on ALOHA bimanual continuous tasks, while the abstract reports 12% and 40% improvements for the two tasks; state the aggregation rule (presumably the mean of the two per-task deltas) so the numbers are unambiguous.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: GPA-RAM's headline gains are measured on held-out episodes and external benchmarks, and the GPA module's detector is an external pretrained feature extractor, not a parameter fitted to the reported success rates.

full rationale

The central claims are empirical outcomes on RLBench, ALOHA, and real-robot trials, not quantities defined by the paper's equations. Equation (15) defines F_GPA = phi_GPA(O_t, F_L, F_RAM) as a feature-fusion operation, and Equations (16)-(23) train the action heads end-to-end against ground-truth actions; no equation defines success rate as the detector's output or as a fitted parameter. The GPA module concatenates features from externally published detectors (M2T2 [49] for RLBench and ResNet [50] for ALOHA) that are pretrained on grasp poses extracted from expert demonstrations, and the paper does not disclose the detector's training data or its overlap with the policy training demonstrations. That omission is a potential data-leakage or benchmark-contamination concern, and therefore a correctness risk, but it is not circularity under the required standard: the detectors are not fitted to the test episodes, their features do not by construction determine the held-out success rates, and the +2.8 point RAM-to-GPA-RAM ablation is an empirically measured difference, not an identity. The paper's own limitation statement that GPA slightly hurts 'Place Cups' and 'Push Buttons' further shows the augmentation is not definitionally forced to improve every reported result. There is no load-bearing self-citation chain: the key baselines and detectors (RVT2, ARP+, ACT, M2T2, ResNet) are external works, and the only overlapping-author citation [40] is used for keyframe selection and general related-work framing, not to justify the central improvement. No uniqueness theorem is imported from the authors, no ansatz is smuggled in via citation, and no known result is simply renamed. The evaluation is self-contained against external benchmarks with held-out test scenarios, so the appropriate circularity finding is 0.

Assumptions & free parameters 4 free parameters · 5 assumptions · 2 invented entities

The paper's empirical claims rest on four unstated or hand-chosen settings (loss weights, detector freeze/fine-tune status, ALOHA hyperparameters, rotation bin count), five domain assumptions inherited from prior imitation-learning pipelines or introduced here (keyframe sufficiency, CLIP alignment, Mamba expressiveness, grasp-pose supervision sufficiency, and no-leakage from the benchmark-trained grasp detector), and two new modules whose necessity is supported only by internal ablations. The heaviest item in the ledger is the assumption that a grasp detector trained on the same benchmark's demonstrations provides a generic, transferable grasp prior; the paper provides no evidence for this outside its own benchmark runs.

free parameters (4)
  • Loss weights lambda_trans, lambda_rot, lambda_gripper, lambda_collision = not stated
    Eq. 18 defines the weighted total loss, but no values are reported anywhere in the paper.
  • Grasp detector freeze/fine-tune status = not stated
    Section III.D says the detector can be frozen or fine-tuned; the choice is never pinned down, and it determines how much the pretrained model contributes.
  • ALOHA training hyperparameters = not stated
    RLBench settings are given (batch 24, lr 0.0018, 99 epochs) but the continuous-task training configuration for the ACT-adapted model is not.
  • Rotation discretization bin count = 72 bins
    Eq. 5 discretizes rotation into 72 categories; the resolution of the rotation head is a design choice that affects achievable precision.
assumptions (5)
  • standard math Mamba provides linear-time sequence modeling with sufficient expressiveness for visual token sequences
    Relied on for RAM's design and the real-time claim; cited from [20], not re-derived here.
  • domain assumption CLIP embeddings align language instructions with visual features well enough for multi-task manipulation conditioning
    Used as the language encoder (Eq. 12); inherited from the RVT2 pipeline, no validation here.
  • domain assumption Keyframes at zero velocity or gripper-state changes preserve the essential action structure of demonstrations
    Keyframe discovery follows prior work [12],[13],[15],[16] (Section IV.A); the policy only trains on these frames.
  • domain assumption Grasp poses extracted from expert demonstrations are sufficient supervision for the grasp detector that GPA relies on
    Load-bearing premise of GPA (Section III.D); if the auto-extracted grasp poses do not capture precise grasping, GPA's gains are not grasp skill.
  • domain assumption The pretrained M2T2 grasp detector for RLBench provides grasp priors that generalize to held-out scenarios and other platforms, with no leakage from its own RLBench training
    Section IV.C uses M2T2, an RLBench-trained method, as the detector; its training data and overlap with the policy's training set are never disclosed.
invented entities (2)
  • PLoFusion (Pre-trained Location Fusion)
    purpose: Fuses frozen grasp-detector features with RAM spatial features via concatenation and self-attention before action prediction (Section III.D).
    Its benefit is evidenced only by the paper's internal ablations (GPA-RAM vs RAM, +2.8% on RLBench); no external falsifiable handle is offered.
  • Coarse-to-fine RAM (C-RAM + F-RAM)
    purpose: Two-stage attention-Mamba feature extractor that refines a coarse action heatmap into a fine one (Section III.C).
    Supported only by internal ablations (C-RAM 66.6% vs RAM 84.7% on RLBench); no independent evidence outside this paper.

how reviews work

0 comments
Cite this review

Pith. "Pith review of GPA-RAM: Grasp-Pretraining Augmented Robotic Attention Mamba for Spatial Task Learning." pith.science (2026). https://pith.science/paper/IKFELWPQ

@misc{pith2026250419683,
  author       = {Pith},
  title        = {Pith review of: GPA-RAM: Grasp-Pretraining Augmented Robotic Attention Mamba for Spatial Task Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IKFELWPQ}},
  note         = {Machine review of arXiv:2504.19683}
}
read the original abstract

Fine-grained robotic manipulation often fails when inaccurate initial grasps propagate errors and necessitate complex pose correction. We propose Grasp-Pretraining Augmentation (GPA), which incorporates grasp priors from task demonstrations into imitation policies without additional grasp-pose data or annotation. When added to RVT2, GPA raises the average success rate on RLBench from 79.3% to 84.2%. When added to ACT, it raises success on ALOHA cube transfer and bimanual insertion from 86% and 16% to 98% and 38%, respectively. To offset added computational costs, we develop Robotic Attention Mamba (RAM) for real-time deployment. RAM combines attention-based spatial feature extraction with state-space modeling to capture long-range dependencies efficiently. The resulting GPA-RAM framework supports discrete keyframe prediction and continuous action generation. We evaluate it on four platforms, including physical UR5 and ARX R5 systems. GPA-RAM achieves an average success rate of 87.5% on RLBench, outperforming RVT2 and ARP+ by 8.2 and 2.6 percentage points, respectively. On ALOHA, it achieves 98% success in cube transfer and 56% in bimanual insertion, improvements of 12 and 40 percentage points over ACT, while operating at approximately 71 frames per second. These results demonstrate that GPA-RAM combines precise manipulation with efficient real-time robotic execution. Code is available at https://gpa-ram.github.io/.

Figures

Figures reproduced from arXiv: 2504.19683 by the authors.

Figure 1
Figure 1. Previous imitation learning methods struggle with tasks requiring high [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Overview of the GPA-RAM on RLBench. The GPA-RAM primarily comprises two modules: RAM for spatial features extraction and GPA for grasping [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Network Architecture of Robotic Attention Mamba and Grasp [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Adapt GPA and One-Stage RAM to Continuous Actions Generation. [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Qualitative Comparison of the previous SOTA (ARP [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Qualitative Comparison of the Proposed GPA-RAM and the Previous [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]
Figure 7
Figure 7. Figure 7: Real-world experimental setups for discrete action generation (left) [PITH_FULL_IMAGE:figures/full_fig_p010_7.png]
Figure 8
Figure 8. Figure 8: Examples of GPA-RAM in the real world. Top row: Grasp and [PITH_FULL_IMAGE:figures/full_fig_p010_8.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

50 extracted references · 33 canonical work pages

  1. [1]

    Rog- sam: A language-driven framework for instance-level robotic grasping detection,

    Y . Mei, J. Sun, Z. Peng, F. Deng, G. Wang, and J. Chen, “Rog- sam: A language-driven framework for instance-level robotic grasping detection,”IEEE Transactions on Multimedia, vol. 27, pp. 3057–3068, 2025

    work page 2025

  2. [2]

    Coupling and decoupling: Towards temporal feedback for 3d object detection,

    Y . Cui, Z. Zou, X. Ye, X. Tan, Z. Li, and Z. Fang, “Coupling and decoupling: Towards temporal feedback for 3d object detection,”IEEE Transactions on Multimedia, vol. 27, pp. 7230–7242, 2025

    work page 2025

  3. [3]

    Openslot: Mixed open-set recognition with object-centric learning,

    X. Yin, F. Pan, G. An, Y . Huo, Z. Xie, and S.-E. Yoon, “Openslot: Mixed open-set recognition with object-centric learning,”IEEE Transactions on Multimedia, vol. 27, pp. 6019–6030, 2025

    work page 2025

  4. [4]

    Source- free elastic model adaptation for vision-and-language navigation,

    M. Tan, P. Chen, H. Zhi, J. Mai, B. Rosman, D. Ji, and R. Zeng, “Source- free elastic model adaptation for vision-and-language navigation,”IEEE Transactions on Multimedia, vol. 27, pp. 3953–3965, 2025

    work page 2025

  5. [5]

    Flexvln: Flexible adaptation for diverse vision-and-language navigation tasks,

    S. Zhang, Y . Qiao, Q. Wang, L. Guo, Z. Wei, and J. Liu, “Flexvln: Flexible adaptation for diverse vision-and-language navigation tasks,” IEEE Transactions on Multimedia, vol. 27, pp. 6307–6318, 2025

    2025

  6. [6]

    Attention is all you need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, “Attention is all you need,” inAdvances in Neural Information Processing Systems, vol. 30, 2017, p. 6000–6010

    work page 2017

  7. [7]

    Denoising diffusion probabilistic models,

    J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,” Advances in neural information processing systems, vol. 33, pp. 6840– 6851, 2020

    2020

  8. [8]

    Toward general cross-modal signal reconstruction for robotic teleoperation,

    Y . Chen, A. Li, D. Wu, and L. Zhou, “Toward general cross-modal signal reconstruction for robotic teleoperation,”IEEE Transactions on Multimedia, vol. 26, pp. 3541–3553, 2024

    work page 2024

Show all 50 references
  1. [9]

    Learning fine-grained bimanual manipulation with low-cost hardware,

    T. Z. Zhao, V . Kumar, S. Levine, and C. Finn, “Learning fine-grained bimanual manipulation with low-cost hardware,” inProceedings of Robotics: Science and Systems (RSS), 2023

    2023

  2. [10]

    Universal manipulation interface: In-the-wild robot teach- ing without in-the-wild robots,

    C. Chi, Z. Xu, C. Pan, E. Cousineau, B. Burchfiel, S. Feng, R. Tedrake, and S. Song, “Universal manipulation interface: In-the-wild robot teach- ing without in-the-wild robots,” inProceedings of Robotics: Science and Systems (RSS), 2024

    2024

  3. [11]

    Rlbench: The robot learning benchmark & learning environment,

    S. James, Z. Ma, D. R. Arrojo, and A. J. Davison, “Rlbench: The robot learning benchmark & learning environment,”IEEE Robotics and Automation Letters, vol. 5, no. 2, pp. 3019–3026, 2020

    2020

  4. [12]

    Perceiver-actor: A multi-task transformer for robotic manipulation,

    M. Shridhar, L. Manuelli, and D. Fox, “Perceiver-actor: A multi-task transformer for robotic manipulation,” inProceedings of the Conference on Robot Learning (CoRL), vol. 205, 2022, pp. 785–799

    2022

  5. [13]

    Act3D: 3D feature field transformers for multi-task robotic manipulation,

    T. Gervet, Z. Xian, N. Gkanatsios, and K. Fragkiadaki, “Act3D: 3D feature field transformers for multi-task robotic manipulation,” in7th Annual Conference on Robot Learning, 2023

    2023

  6. [14]

    Polarnet: 3d point clouds for language-guided robotic manipulation,

    S. Chen, R. Garcia, C. Schmid, and I. Laptev, “Polarnet: 3d point clouds for language-guided robotic manipulation,” inIn 7th Annual Conference on Robot Learning (CoRL), 2023

    2023

  7. [15]

    Rvt: Robotic view transformer for 3D object manipulation,

    A. Goyal, J. Xu, Y . Guo, V . Blukis, Y .-W. Chao, and D. Fox, “Rvt: Robotic view transformer for 3D object manipulation,” inProceedings of the Conference on Robot Learning (CoRL). PMLR, 2023

    2023

  8. [16]

    Rvt2: Learning precise manipulation from few demonstrations,

    A. Goyal, V . Blukis, J. Xu, Y . Guo, Y .-W. Chao, and D. Fox, “Rvt2: Learning precise manipulation from few demonstrations,” inRobotics: Science and Systems, 2024

    2024

  9. [17]

    Autoregres- sive action sequence learning for robotic manipulation,

    X. Zhang, Y . Liu, H. Chang, L. Schramm, and A. Boularias, “Autoregres- sive action sequence learning for robotic manipulation,”IEEE Robotics and Automation Letters, vol. 10, no. 5, pp. 4898–4905, 2025

    2025

  10. [18]

    Efficient self-supervised vision transformers for representation learning,

    C. Li, J. Yang, P. Zhang, M. Gao, B. Xiao, X. Dai, L. Yuan, and J. Gao, “Efficient self-supervised vision transformers for representation learning,” inThe International Conference on Learning Representations (ICLR), 2022

    2022

  11. [19]

    Efficiently modeling long sequences with structured state spaces,

    A. Gu, K. Goel, and C. R ´e, “Efficiently modeling long sequences with structured state spaces,” 2022. [Online]. Available: https: //arxiv.org/abs/2111.00396 IEEE TRANSACTIONS ON MULTIMEDIA 12

    2022 arXiv

  12. [20]

    Mamba: Linear-time sequence modeling with selective state spaces,

    A. Gu and T. Dao, “Mamba: Linear-time sequence modeling with selective state spaces,” inConference on Language Modeling, 2024

    2024

  13. [21]

    Rignet: Robot intention grasp for dense stacked targets with multi-task siamese schema through rois learning,

    X. Zhong, T. Gong, J. Yu, C. Zhou, X. Zhong, and Q. Liu, “Rignet: Robot intention grasp for dense stacked targets with multi-task siamese schema through rois learning,”IEEE Transactions on Automation Sci- ence and Engineering, vol. 22, pp. 10 354–10 367, 2025

    2025

  14. [22]

    Foundationgrasp: Generalizable task-oriented grasping with foundation models,

    C. Tang, D. Huang, W. Dong, R. Xu, and H. Zhang, “Foundationgrasp: Generalizable task-oriented grasping with foundation models,”IEEE Transactions on Automation Science and Engineering, vol. 22, pp. 12 418–12 435, 2025

    2025

  15. [23]

    Learning to reorient objects with stable placements afforded by extrinsic supports,

    P. Xu, H. Cheng, J. Wang, and M. Q.-H. Meng, “Learning to reorient objects with stable placements afforded by extrinsic supports,”IEEE Transactions on Automation Science and Engineering, vol. 21, no. 4, pp. 5653–5664, 2024

    2024

  16. [24]

    Watch and act: Learning robotic manipulation from visual demonstration,

    S. Yang, W. Zhang, R. Song, J. Cheng, H. Wang, and Y . Li, “Watch and act: Learning robotic manipulation from visual demonstration,”IEEE Transactions on Systems, Man, and Cybernetics: Systems, vol. 53, no. 7, pp. 4404–4416, 2023

    2023

  17. [25]

    A learning-based two-stage method for submillimeter insertion tasks with only visual inputs,

    J. Zhao, Z. Wang, L. Zhao, and H. Liu, “A learning-based two-stage method for submillimeter insertion tasks with only visual inputs,”IEEE Transactions on Industrial Electronics, vol. 71, no. 7, pp. 7381–7390, 2024

    2024

  18. [26]

    Grasp, see, and place: Efficient unknown object rearrangement with policy structure prior,

    K. Xu, Z. Zhou, J. Wu, H. Lu, R. Xiong, and Y . Wang, “Grasp, see, and place: Efficient unknown object rearrangement with policy structure prior,”IEEE Transactions on Robotics, vol. 41, pp. 464–483, 2025

    2025

  19. [27]

    Diffusion policy: Visuomotor policy learning via action diffusion,

    C. Chi, S. Feng, Y . Du, Z. Xu, E. Cousineau, B. Burchfiel, and S. Song, “Diffusion policy: Visuomotor policy learning via action diffusion,” in Proceedings of Robotics: Science and Systems, 2023

    2023

  20. [28]

    Zero-shot robotic manipulation with pre-trained image- editing diffusion models,

    K. Black, M. Nakamoto, P. Atreya, H. R. Walke, C. Finn, A. Kumar, and S. Levine, “Zero-shot robotic manipulation with pre-trained image- editing diffusion models,” inThe International Conference on Learning Representations (ICLR), 2024

    2024

  21. [29]

    Unleashing large-scale video generative pre-training for visual robot manipulation,

    H. Wu, Y . Jing, C. Cheang, G. Chen, J. Xu, X. Li, M. Liu, H. Li, and T. Kong, “Unleashing large-scale video generative pre-training for visual robot manipulation,” inThe International Conference on Learning Representations (ICLR), 2024

    2024

  22. [30]

    Gr-mg: Leveraging partially-annotated data via multi-modal goal-conditioned policy,

    P. Li, H. Wu, Y . Huang, C. Cheang, L. Wang, and T. Kong, “Gr-mg: Leveraging partially-annotated data via multi-modal goal-conditioned policy,”IEEE Robotics and Automation Letters, 2025

    2025

  23. [31]

    Any-point trajectory modeling for policy learning,

    C. Wen, X. Lin, J. So, K. Chen, Q. Dou, Y . Gao, and P. Abbeel, “Any-point trajectory modeling for policy learning,”arXiv preprint arXiv:2401.00025, 2023

    2023 arXiv

  24. [32]

    Toward cognitive digital twin system of human-robot collaboration manipulation,

    X. Li, B. He, Z. Wang, Y . Zhou, G. Li, and X. Li, “Toward cognitive digital twin system of human-robot collaboration manipulation,”IEEE Transactions on Automation Science and Engineering, vol. 22, pp. 6677– 6690, 2025

    2025

  25. [33]

    Towards general- izable zero-shot manipulation via translating human interaction plans,

    H. Bharadhwaj, A. Gupta, V . Kumar, and S. Tulsiani, “Towards general- izable zero-shot manipulation via translating human interaction plans,” in2024 IEEE International Conference on Robotics and Automation (ICRA), 2024, pp. 6904–6911

    2024

  26. [34]

    Learning manipulation by predicting interaction,

    Z. Jia, B. Qingwen, W. Bangjun, X. Wenke, C. Li, D. Hao, S. Haoming, W. Dong, H. Di, L. Ping, C. Heming, Z. Bin, L. Xuelong, Q. Yu, and L. Hongyang, “Learning manipulation by predicting interaction,” inProceedings of Robotics: Science and Systems (RSS), 2024

    2024

  27. [35]

    Bc-z: Zero-shot task generalization with robotic imitation learning,

    E. Jang, A. Irpan, M. Khansari, D. Kappler, F. Ebert, C. Lynch, S. Levine, and C. Finn, “Bc-z: Zero-shot task generalization with robotic imitation learning,” inProceedings of the 5th Conference on Robot Learning, ser. Proceedings of Machine Learning Research, A. Faust, D. Hsu...

    2022

  28. [36]

    What matters in language conditioned robotic imitation learning over unstructured data,

    O. Mees, L. Hermann, and W. Burgard, “What matters in language conditioned robotic imitation learning over unstructured data,”IEEE Robotics and Automation Letters, vol. 7, no. 4, pp. 11 205–11 212, 2022

    2022

  29. [37]

    Contrastive imitation learning for language-guided multi-task robotic manipulation,

    T. Ma, J. Zhou, Z. Wang, R. Qiu, and J. Liang, “Contrastive imitation learning for language-guided multi-task robotic manipulation,” in8th Annual Conference on Robot Learning (CoRL), 2024

    2024

  30. [38]

    Rt-1: Robotics transformer for real-world control at scale,

    A. Brohan, N. Brown, J. Carbajal, and et al., “Rt-1: Robotics transformer for real-world control at scale,” inProceedings of Robotics: Science and Systems (RSS), 2022

    2022

  31. [39]

    Rt-2: Vision-language-action models transfer web knowledge to robotic control,

    ——, “Rt-2: Vision-language-action models transfer web knowledge to robotic control,”arXiv preprint arXiv:2307.15818, 2023

    2023 arXiv

  32. [40]

    Fusion-perception- to-action transformer: Enhancing robotic manipulation with 3-d visual fusion attention and proprioception,

    Y . Liu, S. Liu, B. Chen, Z.-X. Yang, and S. Xu, “Fusion-perception- to-action transformer: Enhancing robotic manipulation with 3-d visual fusion attention and proprioception,”IEEE Transactions on Robotics, vol. 41, pp. 1553–1567, 2025

    2025

  33. [41]

    Efficiently modeling long sequences with structured state spaces,

    A. Gu, K. Goel, and C. R ´e, “Efficiently modeling long sequences with structured state spaces,” inThe International Conference on Learning Representations (ICLR), 2022

    2022

  34. [42]

    Learning transferable visual models from natural language supervision,

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clarket al., “Learning transferable visual models from natural language supervision,” inInternational Conference on Machine Learning (ICML), 2021, pp. 8748–8763

    2021

  35. [43]

    Coarse-to-fine q- attention: Efficient learning for visual robotic manipulation via discreti- sation,

    S. James, K. Wada, T. Laidlow, and A. J. Davison, “Coarse-to-fine q- attention: Efficient learning for visual robotic manipulation via discreti- sation,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022, pp. 13 739–13 748

    2022

  36. [44]

    Instruction-driven history-aware policies for robotic manipulations,

    P.-L. Guhur, S. Chen, R. G. Pinel, M. Tapaswi, I. Laptev, and C. Schmid, “Instruction-driven history-aware policies for robotic manipulations,” in Proceedings of the Conference on Robot Learning (CoRL). PMLR, 2023, pp. 175–187

    2023

  37. [45]

    V-rep: A versatile and scalable robot simulation framework,

    E. Rohmer, S. P. N. Singh, and M. Freese, “V-rep: A versatile and scalable robot simulation framework,” in2013 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2013, pp. 1321–1326

    2013

  38. [46]

    Mujoco: A physics engine for model-based control,

    E. Todorov, T. Erez, and Y . Tassa, “Mujoco: A physics engine for model-based control,” in2012 IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, 2012, pp. 5026–5033

    2012

  39. [47]

    Behavior transformers: Cloning k modes with one stone,

    N. M. Shafiullah, Z. Cui, A. A. Altanzaya, and L. Pinto, “Behavior transformers: Cloning k modes with one stone,” inAdvances in Neural Information Processing Systems, S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, Eds., vol. 35. Curran Associates, Inc., 202...

    2022

  40. [48]

    The surprising effectiveness of representation learning for visual imitation,

    J. Pari, N. M. Shafiullah, S. P. Arunachalam, and L. Pinto, “The surprising effectiveness of representation learning for visual imitation,” arXiv preprint arXiv:2112.01511, 2021

    2021 arXiv

  41. [49]

    M2t2: Multi-task masked transformer for object-centric pick and place,

    W. Yuan, A. Murali, A. Mousavian, and D. Fox, “M2t2: Multi-task masked transformer for object-centric pick and place,” inConference on Robot Learning, vol. 229, 2023, pp. 3619–3630

    2023

  42. [50]

    Deep residual learning for image recognition,

    K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 770–778

    2016

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.