Pith. sign in

REVIEW 4 major objections 5 minor 51 references

Knowledge-Driven Imitation Learning: Enabling Generalization Across Diverse Conditions

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A fixed semantic keypoint graph, matched by coarse-to-fine structural and semantic alignment, lets imitation policies learn from a quarter of the demonstrations image-based diffusion needs.

desk verdict A practical keypoint-graph representation for data-efficient imitation learning, with real-robot wins but evaluation reporting that needs tightening. read the letter →

arxiv 2506.21057 v1 pith:UO5TMSTM submitted 2025-06-26 cs.RO

Zhuochen Miao , Jun Lv , Hongjie Fang , Yang Jin , Cewu Lu This is my paper
classification cs.RO
keywords imitationlearningrobotmanipulationsemantickeypointgraphtemplatematchinggeneralizationsampleefficiencydiffusionpolicyDINOv2
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to show that a robot imitation policy can learn manipulation skills from far fewer expert demonstrations when observations are replaced by a category-level semantic keypoint graph instead of raw images. It builds a fixed knowledge template for an object category from one image and matches that template to any new instance through a coarse-to-fine algorithm that balances semantic feature similarity with structural consistency. On three real-world tasks, the resulting knowledge-driven policy outperforms an image-based diffusion policy while using about one quarter of the demonstrations, and the authors report that injecting knowledge raises Diffusion Policy's average success rate by 54%. If correct, this would mean explicit category knowledge can substitute for large amounts of demonstration data in robot manipulation.

What carries the argument

The central object is the category-level semantic keypoint graph $T_C = \{(\hat{f}_k, \hat{p}_k)\}_{k=1}^K$: K semantic keypoints sampled from one object, where $\hat{f}_k$ is a DINOv2 semantic feature and $\hat{p}_k$ a 3D position. The load-bearing mechanism is the coarse-to-fine template matching algorithm, which solves a joint optimization of feature similarity and structural consistency. Coarse matching performs a rigid point-set registration with scaling, solved with the Umeyama algorithm under a RANSAC scheme, with mismatches penalized by a threshold on feature distance, and fine matching refines each keypoint's position within a neighborhood by balancing feature distance, structural position penalty, and a position-distance threshold. This matching output, rather than raw images, is the observation fed into a diffusion policy.

What would settle it

Render a held-out set of object instances from the Google Scanned Objects dataset in unseen poses and measure the fraction of template keypoints matched within 3 cm. If coarse-to-fine template matching does not clearly beat the 11.2% top-1 matching rate on instances where DINOv2 features are not consistent, or if a category with high intra-class variation shows matched rates near chance, the claim that one fixed template spans a category would fail.

Watch

Extended reading notes

Core claim

The central claim is that a single fixed template of K semantic keypoints, generated from one training object and matched to every new observation, is a sufficient and beneficial observation space for manipulation imitation learning. The template is a graph: each keypoint carries a DINOv2 semantic feature and a 3D position, and matching minimizes a combined objective of feature distance and structural deformation after a coarse rigid registration with scaling, followed by a fine feature-aligned refinement. The matched keypoint coordinates are fed to a diffusion policy, which treats a novel object as a deformation of the template object. The paper reports that this scheme surpasses image-based Diffusion Policy on all three tasks and matches its success with roughly one quarter of the expert demonstrations, while also generalizing to unseen objects, backgrounds, and lighting conditions.

Load-bearing premise

The method assumes that one fixed set of keypoints built from a single training object conveys enough semantic and structural information to describe every other object in the same category, and that DINOv2 features give trustworthy matches for those keypoints on novel instances.

Editorial extensions

If this is right

  • A fixed per-category keypoint template can replace image observations, making manipulation policies learnable from about one quarter of the demonstrations required by image-based Diffusion Policy.
  • Template matching that enforces structure reduces keypoint ambiguity: the paper's matching improves the fraction of keypoints matched within 3 cm from 11.2% to 59.2% compared with top-1 keypoint matching.
  • The reported 54% average success-rate increase over Diffusion Policy implies that structured priors can provide some of the benefit that scaling demonstrations provides.
  • Open-loop execution using first-frame keypoint observations is sufficient for the tested tasks because the template anchors the current object to the training object's coordinate frame.
  • Generalization to unseen objects, backgrounds, and lighting follows from using a pre-trained semantic feature encoder whose correspondences are robust to environmental changes.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same template-matching machinery could be run per frame to provide closed-loop keypoint observations, which would address the occlusion cases the paper sidesteps with open-loop control; the reported matching robustness suggests this is a natural extension.
  • Because the template is category-level and built from a single object, the framework is likely to transfer to new object categories if the chosen keypoints cover task-relevant parts; a stress test would be a category with high intra-class shape variation, where fine matching would need to absorb larger deformations.
  • The comparison between DINOv2 and DIFT suggests a testable design rule: feature extractors with more homogeneous feature fields are better suited for structural template matching, while locally discriminative features are better for per-point correspondence; this could be checked on other dense visual features.
  • If the reported gains hold across more tasks, knowledge-driven observation design becomes a complement to data scaling laws in imitation learning, letting practitioners trade demonstrations for a modest amount of per-category annotation.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes a knowledge-driven imitation learning framework for robot manipulation. A category-level knowledge template is defined as a graph of K semantic keypoints sampled from a single training object, with each keypoint carrying a DINOv2 semantic feature and a 3D position. A coarse-to-fine template matching algorithm aligns this template to novel instances by jointly optimizing feature similarity and structural consistency, and the resulting matched keypoint coordinates are fed as observations into a Diffusion Policy. The method is evaluated on three real-world tasks (Mug, Tool, Drawer) against image-based Diffusion Policy and P3-PO, with additional experiments on unseen objects, novel backgrounds, and lighting conditions. The authors claim superior performance over image-based Diffusion Policy using roughly one-quarter of the demonstrations and a 54% increase in average success rate from introducing knowledge.

Significance. If substantiated, the proposed framework would be a meaningful step toward data-efficient and generalizable manipulation: it explicitly injects category-level structural-semantic knowledge into an imitation learning pipeline, uses a principled matching objective, and is evaluated on real hardware. The strengths of the paper include a clearly formulated coarse-to-fine matching scheme, the use of pre-trained DINOv2 features, and real-robot comparisons against reasonable baselines. However, the current evidence does not yet support the headline claims. The data-efficiency comparison in Fig. 4 rests on a one-success difference out of 20 trials, and the matching ablation in Table IV shows an average error of 3.66 cm with only a 59.2% matched rate under a 3 cm threshold, which is difficult to reconcile with the stated 1 cm precision requirement of the Mug task. The paper would need substantially stronger quantitative support to establish that the matching mechanism, rather than policy robustness, drives the observed results.

major comments (4)
  1. [IV-C, Fig. 4] The central data-efficiency claim is not supported by the reported numbers. The comparison of "our method with 24 demonstrations" (35% success) against "Diffusion Policy with 90 demonstrations" (30% success) is a difference of one success in what appear to be 20 trials, which is well within binomial sampling noise. Please report the exact number of trials per configuration, provide confidence intervals or repeated-seed results, and perform a statistical test (e.g., Wilson intervals or a chi-square test) to justify the claim of superiority at one-quarter of the demonstrations.
  2. [IV-E, Table IV] The matching quality reported in Table IV does not establish the mechanism that the paper claims. The template matching achieves an average error of 3.66 cm and a matched rate of 59.2% under a 3 cm threshold, yet Section IV-A2 states that the Mug task requires 1 cm precision. If roughly 40% of keypoints are matched with error exceeding 3 cm under favorable rendered conditions, the observed policy success may be due to the diffusion policy tolerating noisy keypoints rather than to clean, reliable template matching. Please report the full error distribution per keypoint and per object category, and include an ablation that injects controlled keypoint noise to quantify the policy's sensitivity to matching errors.
  3. [III-B and IV-D] The single-template assumption, stated in Section III-B as "we assume that the semantic features and structure of the keypoints can be generalized among objects in the same category," is only weakly tested. In Table II, the proposed method is worse than P3-PO on the Mug task (6/10 vs 7/10) and ties on the Drawer task (5/10 vs 5/10), both on only 10 trials. These results do not demonstrate consistent generalization gains across categories. Please provide per-object results for all unseen objects, specify whether the same template (with the same K keypoints) was used for every object in a category, and report the template-seed variation to assess robustness of the generalization claim.
  4. [III-C] The method as described is not reproducible because the key hyperparameters are never reported. The thresholds δf (feature distance), δp (position distance), the structural weight β, and the number of keypoints K appear in the matching formulation (Eqs. 3–5) and in the implementation (3–20 keypoints), but their values for each experiment are omitted. Please report the values used for all real-world and synthetic experiments, and include a sensitivity analysis for at least δf and β to demonstrate that the results are not sensitive to fine hand-tuning.
minor comments (5)
  1. [Throughout] The manuscript contains several typos and formatting inconsistencies, including "pratice" (Section III-B), "Sematic Keypoint Sampling" (Section III-B2), "carse-to-fine" (Section III-C), "T ool" (Section IV-A2 and Fig. 3), and "Lightning" instead of "Lighting" in Table III and Section IV-D.
  2. [Fig. 4] The percentage labels inside the bars in Fig. 4 are difficult to read and the sample sizes are not stated. Please annotate each bar with the number of trials and the success count.
  3. [IV-B3] The policy implementation states that keypoint observations are obtained from the first frame and the policy runs open-loop. This design choice should be justified more explicitly, since it has important implications for the generalization results, especially under lighting changes that occur during an episode.
  4. [V] The statement that "the introduction of knowledge leads to a 54% increase in the average success rate of Diffusion Policy" is ambiguous: the computed difference is 54 percentage points in average success rate, not a 54% relative increase. Please rephrase to avoid misinterpretation.
  5. [IV-B1] The paper does not specify how point clouds from the two RealSense cameras are fused or how the semantic features are projected to the robot coordinate system beyond "using camera calibration results." A brief description of the fusion and calibration procedure would improve reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central results come from external benchmarks and held-out evaluations, not from quantities defined by the method.

full rationale

The paper's derivation chain is not circular. The knowledge template is constructed from a single training-object image using DINOv2 features and farthest-point sampling (Sec. III-B), and the policy is trained on matched keypoint coordinates (Sec. III-D); the central claims are success rates on held-out objects and environments (Tabs. I-III) and a data-efficiency comparison against image-based Diffusion Policy (Fig. 4). None of these target quantities appears in the template construction loss or in the matching objective (Eqs. 2-5); the thresholds delta_f, delta_p, and beta are not fitted to the reported success rates and are not tuned on the evaluation trials. The matching ablation (Tab. IV) is an external benchmark on Google Scanned Objects, not a restatement of the policy results. The self-citations ([11] AirExo-2, [44] RISE) are used for action-space conventions and related-work context, not to justify the generalization claim; they are independent published system papers. The explicit single-template generalization assumption (Sec. III-B) is an empirical premise that can fail, but an unsupported or weak assumption is a robustness/correctness concern, not a circular reduction.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claims rest on pretrained DINOv2 feature consistency, the single-template-per-category assumption, standard registration math, and several unreported hand-set matching thresholds. These are reasonable domain assumptions but should be stated with sensitivity analysis.

free parameters (4)
  • beta (structure weight) = not reported
    Weight between feature and structure losses in Eq. 2; hand-chosen and controls the matching trade-off.
  • delta_f (feature distance threshold) = not reported
    Threshold in coarse matching (Eq. 4) that decides which keypoints count as correspondences.
  • delta_p (position distance threshold) = not reported
    Threshold in fine matching (Eq. 5) that limits the local search neighborhood for refined keypoints.
  • K (number of keypoints) = 3-20 per category
    Template size, chosen by manual annotation or farthest point sampling depending on object complexity; directly shapes the policy input.
assumptions (4)
  • standard math Umeyama algorithm correctly solves rigid point-set registration with scaling.
    Invoked in Section III-C a for coarse template matching.
  • domain assumption DINOv2 semantic features provide consistent correspondences across object instances within a category.
    The matching and generalization results depend on feature stability; stated as an assumption in Section III-B and used throughout Section III-C.
  • domain assumption A single fixed template per category captures the task-relevant structure of all novel instances.
    Stated in Section III-B: 'We assume that the semantic features and structure of the keypoints can be generalized among objects in the same category.'
  • domain assumption Scenes are effectively static during execution, so first-frame keypoints remain valid for an open-loop policy.
    Section IV-B 4 states keypoint observations are obtained from the first frame only; this assumes no significant object motion or viewpoint change.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Knowledge-Driven Imitation Learning: Enabling Generalization Across Diverse Conditions." pith.science (2026). https://pith.science/paper/UO5TMSTM

@misc{pith2026250621057,
  author       = {Pith},
  title        = {Pith review of: Knowledge-Driven Imitation Learning: Enabling Generalization Across Diverse Conditions},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UO5TMSTM}},
  note         = {Machine review of arXiv:2506.21057}
}
read the original abstract

Imitation learning has emerged as a powerful paradigm in robot manipulation, yet its generalization capability remains constrained by object-specific dependencies in limited expert demonstrations. To address this challenge, we propose knowledge-driven imitation learning, a framework that leverages external structural semantic knowledge to abstract object representations within the same category. We introduce a novel semantic keypoint graph as a knowledge template and develop a coarse-to-fine template-matching algorithm that optimizes both structural consistency and semantic similarity. Evaluated on three real-world robotic manipulation tasks, our method achieves superior performance, surpassing image-based diffusion policies with only one-quarter of the expert demonstrations. Extensive experiments further demonstrate its robustness across novel objects, backgrounds, and lighting conditions. This work pioneers a knowledge-driven approach to data-efficient robotic learning in real-world settings. Code and more materials are available on https://knowledge-driven.github.io/.

Figures

Figures reproduced from arXiv: 2506.21057 by the authors.

Figure 1
Figure 1. Overview of our Knowledge-Driven Imitation Learning [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Keypoint Graph as Knowledge Template. The knowledge template is represented as a graph, explicitly encoding structural information, while each keypoint is associated with a semantic feature, implicitly conveying semantic information. a) Semantic Feature Generation: To obtain the semantic feature f of a specific point p, we first employ a pre￾trained semantic feature extractor. Given an RGB image I ∈ R H×W×3 of a cer… view at source ↗
Figure 3
Figure 3. Tasks. We design three manipulation tasks (Mug, Tool and Drawer) for real-world evaluations. Considering the presence of occlusions, some keypoints may always be unmatched. To address this, we employ the RANSAC strategy for the matching process. This enables the algorithm to rely on a small number of accurately matched keypoints to infer the positions of occluded keypoints, en￾abling more robust template matching. b… view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Data Efficiency Comparisons. Evaluation results of policies trained with varying numbers of demonstrations highlight the data efficiency of our proposed method compared to Diffusion Policy. D. Generalization Evaluations To evaluate the generalization ability of our met…
Figure 5
Figure 5. Figure 5: Experimental Setup on Generalization Evaluations. [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 7
Figure 7. Figure 7: Illustration of Fine Template Matching. Fine template matching allows for the successful matching of mugs with handles of different sizes. manipulation learning method. We define a novel knowledge template format with a coarse-to-fine template matching al￾gorithm, cons…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

51 extracted references · 34 canonical work pages

  1. [1]

    Track2Act: Predicting Point Tracks from Internet Videos Enables Generalizable Robot Manipulation

    Homanga Bharadhwaj et al. “Track2Act: Predicting Point Tracks from Internet Videos Enables Generalizable Robot Manipulation”. In: ECCV. 2024

    work page 2024

  2. [2]

    π0: A Vision-Language-Action Flow Model for General Robot Control

    Kevin Black et al. “ π0: A Vision-Language-Action Flow Model for General Robot Control”. In: arXiv preprint arXiv:2410.24164 (2024)

    arXiv 2024

  3. [3]

    Rt-2: Vision-language-action models transfer web knowledge to robotic control

    Anthony Brohan et al. “Rt-2: Vision-language-action models transfer web knowledge to robotic control”. In: arXiv preprint arXiv:2307.15818 (2023)

    arXiv 2023

  4. [4]

    Language models are few-shot learners

    Tom Brown et al. “Language models are few-shot learners”. In: Advances in neural information processing systems 33 (2020), pp. 1877–1901

    work page 2020

  5. [5]

    Diffusion Policy: Visuomotor Policy Learn- ing via Action Diffusion

    Cheng Chi et al. “Diffusion Policy: Visuomotor Policy Learn- ing via Action Diffusion”. In: The International Journal of Robotics Research (2024)

    work page 2024

  6. [6]

    Universal Manipulation Interface: In-The- Wild Robot Teaching Without In-The-Wild Robots

    Cheng Chi et al. “Universal Manipulation Interface: In-The- Wild Robot Teaching Without In-The-Wild Robots”. In: Pro- ceedings of Robotics: Science and Systems (RSS) . 2024

    work page 2024

  7. [7]

    Keypoint action tokens enable in-context imitation learning in robotics

    Norman Di Palo and Edward Johns. “Keypoint action tokens enable in-context imitation learning in robotics”. In: arXiv preprint arXiv:2403.19578 (2024)

    arXiv 2024

  8. [8]

    Google Scanned Objects: A High-Quality Dataset of 3D Scanned Household Items

    Laura Downs et al. Google Scanned Objects: A High-Quality Dataset of 3D Scanned Household Items . 2022

    work page 2022

Show all 51 references
  1. [9]

    One-Shot Imitation Learning

    Yan Duan et al. One-Shot Imitation Learning . 2017

    2017

  2. [10]

    RH20T: A Comprehensive Robotic Dataset for Learning Diverse Skills in One-Shot

    Haoshu Fang et al. “RH20T: A Comprehensive Robotic Dataset for Learning Diverse Skills in One-Shot”. In: IEEE International Conference on Robotics and Automation, ICRA 2024, Yokohama, Japan, May 13-17, 2024 . IEEE, 2024, pp. 653–660

    2024

  3. [11]

    AirExo-2: Scaling up Generalizable Robotic Imitation Learning with Low-Cost Exoskeletons

    Hongjie Fang et al. “AirExo-2: Scaling up Generalizable Robotic Imitation Learning with Low-Cost Exoskeletons”. In: arXiv preprint arXiv:2503.03081 (2025)

    2025 arXiv

  4. [12]

    Keypoint Abstraction using Large Models for Object-Relative Imitation Learning

    Xiaolin Fang et al. “Keypoint Abstraction using Large Models for Object-Relative Imitation Learning”. In: arXiv preprint arXiv:2410.23254 (2024)

    2024 arXiv

  5. [13]

    Implicit behavioral cloning

    Pete Florence et al. “Implicit behavioral cloning”. In: Confer- ence on Robot Learning . PMLR. 2022, pp. 158–168

    2022

  6. [14]

    Act3D: 3D Feature Field Transform- ers for Multi-Task Robotic Manipulation

    Théophile Gervet et al. “Act3D: 3D Feature Field Transform- ers for Multi-Task Robotic Manipulation”. In: Conference on Robot Learning, CoRL 2023, 6-9 November 2023, Atlanta, GA, USA. Ed. by Jie Tan, Marc Toussaint, and Kourosh Darvish. V ol. 229. Proceedings of Machine Learni...

    2023

  7. [15]

    BAKU: An Efficient Transformer for Multi-Task Policy Learning

    Siddhant Haldar, Zhuoran Peng, and Lerrel Pinto. BAKU: An Efficient Transformer for Multi-Task Policy Learning . 2024

    2024

  8. [16]

    SPOT: SE(3) Pose Trajectory Dif- fusion for Object-Centric Manipulation

    Cheng-Chun Hsu et al. “SPOT: SE(3) Pose Trajectory Dif- fusion for Object-Centric Manipulation”. In: arXiv preprint arXiv:2411.00965 (2024)

    2024 arXiv

  9. [17]

    CoTracker: It is Better to Track To- gether

    Nikita Karaev et al. “CoTracker: It is Better to Track To- gether”. In: Proc. ECCV. 2024

    2024

  10. [18]

    DROID: A large-scale in-the- wild robot manipulation dataset

    Alexander Khazatsky et al. “DROID: A large-scale in-the- wild robot manipulation dataset”. In: Robotics: Science and Systems. 2024

    2024

  11. [19]

    OpenVLA: An Open-Source Vision-Language-Action Model

    Moo Jin Kim et al. “OpenVLA: An Open-Source Vision-Language-Action Model”. In: arXiv preprint arXiv:2406.09246 (2024)

    2024 arXiv

  12. [20]

    Segment Anything

    Alexander Kirillov et al. “Segment Anything”. In: arXiv:2304.02643 (2023)

    2023 arXiv

  13. [21]

    P3-PO: Prescriptive Point Priors for Visuo- Spatial Generalization of Robot Policies

    Mara Levy et al. P3-PO: Prescriptive Point Priors for Visuo- Spatial Generalization of Robot Policies . 2024

    2024

  14. [22]

    Data scaling laws in imitation learning for robotic manipulation

    Fanqi Lin et al. “Data scaling laws in imitation learning for robotic manipulation”. In: arXiv preprint arXiv:2410.18647 (2024)

    2024 arXiv

  15. [23]

    SpawnNet: Learning Generalizable Visuo- motor Skills from Pre-trained Network

    Xingyu Lin et al. “SpawnNet: Learning Generalizable Visuo- motor Skills from Pre-trained Network”. In: IEEE Interna- tional Conference on Robotics and Automation, ICRA 2024, Yokohama, Japan, May 13-17, 2024 . IEEE, 2024, pp. 4781– 4787

    2024

  16. [24]

    Rdt-1b: a diffusion foundation model for bimanual manipulation

    Songming Liu et al. “Rdt-1b: a diffusion foundation model for bimanual manipulation”. In: arXiv preprint arXiv:2410.07864 (2024)

    2024 arXiv

  17. [25]

    What matters in learning from offline human demonstrations for robot manipulation

    Ajay Mandlekar et al. “What matters in learning from offline human demonstrations for robot manipulation”. In: arXiv preprint arXiv:2108.03298 (2021)

    2021 arXiv

  18. [26]

    Open X-Embodiment: Robotic Learning Datasets and RT-X Models : Open X-Embodiment Collabora- tion

    Abby O’Neill et al. “Open X-Embodiment: Robotic Learning Datasets and RT-X Models : Open X-Embodiment Collabora- tion”. In: ICRA. IEEE, 2024, pp. 6892–6903

    2024

  19. [27]

    Octo: An Open-Source Generalist Robot Policy

    Octo Model Team et al. “Octo: An Open-Source Generalist Robot Policy”. In: Proceedings of Robotics: Science and Systems (RSS). 2024

    2024

  20. [28]

    DINOv2: Learning Robust Visual Fea- tures without Supervision

    Maxime Oquab et al. DINOv2: Learning Robust Visual Fea- tures without Supervision . 2023

    2023

  21. [29]

    R+x: Retrieval and execu- tion from everyday human videos

    Georgios Papagiannis et al. “R+x: Retrieval and execu- tion from everyday human videos”. In: arXiv preprint arXiv:2407.12957 (2024)

    2024 arXiv

  22. [30]

    ALVINN: An Autonomous Land Vehicle in a Neural Network

    Dean Pomerleau. “ALVINN: An Autonomous Land Vehicle in a Neural Network”. In: Advances in Neural Information Processing Systems 1, [NIPS Conference, Denver, Colorado, USA, 1988] . Ed. by David S. Touretzky. Morgan Kaufmann, 1988, pp. 305–313

    1988

  23. [31]

    Recasting Generic Pretrained Vision Transformers As Object-Centric Scene Encoders For Manipulation Policies

    Jianing Qian, Anastasios Panagopoulos, and Dinesh Jayara- man. “Recasting Generic Pretrained Vision Transformers As Object-Centric Scene Encoders For Manipulation Policies”. In: IEEE International Conference on Robotics and Automation, ICRA 2024, Yokohama, Japan, May 13-17, 20...

    2024

  24. [32]

    Behavior transformers: Cloning k modes with one stone

    Nur Muhammad Shafiullah et al. “Behavior transformers: Cloning k modes with one stone”. In: Advances in neural information processing systems 35 (2022), pp. 22955–22968

    2022

  25. [33]

    Perceiver- Actor: A Multi-Task Transformer for Robotic Manipulation

    Mohit Shridhar, Lucas Manuelli, and Dieter Fox. Perceiver- Actor: A Multi-Task Transformer for Robotic Manipulation . 2022

    2022

  26. [34]

    Denoising Diffusion Implicit Models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. “Denoising Diffusion Implicit Models”. In: 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Aus- tria, May 3-7, 2021 . OpenReview.net, 2021

    2021

  27. [35]

    Emergent Correspondence from Image Diffusion

    Luming Tang et al. “Emergent Correspondence from Image Diffusion”. In: Thirty-seventh Conference on Neural Informa- tion Processing Systems . 2023

    2023

  28. [36]

    Deep Object Pose Estimation for Semantic Robotic Grasping of Household Objects

    Jonathan Tremblay et al. Deep Object Pose Estimation for Semantic Robotic Grasping of Household Objects . 2018

    2018

  29. [37]

    6-DoF Pose Estimation of Household Objects for Robotic Manipulation: An Accessible Dataset and Benchmark

    Stephen Tyree et al. 6-DoF Pose Estimation of Household Objects for Robotic Manipulation: An Accessible Dataset and Benchmark. 2022

    2022

  30. [38]

    Least-squares estimation of transformation pa- rameters between two point patterns

    S. Umeyama. “Least-squares estimation of transformation pa- rameters between two point patterns”. In: IEEE Transactions on Pattern Analysis and Machine Intelligence 13.4 (1991), pp. 376–380

    1991

  31. [39]

    One- shot imitation learning: A pose estimation perspective

    Pietro Vitiello, Kamil Dreczkowski, and Edward Johns. “One- shot imitation learning: A pose estimation perspective”. In: arXiv preprint arXiv:2310.12077 (2023)

    2023 arXiv

  32. [40]

    Few-Shot In-Context Imitation Learning via Implicit Graph Alignment

    Vitalis V osylius and Edward Johns. “Few-Shot In-Context Imitation Learning via Implicit Graph Alignment”. In: Con- ference on Robot Learning. V ol. 229. Proceedings of Machine Learning Research. PMLR, 2023, pp. 3194–3213

    2023

  33. [41]

    Instant Policy: In-Context Imitation Learning via Graph Diffusion

    Vitalis V osylius and Edward Johns. “Instant Policy: In-Context Imitation Learning via Graph Diffusion”. In: arXiv preprint arXiv:2411.12633 (2024)

    2024 arXiv

  34. [42]

    BridgeData V2: A Dataset for Robot Learning at Scale

    Homer Rich Walke et al. “BridgeData V2: A Dataset for Robot Learning at Scale”. In: Conference on Robot Learning, CoRL 2023, 6-9 November 2023, Atlanta, GA, USA . Ed. by Jie Tan, Marc Toussaint, and Kourosh Darvish. V ol. 229. Proceedings of Machine Learning Research. PMLR, 20...

    2023

  35. [43]

    DexCap: Scalable and Portable Mocap Data Collection System for Dexterous Manipulation

    Chen Wang et al. “DexCap: Scalable and Portable Mocap Data Collection System for Dexterous Manipulation”. In: arXiv preprint arXiv:2403.07788 (2024)

    2024 arXiv

  36. [44]

    RISE: 3D Perception Makes Real-World Robot Imitation Simple and Effective

    Chenxi Wang et al. “RISE: 3D Perception Makes Real-World Robot Imitation Simple and Effective”. In: 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). 2024, pp. 2870–2877

    2024

  37. [45]

    Scaling proprioceptive-visual learning with heterogeneous pre-trained transformers

    Lirui Wang et al. “Scaling proprioceptive-visual learning with heterogeneous pre-trained transformers”. In: arXiv preprint arXiv:2409.20537 (2024)

    2024 arXiv

  38. [46]

    CAGE: Causal Attention Enables Data-Efficient Generalizable Robotic Manipulation

    Shangning Xia et al. “CAGE: Causal Attention Enables Data-Efficient Generalizable Robotic Manipulation”. In: arXiv preprint arXiv:2410.14974 (2024)

    2024 arXiv

  39. [47]

    3D Diffusion Policy: Generalizable Visuo- motor Policy Learning via Simple 3D Representations

    Yanjie Ze et al. “3D Diffusion Policy: Generalizable Visuo- motor Policy Learning via Simple 3D Representations”. In: Proceedings of Robotics: Science and Systems (RSS) . 2024

    2024

  40. [48]

    SAM-E: Leveraging Visual Foundation Model with Sequence Imitation for Embodied Manipulation

    Junjie Zhang et al. “SAM-E: Leveraging Visual Foundation Model with Sequence Imitation for Embodied Manipulation”. In: Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024 . OpenRe- view.net, 2024

    2024

  41. [49]

    Learning Fine-Grained Bimanual Manip- ulation with Low-Cost Hardware

    Tony Z. Zhao et al. “Learning Fine-Grained Bimanual Manip- ulation with Low-Cost Hardware”. In: Robotics: Science and Systems. 2023

    2023

  42. [50]

    On the Continuity of Rotation Representations in Neural Networks

    Yi Zhou et al. “On the Continuity of Rotation Representations in Neural Networks”. In: Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition . 2019, pp. 5745–5753

    2019

  43. [51]

    RT-2: Vision-Language-Action Mod- els Transfer Web Knowledge to Robotic Control

    Brianna Zitkovich et al. “RT-2: Vision-Language-Action Mod- els Transfer Web Knowledge to Robotic Control”. In: Confer- ence on Robot Learning, CoRL 2023, 6-9 November 2023, Atlanta, GA, USA . Ed. by Jie Tan, Marc Toussaint, and Kourosh Darvish. V ol. 229. Proceedings of Machi...

    2023

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.