Pith. sign in

REVIEW 5 major objections 6 minor 58 references

L2D2: Robot Learning from 2D Drawings

T0 review · 5 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read L2D2 argues that teaching robot arms by drawing on synthetically varied images, then grounding those drawings with a few physical corrections, yields policies as effective as full teleoperation while requiring far less human time and…

desk verdict Short-horizon results genuinely support L2D2's main claim; the long-horizon result is currently unsupported because of an impossible t-test statistic. read the letter →

arxiv 2505.12072 v1 pith:PS35SPIU submitted 2025-05-17 cs.RO

Shaunak A. Mehta , Heramb Nemlekar , Hari Sumant , Dylan P. Losey This is my paper · ORCID
classification cs.RO
keywords learningfromdemonstrationsketch-basedteaching2Dto3Dmappingsyntheticdataaugmentationbehaviorcloninghuman-robotinteractioncameraplacementvision-languagesegmentation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

L2D2 is a sketching interface and imitation learning pipeline that lets a human teach a robot arm by drawing trajectories on images of the workspace instead of physically guiding the robot. The paper's central claim is that combining many such drawings, collected quickly on synthetically varied images, with a small set of physical demonstrations produces robot policies as effective as those trained on much larger sets of full teleoperated demonstrations. In the reported experiments, 50 drawings plus 10 physical corrections matched 60 teleoperated demonstrations on Lift and Push tasks, and similar results appeared in a novice user study and a two-subtask long-horizon task. The reason this matters is that drawings remove the most costly parts of conventional data collection: resetting the environment between examples and physically operating the robot. If the claim holds, scaling up demonstration data for manipulation no longer requires proportional human physical effort.

What carries the argument

The central object is the L2D2 pipeline, which has four load-bearing parts. The sketching interface lets a person draw a 2D path for the end-effector, attach orientation and gripper open/close commands at selected points, and label task-relevant objects through text prompts. A camera-placement rule, derived from a PCA variance model, positions the camera so that the depth axis carries as little of the robot's motion variance as possible, minimizing the information lost when 3D states are projected to 2D pixels. A neural network $f_\phi$ maps drawn pixel points back to 3D robot positions, trained on task-agnostic pairs generated by projecting uniformly sampled workspace points. Finally, a vision-language object detector segments the labeled objects and repositions them in the image, so users can draw many demonstrations of the same task in varied scenes; a small physical dataset is then used to fine-tune $f_\phi$ and the behavior cloning policy sequentially. The information-loss identity $I_{\mathrm{loss}}(C)=\mathrm{Var}(z_C)/(\mathrm{Var}(x_C)+\mathrm{Var}(y_C)+\mathrm{Var}(z_C))$ is what connects camera placement to reconstruction quality.

What would settle it

Run a controlled comparison where the same set of object poses is presented either by physically resetting the workspace or by L2D2's random mask repositioning; if policies trained on drawings from synthetic images are systematically worse than those trained on drawings from real images at the same poses, or if users' drawings on synthetic images frequently intersect objects or leave the table, the claim that synthetic drawings are sufficient would be refuted.

Watch

Extended reading notes

Core claim

The paper claims that a robot can learn manipulation tasks from static, 2D drawings on synthetic images as effectively as from physical demonstrations, provided the drawings are grounded by a small real-world dataset. The grounding works in two steps: physical demonstrations are used to fine-tune the 2D-to-3D mapping, so that the drawing dataset is reconstructed more accurately, and then the policy is trained sequentially on the reconstructed drawings and the physical data. In the experiments, policies trained with 50 drawings and 10 physical corrections were statistically indistinguishable from policies trained with 60 teleoperated demonstrations in short-horizon tasks, and comparable performance held in a novice user study and a two-subtask long-horizon setting. The intended contribution is a demonstration that low-information, static sketches can be converted into dynamic, high-dimensional robot behavior with only a few real-world interventions.

Load-bearing premise

The load-bearing premise is that randomly repositioning object masks in the image creates scenes where users' drawings are still valid demonstrations of the task; if the synthetic scenes are physically implausible or if users draw inaccurately on them, the large drawing dataset will contain misleading examples that the few physical corrections may not be able to fix.

Editorial extensions

If this is right

  • Teaching effort and time scale more favorably because environment resets are replaced by synthetic image variation, so the human cost per additional demonstration is mostly the time to draw.
  • A small amount of physical data can anchor a much larger drawing dataset, since the physical corrections are used to recalibrate the 2D-to-3D mapping and then to fine-tune the policy trained on reconstructed drawings.
  • Drawing-based teaching is not limited to short single-object tasks: the long-horizon experiment shows comparable success to teleoperation when the task consists of two sequential subtasks.
  • User preference and perceived effort shift in favor of drawing interfaces: in the user study, most participants preferred drawing over teleoperation and preferred L2D2 over the other drawing baselines.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The information-loss analysis implies a general design rule for any sketch-based teaching system: position the camera so that task-relevant motion is spread across the image plane rather than along the viewing axis, a rule that could transfer to other interfaces without L2D2's other components.
  • Random mask repositioning likely works when the task is roughly planar and the physical correction set is diverse enough to correct errors; tasks with stacking, containment, or coupling between objects would probably need physics-aware augmentation or filtering.
  • The gripper-based heuristic for simulating object motion is a simplification; replacing it with learned video prediction or physics simulation is a natural testable extension that could reduce the number of physical corrections required.
  • A direct stress test would be to increase the distance between training and test object distributions to see whether the diversity from synthetic drawings generalizes beyond the range of object positions covered by the physical corrections.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper proposes L2D2, a sketching interface and imitation learning pipeline in which human users draw end-effector trajectories, orientations, and gripper commands on images of the robot workspace. A vision-language detector segments task-relevant objects, and the system generates synthetic images with varied object placements so that users can draw diverse demonstrations without physically resetting the environment. A task-agnostic neural mapping converts 2D sketch points to 3D positions, object dynamics are approximately simulated, and a small set of physical demonstrations is used to fine-tune both the mapping and the behavior-cloned policy. The manuscript reports short-horizon and long-horizon tabletop experiments with expert users, a 12-participant user study comparing L2D2 with teleoperation and two sketch-based baselines, and a PCA-based camera placement analysis.

Significance. If the results hold, L2D2 addresses a real bottleneck in imitation learning: the cost of collecting diverse physical demonstrations. The reported short-horizon experiments and user study support the claim that drawings plus a few physical corrections can match teleoperation on single-object tasks while reducing user time and effort, and the subjective results are consistent with that conclusion. The PCA-based camera placement derivation is internally coherent and has no fitted parameters, and the paper makes its code and project materials available, which are strengths. The ablations (L2D2-D and Teleop-min) are well chosen and help separate the contributions of diverse drawings and physical grounding. However, the long-horizon generalization claim in the abstract and conclusion is currently supported only by an internally inconsistent test statistic, so the overall evidence for the broadest claim is incomplete.

major comments (5)
  1. [§5.3, Results] The reported test statistic is internally inconsistent: for an independent-samples t-test with 18 degrees of freedom, t = 11.37 corresponds to p < 0.001 (two-tailed), while p = 0.441 would require |t| ≈ 0.79. Because no means, SEMs, or effect sizes are reported for the Long Horizon rollouts, the current text does not establish that L2D2 and Teleop performed similarly. Since the abstract and conclusion explicitly extend the claim to longer-horizon tasks, this is a load-bearing issue; please correct the statistic or provide the raw data and effect sizes.
  2. [§4.2, Eq. (9)] The inequality Inonlinear_loss(C) ≤ Iloss(C) is presented as a consequence of using a nonlinear mapping, and the abstract says that an upper bound is established, but no proof or precise formal statement is given. The cited prior work concerns autoencoders versus PCA in general, not the specific perspective-projection reconstruction problem here; moreover the inequality cannot hold for arbitrary learned f_phi without assumptions about the function class and training optimality. Please either provide a proof for the stated setup or reclassify this as an empirical observation with supporting experiments.
  3. [§5, Experimental Setup] The camera placement analysis in Section 4.2 is a stated contribution, but no experiment varies the camera position to test whether the PCA-optimal placement improves reconstruction accuracy or policy success relative to alternative placements. The paper only says that the camera was placed 'as best as we can'. A comparison (e.g., PCA-optimal versus a fixed non-optimal viewpoint, holding all else equal) would be needed to validate Eq. (7) empirically.
  4. [§4.1, Diverse Images] The synthetic-image pipeline randomly repositions object masks and inpaints the vacated regions, but the manuscript does not validate that the resulting scenes are physically plausible or that user drawings on synthetic images are as accurate as drawings on real images. The assertion that 'moving objects to random image locations introduces sufficient variety' is not supported by quantitative evidence. Since the diversity claim is central to the data-collection contribution, please validate this step (e.g., by comparing drawing accuracy or policy success when synthetic images are replaced by real images with the same object placements, or by filtering implausible placements).
  5. [§5.1, Baselines] The RT-Traj baseline is simplified by feeding sketched trajectories directly instead of sketch images as in the original paper, and both RT-Traj and S2S use 160 task-specific demonstrations to train the mapping. This may be a reasonable data-budget choice, but as described it weakens the claim that L2D2 outperforms the state-of-the-art sketch-based methods, because the comparison may be against degraded versions of those methods. Please justify that the simplified baselines are representative, or report their original-form performance where feasible.
minor comments (6)
  1. [§5.2, Dependent Variables] The text says 'Similar to Section 5.2' when describing the evaluation; this should refer to Section 5.1.
  2. [§5.2, Results] The intuitiveness ANOVA is reported as 'F (2.01, 24.1), p < 0.05' without the F-statistic value; please insert it.
  3. [§5.1 and §5.3] Post-hoc comparisons are reported without a stated multiple-comparison correction; please state the procedure used (e.g., Tukey, Bonferroni, or the specific method embedded in the software).
  4. [Figure 2] The axis label 'rotxrotyrotz' appears to be a concatenation of rotx, roty, and rotz; format it clearly for readability.
  5. [References] Reference [24] (Jolliffe and Cadima) has incomplete bibliographic information; please add the volume, article number, or page range.
  6. [§4.2, Equations (5)-(7)] The notation P_C is used both for the set of robot states in the camera frame and for the data matrix in Eq. (6); please distinguish the two to avoid confusion.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: the main claims are grounded in physical rollouts and external baselines; self-citations are not load-bearing.

full rationale

L2D2's claimed derivation chain does not reduce to its inputs by construction. The information-loss analysis (Eq. 4) defines loss as the variance fraction along the camera's z-axis; minimizing it via PCA (Eqs. 5-7) is a standard optimization on robot-state samples, and the resulting camera placement is evaluated independently by physical rollouts. The 2D-to-3D mapping f_phi is learned on a task-agnostic calibration dataset D_map (Eq. 8, Algorithm 1); it is later fine-tuned with D_R (Eq. 10), but the final policy is trained on both reconstructed drawing data and real physical demonstrations (Eq. 11) and scored by real robot executions. Success rates in Sections 5.1-5.3 are measured outcomes, not values fitted into the derivation. No load-bearing step is justified solely by a self-citation: references to the authors' prior work ([25], [31]-[33]) appear in related-work or background contexts, and the method cites external sources for PCA, behavior cloning, vision-language segmentation, and the sketch baselines. The only notable problem is a statistical-reporting error in Section 5.3: t(18)=11.37 with p=0.441 are mutually inconsistent, so the long-horizon equivalence claim is not numerically supported as printed; this is a correctness issue, not a circularity issue.

Assumptions & free parameters 4 free parameters · 6 assumptions · 0 invented entities

The central claim does not rest on any new physical entity. It rests on two learned models (the 2D-to-3D mapping f_phi and the behavior-cloning policy), a handful of hand-chosen experimental parameters (number of physical demos, object-attachment distance threshold), and several domain assumptions, most notably that randomly repositioned objects in synthetic images remain interpretable and physically plausible for drawing demonstrations, and that a few physical corrections suffice to ground the drawings. The PCA camera placement uses standard linear algebra and is not a free parameter.

free parameters (4)
  • f_phi network weights (2D-to-3D mapping) = Learned from calibration set Dmap; fine-tuned on DR
    The conversion of drawn pixels to 3D robot positions depends entirely on this learned mapping. Its accuracy is central to the method, but it is fitted to synthetic calibration data and a few physical demos, not derived.
  • pi_theta policy network weights = Trained via behavior cloning on D~R then DR
    The final policy is fit to reconstructed drawing data and physical corrections. Performance claims depend on this fit, which is standard for learned policies.
  • Object-attachment distance threshold = Pre-specified, exact value not reported
    Section 4.3 couples object motion to the gripper when closed within a pre-specified distance. This hand-chosen threshold determines the simulated object dynamics in the reconstructed dataset.
  • Ratio of drawings to physical demos (e.g., 50/10, 4/1, 100/25) = Varies by experiment: 50/10 in Sec 5.1; 4/1 in Sec 5.2; 100/25 in Sec 5.3
    The claim that a 'small set' of physical corrections suffices is evaluated at these specific ratios, which are chosen by the authors and not swept.
assumptions (6)
  • standard math PCA eigenvectors identify the 2D projection plane that minimizes variance loss
    Used in Eq. 6-7 to derive the optimal camera orientation via the last eigenvector.
  • standard math Pinhole camera model with known intrinsics and fixed pose; depth is the only lost information
    Eq. 2-3 define the projection; the information-loss measure in Eq. 4 follows from dropping zC.
  • domain assumption Randomly repositioned object masks yield synthetic images that users can draw valid trajectories on
    Section 4.1 states this is 'sufficient variety'; no feasibility filtering or accuracy validation is provided.
  • domain assumption A small set of physical demonstrations can correct systematic errors in f_phi via fine-tuning (Eq. 10) and ground the policy via sequential BC (Eq. 11)
    The grounding stage assumes few physical demos are enough to remove reconstruction bias and teach contact dynamics.
  • domain assumption Task states are well-represented by end-effector pose, gripper state, and detected object positions
    Section 3 definition of state s=(sR,o); the object dynamics model in Section 4.3 relies on this.
  • standard math Behavior cloning with an MLP can represent the demonstrated manipulation policies
    Eq. 1 is used for all experiments; this is a standard assumption in imitation learning.

how reviews work

0 comments
Cite this review

Pith. "Pith review of L2D2: Robot Learning from 2D Drawings." pith.science (2026). https://pith.science/paper/PS35SPIU

@misc{pith2026250512072,
  author       = {Pith},
  title        = {Pith review of: L2D2: Robot Learning from 2D Drawings},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PS35SPIU}},
  note         = {Machine review of arXiv:2505.12072}
}
read the original abstract

Robots should learn new tasks from humans. But how do humans convey what they want the robot to do? Existing methods largely rely on humans physically guiding the robot arm throughout their intended task. Unfortunately -- as we scale up the amount of data -- physical guidance becomes prohibitively burdensome. Not only do humans need to operate robot hardware but also modify the environment (e.g., moving and resetting objects) to provide multiple task examples. In this work we propose L2D2, a sketching interface and imitation learning algorithm where humans can provide demonstrations by drawing the task. L2D2 starts with a single image of the robot arm and its workspace. Using a tablet, users draw and label trajectories on this image to illustrate how the robot should act. To collect new and diverse demonstrations, we no longer need the human to physically reset the workspace; instead, L2D2 leverages vision-language segmentation to autonomously vary object locations and generate synthetic images for the human to draw upon. We recognize that drawing trajectories is not as information-rich as physically demonstrating the task. Drawings are 2-dimensional and do not capture how the robot's actions affect its environment. To address these fundamental challenges the next stage of L2D2 grounds the human's static, 2D drawings in our dynamic, 3D world by leveraging a small set of physical demonstrations. Our experiments and user study suggest that L2D2 enables humans to provide more demonstrations with less time and effort than traditional approaches, and users prefer drawings over physical manipulation. When compared to other drawing-based approaches, we find that L2D2 learns more performant robot policies, requires a smaller dataset, and can generalize to longer-horizon tasks. See our project website: https://collab.me.vt.edu/L2D2/

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

58 extracted references · 53 canonical work pages

  1. [1]

    In: IEEE International Conference on Robotics and Automation, pp

    Alakuijala, M., Dulac-Arnold, G., Mairal, J., Ponce, J., Schmid, C.: Learning reward functions for robotic ma- nipulation by observing humans. In: IEEE International Conference on Robotics and Automation, pp. 5006–5012 (2023)

    work page 2023

  2. [2]

    Robotics and Autonomous Systems 57(5), 469–483 (2009)

    Argall, B.D., Chernova, S., Veloso, M., Browning, B.: A survey of robot learning from demonstration. Robotics and Autonomous Systems 57(5), 469–483 (2009)

    work page 2009

  3. [3]

    Deep Learning Applications pp

    Bahadur, N., Lewandowski, B., Paffenroth, R.: Dimen- sion estimation using autoencoders and application. Deep Learning Applications pp. 95–121 (2022)

    work page 2022

  4. [4]

    In: Robotics: Science and Systems (2022)

    Bahl, S., Gupta, A., Pathak, D.: Human-to-robot imi- tation in the wild. In: Robotics: Science and Systems (2022)

    work page 2022

  5. [5]

    Advances in Neural Information Processing Systems pp

    Belkhale, S., Cui, Y., Sadigh, D.: Data quality in imita- tion learning. Advances in Neural Information Processing Systems pp. 80375–80395 (2023)

    work page 2023

  6. [6]

    In: 1st Workshop on X-Embodiment Robot Learning (2024)

    Bharadhwaj, H., Dwibedi, D., Gupta, A., Tulsiani, S., Doersch, C., Xiao, T., Shah, D., Xia, F., Sadigh, D., Kir- mani, S.: Gen2Act: Human video generation in novel sce- narios enables generalizable robot manipulation. In: 1st Workshop on X-Embodiment Robot Learning (2024)

    work page 2024

  7. [7]

    In: European Conference on Computer Vision, pp

    Bharadhwaj, H., Mottaghi, R., Gupta, A., Tulsiani, S.: Track2Act: Predicting point tracks from internet videos enables generalizable robot manipulation. In: European Conference on Computer Vision, pp. 306–324 (2024)

    work page 2024

  8. [8]

    arXiv preprint arXiv:2212.06817 (2022)

    Brohan, A., Brown, N., Carbajal, J., Chebotar, Y., Dabis, J., Finn, C., Gopalakrishnan, K., Hausman, K., Herzog, A., Hsu, J., et al.: RT-1: Robotics transformer for real- world control at scale. arXiv preprint arXiv:2212.06817 (2022)

    arXiv 2022

Show all 58 references
  1. [9]

    Quality Engineering pp

    Cacciarelli, D., Kulahci, M.: Hidden dimensions of the data: PCA vs autoencoders. Quality Engineering pp. 741–750 (2023)

    2023

  2. [10]

    In: Conference on Robot Learning, pp

    Dasari, S., Gupta, A.: Transformers for one-shot visual imitation. In: Conference on Robot Learning, pp. 2071– 2084 (2021)

    2021

  3. [11]

    Advances in Neural Infor- mation Processing Systems (NeurIPS) 30 (2017)

    Duan, Y., Andrychowicz, M., Stadie, B., Jonathan Ho, O., Schneider, J., Sutskever, I., Abbeel, P., Zaremba, W.: One-shot imitation learning. Advances in Neural Infor- mation Processing Systems (NeurIPS) 30 (2017)

    2017

  4. [12]

    In: Robotics: Science and Systems (RSS) (2022)

    Ebert, F., Yang, Y., Schmeckpeper, K., Bucher, B., Geor- gakis, G., Daniilidis, K., Finn, C., Levine, S.: Bridge data: Boosting generalization of robotic skills with cross- domain datasets. In: Robotics: Science and Systems (RSS) (2022)

    2022

  5. [13]

    In: IEEE International Conference on Robotics and Automation, pp

    Fang, H., Fang, H.S., Wang, Y., Ren, J., Chen, J., Zhang, R., Wang, W., Lu, C.: AirExo: Low-cost exoskeletons for learning whole-arm manipulation in the wild. In: IEEE International Conference on Robotics and Automation, pp. 15031–15038 (2024)

    2024

  6. [14]

    In: IEEE International Conference on Artifi- cial Intelligence and Knowledge Engineering, pp

    Fournier, Q., Aloise, D.: Empirical comparison between autoencoders and traditional dimensionality reduction methods. In: IEEE International Conference on Artifi- cial Intelligence and Knowledge Engineering, pp. 211–214 (2019)

    2019

  7. [15]

    In: Conference on Robot Learning (2024)

    Fu, Z., Zhao, T.Z., Finn, C.: Mobile ALOHA: Learn- ing bimanual mobile manipulation using low-cost whole- body teleoperation. In: Conference on Robot Learning (2024)

    2024

  8. [16]

    SoftwareX 29, 102054 (2025)

    George, A., Bartsch, A., Farimani, A.B.: Openvr: Teleop- eration for manipulation. SoftwareX 29, 102054 (2025)

    2025

  9. [17]

    In: Interna- tional Conference on Learning Representations (2024)

    Gu, J., Kirmani, S., Wohlhart, P., Lu, Y., Arenas, M.G., Rao, K., Yu, W., Fu, C., Gopalakrishnan, K., Xu, Z., Sundaresan, P., Xu, P., Su, H., Hausman, K., Finn, C., Vuong, Q., Xiao, T.: RT-trajectory: Robotic task gener- alization via hindsight trajectory sketches. In: Interna...

    2024

  10. [18]

    Cambridge University Press (2003)

    Hartley, R., Zisserman, A.: Multiple view geometry in computer vision. Cambridge University Press (2003)

    2003

  11. [19]

    In: First Work- shop on Out-of-Distribution Generalization in Robotics at CoRL 2023 (2023)

    Hoque, R., Mandlekar, A., Garrett, C.R., Goldberg, K., Fox, D.: Interventional data generation for robust and data-efficient robot imitation learning. In: First Work- shop on Out-of-Distribution Generalization in Robotics at CoRL 2023 (2023)

    2023

  12. [20]

    In: Conference on Robot Learning (2024)

    Iyer, A., Peng, Z., Dai, Y., Guzey, I., Haldar, S., Chin- tala, S., Pinto, L.: OPEN TEACH: A versatile teleopera- tion system for robotic manipulation. In: Conference on Robot Learning (2024)

    2024

  13. [21]

    In: Robotics: Science and Systems (RSS) (2024)

    Jain, V., Attarian, M., Joshi, N.J., Wahid, A., Driess, D., Vuong, Q., Sanketi, P.R., Sermanet, P., Welker, S., Chan, C., et al.: Vid2Robot: End-to-end video-conditioned policy learning with cross-attention transformers. In: Robotics: Science and Systems (RSS) (2024)

    2024

  14. [22]

    In: Con- ference on Robot Learning, pp

    James, S., Bloesch, M., Davison, A.J.: Task-embedded control networks for few-shot imitation learning. In: Con- ference on Robot Learning, pp. 783–795 (2018)

    2018

  15. [23]

    In: Con- ference on Robot Learning, pp

    Jang, E., Irpan, A., Khansari, M., Kappler, D., Ebert, F., Lynch, C., Levine, S., Finn, C.: BC-Z: Zero-shot task generalization with robotic imitation learning. In: Con- ference on Robot Learning, pp. 991–1002 (2022)

    2022

  16. [24]

    Philosophical trans- actions of the royal society A: Mathematical, Physical and Engineering Sciences

    Jolliffe, I.T., Cadima, J.: Principal component analysis: A review and recent developments. Philosophical trans- actions of the royal society A: Mathematical, Physical and Engineering Sciences

  17. [25]

    ACM Transac- tions on Human-Robot Interaction (2024)

    Jonnavittula, A., Mehta, S.A., Losey, D.P.: SARI: Shared autonomy across repeated interaction. ACM Transac- tions on Human-Robot Interaction (2024)

    2024

  18. [26]

    Autonomous Robots 49(1), 1–26 (2025)

    Jonnavittula, A., Parekh, S., P Losey, D.: VIEW: Visual imitation learning with waypoints. Autonomous Robots 49(1), 1–26 (2025)

    2025

  19. [27]

    In: Conference on Robot Learning (2017)

    Laskey, M., Lee, J., Fox, R., Dragan, A., Goldberg, K.: DART: Noise injection for robust imitation learning. In: Conference on Robot Learning (2017)

    2017

  20. [28]

    Advances in Neural Informa- tion Processing Systems (2024)

    Liu, B., Zhu, Y., Gao, C., Feng, Y., Liu, Q., Zhu, Y., Stone, P.: LIBERO: Benchmarking knowledge transfer for lifelong robot learning. Advances in Neural Informa- tion Processing Systems (2024)

    2024

  21. [29]

    In: Robotics: Science and Systems (2021)

    Lynch, C., Sermanet, P.: Language conditioned imitation learning over unstructured data. In: Robotics: Science and Systems (2021)

    2021

  22. [30]

    In: CoRL 2022 Workshop on Pre-Training Robot Learning (2022) 18 Shaunak A

    Mandi, Z., Bharadhwaj, H., Moens, V., Song, S., Ra- jeswaran, A., Kumar, V.: CACTI: A framework for scal- able multi-task multi-scene visual imitation learning. In: CoRL 2022 Workshop on Pre-Training Robot Learning (2022) 18 Shaunak A. Mehta et al

    2022

  23. [31]

    IEEE Robotics and Automation Letters (2025)

    Mehta, S.A., Ciftci, Y.U., Ramachandran, B., Bansal, S., Losey, D.P.: Stable-BC: Controlling covariate shift with stable behavior cloning. IEEE Robotics and Automation Letters (2025)

    2025

  24. [32]

    ACM Transactions on Human- Robot Interaction 13(3) (2024)

    Mehta, S.A., Losey, D.P.: Unified learning from demon- strations, corrections, and preferences during physical human–robot interaction. ACM Transactions on Human- Robot Interaction 13(3) (2024)

    2024

  25. [33]

    In: IEEE Inter- national Conference on Robotics and Automation, pp

    Mehta, S.A., Parekh, S., Losey, D.P.: Learning latent ac- tions without human demonstrations. In: IEEE Inter- national Conference on Robotics and Automation, pp. 7437–7443 (2022)

    2022

  26. [34]

    In: IEEE/CVF Conference on Computer Vision and Pattern Recogni- tion, pp

    Nguyen, K., Dey, D., Brockett, C., Dolan, B.: Vision- based navigation with language-based assistance via imi- tation learning with indirect intervention. In: IEEE/CVF Conference on Computer Vision and Pattern Recogni- tion, pp. 12527–12537 (2019)

    2019

  27. [35]

    Foundations and Trends in Robotics 7(1-2), 1–179 (2018)

    Osa, T., Pajarinen, J., Neumann, G., Bagnell, J.A., Abbeel, P., Peters, J.: An algorithmic perspective on im- itation learning. Foundations and Trends in Robotics 7(1-2), 1–179 (2018)

    2018

  28. [36]

    In: IEEE International Conference on Robotics and Automation, pp

    O’Neill, A., Rehman, A., Maddukuri, A., Gupta, A., Padalkar, A., Lee, A., Pooley, A., Gupta, A., Mandlekar, A., Jain, A., et al.: Open X-Embodiment: Robotic learn- ing datasets and RT-X models. In: IEEE International Conference on Robotics and Automation, pp. 6892–6903 (2024)

    2024

  29. [37]

    In: Robotics: Science and Systems (2022)

    Pari, J., Shafiullah, N.M.M., Arunachalam, S.P., Pinto, L.: The surprising effectiveness of representation learning for visual imitation. In: Robotics: Science and Systems (2022)

    2022

  30. [38]

    Advances in Neural Information Processing Systems 1 (1988)

    Pomerleau, D.A.: Alvinn: An autonomous land vehicle in a neural network. Advances in Neural Information Processing Systems 1 (1988)

    1988

  31. [39]

    Annual Review of Control, Robotics, and Au- tonomous Systems 3(1), 297–330 (2020)

    Ravichandar, H., Polydoros, A.S., Chernova, S., Billard, A.: Recent advances in robot learning from demonstra- tion. Annual Review of Control, Robotics, and Au- tonomous Systems 3(1), 297–330 (2020)

    2020

  32. [40]

    In: Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp

    Rolinek, M., Zietlow, D., Martius, G.: Variational autoen- coders pursue PCA directions (by accident). In: Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12406–12415 (2019)

    2019

  33. [41]

    In: International Conference on Artificial Intelligence and Statistics (2011)

    Ross, S., Gordon, G., Bagnell, D.: A reduction of imita- tion learning and structured prediction to no-regret on- line learning. In: International Conference on Artificial Intelligence and Statistics (2011)

    2011

  34. [42]

    arXiv preprint arXiv:2311.16098 (2023)

    Shafiullah, N.M.M., Rai, A., Etukuru, H., Liu, Y., Misra, I., Chintala, S., Pinto, L.: On bringing robots home. arXiv preprint arXiv:2311.16098 (2023)

    2023 arXiv

  35. [43]

    Proceedings of the IEEE 100(3), 604–622 (2012)

    Shah, D., Schneider, J., Campbell, M.: A sketch interface for robust and natural robot control. Proceedings of the IEEE 100(3), 604–622 (2012)

    2012

  36. [44]

    Journal of the Robotics Society of Japan 22(4), 519–527 (2004)

    Sian, N.E., Yokoi, K., Kajita, S., Kanehiro, F., Tanie, K.: Whole body teleoperation of a humanoid robot develop- ment of a simple master device using joysticks. Journal of the Robotics Society of Japan 22(4), 519–527 (2004)

    2004

  37. [45]

    IEEE Robotics and Automation Letters 5(3), 4978–4985 (2020)

    Song, S., Zeng, A., Lee, J., Funkhouser, T.: Grasping in the wild: Learning 6DOF closed-loop grasping from low- cost demonstrations. IEEE Robotics and Automation Letters 5(3), 4978–4985 (2020)

    2020

  38. [46]

    Advances in Neural Information Processing Systems pp

    Stepputtis, S., Campbell, J., Phielipp, M., Lee, S., Baral, C., Ben Amor, H.: Language-conditioned imitation learn- ing for robot manipulation tasks. Advances in Neural Information Processing Systems pp. 13139–13150 (2020)

    2020

  39. [47]

    In: Conference on Robot Learning (2024)

    Sundaresan, P., Vuong, Q., Gu, J., Xu, P., Xiao, T., Kirmani, S., Yu, T., Stark, M., Jain, A., Haus- man, K., Sadigh, D., Bohg, J., Schaal, S.: RT-sketch: Goal-conditioned imitation learning from hand-drawn sketches. In: Conference on Robot Learning (2024)

    2024

  40. [48]

    arXiv preprint arXiv:2412.19153 (2024)

    Tanada, K., Iwanaga, Y., Tsuchinaga, M., Nakamura, Y., Mori, T., Sakai, R., Yamamoto, T.: Sketch-MoMa: Tele- operation for mobile manipulator via interpretation of hand-drawn sketches. arXiv preprint arXiv:2412.19153 (2024)

    2024 arXiv

  41. [49]

    In: IEEE/RSJ International Con- ference on Intelligent Robots and Systems, pp

    Wu, P., Shentu, Y., Yi, Z., Lin, X., Abbeel, P.: GELLO: A general, low-cost, and intuitive teleoperation framework for robot manipulators. In: IEEE/RSJ International Con- ference on Intelligent Robots and Systems, pp. 12156– 12163 (2024)

    2024

  42. [50]

    In: Conference on Robot Learning, pp

    Young, S., Gandhi, D., Tulsiani, S., Gupta, A., Abbeel, P., Pinto, L.: Visual imitation made easy. In: Conference on Robot Learning, pp. 1992–2005 (2021)

    2021

  43. [51]

    arXiv preprint arXiv:2503.11918 (2025)

    Yu, P., Bhaskar, A., Singh, A., Mahammad, Z., Tokekar, P.: Sketch-to-Skill: Bootstrapping robot learning with human drawn trajectory sketches. arXiv preprint arXiv:2503.11918 (2025)

    2025 arXiv

  44. [52]

    Yu, T., Xiao, T., Stone, A., Tompson, J., Brohan, A., Wang, S., Singh, J., Tan, C., Peralta, J., Ichter, B., et al.: Scaling robot learning with semantically imagined expe- rience (2023)

    2023

  45. [53]

    In: IEEE International Conference on Robotics and Automation, pp

    Zhang, T., McCarthy, Z., Jow, O., Lee, D., Chen, X., Goldberg, K., Abbeel, P.: Deep imitation learning for complex manipulation tasks from virtual reality teleop- eration. In: IEEE International Conference on Robotics and Automation, pp. 5628–5635 (2018)

    2018

  46. [54]

    In: Robotics: Science and Systems (2024)

    Zhang, X., Chang, M., Kumar, P., Gupta, S.: Diffu- sion meets DAgger: Supercharging eye-in-hand imitation learning. In: Robotics: Science and Systems (2024)

    2024

  47. [55]

    arXiv preprint arXiv:2306.12156 (2023)

    Zhao, X., Ding, W., An, Y., Du, Y., Yu, T., Li, M., Tang, M., Wang, J.: Fast segment anything. arXiv preprint arXiv:2306.12156 (2023)

    2023 arXiv

  48. [56]

    ACM Transactions on Graphics (2012)

    Zheng, Y., Chen, X., Cheng, M.M., Zhou, K., Hu, S.M., Mitra, N.J.: Interactive images: Cuboid proxies for smart image manipulation. ACM Transactions on Graphics (2012)

    2012

  49. [57]

    In: IEEE Inter- national Conference on Robotics and Automation, pp

    Zhi, W., Zhang, T., Johnson-Roberson, M.: Instructing robots by sketching: Learning from demonstration via probabilistic diagrammatic teaching. In: IEEE Inter- national Conference on Robotics and Automation, pp. 15047–15053 (2024)

    2024

  50. [58]

    In: European Conference on Computer Vi- sion, pp

    Zhou, X., Girdhar, R., Joulin, A., Kr¨ ahenb¨ uhl, P., Misra, I.: Detecting twenty-thousand classes using image-level supervision. In: European Conference on Computer Vi- sion, pp. 350–368 (2022)

    2022

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.