Deep Generative Models in Robotics: A Survey on Learning from Multimodal Demonstrations
Reviewed by Pith T0 review T1 audit T2 compute T3 formal T4 reserved pith:7ECBK2BArecord.jsonopen to challenge →
read the original abstract
Learning from Demonstrations, the field that proposes to learn robot behavior models from data, is gaining popularity with the emergence of deep generative models. Although the problem has been studied for years under names such as Imitation Learning, Behavioral Cloning, or Inverse Reinforcement Learning, classical methods have relied on models that don't capture complex data distributions well or don't scale well to large numbers of demonstrations. In recent years, the robot learning community has shown increasing interest in using deep generative models to capture the complexity of large datasets. In this survey, we aim to provide a unified and comprehensive review of the last year's progress in the use of deep generative models in robotics. We present the different types of models that the community has explored, such as energy-based models, diffusion models, action value maps, or generative adversarial networks. We also present the different types of applications in which deep generative models have been used, from grasp generation to trajectory generation or cost learning. One of the most important elements of generative models is the generalization out of distributions. In our survey, we review the different decisions the community has made to improve the generalization of the learned models. Finally, we highlight the research challenges and propose a number of future directions for learning deep generative models in robotics.
This paper has not been read by Pith yet.
Forward citations
Cited by 6 Pith papers
-
Optimal Transport Q-Learning for Flow Policy Steering and Acceleration
Advantage-weighted conditional optimal transport flow matching simultaneously steers flow policies toward high-value actions and straightens their integration paths, enabling 2-3 step inference while improving task success.
-
Spacetime Optimal-Transport Attention for Visuo-Haptic Imitation Learning of Contact-Rich Manipulation
SO-TA replaces standard attention with optimal-transport alignment across vision, force/torque, and proprioception to improve diffusion-policy performance on real-robot insertion and wiping tasks.
-
Encoding Predictability and Legibility for Style-Conditioned Diffusion Policy
SCDP adds lightweight scene encoding and ambiguity detection to pre-trained diffusion policies so they produce legible motions only when goals are unclear and efficient motions otherwise.
-
CLARE: Continual Learning for Vision-Language-Action Models via Autonomous Adapter Routing and Expansion
CLARE is an exemplar-free continual learning framework for VLAs that autonomously expands modular adapters based on feature similarity and uses autoencoder routing for label-free deployment.
-
From Action Labels to Sets: Rethinking Action Supervision for Imitation Learning from Corrective Feedback
CLIC uses set-valued action targets from interactive human corrections instead of pointwise labels to train more robust imitation learning policies.
-
Surface Constraint Policy for Learning Surface-Constrained and Dynamically Feasible Robot Skills
SCP integrates diffusion-based imitation learning with Gaussian-encoded surface constraints and dynamic movement primitives to generate actions with improved success rates and contact stability on free-form surfaces.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.