REVIEW 14 cited by
A Survey of Sim-to-Real Methods in RL: Progress, Prospects and Challenges with Foundation Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
A Survey of Sim-to-Real Methods in RL: Progress, Prospects and Challenges with Foundation Models
read the original abstract
Deep Reinforcement Learning (RL) has been explored and verified to be effective in solving decision-making tasks in various domains, such as robotics, transportation, recommender systems, etc. It learns from the interaction with environments and updates the policy using the collected experience. However, due to the limited real-world data and unbearable consequences of taking detrimental actions, the learning of RL policy is mainly restricted within the simulators. This practice guarantees safety in learning but introduces an inevitable sim-to-real gap in terms of deployment, thus causing degraded performance and risks in execution. There are attempts to solve the sim-to-real problems from different domains with various techniques, especially in the era with emerging techniques such as large foundations or language models that have cast light on the sim-to-real. This survey paper, to the best of our knowledge, is the first taxonomy that formally frames the sim-to-real techniques from key elements of the Markov Decision Process (State, Action, Transition, and Reward). Based on the framework, we cover comprehensive literature from the classic to the most advanced methods including the sim-to-real techniques empowered by foundation models, and we also discuss the specialties that are worth attention in different domains of sim-to-real problems. Then we summarize the formal evaluation process of sim-to-real performance with accessible code or benchmarks. The challenges and opportunities are also presented to encourage future exploration of this direction. We are actively maintaining a repository to include the most up-to-date sim-to-real research work to help domain researchers.
Forward citations
Cited by 14 Pith papers
-
Targeting World Models to Compromise Robot Learning Pipelines
World models introduce a stealthy poisoning vector into robot learning pipelines where malicious prompts or dynamics in teleoperated data activate only during synthetic trajectory generation, enabling backdoors in dow...
-
EgoWalk: A Multimodal Dataset for Robot Navigation in the Wild
EgoWalk supplies 50 hours of real-world multimodal human navigation data in varied indoor/outdoor settings together with open pipelines that auto-generate language goal annotations and traversability masks.
-
FADA: Few-Shot Domain Adaptation via Dynamics Alignment for Humanoid Control
FADA is a three-stage Planner-IDM method that achieves few-shot domain adaptation for humanoid control by distilling an oracle policy then finetuning only the IDM on short target-domain rollouts via supervised learning.
-
Tac-DINO: Learning Vision-Tactile Features with Patch Alignment
Tac-DINO constructs a large tactile dataset and Vis-Tac Holographic Matching Benchmark, then proposes Vision-Tactile Patch Alignment (VTPA) methods that outperform non-aligned baselines on local-to-global feature matching.
-
In LLM Reasoning, there is Irrationality on top of Value Misalignment
LLMs display widespread rational value risk in reasoning that value alignment reduces but does not remove, with risk sensitive to inference strategy and showing diminishing returns from longer reasoning.
-
Application of Deep Reinforcement Learning to Event-Triggered Control for Networked Artificial Pancreas Systems
A DRL-based event-triggered controller for networked artificial pancreas systems uses blood glucose change rules to formulate control as a semi-Markov decision process, improving communication efficiency.
-
Mitigating Data Scarcity in Spaceflight Applications for Offline Reinforcement Learning Using Physics-Informed Deep Generative Models
MI-VAE generates physics-constrained synthetic trajectories from scarce real data to improve offline RL policy performance on planetary lander tasks over standard VAEs.
-
Rationality Measurement and Theory for Reinforcement Learning Agents
RL agents' rationality is quantified via expected value discrepancy to optimal actions, with the training-deployment gap decomposed and bounded by Wasserstein distance and Rademacher complexity, supported by experimen...
-
Decoupled Delay Compensation: Enhancing Pre-trained MARL Policies via Learned Dynamics Filtering
A decoupled estimator combining gated dynamics learning and recursive Kalman filtering improves robustness of pre-trained MARL policies under stale observations and message loss.
-
Sim-to-Real Transfer and Robustness Evaluation of Reinforcement Learning Control with Integrated Perception on an ASV for Floating Waste Capture
A DRL controller for ASV floating waste capture, trained in simulation with a perception abstraction module, achieves centimeter-level accuracy in real-world field experiments across 14 disturbance regimes.
-
Application of Deep Reinforcement Learning to Event-Triggered Control for Networked Artificial Pancreas Systems
A DRL-based event-triggered controller for artificial pancreas systems uses blood glucose change rules to reduce communication frequency while maintaining control performance via an SMDP formulation.
-
UniCon: A Unified System for Efficient Robot Learning Transfers
UniCon standardizes states and control logic into modular execution graphs for efficient transfer of learning controllers across heterogeneous robots, with lower latency than ROS.
-
The Sim-to-Real Gap of Foundation Model Agents: A Unified MDP Perspective
The paper proposes a unified MDP-based research agenda for addressing sim-to-real gaps in foundation model agents and advocates adopting classical solutions such as domain randomization.
-
IDEA: Insensitive to Dynamics Mismatch via Effect Alignment for Sim-to-Real Transfer in Multi-Agent Control
IDEA elevates multi-agent policies to semantic actions with effect alignment and synchronization for improved sim-to-real robustness on navigation tasks.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.