D4RL supplies new offline RL benchmarks and datasets from expert and mixed sources to expose weaknesses in existing algorithms and standardize evaluation.
hub
Challenges of Real-World Reinforcement Learning
17 Pith papers cite this work. Polarity classification is still indexing.
17
Pith papers citing it
abstract
Reinforcement learning (RL) has proven its worth in a series of artificial domains, and is beginning to show some successes in real-world scenarios. However, much of the research advances in RL are often hard to leverage in real-world systems due to a series of assumptions that are rarely satisfied in practice. We present a set of nine unique challenges that must be addressed to productionize RL to real world problems. For each of these challenges, we specify the exact meaning of the challenge, present some approaches from the literature, and specify some metrics for evaluating that challenge. An approach that addresses all nine challenges would be applicable to a large number of real world problems. We also present an example domain that has been modified to present these challenges as a testbed for practical RL research.
hub tools
citation-role summary
background 1
citation-polarity summary
roles
background 1polarities
unclear 1representative citing papers
PromptPO shows LLMs can act as black-box policy optimizers for sequential RL when leveraging prior knowledge, matching baselines in exploration and robotics but underperforming in MuJoCo.
UniSim learns a universal real-world simulator from orchestrated diverse datasets, enabling zero-shot deployment of policies trained purely in simulation.
A scalar-projection federated zeroth-order method for model-free LQR policy learning that reduces per-agent communication from O(d) to O(1) with convergence rate improving in the number of agents.
AutoSERL achieves strong performance on six real-world robot manipulation tasks using RL guided by a single demonstration via sliding-window intervention, safety recovery, and automatic termination.
Deep RL with action decomposition and reward shifting learns a symbolic multi-parameter policy for (1+(λ,λ))-GA on OneMax that outperforms baselines across problem sizes.
PROCO generates synthetic unsafe samples via model-based rollouts and LLM-grounded costs to enable safer policy learning from offline datasets containing few or no violations.
Introduces relativised options and hierarchical abstraction to reuse experience across similar contexts in offline GCRL, with two algorithms demonstrating performance gains.
Echo is a framework that harvests user-driven refinements of agent proposals as training signals to align models with real-world needs, demonstrated by raising code completion acceptance from 25.7% to 35.7% in production.
A three-stage framework pre-trains multi-agent RL agents on real safety-critical data, refines them via online learning in CARLA, and generates the VPSCI dataset of over 198,000 realistic vehicle-pedestrian interaction episodes.
LANTERN improves RL sample efficiency by 40-60% via LLM-generated task automata, semantic multi-source policy aggregation, and experience-gated adaptive transfer.
A hierarchical RL policy paired with a runtime safety shield using forward simulation achieves longer survival, lower line loading, and zero-shot generalization on Grid2Op benchmarks including stress tests and unseen large grids.
A hierarchical DRL method (TBH-DDPG) optimizes UAV trajectories at coarse granularity and bandwidth allocation at fine granularity, reporting 44.44% faster convergence and 58.05% lower computational cost than a non-hierarchical baseline in simulations.
The paper claims a fully quantum MDP model for RL with quantum state transitions, return calculation, and trajectory search that achieves quantum enhancement.
A survey reviewing the integration of generative models with connected and automated vehicles to enhance predictive modeling, simulation accuracy, and decision-making.
citing papers explorer
-
D4RL: Datasets for Deep Data-Driven Reinforcement Learning
D4RL supplies new offline RL benchmarks and datasets from expert and mixed sources to expose weaknesses in existing algorithms and standardize evaluation.
-
When are LLMs Sufficient Policy Optimizers for Sequential RL Tasks?
PromptPO shows LLMs can act as black-box policy optimizers for sequential RL when leveraging prior knowledge, matching baselines in exploration and robotics but underperforming in MuJoCo.
-
Learning Interactive Real-World Simulators
UniSim learns a universal real-world simulator from orchestrated diverse datasets, enabling zero-shot deployment of policies trained purely in simulation.
-
Scalar Federated Learning for Linear Quadratic Regulator
A scalar-projection federated zeroth-order method for model-free LQR policy learning that reduces per-agent communication from O(d) to O(1) with convergence rate improving in the number of agents.
-
One Demonstration Is Enough for Real-World Robotic Reinforcement Learning
AutoSERL achieves strong performance on six real-world robot manipulation tasks using RL guided by a single demonstration via sliding-window intervention, safety recovery, and automatic termination.
-
Discovering Interpretable Multi-Parameter Control Policies for Evolutionary Algorithms Using Deep Reinforcement Learning
Deep RL with action decomposition and reward shifting learns a symbolic multi-parameter policy for (1+(λ,λ))-GA on OneMax that outperforms baselines across problem sizes.
-
Model-Based Proactive Cost Generation for Learning Safe Policies Offline with Limited Violation Data
PROCO generates synthetic unsafe samples via model-based rollouts and LLM-grounded costs to enable safer policy learning from offline datasets containing few or no violations.
-
Abstraction for Offline Goal-Conditioned Reinforcement Learning
Introduces relativised options and hierarchical abstraction to reuse experience across similar contexts in offline GCRL, with two algorithms demonstrating performance gains.
-
Echo: Learning from Experience Data via User-Driven Refinement
Echo is a framework that harvests user-driven refinements of agent proposals as training signals to align models with real-world needs, demonstrated by raising code completion acceptance from 25.7% to 35.7% in production.
-
Generating Realistic Safety-Critical Scenarios for Vehicle-Pedestrian Interactions
A three-stage framework pre-trains multi-agent RL agents on real safety-critical data, refines them via online learning in CARLA, and generates the VPSCI dataset of over 198,000 realistic vehicle-pedestrian interaction episodes.
-
LANTERN: LLM-Augmented Neurosymbolic Transfer with Experience-Gated Reasoning Networks
LANTERN improves RL sample efficiency by 40-60% via LLM-generated task automata, semantic multi-source policy aggregation, and experience-gated adaptive transfer.
-
Hierarchical Reinforcement Learning with Runtime Safety Shielding for Power Grid Operation
A hierarchical RL policy paired with a runtime safety shield using forward simulation achieves longer survival, lower line loading, and zero-shot generalization on Grid2Op benchmarks including stress tests and unseen large grids.
-
UAV Trajectory and Bandwidth Allocation for Efficient Data Collection in Low-Altitude Intelligent IoT: A Hierarchical DRL Approach
A hierarchical DRL method (TBH-DDPG) optimizes UAV trajectories at coarse granularity and bandwidth allocation at fine granularity, reporting 44.44% faster convergence and 58.05% lower computational cost than a non-hierarchical baseline in simulations.
-
Quantum framework for Reinforcement Learning: Integrating Markov decision process, quantum arithmetic, and trajectory search
The paper claims a fully quantum MDP model for RL with quantum state transitions, return calculation, and trajectory search that achieves quantum enhancement.
-
Generative Models and Connected and Automated Vehicles: A Survey in Exploring the Intersection of Transportation and AI
A survey reviewing the integration of generative models with connected and automated vehicles to enhance predictive modeling, simulation accuracy, and decision-making.
- RESample: A Robust Data Augmentation Framework via Exploratory Sampling for Robotic Manipulation
- Higher Resolution, Better Generalization: Unlocking Visual Scaling in Deep Reinforcement Learning