CBF-RL: Safety Filtering Reinforcement Learning in Training with Control Barrier Functions
read the original abstract
Reinforcement learning (RL), while powerful and expressive, can often prioritize performance at the expense of safety. Yet safety violations can lead to catastrophic outcomes in real-world deployments. Control Barrier Functions (CBFs) offer a principled method to enforce dynamic safety -- traditionally deployed online via safety filters. While the result is safe behavior, the fact that the RL policy does not have knowledge of the CBF can lead to conservative behaviors. This paper proposes CBF-RL, a framework for generating safe behaviors with RL by enforcing CBFs in training. CBF-RL has two key attributes: (1) minimally modifying a nominal RL policy to encode safety constraints via a CBF term, (2) and safety filtering of the policy rollouts in training. Theoretically, we prove that continuous-time safety filters can be deployed via closed-form expressions on discrete-time roll-outs. Practically, we demonstrate that CBF-RL internalizes the safety constraints in the learned policy -- both enforcing safer actions and biasing towards safer rewards -- enabling safe deployment without the need for an online safety filter. We validate our framework through ablation studies on navigation tasks and on the Unitree G1 humanoid robot, where CBF-RL enables safer exploration, faster convergence, and robust performance under uncertainty, enabling the humanoid robot to avoid obstacles and climb stairs safely in real-world settings without a runtime safety filter.
This paper has not been read by Pith yet.
Forward citations
Cited by 10 Pith papers
-
MPC-Injection: Biasing Off-Policy Locomotion RL Toward Controller-Induced Behavior Basins
MPC-Injection biases off-policy RL locomotion policies toward controller-induced behavior basins by injecting MPC transitions into the replay buffer.
-
LIBERO-Safety: A Comprehensive Benchmark for Physical and Semantic Safety in Vision-Language-Action Models
LIBERO-Safety supplies a scalable benchmark, data-generation pipeline, and 19,664-demonstration dataset that exposes a generalization-safety tension in current VLA models where diverse training improves collision avoi...
-
Model-free LQG Control with Chance Constraints
NPG-based actor-critic with Lagrangian for model-free chance-constrained LQG, claiming linear convergence, critic convergence via TD(0), and no duality gap.
-
Explicit Control Barrier Function-based Safety Filters and their Resource-Aware Computation
The paper gives explicit closed-form controllers for control barrier function safety filters via state-space partitioning and a switching implementation that recomputes only on region changes.
-
LIBERO-Safety: A Comprehensive Benchmark for Physical and Semantic Safety in Vision-Language-Action Models
Introduces LIBERO-Safety benchmark with parametric scenario generation and 19,664 collision-free demonstrations, then evaluates VLA models to reveal a generalization-safety tension.
-
World-Task Factorization for Robot Learning
Introduces world-task factorization for robot policies using Bayesian evidence and AICON graph plus learned modulator, outperforming baselines with zero-shot generalization in heterogeneous robotics settings.
-
Safety-Critical Whole-Body Control for Humanoid Robots via Input-to-State Safe Control Barrier Functions
A hierarchical framework integrates kinematic whole-body control, an ISSf-CBF safety filter, and dynamic whole-body control to enforce multiple kinematic safety constraints on humanoid robots under bounded disturbance...
-
Geometric Pareto Control: Riemannian Gradient Flow of Energy Function via Lie Group Homotopy
Geometric Pareto Control embeds Pareto solutions in a Lie group submanifold and navigates via Riemannian gradient flow to achieve 100% feasibility and low suboptimality in control tasks without retraining.
-
HANDOFF: Humanoid Agentic Task-Space Whole-Body Control via Distilled Complementary Teachers
HANDOFF is a distilled mixture-of-experts humanoid whole-body controller that follows a compact task-space interface, matches SOTA velocity tracking, provides large manipulation workspace on Unitree G1, and supports V...
-
Constrained Whole-Body Tracking for Humanoid Robots
ConstrainedMimic integrates operational space control and control barrier functions into RL tracking policies to enforce arbitrary runtime constraints on humanoid kinematics and dynamics while preserving contact modes...
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.