pith. sign in

arxiv: 2604.06463 · v1 · submitted 2026-04-07 · 📡 eess.SY · cs.SY

A Control Barrier Function-Constrained Model Predictive Control Framework for Safe Reinforcement Learning

Ali Umut Kaypak , Prashanth Krishnamurthy , Farshad Khorrami This is my paper

Pith reviewed 2026-05-10 18:22 UTC · model grok-4.3

classification 📡 eess.SY cs.SY
keywords safe reinforcement learningcontrol barrier functionsmodel predictive controlprobabilistic neural networkstrajectory samplingstochastic dynamicssafety constraints
0
0 comments X

The pith

Joint learning of probabilistic dynamics and control barrier functions allows MPC to enforce probabilistic safety by sampling only safe trajectories in reinforcement learning under uncertainty.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The authors propose PECTS to handle safety when system dynamics are unknown and stochastic. Probabilistic neural networks learn the dynamics while Lipschitz-bounded networks learn control barrier functions that define safe regions. These learned elements are embedded as constraints inside a model predictive controller. A sampling optimizer then generates candidate trajectories and discards those that violate the learned barrier conditions under the probabilistic model. The result is a method that aims to keep the learning process safe without relying on a perfect prior model.

Core claim

PECTS jointly learns stochastic system dynamics with probabilistic neural networks and control barrier functions with Lipschitz-bounded neural networks. Safety is enforced by incorporating learned CBF constraints into the MPC formulation while accounting for the model stochasticity. This enables probabilistic safety under model uncertainty. To solve the resulting MPC problem, a sampling-based optimizer is used together with a safe trajectory sampling method that discards unsafe trajectories based on the learned system model and CBF.

What carries the argument

The CBF-constrained MPC solved by safe trajectory sampling, where learned probabilistic dynamics and Lipschitz-bounded barriers are used to filter out unsafe rollouts before execution.

If this is right

  • The framework lets reinforcement learning agents explore while maintaining a quantifiable level of safety even when the true dynamics are stochastic and initially unknown.
  • Embedding learned CBFs directly into MPC replaces the need for hand-crafted safety constraints that may not match the actual system.
  • Safe trajectory sampling reduces the computational burden of solving constrained optimization by rejecting bad candidates early.
  • The approach scales to tasks where model uncertainty must be handled explicitly rather than through worst-case robust formulations.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the learned barriers prove reliable across environments, the method could reduce the performance penalty often paid for conservative safety margins in learned controllers.
  • The same joint-learning structure might be tested on systems that change slowly over time by periodically updating the neural models without restarting from scratch.
  • Combining this sampling filter with standard RL reward shaping could produce agents that both stay safe and reach higher returns than purely constrained baselines.

Load-bearing premise

The learned dynamics and barrier functions stay accurate enough during operation to correctly flag and reject unsafe trajectories without missing real violations or rejecting too many safe ones.

What would settle it

An experiment on a physical system in which the agent still collides or violates safety limits after the method has filtered all sampled trajectories.

Figures

Figures reproduced from arXiv: 2604.06463 by Ali Umut Kaypak, Farshad Khorrami, Prashanth Krishnamurthy.

Figure 1
Figure 1. Figure 1: High-level flow of the optimization in PECTS. Candidate input [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Task environments used in simulation studies. In both environments, [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Trained CBF for a unicycle in the goal-reaching environment. The [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Examples of unicycle paths in the goal-reaching task. [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
read the original abstract

Ensuring safety under unknown and stochastic dynamics remains a significant challenge in reinforcement learning (RL). In this paper, we propose a model predictive control (MPC)-based safe RL framework, called Probabilistic Ensembles with CBF-constrained Trajectory Sampling (PECTS), to address this challenge. PECTS jointly learns stochastic system dynamics with probabilistic neural networks (PNNs) and control barrier functions (CBFs) with Lipschitz-bounded neural networks. Safety is enforced by incorporating learned CBF constraints into the MPC formulation while accounting for the model stochasticity. This enables probabilistic safety under model uncertainty. To solve the resulting MPC problem, we utilize a sampling-based optimizer together with a safe trajectory sampling method that discards unsafe trajectories based on the learned system model and CBF. We validate PECTS in various simulation studies, where it outperforms baseline methods.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 1 minor

Summary. The paper proposes PECTS, an MPC-based safe RL framework that jointly learns stochastic system dynamics via probabilistic neural networks (PNNs) and control barrier functions via Lipschitz-bounded neural networks. Learned CBF constraints are incorporated into the MPC formulation to account for model stochasticity, and a sampling-based optimizer with safe trajectory sampling discards unsafe trajectories according to the learned model. This is claimed to yield probabilistic safety under model uncertainty. The approach is validated in simulation studies where it outperforms baselines.

Significance. If the probabilistic safety claims hold with the stated learning components, the work could meaningfully advance safe RL by integrating data-driven dynamics and barrier functions into a receding-horizon optimizer with explicit trajectory filtering. The simulation validation demonstrating outperformance over baselines is a concrete strength that supports practical utility, provided the learned models generalize.

major comments (2)
  1. The central claim that PECTS achieves probabilistic safety under model uncertainty rests on the learned PNN dynamics and Lipschitz-bounded CBFs remaining sufficiently accurate for online trajectory filtering. No generalization bounds, Lipschitz-constant analysis, or robustness guarantees are supplied to bound the probability of false-negative safety violations when test-time dynamics differ from training data; this assumption is load-bearing for the safety guarantee.
  2. The safe trajectory sampling procedure discards trajectories predicted to violate the learned CBF, yet the manuscript provides no quantitative analysis (e.g., via concentration inequalities or empirical coverage) of how model mismatch propagates into missed unsafe trajectories or excessive conservatism; without this, the probabilistic safety statement cannot be verified from the given validation.
minor comments (1)
  1. [Abstract] The abstract states that PECTS 'outperforms baseline methods' in 'various simulation studies' but supplies neither the specific environments, quantitative metrics (e.g., safety violation rates, cumulative reward), nor ablation results; adding these details would strengthen the empirical section.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the thoughtful and constructive comments on our manuscript. We have carefully considered the major concerns raised regarding the probabilistic safety claims and the analysis of model mismatch. Our responses to each point are provided below, and we outline the revisions we plan to make.

read point-by-point responses

  1. Referee: The central claim that PECTS achieves probabilistic safety under model uncertainty rests on the learned PNN dynamics and Lipschitz-bounded CBFs remaining sufficiently accurate for online trajectory filtering. No generalization bounds, Lipschitz-constant analysis, or robustness guarantees are supplied to bound the probability of false-negative safety violations when test-time dynamics differ from training data; this assumption is load-bearing for the safety guarantee.

    Authors: We acknowledge that our manuscript does not provide formal generalization bounds or a detailed Lipschitz-constant analysis for the learned models under potential distribution shifts. The probabilistic safety is established with respect to the uncertainty captured by the PNNs within the training distribution, and the Lipschitz-bounded networks are used to ensure the CBF property holds for the learned function. However, we agree that bounding the probability of safety violations due to model mismatch at test time is an important open aspect not addressed in the current work. In the revised manuscript, we will expand the discussion section to explicitly state this assumption and its implications for the safety guarantees. Additionally, we will include new empirical results evaluating the framework's performance when the test environment dynamics are perturbed from the training data to provide quantitative insight into robustness. revision: yes

  2. Referee: The safe trajectory sampling procedure discards trajectories predicted to violate the learned CBF, yet the manuscript provides no quantitative analysis (e.g., via concentration inequalities or empirical coverage) of how model mismatch propagates into missed unsafe trajectories or excessive conservatism; without this, the probabilistic safety statement cannot be verified from the given validation.

    Authors: We agree that a quantitative analysis of how model mismatch affects the safe trajectory sampling—such as the rate of missed unsafe trajectories or the degree of conservatism—is not present in the current manuscript. The validation relies on simulation studies where the learned models are trained and tested in the same environment, demonstrating outperformance over baselines. To address this, we will add in the revision an empirical study that measures the coverage of safe trajectories and the impact of varying levels of model uncertainty or mismatch on the filtering process. This will help substantiate the probabilistic claims under the observed conditions. revision: yes

Circularity Check

0 steps flagged

No circularity in proposed safe RL framework

full rationale

The manuscript presents PECTS as a combined learning-and-control architecture: PNNs for stochastic dynamics, Lipschitz-bounded NNs for CBFs, and a sampling-based MPC that discards trajectories violating the learned CBF. No derivation chain is offered that reduces a claimed prediction or safety guarantee to a fitted parameter, self-citation, or definitional tautology. The central claims rest on empirical validation in simulation rather than on any algebraic identity or load-bearing self-reference. The reader's assessment of score 1 is therefore conservative; the paper contains no load-bearing circular step.

Axiom & Free-Parameter Ledger

1 free parameters · 1 axioms · 0 invented entities

Abstract provides insufficient detail to enumerate all free parameters or axioms; the framework implicitly relies on the existence of Lipschitz-bounded networks that can represent valid CBFs and on the sampling procedure correctly approximating probabilistic safety.

free parameters (1)
  • Neural network weights for PNN dynamics and CBF approximators
    Learned from data; number and initialization not specified.
axioms (1)
  • domain assumption Lipschitz-bounded neural networks can serve as valid control barrier functions for the learned dynamics
    Invoked when incorporating learned CBF constraints into MPC.

pith-pipeline@v0.9.0 · 5449 in / 1302 out tokens · 72293 ms · 2026-05-10T18:22:18.492415+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

27 extracted references · 27 canonical work pages

  1. [1]

    A review of safe reinforcement learning: Methods, theories, and applica- tions,

    S. Gu, L. Yang, Y . Du, G. Chen, F. Walter, J. Wang, and A. Knoll, “A review of safe reinforcement learning: Methods, theories, and applica- tions,”IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 46, no. 12, pp. 11 216–11 235, 2024

    work page 2024

  2. [2]

    Discrete control barrier functions for safety-critical control of discrete systems with application to bipedal robot navigation

    A. Agrawal and K. Sreenath, “Discrete control barrier functions for safety-critical control of discrete systems with application to bipedal robot navigation.” inProc. Robotics: Science and Systems, vol. 13, Cambridge, MA, US, July 2017, pp. 1–10

    work page 2017

  3. [3]

    Safety-critical model predictive control with discrete-time control barrier function,

    J. Zeng, B. Zhang, and K. Sreenath, “Safety-critical model predictive control with discrete-time control barrier function,” inProc. American Control Conference, New Orleans, LA, US, May 2021, pp. 3882–3889

    work page 2021

  4. [4]

    Safe multi-robotic arm interaction via 3D convex shapes,

    A. U. Kaypak, S. Wei, P. Krishnamurthy, and F. Khorrami, “Safe multi-robotic arm interaction via 3D convex shapes,”Robotics and Autonomous Systems, vol. 196, p. 105263, 2026

    work page 2026

  5. [5]

    Deep reinforce- ment learning in a handful of trials using probabilistic dynamics models,

    K. Chua, R. Calandra, R. McAllister, and S. Levine, “Deep reinforce- ment learning in a handful of trials using probabilistic dynamics models,” inProc. Advances in Neural Information Processing Systems, vol. 31, Montreal, QC, Canada, December 2018, p. 4759–4770

    work page 2018

  6. [6]

    Bounding stochastic safety: Leveraging freedman’s inequality with discrete-time control barrier functions,

    R. K. Cosner, P. Culbertson, and A. D. Ames, “Bounding stochastic safety: Leveraging freedman’s inequality with discrete-time control barrier functions,”IEEE Control Systems Letters, vol. 8, pp. 1937–1942, 2024, extended version available at arXiv:2403.05745

    work page arXiv 1937

  7. [7]

    Deep dynamics models for learning dexterous manipulation,

    A. Nagabandi, K. Konolige, S. Levine, and V . Kumar, “Deep dynamics models for learning dexterous manipulation,” inProc. Conference on Robot Learning, vol. 100, October 2020, pp. 1101–1112

    work page 2020

  8. [8]

    Deep reinforcement learning: A brief survey,

    K. Arulkumaran, M. P. Deisenroth, M. Brundage, and A. A. Bharath, “Deep reinforcement learning: A brief survey,”IEEE Signal Processing Magazine, vol. 34, no. 6, pp. 26–38, 2017

    work page 2017

  9. [9]

    Safe reinforcement learning using robust control barrier functions,

    Y . Emam, G. Notomista, P. Glotfelter, Z. Kira, and M. Egerstedt, “Safe reinforcement learning using robust control barrier functions,”IEEE Robotics and Automation Letters, vol. 10, no. 3, pp. 2886–2893, 2022

    work page 2022

  10. [10]

    Probabilistically safe and efficient model-based reinforcement learning,

    F. Airaldi, B. D. Schutter, and A. Dabiri, “Probabilistically safe and efficient model-based reinforcement learning,” inProc. Conference on Decision and Control, Rio de Janeiro, Brazil, December 2025, pp. 5853– 5860

    work page 2025

  11. [11]

    Reinforcement learning-based receding horizon control using adaptive control barrier functions for safety-critical systems,

    E. Sabouni, H. Sabbir Ahmad, V . Giammarino, C. G. Cassandras, I. C. Paschalidis, and W. Li, “Reinforcement learning-based receding horizon control using adaptive control barrier functions for safety-critical systems,” inProc. Conference on Decision and Control, Milan, Italy, December 2024, pp. 401–406

    work page 2024

  12. [12]

    Enforcing hard constraints with soft barriers: Safe reinforcement learning in unknown stochastic environments,

    Y . Wang, S. S. Zhan, R. Jiao, Z. Wang, W. Jinet al., “Enforcing hard constraints with soft barriers: Safe reinforcement learning in unknown stochastic environments,” inProc. International Conference on Machine Learning, vol. 202, Honolulu, HI, July 2023, pp. 36 593–36 604

    work page 2023

  13. [13]

    Learning barrier certificates: Towards safe reinforce- ment learning with zero training-time violations,

    Y . Luo and T. Ma, “Learning barrier certificates: Towards safe reinforce- ment learning with zero training-time violations,” inProc. Advances in Neural Information Processing Systems, vol. 34, December 2021, pp. 25 621–25 632

    work page 2021

  14. [14]

    Model-free safe reinforcement learning through neural barrier certificate,

    Y . Yang, Y . Jiang, Y . Liu, J. Chen, and S. E. Li, “Model-free safe reinforcement learning through neural barrier certificate,”IEEE Robotics and Automation Letters, vol. 8, no. 3, pp. 1295–1302, 2023

    work page 2023

  15. [15]

    Re- inforcement learning for safe robot control using control lyapunov barrier functions,

    D. Du, S. Han, N. Qi, H. B. Ammar, J. Wang, and W. Pan, “Re- inforcement learning for safe robot control using control lyapunov barrier functions,” inProc. International Conference on Robotics and Automation, London, UK, May 2023, pp. 9442–9448

    work page 2023

  16. [16]

    Learning a better control barrier function,

    B. Dai, P. Krishnamurthy, and F. Khorrami, “Learning a better control barrier function,” inProc. Conference on Decision and Control, Cancun, Mexico, December 2022, pp. 945–950

    work page 2022

  17. [17]

    Learning control barrier functions from expert demonstrations,

    A. Robey, H. Hu, L. Lindemann, H. Zhang, D. V . Dimarogonas, S. Tu, and N. Matni, “Learning control barrier functions from expert demonstrations,” inProc. Conference on Decision and Control, Jeju, Korea, December 2020, pp. 3717–3724

    work page 2020

  18. [18]

    Data-efficient control barrier function refinement,

    B. Dai, H. Huang, P. Krishnamurthy, and F. Khorrami, “Data-efficient control barrier function refinement,” inProc. American Control Confer- ence, San Diego, CA, US, May 2023, pp. 3675–3680

    work page 2023

  19. [19]

    Safe reinforcement learning for lidar- based navigation via control barrier function,

    L. Song, L. Ferderer, and S. Wu, “Safe reinforcement learning for lidar- based navigation via control barrier function,” inProc. International Conference on Machine Learning and Applications, Nassau, Bahamas, December 2022, pp. 264–269

    work page 2022

  20. [20]

    Path integral methods with stochastic control barrier functions,

    C. Tao, H.-J. Yoon, H. Kim, N. Hovakimyan, and P. V oulgaris, “Path integral methods with stochastic control barrier functions,” inProc. Conference on Decision and Control, Cancun, Mexico, December 2022, pp. 1654–1659

    work page 2022

  21. [21]

    Guaranteed-safe MPPI through composite control barrier functions for efficient sampling in multi-constrained robotic systems,

    P. Rabiee and J. B. Hoagg, “Guaranteed-safe MPPI through composite control barrier functions for efficient sampling in multi-constrained robotic systems,” inProc. Conference on Decision and Control, Rio de Janeiro, Brazil, December 2025, pp. 5515–5520

    work page 2025

  22. [22]

    Safe Beyond the Horizon: Efficient Sampling-based MPC with Neural Control Barrier Functions,

    J. Yin, O. So, E. Y . Yu, C. Fan, and P. Tsiotras, “Safe Beyond the Horizon: Efficient Sampling-based MPC with Neural Control Barrier Functions,” inProc. Robotics: Science and Systems, LosAngeles, CA, June 2025

    work page 2025

  23. [23]

    Direct parameterization of Lipschitz- bounded deep networks,

    R. Wang and I. Manchester, “Direct parameterization of Lipschitz- bounded deep networks,” inProc. International Conference on Machine Learning, vol. 202, Honolulu, HI, July 2023, pp. 36 093–36 110

    work page 2023

  24. [24]

    Mbrl-lib: A modular library for model-based reinforcement learning.arXiv preprint arXiv:2104.10159, 2021

    L. Pineda, B. Amos, A. Zhang, N. O. Lambert, and R. Calandra, “MBRL-Lib: A modular library for model-based reinforcement learn- ing,”arXiv preprint arXiv:2104.10159, 2021

    work page arXiv 2021

  25. [25]

    Constrained policy optimization,

    J. Achiam, D. Held, A. Tamar, and P. Abbeel, “Constrained policy optimization,” inProc. International Conference on Machine Learning, vol. 70, Sydney, Australia, August 2017, pp. 22–31

    work page 2017

  26. [26]

    Benchmarking safe exploration in deep reinforcement learning,

    A. Ray, J. Achiam, and D. Amodei, “Benchmarking safe exploration in deep reinforcement learning,” Preprint, 2019. [Online]. Available: https://cdn.openai.com/safexp-short.pdf

    work page 2019

  27. [27]

    Constrained update projection approach to safe policy optimization,

    L. Yang, J. Ji, J. Dai, L. Zhang, B. Zhouet al., “Constrained update projection approach to safe policy optimization,” inProc. Advances in Neural Information Processing Systems, vol. 35, New Orleans, LA, November 2022, pp. 9111–9124

    work page 2022