Pith

open record

sign in

arxiv: 2606.31320 · v1 · pith:GH5YRNZF · submitted 2026-06-30 · cs.LG · cs.RO

Safe Online Learning via Smooth Safety-Structured Policy Composition

Hongpeng Cao , Liqun Zhao , Yuliang Gu , Naira Hovakimyan , Lui Sha , Marco Caccamo This is my paper

Reviewed by Pith T0 review T1 audit T2 compute T3 formal T4 reserved 2026-07-01 06:10 UTCgrok-4.3pith:GH5YRNZFrecord.jsonopen to challenge →

Figure 1
Figure 1. Figure 1: Diagram of a conventional safety filter architecture and the proposed [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] reproduced from arXiv: 2606.31320
classification cs.LG cs.RO
keywords safe reinforcement learningonline learningsafety constraintspolicy architecturecontinuous controlsmooth optimizationrisk-dependent transitions
0
0 comments X

The pith

AutoSafe embeds structured safety monitoring directly into policy action generation to enable smooth, risk-dependent transitions between performance and safety behaviors.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper proposes a new policy architecture called AutoSafe for safe online reinforcement learning. It addresses the tension between strict safety enforcement, which creates discontinuities that disrupt learning, and soft constraints, which offer weaker guarantees but keep optimization smooth. By integrating safety monitoring and intervention into the action generation process itself, the design produces continuous interaction dynamics that support ongoing learning. Empirical tests on continuous-control benchmarks and a physical cart-pole system show safety is maintained while smoothness is preserved.

Core claim

AutoSafe is a safety-aware policy architecture that integrates structured safety monitoring and intervention directly into the action generation process. This produces smooth, risk-dependent transitions between performance-driven and safety-preserving behaviors, resulting in continuous online interaction and learning dynamics that avoid the discontinuities of prior strict intervention methods.

What carries the argument

AutoSafe, the safety-aware policy architecture that embeds structured safety monitoring and intervention directly into the action generation process to yield risk-dependent outputs.

Load-bearing premise

Embedding structured safety monitoring and intervention directly into the action generation process can simultaneously enforce safety constraints and preserve the smoothness required for stable online learning without introducing new discontinuities or safety gaps.

What would settle it

An experiment on the cart-pole system or a benchmark where activating safety interventions produces measurable discontinuities in policy actions or instability in the learning curves would falsify the claim.

Figures

Figures reproduced from arXiv: 2606.31320 by Hongpeng Cao, Liqun Zhao, Lui Sha, Marco Caccamo, Naira Hovakimyan, Yuliang Gu.

Figure 2
Figure 2. Figure 2: (Left) Distributional Shaping (Sec. 4.2): As the intervention weight λ increases, the policy distribution gradually shifts its mean toward the safe anchor a safe, while its variance decreases quadratically ( Eq. 6). (Right) Learnable Sharpness (Sec. 4.4): The sharpness parameter p controls the intervention profile. A larger p creates a “sharper” boundary that delays intervention to prioritize performance, … view at source ↗
Figure 3
Figure 3. Figure 3: Illustration of a Simplex-style safety mechanism with an inner safe set Ω∆min and switching to a certified safe policy at the safety boundary in a static 2D case. The proposed policy composition diagram is safety-filter ag￾nostic. As a representative instantiation, we adopt the widely used Simplex architecture (Sha, 2001) as the baseline safety design. Simplex employs a robust, certified-safe fallback pol￾… view at source ↗
Figure 4
Figure 4. Figure 4: Comparison of learning dynamics and safety intervention behavior. [PITH_FULL_IMAGE:figures/full_fig_p010_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: CartPole safety envelope visualization. Visualization of λ(∆( ˜ s)) over the x and θ dimensions for different learned values of p across CartPole tasks. The parameter p adapts to the level of goal conflict: after the agent has learned to remain safe, increasing p reduces λ at the same state, downweighting the safety policy to favor higher task performance within the safety envelope. 0 25 50 75 100 125 150 … view at source ↗
Figure 6
Figure 6. Figure 6: AutoSafe-controlled trajectories for CartPole ( [PITH_FULL_IMAGE:figures/full_fig_p011_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Real-world experiments with AutoSafe. The agent is deployed on an embedded device, enabling [PITH_FULL_IMAGE:figures/full_fig_p012_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: AutoSafe: The policy uses the state vector st, included in the observation, to assess risk ∆( ˜ st) and generate safe action a safe. In parallel, the learning-based component processes the full observation ot = (st, it) to produce high-performance but potentially unsafe actions a θ . The two action outputs are fused through a weighted summation, where the weights are determined by a learnable exponential a… view at source ↗
Figure 9
Figure 9. Figure 9: Experimental setup illustration of quadrotor and quadruped robots [PITH_FULL_IMAGE:figures/full_fig_p026_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Performance return curve and the accumulated safety violation for all algorithms in the considered [PITH_FULL_IMAGE:figures/full_fig_p029_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Critic Loss and Actor Loss of AutoSafe and Simplex-Based Method [PITH_FULL_IMAGE:figures/full_fig_p030_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: The evolution of sharpness parameter p during policy learning for studied applications 0 50 100 150 200 Training Episodes 0 100 200 300 400 500 Performance Return Cartpole AutoSafe AutoSafe (exp) AutoSafe (linear) 0 50 100 150 200 Training Episodes 1400 1200 1000 800 600 400 200 Performance Return Glucose 0 200 400 600 800 1000 Training Episodes 0 50 100 150 200 250 300 Performance Return 3D Quadrotor 0 5… view at source ↗
Figure 13
Figure 13. Figure 13: Ablation Study of sharpness parameter p: Learning-based vs. Schedule-based. 30 [PITH_FULL_IMAGE:figures/full_fig_p030_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: Ablation of the adaptation for λ in AdaLam D.4 Ablation of AdaLam In this study, we investigate several ways of setting the weights between safe action and learning-based action, as shown in [PITH_FULL_IMAGE:figures/full_fig_p031_14.png] view at source ↗
Figure 15
Figure 15. Figure 15: State trajectories over an evaluation episode. The dashed lines indicate the time-varying con [PITH_FULL_IMAGE:figures/full_fig_p031_15.png] view at source ↗
Figure 16
Figure 16. Figure 16: example evaluation trajectories with randomized obstacle positions. The adapted safe recoverable [PITH_FULL_IMAGE:figures/full_fig_p032_16.png] view at source ↗
read the original abstract

Safe online reinforcement learning requires policies to respect safety constraints while maintaining smooth optimization dynamics. Existing approaches typically rely on either strict safety enforcement via action interventions, which introduce discontinuities in system interaction and learning, or soft safety constraint formulations, which preserve smooth learning but provide limited safety assurance. We propose AutoSafe, a safety-aware policy architecture that integrates structured safety monitoring and intervention directly into the action generation process. This design enables smooth, risk-dependent transitions between performance-driven and safety-preserving behaviors, resulting in continuous online interaction and learning dynamics. Empirical results across a suite of continuous-control benchmarks demonstrate strong safety enforcement without sacrificing learning smoothness. We further validate AutoSafe on a physical cart-pole system, highlighting its practical effectiveness for safe online learning in the real world.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

1 major / 0 minor

Summary. The paper proposes AutoSafe, a safety-aware policy architecture for safe online reinforcement learning that integrates structured safety monitoring and intervention directly into the action generation process. This is claimed to enable smooth, risk-dependent transitions between performance-driven and safety-preserving behaviors, supporting continuous online interaction and learning. The approach is positioned as addressing limitations of strict action interventions (which introduce discontinuities) and soft constraint formulations (which offer limited safety). Empirical validation is asserted on continuous-control benchmarks and a physical cart-pole system.

Significance. If the central claims on smoothness and safety enforcement hold with rigorous evidence, the result would be significant for safe online RL by providing a structured way to balance constraint satisfaction with stable learning dynamics. The inclusion of physical system validation would strengthen applicability claims if accompanied by quantitative details.

major comments (1)
  1. [Abstract] Abstract: the assertion of 'strong safety enforcement without sacrificing learning smoothness' and 'practical effectiveness' on benchmarks and a physical cart-pole is unsupported by any metrics, baselines, ablation studies, or failure-mode discussion. This directly undermines evaluation of the central claim that the architecture achieves both safety and continuous learning dynamics.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for the detailed review and the opportunity to clarify the presentation of our results. We address the single major comment below.

read point-by-point responses

  1. Referee: [Abstract] Abstract: the assertion of 'strong safety enforcement without sacrificing learning smoothness' and 'practical effectiveness' on benchmarks and a physical cart-pole is unsupported by any metrics, baselines, ablation studies, or failure-mode discussion. This directly undermines evaluation of the central claim that the architecture achieves both safety and continuous learning dynamics.

    Authors: We agree that the abstract, as currently written, states strong empirical claims at a high level without referencing specific supporting quantities. The experimental sections of the manuscript do contain quantitative safety-violation rates, return curves, baseline comparisons (including Lagrangian, constrained-policy, and intervention-based methods), and ablation results on the smoothness of the safety intervention. However, these details are not summarized in the abstract itself. We will revise the abstract to include concrete metrics (e.g., average safety violations per episode and smoothness of action trajectories) and will add a short sentence referencing the baselines and the physical-cart-pole validation. We will also expand the failure-mode discussion in Section 5 and ensure the abstract points to it. These changes will be made in the revised manuscript. revision: yes

Circularity Check

0 steps flagged

No significant circularity detected

full rationale

The abstract and description contain no equations, derivations, or load-bearing claims that reduce to self-definition, fitted inputs renamed as predictions, or self-citation chains. The central claim describes an architectural integration of safety monitoring into action generation, supported by empirical results on benchmarks and hardware, without any internal reduction to its own inputs by construction. This is the expected honest non-finding for a paper whose contribution is presented as an engineering design rather than a mathematical derivation.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract-only review supplies no explicit free parameters, axioms, or invented entities.

pith-pipeline@v0.9.1-grok · 5665 in / 914 out tokens · 21762 ms · 2026-07-01T06:10:24.554708+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

29 extracted references · 29 canonical work pages · 5 internal anchors

  1. [1]

    Control Barrier Functions: Theory and Applications

    arXiv:1903.11199 [eess]. Brandon Amos and J. Zico Kolter. OptNet: Differentiable Optimization as a Layer in Neural Networks. In Proceedings of the 34th International Conference on Machine Learning, pp. 136–145. PMLR, July

    work page internal anchor Pith review Pith/arXiv arXiv 1903

  2. [2]

    doi: 10.1145/3744351

    ISSN 2378-962X. doi: 10.1145/3744351. Just Accepted. Hongpeng Cao, Yanbing Mao, Lui Sha, and Marco Caccamo. Physics-regulated deep reinforcement learning: Invariant embeddings. InThe Twelfth International Conference on Learning Representations. Hongpeng Cao, Mirco Theile, Federico G. Wyrwal, and Marco Caccamo. Cloud-edge training architecture for sim-to-r...

    work page doi:10.1145/3744351

  3. [3]

    Generalization in Reinforcement Learning by Soft Data Augmentation

    IEEE. ISBN 978-1- 72819-077-8. doi: 10.1109/ICRA48506.2021.9561510. URLhttps://ieeexplore.ieee.org/document/ 9561510/. Sehoon Ha, Peng Xu, Zhenyu Tan, Sergey Levine, and Jie Tan. Learning to Walk in the Real World with Minimal Human Effort, November

    work page doi:10.1109/icra48506.2021.9561510 2021

  4. [4]

    Learning to walk in the real world with minimal human effort

    arXiv:2002.08550 [cs]. Tuomas Haarnoja, Aurick Zhou, Pieter Abbeel, and Sergey Levine. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. In Jennifer Dy and Andreas Krause (eds.), Proceedings of the 35th International Conference on Machine Learning, volume 80 ofProceedings of Ma- chine Learning Research, pp....

    work page arXiv 2002

  5. [5]

    doi: 10.1146/annurev-control-071723-102940

    ISSN 2573-5144. doi: 10.1146/annurev-control-071723-102940. 14 JulianIbarz, JieTan, ChelseaFinn, MrinalKalakrishnan, PeterPastor, andSergeyLevine. Howtotrainyour robot with deep reinforcement learning: lessons we have learned.The International Journal of Robotics Research, 40(4-5):698–721, April

    work page doi:10.1146/annurev-control-071723-102940

  6. [6]

    doi: 10.1177/0278364920987859

    ISSN 0278-3649, 1741-3176. doi: 10.1177/0278364920987859. Wanxin Jin, Shaoshuai Mou, and George J. Pappas. Safe pontryagin differentiable programming.Advances in Neural Information Processing Systems, 34:16034–16050,

    work page doi:10.1177/0278364920987859

  7. [7]

    cc/paper/2021/hash/85ea6fd7a2ca3960d0cf5201933ac998-Abstract.html

    URLhttps://proceedings.neurips. cc/paper/2021/hash/85ea6fd7a2ca3960d0cf5201933ac998-Abstract.html. Tobias Johannink, Shikhar Bahl, Ashvin Nair, Jianlan Luo, Avinash Kumar, Matthias Loskyll, Juan Aparicio Ojea, Eugen Solowjow, and Sergey Levine. Residual reinforcement learning for robot control. pp. 6023– 6029–6023–6029. IEEE,

    work page 2021

  8. [8]

    arXiv:2205.06750 [cs]

    URL http://arxiv.org/abs/2205.06750. arXiv:2205.06750 [cs]. Timothy P Lillicrap, Jonathan J Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, and Daan Wierstra. Continuous control with deep reinforcement learning.arXiv preprint arXiv:1509.02971,

    work page arXiv

  9. [9]

    Safe Reinforcement Learning using Action Projection: Safeguard the Policy or the Environment?

    URLhttp://arxiv.org/abs/2509.12833. arXiv:2509.12833 [cs]. Federico Nesti, Niko Salamini, Mauro Marinoni, Giorgio Maria Cicero, Gabriele Serra, Alessandro Biondi, and Giorgio Buttazzo. The Use of the Simplex Architecture to Enhance Safety in Deep-Learning-Powered Autonomous Systems, September

    work page internal anchor Pith review Pith/arXiv arXiv

  10. [10]

    arXiv:2509.21014 [eess]. Dung T. Phan, Radu Grosu, Nils Jansen, Nicola Paoletti, Scott A. Smolka, and Scott D. Stoller. Neural Simplex Architecture. In Ritchie Lee, Susmit Jha, Anastasia Mavridou, and Dimitra Giannakopoulou (eds.),NASA Formal Methods, volume 12229, pp. 97–114. Springer International Publishing, Cham,

    work page arXiv

  11. [11]

    doi: 10.1007/978-3-030-55754-6_6

    ISBN 978-3-030-55753-9 978-3-030-55754-6. doi: 10.1007/978-3-030-55754-6_6. Series Title: Lecture Notes in Computer Science. Krishan Rana, Vibhavari Dasagi, Jesse Haviland, Ben Talbot, Michael Milford, and Niko Sünderhauf. Bayesian controller fusion: Leveraging control priors in deep reinforcement learning for robotics.The International Journal of Robotic...

    work page doi:10.1007/978-3-030-55754-6_6

  12. [12]

    Benchmarking Batch Deep Reinforcement Learning Algorithms

    ISSN 0278-3649. doi: 10.1177/ 02783649231167210. Publisher: SAGE Publications Ltd STM. Alex Ray, Joshua Achiam, and Dario Amodei. Benchmarking safe exploration in deep reinforcement learning. arXiv preprint arXiv:1910.01708, 7(1):2,

    work page internal anchor Pith review Pith/arXiv arXiv 1910

  13. [13]

    Proximal Policy Optimization Algorithms

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimiza- tion algorithms.arXiv preprint arXiv:1707.06347,

    work page internal anchor Pith review Pith/arXiv arXiv

  14. [14]

    doi: 10.1109/MS.2001. 936213. Wesley Suttle, Vipul Kumar Sharma, Krishna Chaitanya Kosaraju, Sivaranjani Seetharaman, Ji Liu, Vijay Gupta, and Brian M. Sadler. Sampling-based safe reinforcement learning for nonlinear dynamical systems. InInternational Conference on Artificial Intelligence and Statistics, pp. 4420–4428. PMLR,

    work page doi:10.1109/ms.2001 2001

  15. [15]

    15 Mark Towers, Ariel Kwiatkowski, Jordan Terry, John U Balis, Gianluca De Cola, Tristan Deleu, Manuel Goulão, Andreas Kallinteris, Markus Krimmel, Arjun KG, et al

    arXiv:2404.15199 [cs]. 15 Mark Towers, Ariel Kwiatkowski, Jordan Terry, John U Balis, Gianluca De Cola, Tristan Deleu, Manuel Goulão, Andreas Kallinteris, Markus Krimmel, Arjun KG, et al. Gymnasium: A standard interface for reinforcement learning environments.arXiv preprint arXiv:2407.17032,

    work page arXiv

  16. [16]

    Linear model predictive safety certification for learning-based control

    arXiv:1803.08552 [cs]. Akifumi Wachi and Yanan Sui. Safe reinforcement learning in constrained Markov decision processes. pp. 9797–9806–9797–9806,

    work page internal anchor Pith review Pith/arXiv arXiv

  17. [17]

    ISBN 978-1-4503-1996-6

    ACM. ISBN 978-1-4503-1996-6. doi: 10.1145/2502524. 2502531. URLhttps://dl.acm.org/doi/10.1145/2502524.2502531. Tyler Westenbroek, Fernando Castaneda, Ayush Agrawal, Shankar Sastry, and Koushil Sreenath. Lyapunov Design for Robust and Efficient Robotic Reinforcement Learning.arXiv preprint arXiv:2208.06721,

    work page doi:10.1145/2502524 1996

  18. [18]

    Linhai Xie, Sen Wang, Stefano Rosa, Andrew Markham, and Niki Trigoni

    URLhttps://ieeexplore.ieee.org/abstract/document/10077790/. Linhai Xie, Sen Wang, Stefano Rosa, Andrew Markham, and Niki Trigoni. Learning with training wheels: speeding up training with a simple controller for deep reinforcement learning. pp. 6276–6283–6276–6283,

    work page arXiv

  19. [19]

    A Barrier-Lyapunov Actor-Critic Rein- forcement Learning Approach for Safe and Stable Control.arXiv preprint arXiv:2304.04066,

    Liqun Zhao, Konstantinos Gatsis, and Antonis Papachristodoulou. A Barrier-Lyapunov Actor-Critic Rein- forcement Learning Approach for Safe and Stable Control.arXiv preprint arXiv:2304.04066,

    work page arXiv

  20. [20]

    That is, there exist b(s)∈Randg(s)∈Rm such that ˆ∆(s,a) =b(s) +g(s) ⊤a+εlin(s,a),|εlin(s,a)|≤ϵlin(s)(22) for all actionsaon this segment

    Assumption A.1.For fixeds, the relative margin ˆ∆(s,a)is locally approximated by an affine function ofaon a neighborhood containing the interpolation segment betweenaθanda safe(s). That is, there exist b(s)∈Randg(s)∈Rm such that ˆ∆(s,a) =b(s) +g(s) ⊤a+εlin(s,a),|εlin(s,a)|≤ϵlin(s)(22) for all actionsaon this segment. Whenϵ lin(s) = 0, the margin is exactl...

    work page 2018

  21. [21]

    ˙s=As+Ba.(45) We parameterize the safe controller as a linear feedback policy a=F s,(46) whereF∈Rm×nis the controller gain matrix. The resulting closed-loop dynamics become ˙s= ¯As, ¯A=A+BF.(47) To jointly enforce state and action constraints, we define the unified constraint matrix D= [ As diag(bs)−1 AaFdiag(ba)−1 ] ,(48) such that the constraints can be...

    work page 1999

  22. [22]

    and (Fujimoto et al., 2018), with default parameters summarized in Table

    work page 2018

  23. [23]

    qCMdcjpSuxuYQzuZyBqqjO9S8DY=

    23 Action Deep Neural Policy State Vector Other Information x… 𝒏 × 1 Safety Matrix ……… ……… … 1 × 𝒏𝒏 × 𝒏𝒏 × 1 …••… 𝒏 × 1 •… …… 𝒎 ×𝒏 … 𝒎 × 1 Safety Monitoring Safe Action Generation Sharpness value Exponential Ramp …… <latexit sha1_base64="qCMdcjpSuxuYQzuZyBqqjO9S8DY=">AAACJHicbVC7TsMwFHXKq5RXCyNLRIXEVCUICmMFC2OR6ENqospxb1qrjh3ZDlBF/Q1WmPgaNsTAwrfgtBloy5EsH...

    work page 2001

  24. [24]

    by controlling insulin injectionaI. The dynamics of the glucose control problem are governed by the following ODEs (Tian et al., 2024), ˙G=−p1(G−Gb)−GX+Dt, ˙X=−p2X+p 3(I−Ib), ˙I=−n(I−Ib) +aI Here,Grepresents the amount of glucose in the blood, andIrepresents the amount of insulin in the blood. Xdescribes the delayed effect of insulin on lowering blood glu...

    work page 2024

  25. [25]

    The system model can be found at (Tian et al., 2024)

    Moded-based DesignThe safety envelope and safe policy are obtained from solving an LMI problem, as discussed in Section B. The system model can be found at (Tian et al., 2024). The linearized model and the code to calculate the matrixPandKare available in the attached supplementary files. C.2.3 3D Quadrotor Goal Reaching Task DefinitionThe goal of this ta...

    work page 2024

  26. [26]

    In our case study, we set the initial position of the quadrotor ass0xyz ={1.5,1.5,1.5}and the target position of the quadrotor asˆsxyz ={2.5,2.5,2.5}

    as: r=e −α·(∥x−ˆx∥2+∥y−ˆy∥2+∥z−ˆz∥2)−β·∥a∥2 , 26 whereα= 1.0andβ= 1e−4are the weights to balance the distance-related reward and action penalty. In our case study, we set the initial position of the quadrotor ass0xyz ={1.5,1.5,1.5}and the target position of the quadrotor asˆsxyz ={2.5,2.5,2.5}. The control loop is running at 50Hz. Safety ConstraintsThe sa...

    work page 2022

  27. [27]

    We observe that training of theSimplexis graduallydivergingwitha largecriticloss, asshown inFig.11

    0 50k 100k 150k 200k Training Steps 300 325 350 375 400 425 450 475 500Performance Return Cartpole 0 20k 40k 60k 80k 100k Training Steps 1000 800 600 400 200 Glucose 0 200k 400k 600k 800k 1M Training Steps 0 100 200 300 400 500 600 3D Quadrotor 0 100k 200k 300k 400k 500k Training Steps 0 500 1000 1500 2000 2500 3000 3500 4000 Quadruped Navigation 1 2 3 4 ...

    work page 2000

  28. [28]

    It can be seen that the learned sharpness converges to different values across tasks, suggesting that it may not be trivial to manually set the "right" parameter using heuristics. 29 0.0 0.5 1.0 1.5 2.0 Training Steps 1e5 0.0 0.5 1.0 1.5 2.0 2.5Critic Loss 1e15 Cartpole AutoSafe Simplex 0.0 0.2 0.4 0.6 0.8 1.0 Training Steps 1e5 0 1 2 3 4 5 6 7Critic Loss...

    work page 2000

  29. [29]

    For simple tasks, such as cartpole and glucose, the agent could learn using the data generated by the safe policy

    We found out that initializing theλto close to1at the beginning of the training enables safe interactions. For simple tasks, such as cartpole and glucose, the agent could learn using the data generated by the safe policy. However, we found that it is not effective in high-dimensional cases. For a learning-based setting, the exploration is not effective; t...

    work page 2072