Neural Simplex Architecture

Dung T. Phan; Nicola Paoletti; Nils Jansen; Radu Grosu; Scott A. Smolka; Scott D. Stoller

REVIEW 4 major objections 5 minor 1 cited by

Neural Simplex Architecture

T0 review · 4 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that a runtime watchdog can guarantee safety for a neural controller by switching to a certified baseline, and that online retraining with penalized unsafe actions lets the neural controller retake control without…

desk verdict A solid Simplex extension with real experiments; the safety guarantee is conditional on an exact model, and the empirical work needs artifacts, but it deserves review. read the letter →

arxiv 1908.00528 v2 pith:KJBERKRN submitted 2019-08-01 cs.AI cs.SYeess.SY

Dung T. Phan , Radu Grosu , Nils Jansen , Nicola Paoletti , Scott A. Smolka , Scott D. Stoller This is my paper · ORCID

classification cs.AIcs.SYeess.SY

keywords runtimeassurancesimplexarchitectureneuralcontrollerssafereinforcementlearningonlineretrainingreverseswitchingforwardconditioncontinuouscontrol

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper introduces the Neural Simplex Architecture (NSA), a runtime-safety layer for systems controlled by neural networks. Its central claim is that a pre-certified baseline controller plus a sound forward switching condition guarantees that the plant's safety is never violated, even when the neural controller proposes unsafe actions. Unlike the classical Simplex architecture, NSA also supports reverse switching back to the neural controller and retrains the controller online while the baseline is in control, using penalized unrecoverable actions as training samples. The paper argues that this online retraining eliminates the neural controller's unsafe actions without sacrificing, and in the case studies improving, performance. If correct, this gives a practical route to deploying neural controllers in safety-critical continuous-control systems without requiring the neural network itself to be fully verified.

What carries the argument

The core object is the pair $(R_{BC}, \mathrm{FSC})$: $R_{BC}$ is the largest set of states from which the baseline controller's control law $\beta$ keeps the plant inside the set forever and avoids the unsafe set $U$, and the forward switching condition $\mathrm{FSC}(s,a)$ is required to be true whenever $f(s,a)\notin R_{BC}$—that is, whenever one step under the proposed action would leave the recoverable region. This is the mechanism that transfers the baseline's safety certificate to the whole architecture: because the decision module interposes before an unrecoverable action reaches the plant, the plant's trajectory is a concatenation of baseline-controlled segments and neural-controlled segments that the FSC certifies stay inside $R_{BC}$. The reverse switching condition $\mathrm{RSC}(s)$ and the adaptation module are secondary machinery: RSC returns control to the neural controller when a simulated lookahead (or a distance-to-boundary test) predicts no near-term FSC violations, and the adaptation module retrains the policy from shadow-mode samples $(s_t, a_t^{NC}, s', r')$ with a large negative reward for unrecoverable actions.

What would settle it

Run the rover case study with the real rover's braking acceleration reduced by, say, 10% relative to the value used in $f$, and check whether any trajectory that triggers the FSC still enters a state with $l_{\min} < d_{\text{safe}} + d_{\text{br}} + \epsilon$; if even one does, the 'never violated' guarantee is false.

Watch

Extended reading notes

Core claim

NSA's central claim is that a neural controller can be made safe at runtime without being verified and without being permanently locked out after a mistake. The decision module continuously checks each proposed action against a forward switching condition derived from a pre-certified baseline controller and a recoverable region: if one step of simulated plant evolution under the proposed action leaves the region from which the baseline can still guarantee safety, control passes to the baseline before the unsafe action is executed. While the baseline is in control, the neural controller runs in shadow mode and the adaptation module retrains it, using the intercepted unrecoverable action as a training sample with a large negative reward. The reverse switching condition then returns control to the retrained controller when a lookahead or distance-to-boundary test indicates that future forward switches are unlikely. In the inverted-pendulum, rover, and artificial-pancreas case studies, retraining drove the number of unrecoverable actions to zero and improved average return, whereas training on filtered recoverable actions alone left every evaluated trajectory unsafe.

Load-bearing premise

The load-bearing premise is that the plant dynamics used in the decision module's one-step simulation are deterministic and exactly correct; if the real plant deviates from that model, an unsafe action can be misclassified as safe and the safety guarantee fails.

Editorial extensions

If this is right

If the central claim holds, a neural-network controller can be deployed in a safety-critical continuous-control system without a formal proof of the network's correctness, because the decision module converts the baseline controller's safety certificate into an end-to-end guarantee.
The penalized-unrecoverable-action training strategy implies that unsafe proposed actions are not wasted: used as shadow-mode training samples with a large negative reward, they teach the neural controller to avoid the switching boundary, reducing future failovers.
Reverse switching means a failover to the baseline need not end the mission; control can return to the neural controller once a lookahead or distance-to-boundary test indicates that the unsafe behavior is unlikely to recur.
In the reported case studies, online retraining with only tens of thousands of updates outperformed extending initial training by millions of samples on both safety and average return, suggesting that focused retraining near visited regions of the state space is more sample-efficient than random initial training.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A natural extension is to replace the one-step deterministic simulation in the FSC with reachability or probabilistic model checking, which would let the same architecture carry a safety guarantee for stochastic or nondeterministic plants instead of relying on exact dynamics.
The architecture's safety proof is only as strong as the baseline controller's certificate; for systems where no tight recoverable region can be computed, the FSC will be conservative and the neural controller will spend more time shadowed, so the practical payoff depends on how well the baseline's safe set can be characterized.
The retraining effect could be studied as a form of curriculum: shadow-mode sampling concentrates updates in exactly the state-action regions near the forward-switching boundary, which may explain the large gains observed in the case studies and suggests a general recipe for safe reinforcement learning.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Your conditional verdict is about right. The real contribution is the combination of Simplex-style runtime assurance with reverse switching and online retraining of the neural controller, and the SRL-PUA comparison is a useful empirical datapoint. The architecture is clearly described, the case studies are appropriate, and the related work is honest.

The main soft spot is the safety guarantee. The abstract says NSA 'guarantees that the safety of the plant is never violated,' but Section 3 explicitly assumes deterministic dynamics with known f and full observability. The FSC is implemented by one-step simulation of that same f, so any model error invalidates the guarantee. Section 7 defers model deviation to L1Simplex future work, and the artificial pancreas case study is motivated by training-time model mismatch, which is precisely a regime where the guarantee is not established. This is an overclaim, though a standard one in the Simplex literature. The authors should qualify the abstract and add an explicit 'conditions for the guarantee' statement.

The empirical work is decent but thinner than I'd like. All results are from simulation, every table reports single runs without error bars, and no code or data is released. The differences are dramatic (0 vs 1000 unrecoverable trajectories), so the qualitative findings are probably robust, but for a paper about safe RL, code and seeds should be standard. Hyperparameters beyond architecture are delegated to rllab defaults, which is acceptable but weak for a journal.

The SRL-PUA result is not particularly deep: if you train with a huge negative reward for unrecoverable actions, the policy learns to avoid them. But the comparison against filtering-based approaches is still a useful warning, since many practitioners do exactly that and it fails here.

The citation pattern looks fine. The related work is thorough and the paper is honest about what is new.

Overall, this is a worth-engaging paper. The safety claim needs qualification and the reproducibility needs work, but the architecture is coherent, the experiments are relevant, and the direction is useful. I'd send it to peer review.

Referee Report

4 major / 5 minor

Summary. The paper introduces the Neural Simplex Architecture (NSA), a runtime assurance framework that combines a neural controller (NC) with a pre-certified baseline controller (BC) and a decision module (DM) that switches between them. Unlike the classical Simplex architecture, NSA supports reverse switching from the BC back to the NC and includes an adaptation module (AM) that retrains the NC online, in shadow mode, using reinforcement learning with penalties for unrecoverable actions. The authors formulate a forward switching condition (FSC) based on one-step simulation of a deterministic plant model, define recoverable regions for the BC, and propose two reverse switching conditions (RSC). They evaluate NSA on three continuous-control benchmarks: an inverted pendulum, a target-seeking rover with obstacle avoidance, and an artificial pancreas. The empirical results show that online retraining reduces the number of unrecoverable actions and improves average return compared with the initially trained controllers.

Significance. If the safety guarantee and the empirical benefits hold, NSA is a useful contribution to runtime assurance and safe reinforcement learning, because it directly addresses two limitations of the classical Simplex architecture: the absence of a principled reverse-switching mechanism and the lack of a way to correct an unsafe advanced controller online. The paper is clearly written, the architecture is well structured, and the case studies are relevant and nontrivial. The central safety argument is logically sound under the stated assumptions of exact deterministic plant dynamics and full observability, and the empirical comparisons show large and consistent improvements in the reported tables. However, the unconditional wording of the safety guarantee in the abstract and introduction is not supported by the formal development, and the empirical evaluation relies on single runs of stochastic RL algorithms without statistical measures. The paper's contribution is significant but needs qualification and additional evidence to fully support its claims.

major comments (4)

[Abstract and §1, vs. §3 and §7] The statement that NSA 'guarantees that the safety of the plant is never violated' is stated unconditionally in the abstract and Section 1, but the formal safety argument in Section 3 is explicitly conditional on the plant dynamics being exactly f, on full observability, and on the FSC/BC certification being derived from the same model. Section 7 defers deviations of the plant model from actual behavior to future L1Simplex work. Since the FSC in the rover and artificial pancreas case studies is implemented by one-step simulation using the same f, any model mismatch can cause an unsafe action to be classified as recoverable, and the guarantee fails. Please restate the guarantee with these assumptions made explicit, and in Section 5.2 note that the motivating scenario of training-time model parameters not matching the patient is precisely outside the regime where the current guarantee applies.
[§3, Switching logic] The formal condition imposed on the RSC, namely RSC(s) ⇒ ¬FSC(s,a) for all s∈S and a∈A, is stronger than what the proposed reverse switching strategies actually provide and stronger than what is needed. The simulation-based RSC checks that the FSC does not hold along a finite horizon for the actions generated by the current policy, not for all actions; the margin-based rover RSC depends only on the current state and does not bound the NC's next action in the way the stated condition requires. Please correct the condition to refer to the policy's action, e.g., RSC(s) ⇒ ¬FSC(s, π_θt(s)) for the deterministic policy used in the evaluation, or provide a separate argument if an action-independent RSC is intended.
[§5.2, Table 4 and Appendix C, Table 7] There is a numerical inconsistency that affects the reported effectiveness of AP retraining. The main text reports that among the first 400 AP trajectories, 250 led to forward switching and hence retraining, but Table 4 (and Table 7 in the appendix) states that there were only 61 updates to the NC. Under the update rule in Section 3, which performs one RL update per BC-controlled time step, 250 forward switches should produce at least 250 updates. Please clarify whether 'updates' means DDPG mini-batch gradient steps rather than per-time-step updates, and report the number of retraining samples and the number of gradient updates separately. As written, the claim that 61 updates eliminate all unrecoverable actions is not reproducible from the described setup.
[§4 and §5, Tables 1–7] All reported results are single runs of stochastic RL algorithms (DDPG and TRPO) evaluated on one set of random initial states, with no error bars, confidence intervals, or multiple seeds. Since the central empirical claim is that online retraining improves both safety and performance, the comparison between initially trained and retrained controllers needs to be supported by multiple independent runs or an equivalent statistical treatment. This is particularly important for the rover results in Table 3, where the differences in some metrics (e.g., Avg. Len. between 50K and 71K updates) are not monotonic and could be within seed-to-seed variation.

minor comments (5)

[§6, Related Work] There is a typo in the sentence 'we discuss in detail how the NSA approach guarantees runtime safety and how SRL is is used for online retraining': 'is is' should be 'is'.
[§6, Related Work] The text refers to 'Lyapuanov functions' in the discussion of [4] and [9]; this should be 'Lyapunov functions'.
[§5.1, Table 3] The average trajectory length after 71K retraining updates is lower than after 50K updates (146.56 vs. 156.13), even though the number of target arrivals increases. Since reaching the target earlier truncates the episode, this non-monotonicity is understandable, but it would be helpful to comment on it or to report a separate median episode length.
[§4] The DDPG and TRPO hyperparameters are described as following the rllab examples, but the specific hyperparameter values (learning rates, batch sizes, discount factor, exploration noise parameters) are not given. For reproducibility, please include a table of these settings for each case study.
[Appendix B] The videos are a nice supplement, but the claim that the initially trained NC generalizes 'reasonably well' to random obstacle fields is anecdotal; if this generalization claim is part of the evaluation, it should be quantified with a defined success rate over a fixed set of unseen fields.

Circularity Check

1 steps flagged · score 2.0 of 10

The runtime-safety switching argument is self-contained; only the SRL-PUA retraining evaluation is partly circular because the reported metric is the same predicate used to define the training penalty.

fitted input called prediction [Section 4, Eq. (1); rover reward Eq. (2) in Section 5.1; AP reward in Section 5.2]
"The reward function for retraining can be designed as follows. r(s,a,s′) = { r_unrecov, if FSC(s,a); r_perf(s,a,s′), otherwise } (1)"

The paper's retraining experiments are presented as showing that the neural controller becomes safer, with the key metric being the number of FSC-triggering or unrecoverable actions (e.g., Tables 3-5). But the FSC predicate is exactly the predicate used to define the training reward: actions satisfying FSC(s,a) receive a large negative reward, r_unrecov, and the rover's Eq. (2) assigns -20,000 precisely when FSC(s,a) holds. A policy trained to maximize this reward is therefore being evaluated on the very term it was optimized to minimize. The reported reduction from 1,000 unrecoverable trajectories to 0 is substantially the training objective restated as an outcome rather than an independent prediction of a separate safety property.

full rationale

The central safety claim of NSA is not circular. It is the standard Simplex argument: the BC is pre-certified to keep the plant inside the recoverable region RBC (an invariant), the forward switching condition is required to satisfy f(s,a) ∉ RBC ⇒ FSC(s,a), and the switching logic prevents unrecoverable NC actions from being applied. This derivation is self-contained and does not depend on the case-study data or on fitting any parameter to the reported outcomes. The unconditional wording of the guarantee is limited by assumptions of exact plant dynamics and full observability, and the paper itself defers model deviation to future L1Simplex work; that is an assumption/scope issue, not a circularity issue. The only identifiable circular content is in the SRL-PUA evaluation: the safety metric being improved is the same FSC predicate used to define the training reward, so the observed elimination of unrecoverable actions is partly by construction. The reuse of the authors' own AP plant model [6] and reward design [26] in a case study is a validation choice, not a load-bearing derivation step, so it does not raise the score further. Overall: central architecture independent; one partial self-referential evaluation; score 2.

Assumptions & free parameters 8 free parameters · 6 assumptions · 0 invented entities

The paper's empirical claims rest on hand-chosen reward constants, simulation horizons, and margins, plus a set of standard Simplex and plant-model assumptions. There are no invented physical entities. The load-bearing inputs are the exact-model assumption for f and the certified baseline controller assumption.

free parameters (8)

Rover unrecoverable-action reward penalty = -20,000
Chosen by hand in Eq. (2); huge penalty shapes policy away from FSC-triggering actions, central to SRL-PUA results.
Rover target-arrival reward = 10,000
Chosen by hand in Eq. (2) to reward reaching target; affects policy performance.
Rover distance penalty coefficients = -1 and -20
In Eq. (2), penalizes distance to target, shaping mission performance.
Rover reverse-switch margin m = 5
In Section 5.1 RSC condition lmin >= m*vmax*dt + dsafe + dbr_max + eps; chosen to avoid frequent switching; authors state behavior is not sensitive.
AP reward constants = 10, 14, 26.8, 16, 65.4 and factors 5, 7, 9, 20
In Eq. (6) and Appendix C, hand-chosen to penalize glycemic deviations; affects training and retraining.
IP reward constants = 10 for velocity penalty and 1 for angle penalty
Appendix A reward function chosen by hand; affects policy.
RSC simulation horizon T = 10
For inverted pendulum and artificial pancreas; authors say behavior is not sensitive to T.
DDPG/TRPO hyperparameters = rllab defaults
Not fully specified; reproducibility depends on defaults.

assumptions (6)

domain assumption Plant dynamics are deterministic and exactly known: s_{t+1}=f(s_t,a_t), with full observability of st.
Invoked in Section 3 to define FSC by one-step simulation and to generate shadow-mode retraining samples; if the model is wrong, FSC can misclassify unsafe actions and the safety guarantee breaks.
domain assumption Baseline controller is pre-certified to keep the plant inside recoverable region RBC and avoid unsafe states whenever it takes over within RBC.
Standard Simplex assumption (Section 2), load-bearing for the claim that safety is never violated.
domain assumption Forward switching condition is sound: FSC holds for any action that would leave RBC within one step, and the DM switches before the unsafe action reaches the plant.
Section 3 defines the invariant; if FSC is unsound due to discretization or model error, unsafe actions can be executed.
domain assumption For stochastic or nondeterministic plants, a model checker can be used instead of simulation (footnote 6); in the case studies, deterministic simulation is assumed sufficient.
The given rover RSC uses bounded worst-case speed and a sensor model, and AP/IP use deterministic simulators.
domain assumption The linear AP model from [6,7] and its ODEs describe patient physiology accurately enough for safety analysis.
Appendix C uses the model; if inaccurate, hypoglycemia predictions and FSC are wrong.
domain assumption Sensor model for the rover: the conservative minimum distance inside the cone of two adjacent sensors is min{li,lj} minus epsilon, with epsilon bounding obstacle protrusion.
Underlies rover FSC and RSC; if the sensor field-of-view assumption is violated, safety margins are mis-computed.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Neural Simplex Architecture." pith.science (2026). https://pith.science/paper/KJBERKRN

@misc{pith2026190800528,
  author       = {Pith},
  title        = {Pith review of: Neural Simplex Architecture},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KJBERKRN}},
  note         = {Machine review of arXiv:1908.00528}
}

read the original abstract

We present the Neural Simplex Architecture (NSA), a new approach to runtime assurance that provides safety guarantees for neural controllers (obtained e.g. using reinforcement learning) of autonomous and other complex systems without unduly sacrificing performance. NSA is inspired by the Simplex control architecture of Sha et al., but with some significant differences. In the traditional approach, the advanced controller (AC) is treated as a black box; when the decision module switches control to the baseline controller (BC), the BC remains in control forever. There is relatively little work on switching control back to the AC, and there are no techniques for correcting the AC's behavior after it generates a potentially unsafe control input that causes a failover to the BC. Our NSA addresses both of these limitations. NSA not only provides safety assurances in the presence of a possibly unsafe neural controller, but can also improve the safety of such a controller in an online setting via retraining, without overly degrading its performance. To demonstrate NSA's benefits, we have conducted several significant case studies in the continuous control domain. These include a target-seeking ground rover navigating an obstacle field, and a neural controller for an artificial pancreas system.

Figures

Figures reproduced from arXiv: 1908.00528 by the authors.

**Figure 1.** The Neural Simplex Architecture. The green check marks indicate pre-certified components. including neural controllers, of autonomous and other complex systems without unduly sacrificing performance. NSA is inspired by Sha et al.’s Simplex control architecture [30,28], where a pre-certified decision module (DM) switches control from a high-performance but unverified (hence potentially unsafe) advanced controller (A… view at source ↗

**Figure 2.** Switching boundaries. The blue region is a subset of the orange area, which in turn is a subset of the green region. To ensure safety when returning control to the NC, the FSC must not hold if the RSC is satisfied, i.e., RSC(s) ⇒ ¬FSC(s, a), for all s ∈ S, a ∈ A. We seek to develop reverse switching logic that returns control to NC when it is safe to do so and which avoids frequent back-and-forth switching between t… view at source ↗

**Figure 3.** Illustration of assumptions about obstacle shapes. We assume the sensors have a small angular field-of-view so that each sensor reading reflects the distance from the rover to an obstacle along the sensor’s center line of sight. If a sensor does not detect an obstacle, its reading is lmax. We also assume that when the sensor readings of two adjacent sensors si and sj are li and lj , respectively, then the (conservat… view at source ↗

Figures from the paper (3 more)

**Figure 3.** Figure 3: The NC is a DNN with two ReLU hidden layers, each of size 64, and a tanh output layer. We used the DDPG algorithm for both initial training and online retraining of the NC. For initial training, we ran DDPG for 5 million time steps. The reward function for initial trai…

**Figure 4.** Figure 4: Schematic of the inverted pendulum on a cart. Source: Wikipedia. We consider the classic control problem of keeping an inverted pendulum upright on a movable cart. We describe the problem briefly here; a detailed exposition is available in [29]. A schematic diagram app…

**Figure 5.** Figure 5: Training and testing setup for the rover case study. The red disks are obstacles, the black dot with an inscribed white triangle is the rover, and the blue dot is the target. The spokes coming out of the rover represent the distance sensors. The rover’s heading angle i…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Trusted Floors Under Untrusted Learners: A Runtime Assured-SLO Guard for ML Serving
cs.DC 2026-07 conditional novelty 6.0 of 10

Reservation plus assured-first priority holds admitted assured-class miss at 0.0 on real 2xV100 under every miscalibration of a learned admitter, while GAIE Flow Control fails under label swap.

Reference graph

Works this paper leans on

40 extracted references · 29 canonical work pages · cited by 1 Pith paper

[1]

In: International Conference on Machine Learning

Achiam, J., Held, D., Tamar, A., Abbeel, P.: Constrained policy optimization. In: International Conference on Machine Learning. pp. 22–31 (2017)

work page 2017
[2]

arXiv preprint arXiv:1708.08611 (2017) 16 D

Alshiekh, M., Bloem, R., Ehlers, R., K¨ onighofer, B., Niekum, S., Topcu, U.: Safe reinforcement learning via shielding. arXiv preprint arXiv:1708.08611 (2017) 16 D. Phan et al

arXiv 2017
[3]

In: AAAI (2018), https://www.aaai.org/ ocs/index.php/AAAI/AAAI18/paper/view/17211

Alshiekh, M., Bloem, R., Ehlers, R., K¨ onighofer, B., Niekum, S., Topcu, U.: Safe reinforcement learning via shielding. In: AAAI (2018), https://www.aaai.org/ ocs/index.php/AAAI/AAAI18/paper/view/17211

work page 2018
[4]

In: Advances in neural information processing systems

Berkenkamp, F., Turchetta, M., Schoellig, A., Krause, A.: Safe model-based rein- forcement learning with stability guarantees. In: Advances in neural information processing systems. pp. 908–918 (2017)

work page 2017
[5]

CoRR abs/1904.07189 (2019)

Bouton, M., Karlsson, J., Nakhaei, A., Fujimura, K., Kochenderfer, M.J., Tumova, J.: Reinforcement learning with probabilistic guarantees for autonomous driving. CoRR abs/1904.07189 (2019)

arXiv 2019
[6]

In: American Control Conference (ACC 2019)

Chen, H., Paoletti, N., Smolka, S.A., Lin, S.: Committed moving horizon estima- tion for meal detection and estimation in type 1 diabetes. In: American Control Conference (ACC 2019). pp. 4765–4772 (2019)

work page 2019
[7]

Medical Cyber-physical System Workshop (2015)

Chen, S., Weimer, J., Rickels, M., Peleckis, A., Lee, I.: Towards a model-based meal detector for type I diabetics. Medical Cyber-physical System Workshop (2015)

work page 2015
[8]

AAAI (2019)

Cheng, R., Orosz, G., Murray, R.M., Burdick, J.W.: End-to-end safe reinforce- ment learning through barrier functions for safety-critical continuous control tasks. AAAI (2019)

work page 2019

Show all 40 references

[9]

In: Advances in Neural Information Processing Systems

Chow, Y., Nachum, O., Duenez-Guzman, E., Ghavamzadeh, M.: A Lyapunov- based approach to safe reinforcement learning. In: Advances in Neural Information Processing Systems. pp. 8103–8112 (2018)

2018
[10]

ArXiv e-prints (Jan 2018)

Dalal, G., Dvijotham, K., Vecerik, M., Hester, T., Paduraru, C., Tassa, Y.: Safe exploration in continuous action spaces. ArXiv e-prints (Jan 2018)

2018
[11]

In: IEEE/IFIP International Conference on Dependable Systems and Networks (DSN) (Jun 2019)

Desai, A., Ghosh, S., Seshia, S.A., Shankar, N., Tiwari, A.: A runtime assurance framework for programming safe robotics systems. In: IEEE/IFIP International Conference on Dependable Systems and Networks (DSN) (Jun 2019)

2019
[12]

In: Proceedings of the 33rd Inter- national Conference on Machine Learning - Volume 48

Duan, Y., Chen, X., Houthooft, R., Schulman, J., Abbeel, P.: Benchmarking deep reinforcement learning for continuous control. In: Proceedings of the 33rd Inter- national Conference on Machine Learning - Volume 48. pp. 1329–1338. ICML’16 (2016), http://dl.acm.org/citation.cfm?i...

2016
[13]

In: 2014 Robotics: Science and Systems Conference (2014)

Fu, J., Topcu, U.: Probably approximately correct MDP learning and control with temporal logic constraints. In: 2014 Robotics: Science and Systems Conference (2014)

2014
[14]

In: AAAI’18 (2018)

Fulton, N., Platzer, A.: Safe reinforcement learning via formal methods. In: AAAI’18 (2018)

2018
[15]

In: Vojnar, T., Zhang, L

Fulton, N., Platzer, A.: Veriﬁably safe oﬀ-model reinforcement learning. In: Vojnar, T., Zhang, L. (eds.) Tools and Algorithms for the Construction and Analysis of Systems. pp. 413–430. Springer International Publishing, Cham (2019)

2019
[16]

Garc´ ıa, J., Fern´ andez, F.: A comprehensive survey on safe reinforcement learning. J. Mach. Learn. Res. 16(1), 1437–1480 (Jan 2015), http://dl.acm.org/citation. cfm?id=2789272.2886795

2015
[17]

ACM Transactions on Autonomous and Adaptive Systems (TAAS) 13(3), 14 (2019)

Garc´ ıa, J., Fern´ andez, F.: Probabilistic policy reuse for safe reinforcement learning. ACM Transactions on Autonomous and Adaptive Systems (TAAS) 13(3), 14 (2019)

2019
[18]

Automatica 71, 237–246 (2016)

Gondhalekar, R., Dassau, E., Doyle III, F.J.: Periodic Zone-MPC with asymmetric costs for outpatient-ready safety of an artiﬁcial pancreas to treat type 1 diabetes. Automatica 71, 237–246 (2016)

2016
[19]

CoRR abs/1801.08099 (2018)

Hasanbeig, M., Abate, A., Kroening, D.: Logically-correct reinforcement learning. CoRR abs/1801.08099 (2018)

2018 arXiv
[20]

ACM Trans

Johnson, T., Bak, S., Caccamo, M., Sha, L.: Real-time reachability for veriﬁed Simplex design. ACM Trans. Embed. Comput. Syst. 15(2), 26:1–26:27 (Feb 2016). https://doi.org/10.1145/2723871, http://doi.acm.org/10.1145/2723871 Neural Simplex Architecture 17

2016 doi
[21]

arXiv preprint arXiv:1509.02971 (2015)

Lillicrap, T., Hunt, J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., Wierstra, D.: Continuous control with deep reinforcement learning. arXiv preprint arXiv:1509.02971 (2015)

2015 arXiv
[22]

In: ICAART (2)

Mason, G., Calinescu, R., Kudenko, D., Banks, A.: Assured reinforcement learning with formally veriﬁed abstract policies. In: ICAART (2). pp. 105–117. SciTePress (2017)

2017
[23]

In: ICML

Mnih, V., Badia, A., Mirza, M., Graves, A., Lillicrap, T., Harley, T., Silver, D., Kavukcuoglu, K.: Asynchronous methods for deep reinforcement learning. In: ICML. pp. 1928–1937 (2016)

2016
[24]

In: ICML

Moldovan, T.M., Abbeel, P.: Safe exploration in Markov decision processes. In: ICML. icml.cc / Omnipress (2012)

2012
[25]

IEEE Transactions on Robotics pp

Ohnishi, M., Wang, L., Notomista, G., Egerstedt, M.: Barrier-certiﬁed adaptive re- inforcement learning with applications to brushbot navigation. IEEE Transactions on Robotics pp. 1–20 (2019). https://doi.org/10.1109/TRO.2019.2920206

2019
[26]

In: Computational Methods in Systems Biology

Paoletti, N., Liu, K., Smolka, S., Lin, S.: Data-driven robust control for type 1 diabetes under meal and exercise uncertainties. In: Computational Methods in Systems Biology. pp. 214–232. Springer (2017)

2017
[27]

In: ICML

Schulman, J., Levine, S., Abbeel, P., Jordan, M., Moritz, P.: Trust region policy optimization. In: ICML. pp. 1889–1897 (2015)

2015
[28]

In: Proc

Seto, D., Krogh, B., Sha, L., Chutinan, A.: The Simplex architecture for safe online control system upgrades. In: Proc. 1998 American Control Conference. vol. 6, pp. 3504–3508 (1998). https://doi.org/10.1109/ACC.1998.703255

1998
[29]

Seto, D., Sha, L., Compton, N.: A case study on analytical analysis of the inverted pendulum real-time control system (1999)

1999
[30]

IEEE Software18(4), 20–28 (2001)

Sha, L.: Using simplicity to control complexity. IEEE Software18(4), 20–28 (2001). https://doi.org/10.1109/MS.2001.936213

2001
[31]

arXiv preprint arXiv:1712.01815 (2017)

Silver, D., Hubert, T., Schrittwieser, J., et al.: Mastering chess and shogi by self-play with a general reinforcement learning algorithm. arXiv preprint arXiv:1712.01815 (2017)

2017 arXiv
[32]

Nature 550(7676), 354 (2017)

Silver, D., Schrittwieser, J., Simonyan, K., et al.: Mastering the game of Go without human knowledge. Nature 550(7676), 354 (2017)

2017
[33]

In: AAAI

Sim˜ ao, T.D., Spaan, M.T.J.: Safe policy improvement with baseline bootstrapping in factored environments. In: AAAI. pp. 4967–4974. AAAI Press (2019)

2019
[34]

MIT Press, Cam- bridge (1998)

Sutton, R., Barto, A.: Reinforcement Learning: An Introduction. MIT Press, Cam- bridge (1998)

1998
[35]

ArXiv e-prints (May 2018)

Tessler, C., Mankowitz, D.J., Mannor, S.: Reward constrained policy optimization. ArXiv e-prints (May 2018)

2018
[36]

In: 2016 IEEE 22nd International Conference on Embedded and Real-Time Computing Systems and Applications (RTCSA)

Vivekanandan, P., Garcia, G., Yun, H., Keshmiri, S.: A Simplex architecture for intelligent and safe unmanned aerial vehicles. In: 2016 IEEE 22nd International Conference on Embedded and Real-Time Computing Systems and Applications (RTCSA). pp. 69–75 (Aug 2016). https://doi.or...

2016 doi
[37]

In: 2013 ACM/IEEE International Conference on Cyber-Physical Systems (ICCPS)

Wang, X., Hovakimyan, N., Sha, L.: L1Simplex: Fault-tolerant control of cyber- physical systems. In: 2013 ACM/IEEE International Conference on Cyber-Physical Systems (ICCPS). pp. 41–50 (April 2013)

2013
[38]

arXiv preprint arXiv:1611.01224 (2016)

Wang, Z., Bapst, V., Heess, N., Mnih, V., Munos, R., Kavukcuoglu, K., de Fre- itas, N.: Sample eﬃcient actor-critic with experience replay. arXiv preprint arXiv:1611.01224 (2016)

2016 arXiv
[39]

In: IROS

Wen, M., Ehlers, R., Topcu, U.: Correct-by-synthesis reinforcement learning with temporal logic constraints. In: IROS. pp. 4983–4990. IEEE Computer Society Press (2015) 18 D. Phan et al

2015
[40]

ArXiv e-prints (Oct 2018) Neural Simplex Architecture 19 Appendix A Inverted Pendulum Case Study y x M F➝ l m Fig

Xiang, W., Musau, P., Wild, A.A., Manzanas Lopez, D., Hamilton, N., Yang, X., Rosenfeld, J., Johnson, T.T.: Veriﬁcation for machine learning, autonomy, and neural networks survey. ArXiv e-prints (Oct 2018) Neural Simplex Architecture 19 Appendix A Inverted Pendulum Case Study ...

2018

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [1]

In: International Conference on Machine Learning

Achiam, J., Held, D., Tamar, A., Abbeel, P.: Constrained policy optimization. In: International Conference on Machine Learning. pp. 22–31 (2017)

work page 2017

[2] [2]

arXiv preprint arXiv:1708.08611 (2017) 16 D

Alshiekh, M., Bloem, R., Ehlers, R., K¨ onighofer, B., Niekum, S., Topcu, U.: Safe reinforcement learning via shielding. arXiv preprint arXiv:1708.08611 (2017) 16 D. Phan et al

arXiv 2017

[3] [3]

In: AAAI (2018), https://www.aaai.org/ ocs/index.php/AAAI/AAAI18/paper/view/17211

Alshiekh, M., Bloem, R., Ehlers, R., K¨ onighofer, B., Niekum, S., Topcu, U.: Safe reinforcement learning via shielding. In: AAAI (2018), https://www.aaai.org/ ocs/index.php/AAAI/AAAI18/paper/view/17211

work page 2018

[4] [4]

In: Advances in neural information processing systems

Berkenkamp, F., Turchetta, M., Schoellig, A., Krause, A.: Safe model-based rein- forcement learning with stability guarantees. In: Advances in neural information processing systems. pp. 908–918 (2017)

work page 2017

[5] [5]

CoRR abs/1904.07189 (2019)

Bouton, M., Karlsson, J., Nakhaei, A., Fujimura, K., Kochenderfer, M.J., Tumova, J.: Reinforcement learning with probabilistic guarantees for autonomous driving. CoRR abs/1904.07189 (2019)

arXiv 2019

[6] [6]

In: American Control Conference (ACC 2019)

Chen, H., Paoletti, N., Smolka, S.A., Lin, S.: Committed moving horizon estima- tion for meal detection and estimation in type 1 diabetes. In: American Control Conference (ACC 2019). pp. 4765–4772 (2019)

work page 2019

[7] [7]

Medical Cyber-physical System Workshop (2015)

Chen, S., Weimer, J., Rickels, M., Peleckis, A., Lee, I.: Towards a model-based meal detector for type I diabetics. Medical Cyber-physical System Workshop (2015)

work page 2015

[8] [8]

AAAI (2019)

Cheng, R., Orosz, G., Murray, R.M., Burdick, J.W.: End-to-end safe reinforce- ment learning through barrier functions for safety-critical continuous control tasks. AAAI (2019)

work page 2019