Necessary and Sufficient Conditions for the Optimization-Based Concurrent Execution of Learned Robotic Tasks

Gennaro Notomista; Sheikh A. Tahmid

REVIEW 4 major objections 4 minor 16 references

This paper proves necessary and sufficient conditions for concurrently executing multiple RL-learned tasks with a min-norm controller, and extends the controller to discounted value functions.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 23:48 UTC pith:4NOJRMOM

load-bearing objection The discounted-sigma extension is solid, but Proposition 3 – the central sufficiency result – is false as stated; the Taylor-linearization step cannot be justified, and the stress-test counterexample holds up. the 4 major comments →

arxiv 2511.04054 v2 pith:4NOJRMOM submitted 2025-11-06 eess.SY cs.SY

Necessary and Sufficient Conditions for the Optimization-Based Concurrent Execution of Learned Robotic Tasks

Sheikh A. Tahmid , Gennaro Notomista This is my paper

classification eess.SY cs.SY

keywords multi-task controlreinforcement learningvalue functionsmin-norm controllerconcurrent executioncontrol Lyapunov functionsoptimization-based controlredundant robots

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper asks when a robot trained on several separate reinforcement-learning tasks can execute them all at once through a single optimization-based controller. It defines two precise properties—concurrent controllability and concurrent executability—and then gives necessary conditions (a shared goal equilibrium and a reachability condition on sublevel sets) and a sufficient condition (near a shared goal, with a fully actuated system, the tasks are guaranteed to be jointly executable). It also modifies the controller's constraint right-hand side to handle value functions trained with a discount factor, which is standard in RL but was not covered by the original framework. If correct, these results let engineers check learned task sets ahead of time for compatibility and combine them without retraining.

Core claim

The paper's central claim is that a set of value functions learned by reinforcement learning can be checked, using only their values and gradients, for whether they can be executed concurrently by an optimization-based min-norm controller. The authors define concurrent controllability as the absence of two disjoint subsets of task gradients being exact negative scalings of each other, and concurrent executability as the existence of an input that decreases every unfinished task's value. They prove necessary conditions: a common goal equilibrium must exist and be reachable without leaving the sublevel sets, and they prove a sufficient condition: whenever all value functions share a goal equil

What carries the argument

The central object is the pointwise min-norm controller: a quadratic program that at each state chooses the smallest input satisfying one inequality per task, each requiring the Lie derivative of that task's value function to decrease fast enough. The learned value functions act as candidate control Lyapunov functions—certificates that progress can be made—and their gradients enter the constraints. The argument also relies on a modified right-hand-side expression (the sigma-bar formula) that changes the prescribed decrease rate to match the Hamilton-Jacobi-Bellman equation, including when the value function was learned with a discount factor. The new task-relationship definitions, concurrent

Load-bearing premise

The proof of the sufficient condition assumes that arbitrarily close to a common goal, the linear (first-order) terms of each value function's Taylor expansion determine the sign of its decrease, so no two gradient sums can point in exactly opposite directions; this can fail for smooth value functions that are very flat at the goal.

What would settle it

Take a planar single-integrator robot and two smooth nonnegative value functions that both vanish only at the origin, but whose gradients at points x_n toward the origin alternate between alignment and opposition, for example V_i(x)=exp(-1/|x|^2)(2+sin(1/|x|)) with different phase offsets. Run the min-norm controller from points arbitrarily close to the origin. If the trajectory cannot make progress on both tasks in any ball around the origin, Proposition 3's existence claim fails, showing the sufficiency part needs extra regularity.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

If tasks share a goal and the robot is sufficiently overactuated, there is a guaranteed region around the goal where all tasks can be progressed simultaneously.
If no state exists where all value functions vanish, no compact region supports concurrent execution, so such task sets can be flagged as incompatible before deployment.
If a task's sublevel sets trap the dynamics, concurrent execution fails; this gives a design check for reward shaping and task training.
The discounted-value modification makes the framework compatible with standard RL discount factors, so policies trained with gamma < 1 can be plugged into the controller without losing optimal-input recovery.
The conditions are stated in terms of Lie derivatives and gradients, so they can be computed automatically from neural-network value functions via backpropagation.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The sufficient condition is local: it guarantees a neighborhood around the shared goal, not a global region; extending it to global guarantees would require showing the concurrently controllable region covers the whole operating set.
The sigma-bar correction is not tied to the specific min-norm formulation and could be applied to any constraint-based controller that uses value functions as control Lyapunov functions.
Enforcing a common goal and a positive-definite Hessian at that goal during RL training would make the local sufficient condition rigorous and could enlarge the guaranteed region substantially.
The necessary conditions suggest a practical diagnostic: task value functions intended to be combined should be trained with overlapping goals, otherwise the quadratic program will eventually deadlock at states where gradients oppose.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

The discounted-sigma extension is solid, but Proposition 3 – the central sufficiency result – is false as stated; the Taylor-linearization step cannot be justified, and the stress-test counterexample holds up.

read the letter

The headline you should know: the paper's new discounted-sigma expression is correct and genuinely useful, but the central sufficiency theorem (Proposition 3) is false as stated, and the proof's linearization step fails. The stress-test counterexample is valid; I checked it against the assumptions in Section II and it satisfies the discounted HJB.

What's actually new: Definitions 3 and 4 (concurrent executability and controllability) are natural generalizations of independence/orthogonality from [5], [6], [8], and they are a real step forward for talking about when learned value functions can be combined in a min-norm QP. Section IV.C's derivation of \bar{\sigma}_i is clean and correct. The discounted correction term involving \beta \tilde{J}_i makes the controller recover the optimal input for discounted value functions, which is a practical improvement over [7].

Where it falls apart: Proposition 3 claims that if smooth value functions share a common zero equilibrium point and g has full row rank, then there is a neighborhood of that point where the tasks are concurrently controllable. The proof drops higher-order Taylor terms without any error estimate, and even the first-order algebra is not enough: from \langle v, d\rangle = k\langle w, d\rangle you cannot conclude v and w don't point in opposite directions. More importantly, the claim itself is false under the stated smoothness assumptions. The counterexample with J_1,J_2 flat and oscillatory at the origin is exactly the right construction: both functions are C^\infty, nonnegative, zero only at 0, and their derivatives have opposite signs at points arbitrarily close to 0, so Definition 4 is violated in every neighborhood. These functions do come from a discounted HJB with q = \beta J + (J')^2/4. So the "necessary and sufficient" package in the abstract isn't supported. To salvage the theorem you'd need something like analyticity or a uniform positive-definite Hessian at the goal, and the paper has neither.

Other soft spots are smaller. Lemma 1 invokes LaSalle without fully checking the compactness/invariance details, and Proposition 2's proof is incoherent in a couple of places. Those could be fixed, but the Proposition 3 gap is load-bearing.

Who gets value: someone working on multi-task RL/control who wants the definitions and the discounted-sigma fix will get something out of it, and anyone who might rely on this kind of local argument should see the counterexample. But the central theorem doesn't hold, so it's not a paper to build on.

Recommendation: Send it to review if an editor wants a careful check, but I would not accept it without a major revision that adds assumptions and a real proof. If the authors can fix Proposition 3, the core idea has merit.

Referee Report

4 major / 4 minor

Summary. The paper studies the concurrent execution of multiple learned value functions using the pointwise min-norm controller from prior work [7], [8]. It introduces definitions of concurrent executability and concurrent controllability, proves necessary conditions (Propositions 1-2), and states a sufficiency result (Proposition 3) claiming that if learned value functions share a common zero at an equilibrium and the system is overactuated (g full row rank), then there is a neighborhood of that state in which the tasks are concurrently controllable and hence executable. It also proposes a modified slack/right-hand-side expression σ̅_i in the min-norm QP to handle discount factors. The paper includes simulations with a planar single-integrator robot illustrating the propositions.

Significance. If the main characterization were correct, it would fill a real gap in the optimization-based multi-task RL literature by telling practitioners when learned value functions can be combined. The discounted-sigma extension (Section IV.C) is a modest but sound contribution: substituting the optimal input into the HJB equation verifies the proposed expression. However, the central sufficiency theorem is false as stated, and the proposed definition of concurrent controllability does not actually imply concurrent executability even at the level of linear inequalities. These are load-bearing errors, not presentation issues. The paper therefore does not deliver the advertised necessary-and-sufficient characterization.

major comments (4)

[Section III, Definition 4 and Remark 2] Concurrent controllability does not imply concurrent executability. Let n=m=2 and consider three gradient vectors v1=(2,0), v2=(-1,1), v3=(-1,-2). No subset sum is a negative multiple of another disjoint subset sum, so Definition 4 holds. But 0 lies in the convex hull of {v1,v2,v3}, so by Farkas' lemma there is no u with v_i · u < 0 for all i. Thus the tasks are not concurrently executable in the sense of Definition 3, contradicting Remark 2. This invalidates the foundational link on which the sufficiency result is built.
[Section IV.B, Proposition 3 proof (Eqs. (6)-(9))] The proof discards higher-order Taylor terms without any error bound. Smoothness alone does not make the first-order terms dominate near a common zero. For a scalar system f=0, g=1, define J1(r)=e^{-1/r}(2+sin(1/r^2)) and J2(r)=e^{-1/r}(2+sin(1/r^2+π/2)), with J_i(0)=0. These are C^∞, nonnegative, zero only at r=0, and satisfy the HJB equation (2) with q_i=βJ_i+(J_i')^2/4. Yet their derivatives change sign and take opposite signs at points arbitrarily close to 0, so no neighborhood of the common goal is concurrently controllable. Proposition 3 is false under the stated assumptions; it requires an additional hypothesis such as analyticity or a uniform positive-definite Hessian at the goal.
[Section IV.B, Eq. (8)-(9)] Even the local algebra in the proof is not a valid inference. Equation (9) only gives ⟨Σ_{i∈P}∇J_i - k Σ_{i∈Q}∇J_i, x-a⟩ = 0 for one particular vector x-a. This does not prevent the two gradient sums from being opposite vectors. The proof needs a uniform angle bound over the neighborhood, which is exactly what the Taylor argument fails to provide.
[Section IV.A, Lemma 1] The proof asserts that L_\bar f ̅V(x)=0 if and only if J_1=···=J_N=0 and x is an equilibrium state. The first implication is fine under the strict-progress assumption, but the "x is an equilibrium state" part is not established: if the common zero set of the J_i is not a singleton, a trajectory could converge to and move within that set without x being an equilibrium. The proof cites LaSalle's theorem and appeals to an implicit argument in [16], but the claimed characterization of the invariant set is not a standard consequence unless the zero set is isolated or additional conditions are imposed.

minor comments (4)

[Section IV.A / IV.B] Typos: "F or" in the statement of Proposition 1 and "dyanimics" in the statement of Proposition 3 should be corrected.
[Section IV.C, Eq. (10)] The expression uses q(x) without a task index; it should be q_i(x). Also, the sentence "When R(x)=I in (3)" should refer to (2), since (3) is the controller and does not contain R.
[Section III, Definition 4] The definition does not specify whether P and Q are required to be disjoint. Clarify this, and note the implicit nonempty/nonzero conditions, to avoid ambiguity in the main condition.
[Section IV.A, Lemma 1 proof] The reference to "the proof in [16] implicitly proves that the invariant set is nonempty" is too vague. A direct argument using compactness and the omega-limit set would be more appropriate for a formal proof.

Circularity Check

0 steps flagged

No significant circularity: new conditions are derived from LaSalle, Taylor expansions, and HJB algebra, not from the conclusions they purport to establish.

full rationale

Walking the derivation chain: Section IV.A's necessity results are derived, not assumed. Lemma 1 applies LaSalle's theorem to the sum of value functions to conclude existence of a common zero set, and Propositions 1 and 2 are contrapositive applications of that lemma. These are conditional mathematical statements, not renamed inputs. Section IV.B's Proposition 3 attempts to prove existence of a concurrently controllable neighborhood from Taylor expansions around a common zero. Even if the Taylor linearization step (Eqs. (6)-(9)) is mathematically unjustified, that is a correctness gap, not circularity: the conclusion is not built into the hypotheses, and no fitted parameter is later relabeled as a prediction. Section IV.C derives the discounted sigma-bar expression by substituting the optimal input u* = -1/2 g^T grad J into the HJB equation; the displayed algebra explicitly yields q - beta J = (1/4) LgJ LgJ^T - LfJ, which is what makes the constraint active. This is a functional derivation, not a definition of the target in terms of itself. The reliance on [7] and [8] supplies the min-norm controller under analysis rather than evidence for the new theorems; no uniqueness theorem from the authors' prior work is imported to force a choice, and no ansatz is smuggled in through self-citation. The discounted sigma-bar claim is also independently checkable from the HJB equation. I separately note that Proposition 3's assertion that higher-order terms can be ignored near a common zero is not implied by smoothness alone; that is a soundness/falsity concern, not a circularity concern. Overall, no circular step is present.

Axiom & Free-Parameter Ledger

0 free parameters · 5 axioms · 0 invented entities

The paper introduces no fitted parameters; its free-parameter count is zero. The main unproved inputs are the control-affine system assumption, the strong assumption that L_g J_i=0 iff J_i=0, the full-row-rank assumption, smoothness of learned value functions (which conflicts with practical neural networks and is insufficient for Prop 3), and LaSalle's theorem applied in a way that needs extra justification.

axioms (5)

domain assumption System is deterministic, control-affine, and f,g are continuously differentiable (Eq. 1, Section II.A)
Foundation of the min-norm controller and all propositions.
domain assumption For each learned value function J_i, L_g J_i(x)=0 iff J_i(x)=0 (Definition 1 and Definition 4)
Used to relate zero gradient to completed task; strong for learned approximations.
domain assumption g(x) has full row rank (Proposition 3)
Needed to reduce L_g J_i to gradient of J_i; justified by redundant systems.
standard math LaSalle's theorem (Khalil Thm 4.4) applies to the closed-loop dynamics and implies nonempty invariant set G (Lemma 1)
Used to derive necessary conditions; application is not fully justified.
domain assumption Learned value functions are smooth and not identically zero (Proposition 3)
Taylor expansion used; neural-network value functions are not necessarily smooth or without spurious zeros.

pith-pipeline@v1.3.0-alltime-deepseek · 9970 in / 35893 out tokens · 320766 ms · 2026-08-03T23:48:37.962509+00:00 · methodology

0 comments

read the original abstract

In this work, we consider the problem of executing multiple tasks encoded by value functions, each learned through Reinforcement Learning, using an optimization-based framework. Prior works develop this framework but did not address when learned value functions can be concurrently executed. This work's main contributions consist of theorems which provide necessary and sufficient conditions to concurrently execute sets of learned tasks within subsets of the state space using the previously proposed min-norm controller. These theorems provide insight into when learned control tasks can be made concurrently executable, when they may already be so, and when concurrent execution is not possible under the proposed framework. We also extend the proposed framework to account for value functions trained with a discount factor, making it more compatible with standard RL practices.

Figures

Figures reproduced from arXiv: 2511.04054 by Gennaro Notomista, Sheikh A. Tahmid.

**Figure 1.** Figure 1: Heatmaps of J˜1, J˜2 and J˜3, respectively. V. SIMULATION RESULTS In this section, we provide illustrative examples demonstrating the propositions presented in the previous section with scenarios involving a planar mobile robot. Let X = R 2 be the robot position and the input space U = R 2 , so that x˙(t) = u(t). By picking a two-dimensional example, we are able to clearly illustrate the implications of P… view at source ↗

**Figure 2.** Figure 2: Heatmap of angles between the gradients of [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗

discussion (0)

Reference graph

Works this paper leans on

16 extracted references · 2 linked inside Pith

[1]

Bertsekas,Reinforcement Learning and Optimal Control

D. Bertsekas,Reinforcement Learning and Optimal Control. Athena Scientific optimization and computation series, Athena Scientific, 2019

2019
[2]

Milutinovic and J

D. Milutinovic and J. Rosen,Redundancy in Robot Manipulators and Multi-Robot Systems. Springer Publishing Company, Incorporated, 2012

2012
[3]

Active exploitation of redundancies in reconfigurable multirobot systems,

T. M. Roehr, “Active exploitation of redundancies in reconfigurable multirobot systems,”IEEE Transactions on Robotics, vol. 38, no. 1, pp. 180–196, 2022

2022
[4]

Task-priority based redundancy control of robot manipulators,

Y . Nakamura, H. Hanafusa, and T. Yoshikawa, “Task-priority based redundancy control of robot manipulators,”The International Journal of Robotics Research, vol. 6, no. 2, pp. 3–15, 1987

1987
[5]

Stability analysis for prioritized closed-loop inverse kinematic algorithms for redundant robotic systems,

G. Antonelli, “Stability analysis for prioritized closed-loop inverse kinematic algorithms for redundant robotic systems,”IEEE Transac- tions on Robotics, vol. 25, no. 5, pp. 985–994, 2009

2009
[6]

Beyond jacobian-based tasks: Extended set- based tasks for multi-task execution and prioritization,

G. Notomista, M. Selvaggio, M. Santos, S. Mayya, F. Pagano, V . Lip- piello, and C. Secchi, “Beyond jacobian-based tasks: Extended set- based tasks for multi-task execution and prioritization,” 2023

2023
[7]

A constrained-optimization approach to the execution of prioritized stacks of learned multi-robot tasks,

G. Notomista, “A constrained-optimization approach to the execution of prioritized stacks of learned multi-robot tasks,” inDistributed Autonomous Robotic Systems(J. Bourgeois, J. Paik, B. Piranda, J. Werfel, S. Hauert, A. Pierson, H. Hamann, T. L. Lam, F. Matsuno, N. Mehr, and A. Makhoul, eds.), (Cham), pp. 479–493, Springer Nature Switzerland, 2024

2024
[8]

Value iteration for learning con- currently executable robotic control tasks,

S. A. Tahmid and G. Notomista, “Value iteration for learning con- currently executable robotic control tasks,” inProceedings of the 24th International Conference on Autonomous Agents and Multiagent Systems, AAMAS ’25, (Richland, SC), p. 2006–2014, International Foundation for Autonomous Agents and Multiagent Systems, 2025

2006
[9]

A unified approach for motion and force control of robot manipulators: The operational space formulation,

O. Khatib, “A unified approach for motion and force control of robot manipulators: The operational space formulation,”IEEE Journal on Robotics and Automation, vol. 3, no. 1, pp. 43–53, 1987

1987
[10]

An op- timal task allocation strategy for heterogeneous multi-robot systems,

G. Notomista, S. Mayya, S. Hutchinson, and M. Egerstedt, “An op- timal task allocation strategy for heterogeneous multi-robot systems,” in2019 18th European Control Conference (ECC), pp. 2071–2076, 2019

2071
[11]

Continuous control with deep reinforce- ment learning,

T. P. Lillicrap, J. J. Hunt, A. Pritzel, N. Heess, T. Erez, Y . Tassa, D. Silver, and D. Wierstra, “Continuous control with deep reinforce- ment learning,” 2019

2019
[12]

Soft actor-critic: Off- policy maximum entropy deep reinforcement learning with a stochastic actor,

T. Haarnoja, A. Zhou, P. Abbeel, and S. Levine, “Soft actor-critic: Off- policy maximum entropy deep reinforcement learning with a stochastic actor,”arXiv preprint arXiv:1801.01290, 2018

Pith/arXiv arXiv 2018
[13]

Asynchronous methods for deep reinforcement learning,

V . Mnih, A. P. Badia, M. Mirza, A. Graves, T. Harley, T. P. Lillicrap, D. Silver, and K. Kavukcuoglu, “Asynchronous methods for deep reinforcement learning,”arXiv preprint arXiv:1602.01783, 2016

Pith/arXiv arXiv 2016
[14]

A survey of actor-critic reinforcement learning: Standard and natural policy gradients,

I. Grondman, L. Busoniu, G. A. D. Lopes, and R. Babuska, “A survey of actor-critic reinforcement learning: Standard and natural policy gradients,”IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), vol. 42, no. 6, pp. 1291–1307, 2012

2012
[15]

Value iteration in continuous actions, states and time,

M. Lutter, S. Mannor, J. Peters, D. Fox, and A. Garg, “Value iteration in continuous actions, states and time,” inProceedings of the 38th International Conference on Machine Learning(M. Meila and T. Zhang, eds.), vol. 139 ofProceedings of Machine Learning Research, pp. 7224–7234, PMLR, 18–24 Jul 2021

2021
[16]

H. K. Khalil,Nonlinear systems; 3rd ed.Upper Saddle River, NJ: Prentice-Hall, 2002. The book can be consulted by contacting: PH- AID: Wallet, Lionel

2002

[1] [1]

Bertsekas,Reinforcement Learning and Optimal Control

D. Bertsekas,Reinforcement Learning and Optimal Control. Athena Scientific optimization and computation series, Athena Scientific, 2019

2019

[2] [2]

Milutinovic and J

D. Milutinovic and J. Rosen,Redundancy in Robot Manipulators and Multi-Robot Systems. Springer Publishing Company, Incorporated, 2012

2012

[3] [3]

Active exploitation of redundancies in reconfigurable multirobot systems,

T. M. Roehr, “Active exploitation of redundancies in reconfigurable multirobot systems,”IEEE Transactions on Robotics, vol. 38, no. 1, pp. 180–196, 2022

2022

[4] [4]

Task-priority based redundancy control of robot manipulators,

Y . Nakamura, H. Hanafusa, and T. Yoshikawa, “Task-priority based redundancy control of robot manipulators,”The International Journal of Robotics Research, vol. 6, no. 2, pp. 3–15, 1987

1987

[5] [5]

Stability analysis for prioritized closed-loop inverse kinematic algorithms for redundant robotic systems,

G. Antonelli, “Stability analysis for prioritized closed-loop inverse kinematic algorithms for redundant robotic systems,”IEEE Transac- tions on Robotics, vol. 25, no. 5, pp. 985–994, 2009

2009

[6] [6]

Beyond jacobian-based tasks: Extended set- based tasks for multi-task execution and prioritization,

G. Notomista, M. Selvaggio, M. Santos, S. Mayya, F. Pagano, V . Lip- piello, and C. Secchi, “Beyond jacobian-based tasks: Extended set- based tasks for multi-task execution and prioritization,” 2023

2023

[7] [7]

A constrained-optimization approach to the execution of prioritized stacks of learned multi-robot tasks,

G. Notomista, “A constrained-optimization approach to the execution of prioritized stacks of learned multi-robot tasks,” inDistributed Autonomous Robotic Systems(J. Bourgeois, J. Paik, B. Piranda, J. Werfel, S. Hauert, A. Pierson, H. Hamann, T. L. Lam, F. Matsuno, N. Mehr, and A. Makhoul, eds.), (Cham), pp. 479–493, Springer Nature Switzerland, 2024

2024

[8] [8]

Value iteration for learning con- currently executable robotic control tasks,

S. A. Tahmid and G. Notomista, “Value iteration for learning con- currently executable robotic control tasks,” inProceedings of the 24th International Conference on Autonomous Agents and Multiagent Systems, AAMAS ’25, (Richland, SC), p. 2006–2014, International Foundation for Autonomous Agents and Multiagent Systems, 2025

2006

[9] [9]

A unified approach for motion and force control of robot manipulators: The operational space formulation,

O. Khatib, “A unified approach for motion and force control of robot manipulators: The operational space formulation,”IEEE Journal on Robotics and Automation, vol. 3, no. 1, pp. 43–53, 1987

1987

[10] [10]

An op- timal task allocation strategy for heterogeneous multi-robot systems,

G. Notomista, S. Mayya, S. Hutchinson, and M. Egerstedt, “An op- timal task allocation strategy for heterogeneous multi-robot systems,” in2019 18th European Control Conference (ECC), pp. 2071–2076, 2019

2071

[11] [11]

Continuous control with deep reinforce- ment learning,

T. P. Lillicrap, J. J. Hunt, A. Pritzel, N. Heess, T. Erez, Y . Tassa, D. Silver, and D. Wierstra, “Continuous control with deep reinforce- ment learning,” 2019

2019

[12] [12]

Soft actor-critic: Off- policy maximum entropy deep reinforcement learning with a stochastic actor,

T. Haarnoja, A. Zhou, P. Abbeel, and S. Levine, “Soft actor-critic: Off- policy maximum entropy deep reinforcement learning with a stochastic actor,”arXiv preprint arXiv:1801.01290, 2018

Pith/arXiv arXiv 2018

[13] [13]

Asynchronous methods for deep reinforcement learning,

V . Mnih, A. P. Badia, M. Mirza, A. Graves, T. Harley, T. P. Lillicrap, D. Silver, and K. Kavukcuoglu, “Asynchronous methods for deep reinforcement learning,”arXiv preprint arXiv:1602.01783, 2016

Pith/arXiv arXiv 2016

[14] [14]

A survey of actor-critic reinforcement learning: Standard and natural policy gradients,

I. Grondman, L. Busoniu, G. A. D. Lopes, and R. Babuska, “A survey of actor-critic reinforcement learning: Standard and natural policy gradients,”IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), vol. 42, no. 6, pp. 1291–1307, 2012

2012

[15] [15]

Value iteration in continuous actions, states and time,

M. Lutter, S. Mannor, J. Peters, D. Fox, and A. Garg, “Value iteration in continuous actions, states and time,” inProceedings of the 38th International Conference on Machine Learning(M. Meila and T. Zhang, eds.), vol. 139 ofProceedings of Machine Learning Research, pp. 7224–7234, PMLR, 18–24 Jul 2021

2021

[16] [16]

H. K. Khalil,Nonlinear systems; 3rd ed.Upper Saddle River, NJ: Prentice-Hall, 2002. The book can be consulted by contacting: PH- AID: Wallet, Lionel

2002