RECORD
On Entropy Regularized Path Integral Control for Trajectory Optimization
Optimization and Control · Submitted 8 Jul 2020
abstract compiled · 2026-09-23 · no language-model review
The authors' main claim We are able to identify a generic design strategy that relies on the existence of an optimal state trajectory distribution and finds a parametric policy by minimizing the cross entropy between the optimal and a state trajectory distribution parametrized through its policy.
The compiled record
The paper's own sentences, arranged. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. Numbers in the tables are the authors', checked by code. No sentence in this record was written by a model.
Open the compiled record
The abstract, sentence by sentence
In this article we present a generalised view on Path Integral Control (PIC) methods. PIC refers to a particular class of policy search methods that are closely tied to the setting of Linearly Solvable Optimal Control (LSOC), a restricted subclass of nonlinear Stochastic Optimal Control (SOC) problems. This class is unique in the sense that it can be solved explicitly to yield a formal optimal state trajectory distribution. In this contribution we first review the PIC theory and discuss related algorithms tailored to policy search in general. We are able to identify a generic design strategy that relies on the existence of an optimal state trajectory distribution and finds a parametric policy by minimizing the cross entropy between the optimal and a state trajectory distribution parametrized through its policy. Inspired by this observation we then aim to formulate a SOC problem that shares traits with the LSOC setting yet that covers a less restrictive class of problem formulations. We refer to this SOC problem as Entropy Regularized Trajectory Optimization. The problem is closely related to the Entropy Regularized Stochastic Optimal Control setting which is lately often addressed by the Reinforcement Learning (RL) community. We analyse the theoretical convergence behaviour of the theoretical state trajectory distribution sequence and draw connections with stochastic search methods tailored to classic optimization problems. Finally we derive explicit updates and compare the implied Entropy Regularized PIC with earlier work in the context of both PIC and RL for derivative-free trajectory optimization.
background contribution method result interpretation main claim
| # | sentence | role | the authors' own work | states the main conclusion | strength |
|---|---|---|---|---|---|
| s0 | In this article we present a generalised view on Path Integral Control (PIC) methods. | contribution 0.97 | 0.83 | 0.05 | asserted 0.98 |
| s1 | PIC refers to a particular class of policy search methods that are closely tied to the setting of Linearly Solvable Optimal Control (LSOC), a restricted subclass of nonlinear Stochastic Optimal Control (SOC) problems. | background 1.00 | 0.10 | 0.04 | asserted 1.00 |
| s2 | This class is unique in the sense that it can be solved explicitly to yield a formal optimal state trajectory distribution. | background 1.00 | 0.12 | 0.06 | asserted 1.00 |
| s3 | In this contribution we first review the PIC theory and discuss related algorithms tailored to policy search in general. | contribution 0.68 | 0.73 | 0.04 | asserted 0.97 |
| s4 | We are able to identify a generic design strategy that relies on the existence of an optimal state trajectory distribution and finds a parametric policy by minimizing the cross entropy between the optimal and a state trajectory distribution parametrized through its policy. | contribution 0.60 | 0.90 | 0.22 | asserted 1.00 |
| s5 | Inspired by this observation we then aim to formulate a SOC problem that shares traits with the LSOC setting yet that covers a less restrictive class of problem formulations. | contribution 0.95 | 0.76 | 0.07 | asserted 0.55 |
| s6 | We refer to this SOC problem as Entropy Regularized Trajectory Optimization. | contribution 0.94 | 0.83 | 0.10 | asserted 0.99 |
| s7 | The problem is closely related to the Entropy Regularized Stochastic Optimal Control setting which is lately often addressed by the Reinforcement Learning (RL) community. | background 0.96 | 0.12 | 0.05 | asserted 0.97 |
| s8 | We analyse the theoretical convergence behaviour of the theoretical state trajectory distribution sequence and draw connections with stochastic search methods tailored to classic optimization problems. | contribution 0.72 | 0.91 | 0.08 | asserted 0.99 |
| s9 | Finally we derive explicit updates and compare the implied Entropy Regularized PIC with earlier work in the context of both PIC and RL for derivative-free trajectory optimization. | contribution 0.86 | 0.95 | 0.22 | asserted 1.00 |
What was examined, and what was not
- arXiv listing abstract v2
- full text
- figures and tables
- references and citing sentences
- relations to other papers
- P abstract
- S sections
- T body sentences
- C citations
- R cross-paper relations
The paper
Original abstract
In this article we present a generalised view on Path Integral Control (PIC) methods. PIC refers to a particular class of policy search methods that are closely tied to the setting of Linearly Solvable Optimal Control (LSOC), a restricted subclass of nonlinear Stochastic Optimal Control (SOC) problems. This class is unique in the sense that it can be solved explicitly to yield a formal optimal state trajectory distribution. In this contribution we first review the PIC theory and discuss related algorithms tailored to policy search in general. We are able to identify a generic design strategy that relies on the existence of an optimal state trajectory distribution and finds a parametric policy by minimizing the cross entropy between the optimal and a state trajectory distribution parametrized through its policy. Inspired by this observation we then aim to formulate a SOC problem that shares traits with the LSOC setting yet that covers a less restrictive class of problem formulations. We refer to this SOC problem as Entropy Regularized Trajectory Optimization. The problem is closely related to the Entropy Regularized Stochastic Optimal Control setting which is lately often addressed by the Reinforcement Learning (RL) community. We analyse the theoretical convergence behaviour of the theoretical state trajectory distribution sequence and draw connections with stochastic search methods tailored to classic optimization problems. Finally we derive explicit updates and compare the implied Entropy Regularized PIC with earlier work in the context of both PIC and RL for derivative-free trajectory optimization.
Human review
How the literature uses it
What this paper does with the arXiv papers it cites
What it cites · 13 sentences without a confident label
Cites as background 4 papers
-
The interested reader is referred to earlier references [ref] .
Lyceum: An efficient and scalable ecosystem for robot learning · Path Integral Control (PIC) Methods · 1.00
-
The most important members of this class are PI2 [ref] , PI2 with Covariance Matrix Adaptation [ref] and PI2 with Population Adaptation [ref] .
Path Integral Policy Improvement with Covariance Matrix Adaptation · Other noteworthy PIC methods · 0.91
-
Nowadays, this is a setting of widespread use in the RL community and was explored extensively by Levine et al. particularly in the context of Actor-Critic RL algorithms [ref] .
Soft Actor-Critic Algorithms and Applications · Entropy Regularized Optimal Control · 0.90
Discussion
Know something about this paper? Leave a comment, or highlight any passage above to comment on it.