Pith. sign in

REVIEW 4 major objections 6 minor 50 references

Bridging Reinforcement Learning and Optimal Control via Feasible Action Mapping

T0 review · 4 major / 6 minor · reviewed 2026-07-31 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read FAOC claims that RL actions can live in a static convex set while a bijective mapping guarantees every induced OCP parameter is feasible, with gains shown in simulated table tennis.

desk verdict Solid theoretical contribution for mapping RL actions to feasible OCP parameters, but the strict continuous-time safety claim leans on an unproved patent result and the empirical section has fixable methodological quirks. read the letter →

arxiv 2607.23930 v1 pith:BRHL4KQO submitted 2026-07-27 eess.SY cs.ROcs.SY

Stefan Richter , Alberto Giammarino , Guillem Torrente , Sam Blakeman , Peter Dürr This is my paper
classification eess.SYcs.ROcs.SY
keywords feasibleactionmappingreinforcementlearningmodelpredictivecontrolconstrainedoptimalbijectivesetconvexanalysisrobottabletennisspacedesign
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that a reinforcement learning agent can explore freely in a static, geometrically simple action set while an optimization-based mapping algorithm translates each abstract action into a feasible parameter of the underlying optimal control problem. If true, the OCP always receives a reachable target, so constraint satisfaction is enforced by the optimal controller rather than learned by the RL agent. The authors prove the feasible parameter set is compact, solid, and convex under mild conditions, then give a bijective radial mapping with efficient implementations for implicitly defined sets. Simulated robot table tennis results support the claim that this design improves sample efficiency and final performance over one-dimensional reachability mappings and a soft-constrained two-dimensional baseline.

What carries the argument

The core mechanism is the radial scaling mapping between two compact, solid, convex sets X and Y. Given interior points xc and yc, the algorithm computes the boundary scaling factor alpha along the direction d = x - xc in X, applies an invertible positively homogeneous directional transformation phi(d), computes the boundary scaling factor beta along the transformed direction in Y, and outputs y = yc + (beta/alpha) phi(d). This map is bijective, and for real-time use the paper derives implicit formulations that compute interior points, linear transformation matrices from ellipsoidal surrogates, and boundary scaling factors directly from OCP constraints, avoiding explicit geometric representa

What would settle it

Search for a counterexample within the feasible set of OCP (9): choose a feasible parameter and initial state that maximize the sampled position or velocity over all 1 ms points of the continuous-time cubic spline; if any sample violates the original, unshrunk position or velocity limits, the paper's claimed constraint guarantee is false. This can be checked computationally by exhaustive or randomized search over the feasible polytope.

Watch

Extended reading notes

Core claim

The central claim is that for any abstract action in a static convex set A-bar, the proposed radial mapping M produces a parameter p in the state-dependent feasible set P(x) such that the parameterized OCP is feasible, and the mapping is bijective: every feasible parameter is reachable from some abstract action and no two abstract actions collapse to the same parameter. This removes the need for heuristic action-space bounds, slack variables, or soft terminal penalties, and it lets the RL agent ignore physical feasibility entirely. The bijectivity also enables inverse operations, such as converting expert demonstrations back into abstract actions.

Load-bearing premise

The guarantee that 1 ms samples of the planned continuous-time spline respect the original position and velocity limits rests on an unproved-in-this-paper conservatism result cited from patent [40] (Section 5.1.1), so if that result fails, the claimed physical constraint satisfaction collapses.

Editorial extensions

If this is right

  • RL agents can sample from a static hyperbox rather than a hand-tuned, state-dependent action space, eliminating a major source of design effort and infeasible exploration.
  • The OCP is never made infeasible by the RL action, so hard terminal constraints can replace soft penalties and slack variables, preserving the controller's optimality guarantees.
  • The bijective mapping enables inverse projection of demonstrated or expert-chosen OCP parameters back into the abstract action space, which supports replay buffers and data augmentation without action aliasing.
  • In two dimensions, the area-matching transformation preserves the target set's marginal area proportions and removes point accumulation; in higher dimensions, the ellipsoidal-surrogate linear transformation approximately achieves the same effect.
  • In the robot table tennis application, FAOC outperforms one-dimensional reachability-based mappings and a two-dimensional soft-constraint baseline in sample efficiency and closed-loop performance, and it degrades less when the RL decision frequency is lowered to 7.8 Hz.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Inference: If the cited conservatism result generalizes beyond the table-tennis setup, the same coarse-horizon-with-fine-sampling technique could apply to other triple-integrator-like systems, letting planners use long OCP intervals while still claiming dense-time constraint satisfaction.
  • Inference: The bijective mapping opens a natural path for expert-in-the-loop transfer: a human or scripted policy that specifies feasible OCP parameters can be projected back to abstract actions and used to pre-train the RL policy, which the paper does not explicitly explore.
  • Inference: A testable extension is applying FAOC to problems where P(x) is nonconvex, such as obstacle-avoidance waypoints; the paper argues the RL handles strategic nonconvexity while the OCP handles local convex constraints, but this decomposition has not been validated beyond table tennis.
  • Inference: The dependence on an interior point and boundary scaling suggests that very thin or nearly degenerate feasible sets may cause numerical sensitivity; the paper's robust inscribed-ball formulations address this, but their practical limits in extreme aspect ratios are not empirically quantified.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes FAOC, a hierarchical RL-OC scheme in which a neural policy outputs abstract actions in a fixed convex set, and an optimization-based mapping converts them into parameters p of a state-dependent OCP. The main theoretical results characterize when the feasible parameter set P(x) is compact/solid/convex (Lemma 1), prove that a radial boundary-scaling algorithm gives a bijection between the abstract set and P(x) (Proposition 1), and propose two shape-aware variants to reduce point accumulation (Propositions 2–4 and 8). The method is demonstrated on simulated robot table tennis with an 8-DoF arm and compared to 1D/2D baselines. The authors claim strict constraint satisfaction and improved sample efficiency.

Significance. If correct, FAOC is a useful bridge between RL and MPC: it removes the need for heuristically bounded action spaces, gives a bijective feasible-parameter mapping with real-time tractability via implicit-set formulations, and shows strong empirical performance. The theoretical framework extends prior 1D mapping [30] and hypercube-based methods [31] to general convex sets, and the open-source release is a strength. However, the headline safety guarantee depends on an unproved conservatism result, and the experimental comparison is weakened by post-hoc seed filtering and checkpoint selection. The core mapping theorem is credible, but the advertised guarantees need repair.

major comments (4)
  1. [Section 5.1.1, Eq. (9)] The claim of strict continuous-time constraint satisfaction is not established in this manuscript. The text states that shrinking the limits in Z_k guarantees, per patent [40], that sampling the piecewise-constant-jerk spline at 1 ms never violates the original limits. This result is not proved here, and the stated guarantee is only for 1 ms samples and backward-difference approximations of velocity, acceleration, and jerk, not for the continuous-time derivatives. Since Eq. (9) only constrains states at 8 ms interval boundaries, the continuous-time trajectory can overshoot. The OCP-level feasibility result (Proposition 1) does not by itself imply physical constraint satisfaction. Please provide a self-contained theorem with explicit conditions, or weaken the safety claims to discrete-time OCP feasibility.
  2. [Section 5.1.4, Figs. 5–6] The empirical evaluation uses two post-hoc selection steps: evaluations with return rate below 50% are excluded, and the best-performing checkpoint per seed is selected. This biases the comparison in favor of methods that happen to have a good evaluation run and weakens the reported Mann-Whitney p-values as evidence of overall performance. Report results for all seeds/checkpoints, or use a pre-specified protocol (e.g., final checkpoint), and state clearly whether the return-rate filter is applied identically across all methods.
  3. [Section 5.1.2] The 2Dsoft baseline is not fully specified: the actual stage/terminal weights (ws, W_t) used for the results in Figs. 5–6 are not reported; Fig. 4 only illustrates four candidate values. Since 2Dsoft performance is sensitive to these weights, and since the authors claim FAOC requires no tuning, the comparison is not reproducible without this information. Please report the hyperparameters and the tuning procedure.
  4. [Section 4.2, Proposition 3] Proposition 3's point-accumulation guarantee assumes that X is uniformly distributed over the abstract set. The SAC policy used in the experiments samples actions from a transformed Gaussian, not a uniform distribution. Thus the theoretical justification for the area-matching/linear transformation does not directly apply to the reported training procedure. Please either state this limitation explicitly and provide empirical evidence of the mapped action distribution, or extend the analysis to a broader class of distributions.
minor comments (6)
  1. [Section 3.2, Lemma 1] The proof says 'By Assumption 1' referring to the lemma's own numbered assumption, which conflicts with the global Assumption 1 introduced in Section 4.3. Please rename one of them to avoid ambiguity.
  2. [Section 4.1, Algorithm 1] The inverse mapping is described in the proof of Proposition 1 but not as an explicit algorithm. Provide pseudocode for M^{-1} for clarity and reproducibility.
  3. [Section 4.4.1, Propositions 6–7] The user-defined shape matrix G is a free parameter. The claim that the action space 'does not require expert or heuristic design' should be reconciled with the choice of G and of the surrogate ellipsoids.
  4. [Section 5.1.1] The notation P(x)⊂R^2 may confuse the reader because the RL action is 16-dimensional while the per-joint parameter is 2D. Please clarify the per-joint decomposition.
  5. [General] Reference [1] is cited for many experimental details. The present paper should be self-contained for its core claims, especially the reward design and the MCIS computation, or explicitly state which details are deferred to the companion paper.
  6. [Figure 4] The axes of the 'Action grid' panel are not labeled consistently with the other panels; clarify which quantity is position and which is velocity in each panel.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the FAOC mapping is a constructive theorem; self-citations supply experimental infrastructure and an external conservatism bound, not the central derivation.

full rationale

The central claim—that the RL abstract action set can be bijectively mapped onto the state-dependent feasible parameter set of the OCP—is a constructive mathematical result. Proposition 1 proves invertibility of the radial scaling algorithm under explicit assumptions (compact, solid, convex sets and a continuous, invertible, positively homogeneous directional transformation). The 'all valid parameters remain reachable' statement follows from surjectivity of the constructed map, not from any fitted parameter or empirical prediction. Similarly, Lemma 1 establishes compactness, solidness, and convexity of P(x) from stated geometric conditions; Corollary 1 specializes this to linear MPC. These are self-contained derivations, not renamings or tautologies. The area-matching transformation is defined to match angular area proportions, and Proposition 3 proves that property; this is a quantile-coupling construction, not an empirical pattern renamed as a discovery. The linear ellipsoidal-surrogate transformation is also derived, not fitted. The experimental comparison uses baselines constructed by the authors, but FAOC's advantage is an empirical result, not an artifact of the derivation chain. The main self-citations are [1] (the authors' Nature paper) for experimental infrastructure and [40] (a patent by the first author) for the conservatism bound that guarantees 1 ms sampling of an 8 ms piecewise-constant-jerk spline never violates original limits. These are load-bearing for the continuous-time safety guarantee and are not proved in this paper, making that guarantee a correctness risk. However, circularity requires the reduction of a claimed result to its own inputs; the cited conservatism result is an externally checkable mathematical claim, not a fitted input or an assumption equivalent to the paper's conclusion. Therefore, the paper has minor self-citations but no significant circularity in its central derivation.

Assumptions & free parameters 3 free parameters · 7 assumptions · 0 invented entities

The framework's central guarantee is conditional on convexity and strict feasibility of the OCP parameter set, plus the cited conservatism result for continuous-time sampling. No new physical entities are introduced. The main hidden degrees of freedom are the unstated baseline weights and shrink factors, which affect the empirical and safety claims.

free parameters (3)
  • 2Dsoft terminal/stage cost weights W_t, w_s = not reported
    Used to balance waypoint tracking vs jerk in the baseline; Fig. 4 shows multiple values but the main experiments do not state which were used. This is a hand-chosen parameter affecting fairness of the comparison.
  • Kinematic limit shrink factors in Z_k = not reported (from [40])
    The per-joint OCP shrinks position/velocity limits to guarantee that 1 ms resampling respects original limits. The shrink amount is a design choice cited to patent [40]; without its value the continuous-time safety guarantee cannot be independently checked.
  • User-defined shape matrix G (Propositions 6-7) = user-specified
    Chooses the geometry of the inscribed ball used for interior-point computation; affects numerical behavior, not bijectivity. It is an algorithmic degree of freedom rather than a fitted value.
assumptions (7)
  • domain assumption Lemma 1 conditions: Z compact convex; g jointly convex; S(x) has no directions of recession in p; Slater point exists; rg(H_p) ⊆ H_z T.
    These are necessary for P(x) to be compact, solid, and convex. If any fails, the FAOC guarantee does not hold.
  • ad hoc to paper Assumption 1: affinely related surrogate sets exist for X and Y.
    The linear transformation for point-accumulation mitigation assumes the true sets are well approximated by affinely related ellipsoids/polytopes; the paper obtains them via MVIE or lifted ellipsoids.
  • domain assumption Assumption 2: E(aff W) = R^n.
    Needed for the implicitly defined target set Y=EW+e to be solid; equivalent to E full row rank (sufficiently, W solid).
  • domain assumption Strictly feasible trajectory exists for every reachable initial state in table tennis.
    Corollary 1 needs strict feasibility so P(x) has nonempty interior; if the ball is on a boundary-feasible state, the mapping may break.
  • domain assumption The continuous-time guarantee follows from the shrunk-limit construction of [40].
    Used in Section 5.1.1 to claim that 1 ms sampling of the 8 ms-interval OCP never violates original position/velocity/jerk limits. The result is cited, not proved here; if it fails, the strict constraint-satisfaction claim on the physical system fails.
  • ad hoc to paper Uniform sampling in the abstract set is assumed in Proposition 3's point-accumulation analysis.
    The area-matching transform is proven to match angular marginals for uniform samples; SAC's bounded Gaussian policy is not uniform, so the theoretical justification for reduced point accumulation is approximate in practice.
  • domain assumption Each joint is modeled as an independent triple integrator with jerk input.
    OCP (9) ignores inter-joint coupling and actuator dynamics; this is the model in simulation and an approximation on hardware.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Bridging Reinforcement Learning and Optimal Control via Feasible Action Mapping." pith.science (2026). https://pith.science/paper/BRHL4KQO

@misc{pith2026260723930,
  author       = {Pith},
  title        = {Pith review of: Bridging Reinforcement Learning and Optimal Control via Feasible Action Mapping},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BRHL4KQO}},
  note         = {Machine review of arXiv:2607.23930}
}
read the original abstract

Operating constrained dynamical systems requires controllers to efficiently solve complex tasks while enforcing recursive feasibility and safety constraints. To address these competing requirements, we present Feasible Action for Optimal Control (FAOC), a novel control framework integrating Reinforcement Learning (RL) and Optimal Control (OC). The key contribution is a computationally efficient, optimization-based mapping algorithm that transforms the RL agent's action from a static abstract set into a state-dependent feasible parameter set of the Optimal Control Problem (OCP), guaranteeing strict satisfaction of the dynamical system's constraints. Thus, FAOC effectively combines the predictable safety of OC with the flexibility of RL. In contrast to prior work, the abstract action space of the RL agent does not require expert or heuristic design, and the OCP formulation is not compromised by the inability of RL to guarantee feasibility. We apply our approach to real-time motion planning for robot table tennis, which encapsulates these challenges. Via simulated experiments, we show that FAOC outperforms state-of-the-art baselines in both sample efficiency and closed-loop performance.

Figures

Figures reproduced from arXiv: 2607.23930 by the authors.

Figure 1
Figure 1. Overview of the FAOC framework illustrated for a 2D action space. From left to right: [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Visualization of the proposed mapping approaches. [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗
Figure 3
Figure 3. Illustration of FAOC deployed during a table tennis game. Top: the executed sequence of [PITH_FULL_IMAGE:figures/full_fig_p018_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Actions sampled in a 15 × 15 grid for FAOC and 2Dsoft (left). Comparison of FAOC (right￾most column) vs 2Dsoft (four left columns, for different combinations of cost weights Wt and ws) trajectories generated from those sampled actions for two different initial states (…
Figure 5
Figure 5. Figure 5: Statistical comparison of performance metrics (plot rows) for the five methods (five seeds [PITH_FULL_IMAGE:figures/full_fig_p021_5.png]
Figure 6
Figure 6. Figure 6: Means (bold lines) and standard deviations (shaded areas) of normalized reward as a function [PITH_FULL_IMAGE:figures/full_fig_p022_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

50 extracted references · 2 linked inside Pith

  1. [40]

    Method and apparatus for low-complexity trajectory planning,

    S. Richter, “Method and apparatus for low-complexity trajectory planning,” Patent WO2 025 011 908A1, 2025. [Online]. Available: https://patents.google.com/patent/ WO2025011908A1/en

    2025

  2. [30]

    Learning robot trajectories subject to kinematic joint constraints,

    J. C. Kiemel and T. Kröger, “Learning robot trajectories subject to kinematic joint constraints,” in 2021 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2021, p. 4799–4805

    2021

  3. [31]

    Action mapping: A reinforcement learning method for constrained-input systems,

    X. Yuan, Y. Wang, J. Liu, and C. Sun, “Action mapping: A reinforcement learning method for constrained-input systems,”IEEE Transactions on Neural Networks and Learning Systems, vol. 34, no. 10, pp. 7145–7157, 2023

    2023

  4. [1]

    Outplaying elite table tennis players with an autonomous robot,

    P . Dürr, M. El Gheche, G. J. Maeda, N. Mukai, N. Takahashi, S. Heusser, H. Sahloul, Y. Saraiji, P . Adodin, Y. Bi, S. Blakeman, C. Conti, D. Fuentes Hitos, Y. Hu, F. Khadivar, R. Kreiser, L. Martinez, F. Schilling, R. Tapiador Morales, G. Torrente, M. Ynocente Castro, L. Abecassis, A. Giammarino, Y.-T. Huang, Y. Nagel, A. Scotti, A. Sigrist, T. Silva, E....

    2026

  5. [2]

    Champion- level drone racing using deep reinforcement learning,

    E. Kaufmann, L. Bauersfeld, A. Loquercio, M. Mueller, V . Koltun, and D. Scaramuzza, “Champion- level drone racing using deep reinforcement learning,”Nature, vol. 620, pp. 982–987, 08 2023

    2023

  6. [3]

    Precise and dexterous robotic manipulation via human-in-the- loop reinforcement learning,

    J. Luo, C. Xu, J. Wu, and S. Levine, “Precise and dexterous robotic manipulation via human-in-the- loop reinforcement learning,”Science Robotics, vol. 10, no. 105, p. eads5033, 2025

    2025

  7. [4]

    Outracing champion gran turismo drivers with deep reinforcement learning,

    P . Wurman, S. Barrett, K. Kawamoto, J. MacGlashan, K. Subramanian, T. Walsh, R. Capobianco, A. Devlic, F. Eckert, F. Fuchs, L. Gilpin, P . Khandelwal, V . Kompella, H. Lin, P . MacAlpine, D. Oller, T. Seno, C. Sherstan, M. Thomure, and H. Kitano, “Outracing champion gran turismo drivers with deep reinforcement learning,”Nature, vol. 602, pp. 223–228, 02 2022

    2022

  8. [5]

    Anymal parkour: Learning agile navigation for quadrupedal robots,

    D. Hoeller, N. Rudin, D. Sako, and M. Hutter, “Anymal parkour: Learning agile navigation for quadrupedal robots,”Science Robotics, vol. 9, no. 88, p. eadi7566, 2024

    2024

Show all 50 references
  1. [6]

    Learning agile and dynamic motor skills for legged robots,

    J. Hwangbo, J. Lee, A. Dosovitskiy, D. Bellicoso, V . Tsounis, V . Koltun, and M. Hutter, “Learning agile and dynamic motor skills for legged robots,”Science Robotics, vol. 4, no. 26, p. eaau5872, 2019

    2019

  2. [7]

    Safe reinforcement learning for arm manipulation with constrained markov decision process,

    P . Adjei, N. Tasfi, S. Gomez-Rosero, and M. A. M. Capretz, “Safe reinforcement learning for arm manipulation with constrained markov decision process,”Robotics, vol. 13, no. 4, 2024

    2024

  3. [8]

    Safety-polarized and prioritized reinforcement learning,

    K. Fan, J. Zhang, X. Zhang, Y. Wu, J. Cao, Y. Zhou, and J. Ma, “Safety-polarized and prioritized reinforcement learning,” inProceedings of the 42nd International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, vol. 267. PMLR, 2025, pp. 15 862–15 886

    2025

  4. [9]

    Safe reinforcement learning via projection on a safe set: How to achieve optimality?

    S. Gros, M. Zanon, and A. Bemporad, “Safe reinforcement learning via projection on a safe set: How to achieve optimality?”IFAC-PapersOnLine, vol. 53, no. 2, pp. 8076–8081, 2020. 23

    2020

  5. [10]

    Data-driven mpc for quadrotors,

    G. Torrente, E. Kaufmann, P . Foehn, and D. Scaramuzza, “Data-driven mpc for quadrotors,”IEEE Robotics and Automation Letters, vol. 6, no. 2, pp. 3274–3281, 2021

    2021

  6. [11]

    Learning-based model predictive control: Toward safe learning in control,

    L. Hewing, K. P . Wabersich, M. Menner, and M. N. Zeilinger, “Learning-based model predictive control: Toward safe learning in control,”Annual Review of Control, Robotics, and Autonomous Systems, vol. 3, no. 1, pp. 269–296, 2020

    2020

  7. [12]

    Model predictive control: Theory and practice—a survey,

    C. E. García, D. M. Prett, and M. Morari, “Model predictive control: Theory and practice—a survey,”Automatica, vol. 25, no. 3, pp. 335–348, 1989

    1989

  8. [13]

    Constrained model predictive control: Stability and optimality,

    D. Mayne, J. Rawlings, C. Rao, and P . Scokaert, “Constrained model predictive control: Stability and optimality,”Automatica, vol. 36, no. 6, pp. 789–814, 2000

    2000

  9. [14]

    Borrelli, A

    F. Borrelli, A. Bemporad, and M. Morari,Predictive Control for Linear and Hybrid Systems. Cambridge University Press, 2017

    2017

  10. [15]

    Synthesis of model predictive control and reinforcement learning: Survey and classification,

    R. Reiter, J. Hoffmann, D. Reinhardt, F. Messerer, K. Baumgärtner, S. Sawant, J. Boedecker, M. Diehl, and S. Gros, “Synthesis of model predictive control and reinforcement learning: Survey and classification,”arXiv preprint arXiv:2502.02133, 2025

    2025 arXiv

  11. [16]

    Value function approximation and model predictive control,

    M. Zhong, M. Johnson, Y. Tassa, T. Erez, and E. Todorov, “Value function approximation and model predictive control,” in2013 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL). IEEE, 2013, pp. 100–107

    2013

  12. [17]

    Ac4mpc: Actor-critic reinforcement learning for guiding model predictive control,

    R. Reiter, A. Ghezzi, K. Baumgärtner, J. Hoffmann, R. D. McAllister, and M. Diehl, “Ac4mpc: Actor-critic reinforcement learning for guiding model predictive control,”IEEE Transactions on Control Systems Technology, 2025

    2025

  13. [18]

    Plan online, learn offline: Efficient learning and exploration via model-based control,

    K. Lowrey, A. Rajeswaran, S. Kakade, E. Todorov, and I. Mordatch, “Plan online, learn offline: Efficient learning and exploration via model-based control,” inInternational Conference on Learning Representations, 2019

    2019

  14. [19]

    An overview of the action space for deep reinforcement learning,

    J. Zhu, F. Wu, and J. Zhao, “An overview of the action space for deep reinforcement learning,” in Proceedings of the 2021 4th International Conference on Algorithms, Computing and Artificial Intelligence. Association for Computing Machinery, 2022

    2021

  15. [20]

    Where to go next: Learning a subgoal recommendation policy for navigation in dynamic environments,

    B. Brito, M. Everett, J. P . How, and J. Alonso-Mora, “Where to go next: Learning a subgoal recommendation policy for navigation in dynamic environments,”IEEE Robotics and Automation Letters, vol. 6, no. 3, pp. 4616–4623, 2021

    2021

  16. [21]

    Learning interaction-aware guidance for trajectory optimization in dense traffic scenarios,

    B. Brito, A. Agarwal, and J. Alonso-Mora, “Learning interaction-aware guidance for trajectory optimization in dense traffic scenarios,”IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 10, pp. 18 808–18 821, 2022

    2022

  17. [22]

    A hierarchical approach for strategic motion planning in autonomous racing,

    R. Reiter, J. Hoffmann, J. Boedecker, and M. Diehl, “A hierarchical approach for strategic motion planning in autonomous racing,” in2023 European Control Conference (ECC). IEEE, 2023, pp. 1–8

    2023

  18. [23]

    Parameterization approach of the frenet transformation for model predictive control of autonomous vehicles,

    R. Reiter and M. Diehl, “Parameterization approach of the frenet transformation for model predictive control of autonomous vehicles,” in2021 European Control Conference (ECC). IEEE, 2021, pp. 2414–2419

    2021

  19. [24]

    Reinforcement learning and model predictive control for robust embedded quadrotor guidance and control,

    C. Greatwood and A. G. Richards, “Reinforcement learning and model predictive control for robust embedded quadrotor guidance and control,”Autonomous Robots, vol. 43, no. 7, pp. 1681–1693, 2019

    2019

  20. [25]

    Rein- forcement learning-based receding horizon control using adaptive control barrier functions for safety-critical systems,

    E. Sabouni, H. S. Ahmad, V . Giammarino, C. G. Cassandras, I. C. Paschalidis, and W. Li, “Rein- forcement learning-based receding horizon control using adaptive control barrier functions for safety-critical systems,” in2024 IEEE 63rd Conference on Decision and Control (CDC). I...

    2024

  21. [26]

    Adaptive stochastic nonlinear model predictive control with look-ahead deep reinforcement learning for autonomous vehicle motion control,

    B. Zarrouki, C. Wang, and J. Betz, “Adaptive stochastic nonlinear model predictive control with look-ahead deep reinforcement learning for autonomous vehicle motion control,” in2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2024, pp. 12 ...

    2024

  22. [27]

    Learning when to drive in intersections by combining reinforcement learning and model predictive control,

    T. Tram, I. Batkovic, M. Ali, and J. Sjöberg, “Learning when to drive in intersections by combining reinforcement learning and model predictive control,” in2019 IEEE Intelligent Transportation Systems Conference (ITSC). IEEE, 2019, pp. 3263–3268

    2019

  23. [28]

    Safe reinforcement learning with chance- constrained model predictive control,

    S. Pfrommer, T. Gautam, A. Zhou, and S. Sojoudi, “Safe reinforcement learning with chance- constrained model predictive control,” inLearning for Dynamics and Control Conference. PMLR, 2022, pp. 291–303

    2022

  24. [29]

    A safe reinforcement learning driven weights-varying model predictive control for autonomous vehicle motion control,

    B. Zarrouki, M. Spanakakis, and J. Betz, “A safe reinforcement learning driven weights-varying model predictive control for autonomous vehicle motion control,” in2024 IEEE Intelligent Vehicles Symposium (IV). IEEE, 2024, pp. 1401–1408

    2024

  25. [32]

    Action mapping for reinforcement learning in continuous environments with constraints,

    M. Theile, L. Dirnberger, R. Trumpp, M. Caccamo, and A. Sangiovanni-Vincentelli, “Action mapping for reinforcement learning in continuous environments with constraints,”Reinforcement Learning Journal, vol. 6, pp. 343–363, 2025

    2025

  26. [33]

    Optimal transportation by orthogonal coupling dynamics,

    M. Sadr, P . Mohajerin Esfahani, and H. Gorji, “Optimal transportation by orthogonal coupling dynamics,”SIAM Journal on Scientific Computing, 2026

    2026

  27. [34]

    Learning for control: An inverse optimization approach,

    S. A. Akhtar, A. S. Kolarijani, and P . M. Esfahani, “Learning for control: An inverse optimization approach,”IEEE Control Systems Letters, vol. 6, pp. 187–192, 2021

    2021

  28. [35]

    Inverse optimization via learning feasible regions,

    K. Ren, P . Mohajerin Esfahani, and A. Georghiou, “Inverse optimization via learning feasible regions,” inProceedings of the International Conference on Machine Learning (ICML), 2025

    2025

  29. [36]

    R. T. Rockafellar,Convex Analysis. Princeton University Press, 1970

    1970

  30. [37]

    S. P . Boyd and L. Vandenberghe,Convex Optimization. Cambridge University Press, 2004

    2004

  31. [38]

    Spivak,Calculus, 4th ed

    M. Spivak,Calculus, 4th ed. Publish or Perish, Inc., 2008

    2008

  32. [39]

    J. B. Rawlings, D. Q. Mayne, and M. M. Diehl,Model Predictive Control: Theory, Computation, and Design, 2nd ed. Madison, WI: Nob Hill Publishing, 2017

    2017

  33. [41]

    A dual active-set solver for embedded quadratic programming using recursive LDLT updates,

    D. Arnström, A. Bemporad, and D. Axehill, “A dual active-set solver for embedded quadratic programming using recursive LDLT updates,”IEEE Transactions on Automatic Control, vol. 67, no. 8, pp. 4362–4369, 2022

    2022

  34. [42]

    The quickhull algorithm for convex hulls,

    C. B. Barber, D. P . Dobkin, and H. Huhdanpaa, “The quickhull algorithm for convex hulls,”ACM Transactions on Mathematical Software, vol. 22, no. 4, pp. 469–483, 1996

    1996

  35. [43]

    qpsolvers: Quadratic Programming Solvers in Python,

    S. Caron, D. Arnström, S. Bonagiri, A. Dechaume, N. Flowers, A. Heins, T. Ishikawa, D. Kenefake, G. Mazzamuto, D. Meoli, B. O’Donoghue, A. A. Oppenheimer, A. Pandala, J. J. Quiroz Omaña, N. Rontsis, P . Shah, S. St-Jean, N. Vitucci, S. Wolfers, F. Yang et al., “qpsolvers: Quad...

    2024

  36. [44]

    Universal value function approximators,

    T. Schaul, D. Horgan, K. Gregor, and D. Silver, “Universal value function approximators,” in International Conference on Machine Learning. PMLR, 2015, pp. 1312–1320

    2015

  37. [45]

    Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor,

    T. Haarnoja, A. Zhou, P . Abbeel, and S. Levine, “Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor,” inInternational Conference on Machine Learning. PMLR, 2018, pp. 1861–1870. 25

    2018

  38. [46]

    Bigger, regularized, optimistic: scaling for compute and sample efficient continuous control,

    M. Nauman, M. Ostaszewski, K. Jankowski, P . Miło´ s, and M. Cygan, “Bigger, regularized, optimistic: scaling for compute and sample efficient continuous control,”Advances in Neural Information Processing Systems, vol. 37, pp. 113 038–113 071, 2024

    2024

  39. [47]

    Hyperspherical normalization for scalable deep reinforcement learning,

    H. Lee, Y. Lee, T. Seno, D. Kim, P . Stone, and J. Choo, “Hyperspherical normalization for scalable deep reinforcement learning,”arXiv preprint arXiv:2502.15280, 2025

    2025 arXiv

  40. [48]

    Intrinsic benefits of categorical distributional loss: Uncertainty-aware regularized exploration in reinforcement learning,

    K. Sun, Y. Zhao, E. Shi, Y. Wang, X. Yan, B. Jiang, and L. Kong, “Intrinsic benefits of categorical distributional loss: Uncertainty-aware regularized exploration in reinforcement learning,” in Advances in Neural Information Processing Systems, 2025

    2025

  41. [49]

    Stop regressing: Training value functions via classification for scalable deep rl,

    J. Farebrother, J. Orbay, Q. Vuong, A. A. Taïga, Y. Chebotar, T. Xiao, A. Irpan, S. Levine, P . S. Castro, A. Faustet al., “Stop regressing: Training value functions via classification for scalable deep rl,” in International Conference on Machine Learning, 2024

    2024

  42. [50]

    1000 layer networks for self-supervised rl: Scaling depth can enable new goal-reaching capabilities,

    K. Wang, I. Javali, M. Bortkiewicz, B. Eysenbachet al., “1000 layer networks for self-supervised rl: Scaling depth can enable new goal-reaching capabilities,”arXiv preprint arXiv:2503.14858, 2025. 26

    2025

Pith tools

Reviewed July 31, 2026 · model on record in the stance chip above.