Pith. sign in

REVIEW 3 major objections 6 minor 142 references

Geometry of Neural Reinforcement Learning in Continuous State and Action Spaces

T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A policy-gradient-trained two-layer neural policy confines a continuous-control agent's reachable states to a manifold of dimension at most $2d_a+1$, independent of state dimension.

desk verdict A genuinely new geometric bound for neural RL policies, but the proof's central concentration step is asserted rather than derived; worth refereeing carefully. read the letter →

arxiv 2507.20853 v1 pith:PUYAYCP4 submitted 2025-07-28 cs.LG cs.AI

Saket Tiwari , Omer Gottesman , George Konidaris This is my paper
classification cs.LGcs.AI MSC 68T0793B2758A05
keywords reinforcementlearningcontinuouscontrolmanifoldhypothesistwo-layerneuralnetworksgeometrictheoryintrinsicdimensionalityactor-critic
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish a geometric version of the manifold hypothesis for reinforcement learning: when a continuous-control agent's policy is a wide two-layer neural network trained by policy-gradient-style updates, the states the agent actually visits stay close to a low-dimensional manifold inside the nominal state space. The proven bound is that this manifold has dimension at most $2d_a+1$, where $d_a$ is the number of action components, so the bound does not grow with the dimensionality of the state space. A sympathetic reader would care because it suggests that the intrinsic difficulty of a continuous-control problem is set by the action space, not by the apparent size of the state space, which would justify representation-learning methods that compress states. The paper supports the bound with intrinsic-dimension estimates on four MuJoCo tasks and a toy linear system, and shows that a sparse-representation layer exploiting this structure improves SAC in high-dimensional control domains.

What carries the argument

The load-bearing machinery is the exponential map of the parametrized policy's vector field, $X(W) = g(x) + h(x) \Phi(x; W_0) W$, expanded as a Lie series truncated at second order, together with a continuous-time limit of the stochastic semi-gradient updates that tracks a finite set of summary statistics of the policy outputs. Because the linearized policy's outputs $A^\tau_j(s)$ and their quadratic combinations concentrate (Lemmas 8 and 9), the second-order Lie-series terms collapse onto the span of $h_1,\dots,h_{d_a}$, $v^\tau_1,\dots,v^\tau_{d_a}$ where $v^\tau_j = J_{h_j}(s) h(s) \bar a^\tau(s)$, and the paraboloid $tg + t^2 g'$; these give the $2d_a+1$ generators of the local manifold. The role of this machinery is to convert an infinite-dimensional parameter update into a low-dimensional geometric statement about where the agent's state can go.

What would settle it

Run an actor-critic agent (SAC or DDPG) on a deterministic control-affine task with $d_a=1$ and $d_s\ge 64$, record all states visited during training, and estimate their intrinsic dimension with the same two-nearest-neighbor estimator used in the paper; if the estimate grows with $d_s$ and stays well above $3$, the theorem's central bound fails in the regime the abstract describes.

Watch

Extended reading notes

Core claim

The central claim is Theorem 1: for a deterministic continuous-time MDP with smooth control-affine dynamics, a sequence of linearized infinite-width two-layer GeLU policies whose parameters evolve by semi-gradient updates (assuming an oracle supplies the exact action-value gradient $\nabla_a Q^\pi$), the push-forward of the parameter distribution through the exponential map of the policy's vector field is concentrated, at gradient time $\tau$, around a manifold of dimension $m \leq 2d_a+1$, with the probability of straying farther than $D$ decaying exponentially. The manifold is locally spanned by the $d_a$ action vector fields $h_j$, the $d_a$ curvature directions $v^\tau_j = J_{h_j}(s) h(s) \bar a^\tau(s)$ built from the Jacobians of the action fields and the mean quadratic action statistics, and the time-paraboloid direction $t g + t^2 g'$. The paper claims this as the first result linking the geometry of the state space to the dimensionality of the action space, and corroborates it empirically with the two-nearest-neighbor intrinsic-dimension estimator.

Load-bearing premise

The proof assumes an oracle that provides the exact gradient of the action-value function with respect to the action at each update; actual actor-critic algorithms estimate this quantity from noisy, biased data, and the paper itself calls the oracle assumption 'not practical,' so Theorem 1 as proven does not cover the algorithm named in the abstract.

Editorial extensions

If this is right

  • The nominal state-space dimension $d_s$ is the wrong measure of problem complexity: the effective dimension of the states a neural policy reaches is bounded by $2d_a+1$, which can be far smaller.
  • Representation-learning methods that compress states to the intrinsic manifold are justified by the dynamics of training itself, not just by a generic data assumption.
  • Approximation, sample-complexity, and off-policy-evaluation bounds that depend on the state manifold's dimension can be anchored to $2d_a+1$ instead of $d_s$.
  • Adding a layer that learns sparse low-dimensional representations should help in high-dimensional control with small action spaces; its successful use with SAC in the paper is a direct application of the bound.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: if the bound survives replacement of the oracle $\nabla_a Q^\pi$ by an estimated critic, then practical actor-critic methods inherit the same geometry, and the intrinsic dimension of states visited during training should track $2d_a+1$ despite critic noise and bias.
  • Beyond the paper: because the curvature directions $v^\tau_j$ are built from the Jacobians of the action vector fields, environments with strongly coupled joints should saturate the bound while decoupled ones should fall below it; an experiment could order environments by this coupling and check the manifold dimension follows.
  • Beyond the paper: the theorem's concentration rate depends on the operator norms of $J_{h_j}$ and the values of $g_k(s)$, so near equilibria the effective manifold may shrink further, which would predict that late-training trajectories appear even lower-dimensional than $2d_a+1$.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper develops a geometric theory for continuous state and action reinforcement learning. It models the policy as an infinite-width two-layer network linearized in its parameters, uses continuous-time semi-gradient updates with an oracle for the action-value gradient, and proves (Theorem 1) that the push-forward of the random policy parameters under the system's exponential map concentrates around a manifold of dimension at most 2da + 1, independent of the state-space dimension. The authors empirically estimate the intrinsic dimension of evaluation-trajectory states in four MuJoCo environments and in a toy linear system, and they introduce a CRATE-based sparsification layer into SAC to exploit the predicted low-dimensional structure. The main theoretical argument is in Appendix F (following Ben Arous et al. 2022) and Appendix G, where the exponential-map Lie series is combined with limiting ODEs for actions and action products.

Significance. If fully established, Theorem 1 would be a genuinely novel result connecting the geometry of state trajectories, the width-parameterization of neural policies, and the action-space dimension; the use of an independent dimensionality estimator with no fitted constants is a methodological strength, and the CRATE application is a useful practical demonstration. The proof framework built on Ben Arous et al. (2022) is appropriate for the idealized model, and the paper is explicit about several limitations, including the oracle assumption and deterministic transitions. However, the concentration step in Appendix G, which is the load-bearing part of the main theorem, is not proved; the dimension count in Section 4 is internally inconsistent; and the oracle assumption means the theorem does not directly cover the actor-critic algorithms used in the experiments. These issues prevent acceptance in the current form.

major comments (3)
  1. [Appendix G, Eq. (19), Definition 6] The claim that 'the concentration property is a result of the concentration of a^τ_j(s)A^τ(s) around \bar B^τ_j due to the dynamics in 19' is not supported. Equation (19), which is Lemma 9, is a convergence-in-distribution statement for the product process: d(\bar A_j \bar A_{j'}) = (v_j \bar A_{j'} + v_{j'} \bar A_j)dτ, and the limit remains random through the initialization X_0. It provides no variance bound, no large-deviation estimate, and no control on the distance between a^τ_j(s)A^τ(s) and E[a^τ_j(s)A^τ(s)]. Under Lemma 8 and the CLT, each A^τ_j has O(1) variance, so the product has O(1) fluctuations and the distance from a typical realization to the manifold spanned by the mean products does not vanish as n→∞. Thus the exponential probability bound required by Definition 6 is not established for M_{δ',τ}. Please either supply a rigorous concentration argument or add an explicit concentration assumption to Theorem 1 and adjust the statement accordingly.
  2. [Section 4, Theorem 1 and proof sketch] The dimension count is internally inconsistent. Theorem 1 states m ≤ 2da + 1 for varying δt ∈ (0,δ), but the proof sketch immediately after the theorem says 'for variable t there is a 2da + 2 around which the data lie,' and the informal paragraph after the theorem says the trajectories are concentrated around a '2da + 3 manifold.' In addition, Appendix G constructs only a linear span of vectors at a fixed state s and never defines M_{δ',τ} as a submanifold with coordinate charts. Please reconcile the dimension bound and provide a formal definition of M_{δ',τ}.
  3. [Section 3.2 oracle assumption; Section 5.2 empirical validation] Theorem 1 assumes an oracle that provides exact gradients ∇_a Q^{W_{kη}}, with the text noting these gradients 'do not need to be true in practice,' while the abstract and title claim results for actor-critic training. Actual actor-critic algorithms estimate Q from data, so the theorem as proven does not cover the DDPG and SAC algorithms used in the experiments. The Discussion acknowledges the oracle assumption, but the abstract's claim about policies 'trained using an actor-critic algorithm' and the empirical corroboration in Section 5.2 are stronger than what the theorem supports. Please either restrict the claims to the oracle setting or analyze the gap between the oracle and learned Q estimates.
minor comments (6)
  1. [Definition 6] The expression 'D − O(ϵ)' is undefined because no parameter ϵ is introduced; please clarify the quantifiers over ϵ and D.
  2. [Appendix G, \bar B^τ_j definition] The vector \bar B^τ_j is written as [E[A^τ_j A^τ_1], ..., E[A^τ_j A^τ_1]], repeating the first entry; it should presumably be [E[A^τ_j A^τ_1], ..., E[A^τ_j A^τ_{da}]].
  3. [Lemma 8 and Lemma 9] Both lemmas describe the policy as having 'two hidden layers,' although the model in Section 3.1 is a single-hidden-layer two-layer network.
  4. [Section 5.2] The intrinsic dimension is estimated from the full set of states sampled from evaluation trajectories, whereas Theorem 1 is a local statement around a fixed state over a small time interval δ; please report local estimates or explain why the global point-cloud estimate is the relevant quantity.
  5. [Section 3.2 and Section 4] There are several typographical errors, including 'Simiar formulation' in Section 3.2 and 'their is an additional degrees of freedom' in Section 4; a careful proofreading pass is needed.
  6. [Equation (13) and surrounding text] The rate O(1/n) for the strong law of large numbers is used without specifying the mode of convergence or the norm in which the error is measured; please state whether the convergence is in probability, almost surely, or in expectation and under what boundedness conditions.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; Theorem 1 is derived from stated assumptions and external martingale/CLT results, with no fitted constants or load-bearing self-citation.

full rationale

No circular step is present in the paper's derivation chain. Theorem 1 is proved from explicit assumptions (Assumptions 4 and 5, bounded derivatives of the activation) using continuous-time SGD and martingale arguments adapted from Ben Arous et al. (2022), which is an external reference rather than a self-citation chain. The dimension bound 2da+1 is not fitted: the manifold is defined by the vectors h_j, v^tau_j = J h_j(s) h(s) \bar B^tau_j, and the paraboloid tg + t^2 g', where \bar B^tau_j is the expectation of the limiting product process, and no empirical measurement enters the theorem. The empirical validation is corroborative, using the independent intrinsic-dimension estimator of Facco et al. (2017), and the toy experiment samples from the defined linearized-policy family rather than fitting the theory's parameters. The CRATE layer is imported from prior work as an application, not derived from Theorem 1. The only self-citation, Tiwari & Konidaris (2022), appears in related work and is not load-bearing. One genuine concern is that Appendix G asserts concentration of a^tau_j(s) A^tau(s) around \bar B^tau_j 'due to the dynamics in 19,' but the weak-convergence ODE in Eq. 19 does not by itself establish the exponential concentration required by Definition 6; this is a correctness gap in the proof, not a circularity, because the conclusion is not equivalent to an input or a fitted quantity by construction.

Assumptions & free parameters 0 free parameters · 6 assumptions · 0 invented entities

The theory introduces no fitted constants and no new physical or mathematical entities. It relies on the NTK-style linearized policy model, an exact value-gradient oracle, a specific infinite-width scaling, and external limit theorems.

assumptions (6)
  • domain assumption The activation function φ has bounded first and second derivatives everywhere (satisfied by GeLU).
    Used throughout to justify the boundedness and convergence of gradient terms; stated in Section 3.1 and Appendix E.
  • domain assumption The dynamics are deterministic and control-affine: ṡ = g(s) + Σ_i h_i(s) a_i with smooth g, h_i.
    Defines the MDP model in Section 2.1; the Lie-series and manifold arguments rely on smoothness.
  • domain assumption The policy update has access to an oracle providing exact value-function gradients ∇_a Q^π.
    Assumed in Section 3.2; acknowledged as impractical in the Discussion.
  • domain assumption Infinite width limit n→∞ with learning rate η_n → 0 at rate 1/√n, and Assumptions 4 and 5 hold (fourth-moment noise bound and Lipschitz continuity of the mean gradient).
    These scaling conditions are required for the weak convergence of the SGD statistics; they follow the framework of Ben Arous et al. (2022).
  • domain assumption Policy parameters remain in a bounded ball ||W - W0|| ≤ r.
    Definition 3 restricts the policy family; the compactness is used in the convergence arguments.
  • standard math High-dimensional SGD limit theorems of Ben Arous et al. (2022) and the Lindeberg-Lévy CLT hold for the summary statistics.
    The proof of Lemmas 8 and 9 closely follows Ben Arous et al. (2022), treating their theorem as background.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Geometry of Neural Reinforcement Learning in Continuous State and Action Spaces." pith.science (2026). https://pith.science/paper/PUYAYCP4

@misc{pith2026250720853,
  author       = {Pith},
  title        = {Pith review of: Geometry of Neural Reinforcement Learning in Continuous State and Action Spaces},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PUYAYCP4}},
  note         = {Machine review of arXiv:2507.20853}
}
read the original abstract

Advances in reinforcement learning (RL) have led to its successful application in complex tasks with continuous state and action spaces. Despite these advances in practice, most theoretical work pertains to finite state and action spaces. We propose building a theoretical understanding of continuous state and action spaces by employing a geometric lens to understand the locally attained set of states. The set of all parametrised policies learnt through a semi-gradient based approach induces a set of attainable states in RL. We show that the training dynamics of a two-layer neural policy induce a low dimensional manifold of attainable states embedded in the high-dimensional nominal state space trained using an actor-critic algorithm. We prove that, under certain conditions, the dimensionality of this manifold is of the order of the dimensionality of the action space. This is the first result of its kind, linking the geometry of the state space to the dimensionality of the action space. We empirically corroborate this upper bound for four MuJoCo environments and also demonstrate the results in a toy environment with varying dimensionality. We also show the applicability of this theoretical result by introducing a local manifold learning layer to the policy and value function networks to improve the performance in control environments with very high degrees of freedom by changing one layer of the neural network to learn sparse representations.

Figures

Figures reproduced from arXiv: 2507.20853 by the authors.

Figure 1
Figure 1. The surface of an open cylinder of unit radius, denoted by S 2 , in R 3 is a 2D man￾ifold embedded in a 3D space. More formally, S 2 = {(x, y, z)|x 2 +y 2 = 1, z ∈ (−h, h)} where the cylinder’s height is 2h. One can smoothly pa￾rameterise S 2 as ψ(θ, b) = (sin θ, cos θ, b). The coordinate chart is ϕ(x, y, z) = (sin−1 x, z). Curves and tangent spaces in manifolds natu￾rally lead to vector fields. In the same way that… view at source ↗
Figure 2
Figure 2. We observe that the difference between the returns approaches zero as we increase the width. We empirically observe the impact of our choice of lin￾earised policies as a theoretical model for two-layer NNs by measuring the impact on the returns of this choice. We calculate the difference in returns for DDPG using canonical NNs and linearised NNs as parameterisations for its policy network, while only training the we… view at source ↗
Figure 3
Figure 3. Estimated dimensionality of the attainable states, in blue, is far below [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: The intrinsic dimensionality estimate of attainable states linear fully reachable system under linearised pol￾icy on y-axis. A deterministic system is fully reachable if given any start state, s0 ∈ R ds , the system can be driven to any goal state in R ds . To contrast…
Figure 5
Figure 5. Figure 5: We observe that SAC with fully connected network fails to learn in high-dimensional Dog [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]
Figure 6
Figure 6. Figure 6: Comparison of single hidden layer with GeLU activation (blue) and multiple hidden layer [PITH_FULL_IMAGE:figures/full_fig_p035_6.png]
Figure 7
Figure 7. Figure 7: The canonical policy (in red) tracks the returns for linearised policy (in blue) at higher [PITH_FULL_IMAGE:figures/full_fig_p036_7.png]
Figure 8
Figure 8. Figure 8: Ablation over the αQ parameter. (a) Ant (b) Dog Stand [PITH_FULL_IMAGE:figures/full_fig_p037_8.png]
Figure 9
Figure 9. Figure 9: We show the steps per second for SAC (blue) and sparse SAC (red) as training progresses. [PITH_FULL_IMAGE:figures/full_fig_p037_9.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

142 extracted references · 51 canonical work pages

  1. [1]

    The neural tangent kernel in high dimensions: Triple descent and a multi-scale theory of generalization

    Ben Adlam and Jeffrey Pennington. The neural tangent kernel in high dimensions: Triple descent and a multi-scale theory of generalization. In International Conference on Machine Learning, pp.\ 74--84. PMLR, 2020

    2020

  2. [2]

    Agrachev and Yu

    Andrei A. Agrachev and Yu. L. Sachkov. Control theory from the geometric viewpoint. 2004

    2004

  3. [3]

    Akametalu, Shahab Kaynama, Jaime Fern \'a ndez Fisac, Melanie Nicole Zeilinger, Jeremy H

    Anayo K. Akametalu, Shahab Kaynama, Jaime Fern \'a ndez Fisac, Melanie Nicole Zeilinger, Jeremy H. Gillula, and Claire J. Tomlin. Reachability-based safe learning with gaussian processes. 53rd IEEE Conference on Decision and Control, pp.\ 1424--1431, 2014

    2014

  4. [4]

    Learning and generalization in overparameterized neural networks, going beyond two layers

    Zeyuan Allen-Zhu, Yuanzhi Li, and Yingyu Liang. Learning and generalization in overparameterized neural networks, going beyond two layers. Advances in neural information processing systems, 32, 2019 a

    2019

  5. [5]

    A convergence theory for deep learning via over-parameterization

    Zeyuan Allen-Zhu, Yuanzhi Li, and Zhao Song. A convergence theory for deep learning via over-parameterization. In International conference on machine learning, pp.\ 242--252. PMLR, 2019 b

    2019

  6. [6]

    Analytic Manifold Learning: Unifying and Evaluating Representations for Continuous Control

    Rika Antonova, Maksim Maydanskiy, Danica Kragic, Sam Devlin, and Katja Hofmann. Analytic manifold learning: Unifying and evaluating representations for continuous control. ArXiv, abs/2006.08718, 2020

    work page Pith review arXiv 2006

  7. [7]

    Robust locally-linear controllable embedding

    Ershad Banijamali, Rui Shu, Mohammad Ghavamzadeh, Hung Hai Bui, and Ali Ghodsi. Robust locally-linear controllable embedding. In AISTATS, 2018

    2018

  8. [8]

    Ronen Basri and David W. Jacobs. Efficient representation of low-dimensional manifolds using deep networks. ArXiv, abs/1602.04723, 2017

    work page Pith review arXiv 2017

Show all 142 references
  1. [9]

    High-dimensional limit theorems for sgd: Effective dynamics and critical scaling

    Gerard Ben Arous, Reza Gheissari, and Aukosh Jagannath. High-dimensional limit theorems for sgd: Effective dynamics and critical scaling. Advances in Neural Information Processing Systems, 35: 0 25349--25362, 2022

    2022

  2. [10]

    Dynamic programming and optimal control: Volume I, volume 4

    Dimitri Bertsekas. Dynamic programming and optimal control: Volume I, volume 4. Athena scientific, 2012

    2012

  3. [11]

    Model predictive control and reinforcement learning: A unified framework based on dynamic programming

    Dimitri P Bertsekas. Model predictive control and reinforcement learning: A unified framework based on dynamic programming. arXiv preprint arXiv:2406.00592, 2024

    2024 arXiv

  4. [12]

    An introduction to aspects of geometric control theory

    Anthony M Bloch and Anthony M Bloch. An introduction to aspects of geometric control theory. Nonholonomic mechanics and control, pp.\ 199--233, 2015

    2015

  5. [13]

    An introduction to differentiable manifolds and Riemannian geometry

    William M Boothby. An introduction to differentiable manifolds and Riemannian geometry. Academic press, 1986

    1986

  6. [14]

    Wilkinson

    Michael Bowling, Ali Ghodsi, and Dana F. Wilkinson. Action respecting embedding. Proceedings of the 22nd international conference on Machine learning, 2005

    2005

  7. [15]

    Brockett

    Roger W. Brockett. Lie theory and control systems defined on spheres. Siam Journal on Applied Mathematics, 25: 0 213--225, 1973

    1973

  8. [16]

    Openai gym

    Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, and Wojciech Zaremba. Openai gym. ArXiv, abs/1606.01540, 2016

    2016 arXiv

  9. [17]

    Bronstein, Joan Bruna, Taco Cohen, and Petar Velivckovi'c

    Michael M. Bronstein, Joan Bruna, Taco Cohen, and Petar Velivckovi'c. Geometric deep learning: Grids, groups, graphs, geodesics, and gauges. ArXiv, abs/2104.13478, 2021

    2021 arXiv

  10. [18]

    Deep networks and the multiple manifold problem

    Sam Buchanan, Dar Gilboa, and John Wright. Deep networks and the multiple manifold problem. ArXiv, abs/2008.11245, 2021

    2008 arXiv

  11. [19]

    Geometric control of mechanical systems: modeling, analysis, and design for simple mechanical control systems, volume 49

    Francesco Bullo and Andrew D Lewis. Geometric control of mechanical systems: modeling, analysis, and design for simple mechanical control systems, volume 49. Springer, 2019

    2019

  12. [20]

    Manifold embeddings for model-based reinforcement learning under partial observability

    Keith Bush and Joelle Pineau. Manifold embeddings for model-based reinforcement learning under partial observability. In NIPS, 2009

    2009

  13. [21]

    Cai, Zhuoran Yang, Jason Lee, and Zhaoran Wang

    Q. Cai, Zhuoran Yang, Jason Lee, and Zhaoran Wang. Neural temporal-difference and q-learning provably converge to global optima. arXiv: Learning, 2019 a

    2019

  14. [22]

    Lee, and Zhaoran Wang

    Qi Cai, Zhuoran Yang, Jason D. Lee, and Zhaoran Wang. Neural temporal-difference learning converges to global optima. Advances in Neural Information Processing Systems, 32, 2019 b . ISSN 1049-5258. Publisher Copyright: 2019 Neural information processing systems foundation. All...

    2019

  15. [23]

    Carlsson, T

    G. Carlsson, T. Ishkhanov, V. D. Silva, and A. Zomorodian. On the local behavior of spaces of natural images. International Journal of Computer Vision, 76: 0 1--12, 2007

    2007

  16. [24]

    Using bisimulation for policy transfer in mdps

    Pablo Samuel Castro and Doina Precup. Using bisimulation for policy transfer in mdps. In AAAI Conference on Artificial Intelligence, 2010

    2010

  17. [25]

    Redunet: A white-box deep network from the principle of maximizing rate reduction

    Kwan Ho Ryan Chan, Yaodong Yu, Chong You, Haozhi Qi, John Wright, and Yi Ma. Redunet: A white-box deep network from the principle of maximizing rate reduction. Journal of machine learning research, 23 0 (114): 0 1--103, 2022

    2022

  18. [26]

    Efficient approximation of deep relu networks for functions on low dimensional manifolds

    Minshuo Chen, Haoming Jiang, Wenjing Liao, and Tuo Zhao. Efficient approximation of deep relu networks for functions on low dimensional manifolds. ArXiv, abs/1908.01842, 2019

    1908 arXiv

  19. [27]

    Analysis and design of nonlinear control systems

    Daizhan Cheng, Xiaoming Hu, and Tielong Shen. Analysis and design of nonlinear control systems. In Analysis and Design of Nonlinear Control Systems, 2011

    2011

  20. [28]

    Stochastic gradient and langevin processes

    Xiang Cheng, Dong Yin, Peter Bartlett, and Michael Jordan. Stochastic gradient and langevin processes. In International Conference on Machine Learning, pp.\ 1810--1819. PMLR, 2020

    2020

  21. [29]

    On the global convergence of gradient descent for over-parameterized models using optimal transport

    Lenaic Chizat and Francis Bach. On the global convergence of gradient descent for over-parameterized models using optimal transport. Advances in neural information processing systems, 31, 2018

    2018

  22. [30]

    Relu nets adapt to intrinsic dimensionality beyond the target domain

    Alexander Cloninger and Timo Klock. Relu nets adapt to intrinsic dimensionality beyond the target domain. ArXiv, abs/2008.02545, 2020

    2008 arXiv

  23. [31]

    G. Cybenko. Approximation by superpositions of a sigmoidal function. Mathematics of Control, Signals and Systems, 2: 0 303--314, 1989

    1989

  24. [32]

    Pilco: A model-based and data-efficient approach to policy search

    Marc Peter Deisenroth and Carl Edward Rasmussen. Pilco: A model-based and data-efficient approach to policy search. In ICML, 2011

    2011

  25. [33]

    Reinforcement learning in continuous time and space

    Kenji Doya. Reinforcement learning in continuous time and space. Neural Computation, 12: 0 219--245, 2000 a

    2000

  26. [34]

    Reinforcement learning in continuous time and space

    Kenji Doya. Reinforcement learning in continuous time and space. Neural computation, 12 0 (1): 0 219--245, 2000 b

    2000

  27. [35]

    Lee, Haochuan Li, Liwei Wang, and Xiyu Zhai

    Simon Shaolei Du, J. Lee, Haochuan Li, Liwei Wang, and Xiyu Zhai. Gradient descent finds global minima of deep neural networks. ArXiv, abs/1811.03804, 2018

    2018 arXiv

  28. [36]

    Yan Duan, Xi Chen, Rein Houthooft, John Schulman, and P. Abbeel. Benchmarking deep reinforcement learning for continuous control. In ICML, 2016

    2016

  29. [37]

    Estimating the intrinsic dimension of datasets by a minimal neighborhood information

    Elena Facco, Maria d'Errico, Alex Rodriguez, and Alessandro Laio. Estimating the intrinsic dimension of datasets by a minimal neighborhood information. Scientific Reports, 7, 2017

    2017

  30. [38]

    Fefferman, S

    C. Fefferman, S. Mitter, and Hariharan Narayanan. Testing the manifold hypothesis. arXiv: Statistics Theory, 2013

    2013

  31. [39]

    Panangaden, and Doina Precup

    Norm Ferns, P. Panangaden, and Doina Precup. Metrics for finite markov decision processes. In AAAI Conference on Artificial Intelligence, 2004

    2004

  32. [40]

    Deep learning versus kernel learning: an empirical study of loss landscape geometry and the time evolution of the neural tangent kernel

    Stanislav Fort, Gintare Karolina Dziugaite, Mansheej Paul, Sepideh Kharaghani, Daniel M Roy, and Surya Ganguli. Deep learning versus kernel learning: an empirical study of loss landscape geometry and the time evolution of the neural tangent kernel. Advances in Neural Informati...

    2020

  33. [41]

    Ruiqi Gao, Tianle Cai, Haochuan Li, Cho-Jui Hsieh, Liwei Wang, and J. Lee. Convergence of adversarial training in overparametrized neural networks. In Neural Information Processing Systems, 2019

    2019

  34. [42]

    Bellemare

    Carles Gelada, Saurabh Kumar, Jacob Buckman, Ofir Nachum, and Marc G. Bellemare. Deepmdp: Learning continuous latent space models for representation learning. ArXiv, abs/1906.02736, 2019

    1906 arXiv

  35. [43]

    Dean, and Matthew Greig

    Robert Givan, Thomas L. Dean, and Matthew Greig. Equivalence notions and model minimization in markov decision processes. Artif. Intell., 147: 0 163--223, 2003

    2003

  36. [44]

    Modelling the influence of data structure on learning in neural networks

    Sebastian Goldt, Marc M \'e zard, Florent Krzakala, and Lenka Zdeborov \'a . Modelling the influence of data structure on learning in neural networks. ArXiv, abs/1909.11500, 2020

    1909 arXiv

  37. [45]

    Botvinick, H

    Anirudh Goyal, Riashat Islam, Daniel Strouse, Zafarali Ahmed, Matthew M. Botvinick, H. Larochelle, Sergey Levine, and Yoshua Bengio. Infobot: Transfer and exploration via the information bottleneck. ArXiv, abs/1901.10902, 2019 a

    1901 arXiv

  38. [46]

    Reinforcement learning with competitive ensembles of information-constrained primitives

    Anirudh Goyal, Shagun Sodhani, Jonathan Binas, Xue Bin Peng, Sergey Levine, and Yoshua Bengio. Reinforcement learning with competitive ensembles of information-constrained primitives. ArXiv, abs/1906.10667, 2019 b

    1906 arXiv

  39. [47]

    Botvinick, and Sergey Levine

    Anirudh Goyal, Yoshua Bengio, Matthew M. Botvinick, and Sergey Levine. The variational bandwidth bottleneck: Stochastic evaluation on an information budget. ArXiv, abs/2004.11935, 2020

    2004 arXiv

  40. [48]

    Differential Topology

    Victor Guillemin and Alan Pollack. Differential Topology. Prentice-Hall, 1974

    1974

  41. [49]

    Abbeel, and Sergey Levine

    Tuomas Haarnoja, Haoran Tang, P. Abbeel, and Sergey Levine. Reinforcement learning with deep energy-based policies. In ICML, 2017

    2017

  42. [50]

    Abbeel, and Sergey Levine

    Tuomas Haarnoja, Aurick Zhou, P. Abbeel, and Sergey Levine. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. In ICML, 2018

    2018

  43. [51]

    Finite depth and width corrections to the neural tangent kernel

    Boris Hanin and Mihai Nica. Finite depth and width corrections to the neural tangent kernel. arXiv preprint arXiv:1909.05989, 2019

    1909 arXiv

  44. [52]

    Gaussian error linear units (gelus)

    Dan Hendrycks and Kevin Gimpel. Gaussian error linear units (gelus). arXiv: Learning, 2016

    2016

  45. [53]

    Cleanrl: High-quality single-file implementations of deep reinforcement learning algorithms

    Shengyi Huang, Rousslan Fernand Julien Dossa, Chang Ye, Jeff Braga, Dipam Chakraborty, Kinal Mehta, and Jo \ A G o GM Ara \ A s jo. Cleanrl: High-quality single-file implementations of deep reinforcement learning algorithms. Journal of Machine Learning Research, 23 0 (274): 0 ...

    2022

  46. [54]

    Safe reinforcement learning on autonomous vehicles

    David Isele, Alireza Nakhaei, and Kikuo Fujimura. Safe reinforcement learning on autonomous vehicles. 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp.\ 1--6, 2018

    2018

  47. [55]

    Nonlinear control systems: an introduction

    Alberto Isidori. Nonlinear control systems: an introduction. Springer, 1985

    1985

  48. [56]

    Mofijul Islam, Samin Yeasar Arnob, Tariq Iqbal, Xin Li, Anirudh Goyal, Nicolas Manfred Otto Heess, and Alex Lamb

    Riashat Islam, Hongyu Zang, Manan Tomar, Aniket Didolkar, Md. Mofijul Islam, Samin Yeasar Arnob, Tariq Iqbal, Xin Li, Anirudh Goyal, Nicolas Manfred Otto Heess, and Alex Lamb. Representation learning in deep rl via discrete information bottleneck. In International Conference o...

    2022

  49. [57]

    Gabriel, and C

    Arthur Jacot, F. Gabriel, and C. Hongler. Neural tangent kernel: Convergence and generalization in neural networks. In NeurIPS, 2018

    2018

  50. [58]

    Odest Chadwicke Jenkins and Maja J. Mataric. A spatio-temporal extension to isomap nonlinear dimension reduction. In ICML '04, 2004

    2004

  51. [59]

    Sample complexity of nonparametric off-policy evaluation on low-dimensional manifolds using deep networks

    Xiang Ji, Minshuo Chen, Mengdi Wang, and Tuo Zhao. Sample complexity of nonparametric off-policy evaluation on low-dimensional manifolds using deep networks. ArXiv, abs/2206.02887, 2022

    2022 arXiv

  52. [60]

    Policy gradient and actor-critic learning in continuous time and space: Theory and algorithms

    Yanwei Jia and Xun Yu Zhou. Policy gradient and actor-critic learning in continuous time and space: Theory and algorithms. Journal of Machine Learning Research, 23 0 (275): 0 1--50, 2022 a

    2022

  53. [61]

    Policy gradient and actor-critic learning in continuous time and space: Theory and algorithms

    Yanwei Jia and Xun Yu Zhou. Policy gradient and actor-critic learning in continuous time and space: Theory and algorithms. Journal of Machine Learning Research, 23 0 (275): 0 1--50, 2022 b . URL http://jmlr.org/papers/v23/21-1387.html

    2022

  54. [62]

    q-learning in continuous time

    Yanwei Jia and Xun Yu Zhou. q-learning in continuous time. Journal of Machine Learning Research, 24 0 (161): 0 1--61, 2023. URL http://jmlr.org/papers/v24/22-0755.html

    2023

  55. [63]

    Machado, and George Dimitri Konidaris

    Yuu Jinnai, Jee Won Park, Marlos C. Machado, and George Dimitri Konidaris. Exploration in reinforcement learning with deep covering options. In ICLR, 2020

    2020

  56. [64]

    Geometric control theory

    Velimir Jurdjevic. Geometric control theory. Cambridge university press, 1997

    1997

  57. [65]

    Sham M. Kakade. A natural policy gradient. In NIPS, 2001

    2001

  58. [66]

    On the general theory of control systems

    Rudolf E Kalman. On the general theory of control systems. In Proceedings First International Conference on Automatic Control, Moscow, USSR, pp.\ 481--492, 1960

    1960

  59. [67]

    Brownian motion and stochastic calculus, volume 113

    Ioannis Karatzas and Steven Shreve. Brownian motion and stochastic calculus, volume 113. springer, 2014

    2014

  60. [68]

    Champion-level drone racing using deep reinforcement learning

    Elia Kaufmann, Leonard Bauersfeld, Antonio Loquercio, Matthias M \"u ller, Vladlen Koltun, and Davide Scaramuzza. Champion-level drone racing using deep reinforcement learning. Nature, 620 0 (7976): 0 982--987, 2023

    2023

  61. [69]

    Actor-critic algorithms

    Vijay Konda and John Tsitsiklis. Actor-critic algorithms. Advances in neural information processing systems, 12, 1999

    1999

  62. [70]

    Bellemare, and Pablo Samuel Castro

    Charline Le Lan, Marc G. Bellemare, and Pablo Samuel Castro. Metrics and continuity in reinforcement learning. In AAAI Conference on Artificial Intelligence, 2021

    2021

  63. [71]

    Deep neural networks as gaussian processes

    Jaehoon Lee, Yasaman Bahri, Roman Novak, Samuel S Schoenholz, Jeffrey Pennington, and Jascha Sohl-Dickstein. Deep neural networks as gaussian processes. arXiv preprint arXiv:1711.00165, 2017

    2017 arXiv

  64. [72]

    Wide neural networks of any depth evolve as linear models under gradient descent

    Jaehoon Lee, Lechao Xiao, Samuel Schoenholz, Yasaman Bahri, Roman Novak, Jascha Sohl-Dickstein, and Jeffrey Pennington. Wide neural networks of any depth evolve as linear models under gradient descent. Advances in neural information processing systems, 32, 2019

    2019

  65. [73]

    Sergey Levine, Chelsea Finn, Trevor Darrell, and P. Abbeel. End-to-end training of deep visuomotor policies. ArXiv, abs/1504.00702, 2016

    2016 arXiv

  66. [74]

    Convergence analysis of two-layer neural networks with relu activation

    Yuanzhi Li and Yang Yuan. Convergence analysis of two-layer neural networks with relu activation. In NIPS, 2017

    2017

  67. [76]

    Continuous control with deep reinforcement learning

    TP Lillicrap. Continuous control with deep reinforcement learning. arXiv preprint arXiv:1509.02971, 2015

    2015 arXiv

  68. [77]

    Robot reinforcement learning on the constraint manifold

    Puze Liu, Davide Tateo, Haitham Bou-Ammar, and Jan Peters. Robot reinforcement learning on the constraint manifold. In CoRL, 2021

    2021

  69. [78]

    Winfried Lohmiller and Jean-Jacques E. Slotine. On contraction analysis for non-linear systems. Autom., 34: 0 683--696, 1998

    1998

  70. [79]

    Segmentation of multivariate mixed data via lossy data coding and compression

    Yi Ma, Harm Derksen, Wei Hong, and John Wright. Segmentation of multivariate mixed data via lossy data coding and compression. IEEE transactions on pattern analysis and machine intelligence, 29 0 (9): 0 1546--1562, 2007

    2007

  71. [80]

    Machado, Marc G

    Marlos C. Machado, Marc G. Bellemare, and Michael Bowling. A laplacian framework for option discovery in reinforcement learning. ArXiv, abs/1703.00956, 2017

    2017 arXiv

  72. [81]

    Machado, Clemens Rosenbaum, Xiaoxiao Guo, Miao Liu, Gerald Tesauro, and Murray Campbell

    Marlos C. Machado, Clemens Rosenbaum, Xiaoxiao Guo, Miao Liu, Gerald Tesauro, and Murray Campbell. Eigenoption discovery through the deep successor representation. ArXiv, abs/1710.11089, 2018

    2018 arXiv

  73. [82]

    Proto-value functions: developmental reinforcement learning

    Sridhar Mahadevan. Proto-value functions: developmental reinforcement learning. Proceedings of the 22nd international conference on Machine learning, 2005

    2005

  74. [83]

    Proto-value functions: A laplacian framework for learning representation and control in markov decision processes

    Sridhar Mahadevan and Mauro Maggioni. Proto-value functions: A laplacian framework for learning representation and control in markov decision processes. J. Mach. Learn. Res., 8: 0 2169--2231, 2007

    2007

  75. [84]

    Approximate gradient methods in policy-space optimization of markov reward processes

    Peter Marbach and John N Tsitsiklis. Approximate gradient methods in policy-space optimization of markov reward processes. Discrete Event Dynamic Systems, 13: 0 111--148, 2003

    2003

  76. [85]

    A mean field view of the landscape of two-layer neural networks

    Song Mei, Andrea Montanari, and Phan-Minh Nguyen. A mean field view of the landscape of two-layer neural networks. Proceedings of the National Academy of Sciences of the United States of America, 115: 0 E7665 -- E7671, 2018 a

    2018

  77. [86]

    A mean field view of the landscape of two-layer neural networks

    Song Mei, Andrea Montanari, and Phan-Minh Nguyen. A mean field view of the landscape of two-layer neural networks. Proceedings of the National Academy of Sciences, 115 0 (33): 0 E7665--E7671, 2018 b

    2018

  78. [87]

    Rusu, Joel Veness, Marc G

    Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A. Rusu, Joel Veness, Marc G. Bellemare, Alex Graves, Martin A. Riedmiller, Andreas Fidjeland, Georg Ostrovski, Stig Petersen, Charles Beattie, Amir Sadik, Ioannis Antonoglou, Helen King, Dharshan Kumaran, Daan Wierstra, ...

    2015

  79. [88]

    A case study in approximate linearization: The acrobat example

    Richard M Murray and John Edmond Hauser. A case study in approximate linearization: The acrobat example. Electronics Research Laboratory, College of Engineering, University of …, 1991

    1991

  80. [89]

    Nair and Geoffrey E

    V. Nair and Geoffrey E. Hinton. Rectified linear units improve restricted boltzmann machines. In ICML, 2010

    2010

  81. [90]

    Non-linear dynamical control systems

    Henk Nijmeijer and Arjan van der Schaft. Non-linear dynamical control systems. 1990

    1990

  82. [91]

    Geometric compression of invariant manifolds in neural networks

    Jonas Paccolat, Leonardo Petrini, Mario Geiger, Kevin Tyloo, and Matthieu Wyart. Geometric compression of invariant manifolds in neural networks. Journal of Statistical Mechanics: Theory and Experiment, 2021, 2020

    2021

  83. [92]

    Masked completion via structured diffusion with white-box transformers

    Druv Pai, Sam Buchanan, Ziyang Wu, Yaodong Yu, and Yi Ma. Masked completion via structured diffusion with white-box transformers. In The Twelfth International Conference on Learning Representations, 2024

    2024

  84. [93]

    Bronstein, and Ron Kimmel

    Gautam Pai, Ronen Talmon, Alexander M. Bronstein, and Ron Kimmel. Dimal: Deep isometric manifold learning using sparse geodesic sampling. 2019 IEEE Winter Conference on Applications of Computer Vision (WACV), pp.\ 819--828, 2019

    2019

  85. [94]

    A contraction theory approach to stochastic incremental stability

    Quang-Cuong Pham, Nicolas Tabareau, and Jean-Jacques Slotine. A contraction theory approach to stochastic incremental stability. IEEE Transactions on Automatic Control, 54 0 (4): 0 816--820, 2009. doi:10.1109/TAC.2008.2009619

    2009

  86. [95]

    Balaraman Ravindran and Andrew G. Barto. Symmetries and model minimization in markov decision processes. 2001

    2001

  87. [96]

    Balaraman Ravindran and Andrew G. Barto. Model minimization in hierarchical reinforcement learning. In Symposium on Abstraction, Reformulation and Approximation, 2002

    2002

  88. [97]

    Balaraman Ravindran and Andrew G. Barto. Smdp homomorphisms: An algebraic approach to abstraction in semi-markov decision processes. In International Joint Conference on Artificial Intelligence, 2003

    2003

  89. [98]

    Controllability of dynamical systems with constraints

    Jerzy Respondek. Controllability of dynamical systems with constraints. Systems & Control Letters, 54 0 (4): 0 293--314, 2005

    2005

  90. [99]

    Robbin, Uw Madison, and Dietmar A

    Joel W. Robbin, Uw Madison, and Dietmar A. Salamon. INTRODUCTION TO DIFFERENTIAL GEOMETRY. Preprint, 2011

    2011

  91. [100]

    Deep relu network approximation of functions on a manifold

    Johannes Schmidt-Hieber. Deep relu network approximation of functions on a manifold. ArXiv, abs/1908.00695, 2019

    1908 arXiv

  92. [101]

    Levine, P

    John Schulman, S. Levine, P. Abbeel, Michael I. Jordan, and P. Moritz. Trust region policy optimization. ArXiv, abs/1502.05477, 2015

    2015 arXiv

  93. [102]

    Proximal policy optimization algorithms

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. ArXiv, abs/1707.06347, 2017

    2017 arXiv

  94. [103]

    Uri Shaham, Alexander Cloninger, and Ronald R. Coifman. Provable approximation properties for deep neural networks. ArXiv, abs/1509.07385, 2015

    2015 arXiv

  95. [104]

    Reachability-based trajectory safeguard (rts): A safe and fast reinforcement learning safety layer for continuous control

    Yifei Shao, Chao Chen, Shreyas Kousik, and Ram Vasudevan. Reachability-based trajectory safeguard (rts): A safe and fast reinforcement learning safety layer for continuous control. IEEE Robotics and Automation Letters, 6: 0 3663--3670, 2020

    2020

  96. [105]

    Maddison, Arthur Guez, L

    David Silver, Aja Huang, Chris J. Maddison, Arthur Guez, L. Sifre, George van den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Vedavyas Panneershelvam, Marc Lanctot, Sander Dieleman, Dominik Grewe, John Nham, Nal Kalchbrenner, Ilya Sutskever, Timothy P. Lillicrap, Made...

    2016

  97. [106]

    Effective reinforcement learning for mobile robots

    William D Smart and L Pack Kaelbling. Effective reinforcement learning for mobile robots. In Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No. 02CH37292), volume 4, pp.\ 3404--3410. IEEE, 2002

    2002

  98. [107]

    Necessary and sufficient condition for global controllability of planar affine nonlinear systems

    Yimin Sun. Necessary and sufficient condition for global controllability of planar affine nonlinear systems. IEEE transactions on automatic control, 52 0 (8): 0 1454--1460, 2007

    2007

  99. [108]

    Controllability and reachability criteria for switched linear systems

    Zhendong Sun, Shuzhi Sam Ge, and Tong Heng Lee. Controllability and reachability criteria for switched linear systems. Automatica, 38 0 (5): 0 775--786, 2002

    2002

  100. [109]

    Orbits of families of vector fields and integrability of distributions

    H \'e ctor J Sussmann. Orbits of families of vector fields and integrability of distributions. Transactions of the American Mathematical Society, 180: 0 171--188, 1973

    1973

  101. [110]

    A general theorem on local controllability

    Hector J Sussmann. A general theorem on local controllability. SIAM Journal on Control and Optimization, 25 0 (1): 0 158--194, 1987

    1987

  102. [111]

    Introduction to reinforcement learning, volume 135

    Richard S Sutton and Andrew G Barto. Introduction to reinforcement learning, volume 135. MIT press Cambridge, 1998

    1998

  103. [112]

    Sutton, David McAllester, Satinder Singh, and Yishay Mansour

    Richard S. Sutton, David McAllester, Satinder Singh, and Yishay Mansour. Policy gradient methods for reinforcement learning with function approximation. In Proceedings of the 12th International Conference on Neural Information Processing Systems, NIPS'99, pp.\ 1057--1063, 1999

    1999

  104. [113]

    Underactuated Robotics

    Russ Tedrake. Underactuated Robotics. 2023. URL https://underactuated.csail.mit.edu

    2023

  105. [114]

    Tenenbaum

    Joshua B. Tenenbaum. Mapping a manifold of perceptual observations. In NIPS, 1997

    1997

  106. [115]

    Effects of data geometry in early deep learning

    Saket Tiwari and George Konidaris. Effects of data geometry in early deep learning. Advances in Neural Information Processing Systems, 35: 0 30099--30113, 2022

    2022

  107. [116]

    Mujoco: A physics engine for model-based control

    Emanuel Todorov, Tom Erez, and Yuval Tassa. Mujoco: A physics engine for model-based control. 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp.\ 5026--5033, 2012

    2012

  108. [117]

    Information-theoretic limits of control

    Touchette and Lloyd. Information-theoretic limits of control. Physical review letters, 84 6: 0 1156--9, 1999

    1999

  109. [118]

    Information-theoretic approach to the study of control systems

    Hugo Touchette and Seth Lloyd. Information-theoretic approach to the study of control systems. Physica A-statistical Mechanics and Its Applications, 331: 0 140--172, 2001

    2001

  110. [119]

    dm\_control: Software and tasks for continuous control

    Saran Tunyasuvunakool, Alistair Muldal, Yotam Doron, Siqi Liu, Steven Bohez, Josh Merel, Tom Erez, Timothy Lillicrap, Nicolas Heess, and Yuval Tassa. dm\_control: Software and tasks for continuous control. Software Impacts, 6: 0 100022, 2020

    2020

  111. [120]

    Uhlenbeck and Leonard Salomon Ornstein

    George E. Uhlenbeck and Leonard Salomon Ornstein. On the theory of the brownian motion. Physical Review, 36: 0 823--841, 1930

    1930

  112. [121]

    Worrall, Herke van Hoof, Frans A

    Elise van der Pol, Daniel E. Worrall, Herke van Hoof, Frans A. Oliehoek, and Max Welling. Mdp homomorphic networks: Group symmetries in reinforcement learning. ArXiv, abs/2006.16908, 2020

    2006 arXiv

  113. [122]

    High-dimensional probability: An introduction with applications in data science, volume 47

    Roman Vershynin. High-dimensional probability: An introduction with applications in data science, volume 47. Cambridge university press, 2018

    2018

  114. [123]

    Continuous-time mean--variance portfolio selection: A reinforcement learning framework

    Haoran Wang and Xun Yu Zhou. Continuous-time mean--variance portfolio selection: A reinforcement learning framework. Mathematical Finance, 30 0 (4): 0 1273--1308, 2020

    2020

  115. [124]

    Reinforcement learning in continuous time and space: A stochastic control approach

    Haoran Wang, Thaleia Zariphopoulou, and Xun Yu Zhou. Reinforcement learning in continuous time and space: A stochastic control approach. J. Mach. Learn. Res., 21: 0 198:1--198:34, 2020

    2020

  116. [125]

    Neural policy gradient methods: Global optimality and rates of convergence

    Lingxiao Wang, Qi Cai, Zhuoran Yang, and Zhaoran Wang. Neural policy gradient methods: Global optimality and rates of convergence. ArXiv, abs/1909.01150, 2019

    1909 arXiv

  117. [126]

    When and why pinns fail to train: A neural tangent kernel perspective

    Sifan Wang, Xinling Yu, and Paris Perdikaris. When and why pinns fail to train: A neural tangent kernel perspective. Journal of Computational Physics, 449: 0 110768, 2022

    2022

  118. [127]

    Regularization matters: Generalization and optimization of neural nets vs their induced kernel

    Colin Wei, Jason D Lee, Qiang Liu, and Tengyu Ma. Regularization matters: Generalization and optimization of neural nets vs their induced kernel. Advances in Neural Information Processing Systems, 32, 2019

    2019

  119. [128]

    Introduction to applied nonlinear dynamical systems and chaos

    Stephen Wiggins. Introduction to applied nonlinear dynamical systems and chaos. In SPRINGER, 1989

    1989

  120. [129]

    Simple statistical gradient-following algorithms for connectionist reinforcement learning

    Ronald J Williams. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine learning, 8: 0 229--256, 1992

    1992

  121. [130]

    High-Dimensional Data Analysis with Low-Dimensional Models: Principles, Computation, and Applications

    John Wright and Yi Ma. High-Dimensional Data Analysis with Low-Dimensional Models: Principles, Computation, and Applications. Cambridge University Press, 2022

    2022

  122. [131]

    Tucker, and Ofir Nachum

    Yifan Wu, G. Tucker, and Ofir Nachum. The laplacian in rl: Learning representations with efficient approximations. ArXiv, abs/1810.04586, 2019

    2019 arXiv

  123. [132]

    Wurman, Samuel Barrett, Kenta Kawamoto, James MacGlashan, Kaushik Subramanian, Thomas J

    Peter R. Wurman, Samuel Barrett, Kenta Kawamoto, James MacGlashan, Kaushik Subramanian, Thomas J. Walsh, Roberto Capobianco, Alisa Devlic, Franziska Eckert, Florian Fuchs, Leilani Gilpin, Piyush Khandelwal, Varun Kompella, HaoChih Lin, Patrick MacAlpine, Declan Oller, Takuma S...

    2022

  124. [133]

    Deterministic policy gradient: Convergence analysis

    Huaqing Xiong, Tengyu Xu, Lin Zhao, Yingbin Liang, and Wei Zhang. Deterministic policy gradient: Convergence analysis. In Uncertainty in Artificial Intelligence, pp.\ 2159--2169. PMLR, 2022

    2022

  125. [134]

    Tensor programs iv: Feature learning in infinite-width neural networks

    Greg Yang and Edward J Hu. Tensor programs iv: Feature learning in infinite-width neural networks. In International Conference on Machine Learning, pp.\ 11727--11737. PMLR, 2021

    2021

  126. [135]

    Tensor programs v: Tuning large neural networks via zero-shot hyperparameter transfer

    Greg Yang, Edward J Hu, Igor Babuschkin, Szymon Sidor, Xiaodong Liu, David Farhi, Nick Ryder, Jakub Pachocki, Weizhu Chen, and Jianfeng Gao. Tensor programs v: Tuning large neural networks via zero-shot hyperparameter transfer. arXiv preprint arXiv:2203.03466, 2022

    2022 arXiv

  127. [136]

    White-box transformers via sparse rate reduction

    Yaodong Yu, Sam Buchanan, Druv Pai, Tianzhe Chu, Ziyang Wu, Shengbang Tong, Benjamin David Haeffele, and Yi Ma. White-box transformers via sparse rate reduction. In Thirty-seventh Conference on Neural Information Processing Systems, 2023 a

    2023

  128. [137]

    Emergence of segmentation with minimalistic white-box transformers

    Yaodong Yu, Tianzhe Chu, Shengbang Tong, Ziyang Wu, Druv Pai, Sam Buchanan, and Yi Ma. Emergence of segmentation with minimalistic white-box transformers. In Conference on Parsimony and Learning (Proceedings Track), 2023 b

    2023

  129. [138]

    Learning invariant representations for reinforcement learning without reconstruction

    Amy Zhang, Rowan Thomas McAllister, Roberto Calandra, Yarin Gal, and Sergey Levine. Learning invariant representations for reinforcement learning without reconstruction. ArXiv, abs/2006.10742, 2020

    2006 arXiv

  130. [139]

    The ingredients of real-world robotic reinforcement learning

    Henry Zhu, Justin Yu, Abhishek Gupta, Dhruv Shah, Kristian Hartikainen, Avi Singh, Vikash Kumar, and Sergey Levine. The ingredients of real-world robotic reinforcement learning. ArXiv, abs/2004.12570, 2020

    2004 arXiv

  131. [140]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

  132. [141]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...

  133. [142]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...

  134. [143]

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.