Pith. sign in

REVIEW 4 major objections 8 minor 68 references

Hopfield Neural Network Flow: A Geometric Viewpoint

T0 review · 4 major / 8 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper proves the deterministic Hopfield flow is natural gradient descent in a metric fixed by the activation, and the stochastic diffusion machine is a Wasserstein gradient flow of a free energy.

desk verdict A sound geometric dictionary for Hopfield networks: deterministic flow is natural gradient descent and the diffusion machine is a Wasserstein gradient flow, with fixable typos and a real but non-central gap in the binary case study. read the letter →

arxiv 1908.01270 v2 pith:RYGHIOZP submitted 2019-08-04 math.OC cs.LGcs.NE

Abhishek Halder , Kenneth F. Caluya , Bertrand Travacca , Scott J. Moura This is my paper · ORCID
classification math.OCcs.LGcs.NE MSC 49Q2268T0790C26
keywords HopfieldneuralnetworknaturalgradientdescentmirrorWassersteinflowFokker-Planckequationproximalrecursionoptimaltransportactivationfunction
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper explains the continuous-time Hopfield network as a natural object rather than an ad hoc optimizer: its dynamics are steepest descent in a geometry that the activation function itself defines. The deterministic flow is shown to be natural gradient descent on the open unit cube with the diagonal metric $g_{ii}=1/\sigma'_i(\sigma_i^{-1}(x_i))$, and the same flow is then identified with mirror descent on a dual manifold. For the stochastic diffusion machine, the sample paths are not a gradient flow, but the induced density evolution is: the Fokker-Planck equation is the Wasserstein gradient flow of a free energy with a Gibbs stationary density. This geometric reading turns the task of propagating densities into a proximal recursion over probability measures that can be executed on weighted point clouds without spatial grids, which the paper demonstrates numerically on a four-minimum benchmark.

What carries the argument

The load-bearing object is the diagonal metric tensor $G(x)=\operatorname{diag}(1/\sigma'_i(\sigma_i^{-1}(x_i)))$ on $M=(0,1)^n$. It is the Hessian of the convex potential $\psi^*$ whose Legendre-Fenchel conjugate is the mirror map, which is why the same flow can be viewed either as Riemannian steepest descent in the state space or as mirror descent in dual coordinates. In the stochastic setting, the same $G$ defines a Wasserstein metric $W_G$ on the space of probability densities, and the paper's central mechanism is the identification of the Fokker-Planck operator with the negative Wasserstein gradient of the free energy $F$. That identification is what licenses the proximal recursion $\rho_k=\operatorname{arginf}_{\rho}(\tfrac12 W_G^2(\rho_{k-1},\rho)+hF(\rho))$ and the scattered weighted point-cloud computation used in the examples.

What would settle it

Simulate the stochastic diffusion machine (45) with an activation that is strictly increasing on $(0,1)$ but has nonzero derivative at $x_i=0$ and $x_i=1$ (for instance a scaled logistic that does not flatten at the endpoints) and measure the stationary histogram. If it deviates from $\rho_\infty\propto\exp(-f/T)$ with respect to Lebesgue measure, or if boundary flux terms appear in the Fokker-Planck equation, then the claimed $W_G$ gradient-flow representation requires boundary modification, which is exactly the regime the paper's assumption excludes.

Watch

Extended reading notes

Core claim

The central claim is Theorem 1: the flow $\dot{x}=-(G(x))^{-1}\nabla f(x)$ generated by the Hopfield equations is natural gradient descent for $f$ on the manifold $M=(0,1)^n$ with metric tensor $G=\operatorname{diag}(1/\sigma'_i(\sigma_i^{-1}(x_i)))$. Because the activation is componentwise strictly increasing, this diagonal matrix is positive definite, and the geodesic distance induced by the metric gives the natural measure of progress. In the stochastic case the paper claims that the Fokker-Planck equation for the diffusion machine, equation (49), is exactly $\partial \rho/\partial t=-\nabla_{W_G} F(\rho)$, where $F(\rho)=\int_M f\rho\,dx+T\int_M \rho\log\rho\,dx$ is the free energy and $W_G$ is the Wasserstein distance built from the geodesic distance $d_G$ on the ground manifold. The stationary density of this flow is the Gibbs density $\rho_\infty\propto\exp(-f/T)$, so the local minima of $f$ coincide with the modes of the stationary law.

Load-bearing premise

The load-bearing premise is that the activation is a componentwise strictly increasing homeomorphism whose derivative vanishes at the boundary points $x_i=0,1$; if a chosen activation has a nonzero boundary derivative, the clean Fokker-Planck form (49) and the Gibbs stationary measure need boundary corrections, and the Wasserstein gradient-flow reading in the paper must be modified.

Editorial extensions

If this is right

  • The activation function is not a numerical convenience: it chooses the Riemannian metric $g_{ii}=1/\sigma'_i(\sigma_i^{-1}(x_i))$, so different activations produce different descent paths and rates for the same objective $f$.
  • For the soft-projection activation $\sigma_i(z)=\tfrac12\tanh(\beta_i(z-\tfrac12))+\tfrac12$, the geodesic distance is $d_G(x,y)=\|(\arcsin\sqrt{x}-\arcsin\sqrt{y})\oslash\beta\|_2$, giving a closed-form metric in which HNN descent is provably monotone.
  • The stochastic diffusion machine is not a gradient flow for its sample paths, only for its density; this means annealing and global-optimization guarantees are properties of the ensemble, not of any single trajectory.
  • The Fokker-Planck evolution can be solved by the proximal recursion $\rho_k=\operatorname{arginf}_{\rho}(\tfrac12 W_G^2(\rho_{k-1},\rho)+hF(\rho))$, which is zero-th order and can be implemented with weighted point clouds, avoiding spatial discretization.
  • In the deterministic case, Euclidean distance is the wrong monitor of convergence; the paper's numerical case study shows $\|z_k-z^*\|_2$ nonmonotone while $d_G(z_k,z^*)$ decays monotonically.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: for activations other than the soft-projection, the geodesic equation (12) may not decouple, but the proximal-recursion route still applies; the computational question is whether the optimal-transport subproblem with $d_G$ remains cheap enough for point-cloud updates.
  • Editorial inference: the closed-form distance $d_G(x,y)=\|(\arcsin\sqrt{x}-\arcsin\sqrt{y})\oslash\beta\|_2$ coincides with the standard metric on the probability simplex under the arcsine-square-root map, suggesting that HNNs with logistic-type activations are performing mirror descent whose mirror map is the negative entropy; this links the paper's geometry to information-geometric optimization.
  • Editorial inference: the vanishing-derivative boundary condition that makes the Fokker-Planck form clean is not needed for the deterministic theorem; testing activations with nonzero boundary derivatives could separate the deterministic claim from the stochastic one and reveal how much boundary handling matters in practice.
  • Editorial inference: the proximal point-cloud algorithm's runtime advantage, shown on a 2D example, suggests a natural stress test at $n=10$ or $n=100$ with multimodal $f$ to see whether the contraction property used by the algorithm survives when the ground geodesic is nontrivial.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 8 minor

Summary. The paper gives geometric interpretations of continuous-time Hopfield neural network dynamics. For the deterministic HNN, it shows that the flow is Amari's natural gradient descent on the manifold (0,1)^n with a diagonal metric G whose entries are determined by the activation functions, and then derives an equivalent mirror-descent interpretation. For the stochastic HNN (the diffusion machine), it shows that the Fokker–Planck evolution of the joint density is the Wasserstein gradient flow of a free-energy functional, with ground geodesic distance induced by the same metric G, and it proposes a proximal-recursion viewpoint leading to point-cloud-based computation. The paper closes with two numerical illustrations: an economic load dispatch case study and a proximal computation of a multi-modal stationary density.

Significance. If the local corrections below are made, this is a valuable conceptual contribution: it makes explicit that the activation functions choose the Riemannian geometry in which the HNN descends, and it places the diffusion machine in the Jordan–Kinderlehrer–Otto/Wasserstein framework. The core derivations are simple and self-contained: Theorem 1 follows directly from the chain rule, the FPK computation in Section V is algebraically clean, and Appendix A gives a correct dissipation estimate. The paper explicitly states the boundary-degeneracy assumption needed for the stochastic identification, which is a strength. The deterministic result is parameter-free and the stochastic identification yields falsifiable quantitative predictions, such as the explicit geodesic distance for logistic-type activations. The numerical implementation of the proximal recursion relies on the companion paper [54], so the present paper's contribution is the geometric identification rather than a standalone numerical method; that is acceptable but should be stated even more clearly.

major comments (4)
  1. [Section II-A and Eq. (30)] Equation (30) is not the geodesic distance implied by (22)–(29). With u_i := arcsin√γ_i, the soft-projection metric (24) gives g_ii(γ) γdot_i^2 = (2/β_i) u_dot_i^2, so evaluating (13b) along the geodesic (29) yields d_G(x,y) = (Σ_i (2/β_i)(arcsin√x_i − arcsin√y_i)^2)^{1/2} = ‖√2 (arcsin√x − arcsin√y) ⊘ √β‖_2. The printed formula divides by β_i instead of √β_i and omits the factor 2. Since (30) enters the Wasserstein metric (53) and the numerical experiments in Figs. 3 and 5, this needs correction.
  2. [Section V-A, Eq. (54)] The dynamic variational formula (54) has the wrong constant. For G = I, the standard Benamou–Brenier formula is W_2^2(μ,ν) = inf ∫∫ |u|^2 ρ dx dτ (with time horizon 1 and endpoint constraints). The right-hand side of (54) contains an additional 1/2, so for μ = δ_0 and ν = δ_1 it evaluates to 1/2 instead of 1, contradicting (53). Either remove the 1/2 in the integrand or write W_G^2 as twice the infimum of the action with 1/2.
  3. [Section V, Eq. (48)] In the FPK operator (48), the notation g_ii as defined in (4) is the metric entry, but for SDE (45) the correct FPK operator uses the inverse metric entries g^{ii} = 1/g_ii. As typeset, (48) is inconsistent with (45). The subsequent computation (49) is correct only if every g_ii in (48) is read as g^{ii}. Please use superscript notation consistently; the current typesetting makes the central FPK computation ambiguous. The same issue appears in Eq. (63).
  4. [Section II-A, Eqs. (10)–(12)] The Christoffel-symbol list (10) is stated without the necessary index qualifications. Formula (10b) is valid only for i ≠ k; for i = k it has the wrong sign relative to (10d), and (10c) is nonzero when i = k and coincides with (10d). The sentence claiming that (10a)–(10c) are all zero should be restricted to the appropriate off-diagonal cases. In addition, Eq. (12) as typeset contains σ'_i(γ_i) and (σ_i(γ_i))^2, which mix the state and hidden variables; the correct coefficient should be −σ''_i(σ_i^{-1}(γ_i))/(2[σ'_i(σ_i^{-1}(γ_i))]^2). The example equation (25) is correct, but the displayed general expression needs repair.
minor comments (8)
  1. [Footnote before Eq. (47)] The boundary condition should read g^{ii}(x_i) = σ'_i(σ_i^{-1}(x_i)) = 0 at x_i = 0,1; as printed, σ'_i(σ_i(x_i)) is not dimensionally consistent.
  2. [Eq. (28)] The index in 'k = 1,...,n' should be i; the displayed solution is for each coordinate i.
  3. [Eq. (37)] The last denominator appears to be 1 + exp(2β_i z̃_i), not 1 + exp(42β_i z̃_i); please correct the typo and verify the final expression.
  4. [Table I] The equation references are stale: 'F(ρ) given by (45)' should be (51), 'W_G given by (47)' should be (53), and 'SDE (39)' should be (45).
  5. [Section V-C, Eq. (66)] In the Euler–Maruyama update (66), the diffusion term should be evaluated at the old state x^{k-1}, not at x^k; the printed formula evaluates the noise coefficient at the updated state.
  6. [Section V-C, Eq. (64)] In the scalar SDE (64), the expression −2β_i x_i(1−x_i)∇f(x) should be −2β_i x_i(1−x_i)∂f/∂x_i, since the metric is diagonal.
  7. [Figures 5–6] The numerical demonstration is not reproducible from the manuscript: please include the details of the algorithm from [54] used (number of iterations, entropic-regularization update, stopping criteria) and the runtime parameters, or provide code.
  8. [Global] Minor typos: 'Cristoffel' should be 'Christoffel', and reference [19] lists 'Rockafeller' instead of 'Rockafellar'.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: the central results are direct transcriptions of the defining dynamics, with no fitted parameters and no load-bearing self-citation chain.

full rationale

The deterministic claim (Theorem 1) is a direct transcription: substituting x = sigma(x_H) into (1) and using the chain rule gives dx/dt = -Dsigma(sigma^{-1}(x)) grad f(x) = -G^{-1}(x) grad f(x) with G defined in (4), which is exactly Amari's natural gradient dynamics (6) by definition. No parameter is fitted and no external result is needed to force the identification; the metric is read off from the activation function. The mirror-descent interpretation follows from standard Legendre duality and Bregman-divergence identities, and the integration constant in psi* is explicitly left free rather than used to impose the conclusion. For the stochastic case, the FPK computation in (48)-(49) reduces the operator to div(rho G^{-1} grad(f + T log rho)); since the free energy (51) has variational derivative f + T log rho plus a constant, this is precisely the Wasserstein gradient flow -nabla_{W_G} F(rho) under definition (56). The boundary-degeneracy assumption stated before (47) is a cleanly declared hypothesis on the activation, not a hidden input that secretly defines the result. The proximal algorithm used in Section V-C is cited from the authors' companion paper [54], but the geometric identifications that form the paper's central claim do not depend on that citation; [54] is a separate parameter-free derivation, and its use is an application rather than a load-bearing premise. The remaining issues are local typographical errors (e.g., the scaling in Eq. (30)) and absent reproducibility artifacts for Figs. 5-6, which are correctness risks, not circularity.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

No numbers are fitted to data; β_i, T, h, r, and ε are model or algorithm parameters chosen by the user. The analysis assumes standard convex analysis and optimal transport machinery, plus the structural activation and boundary-degeneracy assumptions above.

assumptions (5)
  • domain assumption σ is a homeomorphism, differentiable a.e., componentwise (σ_i depends only on x_{H,i}), strictly increasing
    Assumed in Section II to ensure J_σ is diagonal positive definite, which makes (3) a natural gradient descent with metric (4).
  • standard math Cyclic monotonicity of componentwise increasing σ, implying existence of convex ψ* with ∇ψ* = σ^{-1} (Rockafellar's theorem)
    Invoked in Section III and Remark 2 to obtain the mirror descent dual manifold and the mirror map.
  • standard math JKO/De Giorgi variational scheme for Wasserstein gradient flows: the proximal recursion (60) converges to the FPK solution as h→0
    Cited to [40], [45], [47]; used in Section V-B to justify the proximal recursion as an approximation to (49).
  • domain assumption Boundary degeneracy: g^{-1}_ii = σ'_i(σ_i(x_i)) = 0 at x_i = 0, 1
    Footnoted before (47); ensures no reflection boundary is needed and the stationary density is Gibbs with respect to Lebesgue measure.
  • domain assumption The SDE (45) is taken as the definition of the diffusion machine from Wong [8]
    The whole stochastic analysis starts from this hand-crafted SDE with drift correction T∇(G^{-1}1).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Hopfield Neural Network Flow: A Geometric Viewpoint." pith.science (2026). https://pith.science/paper/RYGHIOZP

@misc{pith2026190801270,
  author       = {Pith},
  title        = {Pith review of: Hopfield Neural Network Flow: A Geometric Viewpoint},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RYGHIOZP}},
  note         = {Machine review of arXiv:1908.01270}
}
read the original abstract

We provide gradient flow interpretations for the continuous-time continuous-state Hopfield neural network (HNN). The ordinary and stochastic differential equations associated with the HNN were introduced in the literature as analog optimizers, and were reported to exhibit good performance in numerical experiments. In this work, we point out that the deterministic HNN can be transcribed into Amari's natural gradient descent, and thereby uncover the explicit relation between the underlying Riemannian metric and the activation functions. By exploiting an equivalence between the natural gradient descent and the mirror descent, we show how the choice of activation function governs the geometry of the HNN dynamics. For the stochastic HNN, we show that the so-called "diffusion machine", while not a gradient flow itself, induces a gradient flow when lifted in the space of probability measures. We characterize this infinite dimensional flow as the gradient descent of certain free energy with respect to a Wasserstein metric that depends on the geodesic distance on the ground manifold. Furthermore, we demonstrate how this gradient flow interpretation can be used for fast computation via recently developed proximal algorithms.

Figures

Figures reproduced from arXiv: 1908.01270 by the authors.

Figure 2
Figure 2. For x ∈ [0, 1]2 , shown above are the contour plots for the geodesic balls of dG given by (30) with βi = 1/4 for i = 1, 2, centered at (0.5, 0.5). with diagonal metric tensor G given by (24), i.e., gii = 1/ (2βixi(1 − xi)), i = 1, . . . , n. In this case, (12) reduces to the nonlinear ODE γ¨i + 2γi − 1 2γi (1 − γi) ( ˙γi) 2 = 0, (25) which solved with the boundary conditions γi(0) = xi , γi(1) = yi , i = 1, . . . , … view at source ↗
Figure 3
Figure 3. For z := (x; y) ∈ R 2nG , and k ∈ N, the distance to minimum dist(z(k), z ∗ ) obtained from the dual Hopfield recursion (44) applied to the economic load dispatch problem in Section IV-B. Results are shown for 100 Monte Carlo simulations and two types of distances: the geodesic distance dG given by (30) (depicted in blue), and the Euclidean distance (depicted in red). For some initial guess of the pair (λ1, λ2), we … view at source ↗
Figure 4
Figure 4. (a) The scaled Himmelblau function f given by (65) has four local minima in [0, 1]2 . (b) The corresponding stationary PDF ρ∞ has four modes, i.e., local maxima, positioned at the arg mins of f. The proximal recursion (60) for the Example in Section V-C1 converges to this stationary PDF ρ∞, as shown via the weighted point cloud evolution in [PITH_FULL_IMAGE:figures/full_fig_p009_4.png] view at source ↗
Figures from the paper (2 more)
Figure 5
Figure 5. Figure 5: The proximal (weighted scattered point cloud) joint PDF evolution over [0, 1]2 associated with (64) and (65), starting from the uniform initial joint PDF. The joint PDF evolution is computed via (60) with WG given by (53), wherein dG is given by (30). The color (red = …
Figure 6
Figure 6. Figure 6: The computational runtime associated with the proximal recursion (60) for the numerical example given in Section V-C1. Here, the physical time-step h = 10−4 , and k ∈ N. VI. CONCLUSIONS In this paper, we highlight the metric geometry of the dynamics associated with the…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

68 extracted references · 68 canonical work pages

  1. [54]

    Mirror descent and nonlinear projected subgradient methods for convex optimization

    A. Beck, and M. Teboulle, “Mirror descent and nonlinear projected subgradient methods for convex optimization”, Operations Research Letters , V ol. 31, No. 3, pp. 167–175, 2003

    work page 2003

  2. [1]

    Neural networks and physical systems with emerging collective computational abilities

    J.J. Hopfield, “Neural networks and physical systems with emerging collective computational abilities”,Proceedings of the National Academy of Sciences, V ol. 79, No. 8, pp. 2554–2558, 1982

    work page 1982

  3. [2]

    Neurons with graded response have collective computa- tional properties like those of two-state neurons

    J.J. Hopfield, “Neurons with graded response have collective computa- tional properties like those of two-state neurons”, Proceedings of the National Academy of Sciences , V ol. 81, No. 10, pp. 3088–3092, 1984

    work page 1984

  4. [3]

    Simple ‘neural’ optimization networks: An A/D converter, signal decision circuit, and a linear programming circuit

    D.W. Tank, and J.J. Hopfield, “Simple ‘neural’ optimization networks: An A/D converter, signal decision circuit, and a linear programming circuit”, IEEE Transactions on Circuits and Systems, V ol. 33, No. 5, pp. 533–541, 1986. Attributes Gradient Flow in Deterministic HNN Gradient Flow in Stochastic HNN Graphical illustration Attributes Gradient Flow in De...

    work page 1986

  5. [4]

    Neural networks for nonlinear program- ming

    M.P. Kennedy, and L.O. Chua, “Neural networks for nonlinear program- ming”, IEEE Transactions on Circuits and Systems , V ol. 35, No. 5, pp. 554–562, 1988

    work page 1988

  6. [5]

    Identification and control of dy- namical systems using neural networks

    K.S. Narendra, and K. Parthasarathy, “Identification and control of dy- namical systems using neural networks”, IEEE Transactions on Neural Networks, V ol. 1, No. 1, pp. 4–27, 1990

    work page 1990

  7. [6]

    Hopfield neural networks for affine invariant matching

    W.-J. Li, and T. Lee, “Hopfield neural networks for affine invariant matching”, IEEE Transactions on Neural Networks , V ol. 12, No. 6, pp. 1400–1410, 2001

    work page 2001

  8. [7]

    Dual Hopfield methods for large-scale mixed-integer programming

    B. Travacca, and S. Moura, “Dual Hopfield methods for large-scale mixed-integer programming”, 2018 IEEE Conference on Decision and Control (CDC), pp. 4959–4966, 2018

    work page 2018

Show all 68 references
  1. [27]

    Aluffi-Pentini, V

    F. Aluffi-Pentini, V . Parisi, and F. Zirilli, “Global optimization and Attributes Gradient Flow in Deterministic HNN Gradient Flow in Stochastic HNN Graphical illustration V ariational problem inf x 2 M f ( x ) inf ⇢ 2 P 2 ( M ) F ( ⇢ ) Objective f ( x ) F ( ⇢ ) given by (45) ...

  2. [28]

    Stochastic neural networks

    E. Wong, “Stochastic neural networks”, Algorithmica , V ol. 6, No. 1–6, pp. 466–478, 1991

    1991

  3. [29]

    Robust stability for interval Hopfield neural networks with time delay

    X. Liao, and J. Y u, “Robust stability for interval Hopfield neural networks with time delay”, IEEE Transactions on Neural Networks , V ol. 9, No. 5, pp. 1042–1045, 1998

    1998

  4. [30]

    Hopfield neural networks for optimization: study of the different dynamics

    G. Joya, M.A. Atencia, F. Sandoval, “Hopfield neural networks for optimization: study of the different dynamics”, Neurocomputing , V ol. 43, No. 1–4, pp. 219–237, 2002

    2002

  5. [31]

    Natural gradient works efficiently in learning

    S-I. Amari, “Natural gradient works efficiently in learning”, Neural Computation , V ol. 10, No. 2, pp. 251–276, 1998

    1998

  6. [33]

    Nemirovskii, and D.B

    A.S. Nemirovskii, and D.B. Y udin, Complexity of problems and effi- ciency of optimization methods , Nauka, Moscow, 1979

    1979

  7. [36]

    do Carmo, Riemannian Geometry , Birkh ¨auser, 1992

    M.P . do Carmo, Riemannian Geometry , Birkh ¨auser, 1992

    1992

  8. [38]

    Convex optimization: algorithms and convexity

    S. Bubeck, “Convex optimization: algorithms and convexity”, F ounda- tions and Trends R in Machine Learning , V ol. 8, No. 3-4, pp. 231–357, 2015

    2015

  9. [39]

    Rockafeller, Convex Analysis , Princeton University Press, 1970

    R.T. Rockafeller, Convex Analysis , Princeton University Press, 1970

    1970

  10. [40]

    Accelerated Mirror Descent in Continuous and Discrete Time

    W. Krichene, A. Bayen, and P .L. Bartlett, “Accelerated Mirror Descent in Continuous and Discrete Time”, Advances in Neural Information Processing Systems , pp. 2845–2853, 2015

    2015

  11. [47]

    Aluffi-Pentini, V

    F. Aluffi-Pentini, V . Parisi, and F. Zirilli, “Global optimization and Variational problem inf x∈M f (x) inf ρ∈P2(M) F (ρ) Objective f (x) F (ρ) given by (51) Distance dG given by (13) WG given by (53) Differential equation dx dt =− (G(x))−1∇f ∂ρ ∂t =∇· ( ρ (G(x))−1 δF δρ ) :=...

  12. [48]

    Stochastic neural networks

    E. Wong, “Stochastic neural networks”, Algorithmica, V ol. 6, No. 1–6, pp. 466–478, 1991

    1991

  13. [49]

    Robust stability for interval Hopfield neural networks with time delay

    X. Liao, and J. Yu, “Robust stability for interval Hopfield neural networks with time delay”, IEEE Transactions on Neural Networks, V ol. 9, No. 5, pp. 1042–1045, 1998

    1998

  14. [50]

    Hopfield neural networks for optimization: study of the different dynamics

    G. Joya, M.A. Atencia, F. Sandoval, “Hopfield neural networks for optimization: study of the different dynamics”, Neurocomputing, V ol. 43, No. 1–4, pp. 219–237, 2002

    2002

  15. [51]

    Natural gradient works efficiently in learning

    S-I. Amari, “Natural gradient works efficiently in learning”, Neural Computation, V ol. 10, No. 2, pp. 251–276, 1998

    1998

  16. [52]

    Shima, The Geometry of Hessian Structures , World Scientific, 2007

    H. Shima, The Geometry of Hessian Structures , World Scientific, 2007

    2007

  17. [53]

    Nemirovskii, and D.B

    A.S. Nemirovskii, and D.B. Yudin, Complexity of problems and effi- ciency of optimization methods , Nauka, Moscow, 1979

    1979

  18. [55]

    The information geometry of mirror descent

    G. Raskutti, and S. Mukherjee, “The information geometry of mirror descent”, IEEE Transactions on Information Theory , V ol. 61, No. 3, pp. 1451–1457, 2015

    2015

  19. [56]

    doCarmo, Riemannian Geometry, Birkh ¨auser, 1992

    M.P. doCarmo, Riemannian Geometry, Birkh ¨auser, 1992

    1992

  20. [57]

    The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programming

    L.M. Bregman, “The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programming”, USSR Comput. Math. Math. Phys. , V ol. 7, pp. 200–217, 1967

    1967

  21. [58]

    Convex optimization: algorithms and convexity

    S. Bubeck, “Convex optimization: algorithms and convexity”, Founda- tions and Trends R© in Machine Learning, V ol. 8, No. 3-4, pp. 231–357, 2015

    2015

  22. [59]

    Rockafeller, Convex Analysis, Princeton University Press, 1970

    R.T. Rockafeller, Convex Analysis, Princeton University Press, 1970

    1970

  23. [60]

    Accelerated Mirror Descent in Continuous and Discrete Time

    W. Krichene, A. Bayen, and P.L. Bartlett, “Accelerated Mirror Descent in Continuous and Discrete Time”, Advances in Neural Information Processing Systems, pp. 2845–2853, 2015

    2015

  24. [61]

    Characterization of the subdifferentials of convex functions

    R.T. Rockafellar, “Characterization of the subdifferentials of convex functions”, Pacific Journal of Mathematics , V ol. 17, No. 3, pp. 497– 510, 1966

    1966

  25. [62]

    On Bregman V oronoi diagrams

    F. Nielsen, J-D. Boissonnat, and R. Nock, “On Bregman V oronoi diagrams”, Proceedings of the eighteenth annual ACM-SIAM symposium on Discrete algorithms , pp. 746–755, 2007

    2007

  26. [63]

    Clustering with Bregman divergences

    A. Banerjee, S. Merugu, I.S. Dhillon, and J. Ghosh, “Clustering with Bregman divergences”, Journal of Machine Learning Research , V ol. 6, pp. 1705–1749, 2005

    2005

  27. [64]

    Economic load dispatch for piecewise quadratic cost function using Hopfield neural network

    J.H. Park, Y .S. Kim, I.K. Eom, and K.Y . Lee, “Economic load dispatch for piecewise quadratic cost function using Hopfield neural network”, IEEE Transactions on Power Systems , V ol. 8, No. 3, pp. 1030–1038, 1993

    1993

  28. [65]

    Optimal environmental dispatching of electric power systems via an improved Hopfield neural network model

    T.D. King, M.E. El-Hawary, F. El-Hawary, “Optimal environmental dispatching of electric power systems via an improved Hopfield neural network model”, IEEE Transactions on Power Systems , V ol. 10, No. 3, pp. 1559–1565, 1995

    1995

  29. [66]

    Adaptive Hopfield neural networks for economic load dispatch

    K.Y . Lee, A. Sode-Yome, and J.H. Park, “Adaptive Hopfield neural networks for economic load dispatch”, IEEE Transactions on Power Systems, V ol. 13, No. 2, pp. 519–526, 1998

    1998

  30. [67]

    Diffusions for global optimization

    S. Geman, and C-R. Hwang, “Diffusions for global optimization”, SIAM Journal of Control and Optimization , V ol. 24, No. 5, pp. 1031–1043, 1986

    1986

  31. [68]

    Analog optimization with Wong’s stochastic neural net- work

    G. Kesidis, “Analog optimization with Wong’s stochastic neural net- work”, IEEE Transactions on Neural Networks , V ol. 6, No. 1, pp. 258– 260, 1995

    1995

  32. [69]

    Global optimization via the Langevin equation

    B. Gidas, “Global optimization via the Langevin equation”, Proceedings of the 24th IEEE Conference on Decision and Control , pp. 774–778, 1985

    1985

  33. [70]

    Global optimization and stochastic differential equations

    F. Aluffi-Pentini, V . Parisi, and F. Zirilli, “Global optimization and stochastic differential equations”, Journal on Optimization Theory and Applications, V ol. 47, No. 1, pp. 1–16, 1985

    1985

  34. [71]

    A learning algorithm for Boltzmann machines

    D.H. Ackley, G.E. Hinton, and T.J. Sejnowski, “A learning algorithm for Boltzmann machines”, Cognitive Science , V ol. 9, No. 1, pp. 147–169, 1985

    1985

  35. [72]

    Risken, The Fokker-Planck equation: Methods of solution and appli- cations

    H. Risken, The Fokker-Planck equation: Methods of solution and appli- cations. Springer, 1996

    1996

  36. [73]

    Exponential convergence to equilibrium for kinetic Fokker-Planck equations

    S. Calogero, “Exponential convergence to equilibrium for kinetic Fokker-Planck equations”, Communications in Partial Differential Equa- tions, V ol. 37, pp. 1357–1390, 2012

    2012

  37. [74]

    Oscillatory descent for function minimization

    R. Brockett, “Oscillatory descent for function minimization”, in: Current and Future Directions in Applied Mathematics , pp. 65–82, Birkh ¨auser, Boston, MA, 1997

    1997

  38. [75]

    Noisy recurrent neural networks: the continuous-time case

    S. Das, and O. Olurotimi, “Noisy recurrent neural networks: the continuous-time case”, IEEE Transactions on Neural Networks , V ol. 9, No. 5, pp. 913–936, 1998

    1998

  39. [76]

    Villani, Topics in optimal transportation , American Mathematical Society, Providence, RI, 2003

    C. Villani, Topics in optimal transportation , American Mathematical Society, Providence, RI, 2003

    2003

  40. [77]

    Convex functionals of probability measures and non- linear diffusions on manifolds

    K.-T. Sturm, “Convex functionals of probability measures and non- linear diffusions on manifolds”, Journal de Math ´ematiques Pures et Appliqu´ees, V ol. 84, No. 2, pp. 149–168, 2005

    2005

  41. [78]

    Gradient flows on Wasserstein spaces over compact Alexan- drov spaces

    S-I. Ohta, “Gradient flows on Wasserstein spaces over compact Alexan- drov spaces”, American Journal of Mathematics , V ol. 131, No. 2, pp. 475–516, 2009

    2009

  42. [79]

    A computational fluid mechanics solution to the Monge-Kantorovich mass transfer problem

    J-D. Benamou, and Y . Brenier, “A computational fluid mechanics solution to the Monge-Kantorovich mass transfer problem”, Numerische Mathematik, V ol. 84, No. 3, pp. 375–393, 2000

    2000

  43. [80]

    Nonlinear diffusion equations with variable coefficients as gradient flows in Wasserstein spaces

    S. Lisini, “Nonlinear diffusion equations with variable coefficients as gradient flows in Wasserstein spaces”, ESAIM: Control, Optimisation and Calculus of Variations , V ol. 15, No. 3, pp. 712–740, 2009

    2009

  44. [81]

    The geometry of dissipative evolution equations: the porous medium equation

    F. Otto, “The geometry of dissipative evolution equations: the porous medium equation”, Communications in Partial Differential Equations , V ol. 26, pp. 101–174, 2001

    2001

  45. [82]

    Villani, Optimal transport: old and new, Springer Science & Business Media, V ol

    C. Villani, Optimal transport: old and new, Springer Science & Business Media, V ol. 338, 2008

    2008

  46. [83]

    Ambrosio, N

    L. Ambrosio, N. Gigli, and G. Savar ´e, Gradient flows: in metric spaces and in the space of probability measures , Springer Science & Business Media, 2008

    2008

  47. [84]

    Notes on stochastic processes on manifolds

    R. Brockett, “Notes on stochastic processes on manifolds”, in Systems and Control in the twenty-first century, edited by C.I. Byrnes, B.N. Datta, D.S. Gilliam, and C.F. Martin, pp. 75–100, Springer, 1997

    1997

  48. [85]

    The variational formulation of the Fokker-Planck equation

    R. Jordan, D. Kinderlehrer, and F. Otto, “The variational formulation of the Fokker-Planck equation”, SIAM Journal on Mathematical Analysis , V ol. 29, No. 1, pp. 1–17, 1998

    1998

  49. [86]

    Constrained Steepest Descent in the 2- Wasserstein Metric

    E.A. Carlen, and W. Gangbo, “Constrained Steepest Descent in the 2- Wasserstein Metric”, Annals of Mathematics , V ol. 157, pp. 807–846, 2003

    2003

  50. [87]

    {Euclidean, metric, and Wasserstein} gradient flows: an overview

    F. Santambrogio, “{Euclidean, metric, and Wasserstein} gradient flows: an overview”, Bulletin of Mathematical Sciences , V ol. 7, No. 1, pp. 87–154, 2017

    2017

  51. [88]

    Proximit ´e et dualit ´e dans un espace hilbertien

    J-J. Moreau, “Proximit ´e et dualit ´e dans un espace hilbertien”, Bulletin de la Soci ´et´e math ´ematique de France , V ol. 93, pp. 273–299, 1965

    1965

  52. [89]

    Monotone operators and the proximal point algo- rithm

    R.T. Rockafellar, “Monotone operators and the proximal point algo- rithm”, SIAM Journal on Control and Optimization , V ol. 14, No. 5, pp. 877–898, 1976

    1976

  53. [90]

    Bauschke, and P.L

    H.H. Bauschke, and P.L. Combettes, Convex analysis and monotone operator theory in Hilbert spaces , V ol. 408, Springer, 2011

    2011

  54. [91]

    Gradient flows in uncertainty propagation and filtering of linear Gaussian systems

    A. Halder, and T.T. Georgiou, “Gradient flows in uncertainty propagation and filtering of linear Gaussian systems”, Proceedings of the 2017 IEEE 56th Annual Conference on Decision and Control (CDC) , pp. 3081– 3088, 2017

    2017

  55. [92]

    Gradient flows in filtering and Fisher- Rao Geometry

    A. Halder, and T.T. Georgiou, “Gradient flows in filtering and Fisher- Rao Geometry”, 2018 Annual American Control Conference (ACC) , pp. 4281–4286, 2018

    2018

  56. [93]

    Proximal recursion for solving the Fokker- Planck equation

    K.F. Caluya, and A. Halder, “Proximal recursion for solving the Fokker- Planck equation”, 2019 Annual American Control Conference (ACC) , 2019

    2019

  57. [94]

    Gradient flow algorithms for density propa- gation in stochastic systems

    K.F. Caluya, and A. Halder, “Gradient flow algorithms for density propa- gation in stochastic systems”, accepted, IEEE Transactions on Automatic Control, available online: https://arxiv.org/pdf/1908.00533.pdf, 2019

    1908 arXiv

  58. [95]

    Proximal algorithms

    N. Parikh, and S. Boyd, “Proximal algorithms”, Foundations and Trends R© in Optimization, V ol. 1, No. 3, pp. 127–239, 2014

    2014

  59. [96]

    Himmelblau, Applied nonlinear programming , McGraw-Hill, 1972

    D. Himmelblau, Applied nonlinear programming , McGraw-Hill, 1972

    1972

  60. [97]

    Kloeden, and E

    P.E. Kloeden, and E. Platen, Numerical solution of stochastic differential equations, Springer Science & Business Media, V ol. 23, 2013

    2013

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.