Pith. sign in

REVIEW 3 major objections 6 minor 44 references

Constrained Optimization via Constraint-Induced Geometry: Implicit Feasible Dynamics and Optimality from Stationarity

T0 review · 3 major / 6 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Stationary flows satisfy KKT with feasibility built in

desk verdict Useful synthesis with a real gap: continuous-time KKT from stationarity works, but the discrete-time simplex/box KKT claim is unproven as written. read the letter →

arxiv 2508.18764 v3 pith:NM5OBTDY submitted 2025-08-26 math.OC

Valentin Leplat This is my paper · ORCID
classification math.OC MSC 90C3090C2565K10
keywords constrainedoptimizationimplicitgradientflowsKKTconditionscomplementaryslacknessBregmanproximalpointStiefelmanifoldA-stabilityreparameterization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that constraints in continuous-time optimization should be encoded into the geometry of the flow rather than enforced by projection or penalty. For the nonnegative orthant and boxes, it uses componentwise reparameterizations whose Jacobians vanish on the boundary; for the simplex, the rank-deficient Fisher-Shahshahani operator $\operatorname{diag}(x)-xx^T$; for orthogonality constraints, the canonical Riemannian structure of the Stiefel manifold. The payoff is a kinematic mechanism: a trajectory that comes to rest can only rest at points where the gradient's normal component vanishes, which is exactly complementary slackness, so stationarity of the dynamics implies the Karush-Kuhn-Tucker conditions. The same logic is shown for the implicit (A-stable) discretizations, which are Bregman proximal steps and therefore come with monotone descent, $O(1/k)$ convex rates, linear contraction under relative strong convexity for any positive step size, and Kurdyka-Lojasiewicz convergence in the nonconvex case.

What carries the argument

The central object is the constraint-induced geometry: the Jacobian-induced metric $G(u)=J_g(u)$ on vector domains (chosen over the pullback metric to keep the u-dynamics simple and to let the Jacobian vanish exactly at active faces), the Fisher-Shahshahani operator $G(x)=\operatorname{diag}(x)-xx^T$ on the simplex, and the canonical metric with skew field $A(X)=\nabla\Phi(X)X^T-X\nabla\Phi(X)^T$ on the Stiefel manifold. This object does three jobs: it keeps every continuous and discrete trajectory feasible by construction; it turns stationarity into KKT through null-speed and rank-deficiency arguments; and it makes implicit discretizations (backward Euler, KL-prox, Cayley update) into Bregman-proximal or resolv

What would settle it

On a small NNLS problem, run the orthant method with a deliberately loose inner tolerance for many outer iterations and monitor the KKT residual: if it stalls above zero while the exact-update version drives it to zero, the transfer of the KKT characterization to inexact solvers is false. Separately, build a convex problem with a rapidly oscillating metric (large drift) and check whether iterates converge to a non-stationary point, which would show the drift condition in the variable-metric theorem is genuinely necessary.

Watch

Extended reading notes

Core claim

At the center is the 'kinematic slackness' mechanism. Feasibility is built into the flow by $x=g(u)$; with the Jacobian-induced metric, the primal velocity is $\dot{x}=-J_g(u)\nabla f(x)$. Since $J_g$ vanishes on active faces or has rank n-1 with kernel $\operatorname{span}\{1\}$ on the simplex, vanishing velocity forces the normal component of the gradient to vanish at the limit: for the exponential map this is written as the identity $x_i \nabla_i f(x) = -\dot{x}_i$, i.e., complementary slackness is a kinematic statement. The paper proves that the same conclusion holds for convergent implicit sequences whose stepsizes are bounded away from zero; on the orthant the discrete update is exactly the multiplicative update

Load-bearing premise

The discrete-time KKT result assumes the inner equations are solved exactly; when they are only solved approximately, the paper shows rates are preserved but does not prove the limit still satisfies KKT, and the variable-metric proof needs the coordinate system to change slowly, a condition it assumes without checking.

Editorial extensions

If this is right

  • On orthant and box domains, backward Euler in the reparameterized variable is a Bregman-prox step, so convex objectives decrease monotonically and converge at O(1/k) with no restriction on step size.
  • On the simplex, the implicit KL-prox step avoids the rank-deficient Jacobian in the inner solve; the Newton-KKT system is SPD, and convergent interior sequences satisfy the simplex KKT conditions.
  • On the Stiefel manifold, the implicit Cayley update preserves X^T X=I exactly, and a zero Riemannian-gradient limit satisfies the first-order optimality condition for orthogonality constraints.
  • When inner solves are inexact but residuals are summable, the convex and nonconvex rates of the exact implicit scheme survive.
  • For nonconvex objectives satisfying Kurdyka-Lojasiewicz, the Bregman scheme converges to a critical point with the standard finite/linear/sublinear rate trichotomy; under a Bregman-PL condition it is linearly convergent for any positive step size.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The null-speed mechanism should transfer to any smooth reparameterization whose Jacobian loses rank exactly on the active set; a testable extension is to cones or product sets built from smooth inequality constraints.
  • The variable-metric theorem's drift condition suggests a practical safeguard: growth of the step size should be gated by the ratio ||G_{k+1}-G_k|| / ||x_{k+1}-x_k||, giving a computable certificate for the no-stepsize-cap regime.
  • The implicit multiplicative update on the orthant identifies the KKT mechanism of classical multiplicative updates and may extend their convergence theory beyond nonnegative least squares to general smooth objectives.
  • Viewing the Stiefel Cayley step as a trapezoidal discretization points toward higher-order implicit integrators (BDF-type) as a route to accelerated feasible manifold methods; the paper itself lists inertial and accelerated variants as open directions.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes a unified geometry-based framework for constrained optimization: feasible sets are encoded via reparameterizations whose Jacobians vanish at the boundary (orthant, box), via the Fisher–Shahshahani/KL geometry (simplex), or via the canonical Riemannian structure (Stiefel). The continuous-time dynamics are shown to be gradient-like, and the paper argues that stationarity of these dynamics implies KKT conditions. Implicit (backward-Euler/Bregman-prox/Cayley) discretizations are introduced, with convergence results in convex, strongly convex, and nonconvex (KL) settings, and numerical experiments compare the resulting algorithms against standard baselines. The paper also states that convergent implicit sequences generated from the interior recover KKT when stepsizes are bounded away from zero.

Significance. If the main claims hold, the paper would provide a useful unifying perspective: a single kinematic mechanism (vanishing Jacobian or rank-deficient Fisher operator) explains complementary slackness across four constraint geometries, and the implicit schemes have strong stability and convergence properties. The Bregman-proximal convergence theory in Sections 7.2–7.5 is standard and appears correct; the continuous-time KKT derivations are elegant, and the Stiefel Cayley construction is a clean implicit-flow interpretation of existing feasible methods. The paper also provides openly available code and reproducible experiments, which is a notable strength. However, two load-bearing parts of the paper are not yet supported: the discrete-time KKT recovery for simplex/box and the variable-metric convergence theorem as stated. These are fixable but require nontrivial additional analysis or explicit conditions.

major comments (3)
  1. [§5.2 B and C] The discrete-time KKT claim for the simplex and box is not proven. For the KL-prox step (17)–(18), all finite iterates are strictly interior, so the nonnegativity multipliers λ_{k+1} in the stated KKT system are identically zero. The paper asserts that the limit 'reproduces the continuous-time conditions above' (Eq. 18 and following text), but no argument is given for the limiting inequalities ∇f_i(x*) ≥ τ when x*_i = 0. This requires controlling limsup (log x_{k+1,i} − log x_{k,i})/η_k, which need not tend to 0 and whose sign determines the limiting multiplier. The same gap appears in §5.2 C for the two-sided entropy on a box. Since the algorithms in §6 use exactly these strict-interior prox steps, the claimed KKT recovery for convergent implicit sequences is unsupported for the simplex and box.
  2. [Theorem 7.8, Eq. (32)] The summed variable-metric inequality contains a positive metric-drift term +(Bτ/2η) Σ ||x_{k+1}−x_k||² that must be dominated by the negative term −(m/2η̄) Σ ||x_{k+1}−x_k||² for the conclusion to yield convergence. No such dominance condition is stated or verified; the theorem only assumes m, M, τ and bounded stepsizes. Without e.g. m η > B τ η̄, Eq. (32) is compatible with divergent increments, and the 'o(1)' in Corollary 7.10 does not rescue the argument because summability of ||x_{k+1}−x_k||² is part of what needs to be proved. This gap affects the claimed global convergence guarantees for the variable-metric (reparameterized orthant/box) constructions.
  3. [§7.5 and §6 inner solvers] Proposition 7.2 transfers convergence rates to inexact inner solves under a summable-residual condition, but it does not establish that limit points of the inexact implicit updates satisfy KKT. The implemented algorithms in §6 (MGN for orthant/box, Newton–KKT with tolerance for the simplex, Newton–Krylov for Stiefel) compute approximate solves. Even if their residuals are summable, the KKT characterization is a limit statement about the discrete scheme, and no theorem connects the inexact iterates to the KKT system. Thus the practical algorithms, as implemented, do not inherit the KKT guarantee stated in the abstract without an additional argument.
minor comments (6)
  1. [Title/abstract] The arXiv abstract describes 'Gravidy' while the full-text title and abstract describe a different framing; the two versions should be harmonized before publication.
  2. [Appendix references] Internal references to 'Appendix 8', 'Appendix 14', 'Appendix 7', and 'Appendix 9' do not match the labeled appendices (A–F) in the compiled text.
  3. [§2.3.2] The displayed linearization '˙δ = −G ⋆−1H⋆ [G] δ' is garbled; the intended generator/metric structure should be written clearly.
  4. [Appendix A] In the 1D KKT check for c<0, the paper states 'choose λ⋆ = ab'; with the paper's convention ∇f(x*) − λ* = 0, the correct multiplier is λ* = −ab > 0. This is a sign error in the example.
  5. [Table 1] Table 1(a) reports a final KKT residual of 5.86e-05 for Gravidy-Pos (MGN), yet Table 1(b) reports a time to tolerance ≤ 1e-8 for the same method. The inconsistency should be resolved or explained.
  6. [§5.2 D] The Stiefel discrete-time statement says 'if the inner Newton solve drives the residual to zero' the limit satisfies KKT; this quantifier is informal and should be replaced by a precise statement about finite inexact solves.

Circularity Check

0 steps flagged · score 2.0 of 10

No circular reduction found; stationarity-to-KKT is derived from explicitly stated reparameterizations and independently benchmarked. One minor non-load-bearing self-citation and two non-circular proof gaps noted.

full rationale

The paper's central claim, that stationarity of the constructed flows and implicit sequences implies KKT, is a direct mathematical consequence of the explicitly defined reparameterizations rather than a fitted input or a renamed prediction. For the orthant and box, the flow is defined as ẋ = −Jg(u)∇Φ(x) (Eq. 10), so stationarity gives Jg(u⋆)∇Φ(x⋆) = 0; with diagonal Jg positive in the interior and vanishing on active faces, complementarity follows by construction. The discrete orthant result (Eq. 15) is then proven from the exact multiplicative update, and convergence to KKT is argued from the sign of the exponential factor. The simplex continuous-time result (Eq. 16) is the standard Fisher–Shahshahani/replicator flow, and stationarity equalizes active gradients, which is exactly simplex KKT. The Stiefel result uses the standard Riemannian-gradient stationarity characterization from external references [17, 1, 11]. No parameter is fitted and then called a prediction, and no load-bearing uniqueness theorem is imported from the author's prior work. The only self-citation, [25] Leplat et al., appears in related work on β-divergence geometries and is not load-bearing. Two non-circular correctness gaps should be flagged: (i) in Section 5.2 B, the finite-step simplex KL-prox KKT system (Eq. 18) has nonnegativity multipliers identically zero at every finite interior iterate, and the statement 'which in the limit reproduces the continuous-time conditions above' is asserted without a proof constructing limiting multipliers or controlling log-ratio terms; (ii) in Theorem 7.8, Eq. (32) contains a positive metric-drift term +(Bτ/2η)Σ||x_{k+1}−x_k||² that must be dominated by the negative term −(m/2η̄)Σ||x_{k+1}−x_k||² for the claimed convergence to follow, but no such condition is stated or verified. These are omitted-proof/robustness issues, not circularity, and do not raise the circularity score above 2.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

No free parameters are fitted to data. The theory relies on standard smoothness, convexity, and KL assumptions (A1-A6) and on the explicit properties of the chosen reparameterizations. The 'Gravidy' framework is a new name, but it does not introduce new physical or mathematical entities beyond those already listed.

assumptions (5)
  • domain assumption Φ ∈ C^1 on an open set containing C; ∇Φ is L-Lipschitz on compact subsets (Assumptions A1, A2)
    Stated in Section 2.1 and used throughout for gradient-flow and Bregman prox analysis.
  • domain assumption Legendre generator h with σh-strong convexity and Lh-smoothness on relevant level sets
    Required for Bregman proximal point convergence rates in Section 7.
  • domain assumption Kurdyka-Lojasiewicz property on bounded level sets (Assumption A6)
    Used for nonconvex convergence rates in Theorem 7.11.
  • domain assumption Reparameterization maps g are C^2 with Jg continuous; g'_i → 0 on active faces for orthant/box; softmax has rank n-1 Jacobian
    Stated in Section 2.1 (A4) and Section 4; this is the load-bearing geometric structure.
  • domain assumption Relative strong convexity of f with respect to h (Assumption 7.2)
    Used to derive the linear contraction factor in Theorem 7.3.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Constrained Optimization via Constraint-Induced Geometry: Implicit Feasible Dynamics and Optimality from Stationarity." pith.science (2026). https://pith.science/paper/NM5OBTDY

@misc{pith2026250818764,
  author       = {Pith},
  title        = {Pith review of: Constrained Optimization via Constraint-Induced Geometry: Implicit Feasible Dynamics and Optimality from Stationarity},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NM5OBTDY}},
  note         = {Machine review of arXiv:2508.18764}
}
read the original abstract

We introduce Gravidy, a geometry-aware framework for constrained optimization in which constraints are encoded directly into the dynamics, so the motion remains feasible by construction. The geometric mechanism depends on the feasible set: componentwise reparameterizations and induced Hessian geometries for the nonnegative orthant and box constraints, Fisher-Shahshahani and KL geometry for the simplex, and canonical Riemannian geometry for the Stiefel manifold. We derive feasible continuous-time flows and implicit discretizations adapted to each geometry. On the vector domains, the implicit updates admit exact Bregman-proximal interpretations, yielding monotone descent and convergence guarantees for convex objectives, linear contraction under relative strong convexity, and a Kurdyka-Lojasiewicz analysis for nonconvex problems under compact-interiority, decrease, and relative-error assumptions. We also show that convergent trajectories generated from the interior recover the Karush-Kuhn-Tucker conditions on the orthant, simplex, and box. The same holds for convergent implicit sequences generated from the interior when their stepsizes are bounded away from zero. On the Stiefel manifold, stationarity of the canonical Riemannian gradient is equivalent to the usual first-order optimality condition. The algorithms combine large implicit outer steps with problem-adapted Newton, modified Gauss-Newton, Newton-KKT, and Newton-Krylov inner solvers. Numerical experiments on nonnegative, simplex-constrained, box-constrained, and orthogonality-constrained problems show rapid convergence to high accuracy in a small number of outer iterations while preserving feasibility of accepted iterates. A sparse elastic-obstacle experiment further shows that the orthant construction can exploit large structured systems directly. This illustrates their accuracy and ability to exploit sparsity in sparse settings.

Figures

Figures reproduced from arXiv: 2508.18764 by the authors.

Figure 1
Figure 1. Results for Orthant (NNLS), n = 120. Left: f(xk) vs iterations. Right: f(xk) vs time. Averages over 10 seeds; shaded ±1 std. Simplex. Observation: Gravidy-∆ (implicit KL–prox) enjoys large η without rank issues and outperforms PGD+Nest and mirror descent in both iterations and time for ill-conditioned A⊤A. Box. Observation: Gravidy-Box matches or exceeds projected methods as stiffness grows, thanks to the implicit s… view at source ↗
Figure 2
Figure 2. Results for Simplex, n = 40–200. Left: f(xk) vs iterations. Right: f(xk) vs time. 0 50 100 150 200 250 300 350 400 Iterations 10 7 10 6 10 5 10 4 10 3 10 2 10 1 10 0 10 1 |f(xk) f * | Box: objective gap vs iterations GRAVIDY box (Newton) GRAVIDY box (MGN) APGD-box (Nesterov) 10 5 10 4 10 3 10 2 10 1 10 0 10 1 Time [seconds] 10 7 10 6 10 5 10 4 10 3 10 2 10 1 10 0 10 1 |f(xk) f * | Box: objective gap vs time GRAVIDY … view at source ↗
Figure 3
Figure 3. Results for Box. Left: objective vs iterations. Right: objective vs time. [PITH_FULL_IMAGE:figures/full_fig_p026_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Results for Stiefel quadratic, n = 200, p = 2. Left: ∥ grad Φ(Xk)∥F vs iterations. Right: feasibility ∥X⊤ k Xk − I∥F vs time. 8.3 Ablations and stiffness We vary κ(A⊤A) (or the spread of eigenvalues of Q(j) on Stiefel) and observe that the iteration counts of explicit …

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

44 extracted references · 42 canonical work pages

  1. [1]

    Absil, R

    P.-A. Absil, R. Mahony, and R. Sepulchre.Optimization Algorithms on Matrix Manifolds. Princeton University Press, 2008. 27 Table 1: Summary across geometries (lower is better). Boldface marks the best (median) among Gravidyvariants within each geometry. (a) Final KKT residual (Stiefel:∥ grad Φ(X)∥F). Method Orthant Simplex Box Stiefel Gravidy-Pos(MGN) 5.8...

    work page 2008

  2. [2]

    Hedy Attouch, Jérôme Bolte, and Benar F. Svaiter. Convergence of descent methods for semi-algebraic and tame problems: Proximal algorithms, forward–backward splitting, and regularized gauss–seidel methods.Mathematical Programming, 137(1–2):91–129, 2013

    work page 2013

  3. [3]

    A dynamical approach to convex minimization coupling approximation with the steepest descent method.Journal of Differential Equations, 128(2):519–540, 1996

    Hedy Attouch and Roberto Cominetti. A dynamical approach to convex minimization coupling approximation with the steepest descent method.Journal of Differential Equations, 128(2):519–540, 1996

    work page 1996

  4. [4]

    A dynamical approach to an inertial forward-backward algorithm for convex minimization.SIAM Journal on Optimization, 24(1):232–256, 2014

    Hédy Attouch, Juan Peypouquet, and Patrick Redont. A dynamical approach to an inertial forward-backward algorithm for convex minimization.SIAM Journal on Optimization, 24(1):232–256, 2014. 28

    work page 2014

  5. [5]

    A continuous dynamical newton-like approach to solving monotone inclusions.SIAM Journal on Control and Optimization, 49(2):574–598, 2011

    Hedy Attouch and Benar Fux Svaiter. A continuous dynamical newton-like approach to solving monotone inclusions.SIAM Journal on Control and Optimization, 49(2):574–598, 2011

    work page 2011

  6. [6]

    Mirror descent and nonlinear projected subgradient methods for convex optimization.Operations Research Letters, 31(3):167–175, 2003

    Amir Beck and Marc Teboulle. Mirror descent and nonlinear projected subgradient methods for convex optimization.Operations Research Letters, 31(3):167–175, 2003

    2003

  7. [7]

    Birgin, José Mario Martínez, and Marcos Raydan

    Ernesto G. Birgin, José Mario Martínez, and Marcos Raydan. Nonmonotone spectral projected gradient methods on convex sets.SIAM Journal on Optimization, 10(4):1196– 1211, 2000

    work page 2000

  8. [8]

    An algorithm, based on singular perturbation theory, for ill-conditioned minimization problems

    Paul T Boggs. An algorithm, based on singular perturbation theory, for ill-conditioned minimization problems. SIAM Journal on Numerical Analysis, 14(5):830–843, 1977

    work page 1977

Show all 44 references
  1. [9]

    The łojasiewicz inequality for nonsmooth subanalytic functions with applications to subgradient dynamical systems.SIAM Journal on Optimization, 17(4):1205–1223, 2007

    Jérôme Bolte, Aris Daniilidis, and Adrian Lewis. The łojasiewicz inequality for nonsmooth subanalytic functions with applications to subgradient dynamical systems.SIAM Journal on Optimization, 17(4):1205–1223, 2007

    2007

  2. [10]

    Proximal alternating linearized minimiza- tion for nonconvex and nonsmooth problems.SIAM Journal on Optimization, 24(3):137–164, 2014

    Jérôme Bolte, Shoham Sabach, and Marc Teboulle. Proximal alternating linearized minimiza- tion for nonconvex and nonsmooth problems.SIAM Journal on Optimization, 24(3):137–164, 2014

    2014

  3. [11]

    N. Boumal. An Introduction to Optimization on Smooth Manifolds. Cambridge University Press, 2023

    2023

  4. [12]

    Absil, and Rodolphe Sepulchre

    Nicolas Boumal, Bamdev Mishra, P.-A. Absil, and Rodolphe Sepulchre. Manopt, a matlab toolbox for optimization on manifolds.Journal of Machine Learning Research, 15(1):1455– 1459, 2014

    2014

  5. [13]

    A fast non-negativity-constrained least squares algorithm

    Rasmus Bro and Sijmen De Jong. A fast non-negativity-constrained least squares algorithm. Journal of Chemometrics: A Journal of the Chemometrics Society, 11(5):393–401, 1997

    1997

  6. [14]

    Some effective methods for uncon- strained optimization based on the solution of systems of ordinary differential equations

    Andrew A Brown and Michael C Bartholomew-Biggs. Some effective methods for uncon- strained optimization based on the solution of systems of ordinary differential equations. Journal of Optimization Theory and Applications, 62:211–224, 1989

    1989

  7. [15]

    Byrd, Peihuang Lu, Jorge Nocedal, and Ciyou Zhu

    Richard H. Byrd, Peihuang Lu, Jorge Nocedal, and Ciyou Zhu. A limited memory algorithm for bound constrained optimization.SIAM Journal on Scientific Computing, 16(5):1190– 1208, 1995

    1995

  8. [16]

    A special stability problem for linear multistep methods.BIT Numerical Mathematics, 3(1):27–43, 1963

    Germund G Dahlquist. A special stability problem for linear multistep methods.BIT Numerical Mathematics, 3(1):27–43, 1963

    1963

  9. [17]

    Edelman, T.A

    A. Edelman, T.A. Arias, and S.T. Smith. The geometry of algorithms with orthogonality constraints. SIAM J. MATRIX ANAL. APPL., 20:303–353, 1998

    1998

  10. [18]

    Févotte and J

    C. Févotte and J. Idier. Algorithms for nonnegative matrix factorization with theβ- divergence. Neural computation, 23(9):2421–2456, 2011

    2011

  11. [19]

    Admm and accelerated admm as continuous dynamical systems

    Guilherme Franca, Daniel Robinson, and Rene Vidal. Admm and accelerated admm as continuous dynamical systems. InInternational Conference on Machine Learning, pages 1559–1567. PMLR, 2018

    2018

  12. [20]

    Proximal gradient flow and douglas– rachford splitting dynamics: Global exponential stability via integral quadratic constraints

    Sepideh Hassan-Moghaddam and Mihailo R Jovanović. Proximal gradient flow and douglas– rachford splitting dynamics: Global exponential stability via integral quadratic constraints. Automatica, 123:109311, 2021. 29

    2021

  13. [21]

    Trust region algorithms and timestep selection.SIAM Journal on Numerical Analysis, 37(1):194–210, 1999

    Desmond J Higham. Trust region algorithms and timestep selection.SIAM Journal on Numerical Analysis, 37(1):194–210, 1999

    1999

  14. [22]

    Jyrki Kivinen and Manfred K. Warmuth. Additive versus exponentiated gradient updates for linear prediction.Information and Computation, 132(1):1–63, 1997

    1997

  15. [23]

    Lawson and R.J

    C.L. Lawson and R.J. Hanson. Solving Least Squares Problems. Classics in Applied Mathematics. Society for Industrial and Applied Mathematics, 1995

    1995

  16. [24]

    Lee and H

    Daniel D. Lee and H. Sebastian Seung. Learning the parts of objects by non-negative matrix factorization. Nature, 401(6755):788–791, 1999

    1999

  17. [25]

    Leplat, N

    V. Leplat, N. Gillis, and C. Févotte. Multi-resolution beta-divergence nmf for blind spectral unmixing. Signal Processing, 2021

    2021

  18. [26]

    Understanding and accelerating particle-based variational inference

    Chang Liu, Jingwei Zhuo, Pengyu Cheng, Ruiyi Zhang, and Jun Zhu. Understanding and accelerating particle-based variational inference. In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, Proceedings of the 36th International Conference on Machine Learning, volume 97 ofProc...

    2019

  19. [27]

    From differential equation solvers to accelerated first-order methods for convex optimization.Mathematical Programming, pages 1–47, 2021

    Hao Luo and Long Chen. From differential equation solvers to accelerated first-order methods for convex optimization.Mathematical Programming, pages 1–47, 2021

    2021

  20. [28]

    From differential equation solvers to accelerated first-order methods for convex optimization.Mathematical Programming, 195(1-2):735–781, 2022

    Hao Luo and Long Chen. From differential equation solvers to accelerated first-order methods for convex optimization.Mathematical Programming, 195(1-2):735–781, 2022

    2022

  21. [29]

    Ronan M. T. Fleming Masoud Ahookhosh and Phan T. Vuong. Finding zeros of hölder metrically subregular mappings via globally convergent levenberg–marquardt methods. Optimization Methods and Software, 37(1):113–149, 2022

    2022

  22. [30]

    Nemirovsky and David B

    Arkadi S. Nemirovsky and David B. Yudin.Problem Complexity and Method Efficiency in Optimization. Wiley, New York, 1983

    1983

  23. [31]

    Modified gauss–newton scheme with worst-case guarantees for its global performance

    Yurii Nesterov. Modified gauss–newton scheme with worst-case guarantees for its global performance. Technical Report CORE Discussion Paper 2003/xx, CORE, Université catholique de Louvain, 2003

    2003

  24. [32]

    Springer, Cham, 2 edition, 2018

    Yurii Nesterov.Lectures on Convex Optimization, volume 137 ofSpringer Optimization and Its Applications. Springer, Cham, 2 edition, 2018

    2018

  25. [33]

    Portugal, Joaquim J

    Luís F. Portugal, Joaquim J. Júdice, and Luís N. Vicente. A comparison of block pivoting and interior-point algorithms for linear least squares problems with nonnegative variables. Math. Comput., 63(208):625–643, oct 1994

    1994

  26. [34]

    Monotone operators and the proximal point algorithm.SIAM journal on control and optimization, 14(5):877–898, 1976

    R Tyrrell Rockafellar. Monotone operators and the proximal point algorithm.SIAM journal on control and optimization, 14(5):877–898, 1976

    1976

  27. [35]

    Integration methods and optimization algorithms.Advances in Neural Information Processing Systems, 30, 2017

    Damien Scieur, Vincent Roulet, Francis Bach, and Alexandre d’Aspremont. Integration methods and optimization algorithms.Advances in Neural Information Processing Systems, 30, 2017

    2017

  28. [36]

    Understanding the acceleration phenomenon via high-resolution differential equations.Mathematical Programming, pages 1–70, 2021

    Bin Shi, Simon S Du, Michael I Jordan, and Weijie J Su. Understanding the acceleration phenomenon via high-resolution differential equations.Mathematical Programming, pages 1–70, 2021. 30

    2021

  29. [37]

    A differential equation for modeling nesterov’s accelerated gradient method: theory and insights.Advances in neural information processing systems, 27, 2014

    Weijie Su, Stephen Boyd, and Emmanuel Candes. A differential equation for modeling nesterov’s accelerated gradient method: theory and insights.Advances in neural information processing systems, 27, 2014

    2014

  30. [38]

    Continuous-time analysis of accelerated gradient methods via conservation laws in dilated coordinate systems

    Jaewook J Suh, Gyumin Roh, and Ernest K Ryu. Continuous-time analysis of accelerated gradient methods via conservation laws in dilated coordinate systems. InInternational Conference on Machine Learning, pages 20640–20667. PMLR, 2022

    2022

  31. [39]

    Taghvaei and P.-G

    A. Taghvaei and P.-G. Mehta. Accelerated flow for probability distributions.arXiv preprint arXiv:1901.03317, 2019

    1901 arXiv

  32. [40]

    The exponentially convergent trapezoidal rule

    Lloyd N Trefethen and JAC Weideman. The exponentially convergent trapezoidal rule. SIAM review, 56(3):385–458, 2014

    2014

  33. [41]

    Wang and W Li

    Y. Wang and W Li. Accelerated information gradient flow.arXiv preprint arXiv:1909.02102, 2020

    1909 arXiv

  34. [42]

    Wen and W

    Z. Wen and W. Yin. A feasible method for optimization with orthogonality constraints. Mathematical Programming, 142:397–434, 2013

    2013

  35. [43]

    A variational perspective on accelerated methods in optimization

    Andre Wibisono, Ashia C Wilson, and Michael I Jordan. A variational perspective on accelerated methods in optimization. proceedings of the National Academy of Sciences, 113(47):E7351–E7358, 2016

    2016

  36. [44]

    null speed / constant speed

    Abbas K Zghier.The use of differential equations in optimization. PhD thesis, Loughborough University, 1981. Appendix A — 1D exponential map: exact solution, descent, and limits We illustrate, in one dimension, the behavior of the exponential reparameterization for the NNLS mo...

    1981

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.