Pith. sign in

REVIEW 2 major objections 5 minor 31 references

Mirror Descent on Riemannian Manifolds

T0 review · 2 major / 5 minor · reviewed 2026-07-13 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Mirror descent extends to Riemannian manifolds by local reparameterization, with non-asymptotic rates and a scalable Stiefel algorithm.

desk verdict Solid local reparameterization framework that recovers CGD and gives the first non-asymptotic rates for this style of Riemannian MD, plus a usable block-stochastic Stiefel method; the rates are local by design and the experiments are thin but not load-bearing. read the letter →

arxiv 2603.17527 v2 pith:JWVRD3L4 submitted 2026-03-18 stat.ML cs.LGmath.OC

Jiaxin Jiang , Lei Shi , Jiyuan Tan This is my paper
classification stat.MLcs.LGmath.OC MSC 65K0590C0690C30
keywords mirrordescentRiemannianoptimizationreparameterizationnon-asymptoticconvergenceStiefelmanifoldCurvilinearGradientstochastic
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Mirror descent is a first-order method that adapts to problem geometry through a potential function, but it has remained largely undeveloped on Riemannian manifolds. This paper fills that gap by treating mirror descent as gradient descent after a local change of coordinates: at each step a local diffeomorphism reparameterizes a neighborhood of the current point, a retraction-based gradient step is taken in the dual space, and the iterate is mapped back. Under standard smoothness and bounded-gradient assumptions, and when iterates stay inside a compact geodesically convex set, the resulting Riemannian Mirror Descent attains the familiar non-asymptotic rates—O(1/T) for the average squared gradient norm in the nonconvex case and O(1/T) objective error under geodesic convexity—together with matching stochastic rates. On the Stiefel manifold the same construction recovers Curvilinear Gradient Descent and yields a new block-randomized stochastic variant that inverts only small linear systems, making large-scale orthogonal-constrained problems practical.

What carries the argument

Local reparameterization: at iterate x_t a diffeomorphism φ_t maps a geodesic ball around x_t onto a simpler manifold M_t; a retraction gradient step is performed on M_t and mapped back by φ_t^{-1}. This construction simultaneously recovers classical mirror descent, geodesic gradient descent, and Cayley-transform Curvilinear Gradient Descent.

What would settle it

On a concrete Stiefel problem (e.g., linear eigenvalue or orthogonal Procrustes) with known optimum, run RMD/SCGD with the stated constant step-size and check whether the average squared Riemannian gradient or the objective gap decays at the predicted 1/T or 1/T^{1/3} rate; a clear asymptotic plateau would refute the claims.

Watch

Extended reading notes

Core claim

The authors show that Euclidean mirror descent can be realized as gradient descent under a suitable reparameterization, and that the same idea works on a general Riemannian manifold by replacing the global mirror map with a sequence of local diffeomorphisms. The resulting Riemannian Mirror Descent (and its stochastic counterpart) possess non-asymptotic convergence guarantees that match the classical Euclidean rates, and specialize on the Stiefel manifold to Curvilinear Gradient Descent and a scalable stochastic extension.

Load-bearing premise

All iterates and a minimizer must lie inside one compact geodesically convex set on which the function behaves well; without that local pocket the global rates do not hold.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper proposes a Riemannian Mirror Descent (RMD) framework that generalizes Euclidean mirror descent to Riemannian manifolds by local reparameterizations (local diffeomorphisms φ_t and retractions on the reparameterized spaces). It also introduces a stochastic variant (SRMD). Under Assumptions 1–4 (L_Φ-regularity of the induced retraction, iterates confined to a compact geodesically convex set A containing a minimizer, L-smoothness, and bounded gradients), Theorems 1–2 give non-asymptotic rates: O(1/T) for average squared gradient norm (nonconvex) and for objective gap (geodesically convex) in the deterministic case, and the corresponding O(1/√T) and O(1/T^{1/3}) rates in the stochastic case. Specializing the framework to the Stiefel manifold recovers Curvilinear Gradient Descent (CGD) of Wen & Yin and yields a new Stochastic Curvilinear Gradient Descent (SCGD) algorithm based on random block-diagonal skew-symmetric approximations. Numerical experiments on linear eigenvalue and orthogonal Procrustes problems compare SCGD to CGD.

Significance. If the results hold as stated, the paper supplies a clean reparameterization-based template that unifies geodesic gradient descent and classical MD, recovers CGD as an exact special case, and produces a practical stochastic extension (SCGD) for large-p Stiefel problems. The non-asymptotic rates under standard first-order assumptions fill a documented gap relative to prior continuous-time or asymptotic interpretations of MD on manifolds. The proofs are elementary descent lemmas plus a standard potential-function argument (A_t = t^{2}); the Stiefel specialization is explicit and the SCGD construction is of independent algorithmic interest. The main limitation is locality (compact A), which the authors already flag in Remark 2.

major comments (2)
  1. Assumption 2 and Remark 2 make the analysis local: all iterates and a minimizer must lie in a single compact geodesically convex set A on which L-smoothness (and geodesic convexity for the O(1/T) gap) hold. Theorems 1–2 are therefore local/tail rates rather than global rates on general complete manifolds. The manuscript should state this limitation more prominently in the abstract and introduction (e.g., “local non-asymptotic rates under a compact geodesically convex container”) so that the claims match the hypotheses.
  2. Section 5 experiments (Tables 1–2) report only 5-run averages of wall-clock time and final error, with no standard deviations, no iteration counts, and no comparison against standard Riemannian baselines (RGD, RCG, or variance-reduced methods). For SCGD the free parameter K is fixed by a heuristic (⌊n/300⌋) without ablation. These omissions make it hard to assess whether the reported speed-ups are robust or statistically meaningful; at least error bars and one additional baseline are needed to support the claim that SCGD “effectively addresses large-scale manifold optimization.”
minor comments (5)
  1. In the proof of Theorem 1 (geodesically convex case) the potential uses A_t = t^{2} and a_t = 2t+1; the resulting O(1/(ηT)) rate is correct under the stated step-size restriction, but the dependence on diam(A) and C_d should be written explicitly once for readability.
  2. Lemma 2 and Definition 6: the L_Φ-regularity condition is standard; a short remark relating it to the usual second-order retraction error bound would help readers familiar with Absil et al.
  3. Example 3 recovers CGD exactly via the Cayley reparameterization; the extension from St(n,n) to St(n,p) by completing to an orthogonal matrix is sketched only briefly—one sentence clarifying that the first p columns of the updated ¯X yield the St(n,p) iterate would remove any ambiguity.
  4. Typos / notation: “Curvilinear Gradient Descent (CGD) method proposed in [26]” is fine, but the MSC codes and keyword list are standard; a few places write “∥·∥_x” inconsistently with the metric notation g_x. Algorithm 2 line “(I_n + η_t/2 W_k)(I_n − η_t W_k/2)^{-1}” should match the order used in (3).
  5. Related-work paragraph: the continuous-time interpretations of Amid & Warmuth and Gunasekar et al. are cited; a brief sentence distinguishing the present discrete non-asymptotic rates from those continuous-time equivalences would strengthen the contribution claim.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: RMD rates are standard descent lemmas under explicit local assumptions; CGD recovery is an exact specialization, not a redefinition.

full rationale

The paper's central claims (Theorems 1–2) are non-asymptotic rates for RMD/SRMD under Assumptions 1–4. The derivation chain is self-contained: Lemma 2 bounds retraction vs. exponential map from L_Φ-regularity; Lemmas 3–4 give one-step progress from L-smoothness and bounded gradients; the potential A_t = t^{2} then yields the O(1/T) (deterministic) and O(1/√T), O(1/T^{1/3}) (stochastic) rates by the usual telescoping argument. No parameter is fitted to the target quantity and then re-presented as a prediction. Example 3 recovers CGD by choosing the Cayley reparameterization and Euclidean retraction on skew-symmetric matrices; the update (3) is algebraically identical to [26], which is an explicit specialization of the abstract framework rather than a circular redefinition. Self-citations (e.g., to [26] for CGD and SMW) are not load-bearing for the rate proofs. Assumption 2 (compact geodesically convex container A) is stated openly and limits the scope of the convex rates, but that is a locality caveat, not circularity. Score 0.

Assumptions & free parameters 2 free parameters · 5 assumptions · 2 invented entities

The central rates rest on standard Riemannian calculus plus four explicit analytic assumptions (retraction regularity, compact geodesically convex container A, L-smoothness, bounded gradient) and the usual unbiased/bounded stochastic oracle. Free parameters are only algorithmic (constant η, block count K). No new physical entities are postulated; RMD/SRMD/SCGD are algorithmic constructions.

free parameters (2)
  • constant step size η = theory: η < min{1/(L_Φ G/2 + L_f + L_f L_Φ²), 2/G, r/G}; experiments 1e-3 or 1e-2
    Chosen by hand inside an explicit upper bound involving L_Φ, L_f, G, r; experiments further pick 10^{-3} or 10^{-2} depending on K.
  • number of blocks K (SCGD) = ⌊n/300⌋
    Heuristic partition size ⌊n/300⌋ used in tables; controls bias–variance and parallel cost of the block-diagonal estimator.
assumptions (5)
  • standard math Manifold is connected and complete so Exp is defined on all of TM (Hopf–Rinow).
    Stated at the opening of §2; used for geodesics and distance.
  • domain assumption Assumption 1: the pulled-back map is an L_Φ-regular retraction within radius r.
    Controls the Exp–retraction gap in Lemmas 2–4; load-bearing for all rates.
  • domain assumption Assumption 2: iterates and a minimizer lie in a compact geodesically convex set A.
    Enables diameter bounds and local geodesic convexity; Remark 2 acknowledges it is essentially local.
  • domain assumption Assumptions 3–4: f is L_f-smooth and ||∇f|| ≤ G everywhere on M.
    Standard first-order hypotheses used in every one-step inequality.
  • domain assumption Stochastic oracle is unbiased, almost-surely bounded by G, and has finite variance σ².
    Required for Theorem 2; stated immediately before the theorem.
invented entities (2)
  • Riemannian Mirror Descent (RMD) / Stochastic RMD template independent evidence
    purpose: Local-diffeomorphism + dual retraction update that generalizes Euclidean MD to manifolds.
    Algorithmic framework, not a physical object; independent evidence is the recovery of known methods and the proved rates.
  • Stochastic Curvilinear Gradient Descent (SCGD) independent evidence
    purpose: Block-diagonal random estimator of the skew-symmetric generator for large-p Stiefel problems.
    New practical specialization; falsifiable via runtime/accuracy experiments (Tables 1–2).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Mirror Descent on Riemannian Manifolds." pith.science (2026). https://pith.science/paper/JWVRD3L4

@misc{pith2026260317527,
  author       = {Pith},
  title        = {Pith review of: Mirror Descent on Riemannian Manifolds},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JWVRD3L4}},
  note         = {Machine review of arXiv:2603.17527}
}
read the original abstract

Mirror Descent (MD) is a scalable first-order method widely used in large-scale optimization, with applications in image processing, policy optimization, and neural network training. This paper generalizes MD to optimization on Riemannian manifolds. In particular, we develop a Riemannian Mirror Descent (RMD) framework via reparameterization and further propose a stochastic variant of RMD. We also establish non-asymptotic convergence guarantees for both RMD and stochastic RMD. As an application to the Stiefel manifold, our RMD framework reduces to the Curvilinear Gradient Descent (CGD) method proposed in [26]. Moreover, when specializing the stochastic RMD framework to the Stiefel setting, we obtain a stochastic extension of CGD, which effectively addresses large-scale manifold optimization problems.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

31 extracted references

  1. [1]

    Momentum improves optimization on Riemannian manifolds

    Foivos Alimisis, Antonio Orvieto, Gary Becigneul, and Aurelien Lucchi. Momentum improves optimization on Riemannian manifolds. InInternational Conference on Artificial Intelligence and Statistics, pages 1351–1359. PMLR, 2021

    2021

  2. [2]

    Reparameterizing mirror descent as gradient descent

    Ehsan Amid and Manfred KK Warmuth. Reparameterizing mirror descent as gradient descent. Advances in Neural Information Processing Systems, 33:8430–8439, 2020

    2020

  3. [3]

    Unitary evolution recurrent neural networks

    Martin Arjovsky, Amar Shah, and Yoshua Bengio. Unitary evolution recurrent neural networks. InInternational Conference on Machine Learning, pages 1120–1128. PMLR, 2016

    2016

  4. [4]

    Can we gain more from orthogonality reg- ularizations in training deep networks?Advances in Neural Information Processing Systems, 31, 2018

    Nitin Bansal, Xiaohan Chen, and Zhangyang Wang. Can we gain more from orthogonality reg- ularizations in training deep networks?Advances in Neural Information Processing Systems, 31, 2018

    2018

  5. [5]

    Mirror descent and nonlinear projected subgradient methods for convex optimization.Operations Research Letters, 31(3):167–175, 2003

    Amir Beck and Marc Teboulle. Mirror descent and nonlinear projected subgradient methods for convex optimization.Operations Research Letters, 31(3):167–175, 2003

    2003

  6. [6]

    Iteration-complexity of gradient, subgradient and proximal point methods on Riemannian manifolds.Journal of Optimization Theory and Applications, 173(2):548–562, 2017

    Glaydston C Bento, Orizon P Ferreira, and Jefferson G Melo. Iteration-complexity of gradient, subgradient and proximal point methods on Riemannian manifolds.Journal of Optimization Theory and Applications, 173(2):548–562, 2017

    2017

  7. [7]

    Lev M Bregman. The relaxation method of finding the common point of convex sets and its ap- plication to the solution of problems in convex programming.USSR Computational Mathematics and Mathematical Physics, 7(3):200–217, 1967

    1967

  8. [8]

    Riemannian dictionary learning and sparse coding for positive definite matrices.IEEE Transactions on Neural Networks and Learning Systems, 28(12):2859– 2871, 2016

    Anoop Cherian and Suvrit Sra. Riemannian dictionary learning and sparse coding for positive definite matrices.IEEE Transactions on Neural Networks and Learning Systems, 28(12):2859– 2871, 2016

    2016

Show all 31 references
  1. [9]

    Negative curvature obstructs acceleration for strongly geodesically convex optimization, even with exact first-order oracles

    Christopher Criscitiello and Nicolas Boumal. Negative curvature obstructs acceleration for strongly geodesically convex optimization, even with exact first-order oracles. InConference on Learning Theory, pages 496–542. PMLR, 2022

    2022

  2. [10]

    Composite objective mirror descent

    John C Duchi, Shai Shalev-Shwartz, Yoram Singer, and Ambuj Tewari. Composite objective mirror descent. InConference on Learning Theory, volume 10, pages 14–26. Citeseer, 2010

    2010

  3. [11]

    OUP Oxford, 2004

    John C Gower and Garmt B Dijksterhuis.Procrustes Problems, volume 30. OUP Oxford, 2004

    2004

  4. [12]

    Mirrorless mirror descent: A natural derivation of mirror descent

    Suriya Gunasekar, Blake Woodworth, and Nathan Srebro. Mirrorless mirror descent: A natural derivation of mirror descent. InInternational Conference on Artificial Intelligence and Statistics, pages 2305–2313. PMLR, 2021

    2021

  5. [13]

    Convergence of online mirror descent.Applied and Computa- tional Harmonic Analysis, 48(1):343–373, 2020

    Yunwen Lei and Ding-Xuan Zhou. Convergence of online mirror descent.Applied and Computa- tional Harmonic Analysis, 48(1):343–373, 2020

    2020

  6. [14]

    Implicit bias of gradient descent on reparametrized models: On equivalence to mirror descent.Advances in Neural Information Processing Systems, 35:34626–34640, 2022

    Zhiyuan Li, Tianhao Wang, Jason D Lee, and Sanjeev Arora. Implicit bias of gradient descent on reparametrized models: On equivalence to mirror descent.Advances in Neural Information Processing Systems, 35:34626–34640, 2022

    2022

  7. [15]

    Robust stochastic approximation approach to stochastic programming.SIAM Journal on Optimization, 19(4):1574– 1609, 2009

    Arkadi Nemirovski, Anatoli Juditsky, Guanghui Lan, and Alexander Shapiro. Robust stochastic approximation approach to stochastic programming.SIAM Journal on Optimization, 19(4):1574– 1609, 2009

    2009

  8. [16]

    Wiley-Interscience, 1983

    Arkadi Semenoviˇ c Nemirovski and David Borisovich Yudin.Problem Complexity and Method Efficiency in Optimization. Wiley-Interscience, 1983

    1983

  9. [17]

    The information geometry of mirror descent.IEEE Transactions on Information Theory, 61(3):1451–1457, 2015

    Garvesh Raskutti and Sayan Mukherjee. The information geometry of mirror descent.IEEE Transactions on Information Theory, 61(3):1451–1457, 2015. 17

    2015

  10. [18]

    Riemannian stochastic variance reduced gra- dient algorithm with retraction and vector transport.SIAM Journal on Optimization, 29(2):1444– 1472, 2019

    Hiroyuki Sato, Hiroyuki Kasai, and Bamdev Mishra. Riemannian stochastic variance reduced gra- dient algorithm with retraction and vector transport.SIAM Journal on Optimization, 29(2):1444– 1472, 2019

    2019

  11. [19]

    Revisit first-order methods for geodesically convex optimization.arXiv preprint arXiv:2504.06814, 2025

    Yunlu Shu, Jiaxin Jiang, Lei Shi, and Tianyu Wang. Revisit first-order methods for geodesically convex optimization.arXiv preprint arXiv:2504.06814, 2025

    2025

  12. [20]

    Mirror descent of Hopfield model

    Hyungjoon Soh, Dongyeob Kim, Juno Hwang, and Junghyo Jo. Mirror descent of Hopfield model. Neural Computation, 35(9):1529–1542, 2023

    2023

  13. [21]

    Sufficient conditions for non-asymptotic convergence of Riemannian optimisation methods.14th Annual Workshop on Optimization for Machine Learning, 2022

    Vishwak Srinivasan and Ashia Wilson. Sufficient conditions for non-asymptotic convergence of Riemannian optimisation methods.14th Annual Workshop on Optimization for Machine Learning, 2022

    2022

  14. [22]

    Mirror descent max- imizes generalized margin and can be implemented efficiently.Advances in Neural Information Processing Systems, 35:31089–31101, 2022

    Haoyuan Sun, Kwangjun Ahn, Christos Thrampoulidis, and Navid Azizan. Mirror descent max- imizes generalized margin and can be implemented efficiently.Advances in Neural Information Processing Systems, 35:31089–31101, 2022

    2022

  15. [23]

    Riemannian pursuit for big matrix recovery

    Mingkui Tan, Ivor W Tsang, Li Wang, Bart Vandereycken, and Sinno Jialin Pan. Riemannian pursuit for big matrix recovery. InInternational Conference on Machine Learning, pages 1539–

  16. [24]

    Averaging stochas- tic gradient descent on Riemannian manifolds

    Nilesh Tripuraneni, Nicolas Flammarion, Francis Bach, and Michael I Jordan. Averaging stochas- tic gradient descent on Riemannian manifolds. InConference On Learning Theory, pages 650–687. PMLR, 2018

    2018

  17. [25]

    Low-rank matrix completion by Riemannian optimization.SIAM Journal on Optimization, 23(2):1214–1236, 2013

    Bart Vandereycken. Low-rank matrix completion by Riemannian optimization.SIAM Journal on Optimization, 23(2):1214–1236, 2013

    2013

  18. [26]

    A feasible method for optimization with orthogonality constraints

    Zaiwen Wen and Wotao Yin. A feasible method for optimization with orthogonality constraints. Mathematical Programming, 142(1-2):397–434, 2013

    2013

  19. [27]

    Policy optimization with stochastic mirror descent

    Long Yang, Yu Zhang, Gang Zheng, Qian Zheng, Pengfei Li, Jianhang Huang, and Gang Pan. Policy optimization with stochastic mirror descent. InProceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 8823–8831, 2022

    2022

  20. [28]

    Policy mirror descent for regularized reinforcement learning: A generalized framework with linear convergence

    Wenhao Zhan, Shicong Cen, Baihe Huang, Yuxin Chen, Jason D Lee, and Yuejie Chi. Policy mirror descent for regularized reinforcement learning: A generalized framework with linear convergence. SIAM Journal on Optimization, 33(2):1061–1091, 2023

    2023

  21. [29]

    Hongchao Zhang and William W. Hager. A nonmonotone line search technique and its application to unconstrained optimization.SIAM Journal on Optimization, 14(4):1043–1056, January 2004

    2004

  22. [30]

    Riemannian svrg: Fast stochastic optimization on Riemannian manifolds.Advances in Neural Information Processing Systems, 29, 2016

    Hongyi Zhang, Sashank J Reddi, and Suvrit Sra. Riemannian svrg: Fast stochastic optimization on Riemannian manifolds.Advances in Neural Information Processing Systems, 29, 2016

    2016

  23. [31]

    First-order methods for geodesically convex optimization

    Hongyi Zhang and Suvrit Sra. First-order methods for geodesically convex optimization. In Conference on Learning Theory, pages 1617–1638. PMLR, 2016. 18

    2016

Pith tools

Reviewed July 13, 2026 · model on record in the stance chip above.