Pith. sign in

REVIEW 2 major objections 5 minor 174 references

Causal Optimizer Interaction Calculus: Hidden Geometric Relaxation and Identifiable Interventions

T0 review · 2 major / 5 minor · reviewed 2026-07-14 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Optimizer interventions have a complete causal calculus: minimal pathwise state, Möbius pure effects, and a five-term law that transfers hidden relaxation to any smooth readout.

desk verdict Solid experimental outer layer for optimizer interventions: clean pathwise/Möbius gauge, a correct five-term readout transfer, and an unusually tight reduced-value factorial closure—general nonconvex readout still untested. read the letter →

arxiv 2607.07206 v2 pith:GBXMSZ43 submitted 2026-07-08 cs.LG math.OC

Zavier Li This is my paper · ORCID
classification cs.LGmath.OC
keywords optimizercalculuscausalinterventionsMöbiuseffectshiddenrelaxationreadouttransferfactorialdesignresponseclassesidentifiability
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Optimizer experiments see only responses to configurations, not the hidden mechanisms that produced them. This paper builds a causal interaction calculus that separates three layers: a pathwise minimal state under a fixed innovation coupling, a Möbius decomposition of pure interventional effects over lower-finite posets, and a linear incidence operator that is the complete observational gauge for any finite effect support and design. That gauge yields exact identifiability, sharp quotient stability, held-out predictions, and noiseless configuration complexity equal to the support size. Taking smooth hidden-state relaxation as a structural input, the paper proves that any smooth update or trace readout inherits an explicit five-term interaction through the first and second hidden responses; Boolean effects remain exact integrals of that continuous curvature and are therefore accessible to factorial designs, even though general readouts need not inherit the negative-semidefinite sign of the reduced optimal value. Gaussian quotient minimax risk, exact confidence sets, residualized sector tests, misspecification decomposition, certified decisions, and optimal replication complete the statistical layer. A controlled strongly convex logistic experiment closes the reduced-value chain to high numerical precision; neural audits show that declared geometric response classes remain informative in nonconvex training.

What carries the argument

The incidence (design) operator XD,K together with the five-term observable-readout transfer: ∂²uiuj FR = Ruiuj + Ruiσ vj + Rσ uj vi + Rσσ[vi,vj] + Rσ wij. The operator is the complete observational gauge; the transfer carries hidden Schur curvature to experimentally chosen updates and traces so that Boolean effects remain integrable and identifiable.

What would settle it

A factorial experiment on a smooth optimizer whose Boolean pair effects disagree with independently integrated five-term curvature of a declared readout, or a finite support and design whose incidence matrix has full column rank yet fails to recover held-out configuration responses within noise.

Watch

Extended reading notes

Core claim

Under a fixed innovation coupling every finite-horizon innovation-driven optimizer admits a behaviorally minimal pathwise realization; for any finite effect support and design the incidence operator is the complete observational gauge with exact identifiability and noiseless complexity equal to support size; and any smooth readout FR(u)=R(u,σ⋆(u)) inherits an explicit five-term mixed derivative through first and second hidden responses whose Boolean pair effect is the exact integral of that curvature, with no universal sign beyond the reduced-value case.

Load-bearing premise

Hidden relaxation must have a unique interior minimizer with positive-definite vertical Hessian, and the experiment must fix the innovation coupling so that pathwise predictive equivalence is a well-defined congruence.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper develops a causal calculus for optimizer experiments that separates pathwise realization under fixed innovation coupling, Möbius pure-effect coordinates on lower-finite intervention posets, and incidence-based observational identification. Theorem 1 gives a behaviorally minimal pathwise/unifilar state, unique pure effects, the complete design gauge, exact identifiability when rank equals support size, sharp quotient stability, held-out falsification, and noiseless configuration complexity |K|. Taking the companion P1 Schur law as input, Theorem 3 transfers first and second hidden responses to an arbitrary smooth readout FR(u)=R(u,σ⋆(u)), yielding an explicit five-term mixed derivative whose Boolean pair effect is the integral of that curvature and need not inherit the reduced-value negative-semidefinite sign. Theorem 4 supplies Gaussian quotient minimax risk, residualized sector tests, misspecification decomposition, certified decisions, and exact continuous A-optimal replication. A controlled 65-dimensional strongly convex logistic experiment closes the reduced-value chain (Boolean vs independent curvature integrals within 4.21e-11; nine held-out continuous intensities within 8.88e-13; Gaussian coverage/power as predicted; real-minibatch rejection of order-two support). Neural trace audits test geometric response-class residuals in nonconvex training.

Significance. If the package holds, the paper supplies a usable experimental outer layer for optimizer mechanism studies: exact incidence gauges and configuration complexity, a transfer law that predicts which observable effects factorial designs can identify, and finite-sample inference/design formulas specialized to intervention masks. Strengths that raise the contribution above a pure restatement of classical incidence algebra and Schur sensitivity include: full proofs in Appendices A–D; an independent Boolean-versus-quadrature closure with machine-auditable artifacts and sub-1e-10 agreement; exact Gaussian risk/coverage Monte Carlo confirmation; and explicit separation of response-class residual tests from causal attribution. The five-term readout law is the main novel interface between hidden relaxation geometry and the traces practitioners actually log.

major comments (2)
  1. The abstract and Contributions package Theorem 1 with Theorem 3 as the experimental interface for 'arbitrary smooth update or trace readouts.' Appendix B.4 derives the five-term law correctly under Assumption 1, but §§8.1–8.5 close only the reduced-value special case F(u)=min_w E(w,u) on a globally strongly convex logistic model. §8.6 and the Discussion explicitly state that the neural audits lack factorial masks and the hidden first/second responses required by Eq. (13). The load-bearing claim that general update/trace readouts inherit identifiable Boolean effects via the same calculus therefore rests on an untested transfer step. Either (i) add a factorial campaign that measures an actual update readout together with the quantities in Eqs. (12)–(14), or (ii) restate the abstract/contributions so that the validated claim is the reduced-value chain plus the analytical transfer theorem, w
  2. Assumption 1 (unique interior minimizer, positive-definite vertical Hessian) and the fixed innovation coupling of §3.1/Theorem 1(i) are load-bearing for both the five-term transfer and the canonical pathwise minimal state. The Discussion notes nonsmooth/multi-valued/hysteretic cases require generalized derivatives, but the manuscript does not quantify how often these fail for standard adaptive optimizers (Adam-style second-moment state, Muon-style operators, clipping). A short scope paragraph or counter-example class would make the boundary of Theorems 1 and 3 precise rather than leaving it as a regularity caveat.
minor comments (5)
  1. Notation for the Hilbert-valued response F(a) versus scalar readouts Fφ and FR is introduced carefully in §3.1 but is easy to lose later; a one-line glossary or consistent subscript convention would help.
  2. Table 1 and Figure 1 report absolute errors of order 1e-11–1e-15; stating the floating-point / solver tolerance used for the Newton solves would make the agreement easier to interpret.
  3. The neural audit tables (Tables 2–6) mix accuracy and TGER residuals; a clearer caption that these are response-class diagnostics, not optimizer rankings, would reduce misreading (the Discussion already says this, but the tables themselves do not).
  4. Cross-references to the companion P1/P2/P4 papers are frequent; a short dependency diagram or table of which structural lemmas are imported versus proved here would improve self-containment for readers who have not read the series.
  5. In §8.3 the Gaussian channel uses σa=0.004(1+0.2|a|); a one-sentence justification for this mild heteroscedasticity model would be useful.

Circularity Check

1 steps flagged · score 2.0 of 10

Minor concurrent same-author self-citation for the P1 Schur structural input; Theorem 1, the five-term readout transfer, incidence gauge, and factorial numerics are independently derived or checked, not forced by fit or definition.

  1. self citation load bearing [Abstract; §1; §5 Theorem 2 (restated from P1); series position §2]
    "Building on this structural law, we prove an observable-readout transfer theorem... The companion P1 paper already answers the structural reduced-value question... it proves that the reduced intervention Hessian is D²_uu Ē = −G*H⁻¹G, and that Boolean reduced-value contrasts integrate this curvature (Li, 2026b). ... Taking the P1 Schur–Möbius theorem as a structural input, we prove an observable readout interaction law."

    The abstract and framing present inverse-stiffness interaction curvature as an established structural law imported from concurrent same-author P1 (and path-space P2), then build the experimental calculus on top. That is a self-citation chain for the reduced-value sign/Gram story. It is only minor/non-load-bearing here because Appendix B fully reproduces the P1 reduction proof, Theorem 3 itself needs only Assumption 1 + chain rule (not the P1 Gram sign), and the factorial numerics independently recompute both sides of the identity rather than fitting free parameters from P1.

full rationale

The load-bearing novel layers do not reduce to their inputs by construction. Theorem 1 (pathwise Nerode-style quotient, Möbius expansion, incidence kernel, exact noiseless complexity |K|) is proved from first principles in Appendix A under the fixed-innovation protocol; Möbius inversion and rank-nullity are classical, not renamed empirical patterns. Theorem 3 is a local chain-rule / IFT calculation (Appendix B.4) under Assumption 1: FR(u)=R(u,σ⋆(u)) yields the explicit five-term mixed derivative whose Boolean pair effect is the FTC integral; this is not defined in terms of the experimental targets. Theorem 2 (reduced-value Schur Gram −G*H−1G and Boolean integral) is attributed to concurrent same-author P1, which is the only self-citation of note, but Appendix B reproduces the full proof so the present manuscript is mathematically self-contained rather than resting on an unverified external uniqueness claim. The digits factorial closure computes Boolean corner effects and independent Gauss–Legendre curvature integrals on separate paths (“Neither path uses the output of the other”) and reports numerical agreement; that is consistency under a declared strongly convex additive model, not a parameter fitted on a subset and re-labeled as prediction. Held-out continuous intensities, Gaussian coverage/power Monte Carlo, and minibatch bootstrap rejection of order-two support are likewise external checks of the incidence/inference layer. Neural audits are explicitly scoped as response-class sensitivity, not as circular confirmation of Theorem 3. No self-definitional loop, no fitted-input-as-prediction of the central claims, and no uniqueness theorem imported solely to forbid alternatives. Score 2 reflects only the concurrent P1 framing of the structural Schur input, which is not load-bearing for the paper’s own proofs or experiments.

Assumptions & free parameters 5 free parameters · 6 assumptions · 3 invented entities

Universality rests on protocol assumptions (fixed innovation coupling, lower-finite poset, standard Borel histories) and standard incidence algebra. Smooth interaction rests on unique positive-definite hidden relaxation (Assumption 1) imported structurally from P1. Inference rests on finite-dimensional linear Gaussian experiments with known V≻0. Experiments add hand-chosen regularization, intervention scales, and design budgets that affect numerical validation but not the abstract theorems. Invented entities are mostly definitional coordinate systems (effect support K, response classes MF,B) rather than new physical mediators.

free parameters (5)
  • ridge λ = 0.15
    Logistic energy uses λ=0.15 to enforce strong convexity Eww ⪰ 0.15 I; chosen for the controlled experiment, not derived.
  • intervention loss scale = 0.75
    Additive interventions enter as 0.75 ∑ ui Δi(w); scale is a design choice for the digits cube.
  • Gaussian channel noise model σa=0.004(1+0.2|a|) = 0.004 base, 0.2 slope
    Heteroscedastic noise law used for power calibration and 100k campaigns; pilot-driven experimental parameter.
  • replication budgets (3780 main / 720 held-out) = 3780 / 720
    Scaled from pilot (420/80) by first integer reaching predeclared 0.95 power; allocation affects empirical confirmation, not theorems.
  • bounded-diagonal audit box [1e-6, 1] = [1e-6, 1]
    Response-class family bounds for neural TGER audits; hand-set geometry constraints.
assumptions (6)
  • domain assumption Fixed innovation coupling: exogenous randomness is coupled across configurations so pathwise predictive equivalence is well-defined.
    Theorem 1(i) and §3.1; without it only distributional realization is available, which the paper explicitly does not claim.
  • standard math Lower-finite intervention poset with Möbius inversion of expected responses.
    Classical incidence algebra; used in eq. (2) and Theorem 1(ii).
  • domain assumption Assumption 1: Cr+1 energy, local uniform inf-compactness, unique interior minimizer, H=D²σσE ≻ 0.
    Required for Theorems 2–3 and the five-term transfer; stated before Theorem 2.
  • domain assumption Finite-dimensional response with known positive-definite observation covariance for exact Gaussian quotient minimax, χ² sets, and A-optimal replication.
    Theorem 4 setup; paper flags estimated covariance and dependent traces as outside exact claims.
  • domain assumption P1 Schur reduced-value law D²uu Ē = −G* H^{-1} G as structural input.
    Imported from companion Li 2026b; restated not re-derived as primary novelty.
  • ad hoc to paper Affine intervention entry before reduction for the negative Gram law and analytical examples.
    Eq. (8); sufficient for sign law and digits model, not necessary for general readout transfer.
invented entities (3)
  • Behaviorally minimal pathwise/unifilar optimizer state under fixed innovation coupling independent evidence
    purpose: Canonical causal realization for finite-horizon configuration responses.
    Quotient of extended histories by pathwise predictive equivalence; standard minimal-realization idea specialized to the protocol.
  • Observable-readout five-term interaction transfer independent evidence
    purpose: Map hidden first/second responses to arbitrary smooth update or trace scalars.
    Theorem 3; falsifiable via factorial integrals vs independent hidden-model predictions when H,G measured.
  • Geometric response class MF,B with TGER residual
    purpose: Passive membership test for declared positive maps along a trace under variation budget B.
    Definitional audit set; informative in neural tables but not a new physical mechanism.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Causal Optimizer Interaction Calculus: Hidden Geometric Relaxation and Identifiable Interventions." pith.science (2026). https://pith.science/paper/GBXMSZ43

@misc{pith2026260707206,
  author       = {Pith},
  title        = {Pith review of: Causal Optimizer Interaction Calculus: Hidden Geometric Relaxation and Identifiable Interventions},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GBXMSZ43}},
  note         = {Machine review of arXiv:2607.07206}
}
read the original abstract

Optimizer experiments observe responses to algorithmic configurations without uniquely revealing hidden mechanisms. We develop a causal optimizer interaction calculus that separates pathwise realization, Mobius decomposition, and experimental identification. Under a fixed innovation coupling, every finite-horizon innovation-driven optimizer admits a behaviorally minimal pathwise realization. For any finite effect support and intervention design, an incidence operator gives the complete observational gauge, exact identifiability, sharp quotient stability, held-out predictions, and exact noiseless configuration complexity. Smooth hidden relaxation generates interactions through inverse hidden-state stiffness. Building on this structural law, we prove an observable-readout transfer theorem: arbitrary smooth update or trace readouts inherit an explicit five-term interaction through first and second hidden responses. Unlike the reduced optimal value, a general readout has no universal interaction sign. Its Boolean effects remain exact integrals of continuous interaction curvature and can therefore be identified by factorial interventions. We also derive Gaussian quotient minimax risk, exact confidence sets and tests, misspecification decomposition, certified downstream decisions, and optimal replication. A controlled real-data experiment on a 65-dimensional strongly convex logistic model validates the complete reduced-value chain. Boolean effects and independently integrated curvature agree within 4.21e-11, while nine held-out continuous intensities agree within 8.88e-13. Gaussian campaigns attain the predicted coverage and power, and 4,500 real-minibatch observations reject an order-two interaction model. Neural trace audits provide complementary evidence that the declared response classes remain informative in nonconvex training.

Figures

Figures reproduced from arXiv: 2607.07206 by the authors.

Figure 1
Figure 1. Boolean pair effects and independently integrated Schur curvature. The largest absolute discrepancy [PITH_FULL_IMAGE:figures/full_fig_p013_1.png] view at source ↗
Figure 2
Figure 2. Convergence of finite-horizon pair effects to the quasistatic effects. The maximum error decreases [PITH_FULL_IMAGE:figures/full_fig_p014_2.png] view at source ↗
Figure 3
Figure 3. Response-class sensitivity. Layer scalar is nested inside bounded diagonal geometry, and the [PITH_FULL_IMAGE:figures/full_fig_p016_3.png] view at source ↗

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

174 extracted references · 5 canonical work pages

  1. [1]

    A Differential Equation for Modeling

    Su, Weijie and Boyd, Stephen and Cand. A Differential Equation for Modeling. Journal of Machine Learning Research , year =

  2. [2]

    Proceedings of the National Academy of Sciences , year =

    A Variational Perspective on Accelerated Methods in Optimization , author =. Proceedings of the National Academy of Sciences , year =

  3. [3]

    Advances in Neural Information Processing Systems , year =

    Accelerated Mirror Descent in Continuous and Discrete Time , author =. Advances in Neural Information Processing Systems , year =

  4. [4]

    Neural Computation , year =

    Natural Gradient Works Efficiently in Learning , author =. Neural Computation , year =

  5. [5]

    IEEE Transactions on Information Theory , year =

    The Information Geometry of Mirror Descent , author =. IEEE Transactions on Information Theory , year =. doi:10.1109/TIT.2015.2391243 , url =

    doi:10.1109/tit.2015.2391243 2015

  6. [6]

    Proceedings of the 24th International Conference on Machine Learning , year =

    Information-Theoretic Metric Learning , author =. Proceedings of the 24th International Conference on Machine Learning , year =. doi:10.1145/1273496.1273523 , url =

    doi:10.1145/1273496.1273523

  7. [7]

    Optimizing Neural Networks with

    Martens, James and Grosse, Roger , booktitle =. Optimizing Neural Networks with. 2015 , series =

    2015

  8. [8]

    Grosse, Roger and Martens, James , booktitle =. A. 2016 , series =. 1602.01407 , archivePrefix =

    arXiv 2016

Show all 174 references
  1. [9]

    Fast Approximate Natural Gradient Descent in a

    George, Thomas and Laurent, C. Fast Approximate Natural Gradient Descent in a. Advances in Neural Information Processing Systems , year =

  2. [10]

    Journal of Machine Learning Research , volume =

    New Insights and Perspectives on the Natural Gradient Method , author =. Journal of Machine Learning Research , volume =. 2020 , url =

    2020

  3. [11]

    Advances in Neural Information Processing Systems , volume =

    Limitations of the Empirical Fisher Approximation for Natural Gradient Descent , author =. Advances in Neural Information Processing Systems , volume =. 2019 , url =

    2019

  4. [12]

    and Pitsianis, Nikos , booktitle =

    Van Loan, Charles F. and Pitsianis, Nikos , booktitle =. Approximation with. 1993 , doi =

    1993

  5. [13]

    Dutilleul, Pierre , journal =. The. 1999 , doi =

    1999

  6. [14]

    On Estimation of Covariance Matrices with

    Werner, Karl and Jansson, Magnus and Stoica, Petre , journal =. On Estimation of Covariance Matrices with. 2008 , doi =

    2008

  7. [15]

    IEEE Transactions on Signal Processing , volume =

    Geodesic Convexity and Covariance Estimation , author =. IEEE Transactions on Signal Processing , volume =. 2012 , doi =

    2012

  8. [16]

    On the Convexity in

    Wiesel, Ami , booktitle =. On the Convexity in. 2012 , doi =

    2012

  9. [17]

    Information Geometry and Asymptotics for

    McCormack, Andrew and Hoff, Peter , year =. Information Geometry and Asymptotics for. 2308.02260 , archivePrefix =

    arXiv

  10. [18]

    2025 , eprint =

    Geodesic Variational Bayes for Multiway Covariances , author =. 2025 , eprint =

    2025

  11. [19]

    Bouchard, Florent and Breloy, Arnaud and Mian, Ammar and Ginolhac, Guillaume , booktitle =. On-line. 2021 , doi =

    2021

  12. [20]

    Journal of Machine Learning Research , year =

    Adaptive Subgradient Methods for Online Learning and Stochastic Optimization , author =. Journal of Machine Learning Research , year =

  13. [21]

    International Conference on Learning Representations , year =

    Adam: A Method for Stochastic Optimization , author =. International Conference on Learning Representations , year =

  14. [22]

    International Conference on Learning Representations , year =

    Decoupled Weight Decay Regularization , author =. International Conference on Learning Representations , year =

  15. [23]

    Proceedings of the 35th International Conference on Machine Learning , year =

    Shampoo: Preconditioned Stochastic Tensor Optimization , author =. Proceedings of the 35th International Conference on Machine Learning , year =

  16. [24]

    and Janson, Lucas , year =

    Morwani, Depen and Shapira, Itai and Vyas, Nikhil and Malach, Eran and Kakade, Sham M. and Janson, Lucas , year =. A New Perspective on. 2406.17748 , archivePrefix =

    arXiv

  17. [25]

    arXiv preprint arXiv:2002.09018 , year =

    Scalable Second Order Optimization for Deep Learning , author =. arXiv preprint arXiv:2002.09018 , year =

    2002 arXiv

  18. [26]

    2023 , eprint =

    A Distributed Data-Parallel PyTorch Implementation of the Distributed Shampoo Optimizer for Training Neural Networks At-Scale , author =. 2023 , eprint =

    2023

  19. [28]

    Machine Learning , year =

    Logarithmic Regret Algorithms for Online Convex Optimization , author =. Machine Learning , year =

  20. [29]

    Proceedings of the 20th International Conference on Machine Learning , year =

    Online Convex Programming and Generalized Infinitesimal Gradient Ascent , author =. Proceedings of the 20th International Conference on Machine Learning , year =

  21. [30]

    Proceedings of the 35th International Conference on Machine Learning , year =

    Adafactor: Adaptive Learning Rates with Sublinear Memory Cost , author =. Proceedings of the 35th International Conference on Machine Learning , year =

  22. [31]

    Advances in Neural Information Processing Systems , year =

    Memory Efficient Adaptive Optimization , author =. Advances in Neural Information Processing Systems , year =

  23. [32]

    2023 , eprint =

    Symbolic Discovery of Optimization Algorithms , author =. 2023 , eprint =

    2023

  24. [33]

    2023 , eprint =

    Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training , author =. 2023 , eprint =

    2023

  25. [34]

    2024 , howpublished =

    Muon: An Optimizer for Hidden Layers in Neural Networks , author =. 2024 , howpublished =

    2024

  26. [35]

    2025 , eprint =

    Practical Efficiency of Muon for Pretraining , author =. 2025 , eprint =

    2025

  27. [36]

    2025 , eprint =

    An Exploration of Non-Euclidean Gradient Descent: Muon and its Many Variants , author =. 2025 , eprint =

    2025

  28. [37]

    Proceedings of the 36th International Conference on Machine Learning , year =

    Efficient Full-Matrix Adaptive Regularization , author =. Proceedings of the 36th International Conference on Machine Learning , year =

  29. [38]

    arXiv preprint arXiv:1912.02928 , year =

    Bregman Dynamics, Contact Transformations and Convex Optimization , author =. arXiv preprint arXiv:1912.02928 , year =

    1912 arXiv

  30. [39]

    arXiv preprint arXiv:2505.12553 , year =

    Hamiltonian Descent Algorithms for Optimization: Accelerated Rates via Randomized Integration Time , author =. arXiv preprint arXiv:2505.12553 , year =

  31. [40]

    arXiv preprint arXiv:2606.17260 , year =

    Accelerated Convex Optimization via Hamiltonian Dynamics with Deterministic Integration Time , author =. arXiv preprint arXiv:2606.17260 , year =

    arXiv

  32. [41]

    arXiv preprint arXiv:2412.02291 , year =

    Conformal Symplectic Optimization for Stable Reinforcement Learning , author =. arXiv preprint arXiv:2412.02291 , year =

    arXiv

  33. [42]

    Journal of Machine Learning Research , year =

    Learning Discretized Neural Networks under Ricci Flow , author =. Journal of Machine Learning Research , year =

  34. [43]

    arXiv preprint arXiv:2509.22362 , year =

    Neural Feature Geometry Evolves as Discrete Ricci Flow , author =. arXiv preprint arXiv:2509.22362 , year =

  35. [44]

    International Conference on Machine Learning , year =

    Revisiting Over-smoothing and Over-squashing Using Ollivier-Ricci Curvature , author =. International Conference on Machine Learning , year =

  36. [45]

    Physical Review Research , year =

    Improving Gradient Methods via Coordinate Transformations: Applications to Quantum Machine Learning , author =. Physical Review Research , year =

  37. [46]

    Journal of Machine Learning Research , year =

    The Z-Gromov-Wasserstein Distance , author =. Journal of Machine Learning Research , year =

  38. [47]

    Optimization Algorithms on Matrix Manifolds , author =

  39. [48]

    International Conference on Learning Representations , year =

    Riemannian Adaptive Optimization Methods , author =. International Conference on Learning Representations , year =

  40. [49]

    2023 , doi =

    An Introduction to Optimization on Smooth Manifolds , author =. 2023 , doi =

    2023

  41. [50]

    An Introduction to Morse Theory , author =

  42. [51]

    2004 , url =

    Convex Optimization , author =. 2004 , url =

    2004

  43. [52]

    SIAM Review , volume =

    Semidefinite Programming , author =. SIAM Review , volume =. 1996 , doi =

    1996

  44. [53]

    Convex Analysis , author =

  45. [54]

    Positive Definite Matrices , author =

  46. [55]

    Arsigny, Vincent and Fillard, Pierre and Pennec, Xavier and Ayache, Nicholas , journal =. Log-. 2006 , doi =

    2006

  47. [56]

    2008 , doi =

    Functions of Matrices: Theory and Computation , author =. 2008 , doi =

    2008

  48. [57]

    1997 , doi =

    Matrix Analysis , author =. 1997 , doi =

    1997

  49. [58]

    International Journal of Computer Vision , volume =

    A Riemannian Framework for Tensor Computing , author =. International Journal of Computer Vision , volume =. 2006 , doi =

    2006

  50. [59]

    Biometrics , volume =

    Covariance Selection , author =. Biometrics , volume =. 1972 , doi =

    1972

  51. [60]

    Linear Algebra and its Applications , volume =

    Positive Definite Completions of Partial Hermitian Matrices , author =. Linear Algebra and its Applications , volume =. 1984 , doi =

    1984

  52. [61]

    Journal of Differential Geometry , volume =

    Three-Manifolds with Positive Ricci Curvature , author =. Journal of Differential Geometry , volume =. 1982 , doi =

    1982

  53. [62]

    Journal of Differential Geometry , volume =

    Deforming Metrics in the Direction of Their Ricci Tensors , author =. Journal of Differential Geometry , volume =. 1983 , doi =

    1983

  54. [63]

    USSR Computational Mathematics and Mathematical Physics , volume =

    Some Methods of Speeding Up the Convergence of Iteration Methods , author =. USSR Computational Mathematics and Mathematical Physics , volume =. 1964 , doi =

    1964

  55. [64]

    Doklady Akademii Nauk SSSR , volume =

    A Method for Solving the Convex Programming Problem with Convergence Rate \(O(1/k^2)\) , author =. Doklady Akademii Nauk SSSR , volume =

  56. [65]

    Problem Complexity and Method Efficiency in Optimization , author =

  57. [66]

    Operations Research Letters , volume =

    Mirror Descent and Nonlinear Projected Subgradient Methods for Convex Optimization , author =. Operations Research Letters , volume =. 2003 , doi =

    2003

  58. [68]

    Foundations and Trends in Optimization , volume =

    Proximal Algorithms , author =. Foundations and Trends in Optimization , volume =. 2014 , doi =

    2014

  59. [69]

    2006 , doi =

    Numerical Optimization , author =. 2006 , doi =

    2006

  60. [70]

    SIAM Review , volume =

    Optimization Methods for Large-Scale Machine Learning , author =. SIAM Review , volume =. 2018 , doi =

    2018

  61. [71]

    SIAM Journal on Optimization , volume =

    Analysis and Design of Optimization Algorithms via Integral Quadratic Constraints , author =. SIAM Journal on Optimization , volume =. 2016 , doi =

    2016

  62. [72]

    Mathematical Programming , volume =

    Performance of First-Order Methods for Smooth Convex Minimization: A Novel Approach , author =. Mathematical Programming , volume =. 2014 , doi =

    2014

  63. [73]

    Mathematical Programming , volume =

    Smooth Strongly Convex Interpolation and Exact Worst-Case Performance of First-Order Methods , author =. Mathematical Programming , volume =. 2017 , doi =

    2017

  64. [74]

    Advances in Neural Information Processing Systems , volume =

    Learning to Learn by Gradient Descent by Gradient Descent , author =. Advances in Neural Information Processing Systems , volume =. 2016 , url =

    2016

  65. [75]

    Journal of Machine Learning Research , volume =

    Learning to Optimize: A Primer and A Benchmark , author =. Journal of Machine Learning Research , volume =. 2022 , url =

    2022

  66. [77]

    IEEE Signal Processing Magazine , volume =

    Algorithm Unrolling: Interpretable, Efficient Deep Learning for Signal and Image Processing , author =. IEEE Signal Processing Magazine , volume =. 2021 , doi =

    2021

  67. [78]

    2024 , eprint =

    The Road Less Scheduled , author =. 2024 , eprint =

    2024

  68. [79]

    Evolutionary Computation , volume =

    Automated Algorithm Selection: Survey and Perspectives , author =. Evolutionary Computation , volume =. 2019 , doi =

    2019

  69. [80]

    Operations Research , volume =

    Inverse Optimization , author =. Operations Research , volume =. 2001 , doi =

    2001

  70. [81]

    Foundations of Computational Mathematics , volume =

    The Convex Geometry of Linear Inverse Problems , author =. Foundations of Computational Mathematics , volume =. 2012 , doi =

    2012

  71. [82]

    TEST , volume =

    Exact Testing with Random Permutations , author =. TEST , volume =. 2018 , doi =

    2018

  72. [83]

    Statistical Applications in Genetics and Molecular Biology , volume =

    Permutation p -values Should Never Be Zero: Calculating Exact p -values When Permutations Are Randomly Drawn , author =. Statistical Applications in Genetics and Molecular Biology , volume =. 2010 , doi =

    2010

  73. [84]

    System Identification: Theory for the User , author =

  74. [85]

    Proceedings of the American Mathematical Society , volume =

    Linear Automaton Transformations , author =. Proceedings of the American Mathematical Society , volume =. 1958 , doi =

    1958

  75. [86]

    2019 , url =

    Schneider, Frank and Balles, Lukas and Hennig, Philipp , booktitle =. 2019 , url =

    2019

  76. [87]

    2023 , eprint =

    Benchmarking Neural Network Training Algorithms , author =. 2023 , eprint =

    2023

  77. [88]

    Numerische Mathematik , volume =

    Condition Numbers and Equilibration of Matrices , author =. Numerische Mathematik , volume =. 1969 , doi =

    1969

  78. [89]

    Numerische Mathematik , volume =

    Optimally Scaled Matrices , author =. Numerische Mathematik , volume =. 1963 , doi =

    1963

  79. [90]

    Numerische Mathematik , volume =

    Optimally Scaled Matrices, Necessary and Sufficient Conditions , author =. Numerische Mathematik , volume =. 1982 , doi =

    1982

  80. [91]

    SIAM Journal on Numerical Analysis , volume =

    Optimal Block Diagonal \(l_2\)-Scaling of Matrices , author =. SIAM Journal on Numerical Analysis , volume =. 1985 , doi =

    1985

  81. [92]

    SIAM Journal on Optimization , volume =

    Optimizing Condition Numbers , author =. SIAM Journal on Optimization , volume =. 2009 , doi =

    2009

  82. [93]

    SIAM Journal on Matrix Analysis and Applications , volume =

    Minimizing Condition Number via Convex Programming , author =. SIAM Journal on Matrix Analysis and Applications , volume =. 2011 , doi =

    2011

  83. [94]

    Operations Research , volume =

    Optimal Diagonal Preconditioning , author =. Operations Research , volume =. 2025 , doi =

    2025

  84. [95]

    Computational Optimization and Applications , volume =

    Scalable Approximate Optimal Diagonal Preconditioning , author =. Computational Optimization and Applications , volume =. 2026 , doi =

    2026

  85. [96]

    2025 , eprint =

    Optimal Preconditioning is a Geodesically Convex Optimization Problem , author =. 2025 , eprint =

    2025

  86. [97]

    Operations Research , year =

    Optimal Diagonal Preconditioning , author =. Operations Research , year =. doi:10.1287/opre.2022.0592 , url =

    2022 doi

  87. [98]

    Pacific Journal of Mathematics , volume =

    Concerning Nonnegative Matrices and Doubly Stochastic Matrices , author =. Pacific Journal of Mathematics , volume =. 1967 , doi =

    1967

  88. [99]

    IMA Journal of Numerical Analysis , volume =

    A Fast Algorithm for Matrix Balancing , author =. IMA Journal of Numerical Analysis , volume =. 2013 , doi =

    2013

  89. [100]

    Foundations and Trends in Machine Learning , volume =

    An Introduction to Matrix Concentration Inequalities , author =. Foundations and Trends in Machine Learning , volume =. 2015 , doi =

    2015

  90. [101]

    Biometrika , volume =

    A Useful Variant of the Davis--Kahan Theorem for Statisticians , author =. Biometrika , volume =. 2015 , doi =

    2015

  91. [104]

    2607.07206 , archivePrefix =

    Li, Zavier , year =. 2607.07206 , archivePrefix =

    arXiv

  92. [105]

    SIAM Journal on Numerical Analysis , volume =

    The Differentiation of Pseudo-Inverses and Nonlinear Least Squares Problems Whose Variables Separate , author =. SIAM Journal on Numerical Analysis , volume =. 1973 , doi =

    1973

  93. [106]

    Michigan Mathematical Journal , volume =

    The Fundamental Equations of a Submersion , author =. Michigan Mathematical Journal , volume =. 1966 , doi =

    1966

  94. [107]

    Annals of Global Analysis and Geometry , volume =

    A Metric Characterization of Riemannian Submersions , author =. Annals of Global Analysis and Geometry , volume =. 2000 , doi =

    2000

  95. [108]

    Submersions , author =

    Submetries vs. Submersions , author =. Revista Matematica Iberoamericana , volume =. 2011 , doi =

    2011

  96. [109]

    Probability Theory and Related Fields , volume =

    Information Geometry and Sufficient Statistics , author =. Probability Theory and Related Fields , volume =. 2015 , doi =

    2015

  97. [110]

    1975 , doi =

    Csiszar, Imre , journal =. 1975 , doi =

    1975

  98. [111]

    IEEE Transactions on Information Theory , volume =

    Information Projections Revisited , author =. IEEE Transactions on Information Theory , volume =. 2003 , doi =

    2003

  99. [112]

    Totally Geodesic Submanifolds in the Manifold

    Tumpach, Alice Barbara and Larotonda, Gabriel , journal =. Totally Geodesic Submanifolds in the Manifold. 2024 , doi =

    2024

  100. [113]

    SIAM Journal on Matrix Analysis and Applications , volume =

    Riemannian Metric and Geometric Mean for Positive Semidefinite Matrices of Fixed Rank , author =. SIAM Journal on Matrix Analysis and Applications , volume =. 2010 , doi =

    2010

  101. [114]

    IMA Journal of Numerical Analysis , volume =

    A Riemannian Geometry with Complete Geodesics for the Set of Positive Semidefinite Matrices of Fixed Rank , author =. IMA Journal of Numerical Analysis , volume =. 2013 , doi =

    2013

  102. [115]

    SIAM Journal on Matrix Analysis and Applications , volume =

    Quotient Geometry with Simple Geodesics for the Manifold of Fixed-Rank Positive-Semidefinite Matrices , author =. SIAM Journal on Matrix Analysis and Applications , volume =. 2020 , doi =

    2020

  103. [116]

    Journal of Multivariate Analysis , volume =

    Multivariate Normal Distributions Parametrized as a Riemannian Symmetric Space , author =. Journal of Multivariate Analysis , volume =. 2000 , doi =

    2000

  104. [117]

    2000 , doi =

    Perturbation Analysis of Optimization Problems , author =. 2000 , doi =

    2000

  105. [118]

    Proceedings of the 20th Conference on Uncertainty in Artificial Intelligence , pages =

    Predictive State Representations: A New Theory for Modeling Dynamical Systems , author =. Proceedings of the 20th Conference on Uncertainty in Artificial Intelligence , pages =. 2004 , publisher =

    2004

  106. [119]

    Journal of Machine Learning Research , volume =

    Explaining Explanations: Axiomatic Feature Interactions for Deep Networks , author =. Journal of Machine Learning Research , volume =. 2021 , eprint =

    2021

  107. [120]

    Dhamdhere, Kedar and Sundararajan, Mukund and Agarwal, Ashish , booktitle =. The. 2020 , publisher =. 1902.05622 , archivePrefix =

    2020 arXiv

  108. [121]

    Zico , booktitle =

    Amos, Brandon and Kolter, J. Zico , booktitle =. 2017 , publisher =. 1703.00443 , archivePrefix =

    2017 arXiv

  109. [122]

    Advances in Neural Information Processing Systems , volume =

    Efficient and Modular Implicit Differentiation , author =. Advances in Neural Information Processing Systems , volume =. 2022 , eprint =

    2022

  110. [123]

    Proceedings of the 35th International Conference on Machine Learning , series =

    Bilevel Programming for Hyperparameter Optimization and Meta-Learning , author =. Proceedings of the 35th International Conference on Machine Learning , series =. 2018 , publisher =. 1806.04910 , archivePrefix =

    2018 arXiv

  111. [124]

    2006 , doi =

    Optimal Design of Experiments , author =. 2006 , doi =

    2006

  112. [125]

    and Rubin, Donald B

    Dasgupta, Tirthankar and Pillai, Natesh S. and Rubin, Donald B. , journal =. Causal Inference from. 2015 , doi =

    2015

  113. [126]

    SIAM Journal on Optimization , volume =

    Conic Geometric Optimization on the Manifold of Positive Definite Matrices , author =. SIAM Journal on Optimization , volume =. 2015 , doi =

    2015

  114. [127]

    Canadian Journal of Mathematics , volume =

    Best Approximation in Riemannian Geodesic Submanifolds of Positive Definite Matrices , author =. Canadian Journal of Mathematics , volume =. 2004 , doi =

    2004

  115. [128]

    SIAM Journal on Optimization , volume =

    Convex Analysis on the Hermitian Matrices , author =. SIAM Journal on Optimization , volume =. 1996 , doi =

    1996

  116. [129]

    2014 , doi =

    Convex Analysis and Optimization in Hadamard Spaces , author =. 2014 , doi =

    2014

  117. [130]

    2014 , doi =

    Implicit Functions and Solution Mappings , author =. 2014 , doi =

    2014

  118. [131]

    2008 , doi =

    Direct Methods in the Calculus of Variations , author =. 2008 , doi =

    2008

  119. [132]

    Absil, Robert Mahony, and Rodolphe Sepulchre

    P.-A. Absil, Robert Mahony, and Rodolphe Sepulchre. Optimization Algorithms on Matrix Manifolds. Princeton University Press, 2008

    2008

  120. [133]

    Ahuja and James B

    Ravindra K. Ahuja and James B. Orlin. Inverse optimization. Operations Research, 49 0 (5): 0 771--783, 2001. doi:10.1287/opre.49.5.771.10607. URL https://doi.org/10.1287/opre.49.5.771.10607

    2001 doi

  121. [134]

    Natural gradient works efficiently in learning

    Shun-ichi Amari. Natural gradient works efficiently in learning. Neural Computation, 10 0 (2): 0 251--276, 1998. doi:10.1162/089976698300017746

    1998 doi

  122. [135]

    Zico Kolter

    Brandon Amos and J. Zico Kolter. OptNet : Differentiable optimization as a layer in neural networks. In Proceedings of the 34th International Conference on Machine Learning, volume 70 of Proceedings of Machine Learning Research, pp.\ 136--145. PMLR, 2017

    2017

  123. [136]

    Hoffman, David Pfau, Tom Schaul, Brendan Shillingford, and Nando de Freitas

    Marcin Andrychowicz, Misha Denil, Sergio Gomez, Matthew W. Hoffman, David Pfau, Tom Schaul, Brendan Shillingford, and Nando de Freitas. Learning to learn by gradient descent by gradient descent. In Advances in Neural Information Processing Systems, volume 29, 2016. URL https:/...

    2016 arXiv

  124. [137]

    Memory efficient adaptive optimization

    Rohan Anil, Vineet Gupta, Tomer Koren, and Yoram Singer. Memory efficient adaptive optimization. In Advances in Neural Information Processing Systems, volume 32, 2019. URL https://proceedings.neurips.cc/paper_files/paper/2019/hash/8f1fa0193ca2b5d2fa0695827d8270e9-Abstract.html

    2019

  125. [138]

    Mirror descent and nonlinear projected subgradient methods for convex optimization

    Amir Beck and Marc Teboulle. Mirror descent and nonlinear projected subgradient methods for convex optimization. Operations Research Letters, 31 0 (3): 0 167--175, 2003. doi:10.1016/S0167-6377(02)00231-6

    2003 doi

  126. [139]

    Efficient and modular implicit differentiation

    Mathieu Blondel, Quentin Berthet, Marco Cuturi, Roy Frostig, Stephan Hoyer, Felipe Llinares-L\'opez, Fabian Pedregosa, and Jean-Philippe Vert. Efficient and modular implicit differentiation. In Advances in Neural Information Processing Systems, volume 35, 2022

    2022

  127. [140]

    Fr\'ed\'eric Bonnans and Alexander Shapiro

    J. Fr\'ed\'eric Bonnans and Alexander Shapiro. Perturbation Analysis of Optimization Problems. Springer New York, 2000. doi:10.1007/978-1-4612-1394-9

    2000 doi

  128. [141]

    An Introduction to Optimization on Smooth Manifolds

    Nicolas Boumal. An Introduction to Optimization on Smooth Manifolds. Cambridge University Press, 2023. doi:10.1017/9781009166164

    2023 doi

  129. [142]

    Combettes and Bang C

    Patrick L. Combettes and Bang C. V \ u . Variable metric forward--backward splitting with applications to monotone inclusions in duality. Optimization, 63 0 (9): 0 1289--1318, 2014. doi:10.1080/02331934.2012.733883. URL https://arxiv.org/abs/1206.6791

    2014 doi

  130. [143]

    Michael Crawshaw, Chirag Modi, Mingrui Liu, and Robert M. Gower. An exploration of non-euclidean gradient descent: Muon and its many variants, 2025. URL https://arxiv.org/abs/2510.09827

    2025

  131. [144]

    Dahl, Frank Schneider, Zachary Nado, Naman Agarwal, Chandramouli Shama Sastry, Philipp Hennig, et al

    George E. Dahl, Frank Schneider, Zachary Nado, Naman Agarwal, Chandramouli Shama Sastry, Philipp Hennig, et al. Benchmarking neural network training algorithms, 2023. URL https://arxiv.org/abs/2306.07179

    2023 arXiv

  132. [145]

    Pillai, and Donald B

    Tirthankar Dasgupta, Natesh S. Pillai, and Donald B. Rubin. Causal inference from 2^K factorial designs by using potential outcomes. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 77 0 (4): 0 727--753, 2015. doi:10.1111/rssb.12085

    2015 doi

  133. [146]

    The road less scheduled, 2024

    Aaron Defazio, Xingyu Alice Yang, Harsh Mehta, Konstantin Mishchenko, Ahmed Khaled, and Ashok Cutkosky. The road less scheduled, 2024. URL https://arxiv.org/abs/2405.15682. arXiv:2405.15682

    2024 arXiv

  134. [147]

    The Shapley--Taylor interaction index

    Kedar Dhamdhere, Mukund Sundararajan, and Ashish Agarwal. The Shapley--Taylor interaction index. In Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pp.\ 2709--2718. PMLR, 2020

    2020

  135. [148]

    Performance of first-order methods for smooth convex minimization: A novel approach

    Yoel Drori and Marc Teboulle. Performance of first-order methods for smooth convex minimization: A novel approach. Mathematical Programming, 145 0 (1--2): 0 451--482, 2014. doi:10.1007/s10107-013-0653-0. URL https://arxiv.org/abs/1206.3209

    2014 doi arXiv

  136. [149]

    Adaptive subgradient methods for online learning and stochastic optimization

    John Duchi, Elad Hazan, and Yoram Singer. Adaptive subgradient methods for online learning and stochastic optimization. Journal of Machine Learning Research, 12: 0 2121--2159, 2011. URL https://jmlr.org/papers/v12/duchi11a.html

    2011

  137. [150]

    Polloreno, Karl Stratos, Philip Monk, Adarsh Chaluvaraju, Andrew Hojel, Andrew Ma, Anil Thomas, Ashish Tanwer, Darsh J

    Essential AI , Ishaan Shah, Anthony M. Polloreno, Karl Stratos, Philip Monk, Adarsh Chaluvaraju, Andrew Hojel, Andrew Ma, Anil Thomas, Ashish Tanwer, Darsh J. Shah, Khoi Nguyen, Kurt Smith, Michael Callahan, Michael Pust, Mohit Parmar, Peter Rushton, Platon Mazarakis, Ritvik K...

    2025 arXiv

  138. [151]

    Bilevel programming for hyperparameter optimization and meta-learning

    Luca Franceschi, Paolo Frasconi, Saverio Salzo, Riccardo Grazzi, and Massimiliano Pontil. Bilevel programming for hyperparameter optimization and meta-learning. In Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learnin...

    2018

  139. [152]

    Golub and Victor Pereyra

    Gene H. Golub and Victor Pereyra. The differentiation of pseudo-inverses and nonlinear least squares problems whose variables separate. SIAM Journal on Numerical Analysis, 10 0 (2): 0 413--432, 1973. doi:10.1137/0710036

    1973 doi

  140. [153]

    Shampoo: Preconditioned stochastic tensor optimization

    Vineet Gupta, Tomer Koren, and Yoram Singer. Shampoo: Preconditioned stochastic tensor optimization. In Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pp.\ 1842--1850. PMLR, 2018. URL https://proceed...

    2018

  141. [154]

    Exact testing with random permutations

    Jesse Hemerik and Jelle Goeman. Exact testing with random permutations. TEST, 27 0 (4): 0 811--825, 2018. doi:10.1007/s11749-017-0571-1. URL https://doi.org/10.1007/s11749-017-0571-1

    2018 doi

  142. [155]

    Janizek, Pascal Sturmfels, and Su-In Lee

    Joseph D. Janizek, Pascal Sturmfels, and Su-In Lee. Explaining explanations: Axiomatic feature interactions for deep networks. Journal of Machine Learning Research, 22 0 (104): 0 1--54, 2021

    2021

  143. [156]

    Muon: An optimizer for hidden layers in neural networks

    Keller Jordan. Muon: An optimizer for hidden layers in neural networks. https://kellerjordan.github.io/posts/muon/, 2024. Blog post

    2024

  144. [157]

    Hoos, Frank Neumann, and Heike Trautmann

    Pascal Kerschke, Holger H. Hoos, Frank Neumann, and Heike Trautmann. Automated algorithm selection: Survey and perspectives. Evolutionary Computation, 27 0 (1): 0 3--45, 2019. doi:10.1162/evco_a_00242. URL https://arxiv.org/abs/1811.11597

    2019 doi arXiv

  145. [158]

    Kingma and Jimmy Ba

    Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In International Conference on Learning Representations, 2015. URL https://arxiv.org/abs/1412.6980. arXiv:1412.6980

    2015 arXiv

  146. [159]

    Analysis and design of optimization algorithms via integral quadratic constraints

    Laurent Lessard, Benjamin Recht, and Andrew Packard. Analysis and design of optimization algorithms via integral quadratic constraints. SIAM Journal on Optimization, 26 0 (1): 0 57--95, 2016. doi:10.1137/15M1009597. URL https://arxiv.org/abs/1408.3595

    2016 doi arXiv

  147. [160]

    Information-Induced Training Geometry: Exact Reduction, Canonical Completion, and Structured Expressivity , 2026 a

    Zavier Li. Information-Induced Training Geometry: Exact Reduction, Canonical Completion, and Structured Expressivity , 2026 a . arXiv preprint

    2026

  148. [161]

    Optimization Geometrodynamics: Variational Reduction and Interaction Curvature , 2026 b

    Zavier Li. Optimization Geometrodynamics: Variational Reduction and Interaction Curvature , 2026 b . URL https://arxiv.org/abs/2607.06723. arXiv:2607.06723

    2026 arXiv

  149. [162]

    Restricted Dynamic Geometric Complexity: Path-Space Reduction and Certificates for Structured Preconditioning , 2026 c

    Zavier Li. Restricted Dynamic Geometric Complexity: Path-Space Reduction and Certificates for Structured Preconditioning , 2026 c . URL https://arxiv.org/abs/2607.07204. arXiv:2607.07204

    2026 arXiv

  150. [163]

    Sophia: A scalable stochastic second-order optimizer for language model pre-training, 2023

    Hong Liu, Zhiyuan Li, David Hall, Percy Liang, and Tengyu Ma. Sophia: A scalable stochastic second-order optimizer for language model pre-training, 2023. URL https://arxiv.org/abs/2305.14342

    2023 arXiv

  151. [164]

    System Identification: Theory for the User

    Lennart Ljung. System Identification: Theory for the User. Prentice Hall, 2 edition, 1999

    1999

  152. [165]

    Optimizing neural networks with Kronecker -factored approximate curvature

    James Martens and Roger Grosse. Optimizing neural networks with Kronecker -factored approximate curvature. In Proceedings of the 32nd International Conference on Machine Learning, volume 37 of Proceedings of Machine Learning Research, pp.\ 2408--2417. PMLR, 2015. URL https://p...

    2015

  153. [166]

    Daniel Freeman, Amil Merchant, Lucas Beyer, James Bradbury, Naman Agrawal, Ben Poole, Igor Mordatch, Adam Roberts, and Jascha Sohl-Dickstein

    Luke Metz, James Harrison, C. Daniel Freeman, Amil Merchant, Lucas Beyer, James Bradbury, Naman Agrawal, Ben Poole, Igor Mordatch, Adam Roberts, and Jascha Sohl-Dickstein. VeLO : Training versatile learned optimizers by scaling up, 2022. URL https://arxiv.org/abs/2211.09760. a...

    2022 arXiv

  154. [167]

    Linear automaton transformations

    Anil Nerode. Linear automaton transformations. Proceedings of the American Mathematical Society, 9 0 (4): 0 541--544, 1958. doi:10.1090/S0002-9939-1958-0135681-9. URL https://doi.org/10.1090/S0002-9939-1958-0135681-9

    1958 doi

  155. [168]

    Nesterov

    Yurii E. Nesterov. A method for solving the convex programming problem with convergence rate \(o(1/k^2)\). Doklady Akademii Nauk SSSR, 269 0 (3): 0 543--547, 1983

    1983

  156. [169]

    Jorge Nocedal and Stephen J. Wright. Numerical Optimization. Springer, 2 edition, 2006. doi:10.1007/978-0-387-40065-5

    2006 doi

  157. [170]

    Proximal algorithms

    Neal Parikh and Stephen Boyd. Proximal algorithms. Foundations and Trends in Optimization, 1 0 (3): 0 127--239, 2014. doi:10.1561/2400000003. URL https://web.stanford.edu/ boyd/papers/prox_algs.html

    2014 doi

  158. [171]

    Belinda Phipson and Gordon K. Smyth. Permutation p -values should never be zero: Calculating exact p -values when permutations are randomly drawn. Statistical Applications in Genetics and Molecular Biology, 9 0 (1), 2010. doi:10.2202/1544-6115.1585. URL https://doi.org/10.2202...

    2010 doi

  159. [172]

    Boris T. Polyak. Some methods of speeding up the convergence of iteration methods. USSR Computational Mathematics and Mathematical Physics, 4 0 (5): 0 1--17, 1964. doi:10.1016/0041-5553(64)90137-5

    1964 doi

  160. [173]

    Optimal Design of Experiments

    Friedrich Pukelsheim. Optimal Design of Experiments. Society for Industrial and Applied Mathematics, Philadelphia, 2006. doi:10.1137/1.9780898719109

    2006 doi

  161. [174]

    Adafactor: Adaptive learning rates with sublinear memory cost

    Noam Shazeer and Mitchell Stern. Adafactor: Adaptive learning rates with sublinear memory cost. In Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pp.\ 4596--4604. PMLR, 2018. URL https://proceedings....

    2018

  162. [175]

    James, and Matthew R

    Satinder Singh, Michael R. James, and Matthew R. Rudary. Predictive state representations: A new theory for modeling dynamical systems. In Proceedings of the 20th Conference on Uncertainty in Artificial Intelligence, pp.\ 512--519. AUAI Press, 2004. doi:10.5555/1036843.1036905

    2004 doi

  163. [176]

    Cand \`e s

    Weijie Su, Stephen Boyd, and Emmanuel J. Cand \`e s. A differential equation for modeling Nesterov 's accelerated gradient method: Theory and insights. Journal of Machine Learning Research, 17 0 (153): 0 1--43, 2016. URL https://arxiv.org/abs/1503.01243

    2016 arXiv

  164. [177]

    Taylor, Julien M

    Adrien B. Taylor, Julien M. Hendrickx, and Fran c ois Glineur. Smooth strongly convex interpolation and exact worst-case performance of first-order methods. Mathematical Programming, 161 0 (1--2): 0 307--345, 2017. doi:10.1007/s10107-016-1009-3. URL https://arxiv.org/abs/1502.05666

    2017 doi arXiv

  165. [178]

    SOAP : Improving and stabilizing Shampoo using Adam , 2024

    Nikhil Vyas, Depen Morwani, Rosie Zhao, Mujin Kwun, Itai Shapira, David Brandfonbrener, Lucas Janson, and Sham Kakade. SOAP : Improving and stabilizing Shampoo using Adam , 2024. URL https://arxiv.org/abs/2409.11321

    2024 arXiv

  166. [179]

    Wilson, and Michael I

    Andre Wibisono, Ashia C. Wilson, and Michael I. Jordan. A variational perspective on accelerated methods in optimization. Proceedings of the National Academy of Sciences, 113 0 (47): 0 E7351--E7358, 2016. URL https://arxiv.org/abs/1603.04245

    2016 arXiv

Pith tools

Reviewed July 14, 2026 · model on record in the stance chip above.