Pith. sign in

REVIEW 4 major objections 5 minor 35 references

Stability annealing steers smoothed sign descent onto a Burg barrier path for max-margin separators

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Memoryless stability-annealed smoothed-sign descent with weighted exponential loss converges in normalized iterates to a Burg-type barrier minimizer on a margin slice, with an explicit S_t^{-1/2} envelope.

T0 review reviewed 2026-07-08 challenge →

load-bearing objection Clean middle-regime implicit-bias result for annealed smoothed-sign: Burg barrier via exact dual rewrite; residual control under time-varying stability is the step to pressure. the 4 major comments →

arxiv 2607.06013 v1 pith:OBDA63IX submitted 2026-07-07 cs.LG math.OC

Stability Annealing Selects the Implicit Bias of Smoothed Sign Descent: A Rate-Indexed Barrier Path on Separable Data

Xiangwu Wang , Chengwei Cao , Yicheng Song , Ran Bi , Peilin Yu This is my paper
classification cs.LG math.OC
keywords smoothed sign descentstability annealingimplicit biasmax-marginBurg barrierentropic mirror ascentseparable datalinear classification
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Adaptive gradient methods can settle on different max-margin separators than plain gradient descent, but a fixed positive stability constant eventually warps the geometry again. This paper isolates the rate-controlled middle regime for full-batch linear classification on separable data: memoryless stability-annealed smoothed-sign descent with weighted exponential loss. The authors prove that the normalized iterates converge to the minimizer of a convex Burg-type barrier over a margin slice. They rewrite the dynamics exactly as entropic mirror ascent on a concave dual objective, control the dual gap by a KL recursion, and obtain an explicit envelope of order S_t^{-1/2} on the normalized iterates, where S_t is cumulative stability mass. The static barrier geometry is fully characterized, including KKT conditions and both endpoint limits. Experiments confirm the dual identities to floating-point error, map the predicted path and rate diagram, and show an empirical fixed-epsilon crossover scaling in cumulative time, while diagnostics mark the boundaries for logistic tails, fixed epsilon, and adaptive second-moment memory.

Core claim

For memoryless stability-annealed smoothed-sign descent with weighted exponential loss on full-batch linearly separable data, the normalized iterates converge to the unique minimizer of a convex Burg-type barrier over a margin slice. The dynamics admit an exact rewrite as entropic mirror ascent on a concave dual objective, a KL recursion controls the dual gap, and the normalized-iterate error is bounded by an explicit envelope of order S_t^{-1/2} under the rate-controlled annealing schedule.

What carries the argument

The exact dual rewrite of annealed smoothed-sign updates as entropic mirror ascent on a concave dual, paired with a KL recursion that controls the dual gap and produces the S_t^{-1/2} normalized-iterate envelope; the static object is the Burg-type barrier minimized over the margin slice.

Load-bearing premise

The theory needs memoryless smoothed-sign updates under a stability schedule whose cumulative mass diverges while instantaneous stability goes to zero, together with weighted exponential loss on full-batch linearly separable data.

What would settle it

On a fixed linearly separable full-batch problem, run memoryless annealed smoothed-sign with weighted exponential loss and check whether the normalized iterates approach the claimed Burg-barrier minimizer at rate consistent with S_t^{-1/2}; failure of the dual identities beyond floating-point error, or divergence under the stated schedule, would refute the claim.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The manuscript studies full-batch linear classification on separable data under memoryless stability-annealed smoothed-sign descent with weighted exponential loss. It claims that, in a rate-controlled middle regime where the instantaneous stability ε_t → 0 while the cumulative mass S_t diverges, the normalized iterates converge to the unique minimizer of a convex Burg-type barrier over a margin slice. The argument rewrites the discrete dynamics exactly as entropic mirror ascent on a concave dual objective, controls the dual gap by a KL recursion, and produces an explicit S_t^{-1/2} envelope on the normalized iterates. The static barrier geometry is characterized via KKT conditions and endpoint limits. Experiments report floating-point validation of dual identities, a path/rate diagram, fixed-ε crossover scaling, and boundary diagnostics for logistic tails, fixed stability, and adaptive-memory variants.

Significance. If the dual rewrite and residual control under annealing hold as stated, the paper supplies a clean, rate-indexed characterization of the implicit bias of smoothed-sign methods in the intermediate stability regime that sits between pure sign/gradient geometry and fixed-ε adaptive geometry. Explicit dual identities validated to floating-point error, an S_t^{-1/2} envelope, and a full KKT description of the Burg barrier are genuine strengths and raise the standard of evidence for this claim class. The scope diagnostics that flag logistic tails, fixed-ε crossover, and second-moment memory are also useful for the community. The result is of clear interest to the implicit-bias and adaptive-optimization literature, provided the time-varying dual analysis is airtight.

major comments (4)
  1. The central load-bearing step is the passage from a frozen-ε dual identity to a time-varying (annealed) dual under ε_t → 0 with S_t → ∞. Because either the dual objective or the Bregman/entropic potential is non-stationary, the discrete three-point/KL identity acquires residual drift terms from the change of objective or potential between successive steps. Floating-point validation of dual identities certifies the algebraic correspondence at fixed ε, but does not by itself bound the telescoping residual along the annealing path. The manuscript must exhibit an explicit residual budget that is summable (or compatible with the claimed S_t^{-1/2} envelope) throughout the rate-controlled middle regime; without that bound, dual-gap vanishing and convergence of normalized iterates to the static Burg minimizer are not secured. This is the single most important technical point to strengthen or cl
  2. The claimed exact rewrite as entropic mirror ascent on a concave dual must be stated for the annealed (time-varying) update, not only for frozen ε. If the dual objective itself depends on ε_t, the limit dual that defines the Burg barrier must be identified carefully, and the argument that the time-varying dual gap still forces the normalized primal iterates onto the static margin-slice barrier needs a precise statement (e.g., which dual sequence is controlled, and how its argmax relates to the static KKT geometry). A short, self-contained lemma isolating this identification would make the central claim checkable.
  3. The S_t^{-1/2} normalized-iterate envelope is a strong quantitative claim. Its derivation should make transparent which step-size and annealing conditions are necessary and sufficient (beyond S_t → ∞ and ε_t → 0), and whether the envelope is sharp. If the envelope is obtained by summing residual and dual-gap terms, those terms should be tracked explicitly so that a reader can verify the rate under the stated free parameters (ε_t / S_t path, step-size sequence, exponential-loss weights).
  4. Scope: the theory is stated for memoryless smoothed-sign with weighted exponential loss. The diagnostics for logistic tails and adaptive second-moment memory are welcome, but the manuscript should state more sharply which pieces of the dual rewrite fail outside this class (e.g., whether the exact entropic dual identity is lost, or only the residual budget). This is needed so that the proved theorem is not over-read as applying to Adam-style methods or logistic loss without further work.
minor comments (5)
  1. Define the Burg-type barrier and the margin-slice constraint in a single displayed equation early in the main theorem statement, so that the limit object is unambiguous before the dual analysis begins.
  2. Clarify notation for the stability schedule: distinguish instantaneous ε_t, cumulative mass S_t, and any normalized time variable used in the rate diagram, and keep that notation consistent between theory and experiments.
  3. In the experimental section, report the precise annealing schedules and step-size sequences used for the floating-point dual-identity checks and for the fixed-ε crossover scaling, so that the dual identities can be reproduced independently.
  4. The endpoint limits of the barrier geometry (as the margin-slice level or weight parameters vary) should be cross-referenced to the KKT characterization so that the geometric picture is easy to navigate.
  5. A short related-work paragraph situating the Burg barrier against known max-margin and entropy-regularized duals for GD, signGD, and fixed-ε adaptive methods would help non-specialist readers place the contribution.

Simulated Author's Rebuttal

4 responses · 0 unresolved

We thank the referee for a careful, technically focused report. The four major comments correctly isolate the load-bearing steps: residual control under annealing, identification of the time-varying dual with the static Burg geometry, transparency of the S_t^{-1/2} envelope, and sharp scope delineation. We believe the dual rewrite and residual analysis already secure dual-gap vanishing and normalized convergence in the rate-controlled middle regime, but we agree that the residual budget, the annealed dual identification, and the free-parameter hypotheses for the envelope must be made more explicit and checkable. We will revise by adding a residual-budget lemma, a self-contained annealed-dual identification lemma, an explicit tracking of envelope contributions, and sharper scope statements for logistic tails and second-moment memory. Each comment is addressed below.

read point-by-point responses

  1. Referee: The central load-bearing step is the passage from a frozen-ε dual identity to a time-varying (annealed) dual under ε_t → 0 with S_t → ∞. Because either the dual objective or the Bregman/entropic potential is non-stationary, the discrete three-point/KL identity acquires residual drift terms. Floating-point validation at fixed ε does not bound the telescoping residual along the annealing path. The manuscript must exhibit an explicit residual budget that is summable (or compatible with the claimed S_t^{-1/2} envelope); without that bound, dual-gap vanishing and convergence of normalized iterates to the static Burg minimizer are not secured.

    Authors: We agree that an explicit residual budget is essential and that fixed-ε floating-point checks do not by themselves control telescoping drift along the annealing path. In the argument the dual identity is applied at the instantaneous ε_t of each step, so the three-point/KL identity is exact for the frozen dual at that step; residuals arise only from the change of dual objective (or potential) between successive ε_t. Under the rate-controlled middle regime (ε_t → 0, S_t → ∞, with the path constraints of the theorem), these residuals are controlled by the variation of ε and are compatible with dual-gap decrease, which yields dual-gap vanishing and the S_t^{-1/2} envelope. We will revise the proof to isolate an explicit residual-budget lemma that (i) writes the telescoping residual term by term, (ii) bounds it under the stated annealing and step-size hypotheses, and (iii) shows compatibility with the claimed envelope. This will make annealed dual-gap control checkable without reconstructing it from the frozen case. revision: yes

  2. Referee: The claimed exact rewrite as entropic mirror ascent on a concave dual must be stated for the annealed (time-varying) update, not only for frozen ε. If the dual objective itself depends on ε_t, the limit dual that defines the Burg barrier must be identified carefully, and the argument that the time-varying dual gap still forces the normalized primal iterates onto the static margin-slice barrier needs a precise statement (e.g., which dual sequence is controlled, and how its argmax relates to the static KKT geometry). A short, self-contained lemma isolating this identification would make the central claim checkable.

    Authors: We agree. The exact entropic-mirror rewrite holds at each step for the instantaneous dual objective associated with ε_t and the weighted exponential loss; the dual is therefore time-varying. The limit dual defining the Burg barrier is the ε → 0 limit of these duals, whose maximizers correspond via the already-characterized KKT geometry to the static margin-slice Burg minimizer. Dual-gap control is applied to the sequence of instantaneous dual gaps; vanishing of that sequence, together with ε_t → 0 and the residual budget, forces the normalized primal iterates onto the static barrier. We will add a short, self-contained lemma that (i) states the exact annealed dual rewrite, (ii) identifies the limit dual and its relation to the static Burg KKT geometry, and (iii) records precisely which dual-gap sequence is controlled and how its vanishing implies normalized primal convergence to the static barrier minimizer. revision: yes

  3. Referee: The S_t^{-1/2} normalized-iterate envelope is a strong quantitative claim. Its derivation should make transparent which step-size and annealing conditions are necessary and sufficient (beyond S_t → ∞ and ε_t → 0), and whether the envelope is sharp. If the envelope is obtained by summing residual and dual-gap terms, those terms should be tracked explicitly so that a reader can verify the rate under the stated free parameters (ε_t / S_t path, step-size sequence, exponential-loss weights).

    Authors: We agree that the derivation should make the free-parameter conditions transparent. The envelope is obtained by summing dual-gap and residual terms under the KL recursion; the S_t^{-1/2} rate follows when the step-size sequence and the annealing path (ε_t relative to S_t) keep those summed contributions of order S_t^{-1/2} (or better). We will revise the statement and proof of the envelope to (i) list the step-size and annealing conditions used, (ii) track residual and dual-gap contributions explicitly so a reader can verify the rate under the free parameters (ε_t/S_t path, step sizes, exponential-loss weights), and (iii) comment on sharpness: the rate is the natural one from the KL/dual-gap recursion under square-summable residuals, and we will note that matching lower bounds are left open. We do not claim the envelope under arbitrary annealing—only under the rate-controlled middle regime of the theorem. revision: yes

  4. Referee: Scope: the theory is stated for memoryless smoothed-sign with weighted exponential loss. The diagnostics for logistic tails and adaptive second-moment memory are welcome, but the manuscript should state more sharply which pieces of the dual rewrite fail outside this class (e.g., whether the exact entropic dual identity is lost, or only the residual budget). This is needed so that the proved theorem is not over-read as applying to Adam-style methods or logistic loss without further work.

    Authors: We agree that the scope must be stated more sharply. The exact entropic dual identity relies on the memoryless smoothed-sign update together with the weighted exponential loss, which produces the dual structure used in the mirror rewrite. For logistic tails the exact dual identity is lost (the loss no longer yields the same closed dual), so residual budget and dual-gap control do not transfer without a new argument; our diagnostics are empirical only. For adaptive second-moment memory (Adam-style), the update is no longer pure memoryless smoothed sign, so the exact entropic dual rewrite fails; again the diagnostics are boundary checks, not theorems. We will revise the scope discussion (introduction, theorem statements, and diagnostics section) to state explicitly which pieces fail outside the memoryless smoothed-sign + weighted-exponential class—exact dual identity versus residual budget—and to warn against over-reading the proved theorem as covering logistic loss or Adam-style methods without further work. revision: yes

Circularity Check

0 steps flagged

No significant circularity: dual rewrite, KL gap control, and Burg barrier limit are derived from the stated update, not fitted or self-defined.

full rationale

The paper’s central claim is a first-principles derivation for memoryless stability-annealed smoothed-sign descent under weighted exponential loss on full-batch linearly separable data: the dynamics are rewritten exactly as entropic mirror ascent on a concave dual, the dual gap is controlled by a KL recursion, and normalized iterates converge to the minimizer of a static Burg-type barrier on a margin slice, with an explicit S_t^{-1/2} envelope and KKT characterization of the barrier geometry. These steps are algebraic and analytic consequences of the stated update rule, loss, and annealing schedule (ε_t → 0 with cumulative mass S_t diverging). Experiments are used only to validate the dual identities to floating-point error and to illustrate the predicted path/rate diagram and boundary diagnostics; no parameter is fitted to the claimed barrier limit and then re-labeled a prediction. There is no self-definitional loop (the barrier is not defined from the iterates it is said to attract), no fitted-input-called-prediction pattern, and no load-bearing uniqueness theorem imported solely by self-citation that forbids alternatives by construction. Scope limitations (logistic tails, fixed-ε crossover, adaptive second-moment memory) are stated as boundaries rather than smuggled into the proved case. The derivation is therefore self-contained against the paper’s own equations and assumptions; residual-control questions under annealing are correctness risks, not circularity. Score 0 with empty steps is the warranted finding.

Axiom & Free-Parameter Ledger

3 free parameters · 4 axioms · 0 invented entities

Abstract-only ledger. Free parameters are those the theory and experiments must choose (annealing schedule shape, step sizes, loss weights, epsilon path) even if some cancel in the limit statement. Axioms are the standard separable linear classification setup, the memoryless smoothed-sign update form, weighted exponential loss, and the rate-controlled annealing regime that makes S_t diverge while instantaneous stability vanishes. No new physical particle or force is invented; the Burg-type barrier is a derived geometric object of the dual, not an ad-hoc entity, provided the dual rewrite holds. Invented-entity list left empty for that reason. Full free-parameter values are not stated in the abstract.

free parameters (3)
  • stability annealing schedule (epsilon_t / cumulative S_t path)
    The middle-regime theory is indexed by a rate-controlled stability schedule; the envelope is S_t^{-1/2}. The concrete schedule family and constants used in proofs and plots are free choices of the paper even if the limit object is schedule-robust within a class.
  • step-size sequence
    Full-batch descent requires a step-size rule for the mirror-ascent rewrite and KL recursion to close; values are not fixed by the abstract.
  • exponential-loss weights / margin-slice level
    Weighted exponential loss and the margin slice on which the Burg barrier is minimized introduce weight and margin parameters that shape the static geometry.
axioms (4)
  • domain assumption Data are linearly separable; analysis is full-batch linear classification.
    Stated problem setting in the abstract; separability is required for max-margin / barrier-on-margin-slice geometry.
  • ad hoc to paper Update is memoryless smoothed-sign descent with stability annealing (no second-moment memory).
    The proved theory is scoped to this memoryless class; adaptive-method variants are only diagnosed experimentally as boundary cases.
  • domain assumption Loss is weighted exponential (logistic treated as robustness diagnostic, not the main theorem).
    Abstract pairs the theorem with weighted exponential loss; logistic tails are reported as diagnostics delineating scope.
  • standard math Standard convex duality / entropic mirror ascent and KL divergence calculus on the dual simplex-like geometry.
    Proof architecture in the abstract relies on rewriting as entropic mirror ascent and controlling dual gap by a KL recursion.

reviewed 2026-07-08 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Stability Annealing Selects the Implicit Bias of Smoothed Sign Descent: A Rate-Indexed Barrier Path on Separable Data." pith.science (2026). https://pith.science/paper/OBDA63IX

@misc{pith2026260706013,
  author       = {Pith},
  title        = {Pith review of: Stability Annealing Selects the Implicit Bias of Smoothed Sign Descent: A Rate-Indexed Barrier Path on Separable Data},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OBDA63IX}},
  note         = {Machine review of arXiv:2607.06013}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Adaptive gradient methods can favor max-margin separators that differ from gradient descent, yet a fixed positive numerical stability constant eventually changes the update geometry again. This paper studies the rate-controlled middle case for full-batch linear classification on separable data. For memoryless stability-annealed smoothed-sign descent with weighted exponential loss, we prove that the normalized iterates converge to the minimizer of a convex Burg-type barrier over a margin slice. The proof rewrites the dynamics exactly as entropic mirror ascent on a concave dual objective, controls the dual gap by a KL recursion, and yields an explicit S_t^{-1/2} normalized-iterate envelope. The static barrier geometry is fully characterized, including KKT conditions and both endpoint limits. Experiments validate the exact dual identities to floating-point error, illustrate the predicted path and rate diagram, and show an empirical fixed-epsilon crossover scaling in cumulative time. We further report robustness and boundary diagnostics for logistic tails, fixed-epsilon crossover, and adaptive-method variants, delineating the scope of the proved smoothed-sign theory.

Figures

Figures reproduced from arXiv: 2607.06013 by Chengwei Cao, Peilin Yu, Ran Bi, Xiangwu Wang, Yicheng Song.

Figure 1
Figure 1. Figure 1: Two-dimensional barrier geometry. Contours show [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Theorem validation dashboard. All panels use theorem-compatible weighted exponential loss, [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Mechanism scorecard. Panel (a) compares final finite-window error ratios against the proposed exponential [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Fixed-ϵ crossover scaling for the memoryless proxy, with τϵ(c) = min{t : ∥ − ∇L(wt)∥∞ ≤ cϵ}. Panel (a) shows residual collapse for the empirical law Sτϵ = γ −1 ∞ log(1/ϵ) + O(1). Panel (b) reports ρ = b1γ∞ by learning-rate power and family, and panel (c) compares normalized threshold-induced intercept shifts with − log c. This remains empirical and conditional; the two-sided tail bound needed for a proof i… view at source ↗
Figure 5
Figure 5. Figure 5: Adaptive-method boundary diagnostics. The main paper reports a compact summary; these appendix panels show [PITH_FULL_IMAGE:figures/full_fig_p014_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Expanded fixed-ϵ crossover diagnostics. These panels stress-test the empirical slope relation by threshold subgroup, ϵ-range removal, and secondary iteration-time scaling. They are empirical sensitivity checks; the two-sided tail bound remains open. E3: optimizer boundary. E3 compares memoryless smoothed-sign dynamics, RMSProp, full Adam, and Adam with and without bias correction. The prespecified grid is … view at source ↗
Figure 7
Figure 7. Figure 7: Extended Theorem 1 validation diagnostics. These panels expand the main theorem-validation dashboard by showing [PITH_FULL_IMAGE:figures/full_fig_p016_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Primary-window finite-horizon margin diagnostics. Complementing the main dashboard and the extended validation [PITH_FULL_IMAGE:figures/full_fig_p017_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Finite-window convergence-factor scorecard. The two panels compare final/first factors for [PITH_FULL_IMAGE:figures/full_fig_p017_9.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

35 extracted references · 35 canonical work pages · 4 internal anchors

  1. [1]

    Journal of Machine Learning Research , volume =

    The Implicit Bias of Gradient Descent on Separable Data , author =. Journal of Machine Learning Research , volume =. 2018 , url =

    work page 2018

  2. [2]

    International Conference on Learning Representations , year =

    Adam: A Method for Stochastic Optimization , author =. International Conference on Learning Representations , year =

    work page

  3. [3]

    Advances in Neural Information Processing Systems , volume =

    The Marginal Value of Adaptive Gradient Methods in Machine Learning , author =. Advances in Neural Information Processing Systems , volume =. 2017 , url =

    work page 2017

  4. [4]

    International Conference on Learning Representations , year =

    On the Convergence of Adam and Beyond , author =. International Conference on Learning Representations , year =

    work page

  5. [5]

    Advances in Neural Information Processing Systems , volume =

    The Implicit Bias of Adam on Separable Data , author =. Advances in Neural Information Processing Systems , volume =. 2024 , doi =

    work page 2024

  6. [6]

    Advances in Neural Information Processing Systems , volume =

    Implicit Bias of Mirror Flow on Separable Data , author =. Advances in Neural Information Processing Systems , volume =. 2024 , doi =

    work page 2024

  7. [7]

    Advances in Neural Information Processing Systems , volume =

    Implicit Bias of Spectral Descent and Muon on Multiclass Separable Data , author =. Advances in Neural Information Processing Systems , volume =. 2025 , url =

    work page 2025

  8. [8]

    Proceedings of the Forty-First Conference on Uncertainty in Artificial Intelligence , pages =

    A Mirror Descent Perspective of Smoothed Sign Descent , author =. Proceedings of the Forty-First Conference on Uncertainty in Artificial Intelligence , pages =. 2025 , editor =

    work page 2025

  9. [9]

    Proceedings of the Thirty-Second Conference on Learning Theory , pages =

    The Implicit Bias of Gradient Descent on Nonseparable Data , author =. Proceedings of the Thirty-Second Conference on Learning Theory , pages =. 2019 , editor =

    work page 2019

  10. [10]

    Proceedings of Thirty Third Conference on Learning Theory , pages =

    Gradient Descent Follows the Regularization Path for General Losses , author =. Proceedings of Thirty Third Conference on Learning Theory , pages =. 2020 , editor =

    work page 2020

  11. [11]

    Journal of Machine Learning Research , volume =

    Adaptive Subgradient Methods for Online Learning and Stochastic Optimization , author =. Journal of Machine Learning Research , volume =. 2011 , url =

    work page 2011

  12. [12]

    Proceedings of the 35th International Conference on Machine Learning , pages =

    Characterizing Implicit Bias in Terms of Optimization Geometry , author =. Proceedings of the 35th International Conference on Machine Learning , pages =. 2018 , editor =

    work page 2018

  13. [13]

    Mirror Descent Maximizes Generalized Margin and Can Be Implemented Efficiently

    Mirror Descent Maximizes Generalized Margin and Can Be Implemented Efficiently , author =. arXiv preprint arXiv:2205.12808 , year =

    work page internal anchor Pith review Pith/arXiv arXiv

  14. [14]

    The Implicit Bias of

    Qian, Qian and Qian, Xiaoyuan , journal =. The Implicit Bias of. 2019 , doi =

    work page 2019

  15. [15]

    Proceedings of the 38th International Conference on Machine Learning , pages =

    The Implicit Bias for Adaptive Optimization Algorithms on Homogeneous Neural Networks , author =. Proceedings of the 38th International Conference on Machine Learning , pages =. 2021 , editor =

    work page 2021

  16. [16]

    and Klusowski, Jason Matthew and Shigida, Boris , booktitle =

    Cattaneo, Matias D. and Klusowski, Jason Matthew and Shigida, Boris , booktitle =. On the Implicit Bias of. 2024 , editor =

    work page 2024

  17. [17]

    Proceedings of the Twenty-Second International Conference on Artificial Intelligence and Statistics , pages =

    Stochastic Gradient Descent on Separable Data: Exact Convergence with a Fixed Learning Rate , author =. Proceedings of the Twenty-Second International Conference on Artificial Intelligence and Statistics , pages =. 2019 , editor =

    work page 2019

  18. [18]

    International Conference on Learning Representations , year =

    Gradient Descent Maximizes the Margin of Homogeneous Neural Networks , author =. International Conference on Learning Representations , year =

    work page

  19. [19]

    USSR Computational Mathematics and Mathematical Physics , volume =

    The Relaxation Method of Finding the Common Point of Convex Sets and Its Application to the Solution of Problems in Convex Programming , author =. USSR Computational Mathematics and Mathematical Physics , volume =. 1967 , doi =

    work page 1967

  20. [20]

    Operations Research Letters , volume =

    Mirror Descent and Nonlinear Projected Subgradient Methods for Convex Optimization , author =. Operations Research Letters , volume =. 2003 , doi =

    work page 2003

  21. [21]

    2004 , url =

    Convex Optimization , author =. 2004 , url =

    work page 2004

  22. [22]

    Foundations and Trends in Machine Learning , volume =

    Convex Optimization: Algorithms and Complexity , author =. Foundations and Trends in Machine Learning , volume =. 2015 , doi =

    work page 2015

  23. [23]

    Information and Computation , volume =

    Exponentiated Gradient versus Gradient Descent for Linear Predictors , author =. Information and Computation , volume =

    work page

  24. [24]

    Theory of Computing , volume =

    The Multiplicative Weights Update Method: A Meta-Algorithm and Applications , author =. Theory of Computing , volume =. 2012 , doi =

    work page 2012

  25. [25]

    Journal of Machine Learning Research , volume =

    Boosting as a Regularized Path to a Maximum Margin Classifier , author =. Journal of Machine Learning Research , volume =. 2004 , url =

    work page 2004

  26. [26]

    Proceedings of the 30th International Conference on Machine Learning , pages =

    Margins, Shrinkage, and Boosting , author =. Proceedings of the 30th International Conference on Machine Learning , pages =. 2013 , editor =

    work page 2013

  27. [27]

    Proceedings of the Twenty-Second International Conference on Artificial Intelligence and Statistics , pages =

    Convergence of Gradient Descent on Separable Data , author =. Proceedings of the Twenty-Second International Conference on Artificial Intelligence and Statistics , pages =. 2019 , editor =

    work page 2019

  28. [28]

    Lexicographic and Depth-Sensitive Margins in Homogeneous and Non-Homogeneous Deep Models

    Lexicographic and Depth-Sensitive Margins in Homogeneous and Non-Homogeneous Deep Models , author =. arXiv preprint arXiv:1905.07325 , year =

    work page internal anchor Pith review Pith/arXiv arXiv 1905

  29. [29]

    Proceedings of Thirty Third Conference on Learning Theory , pages =

    Implicit Bias of Gradient Descent for Wide Two-layer Neural Networks Trained with the Logistic Loss , author =. Proceedings of Thirty Third Conference on Learning Theory , pages =. 2020 , editor =

    work page 2020

  30. [30]

    On Margin Maximization in Linear and

    Vardi, Gal and Shamir, Ohad and Srebro, Nathan , journal =. On Margin Maximization in Linear and. 2021 , url =

    work page 2021

  31. [31]

    Gradient Descent on Two-layer Nets: Margin Maximization and Simplicity Bias

    Gradient Descent on Two-layer Nets: Margin Maximization and Simplicity Bias , author =. arXiv preprint arXiv:2110.13905 , year =

    work page internal anchor Pith review Pith/arXiv arXiv

  32. [32]

    Bernstein, Jeremy and Wang, Yu-Xiang and Azizzadenesheli, Kamyar and Anandkumar, Animashree , booktitle =. sign. 2018 , editor =

    work page 2018

  33. [33]

    Proceedings of the 35th International Conference on Machine Learning , pages =

    Dissecting Adam: The Sign, Magnitude and Variance of Stochastic Gradients , author =. Proceedings of the 35th International Conference on Machine Learning , pages =. 2018 , editor =

    work page 2018

  34. [34]

    A Simple Convergence Proof of Adam and Adagrad

    D. A Simple Convergence Proof of. arXiv preprint arXiv:2003.02395 , year =

    work page internal anchor Pith review Pith/arXiv arXiv 2003

  35. [35]

    Convex Analysis , author =

    work page

This paper was first reviewed by grok-4.5 on July 8, 2026.