REVIEW 4 cited by
A simple uniformly optimal method without line search for convex optimization
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
abstract
Line search (or backtracking) procedures have been widely employed into first-order methods for solving convex optimization problems, especially those with unknown problem parameters (e.g., Lipschitz constant). In this paper, we show that line search is superfluous in attaining the optimal rate of convergence for solving a convex optimization problem whose parameters are not given a priori. In particular, we present a novel accelerated gradient descent type algorithm called auto-conditioned fast gradient method (AC-FGM) that can achieve an optimal $\mathcal{O}(1/k^2)$ rate of convergence for smooth convex optimization without requiring the estimate of a global Lipschitz constant or the employment of line search procedures. We then extend AC-FGM to solve convex optimization problems with H\"{o}lder continuous gradients and show that it automatically achieves the optimal rates of convergence uniformly for all problem classes with the desired accuracy of the solution as the only input. Finally, we report some encouraging numerical results that demonstrate the advantages of AC-FGM over the previously developed parameter-free methods for convex optimization.
Forward citations
Cited by 4 Pith papers
-
Nesterov Finds GRAAL: Optimal and Adaptive Gradient Method for Convex Optimization
Accelerated GRAAL is the first adaptive first-order method that proves near-optimal accelerated complexity for convex L-smooth and (L0,L1)-smooth functions with geometric stepsize growth.
-
Gradient Methods with Online Scaling Part I. Theoretical Foundations
Online scaled gradient methods adapt matrix step sizes via online learning, match the best fixed step size asymptotically, and achieve non-asymptotic superlinear convergence on smooth strongly convex problems.
-
Online Learning-guided Learning Rate Adaptation via Gradient Alignment
GALA adapts the learning rate through online learning on gradient alignment, achieves a data-adaptive convergence rate for normalized SGD, and shows robust empirical performance across initial learning rates.
-
Doubly Smoothed Optimistic Gradients: A Universal Approach for Smooth Minimax Problems
DS-OGDA is a single-loop first-order method claimed to handle a broad class of smooth minimax problems with a universal step-size schedule and best-known iteration complexity in each subclass.
Discussion (0). Continue with ORCID to comment.