Non-quadratic Mirror Descent exhibits exponential initialization sensitivity in convex settings, shown via 3D constructions and KL-regularized simplex examples, with Bregman anchoring proposed for stabilization.
Risk and parameter convergence of logistic regression
5 Pith papers cite this work. Polarity classification is still indexing.
5
Pith papers citing it
abstract
Gradient descent, when applied to the task of logistic regression, outputs iterates which are biased to follow a unique ray defined by the data. The direction of this ray is the maximum margin predictor of a maximal linearly separable subset of the data; the gradient descent iterates converge to this ray in direction at the rate $\mathcal{O}(\ln\ln t / \ln t)$. The ray does not pass through the origin in general, and its offset is the bounded global optimum of the risk over the remaining data; gradient descent recovers this offset at a rate $\mathcal{O}((\ln t)^2 / \sqrt{t})$.
fields
cs.LG 5representative citing papers
EW with Gaussian prior matches the optimal O(d log(Bn)) regret for online logistic regression at O(B^3 n^5) cost and converges geometrically to a truncated Gaussian vote in the large-B separable regime.
Continual classification in homogeneous models is sequential projections onto margin sets, with local linear convergence under regularity properties for random and cyclic tasks, extended to regression.
Mini-batch noise reverses how Adam's β2 controls anti-regularization, making default momentum values suitable for small batches but requiring β1 closer to β2 for large batches to favor flatter minima.
Gradient descent on logistic loss for two-layer nonlinear models reduces to perceptron updates, yielding provable Õ(√d) iteration complexity from nonlinearity versus Ω(d) for linear models.
citing papers explorer
-
Mirror Descent Beyond Euclidean Stability: An Exponential Separation in Initialization Sensitivity
Non-quadratic Mirror Descent exhibits exponential initialization sensitivity in convex settings, shown via 3D constructions and KL-regularized simplex examples, with Bregman anchoring proposed for stabilization.
-
Efficient Logistic Regression with Mixture of Sigmoids
EW with Gaussian prior matches the optimal O(d log(Bn)) regret for online logistic regression at O(B^3 n^5) cost and converges geometrically to a truncated Gaussian vote in the large-B separable regime.
-
Convergence of Continual Learning in Homogeneous Deep Networks
Continual classification in homogeneous models is sequential projections onto margin sets, with local linear convergence under regularity properties for random and cyclic tasks, extended to regression.
-
The Effect of Mini-Batch Noise on the Implicit Bias of Adam
Mini-batch noise reverses how Adam's β2 controls anti-regularization, making default momentum values suitable for small batches but requiring β1 closer to β2 for large batches to favor flatter minima.
-
Gradient Descent as a Perceptron Algorithm: Understanding Dynamics and Implicit Acceleration
Gradient descent on logistic loss for two-layer nonlinear models reduces to perceptron updates, yielding provable Õ(√d) iteration complexity from nonlinearity versus Ω(d) for linear models.