REVIEW 2 cited by
Optimal Convergence for Distributed Learning with Stochastic Gradient Methods and Spectral Algorithms
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
We study generalization properties of distributed algorithms in the setting of nonparametric regression over a reproducing kernel Hilbert space (RKHS). We first investigate distributed stochastic gradient methods (SGM), with mini-batches and multi-passes over the data. We show that optimal generalization error bounds can be retained for distributed SGM provided that the partition level is not too large. We then extend our results to spectral-regularization algorithms (SRA), including kernel ridge regression (KRR), kernel principal component analysis, and gradient methods. Our results are superior to the state-of-the-art theory. Particularly, our results show that distributed SGM has a smaller theoretical computational complexity, compared with distributed KRR and classic SGM. Moreover, even for non-distributed SRA, they provide the first optimal, capacity-dependent convergence rates, considering the case that the regression function may not be in the RKHS.
Forward citations
Cited by 2 Pith papers
-
Random feature approximation for general spectral methods
Under source conditions with smoothness r>0 and capacity 2r+b>1, random features achieve minimax-optimal rates for any spectral regularization method with qualification at least r∨1.
-
Optimal Convergence Rates for Neural Operators
Two-layer neural operators trained with early-stopped gradient descent achieve the same minimax convergence rates as kernel methods in the neural tangent kernel regime.
Discussion (0). Continue with ORCID to comment.