pith. sign in

Gossip training for deep learning

Michael Blot, David Picard, Matthieu Cord, Nicolas Thome · 2016 · cs.CV · arXiv 1611.09726

2 Pith papers cite this work. Polarity classification is still indexing.

2 Pith papers citing it
open full Pith review browse 2 citing papers arXiv PDF
abstract

We address the issue of speeding up the training of convolutional networks. Here we study a distributed method adapted to stochastic gradient descent (SGD). The parallel optimization setup uses several threads, each applying individual gradient descents on a local variable. We propose a new way to share information between different threads inspired by gossip algorithms and showing good consensus convergence properties. Our method called GoSGD has the advantage to be fully asynchronous and decentralized. We compared our method to the recent EASGD in \cite{elastic} on CIFAR-10 show encouraging results.

fields

cs.LG 1 math.OC 1

years

2026 2

verdicts

UNVERDICTED 2

clear filters

representative citing papers

SUDA-Muon: Structural Design Principles and Boundaries for Fully Decentralized Muon

math.OC · 2026-04-27 · unverdicted · novelty 6.0

SUDA-Muon modularizes decentralized Muon via the SUDA template, proving a topology-separated convergence rate of O((1+σ/√N)K^{-1/4}) in nuclear-norm geometry while establishing that tracking-before-polarization is required to avoid non-stationary fixed points and that local-polarize-then-average is

Factored Gossip DiLoCo: Reducing Blocking Communication in DiLoCo

cs.LG · 2026-06-22 · unverdicted · novelty 5.0

Factored Gossip DiLoCo relaxes exact outer synchronization in DiLoCo to approximate gossip-based mixing, enabling non-blocking steps and a tunable trade-off between compute utilization and stability on up to billion-parameter models.

citing papers explorer

Showing 2 of 2 citing papers after filters.

  • SUDA-Muon: Structural Design Principles and Boundaries for Fully Decentralized Muon math.OC · 2026-04-27 · unverdicted · none · ref 2

    SUDA-Muon modularizes decentralized Muon via the SUDA template, proving a topology-separated convergence rate of O((1+σ/√N)K^{-1/4}) in nuclear-norm geometry while establishing that tracking-before-polarization is required to avoid non-stationary fixed points and that local-polarize-then-average is

  • Factored Gossip DiLoCo: Reducing Blocking Communication in DiLoCo cs.LG · 2026-06-22 · unverdicted · none · ref 42 · internal anchor

    Factored Gossip DiLoCo relaxes exact outer synchronization in DiLoCo to approximate gossip-based mixing, enabling non-blocking steps and a tunable trade-off between compute utilization and stability on up to billion-parameter models.