Pith. sign in

REVIEW 4 cited by

Stochastic Weight Averaging in Parallel: Large-Batch Training that Generalizes Well

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2001.02312 v1 pith:HCEV2IUB submitted 2020-01-07 cs.LG stat.ML

Vipul Gupta , Santiago Akle Serrano , Dennis DeCoste This is my paper · ORCID
classification cs.LGstat.ML
keywords averagingmodelsparalleltrainingalgorithmmini-batchesresultingstochastic
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

We propose Stochastic Weight Averaging in Parallel (SWAP), an algorithm to accelerate DNN training. Our algorithm uses large mini-batches to compute an approximate solution quickly and then refines it by averaging the weights of multiple models computed independently and in parallel. The resulting models generalize equally well as those trained with small mini-batches but are produced in a substantially shorter time. We demonstrate the reduction in training time and the good generalization performance of the resulting models on the computer vision datasets CIFAR10, CIFAR100, and ImageNet.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Enough is as good as a feast: A Comprehensive Analysis of How Reinforcement Learning Mitigates Task Conflicts in LLMs

    cs.CL 2026-07 conditional novelty 6.0 of 10

    RL-trained LLMs keep most of their skills after weight merging, while SFT-trained LLMs drop about 19% on average, because RL keeps parameter updates smaller and more task-compatible.

  2. Task Arithmetic Through The Lens Of One-Shot Federated Learning

    cs.LG 2024-11 conditional novelty 5.0 of 10

    Task arithmetic is exactly one-shot FedAvg with outer step size beta = lambda T, and FedNova, FedGMA, Median, and CCLIP can often improve merged model performance.

  3. CLUES: Collaborative High-Quality Data Selection for LLMs via Training Dynamics

    cs.CL 2025-07 conditional novelty 4.0 of 10

    A collaborative data-selection method that scores each private sample's influence on a public anchor set and filters by a global threshold before federated learning or model merging.

  4. Weight Averaging for Out-of-Distribution Generalization and Few-Shot Domain Adaptation

    cs.CV 2025-01 reject novelty 4.0 of 10

    Gradient-similarity-regularized weight averaging and WA+SAM fine-tuning are tested on OOD and few-shot domain adaptation benchmarks, with mixed results that do not support the claimed improvements.

Pith tools