Pith Number

pith:I67QLZK4

pith:2017:I67QLZK4Y7NFKC7T32F2ZNO7XC

not attested not anchored not stored refs resolved

Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour

Aapo Kyrola, Andrew Tulloch, Kaiming He, Lukasz Wesolowski, Pieter Noordhuis, Piotr Doll\'ar, Priya Goyal, Ross Girshick, Yangqing Jia

ResNet-50 reaches full ImageNet accuracy when trained with 8192-image minibatches on 256 GPUs in one hour.

arxiv:1706.02677 v2 · 2017-06-08 · cs.CV · cs.DC · cs.LG

Open paper page JSON Open Graph Bundle Merged state Verified badge What is a Pith Number?

Add to your LaTeX paper

\usepackage{pith}
\pithnumber{I67QLZK4Y7NFKC7T32F2ZNO7XC}

Prints a linked badge after your title and injects PDF metadata. Compiles on arXiv. Learn more · Embed verified badge

Record completeness

1 Bitcoin timestamp

2 Internet Archive

3 Author claim open · sign in to claim

4 Citations open

5 Replications open

✓ Portable graph bundle live · download bundle · merged state

The bundle contains the canonical record plus signed events. A mirror can host it anywhere and recompute the same current state with the deterministic merge algorithm.

Claims

C1strongest claim

we show no loss of accuracy when training with large minibatch sizes up to 8192 images... our Caffe2-based system trains ResNet-50 with a minibatch size of 8192 on 256 GPUs in one hour, while matching small minibatch accuracy.

C2weakest assumption

That the only obstacles to large-minibatch training are early optimization instability and learning-rate magnitude, which can be fixed by a hyper-parameter-free linear scaling rule plus a warmup schedule without harming final generalization on ImageNet.

C3one line summary

Linear learning-rate scaling plus warmup lets minibatch size 8192 train ResNet-50 on ImageNet in one hour at full small-batch accuracy.

References

40 extracted · 40 resolved · 1 Pith anchors

[1] J. Bagga, H. Morsy, and Z. Yao. Opening designs for 6-pack and Wedge 100. https: //code.facebook.com/posts/203733993317833/ opening-designs-for-6-pack-and-wedge-100 , 2016 2016

[2] M. Barnett, L. Shuler, R. van De Geijn, S. Gupta, D. G. Payne, and J. Watts. Interprocessor collective communica- tion library (intercom). In Scalable High-Performance Com- puting Conference, 1994 1994

[3] L. Bottou. Curiously fast convergence of some stochastic gradient descent algorithms. Unpublished open problem of- fered to the attendance of the SLDS 2009 conference, 2009 2009

[4] Optimization Methods for Large-Scale Machine Learning 2016 · arXiv:1606.04838

[5] Revisiting Distributed Synchronous SGD 2016 · arXiv:1604.00981

Formal links

2 machine-checked theorem links

Cited by

111 papers in Pith

Knowledge Cascade: Reverse Knowledge Distillation on Nonparametric Multivariate Functional Estimation

Improving Neural Network Training by Decoupling the Magnitude and Direction of Weight Vectors

A Benchmark for Heterogeneous Stereo Deblurring with Physically- and Epipolar-constrained Cross Attention

TL++: Accuracy and Privacy Preserving Traversal Learning for Distributed Intelligent Systems

Field-level weak lensing cosmology with $<100$ simulations using multifidelity simulation-based inference

Receipt and verification

First computed	2026-07-04T22:44:11.149234Z
Builder	pith-number-builder-2026-05-17-v1
Signature	Pith Ed25519 (`pith-v1-2026-05`) · public key
Schema	pith-number/v1.0

Canonical hash

47bf05e55cc7da550bf3de8bacb5dfb88104da2489a575a9e3e46e749db40899

Aliases

arxiv: 1706.02677 · arxiv_version: 1706.02677v2 · doi: 10.48550/arxiv.1706.02677 · pith_short_12: I67QLZK4Y7NF · pith_short_16: I67QLZK4Y7NFKC7T · pith_short_8: I67QLZK4

Agent API

Resolver JSON Graph JSON Events JSON Schema Signing key

Verify this Pith Number yourself

curl -sH 'Accept: application/ld+json' https://pith.science/pith/I67QLZK4Y7NFKC7T32F2ZNO7XC \
  | jq -c '.canonical_record' \
  | python3 -c "import sys,json,hashlib; b=json.dumps(json.loads(sys.stdin.read()), sort_keys=True, separators=(',',':'), ensure_ascii=False).encode(); print(hashlib.sha256(b).hexdigest())"
# expect: 47bf05e55cc7da550bf3de8bacb5dfb88104da2489a575a9e3e46e749db40899

Canonical record JSON

{
  "metadata": {
    "abstract_canon_sha256": "c070a1b2d0ad9e98b7899ef89dcf650c67ce3a40bd5e302be4d1ede6ab8a8cfb",
    "cross_cats_sorted": [
      "cs.DC",
      "cs.LG"
    ],
    "license": "http://arxiv.org/licenses/nonexclusive-distrib/1.0/",
    "primary_cat": "cs.CV",
    "submitted_at": "2017-06-08T16:51:53Z",
    "title_canon_sha256": "3314173ec920e4e77e84092359619c2ec366f58bc77016c3ee2761b7473aaf9b"
  },
  "schema_version": "1.0",
  "source": {
    "id": "1706.02677",
    "kind": "arxiv",
    "version": 2
  }
}