{"record_type":"pith_number_record","schema_url":"https://pith.science/schemas/pith-number/v1.json","pith_number":"pith:2017:I67QLZK4Y7NFKC7T32F2ZNO7XC","short_pith_number":"pith:I67QLZK4","schema_version":"1.0","canonical_sha256":"47bf05e55cc7da550bf3de8bacb5dfb88104da2489a575a9e3e46e749db40899","source":{"kind":"arxiv","id":"1706.02677","version":2},"attestation_state":"computed","paper":{"title":"Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour","license":"http://arxiv.org/licenses/nonexclusive-distrib/1.0/","headline":"ResNet-50 reaches full ImageNet accuracy when trained with 8192-image minibatches on 256 GPUs in one hour.","cross_cats":["cs.DC","cs.LG"],"primary_cat":"cs.CV","authors_text":"Aapo Kyrola, Andrew Tulloch, Kaiming He, Lukasz Wesolowski, Pieter Noordhuis, Piotr Doll\\'ar, Priya Goyal, Ross Girshick, Yangqing Jia","submitted_at":"2017-06-08T16:51:53Z","abstract_excerpt":"Deep learning thrives with large neural networks and large datasets. However, larger networks and larger datasets result in longer training times that impede research and development progress. Distributed synchronous SGD offers a potential solution to this problem by dividing SGD minibatches over a pool of parallel workers. Yet to make this scheme efficient, the per-worker workload must be large, which implies nontrivial growth in the SGD minibatch size. In this paper, we empirically show that on the ImageNet dataset large minibatches cause optimization difficulties, but when these are address"},"verification_status":{"content_addressed":true,"pith_receipt":true,"author_attested":false,"weak_author_claims":0,"strong_author_claims":0,"externally_anchored":false,"storage_verified":false,"citation_signatures":0,"replication_records":0,"graph_snapshot":true,"references_resolved":true,"formal_links_present":true},"canonical_record":{"source":{"id":"1706.02677","kind":"arxiv","version":2},"metadata":{"license":"http://arxiv.org/licenses/nonexclusive-distrib/1.0/","primary_cat":"cs.CV","submitted_at":"2017-06-08T16:51:53Z","cross_cats_sorted":["cs.DC","cs.LG"],"title_canon_sha256":"3314173ec920e4e77e84092359619c2ec366f58bc77016c3ee2761b7473aaf9b","abstract_canon_sha256":"c070a1b2d0ad9e98b7899ef89dcf650c67ce3a40bd5e302be4d1ede6ab8a8cfb"},"schema_version":"1.0"},"receipt":{"kind":"pith_receipt","key_id":"pith-v1-2026-05","algorithm":"ed25519","signed_at":"2026-07-04T22:44:11.149734Z","signature_b64":"WwPW5y5/PLlmJG58KuGtDulvd7mi3rUglxoiTvwgWRdLW3z/1bBrGcBgAyeuZsnBLhZaMRP+aLg+RchHjsQjAg==","signed_message":"canonical_sha256_bytes","builder_version":"pith-number-builder-2026-05-17-v1","receipt_version":"0.3","canonical_sha256":"47bf05e55cc7da550bf3de8bacb5dfb88104da2489a575a9e3e46e749db40899","last_reissued_at":"2026-07-04T22:44:11.149234Z","signature_status":"signed_v1","first_computed_at":"2026-07-04T22:44:11.149234Z","public_key_fingerprint":"8d4b5ee74e4693bcd1df2446408b0d54"},"graph_snapshot":{"paper":{"title":"Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour","license":"http://arxiv.org/licenses/nonexclusive-distrib/1.0/","headline":"ResNet-50 reaches full ImageNet accuracy when trained with 8192-image minibatches on 256 GPUs in one hour.","cross_cats":["cs.DC","cs.LG"],"primary_cat":"cs.CV","authors_text":"Aapo Kyrola, Andrew Tulloch, Kaiming He, Lukasz Wesolowski, Pieter Noordhuis, Piotr Doll\\'ar, Priya Goyal, Ross Girshick, Yangqing Jia","submitted_at":"2017-06-08T16:51:53Z","abstract_excerpt":"Deep learning thrives with large neural networks and large datasets. However, larger networks and larger datasets result in longer training times that impede research and development progress. Distributed synchronous SGD offers a potential solution to this problem by dividing SGD minibatches over a pool of parallel workers. Yet to make this scheme efficient, the per-worker workload must be large, which implies nontrivial growth in the SGD minibatch size. In this paper, we empirically show that on the ImageNet dataset large minibatches cause optimization difficulties, but when these are address"},"claims":{"count":4,"items":[{"kind":"strongest_claim","text":"we show no loss of accuracy when training with large minibatch sizes up to 8192 images... our Caffe2-based system trains ResNet-50 with a minibatch size of 8192 on 256 GPUs in one hour, while matching small minibatch accuracy.","source":"verdict.strongest_claim","status":"machine_extracted","claim_id":"C1","attestation":"unclaimed"},{"kind":"weakest_assumption","text":"That the only obstacles to large-minibatch training are early optimization instability and learning-rate magnitude, which can be fixed by a hyper-parameter-free linear scaling rule plus a warmup schedule without harming final generalization on ImageNet.","source":"verdict.weakest_assumption","status":"machine_extracted","claim_id":"C2","attestation":"unclaimed"},{"kind":"one_line_summary","text":"Linear learning-rate scaling plus warmup lets minibatch size 8192 train ResNet-50 on ImageNet in one hour at full small-batch accuracy.","source":"verdict.one_line_summary","status":"machine_extracted","claim_id":"C3","attestation":"unclaimed"},{"kind":"headline","text":"ResNet-50 reaches full ImageNet accuracy when trained with 8192-image minibatches on 256 GPUs in one hour.","source":"verdict.pith_extraction.headline","status":"machine_extracted","claim_id":"C4","attestation":"unclaimed"}],"snapshot_sha256":"fcc9be90ef5713cc0ace24f3c1fbc34a2bf999dd13700f2567044adb60e10950"},"source":{"id":"1706.02677","kind":"arxiv","version":2},"verdict":{"id":"e73ca86e-d4a6-40a8-bc42-d72bfff13e1e","model_set":{"reader":"grok-4.3"},"created_at":"2026-05-12T07:15:48.006355Z","strongest_claim":"we show no loss of accuracy when training with large minibatch sizes up to 8192 images... our Caffe2-based system trains ResNet-50 with a minibatch size of 8192 on 256 GPUs in one hour, while matching small minibatch accuracy.","one_line_summary":"Linear learning-rate scaling plus warmup lets minibatch size 8192 train ResNet-50 on ImageNet in one hour at full small-batch accuracy.","pipeline_version":"pith-pipeline@v0.9.0","weakest_assumption":"That the only obstacles to large-minibatch training are early optimization instability and learning-rate magnitude, which can be fixed by a hyper-parameter-free linear scaling rule plus a warmup schedule without harming final generalization on ImageNet.","pith_extraction_headline":"ResNet-50 reaches full ImageNet accuracy when trained with 8192-image minibatches on 256 GPUs in one hour."},"integrity":{"clean":true,"summary":{"advisory":0,"critical":0,"by_detector":{},"informational":0},"endpoint":"/pith/1706.02677/integrity.json","findings":[],"available":true,"detectors_run":[],"snapshot_sha256":"c28c3603d3b5d939e8dc4c7e95fa8dfce3d595e45f758748cecf8e644a296938"},"references":{"count":40,"sample":[{"doi":"","year":2016,"title":"J. Bagga, H. Morsy, and Z. Yao. Opening designs for 6-pack and Wedge 100. https: //code.facebook.com/posts/203733993317833/ opening-designs-for-6-pack-and-wedge-100 , 2016","work_id":"8c5d389c-2ab0-482e-86a4-f9371564b08c","ref_index":1,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":1994,"title":"M. Barnett, L. Shuler, R. van De Geijn, S. Gupta, D. G. Payne, and J. Watts. Interprocessor collective communica- tion library (intercom). In Scalable High-Performance Com- puting Conference, 1994","work_id":"6f03f630-987b-4dde-91e0-67e4beec3cf7","ref_index":2,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2009,"title":"L. Bottou. Curiously fast convergence of some stochastic gradient descent algorithms. Unpublished open problem of- fered to the attendance of the SLDS 2009 conference, 2009","work_id":"adaf2c4f-7afc-4471-ac8c-ab344b2c2b69","ref_index":3,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2016,"title":"Optimization Methods for Large-Scale Machine Learning","work_id":"515ad74f-f13a-41d0-b03a-48223cbe8738","ref_index":4,"cited_arxiv_id":"1606.04838","is_internal_anchor":false},{"doi":"","year":2016,"title":"Revisiting Distributed Synchronous SGD","work_id":"f5e67e6f-5433-4d17-b30a-7343938c7c20","ref_index":5,"cited_arxiv_id":"1604.00981","is_internal_anchor":false}],"resolved_work":40,"snapshot_sha256":"9eb7b24415c9725a4041fa48738cdfee8925e39c6dc3f022c559920ca4145edb","internal_anchors":1},"formal_canon":{"evidence_count":2,"snapshot_sha256":"7be963967129ef25bbb705aa06f3fe81b9745ca88b68728eeacb2f3a2ea61382"},"author_claims":{"count":0,"strong_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"builder_version":"pith-number-builder-2026-05-17-v1"},"aliases":[{"alias_kind":"arxiv","alias_value":"1706.02677","created_at":"2026-07-04T22:44:11.149291+00:00"},{"alias_kind":"arxiv_version","alias_value":"1706.02677v2","created_at":"2026-07-04T22:44:11.149291+00:00"},{"alias_kind":"doi","alias_value":"10.48550/arxiv.1706.02677","created_at":"2026-07-04T22:44:11.149291+00:00"},{"alias_kind":"pith_short_12","alias_value":"I67QLZK4Y7NF","created_at":"2026-07-04T22:44:11.149291+00:00"},{"alias_kind":"pith_short_16","alias_value":"I67QLZK4Y7NFKC7T","created_at":"2026-07-04T22:44:11.149291+00:00"},{"alias_kind":"pith_short_8","alias_value":"I67QLZK4","created_at":"2026-07-04T22:44:11.149291+00:00"}],"events":[],"event_summary":{},"paper_claims":[],"inbound_citations":{"count":111,"internal_anchor_count":111,"sample":[{"citing_arxiv_id":"2606.25927","citing_title":"Knowledge Cascade: Reverse Knowledge Distillation on Nonparametric Multivariate Functional Estimation","ref_index":5,"is_internal_anchor":true},{"citing_arxiv_id":"2606.25971","citing_title":"Improving Neural Network Training by Decoupling the Magnitude and Direction of Weight Vectors","ref_index":154,"is_internal_anchor":true},{"citing_arxiv_id":"2606.25962","citing_title":"A Benchmark for Heterogeneous Stereo Deblurring with Physically- and Epipolar-constrained Cross Attention","ref_index":5,"is_internal_anchor":true},{"citing_arxiv_id":"2606.25627","citing_title":"TL++: Accuracy and Privacy Preserving Traversal Learning for Distributed Intelligent Systems","ref_index":18,"is_internal_anchor":true},{"citing_arxiv_id":"2606.23346","citing_title":"Field-level weak lensing cosmology with $<100$ simulations using multifidelity simulation-based inference","ref_index":199,"is_internal_anchor":true},{"citing_arxiv_id":"2606.24901","citing_title":"LLM Evolution as an Industry-Scale Ecosystem: A Lifecycle Perspective on Continual Learning","ref_index":32,"is_internal_anchor":true},{"citing_arxiv_id":"2606.13625","citing_title":"Revisiting Vehicle Color Recognition in Long-Tailed Surveillance Scenarios","ref_index":8,"is_internal_anchor":true},{"citing_arxiv_id":"2606.11431","citing_title":"Mirror Descent Beyond Euclidean Stability: An Exponential Separation in Initialization Sensitivity","ref_index":45,"is_internal_anchor":true},{"citing_arxiv_id":"2606.09044","citing_title":"A Hybrid Generative Reduced-Order Model for the Minimal Flow Unit","ref_index":64,"is_internal_anchor":true},{"citing_arxiv_id":"2607.00173","citing_title":"TallyTrain: Communication-Efficient Federated Distillation","ref_index":43,"is_internal_anchor":true},{"citing_arxiv_id":"2606.06624","citing_title":"Principles and Practice of Deep Representation Learning: or a Mathematical Theory of Memory","ref_index":30,"is_internal_anchor":true},{"citing_arxiv_id":"2606.02980","citing_title":"A Training-Efficient Transformer-Based Anti-Spoofing Network for Logical Access in ASVspoof 5","ref_index":17,"is_internal_anchor":true},{"citing_arxiv_id":"2606.03052","citing_title":"What Do Students Learn? A Feature-Level Analysis of Dark Knowledge","ref_index":24,"is_internal_anchor":true},{"citing_arxiv_id":"2606.03498","citing_title":"Demystifying Pipeline Parallelism: First Theory for PipeDream","ref_index":4,"is_internal_anchor":true},{"citing_arxiv_id":"2606.02927","citing_title":"SaluNet: Enabling Total Plasticity in Normalization-Free Deep Networks","ref_index":16,"is_internal_anchor":true},{"citing_arxiv_id":"2606.01128","citing_title":"Local MixVR: Breaking the Communication-Sample Dependence in Distributed Learning","ref_index":4,"is_internal_anchor":true},{"citing_arxiv_id":"2606.00293","citing_title":"Accurate Large-sample Uncertainty Quantification using Stochastic Gradient Markov Chain Monte Carlo","ref_index":18,"is_internal_anchor":true},{"citing_arxiv_id":"2606.28486","citing_title":"Spectral phase transitions and trainability in neural network learning dynamics","ref_index":60,"is_internal_anchor":true},{"citing_arxiv_id":"2606.31532","citing_title":"A time-series classification framework for individual-level absenteeism prediction under severe class imbalance","ref_index":10,"is_internal_anchor":true},{"citing_arxiv_id":"2605.19811","citing_title":"LionMuon: Alternating Spectral and Sign Descent for Efficient Training","ref_index":4,"is_internal_anchor":true},{"citing_arxiv_id":"2605.24316","citing_title":"From One-Pass SGD to Data Reuse: Mini-Batch Scaling Laws in Sketched Linear Regression","ref_index":3,"is_internal_anchor":true},{"citing_arxiv_id":"2606.02980","citing_title":"A Training-Efficient Transformer-Based Anti-Spoofing Network for Logical Access in ASVspoof 5","ref_index":17,"is_internal_anchor":true},{"citing_arxiv_id":"2606.29723","citing_title":"ScaleAware-JEPA: Latent Representation for Discovery in Multiscale Physical Fields","ref_index":43,"is_internal_anchor":true},{"citing_arxiv_id":"2606.30559","citing_title":"Convergence of Continual Learning in Homogeneous Deep Networks","ref_index":57,"is_internal_anchor":true},{"citing_arxiv_id":"2605.26248","citing_title":"Unified Neural Scaling Laws","ref_index":8,"is_internal_anchor":true}]},"formal_canon":{"evidence_count":2,"sample":[],"anchors":[]},"links":{"html":"https://pith.science/pith/I67QLZK4Y7NFKC7T32F2ZNO7XC","json":"https://pith.science/pith/I67QLZK4Y7NFKC7T32F2ZNO7XC.json","graph_json":"https://pith.science/api/pith-number/I67QLZK4Y7NFKC7T32F2ZNO7XC/graph.json","events_json":"https://pith.science/api/pith-number/I67QLZK4Y7NFKC7T32F2ZNO7XC/events.json","paper":"https://pith.science/paper/I67QLZK4"},"agent_actions":{"view_html":"https://pith.science/pith/I67QLZK4Y7NFKC7T32F2ZNO7XC","download_json":"https://pith.science/pith/I67QLZK4Y7NFKC7T32F2ZNO7XC.json","view_paper":"https://pith.science/paper/I67QLZK4","resolve_alias":"https://pith.science/api/pith-number/resolve?arxiv=1706.02677&json=true","fetch_graph":"https://pith.science/api/pith-number/I67QLZK4Y7NFKC7T32F2ZNO7XC/graph.json","fetch_events":"https://pith.science/api/pith-number/I67QLZK4Y7NFKC7T32F2ZNO7XC/events.json","actions":{"anchor_timestamp":"https://pith.science/pith/I67QLZK4Y7NFKC7T32F2ZNO7XC/action/timestamp_anchor","attest_storage":"https://pith.science/pith/I67QLZK4Y7NFKC7T32F2ZNO7XC/action/storage_attestation","attest_author":"https://pith.science/pith/I67QLZK4Y7NFKC7T32F2ZNO7XC/action/author_attestation","sign_citation":"https://pith.science/pith/I67QLZK4Y7NFKC7T32F2ZNO7XC/action/citation_signature","submit_replication":"https://pith.science/pith/I67QLZK4Y7NFKC7T32F2ZNO7XC/action/replication_record"}},"created_at":"2026-07-04T22:44:11.149291+00:00","updated_at":"2026-07-04T22:44:11.149291+00:00"}