{"record_type":"pith_number_record","schema_url":"https://pith.science/schemas/pith-number/v1.json","pith_number":"pith:2017:M5BNRHPUWGA7MHPFDKCQKNVN6T","short_pith_number":"pith:M5BNRHPU","schema_version":"1.0","canonical_sha256":"6742d89df4b181f61de51a850536adf4fdf7a8a9c65a8fd6111076b17f2d4ebe","source":{"kind":"arxiv","id":"1711.04623","version":3},"attestation_state":"computed","paper":{"title":"Three Factors Influencing Minima in SGD","license":"http://arxiv.org/licenses/nonexclusive-distrib/1.0/","headline":"","cross_cats":["cs.AI","cs.CV","stat.ML"],"primary_cat":"cs.LG","authors_text":"Amos Storkey, Asja Fischer, Devansh Arpit, Nicolas Ballas, Stanis{\\l}aw Jastrz\\k{e}bski, Yoshua Bengio, Zachary Kenton","submitted_at":"2017-11-13T15:11:56Z","abstract_excerpt":"We investigate the dynamical and convergent properties of stochastic gradient descent (SGD) applied to Deep Neural Networks (DNNs). Characterizing the relation between learning rate, batch size and the properties of the final minima, such as width or generalization, remains an open question. In order to tackle this problem we investigate the previously proposed approximation of SGD by a stochastic differential equation (SDE). We theoretically argue that three factors - learning rate, batch size and gradient covariance - influence the minima found by SGD. In particular we find that the ratio of"},"verification_status":{"content_addressed":true,"pith_receipt":true,"author_attested":false,"weak_author_claims":0,"strong_author_claims":0,"externally_anchored":false,"storage_verified":false,"citation_signatures":0,"replication_records":0,"graph_snapshot":true,"references_resolved":false,"formal_links_present":false},"canonical_record":{"source":{"id":"1711.04623","kind":"arxiv","version":3},"metadata":{"license":"http://arxiv.org/licenses/nonexclusive-distrib/1.0/","primary_cat":"cs.LG","submitted_at":"2017-11-13T15:11:56Z","cross_cats_sorted":["cs.AI","cs.CV","stat.ML"],"title_canon_sha256":"8374a5494e65f1b9671443e920c94106ccb3751de54b7b0fb22be87c2beb9e16","abstract_canon_sha256":"422ac8d1f82e758fbdde6344531805f5be3c1f9e9c312b760582b691292c8113"},"schema_version":"1.0"},"receipt":{"kind":"pith_receipt","key_id":"pith-v1-2026-05","algorithm":"ed25519","signed_at":"2026-05-18T00:05:51.088728Z","signature_b64":"fP2OFxXEOBNgEQkyndIyq5a2KnNbadwB2qVlPEgvS5FTfw5x1r0BN1eB14cgl9ftHSWVnSqJCO8WdvyPxa4TDQ==","signed_message":"canonical_sha256_bytes","builder_version":"pith-number-builder-2026-05-17-v1","receipt_version":"0.3","canonical_sha256":"6742d89df4b181f61de51a850536adf4fdf7a8a9c65a8fd6111076b17f2d4ebe","last_reissued_at":"2026-05-18T00:05:51.088069Z","signature_status":"signed_v1","first_computed_at":"2026-05-18T00:05:51.088069Z","public_key_fingerprint":"8d4b5ee74e4693bcd1df2446408b0d54"},"graph_snapshot":{"paper":{"title":"Three Factors Influencing Minima in SGD","license":"http://arxiv.org/licenses/nonexclusive-distrib/1.0/","headline":"","cross_cats":["cs.AI","cs.CV","stat.ML"],"primary_cat":"cs.LG","authors_text":"Amos Storkey, Asja Fischer, Devansh Arpit, Nicolas Ballas, Stanis{\\l}aw Jastrz\\k{e}bski, Yoshua Bengio, Zachary Kenton","submitted_at":"2017-11-13T15:11:56Z","abstract_excerpt":"We investigate the dynamical and convergent properties of stochastic gradient descent (SGD) applied to Deep Neural Networks (DNNs). Characterizing the relation between learning rate, batch size and the properties of the final minima, such as width or generalization, remains an open question. In order to tackle this problem we investigate the previously proposed approximation of SGD by a stochastic differential equation (SDE). We theoretically argue that three factors - learning rate, batch size and gradient covariance - influence the minima found by SGD. In particular we find that the ratio of"},"claims":{"count":0,"items":[],"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"source":{"id":"1711.04623","kind":"arxiv","version":3},"verdict":{"id":null,"model_set":{},"created_at":null,"strongest_claim":"","one_line_summary":"","pipeline_version":null,"weakest_assumption":"","pith_extraction_headline":""},"references":{"count":0,"sample":[],"resolved_work":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57","internal_anchors":0},"formal_canon":{"evidence_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"author_claims":{"count":0,"strong_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"builder_version":"pith-number-builder-2026-05-17-v1"},"aliases":[{"alias_kind":"arxiv","alias_value":"1711.04623","created_at":"2026-05-18T00:05:51.088156+00:00"},{"alias_kind":"arxiv_version","alias_value":"1711.04623v3","created_at":"2026-05-18T00:05:51.088156+00:00"},{"alias_kind":"doi","alias_value":"10.48550/arxiv.1711.04623","created_at":"2026-05-18T00:05:51.088156+00:00"},{"alias_kind":"pith_short_12","alias_value":"M5BNRHPUWGA7","created_at":"2026-05-18T12:31:28.150371+00:00"},{"alias_kind":"pith_short_16","alias_value":"M5BNRHPUWGA7MHPF","created_at":"2026-05-18T12:31:28.150371+00:00"},{"alias_kind":"pith_short_8","alias_value":"M5BNRHPU","created_at":"2026-05-18T12:31:28.150371+00:00"}],"events":[],"event_summary":{},"paper_claims":[],"inbound_citations":{"count":18,"internal_anchor_count":13,"sample":[{"citing_arxiv_id":"1906.09069","citing_title":"First Exit Time Analysis of Stochastic Gradient Descent Under Heavy-Tailed Gradient Noise","ref_index":7,"is_internal_anchor":true},{"citing_arxiv_id":"2406.09241","citing_title":"What is the long-run distribution of stochastic gradient descent? A large deviations analysis","ref_index":30,"is_internal_anchor":true},{"citing_arxiv_id":"2408.02839","citing_title":"Mini-batch Estimation for Deep Cox Models: Statistical Foundations and Practical Guidance","ref_index":19,"is_internal_anchor":true},{"citing_arxiv_id":"2409.04777","citing_title":"Optimization Hyper-parameter Laws for Large Language Models","ref_index":20,"is_internal_anchor":true},{"citing_arxiv_id":"2605.20296","citing_title":"Spectral Unforgetting: Post-Hoc Recovery of Damaged Capabilities Without Retraining","ref_index":11,"is_internal_anchor":true},{"citing_arxiv_id":"2605.20642","citing_title":"Same Target, Different Basins: Hard vs. Soft Labels for Annotator Distributions","ref_index":38,"is_internal_anchor":true},{"citing_arxiv_id":"2605.19195","citing_title":"The Thermodynamic Costs of Simple Linear Regression","ref_index":62,"is_internal_anchor":true},{"citing_arxiv_id":"2508.17412","citing_title":"A Ridge Too Far: Correcting Over-Shrinkage via Negative Regularization","ref_index":60,"is_internal_anchor":true},{"citing_arxiv_id":"2510.04686","citing_title":"How does the optimizer implicitly bias the model merging loss landscape?","ref_index":5,"is_internal_anchor":true},{"citing_arxiv_id":"2510.21588","citing_title":"Contribution of task-irrelevant stimuli to drift of neural representations","ref_index":20,"is_internal_anchor":true},{"citing_arxiv_id":"2511.00066","citing_title":"Sharpness-Guided Group Relative Policy Optimization via Probability Shaping","ref_index":17,"is_internal_anchor":true},{"citing_arxiv_id":"2511.07308","citing_title":"Can Stationary Distributions of Scale-Invariant Neural Networks Be Described by the Thermodynamics of an Ideal Gas?","ref_index":3,"is_internal_anchor":true},{"citing_arxiv_id":"2603.10079","citing_title":"Large Spikes in Stochastic Gradient Descent: A Large-Deviations View","ref_index":29,"is_internal_anchor":true},{"citing_arxiv_id":"2605.12168","citing_title":"On What We Can Learn from Low-Resolution Data","ref_index":26,"is_internal_anchor":false},{"citing_arxiv_id":"2604.21691","citing_title":"There Will Be a Scientific Theory of Deep Learning","ref_index":112,"is_internal_anchor":false},{"citing_arxiv_id":"2604.20446","citing_title":"The Origin of Edge of Stability","ref_index":8,"is_internal_anchor":false},{"citing_arxiv_id":"2604.21016","citing_title":"SGD at the Edge of Stability: The Stochastic Sharpness Gap","ref_index":60,"is_internal_anchor":false},{"citing_arxiv_id":"2604.20614","citing_title":"Too Sharp, Too Sure: When Calibration Follows Curvature","ref_index":4,"is_internal_anchor":false}]},"formal_canon":{"evidence_count":0,"sample":[],"anchors":[]},"links":{"html":"https://pith.science/pith/M5BNRHPUWGA7MHPFDKCQKNVN6T","json":"https://pith.science/pith/M5BNRHPUWGA7MHPFDKCQKNVN6T.json","graph_json":"https://pith.science/api/pith-number/M5BNRHPUWGA7MHPFDKCQKNVN6T/graph.json","events_json":"https://pith.science/api/pith-number/M5BNRHPUWGA7MHPFDKCQKNVN6T/events.json","paper":"https://pith.science/paper/M5BNRHPU"},"agent_actions":{"view_html":"https://pith.science/pith/M5BNRHPUWGA7MHPFDKCQKNVN6T","download_json":"https://pith.science/pith/M5BNRHPUWGA7MHPFDKCQKNVN6T.json","view_paper":"https://pith.science/paper/M5BNRHPU","resolve_alias":"https://pith.science/api/pith-number/resolve?arxiv=1711.04623&json=true","fetch_graph":"https://pith.science/api/pith-number/M5BNRHPUWGA7MHPFDKCQKNVN6T/graph.json","fetch_events":"https://pith.science/api/pith-number/M5BNRHPUWGA7MHPFDKCQKNVN6T/events.json","actions":{"anchor_timestamp":"https://pith.science/pith/M5BNRHPUWGA7MHPFDKCQKNVN6T/action/timestamp_anchor","attest_storage":"https://pith.science/pith/M5BNRHPUWGA7MHPFDKCQKNVN6T/action/storage_attestation","attest_author":"https://pith.science/pith/M5BNRHPUWGA7MHPFDKCQKNVN6T/action/author_attestation","sign_citation":"https://pith.science/pith/M5BNRHPUWGA7MHPFDKCQKNVN6T/action/citation_signature","submit_replication":"https://pith.science/pith/M5BNRHPUWGA7MHPFDKCQKNVN6T/action/replication_record"}},"created_at":"2026-05-18T00:05:51.088156+00:00","updated_at":"2026-05-18T00:05:51.088156+00:00"}